This paper discusses how open standards aid free-libre / open source software (FLOSS) projects, how FLOSS aids open standards, and then examines a mystery: Today, essentially every successful open standard is implemented by FLOSS. This paper considers why that is, and what it means if this is predictive. In particular, this suggests that an open standard without a FLOSS implementation is probably too risky for users to require, and that developers of open standards should encourage the development of at least one FLOSS implementation.
Open standards are standards that are publicly defined and anyone can implement, where both their process of definition and who can implement the standard is non-discriminatory. More importantly, open standards create an open market where users can switch between competing implementations -- allowing users to fairly select and re-select the best products for their circumstances. In the software world, examples include TCP/IP (the basis of the Internet) and HTML (the key document format of the World Wide Web). A more precise discussion and definition of what an open standard for software is -- and an example of how to use that definition -- is given in “Is OpenDocument an Open Standard? Yes!”; I discuss software open standards and their relationship to security in “Open Standards and Security” (as reported by NewsForge).
In contrast, free-libre / open source software (FLOSS) is software whose license gives users the freedom to run the program for any purpose, to study and modify the program, and to redistribute copies of either the original or modified program (without having to pay royalties to previous developers). Examples include the Linux kernel, Apache web server, and Firefox web browser. Many FLOSS programs are directly developed by for-profit companies, and in any case, FLOSS software is commercial software.
Open standards do not require the use of FLOSS programs; open standards are by definition neutral on whether proprietary or FLOSS implementations are used. Yet there is a relationship between open standards and FLOSS. This paper briefly discusses how open standards aid FLOSS projects, and how FLOSS aids open standards.
I then move on to a mystery: Today, essentially every successful open standard is implemented by FLOSS. In fact, I haven’t found a single counter-example. There’s no fundamental requirement for this, because by definition open standards do not discriminate between FLOSS and proprietary implementations. I discuss what this observation could mean, including the possibility that you may be able to use this observation to predict the future, and even make wise decisions based on those predictions.
Open standards aid FLOSS projects, and it’s not hard to see why:
However, note that open standards aid proprietary projects in exactly the same way, so open standards are very neutral in their help. Open standards make it easy to switch to a proprietary program that implements the standard, and aid implementors of proprietary programs by giving them the same information. If a user uses an open standard, they can switch from a FLOSS implementation to a proprietary one, as well as from proprietary to FLOSS, if there are both proprietary and FLOSS implementations.
FLOSS aids open standards, too:
Indeed, FLOSS products can by themselves essentially define a (de facto) open standard, because their very existence makes it easy to spread them. Paul Vixie said "compared to the power and relevance of open source software, [the internet standards group IETF] may as well be declared dead for all the relevance it has at this stage". Paul Vixie notes that "zero configuration networking" succeeded by creating a FLOSS implementation first, and that FLOSS programs (like BIND and Sendmail) have essentially created or enforced many interfaces.
One interesting property of FLOSS deserves mentioning: Self-competition. From a user's point of view, open standards are really interesting if there is a marketplace of competing component suppliers that implement the open standard, and allows them to switch to a competitor. So if there is only one implementation of an open standard, and it's proprietary, there's still lock-in (with its problems) due to lack of competition. The same thing doesn't happen with FLOSS, even if there's only one FLOSS implementation. It turns out that a FLOSS program embeds the seeds of competition in it, for several reasons:
Of course, there can still be multiple different FLOSS implementations of an open standard, just as there can be multiple proprietary implementations. The HTTP standard has multiple FLOSS implementations, including Apache and lighttpd. The TCP/IP suite has been separately implemented by the *BSDs and by the Linux kernel. Text editors (which depend on the simple standards for text files) have been implemented for a long time with great interoperability; the running competition between the various emacs and vi/vim camps has been going on for a long time.
At the April 2006 LinuxWorld (Boston) someone made an extraordinary claim: “Today, all successful open standards have FLOSS implementations.” I’ve learned that “all” is rarely true for any statement; this statement should be easy to disprove.
Except I can’t disprove it. Typical examples of open standards (such as TCP/IP, HTTP, HTML, DNS, SMTP, POP3, IMAP, IPSec, SSH, SSL, C, C++, and OpenDocument) all have at least one FLOSS implementation. When I started looking at the specialty open standards, I kept finding FLOSS implementations there too! I looked at niche standards such as Z39.50 for search queries... and found that yes, they have them too. Even open standards developed by non-traditional organizations have FLOSS implementations (e.g., GEDCOM for exchanging genealogical data). There are some open standards without a FLOSS implementation, such as some of the OSI protocol stack components, but these are also examples of failures (TCP/IP, which had strong FLOSS support, demolished the OSI stack long ago). Historically this wasn’t a true statement, but at some point in the past this became true, and it shows all the signs of staying true. There may be a successful open standard without a FLOSS implementation, but the difficulty in finding an example suggests that having a FLOSS implementation is very much the rule, not the exception.
In some sense this is very surprising; there is no fundamental requirement that an open standard must have a FLOSS implementation. For example, almost no organization that creates open standards requires this, and many open standards are based on proprietary implementations (not FLOSS ones). The discussions above on how FLOSS helps open standards, and how open standards help FLOSS projects, could explain why they often occur together... but by themselves they fail to predict this 100% (or nearly so) coverage.
I find unanswered questions much more interesting -- they are a mystery, and often we learn about something more fundamental when we examine mysteries.
So why do essentially all open standards have a FLOSS implementation? One possible explanation is that the aid given by FLOSS to open standards is much more important than previously credited. For example, the availability of a reference model has become so important that perhaps open standards creators essentially always ensure there is a FLOSS implementation, through their development process. There is something to this claim, in the sense that I can point to specific standards’ development process which explicitly used FLOSS as a reference model. The development of OpenDocument often depended on FLOSS implementations -- often when a change in the specification was proposed, its impact on a particular reference implementation that could be viewed by all was examined. This is a good thing, because checking out implementation issues before saying the standard is complete is more likely to produce a good standard. The W3C and IETF often use FLOSS implementations as reference implementations, to help ensure that they’re creating a good standard. But this does not happen all the time, which means this would only explain why this circumstance is common -- it still fails to explain the essentially universal coverage of open standards by FLOSS projects.
Another explanation comes to mind, which I think is more likely. If it’s true, it gives us insight into FLOSS’ current role in information technology (IT). Here’s the possible explanation: FLOSS has become extremely widespread in IT, so much so that if an open standard is successful, there will naturally be a FLOSS implementation. This is more insightful than it appears at first. There are many studies showing FLOSS’ strong presence in many specific markets, such as web servers, mail servers, and so on. But it’s much harder to answer the question “how widespread is FLOSS in the entire IT industry, in all segments?” This is especially hard to answer when you’re interested identifying “key” components, because it’s hard to define “key” in an unambiguous way. Saying “key” components have an open standard is imperfect, but it isn’t a bad estimate; people only bother to work to create an open standard if it is a key interface for them. And it’s reasonable to believe that the larger FLOSS community would concentrate on implementing open standards first -- again, efforts to define open standards tend to focus on key interfaces, and since they’re defined they’d tend to be the interfaces that would be the easiest for FLOSS projects to implement as well.
If that’s true, then essentially all successful open standards are implemented by FLOSS because FLOSS is itself so widespread in IT. At some point in the past FLOSS grew so rapidly that we reached a major milestone: total FLOSS implementation of the key interfaces in the IT industry (as identified by open standards). That suggests that FLOSS is not just implemented in a few key places, but in fact has universally taken root across the IT industry... not just in a few widely-trumpeted situations, but the “unsexy” ones as well.
Theories are only interesting if they can be used to predict future events. And this theory makes a prediction! If it’s true that all successful open standards have a FLOSS implementation, it follows that an open standard will not be successful unless and until a FLOSS implementation becomes available. You can’t predict the other way -- having a FLOSS implementation is no guarantee that an open stanard will succeed -- but the lack of a FLOSS implementation appears to be a serious problem. So:
When you think about it, this makes sense. FLOSS is so pervasive in our industry today that if there are no FLOSS implementations, it suggests that it’s not really an open standard (perhaps there are some legal traps), that it’s viewed as likely to fail, or perhaps there’s some other critical problem. Also, if there’s a competing open standard that has a FLOSS implementation, those who prefer FLOSS products may use that instead... creating a move of users away from the FLOSS-less standard. Having competing standards is a bad idea anyway, but if this observation is predictive -- and it’s reasonable to think it might be -- then it gives us at least one tool to help figure out the eventual winner.
Open standards and FLOSS each support the other. But more interestingly, successful open standards seem to always have at least one FLOSS implementation, and above I speculate why this is so.
This suggests a simple rule: An open standard without a FLOSS implementation is probably too risky for users to require. It also suggests that developers of open standards should encourage the development of at least one FLOSS implementation, since they appear to be essential for acceptance of the standard. That doesn’t mean that all users will use the FLOSS implementation, but they will at least have the option.
It’s been noted that creating open standards is not easy -- and that’s true. But there’s also broad agreement that closed standards only benefit the market leaders, not the users -- such situations quickly lead to dependency, stagnation, a lack of innovation, and extremely high unnecessary costs. Nothing worthwhile is easy, but many organizations have realized that open standards are vital to them and work hard to encourage them (examples include the European Commission, as demonstrated by the “European Interoperability Framework”, and the U.S. DoD). We need to continue to encourage open standards, and work to understand their relationship with other IT trends, so we can deploy open standards in an even smarter way in the future.