Figure 1: Typical OSS Development Model
Open source software (OSS) is one of the most important trends in commercial information technology (IT), yet it’s still widely misunderstood. This article explains what OSS is, why essentially all extant OSS is commercial off-the-shelf (COTS) software, how it’s developed, and how it can be applied to U.S. government acquisitions.
Open Source Software (OSS) is simply software that is licensed to users with the following freedoms:
Common synonyms include “libre software”, “Free software” (sic), “Free / open source software” (FOSS), and “Free-libre / open source software” (FLOSS). Many use the older term “Free software” to emphasize user freedom, but many avoid that term because it’s often misunderstood as meaning “no cost” (which is not the intended meaning). OSS is not free of cost when considering the total cost of ownership (TCO), but an OSS approach may have the lowest TCO for a particular situation. Common antonyms of OSS include “proprietary software” and “closed software”.
OSS is widely used. OSS programs lead or have significant market share in many markets, such as web servers (Apache), web browsers (Mozilla Firefox), email servers (Sendmail), DNS servers (bind), server operating systems (GNU/Linux), C/C++ compilers (gcc), Ada compilers (GNAT), and so on. A 2003 MITRE report noted that OSS “plays a more critical role in the [Department of Defense (DoD)] than has generally been recognized.” [MITRE2003] Many quantitative studies have found that some OSS programs do as well or better than their proprietary competitors by various measures [Wheeler2007].
With almost no exceptions, extant OSS is commercial off-the-shelf (COTS) software. The fact that nearly all extant OSS is commercial, or COTS, can be proven from a variety of viewpoints.
For example, most extant OSS is COTS because U.S. acquisition law and its derivative Federal Acquisition Regulation (FAR) define it as such. U.S. Code Title 41, Chapter 7, section 403, and the corresponding FAR text (2.101) define a “commercial item” as an item “customarily used by the general public or by non-governmental entities” (i.e., they have uses not unique to a government) and have been “sold, leased, or licensed to the general public”, or at least “offered for sale, lease, or license to the general public”. That means that nearly all extant OSS is a commercial item, because nearly all extant OSS is licensed to the general public and have uses not unique to a government. What's more, extant OSS is off-the-shelf (OTS), and since nearly all extant OSS are commercial items, nearly all extant OSS is commercial off-the-shelf (COTS). OSS can even have modifications of a type customarily available in the commercial marketplace, or have minor modifications made to meet federal government requirements, and still be considered a commercial item. The U.S. Navy’s June 5, 2007 memo “Department of the Navy Open Source Software Guidance” specifically affirms this understanding.
This is important in U.S. government acquisitions because the FAR (including the DoD implementation, the DFARS) have far-reaching requirements about commercial items. The FAR requires government agencies to “(a) Conduct market research to determine [if] commercial items or non-developmental items are available … (b) Acquire [them] when… available … [and] (c) Require prime contractors and subcontractors at all tiers to incorporate, to the maximum extent practicable, [them] as components...”. An agency that fails to consider OSS options is in direct violation of the FAR, because it would be failing to consider commercial items.
Another reason that most extant OSS is commercial is because U.S. copyright law says so. U.S. Code Title 17, section 101 defines “financial gain” as including “receipt, or expectation of receipt, of anything of value, including the receipt of other copyrighted works.” Most OSS projects are specifically established to encourage others to contribute improvements (which are copyrighted works), a form of financial gain and thus commercial.
Today it is even more obvious that OSS is increasingly commercial. Commercial companies that directly support OSS projects include IBM, Sun, Red Hat, Novell, and Microsoft – in all cases to improve their profits. Microsoft is a particularly interesting case in point; it fights against OSS projects that compete with its own products (e.g., GNU/Linux), of course, but Microsoft also directly sponsors OSS projects (such as WiX and IronPython) and even its own OSS website (Codeplex) when the OSS projects aid their bottom line. In 2004, Andrew Morton noted that 37,000 of the last 38,000 changes to the Linux kernel were made by developers paid to do so.
Many OSS projects specifically welcome commercial sponsorship. Indeed, the two most popular “formal” definitions of OSS – the “Open Source Definition” and the “Free Software Definition” – both specifically note that commercial use and development of OSS is welcome and encouraged.
For more information on why most extant OSS is COTS, see: https://dwheeler.com/essays/commercial-floss.html.
OSS approaches work because they enable a large number of users to pool their resources, lowering the development and maintenance costs for end-users. This is also true for proprietary software approaches, but OSS projects typically spread these costs in a very different manner. To understand this, we need to examine how OSS is typically developed.
Figure 1 shows a development model for a typical OSS project, which may help dispel misunderstandings. Key to understanding OSS development is that most OSS projects have a “trusted repository”, that is, some (web) location where people can get the “official” version of the program, as well as related information (documentation, bug report system, mailing lists, etc.). Users can get their software directly from the trusted repository, or get it through distributors who acquire it (and provide additional value such as integration with other components, testing, special configuration, support, and so on).
Only some developers are allowed to modify the trusted repository directly: the trusted developers. At project start, the project creators (who create the initial trusted repository) are the trusted developers, and they determine who else may become a trusted developer of this initial trusted repository. All other developers can make changes to their local copies, and even post their versions to the Internet (a process made easy by distributed software configuration management tools), but they must submit their changes to a trusted developer to get their changes into the trusted repository.
Figure 1: Typical OSS Development Model
Users can send bug reports to the distributor or trusted repository, just as they could for a proprietary program. But what is radically different is that a user can actually make a change to the program itself (either directly, or by hiring someone to do it). Since users will want to use the improvements made by others, they have a strong financial incentive to submit their improvements to the trusted repository. That way, their improvements will be merged with the improvements of others, enabling them to use all improvements instead of only their own.
This can create an avalanche-like “virtuous cycle”. As the program becomes more capable, more users are attracted to using it. A very small percentage of such users determine that they can make a change valuable to them, and contribute it back (to avoid maintenance costs). As more improvements are made, more people can use the product, creating more potential users as developers - like a snowball that gains mass as it rolls downhill.
This cost-sharing is rather different than proprietary development, because the costs born by a particular organization are typically only those for whatever improvements or services are used. In contrast, typical proprietary software costs are per-seat, not per-improvement or service. To many the proprietary software business model is more comfortable, because it is more familiar, but it must be noted that the OSS model is much more reflective of actual costs borne by development organizations. It costs essentially nothing to send a file or burn a CD-ROM of software; once it exists, all software costs are due to maintenance and support of software. In short, OSS more accurately reflects the economics of software development. I speculate that this is one reason why OSS has become so common so quickly.
Why would a government be interested in using or creating OSS in government acquisitions? Most reasons simply flow from the definition of OSS:
OSS development processes are all about cost-sharing. While the details are different than in proprietary development, both OSS and proprietary models provide a way for organizations to pool their resources so that each receives a product for far less cost than if they had developed it on their own for their sole use. OSS projects are essentially consortia, and the reasons one might start or join a consortia also apply to starting or joining an OSS project.
In a broader sense, OSS presents governments with the promise of controlling their own destiny: freedom from vendor lock-in, vendor abandonment, and conflicting vendor goals. In short, OSS approaches have the potential in many cases to increase functionality, quality, and flexibility, while lowering cost and development time. Some governments are even applying OSS more widely, as a way for their countries to gain independence from foreign suppliers, but I will not address that further in this article. But the potential of OSS, even when more narrowly considered, can sometimes be difficult to realize.
Applying OSS can sometimes be a challenge in U.S. government acquisitions.
One of the biggest problems today is simply ensuring that OSS is fairly considered in government acquisitions at all. Some acquisition processes or policies have not been updated for OSS; many processes implicitly presume that all COTS software is proprietary software. Alternatively, many U.S. government organizations fail to follow appropriate policies. For example, the FAR is very clear that market research of commercial items is required, and both the U.S. and DoD have issued policy memos stating that they are “OSS neutral”, yet many U.S. government organizations still fail to even consider OSS solutions. In part, this is because many program managers (PMs) are unfamiliar with OSS, and so are unable to ask the right questions. Many COTS OSS projects ignore solicitations & RFPs, so a program cannot simply issue an RFP and expect that all reasonable options will participate (which is why failure to follow the FAR policy for market research is such a serious problem).
Another yet related challenge is that OSS has a fundamentally different economic model. In OSS, users primarily pay-up-front for improvements, as well as paying for whatever services they wish. This is fundamentally different than most proprietary COTS’ pay-per-use model. There is flexibility: For OSS projects, payment can be in money or time, it can be competed, and costs can be shared with others. But it is still a different cost model, one that requires rethinking old assumptions and searching for opportunities, not fleeing from change.
Another difficult challenge is transition costs if there is a pre-existing system, especially if that system is dependent on proprietary formats, protocols, or application programmer interfaces (APIs). All too often an alternative system (OSS or not) will have a radically smaller TCO, yet will not be used because of significant transition costs. The solution is to transition to open architectures based on fully open standards. Such standards are truly supported by multiple vendors and do not include legal traps that inhibit full and open competition, such as misleadingly named “reasonable and non-discriminatory” (“RAND”) patents that actually prohibit most OSS implementation. Using web-based applications, including service-oriented architectures (SOAs), can help as long as they are based on open standards so that they are neutral to the web browsers, servers, and underlying platforms used. Note, however, that vendor lock-in often increases TCO over time - so much so that even high transition costs may be worthwhile.
Other claimed “challenges” usually turn out to be non-problems:
OSS use in U.S. government is pervasive. MITRE noted the DoD’s widespread use of OSS, including the Linux kernel, Samba, Apache, Perl, GCC, GNAT, XFree86, OpenSSH, bind, and sendmail. Reasons aren’t difficult to find; for example, government contractor devIS reports that it “saves its clients a minimum of $100,000 per contract by using OSS” [Miller2003a]. Indeed, government users are often unaware when a program they are using is OSS.
The U.S. federal government has also paid for improvements to OSS when it’s in its interests to do so (demonstrating the transition from user to co-developer). These include Bind (to add DNSSEC support), OpenSSL (for a FIPS cryptographic evaluation), GNAT (to implement Ada support in gcc), BSD TCP/IP suite implementation, and Linux (via Security-Enhanced Linux, aka SELinux).
In some cases U.S. governments have even started whole OSS projects when it was in their interest to do so; examples include OpenVista, Expect, and EZRO. The U.S. state of Georgia started the OSS “Evergreen” project after determining that existing commercial products were grossly inadequate; its OSS approach has been very successful [Stutz2006].
U.S. federal policies are explicitly neutral about using OSS or proprietary software: Both OMB memo M-04-16 and the DoD memo “OSS in DoD” make it quite clear that OSS and non-OSS software are equally acceptable, and do not impose any specific license requirements (e.g., they do not forbid or inhibit use of the GPL, the most popular OSS license).
There are two kinds of COTS software: Proprietary COTS software and OSS COTS software. In many ways you evaluate software the same way. Negotiate your best options with all parties, then select the best one. Evaluate them by winnowing out top candidates for your needs, through the typical process of identifying candidates, reading reviews, comparing (briefly) to needs through criteria, and then analyzing top candidates. Your evaluation criteria will generally be the same: Functionality, total cost of ownership (TCO), support, maintenance/ longevity, reliability, performance, scalability, flexibility, legal/license (including rights and responsibilities), market share, and so on (see https://dwheeler.com/oss_fs_eval.html).
They have many of the same issues:
There are certainly differences in COTS OSS, however. Because COTS OSS is normally developed with greater process and code openness, there are more and different sources of information for evaluation (bug databases, mailing list discussions, and so on). As noted earlier, OSS generally involves pay-per-improvement, while proprietary involves pay-per-use, and in OSS, pay can be time or money. Support can be competed and changed. Since OSS can be modified and redistributed, there is a new decision: When should those options be employed?
One option should normally not be used: Creating a “fork”. A “fork” is a competing OSS project, based on a copy of the original project’s code, that is established with the intent to compete with or replace the original project. Creating a fork is often an emotional event, because it is similar to a call for a “vote of no confidence” in a parliament or a call for a labor strike in a labor dispute. Those creating the fork are essentially stating that they believe the project’s current leadership is ineffective, and are asking developers and users to vote against the existing project leadership by agreeing to switch to the fork instead. If the developers and users switch to the fork, the fork becomes the host of the trusted repository.
Most attempts to create forks fail, for there must be a very strong reason for developers and users to switch to a competing project. Serious attempts to create a fork take a great deal of campaign effort (to convince people to switch), and can create ill will in developers whose support will be needed later (because their valuable time can be consumed in discussing governance issues instead of improving the product). In nearly all cases either the original project or the fork collapses, because neither developers nor users want the confusion and division of effort that stem from having competing repositories.
A rare successful fork happened to the XFree86 project. The XFree86 project leader decided to change the project’s license in 2004 to one that would have made XFree86 illegal to use by many of its users. The XFree86 leadership ignored many attempts to resolve this issue, so a fork was established at X.org. The XFree86 project was then quickly abandoned by essentially all users and developers.
Forks are rarely necessary. The fact that OSS projects can be forked tends to force OSS project leaders to listen to the needs of their customers, just as free elections encourage political leaders to listen to their constituents. The knowledge that a fork can be created encourages leaders to be interested in contributions from others.
Starting an OSS project requires that you check the usual project-start requirements: Is there a need, is the proposal significantly better than alternatives, what is the expected TCO, and so on. In many ways, the OSS approach is like a government off-the-shelf (GOTS) solution, in that the government can directly control the result by investing its own efforts. Unlike a GOTS effort, there is greater opportunity for cost-sharing, with the trade that there is greater openness.
Fundamentally, OSS development processes are about cost-sharing with other interested users. This means that there needs to be at least the possibility of other users, and that any such project must work hard to remove barriers to entry. Such an OSS project should:
These are fundamentally the same issues as for proprietary software. There are many articles and books on how to run an OSS project; consult them.
Neither OSS nor proprietary software is always more secure. Yet OSS does have a potential advantage: OSS better meets the “open design principle”, a fundamental principle in developing secure software that any “protection mechanism must not depend on attacker ignorance.”
It’s critical to understand that hiding source code, as proprietary software vendors typically do, does not really impede attacks. “Security by obscurity” approaches rarely work, and to work at all everything must be kept secret: source code, binaries, and network traffic in and out of the software must be kept inaccessible to attackers.
It’s true that attackers can modify OSS, but they can also modify proprietary software – a hex editor is all you need. The trick is for an attacker to get that modified version into the supply chain. For OSS, this requires subverting, misleading, or becoming the trusted developers or trusted repository/distribution, and for no one to notice the attack later. An attack on the Linux kernel source code demonstrated that this is not necessarily easy, and that widely-reviewed OSS projects can resist such attacks [Miller2003b].
For OSS to be secure from unintentional vulnerabilities, its developers/reviewers must have security knowledge, there must be people who review the code (yes, it does happen), and problems must be fixed and deployed. When examining an OSS option, examine public information (such as websites and mailing lists) to see if these conditions are met. Tools such as static analysis (aka vulnerability scanning) tools and fuzz testing tools can be very helpful in finding such vulnerabilities; DHS and Coverity run a project that scans the source code of major OSS projects and sends to the respective project; Fortify scans many major Java OSS projects and sends them reports.
A foolish policy would say “no OSS” or “no GPL”. This would create a tremendous competitive and strategic disadvantage, and is essentially the same as the foolish “no COTS” policies of some organizations decades ago. Note that U.S. government policy specifically forbids a “no OSS” policy, and intentionally does not forbid GPL use.
Often foolish policies focus on forbidding the GPL, an OSS license that in essence forbids direct linking with proprietary software and requires that someone given a binary must be able to get the source code. The GPL is by far the most popular OSS license; anywhere from 52%-88% of all programs use this license (the next most popular are around 10%). The MITRE study found that, because of the widespread use of the GPL, forbidding GPL use was essentially the same as forbidding use of OSS. Some proprietary companies advocate “no GPL” as a veiled anti-OSS campaign, to inhibit competition. Certainly there will be cases where a GPL’ed program will be inappropriate, but those decisions need to be made on a case-by-case basis, not on emotionalism.
A far more sensible policy requires that all commercial options be considered, both OSS and proprietary, including a review of the licensing requirements. As clearly required by U.S. policy, acquirers need to ensure that all licenses (proprietary and OSS) are compatible with their intended use. It’s fair to note that proprietary end-user license agreements (EULAs) are often more restrictive than OSS licenses in areas that matter to actual use, and that the GPL is often fine for the intended use.
There are two kinds of COTS products: Proprietary and OSS. Do not make the mistake of thinking that “commercial” is the opposite of OSS—instead, OSS is a different kind of commercial approach. Those who use the term “commercial” as an antonym for OSS often find it difficult to understand OSS, since commercial companies are now investing billions of dollars to develop OSS—and you do not want decision-makers or advisors with a poor understanding of this fundamental shift in commercial IT.
OSS options should always be considered during acquisitions. In fact, U.S. government policies mandate that government acquisitions consider OSS approaches, through the FAR and OSS policies. OSS approaches can be applied by selecting a COTS OSS program, modifying an OSS program, or even starting a whole new OSS project. OSS may be components or even the entire solution. OSS approaches are not always the best choice, but it is foolish to ignore them. After all, OSS is very flexible, and it often lowers costs (not only directly, but as competition to other alternatives).
OSS does raise some strategic questions for governments: How can acquirers be pooled so they can start OSS projects when appropriate? How can PMs be educated about OSS, open architectures, and open standards? Should software produced by government research default to OSS, so that all who pay for research can receive the benefit? Should software patents be eliminated (they threaten some OSS projects, have little evidence of benefit, have great evidence of harm, and most software developers consider them inappropriate)?
Certainly projects should change to consider OSS approaches. This will involve educating PMs about the differences in OSS and addressing their fears, emphasizing that OSS options must always be considered (as already required by U.S. policy). Classified systems should work to separate data and program, and layer their programs, to separate classified from unclassified components. Open architectures based on fully open standards should be deployed to enable later change, including moving to web- browser-neutral systems. A part of this will be including requirements that suppliers operationally demonstrate that they can swap components because they use open standards.
OSS approaches present an opportunity, as well as a challenge, for those who can best determine how to apply them.
[MITRE2003] MITRE. “Use of Free and Open Source Software in the US Dept. of Defense.” http://www.terrybollinger.com/dodfoss/dodfoss_pdf_hyperlinked.pdf
[Miller2003a] Miller, Robin. January 29, 2003. Making a Living Saving the Government Money. NewsForge. http://www.newsforge.com/business/03/01/28/1829254.shtml
[Miller2003b] Miller, Robin and Joe Barr. November 6, 2003. “Linux kernel development process thwarts subversion attempt”. NewsForge. http://www.newsforge.com/article.pl?sid=03/11/06/1532223
[Stutz2006] Stutz, Michael. Dec. 21, 2006. Librarians stake their future on open source. Linux.com. http://www.linux.com/article.pl?sid=06/12/04/1538214
[Wheeler2007] Wheeler, David A. April 16, 2007. “Why OSS/FS? Look at the Numbers!” https://dwheeler.com/oss_fs_why.html
This article was originally published in the Software Tech News, Vol. 10, No. 2. It is also available as a presentation, originally given in March 2007 at the "Open Source - Open Standards - Open Architecture" conference in Arlington, Virginia (Crystal City), that was hosted by the non-profit Association for Enterprise Integration (AFEI). It was noted in various places, e.g., Government Computer News' Tech Blog. You may also want to examine the Department of Defense (DoD)'s Open Technology Development Roadmap.
Please see David A. Wheeler's home page for other related articles, such as “Commercial” is not the opposite of Free-Libre / Open Source Software (FLOSS): Most FLOSS is Commercial, Why Open Source Software / Free Software (OSS/FS, FLOSS, or FOSS)? Look at the Numbers!, and How to Evaluate Open Source Software / Free Software (OSS/FS) Programs. More complex issues involving OSS and the U.S. government/DoD are addressed in the page "Questions and Answers for 2008 'Open Source Software and DoD' Webinar, 2008".