David A. Wheeler's Blog

Mon, 12 Dec 2005

Countering Trusting Trust through Diverse Double-Compiling, ACSAC 2005

Something new: I have a section about my work to counter the “Trusting Trust” computer security attack. The “Trusting Trust” attack is a very old and incredibly nasty attack in computer security. Karger and Schell published information about this attack in 1974, and Ken Thompson (of Unix fame) made it much more widely known in 1984 in his Turing award speech “Reflections on Trusting Trust.” Ken Thompson even demonstrated it; he gained complete control over another system, and that system’s owners never detected the subversion. Up to now it’s been presumed that the “Trusting Trust” attack is the essential uncounterable attack.

What exactly is the trusting trust attack? Basically, if an attacker can get a Trojan Horse into the binary of a compiler, at any time, you’re essentially doomed. The subverted compiler can subvert itself, indefinitely into the future, as well as anything else it compiles.

I’ve worried about this attack for a long time, essentially since Thompson made his report. If there’s a known attack that cannot be effectively countered, even in theory, should we really be using computers at all? My hope is that my work in this areas aids the computer security field writ large.

The reason I note this in my blog is that I’ve finally formally published my paper that describes a technique for countering this attack. The paper is Countering Trusting Trust through Diverse Double-Compiling (DDC), and it was published by ACSAC 2005. Here’s a local copy, along with more info and material. Here’s the abstract of that paper:

An Air Force evaluation of Multics, and Ken Thompson’s famous Turing award lecture “Reflections on Trusting Trust,” showed that compilers can be subverted to insert malicious Trojan horses into critical software, including themselves. If this attack goes undetected, even complete analysis of a system’s source code will not find the malicious code that is running, and methods for detecting this particular attack are not widely known. This paper describes a practical technique, termed diverse double-compiling (DDC), that detects this attack and some compiler defects as well. Simply recompile the source code twice: once with a second (trusted) compiler, and again using the result of the first compilation. If the result is bit-for-bit identical with the untrusted binary, then the source code accurately represents the binary. This technique has been mentioned informally, but its issues and ramifications have not been identified or discussed in a peer-reviewed work, nor has a public demonstration been made. This paper describes the technique, justifies it, describes how to overcome practical challenges, and demonstrates it.

I just got back from the ACSAC 2005 computer security conference. Several interesting papers there, on a variety of topics.

An aside: At ACSAC 2005, Aleks Kissinger (from the University of Tulsa) also presented work that he and I had done on micro-tainting. This was the presentation “Fine-Grained Taint Analysis using Regular Expressions,” which was part of the Works in Progress. Basically, we noted that instead of assigning “taint” to a whole value, such as a string, you could assign taint on subcomponents, such as each character. Then you could assign rules that identified the input paths and what could come in — typically zero or more tainted characters — and rules on output paths. We concentrated on defining regular expressions for what is legal, though any other expression for patterns such as BNFs would be fine too. We noted that you could then check statically or dynamically. For the static case, when you work backwards, if the check “fails” you can even trivially derive the input patterns that cause security failures (and from that information it should be easy to figure out how to fix it). Aleks has recently made some good progress by transforming the regular expressions into DFAs. There was another ACSAC presentation on doing taint analysis with Java, but this was the traditional “whole variable” approach that is used in many languages, but through which many vulnerabilities slip by. We hope this micro-tainting approach will lead to improved tools for detecting security vulnerabilities in software, before that software is delivered to end-users.

path: /security | Current Weblog | permanent link to this entry

Wed, 30 Nov 2005

How to revive an old laptop

A relative gave me an old laptop with Windows 98, Which led me to the question: can you take a castoff laptop and, with zero or very little money, improve it so it’s more useful and can run more “modern” software? The answer is: yes! In fact, this turned into a little game/project for me, and I learned a few things along the way. So I wrote down what I decided to do, in the hopes that you may find these ideas useful for reviving an old laptop yourself:

How to revive an old laptop

I think it’s a shame that older machines sometimes rot in closets instead of helping people, and I hope that this document will help change that. With a little elbow grease (and adjusted expectations), you can still get mileage out of an older laptop.

I talk about getting new hardware (keep it cheap!), buying a wireless card, making backups and moving windows to a new disk, installing GNU/Linux, updating a Windows 98 System, and making a boot floppy for windows.

path: /misc | Current Weblog | permanent link to this entry

Sun, 27 Nov 2005

FLOSS Java Roadmap published

Ever since Richard Stallman wrote his article Free But Shackled - The Java Trap, many developers have avoided using Java. Why? At the time, there was no practical way to delivery fully free-libre / open source software (FLOSS) using Java while still being fully functional. Not because it was illegal to have a FLOSS Java implementation, but simply because the FLOSS tools and libraries weren’t available.

But things have been moving quickly; many developers have been working hard to develop an implementation of Java that doesn’t depend on proprietary software. The problem is that there hasn’t been a simple way to understand what’s going on — unless you’re an “insider”.

Thankfully, that’s changed. Escaping the Java Trap: A practical road map to the Free Software and Open Source alternatives is a simple 3-page summary that surveys the many different FLOSS projects that are building, testing, and distributing a complete FLOSS Java implementation (including mountains of libraries). As the roadmap notes, “Important large applications like JOnAS, OpenOffice.org 2, Eclipse 3 and Tomcat 5 are known to work. This document provides a road map of the various projects; how they work together, where they are, where they’re going, and how we make sure that they work well and are compatible.”

This is the roadmap I noted earlier as part of my FISL 2005 travelogue. Although I helped the other authors write it, I really operated as a ghost writer rather than speaking with my own voice. Basically, I really wanted to know what the state of FLOSS Java implementations was, and I was fortunate to be able to talk with the top experts at FISL. I promised them if they told me about the various parts, I would in turn help them describe it in a simple way. So the material is really all theirs — I was just lucky enough to be the first recipient of it.

Other articles also help give more perspectives on the topic, too. The state of Java on Linux by Tom Tromey has some interesting material, for example. But I know of no other document that gives such a wide overview of how a full FLOSS implementation of Java (TM) is getting built, tested, and distributed.

Again, take a peek: Escaping the Java Trap: A practical road map to the Free Software and Open Source alternatives

path: /oss | Current Weblog | permanent link to this entry

Fri, 25 Nov 2005

OpenDocument Accessibility

There has been a lot of virtual ink spent on OpenDocument accessibility. I’ve written up a short essay on OpenDocument accessibility, where I point to some other resources that talk about OpenDocument accessibility, and point out that there are lots of ways to get it. For a vast number of cases, products that natively support OpenDocument do just fine today. For some cases, just use Microsoft Office with an OpenDocument plug-in; you already have to use a third party plug-in to add accessibility in those cases, so saying that you can’t add a third-party plug-in for OpenDocument as well is simply hypocritical.

I also post a lengthy letter from Wesley Parish, who is disabled and yet is a strong supporter of OpenDocument. The article has more, but here are a few quotes: “It is necessary for the disabled to have access to all government information relevant to them, in a file format that is readily available for as many different applications from as wish it, one that does not insist that one jump through licensing hoops in order to implement it, one that can be readily extended in the future according to need - and one that can not be used as an excuse by lazy bureaucrats to deny me my rights! The question currently buzzing in Massachussetts is , “Does Open Document Format limit accessibility?” For myself, I find it does not. [In Computer Science] I found one of the most persistent concepts was a strict separation between data and executable code. ODF provides that strict separation, defining data separately from the code. … An open specification that allows ANYONE to implement accessibility solutions is the way to solve the problems of access by the the blind and other disabled. Otherwise, government data will be tied to specific programs and NOT accessible to all, and in time, NOT accessible at all.”

So go take a peek at my short essay on OpenDocument accessibility.

path: /misc | Current Weblog | permanent link to this entry

Sun, 13 Nov 2005

November 2005 release of “Why OSS/FS? Look at the Numbers!”

It’s November, and I’m putting out another release of “Why Open Source Software / Free Software (OSS/FS, FLOSS, FOSS)? Look at the Numbers!” This paper continues to provide “quantitative data that, in many cases, using open source software / free software (abbreviated as OSS/FS, FLOSS, or FOSS) is a reasonable or even superior approach to using their proprietary competition according to various measures. This paper’s goal is to show that you should consider using OSS/FS when acquiring software.”

The big news is that I’m releasing a presentation based on this report. The presentation is at http://www.dwheeler.com/numbers — and you can use it as-is or as the starting point for own presentations. The presentation is being released in two formats, PDF (for reading) and OpenDocument (for presenting or editing). I’m hoping that many other people will be willing to create translations of this presentation. The presentation is much smaller, and thus much easier to translate, than my thorough (but much larger) work.

I’ve made a number of changes since May as well. Here are some of the highlights:

  1. Added a new performance report on Unix and Linux beat Windows… the funny thing is that it’s from Microsoft.
  2. Various URL fixups, as websites change their address. For example, the URL of the Netcraft survey archive is now http://survey.netcraft.com/.
  3. Information Week Research’s study of corporate use of Linux and Open Source software. They found that “Nearly 90% of companies we surveyed anticipate a jump in server licenses for Linux. No other product comes close to these expectations - not Windows, Macintosh or Unix.”
  4. Added references to BusinessWeek’s interesting market figures.
  5. Noted Symantec’s newest security study, and its misunderstood results. Firefox has more vulnerabilities than Internet Explorer, UNLESS you count vulnerabilities Microsoft hasn’t fixed, in which case IE has more vulnerabilities. Some people haven’t looked at these numbers closely, and think that they are an argument for the security of IE, which is nonsense. I think vulnerability counts are a lousy metric, so it’s fine to complain about that… I do! But DON’T rig the metric and ignore known vulnerabilities just because a supplier does… that is rediculous.
  6. Added a reference to an Investors article on OSS/FS commercialization.
  7. Added RFG’s August 2005 TCO report
  8. Updated the OpenSSH marketshare data. Their September 2004 survey shows them dominating the SSH market, with 87.9% share. What’s possibly more important is the trend line; there’s no evidence that OpenSSH is being eliminated by proprietary products at all.
  9. Noted newer Coverity study of Linux kernel defect rate (August 2005), which complements their earlier studies).
  10. Referenced more recent summaries as of August 2005, which suggest that Internet Explorer is still more dangerous than the OSS/FS Firefox. David Hammond’s Internet Explorer is dangerous examined the Secunia reports on Internet Explorer, Firefox, and Opera, as of August 4, 2005. Firefox did far better than IE, both historically and currently.
  11. Added statistics by Scanit’s Browser Security Test group, which found that 98% of time in 2004 Internet Explorer was vulnerable to dangerous known remote attacks, compared to 17% for Opera and 15% for Mozilla/Firefox. I also added a link to an article that explains the context of DHS’s earlier warnings.
  12. Added a reference to an paper about Multics, which shows that even back then, people thought that it was critical to have source code publicly avaiable if security was important. Here’s what it said: “It is expected that the Multics system will be published when it is operating substantially and will therefore be available for implementation on any equipment with suitable characteristics. Such publication is desirable for two reasons: First, the system should withstand public scrutiny and criticism volunteered by interested readers; second, in an age of increasing complexity, it is an obligation to present and future system designers to make the inner operating system as lucid as possible so as to reveal the basic system issues… The system will evolve under the influence of the users and their activities for a long time and in directions which are hard to predict at this time… It is expected that most of the system additions will come from the users themselves and the system will eventually become the repository of the procedure and data knowledge of the community.”
  13. Added reference to Calculating the True Price of Software by Robert Lefkowitz.
  14. Added a reference to Tom Adelstein’s “Linux in Government: Outside the US, People Get it”.
  15. Noted Microsoft’s increasing development of OSS/FS, as reported by ZDNet.
  16. Noted that GNU/Linux basically owns the high-end computing area. 60% of all supercomputers run GNU/Linux, including the world’s fastest, as of March 2005; 80% of the top ten supercomputers run GNU/Linux, and Noted that the Internet Archive (the world’s biggest library counting by text) uses GNU/Linux.

Were I to start now, I think I’d use the term “FLOSS” (Free-Libre / Open Source Software) as my all-encompassing term, so I mention that at the beginning. FLOSS is much easier to say than some of the alternatives. The term “Free Software” is widely misunderstood as being “no cost”, so by itself I find that it’s not helpful for explaining things. The term Free-Libre is a big improvement because it at least hints at what its promulgators intended the term to mean. However, I’ve used the term OSS/FS all over, and it’s awkward to change now (and people might not find the document they were looking for), so I haven’t changed my own documents.

Enjoy!

path: /oss | Current Weblog | permanent link to this entry

Sat, 06 Aug 2005

Internet Explorer: So insecure, it’s only safe 7 days a year?!?

I recently learned some amazing — unbelievable — shocking data. It turns out that there were only 7 days in 2004 that you could have somewhat safely used Internet Explorer (it was October 12-17), even assuming that attackers only used publicly-known attacks, and that you were only worried about the worst kind of attacks. What does that mean? Let me set the stage first… and I’ll conclude what to do at the end.

In my article how to secure Microsoft Windows (for home and small business users), I give advice for people who want to keep using Windows but have some security. One piece of advice: stop using some of the most vulnerable programs, such as Internet Explorer (IE) and Outlook, and instead more secure alternatives (such as the freely-available Firefox and Thunderbird). It should be self-evident that replacing insecure programs with more secure programs will make you more secure! But let me deal with two complaints: (1) why should I change, and (2) is Internet Explorer (IE) really so much worse?

First - why should you change to using more secure software? Because if you’re not willing to select a more secure program, then you are part of the problem — you are causing everyone to have insecure programs, as well as causing your own misfortune. Why? Because vendors will not make secure products unless customers prefer them. “The marketplace” decides what’s successful, and you are part of it. I’m tired of hearing “my machine is full of spyware”; if you chose to use a product that is known to have that problem, then you need accept the consequences of your choices. You can’t claim ignorance at this point, the news has been circling for a long time. Sure, the attackers should be convicted. But since there are prowlers in the alleyway, please don’t invite them into your house, and then act surprised when they take the silverware. Yes, you can’t spend all your time on securing things, and you need to have useful (not just secure) products, but it’s easy to replace these programs with perfectly good alternatives.

And second — IE really is worse. This isn’t just a random opinion, and it’s not Microsoft-bashing. There is lots of evidence that, in particular, Internet Explorer has become a malware delivery system. See, for example, David Hammond’s comments on Internet Explorer.

But I’m blown away by one particular study I just learned about, which shows the problem is even more serious than I thought. Scanit’s Browser Security Test group “A Year of Bugs” analyzed the vulnerability reports in 2004 for three popular browsers: Microsoft’s Internet Explorer, Mozilla-based browsers (including Firefox and Netscape), and Opera. Since not all vulnerabilities are equal, they only considered the especially dangerous “remote code execution” vulnerabilities, i.e., defects that allow a “malicious web page or e-mail message to execute arbitrary code or OS commands on the viewer’s computer.” They then compared the time from the “public announcement of the vulnerability to the time when the fix is available to the general user population.” They had an incredibly simple metric: every day there’s a publicly-known vulnerability, for which there is no patch available from the vendor, is an unsafe day. That’s a metric anyone can understand: how many days are you vulnerable to the worst attacks that are (1) known worldwide but (2) there’s nothing you can do about it?

Their results: there were only 7 days Internet Explorer was safe to use in the entire year of 2004. That means that 98% of the year, Internet Explorer was not safe to use. Is it any wonder people like me say “please don’t use it?”

Let me quote their study: “there was only one period in 2004 when there were no publicly known remote code execution bugs - between the 12th and the 19th of October - 7 days in total.” That means that someone who diligently kept their installation patched every day of the year (do you install the latest patches every day?) was still known to be vulnerable 98% of the time in 2004. The rediculous excuse “well, it wasn’t exploitable” doesn’t work, either; they found that for “200 days (that is, 54% of the time) there was a [known] worm or virus in the wild exploiting one of those unpatched vulnerabilities.” And that only counts known attacks. Frankly, 2004 was a disturbing year for IE; at the beginning of the year there were two known unpatched vulnerabilities, and 2004 ended with an “unpatched HTML Help ActiveX control vulnerability and [the worm] Trojan.Phel using it to install a backdoor.” And remember, this is only the publicly-known attacks, of the worst kind.

Now let’s not let alternatives off the hook; Mozilla-based programs and Opera had unsafe days too. But compared to IE’s “98% unsafe” value, Opera had unsafe days only 17% of the time, and the Mozilla/Firefox were only unsafe 15% of the time (and about half of that 15% only affected MacOS users). Let’s look at the details:

On June 28, 2004, Microsoft’s Bill Gates told Australians that while other operating system vendors took 90-100 days to release a security patch, Microsoft had this time “down to less than 48 hours.” And Microsoft has clearly stated that IE is part of their operating system. Yet ZDNet found that Microsoft had failed to fix a critical known IE vulnerability for nearly nine months Things got so bad that in late June 2004, the U.S. Department of Homeland Security’s Computer Emergency Readiness Team (CERT) recommended using browsers other than Microsoft Corp.’s Internet Explorer (IE) for security reasons. (That’s not exactly how they officially worded it… but I think many people correctly realized that that was the subtext). And even after all that, IE still had unpatched vulnerabilities for the worst kind of vulnerabilities through most of the rest of the year.

Let me throw in an aside about reporting vulnerabilities. Some companies try to convince vulnerability reporters to “keep quiet” until they fix the problem… and then just never fix it. The vulnerability is still there, though it’s officially not publicly known… and if one person can find it, others will too. That head-in-the-sand approach used to be common, but our systems are just too important to allow that to continue. That’s why I think it’s a good idea for vulnerability reporters to give suppliers 14 days to fix the problem, with a few more days if there’s a really good reason to allow unusual delays. Fourteen days should be more than enough time to fix a critical problem in the vast number of cases, but it puts the supplier on notice that leaving its customers permanently vulnerable to a known weakness is unacceptable. Certainly 30 days should be plenty for even complex problems. If your supplier can’t normally turn around patches for critical fixes in 14 days or less — and certainly by 30 days — perhaps you need a new supplier. Gates says 48 hours is enough, half of the Mozilla problems had one-day turnaround times, and all the Mozilla problems (even the complex ones) were fixed within 30 days of a confirming report.

I will say, with relief, that Microsoft is finally going to release a new version of Internet Explorer, with some attempt at fixing the security problems. But the reports worry me. CERT’s July 2, 2004, notification noted some of the major design decisions that make Internet Explorer so easy to exploit: “There are a number of significant vulnerabilities in technologies relating to the IE domain/zone security model, the DHTML object model, MIME type determination, and ActiveX.” Yet everything I’ve read suggests that they will not fundamentally change all of these major design decisions, so at least some of their fundamental design weaknesses will probably still be there. Disabling ActiveX completely by default for all sites would be a help, for example: The “zone” model doesn’t work (it’s too easy to fool), a massive number of signed or pre-installed ActiveX components are vulnerable, and people just click “ok” when another ActiveX component is sent that ActiveX is a synonym for “send me malware”. I really hope that IE is much more secure, but we’ll see. The past does not necessarily predict the future.. but it’s usually a good way to bet.

And the next version of Internet Explorer will still not support the Internet standards. This was reported by Paul Thurrott in Windows IT Pro, among others. So many standards-compliant web sites will still be inaccessible to Internet Explorer users.

But even worse… the next version of Internet Explorer is only going to go to XP Service Pack 2 users. Microsoft has various excuses for this. That’s rediculous; why does everyone else, who already paid for Internet Explorer, have to suffer? Unless you pirated Windows, Internet Explorer was part of the purchase price of your machine or included in a separate license; it’s actually possible you paid for Windows several times. But most Microsoft Windows users don’t use XP Service Pack 2; even many XP users haven’t installed Service Pack 2 because of the legion of incompatible changes and machine lockups it caused many. A vast number of people do not have Windows XP; Windows 2000 is in widespread use, and even Windows 98/ME have significant use (25% by some measures). It’s not true that a secure browser requires Service Pack 2; other browser makers manage it.

Don’t use the current versions of Internet Explorer normally, and wait a number of months before thinking about using the new version. In particular:

  1. If you use Windows XP, upgrade to Service Pack 2, and upgrade Internet Explorer when the new one becomes available. But for heaven’s sake, don’t use the new version of Internet Explorer in normal operation until it’s been proven to be relatively safe by many months of relatively safe operation. You probably shouldn’t use Internet Explorer at all until it adds better standards support, too; hopefully Internet Explorer version 8 or so will do so.
  2. If you use any version of Windows other than Windows XP, or won’t use Service Pack 2, then abandon hope of ever using Internet Explorer (except to download a better browser). For those machines, there doesn’t seem to be much hope that you will every be able to use Internet Explorer safely, it’s been just one problem after another and the vendor will not even offer a replacement that they hope is safer.
  3. If you use a site that requires IE, try to change to someone who accepts alternatives. If it’s a company-internal site, then you could certainly consider using IE for just that site. In the meantime, continue to use an alternative (like the freely-available Firefox) for all other browsing.
  4. If your bank or other security-critical site actually requires IE, switch to a bank that takes the security of your money and identity seriously, now, and make sure they know why. See Can You Bank on IE Security? from Bankers Online, a magazine for bankers. They say, “No longer are the major organizations suggesting that users merely download the latest patches, check their security settings, and scan their systems for viruses, this time the advice is - CHANGE TO A DIFFERENT BROWSER! And the advice is not coming from any lightweight organization with a bias. This is coming from the most respected international security watchdog organizations. [including CERT, SANS, NIPC]”
  5. If you develop a website, make sure that it’s standards-compliant so that any standards-compliant browser can view it. Internet Explorer has been losing marketshare to other web browsers (such as Mozilla Firefox) since mid-2004, so customers may start avoiding your site because they will probably increasingly not be using IE. It really makes no sense to tie your website to any browser; it’s unnecessary, and creates a situation where your customers may be unable to securely use your website.

Note: I don’t make any money no matter what web browser or operating system you choose. I suggest preferring advice about this topic from others who can say the same. And obviously I’m speaking only for myself, not anyone else, though it’s clear that many, many others have come to the same conclusions.

path: /security | Current Weblog | permanent link to this entry

Thu, 16 Jun 2005

Travelogue Available of International Free Software Forum (FISL), Brazil

As I noted earlier, I spoke at the “6th International Free Software Forum” / Fórum Internacional Software Livre (FISL). The conference was 1-4 June, 2005.

I’ve just posted my travelogue of the 6th International Free Software Forum in Porto Alegre, Brazil. If you didn’t get to go, this may give you an idea what it was like. I also try to make some observations along the way, which hopefully you’ll find interesting. For example, I comment about Brazil’s relationship with open source software / Free software (OSS/FS), which I found very interesting. I also try to explain how I ended up helping to document the complicated inter-relationships between some of the many OSS/FS Java projects.

path: /oss | Current Weblog | permanent link to this entry

Fri, 27 May 2005

I’ll be speaking 3 June at the International Free Software Forum (FISL), Brazil

I’ll be speaking at the “6th International Free Software Forum” in Porto Alegre, Brazil. Its Portuguese name is 6° Fórum Internacional Software Livre, so this is abbreviated as “FISL”. The conference itself is 1-4 June, 2005.

I’ll be speaking on June 3, 2005, from 17:15-18:15. I’ll be presenting in room 41A (in the 41 building). This is their biggest room, with 1000 person capacity and sessions with simultaneous translation. That may sound like a lot, but as of May 27 there were 3,180 attendees registered, and I’m sure there will be more at the door. So if you’re interested, please come early!

My presentation will summarize my work, Why Open Source Software / Free Software (OSS/FS)? Look at the Numbers! Here’s the summary:

“The goal of this lecture is to convince you to consider using OSS/FS when you’re looking for software, using quantitive measures. Some sites provide a few anecdotes on why you should use OSS/FS, but for many that’s not enough information to justify using OSS/FS. Instead, this paper emphasizes quantitative measures (such as experiments and market studies) to justify why using OSS/FS products is in many circumstances a reasonable or even superior approach.”

I hope to see you there!

path: /oss | Current Weblog | permanent link to this entry

Tue, 03 May 2005

In memorium: Dennis W. Fife

My long-time boss, mentor, and friend, Dr. Dennis W. Fife, passed away on April 28, 2005. His funeral was held on May 2, 2005.

Dennis was a good man, who dearly loved truth. On many occasions he spoke the truth to those he worked for, even when the truth was very unpopular. Yet he never did this maliciously; he always did this gently, trying to help people move to a more productive path.

He was smart, and enjoyed learning new things. He wrote important documents of their time on approaches to managing software development; many of his ideas are still applied today (though not always acknowledged!). He later helped many others apply computing to solve real problems. Indeed, he was always willing to share his knowledge with others, including the mentoring of many who are very grateful for his guidance. When asked a question, he’d think about it for a moment, and afterwards he’d often reply with a helpful insight into the situation.

Dennis also had a dry, subtle wit that many of us grew to love. Dennis enjoyed comics like the “Far Side”, with its often twisted view of the world. He would often would say subtle things with a twinkle in his eye… it might take you a moment to get it, and then you’d laugh out loud.

I will greatly miss him.

path: /misc | Current Weblog | permanent link to this entry

Mon, 02 May 2005

Trend: Simple, readable text markup languages

Here’s a new(?) trend, that shows that everything old really is sometimes new again. What’s the trend? Simple, highly readable markup languages.

In some situations, typical document formats (such as OpenDocument or Word .doc format) simply don’t work well. This includes massive collaboration over an internet, or for creating relatively simple/short documents. Although existing markup languages like DocBook, HTML/XHTML, LaTex, and nroff/man all work, they’re often complicated to write and read. You could use SGML or XML to create your own markup language, but that doesn’t really address the need for simplicity. None of these work very well if you expect to have many users who don’t really understand computers deeply (HTML comes closest, but complicated HTML documents become unreadable in a hurry).

Thus, there’s been a resurgence of new markup languages that are really easy to read and write, which can then be automatically translated to other formats. Two especially capable examples of this trend seem to be AsciiDoc and MediaWiki:

  1. AsciiDoc looks very reasonable if you want to create documents or websites; it can generate HTML, XML, DocBook, PDF, and man pages (DocBook can in turn generate other formats; it can also generate the obsolete LinuxDoc format). This is no trivial capability; it can handle cross-links, tables, and so on. Technically, AsciiDoc processing requires an implementation to look ahead to the next line to understand text; some find this annoying, but if it makes the language easy to read, I think that’s quite reasonable.
  2. Wikipedia’s markup language (supported by MediaWiki) has grown a lot of capabilities (to support creating an encyclopedia), yet it’s still easy to use (and thus is a really capable example of this). There are a vast number of users of this notation, but setting up a processor for it isn’t so easy.

The various Wiki languages, such as MoinMoin’s, etc., are also examples of this. But there are a lot of different ones, all incompatible. Here’s some text on StructuredText, ReStructuredText, and WikiText. Many Wiki languages use CamelCase to create links, unfortunately; a lot of people (including me) find that convention ugly and awkward (MediaWiki dumped CamelCase years ago; MediaWiki internal links look like this: [[MediaWiki link]]). Most Wiki languages are too limiting for wider use.

No doubt there are others. One I learned about recently is Markdown. Markdown is a notation for simply writing text and generating HTML or XHTML; it seems to be focused on helping bloggers.

Anyway, it’s an interesting trend! I’ve created a new essay about this at http://www.dwheeler.com/essays/simple-markup.html; if I learn about interesting new links related to this, I’ll add them there.

path: /misc | Current Weblog | permanent link to this entry

Thu, 07 Apr 2005

Updated: Comments on OSS/FS Software Configuration Management (SCM) Systems

I’ve updated my paper Comments on Open Source Software / Free Software (OSS/FS) Software Configuration Management (SCM) Systems. This is basically a review of several of these systems. I can’t possibly look at them all, but I intend for it to be a useful place to start. Given the recent issues with BitMover (maker of BitKeeper), causing Linus Torvalds to look at other SCM tools to manage Linux kernel development, this seems pretty timely.

path: /oss | Current Weblog | permanent link to this entry

Sat, 02 Apr 2005

April 2, 2005 release of “Why OSS/FS? Look at the Numbers!”

I’ve posted an update of “Why Open Source Software / Free Software (OSS/FS, FLOSS, FOSS)? Look at the Numbers!”

The biggest change? I’ve added a large set of studies about the market penetration of the Mozilla web browsers (primarily the newer Mozilla Firefox, but the older Mozilla suite is also in use), as compared to Internet Explorer (IE). A multitude of studies show that IE is losing market share, while OSS/FS web browsers (particularly Firefox) are gaining market share. Sources of data include general market surveys like WebSideStory, OneStat, Information Week/Net Applications, thecounter.com, and quotationspage.com, as well as more specialized sources such as W3Schools (web developers) and Ars Technica (computer technologists). The figure below extracts data from several sources (there are far more in my paper than I can legibly show), but they all show the market trend over time. The red squares are Internet Explorer’s market share (all versions), and the blue circles are the combination of the older Mozilla suite and the newer Mozilla Firefox web browser (both of which are OSS/FS):

Web Browser Market Share (Wheeler Summary) - IE is losing market share while Mozilla Firefox gains market share

The data seems to show a small, gradual trend in the general web browsing community, with a much larger and more rapid move towards Mozilla and Mozilla Firefox in the home user, technical, web development, and blogging communities. In some cases (such as the Ars Technica technical site and the Boing Boing blog site), Firefox has become the leading web browser! That’s interesting particularly because it can be easily argued that the technical, web development, and blogging communities are leading indicators; these are the developers of the web sites you’ll see tomorrow and some of the heaviest users of the web, all making a switch.

One study not shown in the figure above (because it’s a single point of data) is from XitiMonitor. They surveyed a sample of websites used on a Sunday (March 6, 2005), totalling 16,650,993 visits, and categorized various European users. By surveying on Sunday, they intended to primarily find out what people choose to use, from their homes. Of the German users, an astonishing 21.4% were using Firefox. The other countries surveyed were France (12.2%), England (10.9%), Spain (9%), and Italy (8.6%). Here is the original XitiMonitor study of 2005-03-06, an automated translation of the XitiMonitor study, and a blog summary of the XitiMonitor study observing that, “Web sites aiming at the consumer have [no] other choice but [to make] sure that they are compatible with Firefox … Ignoring compatibility with Firefox and other modern browsers does not make sense business-wise.”

I analyzed this data to determine that 13.3% of European home users were using Firefox on this date in March 2005. How can I justify that figure? Well, we can use these major European countries as representatives of Europe as a whole; they’re certainly representative of western Europe, since they’re the most populous countries. Presuming that the vast majority of Sunday users are home users is quite reasonable for Europe. We can then make the reasonable presumption that the number of web browser users is proportional to the general population. Then we just need to get the countries’ populations; I used the CIA World Fact Book updated to 2005-02-10. These countries’ populations (in millions) are, in the same order as above, 82, 60, 60, 40, and 58; calculating (21.4%*82 + 12.2%*60 + 10.9%*60 + 9%*40 + 8.6%*58) / (82+60+60+40+58) yields 13.3%. This is something you won’t find on other pages; this is new analysis unique to my paper.

For all the detail on web browser market surveys, see http://www.dwheeler.com/oss_fs_why.html#browser-marketshare.

And yes, I’ve made lots of other improvements to the paper. Here are a few examples:

  1. I made a number of small improvements to the government section. For example, I expanded the introductory section, and I added a reference to the collection of essays “Government Policy toward Open Source Software” I noted that many governments like the ability to internationalize their software. I also noted that the United States’ Federal Enterprise Architecture includes the Technical Reference Model (TRM), and TRM version 1.1 (August 2003) includes both Linux and Apache.
  2. Added a reference to happy customers of Microsoft’s OSS/FS WiX project — even Microsoft finds that this OSS/FS seems to work for them.
  3. I added a reference to a book on OSS/FS licensing, Understanding Open Source and Free Software Licensing by Andrew M. St. Laurent.
  4. Added references to Lessig’s “Code and Other Laws of Cyberspace”
  5. I added a discussion about Visual Basic for .NET, aka Visual Fred. This is a particularly shocking example: here we have a proprietary vendor who is essentially abandoning a common, widely-used product, even after there’s been a public outcry. The “new” Visual Basic is completely incompatible with the old version of Visual Basic, so different that people now call the new version “Visual Fred”. There’s no practical upgrade path; Visual Basic programs have to be essentially rewritten to use the “new” version, and about 2/3 of current Visual Basic programmers end up switching to a new language (not Visual Fred, which is just as proprietary and risky as the previous Visual Basic). Everyone knows that if a proprietary company goes out of business, people who depend on their software will be in trouble. But people often forget that a proprietary company can choose to go in a different direction for their own reasons, even if that will cost you billions of dollars… and there’s little you can do about it if there are no alternatives. This is a good cautionary tale about why people should choose languages that are standardized with multiple vendors or have a viable OSS/FS implementation; that way, users have a way to continue maintenance if the original developer decides to go in a way against the users’ interests. Depending on a proprietary computer language, with a single proprietary implementation, has been known to be a risky decision for decades. Unfortunately, it appears that this lesson never really gets learned.
  6. Added information on an unintentional reliability study done by Heinz Trober.
  7. Added a quote from Unisys’ Steve Rawsthorn. Years ago Unisys was a strong holdout against supporting Linux, compared to most companies. But Unisys found that they were repeatedly losing competitive bids to other companies who did support Linux. Finally, Unisys decided to support Linux systems because they wanted to be able to make money; Linux was what their potential customers wanted to buy. Hmm, supporting Linux so that you can make money? Now that’s a reason for doing something that everyone understands!
  8. I’ve improved the text explaining the Fuzz results, and added a chart to make their results clearer. This is all part of the reliability section.
  9. And a trivial change: clarified that “GPL” stands for the “GNU General Public License”. There are other “General Public Licenses” out there, but everyone means the GNU license unless otherwise stated. That way, I can just say “GPL” and appeal to the definition at the top of the paper, which clarifies that it’s the GNU version.

path: /oss | Current Weblog | permanent link to this entry

Wed, 23 Mar 2005

E-Password comment deadline (April 4) looms - COMMENT NOW

As noted in a Wired article, the U.S. Department of State plans to issue U.S. passports that can be read wirelessly (remotely), and it won’t even encrypt this extremely personal data. This plan is absurd; it appears to give terrorists and organized crime a way to remotely identify U.S. citizens (for murder or kidnapping) and to provide enough detailed personal information to significantly aid identity theft.

The Department of State claims that the new passports can only be read from 10 centimeters and that fibers will prevent any reading while closed. However, most security experts scoff at these claims, noting that people have to open their passports eventually, and doubting that the fiber’s protection will be perfect anyway in real life. Lee Tien, an attorney at the Electronic Frontier Foundation, reports the reading distance as more like 10-30 feet. Bruce Schneier, who just renewed his passport to make sure he will not have an unencrypted passport for another 10 years, says he has yet to hear a good argument as to why the government is requiring remotely readable chips instead of a contact chip — which could hold the same information but would not be skimmable. “A contact chip would be so much safer.”

I think this Department of State plan is going to kill people. There are people in this world who want to hurt or kill Americans, or citizens of some other countries — now we’re giving them an easy tool to help them find Americans (or citizens of some other countries) in foreign countries so that they can be murdered, tortured, raped, or kidnapped for ransom. The ransom stuff alone would fund huge efforts to use this technology in foreign countries to target victims, because it’d be insanely profitable for the immoral.

In my mind, the real problem is the use of wireless technology. This is an area where the convenience of wireless is far outweighed by the disadvantages of getting murdered. Frankly, for data storage, a 2D barcode (which is MUCH cheaper) would have all the advantages of permitting quick storage of a lot of data. If the purpose of the chip is to make forgery harder, then requiring contact would be sufficient.

Is the lack of encryption a problem? Not necessarily, as long as contact is required. After all, if there’s no encryption, then it’s easier to see exactly what data is on the passport (e.g., to verify that it’s correct for you), and the data is supposed to be the same as what’s already on the passport. But it’s a disaster if it’s wireless, because then people who have no business getting the data will be able to retrieve it. Indeed, it’s a disaster that this is wireless at all.

Those who wish to protest this plan have until April 4, 2005, to send their comments to PassportRules@state.gov. I urge you to send in emails asking State to abandon this wireless approach, and that they instead use a system that requires contact. Do it, before someone dies.

path: /security | Current Weblog | permanent link to this entry

Sun, 06 Mar 2005

March 2005 release of “Why OSS/FS? Look at the Numbers!”

For March - another release of “Why Open Source Software / Free Software (OSS/FS, FLOSS, FOSS)? Look at the Numbers!” I made many changes, here are some of the highlights:

  1. I modified the introduction section to create subsections in it (it was getting too long to read without some breaking up). Since I was modifying the intro anyway, I added to it some evidence that this paper is worth reading (e.g., the reference to it by the state of California). I highlighted “commons-based peer production” Coase’s Penguin, a new reference, as part of a new section on the “bigger picture”. I also mentioned there the “freedom to tinker” and Wikipedia.
  2. Added reference to Bruce Perens’ The Emerging Economic Paradigm of Open Source
  3. Added reference to Black Duck presentation on OSS/FS
  4. Added a quote from Craig Mundie, who in a moment of honesty admitted that for many years Microsoft had been much more interested in functionality than security. Hopefully this is changing, but I believe it’ll take years to really address that. In any case, I think this explains much of where they’ve been, though hopefully not where they’re going.
  5. Noted Novell/SUSE met CAPP/EAL4+, according to Government Computer News.
  6. Added references to some common starting places, such as the OpenCD (with OpenOffice.org and Firefox), Gnoppix, Knoppix.
  7. Tweaked the text about self-sponsored “studies”, which I don’t have a lot of respect for. I noted a wonderful Steve Taylor phrase from the song “Meat the Press” — it’s a phrase I always think of when I think about that text. It is: “They can state the facts while telling a lie.”
  8. Changed any “Open Office” to “OpenOffice.org”. The latter is the official name, due to trademark issues.

path: /oss | Current Weblog | permanent link to this entry

Wed, 23 Feb 2005

OWASP Legal Project - Secure Software Development Contract Annex

The Open Web Application Security Project (OWASP) Legal Project has just announced the “Secure Software Development Contract Annex”. This is basically a starting point for a contract to do software development; it tries to spell out exactly what’s required so that the results are secure.

I didn’t develop this text, but I’m glad to see that some people are working on it. In the contracting world, if you don’t specifically ask for it, you don’t get it. Since most contracts today don’t specifically say that a secure result is needed (and what that means), currently the person paying for the software isn’t getting a secure product. Hopefully this sort of thing will solve the problem.

Personally, I think this is a “first draft”; there are things I’d like to see made more specific. For example, I think it should clearly state that in the development environment it should be possible to determine specifically, by name, who wrote any given line of code. And there are many other issues (like automated examination of code) that aren’t covered. In particular, there are many more common vulnerabilities than the top ten list of OWASP. But this is a very interesting and encouraging first start, and I’m glad to see it.

path: /security | Current Weblog | permanent link to this entry

Sat, 15 Jan 2005

January 2005 release of “Why OSS/FS? Look at the Numbers!”

I’ve made another release of my paper “Why Open Source Software / Free Software (OSS/FS, FLOSS, FOSS)? Look at the Numbers!” I made many changes, here are some of the highlights:

  1. In the section on governments, noted various documents useful for governments who choose to use OSS/FS, including the short article by Adelstein: and the European IDA’s migration guidelines.
  2. Added a longer explanatory essay, noting that software isn’t normally owned by its users, and thus the term “total cost of ownership” is misleading. A proprietary software user, in particular, doesn’t have the normal rights of ownership: they can’t view for understanding, modify, or redistribute. An OSS/FS user isn’t an owner either, but their rights are at least somewhat more similar to an owner’s. Included a link to the trusted computing FAQ by Ross Anderson; see the text for details.
  3. Added a section on the relationship of standards and OSS/FS, and in particular noted that OSS/FS can sometimes be considered an “executable standard”. After all, you can use it (so it’s useful as it is), AND you can also see EXACTLY how it works (helping to counter the problem of ambiguity that occurs in far too many standards). This is particularly obvious when a standards group creates an OSS/FS project to showcase how to implement a standard.
  4. I noted some alternative abbreviations of OSS/FS in the title. I’ve noted them for years in the text, but thought it’d help some people if the title itself acknowledged them. I actually like “FLOSS” (Free/Libre Open Source Software) as an abbreviation; but I didn’t think of that when I originally wrote this paper, and I figure that changing its title (or content) now would simply make the paper harder to find, as well as being a pain for me.
  5. Added info from Massachusetts on OSS/FS legal issues, and quoted its conclusion: “Use of either open source or proprietary software poses some legal risk to states. States face fewer risks in connection with the use of open source software compared to their private sector counterparts, and the risks that they do face can be managed.”
  6. Noted Torvalds is named one of the best managers of the year.
  7. Noted Chicago Mercantile Exchange example.
  8. Referenced Committee for Economic Development, which mentions OSS/FS relationship to innovation. See http://www.ced.org/docs/report/report_dcc.pdf or http://lwn.net/Articles/73678/.
  9. Added reference to http://searchvb.techtarget.com/originalContent/0,289142,sid8_gci1036918,00.html
  10. Gave examples under support of some companies that provide commercial support for OSS/FS; including MozSource, AdaCore, MySQL AB, various Linux distributions, etc. Noted the lists of consultants for Debian and OpenBSD. I can’t list everyone; the point is just that this is an option.
  11. Added information on bounty/sponsor systems and software ransoms.
  12. Added reference to Coverity study on flaws.
  13. Improved the TCO section, e.g., noted Cybersource update to their TCO study. Noted switching costs issues; this drives most companies to start using OSS/FS on new deployments instead of existing ones to start with, since then there’s no switching cost to pay.
  14. Noted the humorous article “Total Cost of 0wnership” (note the zero), and added reference to “Wisdom of the Crowds” book.
  15. Noted various OSS/FS business opportunity, and an interesting report that salaries of core contributors are 5-15% higher.
  16. Added reference to Koders.com, and an interview about it. I put it in the innovation section - it’s much easier to innovate by being able to reuse all that pre-existing code for the “other stuff” — all you have to implement is the new idea, not the piles of “other” stuff.
  17. Referenced IBM’s Blue Gene/L supercomputer.

path: /oss | Current Weblog | permanent link to this entry