6.3. Compilation Solutions in C/C++

A completely different approach is to use compilation methods that perform bounds-checking (see [Sitaker 1999] for a list). In my opinion, such tools are very useful in having multiple layers of defense, but it’s not wise to use this technique as your sole defense. Many such tools only provide a partial defense. More-complete defenses tend to be slower (and generally people choose to use C/C++ because performance is important for their application). Also, for open source programs you cannot be certain what tools will be used to compile the program; using the default “normal” compiler for a given system might suddenly open security flaws.

Historically a very important tool is “StackGuard”, a modification of the standard GNU C compiler gcc. StackGuard works by inserting a “guard” value (called a “canary”) in front of the return address; if a buffer overflow overwrites the return address, the canary’s value (hopefully) changes and the system detects this before using it. This is quite valuable, but note that this does not protect against buffer overflows overwriting other values (which they may still be able to use to attack a system). There is work to extend StackGuard to be able to add canaries to other data items, called “PointGuard”. PointGuard will automatically protect certain values (e.g., function pointers and longjump buffers). However, protecting other variable types using PointGuard requires specific programmer intervention (the programmer has to identify which data values must be protected with canaries). This can be valuable, but it’s easy to accidentally omit protection for a data value you didn’t think needed protection - but needs it anyway. More information on StackGuard, PointGuard, and other alternatives is in Cowan [1999]. StackGuard inspired the development of many other run-time mechanisms to detect and counter attacks.

IBM has developed a stack protection system called ProPolice based on the ideas of StackGuard. IBM doesn’t include the ProPolice name in its current website - it’s just called a "GCC extension for protecting applications from stack-smashing attacks". However, it’s hard to talk about something without using a name, so I’ll continue to use the name ProPolice. Like StackGuard, ProPolice is a GCC (Gnu Compiler Collection) extension for protecting applications from stack-smashing attacks. Applications written in C are protected by automatically inserting protection code into an application at compilation time. ProPolice is slightly different than StackGuard, however, by adding three features: (1) reordering local variables to place buffers after pointers (to avoid the corruption of pointers that could be used to further corrupt arbitrary memory locations), (2) copying pointers in function arguments to an area preceding local variable buffers (to prevent the corruption of pointers that could be used to further corrupt arbitrary memory locations), and (3) omitting instrumentation code from some functions (it basically assumes that only character arrays are dangerous; while this isn’t strictly true, it’s mostly true, and as a result ProPolice has better performance while retaining most of its protective capabilities).

Red Hat engineers in 2005 re-implemented buffer overflow countermeasures in GCC based on lessons learned from ProPolice. They implemented the GCC flags -fstack-protector flag (which only protects some vulnerable functions), and the -fstack-protector-all flag (which protects all functions). In 2012, Google engineers added the -fstack-protector-strong flag that tries to strike a better balance (it protects more functions than -fstack-protector, but not all of them as -fstack-protector-all does). Many Linux distributions use one of these flags, as a default or for at least some packages, to harden application programs.

On Windows, Microsoft’s compilers include the /GS option to include StackGuard-like protection against buffer overflows. However, it’s worth noting that at least on Microsoft Windows 2003 Server these protection mechanisms can be defeated.

An especially strong hardening approach is "Address Sanitizer" (ASan, also referred to as ASAN and AddressSanitizer). ASan is available in LLVM and gcc compilers as the "-fsanitize=address" flag. ASan counters buffer overflow (global/stack/heap), use-after-free, and double-free based attacks. It can also detect use-after-return and memory leaks. It can also counters some other C/C++ memory issues, but due to its design it cannot detect read-before-write. Its has a measured overhead of 73% average CPU overhead (often 2x), with 2x-4x memory overhead; this is low compared to previous approaches, but it is still significant. Still, this is sometimes acceptable overhead for deployment, and it is typically quite acceptable for testing including fuzz testing. The development processes for Chromium and Firefox, for example, use ASan. Details of how ASan works is available at http://code.google.com/p/address-sanitizer/, particularly in the paper "AddressSanitizer: A Fast Address Sanity Checker" by Konstantin Serebryany, Derek Bruening, Alexander Potapenko, and Dmitry Vyukov (Google), USENIX ATC 2012 Fundamentally ASan uses "shadow bytes" to record memory addressability. ASan tracks addressability of memory, where addressability means if a read or write is permitted. All memory allocations (global, stack, and heap) are aligned to (at least) 8 bytes, and every 8 bytes of memory's addressability is represented by a "shadow byte". In the shadow byte, a 0 means all 8 bytes addressable, 1..7 means only next N are addressable, and negative (high bit) means no bytes are addressable. All allocations are surrounded by inaccessible "red zones" (with a default size of 128 bytes). Every allocation/deallocation in stack and heap manipulates the shadow bytes, and every read/write first checks the shadow bytes to see if access is allowed. This countermeasure is very strong, though it can be fooled if a calculated address is in a different valid region. That said, ASan is a remarkably strong defense for applications written in C or C++, in cases where these overheads are acceptable.

A "non-executable segment" approach was developed by Ingo Molnar, termed Exec Shield. Molnar’s exec shield limits the region that executable code can exist, and then moves executable code below that region. If the code is moved to an area where a zero byte must occur, then it’s harder to exploit because many ASCII-based attacks cannot insert a zero byte. This isn’t foolproof, but it does prevent certain attacks. However, many programs invoke libraries that in aggregate are so large that their addresses can have a non-zero in them, making them much more vulnerable.

A different approach is to limit transfer of control; this doesn’t prevent all buffer overflow attacks (e.g., those that attack data) but it can make other attacks harder [Kiriansky 2002]

In short, it’s better to work first on developing a correct program that defends itself against buffer overflows. Then, after you’ve done this, by all means use techniques and tools like StackGuard as an additional safety net. If you’ve worked hard to eliminate buffer overflows in the code itself, then StackGuard (and tools like it) are are likely to be more effective because there will be fewer “chinks in the armor” that StackGuard will be called on to protect.