Static analysis tools for security

David A. Wheeler

2015-08-16

There are many static analysis tools, and many of them look for security vulnerabilities. NIST’s Software Assurance Metrics and Tool Evaluation (SAMATE) project posts a general list of static analysis tools focused on finding security vulnerabilities. Wikipedia's pages on static analysis and dynamic analysis may prove helpful. GNU Hurd (!) has a list of analysis programs. John Carmack (founder and former technical director of Id Software)’s post “Static Code Analysis” discusses static analysis in general: “Automation is necessary... I feel the success that we have had with code analysis has been clear enough that I will say plainly it is irresponsible to not use it.” Carmack also quotes Dave Revell, “the more I push code through static analysis, the more I’m amazed that computers boot at all.”

If you’re looking for another FLOSS tool to help you find security problems in your C programs more in depth, for now I particularly suggest that you look at my own flawfinder, the Clang Static Analyzer, SPLINT, and cppcheck. Cppcheck focuses on low false positives so it should be easy to examine its reports. Flawfinder (my own tool) uses a very simple naive approach (it is basically a simple lexer), but that simple approach has its own advantages. RATS is the project most similar to flawfinder; it uses the same basic technique, and is also released under the GPL.

I’m a big fan of using multiple tools to find security vulnerabilities. My own Flawfinder is intentionally simple, easy-to-use, and easy-to-understand. It is certainly not the be-all of tools, but that is not the point. My hope is that flawfinder will encourage people to start looking into the various tools available, and trying some out. Software is complex; we need tools to help us find vulnerabilities ahead-of-time in software we develop.

OSS tools

Other OSS/FS tools/projects that statically analyze programs for security issues (besides flawfinder) include:

  1. flawfinder, my security static analysis tool for analyzing C/C++ programs (GPL license).
  2. Clang Static Analyzer (BSD-like license) can find bugs in C, C++, and Objective-C programs. Here are a few comments about Clang Static Analyzer from a user. It does inter-procedural analysis with contraint modeling, so it can do far more in-depth analysis of software.
  3. Railroader, a static analysis tool for finding security vulnerabilities in Ruby-on-Rails applications (MIT license). Full disclosure: I am the lead maintainer.
  4. OWASP LAPSE+, a static security analyzer for Java web applications that is a successor to the LAPSE project (GPL).
  5. FindSecurityBugs (LGPL) is a plug-in for FindBugs for finding security-related defects.
  6. SPLINT (GPL license). This works somewhat like lint, searching for probable errors; to really use it, developers need to add additional annotations to help the tool identify problems. This is a very mature program, widely used, and one you can start using right away on ‘real programs”.
  7. (Facebook) Infer (BSD license) is a static analyzer that looks for defects in Java, C, and Objective-C code code. It does interprocedural analysis and is based on separation logic (a logic system with additions specifically for reasoning about programs). It focuses primarily on quality issues like resource leaks and null dereferences, rather than security issues, but it seems promising.
  8. Cqual (GPL license). “Cqual is a type-based analysis tool that provides a lightweight, practical mechanism for specifying and checking properties of C programs. Cqual extends the type system of C with extra user-defined type qualifiers. The programmer adds type qualifier annotations to their program in a few key places, and Cqual performs qualifier inference to check whether the annotations are correct. The analysis results are presented with a user interface that lets the programmer browse the inferred qualifiers and their flow paths.”
  9. MOPS (old BSD license) “MOPS is designed to check for violations of rules that can be expressed as temporal safety properties. A temporal safety property dictates the order of a sequence of operations. For example, in Unix systems, we might verify that the C program obeys the following rule: a setuid-root process should not execute an untrusted program without first dropping its root privilege.” It uses a model checking approach.
  10. RIPS does static code analysis on PHP code. It’s currently in PHP, but RIPS is being rewritten.
  11. CIL is a framework for analyzing C programs.
  12. BLAST (Berkeley Lazy Abstraction Software Verification Tool). “BLAST is a software model checker for C programs. The goal of BLAST is to be able to check that software satisfies behavioral properties of the interfaces it uses. BLAST uses counterexample-driven automatic abstraction refinement to construct an abstract model which is model checked for safety properties. The abstraction is constructed on-the-fly, and only to the required precision.” Note: The first version of BLAST was developed at UC Berkeley, but follow-on work is going on at EPFL.
  13. BOON (BSD-like license). BOON stands for “Buffer Overrun detectiON”. “BOON is a tool for automatically finding buffer overrun vulnerabilities in C source code. Buffer overruns are one of the most common types of security holes, and we hope that BOON will enable software developers and code auditors to improve the quality of security-critical programs.”
  14. ggcc is an extension of the gcc compiler suite that will do static checking of various kinds. As of May 2008 it was in early development.
  15. Stanse (GPLv2) is a static analysis framework to find bugs in C code. It’s written in Java, plus some perl.
  16. The Spike PHP Security Audit Tool is for analyzing PHP programs.
  17. Pixy scans PHP programs for XSS and SQLI vulnerabilities; it is written in Java.
  18. Orizon is a general-purpose code analysis system (though their primary interest is security scanning). Milk is a Java source code security scanner built on top of Orizon. They are connected to OWASP.
  19. PScan (GPL license) is a source code scanner like flawfinder and RATS, but has only a limited capability. It’s really only intended to find format string problems. In contrast, both flawfinder and RATS can find format string problems and many other problems as well.
  20. The Open Source Quality Project at Berkeley is investigating tools and techniques for assuring software quality (not just security) of OSS/FS programs.
  21. Project pedantic’s Czech by Jose Nazario might become interesting, but as of April 2004 it looks like that project has halted, with only a buggy not-ready prototype so far (which is too bad!).
  22. smatch. is a general-purpose tool for statically analyzing programs, and could be used to build vulnerability scanners. Indeed, there are lots of tools for statically analyzing programs in a general way, this is only one example.
  23. Sparse is a specialized static analysis tool that does additional type-checking, including checks related to security. It was originally designed to check the Linux kernel source code. Sparse finally has its own web page. More information on sparse is available from the CE Linux forum, the Quick sparse HOWTO by Randy Dunlap, and the sparse mailing list. You can download older snapshots of sparse’s code from codemonkey.
  24. Oink (including Cqual++) (BSD-like license). (a Collaboration of C++ Static Analysis Tools).
  25. Yasca (BSD license) is a “simple static analysis tool designed to analyze source code and for a variety of errors. It is both a framework and an implementation, and leverages other open source code scanners where applicable.” You can also see the Yasca Github site.
  26. Frama-C (LGPL) is a framework for the development of collaborating static analyzers for the C language. Many analyzers are provided in the distribution, including a value analysis plug-in that provides variation domains for the variables of the program, and Jessie, a plug-in for computing Hoare style weakest preconditions. It provides a formal behavioral specification language for C programs named ACSL.
  27. RTL-check “RTL-check is an extensible and powerful abstract interpretation framework for static analysis of programs from a safety and security perspective. It performs analysis on RTL, which is the low-level intermediate representation generated by GCC. See the documentation section for more information.” The code is on SourceForge; a good first start to learning about it is to read Patrice Lacroix master’s thesis.
  28. PMD looks for potential problems in Java code. Not specific to security. (BSD-style license) There are other Java program analyzers too.
  29. Findbugs also looks for potential problems in Java code. Not specific to security (LGPL license).
  30. cppcheck searches for defects in C/C++ code. It appears to work by tokenizing source files into sequences of tokens and then matching on the tokens; thus it's more like flawfinder and RATS, since it does not have deeper analysis available to it (e.g., it cannot do interprocedural analysis). There’s little documentation, unfortunately, but you can invoke it like this (use the force option “-f” else it will give up on some files, and use -a (“all warnings”) to get all details): 
      cppcheck -a -f ./ 2> cpperr.txt &
  31. PerlCritic analyzes perl programs. It’s really a style checker, not so much a vulnerability scanner.
  32. Agnitio is a tool to manage checklists when doing manual reviews. It’s a different kind of tool, but I thought it’d be worth noting. Warning: it needs .NET and doesn’t run on Mono as of 2011-09-15 (though they are working on that).
  33. Treehydra is a GCC plugin that provides a low level JavaScript binding to GCC’s GIMPLE AST representation. Treehydra is intended for precise static analyses. Most of Treehydra is generated by Dehydra. A Dehydra script walks the GCC tree node structure using the GTY attributes present in GCC. Treehydra is included in Dehydra source, and is built when a plugin-enabled CXX is detected.
  34. Coccinelle aka spatch Coccinelle, also known as spatch, is a source-to-source translator available under GPLv2. Valerie Henson (now Valerie Aurora) has written an article about Coccinelle, and here’s another article about it.
  35. bddbddb / bddshell. bddbddb (aka b5b) is a general-purpose tool for analyzing big programs. It lets you read in a program and then enter queries in a Prolog-like language, and its internals use the BDD datastructure to make all of this work for large programs. bddshell lets you use it interactively. These are more “tools for building analysis tools”, rather than analysis tools themselves.
  36. LLVM. LLVM is really a compiler infrastructure project, but among other things it can be used to create analysis tools. But it’s not a security analysis tool by itself.
  37. shellcheck (GNU Affero General Public License version 3) is a static analysis tool that reports on common mistakes in (Bourne) shell scripts. It is not specific to security, but several of its reports are security-related.
  38. Elsa. Elsa (BSD license) is a C/C++ parser based on Elkhound. GCC also has a parser.

check-all-the-things is a project that tries to run as many tools as possible on some software (be prepared for a lot of output!). The Software Assurance Market Place (SWAMP) can run a number of tools on software.

There is a similar program to flawfinder, ITS4 (from Cigital), but it isn’t open source software or Free Software (OSS/FS) as defined above, and as far as I know it isn’t maintained.

Of course, you could go the other way: Instead of looking for specific common weaknesses, you could prove that the program actually meets (or does not meet) certain requirements. If you’re interested in open source software tools related to proving programs correct, seej High Assurance (for Security or Safety) and Free-Libre / Open Source Software (FLOSS)... with Lots on Formal Methods / Software Verification and the Open Proofs website.

Quasi-open tools

  1. CERT ROSE checkers checks C and C++ against a subset of the rules in the CERT Secure Coding Standards for C and C++. The ROSE checkers are themselves open source, and build on the open source ROSE, but ROSE itself is fundamentally dependent on a a proprietary component (Edison Design Group’s C/C++ compiler), so the whole stack is in fact proprietary.
  2. ROSE/Compass (BSD license) is a source-to-source translator that can be used to build analysis programs. It includes Compass, which reports violations of a number of rules that relate to security.
  3. VisualCodeGrepper - this is a code security review tool for C/C++, C#, VB, PHP, Java, and PL/SQL. The EULA says it's under the GPL.. but I can't find any actual source code. As far as I can tell it's a lexically-based tool, which means it appears to work the same way as flawfinder, RATS, and ITS4.

Proprietary tools

There are various suppliers that sell proprietary programs that do this kind of static analysis. I can't list them all, but I may as well list some. These tools include:

  1. HP/Fortify Software. Their Fortify Source Code Analysis tool is briefly described in the PCWorld article Software Searches for Security Flaws. Fortify Software is now owned by HP (as of 2010).
  2. Coverity’s SWAT tool searches for defects in general, including some security issues. It’s based on previous work on the Stanford checker, which was implemented by xgcc and the Metal language (the Stanford site has lots of interesting papers, but no code as far as I can tell -- please let me know if things are otherwise).
  3. GrammaTech develops and sells “static-analysis and program-transformation tools for C/C++ and Ada”. This include CodeSurfer/CodeSonar (R) for static analysis, and CodeSurfer/x86 for analyzing and rewriting binary executables.
  4. Sourcemeter is a static source code analyzer for Java, C/C++, RPG and Python. The Java code analyzer is able to find source code vulnerabilities like SQL-injection, XSS and some others. It works in a way similar to Lapse+. (There's a free version with a few capabilities, but you need to pay for its more advanced features.)
  5. Veracode has tools to analyze software for security vulnerabilities (including binary analysis).
  6. Sofcheck Inspector performs static analysis on Java and Ada programs to find defects.
  7. Red Lizard Software is an Australian firm that sells Goanna, a tool that analyzes C/C++ code for software quality bugs (including some security vulnerabilities).
  8. Kestrel Institute works to “make formal methods work in practice”; they have various proprietary tools.
  9. Ounce Labs’s product Prexis. Ounce labs was recently bought by IBM.
  10. Klocwork sells various products that do static analysis.
  11. @stake, now owned by Symantec Corporation, sells a tool called the SmartRisk (TM) Analyzer; unlike many tools, this one analyzes binary code.
  12. Parasoft sells some static analysis tools.
  13. Microsoft bought the company Intrinsa, and their product (known as PREfix) is used now to do static analysis of many of their own products.
  14. PVS-Studio is “a static analyzer that detects errors in source code of C, C++/C++0x, C#, and Java applications.” Historically it’ was not specifically focused on security issues, but more recent versions have added that information. Here's an article about PVS-Studio being used to find mistakes in the Linux kernel. Here is more info about PVS-Studio.
  15. Parfait is a Sun research project, which has found some vulnerabilities. An interview discusses Parfait further. At the time of this writing, this is unreleased.
  16. KDM Analytics has developed some prototypes using a standards-based approach. Code is first transformed into KDM (an OMG standard), and rules are defined using SBVR (another OMG standard). Then you can search for matches/violations of rules. One neat thing is that this can analyze (in principle) either binary or source code in arbitrary languages. I know some people are modifying gcc to generate KDM. SBVR (the rule-defining language) is a restricted-English logic language, so the rules are unusually readable. To my knowledge, these are not available on the market yet.

There are of course many companies that sell the service of performing security reviews of source code for a fee; who generally use a combination of tools and expertise. These include Secure Software developer of RATS, and Aspect Security, backers of the Open Web Application Security Project (OWASP).

Arian Evans has announced that he’s working on a list of such tools, and intends to post that list at OWASP; by the time you read this, it may already be available. NIST’s Software Assurance Metrics and Tool Evaluation (SAMATE) project posts a list of static analysis tools, along with a list of related papers and projects. Common Weakness Enumeration (CWE) is developing a standard set of definitions of common weaknesses and their interrelationships.

Other places list security tools, but not really static analysis tools; these include the Talisker Security Wizardry Portal and insecure.org’s survey of the top 75 tools.

Java2s has a list of Java-related tools for source analysis which may be of interest. They make the common mistake of saying “commercial” when they mean “proprietary” (OSS is commercial software too).

There are a vast number of static analysis tools that check for style or for possible errors, which might happen to catch security problems. They’re usually not focused on security issues, though, and there are too many to list anyway, so I don’t try to list them all here.

This list can’t possibly be exhaustive, sorry. My goal here isn’t to provide all possible alternatives, merely to provide useful information pointing to at least some other tools and services. My goal is mainly so you can have an idea of what’s going on in the field.

Be careful defining language subsets

Many people have developed “language subsets” in an effort to reduce the risk of errors. In concept, these can be really helpful, especially for languages like C which are easy to abuse.

Such language subsets should be automated by static analysis tools; then, it’s easy to check if you’ve met the rules. But these only have value if the subset is well-designed. In particular, the subset should be designed to minimize cases where perfectly acceptable constructs are forbidden (essentially false positives), and should maximize detection of actual failures (best done through analysis of real-world failures).

One of the better-known subsets for C is “MISRA C”. Les Hatton has published a detailed and devastating critique of older versions of MISRA C (of both MISRA C 1998 and the later MISRA C 2004). Fundamentally, he believes (and provides good evidence) taht MISRA C’s development was not based on real data on failures, but instead on random rule creation, some of which are absolutely full of false positives, and many have no value. See Les Hatton’s papers, including those showing why those versions of MISRA C are badly flawed. His paper Language subsetting in an industrial context: a comparison of MISRA C 1998 and MISRA C 2004 is “A comparison of real to false positive ratios between the 1998 and 2004 versions of the MISRA C guidelines on a common population of 7 commercial software packages”, and it has devestating conclusions: “On these results, MISRA C 2004 seems a step backwards and attempts at compliance with either document are essentially pointless until something is done about improving the wording of the standard and its match with existing experimental data. In its current form, the complexity and noisiness of the rules suggest that only the tool vendors are likely to benefit.”

An additional problem with MISRA C is that it is not open access (aka Internet-published). That is, you can’t just use Google to find it and then immediately view its contents (without registering or paying for the contents). That makes it hard to apply. Purported standards that aren’t open access are becoming increasingly pointless; IETF, OASIS, W3C, Ecma, and many other bodies already do this.

I’m a fan of Les Hatton’s work, and I particularly like his paper on his EC-- ruleset. The EC-- ruleset is Internet-published, and is much smaller, so it’s actually easier to apply than MISRA C. More importantly, though, the EC-- ruleset appears to be much better matched to the real world for finding failures, so I strongly prefer EC-- over MISRA C. Here were his rules for creating the EC-- ruleset; once you look at this list, I think you’ll see why:

Additional rules specific to security would be a good idea, too, if they’re well-crafted. The CERT C Secure Coding Standard is an effort to craft rules for developing secure C programs. I haven’t had time to evaluate it in-depth, though, so I don’t know what its quality is. Another document you might examine is Microsoft’s Security Development Lifecycle (SDL) Banned Function Calls.

Dynamic/Hybrid Analysis Tools

Static analysis tools are unlikely to catch all problems in practice; they’re best complemented with other approaches. Certainly, having humans look at code is wonderful (this is a manual static analysis approach).

Dynamic analysis tools send data to executing programs as a way to possibly find problems. Many tools are based on the idea of sending random or partly random data for testing; some “randomize” but try to concentrate on patterns most likely to reveal security problems. Dynamic analysis tools include:

  1. American Fuzzy Lop (AFL) is a really good fuzzer, focused on file-level fuzzing.
  2. SPIKE Proxy is an OSS/FS HTTP proxy for finding security flaws in web sites. It is part of the Spike Application Testing Suite and supports automated SQL injection detection, web site crawling, login form brute forcing, overflow detection, and directory traversal detection.
  3. Brute Force Binary Tester (BFBTester) checks for single and multiple argument command line overflows and environment variable overflows, and version 2.0 can also watch for tempfile creation activity.
  4. Michal Zalewski’s mangleme (demo and source code) sends stressing random data for testing web browsers.
  5. iExploder is another tools for testing web browsers by sending random data.
  6. zzuf is a fuzzer (open source, MIT-style license). See the FOSDEM 2007 slides and Joe Barr’s article about zzuf.
  7. OWASP ZAP dynamically probes web applications, looking for indicators of unknown vulnerabilities. There a lot of these kinds of tools.

There are lots of scanning tools for checking for already known specific vulnerabilities, and sometimes they help. Nessus is a widely-used vulnerability assessment tool. Nikto scans web servers for common problems.

There are many, many other tools and techniques available; I can’t list all of them. You can find a few leads from the Top 75 Security Tools survey at insecure.org. ISP planet’s The article Web Vulnerability Assessment Tools.

You might want to look at my Secure Programming HOWTO web page, or some of my other writings such as Open Standards and Security, Open Source Software and Software Assurance (Security), and High Assurance (for Security or Safety) and Free-Libre / Open Source Software (FLOSS).

You can also view my home page.