Thomas Plum, [email protected]
Information-technology standards are the domain of ISO/IEC Joint Technical Committee 1 (JTC 1). Within that domain, programming languages fall within Standing Committee 22 (SC22). Programming languages (and their standardized libraries) play an important role (both positively and negatively) in the areas of Dependability, Security, and Safety. Therefore, SC22 has created Other Working Group: Vulnerabilities [OWGV], to produce a Technical Report entitled “Guidance to Avoiding Vulnerabilities in Programming Languages through Language Selection and Use”.
OWGV was recently informed of the Committee on Certifiably Dependable Software Systems of the National Research Council and of their PDF pre-print “Software for Dependable Systems: Sufficient Evidence?” [SDSSE] (herein “the report”).
These notes provide some comments on the report. They are a personal contribution, not the official contribution of any particular group. These personal opinions do not mean that a consensus view has been adopted yet by OWGV on any issues discussed herein.
The charter of OWGV relates only to a small subset of the report, which deals broadly with the entire software development process. The report highlights the importance of the “culture” within specific communities, e.g. “in the domain of avionics software, the certification process (and the culture that surrounds it) is held in high regard and is credited by many for an excellent safety record, with software implicated in only a handful of incidents”. The report seems highly compatible with some of the other overview reports already communicated to OWGV (e.g., [Jarzombek]).
A software-development culture includes shared experience with methods, languages, and tools. The report reflects a nuanced concern for the costs of deveoping dependable software, and those costs, and associated benefits, depend upon those shared experiences. For example, in Japan, the community of embedded-system producers share an opinion that compilers should be subjected to a commercial Compiler Qualification Service (CQS), which provides extensive compiler regression-testing. When bugs are discovered, an associated test goes into the test database, so it won't occur again or elsewhere. Our company undertook to provide distribution of CQS into English-speaking countries [Plum#3], expecting that, in any culture, organizations with the highest dependability goals would naturally require tests such as CQS. However, many English-speaking software-development cultures share an attitude that “compiler bugs are outside the scope of our concerns". Perhaps among the software cultures most involved with the “Software for Dependable Systems" report we will find the audience that would support a service like CQS.
One major conclusion of the report is that dependability will not automatically result from using a specific methodology, tool, or language, but rather requires the construction of an explicit “dependability case arguing that the required properties follow from the combination of the properties of the system itself (that is, the implementation) and the environmental assumptions”. One or more examples, or links to examples, would greatly assist the reader of the report. The report is consistent with other descriptions of the current state of methods for developing dependable software, namely the use of carefully-chosen subsets of the implementation languages and enforcement using static analysis (which could be called the “subset-plus-static” approach).
The report does not mention the possibility of augmenting “subset-plus-static” methods with carefully-chosen dynamic methods. Here one might mention the recent Bounds-Checking Library Technical Report from the C standards committee, SC22/WG14, addressing the issue of buffer overflows occurring in the lowest-level standard C library. (For a recent draft copy, see [BCL-TR].) This TR introduces the term runtime-constraint for the runtime requirements that must be met to avoid buffer overflows (or other undefined behaviors). Whenever one of the new library functions discovers a violation of a runtime-constraint, it invokes a runtime-constraint handler. Given an interactive user of a desktop application, the simplest handler is the abort handler which terminates execution (or in an interactive debugger, might interrupt with a breakpoint). However, when designing cost-effective and dependable server software, an optimal design might terminate one transaction, re-initialize, and proceed to the next, whenever the offending transaction exceeds any capacity limit. The results described in [Rinard] indicate that this approach could have prevented costly security vulnerabilities in the open-source server applications Apache, Sendmail, etc.
In the report's discussion of “subset-plus-static” approach, one could mention the influential “Joint Strike Fighter Air Vehicle C++ Coding Standard” [JSF++]. (Click on “JSF_AV_C++_Coding_Standards_Rev_C.doc” at http://www.jsf.mil/downloads/down_documentation.htm.) Indeed, we noted that although the “members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance”, the report omits any mention of the C++ programming language. C++ is among the programming languages in the SC22 program of work, and is arguably the most widely-used implementation language.
The C programming language fares not much better: one favorable mention and five unfavorable. For comparison, there are five mentions of SPARK (all favorable) and 13 mentions of Java (all in favorable contexts). Noting that “[t]he work of the expert committee will culminate in a written report with recommendations, which will be subject to National Research Council review processes”, we would appreciate information regarding the National Research Council review process, so that appropriate balance can be achieved in the technical issues that directly involve programming languages.
We have two minor editorial issues to report: On PDF page 30, there is a spurious paragraph break in a bullet list, which currently reads
And on PDF page 96, the text reads “For a standard test suite (such as the Java Compatibility Kit ...)” where “widely-used” might be a more accurate replacement for “standard”.
In summary, the members of the SC22 programming language committees should read and study this important report.
We close with a brief disclosure of Plum Hall’s commercial interests. We are working on methods to attack the problems of security, safety and quality at the language-foundation level by preventing undefined behaviorss (especially critical undefined behaviors) with definitive methods, rather than incremental or heuristic methods. A combination of methods are required, at compile-time, link-time, and (to a surprisingly small extent) at run-time [Plum#1].
Conformance test suites for C, C++, Java, and C# are developed and distributed by Plum Hall. These suites (especially the C suite) are sometimes used in the process of qualifying compilers for use in safety-critical applications [Plum#2].
Projects with the highest requirements for safety and quality can make use of commercially-available compiler-testing services which maintain huge databases of regression tests for error-free code generation. A service such as the Compiler Qualification Service (CQS) from Japan Novel has for many years been routinely used by the large producers of embedded systems in Japan. The CQS service is available in Europe and the English-speaking countries, where it is currently distributed by Plum Hall [Plum#3].
[BCL-TR] “Extensions to the C Library - Part 1: Bounds-checking Interfaces”, ISO/IEC JCT 1 SC22 WG14 N1146. "http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1146.pdf
[Jarzombek] OWGV-N0018 Joe Jarzombek, US Department of Homeland Security, "Considerations in Advancing the National Strategy to Secure Cyberspace," presented at Meeting #1 of OWGV, 27 June 2006. ISO/IEC JTC 1/SC 22/OWGV N 0018
[JSF++] “JOINT STRIKE FIGHTER AIR VEHICLE C++ CODING STANDARDS FOR THE SYSTEM DEVELOPMENT AND DEMONSTRATION PROGRAM”, Document Number 2RDU00001 Rev C, December 2005”. Click on “JSF_AV_C++_Coding_Standards_Rev_C.doc” at http://www.jsf.mil/downloads/down_documentation.htm
[OWGV] ISO/IEC JTC 1/SC 22/OWG:Vulnerabilities http://www.aitcnet.org/isai/
[Plum#1] http://www.plumhall.com/sscc.html
[Plum#2] http://www.plumhall.com/descr.html
[Plum#3] http://www.plumhall.com/cqs/cqs.html
[Rinard] Martin Rinard, et.al. “Enhancing Server Availability and Security Through Failure-Oblivious Computing”, http://web.mit.edu/droy/www/papers/osdi04.pdf