Ask what the percentage measures
A claim of complete code coverage sounds like a complete answer. Before judging it, identify the question behind the number. Did the tests execute every statement, exercise decision outcomes, or satisfy a more demanding criterion? Those are different descriptions of a test campaign. None tells a reader, by itself, whether the expected answers were correct or the requirements captured the intended job.
NASA's 2001 aviation tutorial distinguishes requirements coverage from structural coverage. The first concerns how tests verify the software requirements; the second examines the code structure exercised by those tests. Written in the DO-178B context, it explains why the two contribute different evidence. Treating one percentage as a substitute for both loses that distinction.
Execution and correctness need separate evidence
The tutorial distinguishes statement coverage, decision coverage and modified condition/decision coverage, usually abbreviated MC/DC. Statement coverage concerns executable statements reached during testing. Decision coverage examines decision outcomes; MC/DC additionally examines whether individual conditions can independently affect a decision. These criteria describe the structure exercised. The tutorial also emphasizes comparing observed outputs with the expected requirements-based results.
Consider a hypothetical museum display that converts an object's mass from one unit to another. A test can execute the conversion statement while the application uses an incorrect conversion factor. Recording that the statement ran does not settle whether the displayed number was right. This deliberately simple example illustrates why a report needs both execution evidence and an independently justified expected result.
An uncovered region is a question to investigate
NASA's software engineering requirements, NPR 7150.2D, distinguish several explanations for incomplete structural coverage: a missing requirement, a missing test, unnecessary code, or code associated with another configuration. The document calls for coverage data and explanations of uncovered code to be reviewed. These are NASA requirements, rather than an assertion about the approval basis of a particular aircraft.
For readers, the interesting evidence is the explanation and its resolution. A bare percentage cannot distinguish an omitted test from functionality whose purpose was never documented. Equally, an uncovered region should not automatically be reported as a demonstrated operational failure. Ask what the investigation found and how the final account connects that finding to the stated software behavior.
The measurement method also belongs in the report
A June 2007 FAA-sponsored study investigated how structural-coverage tools interpret and implement coverage criteria. Its prototype test suite identified anomalies in each of three tools examined. The report explicitly says it is research, not FAA certification policy. The dated result illustrates a measurement question; it does not establish that any present-day tool has the same problems.
When comparing two announcements, look for a named criterion and a defined measurement scope before comparing their percentages. One number might concern a selected component while another concerns a much larger software baseline. A useful account also identifies the tool and configuration behind the measurement. Otherwise, superficially identical figures may describe different amounts and kinds of verification work.
Keep coverage inside the wider assurance claim
FAA AC 20-115D describes software assurance through applicable objectives, planned activities and supporting life-cycle evidence. A coverage result contributes to that wider body of work; quoting it alone does not establish that every applicable objective has been satisfied. The circular describes an acceptable compliance approach, rather than making a coverage score an aircraft approval.
Our reading questions are concrete: what required behavior was examined, what criterion was measured, and what happened when expected and observed results differed? Then ask how uncovered structures were explained. This approach gives a strong coverage result its proper value while keeping the limits visible. It evaluates the public explanation without pretending to perform an aircraft's certification review.
Public primary sources reviewed 18 October 2026. The 2001 tutorial and 2007 study explain coverage concepts in the historical DO-178B context; the latter is research, not certification policy. NASA's own engineering requirements and FAA civil-aviation guidance have different scopes. The museum example and reading questions are editorial illustrations. The paid DO-178C text was not consulted; no product, tool, aircraft approval or military compliance assessment is provided.
Read the evidence.
- 01
- 02
- 03Software Verification Tools Assessment Study, DOT/FAA/AR-06/54, June 2007Federal Aviation Administration / NASA Langley Research Center
- 04AC 20-115D — Airborne Software Development AssuranceFederal Aviation Administration · 21 Jul 2017