ACATS 3.0 User's Guide
3.2.1 Coverage Documents
ACVC 2.0 originated coverage documentation for what
is now the ACATS. That coverage document mapped tests to paragraph numbers
in the
[Ada95] Standard. Over time, a number
of deficiencies in that approach have been identified:
Where a paragraph contains multiple rules, it is
not possible to tell if all of the rules are tested. Indeed, a number
of instances where important rules were untested have been identified.
Since there is not an indication of the type of
rule represented by a paragraph, it is not obvious whether the rule is
appropriately tested. For instance, a Legality Rule tested only with
C-Tests is not appropriately tested (see
3.2.3
for more on this topic).
Rules not modified in
[Ada95]
were not considered. However, sometimes additional test objectives are
needed for existing rules because of other changes to the language. Moreover,
there is no determination of whether the legacy tests adequately tested
the original rule.
For these reasons, with ACATS 3.0, the development
of new and much more detailed coverage documentation was started. The
new documentation considers each sentence of the standard individually
if necessary, listing the test objectives for each sentence, and documenting
why no objectives are needed if none are provided.
It also lists tests for each objective (including
Legacy tests), testing priorities for untested objectives, notes on each
objective (including any untested cases), and any additional notes needed.
There is also a summary document that summarizes
totals for important metrics for the coverage documents as a whole.
These documents are provided in Adobe Portable Document
Format (PDF).
For ACATS 3.0, the
following clauses of the Ada Standards are included in the new coverage
documents:
3.9.3 through 3.9.4
4.1.3 through 4.4
6.5 through 6.7
7.4 through 7.6.1
8.3.1 through 8.5
Section 10 (10 through 10.2.1)
These clauses were selected because of their importance
in rule changes in
[Amend1]The original coverage document is also included with
the ACATS for those portions of the Ada Standards that have not yet had
new coverage documents constructed. This document is of primary use for
features of
[Ada95], as it does not cover
Legacy tests nor any features added or modified by
[Amend1].
As noted above, it is being replaced, so it will disappear completely
in a future version of the ACATS.
