ACATS 3.0 User's Guide
Title Page
Contents   Index   References   Next 

Section 1: Introduction

The Ada Conformity Assessment Test Suite (ACATS) provides the official tests used to check conformity of an Ada implementation with the Ada programming language standard (ANSI/ISO/IEC 8652:1995 and later corrigenda and amendments). The ACATS User's Guide is part of the ACATS and is distributed with the test programs and testing support packages. It explains the contents and use of the test suite.
The ACATS is an important part of the conformity assessment process described in ISO/IEC-18009, Ada: Conformity of a Language Processor [ISO99]. This standard provides a framework for testing language processors, providing a stable and reproducible basis for testing. The Ada Resource Association (ARA) has sponsored an instantiation of that process since October 1998. The process is managed by the Ada Conformity Assessment Authority (ACAA).
Prior to the ISO standard, the U.S. Department of Defense sponsored a similar conformity assessment process under the Ada Joint Program Office (AJPO). The test suite for that process was known as the Ada Compiler Validation Capability (ACVC). The AJPO developed ACVC versions based on ANSI/MIL-STD-1815A-1983, ISO/8652:1987 (Ada 83), which were numbered 1.x where x ranged from 1 to 11. It later developed ACVC versions based on ANSI/ISO/IEC 8652:1995 ([Ada95]), numbered 2.0, 2.0.1, 2.1, and 2.2.
When the ACAA took over Ada conformity assessment, it adopted the ACVC as the basis for its test suite. The ACAA determined to continue to use the same version numbering for the test suite in order to avoid confusion. The version of the ACVC current at the time (2.1) was initially used as ACATS 2.1. Later, the already developed but unreleased ACVC 2.2 was released and used as ACATS 2.2. The ACAA later released ACATS 2.3, ACATS 2.4, ACATS 2.5, and then ACATS 2.6 to include maintenance changes and a few new tests.
This version of the ACATS is version 3.0. As with ACATS 2.3 and later, this version was completely developed under the auspices of the ACAA. ACATS 3.0 contains test programs to check for conformity to new language features defined in ISO/IEC 8652:1995/AMD 1:2007 ([Amend1]), as well as test programs to check for conformity to language features defined in earlier versions of Ada, including [Ada95] and [Ada83]. Subsequent maintenance or enhancement versions of the suite, if they are required, will be numbered 3.1, etc.
The ACATS User's Guide describes the set of ACATS tests and how they are to be used in preparation for conformity assessment. The formal procedures for conformity assessment are described in [Pro01], and the rules in that document govern all conformity assessments, notwithstanding anything in this document that may be interpreted differently. Moreover, this guide does not discuss specific requirements on processing of the ACATS test suite, or submission and grading of results that an Ada Conformity Assessment Laboratory (ACAL) may impose.
The User's Guide is intended to be used by compiler implementers, software developers who maintain a version of the ACATS as a quality control or software acceptance tool, and third-party testers (e.g., Ada Conformity Assessment Laboratories).
Section 2 of the User's Guide for ACATS 3.0 summarizes the changes between ACATS 2.6 and ACATS 3.0. Section 3 describes test objectives and their relationship to ACATS tests and to the rules of the Ada Standards documents. Section 4 describes the configuration of the ACATS, including a description of the ACATS software and delivery files. Section 5 provides step-by-step instructions for installing and using the test programs and test support packages, and for grading test results. The appendices include other information that characterizes the ACATS 3.0 release, along with information on test construction.
Refer to Annex F and Section 5.6 for the definition of an acceptable result and the rules for grading ACATS 3.0 test program results. Section 5.7.2 provides instructions for submitting a petition against a test program if a user believes that a deviation from the acceptable results for a given test program is in fact conforming behavior.
The ACATS test suite is available from any ACAL and from the Ada Information Clearinghouse (sponsored by the ARA). See http://www.adaic.org/compilers/testing.html.

Contents   Index   References   Next