DO-254: Increasing verification coverage by test

Verification coverage by test is essential to satisfying both the objectives of DO-254 and interpretation in FAA Order 8110.105. However, verification of requirements by test during final board testing is challenging and time-consuming in most cases. This whitepaper explains the reasons behind these challenges, and provides recommendations how to overcome them. The recommendations center around Aldecís unique device testing methodology that can significantly increase verification coverage by test.

Introduction

DO-254/ED-80 is a means of compliance to aviation regulations for all airborne electronic hardware classified as custom micro-coded devices such as FPGAs, ASICs and PLDs [1]. These devices are often as complex as software controlled microprocessor-based systems, therefore, they need a stringent development approach to satisfy airworthiness requirements. The main purpose of the guidance is to ensure that the device built meets the requirements and safely performs all intended functions under normal and abnormal operating conditions [2]. In order to obtain compliance, the applicant must implement the stringent development and verification process of DO-254, and satisfy the underlying objectives at the device level. The hardware design life cycle and the data generated in the process are for and specific to the device itself.

Device verification by test for DO-254 compliant designs is critical in proving that the device meets the requirements. But testing the device at the board level is quite challenging and in most cases restrictive in verifying specific requirements. In this whitepaper the reasons behind these challenges are investigated and recommendations how to overcome them are provided. Central to the recommendations provided is Aldecís unique approach of testing the target device in isolation prior to final board testing. The key benefits of deploying such an approach are also presented.

Verification process

The verification process is vital and a key element to proving that the device built meets the requirements. The verification process provides a technical assessment of correctness of the design against the requirements. The key factor in formulating an efficient verification plan is to understand the purpose, objectives and activities of the verification process as defined in the guidance.

The purpose of the verification process is to provide assurance that the device meets the requirements [3]. The applicant must satisfy the objectives at the device level such that the device implementation meets the requirements. The verification activities can be a combination of reviews, analyses and tests as defined by the applicant in the verification plan and they take place during all phases of the hardware life cycle. It is important to distinguish that simulation is considered as analysis, and not test. Test is a functional test on the device itself that confirms correct operations in response to a series of stimuli [4].

The verification process as defined in DO-254 includes verification of the design and verification of the implementation. The verification of the design description is satisfied by review and analysis of the design at the Hardware Description Language (HDL) level [5]. The verification of the implementation is satisfied by verification after Place and Route and verification of the device itself [6].

For the Design Under Test (DUT) at the HDL level, verification is done via HDL coding standard reviews, functional simulation and elemental analysis (which can be code coverage analysis when done correctly). For the DUT at the post-layout level, verification is done via dynamic timing simulation and/or static timing analysis. For the verification of the DUT within the component itself, verification is done via device testing. It is important to remember that testing the device or component itself is the real test of the design, and applicants are required to verify all FPGA requirements by test in order to comply with DO-254 and meet the criteria found in FAA Order 8110.105.

All of the outputs from the verification activities performed at these three levels must meet the requirements. Hence, the results from functional simulation, post-layout timing simulation and device testing must meet the requirements.

For functional and timing simulation it is quite easy to exercise or stimulate the device inputs and capture the results at the outputs of the DUT. This type of verification can easily be achieved with the help of industry-leading simulators. However for hardware device testing, this is not so easy, and in fact itís quite challenging to verify the same requirements during device testing.

As described in FAA Order 8110.105, any inability to verify specific requirements by test on the device itself must be justified and alternative means of verification provided [7]. Certification authorities favor verification by test for formal verification because of the simple fact that hardware flies, not simulation models. Requirements describing FPGA pin level behavior must be verified by test.