S4 Suite Testing – Suggestions and Approaches

Zusammenfassung:Since the S4 Program has evolved from a standalone product (NABK) to a whole software suite (NAS4 - NATO Armaments Sub-Group 2 Sharable Software Suite) uniting multiple products, ensuring the quality of the S4 software had become more difficult. An extensive amount of software tests were written during the software development to verify this quality. Thereby nearly all tests are situated at the product level and below. Because the software is designed highly modular and thus configurable performing tests on the whole suite in a certain configuration is an time extensive job. As well, despite the big amount of performed tests, software problem reports increase leading to a overflow of the developers workload. So a main goal is to detect leaks in current testing by also increasing the effectiveness due to the avoidance of test case overlapping during the development of new test cases. The right instrument to realize that is to assess the code coverage of the current tests. It is suggested to develop a tool allowing easily to assess code coverage of the S4 that is fitted and applicable for the existing S4 Autotest framework in which most tests are run. With an included configuration management this new tool should allow to buildspecific test profiles for the different suite configurations. A well structured preparation of the coverage results according to the suite structure saved in the Extensbile Markup Language (XML) format should allow an easy inspection.