Comparing the Robustness of POSIX Operating Systems

Published in the Proceedings of FTCS99, 15-18
June 1999, Madison, Wisconsin.

Abstract

Critical system designers are turning to off-the-shelf operating system (OS)
software to reduce costs and time-to-market. Unfortunately, general-purpose
OSes do not always respond to exceptional conditions robustly, either accepting
exceptional values without complaint, or suffering abnormal task termination.
Even though direct measurement is impractical, this paper uses a multi-version
comparison technique to reveal a 6% to 19% normalized rate at which exceptional
parameter values cause no error report in commercial POSIX OS implementations.
Additionally, 168 functions across 13 OSes are compared to reveal common mode
robustness failures. While the best single OS has a 12.6% robustness failure
rate for system calls, 3.8% of failures are common across all 13 OSes examined.
However, combining C library calls with system calls increases these rates to
29.5% for the best single OS and 17.0% for common mode failures. These results
suggest that OS implementations are not completely diverse, and that C library
functions are both less diverse and less robust than system calls.