Fault Coverage and Diagnosis of Protocols and Systems Modeled As Extended Finite State Machines

Abstract

Automatic test derivation from formal specifications offers a rigorous discipline to functional conformance testing. In various application domains, such as communication protocols and other reactive systems, the specification can be represented in the form of an Extended Finite State Machine (EFSM). Many methods can be used for deriving test suites from an EFSM specification. In practice, developing and applying these test suites to an Implementation Under Test (IUT) is time consuming and costly. Thus, it is desirable to determine high quality test suites in order to reduce the cost of testing. To this end, in the first part of this thesis, using six realistic application examples, we conduct experiments, assess, determine the fault coverage, and accordingly rank various known types of EFSM-based test suites. While the purpose of conformance testing is to check if an IUT is different from its specification, an interesting, complementary, yet more complex step, is called fault diagnosis or diagnostic testing. The objective of fault diagnosis is to determine the faulty implementation, and thus find the differences between the specification and its implementation. In the second part of this thesis, we present a diagnostic method, conduct experiments, and assess the fault localization capabilities of the EFSM-based test suites considered in the first part of the thesis. The fault localization capability of a test suite is determined for many types of diagnostic candidates, representing possibly faulty EFSM implementations, such as candidates with single or double transfer faults, candidates with single assignment faults, and many other types of candidates. In addition, for each considered test suite, the method determines the diagnostic tests required, in addition to the considered test suite, for locating a faulty EFSM IUT.