Research Seminar - August 30, 2000

Abstract

Over the software life cycle - from conceptualisation to definition phases through to leave-in operational system phase - testing alone occupies 50% of the entire life cycle time. However, software testing is a difficult and incomplete process. Indeed the famous maxim says, "Testing confirms the presence of errors, not their absence -:)".

This talk will present the design of a software system for generating test data for a broad class of programs. The key feature of this system is that the problem of test data generation is treated entirely as a numerical optimisation problem and, as a consequence, this method does not suffer from difficulties commonly found in symbolic execution systems. Instead program instrumentation is used to solve a set of path constraints without explicitly knowing their form. The system supports not only the generation of integer and real data types, but also non-numerical discrete types such as characters and enumerated types. (Several key theorems in testing have also been developed and proved). The system has also been extended to incorporate domain testing strategy. Furthermore, the system provides a number of useful metrics and visualisation capabilities to large software systems (including objected-oriented based software systems). The system has been tested on large programs (60,000 lines of code) and found to reduce the effort required to test programs as well as provide an increase in test coverage. Some of the actual operational results will also be presented briefly.