We are all heavily dependent on software in our daily life. Software control our cars, phones, bank accounts etc. Failures are frequent and sometimes catastrophic, for example the recent plane crash. Software testing aims to reveal faults before the software comes into use. Mutation testing is one of the most effective software testing techniques.
Unfortunately it is computationally expensive because it creates many versions of the software (mutants) that all need to be analyzed. Our goal in this project is to reduce the number of mutants to be analyzed by one or two magnitudes while maintaning the effectiveness of full mutation analysis.
Software testing is crucial
Software is increasingly crucial to our civilization and software developers depend on effective software engineering techniques to achieve, assess, and maintain high quality software. A principle activity to achieve quality is software testing, where tests are designed to meet goals (test requirements) based on software artifacts such as functional requirements, structural code coverage, and certain types of faults.
Mutation testing as effective testing technique
One of the most effective testing techniques is mutation testing, which creates many slightly modified (mutated) versions of the program under test, then asks the engineer to design tests that cause each mutated program to fail (killing the mutant).
The problem with too many mutants
Currently, however, mutation testing systems create too many mutants for efficient use. For example, thousands of mutants might be created for a 30-statement Java method. This creates both computational and human expense, as human effort is needed during mutation testing. Luckily, researchers have found that we normally create many redundant mutants. A mutant mi is redundant if any test that kills another mutant, say mj, is guaranteed to also kill mi. Recent work has found that at least 90%, and sometimes up to 99% of mutants are redundant. The costs of excessive redundancy in mutation have been documented in two common applications, test case design and the empirical validation of other testing techniques.
Finding minimal sets of mutants
Thus, a major step forward in the usefulness of mutation testing, and therefore testing in general, would be the ability to eliminate redundant mutants. This project seeks to develop algorithms and techniques to reduce redundancy among mutants with the ultimate goal of finding minimal sets of mutants. Removing redundancy will not only reduce the cost of mutation testing, but will also improve the accuracy of the assessments of other testing techniques.