Event

PhD Defense: Assessment and Improvement of the Practical Use of Mutation for Automated Software Testing

  • Conférencier  Thierry Titcheu Chekam

  • Lieu

    Room E004 JFK Building 29 Avenue J.F. Kennedy L-1855 Kirchberg

    LU

Members of the defense committee:

  • Dr. Jacques Klein, Université du Luxembourg, chairman
  • Dr. Mike Papadakis, Université du Luxembourg, vice chairman
  • Prof. Dr. Yves Le Traon, Université du Luxembourg, supervisor
  • Prof. Dr. Serge Demayer, University of Antwerp, Belgium, member
  • Prof. Dr. Paolo Tonella, Università della Svizzera Italiana, Switzerland, member

Abstract:

Software testing is the main quality assurance technique used in software engineering. In fact, companies that develop software and open-source communities alike actively integrate testing into their software development life cycle. Mutation is a testing criterion that gives objectives to software testing by letting testers create tests in order to reveal artificial error (mutants) induced in the program under test. Industrial studies show that practitioners find mutation testing useful. Nonetheless, the adoption of mutation testing in practice is still slow, because of its cost and the existence of cheaper test criteria such as statement coverage. In this dissertation, we tackle this problem by evaluating the effectiveness of mutation for software testing as compared with the most used testing criteria and, by devising techniques to reduce the high cost of mutation, mainly due to the high number of mutants and the manual effort required from practitioners.

Regarding the evaluation of testing criteria effectiveness, we made an empirical study to evaluate the fault revelation effectiveness of test suites with regards to their achieved coverage of test criteria test objectives. Our results show that mutation has higher fault revelation effectiveness than the widely used statement and branch coverage. Thus mutation should be used in practice.

We tackle the cost reduction problem of mutation in two ways:

  • first, we reduce the number of mutants, that need to be analyzed, by leveraging supervised machine learning to learn fault revealing mutants (mutants that lead to real program faults) from existing faulty programs and predict them on the program under test. 
  • Second, we reduce the manual effort needed to create tests that reveal mutants by leveraging symbolic execution to generate tests to kill “stubborn” mutants in an automated and cost-effective manner. Stubborn mutants are mutants hard to reveal (not revealed by existing tests).