Recommendations for Mutation Testing as Part of a Continuous Integration Pipeline: With a focus on C++
Ladda ner
Typ
Examensarbete för masterexamen
Program
Publicerad
2022
Författare
ÖRGÅRD, JONATHAN
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Mutation testing aims to judge the quality of a test suite by evaluating the ability
of the tests to detect subtle changes in the code. Altered versions of the code called
mutants are generated by a mutation tool, often a subtle change based on common
syntactic mistakes. Each mutant is then run against the original test suite to see if
the test results change.
This thesis aims to identify recommendations to mitigate the fact that mutation
testing can be notoriously expensive, making it difficult to integrate as part of a
continuous integration (CI) pipeline.
An evaluation of existing mutation testing tools for C++ was done to better understand
the functionality and limitations of the available tools. Simultaneously,
a case study was conducted at Zenseact to identify suggestions on how mutation
testing can best be used within CI. First, a literature review was conducted to examine
observations made by other studies on the application of mutation testing in
practice, and to identify techniques and practices to perform mutation testing effectively.
Next, stakeholder interviews were conducted to assess the developer’s view
of effective mutation testing. The result of the tool evaluation showed that existing
mutation tools for C++ differ in their capabilities, and the case study result showed
that there exist plenty of techniques to effectively integrate mutation testing into
CI from the developers’ perspective.
Based on the result, only the mutation tools Dextool and Mull were recommended for
CI integration. Furthermore, guidelines were developed such as prioritizing mutation
testing on essential areas of the code, performing mutation testing on old code only
when there is free machine time, and that inexperienced developers should be trained
to understand the mutation result better.
Beskrivning
Ämne/nyckelord
computer science , engineering , project , thesis , mutation testing , continuous integration , C++ , case study , dextool , mull