When I thought of the word “mutation,” I tend to think of it with negative connotation as something that changes internal structure and can potentially cause problems with the structure later on. However, in software testing, mutations have a positive function. This post will go over how mutation testing works, and some advantages and disadvantages of using it. My reference can be found here: https://www.guru99.com/mutation-testing.html. When going over mutation testing during lecture, I found the subject very interesting, and this website provided great further information about it.
Mutation testing takes a program and changes around various statements (introducing faults) in the source code, so that when running unit testing or other testing suites on the program, the tests written for the original program should fail, if they had already passed before. So, when the mutant program fails the test case, then the mutation is “killed.” If the mutant program still passes the test case, then the mutant “survives,” and the tests should be written differently so that they not only pass the code that is originally written, but they also fail other possible scenarios, whether or not those other scenarios were explicitly written in the program.
There are many different kinds of mutations that can be created and used, and they fall under 3 categories: operand replacement operators (which replaces operands in an expression with another operand or with a constant value, like changing x > y to x > 5), expression modification operators (which modifies an operator in an expression, like changing x > y to x <= y), and statement modification operators (which modifies the logic of the program by changing or deleting different statements, like removing an “else” branch of an if-else conditional).
This method of testing provides several advantages. It can uncover missing test cases that weren’t already included in the test suite. It also points out the different places in source code which may be vulnerable to error. It looks beyond simply statement and branch coverage to find any faults which may be overlooked. This enhances the quality of software testing taking place on the program.
However, there are still some disadvantages of using this testing strategy. Because there are many places in source code where these mutations may take place, it would be tedious to manually insert the faults, and automation is a necessity. Time may also become an issue, because each mutation requires a run through the test suite. Finally, this method would not be appropriate for black-box testing, as it involves directly examining the source code.