What Is Mutation Testing?
Mutation testing measures how good your tests are by deliberately introducing small bugs and checking whether the tests catch them.
Coverage tells you which code ran during tests, but not whether the tests would notice a bug. Mutation testing answers that harder question. It mutates your code, flipping an operator or changing a constant, and reruns the suite. If a test fails, the mutant is "killed"; if all tests still pass, the mutant "survived," exposing a weak spot.
How mutants are made
A mutation tool generates many slightly altered versions of your code, each with one small change: a plus becomes a minus, a less-than becomes a less-than-or-equal, a return value is removed. Each mutant represents a realistic bug a developer might introduce.
Killed versus survived
For each mutant the suite runs again. If at least one test fails, your tests detected the bug and the mutant is killed. If every test still passes, the mutant survived, meaning your tests would not have caught that bug. Survivors point precisely at weak or missing assertions.
The mutation score
The mutation score is the percentage of mutants killed. Unlike coverage, it reflects assertion quality, not just execution. A high mutation score is strong evidence your tests actually verify behavior, which is why it is a more demanding metric.
A quick example
If a tool changes a > to a >= and no test fails, the boundary case is untested, a survivor worth fixing.
if (age > 18) { ... } // original
if (age >= 18) { ... } // mutant: did any test catch it?Mutation testing and CI cost
Mutation testing is expensive: it reruns the suite once per mutant, so it can be orders of magnitude slower than a normal test run. Teams often run it nightly or on a schedule rather than on every push, and parallelize it heavily. Spreading mutants across fast runners is what keeps it feasible.
Key takeaways
- Mutation testing grades tests by seeing if they catch injected bugs.
- Surviving mutants reveal weak or missing assertions.
- It is costly, so run it on a schedule and parallelize across runners.