Mutation Testing Explained

Evaluate your test suite quality by introducing code mutations — finding gaps where tests execute code but do not actually verify its correctness.

Mutation Testing

Mutation testing evaluates test suite quality by automatically introducing small changes (mutations) into source code and checking whether tests detect them — surviving mutations indicate gaps in test coverage.

Explanation

Code coverage measures which lines tests execute, but not whether tests actually verify correctness. Mutation testing goes further: it modifies source code (replacing + with -, flipping conditions, removing statements) and re-runs the test suite. If tests pass with the mutation (the mutant "survives"), the tests are not checking the behavior that changed — revealing a coverage gap. If tests fail (the mutant is "killed"), the tests properly verify that code. The mutation score (killed / total mutants) provides a more meaningful quality metric than line coverage. Tools include Stryker (JavaScript/TypeScript), PITest (Java), and mutmut (Python).

Bookuvai Implementation

Bookuvai uses Stryker for mutation testing on critical business logic modules. We run mutation tests periodically (not on every commit due to runtime) to identify test suite weaknesses. High-value code paths (payment, authentication, authorization) are prioritized for mutation analysis.

Key Facts

  • Introduces small code changes and checks if tests catch them
  • Surviving mutants reveal gaps in test effectiveness, not just coverage
  • Mutation score: percentage of mutants killed by the test suite
  • More meaningful than line coverage for measuring test quality
  • Tools: Stryker (JS/TS), PITest (Java), mutmut (Python)

Related Terms

Frequently Asked Questions

Is mutation testing slow?
Yes — it re-runs the test suite for every mutation, which can mean thousands of test runs. Optimizations include incremental mutation (only mutate changed code), parallel execution, and focusing on critical modules. Run mutation tests periodically, not on every commit.
What is a good mutation score?
Aim for 80%+ mutation score on critical business logic. 100% is not practical — some mutations (e.g., changing log messages) do not warrant tests. Focus on achieving high scores for code that handles money, security, and core business rules.
How is mutation testing different from code coverage?
Code coverage measures which lines are executed during tests. Mutation testing measures whether tests actually verify behavior. You can have 100% coverage but a low mutation score if tests run code without meaningful assertions.