Mutineer
Find bugs your tests would miss. Mutineer changes your Ruby code and runs your Minitest or RSpec tests. Changes that still pass reveal possible test gaps. This is mutation testing.
Features
- Minitest and RSpec. Detected from test filenames; override with
--framework. - Rails support.
--railsboots once. With--daemon, each Minitest worker gets its own SQLite file, PostgreSQL, or MySQL 8 test database. - Focused tests. Runs tests that reach each changed line, likely and fast tests first. Stops at the first failure by default.
- PR checks.
--since origin/mainchecks changed lines;--baselinegates new survivors, and score drops on full runs. When tests change, run a full scan (--no-since). - CI and agents. GitHub Action, versioned JSON, stable mutant ids, clear exit codes, and zero runtime gem dependencies.
📖 davidteren.github.io/mutineer →: overview, operators, and usage.
Quick start
Plain Ruby:
gem install mutineer
mutineer run lib/discount.rb --test test/discount_test.rb
Rails: add to your Gemfile test group:
gem "mutineer", group: :test, require: false
Install, create .mutineer.yml, and check one model:
bundle install
RAILS_ENV=test bundle exec mutineer init --rails
RAILS_ENV=test bundle exec mutineer run app/models/order.rb --rails
Without --test, app/models/order.rb pairs with test/models/order_test.rb or spec/models/order_spec.rb.
A concern at app/models/concerns/storage/totaled.rb pairs with test/models/concerns/storage/totaled_test.rb or an unclaimed split test such as test/models/concerns/storage/totaled_bytes_test.rb.
With no test there, it pairs with test/models/storage/totaled_test.rb, an unclaimed split test such as test/models/storage/totaled_bytes_test.rb, test/models/storage/test_totaled.rb, or spec/models/storage/totaled_spec.rb.
See Quick start.
Documentation
| Guide | What it covers |
|---|---|
| Quick start | Install, run, fix a survivor |
| First run to steady state | One file through changed-code CI |
| Reading results | Reports, survivors, suppression, mutant ids, kill matrix |
| Set up Rails, the daemon, and databases | Rails, daemon, databases, Spring, Bootsnap, Ruby < 3.4 |
| Roll out on a Rails app | Four stages from model to CI gate |
| CI: changed code and baselines | --since, --baseline, Action inputs, timeouts |
| CLI reference | Flags, defaults, .mutineer.yml |
| Comparison | Mutineer, Mutant, other Ruby tools, fit-check prompt |
| AI agents and CI | Agent loop and CI recipes |
| JSON schema | Versioned report contract |
| STABILITY.md | Release guarantees |
Is it production-ready?
Check your database and framework before adding a CI gate.
Mutineer requires Ruby 3.4 or later. In-process runs require Linux or macOS for forks.
Plain Ruby projects without databases support parallel runs.
In the Status column, supported means parallel runs work and serial only runs one mutant at a time.
| Ruby | Database | Framework | Parallelism | Status |
|---|---|---|---|---|
| 3.4 or later | SQLite file | Minitest | supported with --daemon |
supported |
| 3.4 or later | SQLite file | RSpec | not supported | serial only |
| 3.4 or later | PostgreSQL | Minitest | supported with --daemon |
supported |
| 3.4 or later | PostgreSQL | RSpec | not supported | serial only |
| 3.4 or later | MySQL 8 | Minitest | supported with --daemon |
supported |
| 3.4 or later | MySQL 8 | RSpec | not supported | serial only |
| 3.4 or later | MariaDB | Minitest | not tested | not tested |
| 3.4 or later | MySQL 9 or later | Minitest | not tested | not tested |
| 3.4 or later | SQLite in memory | Minitest | not supported | serial only |
| 3.4 or later | SQLite in memory | RSpec | not supported | serial only |
| 3.4 or later | more than one database | Minitest | one worker | serial only |
| 3.4 or later | more than one database, Rails 7.1 or later, the extra ones in shared_databases |
Minitest | supported with --daemon |
supported, unless the test environment registers a pool for a shared database |
| 3.4 or later | more than one database | RSpec | not supported | serial only |
This matrix is for a Rails app on Ruby 3.4 or later. A plain Ruby project
with no database runs in parallel with either framework.
not supported means --daemon cannot run that cell; not tested means no CI
job runs it. SQLite file means a database file, not :memory:.
--railswithout--daemonis serial on every row.--daemonis Minitest only.- MySQL older than 8.0 stops
--daemon. MariaDB and MySQL 9 or later warn and run. - An app with more than one database runs one worker:
--daemonisolates only the primary connection. On Rails 7.1 or later, list the databases your tests never use inshared_databases, and the workers run in parallel. CI tests that row on SQLite. See More than one database. - Postgres worker databases are copied once at the start; MySQL 8 worker databases are created then and the rows copied.
- A test that commits rows outside a transaction leaves them in that worker's database, and the next mutant on that worker sees them. Rails parallel tests have the same limit.
- RSpec: the first failed example with an Active Record connection or lock
error scores
errored, on the same classes as Minitest (#347). On Rails 8,ConnectionNotDefinedis a code error and still scoreskilled. On Rails 7.2 that case raisesConnectionNotEstablishedand scoreserrored. Other statement errors still scorekilled.
Mutineer sets PARALLEL_WORKERS=1, so a parallelize call in test_helper.rb starts no Rails workers during a mutant run.
See Rails parallel tests, Spring, Bootsnap, and Redis.
Older Ruby apps use --test-command; Mutineer still requires Ruby 3.4 or later.
This path uses one Mutineer worker and no coverage-based test selection by default; a wrapper can re-enable Rails test workers by setting PARALLEL_WORKERS.
| Subprocess result | Score |
|---|---|
| Exit 1, normally | killed |
| SimpleCov gate exit with a failing test | killed |
| Other exit code | error |
| Crash before a test fails | Can score error |
| Only a SimpleCov gate fails, with its gate message and a passing summary | survived |
See Apps on Ruby < 3.4 and release guarantees in STABILITY.md.
What is mutation testing?
Mutation testing changes code to check whether your tests notice.
Each changed copy is a mutant, such as replacing >= with > or && with ||.
| Result | Meaning |
|---|---|
| Killed | A test failed and caught the change. |
| Survived | All tests passed. Inspect the change for a test gap. |
A small example
# lib/discount.rb
class Discount
def rate(qty)
return 0.1 if qty >= 10
0
end
end
Tests check quantities 1 and 50: 100% line coverage. Mutineer changes >= to >; both tests pass:
lib/discount.rb
Discount#rate (discount.rb:3)
Operator: comparison (>= -> >)
- return 0.1 if qty >= 10
+ return 0.1 if qty > 10
No test checks exactly 10. Testing 10 kills the mutant. Run the example, tests and report in Mutation testing, explained.
Coverage and mutation testing
| Check | What it shows |
|---|---|
| Line coverage | Which lines ran |
| Mutation testing | Whether tests detect changes to those lines |
Tests without assertions add coverage but kill mutants only when changes cause errors. This gap is common in rushed tests and tests written by AI agents.
How it works
- Parse. Prism, included in Ruby 3.4, parses source files. Mutineer uses only Prism and the standard library.
- Mutate. Operators make one change per mutant. Mutineer parses each result again and skips invalid Ruby.
- Map coverage and check unchanged tests. Each test file runs with coverage to map lines to tests.
.mutineer/coverage.jsoncaches the map. A method-body edit with no added or removed line recaptures the tests that cover that source; an added or edited test file recaptures that test. A class-body edit, a load-time method, a--requirefile, or a project file the tests load recaptures every test. A fresh capture runs every test alone; a failure stops the run. With a cached map, only the tests that cover the sources run again, alone and together, because a mutant runs only those tests. In a standalone run, a line that runs only while the source loads does not count as coverage of every test. The conventionally paired test still covers that line. Atests:map entry does not count. With no conventionally paired test in the run, every test still covers it. - Run each mutant in a fork. The parent loads code once, including the app with
--rails. A source that needs other files loaded first can use--bootorboot:, or be listed after the file that loads them. Without a boot file,require:files load after the sources. Children run in parallel on Linux and macOS. Each loads the changed file (reload) or replaces the changed method (redefine). Only covering test files run: paired tests first, then fastest to slowest. In-process runs stop at the first failure unless--matrixrequests every covering test. The clean check and each mutant run share one seed: Minitest usesSEEDor1, and a random RSpec order uses seed1unless the suite sets one. See the CLI reference. - Score. A failing test kills the mutant; all tests passing means it survived.
Score:
killed / (killed + survived). In coverage-guided runs, timeouts, errors, and uncovered lines count as neither;--test-commandhas no coverage, so a mutant no test reaches survives.
| Alternative backend | Changes to step 4 |
|---|---|
--daemon |
Boots Rails once in a persistent process; isolates each worker's test database |
--test-command |
Runs tests in another Ruby subprocess; writes mutants to the real file, then restores it. Run the tests in a fixed order (SEED=1 mutineer run ... for Minitest, --order defined for RSpec, with no --order rand in SPEC_OPTS) |
| At a glance | |
|---|---|
| Parser | Prism, with no runtime gem dependencies |
| Ruby | 3.4 or later; older apps use --test-command |
| Operators | 22 (5 by default, 17 opt-in) |
| Isolation | One fork per mutant on Linux and macOS |
| Report formats | human, JSON (versioned schema), HTML |
Usage
mutineer run <source...> [--test <test>...] [options]
mutineer init [--rails] [--force]
mutineer triage REPORT.json --reason TEXT (--id ID ... | --all)
mutineer migrate <source...> [--dry-run] [--operators LIST]
mutineer config cache_dir|baseline [run options]
mutineer triage REPORT.json --reason TEXT prints ignore: entries and does not edit files.
--id ID selects one survivor. Repeat --id for several. --all selects every survivor.
A copy id selects that survivor and every id in its copies.
Paste every entry. An entry ignores only its own id.
migrate --dry-run lists ignore: entries that match no mutant in the sources and operators you pass. It writes nothing and exits 1 when any entry matches nothing. Run it over every source. run warns once for each such entry in that run, lists the ids under summary.unmatched_ignores, and still exits 0. --since does not warn about an entry whose mutant sits outside the diff. A run of some files warns only about entries those files and operators do not match. A source you passed that has no paired test is skipped, but its mutants still count for ignore matching, so run and run --dry-run agree; they stay out of the score and cannot fail --threshold (see unpaired counts). --only skips the check. See Mutant ids.
Options
Full flags, defaults, and .mutineer.yml keys: CLI reference.
| Flag | Meaning | ||
|---|---|---|---|
--test PATH |
Test file, directory, or quoted glob. Repeat the flag. Omit it to pair tests by convention and apply any tests: map. Unmatched sources are skipped and counted in the report. |
||
--threshold FLOAT |
Exit 1 when the score is below FLOAT, or when nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted (default: 0 = off) | ||
--since REF |
Mutate only lines changed since git REF, such as origin/main |
||
--baseline FILE |
Exit 1 for new survivors, or score drops on full runs, against a prior JSON report from a full run | ||
--rails |
Boot config/environment once; serial without --daemon |
||
--boot FILE |
Require this file once before the sources. An app entry point, or a gem's lib/<name>.rb. require: files load after this file |
||
--[no-]daemon |
Enable or disable daemon; isolated worker databases, Minitest only | ||
--jobs N |
Worker count; default: one per CPU | ||
| `--format human\ | json\ | html` | Report format; default: human |
--dry-run |
List mutants without running tests. With --format json, print them with their ids |
Exit codes
| Code | Meaning |
|---|---|
0 |
Score ≥ threshold (or no gate) and no baseline regression. |
1 |
Score below --threshold, OR nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted, OR a --baseline regression, OR a runtime error, including a boot, source or require: file that raises while loading (except an in-process --since run with no mutant on a changed line, which checks that those files exist but does not load them). |
2 |
Usage / invalid-flag error, including a mistyped flag, a bad path (such as a missing --boot or require: file), or an unreadable baseline. |
Operators
List with mutineer --list-operators. Enable specific operators with --operators.
| Group | Operators |
|---|---|
| Tier 1 (default) | arithmetic, comparison, boolean_connector, boolean_literal, statement_removal |
| Tier 2 (opt-in) | return_nil, literal_mutation, condition_negation, string_literal, regex, collection_method, safe_navigation, range, negation_removal, chain_link, call_to_receiver, operand_removal, array_literal, condition_true, condition_false, operator_assignment, rescue_body |
The default operators give no mutant to some methods, such as a method whose body is one if with a rescue.
The report counts these methods on its No mutants: line. --dry-run --verbose and the JSON report list them.
A method listed there is not proven tested. For a method with an if, unless, ternary or modifier condition,
add condition_true and condition_false to force its branches. They do not help an empty method or a case.
The --operators list replaces the default set, so name the default operators too:
mutineer run lib --operators arithmetic,comparison,boolean_connector,boolean_literal,statement_removal,condition_true,condition_false
For a rescue clause, add rescue_body. It replaces each rescue body with nil, even a one-statement body,
and turns x rescue y into x rescue nil. return_nil also reaches a method with a def-level rescue:
it gives nil for the final expression of the body (or of the else clause when there is one) and of each rescue clause (#418). A final explicit begin ... end block gets the same treatment (#470).
For AI agents & pipelines
Use versioned JSON, structured exit codes, and runs scoped to changed lines. Mutant ids survive unrelated edits.
Mutant ids and equivalent-mutant suppression moved to Reading results. Apps on Ruby < 3.4 moved to Rails setup.
| Resource | Links and use |
|---|---|
| AI agents & CI recipes | rendered · source: agent loops, CI gates, avoiding equivalent-mutant loops |
| JSON schema reference | rendered · source: --format json schema and versioning |
| Ruby API (YARD) | https://davidteren.github.io/mutineer/api/: shipped classes |
| Agent skill | skills/mutineer/SKILL.md: installation, agent loop, exit codes |
Install the skill with either command:
gh skill install davidteren/mutineer mutineernpx skills add davidteren/mutineer --skill mutineer
Mutineer 1.8.0 and later bundle the skill and offer
Rails Hyperdrive companion installation.
For Rails apps bundling Mutineer, bin/rails hyperdrive:init installs it to .claude/skills/mutineer/SKILL.md.
License
MIT. See LICENSE.