Quality gates

Three thresholds

Each threshold is a percent, decimals allowed, and 0 turns it off. Gooze compares the score as it prints it, rounded to two decimals:

In the calc story from the quick start, the first run scores 84.62%. With a total threshold of 90, the same run fails:

gooze run --threshold-total 90 ./...

Mutation score: 84.62% (11 killed, 2 survived; 13 cached, 0 tested)
Error: mutation score 84.62% is below threshold.total 90%
exit code 3

The score line comes first, then one line for each score that is too low, then exit code 3. The reports are written either way, so the next run still uses the cache. At 80 the run passes:

Mutation score: 84.62% (11 killed, 2 survived; 13 cached, 0 tested)
exit code 0

A file threshold names each file below it:

Mutation score: 84.62% (11 killed, 2 survived)
Error: calc.go: score 84.62% is below threshold.file 90%
exit code 3

A file whose mutations all fail to compile has no score: it never fails the file threshold and doesn't count towards the average. A set without any scored mutation meets every threshold.

Set them once in gooze.yaml

Flags are handy in a pipeline step. To give every run the same gate, locally and in CI, put the thresholds in gooze.yaml:

threshold:  total: 80       # the score of all mutations  average: 75     # the mean of the file scores  file: 50        # the score every file must reach  diff:           # --diff runs only, on the changed lines    total: 100

A flag overrides the file for one run (--threshold-total 0 turns the total off), and so does an environment variable such as GOOZE_THRESHOLD_TOTAL or GOOZE_THRESHOLD_DIFF_FILE. A value outside 0 to 100 is a usage error:

Error: invalid --threshold-total: 101 is out of range: must be between 0 and 100
exit code 2

All keys: configuration. goozels marks a bad threshold in gooze.yaml while you type, and an editor run below a threshold shows as a warning, with the markers refreshed.

Pull requests: a threshold for the changed lines

A pull request run with diff mode scores only the lines it changes. That score deserves its own bar: new code can be held to 100% while the codebase as a whole sits lower. The threshold.diff keys (flags --threshold-diff-total, --threshold-diff-average, --threshold-diff-file) apply only to gooze run --diff, and they replace the plain thresholds there. Every other run, view and merge uses the plain ones.

With this gooze.yaml, main needs 80% and a branch's changed lines need 100%:

threshold:
  total: 80
  diff:
    total: 100

The boundary-tests branch kills every mutant on its changed lines, so its diff run passes the stricter gate:

gooze run --diff main ./...

Mutation score: 100.00% (13 killed, 0 survived; 0 cached, 13 tested; changed lines since main)
Reports of this run are in .gooze-reports/run; add them to the main reports with gooze report merge.
exit code 0

Sharded runs: gate at the merge

A shard tests only its slice, so its score says little about the codebase. A sharded run doesn't check the thresholds; it says so and exits 0:

Mutation score: 83.33% (5 killed, 1 survived; 0 cached, 6 tested)
Thresholds are not checked on a shard; gooze report merge checks them on all the results.
exit code 0

Put the thresholds on gooze report merge in the job that collects the shards. It checks them on all the results:

Mutation score: 84.62% (11 killed, 2 survived)
Error: mutation score 84.62% is below threshold.total 90%
exit code 3

Which commands check them

Exit codes

Gooze exits with one code per outcome, so a pipeline can tell a low score (3) from a broken setup. What each code means and what to do: CLI reference.

Exit codes in your pipeline

Every code but 0 fails a CI step. That is right for errors. Code 3 is different, because the run finished and wrote its reports, so you choose what a low score does. Each CI guide shows how, for its own system:

• GitHub Actions: accept code 3 in the shell, and keep later steps running with if: ${{ !cancelled() }}.
• GitLab CI: show a low score as a warning with allow_failure: exit_codes: [3], and keep the reports with when: always.
• Google Cloud Build: gate in the last step, or record a low score with allowExitCodes: [3].

FAQ