Skip to content

ci(stryker): use coverage-analysis all in mutation run - #448

Merged
samtrion merged 1 commit into
mainfrom
ci/stryker-coverage-analysis-all
Oct 30, 2026
Merged

samtrion merged 1 commit into
mainfrom
ci/stryker-coverage-analysis-all

Conversation

@samtrion

Copy link
Copy Markdown
Contributor

Problem

With coverage-analysis: perTest, Stryker 5.0.0 (solution mode, MTP runner) captures per-test coverage over the pool-wide set of discovered test assemblies for every mutated project.

Change

stryker-config.json: "coverage-analysis": "perTest" -> "all" (one initial test run per project, covered mutants run against all tests of the project). Same change as dailydevops/extensions.test#1542.

Before / After

Both runs are single workflow_dispatch runs of mutation.yml (before: main, after: this branch), so timing noise of several seconds applies.

metric before (perTest) after (all)
Time Elapsed 04:10 03:39
mutation score 93.21 % 95.92 %

Risk

  • Measured with one run per side.
  • On large solutions all is not safe: in pulse and healthchecks the run finished testing and then failed with OutOfMemoryException while serializing the JSON report; frameshift was cancelled. This repo is small, so it is not affected so far.
  • A higher score can mean mutants that perTest wrongly reported as NoCoverage are now tested; it is not necessarily a stronger test suite.

@codecov

codecov Bot commented Oct 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.15%. Comparing base (1326621) to head (f231657).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #448   +/-   ##
=======================================
  Coverage   93.15%   93.15%           
=======================================
  Files          15       15           
  Lines         336      336           
  Branches       99       99           
=======================================
  Hits          313      313           
  Misses         12       12           
  Partials       11       11           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@samtrion
samtrion merged commit 293295c into main Oct 30, 2026
13 checks passed
@samtrion
samtrion deleted the ci/stryker-coverage-analysis-all branch October 30, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant