ci: run the example tests in the pipeline by vbreuss · Pull Request #903 · Testably/Mockolate

vbreuss · 2026-10-18T16:38:33Z

Mockolate.ExampleTests covers realistic usage (file system, Azure SDK clients) but was not part of the UnitTests target, so a change that broke these examples still passed CI.

Changes

Pipeline: UnitTests runs Mockolate.ExampleTests on all its target frameworks (net10.0, net8.0 and net48 on Windows), with a TRX and a Cobertura coverage file per framework like the other unit test projects.
Docs: Docs/agents/workflows.md no longer says that CI skips the example tests and lists them in the UnitTests target.

Verification

./build.ps1 CodeCoverage runs the example tests (19 on net10.0, 19 on net8.0, 14 on net48, all passing) and writes Mockolate.ExampleTests_<tfm>.trx and .cobertura.xml into TestResults.

`Mockolate.ExampleTests` covers realistic usage (file system, Azure SDK clients) but was not part of the `UnitTests` target, so a change that broke these examples still passed CI. ## Changes - **Pipeline**: `UnitTests` runs `Mockolate.ExampleTests` on all its target frameworks (net10.0, net8.0 and net48 on Windows), with a TRX and a Cobertura coverage file per framework like the other unit test projects. - **Docs**: `Docs/agents/workflows.md` no longer says that CI skips the example tests and lists them in the `UnitTests` target. ## Verification - `./build.ps1 CodeCoverage` runs the example tests (19 on net10.0, 19 on net8.0, 14 on net48, all passing) and writes `Mockolate.ExampleTests_<tfm>.trx` and `.cobertura.xml` into `TestResults`.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

sonarqubecloud · 2026-10-18T17:31:54Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions · 2026-10-18T18:43:15Z

Test Results

31 files 31 suites 36m 44s ⏱️
4 533 tests 4 533 ✅ 0 💤 0 ❌
28 808 runs 28 808 ✅ 0 💤 0 ❌

Results for commit 0ed7a04.

github-actions · 2026-10-18T19:49:51Z

🚀 Benchmark Results

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.5 LTS (Noble Numbat)

Intel Xeon Platinum 8370C CPU 2.80GHz (Max: 2.85GHz), 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.303

[Host] : .NET 10.0.13 (10.0.13, 10.0.1326.50504), X64 RyuJIT x86-64-v4
Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10

Method	N	Mean	Error	StdDev	Ratio	Allocated	Alloc Ratio
baseline*	1	377.9 ns	10.49 ns	9.81 ns	0.95	1.93 KB	1.00
Mockolate	1	399.0 ns	10.13 ns	8.98 ns	1.00	1.93 KB	1.00
Imposter	1	594.8 ns	16.97 ns	15.87 ns	1.49	3.96 KB	2.05
TUnitMocks	1	564.4 ns	7.27 ns	6.45 ns	1.41	2.02 KB	1.05
Moq	1	137,031.0 ns	303.72 ns	269.24 ns	343.58	14.74 KB	7.64
NSubstitute	1	6,084.0 ns	31.83 ns	26.58 ns	15.25	9.12 KB	4.72
FakeItEasy	1	6,480.9 ns	18.61 ns	17.41 ns	16.25	8.06 KB	4.18

baseline*	10	684.1 ns	19.95 ns	18.66 ns	0.80	2.14 KB	1.00
Mockolate	10	856.7 ns	8.23 ns	7.69 ns	1.00	2.14 KB	1.00
Imposter	10	980.6 ns	25.46 ns	22.57 ns	1.14	4.8 KB	2.24
TUnitMocks	10	1,772.1 ns	11.70 ns	10.95 ns	2.07	3.73 KB	1.74
Moq	10	142,944.5 ns	516.95 ns	431.68 ns	166.86	18.64 KB	8.71
NSubstitute	10	8,879.7 ns	19.75 ns	16.49 ns	10.37	12.07 KB	5.64
FakeItEasy	10	9,845.2 ns	75.99 ns	67.36 ns	11.49	15.42 KB	7.20

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.5 LTS (Noble Numbat)

INTEL XEON PLATINUM 8573C 2.26GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.303

[Host] : .NET 10.0.13 (10.0.13, 10.0.1326.50504), X64 RyuJIT x86-64-v4
Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10

Event	Mean	Error	StdDev	Ratio	Allocated	Alloc Ratio
baseline*	169.63 ns	4.339 ns	4.059 ns	0.62	1.7 KB	1.00
Mockolate	272.3 ns	7.86 ns	7.35 ns	1.00	1.7 KB	1.00
Imposter	713.3 ns	12.51 ns	11.70 ns	2.62	7.18 KB	4.22
TUnitMocks	165.4 ns	1.61 ns	1.26 ns	0.61	1.34 KB	0.79
Moq	12,355.3 ns	268.61 ns	251.26 ns	45.40	12.51 KB	7.34
NSubstitute	4,644.1 ns	107.31 ns	100.38 ns	17.07	9.05 KB	5.31
FakeItEasy	119,771.8 ns	3,022.75 ns	2,827.48 ns	440.12	15.22 KB	8.94

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.5 LTS (Noble Numbat)

AMD EPYC 7763 2.45GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.303

[Host] : .NET 10.0.13 (10.0.13, 10.0.1326.50504), X64 RyuJIT x86-64-v3
Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10

Callback	Mean	Error	StdDev	Ratio	Allocated	Alloc Ratio
baseline*	215.7 ns	5.42 ns	4.81 ns	0.68	1.57 KB	1.00
Mockolate	317.0 ns	6.99 ns	6.20 ns	1.00	1.57 KB	1.00
Imposter	422.4 ns	13.07 ns	11.59 ns	1.33	2.37 KB	1.51
TUnitMocks	495.0 ns	5.88 ns	4.91 ns	1.56	2 KB	1.27
Moq	98,715.2 ns	434.07 ns	384.79 ns	311.49	8.88 KB	5.66
NSubstitute	4,794.2 ns	55.97 ns	46.74 ns	15.13	7.71 KB	4.91
FakeItEasy	4,796.0 ns	98.71 ns	87.50 ns	15.13	6.81 KB	4.33

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.5 LTS (Noble Numbat)

AMD EPYC 9V74 2.60GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.303

[Host] : .NET 10.0.13 (10.0.13, 10.0.1326.50504), X64 RyuJIT x86-64-v4
Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10

Indexer	N	Mean	Error	StdDev	Ratio	Allocated	Alloc Ratio
baseline*	1	921.1 ns	22.93 ns	21.44 ns	1.25	3.77 KB	1.00
Mockolate	1	734.2 ns	11.84 ns	9.89 ns	1.00	3.77 KB	1.00
Imposter	1	627.8 ns	7.39 ns	6.17 ns	0.86	4.99 KB	1.33
Moq	1	135,241.3 ns	883.52 ns	826.45 ns	184.22	20.37 KB	5.41
NSubstitute	1	7,432.6 ns	119.74 ns	112.00 ns	10.12	12.84 KB	3.41
FakeItEasy	1	7,673.5 ns	107.80 ns	100.84 ns	10.45	13.63 KB	3.62

baseline*	10	2,396.3 ns	12.49 ns	11.68 ns	1.28	4.82 KB	1.00
Mockolate	10	1,875.4 ns	3.40 ns	3.18 ns	1.00	4.82 KB	1.00
Imposter	10	1,197.9 ns	8.65 ns	7.67 ns	0.64	6.47 KB	1.34
Moq	10	142,064.6 ns	367.20 ns	325.51 ns	75.75	29.89 KB	6.20
NSubstitute	10	18,879.8 ns	149.92 ns	140.23 ns	10.07	25.63 KB	5.32
FakeItEasy	10	16,194.5 ns	91.76 ns	85.84 ns	8.64	32.97 KB	6.84

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.5 LTS (Noble Numbat)

INTEL XEON PLATINUM 8573C 2.30GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.303

[Host] : .NET 10.0.13 (10.0.13, 10.0.1326.50504), X64 RyuJIT x86-64-v4
Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10

CreateMock	Mean	Error	StdDev	Ratio	Allocated	Alloc Ratio
baseline*	20.91 ns	1.009 ns	0.944 ns	0.86	160 B	1.00
Mockolate	24.31 ns	0.579 ns	0.542 ns	1.00	160 B	1.00
Imposter	357.25 ns	2.617 ns	2.320 ns	14.70	2240 B	14.00
TUnitMocks	44.52 ns	0.359 ns	0.300 ns	1.83	200 B	1.25
Moq	1,195.30 ns	6.015 ns	5.332 ns	49.20	2096 B	13.10
NSubstitute	1,966.64 ns	17.083 ns	15.144 ns	80.95	5048 B	31.55
FakeItEasy	1,459.45 ns	11.580 ns	10.265 ns	60.07	2759 B	17.24

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.5 LTS (Noble Numbat)

AMD EPYC 7763 2.60GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.303

[Host] : .NET 10.0.13 (10.0.13, 10.0.1326.50504), X64 RyuJIT x86-64-v3
Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10

Property	N	Mean	Error	StdDev	Ratio	Allocated	Alloc Ratio
baseline*	1	593.3 ns	11.87 ns	11.10 ns	1.01	2.41 KB	1.00
Mockolate	1	585.7 ns	13.06 ns	12.21 ns	1.00	2.41 KB	1.00
Imposter	1	443.8 ns	15.21 ns	14.23 ns	0.76	2.98 KB	1.23
TUnitMocks	1	518.3 ns	6.81 ns	6.04 ns	0.89	1.64 KB	0.68
Moq	1	11,982.1 ns	42.98 ns	38.10 ns	20.47	10.27 KB	4.25
NSubstitute	1	8,130.0 ns	49.04 ns	45.87 ns	13.89	11.45 KB	4.74
FakeItEasy	1	8,669.7 ns	53.90 ns	47.78 ns	14.81	11.24 KB	4.66

baseline*	10	959.2 ns	15.07 ns	14.09 ns	0.85	2.91 KB	1.00
Mockolate	10	1,124.0 ns	14.33 ns	13.41 ns	1.00	2.91 KB	1.00
Imposter	10	697.9 ns	8.87 ns	8.30 ns	0.62	3.26 KB	1.12
TUnitMocks	10	1,729.9 ns	10.31 ns	9.65 ns	1.54	3.94 KB	1.35
Moq	10	18,814.7 ns	88.76 ns	78.68 ns	16.74	17.03 KB	5.86
NSubstitute	10	17,972.0 ns	216.70 ns	202.70 ns	15.99	21.08 KB	7.25
FakeItEasy	10	20,263.0 ns	172.14 ns	152.60 ns	18.03	30.81 KB	10.60

baseline* rows show the corresponding Mockolate benchmark from the most recent successful main branch build with results, for regression comparison.

github-actions · 2026-10-18T19:55:37Z

👽 Mutation Results

Mockolate

Details

File	Score	Killed	Survived	Timeout	No Coverage	Ignored	Compile Errors	Runtime Errors	Total Detected	Total Undetected	Total Mutants

The final mutation score is NaN%

Coverage Thresholds: high:80 low:60 break:0

…ne (#903) by Valentin Breuß

vbreuss self-assigned this Oct 18, 2026

Copilot AI balanced review requested due to automatic review settings October 18, 2026 16:38

Copilot AI reviewed Oct 18, 2026

vbreuss merged commit 4ceee94 into main Oct 18, 2026
18 checks passed

vbreuss deleted the ci/run-example-tests branch October 18, 2026 23:23

github-actions Bot added a commit that referenced this pull request Oct 19, 2026

Update benchmark for 4ceee94: ci: run the example tests in the pipeli…

8b2ba08

…ne (#903) by Valentin Breuß

github-actions Bot added a commit that referenced this pull request Oct 19, 2026

Update benchmark for 4ceee94: ci: run the example tests in the pipeli…

ecc8cc5

…ne (#903) by Valentin Breuß

Navigation Menu

Repository navigation

ci: run the example tests in the pipeline - #903

ci: run the example tests in the pipeline#903
vbreuss merged 1 commit into
mainfrom
ci/run-example-tests

vbreuss commented Oct 18, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

sonarqubecloud Bot commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

vbreuss commented Oct 18, 2026

Changes

Verification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud Bot commented Oct 18, 2026

Quality Gate passed

Uh oh!

github-actions Bot commented Oct 18, 2026

Test Results

Uh oh!

github-actions Bot commented Oct 18, 2026

🚀 Benchmark Results

Uh oh!

github-actions Bot commented Oct 18, 2026

👽 Mutation Results

Mockolate

The final mutation score is NaN%

Coverage Thresholds: high:80 low:60 break:0

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants