Skip to content

test: verify the public API with aweXpect.Snapshots.PublicApi - #1868

Merged
vbreuss merged 2 commits into
mainfrom
ci/api-tests-with-snapshots
Oct 18, 2026
Merged

vbreuss merged 2 commits into
mainfrom
ci/api-tests-with-snapshots

Conversation

@vbreuss

@vbreuss vbreuss commented Oct 18, 2026

Copy link
Copy Markdown
Member

The public API was enforced by Microsoft.CodeAnalysis.PublicApiAnalyzers, which needs hand-maintained PublicAPI.Shipped.txt/PublicAPI.Unshipped.txt files per target framework, a pragma in generated code and a downgrade of RS0016/RS0017 for local Debug builds.

aweXpect.Api.Tests now compares the public API of every target framework of aweXpect and aweXpect.Core with snapshots written by aweXpect.Snapshots.PublicApi, so a change shows up as readable C# declarations in the pull request and is accepted like any other snapshot (AcceptApiChanges or AWEXPECT_ACCEPT_SNAPSHOTS=all). The analyzer and its files are removed.

As the check is no longer part of the compilation, the ApiChecks target and the API tests job are back in the CI and build workflows, the release jobs wait for it and Mergify requires it for Dependabot merges. Under CoreOnly the check runs on the Debug build, the only one of aweXpect that references the changed aweXpect.Core.

The public API was enforced by Microsoft.CodeAnalysis.PublicApiAnalyzers, which needs hand-maintained `PublicAPI.Shipped.txt`/`PublicAPI.Unshipped.txt` files per target framework, a pragma in generated code and a downgrade of RS0016/RS0017 for local Debug builds.

`aweXpect.Api.Tests` now compares the public API of every target framework of `aweXpect` and `aweXpect.Core` with snapshots written by `aweXpect.Snapshots.PublicApi`, so a change shows up as readable C# declarations in the pull request and is accepted like any other snapshot (`AcceptApiChanges` or `AWEXPECT_ACCEPT_SNAPSHOTS=all`). The analyzer and its files are removed.

As the check is no longer part of the compilation, the `ApiChecks` target and the `API tests` job are back in the CI and build workflows, the release jobs wait for it and Mergify requires it for Dependabot merges. Under `CoreOnly` the check runs on the Debug build, the only one of `aweXpect` that references the changed `aweXpect.Core`.
@vbreuss vbreuss self-assigned this Oct 18, 2026
@vbreuss
vbreuss enabled auto-merge (squash) October 18, 2026 08:06
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

Test Results

     92 files  +1       92 suites  +1   2h 45m 49s ⏱️ + 21m 9s
 38 387 tests +2   38 386 ✅ +1  1 💤 +1  0 ❌ ±0 
223 504 runs  +7  223 503 ✅ +6  1 💤 +1  0 ❌ ±0 

Results for commit 1a265ab. ± Comparison against base commit 352482a.

This pull request removes 5074 and adds 4937 tests. Note that renamed tests count towards both.

	
	    get;
	"a": 1
	WHERE x = 1, SELECT * FROM t WHERE x = 1)
	await
	bar , foo bar)
	{
	}, 1)

…
aweXpect.Api.Tests.ApiApprovalTests ‑ AcceptApiChanges
aweXpect.Api.Tests.ApiApprovalTests ‑ VerifyPublicApi((aweXpect, net10.0))
aweXpect.Api.Tests.ApiApprovalTests ‑ VerifyPublicApi((aweXpect, net8.0))
aweXpect.Api.Tests.ApiApprovalTests ‑ VerifyPublicApi((aweXpect, netstandard2.0))
aweXpect.Api.Tests.ApiApprovalTests ‑ VerifyPublicApi((aweXpect.Core, net10.0))
aweXpect.Api.Tests.ApiApprovalTests ‑ VerifyPublicApi((aweXpect.Core, net8.0))
aweXpect.Api.Tests.ApiApprovalTests ‑ VerifyPublicApi((aweXpect.Core, netstandard2.0))
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ InnerException_ShouldBeSet(1e0ab80f-e439-4b48-9001-3cae50592c55)
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ InnerException_ShouldBeSet(32a39030-2b16-4041-946d-d5faa0962ec0)
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ InnerException_ShouldBeSet(436cb64a-d41e-4443-abfa-32ecf8afc5b0)
…

@vbreuss
vbreuss merged commit 08682ff into main Oct 18, 2026
18 checks passed
@vbreuss
vbreuss deleted the ci/api-tests-with-snapshots branch October 18, 2026 08:43
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Benchmark Results

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.5 LTS (Noble Numbat)
AMD EPYC 7763 2.69GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.303
[Host] : .NET 8.0.32 (8.0.32, 8.0.3226.50405), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Method Mean vs aweXpect Error StdDev Gen0 Gen1 Allocated vs aweXpect
Contains_aweXpect 351.6 ns 4.73 ns 4.42 ns 0.0443 - 744 B
Contains_FluentAssertions 385.7 ns +10% 7.86 ns 7.35 ns 0.0958 - 1608 B +116%
Contains_TUnit 404.8 ns +15% 5.70 ns 5.05 ns 0.0505 - 848 B +14%
Equivalency_aweXpect 13,361.1 ns 21.61 ns 18.05 ns 0.2289 - 3880 B
Equivalency_FluentAssertions 368,999.9 ns +2662% 2,137.50 ns 1,894.83 ns 35.1563 1.9531 590292 B +15114%
Equivalency_TUnit 19,426.3 ns +45% 135.36 ns 120.00 ns 0.9155 - 15680 B +304%
EventTriggered_aweXpect 735.5 ns 13.84 ns 12.95 ns 0.1106 - 1856 B
EventTriggered_FluentAssertions 83,578.5 ns +11263% 1,101.72 ns 919.98 ns 0.2441 0.1221 5252 B +183%
Int_GreaterThan_aweXpect 152.3 ns 3.41 ns 3.19 ns 0.0319 - 536 B
Int_GreaterThan_FluentAssertions 260.2 ns +71% 5.13 ns 4.55 ns 0.0730 - 1224 B +128%
Int_GreaterThan_TUnit 244.4 ns +60% 2.93 ns 2.74 ns 0.0463 - 776 B +45%
ItemsCount_AtLeast_aweXpect 283.3 ns 5.01 ns 4.69 ns 0.0358 - 600 B
ItemsCount_AtLeast_FluentAssertions 519.3 ns +83% 9.57 ns 8.49 ns 0.1192 - 2008 B +235%
ItemsCount_AtLeast_TUnit 400.6 ns +41% 7.04 ns 6.58 ns 0.0634 - 1064 B +77%
Multiple_aweXpect 797.1 ns 7.38 ns 6.54 ns 0.1001 - 1688 B
Multiple_FluentAssertions 1,979.5 ns +148% 31.85 ns 28.23 ns 0.4005 - 6744 B +300%
Multiple_TUnit 937.5 ns +18% 8.14 ns 7.62 ns 0.1335 - 2240 B +33%
Satisfies_aweXpect 177.8 ns 2.45 ns 2.04 ns 0.0329 - 552 B
Satisfies_FluentAssertions 31,947.1 ns +17864% 290.61 ns 271.84 ns 0.3052 0.2441 5614 B +917%
Satisfies_TUnit 244.9 ns +38% 3.69 ns 3.27 ns 0.0424 - 712 B +29%
String_aweXpect 277.5 ns 2.31 ns 2.16 ns 0.0343 - 576 B
String_FluentAssertions 522.7 ns +88% 10.91 ns 10.20 ns 0.1373 - 2304 B +300%
String_TUnit 347.3 ns +25% 3.77 ns 3.34 ns 0.0544 - 912 B +58%
StringArray_aweXpect 662.0 ns 4.04 ns 3.78 ns 0.0515 - 872 B
StringArray_FluentAssertions 1,378.5 ns +108% 15.79 ns 14.77 ns 0.2480 - 4152 B +376%
StringArray_TUnit 658.7 ns ±0% 4.05 ns 3.78 ns 0.0753 - 1264 B +45%
StringArrayInAnyOrder_aweXpect 654.0 ns 5.98 ns 5.00 ns 0.0544 - 920 B
StringArrayInAnyOrder_FluentAssertions 27,103.5 ns +4044% 199.56 ns 186.67 ns 1.9836 0.0610 33471 B +3538%
StringArrayInAnyOrder_TUnit 791.4 ns +21% 3.73 ns 2.91 ns 0.0839 - 1408 B +53%
Throws_aweXpect 12,660.8 ns 22.48 ns 17.55 ns 0.0916 - 1736 B
Throws_FluentAssertions 41,756.6 ns +230% 174.79 ns 163.50 ns 1.0986 - 18609 B +972%
Throws_TUnit 11,611.6 ns -8% 19.24 ns 16.07 ns 0.1068 - 1824 B +5%
ThrowsAsync_aweXpect 11,968.6 ns 20.08 ns 17.80 ns 0.1221 - 2216 B
ThrowsAsync_FluentAssertions 61,107.5 ns +411% 193.77 ns 171.77 ns 1.3428 - 23554 B +963%
ThrowsAsync_TUnit 22,731.2 ns +90% 46.85 ns 41.54 ns 0.1526 - 2920 B +32%

github-actions Bot added a commit that referenced this pull request Oct 18, 2026
github-actions Bot added a commit that referenced this pull request Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant