Skip to content

chore: upgrade xunit from v2 to v3 - #4623

Merged
MangelMaxime merged 9 commits into
mainfrom
chore/xunit-v3
Oct 12, 2026
Merged

chore: upgrade xunit from v2 to v3#4623
MangelMaxime merged 9 commits into
mainfrom
chore/xunit-v3

Conversation

@dbrattli

@dbrattli dbrattli commented May 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Upgrade the Beam, PHP, Python, and Rust .NET tests to xUnit v3 4.0.2 and Visual Studio runner 4.0.1. Use xunit.v3.mtp-off to preserve VSTest on .NET 10.

  • Generate .NET entry points with xUnit. Remove the empty Beam/PHP/Python Main.fs files and Rust’s application entry point; Rust uses Cargo’s library test harness.
  • Exclude xUnit’s generated entry point, reporters, and result writers from Fable builds.
  • Emit Rust no_main only for crates with an application entry point, so no_std libraries retain Cargo’s test harness.
  • Temporarily run Python/Beam serially to mitigate detached async assertion failures (Beam CI evidence). Remove the workaround once the .NET tests await completion. PHP keeps default parallelism.

Validation:

  • .NET: 8,028 tests passed across all four projects.
  • Rebased Rust validation: 2,793 .NET tests; 2,760 default and 2,719 no_std native tests passed (1 ignored in each). The no_std executable smoke check also passed.
  • Full BEAM suite: 2,790 native tests, entry-point programs, and diagnostic checks passed.
  • Python: 2,507 generated tests passed with two warnings; Python/Rust transpilation passed.
  • PHP transpilation has the same 13 unsupported-feature errors with and without Main.fs.
  • Fantomas and git diff --check passed.

@github-actions

github-actions Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

Python Type Checking Results (Pyright)

Metric Value
Total errors 34
Files with errors 4
Excluded files 4
New errors ✅ No
Excluded files with errors (4 files)

These files have known type errors and are excluded from CI. Remove from pyrightconfig.ci.json as errors are fixed.

File Errors Status
temp/tests/Python/test_hash_set.py 18 Excluded
temp/tests/Python/test_applicative.py 12 Excluded
temp/tests/Python/test_nested_and_recursive_pattern.py 2 Excluded
temp/tests/Python/fable_modules/thoth_json_python/encode.py 2 Excluded

@MangelMaxime
MangelMaxime force-pushed the main branch 2 times, most recently from 7ceced2 to c85af32 Compare June 30, 2026 21:38
@MangelMaxime
MangelMaxime force-pushed the main branch 2 times, most recently from 453bed5 to 19a6522 Compare September 8, 2026 07:01
ncave
ncave reviewed Oct 11, 2026
View reviewed changes
Comment thread tests/Rust/tests/src/main.fs Outdated
()

// [<OuterAttr("cfg", [|"not(feature = \"no_std\")"|])>]
#if FABLE_COMPILER

@ncave ncave Oct 11, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you rebase, you can just delete the main endpoint. See #5055.

@dbrattli dbrattli Oct 12, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ncave. I tried to get this working again. Been uncertain about merging this since its technically unsupported by Xunit, but I guess we do not have much alternative 🤔 Ref: xunit/xunit#3455

dbrattli and others added 9 commits October 12, 2026 08:06
@dbrattli @claude
chore: upgrade xunit from v2 to v3
0aef88d 
Replace the deprecated xunit v2 (`XUnit` 2.9.3) package with `xunit.v3`
3.2.2 in the Beam, Php, Python and Rust test projects.

v3 test projects are self-executing, so each project now sets
`OutputType=Exe` (added to Rust and Php; Python and Beam already had it)
and the .NET-side `[<EntryPoint>]` is dropped in favour of the runner
entry point generated by xunit.v3:

- Rust `main.fs` is also transpiled to Rust's `fn main`, so its entry
  point is guarded with `#if FABLE_COMPILER` rather than removed.
- Python/Beam/Php had the entry point only in the .NET (`#else`) branch
  of `Main.fs`, which is removed.

`xunit.runner.visualstudio` 3.1.5 is already v3-compatible, so
`dotnet test` continues to work. Test sources need no changes:
`Xunit.FactAttribute`, `Assert.Equal` and `Assert.NotEqual` are
unchanged in v3.

Verified with `dotnet test -c Release`: Rust (2449), Python (2266) and
Beam (2439) pass. The Php project has pre-existing compile errors on
.NET unrelated to xunit and is not run by the build system.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dbrattli @claude
chore: stop xunit.v3 attaching runner sources to the Fable build
26e15fc 
CI revealed that the Fable transpilation of the Rust tests failed with
FS0433 ("EntryPointAttribute must be the last declaration in the last
file"). When Fable cracks a project it defines FABLE_COMPILER and runs
the CoreCompile target, which triggers xunit.v3's _XunitAttachSourceFiles
target. That appends a generated entry point and the built-in runner
reporters after the project's own source files. For Rust this collides
with the `[<EntryPoint>]` in tests/src/main.fs (and those sources also
reference runtime types Fable cannot transpile).

Disable both attached source files for the Fable build only, keyed off
the FABLE_COMPILER MSBuild property that Fable's MSBuildCrackerResolver
passes (`/p:FABLE_COMPILER=True`). On .NET the properties keep their
defaults, so xunit.v3 still generates the test runner entry point.

Verified `dotnet test -c Release` (Rust 2449 passing) and the Fable
transpile (Rust/Python/Beam) all succeed with no xunit sources leaking
into the transpiled output.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dbrattli @claude
chore: run xunit serially in Python tests to avoid Windows flake
782e103 
Several MailboxProcessor/async tests assert from inside a fire-and-forget
`Async.StartImmediate`, whose continuation can run after its test
collection has been torn down. xunit.v3 reports such background-thread
exceptions as catastrophic failures and exits non-zero (xunit v2 silently
ignored them). With parallel test collections this surfaces on Windows as
"Catastrophic failure: Assert.Equal() ... Expected 42, Actual 0".

Add tests/Python/xunit.runner.json disabling test-collection
parallelization so each collection fully drains before the next runs.
This is the only xunit suite that runs on Windows in CI (Rust/Beam/Php
.NET tests run on Linux only; the JS suite uses Expecto).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dbrattli @claude
chore(tests): extend serial xunit config to Beam/Php and tidy review …
f348f64 
…feedback

- Add xunit.runner.json to Beam and Php: their tests use the same
  fire-and-forget Async.StartImmediate pattern as Python, so they are
  exposed to the same xunit.v3 catastrophic background-exception failures.
- Collapse the now-identical #if FABLE_COMPILER branches in
  Beam/Php/Python Main.fs and add the missing trailing newlines.
- Compress the .fsproj comments explaining the cracking workaround.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dbrattli
chore(tests): upgrade xunit v3 to 4.0.2
adcdaf3
@dbrattli
chore(tests): limit xunit serialization to Python
c009e72
@dbrattli
chore(tests): remove empty entry files and restore Beam CI workaround
e88ac2d
@dbrattli
chore(tests): remove empty Python entry file
7f43878
@dbrattli
fix(rust): retain test harness without an application entry point
a47b83e
@MangelMaxime
MangelMaxime merged commit 148f989 into main Oct 12, 2026
53 checks passed
@MangelMaxime
MangelMaxime deleted the chore/xunit-v3 branch October 12, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ncave ncave ncave left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dbrattli @ncave @MangelMaxime