Test MSHV 6.18 weekly and benchmark it daily by ludfjig · Pull Request #1941 · hyperlight-dev/hyperlight

ludfjig · 2026-10-15T17:55:10Z

Tests MSHV 6.18 weekly and benchmarks it daily on Azure Linux 3 with the released MSHV 6.18 kernel

Adds WeeklyMshv618.yml job to run tests and examples
Adds a benchmarks-mshv-6-18 job to DailyBenchmarks.yml
Adds an optional mshv_kernel input to dep_build_test.yml, dep_run_examples.yml, and dep_benchmarks.yml. When it is set, mshv3 jobs run on hld-azlinux3-mshv-<mshv_kernel>-<vendor>. Existing callers do not pass it and keep their pools.
When mshv_kernel is set, benchmark artifacts are named benchmarks_Linux_mshv3-<mshv_kernel>_<vendor>. This avoids a name collision with the 6.6 artifacts in the same daily run. bench-report reads the label as its own hypervisor, so 6.6 results never match a 6.18 baseline.

The new pools hld-azlinux3-mshv-6.18-amd and hld-azlinux3-mshv-6.18-intel hold up max 3 VMs each to save azure quota. This is suffient since we don't run per pr

passed runs: https://github.com/hyperlight-dev/hyperlight/actions/runs/38781317655?pr=1941

Add WeeklyMshv618.yml, which runs the mshv3 build-test, examples, and benchmarks on the hld-azlinux3-mshv-6.18-{amd,intel} pools every Monday. A failure opens or updates a release-blocking issue labeled area/ci-periodics-mshv-6.18. Add an optional mshv_kernel input to dep_build_test, dep_run_examples, and dep_benchmarks. It selects the hld-azlinux3-mshv-<kernel>-<vendor> pool. Callers that omit it keep the existing pools. The run keeps hypervisor mshv3, so it uses the mshv3 feature, Justfile recipes, and Rust cache keys. Both pools run Azure Linux 3. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

Move the MSHV 6.18 benchmarks from WeeklyMshv618.yml to a separate DailyBenchmarks.yml job. Daily runs measure 6.6 and 6.18 on the same commit, in the same mode, at about the same time. A separate job keeps a 6.18 failure from cancelling the existing fail-fast matrix. Name benchmark artifacts benchmarks_<OS>_<hypervisor>-<kernel>_<vendor> when mshv_kernel is set, so 6.18 results do not collide with the 6.6 artifacts in the same run. bench-report reads the new label as its own hypervisor, so 6.6 results never match a 6.18 baseline. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

Copilot

🟢 Approval recommended

The scoped workflow changes preserve default behavior, isolate benchmark artifacts, and have no identified blocking issues.

0 open findings

What changed in this PR

Adds scheduled MSHV 6.18 coverage on Azure Linux 3 while preserving existing runner pools for other callers.

Changes:

Runs tests and examples weekly on AMD and Intel pools.
Runs daily benchmarks with kernel-specific artifact names.
Documents benchmark coverage and the dedicated weekly failure label.

File	Description
docs/github-labels.md	Documents the weekly failure label.
docs/benchmarking-hyperlight.md	Documents daily MSHV 6.18 benchmarks.
.github/workflows/WeeklyMshv618.yml	Schedules tests, examples, and failure reporting.
.github/workflows/dep_run_examples.yml	Supports kernel-specific example runner pools.
.github/workflows/dep_build_test.yml	Supports kernel-specific test runner pools.
.github/workflows/dep_benchmarks.yml	Supports kernel-specific pools and artifact names.
.github/workflows/DailyBenchmarks.yml	Adds MSHV 6.18 benchmarks and failure notification wiring.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

hyperlight-gh-bot · 2026-10-15T19:59:39Z

Benchmark Results

Measured commit: d784c30009c1
Baseline commit: 83849abf1ab4

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`764.90 ns` (➖ 1.00x slower)
`vec_bytes`	`594.35 ns` (➖ 1.00x slower)
		`380.17 µs` (➖ 1.02x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`552.94 ns` (➖ 1.06x slower)
`65536`	`141.84 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`81.90 ms` (➖ 1.00x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`8.73 ns` (➖ 1.12x slower)	`7.71 ns` (➖ 1.00x slower)	`7.73 ns` (➖ 1.06x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`79.94 µs` (➖ 1.16x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`7.12 µs` (➖ 1.02x slower)	`8.98 µs` (➖ 1.23x slower)
`65536`	`2.06 µs` (➖ 1.06x slower)	`1.93 µs` (➖ 1.09x faster)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`6.30 µs` (➖ 1.07x faster)	`6.33 µs` (➖ 1.00x faster)
`8192`	`1.10 µs` (➖ 1.04x slower)	`1.06 µs` (➖ 1.01x faster)
`262144`		`26.98 µs` (➖ 1.03x faster)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`692.45 ns` (➖ 1.01x slower)
`vec_bytes`	`526.48 ns` (➖ 1.25x faster)
		`671.96 µs` (➖ 1.00x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`566.56 ns` (➖ 1.09x slower)
`65536`	`154.08 ns` (➖ 1.09x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`78.49 ms` (➖ 1.04x slower)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`7.75 ns` (➖ 1.07x slower)	`7.78 ns` (➖ 1.08x slower)	`7.79 ns` (➖ 1.07x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`45.49 µs` (➖ 1.07x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`8.50 µs` (➖ 1.02x slower)	`8.46 µs` (➖ 1.03x slower)
`65536`	`2.41 µs` (➖ 1.03x slower)	`2.42 µs` (➖ 1.06x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`7.90 µs` (➖ 1.01x slower)	`7.96 µs` (➖ 1.02x slower)
`8192`	`858.07 ns` (➖ 1.04x slower)	`832.99 ns` (➖ 1.04x slower)
`262144`		`32.59 µs` (➖ 1.03x faster)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`950.34 ns` (➖ 1.02x slower)
`vec_bytes`	`717.54 ns` (➖ 1.00x slower)
		`279.89 µs` (➖ 1.03x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`717.82 ns` (➖ 1.00x slower)
`65536`	`190.74 ns` (➖ 1.04x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`59.48 ms` (➖ 1.05x slower)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`10.18 ns` (➖ 1.07x slower)	`9.49 ns` (➖ 1.00x slower)	`9.47 ns` (➖ 1.01x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`86.86 µs` (➖ 1.02x slower)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`10.24 µs` (➖ 1.01x faster)	`10.16 µs` (➖ 1.05x slower)
`65536`	`2.40 µs` (➖ 1.02x faster)	`2.24 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`8.55 µs` (➖ 1.17x faster)	`8.60 µs` (➖ 1.12x faster)
`8192`	`1.30 µs` (➖ 1.00x slower)	`1.32 µs` (➖ 1.06x slower)
`262144`		`35.80 µs` (➖ 1.05x faster)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`919.20 ns` (➖ 1.00x faster)
`vec_bytes`	`654.94 ns` (➖ 1.00x slower)
		`752.45 µs` (➖ 1.11x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`649.12 ns` (➖ 1.00x slower)
`65536`	`172.27 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`70.08 ms` (➖ 1.09x slower)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`9.41 ns` (➖ 1.01x slower)	`9.65 ns` (➖ 1.01x slower)	`9.39 ns` (➖ 1.02x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`45.93 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`7.93 µs` (➖ 1.01x slower)	`7.97 µs` (➖ 1.02x slower)
`65536`	`2.26 µs` (➖ 1.00x faster)	`2.25 µs` (➖ 1.02x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`7.59 µs` (➖ 1.00x faster)	`7.52 µs` (➖ 1.02x faster)
`8192`	`949.84 ns` (➖ 1.01x faster)	`920.85 ns` (➖ 1.01x faster)
`262144`		`38.78 µs` (➖ 1.02x faster)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.14 µs` (➖ 1.05x faster)
`vec_bytes`	`769.62 ns` (➖ 1.03x faster)
		`1.84 ms` (➖ 1.00x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`788.66 ns` (➖ 1.00x slower)
`65536`	`236.30 ns` (➖ 1.02x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`75.48 ms` (➖ 1.07x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`10.86 ns` (➖ 1.02x faster)	`11.31 ns` (➖ 1.04x slower)	`10.86 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`624.04 µs` (➖ 1.17x slower)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`8.60 µs` (➖ 1.01x faster)	`8.55 µs` (➖ 1.00x faster)
`65536`	`2.54 µs` (➖ 1.01x faster)	`2.53 µs` (➖ 1.00x faster)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`9.15 µs` (➖ 1.06x slower)	`8.69 µs` (➖ 1.04x faster)
`8192`	`1.39 µs` (➖ 1.01x faster)	`1.44 µs` (➖ 1.04x slower)
`262144`		`39.47 µs` (➖ 1.04x slower)

hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.22 µs` (➖ 1.02x slower)
`vec_bytes`	`785.34 ns` (➖ 1.01x slower)
		`3.22 ms` (➖ 1.01x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`794.99 ns` (➖ 1.08x slower)
`65536`	`216.06 ns` (➖ 1.03x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`115.66 ms` (➖ 1.03x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`10.05 ns` (➖ 1.01x slower)	`10.03 ns` (➖ 1.14x faster)	`9.97 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`544.01 µs` (➖ 1.21x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`7.78 µs` (➖ 1.02x slower)	`7.80 µs` (➖ 1.00x slower)
`65536`	`2.34 µs` (➖ 1.01x slower)	`2.34 µs` (➖ 1.01x faster)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`8.46 µs` (➖ 1.04x faster)	`8.52 µs` (➖ 1.03x faster)
`8192`	`1.13 µs` (➖ 1.11x faster)	`1.09 µs` (➖ 1.01x faster)
`262144`		`50.13 µs` (➖ 1.20x slower)

_{Reported by cargo ci bench-report --candidate run:38792017498 --config-file bench_report.toml.}

ludfjig added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Oct 15, 2026

This comment has been minimized.

Sign in to view

ludfjig changed the title ~~Add weekly MSHV 6.18 test and benchmark workflow~~ Test MSHV 6.18 weekly and benchmark it daily Oct 15, 2026

Remove temporary PR trigger from WeeklyMshv618.yml

d784c30

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

ludfjig marked this pull request as ready for review October 15, 2026 19:16

Copilot AI balanced review requested due to automatic review settings October 15, 2026 19:16

ludfjig requested review from andreiltd, danbugs, dblnz, devigned, jprendes, jsturtevant, simongdavies, squillace and syntactically as code owners October 15, 2026 19:16

Copilot started reviewing on behalf of ludfjig October 15, 2026 19:17

View session

Copilot AI reviewed Oct 15, 2026

View reviewed changes

jsturtevant approved these changes Oct 15, 2026

View reviewed changes

ludfjig merged commit 4cf3f9a into hyperlight-dev:main Oct 15, 2026
62 checks passed

Conversation

ludfjig commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

hyperlight-gh-bot Bot commented Oct 15, 2026

Benchmark Results

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ludfjig commented Oct 15, 2026 •

edited

Loading