Update pinned uruntime to v0.8.1 and add an end-to-end AppImage smoke test

[Samueru-sama]

Two independent changes, one commit each.

1. Update the pinned uruntime from v0.7.1 to v0.8.1 (URL + all six SHA-256 pins) — ba468e4
2. Add an end-to-end AppImage smoke test and a CI workflow that runs it for every architecture we ship — 86d6f04

They go together in one PR because the smoke test is what gives us confidence in the runtime bump on the architectures upstream no longer executes, but they are cleanly separable if you'd rather split them.

---

1. uruntime v0.7.1 → v0.8.1

What changed upstream

v0.8.0 is not a routine bump — it is a rewrite of the mount/namespace/lifecycle core (src/main.rs +3490/-366 across the two tags). v0.8.1 is a follow-up compatibility/correctness patch.

Execution integrity (0.8.0)

• The runtime and the appended image are now retained as open descriptors. Re-exec prefers execveat(AT_EMPTY_PATH), then falls back through verified /proc/self/fd or an exact dev/ino-matched pathname. Renaming, deleting or replacing the file mid-run can no longer redirect a running runtime or its helpers to different content.

Authenticated reusable mounts (0.8.0)

• Trusted mount records now bind helper PID + process start time + effective UID/GID mapping + namespace kind + concrete user/mount namespace identities. Legacy PID-only records stay readable but are no longer trusted for private-namespace reuse.
• Private reuse validates identity (pidfd_open + pre-transition setns) before any irreversible transition.
• Explicit root/UID/GID mapping now fails closed instead of silently launching with weaker isolation.

No-procfs / restricted environments (0.8.0 + 0.8.1)

• Procfs is now detected functionally (/proc/self/stat readable) with namespace identity probed separately, so "procfs present but ns/* missing" (Linuxulator, older kernels) is handled.
• Without procfs, persistent reuse for private-namespace launches is disabled; visible current-namespace mounts and extracted dirs keep hash-based reuse.
• 0.8.1 fixed create_dir_all on an existing lock parent (broke restricted roots with a usable /tmp), a /dev/null-less supervisor detach that could close fd 0/1/2, and a no-FUSE re-exec dropping an explicit target dir.

Locking & cleanup

• Cross-launch lifetime leases on OFD-lock byte ranges, with a conservative .lease/.lease.lock flock fallback on older kernels. Zero-length lock inodes intentionally remain next to reusable targets.
• MNT_EXPIRE-based delayed unmount; busy or failed mounts are retained rather than force-unmounted.
• Child-subreaper supervision for daemonized descendants; when visibility is unavailable the extracted target is retained instead of deleted.

CLI (0.8.0) — a universal --uruntime-* prefix was added; the existing --appimage-* and --runtime-* forms are retained.

Checksum update

src/pinned.rs is the only place the uruntime is pinned. URUNTIME_URL_TEMPLATE moves to v0.8.1 and all six URUNTIME_CHECKSUMS entries are replaced:

arch	new SHA-256
aarch64	c1641dfe465f4cb70ae545fbfb9de7a221aa6e9a0b1d7ea600b213a1f10738a0
loongarch64	7f149441fbb772477c8748e58c1233e4e0e141d6ca2507c4ddad97b69b4008bc
ppc64	db7a7834c1cb657c2708eab206d9a140231962d22f776cd0ea4fd9a482be3037
ppc64le	0fa3983c5c18794e841d2ad20938b87c467cbf24034a0fad608c5fe00a1668a9
riscv64	25d1137942a24fed3d1126686960de5dc5ec44070d5e3a6c3544cca229d230cb
x86_64	b3c2916153e089d703cee5a7ebc540941d2f1d71baa90eb3092b15eef48345a8

Each digest was read from the release asset digest metadata via the GitHub API and independently confirmed by downloading the asset and re-hashing it (sha256sum); sizes matched too, so no download was truncated. The mkdwarfs pin is untouched.

Why the build-time patching still works

This is the part that actually mattered to check, because the tool patches the runtime binary rather than just downloading it: it writes .upd_info and .envs, and rewrites the URUNTIME_MOUNT= marker for --keep-mount (src/uruntime.rs, src/elf.rs). Verified against the real v0.8.1 binary:

• .upd_info and .envs section names and sizes are byte-identical to v0.7.1 (1024 / 16384) — no new SectionOverflow risk.
• The URUNTIME_MOUNT= marker is still present and --keep-mount still rewrites both occurrences to =0 exactly as before.
• .sha256_sig is only read/written by --appimage-signature / --appimage-addsign and is not verified at startup, so patching sections cannot invalidate anything.
• Embedded helper stamps are unchanged (DWARFS_VERSION 0.15.7, squashfs-tools 4.7.5.r2, squashfuse 0.6.3.r2), so image-format compatibility is unaffected.
• The licensing boundary is intact: the lite variant still embeds dwarfs-fuse-extract (MIT); the non-lite variant embeds dwarfs-universal (GPL-3.0).

Regression assessment

Low risk for this tool's own behaviour — the build-time integration surface is unchanged and was exercised end to end (see Verification). The behavioural changes are all on the AppImage runtime side:

• Fail-closed isolation: an AppImage that requests explicit unshare/UID/GID mapping now exits where v0.7.1 silently degraded. Intended hardening, but a visible change for anyone relying on it.
• No reuse in restricted containers: no procfs + private namespace means fresh mounts per launch.
• One-time fresh state: mounts recorded by v0.7.1 (legacy PID-only records) aren't trusted for reuse. Benign, self-heals.
• More retained artifacts: conservative cleanup can leave busy mounts, extracted dirs and zero-length .lock/.lease sidecars behind. Safe direction, but expect more /tmp residue.
• New syscalls (execveat, pidfd_open, OFD locks, subreaper, MNT_EXPIRE) all have fallbacks or fail closed — least-tested path, but degrade rather than break.

Upstream CI gap (motivation for change 2)

v0.8.1 disabled the foreign QEMU execution gates in its own CI:

# Temporarily disabled: foreign QEMU smoke tests make the matrix excessively slow.
# - name: Install QEMU for foreign smoke tests
# - run: cargo --locked xtask artifacts validate-arch '${{ matrix.arch }}' dist --smoke

(and the AArch64 QEMU quality lane added in 0.8.0 is commented out too). The --smoke flag is gone for every arch including x86_64, and an xtask workflow-contract test now asserts the QEMU step is absent, so the disable is deliberate. What remains is: the native x86_64 quality+lifecycle gate, and manifest/ELF validation for all six arches. riscv64, loongarch64, ppc64, ppc64le and aarch64 were built but never executed upstream for this release — exactly the arches we ship, with big-endian ppc64 historically the fragile one (our own 1ed1dae).

---

2. End-to-end smoke test

smoke_builds_and_runs_appimage in tests/integration.rs:

• writes a minimal AppDir whose AppRun is #!/bin/sh + echo success,
• builds the AppImage through appimagetool::appimage::build,
• executes the resulting AppImage and asserts it prints success.

It is the only test that exercises a finished AppImage the way a user does, so it catches runtime/packaging regressions nothing else can see. It is #[ignore]d by default (it needs mkdwarfs and network access), matching the existing pipeline test.

Knobs:

env var	meaning
APPIMAGETOOL_SMOKE_ARCH	target runtime arch (default: host)
APPIMAGETOOL_SMOKE_MKDWARFS	host mkdwarfs path (otherwise $PATH, then download)

APPIMAGETOOL_SMOKE_ARCH=aarch64 cargo test --test integration -- \
  smoke_builds_and_runs_appimage --ignored --exact

It runs with APPIMAGE_EXTRACT_AND_RUN=1: FUSE isn't usable on CI runners or under QEMU user emulation, so it drives the runtime's extraction path. The AppImage is still executed end to end.

It also checks that the AppImage carries the architecture that was asked for, by reading EI_DATA (offset 5) and e_machine (offset 18) out of the output. An arch mix-up still runs natively on the machine that produced it and prints success, so without this the test cannot tell x86_64 from aarch64 — precisely the 1ed1dae shape, where ppc64le was handed a big-endian ppc64 runtime. ppc64 and ppc64le share EM_PPC64, so the endianness byte is what separates them.

If a foreign arch is requested and no usable handler is registered, the test fails with an actionable message rather than silently skipping; when a handler matches but yields no interpreter, it adds the EI_PAD explanation.

3. CI

New .github/workflows/ci.yml, triggered on push to main, on PRs, and manually. Actions are pinned to the same SHAs already used in release.yml.

• A Unit tests job runs cargo test --locked. The smoke matrix only runs the --ignored test, so without this the unit tests that encode arch resolution and the pins in src/pinned.rs would never run in CI.
• A Smoke <arch> matrix over all six release architectures, fail-fast: false.
• Foreign targets install qemu-user-static, replace its handlers with ones that match AppImages (see below), and assert the handler is live (grep -q enabled /proc/sys/fs/binfmt_misc/qemu-<arch>).
• Host mkdwarfs is fetched and checksum-verified by reading the pin out of src/pinned.rs — the same pattern release.yml already uses for the DwarFS source tarball — so images are never built with a different tool than releases use.

Why the F flag matters: the runtime executes its embedded helpers from a memfd. Only a fix-binary (F) binfmt handler can exec from a memfd. Without it you get ENOEXEC, the runtime then falls back to writing a temp file with a 2 ms cleanup race that QEMU reliably loses, and the launch fails with ENOENT. Debian/Ubuntu's qemu-user-static registers with F.

Why the shipped handlers had to be replaced: qemu-user-static's handlers also require the eight EI_PAD bytes at offset 8 to be zero, but an AppImage writes its AI\x02 magic there. The stock entries therefore never match an AppImage — even while /proc/sys/fs/binfmt_misc/qemu-<arch> cheerfully reports enabled — and exec fails with Exec format error. CI registers handlers that wildcard the padding, and the smoke test now says so when it hits ENOEXEC. The first CI run on this PR failed exactly this way, which is how it got caught.

---

Verification

• uruntime pins: all six assets downloaded and re-hashed; every digest matches both the API metadata and the file.
• Smoke test, all six arches: x86_64 native, and aarch64/riscv64/loongarch64/ppc64/ppc64le with binfmt_misc registered — all pass through the real test in this PR. (Verified locally by registering binfmt inside a user namespace, since the dev box has no root; CI does it properly.)
• Arch assertion: each of the six built AppImages matches exactly one architecture's (EI_DATA, e_machine) pair, and making the test expect big-endian for ppc64le fails it — so the check is not vacuous.
• Full suite: cargo test --locked → 64 + 18 pass, smoke correctly skipped by default; cargo clippy --locked --all-targets clean.
• Manual end-to-end with the new runtime and mkdwarfs 0.15.6: correct --appimage-offset boundary, --appimage-updateinfo / --appimage-envs round-trip, --appimage-extract, direct launch, and APPIMAGE_EXTRACT_AND_RUN=1 all succeed.

Follow-ups

• The upstream foreign-arch execution gap is a standing risk while we ship those binaries; this PR's CI partly compensates, but it is emulation, not bare metal.
• The v0.8.1 FUSE mount path — mount records, lifetime leases, MNT_EXPIRE, reuse, i.e. the bulk of the upstream rewrite — is still executed nowhere: the smoke runs use APPIMAGE_EXTRACT_AND_RUN=1, hosted runners have no /dev/fuse, and QEMU user emulation cannot mount either. Only x86_64 reaches the FUSE fallback chain, and it falls through to extraction. A lifecycle check on a FUSE-capable runner would close it, as would running the aarch64 smoke on a native ubuntu-24.04-arm runner (the release matrix already uses one) instead of QEMU.

[Nemo-010]

Reviewed the two changes against the artifacts, not just the description.

Verified

• Fetched all six uruntime-appimage-dwarfs-lite-* assets from the v0.8.1 tag and hashed them: every digest matches src/pinned.rs (aarch64 c1641d…, loongarch64 7f1494…, ppc64 db7a78…, ppc64le 0fa398…, riscv64 25d113…, x86_64 b3c291…).
• .upd_info and .envs are still present in all six v0.8.1 runtimes, so elf::write_section can't fail closed on the new binaries; the aarch64 CI log shows Adding update information to runtime... and a green build.
• Keep-mount semantics did not move with the rewrite: both v0.7.1 and v0.8.1 src/main.rs have the identical "=0" => (true, Some("inf")) arm, so patch_keep_mount still means "keep the FUSE mount". Worth stating explicitly, because the checksum bump alone doesn't show it.
• Run 36029342344 is green on all six Smoke <arch> jobs, and the aarch64 log shows the AppImage really built and exec'd (test result: ok. 1 passed).

Two things I'd change before merge

1. The smoke test can't see an arch mix-up. It forwards APPIMAGETOOL_SMOKE_ARCH straight into Config (tests/integration.rs:479), so on the x86_64 runner an accidentally-x86_64 runtime still prints success — it just runs natively — and the test stays green. That is exactly the 1ed1dae shape (ppc64le handed a big-endian ppc64 runtime), and this is the only test positioned to catch it. After build() (tests/integration.rs:518) you have the AppImage in hand: read e_machine at offset 18, respecting EI_DATA at offset 5, and assert it equals the requested arch's machine. Cheap, and it makes the "we ship what upstream no longer runs" claim real.

2. ci.yml only runs the --ignored smoke test (ci.yml:96), so the unit tests that encode arch resolution — target_arch_resolves_powerpc_by_endianness and every_release_target_has_both_pins in src/pinned.rs:149,160 — never run in CI, nor does anything else. A second job running cargo test --locked would close that; it's the repo's only test coverage today.

Nits in the description (it becomes the record): there is no systemd-binfmt restart anywhere in ci.yml, and the quoted "cannot run the aarch64 AppImage…" text does not match the panic text in tests/integration.rs:485.

The EI_PAD-masked binfmt magic is a good catch — the sort of long-tail detail that costs a Neucom engineer a week if nobody finds it.

[talaria0101]

Reviewed at b1f8529, re-fetched right before reviewing. Everything below was measured on this head, not taken from the description.

Verdict: no regression measured on any previously working path. Both commits are sound. One coverage gap worth naming.

Pins

• Downloaded all six v0.8.1 assets myself and re-hashed: every digest matches the new URUNTIME_CHECKSUMS table.
• Re-hashed v0.7.1 x86_64: it matches the old pin (f19d2a58...), so the diff replaces the right values.
• The versionless cache name cannot shadow the bump: ensure_cached_binary (src/util.rs) re-verifies the cached file against the pin on every build and re-downloads on mismatch.
• MKDWARFS_CHECKSUMS untouched, as stated.

Build-time integration with the new runtime

• .upd_info is 0x400 and .envs is 0x4000 in both v0.7.1 and v0.8.1 (readelf on both binaries). URUNTIME_MOUNT= appears exactly twice in both.
• Built the same AppDir with tool@main (runtime 0.7.1) and tool@branch (runtime 0.8.1): --appimage-offset equals the runtime size in both, --appimage-updateinfo round-trips on the new runtime, and --keep-mount leaves both occurrences as URUNTIME_MOUNT=0 (checked the bytes before the squashfs offset).
• Swapped runtimes: new tool + old runtime and old tool + new runtime both build and run clean, so the tool side is behavior-compatible in both directions.

Regression measurement, both paths

Same AppDir, both runtimes, on this x86_64 host:

path	v0.7.1	v0.8.1
direct run	exit 0, prints success, FUSE fail then fallback	exit 0, prints success, FUSE fail then fallback
APPIMAGE_EXTRACT_AND_RUN=1	exit 0, prints success	exit 0, prints success

Identical payload block counts and compressed size (189 B in, 1 block, 958 B out). The payload bytes differ between builds, but a control run of the same tool twice also differs (build timestamp near the DWARFS trailer), so that is mkdwarfs nondeterminism, not a version change.

Test and CI

• Full suite locally at head: 64 lib + 18 integration pass, smoke correctly skipped by default; cargo clippy --locked --all-targets clean.
• Ran smoke_builds_and_runs_appimage natively: pass.
• The EI_PAD diagnosis is confirmed from qemu's own source: upstream qemu-binfmt-conf.sh masks bytes 8..15 as \x00, and an AppImage writes AI\x02 there, so stock handlers never match. The CI replacement handlers mask bytes 7..17 and match e_machine exactly (checked the magic, mask and e_machine bytes for all five foreign arches, including big-endian ppc64).
• The upstream quote is accurate: v0.8.1 has the foreign QEMU smoke steps commented out, and xtask/src/workflow_contract.rs:166 asserts the step is absent, so the disable is deliberate. Foreign-arch binaries were genuinely never executed upstream this release.
• The branch's own CI history corroborates the story: the run on 86d6f04 failed with Exec format error (os error 8) on foreign arches, the run on b1f8529 is green, all seven checks.
• The mkdwarfs fetch step in ci.yml extracts a valid URL and checksum from the real pinned.rs, and the action SHAs match release.yml.

Not verified by me

• The specific embedded helper stamps (DWARFS 0.15.7, squashfs-tools 4.7.5.r2, squashfuse 0.6.3.r2) unchanged between versions: neither binary carries greppable version strings, so I could not confirm the versions. The substance is covered anyway: the end to end build and extract tests exercise image-format compatibility with real mkdwarfs 0.15.6.
• Foreign-arch smoke results are verified through this PR's green CI runs, not on bare metal here.

One gap

The foreign smoke runs use APPIMAGE_EXTRACT_AND_RUN=1, so the v0.8.1 FUSE mount path, which is the bulk of the rewrite (mount records, leases, MNT_EXPIRE, reuse), is exercised nowhere in this PR: extract-and-run bypasses mounting, hosted runners have no /dev/fuse, and QEMU user emulation cannot mount either. Only x86_64 ever reaches the FUSE fallback chain in CI, and on my host it fell through to extraction like on runners. The tool's own job (packaging, patching, integrity) is fully covered, and the body already flags emulation versus bare metal, but the mount-path rewrite itself currently has no execution test anywhere. Worth a follow-up, for example a lifecycle check on a runner with FUSE, or the native arm runner idea from the follow-ups.

Ready to merge from my side.

[Samueru-sama]

Addressed both reviews at head aa5017f. All seven checks are green (6 × Smoke, plus the new Unit tests).

@Nemo-010

1. The smoke test couldn't see an arch mix-up — fixed.

After build() the test now reads EI_DATA (offset 5) and e_machine (offset 18) from the produced AppImage and asserts both against the requested arch. I assert the endianness byte explicitly, not just e_machine, because ppc64 and ppc64le both carry EM_PPC64 (0x15) and differ only in EI_DATA — reading e_machine "respecting EI_DATA" alone would still have let a ppc64le/ppc64 swap through, which is the exact 1ed1dae failure.

Evidence it isn't vacuous:

• Each of the six built AppImages matches exactly one architecture's (EI_DATA, e_machine) pair — ppc64 is the only EI_DATA=2.
• Making the test expect big-endian for ppc64le fails it (panicked at tests/integration.rs:584), then passes again once reverted.

2. Unit tests now run in CI — fixed.

Added a Unit tests job running cargo test --locked, so target_arch_resolves_powerpc_by_endianness, every_release_target_has_both_pins and the rest of the suite are covered. At aa5017f it reports 64 lib + 18 integration passing; the smoke test is correctly still skipped there (it runs in the matrix).

3. Description nits — both fixed.

• The systemd-binfmt restart sentence is gone; it described the first draft of the step, which I later replaced with explicit handler registration.
• I dropped the quoted panic text entirely and describe the behaviour in prose instead, so it can't drift out of sync with tests/integration.rs again.

@talaria0101

FUSE coverage gap — agreed, recorded as a follow-up.

Your read is right: APPIMAGE_EXTRACT_AND_RUN=1 bypasses the whole mount path, runners have no /dev/fuse, and QEMU user emulation can't mount either, so the bulk of the v0.8.1 rewrite is executed nowhere. I've added it to the PR's follow-ups, including the two ways out you and I both landed on — a lifecycle check on a FUSE-capable runner, or running the aarch64 smoke natively on ubuntu-24.04-arm instead of QEMU. I'd rather land that as its own change than widen this PR's blast radius.

On your "not verified" item — the helper stamps.

The provenance is upstream's build_support.rs at each tag (repo source, not the compiled artifact), and DWARFS_VERSION is "0.15.7" at both v0.7.1 and v0.8.1. That is the only helper that concerns us: we pin uruntime-appimage-dwarfs-lite-{arch}, which upstream builds from ["appimage", "dwarfs", "lite"], so the squashfs helpers are compiled out and only dwarfs-fuse-extract is embedded. (My earlier message also listed SQUASHFS_TOOLS_VERSION and SQUASHFUSE_VERSION because I quoted the surrounding block; those belong to the squashfs-flavoured runtimes we do not use.)

Consistent with your finding that neither binary carries greppable version strings — the helpers are embedded zstd-compressed, so there's nothing to grep. Your end-to-end build/extract against real mkdwarfs 0.15.6 is the stronger evidence anyway.

---

One thing neither of you raised that I want on the record: the binfmt handlers CI registers are a test harness detail only. The AppImage's bytes at offset 8 stay AI\x02 — that comes from the upstream runtime and the tool never rewrites it — so nothing in this PR affects other tools' ability to recognise the file as an AppImage.

[Nemo-010]

Confirmed at aa5017f — both fixes land and the evidence holds.

• The EI_DATA assertion is the right call, and your correction of my wording is correct: decoding e_machine "respecting EI_DATA" still yields 0x15 for both ppc64 and ppc64le, so the endianness byte has to be asserted on its own. expected_elf_data/expected_emachine are right for all six, including loongarch64 0x0102 and the shared EM_PPC64.
• Run 36034039607 is green on all seven jobs: six Smoke plus Unit tests, which reports 64 lib + 18 integration passing, 2 ignored. That job is what stops target_arch_resolves_powerpc_by_endianness from being decoration.
• Both description nits are gone.

The FUSE-mount hole talaria0101 raised is the honest remaining gap, and a follow-up is the right place for it. Nothing else from me. Good work — this is the sort of pre-flight gate a Neucom rollout wants in place before it leaves Port Edwards.

[talaria0101]

Re-reviewed at aa5017f, re-fetched. Everything in the reply checks out against the tree.

The arch assertion is real, not decoration. I dumped EI_DATA/e_machine from all six v0.8.1 runtime binaries: exactly one identity per arch, and the six expected pairs are unique (x86_64 0x3e LE, aarch64 0xb7 LE, riscv64 0xf3 LE, loongarch64 0x0102 LE, ppc64 0x15 BE, ppc64le 0x15 LE). So the test can in principle tell all six apart, including the ppc64/ppc64le pair that differs only in EI_DATA. Negative control on my side: injecting a wrong EI_DATA expectation for x86_64 fails the test at the new assert (tests/integration.rs:585, same line the description cites), reverting restores green.

Unit tests job: exists in ci.yml, runs cargo test --locked, green in run 36034039607. Locally at aa5017f I reproduce 64 lib + 18 integration passing, 2 ignored, clippy clean, and the smoke test passes natively with the new assertion in place.

Helper stamps: confirmed from upstream build_support.rs at both tags. DWARFS_VERSION is "0.15.7" at v0.7.1 and v0.8.1 alike, and asset_urls with dwarfs + lite selects only dwarfs-fuse-extract, so the squashfs stamps are genuinely compiled out of the flavour this crate pins. Your correction stands.

The binfmt harness note is right: I dumped the header of an AppImage built by the tool at this branch, AI\x02 sits at offset 8, and the tool's patching path touches only .upd_info, .envs and URUNTIME_MOUNT (src/uruntime.rs). Nothing in this PR changes how other tools recognise the file.

Description: both nits verified gone, and the FUSE mount-path gap is in the follow-ups with both remedies.

git diff pr35..pr35b touches only tests/integration.rs and ci.yml, so no product source changed since the head I measured for regressions. Still ready to merge.