Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions artifacts/findings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9417,13 +9417,13 @@ artifacts:
- id: AFD-150
type: ai-found-defect
title: "The Renode smoke job's Mono negative control is nondeterministic: NC1 expected exit 3, got rc=1, then an exit 134 abort"
status: open
description: "Main push run 40664674271 (b810466, #304) failed 'Renode RT1176 smoke' with 'NC0 the real archive -> PASS', then 'NC1 expected an exit-3 failure, got rc=1' and 'FAIL: unexpected exit 134: SMOKE FAIL: boot line not seen within 2 s'. The same commit's PR run passed, and every later main run passed both runtimes. 134 is SIGABRT. The failure path is the boot-failure exit (Environment.Exit(3)) on the legacy Mono runtime, where rt1176-smoke-check.resc already documents that exit codes are unreliable on load errors. DENOMINATOR: 1 run with this signature of the 12 CI runs since 2026-10-28 whose logs could be read for it; 3 other Renode failures in that window were PR iterations on #299 whose logs no longer carry a signature. With the job now REQUIRED (AFD-149), a recurrence blocks a merge until re-run. Not yet diagnosed; not yet reproduced locally (local Renode is the .NET build)."
status: resolved
description: "Main push run 40664674271 (b810466, #304) failed 'Renode RT1176 smoke' with 'NC0 the real archive -> PASS', then 'NC1 expected an exit-3 failure, got rc=1' and 'FAIL: unexpected exit 134: SMOKE FAIL: boot line not seen within 2 s'. The same commit's PR run passed, and every later main run passed both runtimes. 134 is SIGABRT. The failure path is the boot-failure exit (Environment.Exit(3)) on the legacy Mono runtime, where rt1176-smoke-check.resc already documents that exit codes are unreliable on load errors. DENOMINATOR: 1 run with this signature of the 12 CI runs since 2026-10-28 whose logs could be read for it; 3 other Renode failures in that window were PR iterations on #299 whose logs no longer carry a signature. With the job now REQUIRED (AFD-149), a recurrence blocks a merge until re-run. Not yet diagnosed; not yet reproduced locally (local Renode is the .NET build). DIAGNOSED AND FIXED 2026-11-03: the failure is not in the boot assertion but in NC1's demand for the literal exit code. In both CI failures verify.sh rejected the mutated archive (rc=1) and the .resc printed 'SMOKE FAIL: boot line not seen within 2 s' before Mono aborted in Environment.Exit(3) with 134 (SIGABRT); NC1 grepped for 'exit 3' and so reported the control as broken although the assertion had bitten. Same Mono exit-channel unreliability the .resc documents for load errors and NC2 already allows for. FIX (tools/renode-model/verify.sh): NC1 accepts, ONLY for a runtime detected as Mono, rejection AND the FAIL verdict line AND no PASS line with exit 134; .NET keeps exact exit 3. Measured: real .NET Renode 1.16.1 self-test NC0-NC5 pass via the exit-3 path; a stand-in Mono Renode that prints the verdict then SIGABRTs passes via the new branch; one-variable controls stay red: the abort WITHOUT the verdict line (runtime mono) and the same abort from a runtime detected as dotnet. Not reproduced on real Mono locally (local Renode is the .NET build); the CI Mono runtime is the confirmation."
fields:
detected-by: the AFD-149 audit of main's push runs
discovered-at: 2026-10-30
severity: minor
triage-status: open
triage-status: resolved
provenance:
created-by: ai
model: claude-opus-5-5
Expand Down
18 changes: 17 additions & 1 deletion tools/renode-model/verify.sh
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,23 @@ if [ "${1:-}" = "--self-test" ]; then
ctl() { rc=0; RENODE="$RENODE" "$me" "$1" > "$tmp/o" 2>&1 || rc=$?; echo $rc; }
r=$(ctl "$good"); [ "$r" = 0 ] && echo "NC0 the real archive -> PASS (exit 0) [positive control]" || { echo "NC0 expected 0 got $r"; cat "$tmp/o"; exit 1; }
mutate "$tmp/nc1.tgz" "p=d+'/rt1176-smoke-check.resc'; s=open(p).read().replace('JESS-RT1176 boot OK\" \"','JESS-RT1176 NEVER PRINTED\" \"'); open(p,'w').write(s)"
r=$(ctl "$tmp/nc1.tgz"); [ "$r" = 1 ] && grep -q 'exit 3' "$tmp/o" && echo "NC1 check asserts a line never printed -> FAIL, exit 3 [the assertion inside the .resc bites]" || { echo "NC1 expected an exit-3 failure, got rc=$r"; cat "$tmp/o"; exit 1; }
# NC1 — the boot assertion bites. On .NET the .resc exits 3, exactly. On MONO the same
# Environment.Exit(3) sometimes ABORTS (exit 134, SIGABRT) AFTER the verdict line is printed:
# measured twice in CI (AFD-150: 2 of 13 runs), each time with 'SMOKE FAIL: boot line not seen'
# in the output and verify.sh rejecting (rc=1). The assertion bit; the exit channel is what
# Mono garbles, exactly as the .resc documents for load errors. So, as in NC2, the Mono allowance
# is granted only to a DETECTED Mono, and it still requires the rejection AND the FAIL line AND
# no PASS line — it accepts 3 or 134 as the code, nothing else. .NET keeps the exact exit 3.
rt=$(runtime_of "$RENODE")
r=$(ctl "$tmp/nc1.tgz")
if [ "$r" = 1 ] && grep -q 'exit 3' "$tmp/o"; then
echo "NC1 check asserts a line never printed -> FAIL, exit 3 [the assertion inside the .resc bites]"
elif [ "$r" = 1 ] && [ "$rt" = mono ] && grep -q 'unexpected exit 134' "$tmp/o" \
&& grep -q 'SMOKE FAIL: boot line not seen' "$tmp/o" && ! grep -q "$PASS_LINE" "$tmp/o"; then
echo "NC1 check asserts a line never printed -> FAIL, verdict printed then exit 134 [mono aborted in Exit(3) — AFD-150; the assertion still bit]"
else
echo "NC1 expected the boot assertion to bite (exit 3, or on Mono the FAIL line then exit 134); runtime=$rt rc=$r"; cat "$tmp/o"; exit 1
fi
mutate "$tmp/nc2.tgz" "p=d+'/rt1176-smoke-check.resc'; s=open(p).read().replace('\$ORIGIN/','@hardware/renode/'); open(p,'w').write(s)"
# NC2 — a LOAD error. On EVERY runtime verify.sh must REJECT it (rc=1). On .NET the .resc
# itself exits 4. On Mono it exits 0 without the PASS line (the documented limitation), and
Expand Down
Loading