Skip to content

fix(vibe): criteria audit, rulings, evidence and e2e lane checks - #332

Closed
shafty023 wants to merge 2 commits into
fix/vibe-lessons-20261021bfrom
fix/vibe-lessons-20261021c
Closed

shafty023 wants to merge 2 commits into
fix/vibe-lessons-20261021bfrom
fix/vibe-lessons-20261021c

Conversation

@shafty023

@shafty023 shafty023 commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Purpose

vibe 1.0.75 covers every lessons-ledger entry that #331 does not. This branch is stacked on #331 (fix/vibe-lessons-20261021b) because it extends code that only exists there. It is retargeted to main once #331 merges. Each lesson got a check inside the plugin first. Prose only points at the check.

No test in this PR sends a signal to a process it did not start. Every run happened on a second machine in a throwaway copy.

Rung table

Ledger entry Rung Check Prose
Codex session state RUNNING with an idle thread code fix + test turn-start on a RUNNING record without a pending operation runs reconcileSession against the live thread first, then decides; a thread with a turn in progress is still refused none
#8960 merged with 8 audit findings: (1) independent criteria audit check criteria-audit.mjs: a fresh-context MET/PARTIAL/MISSED table on the current head, every criterion MET handoff: one paragraph
(2) coverage matrix check same script: every criterion x surface x input x flag cell needs evidence same paragraph
(3) flag-off parity check same script: a flag-off cell needs no behavior change: (a diff against the base) or ruling: same paragraph
(4) diff checks diff rules comparator-tie (a new comparator needs a recorded tie test that exists) and unused-export (a new export nothing else names blocks unless recorded). Agent-authored aria copy is the existing copy-provenance rule none
(4) removed props still defined but never passed none fits prose only, and none added. A typed host object is a repo type-check rule, not something a text diff can see none
(5) client/server capability parity none fits prose only, and none added. Which rows a mutation should offer is domain knowledge, so the criteria audit and matrix carry it none
Rulings made mid-work never reach the criteria check rulings-check.mjs: every Ruling line in the QA record or resume log is on the ticket with amends:; the ticket template carries the line fleet principle 1: extended
QA findings sent to a remote writer without evidence script + check finding-brief.mjs refuses a finding with no story or route, viewport, and selector or screenshot; it copies screenshots to <prompt>.evidence/, which fleet.sh now ships fleet QA: one sentence
Precedent check skipped the disputed sub-question check precedent-check.mjs --behavior (repeatable) puts the items in the brief and checklist.json; check refuses until each is observed: ...; screenshot: ... or needs a write to test quality-loop: one sentence

Closeout per class

  • Stale RUNNING state: the instance is fixed by trusting the thread over the record. The same-shaped run-turn path refuses from RUNNING too, but it starts and waits on a turn and was not in the incident, so it is unchanged. The fix is made in the plugin's copy. The user-level app-server-orchestrator skill outside this repo carries the same code and needs the same change there.
  • Audit gaps: the audit, the matrix and the flag-off rule are one script, so handoff runs one command. The two new diff rules block with a fix line, like the existing ones. Existing fixtures in two diff-rules tests and three handoff-inventory tests added exports that nothing used, which the new rule flags. The diff-rules fixtures gained a file that imports the export. In the inventory fixtures the incidental export keyword was dropped. No assertion changed.
  • Rulings, evidence, precedent: each has a refusing check plus tests for both the refused and the accepted shape.

Ledger entries

Taken: every entry without "in PR #331", except one. Skipped: "our merges broke main's Desktop E2E; heavy suites never ran on the PR". The operator ruled that skipping pre-merge e2e was a deliberate trade for a deadline and not a standing plugin rule, so this PR adds no check, gate or prose for it. Two sub-items of the #8960 entry have no check, with reasons above.

Commands run

All on the second machine, in a throwaway copy: node --test over every plugins/**/*.test.{mjs,cjs,js}: 645 of 648 passed. The 3 failures were handoff-inventory fixtures the new unused-export rule flagged; after the fixture fix above, that file and its sibling ran 28 of 28. uv run pytest plugins/: 2553 passed. uv run ruff check . and uv run pyright: clean. tools/vibe-workers is unchanged.

- turn-start on a RUNNING record whose thread is idle reconciles
  against the live thread before starting, instead of refusing.
- handoff: criteria-audit.mjs (fresh-context MET audit on the final
  head, criteria x surface x input x flag matrix, flag-off parity) and
  rulings-check.mjs (every ruling on the ticket with what it amends);
  qa-merge-order.mjs --evidence needs each changed e2e spec passing at
  the head in its lane; diff rules for comparators without a tie test
  and new exports nothing uses.
- precedent-check.mjs --behavior makes each disputed behavior a
  required checklist item.
- fleet-orchestrator: finding-brief.mjs refuses findings without
  evidence and fleet.sh ships the screenshots with the prompt.
- vibe 1.0.75 and its changelog.

Testing: node --test and pytest over the plugins on a second machine.
Risks: the unused-export rule blocks a handoff that adds an export
nothing names until it is removed or recorded with Export:.
- The operator ruled that skipping pre-merge e2e was a deliberate
  trade for a deadline, not a standing plugin rule, so the check,
  its prose and its changelog line come out.

Testing: qa-merge-order tests and the vibe contract tests on a second
machine.
Risks: None identified.
@shafty023

Copy link
Copy Markdown
Collaborator Author

Folded into #334 as part of the combined vibe 1.0.74 lessons release.

@shafty023 shafty023 closed this Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant