Skip to content

ISS-12135: fix(vibe): gate pull request bodies on facts from the diff - #313

Closed
shafty023 wants to merge 3 commits into
mainfrom
fix/vibe-pr-facts-gate
Closed

shafty023 wants to merge 3 commits into
mainfrom
fix/vibe-pr-facts-gate

Conversation

@shafty023

@shafty023 shafty023 commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Purpose

Vibe pull request bodies kept stating the wrong facts: overlaps listed from memory, flipped and deleted assertions left unnamed, a test change attributed to a file whose diff only removed a comment, "additive" on bodies that added refusals, ticket asks left unanswered, and pull requests opened while device and play runs were still pending. The prose rules for these did not hold, so this replaces them with a check in a step vibe already runs.

The handoff inventory (handoff-inventory.mjs, already run in step 1 and with --phase pull-request on the pull request path) now reports the facts from the diff: serverOnly, requiredLanes, assertionChanges, contractChanges, refusals, asks (with --ticket) and overlap (with --overlap: one gh pr list call). In the pull-request phase, --body adds blocking.prBody and --runs --commit adds blocking.prEvidence. The environment worker builds the body from those facts and runs the gate in its existing pull-request step; it creates nothing while either is false. The logic lives in pr-facts.mjs, an imported module with no CLI, because folding it in would take the inventory far past 500 lines. No new handoff step: a server-only diff now skips Storybook, Desktop runs and visual QA through the step 1 inventory.

Size

  • Whole PR: 19 files, +517 / -41 lines (module 235, its tests 121, inventory +48).
  • Instruction length (all vibe agents/*.md and skills/**/*.md, evals excluded), merge-base f65aec2 vs head: 15,647 to 15,646 lines, 149,444 to 149,453 words (-1 line, +9 words). The newly requested PostHog line accounts for about 19 words; without it the gate's net is about -10 words. Removed: the handoff hard-gate and play-evidence paragraphs and the verify worker's evidence sentence, which the gate now enforces.
  • Session wall time: shorter for server-only sessions, which skip handoff step 4 (Storybook), the Desktop build and control runs, and visual QA (not measured). Every handoff-phase inventory run adds one git diff -U0 (subsecond). The pull request path adds two gh pr list calls (seconds). A pull request with pending runs now waits for them instead of opening.

Ledger lines

Ledger lesson Encoding
#8947 overlap missed open #8938 changing a type the diff consumes overlap includes imported files; requirements worker step 4
#8947 body attributed a test change to a file that only removed a comment blocking.prBody: every cited path:line must be a changed line
#8948 / #8953 removed guards and inverted expectations unnamed, "presentation-only" assertionChanges gated; summarizer lists removed guards and gate predicates
#8956 changed an undo inverse while claiming contracts compatible contractChanges gated
#8956 / #8957 / #8962 / #8963 / #8965 overlaps understated or missed overlap from gh pr list files, each PR number required by the gate
#8956 ticket asked "say why POST vs DELETE in the PR" asks gated: quoted and answered
#8957 / #8959 / #8965 fixture flips and deleted assertions hidden, "additive" assertionChanges (incl. deleted test files) gated
#8962 cited a TRIAGE ticket as the ruling summarizer: rulings only from an approved artifact or a quoted operator message
#8963 new 409 on generic link delete, "nothing breaks" refusals gated; Breaking changes "none" only when empty
#8965 body claimed mechanisms the code does not have cited lines must be changed lines
#8967 claimed CI authoritative on a stacked PR gate requires the stacked-base sentence when the base is not main
#8948 Desktop and Storybook ceremony for a server-only diff serverOnly from the step 1 inventory
#8958 / #8963 dependency and open PRs over the same service (overlap half) requirements worker: resolve "Roadmap N" and dependencies to PRDs and open PRs
#8987 bodies kept "(flag-pending)" after the PostHog key existed environment worker re-evaluates PostHog before marking a key pending
#8953 / #8956 / #8960 hardening of the shipped device gate blocking.prEvidence: pending, running, missing or wrong-commit runs block

Not encoded here: the "never build a parallel path" half of #8958/#8963, which belongs to the separate tech-debt guardrail PR.

Affected files

  • plugins/vibe/skills/handoff/scripts/handoff-inventory.mjs; pr-facts.mjs and pr-facts.test.mjs (new)
  • plugins/vibe/skills/handoff/SKILL.md; agents vibe-environment-worker, vibe-handoff-summarizer, vibe-requirements-worker, vibe-verify-worker
  • plugins/vibe/tools/python/test_handoff_skill_contract.py (one added test)
  • plugins/vibe/evals/pm-pr-body-facts-block-create/ (new), plugins/vibe/evals/codex/test_run.py (71 cases)
  • vibe 1.0.66 in both manifests, CHANGELOG.md, plugins/vibe/README.md

The vibe version may need re-resolving at merge: #310 claims 1.0.59, #312 claims 1.0.60, and other open vibe pull requests claim nearby versions.

Commands run

  • node --test plugins/vibe/skills/handoff/scripts/*.test.mjs: 53 passed
  • uv run pytest plugins/vibe: 335 passed
  • uv run ruff check .: clean; uv run pyright: 0 errors
  • tools/vibe-workers vitest, the two main-sync.test.ts cases that spawn the inventory: 2 passed

- Add pr-facts.mjs: overlap by shared and imported files, assertion and
  contract changes, new refusals, ticket asks, server-only lanes and run
  evidence, with a check that refuses an incomplete or miscited body
- Build and refresh pull request bodies from those facts and block creation
  while required runs are pending
- Run the same overlap at requirements time, including named dependencies
- Add node tests, contract tests and two shared eval cases
- Report the facts from handoff-inventory.mjs and gate the body and run
  evidence as pull-request phase options, through an imported module with
  no CLI of its own
- Delete the prose the gate replaces and the separate lanes step
- Re-evaluate PostHog before marking a flag key pending
- Keep one eval case and pin the wiring in the existing handoff contract test
# Conflicts:
#	CHANGELOG.md
#	plugins/vibe/.claude-plugin/plugin.json
#	plugins/vibe/.codex-plugin/plugin.json
#	plugins/vibe/README.md
#	plugins/vibe/agents/vibe-environment-worker.md
@shafty023

Copy link
Copy Markdown
Collaborator Author

Folded into #317.

@shafty023 shafty023 closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant