From d40ef963e9a6798245c3a46c66e8f7ededbcebf0 Mon Sep 17 00:00:00 2001 From: Lucas Faria Date: Thu, 5 Nov 2026 16:23:04 -0300 Subject: [PATCH 1/2] feat(interrogate): add exposure, isolation, and upgrade lenses for SDK changes Review bots on Lucas's MCP SDK PRs (posthog-js, posthog-python) kept finding three bug classes the gate's lenses never point at: captured data that skips redaction, state leaking between callers or instances, and default behavior that changes for users who upgrade without touching code. - blind-spots.md gains Exposure, Isolation, and Upgrade lenses. - Frame makes those three mandatory for SDK and package changes, and folds the touched directories' ADRs into the intent, so a change that reverses a recorded decision breaks the intent. - Babysit step 9 runs the correct skill once a review finding class repeats. - translation.md records both as deliberate departures. Tested: bun scripts/check-port.ts -> 0 findings. --- .../skills/sync-upstream/references/translation.md | 3 ++- skills/interrogate/SKILL.md | 4 +++- skills/interrogate/references/blind-spots.md | 12 ++++++++++++ skills/poteto-mode/playbooks/babysit.md | 2 +- 4 files changed, 18 insertions(+), 3 deletions(-) diff --git a/.claude/skills/sync-upstream/references/translation.md b/.claude/skills/sync-upstream/references/translation.md index 8850e41..6e8adc4 100644 --- a/.claude/skills/sync-upstream/references/translation.md +++ b/.claude/skills/sync-upstream/references/translation.md @@ -77,8 +77,9 @@ A sync must keep these even when upstream's line differs. - Orchestrate keeps its state in `~/.claude/projects//pstack/orchestrate//notes.md`, where upstream uses Cursor's Project tracker and store. Owners merge through `merge-gate`. In a `gh stack`, one owner is the single writer, because `gh stack sync` and `rebase` push every branch. The verifier runs on the other of `opus` and `sonnet` from the owner, since upstream's different-family rule has no all-Claude equivalent. - Arena's cross-judge is the first `arena cross-judge pool` entry whose model differs from the parent's. Upstream picks by model family, and the port's pool is all Claude, so a family rule would always skip the judge. Opus and sonnet count as different judges. - `.claude-plugin/plugin.json` carries no `version`, where upstream's `.cursor-plugin/plugin.json` does. Claude Code then versions a GitHub-hosted install by commit SHA, so every merge to `main` reaches `claude plugin update`. A version string that does not change blocks updates. The `plugin-version` rule in `scripts/check-port.ts` keeps it out. `claude plugin validate` warns about the missing version. That warning is expected. -- interrogate adds the blind-spot lenses (`references/blind-spots.md`, chosen in Frame and passed to every adversary), a PR number or URL as scope with each adversary's worktree on the PR head, and the someone-else's-PR flow (`references/review-comments.md`), all folded in from Lucas's retired pr-review-swarm skill. When a playbook runs it as the pre-PR gate, it hands the findings back instead of ending the turn. The gate's rules live in Opening a PR step 2, which applies every verified finding or treats it as a blocker. Upstream dropped interrogate from opening a PR. The port keeps it as the gate. It runs no tool-based analyzers: a CodeScene change-set step was tried and removed after an eval showed the reviewers already covered everything it found (PR #2). +- interrogate adds the blind-spot lenses (`references/blind-spots.md`, chosen in Frame and passed to every adversary; Exposure, Isolation, and Upgrade are mandatory for SDK and package changes), a Frame step that folds the touched directories' decision records into the intent, a PR number or URL as scope with each adversary's worktree on the PR head, and the someone-else's-PR flow (`references/review-comments.md`), all folded in from Lucas's retired pr-review-swarm skill. When a playbook runs it as the pre-PR gate, it hands the findings back instead of ending the turn. The gate's rules live in Opening a PR step 2, which applies every verified finding or treats it as a blocker. Upstream dropped interrogate from opening a PR. The port keeps it as the gate. It runs no tool-based analyzers: a CodeScene change-set step was tried and removed after an eval showed the reviewers already covered everything it found (PR #2). - `skills/poteto-mode/scripts/merge-gate/` is port-only. It reuses watch-pr's readiness verdict without editing watch-pr, and adds the policy: allowed repos, human-only paths needing a human approval on the head, and head pinning with `--match-head-commit`. A PR in a GitHub native stack refuses `gh pr merge`, so the gate falls back to the merge-async REST endpoint with the same head SHA. `--stack` gates native stack positions 1 to the target as one unit: each open PR runs the same per-PR decision, with its expected base being the head of the open PR below it instead of trunk, and one merge-async call on the target, pinned to its head, lands the range atomically. merge-async pins only that head, so the gate re-reads every head and the stack just before the call and refuses a merged head it did not gate (`merged-ungated-head`). Upstream has no native stacks. Every playbook merges only through it, and a refusal is final: human-only changes are the user's to merge. The policy lives outside git, and editing it is on poteto-mode's always-pause list. +- Babysit step 9 runs the **correct** skill when a fixed review finding is the second of its class in the repo. Upstream has no link from review triage to correct. - `principle-never-block-on-the-human` narrows its force-push example to "force-push to a shared branch", matching poteto-mode's Autonomy. Autopilot owners force-push their own branches with a lease on every rebase. - models.md uses only `opus` and `sonnet`. `fable` is dropped for cost: every role that upstream maps to it, and every panel slot, goes to `opus` or `sonnet`. interrogate runs two adversaries (`opus`, `sonnet`), accepting that both share one family and give less independent review than a cross-family pair. - Autopilot-stack owners never force-push a branch once the root links it into the `gh stack`. They only add commits, and take the root's push as their new base and lease. Moving a PR out of a stack is the user's, since `gh stack modify` is interactive. diff --git a/skills/interrogate/SKILL.md b/skills/interrogate/SKILL.md index f4255c0..7a20cca 100644 --- a/skills/interrogate/SKILL.md +++ b/skills/interrogate/SKILL.md @@ -11,7 +11,9 @@ Adversaries try to break a change or a design. A finding survives only when a ru The scope is what the user names. Given a PR number or URL, it is that PR: read it with `gh pr view --json title,body,baseRefName,headRefOid,files` and `gh pr diff `. Otherwise it is the branch diff from its merge base with the base branch, through the working tree. State the intent with the **in-your-own-words** skill (`${CLAUDE_SKILL_DIR}/../in-your-own-words/SKILL.md`), from the user's message, the commits, and the PR description. Ask the user when the intent is unclear. -Pick the blind-spot lenses from `${CLAUDE_SKILL_DIR}/references/blind-spots.md` that the change calls for, or none. Diff-scoped attacks cannot see what is missing, and the lenses cover that gap. +Pick the blind-spot lenses from `${CLAUDE_SKILL_DIR}/references/blind-spots.md` that the change calls for, or none. Diff-scoped attacks cannot see what is missing, and the lenses cover that gap. A change to an SDK or another package that other code installs always takes Exposure, Isolation, and Upgrade. Its users run it in setups no test of the author's covers, so those failures surface first in their production, not in review. + +Read the decision records (ADRs, design docs) in the directories the change touches, and add the decisions that bear on it to the intent. A change that quietly reverses one breaks the intent. ## Fan out diff --git a/skills/interrogate/references/blind-spots.md b/skills/interrogate/references/blind-spots.md index ff30e1c..123a3cc 100644 --- a/skills/interrogate/references/blind-spots.md +++ b/skills/interrogate/references/blind-spots.md @@ -13,3 +13,15 @@ Use when the code handles a protocol, a state machine, a lifecycle, or an input ## Interactions Use when options or features can combine, or a resource has a create, use, dispose lifecycle. List the option combinations and check each one end to end, not each option alone. Follow every created, scheduled, or acquired thing (tasks, futures, files, locks, connections, subscriptions) to its await, close, or release. Flag type mismatches and paths that skip cleanup. + +## Exposure + +Use when the change writes anything that leaves the process: telemetry, logs, error reports, responses, files. List every value the change sends to such a destination and trace each one back to its source. Put a canary secret in each untrusted input, including metadata, headers, identifiers, URLs, and error text, and run every path that writes to the destination, not only the new one. Report every place the canary arrives unredacted, and every new field that skips the redaction the existing fields pass through. + +## Isolation + +Use when the change touches state that outlives one request or call: instance fields, module-level caches, maps keyed by name, anything cleared and rebuilt. Interleave two callers with different identities and inputs, once on one shared instance and once on a fresh instance per request. Report any value from one caller that shows up in the other's output, any result that depends on which instance served an earlier request, and any window where a reader sees state half rebuilt. + +## Upgrade + +Use when the change ships in a package other code installs. Act as a user who upgrades and changes no code. Run their setup with default options before and after the change, and diff everything they can observe: payloads, schemas, return values and their identity, the arguments that reach their own handlers, thrown errors. Hold the change to every promise the docs make, such as "never throws" or "leaves your input unchanged". Report each difference the changelog does not name, and each one that breaks the promise of the version bump it ships under. diff --git a/skills/poteto-mode/playbooks/babysit.md b/skills/poteto-mode/playbooks/babysit.md index 3a79611..0b1e419 100644 --- a/skills/poteto-mode/playbooks/babysit.md +++ b/skills/poteto-mode/playbooks/babysit.md @@ -18,7 +18,7 @@ Babysitting starts when the user asks for it, which is normally once a phase or Answer a user question mid-loop and continue. Only an explicit stop ends the loop before the watcher's stop condition: `READY` in single or stack mode, or a `WAITING`/`merge-queue` report or `COMPLETE` in queued mode. For a queued stack, capture the PR list bottom-to-top once and pass the same frozen list to every rearm. Revise the list only for the sanctioned follow-up PR from step 4. Append it at the end, drop the merged owner, and rearm with the corrected snapshot. 7. **Classify CI before any retrigger.** Flake or infrastructure earns one fresh build, never a job retry. One retry only. An identical second failure means it was never flake, so reclassify and read the child logs instead of retrying blind. A failure in code the diff never touches means a stale base, so check with `git merge-base --is-ancestor` before assuming flake. Report a stale base as needing a rebase instead of burning retries. Only a failure in the diff's own code gets a commit. 8. **Review bots are triaged skeptically, always.** Verify each claim against the code per `../references/bugbot-triage.md`. Fix real findings with a red-first proof in the lowest PR that owns the code, never at the tip unless the owning PR has merged. In that case, use step 4's sanctioned follow-up PR. Per step 2, upstack fixes wait for step 5's next frontier-driven push wave. Push that wave before replying so the reply cites the commit. Reply with `gh api --method POST "repos///pulls//comments//replies" --input ` and put the reply body in the JSON file as data. Never interpolate comment text or a reply into a shell command. Dismiss noise with the concrete disproof on the thread. Count passes with the watcher's `bugbotReviewPasses`, or from the review history for other bots. From the third pass on, lean toward dismissing documented patterns, still escalating anything touching security, auth, billing, data, or migrations rather than dismissing it yourself. Never churn code to quiet a bot. -9. **Stop at the human's line.** Owner approval is a wait, not a blocker to fix. Babysitting merges only through merge-gate, and its refusal is final. Any other merge needs an explicit request to merge, land, ship, or merge when ready. Route that request to Shipping. Surface the escalation and keep working the rest. After the watcher reports `READY`, a queued `WAITING`/`merge-queue` stop, or `COMPLETE`, sweep the run's triage decisions once. Offer any team-useful dismissal pattern as a candidate entry in the shared rubric (`../references/bugbot-triage.md`) and its own PR. Never keep it only in private memory. +9. **Stop at the human's line.** Owner approval is a wait, not a blocker to fix. Babysitting merges only through merge-gate, and its refusal is final. Any other merge needs an explicit request to merge, land, ship, or merge when ready. Route that request to Shipping. Surface the escalation and keep working the rest. After the watcher reports `READY`, a queued `WAITING`/`merge-queue` stop, or `COMPLETE`, sweep the run's triage decisions once. Offer any team-useful dismissal pattern as a candidate entry in the shared rubric (`../references/bugbot-triage.md`) and its own PR. Never keep it only in private memory. When a fixed finding is a class of mistake that has now reached review twice in this repo, run the **correct** skill on that class in its own PR, so the repo stops the next one before review. `drive` ends at merge-ready. Landing the stack is `playbooks/shipping.md`. From 950f8ab082fafada2be4897cbf49be512b034fb0 Mon Sep 17 00:00:00 2001 From: Lucas Faria Date: Thu, 5 Nov 2026 16:25:52 -0300 Subject: [PATCH 2/2] fix(babysit): run correct only in modes that may change things check and threads-only touch nothing (step 6), so they offer the correct PR instead of opening it. Found by the pre-PR gate's opus adversary. Tested: bun scripts/check-port.ts -> 0 findings. --- skills/poteto-mode/playbooks/babysit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/poteto-mode/playbooks/babysit.md b/skills/poteto-mode/playbooks/babysit.md index 0b1e419..5f65425 100644 --- a/skills/poteto-mode/playbooks/babysit.md +++ b/skills/poteto-mode/playbooks/babysit.md @@ -18,7 +18,7 @@ Babysitting starts when the user asks for it, which is normally once a phase or Answer a user question mid-loop and continue. Only an explicit stop ends the loop before the watcher's stop condition: `READY` in single or stack mode, or a `WAITING`/`merge-queue` report or `COMPLETE` in queued mode. For a queued stack, capture the PR list bottom-to-top once and pass the same frozen list to every rearm. Revise the list only for the sanctioned follow-up PR from step 4. Append it at the end, drop the merged owner, and rearm with the corrected snapshot. 7. **Classify CI before any retrigger.** Flake or infrastructure earns one fresh build, never a job retry. One retry only. An identical second failure means it was never flake, so reclassify and read the child logs instead of retrying blind. A failure in code the diff never touches means a stale base, so check with `git merge-base --is-ancestor` before assuming flake. Report a stale base as needing a rebase instead of burning retries. Only a failure in the diff's own code gets a commit. 8. **Review bots are triaged skeptically, always.** Verify each claim against the code per `../references/bugbot-triage.md`. Fix real findings with a red-first proof in the lowest PR that owns the code, never at the tip unless the owning PR has merged. In that case, use step 4's sanctioned follow-up PR. Per step 2, upstack fixes wait for step 5's next frontier-driven push wave. Push that wave before replying so the reply cites the commit. Reply with `gh api --method POST "repos///pulls//comments//replies" --input ` and put the reply body in the JSON file as data. Never interpolate comment text or a reply into a shell command. Dismiss noise with the concrete disproof on the thread. Count passes with the watcher's `bugbotReviewPasses`, or from the review history for other bots. From the third pass on, lean toward dismissing documented patterns, still escalating anything touching security, auth, billing, data, or migrations rather than dismissing it yourself. Never churn code to quiet a bot. -9. **Stop at the human's line.** Owner approval is a wait, not a blocker to fix. Babysitting merges only through merge-gate, and its refusal is final. Any other merge needs an explicit request to merge, land, ship, or merge when ready. Route that request to Shipping. Surface the escalation and keep working the rest. After the watcher reports `READY`, a queued `WAITING`/`merge-queue` stop, or `COMPLETE`, sweep the run's triage decisions once. Offer any team-useful dismissal pattern as a candidate entry in the shared rubric (`../references/bugbot-triage.md`) and its own PR. Never keep it only in private memory. When a fixed finding is a class of mistake that has now reached review twice in this repo, run the **correct** skill on that class in its own PR, so the repo stops the next one before review. +9. **Stop at the human's line.** Owner approval is a wait, not a blocker to fix. Babysitting merges only through merge-gate, and its refusal is final. Any other merge needs an explicit request to merge, land, ship, or merge when ready. Route that request to Shipping. Surface the escalation and keep working the rest. After the watcher reports `READY`, a queued `WAITING`/`merge-queue` stop, or `COMPLETE`, sweep the run's triage decisions once. Offer any team-useful dismissal pattern as a candidate entry in the shared rubric (`../references/bugbot-triage.md`) and its own PR. Never keep it only in private memory. When a fixed finding is a class of mistake that has now reached review twice in this repo, `drive` and `background` run the **correct** skill on that class in its own PR, so the repo stops the next one before review. `check` and `threads-only` offer that PR instead. `drive` ends at merge-ready. Landing the stack is `playbooks/shipping.md`.