Skip to content

fix(vibe): signal safety after kill(-1), and the 2026-10-21 fleet lessons - #331

Closed
shafty023 wants to merge 5 commits into
mainfrom
fix/vibe-lessons-20261021b
Closed

fix(vibe): signal safety after kill(-1), and the 2026-10-21 fleet lessons#331
shafty023 wants to merge 5 commits into
mainfrom
fix/vibe-lessons-20261021b

Conversation

@shafty023

@shafty023 shafty023 commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Incident: a plugin test ended every process on the operator's laptop

On 2026-10-21, at 15:38 and again at 15:48, every app on Daniel's laptop (Slack, Chrome, iTerm, Claude) died at once. The cause was the in-progress version of this PR. Its fleet-orchestrator helpers.test.mjs recorded pid 1 for a helper and then ran helpers.mjs stopped. endProcesses counted EPERM as "alive" and sent process.kill(-pid, SIGTERM), then SIGKILL. With pid 1 that call is kill(-1), which signals every process the user owns. Nothing was lost on disk, but every open session and app was killed, twice.

This PR fixes it, and nothing that sends signals was run on that laptop again. Every test run behind this PR happened on a second machine in a throwaway copy, and every safety check below was shown to fail with its fix removed. Row (e) was added after review:

Check What it enforces Shown failing by
endProcesses own-group rule A whole group is signalled only for a pid recorded with helpers.mjs pid --group (the helper spawned it detached) whose process group id (ps -o pgid=) still equals the pid. Any other pid is signalled alone putting back the old group-first signalling: the group test fails
EPERM is not ours A pid the system reports as not ours (EPERM, for example a reused pid) is never signalled and does not count as a survivor. isAlive still reports it counting EPERM as alive again: the EPERM test fails
Unsafe pids pid refuses to record 1 or lower. endProcesses never probes or signals a pid of 1 or lower, its own pid or its parent's removing both guards: the unsafe-pid test fails. endProcesses takes an injected kill, and the fake pids sit above every real pid_max, so even that failing run sent no real signal
(e) Pid reuse pid stores each process's start time and command (ps -o lstart=,command=) and refuses a pid it cannot read. Every signal re-reads them first, through an injected reader, and signals only on an exact match. A reused pid (one of the user's own apps), a reused pid that leads its own group the way a launchd app does, a pid reused during the grace, and an older record with no identity are each treated as not ours and never signalled making the identity check always pass: all four identity tests fail. The tests use the injected kill and reader, so no real signal was sent
Guard test (c) test_vibe_signal_safety_contract.py: no plugin test passes 0, 1, a negative literal, process.pid or process.ppid to a call that sends a signal, writes one into a pids ledger, or calls endProcesses without an injected kill putting back process.kill(process.pid, 'SIGKILL') in cli-worker-session.test.mjs and pids: [process.pid] in helpers.test.mjs: the test fails
Static scan (d) The same file, in the plugin's source-guard style: no source signals a group (process.kill(-x), kill(-x), kill -- -x) unless x is checked above 1 in the lines just before removing the guard in run-bounded.mjs: the test fails

The scan covers plugins/vibe and tools/vibe-workers/src. Every existing group signal now has a guard: run-bounded.mjs, precedent-check.mjs, the vibe-workers ledger.ts and claude-worker.ts (dist rebuilt), and two tests. The fake Codex in cli-worker-session.test.mjs used to SIGKILL itself. It now ends with process.abort() after closing stderr, and the assertion expects SIGABRT. The contract (a process killed by a signal reports that signal) is unchanged. Only the signal changed, so that no test signals its own pid. The rule for agents is in fleet-orchestrator: never signal pid 1 or lower or a group you did not spawn, and run signal-sending tests off the operator's daily machine.

Purpose

vibe 1.0.74: the signal safety work above, plus every ledger entry added after #329 was opened (vibe, fleet-orchestrator and codex-fleet-monitor). Each lesson got the highest rung that fits, which is a script or test inside the plugin. Prose was added only to point readers at the check.

Rung table

Lesson class Rung Check Prose
A test signalled pid 1 (P0) guard test + static scan + safe API helpers.mjs own-group and EPERM rules; test_vibe_signal_safety_contract.py fleet Machines: one sentence
Writers could not see each other; a QA guide was not told its checkpoint was ready script with a status gate board.mjs (STALE rows, UNTOLD deliveries, exit 1) fleet "Fleet board and direct messages"; worker preamble gives the messaging recipe
A UX ruling reached one writer as text script board.mjs deliver --kind ruling needs --reference, and stays UNTOLD until every renderer's writer and guide is told same section (code_grep for renderers, one shared component)
Precedent means the live app check precedent-check.mjs check refuses a public or undocumented answer unless --no-browser <reason> quality-loop: one sentence
Push cadence check on every commit push-cadence.mjs; commit-worktree.mjs reports pushDue vibe step 3 and fleet QA: one sentence each
Final main merge blocked on e2e none fits prose only: the order of push and e2e is a session choice that no script sees before it happens handoff and fleet: one sentence
Self-shepherding script shepherd.mjs (two-minute polls; exits on conflict, failed checks, merge or close) handoff and fleet: one sentence
Human QA only when necessary check qa-plan-check.mjs refuses a human checkpoint without a reason an agent cannot cover, a visual one included fleet agent-first QA line rewritten (not added)
QA reruns used the stack handed to the human check helpers.mjs hand/unhand; start --stack on a handed stack exits 1 fleet QA: one sentence
Stopped helpers left processes running check helpers.mjs pid, stopped ends them and refuses while any lives, status lists ORPHAN fleet Claude workers: extended
Watcher lapse unnoticed check watch.sh writes watcher.json; watcher.mjs prints MISSING (exit 1) monitor Run and fleet Monitoring: one sentence each
Containerized e2e collided on a fixed compose project script e2e-lock.mjs run sets a per-run COMPOSE_PROJECT_NAME; --foreign-compose keeps the stack busy while an outside stack runs handoff e2e and fleet Machines: one sentence each
Supervisor restart and throttle template + test plist: bootout then bootstrap, ThrottleInterval 30; templates.test.mjs monitor supervisor: one sentence
Migration timestamps out of order gate before push main-sync-push refuses (migration-order.mjs) environment.md: one sentence
A file-scanning gate no selection picks (prototype import) gate handoff diff rules read the repo's .vibe/path-gates.json; a matching gate is a required run on the published commit verify worker: one sentence

Instruction length: about 25 lines of skill and agent prose added, all of it pointing at a check. No existing rule was replaced, because none of these lessons had prose before.

Closeout per class

  • P0 signals: instance fixed (pid 1 refused, own-group rule, EPERM skipped). Same mistake elsewhere: every group signal in the plugin and vibe-workers is now guarded, and the two tests that used their own pid as a signal target or ledger entry were changed. Prevention: the guard test and the scan.
  • Board, rulings, checkpoints: before, nothing recorded who had to be told. Now board.mjs status fails until each recipient is marked told.
  • Precedent: run already looked through the browser first. The gap was recording a help-center answer, and check now refuses it.
  • Push cadence, shepherding, final merge: scripts in vibe, so a person using vibe without an orchestrator gets them too. The fleet side only points at them.
  • Human QA: qa-plan-check.mjs reads the guided-manual-qa record's Routing line. closedloop-core is not changed here.
  • Processes, watcher, stacks, compose, supervisor, migrations, path gates: each has a script plus a failing-first test in its own *.test.mjs or pytest file.

Ledger entries

Taken (heading date 2026-10-21):

  • launchctl kickstart hung the supervisor → plist and monitor note.
  • "what does Asana do" means the live app → precedent-check.mjs check.
  • stopped QA agents left app stacks running → helpers.mjs pid/stopped/ORPHAN.
  • local containerized e2e tore down another session's stack → per-run compose project, --foreign-compose.
  • QA guide never told its checkpoint was ready → board.mjs deliver --kind checkpoint.
  • the fleet watcher lapsed for 2 hours → watcher.json, watcher.mjs.
  • a UX precedent reached one writer as text → board.mjs deliver --kind ruling --reference.
  • writers didn't know what other writers were doing → board.mjs, messaging recipe.
  • the critical-path PR sat unpushed for 5 hours → push-cadence.mjs, pushDue.
  • a plugin test killed every process (P0) → the incident section.
  • #8947's prototype import broke a file-scanning gate → .vibe/path-gates.json.

From the brief, without their own ledger heading: final merge does not block on e2e, self-shepherding, zero human QA by default, QA reruns never use the handed stack, migration order before every push. Each is covered above.

Skipped: none. Every entry without an "in PR" or "not a plugin change" status is taken. Entries marked "in PR #329" or earlier were checked against origin/main (vibe 1.0.73) and are already encoded.

Commands run

All on the second machine, in a throwaway copy of this branch, never on the operator's laptop:

  • node --test over every plugins/**/*.test.{mjs,cjs,js}, as CI runs it: 626 of 627 passed. The one failure was the fleet skill's generic test reading QA-001 fixture ids as ticket ids, which the last commit fixes and which passed on rerun. An earlier pass at load average 46 also failed two git clone fixtures (maintenance.lock) and the run-bounded group test. All three passed on the next pass, and none touches changed code.
  • uv run pytest plugins/: 2544 passed. uv run ruff check . and uv run pyright: clean.
  • tools/vibe-workers: npm run typecheck clean, and npm run build produced the dist bundles committed here. npm test timed out in five tests while the load average was 46. GitHub's vibe-workers job runs the same suite and the dist check, and both are green.
  • The safety mutations in the incident section, each run only against the test that catches it.
  • After the pid-reuse fix, helpers.test.mjs and run-bounded.test.mjs again: 19 of 19 passed. The run-bounded group test now waits, bounded (every 50 ms for up to 5 s), for the killed grandchild to be reaped before it asserts ESRCH; it had failed once at load 46, then passed 3 of 3 alone at lower load.

Static checks (ruff and the two new pytest files) also ran on the primary laptop; they send no signals.

@shafty023
fix(vibe): never signal pid 1 or an unowned process group
a5e5a0b 
- A helpers.test.mjs run recorded pid 1 and stopped it; endProcesses
  counted EPERM as alive and sent kill(-1), which signals every process
  the user owns. It ended every app on the operator's machine twice.
- helpers.mjs refuses to record a pid of 1 or lower. endProcesses never
  probes or signals one, nor its own or its parent's pid. It signals a
  whole group only for a pid recorded with --group whose pgid still
  equals the pid, and never signals a pid that EPERM says is not ours.
  It takes an injected kill, so its tests send no real signal.
- Every other group signal (run-bounded, precedent-check, the
  vibe-workers ledger and Claude worker, two tests) checks the pid
  above 1 first; dist rebuilt.
- test_vibe_signal_safety_contract.py fails when a plugin test signals
  0, 1, a negative literal or its own pid, writes one into a pids
  ledger, or calls endProcesses without a fake kill, and when any
  source signals a group without checking the pid above 1.
- The cli-worker fake Codex ends with process.abort() instead of
  signalling its own pid.

Testing: node --test, pytest, ruff, pyright and the vibe-workers
vitest and build on a second machine; each safety check shown failing
with its fix removed.
Risks: a helper started without --group now has only its own pid
signalled, so children it spawned in another group need their own
recorded pids.
@shafty023
fix(vibe): encode the open 2026-10-21 fleet lessons
a70cbb3 
- fleet-orchestrator: board.mjs keeps the fleet board, flags rows
  behind the pushed head, and tracks rulings and ready QA checkpoints
  until every writer and QA guide is told directly. qa-plan-check.mjs
  defaults to zero human checkpoints. helpers.mjs records and ends a
  helper's processes, lists orphans, and keeps a handed stack from QA
  reruns.
- codex-fleet-monitor: watch.sh writes watcher.json and watcher.mjs
  reports a lapsed watcher; the supervisor template restarts with
  bootout then bootstrap and throttles 30 seconds.
- vibe: push-cadence.mjs and pushDue on every commit, shepherd.mjs for
  the final stage, main-sync-push refuses out-of-order migrations,
  e2e-lock.mjs gives each run its own compose project, and
  precedent-check.mjs check refuses an answer no live look settled.
- handoff: repo-declared path gates (.vibe/path-gates.json) become
  required runs on the published commit.
- Skill prose points at each check; vibe 1.0.74 and its changelog.

Testing: node --test, pytest, ruff, pyright and the vibe-workers
vitest and build on a second machine.
Risks: None identified; every new check is additive and a repo
without .vibe/path-gates.json requires no gate.
@shafty023
test(vibe): use plain checkpoint ids in the QA plan check test
4669022 
- The fleet skill's generic test reads QA-001 as a ticket id; the
  check accepts any checkpoint id, so the fixtures use C1 to C8.

Testing: qa-plan-check, board and generic tests on a second machine.
Risks: None identified.
@shafty023
fix(vibe): never signal a reused pid from the helper ledger
9cfa37b 
- A recorded pid can be handed to one of the user's own apps after
  the helper exits; probe() saw it as alive, and a launchd app leads
  its own group, so stopped would have killed it.
- pid records each process's start time and command and refuses one
  it cannot read; endProcesses re-reads them before every signal and
  treats a mismatch, an unreadable value or an older record without
  identity as not ours. The ps reader is injected like groupOf.
- run-bounded's group test waits, bounded, for the killed grandchild
  to be reaped before asserting it is gone.

Testing: helpers and run-bounded tests on a second machine; making the
identity check always pass fails all four identity tests.
Risks: a process that execs into another command after it is recorded
is no longer signalled by stopped; record the final process instead.
This was referenced Oct 21, 2026
Closed
Closed
Merged
@shafty023

shafty023 commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Folded into #334 as part of the combined vibe 1.0.74 lessons release.

@shafty023 shafty023 closed this Oct 22, 2026
shafty023 added a commit that referenced this pull request Oct 22, 2026
@shafty023
fix(vibe): ship combined 1.0.74 lessons release (#334)
098915f 
* fix(vibe): never signal pid 1 or an unowned process group

- A helpers.test.mjs run recorded pid 1 and stopped it; endProcesses
  counted EPERM as alive and sent kill(-1), which signals every process
  the user owns. It ended every app on the operator's machine twice.
- helpers.mjs refuses to record a pid of 1 or lower. endProcesses never
  probes or signals one, nor its own or its parent's pid. It signals a
  whole group only for a pid recorded with --group whose pgid still
  equals the pid, and never signals a pid that EPERM says is not ours.
  It takes an injected kill, so its tests send no real signal.
- Every other group signal (run-bounded, precedent-check, the
  vibe-workers ledger and Claude worker, two tests) checks the pid
  above 1 first; dist rebuilt.
- test_vibe_signal_safety_contract.py fails when a plugin test signals
  0, 1, a negative literal or its own pid, writes one into a pids
  ledger, or calls endProcesses without a fake kill, and when any
  source signals a group without checking the pid above 1.
- The cli-worker fake Codex ends with process.abort() instead of
  signalling its own pid.

Testing: node --test, pytest, ruff, pyright and the vibe-workers
vitest and build on a second machine; each safety check shown failing
with its fix removed.
Risks: a helper started without --group now has only its own pid
signalled, so children it spawned in another group need their own
recorded pids.

* fix(vibe): encode the open 2026-10-21 fleet lessons

- fleet-orchestrator: board.mjs keeps the fleet board, flags rows
  behind the pushed head, and tracks rulings and ready QA checkpoints
  until every writer and QA guide is told directly. qa-plan-check.mjs
  defaults to zero human checkpoints. helpers.mjs records and ends a
  helper's processes, lists orphans, and keeps a handed stack from QA
  reruns.
- codex-fleet-monitor: watch.sh writes watcher.json and watcher.mjs
  reports a lapsed watcher; the supervisor template restarts with
  bootout then bootstrap and throttles 30 seconds.
- vibe: push-cadence.mjs and pushDue on every commit, shepherd.mjs for
  the final stage, main-sync-push refuses out-of-order migrations,
  e2e-lock.mjs gives each run its own compose project, and
  precedent-check.mjs check refuses an answer no live look settled.
- handoff: repo-declared path gates (.vibe/path-gates.json) become
  required runs on the published commit.
- Skill prose points at each check; vibe 1.0.74 and its changelog.

Testing: node --test, pytest, ruff, pyright and the vibe-workers
vitest and build on a second machine.
Risks: None identified; every new check is additive and a repo
without .vibe/path-gates.json requires no gate.

* test(vibe): use plain checkpoint ids in the QA plan check test

- The fleet skill's generic test reads QA-001 as a ticket id; the
  check accepts any checkpoint id, so the fixtures use C1 to C8.

Testing: qa-plan-check, board and generic tests on a second machine.
Risks: None identified.

* fix(vibe): never signal a reused pid from the helper ledger

- A recorded pid can be handed to one of the user's own apps after
  the helper exits; probe() saw it as alive, and a launchd app leads
  its own group, so stopped would have killed it.
- pid records each process's start time and command and refuses one
  it cannot read; endProcesses re-reads them before every signal and
  treats a mismatch, an unreadable value or an older record without
  identity as not ours. The ps reader is injected like groupOf.
- run-bounded's group test waits, bounded, for the killed grandchild
  to be reaped before asserting it is gone.

Testing: helpers and run-bounded tests on a second machine; making the
identity check always pass fails all four identity tests.
Risks: a process that execs into another command after it is recorded
is no longer signalled by stopped; record the final process instead.

* fix(vibe): encode the open lessons after #331

- turn-start on a RUNNING record whose thread is idle reconciles
  against the live thread before starting, instead of refusing.
- handoff: criteria-audit.mjs (fresh-context MET audit on the final
  head, criteria x surface x input x flag matrix, flag-off parity) and
  rulings-check.mjs (every ruling on the ticket with what it amends);
  qa-merge-order.mjs --evidence needs each changed e2e spec passing at
  the head in its lane; diff rules for comparators without a tie test
  and new exports nothing uses.
- precedent-check.mjs --behavior makes each disputed behavior a
  required checklist item.
- fleet-orchestrator: finding-brief.mjs refuses findings without
  evidence and fleet.sh ships the screenshots with the prompt.
- vibe 1.0.75 and its changelog.

Testing: node --test and pytest over the plugins on a second machine.
Risks: the unused-export rule blocks a handoff that adds an export
nothing names until it is removed or recorded with Export:.

* fix(vibe): drop the pre-enqueue changed-spec lane check

- The operator ruled that skipping pre-merge e2e was a deliberate
  trade for a deadline, not a standing plugin rule, so the check,
  its prose and its changelog line come out.

Testing: qa-merge-order tests and the vibe contract tests on a second
machine.
Risks: None identified.

* fix(vibe): judge story plays by events and run every shared host

- story-play-verdict.mjs passes a play only when its completion was
  observed and no exception or error event fired; storyFinished's
  status reported success for a play that threw. The evidence gate
  needs playVerdicts, all PASS, on the storybook-plays run.
- A changed UI component two or more hosts import requires every test
  and story covering it or a host, in both flag states when the
  change reads a flag, and blocks until the change log records the
  host the behavior is keyed on.
- vibe 1.0.76 and its changelog.

Testing: node --test (651 of 651) and pytest (2554) over the plugins
on a second machine.
Risks: a shared component change now needs every host's suites run
before a pull request opens.

* fix(fleet-orchestrator): audit ticket statuses against their evidence

- status-audit.mjs reads a snapshot of each in-scope PRD's children:
  a merged pull request means DONE, an open one or an assigned writer
  means IN_PROGRESS or IN_REVIEW, and neither means UNOWNED. It fails
  until each mismatch is fixed or ruled out of scope with a reason.
- The fleet skill runs it at every writer assignment and every merge.
- vibe 1.0.77 and its changelog.

Testing: node --test (654 of 654) and pytest (2555) over the plugins
on a second machine.
Risks: None identified; the audit reads a snapshot and changes nothing.

* fix(vibe): run a repo's Storybook rules as handoff diff rules

- storybook-rules.mjs reads .vibe/storybook-rules.json and blocks a
  story change with a missing or too-low Storybook release, a reused
  release version, new stories under a frozen root, an edit handler
  wired to a no-op, a hand-rolled table row or cell, a touched story
  with no controls decision, a lifecycle tag outside its roots, a
  computed options list, and a story-local copy of a deleted
  component. Every repo fact lives in the declaration.
- The declaration's checks become required tests whenever a story
  changes, so the evidence gate needs a passing run of each.
- An example declaration (symphony-alpha's), vibe 1.0.78 and its
  changelog.

Testing: each rule flags the real diffs of symphony-alpha #8953,
#8959, #8960, #8962, #8963 and #8980; removing any rule fails its
unit test.
Risks: a repo that adds the declaration blocks story changes that
skip a release until it is cut or a ruling is recorded.

* fix(vibe): scope the options rule to list controls and spare frozen files

- story-computed-options counts an options value only in an argTypes
  entry with a select, radio or check control. A prop named options
  ({ control: false }) and an options value under args no longer
  match (symphony-alpha #9102).
- story-no-op-handler skips files under the frozen roots: a v1.0 file
  restored to its earlier bytes keeps its handlers.

Testing: #9102's diff gives only its computed options finding and
#9104's gives none; each new condition fails a fixture when removed.

* fix(vibe): combine lessons as 1.0.74

- Fold all lesson contracts and release notes into one release
- Encode program rulings, attention, budget, VQA, and review IDs
- Remove the deadline-only final-merge E2E exception

Testing: focused vibe 1.0.74 and signal-safety contracts pass

Risks: New orchestration guidance may surface more required fixes

* fix(vibe): preserve VQA handoff contract

- Keep confirmed findings on the same writer
- Extend the rule to pre-existing findings on touched surfaces

Testing: 17 focused handoff and combined-release contracts pass

Risks: None identified

* docs(vibe): clarify confirmed VQA fixes

- State that confirmed findings on touched surfaces are fixed
- Keep the change within the single vibe 1.0.74 entry

Testing: 17 focused handoff and combined-release contracts pass

Risks: None identified

* fix(vibe): order product research

- Check feasibility and sourced decisions before precedent
- Use live Asana through the actor with the signed-in browser
- Return unresolved browser work to the fleet orchestrator

Testing: 29 product-question and combined-release contracts pass

Risks: Product questions may wait when no signed-in browser is available

* fix(vibe): enforce lesson prevention gates

- Gate product questions, live precedent, VQA dispositions, rulings, and review IDs
- Surface idle-worker questions and mirror program rulings to remote writers
- Align docs and eval twins with the enforced contracts

Testing: focused Node and Python contract suites; negative and mutation proofs

Risks: workflow inputs now require explicit evidence and dispositions

* test(vibe): keep prevention fixtures generic

- Avoid ticket-shaped values in fleet-orchestrator test fixtures
- Preserve VQA disposition and remote rulings coverage

Testing: generic guard and app-server recovery black-box tests

Risks: None identified

* docs(vibe): note generic guard fixtures

- Keep the combined 1.0.74 entry aligned with its guard coverage

Testing: generic guard and app-server recovery black-box tests

Risks: None identified

* docs(vibe): preserve research safeguards

Restore the settled-decision and narrow-layout instructions while keeping live Asana as the only precedent source.

Testing: 54 focused Python contract tests

Risks: None identified

* fix(vibe): resolve main-sync owner versions

Read catalog-backed generator versions from the target workspace and bind other owners to the installed package instead of plugin pins.

Testing: 63 main-sync input-boundary tests; negative proof against old bundle; vibe-workers typecheck

Risks: catalog changes now participate in producer input identity
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@shafty023