Skip to content

feat(core): Make dependency graph pages portable - #308

Merged
shafty023 merged 1 commit into
mainfrom
feat/prd-graph-portable
Oct 19, 2026
Merged

shafty023 merged 1 commit into
mainfrom
feat/prd-graph-portable

Conversation

@shafty023

Copy link
Copy Markdown
Collaborator

Purpose

A prd-dependency-graph page was self-contained to view but not to refresh: the refresh depended on ~/.local/share/prd-dependency-graph/<graph-id>.json, which held the scope and options, labels and columns, me, a per-graph token the page also embedded, and absolute output paths on the author's machine. A shared page could not be refreshed anywhere else.

This makes a page (or a report with embedded graphs) portable: copy the HTML file to any machine with the skill and a CLOSEDLOOP_API_KEY, click refresh, and that copy is rewritten in place. There is no side file, and nothing from the author's machine or identity.

Design

  • Config in the page. Each graph block carries <template class="dg-config" data-dg-graph="<id>"> with HTML-escaped JSON: scope, options, labels, column overrides, column_labels, nodes, edges, PRODUCES links and the last refresh's changes. It holds no token, output paths, API base or me. A <template> is used instead of <script type="application/json"> because pages rendered by the previous version re-create every <script> they swap in as executable, which would turn the JSON into a syntax error in tabs that are already open. Standalone pages now get depgraph markers too, so every refresh replaces only the region between the markers.
  • POST /v2/refresh (JSON body: graph, scope, optional outside):
    • a page opened from a file sends path (its own location.pathname). The helper validates the file, reads the embedded config, re-reads the scope with this machine's key, rewrites only that graph, and answers {ok, written, refresh_id} with no ticket data. The page stores the refresh id and zoom in sessionStorage and reloads; on load it shows the change dialog from last_refresh in its own config and restores the zoom;
    • a page that is not a file sends config instead. The helper answers with the rendered block only for loopback origins (or a local program with no Origin), and the page swaps it in with a note that no file on disk changed;
    • an older helper refuses the JSON preflight, so the page probes /health and tells "out of date, run prd-graph-helper --ensure" apart from "not running".
  • "Yours" comes from GET /me (the API key's owner), cached for 10 minutes per key. --me is now only a fallback for when that lookup fails. A copy shows the author's highlighting until its new owner refreshes it.
  • CLI: --once <page.html> (plus --graph for reports), --export-config <page.html>, --once <page.html> --apply-config <json> for label and column edits, and render_graph.py <page.html> --print-columns. --new writes no side file.

Security model (SKILL.md "Security", pinned by test_prd_dependency_graph_security.py)

Any website open in the browser can send requests to 127.0.0.1:8765, and pages no longer carry a token.

  • Exfiltration: a file refresh returns no ticket data; the data goes into the file, which only the page itself can read from disk. Ticket data goes back only to loopback origins and to local programs with no Origin. Origin null is refused too, since a sandboxed frame on any website sends it. Errors carry no ticket data, and /health and the old token endpoint no longer reveal the home path to other origins. The remaining signal is whether a guessed path holds a given graph.
  • Arbitrary writes: only an absolute .html/.htm path (checked before and after resolving symlinks) that is an existing regular file owned by the user, up to 64 MB, holding this graph's markers and an embedded config for the requested scope. Only the text between the markers is replaced, atomically, and the write is refused if the file changed during the refresh.
  • Cost: the request must name a graph id and a strictly formatted scope that the file holds, and these checks run before any API call. JSON is required, so browsers preflight the request, and the preflight is granted only to null and loopback origins. There is a 3 second cooldown per target, at most 2 refreshes at once, and bodies are capped at 4 MB.
  • API key: never written to any file, and only sent to https://api.closedloop.ai. An api_base in a page is ignored; only an older local config can set one.

Backward compatibility

  • The old token endpoint (/refresh?graph=&token=) is unchanged, so pages from 1.3.0 and 1.4.x keep refreshing through their running helper or the new one.
  • While an older config in ~/.local/share/prd-dependency-graph/ lists a file, that config stays the file's source of truth whichever way the refresh arrives, and --once <graph-id> still works.
  • Every refresh writes the portable format into the listed files, which migrates the page and removes its token. The config file is kept.
  • A copied pre-1.5.0 page with no config on the new machine gets a clear message to have its author refresh it once.

Files changed

  • skills/prd-dependency-graph/scripts/refresh_server.py: v2 endpoint, path, origin and scope validation, rate gate, /me lookup, file and config refresh, CLI additions, legacy routing.
  • skills/prd-dependency-graph/scripts/render_graph.py: embedded config, markers on standalone pages, a portable start command, page JS for v2 (reload, report and old-helper detection), HTML input.
  • skills/prd-dependency-graph/scripts/check_graph.cjs: works on a copy; checks the file refresh (request shape, reload, dialog, zoom) and the served-page swap.
  • skills/prd-dependency-graph/SKILL.md: steps, refresh, sharing, older graphs, security, config.
  • tools/python/conftest.py, test_prd_dependency_graph_security.py, test_prd_dependency_graph_portable.py: new tests with a stubbed API and an isolated HOME.
  • README.md, CHANGELOG.md, .claude-plugin/plugin.json, .codex-plugin/plugin.json: closedloop-core 1.5.0.

Commands run

uv run ruff check .                                  # All checks passed
uv run pyright                                       # 0 errors
uv run pytest plugins/                               # 2423 passed, 3 skipped
uv run pytest plugins/closedloop-core/tools/python   # 80 passed
(cd tools/plugin-manifests && npm ci && npm test)    # 6 passed
python3 scripts/render_graph.py <example+refresh>.json -o page.html
python3 scripts/render_graph.py <example+refresh>.json --embed-into report.html
node scripts/check_graph.cjs page.html ISS-10797 shot.png     # ok, no errors
node scripts/check_graph.cjs report.html ISS-10797 shot.png   # ok, no errors

Manual end to end against the real API, on scratch copies only, with a helper on port 8799 under a scratch HOME (the user's helper on 8765 and their files were not touched):

  • Old PRD-800 page and old PRD-751 briefing, each with its old config: the first click refreshed through the token URL in the old page script and migrated the file. The second click went through /v2/refresh with a reload and the change dialog. The briefing's prose outside the markers was byte-identical, and its labels and depth all were kept.
  • Copies of the migrated pages in another folder, refreshed by a helper with an empty HOME (no configs): refresh with the dialog and zoom kept, the gear toggle to depth all, and the original file unchanged.
  • --once <file>, --export-config, --new (and its refusal to overwrite), --ensure and --print-columns on HTML. me resolved from the key.
  • curl probes: an evil origin got 403 with no CORS header, null was refused ticket data via config, and file mode answered {ok, written, refresh_id} only.

- Embed each graph's config (scope, options, labels, columns, nodes,
  links, last refresh) in its page in an inert template; no token,
  output paths, API base or viewer name are stored anywhere
- Add POST /v2/refresh: a file page sends its own path, the helper
  validates the file, rewrites only that graph between its markers and
  answers without ticket data so the page reloads from disk; pages
  served from localhost get an in-place swap that says no file changed
- Resolve "yours" from the API key's owner via GET /me
- Enforce the threat model: JSON-only requests preflighted for file and
  loopback origins only, no ticket data to null or other origins, path
  and marker checks before any API call, scope must match the file,
  per-target cooldown and concurrency cap, key only sent to the
  ClosedLoop API whatever a page says
- Keep older graphs working: token URLs, --once <graph-id> and files
  listed in an older config still refresh from it and are migrated to
  the portable format on refresh
- Add --once <page.html>, --export-config and --apply-config, and let
  render_graph read a page for --print-columns
- Stop /health and the token endpoint revealing the home path
- Bump closedloop-core to 1.5.0; update SKILL.md, README, CHANGELOG

Testing: uv run ruff check . and uv run pyright (clean); uv run pytest
plugins/ (2423 passed, 3 skipped), including new security and
portability suites with a stubbed API; check_graph.cjs on fresh
standalone and embedded pages; manual refresh against the real API on
scratch copies of the PRD-800 page and PRD-751 briefing, a copied page
refreshed by a helper with an empty HOME, and the CLI paths

Risks: Pages open in a tab from before this version refresh through
the token URL once, then use the new protocol. A helper older than a
new page is reported as out of date until replaced with --ensure.
@shafty023
shafty023 merged commit 788538e into main Oct 19, 2026
10 checks passed
@shafty023
shafty023 deleted the feat/prd-graph-portable branch October 19, 2026 19:53
shafty023 added a commit that referenced this pull request Oct 20, 2026
* feat(vibe): Add codex-fleet-monitor skill (#308)

- Background watcher for a fleet of Codex App Server threads (local
  and one ssh host) plus their pull requests; exits with a diff on the
  first change
- Reads liveness from each thread's last turn and item count, so a
  turn left open by a silent usage limit shows as STALLED
- vibe 1.0.57, README and changelog entries

Testing: pytest plugins/ (2430 passed), ruff, plugin-manifests
typecheck and tests, node --check and zsh -n on the scripts
Risks: None identified, the skill only reads thread and PR state

* Rewrite codex-fleet-monitor skill description

- Describe when to reach for the skill and why a turn-only watcher misses a usage-limit stall

* Note skill trigger in changelog

* Make codex-fleet-monitor self-contained and accept thread ids

liveness.mjs no longer imports the app-server-orchestrator skill from
~/.codex/skills. It carries a small App Server client (WebSocket over
codex app-server proxy, initialize, thread/read) in one file so the ssh
copy to a remote host still works. sessions.txt takes
'local|remote <label> <threadId>' as well as the older '<name>' form that
reads a state file. Adds a node:test suite with a fake server.

* feat(vibe): Report Codex usage and request files in codex-fleet-monitor

- ratelimit.mjs reads account/rateLimits/read through the client liveness.mjs
  already exports and prints codex-usage <where> <band>% <state> resets=<n>.
  snapshot.sh runs it locally and on REMOTE_HOST, copying both scripts there.
- Optional REQUESTS_DIR and REMOTE_REQUESTS_DIR print a requests line so the
  watcher wakes when a session drops a delegated-work request.
- watch.sh keeps the previous run's last-snapshot.txt as its base, so a change
  between a watcher exit and its re-arm is still reported.
- PR check counts are only updated once settled; while checks are pending the
  last settled counts show with a (running) suffix.
- SKILL.md documents the new lines and what to do about each.
shafty023 added a commit that referenced this pull request Oct 20, 2026
* feat(vibe): Add codex-fleet-monitor skill (#308)

- Background watcher for a fleet of Codex App Server threads (local
  and one ssh host) plus their pull requests; exits with a diff on the
  first change
- Reads liveness from each thread's last turn and item count, so a
  turn left open by a silent usage limit shows as STALLED
- vibe 1.0.57, README and changelog entries

Testing: pytest plugins/ (2430 passed), ruff, plugin-manifests
typecheck and tests, node --check and zsh -n on the scripts
Risks: None identified, the skill only reads thread and PR state

* Rewrite codex-fleet-monitor skill description

- Describe when to reach for the skill and why a turn-only watcher misses a usage-limit stall

* Note skill trigger in changelog

* Make codex-fleet-monitor self-contained and accept thread ids

liveness.mjs no longer imports the app-server-orchestrator skill from
~/.codex/skills. It carries a small App Server client (WebSocket over
codex app-server proxy, initialize, thread/read) in one file so the ssh
copy to a remote host still works. sessions.txt takes
'local|remote <label> <threadId>' as well as the older '<name>' form that
reads a state file. Adds a node:test suite with a fake server.

* feat(vibe): Report Codex usage and request files in codex-fleet-monitor

- ratelimit.mjs reads account/rateLimits/read through the client liveness.mjs
  already exports and prints codex-usage <where> <band>% <state> resets=<n>.
  snapshot.sh runs it locally and on REMOTE_HOST, copying both scripts there.
- Optional REQUESTS_DIR and REMOTE_REQUESTS_DIR print a requests line so the
  watcher wakes when a session drops a delegated-work request.
- watch.sh keeps the previous run's last-snapshot.txt as its base, so a change
  between a watcher exit and its re-arm is still reported.
- PR check counts are only updated once settled; while checks are pending the
  last settled counts show with a (running) suffix.
- SKILL.md documents the new lines and what to do about each.

* feat(vibe): Add fleet-orchestrator skill

- Add the fleet-orchestrator skill: rules for running Codex and Claude
  workers on parallel PRs across machines for an operator who does QA
  and merges, with state, template and log formats under references/
- Add codex-session.mjs to start or resume a Codex thread bound to a
  worktree, start and steer turns from prompt files and read the last
  messages, built on the codex-fleet-monitor App Server client
- Add slot-lock.sh for a machine-wide e2e stack slot and an N-slot
  heavy-check cap, with stale pid cleanup and release on exit or signal
- Bump vibe to 1.0.65 and document the skill

Testing: node --test over all vibe test files, uv run pytest
plugins/vibe, ruff, pyright, plugin-manifests tests; one live turn,
resume, steer and last against a real Codex App Server

Risks: None identified; new files only

* Make fleet-orchestrator config driven and machine neutral

- Read hosts, paths, model names, effort and slot counts from the
  orchestration dir's config.env through a new fleet.sh, which runs
  sessions and slots locally or over ssh on another machine
- Require an effort for a new Codex session instead of a built in
  default, and create the state dir when saving
- Add a setup reference with prerequisites, a placeholder config and a
  checklist, and replace program specific examples with generic ones
- Add a test that fails on home paths, IPs, emails, ticket ids and
  repo references in the skill

Testing: node --test over the skill scripts under bash 5 and macOS
bash 3.2

Risks: None identified

* Use generic wording for repo docs and UX precedent

- Point repo specifics at the repo's own docs or the team knowledge
  base and name UX precedent generically
- Fail the generic check on this organization's and repository's names

* Add judgment patterns to fleet-orchestrator

- Add references/judgment.md with the patterns from the brief's
  addendum that SKILL.md did not already carry
- Fold the overlapping ones into their existing SKILL.md rules: usage
  rate, measured QA checks, baseline tracing, precedent rulings

* Add the per-PR pipeline and remote reboot recovery

- Name the order each PR moves through before enqueue, and that a
  head change sends it back to the earliest affected step
- Say how to recover Codex threads on a remote machine that rebooted

* Require vibe for every fleet writer

- Every writer is a vibe session: Codex writers start with a $vibe
  prompt, which codex-session.mjs now requires for a new session, and
  Claude writers invoke vibe:vibe
- Route reviews, handoff, ticket, environment and seed work through
  vibe's own workers and scripts
- Start the worker preamble with invoking vibe, keep a Claude takeover
  inside vibe and log the missing migration as a gap

* List the vibe plugin as a prerequisite on every writer machine

* Add moving work between runtimes and accounts

- One section for moves in both directions between Codex machines or
  accounts and Claude, triggered by MOVE_AT_PERCENT, Codex limit lines
  or Claude limit errors, with the old writer always stopped first
- Add a resume prompt template and the MOVE_AT_PERCENT config key
- List the plugin among the per-machine prerequisites in SKILL.md

* Open fleet-orchestrator with its two principles

- State up front that the operator's attention is the scarcest
  resource and that every worker output gets a sanity check
- Point Roles, Pushback, Reporting and judgment.md at the principles
  instead of restating them

* Require closedloop-graph research in fleet-orchestrator

- Add a research section: overlap and collisions before assigning or
  ruling, ticket lineage, callers and importers before a shared change,
  code_tests_for for tests, and tracing a surprising diff to its ticket
- Carry graph-first research in the worker preamble and resume prompt
- List the closedloop-graph server as a prerequisite and check it in
  the setup checklist
- Let the generic check allow the plugin family's own tool names

* Check ticket scope and human threads before ruling by precedent

* Make the pre-ruling read a numbered checklist under principle 1

- Read acceptance criteria and scope, the PRD, human review threads
  including resolved ones, and closedloop-graph before precedent
- Send a conflicting human-sourced line to the operator and record
  every ruling with its source for the writer to cite

* Add a first-run setup interview to fleet-orchestrator

- Ask once for machines, writer runtimes, Codex budget and move
  threshold, and the UX precedent product when config.env lacks them,
  with single machine and Claude only as defaults
- Write config.env from the answers, verify, and report; later runs
  ask again only when a verification fails

* Skip local reruns after text-only merge conflicts

* Restore the blank line between changelog entries

* Gate visual requests on the pushed head and scope test reruns

- Add request-visual.mjs, which files a visual request only when its
  head equals the PR's pushed head and the worktree HEAD, and refuses
  when gh cannot answer; ship it to remote machines with fleet.sh
- Let code_tests_for over the touched files decide which tests run
  after any change, fix or merge, in SKILL.md and both worker prompts

* Add load-balanced heavy runs and per-PR QA-guide sessions

- Send each heavy run to the machine with the lower load per core
- Run human QA in a headless, primed Claude QA-guide session per PR
  that the operator resumes, with a QA guide prompt template

* Leave sub-agent models to vibe's worker-model policy

- Drop CODEX_EXPLORER_MODEL from the config example, the first-run
  interview and the resume prompt; keep CODEX_MODEL and CODEX_EFFORT
  for the thread
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant