Skip to content

feat(gateway): a waiting request says why each worker cannot take it - #693

Merged
V3RON merged 0 commit into
task/636from
task/637
Oct 19, 2026
Merged

V3RON merged 0 commit into
task/636from
task/637

Conversation

@V3RON

@V3RON V3RON commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Closes #637

Status

Implement: done Review: passed (round 2 + 2 fix-checks; second allowed by the maintainer) Hardware: n/a Gate: waiting for the stack

Assumptions

  • Assumption: a worker that answered PROVISION_FAILED lists not-chosen while the request waits — the spec's reason table has no row for it, and not-chosen is the existing default for a worker that passed every check.
  • Assumption: RoutingPolicy.explain returns { workerId, filter?, zero? } rather than the pipeline's { workerId, reason? } — the coordinator must tell a filter reason (before the memos) from a rank zero reason (after them), and the policy does it from two explainStages calls without naming a stage.
  • Assumption: the dispatcher test's worker fixture gains health: "running" and queueDepth: 0 — without them healthy and idle-queue (earlier in stage order) give the reason, and its full expectation is unchanged.

Review

Spec review: 2 blocking, 0 fixed, 1 notes. Code review: 7 blocking, 6 fixed, 2 notes. Claims review: 4 blocking, 4 fixed, 3 notes.
Rounds: 2. Tokens: 19.4M.
Assessment: complete

Fixed unreviewed (text, rule 14): the comment at fleet-coordinator.ts:116 (e79389b); the not-ready and unhealthy tests (50e1fe3).

Rejected:

  • spec: Assumption 1 lists a worker that failed provisioning as not-chosen — [intentional] the spec table has no row for it, and full would be wrong for a failure that isn't about capacity.
  • spec: a stale PROVISION_FAILED refusal still counts after the view key changed — [nonblocking] needs narrow claim timing, and the spec leaves "in this walk" loose.
  • code: a waiting request is rejected on an old refusal after the other worker is drained — [nonblocking] the worker is excluded either way.

Written by an agent.

@V3RON

V3RON commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Decision taken

Question: a worker reports "provisioning" progress to the gateway before it creates a device (src/core/device-provisioner.ts:75-79). Once progress has arrived, the gateway has bound the request to that worker. So a real PROVISION_FAILED nearly always arrives after progress, and the "walk on to the next worker" rule (item 9) almost never fires.

Options:

  1. Keep the spec as worded. The gateway walks on only on a PROVISION_FAILED that comes before progress, and after progress it relays the error to the caller. In practice the caller gets the first worker's setup error.
  2. Walk on even after progress. The gateway unbinds the request from the worker and tries the next worker. The caller only gets PROVISION_FAILED once every worker has failed, which is what the maintainer asked for. This changes how progress binds a request (ADR 0021) and needs its own task.

Taken: 1. It is what the spec says and closest to today's code, and the docs now say "before progress". Reverting means a new task for option 2.

Written by an agent.

@V3RON

V3RON commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Review notes

Not blocking, not verified. Each is one reviewer's claim.

  • spec: src/gateway/fleet-coordinator.ts:866 — a PROVISION_FAILED refusal still counts after its view key changed, so a lone worker that an older request claimed this pass can end a later request.
  • code: src/gateway/fleet-coordinator.ts:789 — an old PROVISION_FAILED refusal plus a later drain of the other worker rejects a request that had been waiting.
  • code: ui/src/screens/waiting/index.tsx:139 — the worker label is used as the React key, but labels need not be unique.
  • claims: src/gateway/AGENTS.md:80, docs/CLI.md:954, docs/HTTP-API.md:576 — each names only one of the states (full, busy, not ready, unhealthy) that keep the request queued.

Written by an agent.

@V3RON
V3RON marked this pull request as ready for review October 18, 2026 19:22
@V3RON
V3RON added this pull request to stack #691 October 18, 2026 19:22
@V3RON
V3RON merged commit d71eb83 into main Oct 19, 2026
@V3RON
V3RON deleted the task/637 branch October 19, 2026 12:29
V3RON added a commit that referenced this pull request Oct 19, 2026
…stats and console figures (#623) (#626)

Delivers #623: the event history moves to `history.db`, with usage
projections, `simlock stats`, `GET /v1/stats` and the console's figures.
It also includes ADR 0032.

This PR carries the whole feature. The stack's PRs were reviewed one by
one, and then a stack sync pushed every branch to one commit. GitHub
then marked them as merged into each other, never into `main`. Their
reviews stay there:

- ADR 0032 (this PR)
- #635 → #690
- #636 → #692
- #637 → #693
- #629 → #694
- #630 → #695
- #631 → #711
- #632 → #724
- #633 → #727
- #634 → #731

After the sync onto `main`, a fix-check reviewed the conflict
resolutions. Its fix makes `history.db` and its WAL/SHM files owner-only
(0600), as #685 did for `events.jsonl`.

Closes #629, closes #630, closes #631, closes #632, closes #633, closes
#634, closes #635, closes #636, closes #637.

### Status
Implement: done Review: per-task PRs passed; sync fix-check passed
Hardware: n/a Gate: running

## Review

Spec review: skipped. Code review: 4 blocking, 4 fixed, 1 notes. Claims
review: 0 blocking, 0 fixed, 0 notes.
Rounds: 1. Tokens: 29.0M.

*Written by an agent.*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A waiting request says why each worker cannot take it, on a gateway

1 participant