feat(isolation): new models orchestrator kit — discover, cache/TTL, aggregate neutral catalog (ADR 0004 Layer 2) #425

Summary

Layer 2 of ADR 0004 (integrations/isolation/docs/decisions/0004-neutral-model-provider-discovery.md, PR #423). Depends on #424 (at least one provider kit self-registering) landing first, or can be built against a fixture provider-facts directory for offline testing.

Scope

A new kit, integrations/isolation/acq-kits/models/ (name TBD — bikeshed welcome), placed early in kit order (after provider kits, before harness kits — same positional convention as zscaler-ca-certificate; see acq.backends/common.sh:47-58 in quickstart and its bats-asserted invariant at test/bats/50-kit-list-completeness.bats:26-28).

This kit carries zero vendor-specific knowledge — pure orchestration. Its startup phase:

1. Globs /var/lib/acq/models/providers/*.json.
2. For each discovered provider, checks a cache file (embedded fetch timestamp) at a host-mounted volume path, keyed by providerId. (Per maintainer decision: NOT guest-local-only — a guest-local cache makes TTL meaningless across acq run invocations, since every fresh sandbox starts cold. Use the neutral volumes[] vocabulary from hybrid/v1/ADR 0001, mounted via msb --mount-dir / sbx's volume primitive.)
3. If stale (default TTL 24h, overridable by a config-kit default — separate issue), invokes that provider's own normalizer (from feat(usai-provider): self-register provider facts + own normalizer + vendored fallback (ADR 0004 Layer 1) #424) via a shared bounded-fetch helper (timeout + response-size cap — extract/canonicalize from the existing fetchJsonBounded in usai-provider/scripts/sync-usai-models.mjs, which has already been copy-pasted once into build-catalog.mjs and already drifted — this issue should leave exactly ONE canonical implementation).
4. Validates the normalizer's OUTPUT against the neutral catalog schema before accepting it — not just checking the fetch succeeded. A structurally-valid-but-wrong response is still a routing hazard.
5. On ANY failure (timeout, oversized, malformed, validation failure, no provider present): falls back to last-good cache, then to that provider's own vendored snapshot (feat(usai-provider): self-register provider facts + own normalizer + vendored fallback (ADR 0004 Layer 1) #424).
6. Aggregates every provider's neutral output into /var/lib/acq/models/catalog.json.
7. Records provenance per provider ("source": "live" | "cache" | "snapshot", with a timestamp) — a permanently degraded fallback must be visible, not silently indistinguishable from a fresh fetch.

Security requirements (binding, from ADR 0004's incorporated dissent)

• Env-var-ownership check: before reading keyEnv's value, confirm the SAME provider kit's own declared environment names that variable (see feat(usai-provider): self-register provider facts + own normalizer + vendored fallback (ADR 0004 Layer 1) #424). Never trust an arbitrary name from a facts file.
• SSRF hardening: modelsUrl must be https://, and its host must equal the facts file's own declared host, which must itself be on the sandbox's already-effective caps.network.allow union (ADR 0002). Never fetch an arbitrary URL a facts file supplies without this cross-check.
• Atomicity: facts/cache/aggregate writes are temp-file + rename; no reader observes a partial write.
• Bounded resource use: per-provider timeout + size cap, PLUS a total wall-clock budget across all discovered providers so a slow/hanging endpoint cannot stall sandbox startup indefinitely.

Neutral catalog schema

Versioned from v1 (schemaVersion: "acq-neutral-model-catalog/v1"). Per maintainer decision: ship v1 now, no multi-version negotiation mechanism ahead of a real second version existing. A breaking change later is a coordinated bump in a follow-up ADR — accepted as a rare cost.

Explicitly out of scope for this issue

• Any harness kit consuming the aggregate (separate issue, Layer 3 — usai-provider/OpenCode first, since it exists today).
• The config kit's cross-harness defaults (model-role priority) — separate issue, referenced but not blocking.
• Promotion of this kit into any "core, always-on" built-in bundle — per the ADR, that needs its own human/CODEOWNERS review, same process as ADR 0002's balanced allowlist.

Verification

• Offline bats coverage (fits the existing scripts/test-acq-bats / test/bats/*.bats stubbed pattern): fresh-cache / stale-cache-refresh-success / stale-cache-refresh-fail-fallback-to-last-good / no-cache-fallback-to-snapshot / malformed-response-rejected / zero-providers-present, all against a fixture provider-facts directory and a stubbed fetch — no live network required for the default test path.
• shellcheck --severity=warning, scripts/scan_unsafe_shell.py, validate-kits.py --strict.
• Live RUN_ACQ=1 scripts/verify end-to-end against a real sandbox with usai-provider + this kit stacked, once feat(usai-provider): self-register provider facts + own normalizer + vendored fallback (ADR 0004 Layer 1) #424 lands.

References

• ADR 0004: docs(isolation): add ADR 0004 for neutral model-provider discovery #423
• Depends on: feat(usai-provider): self-register provider facts + own normalizer + vendored fallback (ADR 0004 Layer 1) #424