Skip to content

ADR 0004 phase 7: expose unified catalogue through /reflect, snapshots, CLI, and package #10297

Open

Description

@lpcox

Objective and dependencies

Implement ADR 0004 Gate 7 with the user-directed reflection amendment. Parent: #10290. Reconciled on 2026-11-10 UTC against merged PRs, released versions and trusted live-smoke artifacts. Agent-facing decision diagnostics are delivered and live-accepted within the source-built Docker scope; broader export/persistence/downstream requirements remain separate. Coordinate adoption with github/gh-aw and gh-aw-router-training; do not infer completion from an open/closed issue or a version bump.

Use /reflect only. Do not implement /reflect/v2, endpoint-version negotiation, or v2-to-v1 fallback. Preserve existing fields and their meanings; add the canonical catalogue and provenance fields additively. Versioned schemas/catalogue revisions do not require a new endpoint.

Delivered agent-facing reflection and release boundary

#10494 already exposes active catalogue source/revision/freshness/failure/fallback state additively through /reflect, with request-pinned pricing provenance and producer/loader/accounting/packaging regressions. #10681 adds shared provider-qualified identity/match provenance and generated naming assets packaged for host and proxy execution. Reuse these contracts/assets rather than reconstructing source labels or creating another endpoint.

#11323 adds actionable model-decision diagnostics to existing /reflect: current metadata, captured request outcomes and aggregate accounting remain distinct; observations are bounded/sanitized with conservative action/retry guidance. #11328 adds actual-request compatibility admission outcomes, including accepted/skipped/exhausted fallback decisions, rather than merely exporting unused translator contracts. #11320 supplies shared exact-model reasoning; #11325 supplies directed executable compatibility evidence; #11334 activates the reviewed native reasoning/context subset. These are implemented consumers, not only reusable inputs.

Those model slices ship in AWF v0.29.11. gh-aw v1.2.2 pins that release; #11412 recompiles this repository's workflows. This is release inclusion, not proof of compiler-to-AWF payload/export adoption.

The full persisted catalogue schema, public read-only package APIs and downstream export adoption are not established by these PRs. The original CLI items remain unverified/deferred; the user's current priority is improving agent behavior, not an operator CLI. Do not make a new CLI prerequisite for agent diagnostics acceptance or assign one as the next task.

Live agent diagnostics acceptance

#11371 adds the real-agent utilization smoke; #11429 improves sanitized failure evidence; #11431 publishes the exact request-only initial handoff and clarifies evidence-backed correction versus permanent denial. These two harness fixes postdate v0.29.11.

Run 42904995184, source 6454ec273f0289ed5788475495d22a42c606ff15, passed the provider-free gate and all six Copilot/Claude/Codex scenarios. Each recovery observed the required 400 -> GET /reflect 200 -> effort-only correction -> 200 with exactly two POSTs, one upstream call and verified result. Each denial observed 403 -> GET /reflect 200 -> explicit retry: false blocked report, one POST and zero upstream calls. Trusted host grading, not agent self-report alone, establishes the pass.

This verifies observable use of production diagnostics with real CLIs on Docker and deterministic challenge inference. It is not causal proof that diagnostics were necessary, real-provider capability verification, released-artifact acceptance, gh-aw compilation acceptance or coverage of other runtime topologies. The next acceptance slice uses explicit released versions through the native compiler path; no live dispatch is authorized by updating this issue.

Other inputs and unresolved boundaries

#10878 supplies scoped eligibility/reasons and immutable request pins. #11057/#11064/#11072 supply observation contracts/producers. #10294 and the runtime-validation children #11091/#11092/#11068 are closed, but their closure does not establish every original unchecked criterion. github/gh-aw#70728 remains open without linked model-payload round-trip completion in its body. Verify actual compiler/export adoption before declaring that boundary complete.

User-confirmed supported targets: cloud-hypervisor, nvx, docker, and arc/dind only. ARC/DinD is a Docker deployment topology, not a separate runtime backend. sbx is unsupported; historical enum/backend names (including runc/gVisor/Kata) are not additional product support targets. Do not count sbx coverage toward phase completion or plan new sbx work.

Preserve observed/unknown/unverified distinctions, exact topology/scope and no-constraint versus verified compatibility. Do not leak primary-agent evidence into dedicated enclaves or infer support for opaque sessions. The six Docker diagnostics scenarios do not establish KVM, NVX, Kubernetes ARC, per-enclave observation or all-runtime harness enforcement. Unsupported feed constraints must not be activated incidentally.

The earlier loader smoke 40236788522 verified loading/fallback, not central-feed billing or the full later diagnostic behavior. Publication v1 does not distribute remote naming/availability rules. Pricing is now authoritative through #11113/#11117/#11157/#11164; do not describe the shared resolver as still awaiting initial shadow activation.

Scope

Expose provider-qualified resolved records, static catalogue/schema revisions, source-derived generation time, runtime observation times, per-value/component source/freshness, estimates/staleness/incomplete/conflict warnings, selectability reasons, harness compatibility, workflow provenance, and central source/refresh/fallback state.

Version models.json compatibly or introduce sibling model-catalog.json when required to retain current persisted contracts. Ship generated static data/read-only identity, capability, and pricing APIs through the package, keeping overlays run-local. Implement awf models list/show/export/validate/sources; diff is optional and not required for completion. Ensure exports are actually present in published/container assets, not just source-tree fixtures.

Entry points: containers/api-proxy/{management.js,model-discovery.js}, snapshot persistence, src/commands/, src/cli.ts, package.json, bundle/packaging scripts, and documentation.

Acceptance criteria

  • Additive catalogue source/revision/freshness/failure/fallback reflection and packaged loader/naming assets are delivered. feat: load reviewed model catalogue prices at runtime #10494/Unify model resolution around canonical provider-qualified identities #10681. These are reusable inputs; the complete resolved export criteria below remain open.
  • Internal shared eligibility/re-evaluation and sanitized observation contracts have packaged host/proxy inputs. [WIP] Implement shared scoped eligibility decisions and refresh re-evaluation #10878/feat: observe actual-environment harness versions #11057/fix: correct harness observation runtime support boundary #11064. No full public export/CLI/API or downstream adoption is inferred from this.
  • Existing /reflect gains resolved model-decision diagnostics without a new endpoint; current metadata, captured outcomes and aggregate accounting are distinct and regression-covered. feat: expose actionable model-decision diagnostics through /reflect #11323/Apply directed compatibility contracts to fallback decisions and /reflect diagnostics #11328.
  • Real Copilot, Claude and Codex utilize /reflect before correction or policy-stop in all six source-built Docker scenarios. Run 42904995184; this is not released/compiler-path acceptance.
  • Every exported price component/capability requiring provenance carries its source/freshness/estimated state; no raw provider payloads, secrets, private headers, or arbitrary fields leak.
  • Static times/revisions remain deterministic; live times are distinct. Workflow overlays remain run-local in reflection and persistence.
  • Persisted snapshots are schema-validated and distinguish static facts from run observations; legacy readers retain documented behavior.
  • Public read-only package APIs and broader built/package export contracts have explicit adoption evidence. Original CLI commands remain deferred/unverified, not the next agent-behavior task.
  • Tests cover central refresh failures, stale/estimated facts, unknown efforts, and incompatible harness diagnostics in exported views.
  • Downstream handoff documents additive-field detection and behavior with older pinned AWF /reflect responses; missing new fields are not silently interpreted as known support or zero prices.
  • gh-aw AIC data and router-training pricing/efforts adopt generated exports after export/accounting parity checks, or this issue remains explicitly blocked on linked companion work.
  • The same recovery/denial contract passes through normal compiler-generated execution with explicitly identified released AWF CLI/runtime images and gh-aw version.

Non-goals

No new endpoint, independent downstream model tables, raw runtime payload access, or removal of compatibility adapters before Phase 9.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions