[WIP] Implement shared scoped eligibility decisions and refresh re-evaluation by Copilot · Pull Request #10878 · github/gh-aw-firewall

Copilot · 2026-10-31T02:04:26Z

Summary

Add one provider/channel/deployment-scoped model eligibility contract with structured reasons, sanitized provenance, semver/numeric harness comparisons, and verified actual-execution-environment evidence requirements.
Integrate host validation, explicit requests, selectors, routing, fallback and compatible recovery; add immutable request-pinned refresh snapshots while preserving existing policy, lifecycle, protocol, pricing and recovery opt-outs.
Generate/package host and API-proxy modules and assets, add contract/consumer/refresh/packaging regressions, and document remaining probe and compiler/overlay handoffs.
Fix deployment-scoped recovery evidence, include eligibility assets in publication fixtures, and shut down integration-test telemetry to prevent cross-suite leakage.

Validation

Root suite: 10,929 passed, 66 skipped (532 suites passed, 5 skipped).
API-proxy suite: 2,977 passed across 135 suites.
Focused contract/host/compiler tests: 190 passed, including host and proxy execution outside the source checkout.
Standalone CLI bundle built and executed outside the checkout.
Build, lint (warnings only), type-check, generated-output checks, diff checks and secret scanning passed.

Remaining handoff

Local CodeQL was requested but analysis was skipped because the database is too large; successful CI CodeQL verification is still required.
A maintainer must approve/re-trigger required CI on HEAD 838487bd693c0b18f23751d280387221a3e07230, review the results, and resolve the addressed provenance-disclosure thread.
Actual-environment observation producers and workflow/operator overlay/compiler integration remain follow-ups under ADR 0004 phase 4: migrate lifecycle validation, overlays, and harness eligibility #10294 and Agree and implement opt-in model-information compiler-to-AWF contract (ADR 0004 Phase 4) gh-aw#70728. Feed-supplied harness constraints remain unsupported; this bounded slice does not complete ADR 0004 phase 4: migrate lifecycle validation, overlays, and harness eligibility #10294, ADR 0004 phase 5: unify pricing resolution and offline-first catalogue refresh #10295 or Implement ADR 0004: unified model information subsystem #10290.

Fixes ADR 0004 phase 4 slice: shared scoped eligibility decisions and refresh re-evaluation #10877

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot · 2026-10-31T04:02:31Z

+      model: decision.identity?.key,
+      scope: decision.scope,
+      reasons: decision.reasons.filter(reason => reason.code.startsWith('harness_')).map(reason => ({
+        ...reason,


Addressed in b8d2eee: buildEligibilityError now constructs an explicit client-facing field allowlist rather than spreading reason objects. Provenance is excluded, and shared URL sanitization also drops pathname/query/credentials. The real HTTP ingress regression asserts that internal provenance hostnames and revision values are absent from the returned error. I am running the full local validation suites on the latest code.

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/39bb6d5c-feec-4ae5-b11b-7cf97937e6cb Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

lpcox · 2026-10-31T03:13:08Z

@copilot address all review feedback

Copilot · 2026-10-31T03:19:01Z

@copilot address all review feedback

Addressed in b8d2eee: eligibility errors now use a safe reason-field allowlist, and provenance URLs retain only their origin.

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot

🟡 Changes recommended

Query-string decoys can bypass model eligibility checks in both HTTP and WebSocket requests, allowing constrained models to pass through unchecked.

5 open issues

What changed in this PR

Introduces a shared, scoped model-eligibility boundary for AWF’s host validation and API proxy, advancing ADR 0004 Phase 4 while leaving environment observation producers and remote constraint activation for later work.

Changes:

Adds version-constraint evaluation, structured evidence diagnostics, and immutable request snapshots.
Integrates eligibility with explicit requests, selectors, routing, fallback, recovery, and visible WebSocket handshake models.
Packages shared contracts and generated projections, adds regression coverage, and documents remaining handoffs.

File	Description
`src/model-information.eligibility.generated.json`	Adds the host harness-constraint projection.
`src/model-information-eligibility.test.ts`	Tests scope, version rules, evidence, and snapshots.
`src/model-information-contract/eligibility.js`	Packages the shared runtime for host use.
`src/model-information-contract/eligibility.d.ts`	Packages host eligibility types.
`src/copilot-model.ts`	Applies eligibility during CLI model validation.
`src/copilot-model.test.ts`	Tests CLI constraints and pinned evidence.
`shared/model-information/eligibility.js`	Implements eligibility decisions and evidence snapshots.
`shared/model-information/eligibility.d.ts`	Defines requirements, observations, and decision contracts.
`scripts/model-information/compile.test.ts`	Checks generated projections and standalone packaging.
`scripts/model-information/compile.cjs`	Generates host and proxy eligibility assets.
`docs/adr/0004-model-information-management.md`	Documents semantics, integrations, and remaining handoffs.
`containers/api-proxy/websocket-proxy.js`	Pins evidence and identifies visible handshake models.
`containers/api-proxy/websocket-guards.test.js`	Covers opaque handshake compatibility.
`containers/api-proxy/websocket-guards.js`	Adds handshake harness admission checks.
`containers/api-proxy/upstream-http.js`	Filters fallback attempts through eligibility.
`containers/api-proxy/server-factory.js`	Pins evidence at HTTP and upgrade ingress.
`containers/api-proxy/routing-runtime.js`	Passes trusted eligibility dependencies into routing.
`containers/api-proxy/routing-provider-executor.js`	Preserves snapshots through classification and final checks.
`containers/api-proxy/routing-controller.test.js`	Tests snapshot consistency across routing retries.
`containers/api-proxy/routing-controller.js`	Shares one snapshot throughout routing decisions.
`containers/api-proxy/routing-candidates.js`	Filters routing candidates by eligibility.
`containers/api-proxy/proxy-request.js`	Pins eligibility before HTTP processing.
`containers/api-proxy/proxy-guards.js`	Adds explicit-request harness rejection.
`containers/api-proxy/model-selector.js`	Attaches eligibility decisions to selector results.
`containers/api-proxy/model-resolver.js`	Filters automatic candidates and records rejections.
`containers/api-proxy/model-recovery.js`	Integrates availability and recovery candidate decisions.
`containers/api-proxy/model-information.eligibility.generated.json`	Adds the proxy harness-constraint projection.
`containers/api-proxy/model-identity.js`	Accepts scoped identity-resolution options.
`containers/api-proxy/model-eligibility.js`	Adapts shared evidence, scope, and diagnostics for proxy consumers.
`containers/api-proxy/model-eligibility.integration.test.js`	Exercises consumers, refresh behavior, and request pinning.
`containers/api-proxy/model-config.js`	Applies pinned evidence during model transformations.
`containers/api-proxy/model-body-rewriter.js`	Propagates eligibility context and decisions.
`containers/api-proxy/Dockerfile`	Includes eligibility runtime and generated assets.
`containers/api-proxy/cross-provider-fallback.js`	Preserves pinned evidence and catalogue context.
`containers/api-proxy/catalogue-contract/eligibility.js`	Packages the shared runtime for standalone proxy use.
`containers/api-proxy/catalogue-contract/eligibility.d.ts`	Packages proxy eligibility type declarations.

_{Review effort: Max (auto)}

  }

+  const eligibilityModel = checkModelMultiplier
+    ? getRequestModel(null, req.url) || getRequestModel(body, req.url)


+  const records = req.awfRuntimePricing?.[provider] || [];
+  const available = identity && records.some(record =>
+    resolveModelIdentity(record.id, provider, { scope: consumerScope, policy: { providerPrefix: false } }).identity?.key === identity.key);


+      status: unavailable ? 'unavailable' : 'unknown', authoritative: unavailable === true,
+      scope: { provider, channel: provider === 'copilot' ? 'copilot-api' : 'native-api' },


+    [currentProvider]: providerModels.filter(m =>
+      _isModelPermittedByPolicy(m, modelPolicyConfig, currentProvider) &&
+      _isModelHarnessPermitted(m, currentProvider, resolutionState)),


+  const eligibilityModel = checkModelMultiplier
+    ? getRequestModel(null, req.url) || getRequestModel(body, req.url)
+    : null;


Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

lpcox · 2026-10-31T04:19:15Z

@copilot address the review feedback

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

github-actions · 2026-10-31T05:10:03Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Smoke Codex

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Smoke Codex · gpt54 · ~10 AIC · ⊞ 22.7K · ◷

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-10-31T05:10:03Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-10-31T05:10:04Z

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

🛡️ Egress verdict from Smoke Copilot Network Isolation

github-actions · 2026-10-31T05:10:07Z

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-10-31T05:10:16Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

github-actions · 2026-10-31T05:10:26Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-10-31T05:22:24Z

OTEL smoke test

✅ S1 Module loads; exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, routerCallTracing, shutdown, isEnabled
✅ S2 OTEL tests: 75 passed, 0 failed (3 suites)
✅ S3 Env forwarding: trace ID/parent span ID in env-passthrough.ts; OTLP vars + trace context in api-proxy-env-config.ts
✅ S4 onUsage hook present in token-tracker-http.js
⚠️ S5 No exported-span diagnostics checked; treated as pending (no OTLP endpoint verification here)

📡 OTel tracing validated by Smoke OTel Tracing
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:23:09Z

Services smoke test: FAIL

❌ Redis PING: connection refused
❌ pg_isready: no response
❌ psql SELECT 1: connection refused

🔌 Service connectivity validated by Smoke Services
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:24:01Z

Smoke Test (Copilot)
PRs: "Expose API proxy URLs independently of host-service access"; "Add scoped GitHub capability handoff for Cloud Hypervisor agent enclaves"

GitHub MCP: ✅
GitHub.com connectivity: ✅
File write/read: ✅

Overall: PASS — @Copilot (author), assignees @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:24:11Z

Cloud Hypervisor + Copilot smoke test

Merged PR lookup: PASS
github.com connectivity: PASS (HTTP 200)
File write/read: PASS
example.com blocked: PASS (HTTP 000)

Overall: PASS

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:24:21Z

Smoke Test: Copilot BYOK (Direct) Mode ✅

Tests: All passing

✅ GitHub MCP connectivity
✅ github.com HTTP 200
✅ File write/read
✅ BYOK inference via api-proxy → api.githubcopilot.com

Status: PASS | Direct BYOK mode active (COPILOT_PROVIDER_API_KEY)

/cc @lpcox @Copilot

🔑 BYOK report filed by Smoke Copilot BYOK
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:25:21Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1	✅ PASS
Bun	hono	✅	1/1	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1	✅ PASS
Deno	std	N/A	1/1	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	pass	✅ PASS
Go	env	✅	pass	✅ PASS
Go	uuid	✅	pass	✅ PASS
Java	gson	❌	0 run	❌ FAIL
Java	caffeine	❌	0 run	❌ FAIL
Node.js	clsx	✅	pass	✅ PASS
Node.js	execa	✅	pass	✅ PASS
Node.js	p-limit	✅	pass	✅ PASS
Rust	fd	✅	1/1	✅ PASS
Rust	zoxide	✅	1/1	✅ PASS

Overall: 7/8 ecosystems passed — FAIL

Java failure details: Maven could not use ~/.m2 (LocalRepositoryNotAccessibleException). With an alternate local repo, dependency resolution failed: InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty (the Java truststore is empty or inaccessible).

.NET needed DOTNET_CLI_HOME and XDG_RUNTIME_DIR set to work around a missing /run/user/1001.

Generated by Build Test Suite for #10878 · copilot · auto · ~24 AIC · ⊞ 5.7K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:25:54Z

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com → 200)
✅ Blocked domain (example.com blocked)

Overall: PASS — cc @lpcox

🛡️ Egress verdict from Smoke Copilot Network Isolation
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:25:57Z

Chroot version comparison

Runtime	Host Version	Chroot Version	Match?
Python	3.12.15	3.12.15	✅
Node.js	v24.22.1	v22.23.2	❌
Go	go1.22.12	go1.22.12	✅

Node.js differs between host and chroot, so the tests did not all pass and the smoke-chroot label was not added.

Tested by Smoke Chroot
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:26:17Z

@Copilot @lpcox
PR Titles:

Expose API proxy URLs independently of host-service access
Add scoped GitHub capability handoff for Cloud Hypervisor agent enclaves

✅ MCP / GitHub.com / File / BYOK tests all passed
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
Overall status: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:27:01Z

Smoke Claude Results

API status: ✅ PASS
gh check: ✅ PASS
File status: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #10878 · claude · haiku45 · ~34 AIC · ⊞ 6.1K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-10-31T05:27:50Z

Smoke Test

Expose API proxy URLs independently of host-service access ✅
Add scoped GitHub capability handoff for Cloud Hypervisor agent enclaves ✅
Playwright title check ❌
File write/read ✅
Build AWF ✅
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex
_{Add label ready-for-aw to run again}

Initial plan

f6c4c98

Copilot AI assigned Copilot and lpcox Oct 31, 2026

Copilot started work on behalf of lpcox October 31, 2026 02:04

View session

Copilot AI linked an issue Oct 31, 2026 that may be closed by this pull request

ADR 0004 phase 4 slice: shared scoped eligibility decisions and refresh re-evaluation #10877

Closed

10 tasks

feat(models): add scoped eligibility and pinned refresh decisions

d94be82

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

github-advanced-security AI found potential problems Oct 31, 2026

View reviewed changes

Copilot finished work on behalf of lpcox October 31, 2026 03:04

Changes before error encountered

d4e084b

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/39bb6d5c-feec-4ae5-b11b-7cf97937e6cb Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI requested a review from lpcox October 31, 2026 03:04

github-advanced-security AI found potential problems Oct 31, 2026

View reviewed changes

Comment thread shared/model-information/eligibility.js Resolved

Copilot started work on behalf of lpcox October 31, 2026 03:13

View session

fix: redact eligibility diagnostics

b8d2eee

Copilot finished work on behalf of lpcox October 31, 2026 03:19

Copilot started work on behalf of lpcox October 31, 2026 03:48

View session

Copilot started work on behalf of lpcox October 31, 2026 03:56

View session

lpcox marked this pull request as ready for review October 31, 2026 03:58

Copilot AI auto review requested due to automatic review settings October 31, 2026 03:58

Copilot finished work on behalf of lpcox October 31, 2026 03:58

Copilot started reviewing on behalf of lpcox October 31, 2026 03:58

View session

Copilot started work on behalf of lpcox October 31, 2026 03:59

View session

fix: preserve deployment scope in model recovery

3f48c9a

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI reviewed Oct 31, 2026

View reviewed changes

test: drain eligibility integration tracing on teardown

838487b

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot finished work on behalf of lpcox October 31, 2026 04:17

Copilot started work on behalf of lpcox October 31, 2026 04:19

View session

fix: address model eligibility review feedback

89f47dc

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

lpcox had a problem deploying to aoai-model October 31, 2026 05:10 — with GitHub Actions Failure

lpcox enabled auto-merge (squash) October 31, 2026 05:10

github-actions Bot added the smoke-copilot label Oct 31, 2026

github-actions Bot added the smoke-cloud-hypervisor label Oct 31, 2026

github-actions Bot added the smoke-copilot-byok label Oct 31, 2026

github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Oct 31, 2026

github-actions Bot added the smoke-copilot-byok-aoai-apikey label Oct 31, 2026

github-actions Bot added the smoke-claude label Oct 31, 2026

lpcox deployed to aoai-model October 31, 2026 05:27 — with GitHub Actions Active

lpcox had a problem deploying to aoai-model October 31, 2026 05:27 — with GitHub Actions Failure

github-actions Bot mentioned this pull request Oct 31, 2026

[aw] Smoke Copilot BYOK AOAI (Entra): timed out #10893

Closed

lpcox merged commit dd19bf4 into main Oct 31, 2026
141 of 147 checks passed

lpcox deleted the copilot/adr-0004-phase-4-slice branch October 31, 2026 05:51

		status: unavailable ? 'unavailable' : 'unknown', authoritative: unavailable === true,
		scope: { provider, channel: provider === 'copilot' ? 'copilot-api' : 'native-api' },

Conversation

Copilot AI commented Oct 31, 2026 • edited by lpcox Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Remaining handoff

Uh oh!

Copilot AI Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lpcox commented Oct 31, 2026

Uh oh!

Copilot AI commented Oct 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

lpcox commented Oct 31, 2026

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Smoke Codex

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 31, 2026

Uh oh!

github-actions Bot commented Oct 31, 2026

Uh oh!

github-actions Bot commented Oct 31, 2026

Uh oh!

github-actions Bot commented Oct 31, 2026

Uh oh!

github-actions Bot commented Oct 31, 2026

Smoke Test: Copilot BYOK (Direct) Mode ✅

Uh oh!

github-actions Bot commented Oct 31, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions Bot commented Oct 31, 2026

Uh oh!

github-actions Bot commented Oct 31, 2026

Chroot version comparison

Uh oh!

github-actions Bot commented Oct 31, 2026

Uh oh!

github-actions Bot commented Oct 31, 2026

Smoke Claude Results

Uh oh!

github-actions Bot commented Oct 31, 2026

Smoke Test

Uh oh!

Uh oh!

This branch had an error being deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Oct 31, 2026 •

edited by lpcox

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading