Skip to content

feat(benchmark): add Cloudflare Clef benchmark providers - #716

Draft
zzwong wants to merge 6 commits into
zzwong/offline-openai-decisions-identityfrom
zzwong/offline-clef-identity
Draft

zzwong wants to merge 6 commits into
zzwong/offline-openai-decisions-identityfrom
zzwong/offline-clef-identity

Conversation

@zzwong

@zzwong zzwong commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add the cloudflare_clef provider type to offline report-identity suites, with explicit clef or clef-flash model selection. cr benchmark run resolves the configured Cloudflare account and credential source while reusing the shared System One codec, identity prompt, runner and replay.

Cloudflare can truncate long state without a completeness receipt. Runs require --allow-unverified-context and record context_status: unverified; explicit truncation signals fail. A valid returned choice does not establish that the model consumed the complete input.

This draft is stacked on #715, following #714. The unshipped infer-clef command is removed without an alias. Live review behavior is unchanged.

Validation

  • Full go test -p 1 ./..., focused uncached provider/command/identity/architecture checks, lint, module tidy and build passed. Independent final source review is clear.
  • Compiled CLI checks exercised direct and saved candidates, empty fixtures and whole-budget failures. Missing context acknowledgement and invalid selectors fail before output. All checks made zero provider calls and verified fixed routes, Bearer provenance, unverified context, suite hashes, original retention, independent replay, private permissions and manifest hashes.
  • Tests cover both Clef selectors, strict account and candidate validation, explicit truncation, usage metadata and response bounds. All client, codec, shared projection and replay source files match the preserved implementation.

Live Cloudflare smoke remains pending because no scoped credential is configured. Injected transports and compiled zero-call CLI checks validate local behavior. Prior local evidence is preserved, no new inference calls or full matrix were run, and no complete-context model-quality claim is made.

Reuse the System One codec for Clef and Clef-flash requests while validating Cloudflare envelopes, routing and usage. Mark provider context completeness as unverified.
Replace only the usage value so valid response metadata cannot inflate past the wire size guard. Retain known usage and the original response digest for ASCII and Unicode metadata.
Reuse the shared identity question and replay runner for Cloudflare models. Require an explicit context acknowledgement and retain unverified context, usage and complete original records in private receipts.
@zzwong zzwong changed the title feat(benchmark): add Cloudflare Clef identity inference feat(benchmark): add Cloudflare Clef benchmark providers Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant