Skip to content

Add explicit adapter delivery compression support - #452

Draft
ChristianPavilonis wants to merge 3 commits into
mainfrom
feature/362-delivery-compression
Draft

ChristianPavilonis wants to merge 3 commits into
mainfrom
feature/362-delivery-compression

Conversation

@ChristianPavilonis

@ChristianPavilonis ChristianPavilonis commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Give applications an explicit delivery-compression contract before they choose their output encoder, without changing unmarked responses.
  • Share Fastly's final header handling between standard response conversion and custom lazy senders. This extracts the platform mechanism needed by Trusted Server without moving SSAT, privacy, or auction policy into EdgeZero.
  • Keep the existing standard-converter buffering and all other-adapter behavior unchanged.

Changes

Crate / file Change
edgezero-core/src/delivery_compression.rs Unavailable-by-default request capability, checked identity-response marking, opaque intent, and native tests
edgezero-adapter-fastly/src/delivery_compression.rs Feature-independent, idempotent header finalizer that never polls or collects the body
Fastly request/response converters Advertise capability before application processing and apply the shared finalizer at standard conversion
Fastly contract tests Real SDK/Viceroy coverage for conversion/dispatch, marked buffered and streamed bodies, genuine encoded passthrough, stream errors, and custom no-poll finalization
Spec, implementation plan, Fastly guide Ownership split, identity-before-intent invariant, and standard/custom sender usage

Design and implementation plan.

Two independent design checks and a separate implementation inspection found no blockers. Applications must install identity bytes or their final identity-producing stream before marking. Body-transforming middleware must remove the intent before re-encoding. Reader negotiation/fallback remains application-owned; Fastly uses the matching original client request.

Closes

Closes #362.

Paired consumer: IABTechLab/trusted-server#884, tracked in trusted-server#858. The consumer now pins this PR's exact commit 157f091e42b5c584898a5d0f8a0d3ddc3a4dd5ec in all six declarations and eight locked packages. Both PRs remain draft for joint review. EdgeZero merges first; replace the consumer's temporary revision pin with a merged release before making it merge-ready.

Test plan

Passed locally with Rust 1.98.1 and Viceroy 0.21.0:

  • cargo test --workspace --all-targets --locked
  • cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
  • cargo fmt --all -- --check
  • cargo check --workspace --all-targets --features "fastly cloudflare spin" --locked
  • cargo check -p edgezero-adapter-spin --target wasm32-wasip2 --features spin --locked
  • Actual Viceroy Fastly contracts, 12 passed, and SDK response-converter neighbors, 4 passed
  • Focused core, 510 passed; native Fastly, 68 passed; new rustdoc example
  • Strict native and WASM Clippy on the changed crates/contracts
  • Placeholder/legacy/shared-pin guards, shared-pin regression tests, and core documentation build
  • Docs formatting, lint, and build

The new core, helper, request-conversion and SDK-converter regressions were observed failing before their implementation and passing on the final source. No manifest, lockfile, new dependency, or unrelated fixture changes were needed. All GitHub checks on the original 63c7b63 implementation passed, including demo/generated-project smoke checks. The test-only review follow-up triggers fresh CI.

Checklist

  • Follow repository conventions; no new dependencies, Tokio, configuration, route, or store-wiring changes
  • Use portable core HTTP types and preserve unmarked/unsupported behavior
  • Add regression tests through shipping helpers and actual Fastly SDK conversion
  • Document custom sender ordering without a transport rewrite
  • No credentials or production data committed
  • Keep this PR draft for joint review with the Trusted Server integration

Paired consumer integration

Trusted Server #884 is now a feature-only rebase onto main 51730bd, published as 85a9b9184a67a9c827be9b59ead95a801fb07faf. It removes its duplicate mechanism and keeps a pending output plan until identity output is installed. Its custom Fastly sender calls this shared finalizer after late effects and privacy enforcement, without collecting lazy bodies.

Before this review follow-up, consumer db27e738 with framework 63c7b63 passed all 32 local Rust/repository gates, 1,838 JS tests, docs checks, actual deadline fixtures and actual Axum/release-Fastly EOF controls. Independent source review found no blockers. Its required Fastly aliases and runtime controls passed under default Viceroy 0.21.0 with no runner override or SDK stub. Review reruns also found current local SDK contract and both consumer Fastly alias builds link-blocked on pending_req_header_insert. Same-tree hosted CI passed those paths. The discrepancy is unresolved, not a proven main or production failure.

R1 review follow-up

Resolved local review R1 in framework 1c0f475097a813a3a9cbb7a4a121ca0523d5ed64: Vary test rows now contain literal expected header fields instead of copying the production matching predicate. No shipping SDK logic changed. Consumer 5298c0a20b664394d6ec218862296ca1ba9463b8 refreshes six declarations and eight lock sources to that exact commit, with no registry or SDK version drift.

Fresh checks passed: 13 framework native helper/core tests, SDK formatting and scoped strict Clippy; six consumer delivery tests, ten build-digest tests, consumer formatting and changed Markdown formatting. Earlier broad verification belongs to the original revisions noted above. New commits trigger fresh hosted CI.

This is not a Viceroy runtime fix. Current local loader divergence remains unresolved; no SDK stub, runtime-source/pin workaround or trap override was introduced. Real Fastly staging remains open.

CI smoke fixture fix

Framework 157f091e42b5c584898a5d0f8a0d3ddc3a4dd5ec fixes the failed reusable HTTP smoke probe. Delivery contracts had passed 12/12 and Fastly library tests 103/103; uncontrolled origin EOF could race downstream receipt.

The post-commit origin now waits for the existing client first-body-byte signal before closing short. Plain immediate aborts still serve pre-commit fault tests. Status 200, exact partial, interrupted-response type and matching guest error remain mandatory. No sleeps, retries, SDK stubs, trap overrides or runtime changes were added. Cleanup releases waiters after failed probes.

A regression failed before the fix. Final checks pass 2 core + 27 harness tests, formatting, strict Clippy and the full pinned-Viceroy Fastly smoke. All three custom variants observed the required bytes and correlated error. Two independent source investigations/reviews found no blocking issue.

Consumer 85a9b9184a67a9c827be9b59ead95a801fb07faf refreshes all six declarations/eight lock sources. Six delivery tests, ten build-digest tests and formatting pass, with no registry/SDK drift. GitHub's test workflow now passes at 157f091e, including the previously failing Fastly job. Retained CI artifacts confirm 200/partial and matching guest errors for all three custom variants. Both PRs remain draft; consumer CI runs separately.

This does not resolve the separate local debug/WASM loader discrepancy or prove live Fastly negotiation/performance.

Remaining validation

No live deployment or performance measurement ran. Viceroy establishes guest bytes/headers and SDK behavior, not real Fastly dynamic compression, final wire negotiation, service-chain behavior, progressive wire delivery, or transfer/vCPU savings. Those remain explicit staging work with the consumer issue.

Expose portable request capability and checked identity-response intent.
Share Fastly header finalization between standard conversion and custom
streaming senders without changing unmarked output or collecting bodies
in the reusable finalizer.

Keep negotiation, rewriting, privacy and cache policy application-owned.
Cover native contracts and actual Fastly SDK conversion under Viceroy.
Record the reviewed design and cross-repository implementation plan.

Refs #362
Gate the post-commit abort origin on observed downstream body bytes. Preserve immediate pre-commit fault probes and strict partial-response and correlated-error assertions. Add ordering and pre-header-close regressions.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add reusable Fastly delivery-compression support

1 participant