Skip to content

Offload Fastly SSAT delivery compression from the Compute guest #858

New issue
New issue
Open
Task
#884
#894
Open
Offload Fastly SSAT delivery compression from the Compute guest#858
Task
#884
#894
Assignees
ChristianPavilonis
Labels
rustPull requests that update rust code

Description

@ChristianPavilonis
ChristianPavilonis
opened on Jul 7, 2026

Summary

Offload final SSAT HTML compression from the Fastly Compute guest to Fastly dynamic delivery compression on current main.

The current implementation still compresses assembled HTML in the guest, including cached-template delivery. Streaming and template caching have changed since the original experiment, so the implementation and measurements must use today's paths rather than revive the closed parser-safe streaming stack from #872.

Agreed scope

  • Automatically offload eligible Fastly SSAT HTML delivery compression. No new publisher opt-in or configuration flag.
  • Preserve existing origin Accept-Encoding negotiation. Decode supported gzip, deflate, and Brotli origin responses, process HTML as today, then emit identity output with X-Compress-Hint: on.
  • Preserve final response framing: no stale Content-Encoding or Content-Length; retain existing Vary fields and cover Accept-Encoding.
  • Explicitly declare delivery-compression capability at adapter boundaries. Axum, Cloudflare, and Spin retain current behavior. Fastly dynamic compression supports gzip/Brotli; retain in-guest delivery when reader negotiation selects deflate or fails, without changing origin negotiation.
  • Cover current inline streaming, buffered HTML, template-cache miss, warm hit, and assembly fallback delivery without changing parser/EOF behavior, auction timing, or consent gates.
  • Preserve current terminal privacy rules, including stricter no-store paths. Do not change template admission, stored representations, cache keys, origin readthrough, TTLs, or purge.

Shared EdgeZero mechanism

The P3 extraction work is tracked in IABTechLab/edgezero#362 and implemented in draft EdgeZero #452, created before the consumer migration. Draft Trusted Server #884 now pins its exact published commit 157f091e42b5c584898a5d0f8a0d3ddc3a4dd5ec across six workspace declarations and eight lockfile packages.

EdgeZero owns typed request capability, checked opaque identity-output intent and the pure Fastly finalizer. Trusted Server owns SSAT eligibility, consent/auction gates, origin decoding, output planning, HTML/EOF processing, template neutrality and privacy. The local marker, generic capability/options wrapper and header-finalization helper are removed.

Trusted Server keeps its custom lazy sender and calls the shared finalizer after late effects and privacy enforcement, before consuming response parts. It does not use the standard collecting converter. Neither project enables compression globally or relabels compressed bytes as identity.

Both PRs remain draft for joint review. Merge EdgeZero first, then replace the temporary consumer pin with a merged release before merge-ready. The existing Fastly staging criteria below remain unchanged.

Historical evidence

The July identity-origin plus Fastly delivery-compression experiment measured approximately 44.6 ms versus 9.0 ms guest processing and 47 ms versus 19 ms Fastly vCPU, with Brotli browser delivery. Earlier local fixture measurements also showed decode-to-identity substantially cheaper than mirror recompression.

These measurements justify fresh testing; they are not performance claims for the current output-only implementation.

The former blanket 10 MiB origin-response rationale is stale. Current Fastly adapter materialization checks apply to buffered responses, not its streamed-response branch. Publisher pipeline limits remain separate and unchanged. This scope does not request larger identity bodies from origin.

Acceptance criteria

  • Eligible Fastly SSAT HTML selects guest identity output and the Fastly delivery-compression hint automatically, without a publisher flag. Actual platform delivery remains covered by the staging criteria below.
  • Origin request encoding negotiation remains unchanged.
  • Identity, gzip, deflate, and Brotli inputs produce equivalent decoded rewritten HTML through affected streaming and buffered finalizers.
  • Template misses, hits, and assembly fallbacks retain existing template and privacy contracts while selecting offloaded final delivery encoding.
  • Non-Fastly, non-SSAT, non-HTML, unsupported encoding, bodyless, redirect, and error paths retain existing behavior.
  • Headers, lazy streaming, auction timing/cleanup, byte limits, and terminal privacy have regression coverage.
  • Required local adapter tests/lints and repository CI checks pass. Hosted GitHub CI on the updated PR is pending.
  • Fresh Fastly staging measurements cover inline, template-miss, and template-hit paths, recording vCPU, TTFB, origin bytes, browser bytes, actual delivery encoding, and decoded HTML equivalence.
  • Real Fastly reader negotiation is verified for absent encoding headers, identity-only, supported compressed encodings, and identity-refusing requests. Local/Viceroy results are not presented as proof of dynamic compression.

Out of scope

Identity-origin fetching, new compression flags, new caching behavior, parser-safe streaming/EOF refactors, static asset compression, new retries, and new body-size limits.

Delivery

Draft PR #884 is now a feature-only branch at 85a9b9184a67a9c827be9b59ead95a801fb07faf, rebased onto main 51730bdaf841e4d7bb96c2258e5e98169bdff9a7. It preserves current-main trace, ingress and absolute-deadline behavior. The obsolete parser-stack and merge history are not replayed. The previous 5b663ee2d implementation is superseded and retained in a local backup branch.

Design: SSAT delivery compression. Plan and local verification: EdgeZero consumer integration. Consumer contract checks and independent implementation review found no blockers.

The original db27e738/63c7b63 integration validation passed all 32 Rust/repository gates, including 3,345 native core tests, normal/reusable Fastly aliases, all adapter tests/lints, parity/digest/clock/host checks, actual Fastly/Cloudflare/Spin deadline fixtures and actual Axum plus release-Fastly EOF/auction/prefix controls. JS passed 1,838 tests with one skipped. Documentation formatting/lint/build and script/Node-pin checks passed. Tool versions were Rust 1.98.1, Node 24.20.0 and Viceroy 0.21.0.

Review R1 now uses literal expected Vary fields in the framework test. The consumer pins the resulting exact framework head. Fresh framework native tests, consumer delivery/build-digest tests, formatting and SDK scoped Clippy passed. Review reruns also found both current local Fastly aliases and the SDK contract link-blocked on pending_req_header_insert; verified same-tree hosted runs passed, and the discrepancy remains unresolved. No runtime workaround was introduced. Fresh hosted CI runs on the updated heads.

These local results prove guest behavior, not actual CDN compression or performance. No deployment or merge ran. The two staging criteria above remain open.

Related: #858's historical measurement comments below remain useful background; #894 is a separate specification-only caching PR and is not modified by this work.

CI smoke fixture fix

Framework 157f091e42b5c584898a5d0f8a0d3ddc3a4dd5ec fixes the failed reusable HTTP smoke probe. Delivery contracts had passed 12/12 and Fastly library tests 103/103; uncontrolled origin EOF could race downstream receipt.

The post-commit origin now waits for the existing client first-body-byte signal before closing short. Plain immediate aborts still serve pre-commit fault tests. Status 200, exact partial, interrupted-response type and matching guest error remain mandatory. No sleeps, retries, SDK stubs, trap overrides or runtime changes were added. Cleanup releases waiters after failed probes.

A regression failed before the fix. Final checks pass 2 core + 27 harness tests, formatting, strict Clippy and the full pinned-Viceroy Fastly smoke. All three custom variants observed the required bytes and correlated error. Two independent source investigations/reviews found no blocking issue.

Consumer 85a9b9184a67a9c827be9b59ead95a801fb07faf refreshes all six declarations/eight lock sources. Six delivery tests, ten build-digest tests and formatting pass, with no registry/SDK drift. GitHub's test workflow now passes at 157f091e, including the previously failing Fastly job. Retained CI artifacts confirm 200/partial and matching guest errors for all three custom variants. Both PRs remain draft; consumer CI runs separately.

This does not resolve the separate local debug/WASM loader discrepancy or prove live Fastly negotiation/performance.

Activity

  1. ChristianPavilonis
    added
    enhancement
    rustPull requests that update rust code
    on Jul 7, 2026

  2. ChristianPavilonis commented on Jul 9, 2026

    @ChristianPavilonis
    ChristianPavilonis
    on Jul 9, 2026
    CollaboratorAuthor

    Local measurement update from a native release-mode fixture test.

    I added an ignored local fixture harness at crates/trusted-server-core/tests/ssat_compression_fixtures.rs. It reads representative local HTML fixtures, runs them through the existing StreamingPipeline + HtmlProcessor, and verifies that:

    • gzip origin mirror output decompresses to the same bytes as identity-origin processed output;
    • gzip decode-to-identity output matches identity-origin processed output;
    • brotli origin mirror output decompresses to the same bytes as identity-origin processed output;
    • brotli decode-to-identity output matches identity-origin processed output.

    Command used:

    cargo test -p trusted-server-core \
  --target aarch64-apple-darwin \
  --release \
  --test ssat_compression_fixtures \
  -- --ignored --nocapture

    Representative local results:

    Fixture Identity input Origin gzip Origin br Identity process gzip mirror br mirror gzip decode→identity br decode→identity
    Home page 1,330,005 B 181,161 B 112,086 B 0.772 ms 15.463 ms 7.425 ms 2.654 ms 2.260 ms
    Article page 668,249 B 100,129 B 69,891 B 0.512 ms 8.624 ms 4.641 ms 1.538 ms 1.495 ms

    Local interpretation:

    • The existing mirror-compression path is materially more expensive than identity-origin processing on these representative pages.
    • Decode-to-identity is much closer to identity processing than mirror recompression, which points to in-guest recompression as the main local cost.
    • These tests also give us a reusable local regression/measurement harness while keeping the large representative HTML fixtures out of the repo and out of default CI.

    Caveats:

    • These are native local release timings, not Fastly Compute vCPU-ms.
    • Local/Viceroy cannot validate Fastly dynamic delivery compression from X-Compress-Hint: on.
    • The delivery-compressed sizes in this local harness are only approximations; the final decision still needs a Fastly staging/canary measurement for real Content-Encoding, transfer size, and Compute vCPU-ms.
  3. ChristianPavilonis
    self-assigned this
    on Jul 9, 2026
  4. aram356
    added the
    Task
    issue type on Jul 9, 2026

  5. ChristianPavilonis commented on Jul 9, 2026

    @ChristianPavilonis
    ChristianPavilonis
    on Jul 9, 2026
    CollaboratorAuthor

    Measurement update from the SSAT compression experiment branch:

    Tested the guarded query parameter modes:

    • ?__ts_ssat_compression=identity
    • ?__ts_ssat_compression=mirror

    Observed headers for identity mode:

    content-encoding: br
x-ts-ssat-compression-mode: identity
x-ts-ssat-delivery-compression: fastly-dynamic
x-ts-ssat-fastly-vcpu-ms: 19
x-ts-ssat-origin-accept-encoding: identity
x-ts-ssat-origin-content-encoding: identity
x-ts-ssat-output-content-encoding: identity
x-ts-ssat-processed-bytes: 1365106
x-ts-ssat-processing-ms: 9.045

    Observed headers for mirror mode:

    x-ts-ssat-compression-mode: mirror
x-ts-ssat-delivery-compression: in-guest
x-ts-ssat-fastly-vcpu-ms: 47
x-ts-ssat-origin-accept-encoding: gzip, deflate, br
x-ts-ssat-origin-content-encoding: gzip
x-ts-ssat-output-content-encoding: gzip
x-ts-ssat-processed-bytes: 178351
x-ts-ssat-processing-ms: 44.630

    Initial finding: identity mode is materially better for Compute-side processing on this page:

    • Processing wall time: ~9 ms vs ~45 ms (identity is about 5x faster)
    • Fastly vCPU: 19 ms vs 47 ms
    • Final browser delivery is still compressed with Brotli (content-encoding: br) via Fastly dynamic compression

    x-ts-ssat-processed-bytes is larger in identity mode because it measures the uncompressed body handled by the guest before Fastly delivery compression. It does not represent final wire size.

    Also noted: x-compress-hint was not visible in the browser response, but the response had:

    x-ts-ssat-delivery-compression: fastly-dynamic
content-encoding: br

    So Fastly delivery compression appears to have been applied successfully despite the control hint not being exposed back to the client.

  6. ChristianPavilonis commented on Jul 9, 2026

    @ChristianPavilonis
    ChristianPavilonis
    on Jul 9, 2026
    CollaboratorAuthor

    Final decision: proceed with compression offload as a publisher-level Fastly opt-in, not as the global default yet.

    PR: #884

    The implementation adds:

    [publisher]
ssat_compression_offload_enabled = true

    When enabled on Fastly and the existing SSAT eligibility gates pass, Trusted Server requests Accept-Encoding: identity, emits processed identity HTML with X-Compress-Hint: on, and lets Fastly dynamically compress delivery. Origins that ignore the identity request and return gzip, deflate, or Brotli are decoded safely to identity output.

    Axum, Cloudflare, and Spin explicitly declare the capability unavailable and keep the current mirrored compression behavior. The feature defaults to false everywhere.

    Rationale:

    • measured guest processing improved from ~44.6 ms to ~9.0 ms;
    • measured Fastly vCPU improved from 47 ms to 19 ms;
    • browser delivery remained Brotli-compressed;
    • identity origin bodies are larger and count against the current 10 MiB raw origin-response limit, so an opt-in rollout is safer than changing the global default.

    The PR removes the temporary measurement query parameters and diagnostic headers; those findings remain recorded in this issue.

  7. ChristianPavilonis
    linked a pull request that will close this issueDesign TS-owned dynamic response caching #894on Jul 13, 2026
  8. aram356
    removed
    enhancement
    on Sep 26, 2026
  9. ChristianPavilonis
    changed the title [-]Measure and offload SSAT HTML compression from the Compute guest[/-] [+]Offload Fastly SSAT delivery compression from the Compute guest[/+] on Oct 29, 2026
  10. ChristianPavilonis
    added a commit that references this issue on Oct 29, 2026

    Refresh Fastly SSAT delivery compression

    5b663ee
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

rustPull requests that update rust code

Type

Task

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions