Skip to content

feat(compaction): stream the summary into the chat as it generates - #252

Merged
hakula139 merged 3 commits into
mainfrom
feat/stream-compaction-summary
Oct 16, 2026
Merged

hakula139 merged 3 commits into
mainfrom
feat/stream-compaction-summary

Conversation

@hakula139

Copy link
Copy Markdown
Owner

Summary

/compact and auto-compaction used to show only a spinner while the summary was being generated, which could take tens of seconds in a long session. Now the summary streams into the chat as it is generated, using the layout of the compacted block it will become. Every compaction outcome also removes or replaces it cleanly.

  • Protocol: Two new events, Event::SummaryToken(String) and Event::SummaryDiscarded, carry summary updates. A sequence of SummaryToken chunks ends with SessionCompacted on success or SummaryDiscarded otherwise. A discard precedes the TurnFailed, Cancelled, or Error event that reports the outcome. If an overflow triggers a retry, the discard comes before streaming resumes, so the frontend never shows two attempts at once. Both events are included in the every_event round-trip table.
  • Engine: compact_session takes the event sink and reads the reply through a SummaryStream sink, which forwards text as SummaryToken, passes retry notices through, and drops thinking. It emits SummaryDiscarded only if text has streamed. Its Drop implementation handles cancellation, since await_unless_aborted drops the summarization future before the caller reports Cancelled. replace_session_with_summary discards the summary before reporting a failed boundary write.
  • TUI: ChatView keeps the in-flight summary in its own buffer and displays it a blank row below the chat using CompactedBlock::render_streaming, under a Summarizing the conversation... header. SessionCompacted replaces it with the final block. SummaryDiscarded removes it and subtracts it from the paused new-content count.
  • Stdio: The REPL and headless frontends ignore both events and keep their one-line boundary notice. Headless runs never auto-compact because the engine starts without usage to trigger it.
  • Docs: The /compact design, including its component notes and design decision 12, the auto-compaction steps, and the welcome screen's emptiness rule now describe the stream.

Closes #186

Test plan

  • cargo fmt --all --check
  • cargo build
  • cargo clippy --all-targets -- -D warnings
  • cargo test: 2659 unit and 4 CLI tests pass.
    • Compaction tests assert the exact event sequence for the initial text, thinking, overflow retries, prompt-too-long, the newest-round overflow, empty and unfinished summaries, retry notices, and a future dropped mid-summary.
    • Auto-compaction tests assert the full sequence for an overflow retry and an unsaved boundary. Engine tests cover manual success, failure, a dead session, and cancellation after the first token, where the discard arrives before Cancelled.
    • TUI snapshots cover streaming, completed, and cancelled states, and a text test covers a failure.
  • cargo llvm-cov: Every changed line is covered except the closing line of the test server task in serve_stalled_summary, which keeps its connection open by never finishing.
  • pnpm format, pnpm lint, pnpm spellcheck

/compact and auto-compaction showed only a spinner until the whole
summary arrived, which can take tens of seconds on a long session.

compact_session now streams the summary's text to the frontend as
Event::SummaryToken through a SummaryStream sink, which passes retry
notices through and drops the reply's thinking. When any text has
streamed, an overflow retry, a failed or empty summary, or a dropped
future (cancellation) emits Event::SummaryDiscarded before the event
that reports the outcome. A summary whose compact boundary fails to save
is discarded the same way, and a saved one ends in SessionCompacted as
before.

The TUI renders the summary in flight below the chat under a
"Summarizing the conversation..." header, laid out like the
CompactedBlock it becomes, so success swaps it for the final block in
the same layout. A discard removes it. The stdio frontends ignore both
events.
@hakula139 hakula139 added this to the v0.1.0 milestone Oct 16, 2026
@hakula139 hakula139 added enhancement A new user-facing capability or an extension of one (feat) area: agent Touches the agent turn loop, compaction, or agent events area: tui Touches terminal UI rendering, input, or theming area: engine Touches the engine, its protocol, or the drivers that run it labels Oct 16, 2026
@hakula139 hakula139 self-assigned this Oct 16, 2026
@codecov

codecov Bot commented Oct 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.77629% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/oxide-code/src/engine.rs 98.63% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@hakula139
hakula139 merged commit b9d252b into main Oct 16, 2026
10 checks passed
@hakula139
hakula139 deleted the feat/stream-compaction-summary branch October 16, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: agent Touches the agent turn loop, compaction, or agent events area: engine Touches the engine, its protocol, or the drivers that run it area: tui Touches terminal UI rendering, input, or theming enhancement A new user-facing capability or an extension of one (feat)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stream the compaction summary into the chat as it generates

1 participant