Repository navigation
feat(compaction): stream the summary into the chat as it generates - #252
Merged
Merged
Conversation
/compact and auto-compaction showed only a spinner until the whole summary arrived, which can take tens of seconds on a long session. compact_session now streams the summary's text to the frontend as Event::SummaryToken through a SummaryStream sink, which passes retry notices through and drops the reply's thinking. When any text has streamed, an overflow retry, a failed or empty summary, or a dropped future (cancellation) emits Event::SummaryDiscarded before the event that reports the outcome. A summary whose compact boundary fails to save is discarded the same way, and a saved one ends in SessionCompacted as before. The TUI renders the summary in flight below the chat under a "Summarizing the conversation..." header, laid out like the CompactedBlock it becomes, so success swaps it for the final block in the same layout. A discard removes it. The stdio frontends ignore both events.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/compactand auto-compaction used to show only a spinner while the summary was being generated, which could take tens of seconds in a long session. Now the summary streams into the chat as it is generated, using the layout of the compacted block it will become. Every compaction outcome also removes or replaces it cleanly.Event::SummaryToken(String)andEvent::SummaryDiscarded, carry summary updates. A sequence ofSummaryTokenchunks ends withSessionCompactedon success orSummaryDiscardedotherwise. A discard precedes theTurnFailed,Cancelled, orErrorevent that reports the outcome. If an overflow triggers a retry, the discard comes before streaming resumes, so the frontend never shows two attempts at once. Both events are included in theevery_eventround-trip table.compact_sessiontakes the event sink and reads the reply through aSummaryStreamsink, which forwards text asSummaryToken, passes retry notices through, and drops thinking. It emitsSummaryDiscardedonly if text has streamed. ItsDropimplementation handles cancellation, sinceawait_unless_aborteddrops the summarization future before the caller reportsCancelled.replace_session_with_summarydiscards the summary before reporting a failed boundary write.ChatViewkeeps the in-flight summary in its own buffer and displays it a blank row below the chat usingCompactedBlock::render_streaming, under aSummarizing the conversation...header.SessionCompactedreplaces it with the final block.SummaryDiscardedremoves it and subtracts it from the paused new-content count./compactdesign, including its component notes and design decision 12, the auto-compaction steps, and the welcome screen's emptiness rule now describe the stream.Closes #186
Test plan
cargo fmt --all --checkcargo buildcargo clippy --all-targets -- -D warningscargo test: 2659 unit and 4 CLI tests pass.Cancelled.cargo llvm-cov: Every changed line is covered except the closing line of the test server task inserve_stalled_summary, which keeps its connection open by never finishing.pnpm format,pnpm lint,pnpm spellcheck