Skip to content

feat(agents-ui): announce new messages in the chat transcript - #1502

Draft
jeffdaley wants to merge 2 commits into
jeffdaley/agents-ui-composerfrom
jeffdaley/agents-ui-announce-messages
Draft

jeffdaley wants to merge 2 commits into
jeffdaley/agents-ui-composerfrom
jeffdaley/agents-ui-announce-messages

Conversation

@jeffdaley

@jeffdaley jeffdaley commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Ticket

https://linear.app/livekit/issue/DE-211/agents-ui-announce-new-messages-to-screen-readers

Builds on #1500, which adds the composer and the transcript's labels. This PR targets that branch, so the diff shows only the announcements.

Context / reason for change

Screen reader users don't hear when the agent replies in the chat transcript. They have to go looking for new messages, which makes a text conversation with an agent hard to follow.

Summary of changes

  • New messages from the agent and other participants are announced, through a new live region in the transcript.
  • Announcements wait until a message stops changing for a second, so streaming replies aren't read word by word. A reply that continues after a pause announces only the new text. Transcripts are tracked by segment, since each update arrives as a new stream with a new ID.
  • Each message waits for its own pause, so a long reply doesn't hold back others. A rewritten message is announced again in full.
  • History and your own messages stay quiet. That covers messages on screen at mount, messages that arrive together, and messages loaded above the conversation. History is detected by order and batch rather than timestamps, since remote clocks can be off.
  • Unfinished links and code spans wait up to 3 seconds to finish, so they aren't read in pieces.
  • Markdown is read as plain text. Links, emphasis, and code keep their words and lose the punctuation; lone symbols like 5 * 3 stay.
  • announceMessages chooses what's announced: 'all' (default), 'typed', or 'none'. AgentSessionView_01 uses 'typed', since the agent's voice already says its transcripts aloud.
  • labels.messageAnnouncement gets the text and its message. The default says "Agent" for agents and the participant's name for anyone else.

Screenshots / demos

No visual change.

How to verify

Run Storybook (pnpm --filter @livekit/component-docs-storybook dev) with VoiceOver on (Cmd+F5), and open Agents UI / AgentChatTranscript → InjectMessages:

  1. Press the add button. The first message is from the agent, and VoiceOver reads "Agent: This is a message from the agent." about a second later.
  2. Press it again. That message is yours, and nothing is read.

Submitter checklist

  • I ran the change locally and inspected the affected UI/flow myself, if applicable.
  • I added or updated tests covering this change (unit, play function, or E2E), or explained why none are needed.
  • I attached screenshots or a demo for user-facing changes (ideally before + after), if applicable.
  • I self-reviewed the diff and removed debug code, dead code, and stray comments.

🤖 Generated with Claude Code

@changeset-bot

changeset-bot Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 6bd612c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@livekit/agents-ui Minor
@livekit/component-docs-storybook Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
components-js-storybook-5kld Ready Ready Preview Nov 9, 2026 3:25pm UTC

Request Review

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

jeffdaley and others added 2 commits November 9, 2026 10:24
AgentChatTranscript announces messages from the agent and other participants to screen readers
once they stop changing for a second, so streaming replies aren't read word by word. A reply that
continues after a pause announces only the new text, tracked by transcription segment since each
update arrives as a new stream. History on mount and your own messages stay quiet, and Markdown is
read as plain text.

`announceMessages` chooses what's announced, and AgentSessionView_01 skips spoken transcripts,
which the agent's voice already says aloud. The default label names non-agent participants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…essage on its own

- Each message waits for its own pause, so a long reply no longer holds back other messages.
- A rewritten message, including one edited to the same length, is announced again in full.
- History that loads after mount stays quiet: messages that arrive together, or that are placed
  before ones already on screen.
- An unfinished link or code span waits up to 3 seconds to finish, so it isn't read in pieces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 6bd612cb Deployed Nov 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants