feat(core): Show sent chat messages right away, with new `status` attribute and `resend` function by jeffdaley · Pull Request #1497 · livekit/components-js

jeffdaley · 2026-11-02T21:22:00Z

Ticket

https://linear.app/livekit/issue/DE-206/agents-ui-add-agentcomposer-text-input-with-microphone

This is the SDK half of the Agents UI composer.

Context / reason for change

When you send a chat message, it doesn't appear until the server has accepted it, and if sending fails it never appears at all. In a chat UI, that makes sending feel slow, and a failed message disappears.

Chat apps usually show your message the moment you send it, then mark it if it fails and let you try again. Agents UI's new composer plans for that, and every other chat UI built on useChat benefits too.

Summary of changes

Messages appear right away: send adds the message as soon as it's called, with a status of sending. It becomes sent once the server accepts it, or failed if it doesn't.
- sent means the server accepted the message, not that the agent read it.
- send still rejects when sending fails, as before.
Failed messages stay and can be resent: the new resend(messageId) from useChat and useSessionMessages sends a failed message again in place, keeping its original position and timestamp. It throws for a message that isn't failed or is already sending.
Your messages keep one ID: a message sent from this client gets a local ID when you send it and keeps it, instead of switching to the text stream's ID once sent. The browser can't choose the stream ID before sending, so this is what keeps the message from jumping or showing twice.
- This changes the ID of every message you send. Code that matched a sent message to its stream ID will need to change.
Only your messages have a status. Messages from other participants have none.
Status changes aren't edits. A message moving from sending to sent replaces it without setting editTimestamp.
React's prefab <Chat> now shows a message as soon as you send it. A failed message stays in the list, with no indicator yet.

Screenshots / demos

No UI in this PR. The composer PR that builds on it shows the sending, failed, and retry states.

Rough idea of how it could manifest:

CleanShot.2026-11-03.at.2.01.22.PM.mp4

And a live demo from #1500: https://components-js-storybook-5kld-git-jeffdaley-agents-ui-composer.staging.livekit.io/?path=/story/agents-ui-agentcomposer--default

How to verify

CI runs the new tests. To check by hand, run pnpm --filter @livekit/components-core test and pnpm --filter @livekit/components-react test. The tests cover:

A message going from sending to sent, and to failed.
Resending a failed message in place, and rejecting a resend while it's sending or after it's sent.
Two setupChat calls for the same room sharing state.
Remote messages having no status.

Submitter checklist

I ran the change locally and inspected the affected UI/flow myself, if applicable.
I added or updated tests covering this change (unit, play function, or E2E), or explained why none are needed.
I attached screenshots or a demo for user-facing changes (ideally before + after), if applicable.
I self-reviewed the diff and removed debug code, dead code, and stray comments.

🤖 Generated with Claude Code

Messages sent from this client appear right away as sending, then become sent or failed. A failed message stays in place and can be resent with resend, exposed from useChat and useSessionMessages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vercel · 2026-11-02T21:22:02Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
components-js-storybook-5kld	Ready	Preview	Nov 2, 2026 9:23pm UTC

CLAassistant · 2026-11-02T21:22:06Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

github-actions · 2026-11-02T21:23:18Z

size-limit report 📦

Path	Size
LiveKitRoom only	7.88 KB (0%)
LiveKitRoom with VideoConference	32.81 KB (+0.57% 🔺)
All exports	44.67 KB (+0.55% 🔺)

thomasyuill-livekit · 2026-11-03T22:02:04Z

This is a little out of my wheel house as I have less familiarity with core and react packages

But can we extend the Prefab Chat story to show a live demo?

jeffdaley · 2026-11-04T17:57:27Z

The best demo at the moment is probably at https://components-js-storybook-5kld-git-jeffdaley-agents-ui-composer.staging.livekit.io/?path=/story/agents-ui-agentcomposer--default (from #1500).

I'll also add this link to the PR description.

thomasyuill-livekit · 2026-11-04T17:59:41Z

right but we need to confirm there are no regressions in the separate react component package (outside of Agents UI)

Maybe @lukasIO or @1egoman can help review this code

1egoman

Generally I think this makes sense to me, and the optimistic message adding is a great improvement! Not going to approve immediately because I'd like to get both of your takes on some of the comments I left and the below.

One bit of very important context though: This code path is about to (like in the next month) change substantially. With the new agents 2.0 A2A features, there's likely going to be a deprecation of useSessionMessages and all associated data channel based text message logic and a new hook will likely be introduced called something like useThreadMessages which will be the new way users will be recommended to implement text based agent interfaces.

Given this, and assuming this is a worthwhile improvement, I wonder if it makes sense to defer this into useThreadMessages. As is I'd likely have to duplicate and rework this code a bit to get it to fit into that new hook so if this isn't urgent, I think my preference would be for this to end up in the new stuff only. But open to other perspectives here.

1egoman · 2026-11-04T19:43:48Z

      try {
        await sendMessage(room.localParticipant, encodedLegacyMsg, {
          reliable: true,
          topic: legacyTopic,
        });
      } catch (error) {
        log.info('could not send message in legacy chat format', error);
      }


thought: I think today, if the modern version of the message were to get sent and the legacy version were to not get sent, then the legacy version of the message would get dropped and not be tracked by the unsent message infrastructure.

I'm not convinced there's really a good way to solve this because there isn't a way to send both idempotently / in a "transaction" today but maybe just something to be aware of.

Agreed, seems fine to keep this as a log.

1egoman · 2026-11-04T19:46:31Z

+function createLocalMessageId() {
+  localMessageCount += 1;
+  return `local-${Date.now().toString(36)}-${localMessageCount}`;
+}


suggestion: Today, stream ids are uuids: https://github.com/livekit/client-sdk-js/blob/4721d094b2062b08b12093f7eb8f5597bbff235c/src/room/data-stream/outgoing/OutgoingDataStreamManager.ts#L84. If you made this a uuid then from the perspective of a downstream user, the api interface would appear exactly the same.

1egoman · 2026-11-04T19:52:14Z

 type ReceivedChatMessageWithRequiredType = ReceivedMessageWithType<
  'chatMessage',
  ChatMessage & {
    from?: Participant;
    attributes?: Record<string, string>;
+    /** Set only on messages sent from this client. */
+    status?: ChatMessageStatus;
  }
 >;


suggestion: I don't necessarily like some messages having a field and some not having a different field without it being clear in what cases that field can be set.

What if you were to switch this to be some sort of discriminated union based on from? Maybe not exactly this but something conceptually like this:

Suggested change

type ReceivedChatMessageWithRequiredType = ReceivedMessageWithType<

'chatMessage',

ChatMessage & {

from?: Participant;

attributes?: Record<string, string>;

/** Set only on messages sent from this client. */

status?: ChatMessageStatus;

}

>;

type ReceivedChatMessageWithRequiredType = ReceivedMessageWithType<

'chatMessage',

ChatMessage & ({

from: Participant;

attributes?: Record<string, string>;

} | {

from?: never;

attributes?: Record<string, string>;

/** Set only on messages sent from this client. */

status: ChatMessageStatus;

})

>;

Agree the type should make it clear when status exists. One catch is that local messages do have from (LocalParticipant, which extends Participant), so keying the union on from wouldn't work. Maybe this?

type Message = ChatMessage & { from?: Participant } & ( | { isLocal: true; status: 'sending' | 'sent' } | { isLocal?: false } );

Something to settle in useThreadMessages.

so keying the union on from wouldn't work.

This was what I was trying to accomplish with from?: never; - this does work, here is a link if you want to play around with it. It's a little confusing though and I agree maybe an isLocal / some sort of enum of options could maybe be better.

1egoman · 2026-11-04T20:03:05Z

+  /**
+   * Adds the message as `sending`, then marks it `sent` or `failed`. Rejects if sending fails,
+   * leaving the failed message in place for `resend`.
+   */
+  const deliver = async (unsentMessage: ReceivedChatMessage, options: SendTextOptions) => {
+    const { id, message } = unsentMessage;
+    unsentMessages.set(id, { message: unsentMessage, options, isSending: true });
    pendingSends += 1;
    isSending$.next(true);
+    messageSubject.next({ ...unsentMessage, status: 'sending' });

    try {
-      const info = await room.localParticipant.sendText(message, options);
+      let info: TextStreamInfo;
+      try {
+        info = await room.localParticipant.sendText(message, options);
+      } catch (error) {
+        unsentMessages.set(id, { message: unsentMessage, options, isSending: false });
+        messageSubject.next({ ...unsentMessage, status: 'failed' });
+        throw error;
+      }


question: Just generally, I'm curious if you have any data around how often chat message sending fails. Even anecdotally is fine - I've yet to see one fail in local development. But it's possible I just have been treating it with kid gloves, and I haven't deeply tested cases like switching wifi networks / etc

Broader context: It's definitely possible for a data stream to fail but the bar for that has gotten a lot higher now with data streams v2. I'm actually going to be doing an EPD all hands presentation about this next month (and can send you all over the slides early if you'd be interested in learning more) but the TLDR is data streams are now over 50% smaller than they used to be, which means that on phones and poor bandwidth network connections it becomes substantially harder for them to drop (though not impossible).

I bring it up because while I think this is cool and lets a user implement a pretty nice UX, I also wonder if the complexity is really warranted if a data stream delivery failure is that much harder now. Something like interface pattern that the existing send function provides (wrap in a try/catch, show a toast or alert) could be good enough if it is really an abnormal case.

No data on my end, just seemed like something to think through, even if rare (and even less rare with data streams v2 — nice). I'm fine keeping it out for now. If it becomes an issue we can always take care of it then.

jeffdaley · 2026-11-06T19:31:22Z

Good context to know. Let's wait until useThreadMessages. I moved the PR into draft — anything we like can be worked into the new hook when the time comes.

… and transcript The composer no longer depends on #1497. A message that fails to send goes back in the composer, unless something new has been typed, and onError reports the error. The transcript no longer marks failed messages or offers a retry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vercel Bot deployed to Preview November 2, 2026 21:23

View deployment

jeffdaley changed the title ~~feat(core): show sent chat messages right away, with status and resend~~ feat(core): show sent chat messages right away, with new status attribute and resend function Nov 2, 2026

jeffdaley changed the title ~~feat(core): show sent chat messages right away, with new status attribute and resend function~~ feat(core): Show sent chat messages right away, with new status attribute and resend function Nov 2, 2026

jeffdaley marked this pull request as ready for review November 3, 2026 17:54

jeffdaley requested review from 1egoman, lukasIO and thomasyuill-livekit as code owners November 3, 2026 17:54

jeffdaley mentioned this pull request Nov 3, 2026

feat(agents-ui): add AgentComposer, a text input with a microphone #1500

Draft

4 tasks

1egoman reviewed Nov 4, 2026

View reviewed changes

jeffdaley marked this pull request as draft November 6, 2026 19:29

Conversation

jeffdaley commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ticket

Context / reason for change

Summary of changes

Screenshots / demos

How to verify

Submitter checklist

Uh oh!

vercel Bot commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Nov 2, 2026

Uh oh!

github-actions Bot commented Nov 2, 2026

size-limit report 📦

Uh oh!

thomasyuill-livekit commented Nov 3, 2026

Uh oh!

jeffdaley commented Nov 4, 2026

Uh oh!

thomasyuill-livekit commented Nov 4, 2026

Uh oh!

1egoman left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffdaley commented Nov 6, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jeffdaley commented Nov 2, 2026 •

edited

Loading

vercel Bot commented Nov 2, 2026 •

edited

Loading

1egoman left a comment •

edited

Loading