Skip to content

[Brook S8] Added Activity Streams, SyncStream and BufferedPublisher. - #378

Open
moetemp wants to merge 7 commits into
moe/brook/review/s7-workflow-streamsfrom
moe/brook/review/s8-activity-handles
Open

moetemp wants to merge 7 commits into
moe/brook/review/s7-workflow-streamsfrom
moe/brook/review/s8-activity-handles

Conversation

@moetemp

@moetemp moetemp commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

What was changed

This PR adds the Stream calls in Activities, and BufferedPublisher, SyncStream and SyncSubscription in temporalio.client.

  • In an Activity, a Stream of the Worker's client, such as from activity.client().to_stream(handle), writes as the attempt. So a retry withdraws the Records of the earlier attempts. A Stream of another client writes as that client.
  • A sync Activity has no activity.client(), so it keeps the Worker's client, such as in an Activity class, and wraps a Stream in SyncStream(stream), whose calls block. publish and subscribe of a Stream in a sync Activity raise StreamUnsupportedError, and the message names SyncStream.
  • publish starts the write at the call. The end of the attempt waits for the attempt's writes, and a write error that no caller got fails the attempt.
  • BufferedPublisher(stream, *, max_delay=...) takes a Stream and batches its publishes, in order, and flushes at the end of an Activity attempt. Workflow code has no Stream, and a WorkflowStream passed with no type checks raises StreamUnsupportedError.
  • close, async with and SyncStream.close raise StreamNotOwnerError in an Activity with no call, and async with raises at __aenter__. Only the creator's side closes a Stream, so a Workflow's Stream closes from Workflow code and a client's Stream from a client. The methods stay, since Stream is also the client's class. producer raises StreamUnsupportedError, since the attempt is the Producer.
  • A sync call runs on the Worker's loop. _run makes the call's future before it schedules the call, so a cancel of the Activity thread after the schedule also cancels the call. With asyncio.run_coroutine_threadsafe, a cancel can leave a long poll running. A call when the Worker's loop is already closed closes its coroutine, so nothing warns at shutdown.

Depends on S7 (#377). No changelog fragment, since S9 has the Streams entry.
Part of the Brook review series.

Why?

Agents stream model tokens from an Activity. The Workflow keeps the Stream, and the Activity writes to it.

Checklist

  1. How was this tested:
    • poe lint, poe gen-docs and pytest tests/contrib/streams on both backends. The sync cancel tests ran in a loop on Python 3.13 and 3.14, alone and in parallel.
    • Fork CI passes, except the arm jobs, since the fork has no arm runners.

@moetemp moetemp added the skip-changelog Changelog entry rides another PR label Oct 25, 2026
@moetemp
moetemp force-pushed the moe/brook/review/s7-workflow-streams branch from b298e44 to 90966d6 Compare October 26, 2026 00:14
@moetemp
moetemp force-pushed the moe/brook/review/s8-activity-handles branch 4 times, most recently from 24f6a90 to 79cf55a Compare October 26, 2026 04:52
@moetemp
moetemp force-pushed the moe/brook/review/s7-workflow-streams branch from 1c5e48f to 84f1587 Compare October 26, 2026 08:44
@moetemp
moetemp force-pushed the moe/brook/review/s8-activity-handles branch from 79cf55a to 15eebd7 Compare October 26, 2026 08:44
@moetemp
moetemp force-pushed the moe/brook/review/s7-workflow-streams branch from 84f1587 to c3f30a4 Compare October 27, 2026 09:07
@moetemp
moetemp force-pushed the moe/brook/review/s8-activity-handles branch from 15eebd7 to b3da39a Compare October 27, 2026 09:07
@moetemp
moetemp force-pushed the moe/brook/review/s7-workflow-streams branch from c3f30a4 to b9572b1 Compare October 27, 2026 09:17
@moetemp
moetemp force-pushed the moe/brook/review/s8-activity-handles branch from b3da39a to 212ef71 Compare October 27, 2026 09:17
@moetemp
moetemp force-pushed the moe/brook/review/s7-workflow-streams branch from b9572b1 to 925cf34 Compare October 28, 2026 01:33
@moetemp
moetemp force-pushed the moe/brook/review/s8-activity-handles branch from 212ef71 to 4a67b4e Compare October 28, 2026 01:33
@moetemp
moetemp force-pushed the moe/brook/review/s7-workflow-streams branch from 925cf34 to cc8e51c Compare October 28, 2026 07:54
@moetemp
moetemp force-pushed the moe/brook/review/s8-activity-handles branch from 4a67b4e to 0c7ec5b Compare October 28, 2026 07:54
An Activity writes as its attempt, so a retry withdraws the Records of
the earlier attempts. An Activity's `BufferedPublisher` flushes at the
end of the attempt.
@moetemp
moetemp force-pushed the moe/brook/review/s7-workflow-streams branch from cc8e51c to e7710c0 Compare October 28, 2026 08:05
@moetemp
moetemp force-pushed the moe/brook/review/s8-activity-handles branch from 0c7ec5b to 86d50a1 Compare October 28, 2026 08:05
In an Activity a handle passes each call to the Activity calls, which write as the attempt's
Producer. A write error that no caller got fails the attempt. Only the creator's side closes a
Stream, so a close and `async with` raise before any call. `SyncStreamHandle` wraps a handle for
a sync Activity, and `BufferedPublisher` takes a handle in client and Activity code.
# Conflicts:
#	temporalio/common/__init__.py
#	temporalio/common/_streams/__init__.py
#	temporalio/common/_streams/_handle.py
#	tests/contrib/streams/test_redis.py
@moetemp moetemp changed the title [Brook S8] Added Activity Stream handles and BufferedPublisher. [Brook S8] Added Activity Streams, SyncStream and BufferedPublisher. Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Changelog entry rides another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant