Skip to content

fix(ffi): run a moq-tokio session from C, C++, and Kotlin, and end a destroyed C++ task's executor use - #211

Merged
kixelated merged 9 commits into
mainfrom
quest/m1/ffi-export/session-tokio
Oct 29, 2026
Merged

kixelated merged 9 commits into
mainfrom
quest/m1/ffi-export/session-tokio

Conversation

@kixelated

@kixelated kixelated commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The session-tokio quest's Goal is a binding that connects and accepts moq sessions with one call each, proven from C against a local server, including a server that binds port 0 and reads its address back. Until #196 pinned moq-tokio's fallback runtime (moq-dev/moq#7123) and #190 let C set listen.bind, nothing outside Rust could run a session.

Approach

just ffi moq now runs the same QUIC round trip from each generator. A server binds [::]:0, reads its real port back with local_addr, and serves an origin with the one call serve_publish. A client dials moqt://localhost:<port> with the one call consume, waits for established, requests the broadcast, subscribes, and reads the frame the server published. Then it closes the connection and the client and stops the server. Nothing in the test starts a runtime: moq-tokio's fallback runs the sockets, and its worker threads wake the caller.

  • C (moq/moq.c): the waker signals a condition variable from moq-tokio's threads. An AWAIT loop polls the server's accept loop alongside each client future.
  • C++ (moq/moq.cpp): the server runs through future::then on a worker-thread executor, and the client blocks on future::get() for each step.
  • Kotlin (moq/moq.kt): the server runs in a launched coroutine, and the client's suspend funs resume off Rust's wakes under runBlocking.

Each language fails loudly instead of hanging on a stall. C aborts after 30 seconds without a wake, C++ has a 30-second watchdog thread, and Kotlin uses withTimeout(30.seconds).

Verified:

  • Runs: just ffi moq, just check, just test. Locally C passed 30/30 runs, C++ 30/30 and Kotlin 10/10. C and C++ each finish the whole script in about 50 ms.
  • Sabotage: a server publishing "howdy" instead of "hello" fails each language's read check.

The automated review found a race in the C++ runtime that the C++ round trip exposed, fixed here in runtime/rt.hpp:

  • The race: a wake from Rust can win a task's idle -> queued and pause before posting to the executor. A task destroyed in that window returned at once, and the late post then called an executor its owner may already have destroyed.
  • The fix: each post now holds a lock across the executor call and skips a cancelled task. cancel() takes that lock after marking the task done. Once destroying a task returns, its executor is never called again. This holds for then, via, and a task cancelled from its own callback. The task, then, and via docs and ffi/README.md now say so.
  • The reproduction: the new check in tests/api.cpp uses an executor whose second call, the wake's post, blocks on a latch. It destroys the task on another thread while the executor is blocked, and fails if the destroy returns before the latch is released. Before the fix it failed 20 of 20 runs, and after it passes 20 of 20 at each C++ standard.
  • Why the check waits 100 ms: it can't be fully deterministic. A destroy that correctly waits shows only by not returning, and the runtime has no hook to observe it waiting, so the check gives the destroy 100 ms. The check can never fail with the fix in place. Without the fix it fails whenever the destroying thread runs within 100 ms, which in practice is every run.
  • moq.cpp: with the barrier, its worker no longer needs shared queue state, so it is back to a plain worker joined after its task is destroyed.

The quest is done, so this PR also:

  • deletes quest/m1/ffi-export/session-tokio.md and its links, and drops session-quic.md's dangling link to it;
  • updates ffi/README.md, whose description of moq/moq.c still listed the binary-track publish chore: bump the moq pin past moq-tokio's fallback runtime #196 removed;
  • documents in ffi/README.md that Rust may call a waker's wake and drop from any thread, as runtime/rt.rs already requires.

Impact

  • No public API change, and moq's generated names are unchanged.
  • Behavior: destroying a C++ task, from then or a coroutine awaiting via, now waits out a post to the executor already under way. After that, the executor is never called again, so an executor may be destroyed right after its tasks.
  • ffi/README.md now states the existing waker contract that wake and drop may come from any thread.

Alternatives

  • Driving the server with bind, accept and Request::ok instead of serve_publish. This is more code per language and doesn't exercise the one-call path the quest's Goal names.
  • Leaving a stall to CI's job timeout. A stall would then cost a whole job and say nothing about where it hung.

Follow-ups

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 3 commits October 28, 2026 21:15
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Sessions on moq-tokio test(ffi): serve and read a moq-tokio session from C, C++, and Kotlin Oct 29, 2026
@kixelated
kixelated marked this pull request as ready for review October 29, 2026 04:42
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T14:59:07.745854Z aee07e6 New commits
🔒 Security Review ✅ Completed 2026-10-29T14:58:48.809876Z aee07e6 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 2fbd68c270

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 2fbd68c270

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 2fbd68c2702ab3275d08a9c463d9486a64d6dcaf.

P2: Keep executor state alive through late wake submissions (ffi/moq/moq.cpp:83–89, teardown at 382–384). The executor captures the stack worker by raw this. A Rust wake can win idle -> queued in ffi/runtime/rt.hpp:638, pause before post(), and resume after main cancels serving and destroys the worker. driver::cancel() waits for polling/callbacks, not an in-flight post; the surviving driver can then invoke this executor and access a destroyed mutex/queue. Draining the queue and joining its consumer does not synchronize that producer. Capture shared queue state that survives executor copies and rejects submissions after shutdown, or add an explicit active-post barrier. A deterministic delayed-post/cancel/shutdown test would cover this race.

Direction: useful cross-language coverage of the one-call, fallback-runtime session path; fix the executor lifetime before relying on the new C++ smoke test. Static inspection, including an independent race check; I did not run builds, tests, or the reported repeated runs.

(Written by OpenAI)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Fixed the P2 in 5e2eb96: moq.cpp's worker keeps its queue in shared state that every executor copy holds and that outlives the worker, so a late post after the task and worker are gone lands in a stopped queue and is dropped. Ran 30/30 locally, and just ffi moq and just check pass.

The root cause sits in runtime/rt.hpp: task's destructor doesn't wait for a post that's already in flight. That's listed under Follow-ups as a maintainer decision: either document that the executor must outlive its tasks, or add a barrier in cancel().

(Written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 5e2eb96e970dc68ca9a9c43960b39e69cd2fbe62, compared with 2fbd68c2702ab3275d08a9c463d9486a64d6dcaf.

The previous P2 executor-lifetime race is fixed: executor copies now retain the shared queue after the worker is gone, shutdown rejects late submissions under its mutex, and rejected jobs are destroyed after that lock is released. Already queued work still drains before joining. No new actionable bugs found in this delta.

Direction: the shared shutdown-aware state is a small, appropriate fix without changing the runtime API. Static lifetime/locking review with an independent pass only; I did not run builds, smoke tests or a deterministic race test.

(Written by OpenAI)

@kixelated kixelated changed the title test(ffi): serve and read a moq-tokio session from C, C++, and Kotlin fix(ffi): run a moq-tokio session from C, C++, and Kotlin, and end a destroyed C++ task's executor use Oct 29, 2026

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: c8b1826

No new actionable bugs found in the substantive delta since 5e2eb96, excluding incoming main changes. The earlier fixture-lifetime P2 remains addressed, now by the runtime contract: post checks done while holding the executor-call barrier, and cancellation marks done then drains that barrier after releasing the polling mutex. This covers a wake delayed before posting and an executor call already under way, including cancellation from the completion callback under the documented never-inline executor contract. The fixture can consequently return to a plain worker whose lifetime encloses task cancellation. Kotlin's payload conversion matches the incoming byte-array API.

Direction: enforcing the lifetime guarantee centrally is preferable to requiring every consumer to retain shutdown-aware queue state. The added blocked-post regression targets the relevant race; its 100 ms negative observation remains scheduler-dependent, as documented.

Verification: static delta and locking/lifetime review with an independent pass. No builds, C++ standard matrix, smoke tests, or race tests run; reported results were not independently reproduced.

(Written by review (OpenAI))

@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 24b8abe062

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 24b8abe062

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@kixelated
kixelated merged commit 69247fb into main Oct 29, 2026
1 check passed
@kixelated
kixelated deleted the quest/m1/ffi-export/session-tokio branch October 29, 2026 15:59
kixelated added a commit that referenced this pull request Oct 29, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 29, 2026
… read path (#221)

* chore(quest): claim quest/m1/ffi-export/kotlin.md

Quest-Claim: e8a216b2949abd4f

* perf(ffi): fuse a Kotlin future's first poll into the call that makes it

The call that makes a future polls it once with no waker, returning its result if ready,
and makes the waker node only if it pends. A ready read crosses JNI once rather than five
times, and makes no global reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ffi)!: lend Kotlin returned bytes as a Bytes, and free what nobody closed once collected

Bytes Rust returns cross JNI as their owner: a Bytes lends Rust's buffer as a read-only
direct ByteBuffer until close() frees it. Arguments stay ByteArray. An object or Bytes never
closed is freed once the GC collects it, through one phantom reference queue a daemon
thread drains, which every JVM and Android level has.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ffi): read #213's iterated bytes as a Bytes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ffi): keep a Bytes reachable while it copies its bytes out

A duplicate of the read-only view the GC tracks keeps only the direct buffer under it on
the JVM, so the GC could free the bytes mid-copy. Copying under the Bytes' lock keeps it,
and so its buffer, reachable, and makes close() wait for the copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ffi): check Kotlin's own contract, which hashes its JNI glue

The library's contract hashed only the C declarations, so a Kotlin file and a library
whose JNI signatures differ, as this branch's do, passed the load check. The Kotlin's
contract now hashes the header's with everything the Kotlin and its glue add.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ffi): read #211's session payload as a Bytes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Luke Curley <432854+kixelated@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant