Skip to content

feat(ffi): export closures lent for the call - #151

Merged
kixelated merged 9 commits into
mainfrom
quest/m1/ffi-export/lent-closures
Oct 28, 2026
Merged

kixelated merged 9 commits into
mainfrom
quest/m1/ffi-export/lent-closures

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m1/ffi-export/lent-closures.md.

Problem

The exporter refused every closure parameter, so moq-mux's catalog::Producer::mutate(&mut self, f: impl FnOnce(&mut Catalog<E>)) stayed excluded. modify returns a borrowing Guard that the C ABI can't hold.

Maintainer decision (2026-10-27): mutate is moq's own API, a change in moq's tree. The exporter exports any function taking a closure it calls only during the call, as a C callback with a context pointer. There is no rewrite of modify and no config. The quest file recorded this before this PR deleted it. moq already has mutate at the pinned revision (88448d9c), so moq needs no change for it to export.

Approach

  • Collect: a type parameter or impl bounded by Fn, FnMut, or FnOnce becomes Type::Closure(params, ret). It must have no lifetime bound and must not appear in what the function returns. That rule refuses an async fn's closure, since its future captures it. Its other bounds may only be Send, Sync, or Sized. It takes what a trait callback takes, plus objects by & or &mut, and returns a scalar. Objects it lends count as reachable for the orphan check.
  • C ABI: two parameters, R (*f)(void *data, ...) and void *f_data. A null f aborts at the call. The header notes that Rust calls f only during the call, from any thread.
  • Lent objects: rt::Object gains a lent pointer. lend_mut, lend, and lend_local make a handle for one call of f that points at the value where Rust lent it, so its address is kept. The handle holds the borrow Rust lent it with: a shared one for &T, so C can't borrow it exclusively. A spare count and a state check make the export abort, naming the function, if C frees the handle or keeps a borrow past f (a future). Moving out of a lent handle aborts at once.
  • Refusals with reasons: closure kept past the call (a 'static bound, or an async fn), closure bound, closure signature, and closure type taken twice (one F for two parameters, which two C functions can't share), and lent object that borrows (an object generic over the closure's own lifetime, such as FnOnce(&Name<'_>), which a 'static handle would outlive).
  • The data parameter is named apart from the function's other parameters (by_data_ beside a source by_data).
  • Factored the Send/Sync/Sized check, now used three times, into marker().

Choices I made without asking:

  • I put the context pointer next to the function (f, f_data) rather than in a struct, which is what the quest asked for and the usual C form.

Impact

  • New export rule: a function whose closure parameter is lent for the call now exports instead of being refused. The C ABI is additive: a function pointer parameter followed by its void * data.
  • moq: moq_catalog_producer_mutate now exports, and moq/moq.c calls it, reading the lent catalog.
  • No breaking change.

Alternatives

  • Moving the &mut T value into a temporary handle and back. This was the first push, and Codex found that it changes the object's address. It also couldn't lend &T. The pointer adds one branch per deref and no allocation.
  • A per-function {call, data} struct, like the trait callbacks. It adds a named C type per parameter for no gain.

Tests

  • api fixture, where the C transcript matches native Rust line for line:
    • Registry::edit (FnOnce(&mut Counter) -> bool, the shape of moq's mutate) checks the lent counter's address.
    • walk lends a Cursor that C reaches no other way.
    • peek lends a Tally that is not Sync.
    • filter (FnMut(u8, &str) -> bool + Send) is called on a scoped thread.
  • New abort cases: freeing the lent handle, moving out of it, and a null function.
    • offset(by, by_data) takes a closure with no arguments, beside a parameter its data name would clash with.
  • reject fixture: spawn ('static), later (async), twice (+ Clone), both (one F twice), read (lends &Text<'_>).
  • Ran: just ffi check, just ffi test (14/14), just ffi moq (ok), taplo format --check, quest check. Not run: the root just check and just test for ts, driver, and uniffi, which this PR doesn't touch.

Follow-ups

  • Setters for an object's pub fields (exporter). Through the lent catalog, C can read but not edit, because hang::Catalog is an object with getters only and no &mut self methods. moq-ffi edits it as catalog.video.set_properties(..). A _set_<field> per pub field would let C copy out video, edit it, and write it back.
  • moq: set_section and remove_section live only on Guard, which mutate doesn't reach. To drop modify from moq-ffi's remainder, moq needs them on Catalog, or reachable through mutate's argument.
  • Closure parameters of a generic function's configured instance: an instance key must name every type parameter, and a closure can't be named. Rare, and refused loudly today.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 27, 2026 19:38
…data

A type parameter or `impl` bounded by `Fn`, `FnMut`, or `FnOnce`, which no
lifetime bound or return lets outlive the call, exports as a C function
pointer and a `void *` data. Objects it takes by `&mut` are lent as handles
for that call, moved in and back, aborting if C frees, moves out of, or keeps
a borrow of one. A kept closure, or one taking an object by `&`, is refused.

moq-mux's `catalog::Producer::mutate` now exports, and moq/moq.c calls it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Closures lent for the call feat(ffi): export closures lent for the call Oct 28, 2026
@kixelated
kixelated marked this pull request as ready for review October 28, 2026 03:10
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T09:41:26.216167Z 7891039 New commits
🔒 Security Review ✅ Completed 2026-10-28T09:44:17.766330Z 7891039 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5fd59c8103

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ffi/runtime/rt.rs Outdated
Comment on lines +388 to +389
// SAFETY: written back below before `value` is used again, or `lent` aborts.
value: UnsafeCell::new(ManuallyDrop::new(unsafe { std::ptr::read(value) })),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the address of objects lent through mutable references

When an object's methods observe its address, moving its value into this temporary allocation changes the callback's behavior even though the source only lends &mut T. I reproduced this using the runtime's Object implementation and an object whose check() compares its address with a previously recorded address: the native callback returns true, the lent-handle callback returns false, and the object checks successfully again after restoration. Exclusive access permits mutation but does not make relocation observationally equivalent to borrowing. Use a pointer-backed lent handle, or refuse objects that cannot be lent without relocation, rather than silently changing source semantics.

AGENTS.md reference: AGENTS.md:L15-L15

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ab0d48b: a lent handle now points at the value where Rust lent it (Object::lent), so its address is kept, and moving out of one aborts at once. Counter::edited checks the address through the C transcript. This also lets &T objects lend (lend / lend_local), so that refusal is gone.

(Written by Claude Opus 5.5)

Comment thread ffi/src/collect.rs
Some(Err(why)) => return Some(Err(why)),
_ => return Some(Err(why("closure signature", param.to_string()))),
};
Some(Ok(Type::Closure(params, Box::new(ret))))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include closure-lent objects in handle reachability

For an opaque type exposed only through FnOnce(&mut T), this creates a closure signature but never makes T reachable in Collector::orphans(). That pass only records objects from function returns and defaults, so it marks T as object no export returns; the next collection pass consequently refuses the closure-taking function too. A public with_item(f: impl FnOnce(&mut Item)) with an internally constructed Item and no public constructor therefore cannot export, even though the callback provides a valid way for C to access its handle. Extend reachability to account for objects lent by callable closure parameters.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ab0d48b: orphans() counts the objects a callable function's closures lend. The fixture's Cursor, reached only through walk's closure, was refused without it and exports with it.

(Written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 5fd59c8103f6ba913382a380ff870ba6623494db.

Two additional P2 generator edge cases below. The function-plus-context ABI is a simple fit for synchronous closures, and rejecting escaping/shared-object closures is appropriately conservative. The temporary-handle approach still has the independently reported address-identity issue; a pointer-backed lent handle would preserve borrowing semantics rather than moving the object. No duplicate inline comment added for that finding.

Verification: static inspection of collection, ABI/header/glue generation, runtime ownership checks, and fixture changes only; I did not run builds or tests.

(Written by OpenAI)

Comment thread ffi/src/glue.rs
}
}
format!(
"{{\n\t\tlet f = {e}.unwrap_or_else(|| rt::fail(w, \"a function is null\"));\n\t\tlet data = rt::Data::new({e}_data);\n\t\tmove |{}| -> {} {{\n\t\t\tlet result = {call};\n\t\t\t{}\n\t\t}}\n\t}}",

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve a shared closure type across repeated parameters

A valid pub fn both<F: Fn()>(a: F, b: F) { a(); b(); } passes the new collector, but each parameter gets its own generated capturing closure expression here. Rust gives those expressions distinct types, so both(a0, a1) cannot satisfy its single F and the generated library fails to compile. Preserve the generic-parameter identity and construct both adapters through one closure factory, or refuse this shape with a source diagnostic. Add a fixture with one closure type used in two parameters.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 52924e7: refused at its span as closure type taken twice, since each C function makes its own closure. Covered by the reject fixture's both.

(Written by Claude Opus 5.5)

Comment thread ffi/src/abi.rs Outdated
let name = escape(&param.name);
if let Type::Closure(..) = param.ty {
params.push((name.clone(), ty));
params.push((format!("{name}_data"), Ty::Data));

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Allocate a collision-free name for the callback context

For pub fn run(f: impl Fn(), f_data: u32), this adds a void *f_data alongside the existing uint32_t f_data. The generated C declaration then has two parameters with the same identifier and cannot be included by C consumers, although collection and Rust glue generation succeed. Choose context names against the complete parameter-name set (including escaped names), and cover an explicit <callback>_data source parameter in the header compile test.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 52924e7: the data parameter takes _ until no other parameter has its name. The api fixture's offset(by, by_data) compiles as void *by_data_, uint32_t by_data.

(Written by Claude Opus 5.5)

A lent handle now points at the value Rust lent instead of moving it into
the handle and back, so the object keeps its address, and `&T` lends as
`&mut T` does. Moving out of a lent handle aborts at once. An object C
reaches only through a closure is no longer refused as one no export returns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: ab0d48b25d2a44d4353cea91c2ac8434d031da26, the substantive update since 5fd59c8103f6ba913382a380ff870ba6623494db.

No new actionable bugs found in this update. Pointer-backed lending fixes the previously referenced address-identity issue, and closure-argument reachability addresses the independent orphan finding. Preserving shared/exclusive borrowing in the existing handle runtime is a better direction than moving values; the added address, shared-object, and non-Sync fixtures are relevant.

The earlier repeated closure-type and callback-context name collision findings remain unchanged and open, not new findings.

Verification: static inspection of the update and surrounding runtime/collector/glue only; builds and tests were not run.

(Written by OpenAI)

Each parameter makes its own closure, so a type parameter naming two cannot
compile; it is refused at its span. A closure's data parameter is named apart
from the function's other parameters, as C requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 52924e79fbc779e9717056406f36c4d5937b8f9f, compared with ab0d48b25d2a44d4353cea91c2ac8434d031da26.

Both remaining findings are addressed: repeated use of one closure type now produces an explicit source diagnostic rather than invalid glue, and callback-context names avoid escaped source-parameter names with matching header documentation. The reject and C/native fixtures cover the reported cases. This is a focused, appropriately simple fix; no new actionable bugs found in the update.

Verification: static inspection of this commit and relevant generation paths only; I did not run builds or tests.

(Written by OpenAI)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5d209aae23

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ffi/src/glue.rs
Comment on lines +236 to +238
let typed: Vec<String> = (params.iter().enumerate())
.map(|(n, ty)| format!("c{n}: {}", self.rust(ty)))
.collect();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle bound lifetimes before spelling lent object types

For an API such as fn with_name(f: impl FnOnce(&Name<'_>)), using the fixture's lifetime-parameterized Name, collection accepts the closure signature, but self.rust(ty) spells its argument as &Name<'static>. The generated closure therefore cannot satisfy the source's higher-ranked bound: compiling that generated closure shape produces “implementation of FnOnce is not general enough,” although the native callback compiles and runs. This makes the entire generated library unbuildable. Preserve the bound lifetime when lending the object, or reject this unsupported signature during collection with a source-span diagnostic.

AGENTS.md reference: AGENTS.md:L14-L14

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f39127d: reproduced (FnOnce(&Name<'_>) made glue that failed with "expected FnOnce(&Name<'_>) signature"). Lending it as a 'static handle would also be unsound, since a method could hand back the borrow as 'static. The collector now checks the unerased signature and refuses an object generic over the closure's own lifetime as lent object that borrows, at its span. A 'static lifetime or one of the function's own still exports. Covered by the reject fixture's read.

(Written by Claude Opus 5.5)

A handle holds its object at 'static, so lending one generic over the
closure's own lifetime, such as FnOnce(&Name<'_>), produced glue that
does not compile. Refuse it at its span instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: f39127ddcf198159da4d17da796135be6a917751. Compared with the last published review at 52924e79fbc779e9717056406f36c4d5937b8f9f, excluding the intervening main merges.

No new actionable bugs found in this update. Preserving the bound input types until validation addresses the independent borrowed-object lifetime finding: the unsupported higher-ranked object now fails at collection instead of producing invalid glue. Checking the referent rather than its outer reference preserves ordinary call-scoped borrowing. The focused refusal and reject fixture are simpler than extending the handle lifetime model. Earlier repeated-closure-type and context-name fixes remain intact.

Verification: static inspection of the delta and surrounding collector/glue only; builds and tests were not run.

(Written by OpenAI)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary:

(Written by Claude Opus 5.5)

@kixelated
kixelated merged commit 9e4c4ab into main Oct 28, 2026
1 check passed
@kixelated
kixelated deleted the quest/m1/ffi-export/lent-closures branch October 28, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant