Skip to content

feat(ffi): export IntoIterator arguments and serde_json::Value - #202

Merged
kixelated merged 16 commits into
mainfrom
quest/m1/ffi-export/instance-keys
Oct 29, 2026
Merged

kixelated merged 16 commits into
mainfrom
quest/m1/ffi-export/instance-keys

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes and deletes quest/m1/ffi-export/instance-keys.md.

Problem

moq-net's track::Subscription::with_ranges(impl IntoIterator<Item = Range>) was refused. moq-mux's json_snapshot and json_stream exported at String, which serializes one JSON string rather than a document, and moq's config excluded them and their update/append.

Decision (maintainer, 2026-10-28)

serde_json::Value is a runtime JSON object, as #190's url::Url is, and never a string. This replaces the earlier "JSON text" plan and the parse-error options this PR asked about.

  • API: only a parse and a to-text.
    • C: <prefix>_json_parse(buf, &out, &err), <prefix>_json_to_text(json), <prefix>_json_free.
    • C++: json::parse(view), which returns result<json, buf>, and .to_text().
    • Kotlin: Json.parse(String), which throws StringException, and .toText().
    • Text crosses as String does.
  • Errors: parse errors come only from parse.
  • No contents API: callers read the text with their own JSON library.
  • Signatures: every function taking or returning a Value keeps its Rust signature and passes the handle. Nothing in moq changes, so there is no moq PR.
  • Inference: a serde Serialize, Deserialize, or DeserializeOwned bound infers serde_json::Value, not String.

Approach

  • IntoIterator: choose ranks Vec<X> for a parameter whose IntoIterator bound fixes Item = X. It reuses feat(ffi): export Url, SocketAddr, paths, and Vec<T> across the C ABI #190's Vec<T>.
  • Foreign types: feat(ffi): export Url, SocketAddr, paths, and Vec<T> across the C ABI #190's single url slot becomes a list of foreign types (url::Url, serde_json::Value). Each exports as an object made by parsing text, when some crate builds with it and no export is its crate.
    • Parse reuses Access::Parse; to_text is a new Access::Text, which calls Display.
    • The generated crate depends on serde_json = "1", which cargo unifies with the exports', as with url.
    • A foreign type counts as an instance argument, so json::Snapshot<Value> exports, named moq_json_snapshot_json.
    • Its C name is json, by role, rather than the naming rule's value.
    • prune drops each unused foreign object and renumbers the objects after it. It now also counts one as used when it is only a type argument; before, an instance argument naming Url alone would have been pruned.
  • fix(ffi)!: export byte payloads as bytes #197's From-impl rule: it now also takes a source that is no type parameter as it is. impl<'a, V> From<&'a V> for Timed<&'a V> makes impl Into<Timed<&'a T>> take &T.
  • Ordered choices: a later parameter's bounds now see each earlier one chosen (choose gets the running arguments), so that &T resolves to &Value. The same fix exports moq-tokio's nested tls::Identity::generate and Addrs::collect.
  • Proof:
    • api fixture: C, C++ at 17/20/23, and Kotlin match native Rust's transcript for:
      • encode<T: Serialize>(&T)
      • tagged(Value, &str) -> Value, a handle in and out
      • timed_text<'a, T: Serialize>(impl Into<Timed<&'a T>>), moq's shape
      • a malformed parse in each language
      • summed(impl IntoIterator<Item = Point>) and longest<I: IntoIterator<Item = String>>, empty and non-empty
    • moq/moq.c, moq.cpp, moq.kt, in place of the binary_snapshot block chore: bump the moq pin past moq-tokio's fallback runtime #196's pin removed:
      • Publish a JSON track through json_snapshot; the catalog claims and lists it.
      • update it with a value parsed from text, and read it back off consume()'s subscriber as {"on":true}.
      • Set a catalog section through catalog::hang::Extra::set in the lent catalog, and read it back with get(...).to_text().
      • Check a malformed parse's serde_json message and a reserved section as moq-mux's error.
      • Read back a subscription built with_ranges([live, groups(2..6)]).

Census (moq pin 30f25c04, methods and fields exported / refused, origin/main 320b000 vs head)

Package main this PR
moq-mux 1045 / 70 1053 / 62
moq-net 554 / 21 557 / 20
moq-pattern 28 / 2 29 / 1
moq-tokio 279 / 46 282 / 43

Types and functions are unchanged, and so are hang, kio, and moq-clock.

Newly exported:

  • with_ranges
  • json::Snapshot::update and json::Stream::append
  • catalog::hang::Extra::get/set/remove
  • the av1/h264/h265 Import::decodes
  • Pattern::new
  • connect::Addr::pinned, Addrs::collect, and tls::Identity::generate

moq/ffi.toml drops 13 excludes: with_ranges, three decodes, Pattern::new, json_snapshot, json_stream, json::Snapshot, json::Stream, and Extra::get/set/remove.

Impact

  • New: <prefix>_json with _parse, _to_text, and _free wherever an export names serde_json::Value (C++ json, Kotlin Json). Each impl IntoIterator<Item = X> parameter takes <prefix>_vec_<X> / std::vector<X> / List<X>. An impl Into<X> whose From impl has a concrete source takes that source.
  • Breaking (MIGRATE.md): a serde-bounded type parameter takes <prefix>_json, not a buffer, and a generic type at one is named ..._json, not ..._string. The new names can clash with an export's.

Alternatives

  • Text everywhere, with a parse error converted through the call's error type (From<serde_json::Error>) or carried in an extra error out. The maintainer replaced both with the runtime object.

Follow-ups

  • json::Consumer<T> stays excluded. Its only instance comes through catalog::Entry::subscribe, which moq's config excludes, and an inherent impl's bounds (impl<T: DeserializeOwned> Consumer<T>) don't seed instances. This could be a quest if a binding needs typed JSON reads.
  • Timed<Value> (moq-net's Timed, excluded) would clash moq_timed_json_at between its field and its method if it were ever exported.

(written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 28, 2026 16:51
moq-net's with_ranges takes its ranges as a Vec of Range, and moq-mux's
codec decodes, moq-pattern's Pattern::new, and moq-tokio's
connect::Addr::pinned export by the same rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Iterator and JSON arguments the exporter can't map yet feat(ffi): export IntoIterator and JSON arguments Oct 29, 2026
kixelated and others added 4 commits October 28, 2026 20:15
A serde bound takes serde_json::Value, which exports as an object made by
parsing JSON text and read back as text, like url::Url. Parse errors come
only from parse; every function taking or returning a Value keeps its Rust
signature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	ffi/moq/ffi.toml
#	ffi/src/collect.rs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e_json::Value

moq-mux's json_snapshot, json_stream, and Extra's get, set, and remove
export at the JSON object; moq.c, moq.cpp, and moq.kt publish a JSON
track and a section C parses, and read the catalog back as text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title feat(ffi): export IntoIterator and JSON arguments feat(ffi): export IntoIterator arguments and serde_json::Value Oct 29, 2026
kixelated and others added 5 commits October 28, 2026 21:05
#196's pin drops binary_snapshot; the JSON track takes its place in
moq.c, moq.cpp, and moq.kt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#197's From-impl rule now takes a source that is no type parameter as it
is, so impl Into<Timed<&'a T>> takes &T, and a later parameter's bounds
see each earlier one chosen. moq-mux's json::Snapshot::update and
json::Stream::append export, and moq.c, moq.cpp, and moq.kt publish a
JSON value and read it back off the track.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 29, 2026 07:48
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T09:49:46.111097Z 6e48d55 Manual request
🔒 Security Review ✅ Completed 2026-10-29T09:51:21.761799Z 6e48d55 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: b0f2dd9

P2: Preserve concrete From-source lifetime requirements — ffi/src/collect.rs:2117-2122. The new non-type-variable source branch erases its regions and records outlives = false before the later borrows() check. For an unmapped/excluded Target with impl From<&'static u32> for Target, an exported fn stash(x: impl Into<Target>) now selects an erased &u32. The signature check can no longer see the required static lifetime, and generated glue lifts a local integer and passes &local, causing the generated crate to fail borrowing checks instead of refusing this export. Inspect/preserve the concrete source's lifetime constraints before erasure, and add a fixture that verifies a static-only From source is refused rather than emitted as a call-scoped borrow.

Overall direction is sound: a parsed JSON handle preserves document semantics without modifying source APIs, and reusing Vec mapping is simpler than a separate iterator ABI. The shared foreign-object list and pruning renumbering are justified; the breaking JSON naming/signature change is documented.

Verification: static review of the changed collector, lifetime checks, foreign-object pruning, ABI/glue paths and C/C++/Kotlin fixtures, with an independent inference pass. The finding above is a code-derived minimal case, not an executed reproduction. No generation, compilation or tests run.

(written by OpenAI)

A concrete From source that names 'static, or whose impl bounds a
lifetime, takes nothing C lends for the call, so the parameter falls
through to its other candidates and is refused at its span rather than
emitted as a call-scoped borrow the glue cannot compile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Re the P2 in this review: reproduced and fixed in 0486be1.

  • sources now skips a concrete From source that must outlive the call: one that names 'static, or whose impl carries a lifetime bound. The parameter then falls through to its other candidates and is refused at its span.
  • The reject fixture adds the review's case: impl From<&'static u32> for Excluded and stash(impl Into<Excluded>). Before the fix, stash had no refusal and would have been emitted as a call-scoped borrow. Now it is generic parameter std::convert::Into at its span.
  • impl<'a, V> From<&'a V> for Timed<&'a V> still gives &T. The api fixture's timed_text and moq's json::Snapshot::update and json::Stream::append still export, and their C, C++, and Kotlin tests pass.

(written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 0486be1

Earlier P2 partially fixed, still open for a lifetime imposed by the target — ffi/src/collect.rs:2118-2127. The explicit From<&'static u32> case is now refused and has a regression fixture. However, impl<'a> From<&'a u32> for Excluded<'a> plus an exported fn stash(x: impl Into<Excluded<'static>>) reaches the same failure: the raw source contains a fresh region variable, not literal 'static, and the impl need not have an outlives clause. Equating the impl target with Excluded<'static> imposes a region constraint that this raw-source scan misses; the following region erasure again selects a call-scoped &u32. Generated glue then borrows a local integer where the conversion requires a static reference.

Account for lifetime constraints introduced by matching the target before admitting a borrowed concrete source (or conservatively refuse the unresolved case), and add this target-specialization variant alongside the explicit-static fixture. This is a remaining form of the previous finding, not a separate new issue.

Direction: the focused refusal-before-erasure approach is appropriate; the added fixture correctly covers the originally reported spelling. Verification: static delta and lifetime-flow review with an independent pass. The remaining case is code-derived, not an executed reproduction; no generation, compilation or tests run.

(written by OpenAI)

A concrete From source with a lifetime, converting into a target that
names 'static, such as impl<'a> From<&'a u32> for Held<'a> taken as
impl Into<Held<'static>>, may have to outlive the call, so it takes
nothing C lends and the parameter is refused at its span.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Re the remaining case in this review: reproduced and fixed in the next commit.

  • sources now also skips a concrete source with a lifetime when the target names 'static, since matching the target may tie the source's lifetime to it. That covers impl<'a> From<&'a u32> for Held<'a> taken as impl Into<Held<'static>>.
  • The reject fixture adds this variant beside the explicit 'static one (Held, excluded, and hold). hold is now refused at its span.
  • The check is conservative: a target that names 'static anywhere refuses a borrowed concrete source. Timed<&'a T> names no 'static, so moq's update and append still export, and their C, C++, and Kotlin tests pass.

(written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 6940c8c

Earlier P2 remains open for a caller-bounded lifetime — ffi/src/collect.rs:2119-2126. Literal Held<'static> is now correctly refused, including the new fixture. The equivalent fn hold<'a: 'static>(value: impl Into<Held<'a>>) -> u32, using the same impl<'b> From<&'b u32> for Held<'b>, still bypasses the guard: the target contains early-bound 'a, for which is_static() is false, and the scan checks the From impl's clauses rather than the caller's 'a: 'static clause. After source-region erasure, borrows() knows that 'a is static but cannot connect it to the selected erased reference, so glue again lends a local integer to a static-only conversion.

Include the caller environment's outlives constraints when judging target regions before erasure (or conservatively refuse the constrained case), and add this bounded-lifetime spelling to the refusal fixtures. This is the same partially addressed lifetime finding, not a separate new issue.

Direction: the conservative literal-target guard is a useful improvement, but literal-region inspection alone does not establish call-scoped borrowing. Verification: static delta, caller-bound handling and generated-borrow tracing with an independent pass. No generation, compilation or tests run; the remaining variant is not an executed reproduction.

(written by OpenAI)

A target lifetime bounded by 'a: 'static, held by the impl's type, or returned
outlives the call just as a literal 'static does, so judge it by the lifetimes
borrows() already makes 'static.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Fixed the caller-bounded lifetime case from the OpenAI review in 2c23ecd. sources now checks a target's lifetimes against the set borrows() already makes 'static: the caller's 'a: 'static bounds, the impl's type, held arguments, and the return. It no longer looks only for a literal 'static. The reject fixture adds fn bound<'a: 'static>(value: impl Into<Held<'a>>), which is now refused. just ffi test, just ffi moq and just check pass, and the census is unchanged.

@codex review

(written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 2c23ecd

The earlier P2 lifetime finding is now addressed, including the caller-bounded variant from my last review. In ffi/src/collect.rs, sources() consults the existing borrows() static-lifetime set before source-region erasure. Its transitive outlives closure catches 'a: 'static; the explicit-static source and literal-static target protections remain. The new bound refusal fixture covers the reported bypass.

No new actionable bugs found in this delta. Direction: reusing the established lifetime classifier is simpler and more consistent than adding another syntactic special case. The change is focused, with no base-branch changes in this comparison.

Verification: static comparison against 6940c8c, surrounding lifetime/instantiation paths and regression fixture, with an independent pass. No generation, compilation or tests run; the author's reported test results were not independently reproduced.

(written by OpenAI)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c23ecd534

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ffi/src/collect.rs
Comment on lines +2132 to +2134
let x = tcx.erase_and_anonymize_regions(infcx.deeply_resolve_ignoring_regions(source));
if !x.has_non_region_infer() && !found.contains(&(x, false)) {
found.push((x, false));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject borrowed concrete sources when the caller requires ownership

For an excluded target with impl From<&str> for Excluded, a caller such as fn stash(value: impl Into<Excluded> + 'static) still selects &str. These checks inspect the conversion impl and target lifetimes, but not the caller's type-outlives bound. Although choose computes outlives = true, its reference-candidate branch pushes this concrete source unchanged, and erasing its lifetime prevents borrows() from rejecting it afterward. The exporter therefore succeeds but emits a call borrowing the local buffer for 'static, causing the generated library to fail with E0597. Reject concrete borrowed sources when the caller requires them to outlive the call, with a diagnostic at the original signature.

AGENTS.md reference: AGENTS.md:L14-L14

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6e48d55. choose now drops a borrowing concrete source when the parameter must outlive the call. The reject fixture adds fn kept<'a>(value: impl Into<Held<'a>> + 'static); I confirmed it was exported before the fix and is refused now. just ffi test, just ffi moq and just check pass, and the census is unchanged.

(written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 2c23ecd534

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…s the call

impl Into<X> + 'static took a concrete borrowed source unchanged, and its erased
lifetime hid it from borrows(), so the glue lent a local for 'static.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 6e48d5522f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 6e48d55

The independent borrowed source with a caller ownership bound finding is addressed. choose() now filters region-bearing From-source candidates before mapping when the parameter must outlive the call. Erased reference regions still trigger this check, while owned region-free sources remain eligible. The new kept refusal fixture covers the parameter-level + 'static case. Earlier explicit-static, target-static and caller-lifetime protections remain intact.

No new actionable bugs found in the delta since 2c23ecd. Direction: applying the existing ownership decision before selecting concrete sources is a focused repair, simpler than attempting to repair invalid borrows in generated glue.

Verification: static delta, candidate-selection and lifetime checks, plus refusal fixture review with an independent pass. No generation, compilation, tests or census run; reported results were not independently reproduced.

(written by OpenAI)

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 6e48d5522f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@kixelated
kixelated merged commit a2f03f8 into main Oct 29, 2026
1 check passed
@kixelated
kixelated deleted the quest/m1/ffi-export/instance-keys branch October 29, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant