Skip to content

feat(ffi): export Url, SocketAddr, paths, and Vec<T> across the C ABI - #190

Merged
kixelated merged 15 commits into
mainfrom
quest/m1/ffi-export/foreign-types
Oct 28, 2026
Merged

kixelated merged 15 commits into
mainfrom
quest/m1/ffi-export/foreign-types

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Claims quest/m1/ffi-export/foreign-types.md and finishes it.

Problem

moq-tokio's session config and calls take url::Url, SocketAddr, PathBuf, and Vec<String>, which the exporter refused, so C could not build a session config (connect::Config::url, tls.fingerprint, tls.generate, TLS files, Server::local_addr).

Approach

General rules, no config:

Rust C C++ Kotlin
url::Url object <prefix>_url, made by <prefix>_url_parse(buf, &out, &err), which fails with the parse error's message; _to_string, _eq, _clone, _debug class url, url::parse(view) returns result<url, buf> class Url, Url.parse(String) throws StringException
SocketAddr <prefix>_socket_addr { uint8_t ip[16]; uint16_t port; bool v6; uint32_t flowinfo, scope_id; } socket_addr, the C struct java.net.InetSocketAddress
&Path, PathBuf <prefix>_buf of the OS's bytes on Unix (any bytes), UTF-8 elsewhere view / buf String
Vec<T>, any mapped T but u8 <prefix>_vec_<T> { const T *ptr; size_t len; void *owner; }: a view passed in, an owned array returned, freed by <prefix>_vec_<T>_free with its first len elements; null ptr is None std::vector<T>, of views for bytes passed in List<T>
  • Url is an object because parsing must fail as an error, not an abort: Rust code parses with Url::parse(s)? too, so the error surfaces where Rust surfaces it. It is synthesized as an ordinary IR object with one parse method (Access::Parse, FromStr with the error's Display), so every generator (C, C++, and Kotlin once it lands) gets it with no code of its own. It is dropped when no export names it. The generated crate depends on url = "2", which cargo unifies with the exports' as it does bytes.
  • SocketAddr is a value: no parse, no error path, natural for sockaddr-minded C and for InetSocketAddress. An IPv4 address with stray IPv6 bytes aborts, as other malformed values do.
  • A Vec's shape is shared with what instance keys needs (Vec<Range> now maps). Elements follow the value rules: a Vec<&T> maps only when returned (cloned), and non-Copy handles only where an Option of them would. A binding that takes the elements frees the array alone by setting len to 0, which C++'s std::vector lift uses for zero-copy bufs.
  • C++ lowers a std::vector into an arena that lives for the call, so a vector inside a record or another vector works too.
  • Kotlin (#184 landed first, so this PR adds it): a List crosses JNI encoded as its length and elements, and the JNI reader keeps each array it reads until the call returns. A path Rust returns that is not UTF-8 throws rather than decoding lossily; an InetSocketAddress must be resolved, and one Rust returns with an IPv6 flow label, which Java does not keep, throws. Passing a List of objects by value closes each, as passing one does.

Choices made without asking: Url as an object rather than text; SocketAddr as a struct rather than text, and InetSocketAddress in Kotlin; paths as Kotlin Strings rather than ByteArrays; paths as UTF-8 off Unix (Windows returns WTF-8 via as_encoded_bytes, and only UTF-8 is taken in); <prefix>_vec_<T>_free by value, like <prefix>_buf_free.

Proven by the api fixture's transcript from C, C++ at 17/20/23, and Kotlin (URL parse ok and error, v4 and v6 addresses, paths, Vec<String>, Option<Vec<String>> with Some(empty) vs None, Vec<Vec<u8>>, Vec<Vec<u32>>, Vec<Counter> returned and Option<Vec<Counter>> moved in, Vec<Option<PathBuf>>, a record holding a Vec, a returned Vec<&str>), a non-UTF-8 path round trip from C and its refusal in Kotlin, two new abort cases, a reject case, and moq/moq.c, moq.cpp, and moq.kt against moq, which set connect.url, tls.fingerprint, and listen.bind through #189's setters and read them back, closing field-setters' goal end to end (a Url through auth::Tokens::dial, Versions::alpns, listen::Bind's address, TLS root and fingerprint).

Census of moq's exports (types, functions, methods and fields: exported/refused), against main at 9d94118 with #189's setters:

Before After
Types 448 / 131 461 / 118
Functions 36 / 20 36 / 20
Methods and fields 2157 / 245 2259 / 183

moq-tokio alone: types 60 → 68, methods and fields 221 → 279. moq/ffi.toml drops the 79 excludes these unblock (among them connect::Config::url, bind, tls::Connect::fingerprint, tls::Listen::generate, Server::local_addr), and renames moq-mux's catalog::Consumer to moq_mux_catalog_consumer, since it now clashes with hang's.

Impact

  • New C names: <prefix>_socket_addr; <prefix>_url and its _parse, _to_string, _eq, _clone, _debug, _free where an export names Url; <prefix>_vec_<T> and _free per element type.
  • Breaking (MIGRATE.md): a struct or enum refused as a value only for these types is now a value, not a handle (moq-tokio's listen::Bind); the new names can clash with an export's item.
  • Every header's contract changes, since each declares <prefix>_socket_addr.
  • Kotlin: these types now export as above; nothing existing changes.

Alternatives

  • Url and SocketAddr as text parsed at each call: a parse error would have no home in a function that is not fallible, or one whose error type is not a string, short of an abort.
  • &[T] as a view of the same array: left out, see follow-ups.

Follow-ups

  • The port-0 bind test moves to session-tokio: listen.bind now sets, but Server::new needs tokio, which waits on moq's fallback runtime.
  • Slices of any type, and SocketAddrV4/V6, added to the ffi-export epic here: &[T] of non-byte types (12 refusals, e.g. connect::Addr::addresses, Pattern::segments) as the same array, and SocketAddrV4/V6 (moq-tokio's listen::Config::preferred_v4/_v6) sharing <prefix>_socket_addr.

🤖 Generated with Claude Code

kixelated and others added 2 commits October 28, 2026 14:49
`url::Url` exports as an object parsed from text, failing with the parse
error's message; `SocketAddr` as `<prefix>_socket_addr`; `&Path` and
`PathBuf` as a buffer of the OS's bytes; and `Vec<T>` of any other mapped
`T` as `<prefix>_vec_<T>`, an array C frees with what it holds. C++ gets
`std::vector`, lowered through an arena that lives for the call.

moq's config drops the excludes these unblock and renames moq-mux's
`catalog::Consumer`, which now clashes with hang's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Session config's foreign types feat(ffi): export Url, SocketAddr, paths, and Vec<T> across the C ABI Oct 28, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 28, 2026 21:53
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T23:37:35.629416Z ef674d4 New commits
🔒 Security Review ✅ Completed 2026-10-28T23:39:40.566338Z ef674d4 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: 30642d5

Two actionable issues:

  • P2 — Free the owned clones in returned vectors of references (ffi/src/glue.rs:359–364, list registration at ffi/src/abi.rs:643–645). The list retains its Ref element type, and lower clones that referent, but free(Ref(...)) falls through without releasing anything unless it is a buffer. For example, returning Vec<&Peer> where Peer is the new Clone record containing Vec<String> allocates owned cloned fields, yet the generated vector free drops only the outer array. Normalize returned element ownership or recurse through Ref when freeing; add a C-side allocation/drop regression for a borrowed record/object element, not just Vec<&str>.

  • P2 — Reject references nested inside vector elements too (ffi/src/collect.rs:2352–2358). The check only rejects a directly referenced element, while lends_inside does not descend into List. A parameter such as Vec<Option<&str>> therefore gets accepted, but glue lifts its elements to Option<String> and passes Vec<Option<String>> to the Rust function expecting borrowed strings, making the generated crate fail to compile. Recursively reject these parameter references (including Option/Box inside a Vec), with a reject fixture.

Direction: a fallible URL object and a shared array ABI are sensible; the call-scoped C++ arena supports nested values without special-casing session configs. Ownership normalization should be shared rather than adding per-generator exceptions.

Verification: static review of collection, ABI, Rust/C/C++ generation, runtime ownership, and added fixtures. No builds, tests, census, or runtime probes executed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 30642d5c5c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ffi/src/collect.rs Resolved
Comment thread ffi/src/glue.rs Resolved
Comment thread ffi/src/cpp.rs Resolved
A `Url` field makes its type an object, as any handle C cannot copy does,
so reading the value no longer consumes it. A returned `Vec<&T>`'s free
drops the clones it holds. A parameter's `Vec` refuses a reference nested
in its elements, and C++ takes an optional `Vec` of handles by value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Both findings of the local review are fixed in e9e929d: returned Vec<&T> elements are normalized to their owned form, so the array's free drops the clones (a C and C++ check counts two Token drops), and a parameter's Vec refuses a reference nested anywhere in its elements (Vec<Option<&str>> is in the reject fixture).

(written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: e9e929d

Reviewed the single-commit update since 30642d5; base unchanged. No new actionable bugs found in this delta.

Both findings from my previous review are fixed in code: list registration now normalizes borrowed elements to owned types before generating cleanup, and recursive borrows rejects nested parameter references. The added C/C++ Token drop-count checks and Vec<Option<&str>> reject fixture target those regressions.

The independent URL-field finding and optional-vector ownership finding are also addressed by the non-Copy field restriction and move-aware optional parameter. Direction is sound: reuse the existing ownership rules rather than special-case generated destructors.

Verification: static comparison and inspection of the fixes and regression fixtures only; no builds, tests, census, or runtime probes executed.

kixelated and others added 3 commits October 28, 2026 15:47
…oreign-types

# Conflicts:
#	ffi/fixtures/reject/expected.txt
#	ffi/src/abi.rs
…n export

A path is a String, which throws for one that is not UTF-8; a SocketAddr is an
InetSocketAddress, which throws for an IPv6 flow label Java does not keep; a
Vec<T> is a List, whose objects close once a call moves them in. The JNI reader
keeps each array it reads until the call returns. The api transcript uses a
UTF-8 path, which Kotlin can express, with the non-UTF-8 round trip checked
from C and Kotlin's refusal of one as a broken case; moq.kt mirrors moq.c's
foreign-type checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: 58d657d

Reviewed the Kotlin/JNI foreign-type integration after accounting for the merged main changes. The previously reported borrowed-vector cleanup and nested-reference rejection fixes remain present.

P2 — Close consumed elements when the vector is boxed (ffi/src/kotlin.rs:493–498). The new cleanup gate accepts only top-level List or Option, so an exported parameter Box<Vec<Counter>> skips cleanup even though moved correctly recurses through Box and Rust consumes each Counter. Kotlin exposes that parameter as a List; after the call its elements remain marked open, retain their native handle allocations, and a subsequent method call reaches Rust's moved-out-handle abort instead of Kotlin's closed-object check. Apply the ownership decision through Box as well (the existing close helper already handles it), and add a boxed-vector parameter regression alongside worth(Option<Vec<Counter>>). This is a new Kotlin integration gap, not a recurrence of the prior C/C++ findings.

Direction: keeping JNI array storage alive for the call and explicitly rejecting unrepresentable paths/flow labels is sensible. Reusing moved uniformly is simpler than an additional outer-type whitelist.

Verification: static inspection of the integration delta, codec lifetime/ownership paths, fixtures, and retained fixes; no builds, tests, census, or runtime probes executed.

Gate the close on `moved` alone, which already recurses through Box, so a
Box<Vec<T>> parameter closes each object as Option<Vec<T>> does. The api
fixture's `worth` takes Option<Box<Vec<Counter>>>, and Kotlin checks a counter
it moved in is closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

The local review's P2 on 58d657d is fixed in b368920: the Kotlin close is gated on moved alone, which recurses through Box, and the api fixture's worth now takes Option<Box<Vec<Counter>>>, with Kotlin checking a counter it moved in is closed.

(written by Claude Opus 5.5)

kixelated and others added 2 commits October 28, 2026 16:02
The follow-ups #190 leaves: `&[T]` of any mapped type (12 refusals at the
pinned moq) and the typed socket addresses moq-tokio's listen config holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: b368920

The boxed-vector cleanup finding from my previous review is fixed: ffi/src/kotlin.rs now gates cleanup on moved alone, and both moved and close recurse through Box. Reusing those ownership helpers removes the redundant outer-type whitelist. No new actionable bugs found in this delta; the other changes since the previous review are imported main fixture coverage.

The updated worth fixture takes Option<Box<Vec>> and checks that a consumed element is closed. A direct Box<Vec> fixture would more precisely lock in the original outer-Box regression, because the old whitelist already accepted an outer Option.

Verification: static comparison and inspection of the cleanup path and fixtures only. No builds, tests, census, or runtime probes executed.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: 025c9ca

No actionable bugs found in the new slices/SocketAddrV4/V6 follow-up plan. Reusing the vector ABI, copying returned slices into owned storage, and checking address-family mismatches are sensible extensions without a second representation. Cross-language transcript and census checks are appropriate acceptance criteria; this change plans the work rather than implementing it.

The remaining delta since b368920 imports main's moq-pin plan. No implementation changed, so the prior boxed-vector cleanup fix remains intact.

Verification: static comparison and plan inspection only. No builds, tests, census, or runtime probes executed.

`worth` takes an Option<Vec<Counter>> and a Box<Vec<Counter>> apart, so the
fixture covers a boxed Vec without an Option around it, which the Kotlin close
missed before b368920.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

The local review's note on b368920 is addressed in 555f0a0: worth now takes Option<Vec<Counter>> and a bare Box<Vec<Counter>> apart, so the fixture covers the outer-Box case the old gate missed, and Kotlin checks each counter it moved in is closed.

(written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: 9425dde

The regression-coverage gap noted earlier is addressed: worth now takes separate Option<Vec> and direct Box<Vec> parameters, and Kotlin checks both consumed elements are closed. This exercises the original outer-Box cleanup failure while retaining the optional-vector case. The C, C++, and native transcript calls preserve the same total and ownership cleanup.

No new actionable bugs found. This is a focused test improvement with no added implementation complexity; the remaining delta imports main's quest plans. The previously fixed cleanup finding remains fixed.

Verification: static comparison and cross-language fixture inspection only. Tests, builds, and census were not run.

kixelated and others added 2 commits October 28, 2026 16:35
…oreign-types

# Conflicts:
#	ffi/MIGRATE.md
#	ffi/README.md
#	ffi/fixtures/reject/expected.txt
#	ffi/src/collect.rs
…nd Kotlin

With #189's setters and the foreign types mapped, `connect.url`,
`tls.fingerprint`, and `listen.bind` generate setters. moq's tests set each and
read it back, which closes field-setters' goal end to end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

Merged main with #189's setters. connect.url, tls.fingerprint, and listen.bind now generate setters; moq.c, moq.cpp, and moq.kt set each and read it back.

(written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: ef674d43fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: ef674d4

No new actionable bugs found in the setter integration and added session-config tests, after separating the merged main changes from this PR's update. The C, C++, and Kotlin cases now set and read back the URL, fingerprint list, and bind address through the shared generated accessors. Their ownership handling matches the existing moved-input/cloned-getter contract; Kotlin also reads its URL after closing the source config.

Direction: testing the composition of setters and foreign-type mapping is useful without adding session-specific generator logic. The earlier boxed-vector cleanup fix and direct-Box regression remain unchanged, not new findings. This does not yet test opening a live session or a port-0 listener.

Verification: static comparison, inspection of accessor collection/glue and cross-language test changes only. No builds, tests, census, or network/runtime probes executed.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: ef674d43fa

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant