Skip to content

refactor(ffi): infer instances in one pass before typing - #188

Merged
kixelated merged 7 commits into
mainfrom
quest/m1/ffi-export/inference
Oct 28, 2026
Merged

kixelated merged 7 commits into
mainfrom
quest/m1/ffi-export/inference

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Restructures the exporter's instance inference into one pass that runs before typing, so the cases Codex kept finding on #152 follow from the design instead of being patched one by one. It finishes the quest it opened, quest/m1/ffi-export/inference.md, and deletes it.

Consumer: moq. Its export is byte-identical to main's, and its census changes as listed below. The two round-4 cases from #152 now export, and the api fixture calls each one from C and C++.

Design

Before this PR, mapping a signature discovered instances. When ty() met an instance not yet inferred, it recorded it as pending, and naming() kept it if that was the signature's only refusal. Collection then reran from scratch. So inference saw only what typing reached:

  • A closure or callback parameter of an inferred function instance failed in typing before the instance was named.
  • A concrete implementor such as Wrap<u32> failed each's mapping check before any signature was typed.

One pass, infer, after the walk and before declare. It reads rustc's types only. No mapping outcome feeds back into it.

  • Inputs.
    • The seeds, as before: each generic type's defaults, the implementors of its parameters' bounds, and each inherent impl's concrete self type.
    • Every signature the export maps: each pub fn and const, and each trait's methods.
    • For each type (each plain one, and each instance admitted so far): its pub fields, and the inherent methods and constants that apply to it. An object's private fields are not part of its API.
  • Every choice typing could take. Each function is instantiated by the same instantiate that typing uses, with inferring set:
    • A type that names an export's type counts as mapping, since none is declared yet.
    • A parameter choose fixes takes its ranked candidates, up to the first that names no export. Typing takes a later one only when an earlier export is refused.
    • A parameter an Fn trait bounds stays a parameter. Closures are kept, not typed.
    • A parameter an exported trait bounds takes each implementor, even where typing will take a foreign callback. That includes a generic type's concrete instance that only a trait impl names (impl Seed<u64> for Wrap<u32>).
    • A parameter that one exported trait without type parameters bounds also stays a parameter, for a foreign implementation, even when no Rust type implements it.
    • An argument of an instance the signature names takes each admitted instance's argument.
  • Admission. Each type the instantiated signature names, bounds included, is admitted with the instances nested in it, or rejected with a reason: nesting, an unmet bound, a const argument, or the config's exclude.
  • Fixpoint. The pass repeats over all inputs until a round admits nothing. A new instance adds its fields and methods as inputs, but only if each of its arguments could name an instance ((), a primitive, String, or an export's type, as arg requires). declare refuses any other instance as type argument when it first declares it, before mapping any of its members. The set is sorted by path, so it does not depend on crate or item order, and the shortest-unique names stay as decided.
  • Termination. Only instances whose arguments are (), primitives, String, or export types add inputs. admit rejects an instance holding an instance of its own type at any depth. So on any path from such an instance to its leaves, each generic type appears at most once, depth is bounded by the number of generic types, and leaves come from a finite set ((), primitives, String, the non-generic exported types). Those instances are finite, and so are the ones their signatures name. The reject fixture's Grow<T>::grow -> Grow<(T,)> admits Grow<(u32,)>, which is refused as tuple, and goes no further. The reject fixture's Ping<T>::pong -> Pong<Ping<T>> and Pong<T>::ping -> Ping<Pong<T>> admit Pong<Ping<u8>>. They stop at Ping<Pong<Ping<u8>>>, which holds a Ping inside a Ping.
  • Typing. declare and define read the set and never add to it. The mapping filters (a type that does not map, a refused callback trait) only choose among the instantiations inference took. instantiate returns at its first refusal in both modes, so typing maps no candidate after an error that inference stopped at. A generic type unreachable from its crate's root is now refused as such, not as "instance no export names".
  • Agreement is a quality goal, not a safety property. Inference aims to explore everything typing looks at. Where it misses an instance, typing refuses the item at its span as instance not inferred, with a note at the generic type, and a candidate it tries and finds uninferred is rejected. A gap shows up in the census as that construct and never crashes the exporter. Today no crate in moq's census reports it. Rounds 1 to 3 of review each found such a gap, and one is now a refusal, not a panic.
  • Closure and callback arguments. An instantiation keeps the parameters that stay parameters, and signature maps them. An instance's turbofish names such a parameter _ (Instance::args is now Vec<Option<Type>>), and Rust infers it from the argument.
  • Bounds at an instance. holds used to check every clause with chosen arguments substituted, against the item's identity environment. That environment states each bound only at the item's own parameters. So a clause naming another parameter, such as F: Fn(T) at T = u32 or P: IntoRendition<E, _> at E = Extra, could never hold.
    • holds now skips clauses that still name a parameter, as applies already did, and the final check runs on every instantiation.
    • When none holds, the refusal names the failing bound (unmet bound ...), not "generic parameters no choice meets the bounds of".
    • This is also the root cause the instance-keys quest asked about: moq-mux's json_snapshot<T: Serialize> now takes String. moq's config still excludes it, and the quest's text says so.

Reuse. rustc_monomorphize::collector is pub(crate). It roots at non-generic items and walks MIR bodies for the concrete uses they make, bounded by the recursion limit. It cannot choose arguments for a public generic item that no body uses, which is the whole problem here. The pass keeps reusing rustc's trait solver for every bound, and trait_impls_of for implementors.

Choice made, for review

The instance set now depends only on the signatures, not on what maps. So an instance named only by a function refused for another reason still infers, and then exports or is refused itself. Fixing a mapping gap later never changes the instance set or any instance's name. Removing the typing feedback that would avoid this was the point of the redesign. Effect on moq's census (the export is unchanged):

crate exported types refused types methods and fields why
hang 68 → 72 8 → 8 256 → 266 Report/Snapshot<String> instances, named by publisher/viewer, which Health refuses
kio 4 → 4 20 → 24 10 → 10 Pending<…> instances, refused as objects nothing returns
moq-mux 174 → 177 25 → 22 1054 → 1075 json::{Snapshot,Stream,Consumer}<String>, from json_snapshot/json_stream at String
moq-net 121 → 122 35 → 36 525 → 526 Timed instances
moq-tokio 60 → 60 34 → 36 unchanged failover::Attempt<…> instances, refused as objects nothing returns

Changes

  • ffi/src/collect.rs:
    • infer runs the fixpoint, through named/instantiated.
    • members, shown, spelled, and unmet are split out of methods, functions, each, and holds, so the pass and typing share them.
    • lent, maps, and names_export are new.
    • Instantiation::kept replaces impls, and signature maps the kept parameters.
    • Deleted: pending, named, naming(), UNNAMED and its note, and collection's inner rerun loop.
  • ffi/src/ir.rs, ffi/src/glue.rs: a kept declared parameter is None, spelled _.
  • Fixtures:
    • api adds apply<T, F: Fn(T) -> u32>(Pair<T>, F), report<T, L: Log>(Pair<T>, L), and Wrap<T> with only impl Seed<u64> for Wrap<u32>, each called from C, C++, and the native transcript.
    • api adds keeps(impl Log) -> Boxed<u8>, a callback no Rust type implements on a signature naming an instance nothing else names.
    • reject adds Ping/Pong, Grow, Shell<(u32,)>, whose field names an instance nothing else does, and probe<const N: usize>(impl Into<Probe<u8>>), refused at its const parameter before its other candidate is mapped.
  • ffi/README.md: the inference paragraph. quest/m1/ffi-export/instance-keys.md: the T: Serialize finding.

Net lines (merge base to head): ffi/src +388 −240. Everything, fixtures and tests included: +543 −261.

Public API: additions only. New C/C++ functions appear where a crate has these shapes (the api fixture's api_apply_u32, api_report_u32, api_wrap_u32, and *_wrap_u32 seed instances). Existing names in the api fixture and moq are unchanged. Instance::args is crate-internal. No MIGRATE entry.

Checks

  • just check, just ffi test (17 tests), and just ffi moq (C and C++) pass. just check reran on the merged head; main touched only ts/.
  • Reproduced first: on main the api fixture refused apply (type parameter T), report<u32> (type parameter L), and Wrap (generic type).
  • moq's generated export (headers and glue) is byte-identical to main's. The reject fixture differs by the new Ping/Pong, Grow, Shell, and Probe refusals and the line shifts they cause. One existing message changed: Cell<Vec<String>> is now refused as type argument, like every instance refused for its arguments.
  • Review round 1 (Codex, on 0fb182d): two P1 findings, both reproduced and then fixed in e199039 (unbounded growth through tuples, and callbacks with no Rust implementor).
  • Review round 2 (Codex, on e199039): one P2 finding, a record instance at an unarguable argument settled before it was refused. Reproduced, then fixed in 29d81a1.
  • Review round 3 (Codex, on 29d81a1): one P2 finding, instantiate in typing mode mapped a candidate after a const refusal and hit the unreachable!. Reproduced. The maintainer chose to keep the pass, replace the panic with the instance not inferred refusal, and stop instantiate at its first refusal. Fixed in b53d8ea's parent.
  • Review round 4 (Codex, on b53d8ea): clean, and its security review found nothing. CI Check passed on b53d8ea.

Follow-ups

  • moq-mux json_snapshot/json_stream now choose String. The JSON shape stays with the instance-keys quest.

🤖 Generated with Claude Code

kixelated and others added 3 commits October 28, 2026 11:39
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Inference walks every signature the export maps, at every choice its bounds
allow, before any type is declared, so mapping a signature never discovers an
instance. A function inferred at an instance keeps its lent closure and foreign
callback parameters, and a concrete instance only a trait impl names, such as
Wrap<u32>: Seed<u64>, is inferred. Drops the pending/named bookkeeping and the
rerun of collection per inferred round.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 28, 2026 18:54
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T22:06:01.390109Z b53d8ea Manual request
🔒 Security Review ✅ Completed 2026-10-28T22:07:56.867392Z b53d8ea Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0fb182d0b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ffi/src/collect.rs Resolved
Comment thread ffi/src/collect.rs Resolved

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 0fb182d

Two actionable P1 issues are already independently reported; I agree with them and am referencing rather than duplicating inline comments:

  • Bound non-ADT instance growth, ffi/src/collect.rs:436–438: a seeded Grow returning Grow<(T,)> keeps admitting deeper tuples. The ADT-repeat guard does not bound tuple/array/reference nesting, and inference now expands even instances typing would refuse. Add a terminating structural admission rule and a regression for these constructors.
  • Preserve callback-only inference, ffi/src/collect.rs:502: before declarations, a callback trait with no Rust implementors makes instantiate fail and the signature disappear. Typing later accepts that callback and can panic on its newly encountered return instance. Infer the kept callback alternative independently of the declaration map and cover a callback with no Rust implementors.

Direction: separating inference from mapping is a useful simplification, and sharing instantiation/bound checks avoids duplicated choice rules. The remaining requirement is that this earlier pass be both terminating and a conservative superset of typing; the new closure/implementor and Ping/Pong fixtures do not cover the two cases above. No additional actionable findings.

Verification: static inspection of the collector, glue/IR changes and fixtures only. I did not run tests or independently verify the reported moq export comparison.

…oices while inferring

Inference walks only the members of an instance whose arguments could map,
so a method wrapping its argument in a tuple no longer grows instances
forever. Before traits are declared, a parameter one exported trait bounds
may stay for a foreign implementation, so a signature taking a callback no
Rust type implements still names its instances.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e199039a6e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ffi/src/collect.rs Resolved

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: e199039
Compared with my last review at 0fb182d (one fix commit; unchanged base).

P1 still open, incomplete fix: recursively check exported argument instances before expanding members. In ffi/src/collect.rs:503–507, arguable accepts any exported ADT without checking its arguments. With exported Wrap<T> and Grow<T>, a seed Grow<Wrap<u32>>, and impl<T> Grow<Wrap<T>> { pub fn grow(self) -> Grow<Wrap<(T,)>> { todo!() } }, inference still admits Grow<Wrap<(u32,)>>, then Grow<Wrap<((u32,),)>>, indefinitely. Every outer argument is an exported Wrap, so it passes this filter, and no individual type contains a repeated ADT for admit to reject. Although Wrap<(u32,)> would eventually be refused, typing is never reached. This is a remaining case of the earlier growth finding, not a separate new bug. Recursively establish argument eligibility (including referenced instances' arguments) and add this wrapped-tuple regression alongside direct Grow<(T,)>.

The callback-only inference panic is addressed: structural foreign recognition preserves the parameter during inference, while taken retains declaration validation during typing. The new keeps fixture exercises the previously missing return instance through C/C++ and the native transcript.

Direction: retaining a separate inference pass and narrowing member expansion is sensible, but the termination guard must follow nested argument dependencies, not just the outer constructor. No other actionable findings in this update.

Verification: static inspection of the fix and regression changes only; I did not run tests or execute the wrapped-tuple reproduction.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: e199039a6e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…apping its fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 29d81a1
Compared with my last review at e199039: one fix commit, unchanged base.

  • Fixed: the independently reported field-settling panic. ffi/src/collect.rs:770–774 now marks unsupported argument instances refused before settle visits fields, and the new Shell<(u32,)>/Core<u8> reject fixture covers this case.
  • Still open, not new: the wrapped-tuple inference-growth P1 from my previous review. The renamed unarguable still checks only the outer ADT (ffi/src/collect.rs:507–511), so the exported Wrap case remains unbounded. Recursively validate argument eligibility and add that regression. The callback-only panic remains addressed.

Direction: sharing the early eligibility check between inference and declaration is a focused fix with little extra complexity. No new actionable bugs found in this delta; the existing termination issue remains.

Verification: static inspection of the collector delta, surrounding declaration/settling logic and reject fixture/expectations only. I did not run tests or execute a reproduction.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 29d81a110e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ffi/src/collect.rs Resolved
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 29d81a110e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

kixelated and others added 2 commits October 28, 2026 15:03
… at the first refusal

An instance typing meets that inference did not admit is now refused at the
item's span as `instance not inferred`, with a note at the generic type,
so a gap in inference shows in the census instead of crashing the exporter.
A candidate typing tries and finds uninferred is rejected. `instantiate`
returns at its first refusal, as inference does, so it maps no later
parameter's candidates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: b53d8ea
Compared with my last review at 29d81a1, excluding the merged main changes (base now a4fb96f).

Fixed: the independently reported candidate-selection panic after a refused instantiation. ffi/src/collect.rs:1664–1665,1684–1685 now stops at the first refusal; 2478–2485 turns a missed instance into a diagnostic instead of panicking. The new const-generic probe reject fixture covers the reported ordering.

Still open, not new: the wrapped-tuple inference-growth P1. unarguable at ffi/src/collect.rs:507–511 remains shallow, so exported Wrap arguments can still hide unbounded tuple growth. Recursively validate those argument dependencies and cover the wrapped case. Earlier callback-only and field-settling panic fixes remain intact.

Direction: fail-fast instantiation is simpler than accumulating errors while continuing speculative mapping, and a source-spanned refusal is a reasonable fallback for missed inference. No new actionable bugs found in this delta; the existing termination issue remains.

Verification: static inspection of the FFI delta, surrounding selection/refusal paths and fixture expectations only. I did not run tests or reproductions; merged TypeScript changes were not re-reviewed.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: b53d8ea83f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: b53d8ea83f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merging at b53d8ea, which Codex reviewed clean in round 4, with a direct squash merge (the land queue is not set up yet).

  • Checks: Check passed on b53d8ea. Locally, just check, just ffi test, and just ffi moq passed, and moq's generated export is byte-identical to main's.
  • Reviews: rounds 1 to 3 found 4 gaps, each reproduced and fixed. Round 3's led to the maintainer's choice that an instance inference misses is refused as instance not inferred rather than crashing the exporter.

(Written by Claude Opus 5.5)

@kixelated
kixelated merged commit 3b2230b into main Oct 28, 2026
1 check passed
@kixelated
kixelated deleted the quest/m1/ffi-export/inference branch October 28, 2026 22:29
kixelated added a commit that referenced this pull request Oct 28, 2026
#184 and #188 merged apart: #188 added apply, report, wrapped, and keeps to
the api fixture's native transcript and its C and C++ tests, while #184's
Kotlin test predates them, so `just ffi test` fails on main.

Co-authored-by: Luke Curley <432854+kixelated@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant