Skip to content

feat(ts): translate size_hint and reserve by it in collect and extend - #294

Merged
kixelated merged 7 commits into
mainfrom
quest/m3/ts-size-hints
Oct 30, 2026
Merged

kixelated merged 7 commits into
mainfrom
quest/m3/ts-size-hints

Conversation

@kixelated

@kixelated kixelated commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Finishes quest/m3/ts-size-hints.md.

Iterator::size_hint of any std iterator refused at its span ("unsupported call to <... as Iterator>::size_hint"; 194 refusals in the new fixture on the base). And no collection reserved by the hint, so where std's collect or extend panics with capacity overflow before reading an item, the translation read items instead (1,692 of the new reserve cases differed from native on the base).

Approach

  • Hints. feat(ts): translate collect::<String>, String's Extend, and String::drain #284's rt.Iter.lower() becomes sizeHint(): [lower, upper | undefined], std's for every runtime iterator: chain and flatten add with a checked upper that overflows to None, zip takes the shorter side's, flatten counts [T; N] items and specializes on std's OneShot iterators (Once, Option, [T; 0 | 1]), ranges are exact past 2^53 and usize::MAX, and the rest (Peekable, Fuse, StepBy, TakeWhile, the ? shunt, Chars, B-tree ranges, ...) as std gives them. size_hint maps to it.
  • Reserves. Each collect and extend passes the element size and whether the iterator is TrustedLen:
    • Vec, VecDeque, and BinaryHeap reserve the upper bound up front for a TrustedLen iterator (None panics), else the lower bound plus one at the first item and whenever full, growing as RawVec does. B-tree collects go through the same Vec.
    • HashMap and HashSet reserve the lower bound (half of it once non-empty) against hashbrown's largest table for that layout, computed by the translator, and panic with Hash table capacity overflow.
    • A Result or Option collect is never trusted, as std's GenericShunt isn't.
  • Capacity isn't modeled (m2/ts-model/std-rest), so a collection an extend starts from is taken as full. That matches std after Vec::new, vec!, or a TrustedLen collect. The quest now notes where it doesn't.
  • A crate's own iterator that overrides size_hint stays refused with its span (reject fixture, fixtures/reject/src/lib.rs:203).

Proven by hints.rs in the lang fixture, differential against native wasm32:

  • hints::hints (24,576 cases): every adaptor's lower and upper bound before and after reading from either end, at lengths 0, 1, 5, isize::MAX and past it, usize::MAX and past it, and 2^60.
  • hints::reserves (6,500 cases): 13 collect/extend forms across 5 element layouts, one past each collection's threshold, at u32::MAX, and with an unbounded hint. Each collection reaches its overflow panic, and none allocates natively.

Mutation checks against the generated runtime are each caught: Chain's checked upper (1,078 differences), RawVec's doubling (93), the + 1 on the lower bound (279), hashbrown's half reserve (50), and the minimum non-zero capacity (48).

The collect loop measures the same or faster than the base runtime in Node (ns/item, median of 15): filter to Vec<u8> 6.41 vs 6.45, filter to Vec<u32> 11.63 vs 10.48, map to Vec<u32> 12.66 vs 8.20.

Impact

  • Public API: the translator's output only. rt.Iter.lower() is replaced by sizeHint(). rt.iterToArray, rt.iterToByteVec, rt.iterExtend, rt.iterExtendBytes, rt.dequeExtend, rt.heapExtend, rt.btreeExtend, rt.btreeExtendKeys, rt.hashExtend, rt.hashExtendKeys, rt.iterFlatMap, and rt.iterFlatten take trailing size or trust arguments. The runtime ships with each translation, so there is no MIGRATE.md entry.
  • Consumer: moq's collects and extends now panic where std's do, and any size_hint it reaches translates.

Alternatives

Follow-ups

  • m2/ts-model/std-rest: replace rt.vecCapacity's "full" with the real capacity.
  • New quests:
    • ts-zero-sized: a function returning a unit struct or empty array emits return _0 that tsc rejects, and from_fn of a capture-less closure refuses. Both kept zero-sized elements and [T; 0] flattens out of the fixture.
    • ts-array-keys: collections of [u32; N] keys fail tsc.
    • ts-zip-back: Zip::next_back of random-access sides trims item by item, where std's is O(1).
  • An overflow just at each threshold can't be tested without allocating natively, so thresholds are proven from above only. The mutation checks cover the arithmetic.

Decision

An extend after truncate, clear, pop, drain, with_capacity, or reserve. Decided by the orchestrator on 2026-10-30, pending maintainer review: keep "taken as full" (option a). Exact capacity, which fixes this, is already planned in m2/ts-model/std-rest, and this PR records the gap in that quest's plan.

The runtime keeps no capacity, so these extends diverge from std. The divergence only shows for iterators whose lower bound is past isize::MAX / size_of::<T>(), where std uses its spare room first. This PR removes 1,692 existing divergences and leaves only this one. The rejected options were modeling capacity here (std-rest's whole goal) and reserving only for collects and TrustedLen extends (which drops std's panic for a full collection).

Review

Local quest review (codex gpt-6.1-sol, medium), 2 rounds, last on head 7f22c32969a0c1e782c22459d2d8b54dd09ba276:

  • Round 1 on 355fa842: one finding (medium, ts/runtime/rt/iter.ts:417). After v.truncate(1), extend treats the Vec as full and panics with capacity overflow on an iterator whose lower bound is past isize::MAX. std's uses the spare capacity first, so its closure runs once more and panics with consumed. I refuted it as out of scope: the brief leaves Vec capacity to m2/ts-model/std-rest, and this PR records the gap in that quest's plan.
  • Round 2 on 7f22c329: the reviewer kept the finding. The orchestrator settled it as recorded under Decision; no other findings.

just ci passed on the reviewed head 7f22c32969a0c1e782c22459d2d8b54dd09ba276.

Merge of main at 1d3a8e0f (#297, #299) into 7f22c329, as 46124952: ordered main's iters::peeked_others before hints::* in harness.rs and caller.ts, took main's generated snapshots and regenerated them (this PR's new instances only). Recorded skip, too trivial to review; just ci passed on 46124952.

🤖 Generated with Claude Code

(written by Claude Opus 5.5)

kixelated and others added 5 commits October 30, 2026 10:26
Quest-Claim: 7e3523738fe306f9
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	ts/fixtures/lang/caller.ts
#	ts/fixtures/lang/expected/harness.ts
#	ts/fixtures/lang/src/harness.rs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Size hints of std's iterators feat(ts): translate size_hint and reserve by it in collect and extend Oct 30, 2026
@kixelated
kixelated marked this pull request as ready for review October 30, 2026 19:41
# Conflicts:
#	ts/fixtures/lang/caller.ts
#	ts/fixtures/lang/expected/core/option.ts
#	ts/fixtures/lang/expected/harness.ts
#	ts/fixtures/lang/src/harness.rs

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 7f22c32

  • P2, existing finding still present: ts/runtime/rt/iter.ts:416–417,439–441 treats every non-ZST destination as full. This confirms the local-review finding already described in the PR’s Review/Decision sections, rather than a new independent issue: after truncate/pop or a reservation leaves spare capacity, extending with a non-TrustedLen iterator whose lower bound exceeds the layout limit can panic on the first yielded item, whereas Rust uses the spare slots and can run further iterator side effects (including the documented consumed panic). Recording this in std-rest does not remove the observable divergence. Preserve actual capacity for this decision, or explicitly retain this correctness limitation for maintainer acceptance and add its differential regression alongside the full-destination cases.

No additional actionable bugs found in the reviewed hint propagation, reserve plumbing, or new fixtures. Overall, one sizeHint() API and shared growth helpers are a sensible direction; the differential matrix is valuable, but the stated capacity approximation remains a qualification on parity.

Verification: static review only. I did not run builds, tests, mutation checks, or benchmarks; the PR’s reported results were not independently verified.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: cf1bfa39-8400-4136-904a-4d79faeccd6c
Commits

Reviewing files that changed from the base of the PR and between f7f0e72 and 7f22c32.

Files selected for processing: 60
  • quest/m2/ts-model/std-rest.md
  • quest/m3/README.md
  • quest/m3/ts-array-keys.md
  • quest/m3/ts-size-hints.md
  • quest/m3/ts-zero-sized.md
  • quest/m3/ts-zip-back.md
  • ts/README.md
  • ts/fixtures/codec/expected/bench.ts
  • ts/fixtures/crates/expected/harness.ts
  • ts/fixtures/lang/caller.ts
  • ts/fixtures/lang/expected/bounds.ts
  • ts/fixtures/lang/expected/chars.ts
  • ts/fixtures/lang/expected/chunks.ts
  • ts/fixtures/lang/expected/collections.ts
  • ts/fixtures/lang/expected/core/option.ts
  • ts/fixtures/lang/expected/core/result.ts
  • ts/fixtures/lang/expected/counts.ArcNode.ts
  • ts/fixtures/lang/expected/counts.RcNode.ts
  • ts/fixtures/lang/expected/counts.ts
  • ts/fixtures/lang/expected/defaults.ts
  • ts/fixtures/lang/expected/durations.ts
  • ts/fixtures/lang/expected/dyns.ts
  • ts/fixtures/lang/expected/edits.ts
  • ts/fixtures/lang/expected/errors.ts
  • ts/fixtures/lang/expected/harness.ts
  • ts/fixtures/lang/expected/hashes.ts
  • ts/fixtures/lang/expected/hints.Big.ts
  • ts/fixtures/lang/expected/hints.Countdown.ts
  • ts/fixtures/lang/expected/hints.Endless.ts
  • ts/fixtures/lang/expected/hints.Most.ts
  • ts/fixtures/lang/expected/hints.ts
  • ts/fixtures/lang/expected/holds.ts
  • ts/fixtures/lang/expected/ints.ts
  • ts/fixtures/lang/expected/iters.ts
  • ts/fixtures/lang/expected/keys.ts
  • ts/fixtures/lang/expected/lib.ts
  • ts/fixtures/lang/expected/locals.ts
  • ts/fixtures/lang/expected/nested.ts
  • ts/fixtures/lang/expected/objects.ts
  • ts/fixtures/lang/expected/orders.ts
  • ts/fixtures/lang/expected/pointers.ts
  • ts/fixtures/lang/expected/refs.ts
  • ts/fixtures/lang/expected/shapes.ts
  • ts/fixtures/lang/expected/slices.ts
  • ts/fixtures/lang/expected/statics.ts
  • ts/fixtures/lang/expected/strs.ts
  • ts/fixtures/lang/expected/utf8.ts
  • ts/fixtures/lang/expected/wrappers.ts
  • ts/fixtures/lang/src/harness.rs
  • ts/fixtures/lang/src/hints.rs
  • ts/fixtures/lang/src/lib.rs
  • ts/fixtures/task/expected/drops.ts
  • ts/fixtures/task/expected/drops/closures.ts
  • ts/fixtures/task/expected/drops/folds.ts
  • ts/fixtures/task/expected/drops/iters.ts
  • ts/runtime/rt/collections.ts
  • ts/runtime/rt/iter.ts
  • ts/runtime/rt/str.ts
  • ts/src/lower/collections.rs
  • ts/src/lower/iter.rs
Files with no reviewable changes: 1
  • quest/m3/ts-size-hints.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Walkthrough

The runtime replaces lower-bound-only iterator reporting with lower and optional upper size hints. Collection helpers use these hints, element sizes, and trusted-length information to reserve capacity and check overflow limits. Rust lowering passes the relevant layout and trust data to runtime helpers. New fixtures cover iterator hints and collection reservations, and generated TypeScript outputs include updated helper arguments.

Merge Risk: Minimal · up to 7f22c

The size-hint and reservation changes are mergeable after normal checks. Existing collection capacity remains an explicitly documented approximation.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main changes: translating iterator size hints and using them for reservation during collection and extension.
Docstring Coverage Passed Docstring coverage is 80.95% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentati…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description is specific to the changeset. It names the quest being finished (ts-size-hints), explains the translator changes for Iterator::size_hint and collection reservation (sizeHint, TrustedLe…
Full details: Docstring Coverage

Explanation

Docstring coverage is 80.95% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 126 functions across 50 files. (8 skipped: 6 unsupported, 2 over the file limit.)

Full details: Description check

Explanation

The description is specific to the changeset. It names the quest being finished (ts-size-hints), explains the translator changes for Iterator::size_hint and collection reservation (sizeHint, TrustedLen handling, hashbrown capacity), lists the public runtime signature changes, and records test evidence and follow-ups. It is not off-topic or generic.

Finishing Touches: 💡 1
⚔️ Resolve merge conflicts: 💡
  • Resolve merge conflict in branch quest/m3/ts-size-hints
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated
kixelated merged commit 82fb1fc into main Oct 30, 2026
2 checks passed
@kixelated
kixelated deleted the quest/m3/ts-size-hints branch October 30, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant