Skip to content

feat(ts): lend the byte arrays a collection holds as slices - #295

Merged
kixelated merged 3 commits into
mainfrom
quest/m3/ts-byte-key-refs
Oct 30, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m3/ts-byte-key-refs

Conversation

@kixelated

@kixelated kixelated commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A collection of [u8; N] keys holds each key as a Uint8Array, where translated code reads a &[u8; N] as an rt.ByteSlice. So every call that lends a held key (keys, iter, get_key_value, Entry::key, HashSet::get, B-tree first/range/get, retain's closure) was refused with a span (#276). Auditing them turned up silent mistranslations in the same class: for (k, v) in &map, for k in &set, (&map).into_iter(), and map[&k] with a [u8; N] value handed out the held Uint8Array unconverted, because bytes_across never looked through the & of &HashMap<..> and waved every Index through.

Approach

  • collections::lend converts what a collection's mapping returns, by the call's output type: each &[u8; N] becomes rt.byteSliceFrom(held), through tuples ((&K, &V)), Options (new rt.optionMap), and iterator items (rt.iterMap). Closures std hands a held key or value (retain, or_insert_with_key, and_modify) are wrapped the same way. into_iter of &coll / &mut coll lends its items; the heap's special case folds into this.
  • It applies to HashMap, HashSet, BTreeMap, BTreeSet, entries, VecDeque, and BinaryHeap, so keys and values alike: a & shares the held bytes (nothing writes through it, and *k copies), and a &mut (get_mut, front_mut, values_mut) writes where the collection holds them.
  • collections::lends_call replaces the lookup special case in bytes_across: a collection call is let through only if its output and closure parameters are lendable and no other input takes references to bytes. Everything else stays loud with a span.
  • New mappings: HashMap::get_key_value (rt.hashGetKeyValue) and BTreeSet::get (rt.btreeGetKey). Neither was mapped for any key type.
  • Fixed lookup_ty for Index<&Q>. It returned &Q, not Q, so map[&k] passed a byte-array key's slice unconverted.
  • Vec<u8> and &[u8] keys didn't share the gap: they are held as rt.ByteVec / rt.ByteSlice already. They are covered by the tests anyway.

Tests: keys::lent_keys and keys::lent_values in the lang differential harness against native wasm32 cover:

  • all four maps and sets, including empty ones
  • B-tree order from either end and over ranges
  • lent keys sorted, compared, and looked up by after lending
  • HashSet::get and BTreeSet::get by a borrowed &[u8], plus HashSet<Vec<u8>>::get
  • copies of lent keys written without reaching the stored key, and a write to the key given insert after insertion
  • writes through &mut values, deque items, and heap peek/iter/retain

A mutation check confirms the tests catch a broken lend. With optionMap handing out copies, the lang harness reported 162 differences from native.

Impact

  • Translator output: these calls now translate instead of being refused. Previously silent for-over-&coll and map[&k] cases now emit correct slices.
  • Newly refused with a span, previously mistranslated: extend of a [u8; N] collection by references to arrays (set.extend(&pairs)), which would store slices sharing the source's bytes.
  • Runtime (internal): optionMap, hashGetKeyValue, btreeGetKey. No breaking change, no MIGRATE.md entry.

Decisions made without asking

  • Scope covers values, VecDeque, and BinaryHeap as well as keys. It is the same mechanism and the same bug class ("fix every instance"); restricting it to keys would have meant extra code just to keep refusing values.
  • A collection holding a &[u8; N] beside a [u8; N] (e.g. HashMap<&[u8; 2], [u8; 2]>) is refused outright. Its held and lent &[u8; N] are the same type, and these calls were already refused before.

Alternatives

  • Storing [u8; N] keys as rt.ByteSlice in the collection. That would avoid conversion on lend but convert on every insert and owned yield, and change the collections' TS types. I rejected it as more invasive.
  • Per-method runtime helpers (lentKeys, lentEntry, ...). I rejected these as combinatorial; the type-driven conversion covers every shape with one function.

Follow-ups

  • Set operations: union/intersection/difference/symmetric_difference are unsupported for every element type.
  • References to byte arrays std takes: extend by references, adaptors' closures over held arrays (filter_owned_bytes, map(|(k, v)| ..).collect() over a map), and a collection of both forms.

Review

  • Reviewer: Codex (quest review), gpt-6.1-sol, medium effort.
  • Round 1 on 3fd141fb096722f349c4265cdaf124b7a3ce5aed: no findings.
  • Round 2 on 84262b7cebfb4cb7728114aadbc1149b81e8c9e3 (merge of origin/main with hand-resolved conflicts in collections.rs and the harness case list): no findings.
  • Reviewed head: 84262b7cebfb4cb7728114aadbc1149b81e8c9e3. just ci passed locally on it.

(written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 30, 2026 10:32
What a map, set, deque, or heap of `[u8; N]` keys or values lends out, by `keys`, `iter`,
`get`, `get_key_value`, `range`, an entry's `key`, `Index`, `for` over a reference, or
`retain`'s and `or_insert_with_key`'s closures, is an `rt.ByteSlice` of the bytes it
holds, as a `&[u8; N]` is elsewhere. `HashMap::get_key_value` and `BTreeSet::get`
translate, and `map[&key]` finds a byte-array key. Extending one by references to arrays,
or holding a `&[u8; N]` beside a `[u8; N]`, is refused with a span.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…refs

# Conflicts:
#	ts/fixtures/lang/expected/harness.ts
#	ts/fixtures/lang/expected/keys.ts
#	ts/fixtures/lang/src/harness.rs
#	ts/fixtures/reject/census.md
#	ts/fixtures/reject/expected.txt
#	ts/src/lower/collections.rs
@kixelated kixelated changed the title References to byte-array keys a collection yields feat(ts): lend the byte arrays a collection holds as slices Oct 30, 2026
@kixelated
kixelated marked this pull request as ready for review October 30, 2026 18:30
@kixelated
kixelated merged commit cecbf0d into main Oct 30, 2026
1 of 2 checks passed
@kixelated
kixelated deleted the quest/m3/ts-byte-key-refs branch October 30, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant