Skip to content

fix(resolve): find the token that starts at a point, not the one ending there - #472

Merged
bompus merged 2 commits into
fork/consolidatedfrom
fix/point-lookup-token-boundary
Oct 12, 2026
Merged

bompus merged 2 commits into
fork/consolidatedfrom
fix/point-lookup-token-boundary

Conversation

@bompus

@bompus bompus commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

The kernel's point lookup returned the token that ends at a point instead of the one that starts there, so code written without a space between tokens read the wrong node: a Lua local function right after ;, or a PHP instanceof guard right before $x->bar(), resolved wrongly. Json, jq and CPython index to identical edges (CPython: 325,350 call edges, 0 lost, 0 gained).

What changed

  • descendant_for_position in iteration.rs now requires a child to end after the point, matching the TypeScript twin fixed in fix(tsjs): type receivers from the code that declares them #308. ) then Foo in if(ok)Foo f; finds Foo.
  • 34 callers passed the column plus one to stay inside a token; with the stricter rule a one-character token would then be skipped (a.f() would find the .), so they pass the column itself. A temporary test over every token start in 1,866 real files across 11 grammars found the same node under the old rule at column+1 and the new rule at the column at all 2,373,173 points.
  • The earlier audit's list of exposed callers was partly wrong; the callers that really passed a token start are cpp_types.rs, five in iteration.rs, lua_alias.rs, php_scope.rs, prefilter.rs and dart_fields.rs.
  • Zero-width children follow tree.ts (a child must end after the point), not tree-sitter's own rule; this only matters for parser-inserted missing nodes.

Evidence

Tests that fail on the base kernel and pass here: Rust a_point_where_one_token_ends_and_the_next_begins_finds_the_next (if(ok)Foo f;, if(ok)target();, f(x)), and two TypeScript cases, reads a Lua local function written right after ';' as local and applies a PHP instanceof guard to a call written right after ')' (each with a spaced control).

Corpus Call edges base / fixed Reference edges base / fixed Lost / gained
json 13,382 / 13,382 257 / 257 0 / 0
jq 5,307 / 5,307 326 / 326 0 / 0
CPython 325,350 / 325,350 41,769 / 41,769 0 / 0

Golden dumps unchanged; eval:precision identical on json (11/11 absent held, 7/7 present held; jq has no cases); cargo test --lib 68 passed; full suite 695 files, 8,888 passed, 39 skipped, no worker crash lines; clippy, the kernel-rules check and tsc pass.

Limits

The C++ namespace climb (cpp_types.rs) and the Dart field lookup can in theory change too; I built no fixture for either and none of the three corpora moved. A Kotlin parenthesized infix receiver ((b)put(1)) now resolves by name like its spaced form (b) put 1 already does on the base kernel; that older name fallback is not changed here.

README rows checked, none changed: "Name resolution", "What the fork adds" and the "Measured results" timing table.


Summary by cubic

Fixes the kernel's point lookup so that at a token boundary it reads the token that starts there instead of the one ending there. Previously x=1;local function helper() read the ; and treated the Lua local function as global, and if ($x instanceof Foo)$x->bar(); read the ) and missed the PHP instanceof guard; both now resolve like their spaced forms.

Bug Fixes

  • descendant_for_position now requires a child to end after the point, matching the TypeScript side.
  • The 34 callers that passed column+1 to stay inside a token now pass the column itself, so one-character tokens like f in f(x) are not skipped.
  • Verified at every one of 2,373,173 token starts in 1,866 real files across 11 grammars; json, jq and CPython index to identical call edges.
  • A Kotlin parenthesized infix receiver ((b)put(1)) now resolves by name like its spaced form already did; the C++ namespace climb and Dart field lookup can in theory change too, but no fixture was built and none of the corpora moved.

Written for commit 7fdae16. Summary will update on new commits.

View guided diff Turn on auto-fix

Summary by CodeRabbit

  • Bug Fixes
    • Improved code reference resolution at token boundaries across Lua, PHP, C++, Go, Kotlin, Scala, C#, Swift, and other supported languages.
    • Fixed Lua local-function and PHP instanceof cases, including calls written without whitespace at token boundaries.
    • Added regression coverage for boundary lookups; JSON, jq, and CPython behavior remains unchanged.

…ng there

The kernel's descendant_for_position took the first child whose span held
the point with an inclusive end, so where one token ends exactly where the
next begins it returned the earlier one. Callers that pass a token's start
got the `;` in `x=1;local function helper()`, which made the Lua local read
as global, and the `)` in `if ($x instanceof Foo)$x->bar();`, which hid the
PHP guard. tree.ts got the same fix in #308.

A child must now end after the point, as in tree.ts. Callers that passed
column + 1 to land inside a token now pass the column itself; under the new
rule that finds the same node, checked at all 2.37 million token starts of
1,866 real files in 11 languages. Without that change a one-character name
or receiver (`a.f()`) would have found the token after it.

Golden dumps are unchanged, and json, jq and CPython index to the same edges.
@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8fad5713-c302-43a7-9a46-40463522b026

📥 Commits

Reviewing files that changed from the base of the PR and between ce68093 and 7fdae16.


📒 Files selected for processing (18)
  • CHANGELOG.md
  • __tests__/kernel-point-lookup-boundary.test.ts
  • codegraph-kernel/src/resolve/call_shape.rs
  • codegraph-kernel/src/resolve/cpp_aliases.rs
  • codegraph-kernel/src/resolve/cpp_range_for.rs
  • codegraph-kernel/src/resolve/cpp_receivers.rs
  • codegraph-kernel/src/resolve/cpp_types.rs
  • codegraph-kernel/src/resolve/fields.rs
  • codegraph-kernel/src/resolve/go_asserted.rs
  • codegraph-kernel/src/resolve/iteration.rs
  • codegraph-kernel/src/resolve/kotlin_receiver.rs
  • codegraph-kernel/src/resolve/kotlin_type_scope.rs
  • codegraph-kernel/src/resolve/language_type_scope.rs
  • codegraph-kernel/src/resolve/lua_alias.rs
  • codegraph-kernel/src/resolve/names.rs
  • codegraph-kernel/src/resolve/overloads_upstream.rs
  • codegraph-kernel/src/resolve/receivers.rs
  • codegraph-kernel/src/resolve/scala_type_scope.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.



📝 Walkthrough

Walkthrough

Resolver syntax-tree lookups now use recorded source columns, and boundary lookup can select a token that begins at the queried position. New regression tests cover Lua and PHP call targets at adjacent-token boundaries.

Changes

Token-boundary resolution

Layer / File(s) Summary
Boundary lookup behavior
codegraph-kernel/src/resolve/iteration.rs
The shared descendant lookup includes node boundaries. A unit test checks selection of identifiers that start at the preceding token’s end.
Resolver source columns
codegraph-kernel/src/resolve/call_shape.rs, codegraph-kernel/src/resolve/cpp_*.rs, codegraph-kernel/src/resolve/fields.rs, codegraph-kernel/src/resolve/go_asserted.rs, codegraph-kernel/src/resolve/iteration.rs, codegraph-kernel/src/resolve/kotlin_*.rs, codegraph-kernel/src/resolve/language_type_scope.rs, codegraph-kernel/src/resolve/lua_alias.rs, codegraph-kernel/src/resolve/names.rs, codegraph-kernel/src/resolve/overloads_upstream.rs, codegraph-kernel/src/resolve/receivers.rs, codegraph-kernel/src/resolve/scala_type_scope.rs
Resolver syntax-tree lookups across Rust, C++, Go, Kotlin, Scala, C#, Lua, and Swift use the recorded column instead of adding one.
Call-target regression coverage
__tests__/kernel-point-lookup-boundary.test.ts, CHANGELOG.md
Integration tests check Lua call targets after ; and PHP Foo::bar targets after ) with and without spacing. The changelog records the cases and notes unchanged JSON, jq, and CPython graphs.

Priority: ⬇️ Low

Merge Risk | ⚪ Minimal · up to 7fdae

Merge Risk: ⚪ Minimal · up to 7fdae

This change fixes symbol resolution for tokens that sit directly next to each other, such as in Lua and PHP code, and keeps lookups for one-character tokens working. No actionable merge-blocking risk was found.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 7fdae

The change corrects token selection and aligns existing callers with the corrected behavior. No concrete security regression was identified. Its effects on downstream uses of the resulting graphs are not fully established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — Source text and recorded positions can influence inferred graph targets across the existing language consumers. The evidenced outcome is changed analysis metadata, not additional execution authority. Whether external consumers use those targets for security-sensitive decisions is not established.

Trust Boundaries and Controls

  • inferred — The inspected path remains a source-analysis boundary: existing reference inputs select syntax nodes, language-specific consumers derive resolution evidence, and the public method returns verdicts. No introduced authentication, credential, tenant or privileged-action transition was identified in that path; this conclusion does not cover unrepresented external consumers.

Pre-merge checks | Passed 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly describes the main change: selecting the token that starts at a queried point instead of the token that ends there.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Suppressions Explained Passed The reviewed diff changes the resolver lookup logic, tests, and changelog only. No changed line adds or broadens a lint, type-check, compiler-suppression directive, or ignore entry. A scan of added di…
User-Visible Changes Documented Passed The diff changes token-boundary lookup behavior in existing resolver code, adds regression tests, and adds a CHANGELOG entry. It does not add, remove, or rename a CLI command or flag, MCP tool or argu…

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR



🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@bompus
bompus merged commit e91e630 into fork/consolidated Oct 12, 2026
5 checks passed
@bompus
bompus deleted the fix/point-lookup-token-boundary branch October 12, 2026 02:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant