Repository navigation
Walk get-in and Pointer.get by reference instead of copying the document - #25
Merged
Merged
Conversation
Member
|
port to the new interface. |
JSON.get-in and JSON.Pointer.get seeded their fold with a copy of the whole document, and every step copied the subtree it entered. One lookup of a scalar therefore cost O(document size), and every RFC 6902 test, copy and move operation copied the document once more. Both now use one recursive walker that descends by reference and copies only the addressed value. get-in still indexes arrays with Int.from-string and Pointer.get with array-index, so their token rules are unchanged. bucket-for and member-at move from JSON.Patch to JSON so the walker can use them. They and the walker are hidden but not private, because JSON.Pointer and JSON.Patch call them and a private binding is only visible inside its own module.
hellerve
force-pushed
the
claude/lookup-by-reference
branch
from
October 28, 2026 23:31
8a1268b to
b7ba0ad
Compare
hellerve
marked this pull request as ready for review
October 29, 2026 00:17
hellerve
approved these changes
Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JSON.get-inandJSON.Pointer.getfolded over the path starting from(Maybe.Just @doc), and every step copied the subtree it entered. So a lookup of one scalar copied the whole document plus every subtree on the path. Each RFC 6902test/copy/moveop (they go throughPointer.get) copied the document again, andupdate-inpaid the same cost throughget-in.Both now call one recursive walker,
JSON.get-in-at. It descends by reference and copies only the addressed value. Object steps go throughMap.value-ref!, the same public APIdiff-membersuses; array steps check bounds, then useunsafe-nth. Each entry point passes its own array-token rule, so neither accepts anything new.get-inkeepsInt.from-string, so"01","+1"," 1"and""index as before.Pointer.getkeepsarray-index. Signatures and doc strings are unchanged.The object step lives in a forward-declared
get-in-member, because a recursive call reached through a closure segfaults (the same workaround asserialize-obj-into!).get-in-atishiddenbut notprivate, becauseJSON.Pointercalls it and a private binding is only visible inside its own module.Rebased onto 0.7.2 (core's open-addressing Map): suite 439/0 on carp-lang master cda0b32, angler and
carp-fmt --checkclean. A--optimizererun of 1,000 lookups on the n = 10,000 document went from 394 ms on main to under 1 ms, with the same memory balance on both.Timing, master → branch, on the Pi. The document is
{"meta":{"id":1},"data":[0..n-1]}and the figures are totals over all reps (measured while other builds ran, so they are rough):get-in ["meta","id"]Pointer.get "/meta/id"get-in ["data", last]Patch.apply, 100testopsAt n = 100,000,
Patch.applywith an empty patch costs 9.4 / 9.2 ms; that is the one document copy it always makes. So most of what remains for the 100testops is that copy.Verification
test/json.carp: 439 passed, 0 failed. Eight new assertions pin the token rules:get-in: empty path,"01"on an array, index equal to the length,"-1","-".Differential: 3,000 generated documents, with paths drawn from awkward tokens (
"","01","+1"," 1","-","-1",~0/~1, overflowing integers, numeric object keys, malformed escapes). Each case ran throughget-in,Pointer.get,update-inand atest/copy/movepatch. Master and branch output is byte-identical: 18,001 lines, same md5.Debug.memory-balancereturns to its starting value around 13 lookups and patches, on both master and branch. They cover hit, miss, out of range, into a scalar, empty path/pointer, invalid pointer, test/copy/move and a failing test.7/7 walker mutants are killed:
<→<=>= 0check<→<=Pointer.getusingInt.from-stringNothingThe
>= 0and index-like-token mutants pass master's suite (431/0); only the new assertions catch them.angler and
carp-fmt --checkare clean.carp -x gendocs.carpbuilds, and none of the hidden helpers show up in the docs.apply-teststill copies the value it tests before comparing. Comparing in place would mean passing the leaf action into the walker, which didn't seem worth it to save one copy of the tested value.Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.