Repository navigation
fix: preserve text delta chunks and UTF-8 insertion order - #1230
Merged
Merged
Conversation
Contributor
WASM Size Report
|
See context/text-delta-chunking.md for the adjacent-leaf and end-cursor invariants.
Test the packaged crate against registry generic-btree 0.10.7. See context/text-delta-chunking.md for the workspace patch boundary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Long strings inserted through
loro_delta::text_delta::TextDeltasilently lost their final chunk: 129, 256, and 257 ASCII bytes became 128, 128, and 256 bytes. Middle insertion could also panic when a UTF-8 character crossed chunk capacity. These chunking defects have existed since DeltaRope was introduced in #327.Share byte-capacity, UTF-8-aware chunking between long appends and middle insertion, including the final slice. Preserve the short-string fast path and attributes on every chunk.
Partial UTF-8 chunks expose a B-tree batch ordering defect: inserting 127
xcharacters plusébetween thebandcin ordinaryafollowed by boldbccan move thexcharacters beforeb. FixDeltaRope::insert_valuesin the published crate itself: buffer multiple items and insert back to front using the returned start cursor from single-item insertion. Empty and single-item inputs avoid batch allocation. This works with registrygeneric-btree 0.10.7, without requiring the workspace patch or a new B-tree release.Also fix the in-tree B-tree's adjacent-left-leaf selection and end cursor after merging a batch prefix. The vendored fork is excluded from publishing, so these changes are tested separately and are not the basis for downstream correctness.
Add
scripts/test-delta-published-deps.shto CI. It creates the actual.cratearchive, extracts it outside the workspace, pins registrygeneric-btreeto the minimum supported 0.10.7, and runs all packaged tests independently of[patch.crates-io].Regression coverage includes length boundaries; 2/3/4-byte UTF-8 characters; append and middle insertion; preservation of surrounding attributes; 1,000 sequential UTF-8 edits compared against
String; B-tree leaf/parent boundaries, caches, and subsequent insertion. Include a Rust-only changeset and document the insertion and publishing invariants incontext/text-delta-chunking.md.Validation:
bash scripts/test-delta-published-deps.sh --offline: all 44 packaged tests passed with registrygeneric-btree 0.10.7, including all 22 text integration tests.cargo test --offline -p generic-btree -p loro-delta: 70 passed, 3 existing ignored tests; doctests passed.cargo clippy --offline -p generic-btree -p loro-delta --all-targets -- -D warnings: passed.git diff --check: passed.