Skip to content

Preserve embedded NUL bytes in text values - #366

Merged
quinnj merged 1 commit into
masterfrom
fix/preserve-text-value-bytes
Oct 2, 2026
Merged

quinnj merged 1 commit into
masterfrom
fix/preserve-text-value-bytes

Conversation

@quinnj

@quinnj quinnj commented Oct 2, 2026

Copy link
Copy Markdown
Member

Reading a TEXT value such as "abc\0xyz" currently returns "abc". Julia scalar and aggregate SQL functions also receive the shortened argument, even though SQLite retains all seven bytes.

Read the UTF-8 pointer and its byte count in both text readers so the resulting Julia string contains the complete value. This follows SQLite's documented column extraction order and value extraction contract. SQLite permits embedded NUL characters in stored text.

Validation:

  • The permanent regression fails seven checks before the fix. It covers a stored TEXT column, default and strict queries, scalar and aggregate arguments, NUL positions, Unicode, empty strings, SQL NULL, and materialized values after connection closure. Native hex(value) controls confirm that SQLite retains the full bytes.
  • All 834 package checks pass on Julia 1.9.4 and 1.13.1 with two threads. The original reproduction goes from 30 mismatches to zero on both versions.
  • The existing native statement/BLOB test compiles with JuliaC --trim=safe on Julia 1.13.1 and exits successfully with JULIA_LOAD_CODEGEN_LIB=0 and runtime Unicode input.

This change was researched, implemented, and tested by Codex as part of an AI-driven maintenance review.

Co-authored by Codex

@quinnj
quinnj merged commit 3dc2152 into master Oct 2, 2026
7 checks passed
@quinnj
quinnj deleted the fix/preserve-text-value-bytes branch October 2, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant