Skip to content

fix(jdbc): treat non-ASCII characters as identifier characters - #137

Merged
magnusp merged 1 commit into
mainfrom
fix/i103-non-ascii
Oct 22, 2026
Merged

magnusp merged 1 commit into
mainfrom
fix/i103-non-ascii

Conversation

@magnusp

@magnusp magnusp commented Oct 22, 2026

Copy link
Copy Markdown
Owner

Fixes #103.

Contract: Hrana 3 is the source of truth for transport, values, metadata, batching, transactions, and error reporting. java.sql is the source of truth for the JDBC surface. xerial/sqlite-jdbc is a behavioral reference and conformance foundation, never a copy target.

Change

isWordChar in SqlTableReference and LibsqlResultSetMetaData now treats every character at or above 0x80 as an identifier character, matching SQLite. SqlTableReference.UNQUOTED_IDENTIFIER accepts the same range, because identifierOf re-validates word tokens against that pattern; without it the tokenizer would still split användare. ASCII-only behavior is unchanged.

Base aad9c7e456fad5c874e8efdc6f6c2d368244b654, head c47144e6935ed9b145cf2e58eca7dfd39387d526.

Red and green

  • Red at trunk code with the new tests: 7 failures, Tests run: 66, Failures: 7 (/tmp/xerial-parity-i103/lane-1-regression.log). SqlTableReferenceTest.resolvesNonAsciiTableAndColumn got [null] instead of användare; LibsqlResultSetMetaDataTest.resolvesNonAsciiTableName got an empty table name.
  • Green at head: SqlTableReferenceTest 34/34, LibsqlResultSetMetaDataTest 32/32, BUILD SUCCESS.

Lanes

Lane Command Result
1 targeted tests at trunk vs head red 7 failures, green 66/66
2 LibsqlResultSetMetaDataTest 32/32; getTableName returns användare
3 non-ASCII table + anv shadow (LibsqlResultSetMetaDataTest, prefix case) 32/32; reports användare, not anv
4 ./mvnw -pl libsql-jdbc test 245/245, BUILD SUCCESS
5 ./mvnw -pl libsql-conformance -am verify 123 tests, 0 failures, 1 skipped, BUILD SUCCESS
6 ./mvnw -pl libsql-conformance -am -Pxerial-control verify 100 tests, 0 failures, 1 skipped, BUILD SUCCESS
7 ./mvnw spotless:check BUILD SUCCESS
8 ./mvnw test BUILD SUCCESS
9 parser/result-set suite (98 tests) BUILD SUCCESS; no ASCII case changed outcome
10 ./mvnw -pl libsql-jdbc -am package -DskipTests BUILD SUCCESS; jar and shaded jar built

Perf: ./mvnw -pl libsql-jdbc test medians, three interleaved runs per side: trunk 2.08 s, head 2.08 s (within 10%). Logs in /tmp/xerial-parity-i103/.

Divergences

  • UNQUOTED_IDENTIFIER gained the \x80-\uFFFF range. The plan's Build box names only isWordChar, but identifierOf rejects any word token outside that pattern, so the tokenizer fix alone cannot make användare resolve.
  • LibsqlResultSetMetaDataTest is not in the Files block; lane 2 names it explicitly, so the two metadata cases live there.
  • resolvesQuotedNonAsciiNames mixes quoted names with an unquoted non-ASCII column, so it fails at trunk as lane 1 requires. The fully quoted assertion (SELECT "namn_ö" FROM "användare") already passed at trunk and is kept as regression coverage only.

Tests added: non-ASCII tables, columns, table/column aliases, quoted and bracketed names, SELECT * expansion, a table named anv shadowing användare, and the metadata path.

@magnusp
magnusp merged commit 1bed15e into main Oct 22, 2026
5 checks passed
@magnusp
magnusp deleted the fix/i103-non-ascii branch October 22, 2026 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Table metadata resolution breaks on identifiers with non-ASCII characters

1 participant