Skip to content

Review follow-up: dialect dedup, capabilities and test/CI infrastructure - #578

Open
thomasp85 wants to merge 8 commits into
review/consolidationfrom
review/dedup
Open

thomasp85 wants to merge 8 commits into
review/consolidationfrom
review/dedup

Conversation

@thomasp85

Copy link
Copy Markdown
Collaborator

Stacked on #577 (itself stacked on #569). Second wave of review follow-ups — mostly mechanical deduplication, all tests green (2095 lib tests, goldens unchanged).

  • Dialect fragment dedup: backtick quote_ident (5 dialects), 1/0 boolean literal (3), and the two identical sql_temporal_as_number pairs are now shared helpers in dialects/mod.rs.
  • Static dialects: the registry stores &'static dialect instances (DialectRef) instead of boxing per lookup; the conformance sweep iterates REGISTRY instead of a hand-copied scheme list, so new backends are covered automatically.
  • ODBC fetch policy: integer-scale NUMERIC/DECIMAL binds as Int64 again (precision past 2^53) except where the registry declares odbc_numeric_as_double (Oracle); text buffers size from the driver-reported column size (1 MiB cap) instead of a flat 16 KiB, with overlong values still an explicit error.
  • Test infrastructure: battery::fixture_df is the single fixture builder; StubReader's SQL-scanning helpers have direct unit tests; registry.rs documents the full 'adding a dialect' checklist; Drill's golden-only status is documented.
  • CI: the duplicated dialect-live compile job is now a reusable workflow_call workflow, fixing the save-if cache drift between the PR and nightly workflows.

Deliberately skipped (from the plan): the geom __ggsql_*__ alias-boilerplate helper and the five 'numbered rows × cross join' fragment dedups — cosmetic, consistent as-is, and not worth another golden churn cycle; happy to do it as a separate small PR if wanted. The e2e.rs submodule reorg is inherited structure, also deferred.

Backtick quote_ident (5 dialects), the 1/0 boolean literal (3), and the
two identical sql_temporal_as_number pairs (Postgres/Redshift,
DataFusion/MonetDB) now live as shared helpers in dialects/mod.rs.
Dialects are stateless unit structs, so boxing a fresh trait object per
lookup was pointless churn. The registry now stores &'static dialect
references (DialectRef) and resolve_dialect/detect_or_err/
dialect_override return them; reader structs and test stubs hold the
reference directly. The dialect conformance sweep iterates REGISTRY
instead of a hand-copied ALL_SCHEMES list, so new backends are covered
automatically.
Two behaviors previously changed globally to accommodate Oracle are now
capability-driven:
- NUMERIC/DECIMAL with scale 0 and <= 15 digits binds as Int64 again
  (preserving integer type and precision past 2^53) except where the
  registry declares odbc_numeric_as_double (Oracle, HY090).
- The per-cell text buffer is sized from the driver-reported column
  size (capped at 1 MiB, x4 for UTF-8) instead of a flat 16 KiB, so
  longer values in sized columns fit; unsized columns keep the default
  and overlong values remain an explicit error, never truncated.
- battery::fixture_df is now the single Arrow fixture builder (golden
  tier and the DataFusion live leg); BigQuery's hand-built batch stays,
  documented as needing Int64 ids.
- Unit tests for StubReader's SQL-scanning helpers (select_list_span,
  split_top_level_commas, output_name), previously only exercised
  indirectly through goldens.
- registry.rs gains an 'Adding a dialect' checklist covering all ~7
  touch points; dialect_live.rs documents that Drill is golden-only.
dialect-live.yml and dialect-live-cloud.yml carried near-verbatim copies
of the compile job (already drifting: save-if present in one, missing in
the other, so the nightly could fight over the shared cache key). The
job now lives in dialect-live-compile.yml via workflow_call, with the
save decision as an explicit input.
auto_cache_if_needed reuses cache_uri() instead of a second spelling of
the in-memory URIs (which had drifted: 'sqlite://:memory:' vs
'sqlite://memory'). The two cache-disable knobs (cache=off vs
cache_disabled) and the reader= param's accepted values are now
documented where they bite, as is the deliberate probe/build double
dlopen of ADBC drivers.
@thomasp85
thomasp85 added this pull request to stack #579 October 6, 2026 17:51
@thomasp85
thomasp85 requested a review from teunbrand October 6, 2026 17:55
- Remove redundant &* reborrows now that readers hold DialectRef
- Drop unnecessary usize cast in ODBC text-buffer sizing
- Update ggsql-jupyter data explorer to pass FromItem::Query
- Remove unused Arc import; move assert_dataframes_equal above the
  test module (items_after_test_module)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant