Repository navigation
Share SQL stages and dispatch validated numerical kernels - #302
Draft
cromwellian wants to merge 3 commits into
Draft
cromwellian wants to merge 3 commits into
cromwellian wants to merge 3 commits into
Conversation
Support prepared SQL gradients, arithmetic chunk-grid counts and direct Arrow registration. Preserve Context metadata through extension installation, expose target partitions and guard physical reuse against value-dependent filters. Co-Authored-By: GPT-6 <noreply@openai.com>
Materialize shared reachable CTEs once, reuse stage programs with fresh physical state, and retain ddx differentiation of original views. Add optional session-local float64 CPU and float32 MLX kernel execution with SQL validation and fallback, using the separately maintained EinFold feature build. Co-Authored-By: GPT-6 <noreply@openai.com>
Build the upstream-compatible feature commit from the contribution fork. Co-Authored-By: GPT-6 <noreply@openai.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
This was referenced Nov 2, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🛠️ Codex
Evaluate reachable shared CTEs once per invocation and reuse their stage programs while keeping ddx differentiation of the original SQL/views. Add session-local EinFold numerical dispatch for explicit float64 CPU or float32 MLX GPU execution, deterministic function eligibility, fresh physical state, ordered Arrow providers and bounded coordinate reuse. Ordinary SQL remains the fallback for unsupported semantics.
Depends on CPU setup/preparation #301 and EinFold’s kernel-backend PR. Incremental review: cromwellian/xarray-sql@codex/training-plan-reuse...codex/shared-kernel-execution . GitHub requires fork PRs to target upstream main, so the default diff includes the predecessor until it merges.
Measured tiny-model shared CPU/GPU warm steps improved 314→206 ms (1.53×) and 371→224 ms (1.66×), with fewer repeated kernel dispatches. GPU alone did not solve large-model backward/Arrow overhead. The separate compact-array and resident-training PRs address those remaining costs. The performance/evidence PR retains the detailed workloads and raw results.
The optional installer builds a pinned commit from the separate EinFold fork; native sources and patches are not duplicated here. Validation of the full contribution stack: 714 non-integration tests passed against rebuilt dependency wheels, including SQL masks/NULLs, custom-function overrides, changing inputs, coordinates and snapshots. Rust formatting and changed-file Ruff checks passed. GPU arithmetic remains explicit float32.