Skip to content

DDIR explain: keep the user's key and value whole in demand rows - #929

Merged
frankmcsherry merged 1 commit into
master-nextfrom
explain-shapes
Oct 11, 2026
Merged

frankmcsherry merged 1 commit into
master-nextfrom
explain-shapes

Conversation

@frankmcsherry

@frankmcsherry frankmcsherry commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Explain's demand rows spliced the user's value into one flat tuple with the iteration chain and query id, (K ; V fields…, chain…, q).
That required each collection's field counts, which explain read from projection syntax, and it broke in several ways:

  • A user term reading a whole row saw explain's extra fields. Reach with an always-true filter($1 == tuple($1[0])) gave an insufficient explanation, and a flatmap list reading $0 saw explain's (key, row) pair, so the rewritten program failed to install.
  • A projection given as one term (value($1), and corgi lambdas to come) counted as one field whatever its shape.
  • Keys and values that are neither products nor units (an int source, value($1 * 2), a collect list) were read by field position.

Change

Demand rows are now (K ; (V, chain…, q)): the user's key and value whole, then the chain, then q. No rule takes K or V apart, so explain needs no shapes or field counts.

  • The syntax-based field counting and every width parameter of RowModel and the reverse rules are gone.
  • User terms (filter predicates, projections, flatmap lists, join projections) read $0 as the key and $1 as the value's first field (with_rows).
  • A scope's own sites are wrapped as (V) before their first lift, so the lifts give (V, chain…). Root-level host forms and sources enter the explain scope as (K ; (V)).
  • Demand sets stay the user's rows, for the forward clone and the exports.

Interface

  • explain(p) / explain_with(p, options) take no source shapes; source_arities and export_shape are removed.
  • A query row is (key ; (val, q)).
  • The server's load option is explain or explain=debug (was explain=<arity>[,debug]); README updated. dump_explain takes no arity.

explain/mod.rs goes from 1112 to 973 lines.

Tests

  • whole_row_explanations_sufficient_small: a whole-row filter and a flatmap reading $0.
  • scalar_explanations_sufficient_small, corgi_agrees_on_scalar_explanation: scalar source keys and values, a scalar value and a list at the root, scalar keys in an iterative scope.
  • The reverse-rule contract tests are rewritten for the nested rows.
  • report_demand_excess prints the same demand sizes as master-next, row for row.

cargo test -p interactive --lib --tests, explain's ignored sweeps (--ignored), and the ddir-server tests pass.

🤖 Generated with Claude Code

Explain's demand rows spliced the user's value into one flat tuple with the iteration chain and query id: `(K ; V fields…, chain…, q)`.
That needed each collection's field counts, which explain read from projection syntax, and it broke user terms that read a whole row.
An always-true `filter($1 == tuple($1[0]))` gave an insufficient explanation, a flatmap list reading `$0` saw explain's (key, row) pair, a projection given as one term counted as one field whatever its shape, and keys or values that are not products were read by field position.

Demand rows are now `(K ; (V, chain…, q))`, with the user's key and value whole.
No rule takes them apart, so explain needs no shapes or field counts: the syntax-based counting and every width parameter of the reverse rules are gone.
User terms read `$0` as the key and `$1` as the value's first field.
A scope's own sites are wrapped as `(V)` before their first lift, so lifts give `(V, chain…)`; root-level host forms and sources enter the explain scope as `(K ; (V))`.
Demand sets stay the user's rows, for the forward clone and the exports.

`explain` and `explain_with` take no source shapes, and `source_arities` and `export_shape` are removed.
A query row is `(key ; (val, q))`.
The server's load option is `explain` or `explain=debug`; the arity is no longer needed.
The demand-excess report is unchanged from before, row for row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frankmcsherry frankmcsherry changed the title DDIR explain: take shapes from shape inference DDIR explain: keep the user's key and value whole in demand rows Oct 11, 2026
@frankmcsherry
frankmcsherry merged commit f803531 into master-next Oct 11, 2026
6 checks passed
@frankmcsherry
frankmcsherry deleted the explain-shapes branch October 11, 2026 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant