Repository navigation
Conversation
… cllw-ore's vitaminc schemes Text could not be ordered: no ordering operator was answered on a text field, and the ORE and OPE terms stored for text went through cllw-ore's own collation, which keeps case, so Zoe ordered before alice and é after z. ADR-0002 defines text's order key as its fold (Unicode 16 NFC with case and accents folded), and vitaminc 0.6 ships it in vitaminc-ore. cllw-ore 0.5.2-rc.1 implements vitaminc-ore's `OrderScheme` behind its `vitaminc` feature, so Stack Encrypt encodes nothing for ordering itself. An ORE or OPE term is now cllw-ore's `vitaminc::Ore` or `vitaminc::Ope` encryption of the value: the engine names the value's order target through the new sealed `sem::Orderable` (integers at their own width, `f32`/`f64` as `F64`, text as `Text`, bytes as `Bytes`) and passes the value straight through, on the typed path and for each scalar of a dynamic value alike; cllw-ore encodes it with vitaminc-ore, text by the fold. Café, cafe, CAFÉ and cafe with a combining accent are one term. Text holding a code point Unicode 16 does not assign has no fold and no order term, as it has no equality or match term; ADR-0002 is amended to say vitaminc 0.6 refuses such a code point rather than passing it through. The per-index key derivation is kept: the PRF block keys the scheme inside the visitor and is wiped there, and the column is opened with the value's target and the empty context, since the key is already the index's own. The key labels move to `/v2` because every order term changes bytes (new column salt, cllw-ore's marker byte for text and bytes). Every source type's term is now one ciphertext type (`OreCllw8VariableV1`, `OpeCllw8VariableV1`); `OreTerm<T>`/`OpeTerm<T>` keep `T` as a marker so typed callers compile unchanged, and `sem::CllwOreEncrypt`/`CllwOpeEncrypt` are no longer re-exported. The term pins and the record, query and value-kinds fixtures are re-recorded. op::Ordered covers String, and the data path answers the orderings on text too, refusing order text with no fold at its boundary, for one value and for each bound of a range, through `vitaminc_ore::text`, the encoding the `Text` target uses. The Go generator writes Lt, Lte, Gt, Gte and Between on a string field with ore or ope; its notice is left for order indexes on bytes or a float, and for a second order index. Closes #1234.
An `f32` or `f64` order term is cllw-ore's vitaminc encryption of the value at the `F64` target, whose vitaminc-ore encoding folds `-0.0` into `0.0` and every NaN into one value above `+Infinity`: Postgres's `float8` order. So the ordering operators are now answered over floats, on the typed path (`op::Ordered` for `f32` and `f64`) and over a data plan's `float32` and `float64` fields, and stashgen writes `Lt`, `Lte`, `Gt`, `Gte` and `Between` on a float field with `ore` or `ope`. An `f32` is ordered as the `f64` it widens to: its `Orderable` target is `F64` and vitaminc widens it exactly, so an `f32` and its `f64` are one term and `f32` can join `op::Ordered`. A float `Between` is not refused when inverted (a range from NaN, which sorts last, matches nothing), as SQL's `BETWEEN` is not. The Go query profile gains a sealed `Score float64` with ORE, matching the Rust query fixture, and the live PostgreSQL test stores the score and its ORE term so the rows it reads back open. Float pins, the float ordering baseline, the integer-width lengths and the `score` field in the cross-path fixture are added; the fixtures and UI snapshots are re-recorded. Closes #1236.
🦋 Changeset detectedLatest commit: 82ab168 The changes in this PR will be included in the next version bump. This PR includes changesets to release 12 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
coderdan
left a comment
coderdan
left a comment
There was a problem hiding this comment.
Review of 7b30b04 (this PR's diff against feat/stack-encrypt-order-keys only). No prior review comments existed. I found nothing that blocks; three low-severity notes are inline and summarised below.
Findings (most severe first)
- Low — the ordered fixtures pin order-term bytes that the base PRs may still move.
v4_ordered_queries.json(Rust-derived, checked by Go) andv4_ordered_rows.json(Go-sealed, checked by Rust) both holdophex that depends on how #1235/#1242 settle thestored_rowsquestion. If either base changes the bytes before this merges, the GoTestV4OrderedQueriesMatchTheRustFixture/TestV4OrderedRowsMatchTheCommittedFixtureand the Rustthe_ordered_query_fixture_is_what_the_dispatch_derives/ordered_rows_the_go_sdk_sealed_open_in_rustall fail on this branch after a rebase. Regenerable (EQL_UPDATE_FIXTURES=1on both sides, Go first then Rust, or the other way for the queries file), but worth a line in the PR's rebase notes. The PR already says there is no frozenstored_v4_ordered_rowsfixture yet; the fixtures README (inline) should say so too so nobody treats these two as frozen. - Low —
encryption_gap's fallback comment is stale. Theelsebranch comment says it is reached by an "equality-only domain"; withord/ord_openow inENCRYPTION_DOMAINS, any unlisted OPE-only domain of a family with a plaintext (e.g.smallint's once it gains a kind) lands there too. The message itself is generic enough; only the comment is wrong. Inline. - Low — a README note on indexing would close the loop with the non-superuser story. The Go README says to sort with
ORDER BY eql_v4.ord_term(col)but not how to index it. Since the OPE term is abyteadomain, a plain functional btree oneql_v4.ord_term(col)works with no operator class, which is exactly why these types are the portable choice on a non-superuser install where the ORE opclass is skipped. One sentence would spare a reader reaching for the ORE opclass. Inline; optional.
Informational, no change asked: Eq on a TextOrd/TextOrdOpe field always derives op alongside hm, so =/<> with text that has no fold (an unassigned code point) is refused even though = reads only hm. Harmless in practice because the same text cannot be stored in that column, so no row could match; and the crate README already says "refused, stored or queried".
Verified as sound
- Bytes written to
opare what the SQL reads.OpeVisitorwriteshex::encode(term.as_bytes())of cllw-ore'sOpeCllw8VariableV1;eql_v4_internal.ope_cllw(jsonb)isdecode(val->>'op','hex')::bytea, and everylt/lte/gt/gte/eq/neqwrapper on the*_ord/*_ord_opedomains comparesord_term(a) <op> ord_term(b), i.e. PostgreSQLbyteaorder (memcmp, shorter prefix first). cllw-ore documents the OPE ciphertext as exact under lexicographic byte order (+128signal dominates the backward carry) and its ownOrdis derived byte order, so<on terms is<on order keys. The query side uses the sametarget::ope()plan under the same(table, column)CallerContext, andeach_ordered_type_stores_its_catalog_termspinsop(query) == op(stored)for all ten types. - Order equals plaintext order per family. Integers:
i32/i64go in at their ownOrderTargetwidth (I32/I64), andinteger_terms_order_by_valuecoversMIN < -70000 < -1 < 0 < 1 < ... < MAX, so signed values cross zero correctly. Floats: bothf32andf64map toOrderTarget::F64;float_terms_order_as_postgres_float8_doespins-inf < MIN < ... < -0.0 = 0.0 < ... < +inf < NaN = NaN(other payload), matchingfloat8sort, anda_real_orders_as_the_double_it_widens_topins the widening. Text:text_terms_order_by_the_foldpins the empty string, prefix-first (al<alice), case/accent ties (alice=ALICE,cafe=Café=CAFE+combining), andtext_order_ties_are_not_equalpins thathmstill differs and equalsTextEq's. Fixture lengths agree with cllw-ore's8n+1: 66 hex fori32, 130 fori64/f64and for widenedf32, and folded-text length + 1 for text. - Widths and refusal.
run_targetgoes throughPlaintext::from_frame, which refuses any other kind rather than narrowing or widening; the generated structs pin#[stash(plaintext = i32|i64|f32|f64|String)]; the SQL does not care about width, only the domain name, so aninteger_ordcolumn and abigint_ordcolumn cannot be cross-compared by accident. Go range-checks wider integer fields before sealing (pre-existing). - Twins.
IntegerOrd/IntegerOrdOpe(and the other families) both declareindexes: ["ope"]and store oneop;ord_and_ord_ope_store_one_termpins it.*OrdOrestays unproducible with the block-ORE reason.TextOrd/TextOrdOpestorehm+op, matching thetext_orddomain CHECK (v,i,c,hm,op) and thequery_text_ordCHECK (v,i,hm,op, noc);=/<>on text readeq_term, orderings readord_term. betweencannot leak through.target_operatorsfilterstakes_pair()andanswer()refuses an operator the target does not list asOperatorNotAnswered, so a hand-builtbetweenplan on an OPE target is refused rather than producing a one-bound operand. The Go/guest catalog tests assert exactlylt,lte,gt,gte(+eqfirst on text).- Docs vs non-superuser installs. Nothing here points at the ORE opclass; the Go README's term table already says
Opecompares in SQL with<,>,ORDER BYandOremust not. Consistent with OPE being the portable path. - Changeset. The bot's "No Changeset found" is a false alarm: the PR extends the existing unreleased
@cipherstash/eqlminor changeset rather than adding a file. That is the right package:eql-bindingsships in lockstep with the npm package, and the Go module has no changeset (its CHANGELOG is edited by hand here). No TS package or skill surface changes.
Checks run (no credentials, worktree at 7b30b04)
packages/eql:mise run types:check(regenerates the 27 binding files andlanguages/golang/encrypt/eql/eql_gen.go: no drift) andmise run codegen:parity: both clean.cargo test --locked -p eql-codegen,-p eql-bindings --all-features, and-p eql-encryption-tests --test v4_order --test targets --test v4_types: all pass (v4_order12 passed, the 2 PostgreSQL tests ignored without a database; CI runs them viatest:encryption:postgresintest-eql.yml).cargo clippy --locked -p eql-bindings -p eql-codegen -p eql-encryption-tests --all-features --all-targets -- -D warnings: clean.- Built all four Go guests (
wasm:guest:build,:eql,:deterministic,:eql:deterministic), thengo generate ./...inlanguages/golang: no diff;go vet ./...clean;go test ./encrypt/... ./stashgen/ ./cmd/...: pass, including the three newTestV4Ordered*tests against the deterministic EQL guest. - Not run: the PostgreSQL half of
v4_order(no local EQL v4 database). I read the generated SQL it exercises instead.
| cannot make, so `=` or `<` on short text would be refused; how a query that is not a \ | ||
| match leaves it out is not decided (#1131)" | ||
| } else { | ||
| // Reached by an equality-only domain of a family whose plaintext is |
There was a problem hiding this comment.
Low: this comment is now stale. With ord and ord_ope in ENCRYPTION_DOMAINS, the fallback is reached by any unlisted domain of a family whose plaintext is specified and whose terms are neither ORE nor match-and-order, not only an equality-only one (a future smallint ord once that family gains a kind, for instance). The message string itself is fine; just the comment needs widening.
There was a problem hiding this comment.
Fixed in cc3ea66. The comment now says the fallback is reached by any unlisted domain of a family whose plaintext is specified and whose terms are neither ORE nor match-and-order.
| `float8` sorts them (`-0.0` ties `0.0`, every NaN is one value after | ||
| `+Infinity`), text by its fold (case and accents folded, under Unicode 16). | ||
| A field gets `Lt`, `Lte`, `Gt` and `Gte`, and a text field `Eq` too; sort | ||
| with `ORDER BY eql_v4.ord_term(col)`. On a number column SQL's `=` compares |
There was a problem hiding this comment.
Low, optional: a sentence on indexing would help here. Because ord_term returns a bytea domain, a plain functional btree index on eql_v4.ord_term(col) serves <, > and ORDER BY with no operator class, which is the reason these types are the portable choice on a non-superuser install where the ORE opclass is skipped. As written a reader has to work that out from the Ope vs Ore bullets further up.
There was a problem hiding this comment.
Added in cc3ea66. The Go README now says ord_term returns a bytea domain, so a plain functional btree, CREATE INDEX ON t (eql_v4.ord_term(col)), serves <, > and ORDER BY with no operator class. It also says that's why these types work on an install without superuser rights.
| | `v4_ordered_queries.json` | the by-name dispatch in `tests/v4_order.rs` | yes, with `EQL_UPDATE_FIXTURES=1` | | ||
| | `v4_ordered_rows.json` | the Go SDK's `TestV4OrderedRowsMatchTheCommittedFixture` | yes, with `EQL_UPDATE_FIXTURES=1` | | ||
| | `text_eq_query.json`, `ste_vec_selectors.json` | `tests/text_eq.rs`, `tests/ste_vec_selectors.rs` | yes, see each test | | ||
| | `stored_v4_rows_v1.json` | copied from `v4_rows.json` at the next build | **never: frozen** | |
There was a problem hiding this comment.
Low: the two new rows above are regenerable, and their op bytes depend on how #1235/#1242 settle the stored-rows question. Since this table is also where the frozen fixture is explained, it would be worth one line saying that no frozen stored_v4_ordered_rows_* exists yet and that both ordered fixtures must be regenerated together (Go writes v4_ordered_rows.json, Rust writes v4_ordered_queries.json) after a rebase that moves the order-term bytes. Otherwise the first sign is four cross-language tests failing at once.
There was a problem hiding this comment.
Fixed in 82ab168. The fixtures README now says neither ordered fixture is frozen, and that both are regenerated together when the order-term bytes move: Go writes the rows, Rust writes the queries. It also says no frozen stored_v4_ordered_rows_* exists yet.
IntegerOrd, BigintOrd, RealOrd, DoubleOrd, TextOrd and their OrdOpe twins join ENCRYPTION_DOMAINS, so their v4 structs derive EncryptFrom and DecryptInto and the by-name dispatch runs them. OpeCllw implements EncryptFrom for any orderable plaintext: the hex of Stack Encrypt's OPE term over ADR-0002's order key, which EQL compares as bytea. The engine already answers the four single-bound orderings on a target through its query twin (and Eq on text, through TextEq's equality term), so Go fields typed with these get Lt, Lte, Gt and Gte from stashgen. Between is not offered: a range needs EQL v4's range query domains. Tests: the stored terms sort as the order keys do (integers, floats with -0.0 tied and NaN last, text by its fold, prefixes first), each query twin carries the stored terms, Ord and OrdOpe store one term, text with no fold is refused, and against an EQL v4 database <, <=, >, >=, = and ORDER BY eql_v4.ord_term find exactly the expected rows. A new v4_ordered_queries.json pins one query per type for the Go SDK. TextSearch stays unproducible with its own reason: its query carries a match term, which short text cannot make, so = and < on short text would be refused. The *OrdOre types still wait on block ORE.
…Go's rows in PostgreSQL A Ranking test model holds one field of each OPE-ordered type. Go seals it through generated code over the deterministic EQL guest: each value stores its catalog keys, the order terms sort as the rows do (with the -0.0 and folded-text ties), and everything opens back. Its Lt, Gte and Eq methods derive v4_ordered_queries.json byte for byte. TestV4OrderedRowsMatchTheCommittedFixture writes v4_ordered_rows.json, which the Rust suite opens field by field and inserts into eql_v4 columns, where < with the fixture's query operands finds the rows the plaintexts say. Profile and its frozen rows are untouched.
7b30b04 to
82ab168
Compare
coderdan
left a comment
coderdan
left a comment
There was a problem hiding this comment.
Incremental re-review of 82ab168 against the previous review on 7b30b04 (review 5563357370). Between the two heads the code diff is five files: the changeset split, the Go README indexing note, the encryption_gap comment and the fixtures README note. The Go and PostgreSQL test commit was already on 7b30b04, but I read it again closely here, as asked, and ran its PostgreSQL half locally, which the earlier review could not do.
Prior findings
- Fixtures README note on the ordered fixtures: fixed.
fixtures/README.mdnow says that neither ordered fixture is frozen, that both are regenerated together (Go writes the rows, Rust writes the queries), and that nostored_v4_ordered_rows_*exists yet. One wording nit is inline. - Stale
encryption_gapcomment: fixed. Theelsebranch ineql-codegen/src/bindings.rsnow says it is reached by any unlisted domain whose terms are neither ORE nor match-and-order. - Go README indexing sentence: fixed.
encrypt/README.mdnow saysord_termreturns abyteadomain, so a plainCREATE INDEX ON t (eql_v4.ord_term(col))serves<,>andORDER BYwith no operator class, and that this is why these types work on a non-superuser install.
The changeset is also cleaner now. The PR adds its own .changeset/eql-v4-ordered-ope-types.md (@cipherstash/eql minor) and no longer edits eql-v4-bindings.md, which matches the base branch exactly.
New findings
- Low: the PostgreSQL test of the Go-sealed rows checks only
<, so it never sorts Go's rows in SQL.postgres_compares_rows_the_go_sdk_sealedruns one<query per column. The probe ties row 2, so every column expects[1]. That catches a Go term for row 1 or row 2 on the wrong side of the probe. It cannot catch a row-3 term that ties the probe or row 2, because the only check on row 3 is that it is not below the probe.ORDER BY eql_v4.ord_term(col)over Go's bytes is never run either. The Rust-sealed test covers those operators, and the Go test checks the byte order of Go's terms, so this is a coverage gap rather than a known bug. Adding the other operators costs nothing: I ran them against the committed fixtures (details inline) and they give the expected answers. Details inline. - Nit: "the base build" in the fixtures README will be unclear once this merges. Inline.
Informational, no change asked: unlike v4_rows.json, the ordered rows have no CI step that seals them fresh in Go and hands them to Rust (the EQL_V4_ROWS_OUT / EQL_V4_ROWS step in tests-golang.yml). Rust only opens the committed ciphertexts. I don't think that matters here. TestV4OrderedRowsMatchTheCommittedFixture compares everything except c (so op and hm are drift-checked) and checks that the committed c still opens. A change in how Go writes the plaintext width would also move the op length (66 hex vs 130), which that check catches. The fresh hand-off for v4_rows already covers the AEAD agreement between the Go guest and native Rust.
Verified as sound
- CI runs it.
packages/eql/mise.toml'stest:encryption:postgresnow loops overtext_eq v4_types v4_orderand refuses an empty--ignoredselection. TheRust workspace cratesjob intest-eql.ymlruns it against the EQL v4 bundle built from the tree. On this head, the job log shows bothpostgres_compares_and_orders_ope_rows ... okandpostgres_compares_rows_the_go_sdk_sealed ... ok. - The keys match. The query fixture is derived under
FakeDataKeySource. The rows are sealed under the deterministic guest andtestSeed.TestV4OrderedQueriesMatchTheRustFixturederives the same query bytes from Go'sLt/Gte/Equnder the guest, so both sides use the same index key and<across them means something. - The expectations tell a fold from raw bytes.
ZucchinivsMangoandCarol/Alicevscarolsort differently by byte than by fold, so a case-sensitive term would fail thetitle/handlechecks. The float plaintexts are exact inf32, sof64::from(p.speed) < plain.as_f64()has no rounding edge.!expected.is_empty() && expected.len() < rows.len()stops a column from passing on an empty or full result. - The fixtures are deterministic and drift-checked. The Go side compares the fixture with
cstripped and only rewrites the file underEQL_UPDATE_FIXTURESwhen something other thancmoved. The Rust side asserts the query fixture byte for byte.tests-golang.yml's path filter includespackages/eql/tests/encryption/**, so regenerating the Rust query fixture triggers the Go check. TestV4OrderedFieldsSealToTheirEQLTypesAndOpenBackcovers the cases JSON cannot hold (the infinities and-0.0): strict term order between consecutive rows, the row 2/3 tie, and the sign of zero after a round trip.
Checks run (worktree at 82ab168, no credentials)
cargo test --locked -p eql-codegen,-p eql-bindings --all-features,-p eql-encryption-tests: all pass (v4_order: 12 passed, 2 ignored).mise run test:encryption:postgresagainst a throwaway local PostgreSQL 17 with the tree's v4 bundle:text_eq1,v4_types3,v4_order2, all pass.- Ad hoc SQL over the committed
v4_ordered_rows.jsonandv4_ordered_queries.json, per column:ORDER BY eql_v4.ord_term(col) DESCgives3,2,1,>gives[3]and>=gives[2,3].=gives[2]on the eight number columns and nothing ontitle/handle, as expected, because=reads the case-sensitivehmand row 2 ismango/CarolagainstMango/carol. - Built the four encrypt guests and the auth guest. In
languages/golang:go generate ./...leaves no diff,go vet ./...is clean, andgo test ./encrypt/... ./stashgen/... ./cmd/...passes, including the threeTestV4Ordered*tests run with-v. - Not run: the repo's
scripts/__tests__vitest suite (nonode_modulesin this worktree). CI is green on this head.
| ), | ||
| queries[*column]["query"].as_str().unwrap(), | ||
| ); | ||
| assert_eq!(found, expected, "< over Go-sealed {column}"); |
There was a problem hiding this comment.
Low: this is the only SQL comparison of Go-sealed rows, and with the probe tying row 2 it expects [1] in every column. It catches row 1 or row 2 landing on the wrong side of the probe. Row 3 only has to be not below the probe, so a Go-sealed row-3 term equal to row 2's would still pass. ORDER BY eql_v4.ord_term(col) over Go's bytes is never run.
The extra assertions are free. I ran them by hand against the committed fixtures on a local EQL v4 install: ORDER BY eql_v4.ord_term(col), id gives [1, 2, 3] in every column, > gives [3] and >= gives [2, 3]. = gives [2] on the number columns and [] on title/handle, because = reads the case-sensitive hm and row 2 is mango/Carol against Mango/carol. To cover them, derive an above set from the plaintexts the same way below is derived, assert it for > (and its complement for <=), and assert that SELECT id FROM go_rankings ORDER BY eql_v4.ord_term({column}), id returns the ids sorted by plaintext. That would make this test hold the commit message's claim that Go's rows order in PostgreSQL, rather than just split around one probe.
| | `stored_v4_rows_v1.json` | copied from `v4_rows.json` at the next build | **never: frozen** | | ||
|
|
||
| `v4_ordered_queries.json` and `v4_ordered_rows.json` are not frozen: their | ||
| `op` bytes are the order terms of the base build, so a change that moves |
There was a problem hiding this comment.
Nit: "the order terms of the base build" will be unclear once this merges, because a reader of main has no base build to look at. Something like "the order terms Stack Encrypt writes today" would say the same thing.
fa4652a to
c1bdf66
Compare
This branch has not been deployed
1 participant
Summary
Go developers can now store numbers and text in EQL columns they can sort and range-filter:
IntegerOrd,BigintOrd,RealOrd,DoubleOrd,TextOrdand theirOrdOpetwins. EQL is the SQL library we install into a customer's PostgreSQL database to store and search encrypted values. Stack Encrypt is our Rust encryption engine, which the Go SDK runs as a WebAssembly guest. These types store an OPE term (order-preserving encryption: an encrypted value whose bytes sort in the same order as the plaintexts), and EQL compares it as plain bytes. So<,<=,>,>=andORDER BY eql_v4.ord_term(col)work on them. #1235 and #1242 already made the engine derive these terms over ADR-0002's order keys. What was missing was the EQL side: a list in the code generator, and the step that writes the engine's OPE term into EQL'sopfield.Stacked on #1242 (which is stacked on #1235). Review those first.
Changes
eql-codegen and eql-bindings
ENCRYPTION_DOMAINS. Their v4 structs deriveEncryptFromandDecryptInto, and the by-name dispatch handles them. That takes the producible types from 12 of 51 to 22.OpeCllw(theopterm) implementsEncryptFromfor any orderable plaintext. It writes the hex of Stack Encrypt's OPE term.TextSearchstays unproducible with a new reason. Its query value has to carry a match term, and text shorter than one 3-gram can't make one. So=or<on short text would be refused. How a query that isn't a match leaves that term out still needs deciding (stashgen refuses TextMatch, TextSearch, TextOrd and TextOrdOpe — the engine derives their terms, but no EQL type is built from them #1131).v4/{integer,bigint,real,double,text,targets}.rsand Go'sencrypt/eql/eql_gen.go..changeset/eql-v4-ordered-ope-types.md(@cipherstash/eqlminor), is kept separate from the sharedeql-v4-bindings.mdso this PR and feat(eql-bindings): produce Smallint and SmallintEq from i16 #1267 don't conflict.Tests (
packages/eql/tests/encryption, newtests/v4_order.rs)The stored terms sort as the order keys do:
float8sorts them:-0.0ties0.0, and every NaN is one value after+Infinity;f32orders as thef64it widens to;Each query value carries the terms the stored value holds. A family's
OrdandOrdOpestore one term for one value. Text with no fold (an unassigned code point) is refused on both paths.PostgreSQL, against the v4 bundle built from the tree. On every new column,
<,<=,>,>=,=, the query on the left of the comparison,ORDER BY eql_v4.ord_term(col)andeql_v4.maxeach return exactly the expected rows:=reads the order term, so-0.0 = 0.0;=readsTextEq's equality term, soCaféandcafetie in order but aren't=.test:encryption:postgresnow also runsv4_order.New fixtures:
v4_ordered_queries.json: one query per type, derived in Rust and read by Go;v4_ordered_rows.json: rows sealed by Go, then opened and compared in Rust and PostgreSQL.v4_rows.json,stored_v4_rows_v1.jsonand theProfilemodel are untouched. A separateRankingmodel holds the ordered fields.Go
Lt,Lte,GtandGteon these fields, plusEqon the text ones. The guest already answered these through each type's query value, so the generator needed no change.Rankingtest model.eql_v4_order_test.gochecks:IntegerOrdOre, which is still unproducible.eql_v4.ord_term(col)with a plain btree),doc.go, the stashgen README and the CHANGELOG.Verification
packages/eql:mise run build,test:codegen,test:crates,test:encryption(432 passed),types:check(on the commit) andcodegen:parityall passed.mise run test:encryption:postgrespassed against a disposable PostgreSQL 17.5 (initdbcluster) with the v4 bundle built from the tree:text_eq1,v4_types3 andv4_order2 tests.cargo clippy -p eql-bindings -p eql-codegen -p eql-encryption-tests --all-features --all-targets -- -D warningsand eql-bindings rustdoc with-D warningswere clean, under--all-featuresand understack-encryptalone.cargo publish --dry-run --all-features -p eql-bindingscan't resolvestack-encrypt ^0.3.0from crates.io. That's the same as on the base, and CI reports it as a warning.mise run wasm:guest:testpassed (68 and 71).languages/golang:mise run go:testpassed,go vet ./...was clean andmise run go:lintreported 0 issues.go generate ./...leaves no diff.stored_rows_v1.jsonand the jobs that depend on it, by design.) Nothing here touches that fixture or its test.Related
Refs #1131: makes
TextOrdandTextOrdOpeproducible and checks step 1 for OPE (EQL's SQL compares the engine's OPE terms correctly).TextSearchis left with its reason.Refs #1130: the OPE-ordered number types.
Refs #1132: the
*OrdOretypes still wait on block ORE.Review notes
OpeCllw'sEncryptFromincrates/eql-bindings/src/encryption.rs, andencryption_gapincrates/eql-codegen/src/bindings.rs. The rest is generated or tests.Answersimpls only for single-index types, and its generator panics on an order index. Whichever of feat(eql-bindings): query EQL v4 types through Stack Encrypt's typed query chain #1244 and this PR merges second must extend that rule toope(Lt/Lte/Gt/Gte, andEqforhm+ope).Between. Once both are in, a follow-up builds the{v, i, lo, hi}range operand for these types and stashgen getsBetween. Until then the engine doesn't listbetweenfor a target, so nothing generates it.encryption_gap, an indexing note in the Go README, a fixtures README note that the two ordered fixtures aren't frozen and regenerate together, and a separate changeset.EQL_UPDATE_FIXTURES=1: Go'sTestV4OrderedRows…writes the rows and Rust'sv4_orderwrites the queries. Otherwise four cross-language tests fail at once.stored_rowsquestion. Once those bytes are final,stored_v4_rows_v2.json(or astored_v4_ordered_rows_v1.json) should freeze them.Eqmethods on the numberOrdtypes. SQL's=works on them through the order term, but the engine answersEqonly through an equality index.Smallintfamily.TextSearchand the*OrdOretypes.