Repository navigation
stashgen refuses the storage-only Text type — it is held back until TextEq is proven end to end #1133
Description
Background
EQL (Encrypt Query Language) is the SQL library we install into a customer's PostgreSQL database. It stores encrypted values and lets the database search them without decrypting. Each EQL type (for example TextEq, IntegerOrd, TextMatch) fixes what a column stores: the ciphertext, plus the search terms that type needs. A search term is a small encrypted value the database can compare, such as an HMAC (a keyed hash, used for equality) or an ORE term (order-revealing encryption, used for < and >).
Stack Encrypt (packages/stack-encrypt) is our Rust encryption engine. Since #1095, a plan field can name an EQL type as its target: the engine then writes the exact value that EQL type stores. The Go SDK uses this through its WASI guest (a WebAssembly build of the engine) and the stashgen code generator.
A target type is producible when the engine can write it. The engine runs the type's own Rust plan, which is the EncryptFrom derive on the type's generated struct. eql-codegen adds that derive only to the types listed in ENCRYPTION_DOMAINS (packages/eql/crates/eql-codegen/src/bindings.rs:316), and today that list is only ("text", "eq"). So of the 51 EQL types, only TextEq is producible. The generated table packages/eql/crates/eql-bindings/src/v3/targets.rs records a reason for each of the other 50, and stashgen refuses them with that reason. #1062 scoped this on purpose to TextEq only. This issue tracks one group of the remaining types.
Only EQL v4 types derive Stack Encrypt's traits. ADR-0002 (#1139) installs the EQL SQL twice: eql_v3 for payloads written by cipherstash-client (the engine behind the TypeScript SDK), and eql_v4 for payloads written by Stack Encrypt. Stack Encrypt's output is not byte-for-byte compatible with cipherstash-client's, so deriving EncryptFrom or DecryptInto on an eql_v3 struct would claim a compatibility that doesn't exist. Every type in this issue becomes producible as its eql_v4_* struct, and no eql_v3_* struct derives either trait. Today's v3 TextEq and TextEqQuery derives move to v4 in #1142. File paths above name the current v3 modules; the v4 ones are generated in #1142.
Problem
The storage-only text type Text cannot be written by the engine, although it is the simplest type of all: it stores a ciphertext and no search terms (encrypted at rest, never searched in the database).
The generated table refuses it with: "the storage-only text domain follows once TextEq is proven end to end in PostgreSQL". So this is a deliberate hold-back, not a missing feature: Text needs only the ciphertext that TextEq already writes.
Impact: a Go developer who wants an encrypted text column with no search (notes, free text, anything never queried) must use TextEq instead. That stores an equality HMAC the column never needs, which costs space and reveals which rows hold equal values.
Proposal
- Confirm the condition the hold-back names: the v4
TextEq(after@cipherstash/eql4.0: ship the v3 and v4 bundles together and move Stack Encrypt's targets to v4 #1142) round-trips end to end in PostgreSQL (written from Go through the guest, queried with EQL, read back). Point to the test or CI job that shows it. - Add
("text", "")toENCRYPTION_DOMAINSand regenerate, so the v4Textstruct gets theEncryptFromandDecryptIntoderives andproducible: true. The v3Textstruct stays without them. - Add a round-trip test for
Textbeside theTextEqone: written from Go, stored with no terms, read back.
This is the smallest of the producibility issues and could ship first.
Relationship to other work
- Go and Rust plans cannot target EQL domains, so Go writes rows EQL cannot query — make domains plan field targets #1062: made EQL types plan targets, with
TextEqonly. - stashgen refuses TextMatch, TextSearch, TextOrd and TextOrdOpe — the engine derives their terms, but no EQL type is built from them #1131 (match and OPE text types), stashgen refuses TextOrdOre and TextSearchOre — EQL stores block ORE terms and the engine derives CLLW ORE #1132 (block ORE vs CLLW ORE) and stashgen refuses every number, date and boolean EQL type — produce them in EQL v4 using ADR-0002's encodings #1130 (non-text families) cover the other types.
Activity