Skip to content

EQL v4 has no type for 128-bit integers, and block ORE cannot order them yet #1251

Open

Description

@coderdan

Background

EQL (Encrypt Query Language) is the SQL library we install into a customer's PostgreSQL database. It defines the column types (PostgreSQL domains, such as eql_v4_integer_eq) that hold an encrypted value together with its search terms. Search terms let the database compare values without decrypting them. Equality uses an HMAC term. Ordering (<, >, BETWEEN, ORDER BY) uses ORE (order-revealing encryption) or OPE (order-preserving encryption). EQL v4 is the version Stack Encrypt produces: Stack Encrypt is CipherStash's Rust encryption engine, which the Go SDK runs as a WebAssembly guest.

Stack Encrypt's value model now holds 128-bit integers (i128 and u128, ADR-0002), and the Go SDK can encrypt them (encrypt.Int128 and encrypt.Uint128, #1144 / #1246). EQL v4 cannot store or search them yet.

Problem

A 128-bit integer can be encrypted, but there is no EQL v4 type to store it in, and nothing can order it. In detail:

  1. No EQL v4 type takes a 128-bit integer. The catalog's integer families stop at bigint (smallint, integer, bigint). PostgreSQL has no native 128-bit integer type, so there is no obvious type to model one on. (Verified: the families in languages/golang/encrypt/eql/eql_gen.go.)
  2. Block ORE cannot order a 128-bit integer today. OreEncrypt for u128/i128 panics on OreAes128 (never worked, no tests) ore.rs#105: OreEncrypt for u128/i128 panics on every call under the legacy OreAes128 scheme, because it takes at most 15 blocks and a 128-bit integer needs 16 bytes. The six-bit scheme (Bit6) caps at 14 blocks, which is enough for 64-bit types only. feat(chained): encrypt u128/i128 on the chained scheme; drop the panicking legacy impls ore.rs#113 fixes this by encrypting u128/i128 on the chained scheme (22 six-bit blocks; a full ciphertext is 570 bytes and a left-only ciphertext 378 bytes). That is the ore.rs code Stack Encrypt will target for block ORE.
  3. The Go SDK refuses them on purpose. stashgen refuses a built-in Go integer stored as a 128-bit kind when it generates code (languages/golang/stashgen/engine.go:406, on the feat(stack-encrypt,golang): give every Go integer width, dates, timestamps, decimals and 128-bit integers their own kinds #1246 branch), and gensupport converts built-in integers to 64 bits or fewer only (languages/golang/encrypt/gensupport/widen.go:99). Without a 128-bit EQL type there is nothing to convert to, so these refusals are correct for now. They must change once a type exists.

Impact: a customer with 128-bit identifiers or counters (u128 in Rust, encrypt.Uint128 in Go) can encrypt the value but cannot put it in an EQL v4 column, and cannot search it by equality or range. Nothing fails loudly today only because no producible EQL type names a 128-bit kind.

Proposal

  1. Decide the EQL v4 shape for 128-bit integers. Decide whether they get their own families (for example, signed and unsigned, each with Eq, Ord and OrdOre variants) or ride in an existing one. Decide which PostgreSQL type the domains are built on. Record the decision in an ADR or an amendment to ADR-0002.
  2. Equality. Use ADR-0002's integer equality term, which is the same at every width, so a 128-bit value equal to a 64-bit value gets the same term. Check that this holds for values outside the 64-bit range, and pin the result with fixtures.
  3. Block ORE. Once feat(chained): encrypt u128/i128 on the chained scheme; drop the panicking legacy impls ore.rs#113 is released and Stack Encrypt moves onto it (alongside stashgen refuses TextOrdOre and TextSearchOre — EQL stores block ORE terms and the engine derives CLLW ORE #1132), derive u128/i128 block ORE terms on the chained scheme. Pin the term bytes with fixtures shared by Rust and Go. Note the ore.rs PR's caveat: 0u128 and i128::MIN encrypt to the same bytes under one key, so signed and unsigned 128-bit values must never share a column or a comparison.
  4. CLLW ORE and OPE. Confirm that the CLLW ORE and OPE order terms Stack Encrypt derives for i128/u128 (packages/stack-encrypt/src/dynamic/term.rs:460-493 on the feat(stack-encrypt,golang): give every Go integer width, dates, timestamps, decimals and 128-bit integers their own kinds #1246 branch) follow ADR-0002's order keys, as Stack Encrypt cannot order floats, dates, timestamps or decimals, and its integer order keys predate ADR-0002: derive every order term over vitaminc-ore #1236 requires for the other widths.
  5. eql-bindings and the Go SDK. Generate the new types in eql-bindings, mark them producible, and remove stashgen's 128-bit refusal. Add 128-bit arms to gensupport.integerAs so a built-in integer field can be stored as a 128-bit type, and round-trip a value outside the 64-bit range in the Go tests.

Relationship to other work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions