Skip to content

feat(eql-bindings): produce Smallint and SmallintEq from i16 #1267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
coderdan merged 2 commits into from
Oct 19, 2026
Merged

feat(eql-bindings): produce Smallint and SmallintEq from i16 #1267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d5184c2
feat(eql-bindings): produce Smallint and SmallintEq from i16
coderdan Oct 18, 2026
337e882
test(golang,eql): seal the smallint EQL v4 types from Go, and store G…
coderdan Oct 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/eql-v4-smallint.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@cipherstash/eql': minor
---

**Stack Encrypt produces the EQL v4 types `Smallint` and `SmallintEq`** (`eql-bindings`, `stack-encrypt` feature), from `i16`. `SmallintEq`'s equality term is the integer one, so a smallint and an integer of the same value under one column share it. The `date`, `timestamp` and `numeric` types now say they wait for a choice of Rust plaintext type, not for an engine kind.
18 changes: 11 additions & 7 deletions .github/workflows/tests-golang.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,17 +248,21 @@ jobs:
echo "RLIMIT_MEMLOCK: $(ulimit -l) KiB"
mise run go:test

# The Rust suite opens the committed v4_rows.json, whose ciphertexts the
# Go SDK sealed when the file was last regenerated. This also hands it
# the rows the Go SDK seals in this run, so a Go guest and a native
# Rust build that stop agreeing on today's ciphertexts fail here rather
# than when someone next regenerates the file. The toolchain is the
# The Rust suite opens the committed v4_rows.json and
# v4_smallint_rows.json, whose ciphertexts the Go SDK sealed when each
# file was last regenerated. This also hands it the rows the Go SDK
# seals in this run, so a Go guest and a native Rust build that stop
# agreeing on today's ciphertexts fail here rather than when someone
# next regenerates a file. Both filters are exact, so each test named
# here is one that reads the rows handed to it. The toolchain is the
# root's pin, as for the guests.
- name: Rows the Go SDK seals today open in Rust
run: |
rows="$RUNNER_TEMP/v4_rows.json"
(cd languages/golang && CGO_ENABLED=0 EQL_V4_ROWS_OUT="$rows" go test -count=1 -run TestV4RowsMatchTheCommittedFixture ./encrypt)
EQL_V4_ROWS="$rows" cargo test --locked --manifest-path packages/eql/Cargo.toml -p eql-encryption-tests --test v4_types rows_the_go_sdk_sealed_open_in_rust
smallint_rows="$RUNNER_TEMP/v4_smallint_rows.json"
(cd languages/golang && CGO_ENABLED=0 EQL_V4_ROWS_OUT="$rows" EQL_V4_SMALLINT_ROWS_OUT="$smallint_rows" go test -count=1 -run '^(TestV4RowsMatchTheCommittedFixture|TestV4SmallintRowsMatchTheCommittedFixture)$' ./encrypt)
test -s "$rows" && test -s "$smallint_rows"

@coderdan coderdan Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this line only enforces its second half. shell: bash runs as bash -eo pipefail, and errexit ignores a failing command that isn't the last one in an && list. So if $rows is missing or empty, && stops before the second test, the list fails, and the step keeps going to cargo.

The realistic way to get there is the deterministic EQL guest not being built. deterministicEQLClient then calls t.Skip in both Go tests, go test prints ok, and neither file gets written. That case still fails loudly, but in the wrong place. cargo panics in both tests with fixtures/v4_rows.json is written by the Go SDK's TestV4RowsMatchTheCommittedFixture: ... NotFound (I reproduced the smallint version by pointing the variable at a path that doesn't exist). The message names the committed fixture, which is present, not the $RUNNER_TEMP file that is missing. That sends whoever reads the log to the wrong file.

Splitting it into two lines makes both checks stop the step:

test -s "$rows"
test -s "$smallint_rows"

EQL_V4_ROWS="$rows" EQL_V4_SMALLINT_ROWS="$smallint_rows" cargo test --locked --manifest-path packages/eql/Cargo.toml -p eql-encryption-tests --test v4_types -- --exact rows_the_go_sdk_sealed_open_in_rust smallint_rows_the_go_sdk_sealed_open_in_rust

@coderdan coderdan Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low: --exact makes this step stricter about which tests run, but it doesn't check that they ran. If either test is renamed (say smallint_rows_the_go_sdk_sealed_open_in_rust becomes ..._open_at_16_bits), libtest matches nothing for that name, runs the other test alone, and exits 0. I checked this locally with one name misspelled: 1 passed; 0 failed; 18 filtered out, cargo exit 0. The hand-off this PR adds would then be gone, and CI would stay green. The old substring filter had the same weakness, but --exact makes it break on any rename.

One way to make it fail loudly is to count before running:

filters=(--exact rows_the_go_sdk_sealed_open_in_rust smallint_rows_the_go_sdk_sealed_open_in_rust)
test "$(cargo test --locked --manifest-path packages/eql/Cargo.toml -p eql-encryption-tests --test v4_types -- "${filters[@]}" --list --format terse | grep -c ': test$')" -eq 2
EQL_V4_ROWS="$rows" EQL_V4_SMALLINT_ROWS="$smallint_rows" cargo test ... -- "${filters[@]}"

A small test under scripts/__tests__/ that checks each name in this step against a fn in tests/v4_types.rs would also do it, and wouldn't touch the workflow.


# What is encrypted is fixed before the program ships: every generated
# file is committed, and this fails when `go generate` would change
Expand Down
4 changes: 4 additions & 0 deletions languages/golang/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,10 @@ listed under **Breaking**. Releases are tagged `languages/golang/vX.Y.Z`.
`float32` or `float64` field tagged `encrypt_into=` one of them gets `Eq`.
`=` compares as PostgreSQL compares the plain types: an integer has one
term whatever its width, and for floats `-0.0 = 0.0` and `NaN = NaN`.
- **`Smallint` and `SmallintEq`**: an `int16` field (or any Go integer
type, range checked as for `Integer`) tagged `encrypt_into=` one of them
seals at 16 bits; `SmallintEq` gets `Eq`, with the same equality term as
an integer of the same value under the same column.
- **stashgen notices an index no generated method queries**: an `ore` or
`ope` index on text, bytes or a float (the engine answers the orderings
over integers only), or the second of two order indexes on an integer.
Expand Down
3 changes: 2 additions & 1 deletion languages/golang/cmd/stashgen/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,13 +159,14 @@ The engine produces these EQL types today, each from one Go field type:
| `encrypt_into` | Go field |
|---|---|
| `TextEq`, `Text`, `TextMatch` | `string` |
| `Smallint`, `SmallintEq` | `int16`, or any Go integer type |
| `Integer`, `IntegerEq` | `int32`, or any Go integer type |
| `Bigint`, `BigintEq` | `int64`, or any Go integer type |
| `Boolean` | `bool` |
| `Real`, `RealEq` | `float32` |
| `Double`, `DoubleEq` | `float64` |

The `Eq` types answer `Eq`; `Text`, `Integer`, `Bigint`, `Boolean`, `Real` and `Double` are stored and read only, with no query method.
The `Eq` types answer `Eq`; `Text`, `Smallint`, `Integer`, `Bigint`, `Boolean`, `Real` and `Double` are stored and read only, with no query method.

Any built-in Go integer field takes an integer type (ADR-0002) and seals at the type's width: an `int16` field always fits an `Integer`, and an `int64` field is range checked value by value, `Encrypt` refusing a value that does not fit with `encrypt.ErrEncoding`. `encrypt.Int128` and `encrypt.Uint128` are not converted. A float's width must match: a `float64` field is not a `Real`.
A `TextMatch` field refuses text shorter than three characters, such as an empty string or a two-letter code, because it has no 3-grams to index; its query refuses such text too.
Expand Down
5 changes: 4 additions & 1 deletion languages/golang/encrypt/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,8 @@ The EQL types the engine produces, and the Go field types each takes:
| `TextEq` | `string` | `=`, `<>` |
| `TextMatch` | `string` | `@@` (substring match, 3-grams, case-insensitive) |
| `Text` | `string` | none: stored and read only |
| `Smallint` | `int16`, or any Go integer type (see below) | none |
| `SmallintEq` | the same | `=`, `<>` |
| `Integer` | `int32`, or any Go integer type (see below) | none |
| `IntegerEq` | the same | `=`, `<>` |
| `Bigint` | `int64`, or any Go integer type | none |
Expand All @@ -446,7 +448,8 @@ The EQL types the engine produces, and the Go field types each takes:

Any built-in Go integer field takes an integer EQL type (ADR-0002): the field
is sealed at the type's width, and opens back into the field's type. A type
every value of which fits (an `int16` or a `uint16` field for `Integer`)
every value of which fits (an `int16` or a `uint16` field for `Integer`, an
`int8` for `Smallint`)
always seals; a wider one (an `int64` or a `uint32` field for `Integer`) is
range checked value by value, and `Encrypt` refuses a value that does not fit
with `ErrEncoding`, naming the field's kind and not the value.
Expand Down
11 changes: 6 additions & 5 deletions languages/golang/encrypt/doc.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,15 @@
// }
//
// encrypt_into names the EQL type the field is stored as. The Go field type
// must match it: TextEq, TextMatch and Text take a string, Integer and
// IntegerEq an int32, Bigint and BigintEq an int64, Boolean a bool, Real and
// RealEq a float32, and Double and DoubleEq a float64. An integer EQL type
// must match it: TextEq, TextMatch and Text take a string, Smallint and
// SmallintEq an int16, Integer and IntegerEq an int32, Bigint and BigintEq
// an int64, Boolean a bool, Real and RealEq a float32, and Double and
// DoubleEq a float64. An integer EQL type
// also takes any other built-in Go integer type, sealed at the type's width:
// one every value of which fits it (an int16 into Integer) always seals, and
// a wider one (an int64 into Integer) is range checked, a value that does not
// fit refused with [ErrEncoding]. The *Eq types answer Eq; Integer, Bigint,
// Boolean, Real, Double and Text are stored and read only.
// fit refused with [ErrEncoding]. The *Eq types answer Eq; Smallint,
// Integer, Bigint, Boolean, Real, Double and Text are stored and read only.
//
// 3. Run the generator. It writes user_stash.go beside the struct:
//
Expand Down
Loading
Loading