Repository navigation
json_tree: the smaller performance findings from the first benchmarks #1324
Copy link
Copy link
Open
Labels
Description
Summary
The json_tree family (EQL v4's encrypted JSON document type, written by Stack Encrypt's JSON index) was benchmarked at 10,000 rows against the v3 SteVec type and native jsonb (cipherstash/benches, branch feat/json-tree-benches, report/JSON_TREE_COMPARISON.md). The three large findings, the containment projection being recomputed per recheck, the subtree walk rescanning the document per node, and the range comparison losing its index under a prepared statement, are fixed in #1325, stacked on #1288. Four smaller findings remain and are collected here so they are not lost.
What is left
->scans every entry on every extraction.eql_v4.json_tree_path_query_first(doc, selector)runsjsonb_array_elements(doc -> 'sv')and filters ons. That is 0.18 ms on a five-node document and 0.79 ms on a 220-node one, per row. An index of the entries by selector, stored besidesv(a{selector: [ordinal, …]}object, asqnow is for containment) or built as a jsonb lookup, would make it one->and one array read.json_tree_containsalways plans a second GIN probe foralt. The body isq @> (b - 'alt') OR (b ? 'alt' AND q @> (b -> 'alt')), and the planner emits aBitmapOrof two index probes even when the needle carries noalt(every needle whose root is a container). The second probe is cheap but visible in every plan; a form that lets the planner drop it whenaltis absent, or a separate operator for scalar-root needles, would remove it.ORDER BY eql_v4.ord_term(col -> sel) LIMIT 10measured 0.92 ms against the v3 SteVec's 0.25 ms, both index scans on the functional btree. The plan reads 96 buffers for ten rows; the difference is not explained yet and is worth a look together with item 1, since the index scan re-extracts through->for the output.- A nested document is 24 times its plaintext (96,596 characters against 4,034 for the benchmark's 220-node shape): about 440 characters per entry, from six 32-character hex tags, a base64 AES-GCM ciphertext with a 12-byte nonce and 16-byte tag per node, and the sealed root's copy of the whole document. The design document's open decisions 1 (base64url instead of hex for the sixteen-byte fields, about 40 characters per entry) and 6, and the per-path policy with opaque subtrees, are the levers. The 100,000-row nested table is 8 GB on disk at this size.
Numbers, 10,000 rows
| scenario | v4 json_tree | v3 SteVec | plaintext jsonb |
|---|---|---|---|
-> by id, flat document |
0.18 ms | — | 0.10 ms |
-> by id, nested document |
0.79 ms | — | 0.10 ms |
ORDER BY ord_term(col -> sel) LIMIT 10, flat |
0.92 ms | 0.25 ms | — |
| nested document size | 96,596 chars | — | 4,034 chars |
Related
Refs #1288 and #1325. Design: docs/plans/2026-10-17-json-index-stack-encrypt-spike.md, "Implementation notes (2026-10-21, Phase 3)".
Reactions are currently unavailable
Activity