Call deserialize through invokelatest - #367
Conversation
Every query compiles a path into Serialization.deserialize for BLOB columns. That path is easily invalidated (e.g. by loading LibGit2), which then forces recompiling query code even when no BLOB is ever read. A dynamic call cuts the dependency; deserializing is already slow, so the dispatch cost doesn't matter.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #367 +/- ##
==========================================
+ Coverage 94.13% 96.76% +2.62%
==========================================
Files 4 5 +1
Lines 563 773 +210
==========================================
+ Hits 530 748 +218
+ Misses 33 25 -8 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
quinnj
left a comment
quinnj
left a comment
There was a problem hiding this comment.
Confirmed the invalidation cause and the improvement at ede6a3a against base 3dc2152 on Julia 1.12.6 (macOS ARM64), with identical resolved dependencies. In two fresh-process runs per revision, warming a scalar SELECT and then loading LibGit2 invalidated five SQLite method instances on the base, including sqldeserialize, sqlitevalue, juliatype, and execute. The PR invalidated none of those methods. The next query spent 188–197 ms recompiling and allocated about 31.9 MB on the base; on this head it had zero measured compilation and took 114–136 μs with 5,792 bytes allocated.
Steady-state scalar-query medians stayed around 4.3–4.4 μs, with typical allocations of 1,776 bytes per query. Alternating warmed direct/invokelatest runs on the same small serialized NamedTuple were about 2.4–2.6 μs with 1,680 bytes in both paths; this measurement supports the latency fix without claiming a general steady-state speedup. Raw byte BLOBs and serialized values round-trip correctly. The shared helper also covers aggregate-state and UDF deserialization.
Local Pkg.test(): 802 checks passed, with two existing marked-broken checks. All nine current PR checks are green, including Julia 1.9, nightly, Windows, macOS, and the native statement core on Julia 1.13. No storage-format or exported-API change.
AI disclosure: This work was prepared with assistance from OpenAI Codex.
2 participants
Every
DBInterface.executecompiles a path intoSerialization.deserialize(for possible BLOB columns), even if the query never returns a BLOB. That path gets invalidated easily — loading LibGit2 (e.g. via Pkg) is enough — and then all precompiled query code has to be recompiled on first use.Calling
deserializethroughinvokelatestremoves that dependency. Deserialization is already slow, so the extra dynamic dispatch is negligible.Tests pass locally.