Repository navigation
Hot spots: narrow-leg metadata search, stale-while-revalidate PR titles - #1397
Conversation
Metadata search with several filters (the CLI's staged-branch lookup filters on gh.repo + gh.branch + gh.kind) intersected one leg per filter. Without stats SQLite walked the whole workspace on the PK to satisfy ORDER BY object_key: 14,000 to 45,000 rows read per call. It now walks the narrowest filter on file_metadata_lookup_idx (INDEXED BY) and checks the others with a PK lookup per row. On production data: 45,137 rows / 130 ms before, 22 rows / 0.4 ms after, same results. No new index. /pulls waited up to 1.4 s for PR titles whenever the KV entry for an open PR had expired. When every row already has a stored title it now waits only 250 ms (enough for cache hits), answers with stored titles, and lets the full resolve finish after the response, backfilling the rows. Before, a resolve that missed the budget was dropped and never backfilled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeLbkis33mPMcBF5kRguaM
|
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one): 2
🚫 Excluded labels (none allowed): 1
Please check the settings in the CodeRabbit UI or the
 |
Requested by Zach · project thread
This is the follow-up to #1395 from the same query audit. It covers rows 3 and 4 of the audit's hot spot table.
Metadata search with several filters
findObjectsByMetadatawith two or more filters intersected one subquery per filter. Without table statistics, SQLite satisfiedORDER BY object_keyby walking the whole workspace on the primary key. D1 query insights showed 2,306 calls in a week reading 14,000 to 32,000 rows each. Most came from the CLI's staged-branch lookup, which filters ongh.repo,gh.branchandgh.kind.It now walks one filter on
file_metadata_lookup_idx, pinned withINDEXED BY. That filter is the first one not inBROAD_META_KEYS(gh.kind,gh.repo,gh.status,gh.merged). Every other filter is anEXISTSwith a primary-key lookup per row. The prefix, collapse andafterwrappers are unchanged.I ran the old and new SQL read-only on production D1 for one recent branch, with repo + branch + kind:
There is no new index.
file_metadata_lookup_idxalready exists in production; I checkedsqlite_master. The single-filter form is unchanged.PR titles on
/pulls/pullswaited up to 1.4 s (the public title budget) whenever an open PR's KV title entry, which lasts 1 hour, had expired. The repo page's PR list took 1.6 to 1.9 s today.storedTitleFor), the handler now waits only 250 ms. That is enough time for KV hits. It then answers with the stored titles.afterResponse) and backfillsgithub_pr_activity. Before, a resolve that ran past the budget was dropped and never backfilled, so the same rows stayed slow.Testing
pr-scope-sqlite: a multi-filter search with the broad keys listed first still drives fromgh.branch.EXPLAIN QUERY PLANshowsfile_metadata_lookup_idx (workspace=? AND meta_key=? AND meta_value=?)plus primary-key lookups. Prefix andaftergive correct results.routes-workspace-scope: with a stored title and a title lookup slowed to 600 ms,/pullsanswers in under 550 ms with the stored title. The row is then backfilled with the new title and state. On the old code this test fails at 609 ms.FileMetadataTable, mcp worker) now accept aliasedmeta_key/meta_valuepairs.pnpm test(7913 passed),pnpm typecheck,pnpm check.Generated by Claude Code