From f226c7d7410114ce92cf154eb5c03259b21a5678 Mon Sep 17 00:00:00 2001 From: Aaron Queen Date: Mon, 12 Oct 2026 09:08:42 -0600 Subject: [PATCH 1/4] docs: generalize private product vocabulary and owner-host wording Two explore evidence reports named a private downstream application's components, case names and commit hashes; they now describe the fixture generically. The benchmark setup sections and the strategy note described the maintainer's own tooling and machine; they now state the actual condition (a 16 GiB memory cap, no other heavy job running). The duplicated landing guidance at the top of AGENTS.md is reduced to the branch line. Three tracked Python bytecode files are removed and __pycache__/ and *.pyc are ignored. No measured number, query text or conclusion changes. --- .gitignore | 3 ++- AGENTS.md | 10 +--------- .../generate-waitlist.cpython-313.pyc | Bin 9768 -> 0 bytes docs/benchmarks/explore-requested-evidence.md | 18 +++++++++--------- .../explore-selector-cache-evidence.md | 6 +++--- .../benchmarks/fork-vs-upstream-2026-09-26.md | 7 +++---- .../benchmarks/fork-vs-upstream-2026-09-28.md | 8 ++++---- .../fork-vs-upstream-node-bun-2026-09-28.md | 4 ++-- .../fork-vs-upstream-node-bun-2026-10-03.md | 4 ++-- .../fork-vs-upstream-node-bun-2026-10-12.md | 11 +++++------ docs/design/strategy-2026-09.md | 16 ++++++++-------- .../pick-g1-commit.cpython-314.pyc | Bin 3319 -> 0 bytes .../replay-repowise-g1.cpython-314.pyc | Bin 9175 -> 0 bytes 13 files changed, 39 insertions(+), 48 deletions(-) delete mode 100644 assets/__pycache__/generate-waitlist.cpython-313.pyc delete mode 100644 scripts/precision/__pycache__/pick-g1-commit.cpython-314.pyc delete mode 100644 scripts/precision/__pycache__/replay-repowise-g1.cpython-314.pyc diff --git a/.gitignore b/.gitignore index 4ed3edc198..db4cbeffb2 100644 --- a/.gitignore +++ b/.gitignore @@ -68,7 +68,8 @@ release/ # Local-only: browser-based tmux session launcher (see tmux-web/README.md) tmux-web/ -assets/__pycache__/ +__pycache__/ +*.pyc assets/generate-waitlist.py diff --git a/AGENTS.md b/AGENTS.md index 55cadb3ec8..1f9644ab0d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,15 +4,7 @@ Canonical project guidance for coding agents working in this repository (Codex/A **Instruction budget:** Keep this root file below 32,768 UTF-8 bytes; `npm run check:agent-docs` enforces the limit. Put conditional procedures, evidence, and worked examples in linked documents. Codex user configuration may set `project_doc_max_bytes = 65536` as a safety margin, but the larger limit does not replace the repository guard. -**Completion and branch boundary:** - -- This repository's default integration branch is `fork/consolidated`. -- A worktree or other feature branch is intermediate. -- Before committing or integrating, confirm the checkout is this repository and that its integration target is `fork/consolidated` (worktrees included). Never commit or merge this repository's work into another repository or its default branch (`main` included), and never merge another repository's work here. -- For authorized repository work, finish by committing the verified change, integrating it through any required checks or PR into `fork/consolidated`, pushing, and confirming that its remote head contains the commit. -- Stop earlier only when the user requests it or integration is blocked; report the exact blocker. -- Do not run `npm publish` without explicit authorization. -- Fork GitHub release policy is defined under Releases below. +**Branch:** this repository's default integration branch is `fork/consolidated`, and pull requests go to the fork, never to the upstream repository or `main`. Do not run `npm publish` without explicit authorization; fork release policy is under Releases below. **Agent rules:** [House Rules](https://github.com/bompus/house-rules) is an optional, portable rule set that works well with this repository; nothing here requires it. If your agent loads it or similar always-on rules, this file adds to them and wins for this repository's commands, branches and checks. diff --git a/assets/__pycache__/generate-waitlist.cpython-313.pyc b/assets/__pycache__/generate-waitlist.cpython-313.pyc deleted file mode 100644 index ffb2bd54a99e34cf3e7028d1e7c2f6d3326e82fb..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 9768 zcmb_CTTB~Smbd)C#x}NjlLt^ZB*6sRKmvI*knkpt4&cHG-6S}LU4Ro)cB)Jas2}c+ zQNwnx;z&JeXQw}Cb$__kqn)%gA6aQXGRnuKrCG^d_b5y$tsU*=C;LSvNIja5)t-A@ zWt-4+C!@Iuw@#hcJ?GrI=hi*07bPVI1nuqx+x5yyg#HKLC`~H^FRCbneu)?)AckU8 z=P7}LtNOg^oLW%R7^gX}Ij0r01lFD}I;RtK1TH$SKUXXi6IgfNaLy-G2^CCGohfLj&gyy`G}nsLwFPeb7`0sINr|*BJeJ8K}W#P@sWU!4bmJ8upFJ|1ddoa z9Egl8&d|XS>lYPsL0qK4V4KMK7J*NUupnz#2>Cf%3mu&g_~z-EV90kvq|eNT_=sbc z^>MTdteA1&d!jAC&$VECa^|z33CASF2baORV1S(oa)v0ojC~d717Sj)5m+9)Uk-Cv zsguUh35MoE^a3l~pcevy5CW&N7P)5GTLB0e4mkwU&D~rK2%LC|#zsU!p`h48&j=w1 zA5AxfVSoS+g27;(E|cwsA&@wg^+pUW3O>$pixr54E;bu=x+w@z;K6^0G|esrL_0kz zgcgX^K`4N`- zcDZ<%KQBz;VvrMEWa4&l)ioQyEVAPc;|WpVSWDJ4lE zkBXtDim(FQS|L#}>XT|3<29Jc4GZ|?#;d1D`onmS+F?Y9*LYO!l9%~VZrSr%pkoJK zYD!w*i(ou09*@^eVIN8J6y+?|dk|kdRi;qN(a49uYw#GDB9dlo2u*(K(M~}!&Ne0| zi?!lEMu&Tk#sQb9sw@{m4gefrIbZzd5upC07kTX-gJK!h$s4EkDEd4aM#tzpnr1Y4 zr$de_Z<@S;`P};yuKabXR)Hr!R#L8_o5nsFvEv(fN*erlPr$m<1^93fGLNL0=U6}V3y~#>q6PdM+N}b9NF~kC=*0;^gTbQ9 zgJ*|FCM3fmABc!!oG`c`6=SlOIs5jYWSrw7eViESUtq)JW;nPUo@cm-fCD3Gut}0S zB={xmguA!@yr3g!zraQX9DgFyA6i&|{36;*B%N%qWSC(^NTEFEU6xd{5lOumUXZjR zBzUmghes=#!$XOJm1q%)nqee(Op#a$ z%R7di7F#~ie6&B-_e8TV(YIdp>#7vpdcW~MHK#WWrqxSp&5sTBX+wRi|B0zGpfoPD6&u#FPtAAC87uu| z`MUdy>a^9qs(oszjB|pO^LM3X%raQnQPeVy0-JEN?%f!wDzr<$|?fghG6& z$LcBdlpRwu`n~A7o;U11Zh^hV0xly{oE=Ar32$NyKbCA=h%t^)o?b zDZpc$urX#E8KtUF1Hn}9z|;UH|FjYx2q(6uad&OZ9{ham)LDT4Db4gY&3j?2?a|l= zm|Scb?S#C3lo==E?uYp^*br08)G2uPVMTBLs%(c$Jzm{Jl73CevYcdtELq3cUUN){ z2C?>4CG%q*cui~nk$yJ*-G08R+U4{(E{AMTz=FO2(J$#f$;O5#W?^n5uCH|sq*L<*N^Ogu(!$@ zfLqppal2`N9G&0y$tbO3lyXGkOQxEX%3=#2odQ&4@XM-&UB8{EiHF zot`2yI;ziWnL`bz1u22>!`Wa~MWcE+5Yp&kbGN!4%~JN`7gsc+jH_KTi2ZDMgp|7i z|E@wvAe99Gl(NJBo_@9$?zWW|@Tm&FeZFu**P||ci=I$<&;&VF8f*Ps*wjs>FEW=3VBle!U@6)|e_ixIX^nV$yxD z;a=}I2fmuTcQbSNWa`kVl;w?-{te*>2+sY-A8@NVBcT;n7p_JwEd+V7+cqDGgipC# z(P-2eJ?abzbFTK*)>apA*l2Q6?zXKMZ1g+{F2EKWeJj94`$9|IwpO~8Cho25vC%>( z$aUKS3m}I}SWqCE;esGwvh^5fx+?;OP_Ad?beAg&VO)d5Ay+)1VCi3`9S*`OK#7ya z>YW`ndRe~8wmgF;0j=n5G*qLob}&th2%#G=^vVgFi^yZOq&gUZiq)$qU9rlN`0PwC zwSle@7#Xf@HhQTYZj0a+u5&Orv_GF-0n0*A(hEe0eMdW99-0&;Wq;=KCR!D99ZGDE zih73YSfm6&%)?fLnb_QC)j(M(%i6R}4xe&h zva)e<&5>~V0)h|X zD(&Z>%mC$P-?Ckq5nPj@{Vy-xeS_!j6|?n0+<~`Z`qG zmDpmx3*)k~Hy5jLD7Z}G90vXfED(umnVTi8EFT9Px-Ps(dJHS3TyX>LB`Ug76ov9K zN3W=%z$~=mh#TxQspfm5-S&N?TqNMapQM9I5F3dI0xt3i_#i9b%qvkNQc?fN*)i{B zNjug%;JrY~B$9f#cTCdty4|Cf6^dKZV8;YJv660dvhTv^=zwIB0q;Ps`#e@LHt3cN z(wQAtN;lN6UA(Onjf6A2mFPLi~w#z9lVg98<-c>(7* z0dE*WCv+lC&v>Q^+nF1 zD$#k@@VLg2hW{oBeCAiB2;VzB?fD}eeFo1G+n$GSPjP2nYBal-ZiLu{h*<=U0X%S zS{c9mSu|aFC<(?i$3~tQt?`;fKh%&)GsXk4fgdz`jG7k+x|F$l6JPJM_Zb2_Fp$e1 z+8!Msz#}iYq^?e&ZMsSvQ*zYdsLrl6QOshez<_*O$1O@=M#f@ zwQ5t1kt&syCJ-3gaA=)PQlI;h4PTfN?``TZy$Cr^5D4fePu*|$H}$aFY%u(K=I4X6SlS!&siq6UyP{ZN%o1g0wE1O0W zqC)97tx(QT=cxNZ8Bd)D^;qSW8Pv!7*WO;euxY_Vn9-g?1Om!oN0Ryc`uY`x(tbjr zN9Zr>|bmTn(lc`1I9K>PlD^p>9%^too|m?5)Q3R$+TF^t@^}du2Xs zugpm7bw5%)Jn{RU-}Pkrnbd?Q-G3?7cbRa2(n)GM$L*!AQK@S)DIb@ao_#zWOiu?h z(|jrvPEX%VT@_>zN(6(6{u*p>X}o<$fcId7Yp}r>dR~JKZcDm%1X$+72Fr{;VX(|h z43@cn&fu(w3L8u$_U{(qT5RxsY%qqN@82!LGGBKeMHsjCqkdRykJOKPztuisA39UL zKIVk&_R?^ro(h3MoPjs1((Ms&hDVo>yJKmp4@xRnPI=MSM4^8dJSM zXiznd?*>%vtkh;5gG@rM^v{fxpiL0!pl zUB&UAno&qTY9UABsYNOkTvSN6s3Dc2dpPf?>>>F3CO8UrV`E44`j zWXdXQL?!0cqnih@bB)MSO`xqtR8t$Dld-Bga~9i#EETJlH|?0WiK?_@p=M;QUJYy> z!PsV0Q>Q@n4T%W_wlyh`y={F(fg#z-QYYWI7m;DAt2zs{pql;h*-Z!5+k$HA2(;CL z>KoVl*2T|}67jotOTgu%KY)}M- zaYkTQeB@4zG2dQ-eDv`B)X-&_O+86bTiC8x|JE@CzkMBEJAAukt>qKp)99V(r|;i+ z|I4LJb?ZH6#?T9|JjErezWC9P>k{o7dc$q~nm%6samf#AWT<(nDPA3lAB`W3i;1?x zfyDS{Hxs_~y~&~EiIm~kA2l66Xo2gW>;5L<()o8M$~;c>ADsqImrjCrMEvgM_1bNc z&g=Dud|t0`5x67^{>Wkie`Mi|L@oqovOkUP&12yM3P3xvA3XRtA^U^ro;(A*bBg%W zjj`-U4+1_MOZ#4;M+!cJ$o?c+ljm3nh(2eIgZy|ZoCPhyIozGc-38o@;%)?YW4If~ z9eyh(@6rN(^~}DO_hAUXVhRJe8^qlZ?uK!P%L?S}oVlfk^*GhJ8KIEIw7QZ}@A;EP|BInb KHPY-Q{`^0MK7pP9 diff --git a/docs/benchmarks/explore-requested-evidence.md b/docs/benchmarks/explore-requested-evidence.md index 52df4b8500..3cd0464b39 100644 --- a/docs/benchmarks/explore-requested-evidence.md +++ b/docs/benchmarks/explore-requested-evidence.md @@ -12,7 +12,7 @@ Measured 2026-09-10 (America/Denver). This change fixes omissions of requested s ## Controlled comparison -The 16 queries and expected source ranges come from the private reproduction fixture (private checkout at `74fcd4e`). All 13 referenced source files were checked against their recorded SHA-256 hashes at downstream commit `c570e32b22177ab59d3fa8f4c162419bac196422`. +The 16 queries and expected source ranges come from a private reproduction fixture taken from a downstream application. All 13 referenced source files were checked against the SHA-256 hashes recorded for the pinned downstream revision. Three engines queried the same checkout-local index, with a fresh ToolHandler per query and no explicit `maxFiles` override: @@ -30,15 +30,15 @@ Each cell counts returned / expected nonblank source lines across the fixture's | 2 | Named normalization body | 9/9 | 9/9 | 9/9 | | 3 | Prop wiring and table selectors | 0/24 | 0/24 | 2/24 | | 4 | Cache key and consensus evaluation | 52/123 | 52/123 | 73/123 | -| 5 | Draft identity test | 0/31 | 0/31 | 30/31 | +| 5 | Identity-assertion test | 0/31 | 0/31 | 30/31 | | 6 | Named interface | 0/132 | 0/132 | 132/132 | | 7 | Priority stamping body | 0/33 | 12/33 | 33/33 | -| 8 | ADP mappers and secondary test | 97/117 | 97/117 | 97/117 | -| 9 | TTD receiver/storage, prose query | 0/78 | 0/78 | 0/78 | -| 10 | TTD receiver/storage, event query | 0/78 | 0/78 | 0/78 | +| 8 | Mapper bodies and secondary test | 97/117 | 97/117 | 97/117 | +| 9 | Receiver/storage, prose query | 0/78 | 0/78 | 0/78 | +| 10 | Receiver/storage, event query | 0/78 | 0/78 | 0/78 | | 11 | Hover-related cache key | 0/50 | 0/50 | 0/50 | | 12 | Vue cells and column styles | 0/22 | 0/22 | 18/22 | -| 13 | Draft identity test, broad path control | 0/31 | 0/31 | 23/31 | +| 13 | Identity-assertion test, broad path control | 0/31 | 0/31 | 23/31 | | 14 | Focused stamping control | 33/33 | 33/33 | 33/33 | | 15 | Exact Vue column literal and CSS | 0/4 | 0/4 | 4/4 | | 16 | Explicit sender/receiver/storage bodies | 113/128 | 113/128 | 128/128 | @@ -49,9 +49,9 @@ The decisive assertions are present in cases 1 and 5, including the identity che - Cases 3, 4 and 11 still omit table selectors or the cache key. Case 4's evaluation body is now complete; losing the two formerly returned closing lines of the cache-key range does not establish that the key was previously usable. - Cases 9 and 10 still omit the receiver file unless the query explicitly names its endpoints. Selecting that file and synthesizing the actual message boundary are separate future changes. -- Case 12 returns the survival tag and requested CSS ranges, but still omits the column declaration and ADP/FP cells in that broad query. The exact column-literal control, case 15, is complete. +- Case 12 returns the survival tag and requested CSS ranges, but still omits the column declaration and the remaining column cells in that broad query. The exact column-literal control, case 15, is complete. - Case 13 improves selected test source but still omits the identity assertion. Its broad path-and-function query has less discriminating information than case 5. Do not score it as solved from its filename or partial line count. -- The ADP question remains helpful without the secondary test; no test-inclusion improvement is claimed for case 8. +- The mapper question remains helpful without the secondary test; no test-inclusion improvement is claimed for case 8. ## Validation and reproduction @@ -68,7 +68,7 @@ For a source probe, build the intended engine's TypeScript and copy its assets, ```bash fnm exec --using codegraph node scripts/agent-eval/probe-explore.mjs \ \ - 'src/components/DraftTable.vue "col-w-survival"' + ' ""' ``` Use the fixture's exact queries for the other cases. Match numbered lines inside each file's fenced Source section, not occurrences in the blast-radius header or omitted-symbol pointers. Recheck fixture hashes first. The direct harness binds the project through `CodeGraph.openSync`; it is not a deployed MCP or agent A/B run. diff --git a/docs/benchmarks/explore-selector-cache-evidence.md b/docs/benchmarks/explore-selector-cache-evidence.md index 98a5ff3398..5858f9c93a 100644 --- a/docs/benchmarks/explore-selector-cache-evidence.md +++ b/docs/benchmarks/explore-selector-cache-evidence.md @@ -12,17 +12,17 @@ The existing guards for stale source, file admission, oversized containers, flow ## Fixed-source comparison -The exact 16 queries and 13 source-file hashes are from the private fixture (private checkout at `74fcd4e`), pinned to downstream source `c570e32b22177ab59d3fa8f4c162419bac196422`. Both engines used the same existing index and a fresh ToolHandler per call. Baseline results are the retained `22ae167837e174fa623dc77c2c4435ecadbc37d0` comparison; candidate results were rerun with each referenced source hash checked. This compares source completeness, not agent answer quality or independently rebuilt indexes. +The exact 16 queries and 13 source-file hashes are from the private fixture described in the first report, pinned to a downstream application's source revision. Both engines used the same existing index and a fresh ToolHandler per call. Baseline results are the retained `22ae167837e174fa623dc77c2c4435ecadbc37d0` comparison; candidate results were rerun with each referenced source hash checked. This compares source completeness, not agent answer quality or independently rebuilt indexes. | Case | Evidence | 22ae1678 | Candidate | | --- | --- | --- | --- | | 3 | Popout prop wiring | 2/2 | 2/2 | -| 3 | DraftTable selectors | 0/22 | 22/22 | +| 3 | Table selectors | 0/22 | 22/22 | | 4 | Cache key | 0/50 | 50/50 | | 4 | Consensus evaluator | 73/73 | 73/73 | | 11 | Cache key from broad hover/invalidation query | 0/50 | 50/50 | -All other fixture ranges retain their baseline completeness. The snapshot assertion, draft identity assertion, named interface, stamping body, ADP mapper bodies, exact Vue column/CSS, and explicit TTD endpoint bodies remain passing controls. Cases 9/10 still omit the broad-query receiver; case 12 still omits the column declaration and ADP/FP cells; case 13 still omits the identity assertion. No sender-to-listener edge was added. +All other fixture ranges retain their baseline completeness. The snapshot assertion, identity assertion, named interface, stamping body, mapper bodies, exact Vue column/CSS, and explicit endpoint bodies remain passing controls. Cases 9/10 still omit the broad-query receiver; case 12 still omits the column declaration and the remaining column cells; case 13 still omits the identity assertion. No sender-to-listener edge was added. The candidate's maximum response was 24,989 characters; all 16 responses totalled 392,850 characters. Single-pass median latency was 197.5 ms, range 69–268 ms. The earlier baseline was 202 ms, range 65–258 ms. Runs were not interleaved: these timings do not establish a speedup or regression. diff --git a/docs/benchmarks/fork-vs-upstream-2026-09-26.md b/docs/benchmarks/fork-vs-upstream-2026-09-26.md index 637b407528..d335349541 100644 --- a/docs/benchmarks/fork-vs-upstream-2026-09-26.md +++ b/docs/benchmarks/fork-vs-upstream-2026-09-26.md @@ -13,9 +13,8 @@ n8n (Vue and TypeScript) and re-measures every corpus on one newer fork build. in their own worktrees. - **Host.** WSL2 Ubuntu, 16 vCPUs, 47 GiB RAM, ext4. No swap in use. Each batch started with a 1-minute load average under 2 and Windows under 5% CPU. -- **Commands.** Every run was capped with - `nice -n 10 ~/.local/bin/run-capped 16G` and timed with `/usr/bin/time -v`, - with `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_NO_DAEMON=1` and +- **Commands.** Every run used `nice -n 10` under a 16 GiB memory cap and was + timed with `/usr/bin/time -v`, with `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_NO_DAEMON=1` and `CODEGRAPH_TELEMETRY=0`. The index directory started empty for every run. - Index: `codegraph init --yes .` - One-file sync: an untimed `init`, then a one-line edit to one source file, @@ -98,7 +97,7 @@ README lists as the fork's largest. ## One-file sync: fork vs upstream (Node 24) -Re-measured later the same day on an idle host (no other session's capped jobs +Re-measured later the same day on an idle host (no other memory-capped jobs running, 1-minute load 1.5–4 from the benchmark's own workers), with the fork at `perf/incremental-sync` (the sync skip cache, [`docs/design/incremental-synthesis.md`](../design/incremental-synthesis.md)). Each arm indexes once, then times two one-file syncs, each adding a line to the diff --git a/docs/benchmarks/fork-vs-upstream-2026-09-28.md b/docs/benchmarks/fork-vs-upstream-2026-09-28.md index c27e89bd81..22cb6476eb 100644 --- a/docs/benchmarks/fork-vs-upstream-2026-09-28.md +++ b/docs/benchmarks/fork-vs-upstream-2026-09-28.md @@ -19,10 +19,10 @@ runtime comparison (Node 24, Node 26, Bun), which was not re-run. `npm run build` in their own worktrees. - **Host.** WSL2 Ubuntu, 16 vCPUs, 47 GiB RAM, ext4, Node 24.21.0. No swap in use. Before each corpus's index step and again before its sync step, the - runner waited until no other session held a capped job and the 1-minute load - average was under 2. -- **Commands.** Every run was capped with `nice -n 10 ~/.local/bin/run-capped` - (16G for index, 13G for sync) and timed with `/usr/bin/time -v`, with + runner waited until no other memory-capped job was running and the 1-minute + load average was under 2. +- **Commands.** Every run used `nice -n 10` under a memory cap (16 GiB for + index, 13 GiB for sync) and was timed with `/usr/bin/time -v`, with `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_NO_DAEMON=1` and `CODEGRAPH_TELEMETRY=0`. The index directory started empty for every run. - Index: `codegraph init --yes .` diff --git a/docs/benchmarks/fork-vs-upstream-node-bun-2026-09-28.md b/docs/benchmarks/fork-vs-upstream-node-bun-2026-09-28.md index bdf4926d2a..d284bf58a2 100644 --- a/docs/benchmarks/fork-vs-upstream-node-bun-2026-09-28.md +++ b/docs/benchmarks/fork-vs-upstream-node-bun-2026-09-28.md @@ -36,11 +36,11 @@ The scripts are in [`scripts/benchmarks/runtime/`](../../scripts/benchmarks/runt fork needs no override. - **Host.** WSL2 Ubuntu, 16 vCPUs, 47 GiB RAM, ext4. Swap was cycled before the run and stayed at 0 B. Before each corpus the runner waited until no - other session held a capped job and the 1-minute load average was under 2. + other memory-capped job was running and the 1-minute load average was under 2. Windows-side CPU was about 9%. - **Environment.** `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_TELEMETRY=0`, and `CODEGRAPH_NO_DAEMON=1` for the CLI runs. Every CLI run used - `nice -n 10 ~/.local/bin/run-capped 16G` and `/usr/bin/time -v`. Peak memory + `nice -n 10` under a 16 GiB memory cap and `/usr/bin/time -v`. Peak memory is MaxRSS. Database size is `codegraph.db` after a WAL checkpoint. - **Order.** Per corpus the arms ran in the order upstream Node, fork Node, fork Bun, upstream Bun, then the reverse. Each run did a full index diff --git a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md index d3d03aec01..10bc2a826b 100644 --- a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md +++ b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md @@ -28,12 +28,12 @@ The scripts are in [`scripts/benchmarks/runtime/`](../../scripts/benchmarks/runt Node 25 and newer, so without the override upstream exits with `Unsupported Node.js version: 26.3.0`. The fork needs no override. - **Host.** WSL2 Ubuntu, 16 vCPUs, 47 GiB RAM, ext4. Before each corpus the - runner waited until no other session held a capped job and the 1-minute + runner waited until no other memory-capped job was running and the 1-minute load average was under 2. Unlike the 2026-09-28 run, swap was not cycled first: 2.1 GiB of swap was in use, with about 35 GiB of memory available. - **Environment.** `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_TELEMETRY=0`, and `CODEGRAPH_NO_DAEMON=1` for the CLI runs. Every CLI run used - `nice -n 10 ~/.local/bin/run-capped 16G` and `/usr/bin/time -v`. Peak memory + `nice -n 10` under a 16 GiB memory cap and `/usr/bin/time -v`. Peak memory is MaxRSS. Database size is `codegraph.db` after a WAL checkpoint. - **Order.** Per corpus the arms ran in the order upstream Node, fork Node, fork Bun, upstream Bun, then the reverse. Each run did a full index diff --git a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md index c84c5e0bd7..dffb24d444 100644 --- a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md +++ b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md @@ -31,13 +31,12 @@ The scripts are in [`scripts/benchmarks/runtime/`](../../scripts/benchmarks/runt `Unsupported Node.js version`. The fork needs no override. The previous run used Bun 1.4.2. - **Host.** WSL2 Ubuntu, 16 vCPUs, 62 GiB RAM, ext4. Each run waited until - the host was quiet (below), and the benchmark held the host lock while it + the host was quiet (below), and no other heavy job ran on the host while it ran. - **Environment.** `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_TELEMETRY=0`, and `CODEGRAPH_NO_DAEMON=1` for the CLI runs. Every timed command ran at normal - priority under `~/.local/bin/run-capped 16G` (a 16 GiB memory cap) and - `/usr/bin/time -v`. Peak memory is MaxRSS. Database size is `codegraph.db` - after a WAL checkpoint. + priority under a 16 GiB memory cap and `/usr/bin/time -v`. Peak memory is + MaxRSS. Database size is `codegraph.db` after a WAL checkpoint. - **Order.** Per corpus the arms ran in the order upstream Node, fork Node, fork Bun, upstream Bun, then the reverse. Each run did a full index (`codegraph init --yes .` into an empty directory) and then two one-file @@ -60,8 +59,8 @@ corpus. ### Quiet-host rule A run started only when swap in use was at most 64 MiB, the 1-minute load -average was under 2, no `capped-*` scope was running, and Windows CPU, read from -the Windows side, was under 20% for three samples 20 s apart. An attempt was +average was under 2, no other memory-capped job was running, and Windows CPU, +read from the Windows side, was under 20% for three samples 20 s apart. An attempt was rejected as suspect when the command exited non-zero or a Windows CPU sample before or after it exceeded 30%. Each corpus had one retry; the first accepted attempt is the one reported. The two runs per arm are the spread this page diff --git a/docs/design/strategy-2026-09.md b/docs/design/strategy-2026-09.md index 48fc8bd155..fa9940229e 100644 --- a/docs/design/strategy-2026-09.md +++ b/docs/design/strategy-2026-09.md @@ -6,13 +6,13 @@ ## 1. What we need from a code index -Our use comes first; "best all-around" is the second test. From our private downstream project's code-index research and the fork's own history: +Our use comes first; "best all-around" is the second test. From code-index research done for a downstream application and the fork's own history: -- **Agent-driven work on a shared host.** Many sessions across Claude Code, Codex, Cursor, OpenCode, AGY and Devin, most of them in their own git worktrees, on one WSL machine that has been OOM-killed by aggregate memory pressure. +- **Agent-driven work on a shared machine.** Many sessions across Claude Code, Codex, Cursor, OpenCode, AGY and Devin, most of them in their own git worktrees, share one machine, so aggregate memory pressure is a real constraint. - **One call that answers.** The measured win is an agent that stops reading files because the answer came back complete: verbatim, line-numbered source plus the flow between the named symbols. In the rules-on A/B (Opus, headless, 6 discovery + 3 concept tasks) CodeGraph cut 2 to 2.5 calls per task and kept 6 of 6 correct. Two shapes that return less than source measured worse: a symbols-only answer lost 2 of 18 cells, and gortex's budgeted outline answered 3 of 6. - **The index has to be right.** A wrong caller or a wrong edge is what sends the agent back to Grep; a missing one mostly costs a follow-up call. - **Local, deterministic, no LLM in the index.** Answers must be reproducible, and nothing leaves the machine. -- **Fresh in every checkout.** Worktrees had no index in 9 of 11 cases in the downstream audit; an un-indexed root routes the agent to Grep. +- **Fresh in every checkout.** Worktrees had no index in 9 of 11 cases in an audit of a downstream setup; an un-indexed root routes the agent to Grep. - **Also used:** string literals and storage keys (the `literals` table), Markdown sections, and transcript search across hosts (`codegraph_sessions`). What we do not need, on current evidence: embeddings, LLM-written documentation, large tool catalogues, compact wire formats, PreToolUse hooks. Each was measured null or negative, or has no use here (see the survey's rows L–R). @@ -49,7 +49,7 @@ repowise's [BENCHMARKS.md](https://github.com/repowise-dev/repowise/blob/main/do | Graph build, 35 repositories | 3.65 s median, **757 MB** peak, fastest on 16 | 2.77 s, **75 MB**, fastest on 14 | speed level, memory 10× theirs | | Full index, django | **16.4 s** | 367 s (1,058 s with prose) | our clearest win | -The TypeScript result matters most to us: TypeScript is our downstream project's language and the language of most of the README corpora. +The TypeScript result matters most to us: TypeScript is the language of the downstream application this fork serves and of most of the README corpora. The precision gap has a likely shape. repowise types a receiver from its declaration where it can, and stores every weaker binding under a named origin with a lower confidence (its cross-file "global unique" match is recorded at 0.50 as a guess); CodeGraph's exact-name and fuzzy strategies bind a call to a same-named definition that passes the gates, with nothing marking it as a guess. The fork already found one instance while writing this: the kernel bound a Python call to a same-named Nix binding, which the TypeScript resolver rejects (fix pending). The 1.5.0 numbers predate the fork's binding model (#27–#38) and the member-access kernel work, so some of the 116 wrong rows may already be fixed. That has to be measured, not assumed (§6, step 1). @@ -66,7 +66,7 @@ What it bought, all measured in the ledger (§5.38–5.43): What it did not buy: - **Answer quality or precision.** The kernel resolver is a bug-for-bug mirror of the TypeScript one, so it reproduces the 58.6%. The survey's conclusion still holds: "The language of the engine did not decide any of these outcomes; what the tool returns did." -- **Memory.** Our index run peaks near 400 MB even on a repository of 1,200–1,500 nodes (os-lib, lazy.nvim in the ledger's A/B), so the floor is Node plus the worker pool, not the kernel. repowise builds its graph in 75 MB. On a host that has been OOM-killed by aggregate pressure, this matters more to us than seconds do. +- **Memory.** Our index run peaks near 400 MB even on a repository of 1,200–1,500 nodes (os-lib, lazy.nvim in the ledger's A/B), so the floor is Node plus the worker pool, not the kernel. repowise builds its graph in 75 MB. On a machine shared by many agent sessions, memory matters more than seconds do. - **One implementation (as of 2026-09-25; #192 has since removed the TypeScript resolver spine).** Every kernel punt then ran the TypeScript resolver, so there were two resolvers to keep in step, and a parity bug the dump gate could not see (a language no corpus contains) could ship silently. `import-resolver.ts` remains as an import-path helper for synthesizers and framework resolvers. The greenfield sketch's reopen conditions are close to met: both incremental plans have landed and resolution runs in the kernel with golden-dump parity. Its third condition, "a memory ceiling the daemon cannot meet", now has outside evidence behind it. That argues for measuring the memory floor before deciding anything larger, not for a rewrite. @@ -98,7 +98,7 @@ Take option B: 2. **Keep an upstream channel for small fixes only.** Of the fork's 205 non-merge first-parent commits since the merge-base, about 67 are independent TypeScript fixes or features, and 25 of those cherry-pick onto `upstream/main` without conflict. Send those as goodwill in the shape the maintainer actually merges: one concern, under ~600 lines, a fail-to-pass test, no CHANGELOG hunk. Do not wait on them; nothing in our plan depends on them landing. 3. **Close or re-cut the stalled large PRs** (#1699, #1867, #1737, #1702, #1841) so they stop costing rebases. -This also changes the downstream project's stated goal of keeping the fork thin so that retiring it stays a one-line swap. That goal assumed upstream would absorb our work; the numbers above say it will not. The downstream kernel entry should name the fork as the served engine on purpose, not as a temporary divergence. +This also changes a downstream consumer's stated goal of keeping the fork thin so that retiring it stays a one-line swap. That goal assumed upstream would absorb our work; the numbers above say it will not. The consumer's kernel entry should name the fork as the served engine on purpose, not as a temporary divergence. ## 6. Priorities for the engine @@ -109,7 +109,7 @@ Ordered by what moves "the agent stops reading" and by what the outside benchmar - Tighten the strategies that bind by name alone (exact-name across files, fuzzy) where the precision replay shows them wrong, starting with TypeScript: decline when the receiver's type is unknown rather than pick a same-named method. - Persist a resolution origin with a confidence on every edge, like repowise's 29 origins or gortex's tiers, and show it in explore's flow so the agent verifies the uncertain hop instead of re-reading the neighbourhood. Add the cross-language mis-wire guard with it (survey item C; the Nix bug is this class). Gate: the precision replay improves, the golden dumps change only where intended, and the agent A/B shows no recall loss that costs answers. -3. **Measure and cut the memory floor.** Profile a small-repo index and the idle daemon: Node heap, per-worker node tables and caches, SQLite page cache. Set a target (for example, under 200 MB for a small repository) before optimizing. This is the one cost where we measured 10× worse than a competitor, and it is the one our host actually suffers from. **Done 2026-09-25:** metrics ledger §5.45 (parse pool sized to the project, idle query workers retired; a 3-file project's index run peaked at 176 MB, down from 358 MB). +3. **Measure and cut the memory floor.** Profile a small-repo index and the idle daemon: Node heap, per-worker node tables and caches, SQLite page cache. Set a target (for example, under 200 MB for a small repository) before optimizing. This is the one cost where we measured 10× worse than a competitor, and it is the one a shared machine feels first. **Done 2026-09-25:** metrics ledger §5.45 (parse pool sized to the project, idle query workers retired; a 3-file project's index run peaked at 176 MB, down from 358 MB). 4. **Diff-seeded impact in explore** (survey item A). Every review-oriented competitor ships it; we have the impact machinery and no entry point. Add a "what does this diff break" task class to the agent-eval harness first. **Done 2026-09-25:** explore answers change questions from the diff (#125); the diff-impact task class and its first baseline in the agent-eval harness followed in #460. 5. **Index artifacts for worktrees** (survey item D). Copy an index into a new worktree and catch up by file hash, so a fresh worktree answers from the first call. **Done 2026-09-26:** `codegraph init` seeds from the closest compatible sibling worktree index and syncs (ledger 5.58). 6. **Finish the resolver port, then delete the TypeScript resolver.** Only after steps 1–2, so the kernel moves one design forward instead of two mirrors in step. This removes the dual-maintenance cost for good. **Done 2026-09-26:** the TypeScript resolver spine is deleted; the kernel resolver is the only one (#192). This went ahead of the "only after steps 1–2" order: step 1 was done, and step 2 is still open. @@ -126,6 +126,6 @@ Explicitly deferred: git-history and code-health layers (repowise's strengths, b - CodeGraph upstream releases: https://github.com/colbymchenry/codegraph/releases - codebase-memory-mcp: https://github.com/DeusData/codebase-memory-mcp - Upstream PR and merge statistics: `gh pr list -R colbymchenry/codegraph --state all --limit 200`, 90 days to 2026-09-25; divergence from `git diff --shortstat upstream/main...origin/fork/consolidated` at merge-base `ba3c21e5`. -- Our own measurements: [metrics-ledger.md](metrics-ledger.md) §5.38–5.43; the rules-on A/B in the downstream project's code-index research. +- Our own measurements: [metrics-ledger.md](metrics-ledger.md) §5.38–5.43; the rules-on A/B in a downstream application's code-index research (not published). Limits: competitor facts are from their own documentation except where repowise measured them; the precision and retrieval figures are for CodeGraph 1.5.0, not this fork; the 67 / 25 upstream-able counts come from a path-and-subject classification and a patch-level apply check, with no build or tests run. diff --git a/scripts/precision/__pycache__/pick-g1-commit.cpython-314.pyc b/scripts/precision/__pycache__/pick-g1-commit.cpython-314.pyc deleted file mode 100644 index 2d1c092879a032c8cdd501d4bc2455038d5693e4..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 3319 zcmai0Z)_9U5r6y7-q>C{|G1n_P&S1GEMf;kL_!Ljas=+U9KqQO22`AM@ot<2$Gh2G z6JqI*i&SBvtK#aYl&zn!`SU&GWyK2xXCgOf-W8;tXLD z_Dq_jJyRxS&$LO~Gh;FWX*vX|2boTR2Fk|hNN3aR_B=xjytchFT`^}bin)e}I?r?q zjNlNQF_#A&B14qv5eSouF_E3yI?Yut64)7Lh8Ut6y1tn2C6ernrztryijedfLOW56 zittT)+cS9N5!wWo;EwsQAA%=D4{RSJfrFoSxN&dfhb>2qWu))ByrBlm?22{mYpkXC z_jutG+Jz3GGu9PiUhtr67&3&%ZWLp23}f!02FL7+)SyyZa zX!@GbKTJWaU$JfZVqHRat6r>I*b(c2Gj~%OV(`kabIS)D`-AmeEA2%yPN8SGk>3Gs z`^@xBSm8@0c<4xg7fpU?UWw1Uu;_wn zn0!JPC0XKib;;Pr8){mQ%e*PyFnKU)z$HAMbUCi-SS+ce-1mau{gI<#sRD`NlpwR)P{ZlcW87gEHz>x zT|bEeaMI*%TS7qrr{yx2HQ9*kie_5GV8#cbmt;G>p{s_PvS=JfppXE|UB|*K&Uk>* zaWE}MJ$05NZp@3i7GNy8u9s9r$)Z(F1{+sIL$;`dYz7z|htHw_vKE~bQwfWZEJ8PM zc>8%6R>yZRV6oVTSpc}+4Pw}Up#jOF8tclQ{Hpz6j(+U&t_D^DcgH^(FB~a$6o-oa zYre97q|$w~;u_5nHP&4?@@s#EJNKbKKep!hSSfYiQ{NZYd;iq;Y2OC7IXby2=EM*E z_qlUb_S_2fltCTgCl2KCW#9hqGYWYlgwX+){?Rvfg8g%QH>Lpx(0D6eaO#v)q?0~?JY5Em$cY{f~l1!(m5SZg$~rX@T_w@~^5dZrM%yoI(X zEl>h2Fk4T7yVXX0-E8}jme$vl5*$s8YD|<8oUPvfUwO7E|Lvw8foA{WXx=NdwOZ92 zaF45%a~WsK9?`9e!4G7kTK#d$2{%DM>lQq|LeDh~)%cQjE(1(qf@Ws*(cmcgVV~>V`$QbeuAa_{r$x z*M3+{LVe&mYqLld3Q!Zxc_=MaNrB1@#V`#WUOa>=6Y7jHw4`FfWI`cZR8md_oOY$@ z7*$~5nvVasScGOV*eV#D)5U~=uk5$$j@P@e0AoN`mv#IuWAnzRK%mZ?Z1#<>xQ-n0 z*yUN>cW2+~z@35oQo$_FeSD_k3IX5OvAT3;Y4zrvn+0*x$A9VL@9lr=?aYS@iM2}) zyu+K`WB0wsD&7;D-tqh1@r~(<_w5{0; z!W}N1ta2kcy5{!eqCY=+H(u!b;Oav+Uz{$5OVLk`u1|gPqiXQ=hkdVabZ$(2zT>mb zr*7o!eBwbKZ}!Y{a2|S1LNPOB@MOke=VVa%7kqVw)#dBKB*YPp!kM-pyj##<NuWW$dSVPB!VNrS- zP-K`AIH>ny4iUEqY}wQ;M$?s)spA%5wc#5iIcbW&MLN7%s16eL*n%phlky4uP2k}- z)>r`ggdzyyDM=C$;u{Ad_HCm63hJ*hC$jW2k|BK0JS^dT(uo{xw=e$mVovt}ITAD2fP8AnxOk0+&k*?dpTfsb?pUN**NPm{7Ih?syS9h)K z%A19;!j(!}?*m6~&Cz~4bt{#hD!7V6Yk|^O=`b|^shM)mY<0(MSvAVb?>%tb{01TC zX%dPRC|_1!WIa{8-SY5A($cv9THm1Bi)VY=A+s|z0;T^m~ye@B(Ze(zX2R+JvGTu#b|!EgPS=RxS* zdtJrp-}p=2RrW}YX0kUno&GPK{=(t|XK$76y){-tuG`*Q-rQn-Z@K4CX>?uM82W+?*WAF_9{t%O*xZBeg(ANTRz1MzIs( znJJ6J9^^{0qg2KfWs_`3O-&UaJGIX4PL1bdt30(eyTURH>2j5~YCpEJwI4}5TiMCa z?z!D+31K`t`>buuJ@bE0ZdHPR!gys>0 zL^O;T;zdFvWJrpn3^k%ghFVc8L!GFTAth1_De4(bITG_2EtCeA#<5*RpM^Du(X|sw zTQo5g%*P!=$GP!=)clbMSPktFzFxOQp0S-8o*1&`#tkag_o7 zl8yM8(l*U8@))_XZ|(Yt+nKV%Y04ZYvUAm%DjSDg?a4xB zhf1AT<1zrRYk;Qmjr+SQ9jCJM)f!Wwu2WCt0nLR|y2(;jq+I0xF}(mKv&cTsC_ zr*^ukKyP;)(5NfK`rpR|vB6dFd>+k`pw(v4;;MICQ1{3}W;altqpz+yS3{eo9aiDb z0bJeusk$LrUDk{w|5oO{br@}wli2JkdX_R*vx;4`xojB>pp82bv!|C}Xt8B0O?S2Y zPs`EOvc>5ub0z0$hTF8k27zR1pfBgXL0>KOX*SKPgFfx1zP(_zMmKX5A>dN;Ftu^+ zF+%0cxXhV7h}oy=!txVHKPOE&&;hE+f_*Vnk4c_Klhf;I3!P*jab}>4=4Q^Ju z>mc;DgTA)s;7s42J?LCndNtwjZz=au?5`Lr)4Y{Oacx)k10Ay+&@e5kB!&rQziPoj zH;6i9X=VQgwq>}GpPi$5E)<;g30(6S7jRF|XIkl#K@WF^XG0Ust+s=<{q#7`dN>cw z2WJJE9cO(3L8MvQ8{{WxU%{Nc+BPwPPJs_dAONuL#EF{vjLBSNoYBaCw8@rT4|1j z9o-Y0dlCo@hDD<} z>!AHXH!E_qIKk0w*6*hUAGFNeRn{%iFp&fD{em(c;8>mm_Tp}tMqhw#YHr_eRC$Pz z2*EJ#=78Vq_!Jiqt#pvjAm=zbz)r#12`&J_fc@zSR-gkxb(P3n6)jdH;0eg0=S7&y ziZTLOq+`G$0XKQrB8^$VC^7A6^d0ml?i!^|9VK>8ps zix&%D$Vb+c7Fn<@+|>}r`!Jt`7ElMyGtM=KKr?-UFBoXS zI_*vert+oJK?x=aJ^g1W=8+RQHjc_u;N~FpvgNul2VS}9Q=Q;eV+^6<*=kf>86t+X zDpCif*0Ky3JQueIfu6G+!clPSObBF-b4q5jSTcq0PG`LJ+WJtEZ*1 zEpLyc$Lc#0^zub8tq^{NK`7?YI%T-&d&3vgd_NHDU!}^|s9h`6t`xOrjoPXrgu9lpj5R`_O#v zQ$UW~8|9M+9kQgIAYqmOS(B<$QXJ`dFqOmTJQ0SV`A8{3o)R2*3}y7enbCqHVE8@Y z`gfvUau4d=agjbxKrkRbyzss#C)mO70mbd%9fmSAW$_iG3z`esu*QbM%it1|dQ^wbQ+ zabWvKhHgzbC@0ZpAtVjh2+%KLLegUUs3_PxkCjKt14A)2=3%>}3dP*oj$`Nt)1i*w z*RFt{JO4V5KHiq-Pl%s5Kd$ECX#w~p%_KM1XVFR;KNpY;Tp&Effgj`e?HIde$`^nO zqLa7fF(OHe^;aT%65*G$6FyN8@YSsM)`Q(K9 zz-NQE>i)9r-t_&FUkLZxe{uCMIv>=o9DC_jU9=`@U#yGP!OZ!d6+`cn)`Pd&qT-_p z>myThQhVZFI~2crszJs+Lcrww%4F=`NByd*8kZdfP)5pz%|qv#PFOn3L!KtfkkDwh zHCpyQBR=`odul zz^`x#ih1;y%D?G8V;@* zj;$Dur3{^GhMuIM=l-_&p2wzw#UpPXiG43VkTTUK4y>B$liGUu8stbCtFtWWMzDDQ zMM;BE+KHYc>5T8&U{|)*2~xFSDr_UcZemu-pCcnjwFf_w7y`5}ZJBbk?|+dd2I#VA zzQi6PQaWTdUmotTnk}xT^o6pJeSH>+$A*u$;Gz)K=kLHu#kKc9>o?t z?L-xo&`-crtPyQ6vJ;O^K=F)5=<{MVKW@0w5U+W=@h9gKC7+gmSiV&9QPuk+$&%)z zu^GEe-wj`~cwfSsqMAX6nyg?u&McgX+21<*ju3DC;P88g6Rq!ey#0F8)R5G6E;m8( ztcGY|cUePzAd~UDSP5>dDhC`-;T4;{?41VB2mV&}dtlb5OH`YY_FDPiwFvN2V6*X) zD@=yKgUTKUYT$239@F$4+0g-26IA6}hkeGhL|n~brZy1dr+9F9pew*hoVdfmhh7Oj zVm&FUa^AjPUh(7EJF{`|o$D!6qs)*cVivvfNn?9$`0a4uM zkD*k{qC$0VMei6r*c%+Q$Y0j0$#L1Zlj1;yu$}t@_Rs9}><&&nCr6loCnrJV)E}8u zBSx8$|A?c84VEv@nWygL0h=RJnS*@@(hli!UxHBRI2qXbUu?* z4;h9sYv8OoNi%hika4J5#WA!?tvL*po{M-@W}Vz z&mV{}f^9bOdjS^@ii!AxP)I~eq-@A$b_>@+bJs-S+N5~x3OmDE_Wn7)AIAIoBBZV5 zbYG-I@zo>pyIhWr1}sIJEcqr6&mX`WobW^2<*q*?9AXnE|0bi|p9D1fd z#bKE6BYi|6od+%;e0re={1Z40eZkFZZ(NI=PU)-XyPs56M{5_&3+7nst)|3|HG^%% zU|SMXhW6$DRm0&mL+6U2Gi5k2-?MHkTx?uuj18oWRr9Bw=<{QB@v`{z-HLZwK4v~0 z{Nur7%b5qd2kdY1*7o&3+}FQeP#SB!eJQF<9Jo6e3$GUy$4YLCi`N&f#|J*R{NClI z?WtY1wW5}XMJ=D}9ve$y^>4MjacaJMU2nMYTAWP~cRe3Wy*IVgy|%mM;cl3t4AjK! zclRd9pO`;8nDJ+O6V4Q61%C#Sb$@jF_P{%Bv1!?UPrn^``^2MyT0GepZC^aPa5Q!y zWvWh5JLl~mYvC<0{mH;b#yIQGvhe=8O$ZDwQi6Ix&u=+g7pms1yCSS-Vz#< zkR7seDwyI#*29XB48*xVoLdoSj>Z}d87B`k1RBydoJ}FSQ?U6w8%ITDR8i1Q5k}>- zo2tY(i3Le0!Iq(>^MsuK<81gIW*V3#NZ`@t2ArZ28mtXCMac$4A3Wf2ljppEDBy;M zk_A)_-g!dO;u|9A)7dN8Fi1p4gtXET%0kchSUN(!dR;QAxvK8D$n7-qGDsDk!G>a3 zuzFdaKg@F@JjddUmCToHstdZkLi$*n54GV)=3LxxP{@$w_CbbllA}*pBIG`sq)+G8 zBudGmb@VCLE23m&=9(pa(`|SQf8pTW30f*X&d=z3)~A%g+N~vbBJFnijBkOO}09c{v^)U zalE8~)T~5#Kx=&PYKGk547{mvf)8Hw0H6%A9!VPv!5f_LNCeNH#Ynnw(FKJ>gy2qG z^$FM<0*gJ0&BYEv(m*0oB3=R>K$iGO&cmO=;sxR> zBg!-0jNFJMOB+-Arg;jFHQ=$XZ;T|-^bJ)=Siahgbj54h@`u{;l(uqSlcAH$N42-sVO6l^Y3UO+b|RFkBf0{;^kynd#dOny?yu!P(m$09h^zi^a%MJp^|jTR*S|u@PIi+3PLk)yf5EMnaI2Sm8G}B;LH-t@7l?lH z8%U>-m&tWg>EekuPsE*xp2VS~#h$G0PMJ`JBJc;(bMbsMX~KM zHg0&=4@eoG?=p{xls2;`h3s~~+ z3yT-8EL?%EDjy!0B>fmnkpkXx68^6Ww8G>J2D9WG1`+Z#3|=Sg8kiCR0Ie%zw+09) zI;jD?MK5W93@C9=pC)R5Lf70mnz;Jk;pVSTCLR4rW?D6_$Uk zLj|Q@+Yl(e=10}@_B&kM9(N~d?t1?yxKO>0v^Nbm4AJ^K+vEC#=F_|n^Onj!DoXCz WzpP8{IJ7LJN{-!cT|=j2L-X$|Hhn$- From c133949783dd24774588ecba4ac6c0de04410497 Mon Sep 17 00:00:00 2001 From: Aaron Queen Date: Mon, 12 Oct 2026 09:16:34 -0600 Subject: [PATCH 2/4] docs: keep the benchmark and branch wording to what was verified --- AGENTS.md | 2 +- docs/benchmarks/fork-vs-upstream-2026-09-26.md | 2 +- docs/benchmarks/fork-vs-upstream-2026-09-28.md | 4 ++-- .../fork-vs-upstream-node-bun-2026-09-28.md | 4 ++-- .../fork-vs-upstream-node-bun-2026-10-03.md | 4 ++-- .../fork-vs-upstream-node-bun-2026-10-12.md | 12 ++++++------ 6 files changed, 14 insertions(+), 14 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 1f9644ab0d..86faadb2d0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,7 @@ Canonical project guidance for coding agents working in this repository (Codex/A **Instruction budget:** Keep this root file below 32,768 UTF-8 bytes; `npm run check:agent-docs` enforces the limit. Put conditional procedures, evidence, and worked examples in linked documents. Codex user configuration may set `project_doc_max_bytes = 65536` as a safety margin, but the larger limit does not replace the repository guard. -**Branch:** this repository's default integration branch is `fork/consolidated`, and pull requests go to the fork, never to the upstream repository or `main`. Do not run `npm publish` without explicit authorization; fork release policy is under Releases below. +**Branch:** this repository's default integration branch is `fork/consolidated`; fork work goes there through a pull request to the fork, never to `main`, and focused upstream contributions follow Branch roles below. Do not run `npm publish` without explicit authorization; fork release policy is under Releases below. **Agent rules:** [House Rules](https://github.com/bompus/house-rules) is an optional, portable rule set that works well with this repository; nothing here requires it. If your agent loads it or similar always-on rules, this file adds to them and wins for this repository's commands, branches and checks. diff --git a/docs/benchmarks/fork-vs-upstream-2026-09-26.md b/docs/benchmarks/fork-vs-upstream-2026-09-26.md index d335349541..3b69057872 100644 --- a/docs/benchmarks/fork-vs-upstream-2026-09-26.md +++ b/docs/benchmarks/fork-vs-upstream-2026-09-26.md @@ -97,7 +97,7 @@ README lists as the fork's largest. ## One-file sync: fork vs upstream (Node 24) -Re-measured later the same day on an idle host (no other memory-capped jobs +Re-measured later the same day on an idle host (no other session's memory-capped jobs running, 1-minute load 1.5–4 from the benchmark's own workers), with the fork at `perf/incremental-sync` (the sync skip cache, [`docs/design/incremental-synthesis.md`](../design/incremental-synthesis.md)). Each arm indexes once, then times two one-file syncs, each adding a line to the diff --git a/docs/benchmarks/fork-vs-upstream-2026-09-28.md b/docs/benchmarks/fork-vs-upstream-2026-09-28.md index 22cb6476eb..2b87e27eec 100644 --- a/docs/benchmarks/fork-vs-upstream-2026-09-28.md +++ b/docs/benchmarks/fork-vs-upstream-2026-09-28.md @@ -19,8 +19,8 @@ runtime comparison (Node 24, Node 26, Bun), which was not re-run. `npm run build` in their own worktrees. - **Host.** WSL2 Ubuntu, 16 vCPUs, 47 GiB RAM, ext4, Node 24.21.0. No swap in use. Before each corpus's index step and again before its sync step, the - runner waited until no other memory-capped job was running and the 1-minute - load average was under 2. + runner waited until no other session's memory-capped job was running and the + 1-minute load average was under 2. - **Commands.** Every run used `nice -n 10` under a memory cap (16 GiB for index, 13 GiB for sync) and was timed with `/usr/bin/time -v`, with `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_NO_DAEMON=1` and diff --git a/docs/benchmarks/fork-vs-upstream-node-bun-2026-09-28.md b/docs/benchmarks/fork-vs-upstream-node-bun-2026-09-28.md index d284bf58a2..543331746b 100644 --- a/docs/benchmarks/fork-vs-upstream-node-bun-2026-09-28.md +++ b/docs/benchmarks/fork-vs-upstream-node-bun-2026-09-28.md @@ -36,8 +36,8 @@ The scripts are in [`scripts/benchmarks/runtime/`](../../scripts/benchmarks/runt fork needs no override. - **Host.** WSL2 Ubuntu, 16 vCPUs, 47 GiB RAM, ext4. Swap was cycled before the run and stayed at 0 B. Before each corpus the runner waited until no - other memory-capped job was running and the 1-minute load average was under 2. - Windows-side CPU was about 9%. + other session's memory-capped job was running and the 1-minute load average + was under 2. Windows-side CPU was about 9%. - **Environment.** `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_TELEMETRY=0`, and `CODEGRAPH_NO_DAEMON=1` for the CLI runs. Every CLI run used `nice -n 10` under a 16 GiB memory cap and `/usr/bin/time -v`. Peak memory diff --git a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md index 10bc2a826b..3685020c75 100644 --- a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md +++ b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md @@ -28,8 +28,8 @@ The scripts are in [`scripts/benchmarks/runtime/`](../../scripts/benchmarks/runt Node 25 and newer, so without the override upstream exits with `Unsupported Node.js version: 26.3.0`. The fork needs no override. - **Host.** WSL2 Ubuntu, 16 vCPUs, 47 GiB RAM, ext4. Before each corpus the - runner waited until no other memory-capped job was running and the 1-minute - load average was under 2. Unlike the 2026-09-28 run, swap was not cycled + runner waited until no other session's memory-capped job was running and the + 1-minute load average was under 2. Unlike the 2026-09-28 run, swap was not cycled first: 2.1 GiB of swap was in use, with about 35 GiB of memory available. - **Environment.** `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_TELEMETRY=0`, and `CODEGRAPH_NO_DAEMON=1` for the CLI runs. Every CLI run used diff --git a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md index dffb24d444..3037e1487c 100644 --- a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md +++ b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md @@ -31,8 +31,8 @@ The scripts are in [`scripts/benchmarks/runtime/`](../../scripts/benchmarks/runt `Unsupported Node.js version`. The fork needs no override. The previous run used Bun 1.4.2. - **Host.** WSL2 Ubuntu, 16 vCPUs, 62 GiB RAM, ext4. Each run waited until - the host was quiet (below), and no other heavy job ran on the host while it - ran. + the host was quiet (below), and the benchmark held a lock that other sessions + on the host respect before starting heavy jobs. - **Environment.** `CODEGRAPH_DIR=.cg-bench`, `CODEGRAPH_TELEMETRY=0`, and `CODEGRAPH_NO_DAEMON=1` for the CLI runs. Every timed command ran at normal priority under a 16 GiB memory cap and `/usr/bin/time -v`. Peak memory is @@ -59,10 +59,10 @@ corpus. ### Quiet-host rule A run started only when swap in use was at most 64 MiB, the 1-minute load -average was under 2, no other memory-capped job was running, and Windows CPU, -read from the Windows side, was under 20% for three samples 20 s apart. An attempt was -rejected as suspect when the command exited non-zero or a Windows CPU sample -before or after it exceeded 30%. Each corpus had one retry; the first accepted +average was under 2, no other session's memory-capped job was running, and +Windows CPU, read from the Windows side, was under 20% for three samples 20 s +apart. An attempt was rejected as suspect when the command exited non-zero or +a Windows CPU sample before or after it exceeded 30%. Each corpus had one retry; the first accepted attempt is the one reported. The two runs per arm are the spread this page offers; no wider variation criterion was set before the run, so a difference near the 3% reporting band is labelled no clear change. From 6525734d3871cfc733a62df5aa334097c6d582ec Mon Sep 17 00:00:00 2001 From: Aaron Queen Date: Mon, 12 Oct 2026 09:16:44 -0600 Subject: [PATCH 3/4] docs: split the quiet-host rule sentence --- docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md index 3037e1487c..32b4cff1a7 100644 --- a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md +++ b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md @@ -59,9 +59,9 @@ corpus. ### Quiet-host rule A run started only when swap in use was at most 64 MiB, the 1-minute load -average was under 2, no other session's memory-capped job was running, and -Windows CPU, read from the Windows side, was under 20% for three samples 20 s -apart. An attempt was rejected as suspect when the command exited non-zero or +average was under 2 and no other session's memory-capped job was running. +Windows CPU, read from the Windows side, also had to be under 20% for three +samples 20 s apart. An attempt was rejected as suspect when the command exited non-zero or a Windows CPU sample before or after it exceeded 30%. Each corpus had one retry; the first accepted attempt is the one reported. The two runs per arm are the spread this page offers; no wider variation criterion was set before the run, so a difference From 9b1a81ea41752ad3f264f8a9c081c7ad82f00977 Mon Sep 17 00:00:00 2001 From: Aaron Queen Date: Mon, 12 Oct 2026 09:23:50 -0600 Subject: [PATCH 4/4] docs: name the quiet-host check the runner actually makes --- docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md index 32b4cff1a7..1cd42ecc18 100644 --- a/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md +++ b/docs/benchmarks/fork-vs-upstream-node-bun-2026-10-12.md @@ -59,8 +59,8 @@ corpus. ### Quiet-host rule A run started only when swap in use was at most 64 MiB, the 1-minute load -average was under 2 and no other session's memory-capped job was running. -Windows CPU, read from the Windows side, also had to be under 20% for three +average was under 2 and no memory-capped job (a `capped-*` systemd user scope) +was running. Windows CPU, read from the Windows side, also had to be under 20% for three samples 20 s apart. An attempt was rejected as suspect when the command exited non-zero or a Windows CPU sample before or after it exceeded 30%. Each corpus had one retry; the first accepted attempt is the one reported. The two runs per arm are the spread this page