Repository navigation
feat(bunch): D1–D3 of bunch-real-projects — no cap, allowlist, partitioned search + refinement - #255
Merged
Merged
Conversation
…ioned search + refinement D1 — no community-size cap on Bunch partitions - Cluster.capPartition: Leiden partitions only, keyed on the algorithm that actually ran (Leiden above the Bunch ceiling is still capped) - Pipeline Step 5 uses it; the oversize warning only runs when capped D2 — the MDG folds code dependencies only, and Bunch runs once - foldToFileMDGWith / defaultMDGRelations (imports, calls, references, extends, implements); contains, documents, depends_on and inferred are never folded - clustering.bunch.relations (validated against the forbidden set) - Cluster.stepFiveClustering: Bunch reuses its Step-4 partition on the inference-enriched graph; cohesion and quality re-scored there D3 — real repositories - splitBySubtree: directory sub-trees of <= max files (siblings under max/10 pooled, a single directory cut into consecutive chunks) - refinePartition: next-ascent single-file moves to neighbouring modules over the whole MDG (Bunch's moveEffect, MQ strictly up) - bunchClusterPartitioned: one population search, or one per sub-tree, then global refinement; report line "Bunch runs: N (sizes), refinement: M moves in P passes"; --bunch-max-files-per-run (default 500), clustering.bunch.max_files_per_run; --bunch-max-files default 50,000 Tests: 25 new (BunchSpec, ClusterSpec, ConfigSpec, ParserSpec, ReportSpec). Full suite: 1,653 examples, 5 failures, the 5 GrammarsSpec failures already on main (avi-1440, avi-1765).
The Partition quality line scored every edge of the exported graph, so the depends_on / inferred edges added by edge inference after clustering lowered a Bunch partition's MQ in GRAPH_REPORT.md (nestjs 2.75, src/cli 13.88) below the MQ Bunch optimised (4.20, 18.37). - Cluster.bunchPartitionQuality: MQ, modularity and intra-edge fraction of a Bunch partition scored on the graph restricted to the MDG relations (restrictToRelations); runClustering and stepFiveClustering use it; the pipeline keeps it (a Bunch partition is never re-scored on the enriched graph) - ClusterQuality.cqScoredRelations; the report line reads "Partition quality (bunch, scored over imports, calls, ...)" - report-consistency delta in bunch-real-projects; task 2.4 Real graphs, default flags: nestjs MQ 4.2019, src/cli MQ 18.3652. Full suite: 1,656 examples, the 5 GrammarsSpec failures already on main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements D1–D3 of
openspec/changes/bunch-real-projects(spec #252, updated in #254). D4, import resolution, is not in this PR: it depends onusable-graph-for-code-reviewandavi-1765.What
Cluster.capPartition,Pipeline/Core.hsStep 5imports, calls, references, extends, implements.contains,documents,depends_onandinferredare never folded. Configurable asclustering.bunch.relations, validated. In Step 5, Bunch reuses its Step-4 partition instead of re-searching on the inference-enriched graph.Bunch.foldToFileMDGWith,Config/Clustering.hs,Cluster.stepFiveClustering--bunch-max-files-per-run(default 500), Bunch runs per directory sub-tree. A global refinement then follows, every time: next-ascent single-file moves to neighbouring modules over the whole MDG, using Bunch's ownmoveEffect, with MQ strictly increasing. The hard ceiling--bunch-max-filesis raised to 50,000. The report gains aBunch runs: N (sizes), refinement: M moves in P passesline.Bunch.splitBySubtree,refinePartition,bunchClusterPartitioned,Report.hs,CLI/Parser.hsTests
BunchSpec: allowlist; split fixtures and properties; refinement never lowers MQ, keeps a partition and is idempotent; cross-sub-tree singleton absorbed; partitioned search recovers the cliques; determinism.ClusterSpec: no cap on Bunch; Leiden capped, including above the ceiling; Step-5 reuse; the enriched MDG equals the Step-4 MDG; split dispatch.ConfigSpec,ParserSpec,ReportSpec: the new settings, the flag and the report line.cabal test, skipping theBoundedDeltaCommunitySpecproperty that hangs onmain): 1,656 examples, 5 failures. These are exactly the 5GrammarsSpecfailures already onmain(avi-1440,avi-1765); nothing new fails.Validation on real graphs
Default flags, with no
--no-community-capand no--edge-density Sparse. The inputs are file/import checkpoints built by the external resolver, which isolates D1–D3 from D4.src/cli(159)About the pc26 MQ target. The 370 came from the external Python prototype, which scored directed import records and split slightly differently: 23 runs, 1,515 modules. This implementation scores Bunch's deduplicated undirected MDG pairs. Its result has fewer, larger modules and higher modularity (0.205 against 0.156), but MQ 332. Task 5.3 is left unchecked. The design's target should be restated as Graphos's own measure, or accepted as missed.
Report MQ fixed in this PR (second commit). The
Partition qualityline used to score the exported graph, inferreddepends_on/inferrededges included, so a Bunch partition showed a lower MQ than the one it optimised. A Bunch partition is now scored over its MDG relations (Cluster.bunchPartitionQuality, with areport-consistencydelta inbunch-real-projects), and the line names them:The modularity on that line is still the directed computation that
modularity-undirected-view(#251) fixes; for nestjs it would be 0.364 once undirected. Merge note: that open change modifies the samereport-consistencyrequirement, so whichever change archives second must carry both texts.Tasks
tasks.md: 1.1, 2.1–2.4 and 3.1–3.4 are checked. Still open: 3.5 (optional Lean arm), 4.x (D4), 5.x (full validation, which needs D4) and 6.x.