Skip to content

feat(avi-1765): make PHP, Kotlin, Perl, C# and Java produce real graphs - #256

Merged
jeremiepas merged 1 commit into
mainfrom
feature/AVI-1765-tree-sitter-grammars-php-kotlin-perl-csharp-java-imports
Nov 13, 2026
Merged

jeremiepas merged 1 commit into
mainfrom
feature/AVI-1765-tree-sitter-grammars-php-kotlin-perl-csharp-java-imports

Conversation

@jeremiepas

Copy link
Copy Markdown
Owner

Applies the avi-1765-add-tree-sitter-grammars-php-kotlin-perl-csharp-java-imports openspec change.

The four grammars were already registered (AVI-1716). Checking each task against the spec deltas, and running graphos over a mixed-language fixture corpus, showed the languages still did not produce usable graphs.

What was broken

Gap Effect
.pl/.pm/.psgi absent from fecCode Perl files were never detected — the registered grammar was unreachable
PHP import forms absent from apiSurfaceTypes use Foo\Bar; and include/require emitted nothing
C# root compilation_unit unregistered .cs files had no module node, so their using edges had no source
extractSpecifier only read quoted literals PHP/Perl/C#/Kotlin Import nodes had an empty label and no imports edge — forbidden by the import-resolution delta
Three dead Perl node names use Foo; is use_no_statement, sub f {} is function_definition, quoted require 'x.pl' does not parse at ABI 11
Java resolution walked ancestors only a src/test/java class could never resolve an import of a src/main/java class
Kotlin had no resolver orphan Import nodes — same spec violation
Identifier-less declarations Perl package, PHP/C# namespace, C#/Java fields all got node id "" and collided into one node per graph

What changed

  • Extraction — Perl extensions in fecCode; PHP import forms and C#'s compilation_unit whitelisted; Perl node names corrected against the vendored parser.c and documented in vendor/README.md.
  • Import edges — Convert.bareSpecifier reads unquoted specifiers, including the C# alias directive using A = B.C;. Java consults each ancestor's src/main/java and src/test/java, resolves import static a.b.C.m to the enclosing type, and never targets the importing file. Kotlin (LangKotlin) resolves against src/main/kotlin, then the Java roots for interop, then an external node.
  • Node identity — Core.extractIdentifier learns package_name / namespace_name, plus two fallbacks keyed on the declaration type so a qualified_name or a nested identifier is never mistaken for a name where it is really a reference.

Verification

  • graphos-test: 1663 examples, 1 failure, 4 pending (84 s).
  • graphos over a PHP/Perl/C#/Kotlin/Java fixture corpus: every language produces symbols, 14 imports edges, 0 empty-id nodes. A src/test/java class resolves to both its src/main/java imports; a Kotlin file resolves to a .kt sibling and to a .java class via interop.

Two pre-existing issues, both unrelated to this change

  1. The one test failure is the TypeScript type_identifier case. It belongs to the separate, unapplied avi-1440-ts-declaration-only-type-nodes change: that change's regression test is committed but its task 1.1 (remove type_identifier from apiSurfaceTypes) is not done. Left alone deliberately.
  2. cabal test graphos-test does not terminate on main. BoundedDeltaCommunitySpec.hs:224, "delta update equals full rebuild on small random graphs (property)", spins indefinitely. Verified at 362c1fc with this branch's changes stashed — identical hang — so CI's cabal test all is blocked independently of this work. The figures above come from a run with --skip "small random graphs". This likely needs its own issue.

Apply the avi-1765 openspec change. The four grammars were already registered
(AVI-1716), but checking each task against the spec deltas and against a run
over a mixed-language fixture corpus showed the languages did not yet produce
usable graphs.

Extraction

- Register .pl/.pm/.psgi in fecCode. They were registered as tree-sitter
  extensions but absent from the file-category defaults, so Perl files were
  never detected and the grammar was unreachable.
- Whitelist PHP's import forms (namespace_use_declaration and the
  include/require expressions, which nest inside an expression_statement) and
  C#'s compilation_unit file root, so .cs files get a module node and their
  using edges get a source.
- Correct three dead Perl node names against the vendored ABI 11 parser.c:
  `use Foo;` is use_no_statement, a named `sub f { }` is function_definition,
  and a quoted `require 'x.pl'` does not parse at all. Verified names are
  recorded in vendor/README.md.

Import edges

- Read unquoted specifiers (Convert.bareSpecifier), including the C# alias
  directive `using A = B.C;`. Without it PHP/Perl/C#/Kotlin Import nodes
  carried an empty label and no imports edge, which the import-resolution
  delta forbids.
- Java: consult each ancestor's src/main/java and src/test/java, so a class
  under src/test/java can resolve an import of one under src/main/java;
  resolve `import static a.b.C.m` to the enclosing type; never target the
  importing file.
- Kotlin: resolve imports against src/main/kotlin and src/test/kotlin first,
  then the Java roots for interop, then fall back to an external node.

Node identity

- Perl `package`, PHP/C# `namespace` and C#/Java field declarations resolved
  to no identifier, so each got the node id "" and every such node in a
  corpus collided on it, leaving one survivor for the whole graph.

Verification

- graphos-test: 1663 examples, 1 failure, 4 pending (84 s). The failure is
  pre-existing and belongs to the unapplied
  avi-1440-ts-declaration-only-type-nodes change. The run skips
  BoundedDeltaCommunitySpec's "small random graphs" property, which does not
  terminate on main either (verified at 362c1fc with these changes stashed).
- A graphos run over a PHP/Perl/C#/Kotlin/Java fixture corpus: every language
  produces symbols, 14 imports edges, 0 empty-id nodes.
@jeremiepas

Copy link
Copy Markdown
Owner Author

Independent confirmation of pre-existing issue 2 in the description: the test job on main (run 43291562097, merge of #254) started at 19:40:35Z and was still running 2h37m later. Every other job in that run — build, fixture, haddock, studio-contract — succeeded.

So cabal test all is hung on main right now, and will hit GitHub's 6h job timeout. The cause is tests/Graphos/Domain/Graph/BoundedDeltaCommunitySpec.hs:224, "delta update equals full rebuild on small random graphs (property)", reproduced locally on bare main at 362c1fc with this branch's changes stashed.

This PR's own test job will hang the same way — it is not a signal about this change. Local result with that one property skipped: 1663 examples, 1 failure, 4 pending in 84s, the failure being the unrelated AVI-1440 TypeScript case.

Worth a separate issue; this PR does not touch community-detection code.

@jeremiepas
jeremiepas merged commit 74a82de into main Nov 13, 2026
8 of 9 checks passed
@jeremiepas
jeremiepas deleted the feature/AVI-1765-tree-sitter-grammars-php-kotlin-perl-csharp-java-imports branch November 13, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant