Parse Ruby with Prism instead of lib-ruby-parser

[dduugg]

Summary

• pks parses Ruby with Prism in place of lib-ruby-parser. lib-ruby-parser follows Ruby 3.1's grammar, its repository was archived in 2024, and its last release was in April 2024. So it can't read newer syntax, such as Ruby 3.2's anonymous argument forwarding (def foo(*) = bar(*)). pks lost the references in such files, and since 0.7.0 it warns about them. Prism is the parser Ruby itself has used since 3.4.
• Both tree walkers collect the same references and definitions as before. Where the two parsers' trees differ, the walkers make up for it:
  • lib-ruby-parser gave foo&.bar and foo[1] node types of their own, while Prism makes them calls.
  • lib-ruby-parser made a.b ||= c a call, while Prism doesn't.
  • Prism has ten node types for assigning a constant. They share one macro, which visits only the value assigned, as lib-ruby-parser's casgn handling did.
• A template's yield doesn't warn. Prism reports yield outside a method as an error, and the Ruby that pks makes from an ERB template has its yield at the top level, as a Builder template's is. Rails runs both as the body of a method, so that error is ignored in them.
• Warning text changes. Syntax-error warnings use Prism's first reported error, in Prism's words. ruby-prism's Diagnostic::message panics on a message that isn't UTF-8, which happens when a magic comment names another encoding and Prism's quote of the file ends mid-character, so pks catches that and warns without the message.
• A new CI job builds every release target on the runner dist uses for it, because release builds don't run on pull requests. It builds as dist does, with the dist profile and, on Windows, a static C runtime, and runs each binary on a fixture. A Build job reports whether every target built, so main's ruleset has one name to require once this merges.
• No more Intel macOS binary. Releases drop x86_64-apple-darwin, and the Apple Silicon binary is built on macos-26, the newest arm64 macOS that GitHub's runner images list as generally available, in place of dist's default, the deprecated macos-14. macos-26 is named explicitly rather than as macos-latest. [dist.github-custom-runners] sets it for the release, and the CI job uses the same runner. dist generate leaves release.yml unchanged and dist plan lists macos-26.

Prism version

• ruby-prism = "1.9.0", the newest minor release (January 2026). 1.8.1, from March, is a patch release on the older line.
• The crate's parse takes no options, so Prism reads the newest Ruby syntax it knows, and pks can't pin it to an older version. Dependabot (Warn about files that don't parse, read <%== in ERB, and release 0.7.0 #72) will keep the requirement current.
• So pks warns about syntax that only older Rubies accept, such as a[k: 1] = v, and it in a block is the block's parameter, not a call, which with --experimental-parser can stop a class counting as defined. The changelog says so.

Build requirements and risk per target

• What builds need now: ruby-prism-sys compiles Prism's C source with cc, and generates its bindings with bindgen on every build. So building pks now needs a C compiler and libclang. INSTALLATION.md says so.
• Per target, on the runners dist uses:
  • aarch64-apple-darwin (macos-26): built and tested locally on Apple Silicon.
  • x86_64-unknown-linux-gnu and aarch64-unknown-linux-gnu (ubuntu-22.04 and ubuntu-22.04-arm): both images list Clang 13 to 15. The new CI job builds them.
  • x86_64-unknown-linux-musl (ubuntu-22.04): the most uncertain. dist installs musl-tools, so cc uses musl-gcc, but bindgen has to parse Prism's headers for a musl target. The new job installs musl-tools as dist does, and builds it.
  • x86_64-pc-windows-msvc (windows-2022): the image includes LLVM, and CRuby builds Prism with MSVC. The new job builds it.
• Remaining risk: the new job runs cargo build --profile dist --target … on dist's runners, not dist build itself. This PR's CI run is the evidence for the targets I couldn't build locally. A release dry run on 5ebbfba (run) ran dist build itself on all five targets, plus the installers, and passed.

Equivalence

I built main and this branch side by side with an uncommitted probe. On each run it recorded every file's parsed references, with names, namespaces and positions, plus its definitions. I compared those, and the output of check -o json (violations and stale todos), list-references and list-definitions, all read-only with --no-cache, under both parsers.

• Fixtures: every fixture under tests/fixtures, 72 runs. All 288 parse results and all three commands' output are identical.
  • One fixture panics on both branches, with a message that names a different duplicate constant from run to run on main too.
• Three large Rails apps: 6 runs, 133,098 files parsed.
  • The references and definitions of every file are identical.
  • So are the 192,201 check violations, the 550,054 list-references entries (byte for byte), list-definitions, and stderr. No new warnings appear.
  • 8 files differ only in where a reference's range ends, which pks doesn't use: an association-style call followed by a block, which Prism's range takes in.
  • 2 files collect the same references in a different order, which no output depends on.
  • With Warn about files that don't parse, read <%== in ERB, and release 0.7.0 #72's <%== fix, no file in these apps failed to parse before, so none gains references.
• About 270,000 more files, deduplicated: installed gems, Ruby's stdlib, ruby/ruby's tests and specs, Sorbet's parser test data and other checkouts. Every file that both parsers read without error gives the same references and definitions, apart from order and the ends of ranges. The PR never panicked, and main panicked on 6, each with a module named by an expression, as module self::Foo is.
• Unit tests pin edge cases, and pass under both lib-ruby-parser and Prism. These cover calls that do and don't count as a class changing behavior: self.a ||= 1, self.a, self.b = 1, 2, /(?<a>.)/ =~ BAR, BAR.[](1), BAR[1], BAR[1] = 2 and @a&.b ||= 1.
• Files with a syntax error: Prism recovers more of them. In a class with an unclosed def b(, it still finds a reference earlier in the class that lib-ruby-parser lost.
• Performance: on the largest of the three apps, check takes as long as before. Medians of 8 alternating runs: 5.95 s against 6.03 s by default, 5.23 s against 5.20 s with --experimental-parser. Peak memory is 5 to 7% lower (1,576 MB against 1,655 MB, and 1,130 MB against 1,220 MB). The machine's load average was 30 to 40 throughout, so treat the times as roughly equal rather than exact.

Native-extension risk

Linking a second copy of libprism into a Ruby 3.4+ process, which already contains one, could clash on symbols. I found nothing that loads pks into a Ruby process:

• pks's library builds only as an rlib, and nothing in src/ uses its serde_magnus dependency.
• The one Ruby native extension in the org that uses serde_magnus, code_ownership's, links codeowners-rs, not pks.
• No rubyatscale repository depends on pks as a crate.
• The pks crate on crates.io is the old alexevanczuk/packs 0.2.40, and nothing depends on it.

Out of scope

• Prism's partial_script option would replace matching the "Invalid yield" message, but the crate doesn't expose parse options. A test fails if Prism's wording changes.
• Removing the unused serde_magnus dependency.
• ERB source positions, and the inconsistent ranges for constant definitions: FOO = x spans the assignment, while FOO ||= x spans only the name. Both are kept as they were.

Test plan

• cargo test, with and without RUST_LOG=debug, plus cargo fmt --check and cargo clippy --all-targets --all-features -- -Dwarnings on each commit.
• New tests:
  • references in Ruby 3.2 * and ** forwarding, under both parsers, with no warning (this one fails on main);
  • <%= yield %> without a warning;
  • which calls count as a class changing behavior, passing under both lib-ruby-parser and Prism;
  • each of the ten constant-assignment nodes, under both parsers;
  • module self::Foo and class self::Foo in packwerk mode, &. and block association calls, and the reference that Prism recovers before a syntax error;
  • yield in a Builder template, and a Prism message that isn't UTF-8, without a panic.
• The new Build jobs pass on all five targets.