Repository navigation
Conversation
Prism parses recursively, as the walk of its tree does, so each level of nesting in a file takes stack. A rayon thread has 2 MiB by default, which with Prism holds only about 1,400 to 2,500 levels of nested calls, hashes, arrays or blocks, where lib-ruby-parser held 16,000 or more. Past that, the whole run aborts with a stack overflow. Prism stops nesting after 10,000 levels and reports an error in the file. 16 MiB holds that many levels of each construct tried, where 8 MiB doesn't for arrays, parentheses, `!`, splats or indexes. A thread's stack is reserved, not committed, so peak memory on two large Rails apps is unchanged. A debug build takes more than ten times the stack per level, so the test's 500 levels overflow 2 MiB there, but not 16 MiB.
pks catches the panic that `Diagnostic::message` raises on a message that isn't UTF-8, but Rust's default panic hook had already printed it, with a path into the cargo registry that ruby-prism was built from, so the run looked as if it had crashed. A hook now leaves out a panic raised while a message is read, on that thread only, and passes any other to the hook it replaced.
dduugg
marked this pull request as ready for review
October 13, 2026 01:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two fixes from a review of #73 after it merged. Neither problem has been released, since #73 didn't bump the version.
fatal runtime error: stack overflow. lib-ruby-parser held 16,000 levels or more. The global thread pool's threads now get 16 MiB.Diagnostic::messageraises on a message that isn't UTF-8, but Rust's default panic hook printed it first, with a path into the cargo registry, so the run looked as if it had crashed. Now the warning is all that prints.Design notes
!, splats and indexes. Method chains andelsifchains are flat in Prism's grammar but deep in its tree, so the cap doesn't stop them, but they overflowed 2 MiB only past about 16,000 levels.Diagnostic::messagecould replace invalid bytes instead of panicking.Test plan
list-referenceson 500 nested arrays, under both parsers. It aborts without the stack change.cargo fmt --check,cargo clippy --all-targets --all-features -- -Dwarningsand the full suite pass.