Emit a line break before rescue after an omitted keyword argument value - #10
Open
tarebyte wants to merge 2 commits into
Open
Emit a line break before rescue after an omitted keyword argument value#10tarebyte wants to merge 2 commits into
tarebyte wants to merge 2 commits into
Conversation
…alue A value-omitted keyword argument (`foo key:`, Ruby 3.1+) may take its value from the following line, as in `foo key:\n bar`. To allow that, the pair rule offers the `_no_line_break` external token as a hint, and the scanner responds by withholding the line break that would otherwise terminate the statement. The `rescue` keyword can never begin a value, so withholding the line break before it left the parser in a state where `rescue_modifier` — which requires its body and the keyword to be on the same logical line — was preferred over the `rescue` clause of the enclosing body. `begin ... foo key: ... rescue => e ... end` therefore parsed as a rescue modifier wrapping the call, with the `=> e` reported as an ERROR. The grammar declares no conflicts, so this choice is made statically at generation time and cannot be corrected in `grammar.js`; the only way to distinguish the two is to give the parser the line break. The scanner now records when it withholds a line break and, when the next token turns out to be the `rescue` keyword, produces the line break after all. The identifier lookahead used for that check is shared with the existing hash-key / `identifier!` scanning so that words merely starting with `r` keep working. Fixes tree-sitter#237 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ac21705f-befb-4606-9c83-657e3e9ea0dc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A value-omitted keyword argument (
foo key:, Ruby 3.1+) may take its value from the following line, so thepairrule offers the_no_line_breakexternal token as a hint and the scanner responds by withholding the line break that would otherwise terminate the statement.rescuecan never begin a value, but with no line break emitted the parser preferredrescue_modifier— whose body and keyword must share a logical line — over therescueclause of the enclosing body. So this:parsed as a rescue modifier wrapping the call, with
=> ereported as anERROR. Prism parses it as aBeginNodewith afoo(key:)call and a rescue clause.The grammar declares no
conflicts, so the generated parser resolves that preference statically and always takes the modifier path. Expressing the distinction ingrammar.jswould mean the grammar deciding on a token the scanner has not emitted yet, so the fix is made where the lookahead already lives: the scanner now records when it withholds a line break and, if the next token turns out to be therescuekeyword, emits the line break after all. The identifier lookahead used for that check is shared with the existing hash-key /identifier!scanning so words that merely start withrkeep working.The withheld-line-break flag is deliberately not serialized. It is cleared at the top of every whitespace scan, set only while that scan is consuming whitespace, and consumed later in the same
scan()call, so it never has to survive across a serialize boundary.grammar.js,src/grammar.jsonandsrc/parser.care unchanged — this is a scanner-only fix.Validation
ruby/specfiles withmasterand with this branch produces byte-identical trees; the only diff lines are parse-timing numbers. This is a regression check rather than evidence for the fix: nothing in that corpus reaches the changed path.rescue => eand back again produces trees identical to fresh parses in both directions.Covered by new corpus tests:
rescueon the same line still parses asrescue_modifier(x = compute rescue nil,y = foo(1) rescue nil).rstill parse as values:rescue_handler,reload!,r,Regexp.Checked by direct parsing, not by checked-in corpus cases: rescue clauses in
begin,def,classanddoblocks, plus multiplerescueclauses withensure, all parse with no error nodes.One pre-existing corpus case was renamed and its expected tree updated from
rescue_modifierplusERRORto the corrected call-and-rescue tree.References
Addresses the
foo key:beforerescuecase reported in tree-sitter#237 — one of the two repros @aibaars reduced in that issue. The other repro,... 3, already parses as a beginless range onmasterand is already covered there bytest/corpus/declarations.txt:577; no new test is needed for it.