Skip to content

Inaccurate starting point of terminal tokens, including unwanted whitespaces #3966

@blindFS

Description

@blindFS

Problem

Related grammar:

  extras: ($) => [/[ \t]/, $.comment],
  ...
  ctrl_error: ($) =>
    seq(
      "error",
      "make",
      optional($._flag),  // <--- works fine without this line
      field("error_record", $.val_record),
    ),
image

You can see that the "make" token starts early at [0, 5] instead of [0, 9].

If I remove the line of optional($._flag), it outputs correct region again.

The code of the _flag rule is here, I think it has nothing to do with whitespaces.

Steps to reproduce

git clone https://github.com/nushell/tree-sitter-nu
cd tree-sitter-nu
tree-sitter generate

Expected behavior

The "make" token should start at [0, 9] as expected.

Tree-sitter version (tree-sitter --version)

tree-sitter 0.24.3

Operating system/version

macOS 15.1.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    parserRelated to parsing

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions