Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix misparsing in/after JSX (#6677) #6686

Merged
merged 1 commit into from
Mar 20, 2024
Merged

fix misparsing in/after JSX (#6677) #6686

merged 1 commit into from
Mar 20, 2024

Conversation

tsnobip
Copy link
Contributor

@tsnobip tsnobip commented Mar 19, 2024

Fixes #6677.

The tracking of JSX mode should be more robust, we basically pop JSX mode where it should (inside children for example), and we pop it before checking for the end of JSX (because expect also parses the next token and we don't want it to be parsed in JSX mode).

@tsnobip tsnobip changed the base branch from master to 11.0_release March 19, 2024 17:50
@tsnobip tsnobip marked this pull request as ready for review March 19, 2024 17:51
@tsnobip tsnobip requested a review from zth March 19, 2024 17:59
@tsnobip tsnobip force-pushed the fix_hyphens_in_jsx branch from 1b884e6 to ec4c09f Compare March 19, 2024 18:24
Comment on lines +2615 to 2616
Scanner.popMode p.scanner Jsx;
Parser.expect GreaterThan p;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we expect the end of the JSX so we can pop JSX mode

Comment on lines +2636 to 2637
Scanner.popMode p.scanner Jsx;
Parser.expect GreaterThan p;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

@@ -2687,7 +2689,6 @@ and parseJsxOpeningOrSelfClosingElement ~startPos p =
* jsx-children ::= primary-expr* * => 0 or more
*)
and parseJsx p =
Scanner.popMode p.scanner Jsx;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was not needed

@@ -2700,7 +2701,6 @@ and parseJsx p =
parseJsxFragment p
| _ -> parseJsxName p
in
Scanner.popMode p.scanner Jsx;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed either

Comment on lines +2717 to +2720
if p.token = LessThan then p.token <- Scanner.reconsiderLessThan p.scanner;
Parser.expect LessThanSlash p;
Parser.expect GreaterThan p;
Scanner.popMode p.scanner Jsx;
Parser.expect GreaterThan p;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we first check if the next token is not </ instead of just < to test the end of the fragment with a following > (forming a </>), if it's the case then we can pop JSX mode.

| _ -> None

and parseJsxProps p =
parseRegion ~grammar:Grammar.JsxAttribute ~f:parseJsxProp p

and parseJsxChildren p =
Scanner.popMode p.scanner Jsx;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSX children should not be parsed in JSX mode because they could be any kind of expression between curly braces.

let children = List.rev (loop p []) in
(false, children)
in
Scanner.setJsxMode p.scanner;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

once we're done parsing the children, we reset JSX mode because we have to parse the closing tag.

@mununki
Copy link
Member

mununki commented Mar 20, 2024

side-note: This is off-topic for this PR, but it seems like the current implementation of parsing with Jsx and Diamond as state values for scanner.mode: mode list makes it critical where setMode and popMode are used. I think the parser implementation would be improved if the scan function could scan without branching based on the state of Jsx and Diamond, and only for Jsx, have hipen handle the identifier when parsing.
What do you think? (and I'm not talking about working on this in this PR)

@tsnobip
Copy link
Contributor Author

tsnobip commented Mar 20, 2024

side-note: This is off-topic for this PR, but it seems like the current implementation of parsing with Jsx and Diamond as state values for scanner.mode: mode list makes it critical where setMode and popMode are used. I think the parser implementation would be improved if the scan function could scan without branching based on the state of Jsx and Diamond, and only for Jsx, have hipen handle the identifier when parsing. What do you think? (and I'm not talking about working on this in this PR)

I think the current implementation is far from being ideal. Basically, it was bearable before because the implementation was not really sensitive to the mode. The result was that the JSX mode was leaking most of the time but basically it didn't have any impact. Now that the JSX mode is more different, you have to be careful about when you set and pop it. I even tried modifying the parser in order to reflect the mode in the type of the parser, but it was not really possible because of the way the parser is designed (using mutation).

The easiest solution would likely be as you suggested to just get rid of the mode in the parser and likely add a parameter to Parser.next or something like that. What do you think?

@mununki
Copy link
Member

mununki commented Mar 20, 2024

The easiest solution would likely be as you suggested to just get rid of the mode in the parser and likely add a parameter to Parser.next or something like that. What do you think?

Agreed.
I'm going to try in my local dev, and share you how it would be.

@cknitt
Copy link
Member

cknitt commented Mar 20, 2024

@tsnobip Thanks a lot for the fix! 👍

Regarding the refactoring of the mode handling, I think we should do that on the master branch. For the 11.0_release branch, let's just merge this fix.

The tests are running without errors, and I also just tried compiling a large project with these changes, and everything worked fine.
So good to go from my point of view. @zth what do you think?

Copy link
Collaborator

@zth zth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cknitt yeah let's ship this in a new rc and then work on the rest in master.

@zth
Copy link
Collaborator

zth commented Mar 20, 2024

Great work @tsnobip!

@zth zth merged commit 49212ab into 11.0_release Mar 20, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants