-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix corner cases for the greedy flag #1095
Conversation
Actually this is not a good solution after all. It only works if the comment contains another comment. |
Thank you for taking a look into this! |
Yes I think it is possible to optimize and generalize this approach. I will try a few things and then update this PR. |
I don't think we can make the At some point we have to decide if the extra effort of fixing |
f1d30ac
to
ffc662d
Compare
It should be fixed now. I've also added test cases to a few languages including one test case for template strings in Javascript. |
5a99df7
to
733a955
Compare
@Golmote Do you think this can be merged? |
@zeitgeist87 Again sorry for the long time before review. This looks good to me. Thank you so much for digging into this. Feel free to merge when you can! |
By refactoring the tokenize() method into two separate methods, it is possible to recursively clean up any broken tokens left by the greedy feature. This should fix the issue 1075.
@Golmote Thanks for the review! I've rebased it and solved the merge conflicts. |
By refactoring the tokenize() method into two separate methods, it is
possible to recursively clean up any broken tokens left by the greedy
feature.
This should fix the issue #1075.