Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

librustc_lexer: Refactor the module #66015

Merged
merged 9 commits into from
Nov 6, 2019
Merged

librustc_lexer: Refactor the module #66015

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
0825b35
librustc_lexer: Add methods "first" and "second" to the "Cursor"
popzxc Nov 3, 2019
72767a8
librustc_lexer: Introduce "eat_while" and "eat_identifier" methods
popzxc Nov 3, 2019
e0c45f7
librustc_lexer: Make "eat_float_exponent" return bool instead of result
popzxc Nov 3, 2019
649a524
librustc_lexer: Simplify "single_quoted_string" method
popzxc Nov 3, 2019
d6f722d
librustc_lexer: Simplify "double_quoted_string" method
popzxc Nov 3, 2019
6e350bd
librustc_lexer: Simplify "raw_double_quoted_string" method
popzxc Nov 3, 2019
ecd2673
librustc_lexer: Simplify "lifetime_or_char" method
popzxc Nov 3, 2019
e8b8d2a
librustc_lexer: Reorder imports in lib.rs
popzxc Nov 3, 2019
31735b0
librustc_lexer: Make nth_char method private
popzxc Nov 3, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 15 additions & 11 deletions src/librustc_lexer/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -537,26 +537,30 @@ impl Cursor<'_> {

fn single_quoted_string(&mut self) -> bool {
debug_assert!(self.prev() == '\'');
// Parse `'''` as a single char literal.
if self.nth_char(0) == '\'' && self.nth_char(1) == '\'' {
// Check if it's a one-symbol literal.
if self.second() == '\'' && self.first() != '\\' {
self.bump();
self.bump();
return true;
}

// Literal has more than one symbol.

// Parse until either quotes are terminated or error is detected.
let mut first = true;
loop {
match self.first() {
// Probably beginning of the comment, which we don't want to include
// to the error report.
'/' if !first => break,
// Newline without following '\'' means unclosed quote, stop parsing.
'\n' if self.nth_char(1) != '\'' => break,
// End of file, stop parsing.
EOF_CHAR if self.is_eof() => break,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why the order of match arms was changed here.

Copy link
Contributor Author

@popzxc popzxc Nov 3, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I had two-level motivation here:

  1. I ordered match arms depending on the termination level (first match has return, then go exceptional cases with break, then go char-skipping arms (escaped char and any other char)).
  2. I thought that it's a bit more readable to have the normal exit condition to be the first match arm.

// Quotes are terminated, finish parsing.
'\'' => {
self.bump();
return true;
}
// Probably beginning of the comment, which we don't want to include
// to the error report.
'/' => break,
// Newline without following '\'' means unclosed quote, stop parsing.
'\n' if self.second() != '\'' => break,
// End of file, stop parsing.
EOF_CHAR if self.is_eof() => break,
// Escaped slash is considered one character, so bump twice.
'\\' => {
self.bump();
Expand All @@ -567,8 +571,8 @@ impl Cursor<'_> {
self.bump();
}
}
first = false;
}
// String was not terminated.
false
}

Expand Down