Introduce NonterminalKind for more type-safe mbe parsing #74826

matklad · 2020-07-27T13:19:15Z

It encapsulate the (part of) the interface between the parser and
macro by example (macro_rules) parser.

The second bit is somewhat more general parse_ast_fragment, which is
the reason why we keep some parse_xxx functions as public.

rust-highfive · 2020-07-27T13:19:18Z

r? @ecstatic-morse

(rust_highfive has picked a reviewer for you, use r? to override)

matklad · 2020-07-27T13:22:24Z

r? @petrochenkov

I also want to change MetaVarDecl(Span, Ident /* name to bind */, Ident /* kind of nonterminal */), to MetaVarDecl(Span, Ident /* name to bind */, Option<MbeFragment>),

matklad · 2020-07-27T13:31:07Z

I also want to change MetaVarDecl(Span, Ident /* name to bind /, Ident / kind of nonterminal /), to MetaVarDecl(Span, Ident / name to bind */, Option),

Ah, MetaVarDecl is also serializable. I think it makes sense to keep it as is then and not to mix serialization into parser's interface.

mark-i-m · 2020-07-27T17:10:31Z

This code is quite out of cache for me, but any improvements are very well! 🎉

src/librustc_parse/mbe_fragment.rs

matklad · 2020-08-01T15:04:13Z

Switched to NonterminalKind! This actually I think made it possible to use it in the MetaVarDecl, but that would be a somewhat bigger refactoring with higher risk of breaking something, so it might make sense to review the two separately.

matklad · 2020-08-01T16:02:07Z

Yup, so the second commit uses NonterminalKind for MetaVarDecl as well, ready for review now!

matklad · 2020-08-01T16:11:02Z

The remaining weirdness with Option is due to #40107 I belive. I'll tackle that next, seems fine to try to hard error now.

src/librustc_ast/token.rs

src/librustc_parse/lib.rs

It encapsulate the (part of) the interface between the parser and macro by example (macro_rules) parser. The second bit is somewhat more general `parse_ast_fragment`, which is the reason why we keep some `parse_xxx` functions as public.

This is more type safe and allows us to remove a few dead branches

Seems to be a fallout from rustfmt transition

petrochenkov · 2020-08-02T12:17:20Z

@bors r+

bors · 2020-08-02T12:17:22Z

📌 Commit 1a2d07e has been approved by petrochenkov

matklad · 2020-08-02T12:39:17Z

src/librustc_parse/parser/mod.rs


 use crate::lexer::UnmatchedBrace;
+use diagnostics::Error;
+pub use path::PathStyle;


Are there any word of mouth and/or written guidelines about the ordering of mods and uses? I've fixed this negatively, by not mixing uses and modules in random order, but i am wondering what the positive fix would look like. In rust-analyer, we are somewhat picky about this: https://github.com/rust-analyzer/rust-analyzer/blob/master/docs/dev/style.md#order-of-imports

I don't think there's an official guideline, but the most common order is

use crate::things; use other_crate::things; use std::things; mod foo; /* Other items */

bors · 2020-08-02T17:55:33Z

⌛ Testing commit 1a2d07e with merge 8ae5d23b3623b4622364808137967732e0783d70...

rust-log-analyzer · 2020-08-02T18:01:07Z

The job i686-msvc-1 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

bors · 2020-08-02T18:01:16Z

💔 Test failed - checks-actions

matklad · 2020-08-02T18:03:31Z

Seems to be a spurious network error:

    Updating crates.io index
error: failed to get `cc` as a dependency of package `bootstrap v0.0.0 (D:\a\rust\rust\src\bootstrap)`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  error inflating zlib stream; class=Zlib (5)
failed to run: D:\a\rust\rust\build\i686-pc-windows-msvc\stage0\bin\cargo.exe build --manifest-path D:\a\rust\rust\src/bootstrap/Cargo.toml --locked
Build completed unsuccessfully in 0:00:16
make: *** [Makefile:60: prepare] Error 1
Command failed. Attempt 5/5:
    Updating crates.io index
error: failed to get `cc` as a dependency of package `bootstrap v0.0.0 (D:\a\rust\rust\src\bootstrap)`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  error inflating zlib stream; class=Zlib (5)

@bors retry

bors · 2020-08-02T19:48:49Z

⌛ Testing commit 1a2d07e with merge f042d74...

bors · 2020-08-02T22:07:24Z

☀️ Test successful - checks-actions, checks-azure
Approved by: petrochenkov
Pushing f042d74 to master...

…y, r=matklad Restore public visibility on some parsing functions for rustfmt In rust-lang#74826 the visibility of several parsing functions was reduced. However, rustfmt is an external consumer of some of these functions as well and needs the visibility to be public, similar to other elements in rustc_parse such as `parse_ident` https://github.com/rust-lang/rust/blob/db534b3ac286cf45688c3bbae6aa6e77439e52d2/src/librustc_parse/parser/mod.rs#L433-L436

rust-highfive assigned ecstatic-morse Jul 27, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 27, 2020

rust-highfive assigned petrochenkov and unassigned ecstatic-morse Jul 27, 2020

matklad changed the title ~~WIP: Introduce MbeFragment~~ Introduce MbeFragment Jul 27, 2020

matklad force-pushed the mbe-fragment branch from 51d898c to 65b53aa Compare July 27, 2020 14:32

matklad force-pushed the mbe-fragment branch from 65b53aa to 7efaa2a Compare July 28, 2020 08:06

petrochenkov reviewed Jul 28, 2020

View reviewed changes

src/librustc_parse/mbe_fragment.rs Outdated Show resolved Hide resolved

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 28, 2020

matklad force-pushed the mbe-fragment branch from 7efaa2a to 15caf38 Compare August 1, 2020 14:28

matklad changed the title ~~Introduce MbeFragment~~ Introduce NonterminalKind for more type-safe mbe parsing Aug 1, 2020

matklad added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 1, 2020

petrochenkov reviewed Aug 2, 2020

View reviewed changes

src/librustc_ast/token.rs Outdated Show resolved Hide resolved

src/librustc_parse/lib.rs Outdated Show resolved Hide resolved

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 2, 2020

matklad force-pushed the mbe-fragment branch from 8a478f8 to 60ba566 Compare August 2, 2020 12:06

matklad added 3 commits August 2, 2020 14:09

Introduce NonterminalKind

2595d75

It encapsulate the (part of) the interface between the parser and macro by example (macro_rules) parser. The second bit is somewhat more general `parse_ast_fragment`, which is the reason why we keep some `parse_xxx` functions as public.

Use NonterminalKind for MetaVarDecl

7fdffe1

This is more type safe and allows us to remove a few dead branches

Formatting: don't mix mod and use

1a2d07e

Seems to be a fallout from rustfmt transition

matklad force-pushed the mbe-fragment branch from 60ba566 to 1a2d07e Compare August 2, 2020 12:09

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 2, 2020

matklad commented Aug 2, 2020

View reviewed changes

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 2, 2020

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 2, 2020

bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 2, 2020

bors merged commit f042d74 into rust-lang:master Aug 2, 2020

JohnTitor mentioned this pull request Aug 3, 2020

Fix link/wording for parse_nt rust-lang/rustc-dev-guide#819

Merged

calebcartwright mentioned this pull request Aug 30, 2020

Restore public visibility on some parsing functions for rustfmt #76115

Merged

cuviper added this to the 1.47.0 milestone May 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce NonterminalKind for more type-safe mbe parsing #74826

Introduce NonterminalKind for more type-safe mbe parsing #74826

matklad commented Jul 27, 2020

rust-highfive commented Jul 27, 2020

matklad commented Jul 27, 2020

matklad commented Jul 27, 2020

mark-i-m commented Jul 27, 2020

matklad commented Aug 1, 2020

matklad commented Aug 1, 2020

matklad commented Aug 1, 2020

petrochenkov commented Aug 2, 2020

bors commented Aug 2, 2020

matklad Aug 2, 2020

petrochenkov Aug 2, 2020

bors commented Aug 2, 2020

rust-log-analyzer commented Aug 2, 2020

bors commented Aug 2, 2020

matklad commented Aug 2, 2020

bors commented Aug 2, 2020

bors commented Aug 2, 2020

Introduce NonterminalKind for more type-safe mbe parsing #74826

Introduce NonterminalKind for more type-safe mbe parsing #74826

Conversation

matklad commented Jul 27, 2020

rust-highfive commented Jul 27, 2020

matklad commented Jul 27, 2020

matklad commented Jul 27, 2020

mark-i-m commented Jul 27, 2020

matklad commented Aug 1, 2020

matklad commented Aug 1, 2020

matklad commented Aug 1, 2020

petrochenkov commented Aug 2, 2020

bors commented Aug 2, 2020

matklad Aug 2, 2020

Choose a reason for hiding this comment

petrochenkov Aug 2, 2020

Choose a reason for hiding this comment

bors commented Aug 2, 2020

rust-log-analyzer commented Aug 2, 2020

bors commented Aug 2, 2020

matklad commented Aug 2, 2020

bors commented Aug 2, 2020

bors commented Aug 2, 2020