Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Speed up the macro parser #50855

Merged
merged 3 commits into from
May 20, 2018
Merged

Speed up the macro parser #50855

merged 3 commits into from
May 20, 2018

Conversation

nnethercote
Copy link
Contributor

These three commits reduce the number of allocations done by the macro parser, in some cases dramatically. For example, for a clean check builds of html5ever, the number of allocations is reduced by 40%.

Here are the rustc-benchmarks that are sped up by at least 1%.

html5ever-check
        avg: -6.6%      min: -10.3%     max: -4.1%
html5ever
        avg: -5.2%      min: -9.5%      max: -2.8%
html5ever-opt
        avg: -4.3%      min: -9.3%      max: -1.6%
crates.io-check
        avg: -1.8%      min: -2.9%      max: -0.6%
crates.io-opt
        avg: -1.0%      min: -2.2%      max: -0.1%
crates.io
        avg: -1.1%      min: -2.2%      max: -0.2%

@nnethercote
Change TokenTreeOrTokenTreeVec to TokenTreeOrTokenTreeSlice.
6872377 
This avoids a `to_owned` call that can be hot, speeding up the various
runs of html5ever by 1--5%, and some runs of crates.io by 2--3%.
@rust-highfive
Copy link
Collaborator

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 18, 2018
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-3.9 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log. 

[00:04:56] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:57] tidy error: /checkout/src/libsyntax/ext/tt/macro_parser.rs:237: tab character
[00:04:57] tidy error: /checkout/src/libsyntax/ext/tt/macro_parser.rs:246: tab character
[00:04:58] some tidy checks failed
[00:04:58] 
[00:04:58] 
[00:04:58] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:58] 
[00:04:58] 
[00:04:58] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:58] Build completed unsuccessfully in 0:02:02
[00:04:58] Build completed unsuccessfully in 0:02:02
[00:04:58] Makefile:79: recipe for target 'tidy' failed
[00:04:58] make: *** [tidy] Error 1

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:1e2f5cfb
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@petrochenkov
Copy link
Contributor

@bors r=me after tidy is fixed

@petrochenkov
Copy link
Contributor

@bors delegate+

@bors
Copy link
Contributor

bors commented May 18, 2018

✌️ @nnethercote can now approve this pull request

@petrochenkov petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 18, 2018
@nnethercote
Introduce MatcherPosHandle.
fcf2b24 
This lets us store most `MatcherPos` instances on the stack. This speeds
up various runs of html5ever, the best by 3%.
@nnethercote
Make Directory::path a Cow.
ad47145 
Because we create a lot of these in the macro parser, but only very
rarely modify them.

This speeds up some html5ever runs by 2--3%.
@nnethercote
Copy link
Contributor Author

Thank you for the fast review.

@bors: r=petrochenkov

@bors
Copy link
Contributor

bors commented May 18, 2018

📌 Commit ad47145 has been approved by petrochenkov

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 18, 2018
@bors
Copy link
Contributor

bors commented May 20, 2018

⌛ Testing commit ad47145 with merge 4c26e2e...

bors added a commit that referenced this pull request May 20, 2018
@bors
Auto merge of #50855 - nnethercote:fewer-macro_parser-allocs, r=petro…
4c26e2e 
…chenkov

Speed up the macro parser

These three commits reduce the number of allocations done by the macro parser, in some cases dramatically. For example, for a clean check builds of html5ever, the number of allocations is reduced by 40%.

Here are the rustc-benchmarks that are sped up by at least 1%.
```
html5ever-check
        avg: -6.6%      min: -10.3%     max: -4.1%
html5ever
        avg: -5.2%      min: -9.5%      max: -2.8%
html5ever-opt
        avg: -4.3%      min: -9.3%      max: -1.6%
crates.io-check
        avg: -1.8%      min: -2.9%      max: -0.6%
crates.io-opt
        avg: -1.0%      min: -2.2%      max: -0.1%
crates.io
        avg: -1.1%      min: -2.2%      max: -0.2%
```
@bors
Copy link
Contributor

bors commented May 20, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: petrochenkov
Pushing 4c26e2e to master...

@bors bors merged commit ad47145 into rust-lang:master May 20, 2018
@nnethercote nnethercote deleted the fewer-macro_parser-allocs branch May 21, 2018 02:18
@mark-i-m
Copy link
Member

This code has gotten a lot of love this year :)

@mark-i-m mark-i-m mentioned this pull request Jul 4, 2018
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@petrochenkov petrochenkov

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@nnethercote @rust-highfive @petrochenkov @bors @mark-i-m @nikomatsakis