Always calculate glob map but only for glob uses #57392

Xanewok · 2019-01-06T23:48:06Z

Previously calculating glob map was opt-in, however it did record node id -> ident use for every use directive. This aims to see if we can unconditionally calculate the glob map and not regress performance.

Main motivation is to get rid of some of the moving pieces and simplify the compilation interface - this would allow us to entirely remove CrateAnalysis. Later, we could easily expose a relevant query, similar to the likes of maybe_unused_trait_import (so using precomputed data from the resolver, but which could be rewritten to be on-demand).

r? @nikomatsakis

Local perf run showed mostly noise (except ctfe-stress-*) but I'd appreciate if we could do a perf run run here and double-check that this won't regress performance.

Xanewok · 2019-01-06T23:51:42Z

src/librustc_resolve/lib.rs

-    fn add_to_glob_map(&mut self, id: NodeId, ident: Ident) {
-        if self.make_glob_map {
-            self.glob_map.entry(id).or_default().insert(ident.name);
+    #[inline]


Not sure if premature? Was afraid this might evade inlining and it's small enough but we can get rid of it if needed.

Zoxc · 2019-01-07T00:16:01Z

@bors try

bors · 2019-01-07T00:16:16Z

⌛ Trying commit 0a82177 with merge 4ff2a6e...

@nikomatsakis

Always calculate glob map but only for glob uses Previously calculating glob map was *opt-in*, however it did record node id -> ident use for every use directive. This aims to see if we can unconditionally calculate the glob map and not regress performance. Main motivation is to get rid of some of the moving pieces and simplify the compilation interface - this would allow us to entirely remove `CrateAnalysis`. Later, we could easily expose a relevant query, similar to the likes of `maybe_unused_trait_import` (so using precomputed data from the resolver, but which could be rewritten to be on-demand). r? @nikomatsakis Local perf run showed mostly noise (except `ctfe-stress-*`) but I'd appreciate if we could do a perf run run here and double-check that this won't regress performance.

rust-highfive · 2019-01-07T01:34:36Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:086168f9:start=1546818570894539900,finish=1546818643800232979,duration=72905693079
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-both (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:12:19] 
[01:12:19] running 118 tests
[01:12:46] .iiiii...i.....i..i...i..i.i..i.ii..i.....i..i....i..........iiii..........i...ii...i.......ii.i.i.i 100/118
[01:12:51] ......iii.i.....ii
[01:12:51] 
[01:12:51]  finished in 31.394
[01:12:51] travis_fold:end:test_debuginfo

---
travis_time:end:1c17a7e0:start=1546824872330960908,finish=1546824872335806369,duration=4845461
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:03358ddf
$ ln -s . checkout && for CORE

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-01-07T02:13:08Z

☀️ Test successful - status-travis
State: approved= try=True

Zoxc · 2019-01-07T02:14:13Z

@rust-timer build 4ff2a6e

rust-timer · 2019-01-07T02:14:14Z

Success: Queued 4ff2a6e with parent b92552d, comparison URL.

rust-timer · 2019-01-07T06:52:54Z

Finished benchmarking try commit 4ff2a6e

Xanewok · 2019-01-07T08:13:26Z

Perf seems unaffected, with interesting greens in ctfe-stress-* (due to inlined directive.is_glob() check? No idea 🤷‍♂️ ).

petrochenkov · 2019-01-07T12:01:12Z

Perf seems unaffected, with interesting greens in ctfe-stress-* (due to inlined directive.is_glob() check? No idea

Probably noise.
add_to_glob_map is a drop in the ocean compared to what the rest of resolve does, I wouldn't expect the change to have any noticeable effect.

nikomatsakis · 2019-01-07T14:57:39Z

r? @petrochenkov

Seems good to me, but I would prefer for @petrochenkov to sign off too, though based on their comments in thread I am guessing they are fine with it.

A question though: can the glob map be computed from other bits of data (i.e., lazilly, after the fact)? It seems like the idea scenario longer term would be that there is a "core query" that computes the main name resolution results, and then the glob map is computed only when needed based on those other results.

Note that there is a mild memory usage regression in many cases, though in some cases we see benefits too (presumably because we are computing less data? or noise? not sure).

petrochenkov · 2019-01-08T01:31:01Z

can the glob map be computed from other bits of data (i.e., lazilly, after the fact)?

No, the only similar thing is export_map, but it doesn't keep the mapping "NodeId -> Names" for globs.
This kind of data, in general, doesn't leave resolve because it is not used by later stages, glob map was added solely for save-analysis.

petrochenkov · 2019-01-08T01:31:08Z

@bors r+

bors · 2019-01-08T01:31:09Z

📌 Commit 71c6402 has been approved by petrochenkov

Xanewok · 2019-01-08T09:10:28Z

can the glob map be computed from other bits of data (i.e., lazilly, after the fact)?

I wanted to take this approach but I wasn't sure how to tackle this - it seems that the data from which we can derive this isn't useful outside of the save-analysis, just like @petrochenkov is saying. Maybe another visiting pass would work? However, at a first glance it seems to be inherently tied to the core resolution. Maybe we could see if we can make it on-demand later, as we go?

bors · 2019-01-10T06:06:18Z

⌛ Testing commit 71c6402 with merge e68c08e8f168df8bcad26ef91b9fb1b565ebf631...

bors · 2019-01-10T06:06:28Z

💔 Test failed - status-appveyor

pietroalbini · 2019-01-10T08:19:37Z

@bors retry
AppVeyor... what's wrong with you today?

bors · 2019-01-13T11:01:48Z

📌 Commit b1b64bd has been approved by petrochenkov

Xanewok · 2019-01-13T11:07:11Z

@pietroalbini ah, I thought the r+ rights allowed me to retry in #51487 (comment) (but it seems it was the try permissions 🤔)

Thanks for the heads up!

pietroalbini · 2019-01-13T11:09:01Z

Yeah, retry is granted by the try permission (along with try and p=).

Centril · 2019-01-15T17:47:21Z

@bors p=5

Rollup fairness due to submodule changes.

Xanewok · 2019-01-15T20:15:38Z

Looking at https://buildbot2.rust-lang.org/homu/queue/rust I’m afraid the priority didn’t get assigned.

Xanewok · 2019-01-16T16:58:30Z

Let’s try above again.

@bors p=5

bors · 2019-01-16T19:34:23Z

⌛ Testing commit b1b64bd with merge 59c1cac...

@nikomatsakis

Always calculate glob map but only for glob uses Previously calculating glob map was *opt-in*, however it did record node id -> ident use for every use directive. This aims to see if we can unconditionally calculate the glob map and not regress performance. Main motivation is to get rid of some of the moving pieces and simplify the compilation interface - this would allow us to entirely remove `CrateAnalysis`. Later, we could easily expose a relevant query, similar to the likes of `maybe_unused_trait_import` (so using precomputed data from the resolver, but which could be rewritten to be on-demand). r? @nikomatsakis Local perf run showed mostly noise (except `ctfe-stress-*`) but I'd appreciate if we could do a perf run run here and double-check that this won't regress performance.

bors · 2019-01-16T21:26:14Z

💔 Test failed - checks-travis

rust-highfive · 2019-01-16T21:26:16Z

The job arm-android of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[01:48:47] test string::test_str_truncate ... ok
[01:48:47] test string::test_reserve_exact ... ok
[01:48:47] test string::test_str_truncate_split_codepoint ... ok
[01:48:47] test string::test_str_truncate_invalid_len ... ok
[01:48:47] died due to signal 11
[01:48:47] 
[01:48:47] 
[01:48:47] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "arm-linux-androideabi" "-j" "4" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "-p" "alloc" "--"
[01:48:47] expected success, got: exit code: 3
---
travis_time:end:0e1b8b14:start=1547673899900240099,finish=1547673899920749297,duration=20509198
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:1c36552c
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:324fe60d
travis_time:start:324fe60d
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:18f5d224
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Xanewok · 2019-01-16T21:34:18Z

Segfault in collection tests 🤔 This should be unrelated.

@bors retry

bors · 2019-01-16T23:25:50Z

⌛ Testing commit b1b64bd with merge c40b977...

@nikomatsakis

Always calculate glob map but only for glob uses Previously calculating glob map was *opt-in*, however it did record node id -> ident use for every use directive. This aims to see if we can unconditionally calculate the glob map and not regress performance. Main motivation is to get rid of some of the moving pieces and simplify the compilation interface - this would allow us to entirely remove `CrateAnalysis`. Later, we could easily expose a relevant query, similar to the likes of `maybe_unused_trait_import` (so using precomputed data from the resolver, but which could be rewritten to be on-demand). r? @nikomatsakis Local perf run showed mostly noise (except `ctfe-stress-*`) but I'd appreciate if we could do a perf run run here and double-check that this won't regress performance.

bors · 2019-01-17T02:12:11Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: petrochenkov
Pushing c40b977 to master...

@Zoxc

Move glob map use to query and get rid of CrateAnalysis ~Also includes commits from ~rust-lang#57392 and ~rust-lang#57436 With glob map calculated unconditionally in rust-lang#57392, this PR moves the calculated glob map to `GlobalCtxt` and exposes a relevant query (as we do with other queries which copy precomputed data over from the `Resolver`). This allows us to get rid of the `CrateAnalysis` struct in an attempt to simplify the compiler interface. cc @Zoxc r? @nikomatsakis @Zoxc @petrochenkov

@nikomatsakis

Querify `entry_fn` Analogous to rust-lang#57570 but this will also require few fixups in Miri so I decided to separate that (and it seems [CI doesn't let us break tools anymore](rust-lang#57392 (comment))? Or was that because it was a rollup PR?) r? @nikomatsakis

rust-highfive assigned nikomatsakis Jan 6, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 6, 2019

Xanewok commented Jan 6, 2019

View reviewed changes

Xanewok mentioned this pull request Jan 7, 2019

Add Xanewok to try users rust-lang/rust-central-station#125

Merged

rust-highfive assigned petrochenkov and unassigned nikomatsakis Jan 7, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 8, 2019

Xanewok mentioned this pull request Jan 9, 2019

Move glob map use to query and get rid of CrateAnalysis #57476

Merged

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 10, 2019

Xanewok mentioned this pull request Jan 13, 2019

Querify entry_fn #57573

Merged

Centril mentioned this pull request Jan 13, 2019

Rollup of 7 pull requests #57574

Closed

Centril mentioned this pull request Jan 13, 2019

Rollup of 6 pull requests #57575

Closed

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 16, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 16, 2019

bors mentioned this pull request Jan 16, 2019

Prepare beta 1.33.0 #57670

Closed

bors merged commit b1b64bd into rust-lang:master Jan 17, 2019

bors mentioned this pull request Jan 17, 2019

rustdoc: Add option to persist doc test executables #56189

Merged

Xanewok mentioned this pull request Jan 17, 2019

Don't pass now removed MakeGlob parameter to rustc rust-lang/rls#1238

Merged

Xanewok deleted the always-calc-glob-map branch February 22, 2021 16:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Always calculate glob map but only for glob uses #57392

Always calculate glob map but only for glob uses #57392

Xanewok commented Jan 6, 2019

Xanewok Jan 6, 2019

nikomatsakis Jan 7, 2019

Zoxc commented Jan 7, 2019

bors commented Jan 7, 2019

rust-highfive commented Jan 7, 2019

bors commented Jan 7, 2019

Zoxc commented Jan 7, 2019

rust-timer commented Jan 7, 2019

rust-timer commented Jan 7, 2019

Xanewok commented Jan 7, 2019

petrochenkov commented Jan 7, 2019

nikomatsakis commented Jan 7, 2019

petrochenkov commented Jan 8, 2019

petrochenkov commented Jan 8, 2019

bors commented Jan 8, 2019

Xanewok commented Jan 8, 2019

bors commented Jan 10, 2019

bors commented Jan 10, 2019

pietroalbini commented Jan 10, 2019

bors commented Jan 13, 2019

Xanewok commented Jan 13, 2019

pietroalbini commented Jan 13, 2019

Centril commented Jan 15, 2019

Xanewok commented Jan 15, 2019

Xanewok commented Jan 16, 2019

bors commented Jan 16, 2019

bors commented Jan 16, 2019

rust-highfive commented Jan 16, 2019

Xanewok commented Jan 16, 2019

bors commented Jan 16, 2019

bors commented Jan 17, 2019

Always calculate glob map but only for glob uses #57392

Always calculate glob map but only for glob uses #57392

Conversation

Xanewok commented Jan 6, 2019

Xanewok Jan 6, 2019

Choose a reason for hiding this comment

nikomatsakis Jan 7, 2019

Choose a reason for hiding this comment

Zoxc commented Jan 7, 2019

bors commented Jan 7, 2019

rust-highfive commented Jan 7, 2019

bors commented Jan 7, 2019

Zoxc commented Jan 7, 2019

rust-timer commented Jan 7, 2019

rust-timer commented Jan 7, 2019

Xanewok commented Jan 7, 2019

petrochenkov commented Jan 7, 2019

nikomatsakis commented Jan 7, 2019

petrochenkov commented Jan 8, 2019

petrochenkov commented Jan 8, 2019

bors commented Jan 8, 2019

Xanewok commented Jan 8, 2019

bors commented Jan 10, 2019

bors commented Jan 10, 2019

pietroalbini commented Jan 10, 2019

bors commented Jan 13, 2019

Xanewok commented Jan 13, 2019

pietroalbini commented Jan 13, 2019

Centril commented Jan 15, 2019

Xanewok commented Jan 15, 2019

Xanewok commented Jan 16, 2019

bors commented Jan 16, 2019

bors commented Jan 16, 2019

rust-highfive commented Jan 16, 2019

Xanewok commented Jan 16, 2019

bors commented Jan 16, 2019

bors commented Jan 17, 2019