Make jump threading state sparse #127036

cjgillot · 2024-06-27T10:58:45Z

Continuation of #127024

Both dataflow const-prop and jump threading involve cloning the state vector a lot. This PR replaces the data structure by a sparse vector, considering:

that jump threading state is typically very sparse (at most 1 or 2 set entries);
that dataflow const-prop is disabled by default;
that place/value map is very eager, and prone to creating an overly large state.

The first commit is shared with the previous PR to avoid needless conflicts.

r? @oli-obk

rustbot · 2024-06-27T10:58:53Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

cjgillot · 2024-06-27T10:58:55Z

@bors try @rust-timer queue

Make jump threading state sparse Continuation of rust-lang#127024 Both dataflow const-prop and jump threading involve cloning the state vector a lot. This PR replaces the data structure by a sparse vector, considering: - that jump threading state is typically very sparse (at most 1 or 2 set entries); - that dataflow const-prop is disabled by default; - that place/value map is very eager, and prone to creating an overly large state. The first commit is shared with the previous PR to avoid needless conflicts. r? `@oli-obk`

bors · 2024-06-27T11:00:08Z

⌛ Trying commit 73f860a with merge ca4f312...

bors · 2024-06-27T12:42:20Z

☀️ Try build successful - checks-actions
Build commit: ca4f312 (ca4f31271eb3499af16c1f984419976a4efce1d3)

rust-timer · 2024-06-27T15:50:46Z

Finished benchmarking commit (ca4f312): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.7%	[-1.1%, -0.5%]	4
Improvements ✅ (secondary)	-1.9%	[-1.9%, -1.9%]	1
All ❌✅ (primary)	-0.7%	[-1.1%, -0.5%]	4

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 695.971s -> 696.966s (0.14%)
Artifact size: 326.67 MiB -> 326.70 MiB (0.01%)

oli-obk · 2024-06-28T08:50:43Z

@bors r+

bors · 2024-06-28T08:50:45Z

📌 Commit 73f860a has been approved by oli-obk

It is now in the queue for this repository.

Only track mentionned places for jump threading This PR aims to reduce the state space size in jump threading and dataflow const-prop opts. The current implementation walks the types of all locals, and creates a place for each possible projection. This can easily lead to a large number of places and tracked values, most being useless to the actual pass. With this PR, we instead collect places that appear syntactically in the MIR (first commit). However, this is not sufficient (second commit), and we miss places that we could track in aggregate assignments. The third commit tracks such assignments to mirror place projections, see the inline comment. This is complementary to rust-lang#127036 r? `@oli-obk`

Make jump threading state sparse Continuation of rust-lang#127024 Both dataflow const-prop and jump threading involve cloning the state vector a lot. This PR replaces the data structure by a sparse vector, considering: - that jump threading state is typically very sparse (at most 1 or 2 set entries); - that dataflow const-prop is disabled by default; - that place/value map is very eager, and prone to creating an overly large state. The first commit is shared with the previous PR to avoid needless conflicts. r? `@oli-obk`

bors · 2024-07-01T00:42:16Z

⌛ Testing commit 73f860a with merge 6e68440...

bors · 2024-07-01T00:51:35Z

💔 Test failed - checks-actions

cjgillot · 2024-07-01T15:43:51Z

Removed a duplicated commit.
@bors r=oli-obk

bors · 2024-07-01T15:43:53Z

📌 Commit 76244d4 has been approved by oli-obk

It is now in the queue for this repository.

bors · 2024-07-02T00:12:58Z

⌛ Testing commit 76244d4 with merge b5f0652...

Make jump threading state sparse Continuation of rust-lang#127024 Both dataflow const-prop and jump threading involve cloning the state vector a lot. This PR replaces the data structure by a sparse vector, considering: - that jump threading state is typically very sparse (at most 1 or 2 set entries); - that dataflow const-prop is disabled by default; - that place/value map is very eager, and prone to creating an overly large state. The first commit is shared with the previous PR to avoid needless conflicts. r? `@oli-obk`

rust-log-analyzer · 2024-07-02T01:01:21Z

The job dist-x86_64-msvc-alt failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

 finished in 5.105 seconds
[TIMING] core::build_steps::dist::JsonDocs { host: x86_64-pc-windows-msvc } -- 5.115
[TIMING] core::build_steps::dist::Mingw { host: x86_64-pc-windows-msvc } -- 0.000
thread 'main' panicked at src/lib.rs:1668:17:
failed to copy `C:\a\rust\rust\build\x86_64-pc-windows-msvc\stage2\bin\rustdoc.exe` to `C:\a\rust\rust\build\tmp\tarball\rustc\x86_64-pc-windows-msvc\image\bin\rustdoc.exe`: The process cannot access the file because it is being used by another process. (os error 32)
Build completed unsuccessfully in 0:33:30
  local time: Tue, Jul  2, 2024  1:01:09 AM
  network time: Tue, 02 Jul 2024 01:01:09 GMT
##[error]Process completed with exit code 1.

bors · 2024-07-02T01:01:56Z

💔 Test failed - checks-actions

cjgillot · 2024-07-02T14:08:55Z

@bors retry

bors · 2024-07-03T18:52:07Z

⌛ Testing commit 76244d4 with merge 2b90614...

bors · 2024-07-03T21:09:24Z

☀️ Test successful - checks-actions
Approved by: oli-obk
Pushing 2b90614 to master...

rust-timer · 2024-07-04T00:07:39Z

Finished benchmarking commit (2b90614): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.8%	[-1.1%, -0.5%]	5
Improvements ✅ (secondary)	-0.4%	[-0.4%, -0.4%]	1
All ❌✅ (primary)	-0.8%	[-1.1%, -0.5%]	5

Max RSS (memory usage)

Results (secondary -0.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.6%	[0.6%, 0.6%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.7%	[-1.7%, -1.7%]	1
All ❌✅ (primary)	-	-	0

Cycles

Results (secondary -9.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.5%	[0.5%, 0.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-10.5%	[-10.7%, -10.4%]	6
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 742.585s -> 742.115s (-0.06%)
Artifact size: 327.63 MiB -> 327.64 MiB (0.00%)

Only track mentioned places for jump threading This PR aims to reduce the state space size in jump threading and dataflow const-prop opts. The current implementation walks the types of all locals, and creates a place for each possible projection. This can easily lead to a large number of places and tracked values, most being useless to the actual pass. With this PR, we instead collect places that appear syntactically in the MIR (first commit). However, this is not sufficient (second commit), and we miss places that we could track in aggregate assignments. The third commit tracks such assignments to mirror place projections, see the inline comment. This is complementary to rust-lang#127036 r? `@oli-obk`

Only track mentioned places for jump threading This PR aims to reduce the state space size in jump threading and dataflow const-prop opts. The current implementation walks the types of all locals, and creates a place for each possible projection. This can easily lead to a large number of places and tracked values, most being useless to the actual pass. With this PR, we instead collect places that appear syntactically in the MIR (first commit). However, this is not sufficient (second commit), and we miss places that we could track in aggregate assignments. The third commit tracks such assignments to mirror place projections, see the inline comment. This is complementary to rust-lang/rust#127036 r? `@oli-obk`

rustbot assigned oli-obk Jun 27, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jun 27, 2024

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 27, 2024

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 27, 2024

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 28, 2024

cjgillot mentioned this pull request Jun 28, 2024

Only track mentioned places for jump threading #127087

Merged

cjgillot added the A-mir-opt Area: MIR optimizations label Jun 28, 2024

This comment has been minimized.

Sign in to view

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 1, 2024

cjgillot added 2 commits July 1, 2024 15:39

Swap encapsulation of DCP state.

1834f5a

Make jump threading state sparse.

76244d4

cjgillot force-pushed the sparse-state branch from 73f860a to 76244d4 Compare July 1, 2024 15:43

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 1, 2024

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 2, 2024

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 2, 2024

bors added the merged-by-bors This PR was explicitly merged by bors. label Jul 3, 2024

bors merged commit 2b90614 into rust-lang:master Jul 3, 2024
7 checks passed

rustbot added this to the 1.81.0 milestone Jul 3, 2024

cjgillot deleted the sparse-state branch July 4, 2024 05:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make jump threading state sparse #127036

Make jump threading state sparse #127036

cjgillot commented Jun 27, 2024

rustbot commented Jun 27, 2024

cjgillot commented Jun 27, 2024

This comment has been minimized.

bors commented Jun 27, 2024

bors commented Jun 27, 2024

This comment has been minimized.

rust-timer commented Jun 27, 2024

oli-obk commented Jun 28, 2024

bors commented Jun 28, 2024

bors commented Jul 1, 2024

This comment has been minimized.

bors commented Jul 1, 2024

cjgillot commented Jul 1, 2024

bors commented Jul 1, 2024

bors commented Jul 2, 2024

rust-log-analyzer commented Jul 2, 2024

bors commented Jul 2, 2024

cjgillot commented Jul 2, 2024

bors commented Jul 3, 2024

bors commented Jul 3, 2024

rust-timer commented Jul 4, 2024

Make jump threading state sparse #127036

Make jump threading state sparse #127036

Conversation

cjgillot commented Jun 27, 2024

rustbot commented Jun 27, 2024

cjgillot commented Jun 27, 2024

This comment has been minimized.

bors commented Jun 27, 2024

bors commented Jun 27, 2024

This comment has been minimized.

rust-timer commented Jun 27, 2024

Overall result: ✅ improvements - no action needed

oli-obk commented Jun 28, 2024

bors commented Jun 28, 2024

bors commented Jul 1, 2024

This comment has been minimized.

bors commented Jul 1, 2024

cjgillot commented Jul 1, 2024

bors commented Jul 1, 2024

bors commented Jul 2, 2024

rust-log-analyzer commented Jul 2, 2024

bors commented Jul 2, 2024

cjgillot commented Jul 2, 2024

bors commented Jul 3, 2024

bors commented Jul 3, 2024

rust-timer commented Jul 4, 2024

Overall result: ✅ improvements - no action needed