Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: disable incremental compilation in dogfood to reduce target dir size #4513

Merged
merged 2 commits into from
Sep 14, 2019

Conversation

matthiaskrgr
Copy link
Member

@matthiaskrgr matthiaskrgr commented Sep 6, 2019

tests: execute dogfood tests with incremental compilation disabled reduces target/ dir size of when "cargo test"ing by around 2 gigs.

changelog: none

@llogiq
Copy link
Contributor

llogiq commented Sep 7, 2019

I tried this, and it appeared to slow down the inevitable "no space left on device", though not overcoming it. The dogfood_tests test seems to generate a lot of data on disk.

@llogiq
Copy link
Contributor

llogiq commented Sep 7, 2019

Interestingly, if I run cargo build && for d in clippy_workspace_tests clippy_workspace_tests/src clippy_workspace_tests/subcrate clippy_workspace_tests/subcrate/src clippy_dev rustc_tools_util; do CLIPPY_DOGFOOD=1 target/debug/cargo-clippy clippy --verbose -- -D clippy::all -D clippy::pedantic; done from a clean repo, it "only" takes about 5G.

Perhaps a way to clear the incremental caches between dogfood and dogfood_tests would give us sufficient wins without losing all of the benefits of incremental compilation.

@flip1995 flip1995 added the S-waiting-on-author Status: This is awaiting some action from the author. (Use `@rustbot ready` to update this status) label Sep 11, 2019
@matthiaskrgr
Copy link
Member Author

matthiaskrgr commented Sep 11, 2019

clear the incremental caches between dogfood and dogfood_tests

Hmm, I think the tests currently run in parallel, so we'd have to run them in linear order which will probably worsen execution time.

@matthiaskrgr
Copy link
Member Author

matthiaskrgr commented Sep 11, 2019

Clippy seems to invoke rustc with -C debuginfo=2 and I wonder how much that adds to the target dir size.
Is there a way to override it?
RUSTFLAGS ="-Cdebuginfo=0" or cargo clippy -- -Cdebuginfo=0 does not seem to work.

EDIT: nevermind it seems to work, pebkac.

@matthiaskrgr matthiaskrgr changed the title [WIP] tests: disable incr comp in dogfood tests: disable incr comp in dogfood Sep 11, 2019
@matthiaskrgr matthiaskrgr changed the title tests: disable incr comp in dogfood tests: disable incremental compilation in dogfood to reduce build artifact size Sep 11, 2019
@matthiaskrgr
Copy link
Member Author

previous behaviour:

cargo build
time cargo test
time: 1147 seconds
size target dir: 6503 m
// change a string inside clippy to trigger incr comp
time cargo test
time: 155 seconds
size: 6837 m

with this branch:

cargo build
time cargo test
time: 993 s
size target dir: 4146 m
// change string
time cargo test
time: 230 seconds
size target dir: 4479

@matthiaskrgr matthiaskrgr added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties and removed S-waiting-on-author Status: This is awaiting some action from the author. (Use `@rustbot ready` to update this status) labels Sep 11, 2019
@matthiaskrgr matthiaskrgr changed the title tests: disable incremental compilation in dogfood to reduce build artifact size tests: disable incremental compilation in dogfood to reduce target dir size Sep 11, 2019
Copy link
Member

@flip1995 flip1995 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me. Just deleted a 50GB target directory from Clippy yesterday.

@llogiq
Copy link
Contributor

llogiq commented Sep 14, 2019

@bors r+

@bors
Copy link
Collaborator

bors commented Sep 14, 2019

📌 Commit a1ecf39 has been approved by llogiq

@bors
Copy link
Collaborator

bors commented Sep 14, 2019

⌛ Testing commit a1ecf39 with merge 535bc1d...

bors added a commit that referenced this pull request Sep 14, 2019
tests: disable incremental compilation in dogfood to reduce target dir size

tests: execute dogfood tests with incremental compilation disabled reduces target/ dir size of when "cargo test"ing by around 2 gigs.

changelog: none
@bors
Copy link
Collaborator

bors commented Sep 14, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: llogiq
Pushing 535bc1d to master...

@bors bors merged commit a1ecf39 into rust-lang:master Sep 14, 2019
@llogiq
Copy link
Contributor

llogiq commented Sep 14, 2019

Thanks, @matthiaskrgr!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants