Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use download-rustc = "if-unchanged" in x86_64-gnu #112143

Closed
wants to merge 11 commits into from
Closed

Use download-rustc = "if-unchanged" in x86_64-gnu #112143

wants to merge 11 commits into from

Commits on Jul 9, 2023

  1. Make --dry-run more useful when download-rustc is enabled 

    Previously, it would always treat download-rustc as set to false, which made bootstrap issues with
download-rustc hard to debug.
    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    d0bcbf4 View commit details
    Browse the repository at this point in the history

  2. check ci-rustc options

    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    02bacb2 View commit details
    Browse the repository at this point in the history

  3. fix x test --stage 0 core

    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    93733f9 View commit details
    Browse the repository at this point in the history

  4. don't try to cross-compile sanitizer tests 

    this was a pre-existing latent bug, we just didn't have any CI builders exercising it.

fixes the following errors:
```
 ---- [ui] tests/ui/sanitize/new-llvm-pass-manager-thin-lto.rs#opt1 stdout ----

error in revision `opt1`: test compilation failed although it shouldn't!
status: exit status: 1
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui/sanitize/new-llvm-pass-manager-thin-lto.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "--sysroot" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2" "--target=i686-unknown-linux-gnu" "--cfg" "opt1" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Cstrip=debuginfo" "--remap-path-prefix=/checkout/tests/ui=fake-test-src-base" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/sanitize/new-llvm-pass-manager-thin-lto.opt1/a" "-A" "unused" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/i686-unknown-linux-gnu/native/rust-test-helpers" "-Clinker=x86_64-linux-gnu-gcc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/sanitize/new-llvm-pass-manager-thin-lto.opt1/auxiliary" "-Zsanitizer=address" "-Clto=thin" "-Copt-level=1"
--- stderr -------------------------------
error: linking with `x86_64-linux-gnu-gcc` failed: exit status: 1
   = note: x86_64-linux-gnu-gcc: error: /checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librustc-nightly_rt.asan.a: No such file or directory

failures:
    [ui] tests/ui/sanitize/badfree.rs
    [ui] tests/ui/sanitize/address.rs
    [ui] tests/ui/sanitize/use-after-scope.rs
    [ui] tests/ui/sanitize/new-llvm-pass-manager-thin-lto.rs#opt0
    [ui] tests/ui/sanitize/new-llvm-pass-manager-thin-lto.rs#opt1
```
    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    9cc1765 View commit details
    Browse the repository at this point in the history

  5. allow mixing llvm.assertions and download-rustc 

    by using `rustc-builds-alt` if download-rustc is set

this also changes the download code to use a separate build/cache/ directory and .rustc-stamp stamp file depending on whether assertions are enabled.
    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    7f27750 View commit details
    Browse the repository at this point in the history

  6. Move ci_rustc_dir to Config and use it consistently

    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    96c2e2b View commit details
    Browse the repository at this point in the history

  7. Test x86_64-gnu in PR CI, not x86_64-gnu-llvm-14 

    This is a pre-requisite for rust-lang#112143, which wants to start
using download-rustc in PRs. download-rustc doesn't allow providing an external LLVM.
    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    b80c2f8 View commit details
    Browse the repository at this point in the history

  8. Use download-rustc = "if-unchanged" in PR CI and x86_64-gnu 

    This has two main benefits:

1. It tests that download-rustc doesn't regress. This doesn't reduce our test coverage, since we
   still never use `download-rustc` in a full bors merge, but it should make it a lot less likely that
   this breaks by accident.

2. It greatly speeds up CI when compiler/ and library/ haven't been modified. Once the changes in
   rust-lang/compiler-team#619 land, this will also be faster for changes to
   library/, and only changes to compiler/ will have to rebuild.
    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    a6dcffa View commit details
    Browse the repository at this point in the history

  9. download-rustc: Don't consider the current commit for download if run… 

    …ning in CI

Avoids the following error:
```
curl: (22) The requested URL returned error: 404

error: failed to download pre-built rustc from CI

note: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-rustc` in config.toml:
[rust]
download-rustc = false
```

Note that this strategy is more complicated than the one for download-ci-llvm, which doesn't have to deal with rollup commits.
Eventually we should probably adopt this for LLVM too, to avoid bugs if someone forgets to mark an LLVM bump as rollup=never.

I tested this with `git -c user.email=bors@rust-lang.org -c user.name=bors commit --allow-empty -m 'test bors commit' && GITHUB_ACTIONS=true x check`.
    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    20f9b0c View commit details
    Browse the repository at this point in the history

  10. give a hard error if setting DOWNLOAD_RUSTC in a dist builder 

    these should compile from source; i have reasonable confidence in download-rustc, but not so much that i want to allow shipping it in dist artifacts.
    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    ce127e1 View commit details
    Browse the repository at this point in the history

  11. Don't depend on crate names in tests/ui-fulldeps/missing-rustc-driver… 

    …-error.rs

Not only are they a pain to update, but they differ depending on whether `parallel-rustc` is enabled or not
    @jyn514
    jyn514 committed Jul 9, 2023
    Configuration menu
    Copy the full SHA
    2c3ad2d View commit details
    Browse the repository at this point in the history