rustup breaks with cargo-make due to environment variables

[sffc]

Problem

It appears that a recent rustup update today caused our GitHub Actions CI to start failing to install a pinned nightly version.

Passing (7hrs ago): https://github.com/unicode-org/icu4x/actions/runs/2650696918

Failing (1hr ago): https://github.com/unicode-org/icu4x/actions/runs/2652484278

GitHub Actions config file: https://github.com/unicode-org/icu4x/blob/main/.github/workflows/build-test.yml

CC @Manishearth

Steps

View the above CI reports. See how the jobs start failing with messages such as:

[cargo-make] INFO - Execute Command: "rustup" "run" "nightly-2022-04-05" "cargo" "wasm-build-release" "--package" "icu_capi_cdylib" "--features" "provider_test,smaller_test"

and

[memory] > error: no such subcommand: `+nightly-2022-04-05`

Possible Solution(s)

No response

Notes

No response

Rustup version

latest

Installed toolchains

nightly-2022-04-05

[Manishearth]

Another error that we're seeing is:

error: "/home/runner/.rustup/toolchains/1.61-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/Cargo.lock" does not exist, unable to build with the standard library, try:
        rustup component add rust-src --toolchain nightly-2022-04-05-x86_64-unknown-linux-gnu

Which I think is our fault, and I have a fix in #3029 . However it's unclear how the rustup update broke that (maybe the culprit isn't the rustup update)

---

Apologies for the low-context issue, I figured given that rustup releases are rare it was more valuable for us to report this bug ASAP and work on figuring out more details afterwards. The +nightly thing being broken still seems sus.

[sffc]

Note that the following error only appears on Windows:

[memory] > error: no such subcommand: `+nightly-2022-04-05`

[Manishearth]

error: "/home/runner/.rustup/toolchains/1.61-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/Cargo.lock" does not exist, unable to build with the standard library, try:
        rustup component add rust-src --toolchain nightly-2022-04-05-x86_64-unknown-linux-gnu

Okay, looking at this further this is really weird. In our "Load cortex target for no_std build" CI task section in this failing job we call:

        rustup component add --toolchain nightly-2022-04-05 rust-src
        rustup target add thumbv7m-none-eabi --toolchain nightly-2022-04-05
        rustup target add thumbv8m.main-none-eabihf --toolchain nightly-2022-04-05
        rustup target add x86_64-unknown-linux-gnu --toolchain nightly-2022-04-05

Looking closely at the error, I see a couple things wrong:

• We're installing rust-src for the toolchain version, which seems sensible and correct. rustup is asking us to install it for the toolchain and target, which seems excessive (and a breaking change?)
• It seems like rustup was looking for the file for 1.61-x86_64-unknown-linux-gnu even though that task is running nightly-2022-04-05, and from looking at the error message it clearly wants nightly-2022-04-05, not 1.61. I don't know what's going on here

Both of these things seem to be Rustup bugs. The +nightly-foo failing on Windows seems like another rustup issue but unrelated.

[kinnison]

If you are doing cross-builds from within invocations of cargo are you perhaps not being sure to clean your environment of things like RUSTC RUSTDOC RUSTFLAGS or other environment variables which might affect builds? This might be the equivalent of #3031 if so.

If I'm barking up the wrong tree, please tell me and we can investigate further.

[Kryptos-FR]

Maybe a different issue but since the release of the new rustup version, it fails on Windows CI jobs on GitHub.

Minimal repro

name: Build (Windows)

on:
  workflow_dispatch:
jobs:
  Windows:
    runs-on: windows-2019
    steps:
      - name: Set up rust
        env:
          RUSTUP_USE_CURL: 1
        run: |
          rustup toolchain install nightly
          rustup +nightly target add i686-pc-windows-gnu
          rustup default nightly-i686-pc-windows-gnu

Errors in log

info: syncing channel updates for 'nightly-x86_64-pc-windows-msvc'
info: latest update on 2022-07-12, rust version 1.64.0-nightly (38b72154d 2022-07-11)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: installing component 'cargo'
info: installing component 'rust-std'
info: installing component 'rustc'
  nightly-x86_64-pc-windows-msvc installed - rustc 1.64.0-nightly (38b72154d 2022-07-11)
info: checking for self-updates
info: downloading self-update
error: could not copy file from 'C:\Users\runneradmin\.cargo\bin\rustup-init.exe' to 'C:\Users\runneradmin\.cargo\bin\rustup.exe': Access is denied. (os error 5)
ResourceUnavailable: D:\a\_temp\aaf6a12a-e61a-436f-8bde-96f840d0d276.ps1:3
Line |
   3 | rustup +nightly target add i686-pc-windows-gnu
     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Program 'rustup.exe' failed to run: An error occurred trying to start process
     | 'C:\Users\runneradmin\.cargo\bin\rustup.exe' with working directory 'D:\a\shards\shards'. Access is
     | denied.At D:\a\_temp\aaf6a12a-e61a-436f-8bde-96f840d0d276.ps1:3 char:1 + rustup +nightly target add
     | i686-pc-windows-gnu + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
Error: Process completed with exit code 1.

Workaround

rustup toolchain install --no-self-update nightly

Looks like it is the self update causing havoc.

Note that it only fails on Windows workers. Linux and macOS didn't have that failure.

[kinnison]

That is a problem with Windows locking the file harder. You can add rustup set auto-self-update disable in your CI if you want to prevent this kind of thing.

[Emilgardis]

the windows error is very sporadic, see https://github.com/Emilgardis/rustup-windows-issue/runs/7298752016?check_suite_focus=true

to circumvent, do what @kinnison suggest, example with actions-rs/toolchain (and only doing it on windows since it works well on the unix runners):

- run: rustup set auto-self-update disable
  if: contains(runner.os, 'windows')
  shell: bash
- name: Install Rust toolchain
  uses: actions-rs/toolchain@v1
  with:
    toolchain: stable
    default: true
    profile: minimal