Reland optimized-compiler-builtins config

[jyn514]

this is possible now that compiler-builtins contains the fix in rust-lang/compiler-builtins#532.

in particular, this makes the c feature for compiler-builtins an explicit opt-in, rather than silently detected by whether llvm-project is checked out on disk.
exposing this is necessary because the cc crate doesn't support cross-compiling to MSVC, and we want people to be able to run x check --target foo regardless of whether they have a c toolchain available.

this also uses the new option in CI, where we do want to optimize compiler_builtins.

the new option is off by default for the dev channel and on otherwise.

Fixes #102560. Fixes #101172. Helps with #105065 (although there's some weirdness there - it's still broken when optimized-compiler-builtins is set to true).

r? @tmandry cc @Mark-Simulacrum

[tmandry]

We may not be using a pre-built version of LLVM. Specifically the use case I want to support is cd into src/llvm-project, check out a vanilla commit, and set submodules = false, llvm-has-rust-patches = false in config.

That seems totally bizarre, though. Why would you want that?

To build against a more recent version of LLVM.

If you separately build and supply your own llvm-config, then you hit the other assumptions in here about not needing to distribute the LLVM libraries because it's probably system installed. So basically this was unsupported before.

Plus it's desirable to have bootstrap manage the configuration of LLVM if all you want to vary is the version.

You can use compiler-rt without Rust patches. I think the important thing is that compiler-rt is built from the same sources as LLVM. So.. either llvm-config is unset, or llvm_from_ci is active and we're managing submodules?

Yeah, as I wrote in #101833 (comment) enabling optimized_compiler_builtins is kinda required on some targets, and from my understanding the only requirement is that the sources of compiler-rt has to be the same as the custom LLVM's. We probably need a way to supply the custom sources in those cases.

@rustbot author

☔ The latest upstream changes (presumably #103797) made this pull request unmergeable. Please resolve the merge conflicts.

@tmandry I am still very confused how llvm_has_rust_patches is meant to be used. The only place I see it used is in Build::is_rust_llvm, which is used in three places:

1. rust/src/bootstrap/test.rs

   Lines 1553 to 1554 in c5f92ce

   	if !builder.is_rust_llvm(target) {
   	cmd.arg("--system-llvm");

2. The new assert I just added. Let's ignore that for now.

3. rust/src/bootstrap/compile.rs

   Lines 805 to 806 in b3bc6bf

   	if builder.is_rust_llvm(target) {
   	cargo.env("LLVM_RUSTLLVM", "1");

3 defines LLVM_RUSTLLVM, which is eventually used in

rust/compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp

Lines 308 to 311 in 693c631

	#ifdef LLVM_RUSTLLVM
	/// getLongestEntryLength - Return the length of the longest entry in the table.
	template<typename KV>
	static size_t getLongestEntryLength(ArrayRef<KV> Table) {

; that bit looks correct. But 1. looks absolutely wrong even before this PR: it will pass --system-llvm for an in-tree commit with submodules = false and patches = false. How is llvm_has_rust_patches tested? how is it supposed to work? I ran llvm-config --ldflags --system-llvm locally and it didn't even exit successfully ...

ugh I misread, it's a compiletest flag, not an llvm-config flag. so I think what was happening before is that some tests were silently being ignored, which we don't have any sort of checks for:

rust/src/tools/compiletest/src/header.rs

Lines 1059 to 1061 in d5fd1af

	fn ignore_llvm(config: &Config, line: &str) -> bool {
	if config.system_llvm && line.starts_with("no-system-llvm") {
	return true;

I'll fix 1 at the same time as I address your and pietro's comments.

Ok, pushed a few changes. @pietroalbini @tmandry let me know if you think this will work better.

@rustbot author (I think there are some outstanding concerns from Pietro to be addressed)

Looks good, thanks for checking! I left some nits, feel free to fix or ignore and r=me

☔ The latest upstream changes (presumably #114148) made this pull request unmergeable. Please resolve the merge conflicts.

rust-lang/team#1051

This PR modifies config.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

i am going to remove the check that the in-tree llvm matches the llvm we're linking to, actually. i am not sure the difference matters, and it wasn't enforced before.

r? bootstrap

@rustbot ready

☔ The latest upstream changes (presumably #119373) made this pull request unmergeable. Please resolve the merge conflicts.

No concerns on my side. Currently, Jyn isn't actively working on the project. I will close this PR; open another one to cherry-pick the commits, resolve conflicts, and then r+ it.