compiler-rt: build builtins on darwin #186575

stephank · 2022-08-14T06:47:42Z

Description of changes

On Darwin, compiler-rt is missing builtins. There is supposed to be a libclang_rt.osx.a, but it is missing completely. I'm a little surprised this wasn't an issue before, but I'm trying to build some code that uses if (@available(...)) which is breaking, because it uses __isPlatformVersionAtLeast from compiler-rt.

I believe this is down to the CMake configuration using xcrun to determine what to build for the target SDK, and we simply didn't provide it and ignored the errors. In #185969, I added the necessary functionality to our shell-based xcrun, and more importantly made it available separately from xcbuild as xcbuild.xcrun. This hopefully means it's not an objection to make it part of Darwin stdenv, as it's just a bunch of Nix-generated scripts and files.

I'm also patching out a check that uses PlistBuddy. We could add the field to our SDKSettings.plist, but the output of querying an array with PlistBuddy from facebook/xcbuild is different from Apple's PlistBuddy. I felt it was easier to patch the CMake check.

Finally, I removed the subtitute rule that disabled ARM64, because it is obviously essential. I believe this was just a workaround for the other issues.

I tested buils of compiler-rt itself on x86_64-darwin, versions 12, 13 & 14. On aarch64-darwin, I did a full rebuild of stdenv including version 11, and also tested version 13 (part of my intended build).

Things done

matthewbauer

should be good to go to staging ; make sure to follow status of staging since sometimes these kind of stdenv changes can cause unforeseen consequences

stephank · 2022-08-16T07:37:41Z

should be good to go to staging ; make sure to follow status of staging since sometimes these kind of stdenv changes can cause unforeseen consequences

Thanks for bringing this up, I was wondering where I can follow this? It doesn't look like Hydra builds staging anymore, so is the idea to keep an eye out for the next round of stabilization on staging-next?

SuperSandro2000 · 2022-08-16T13:31:17Z

pkgs/development/compilers/llvm/git/compiler-rt/default.nix

+  ]
+    ++ lib.optionals stdenv.hostPlatform.isDarwin [
+      ../../common/compiler-rt/darwin-plistbuddy-workaround.patch
+      # Prevent a compilation error on darwin
+      ./darwin-targetconditionals.patch
+    ]


Can we make this unconditional?

The armv7l patch below this was specifically made conditional in d616ae8

But I think the idea is to avoid Darwin stdenv rebuilds? And I'm not sure how much we care about other rebuilds. So the Darwin patches could be made non-optional.

The armv7l patch below this was specifically made conditional in d616ae8

That would no longer apply when targeting staging because we need to rebuild it anyway already.

Applying the patches always makes it easier to notice when they no longer apply when updating on other platforms.

I rebased and made the patches non-optional. Verified v11, v12, v13, v14 on aarch64-darwin (with stdenv rebuild), and verified git at least evaluates. Also did a build of v11 on x86_64-linux.

The missing xcrun meant builtins were missing from darwin. This apparently wasn't an issue until now, but is in projects using `@available` checks. (The ARM64 hack was apparently the previous solution to fixing broken SDK detection.)

Atemu · 2022-09-06T12:39:04Z

pkgs/development/compilers/llvm/11/compiler-rt/default.nix

@@ -15,7 +15,8 @@ stdenv.mkDerivation {
  inherit version;
  src = fetch "compiler-rt" "0x1j8ngf1zj63wlnns9vlibafq48qcm72p4jpaxkmkb4qw0grwfy";

-  nativeBuildInputs = [ cmake python3 libllvm.dev ];
+  nativeBuildInputs = [ cmake python3 libllvm.dev ]
+     ++ lib.optional stdenv.isDarwin xcbuild.xcrun;


This won't fly. Hydra doesn't do xcbuild and since this is in the stdenv, we wouldn't get any darwin builds whatsoever.

Oh, what prevents hydra from building xcbuild? I tried to make it stdenv-friendly in #185969 by exposing xcrun separately, which is just some text files generated by Nix. If that doesn't work, then I guess we'll have to patch the CMake files.

The fact that builders don't have no Xcode installed (to my knowledge) and that it's highly impure to run external build tools.

Ah, but we don't really use Xcode or the SDKs, just emulate them. For example, our xcrun is this shell script:

nixpkgs/pkgs/development/tools/xcbuild/wrapper.nix

Lines 54 to 97 in ab26522

xcrun = writeShellScriptBin "xcrun" ''

args=( "$@" )

# If an SDK was requested, check that it matches.

for ((i = 0; i < ''${#args[@]}; i++)); do

case "''${args[i]}" in

--sdk | -sdk)

i=$((i + 1))

if [[ "''${args[i]}" != '${xcrunSdkName}' ]]; then

echo >&2 "xcodebuild: error: SDK \"''${args[i]}\" cannot be located."

exit 1

fi

;;

esac

done

while [ $# -gt 0 ]; do

case "$1" in

--sdk | -sdk) shift ;;

--toolchain | -toolchain) shift ;;

--find | -find | -f)

shift

command -v $1 ;;

--log | -log) ;; # noop

--verbose | -verbose) ;; # noop

--no-cache | -no-cache) ;; # noop

--kill-cache | -kill-cache) ;; # noop

--show-sdk-path | -show-sdk-path)

echo ${sdks}/${sdkName}.sdk ;;

--show-sdk-platform-path | -show-sdk-platform-path)

echo ${platforms}/${xcodePlatform}.platform ;;

--show-sdk-version | -show-sdk-version)

echo ${sdkVer} ;;

--show-sdk-build-version | -show-sdk-build-version)

echo ${sdkBuildVersion} ;;

*) break ;;

esac

shift

done

if ! [[ -z "$@" ]]; then

exec "$@"

fi

'';

And it references fake macOS SDK and platform directories that are generated plist files. Its entire closure is:

$ nix-store -qR /nix/store/7142q0hx4a6pxhy01y3iwhdwngv26a36-xcrun /nix/store/i7ql4rg0667cc3ppp1s0v133cr2414ww-SDKs /nix/store/kvji81lb8cgbzcj59qv8v5lqwmpz1kss-Platforms /nix/store/rdsvxa6vz0hj00jl7f7dn2ik14yfl6p9-bash-5.1-p16 /nix/store/7142q0hx4a6pxhy01y3iwhdwngv26a36-xcrun

I believe it should be fine, but will try to verify. I thought I was already on a clean macOS, working on a machine provisioned by Scaleway, but it seems Xcode is already on it. I'll delete and try a rebuild.

I removed Xcode and Command-Line Tools, cleaned up my Nix store, then did a rebuild, and stdenv builds correctly. (I did find an impurity in my Swift PR, so good thing we tried!)

Okay then, I was concerned because it's located under the xcbuild umbrella. Since it isn't really related to xcode, wouldn't it be best to give it a more fitting name and put it somewhere else?

I guess it's related in the sense that xcbuild, xcrun, xcode-select, etc all emulate Xcode tooling. It's probably called xcbuild because the core tool its packaging is facebook/xcbuild? And the rest of it is small Nix shims.

But facebook/xcbuild is also unmaintained. It looks like we have several dozen packages using it, though not sure how many use the xcbuild parts, and how many just need the Nix shims. Also not sure how well it is capable of building newer Xcode projects (including Swift stuff).

It might make sense to move the non-xcbuild stuff to a separate top-level attribute, like xcode-shims? Maybe @matthewbauer has a stronger opinion. 🙂

stephank · 2022-10-02T10:58:12Z

Ping! Is this okay to merge?

vcunat · 2022-10-20T08:48:30Z

This merge introduced an evaluation issue:

$ nix eval -f pkgs/top-level/release.nix --arg supportedSystems '[ "aarch64-darwin" ]' stdenvBootstrapTools.aarch64-darwin.dist.outPath
error: infinite recursion encountered

       at pkgs/stdenv/generic/make-derivation.nix:295:14:

          294|       args = attrs.args or ["-e" (attrs.builder or ./default-builder.sh)];
          295|       inherit stdenv;
             |              ^
          296|
(use '--show-trace' to show detailed location information)

There's a longer sad story for that job. By an accident it wasn't evaluated and built by Hydra for a few months – and in the meantime it stopped working. Fixing it completely would be nice, as normally the nixpkgs-22.11-darwin channel would block on it (soon to be created).

vcunat · 2022-10-20T08:51:59Z

I guess /cc PR #195862 as this is the reason why an "important job" is missing on Hydra.

(cherry picked from commit 09b8886)

Darwin-specific builtins were present on x86-64-darwin, but not on aarch64-darwin. This is the same issue as #186575, but while the fixes were correctly applied to LLVM 15, we were still disabling the build of builtins on aarch64-darwin. Likely just a confusion.

stephank requested a review from matthewbauer as a code owner August 14, 2022 06:47

ofborg bot added 6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin-stdenv labels Aug 14, 2022

ofborg bot requested review from 7c6f434c, dtzWill, lovek323 and primeos August 14, 2022 08:17

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 0 labels Aug 14, 2022

matthewbauer approved these changes Aug 15, 2022

View reviewed changes

SuperSandro2000 reviewed Aug 16, 2022

View reviewed changes

bobby285271 added the 12.approvals: 1 label Aug 16, 2022

compiler-rt: build builtins on darwin

fb91bfc

The missing xcrun meant builtins were missing from darwin. This apparently wasn't an issue until now, but is in projects using `@available` checks. (The ARM64 hack was apparently the previous solution to fixing broken SDK detection.)

stephank force-pushed the fix/compiler-rt-builtins branch from ec646f4 to fb91bfc Compare August 24, 2022 10:52

ofborg bot added 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ and removed 10.rebuild-linux: 0 labels Aug 24, 2022

bobby285271 removed the 12.approvals: 1 label Aug 24, 2022

stephank mentioned this pull request Sep 6, 2022

swift: split build & Darwin support. #189977

Merged

13 tasks

Atemu reviewed Sep 6, 2022

View reviewed changes

7c6f434c merged commit 711191c into NixOS:staging Oct 5, 2022

stephank deleted the fix/compiler-rt-builtins branch October 5, 2022 18:49

rrbutani mentioned this pull request Oct 10, 2022

llvmPackages_15: init at 15.0.7 #194634

Merged

92 tasks

stephank mentioned this pull request Oct 20, 2022

[staging-next] stdenvBootstrapTools: fix cycle on aarch64-darwin #196909

Merged

13 tasks

rrbutani added a commit to rrbutani/nixpkgs that referenced this pull request Jan 27, 2023

llvmPackages_15.compiler-rt: apply NixOS#186575 to llvmPackages_15

09b8886

github-actions bot pushed a commit that referenced this pull request Jan 28, 2023

llvmPackages_15.compiler-rt: apply #186575 to llvmPackages_15

068139d

(cherry picked from commit 09b8886)

K900 pushed a commit that referenced this pull request Jan 28, 2023

llvmPackages_15.compiler-rt: apply #186575 to llvmPackages_15

694c7b2

xanderio pushed a commit to xanderio/nixpkgs that referenced this pull request Feb 13, 2023

llvmPackages_15.compiler-rt: apply NixOS#186575 to llvmPackages_15

536a3d3

stephank mentioned this pull request Apr 23, 2023

swift: 5.7.3 -> 5.8 #225197

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

compiler-rt: build builtins on darwin #186575

compiler-rt: build builtins on darwin #186575

stephank commented Aug 14, 2022 •

edited

Loading

matthewbauer left a comment

stephank commented Aug 16, 2022

SuperSandro2000 Aug 16, 2022

stephank Aug 16, 2022

SuperSandro2000 Aug 17, 2022

stephank Aug 24, 2022

Atemu Sep 6, 2022

stephank Sep 6, 2022

Atemu Sep 6, 2022

stephank Sep 6, 2022

stephank Sep 7, 2022

Atemu Sep 7, 2022

stephank Sep 7, 2022

stephank commented Oct 2, 2022

vcunat commented Oct 20, 2022

vcunat commented Oct 20, 2022

	xcrun = writeShellScriptBin "xcrun" ''
	args=( "$@" )

	# If an SDK was requested, check that it matches.
	for ((i = 0; i < ''${#args[@]}; i++)); do
	case "''${args[i]}" in
	--sdk \| -sdk)
	i=$((i + 1))
	if [[ "''${args[i]}" != '${xcrunSdkName}' ]]; then
	echo >&2 "xcodebuild: error: SDK \"''${args[i]}\" cannot be located."
	exit 1
	fi
	;;
	esac
	done

	while [ $# -gt 0 ]; do
	case "$1" in
	--sdk \| -sdk) shift ;;
	--toolchain \| -toolchain) shift ;;
	--find \| -find \| -f)
	shift
	command -v $1 ;;
	--log \| -log) ;; # noop
	--verbose \| -verbose) ;; # noop
	--no-cache \| -no-cache) ;; # noop
	--kill-cache \| -kill-cache) ;; # noop
	--show-sdk-path \| -show-sdk-path)
	echo ${sdks}/${sdkName}.sdk ;;
	--show-sdk-platform-path \| -show-sdk-platform-path)
	echo ${platforms}/${xcodePlatform}.platform ;;
	--show-sdk-version \| -show-sdk-version)
	echo ${sdkVer} ;;
	--show-sdk-build-version \| -show-sdk-build-version)
	echo ${sdkBuildVersion} ;;
	*) break ;;
	esac
	shift
	done

	if ! [[ -z "$@" ]]; then
	exec "$@"
	fi
	'';

compiler-rt: build builtins on darwin #186575

compiler-rt: build builtins on darwin #186575

Conversation

stephank commented Aug 14, 2022 • edited Loading

Description of changes

Things done

matthewbauer left a comment

Choose a reason for hiding this comment

stephank commented Aug 16, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephank commented Oct 2, 2022

vcunat commented Oct 20, 2022

vcunat commented Oct 20, 2022

stephank commented Aug 14, 2022 •

edited

Loading