Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdenv: add pure Linux stdenv support for powerpc64le and sparc64 #141448

Draft
wants to merge 27 commits into
base: master
Choose a base branch
from

Conversation

piperswe
Copy link
Contributor

@piperswe piperswe commented Oct 13, 2021

Motivation for this change

I have a sparc64 system I want to try to get NixOS running on, and if I can get NixOS working well in a powerpc64le emulator, I'll consider getting a ppc64le server to play around with (and probably use as a Hydra buildbox so I can add ppc64le to my binary cache).

I've been building things for this branch on my Hydra instance as part of my nixfiles project/jobset: https://hydra.piperswe.me/jobset/nixfiles/main

The results of basically all of my builds go into my binary cache, so if you want to play around with this branch you can use my binary cache to speed things up:

{
  nix = {
    binaryCaches = [
      "https://nix-cache.piperswe.me"
    ];
    binaryCachePublicKeys = [
      "nix-cache.piperswe.me:4r7vyJJ/0riN8ILB+YhSCnYeynvxOeZXNsPNV4Fn8mE="
    ];
  };
}

Since changing stdenv requires rebuilding the world, I might also add a few more interesting architectures to this PR so that we have bootstrap tools for most architectures out there at the cost of one world rebuild.

Fixes #113977. Depends on #141451.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • 21.11 Release Notes (or backporting 21.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: stdenv Standard environment label Oct 13, 2021
@r-burns r-burns added the 6.topic: exotic Exotic hardware or software platform label Oct 13, 2021
@r-burns
Copy link
Contributor

r-burns commented Oct 13, 2021

Cool, I'm glad there are others interested in exotic platforms. Feel free to ping me for review on any related PRs.

W.r.t. bootstrap tools, see also #115406. These always get held up because it's hard to verify that the bootstrap tools are reproducible - how do you do that for native bootstrap tools on a platform we don't support yet? But on the other hand, cross-compiled bootstrap tools won't build stdenv because of #113977, so I'm not really sure how to proceed here.

Your first commit (b8eebed) looks pretty noncontroversial, if you'd like to split that off into a separate PR we could probably get that merged quickly and start playing around with cross-compiled sparc64 packages.

@piperswe
Copy link
Contributor Author

piperswe commented Oct 13, 2021

W.r.t. bootstrap tools, see also #115406. These always get held up because it's hard to verify that the bootstrap tools are reproducible - how do you do that for native bootstrap tools on a platform we don't support yet? But on the other hand, cross-compiled bootstrap tools won't build stdenv because of #113977, so I'm not really sure how to proceed here.

Very true... it seems like there are three stages to building bootstrap tools:

  1. Cross-compile
  2. Native using cross tools
  3. Native using native tools

Maybe there should be more automation around building these - like, on a system with qemu-binfmt, you have a derivation that runs through all three stages, though that requires having a way to inject the newly built bootstrap tools into nixpkgs.

The first approach that comes to mind would be adding an argument to nixpkgsFun for the seeds, which probably wouldn't be a bad idea in general. I'd love to hear other people's thoughts on that though.

stdenv = self.makeStaticLibraries self.stdenv;
}).overrideAttrs (oa: {
# Work around faulty stackprotector support in cross bootstrap tools
hardeningDisable = (oa.hardeningDisable or []) ++ [ "stackprotector" ];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@piperswe piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from 58f75c4 to c1997a4 Compare October 14, 2021 00:00
@piperswe piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from 020ac9e to 9468c68 Compare October 14, 2021 01:02
@piperswe piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from 9468c68 to ad6841c Compare October 14, 2021 01:41
@piperswe piperswe force-pushed the piper/bootstrap-ppc64le-and-sparc64 branch from ad6841c to 1e268db Compare October 14, 2021 04:34
@github-actions github-actions bot added the 6.topic: bsd Running or building packages on BSD label Oct 14, 2021
Comment on lines +151 to +159
platforms = [
"i386-linux" "x86_64-linux"
"armv5tel-linux" "armv6l-linux" "armv7l-linux" "armv7a-linux" "aarch64-linux"
"mipsel-linux"
"powerpc64-linux" "powerpc64le-linux"
"or1k-linux"
"s390x-linux"
"riscv64-linux"
];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is basically platforms.linux? Or which platform is not supported? I can't tell. It would be way cleaner to substract the one or two not working platforms from platforms.linux or just write them into meta.broken.

@@ -1,5 +1,5 @@
{ lib
, localSystem, crossSystem, config, overlays, crossOverlays ? []
, localSystem, crossSystem, config, overlays, crossOverlays ? [], ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we suddenly need to accept any input here and in the other files below? I don't think that is a good idea because it swallows mistakes like typos and makes.

@@ -0,0 +1,18 @@
# Stage 1 is to cross-compile seeds.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and the 3 files below are very similar. Could we combine them?

@@ -0,0 +1,9 @@
# Usage: nix-build ./default.nix --argstr to powerpc64le-linux -A dist
{ from ? builtins.currentSystem, to, nixpkgs ? import ../../../.. }:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could break with flakes because currentSystem is not defined there.

if crossSystem == null
then "-${localSystem.system}"
else "-${localSystem.system}-to-${crossSystem.system}";
stageSuffix = if stage == null then "" else "-${stage}";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
stageSuffix = if stage == null then "" else "-${stage}";
stageSuffix = lib.optionalString (stage != null) "-${stage}";

Comment on lines +10 to +13
systemSuffix =
if crossSystem == null
then "-${localSystem.system}"
else "-${localSystem.system}-to-${crossSystem.system}";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
systemSuffix =
if crossSystem == null
then "-${localSystem.system}"
else "-${localSystem.system}-to-${crossSystem.system}";
systemSuffix = "-${localSystem.system}" + lib.optionalString (crossSystem != null) "-to-${crossSystem.system}";

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 19, 2022
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: merge conflict 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: bsd Running or building packages on BSD 6.topic: exotic Exotic hardware or software platform 6.topic: nixos 6.topic: stdenv Standard environment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cross bootstrap tools have broken stackprotector support
4 participants