Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unpack-bootstrap-tools.sh: use patchelf to undo nuke-refs and nothing else #161925

Closed
wants to merge 3 commits into from
Closed

unpack-bootstrap-tools.sh: use patchelf to undo nuke-refs and nothing else #161925

wants to merge 3 commits into from

Commits on Apr 23, 2022

  1. stdenv/linux: remove perl enableThreading=false override from stage1

    The comment for the line removed by this patch states that "A threaded
    perl build needs glibc/libpthread_nonshared.a, which is not included
    in bootstrapTools".  This is not the reason why it is necessary --
    libpthread_nonshared.a part of the bootstrap-files on many platforms.
    The reason why this workaround was needed is because nuke-refs erases
    the rpath from libpthread.so, and unpack-bootstrap-tools.sh neglects
    to undo this using patchelf.  Let's apply patchelf to libpthread.so,
    so we aren't leaving dangling /nix/store/eeee.... refs in
    libpthread.so, and remove the enableThreading=false override since it
    is no longer necessary.
    
    Additional background:
    
    The enableThreading=false attribute not only removes the
    "-Dusethreads" from the invocation of perl's Configure script, it also
    patches the script like this:
    
      # We need to do this because the bootstrap doesn't have a static libpthread
      sed -i 's,\(libswanted.*\)pthread,\1,g' Configure
    
    Although this prevents the perl interpreter from linking against
    libpthread.so, it does not prevent the libraries bundled with the
    interpreter from doing so.  Time/HiRes/HiRes.so will still try to link
    against libpthread.so, and is only prevented from doing so by the fact
    that unpack-bootstrap-tools.sh neglects to patchelf out all of the
    nuke-refs'd rpaths.  When all of the nuke-refs'd paths are patchelf'ed
    out, HiRes.so links against libpthread.so in the bootstrap tools, and
    the stdenv bootstrap fails:
    
      Can't load '/nix/store/7ny8kmppjzx6kx4xr5kphjz8fqqrlsll-perl-5.34.1/lib/perl5/5.34.1/x86_64-linux/auto/Time/HiRes/HiRes.so' for module Time::HiRes: libpthread.so.0: cannot open shared object file: No such file or directory at /nix/store/7ny8kmppjzx6kx4xr5kphjz8fqqrlsll-perl-5.34.1/lib/perl5/5.34.1/XSLoader.pm line 93.
      at /nix/store/7ny8kmppjzx6kx4xr5kphjz8fqqrlsll-perl-5.34.1/lib/perl5/5.34.1/x86_64-linux/Time/HiRes.pm line 94.
      Compilation failed in require at lib/Autom4te/FileUtils.pm line 42.
      BEGIN failed--compilation aborted at lib/Autom4te/FileUtils.pm line 42.
      Compilation failed in require at bin/autom4te line 46.
      BEGIN failed--compilation aborted at bin/autom4te line 46.
      make[1]: *** [Makefile:2259: tests/wrapper.in] Error 2
      make[1]: *** Waiting for unfinished jobs....
    
    Perl implements its own custom library loading routines for its
    modules, which likely do not fully implement rpath searching.
    
    This problem was revealed during the course of writing PR #161925,
    which depends on this PR in order to pass CI.
    Adam Joseph committed Apr 23, 2022
    Configuration menu
    Copy the full SHA
    df668b7 View commit details
    Browse the repository at this point in the history
  2. unpack-bootstrap-tools.sh: use patchelf to undo nuke-refs and nothing…

    … else
    
    The current unpack-bootstrap-tools.sh wields patchelf selectively,
    guided by unexplained criteria.  Specifically, it applies
    
    patchelf --set-interpreter to:
    
      $out/bin/* $out/libexec/gcc/*/*/* (except */liblto*)
    
    and patchelf --set-interpreter --set-rpath to:
    
      $out/lib/librt-*.so $out/lib/libpcre*
    
    The provenance of these lists is unclear.  Some libraries which never
    had an rpath to begin with (like librt.so) are given an rpath by this
    script, while other libraries (like libbz2.so) are left with dangling
    /nix/store/eeee...'s.
    
    This makes it very difficult to troubleshoot the behavior of the
    bootstrap-tools unpacker when porting to new platforms, where it is
    likely the case that patchelf is being debugged concurrently with
    nixpkgs.
    
    This commit changes unpack-bootstrap-tools.sh to use patchelf *only*
    to replace /nix/store/eeee... references created by nuke-refs, and to
    replace *all* such references except where an exception is clearly
    described and explained.  Currently there are two such exceptions:
    
      1. libgcc_s.so is exempted because its rpath leaks into the final
         stdenv, so we cannot patchelf its rpath without creating a
         fordbidden stdenv-final->bootstrap-tools requisite.
    
      2. libpthread.so and libc.so are exempted from --set-interpreter
         because the ELF interpreter of these files does not matter (they
         are libraries, not executable programs, and unlike ld.so they are
         not meant to play both roles), and because an unresolved bug in
         patchelf causes corruption if it is used to set the interpreter
         here:
    
         NixOS/patchelf#368
    
    Making sure to patchelf away all of the nuke-refs'd paths will make
    the bootstrapping process more robust and less fragile in general.  In
    particular, it will prevent situations like #169746 where the only
    thing preventing a bug from manifesting was the fact that the
    arbitrary choice of which libraries to patchelf had been made in a
    particular way.  Situations like that make things more fragile,
    because now the particular choice which used to be arbitrary has
    become mandatory, and neither that requirement nor the reason for it
    is documented anywhere.
    
    This commit requires #169746 (which is included in this PR) in order
    to pass CI.
    Adam Joseph committed Apr 23, 2022
    Configuration menu
    Copy the full SHA
    8d4ba1e View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2022

  1. remove exception for patchelf#368, as it appears to be fixed (patchel…

    …f#380)
    Adam Joseph committed Jun 19, 2022
    Configuration menu
    Copy the full SHA
    4caa22d View commit details
    Browse the repository at this point in the history