Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3Packages.pybind11: work around clang check #253031

Merged
merged 1 commit into from
Oct 19, 2023

Conversation

reckenrode
Copy link
Contributor

Description of changes

This is required for the upcoming clang 16 bump for Darwin. clang 16 defaults to C++17, which results in aligned allocations in pybind11. These are supported in libc++ with the 10.12 SDK, but clang has a hard-coded check for 10.13.

While the only rebuilds should be x86_64-darwin, targeting staging because there are 2000+.

pybind11’s check phase was run for testing.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Copy link
Contributor

@toonn toonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Builds with current stdenv and the error reproduces when based on the LLVM 16 bump, and this fixes it. LGTM

@reckenrode
Copy link
Contributor Author

According to GitHub, there’s a merge conflict, which I need to fix.

It would also be nice to get feedback from Python maintainers because this is changing mk-python-derivation to allow the stdenv to be overridden.

@toonn
Copy link
Contributor

toonn commented Oct 9, 2023

I rebased on staging to fix the merge conflict and pinged #python:nixos.org for a look.

@toonn
Copy link
Contributor

toonn commented Oct 13, 2023

@FRidh, @Hexa, @jonringer, @tjni, pinging because you're listed as part of the Python team in team-list.nix. I've raised this PR in #python two times but seemingly gotten no response. The changes touch mk-python-derivation.nix so we're looking for acceptance from a Python maintainer.

@toonn
Copy link
Contributor

toonn commented Oct 16, 2023

@dotlambda, I've been told you're the appropriate person to ping for this one.

@FRidh
Copy link
Member

FRidh commented Oct 18, 2023

As mentioned on #173411, I'd like to know whether cross-compilation still functions. I did not test that.

@reckenrode
Copy link
Contributor Author

reckenrode commented Oct 18, 2023

I’m building a cross to Linux now using this PR on my clang 16 branch. I’ll report back when it’s done. I can’t test cross to Darwin (even from Darwin) because it’s not supported by upstream Python (see python/cpython#90905).

@reckenrode
Copy link
Contributor Author

Cross-compiling pybind11 failed because Boost fails to cross-compile from Darwin to Linux. That happened both with this PR and without (on master), so I assume it is not a regression caused by this PR. I started a cross-build from Linux to Linux of this PR to see if it builds there (assuming Boost supports Linux to Linux cross).

@reckenrode
Copy link
Contributor Author

$ nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.57, NixOS, 22.11 (Raccoon), 22.11.20230703.ea4c80b`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.1`
 - nixpkgs: `/etc/nix/inputs/nixpkgs
$ nix build 'github:NixOS/nixpkgs?ref=pull/253031/head#pkgsCross.aarch64-multiplatform.python3Packages.pybind11'
$ nix-store -qR ./result
/nix/store/liw7l94wlss1v9285v2xs26q9ykffssc-libunistring-1.1
/nix/store/i5idc02qlnmfwkkqlafcvcvm72s4rdcp-libidn2-2.3.4
/nix/store/j6gik3dxygd15f2s2ws78rgfr5xpal1r-xgcc-12.3.0-libgcc
/nix/store/10ksw18724qbf44v1v8nz7y4i5m53pm4-glibc-2.38-27
/nix/store/323k9k4fbh490wzrzjf1dfp76pkr7pqh-libgcc-aarch64-unknown-linux-gnu-12.3.0
/nix/store/gcm5cmf8yphxi3xm5qkg364lc6r2fynd-glibc-aarch64-unknown-linux-gnu-2.38-27
/nix/store/1nd1rbhnq5ywk1fklnbnn8219kpw3r9k-bzip2-aarch64-unknown-linux-gnu-1.0.8
/nix/store/1pqv072js8w7092vhs3ysy9i9nsbh6sp-libxcrypt-aarch64-unknown-linux-gnu-4.4.36
/nix/store/2bk3nblajlnnwni404dvjp9wab906v97-xz-aarch64-unknown-linux-gnu-5.4.4
/nix/store/vcs82vr8917g4hpvfkg4z7hcvb8k2kz9-gcc-12.3.0-libgcc
/nix/store/6wlzxf0750hlj933v0z0vxicpsc9hrg8-gcc-12.3.0-lib
/nix/store/9d8z8mx3kn6c757yaafb3ij9y9p58wgv-expat-2.5.0
/nix/store/9n3zjdvv63c3v9sikbx3cq2h8rs39pxk-libffi-3.4.4
/nix/store/9p51mwk8wsb3pqnq3zfhrk7risq4f0a2-openssl-3.0.11
/nix/store/b1c14bgx0h1zipz1f8r5fjs35396vf92-bash-5.2-p15
/nix/store/bkj8c43f9f7a4yid50l5glgz5by0srif-mailcap-2.1.53
/nix/store/cy23qm9hx9qzik4p854gksm687j56snx-libxcrypt-4.4.36
/nix/store/f3sfyxgsh330lp8y5qw7yrkvgnyvqv49-tzdata-2023c
/nix/store/mic15llapbsldm8addghpzfg8srxms9p-gdbm-1.23
/nix/store/v0bxiyawq08rkqjlca3xrqzb7nkz5psg-ncurses-6.4
/nix/store/pasa6jbff4glxpgphxni2gx4v9zz344r-readline-8.2p1
/nix/store/q3sxjj0sr4bvpkvimny4n5bxvrq6rjqh-zlib-1.3
/nix/store/q735vvwqmqfjcccf8q27i08pp3064lbi-sqlite-3.43.1
/nix/store/r567wvncd4pl3sisiy1sh9vbg7hrdpyx-xz-5.4.4
/nix/store/vp124r76hysyxlh0avd4ysx2jrsqkyaf-bzip2-1.0.8
/nix/store/4y9fym6j4065m7bprjl0gfivz8z82h6i-python3-3.11.6
/nix/store/pwsibh0z4a6lzr20r9f1pmvdq9vd84cf-zlib-aarch64-unknown-linux-gnu-1.3
/nix/store/6gflf3pfpy24fk637f6zppq4nk8x9jp5-sqlite-aarch64-unknown-linux-gnu-3.43.1
/nix/store/7rk952bhy8zk0znq9bj2wkpi0k2hq8lp-ncurses-aarch64-unknown-linux-gnu-6.4
/nix/store/82mz6lfi78zwmzrbwbwh3wh56k1mgdhq-gdbm-aarch64-unknown-linux-gnu-1.23
/nix/store/bdb2qqbp0nmj7vrwb3arzklpcyaxk5f0-readline-aarch64-unknown-linux-gnu-8.2p1
/nix/store/frvlmllz9j0if3x5j69qbq5v5vsc9jxs-expat-aarch64-unknown-linux-gnu-2.5.0
/nix/store/j21mj6yh9x8cxgvad490zcjj1h2y7m51-mailcap-aarch64-unknown-linux-gnu-2.1.53
/nix/store/jmwbmvvbrng2kkccphzwl04ij4fixqn2-libffi-aarch64-unknown-linux-gnu-3.4.4
/nix/store/vhl0fvn3f4kbi2kks85bjhs7jhrn6y71-tzdata-aarch64-unknown-linux-gnu-2023c
/nix/store/x7hvd498lb7bmk0jhjdqhk64sc9x4w37-bash-5.2-p15-aarch64-unknown-linux-gnu
/nix/store/xm0y1g1jvxawyg994m4jzzpj3pa2j8fl-openssl-aarch64-unknown-linux-gnu-3.0.11
/nix/store/zwwcsb25mldmmqib7q4kj2zf3xzl3md5-aarch64-unknown-linux-gnu-stage-final-gcc-12.3.0-lib
/nix/store/yv1p4ym5mm1mw5hcam7sk2cpq5f973h6-python3-aarch64-unknown-linux-gnu-3.11.6
/nix/store/97b6h51rdqa4nf8by8mbfkns4wfbq47q-pybind11-setup-hook
/nix/store/i3ax4vsymg0pc3ks5ghiggiajvj0l5zi-python3.11-pybind11-2.11.1-aarch64-unknown-linux-gnu

@reckenrode
Copy link
Contributor Author

It appears that cross-compilation works fine.

This is required for the upcoming clang 16 bump for Darwin. clang 16
defaults to C++17, which results in aligned allocations in pybind11.
These are supported in libc++ with the 10.12 SDK, but clang has a
hard-coded check for 10.13.
@toonn toonn merged commit 026763c into NixOS:staging Oct 19, 2023
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants