Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

universal2 build improvements #495

Merged
merged 1 commit into from
Feb 2, 2023
Merged

Conversation

radarhere
Copy link
Collaborator

Two suggested changes when building universal2 wheels.

  1. Instead of presuming the length of the string to replace in the following code, use sed. This will allow $py_osx_ver to vary in length.

    multibuild/osx_utils.sh

    Lines 513 to 514 in 2e10577

    if [[ "$whl" == *macosx_${py_osx_ver}_x86_64.whl ]]; then
    whl_base=$(echo $whl | rev | cut -c 23- | rev)

  2. When building for universal2 on x86-64, run the cross build first and then the native build. This will mean that the more natural native variables are in place when the code is finished, for anything that might run afterwards.

@isuruf
Copy link
Collaborator

isuruf commented Jan 6, 2023

Some shared libraries require a native version of the library/tools for cross compiling. Changing the order would prevent that.

@radarhere
Copy link
Collaborator Author

But the native version is already built last on arm64?

multibuild/osx_utils.sh

Lines 527 to 532 in 2e10577

function wrap_wheel_builder {
if [[ "${PLAT:-}" == "universal2" ]]; then
if [[ "$(uname -m)" == "arm64" ]]; then
(macos_intel_cross_build_setup && $@)
rm -rf *-stamp
(macos_arm64_native_build_setup && $@)

@isuruf
Copy link
Collaborator

isuruf commented Jan 6, 2023

Ah yes. That should be fixed. Arm64 native CI is untested AFAIK

@radarhere
Copy link
Collaborator Author

I've removed the second change, leaving just the sed change.

@radarhere
Copy link
Collaborator Author

Some shared libraries require a native version of the library/tools for cross compiling. Changing the order would prevent that.

Could you give an example of one such library?

@isuruf
Copy link
Collaborator

isuruf commented Jan 6, 2023

ncurses

@radarhere
Copy link
Collaborator Author

I attempted to test this with a custom

function build_ncurses {
    fetch_unpack https://invisible-island.net/datafiles/release/ncurses.tar.gz
    (cd ncurses-6.3 \
        && ./configure --prefix=$BUILD_PREFIX \
        && make -j4 \
        && make install)
    touch "ncurses-stamp"
}

but it wouldn't even build on arm64. If you're able to demonstrate, that would be helpful.

@radarhere
Copy link
Collaborator Author

radarhere commented Jan 20, 2023

If you aren't able to demonstrate, it would be good if you were able to Approve this PR for my sed change - just to clarify that our discussion is not a blocker.

Copy link
Collaborator

@isuruf isuruf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll have a look at the ordering later this week. Sed change is fine.

@radarhere
Copy link
Collaborator Author

To give more information about my situation, build_libjpeg_turbo is installing cjpeg and djpeg binaries. Because the cross build is run last, the binaries built raise "Bad CPU type in executable" during subsequent testing.

Sure, a workaround for this could be for me to just run build_libjpeg_turbo again after both builds - but running the native build last seemed like the more general solution to this type of problem.

@isuruf
Copy link
Collaborator

isuruf commented Jan 26, 2023

I think the issue with both ncurses and build_libjpeg_turbo is that the PATH is set incorrectly for the cross compilation. The native prefix should be first in the PATH for cross compilation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants