Support cross compilation on macOS

[autoantwort]

BPO	46749
Nosy	@ronaldoussoren, @ned-deily, @moreati, @autoantwort

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-02-14.14:23:11.267>
labels = ['OS-mac', 'type-feature', '3.11']
title = 'Support cross compilation on macOS'
updated_at = <Date 2022-02-14.21:25:37.996>
user = 'https://github.com/autoantwort'

bugs.python.org fields:

activity = <Date 2022-02-14.21:25:37.996>
actor = 'ned.deily'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['macOS']
creation = <Date 2022-02-14.14:23:11.267>
creator = 'autoantwort'
dependencies = []
files = []
hgrepos = []
issue_num = 46749
keywords = []
message_count = 2.0
messages = ['413224', '413265']
nosy_count = 4.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'Alex.Willmer', 'autoantwort']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46749'
versions = ['Python 3.11']

[autoantwort]

Currently you get the following output:

➜  debug git:(main) ✗ ../configure --host=x86_64-apple-darwin --build=arm64-apple-darwin --with-build-python=./python3.11
checking for git... found
checking build system type... aarch64-apple-darwin
checking host system type... x86_64-apple-darwin
checking for --with-build-python... ./python3.11
checking for Python interpreter freezing... ./python3.11
checking for python3.11... (cached) ./python3.11
checking Python for regen version... Python 3.11.0a5+
checking for x86_64-apple-darwin-pkg-config... no
checking for pkg-config... /opt/homebrew/bin/pkg-config
configure: WARNING: using cross tools not prefixed with host triplet
checking pkg-config is at least version 0.9.0... yes
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... configure: error: cross build not supported for x86_64-apple-darwin

Is "needed" for microsoft/vcpkg#22603

[ned-deily]

We don't currently support cross-building a single macOS architecture on another, i.e. building an x86_64 only build on an Apple Silicon Mac. However, we do support building a multi-architecture build on either on current releases of Python 3.9 and 3.10, with what is known as a universal or fat build. In this case, you'd want to add these two options:

./configure --enable-universalsdk --with-universal-archs=universal2 ...

For this to succeed, you will also need to provide universal builds of any of the third-party libraries used by standard modules you need and that are not otherwise provided by macOS, the most common being OpenSSL and XZ (and perhaps newer versions of SQLite and Tcl/Tk). For the most part, those are straightforward to build yourself but also may be available through some third-party distributors like MacPorts and possibly HomeBrew. The Mac/README.rst file in repo has some more details about universal builds and there may be some useful hints at building the third-party libraries in Mac/BuildScript/build-installer.py although it is not intended to be used for this purpose. The resulting universal2 build can be used as is on either Intel-64 or Apple Silicon Macs but, if you truly need to have only Intel binaries you should be able to use the macOS lipo command to separate out the desired architecture from the built executables and shared libraries into other files. If you want to run these binaries on a range of macOS versions, you can also set the MACOSX_DEPLOYMENT_TARGET environment variable when building the libs and Python; we currently test and support back to 10.9.

[directhex]

However, we do support building a multi-architecture build on either on current releases of Python 3.9 and 3.10, with what is known as a universal or fat build.

Strictly speaking, the build system does expose the ability to do a single architecture multi-architecture build - --enable-universalsdk --with-universal-archs=intel-64. Whilst it doesn't technically use the --host and --build flags, the end result is the same: you can build an x64-only Python for Mac on arm64.

The reverse, however, is not true - there is no value for --with-universal-archs in the build system which targets arm64 exclusively.

That said, the --with-universal-archs values in configure are ancient - untouched for a decade. There are 8 values:

• 32-bit: 32-bit POWER, x86
• 64-bit: 64-bit POWER, AMD64
• all: 32-bit POWER, x86, 64-bit POWER, AMD64
• universal2: AARCH64, AMD64
• intel: x86, AMD64
• intel-32: x86
• intel-64: x64
• 3-way: 32-bit POWER, x86, x64

I would question the utility of most of this. Does Python really CI on Mac OS X 10.5 (the last version to run on POWER)? That also means Xcode 3.x - with GCC 4.2.

At the absolute bare minimum, there should be a value for arm64, which provides single-arch universal builds and the equivalent to cross-compilation. And AC_CHECK_SIZEOF does assume the target architecture runs on the build system (which is true for M1 Macs which can execute AMD64 Mac binaries, but the reverse isn't true) - some mild massaging of configure.ac is needed to avoid that requirement.