Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve buildPython discovery on Mac #821

Open
mhsmith opened this issue Mar 6, 2023 · 5 comments
Open

Improve buildPython discovery on Mac #821

mhsmith opened this issue Mar 6, 2023 · 5 comments
Milestone

Comments

@mhsmith
Copy link
Member

mhsmith commented Mar 6, 2023

The default PATH on Mac is /usr/bin:/bin:/usr/sbin:/sbin. However, apps can't install anything into these locations, so the python.org installers use /usr/local/bin instead. This directory may also appear to be on the default PATH, but this is because it's listed in /etc/paths, which only affects shells, not other apps like Android Studio, or its Gradle subprocesses.

This effectively means that we'll never auto-discover the minor-versioned Python executable on Mac, only the major-versioned one in /usr/bin, which may be provided by Xcode or the command line developer tools. A similar problem affects the runtime build process, which needs minor-versioned executables to pyc-compile the bootstrap code.

Both of these could be fixed by either adding /etc/paths to the PATH if it isn't already present, or searching in those directories manually.

Related: #991.

@mhsmith mhsmith added this to the 14.1 milestone Mar 6, 2023
mhsmith added a commit to mhsmith/chaquopy that referenced this issue Aug 1, 2023
@mhsmith
Copy link
Member Author

mhsmith commented Aug 7, 2023

A similar problem affects the runtime build process

This has now been fixed, and a function added to Common.java which could also be used by the Gradle plugin.

@mhsmith mhsmith changed the title Improve Python discovery on Mac Improve buildPython discovery on Mac Sep 18, 2023
@mhsmith
Copy link
Member Author

mhsmith commented Nov 28, 2023

I discovered this issue when working on the product project, which uses Gradle 6.7.1 so the Gradle plugin it builds can still be loaded by AndroidPlugin.test_old (see product/gradle/wrapper/gradle-wrapper.properties). But in all the Gradle versions we do support , the problem doesn't exist anymore.

I've narrowed down the change to Gradle 6.9. In that version, and all higher versions I've tried as far as 8.2, ps -AE (which "does not reflect changes in the environment after process launch") still shows the daemon's PATH as /usr/bin:/bin:/usr/sbin:/sbin when launched from Android Studio Giraffe. And yet, printing System.getenv("PATH") in a build.gradle file shows the full set of directories, including those which are in .zprofile but not in /etc/paths, and it has no problem finding Python.

I have no idea how it's doing this, and I don't see anything relevant in the release notes except maybe Apple Silicon support. But it looks like we don't need to take any more action here.

@mhsmith
Copy link
Member Author

mhsmith commented Mar 9, 2024

I encountered this problem again while testing Android Studio Iguana, and I can reproduce it as follows:

  • Sync the the product project in Iguana, which starts a 6.7.1 daemon with a minimal PATH.
  • Run testIntegration-8.3 from the command line. This reuses the 6.7.1 daemon, and the 8.4 daemon it launches to build the test apps can't find python3.8 on the PATH, causing pyc compilation to fail.

An 8.4 daemon launched directly from Iguana is still fine, as is a 6.7.1 daemon launched from the command line, so there's no need to look into this any further. It'll probably go away when we update the product project to Gradle 6.9 or newer.

@mhsmith
Copy link
Member Author

mhsmith commented Jun 4, 2024

I had a similar problem when testing Android Studio Koala, where even killing all daemons and starting the integration tests from the command line was losing the PATH somehow. The only explanation I can think of is that daemons were being started by a copy of Android Studio in the background, because once I closed all open projects, then killed all daemons again, it stopped happening.

@mhsmith
Copy link
Member Author

mhsmith commented Jul 7, 2024

Reopening this for visibility, because it hits me regularly when running Gradle tests on my Mac.

@mhsmith mhsmith reopened this Jul 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant