Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix build failure when pyproject.toml contains editable path dependencies #689

Merged
merged 3 commits into from
Aug 23, 2022

Conversation

crossing
Copy link
Contributor

mkPoetryApplication fails when a editable path dependency exists in pyproject.toml, similar to #284

This PR fix the problem by removing develop attributes in path / git dependencies in hooks.

@crossing
Copy link
Contributor Author

I don't have a mac to check the failed job. And the message doesn't seem to be related. Any hints what I should do?

@sigprof
Copy link
Contributor

sigprof commented Aug 10, 2022

The issues with the pillow package build on darwin probably appeared a long time before, but remain hidden in most cases, because the package does not get rebuilt. Probably your change touched something that caused a rebuild of that package and uncovered the breakage.

The underlying reason is that the build scripts of the pillow package try to detect the presence of various optional libraries, but do that in a way that is not really compatible with Nix; apparently at some point the GitHub macos-latest environment changed somehow, and those scripts started to detect the presence of the jpeg2000 library in there (installed through homebrew, probably as a dependency of something else), even though that library is not in Nix store, therefore the build process fails later when it attempts to use the library outside of the Nix store.

I'm currently using this override for pillow, which allows it to build on GitHub CI for macos-latest (but I tested only the latest version, which happens to be 9.2.0 at the moment, while the CI job that failed here apparently tried to build 6.2.2 — no idea whether this override would actually work for that version):

      pillow = super.pillow.overridePythonAttrs(old: {
        # Use preConfigure from nixpkgs to fix library detection issues and
        # impurities which can break the build process; this also requires
        # adding propagatedBuildInputs and buildInputs from the same source.
        propagatedBuildInputs = (old.buildInputs or []) ++ pkgs.python3.pkgs.pillow.propagatedBuildInputs;
        buildInputs = (old.buildInputs or []) ++ pkgs.python3.pkgs.pillow.buildInputs;
        preConfigure = (old.preConfigure or "") + pkgs.python3.pkgs.pillow.preConfigure;
      });

But I'm not sure whether this is the right solution (maybe the correct thing would be to copy the required parts from nixpkgs instead of extracting them from the package like that, but they are rather extensive). I also don't have a Mac and can use only the GitHub CI for testing the Mac-specific changes.

@crossing
Copy link
Contributor Author

Thanks @sigprof

Made a change to use preConfigure from nixpkg's pillow and now test passed.

Pushed an empty commit to trigger action re-runs.

Let me know if I should stash the commits.

@adisbladis adisbladis merged commit e1e95bb into nix-community:master Aug 23, 2022
sigprof added a commit to sigprof/nix-devenv-qmk that referenced this pull request May 5, 2023
There is a bug in the `pillow` 9.5.0 release that breaks the build:

  nix-community/poetry2nix#1139
  python-pillow/Pillow#7069

Apply the patch with the upstream fix when building version 9.5.0.

The old override is still kept, even though the main part of that
override was included in nix-community/poetry2nix#689 (the problem is
that `buildInputs` in the poetry2nix override does not include all
packages that are in the nixpkgs package for `pillow`, and therefore
`preConfigure` might add references to nonexistent packages).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants