PlatformEnv solving fails when setup.py relies on binaries present in host environment

[croth1]

To my understanding, at the time of poetry solving, the conda environment is not physically built, just exists as a list of dependencies. The poetry solver works on an imaginary python environment with the conda deps fixed.

This strategy does not seem to work with setup.pys that rely on pre-installed non-pythonic dependencies. In my case, the poetry solver tries to execute the fiona's setup.py, which errors unless it can find gdal-info (which can be satisfied with the gdal package). Even if I have gdal as a conda dependency, the solve wouldn't succeed, because the environment does not really exist. I can solve successfully if I install gdal into the environment that hosts conda-lock, but that doesn't seem to be a proper solution.

In order for the poetry solve to do the right thing, it seems one would have to temporarily build the environment solved with conda, and use it with the poetry solver.

Any thoughts about that? Would probably not be platform independent anymore?

[maresb]

If a package relies on non-python dependencies, that's a good argument for installing it with conda instead of pip. Can you explain why you're not using the Fiona CF package?

[croth1]

I agree, using the conda package would be the better option - however in this case it's a bit tricky: fiona comes as a dependency of a library, I try to install from a private pypi index. Using a private conda-index would be awesome, but is probably not an option at this point. While I could install most of the dependencies with conda and pin the package version, this seems like a lot of manual work - I think I would have to sync the conda dependencies manually every time a constrain on the dependency changes- that sounds a bit painful.

[mariusvniekerk]

Generally you should be able to just add Fiona as a conda dep and since that is around things should solve.

That being said ymmv with using pip. I'd recommend very strongly to move away from setup.py wherever possible in favor of newer fully declarative metadata.

If you want to make a pr to conda lock that performs non-dry-run installations (presumably bound to a given platform) that would be okay