pin ABI version of dll dependency at build time

[minrk]

I want this for pyzmq, but I think it’s pretty general (I know @SylvainCorlay has encountered it).

When a package links a DLL, it will only work with the ABI version of that dependency that it loaded at build time. Many packages (e.g. pyzmq) are quite flexible in what versions of those DLLs they can build against, and conda recipes tend to reflect this flexibility. However, that makes it possible to do things like install pyzmq-15.3 and zeromq-4.0, which won’t work because pyzmq-15.3 was built against zeromq-4.1 which has a different ABI than zeromq-4.0, and will fail to load.

The pattern that makes the most sense to me is to leave the build dependency open, but then pin the runtime dependency based on the version loaded at build time. I think conda doesn’t actually record ABI versions, only package versions, but I’m guessing that’s an okay approximation in most cases.

Right now, what I’m trying is pinning the upstream dependency explicitly in meta.yml, which ought to work, but is more restrictive than should really be necessary at the recipe level.

Here’s what I’m imagining in meta.yml:

requirements:
  build:
    - liba
    - libb
    - libc
  runtime:
    - liba build.x*
    - libb build.xy*
    - libc ==build.xyz

Where liba build.x would resolve as the major version of liba at build time, libb build.xy as major.minor of libb, build.xyz as full version of libc, etc.

Perhaps simpler to implement would be a separate dependency category, along the lines of:

dependencies:
  runtime_build_version:
    major:
      - liba
    minor:
      - libb

but that’s not as nice or flexible as allowing substitutions into the existing version resolution spec.

In general, I think it would be appropriate for conda-build to record the exact versions of all packages loaded in the build env (including channels) in the package metadata, if it’s not already doing so.

[pelson]

There is a colourful history to this topic (e.g. #747). I know the capability is high on @msarahan's list, but I'm not certain exactly what the solution will look like, or indeed where he is on this.

In general, I think it is fair to say there is absolute consensus on being able to pin dependencies at build time, rather than hard-coding the versions in the recipe (which is how we currently do it in conda-forge).

[msarahan]

The issue to follow is at #1142, which has links to the current PR, as well as the history. Current status is that I hope to return to it soon, but have been waylaid by more urgent bugfixes. =(

If anyone has time to push that forward with proposals, specs, or code, I'll be very appreciative.

[msarahan]

This will be fixed by #1585 - should be done early in the new year.

[minrk]

Exciting, thanks @msarahan!

[msarahan]

#1585 is merged. Closing.

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically locked because it has not had recent activity after being closed.

Please open a new issue if needed.

Thanks!