Pre-build wheels not available for FreeBSD and Cygwin

[abravalheri]

It seems that there are no pre-built wheels uploaded to PyPI for platforms other than Windows/Linux/macOS.

People trying to install Pydantic (without a max version cap) on FreeBSD or Cygwin platforms may be surprised by pip failing with an error.

Would it be possible to make pre-built wheels available for these platforms?

Selected Assignee: @davidhewitt

Selected Assignee: @samuelcolvin

[davidhewitt]

@messense do you have any idea if maturin-action can support building for these platforms?

[messense]

It'd be a bit challenging. It was not explored because you can't upload pre-built wheels for FreeBSD and etc to PyPI, which makes the feature less useful.

[samuelcolvin]

@abravalheri are you actually using cygwin and freebsd? Seems like a slightly unusual combination of OSs.

More generally, we have pre build binaries for 99.9895%+ of people, as per pydantic/pydantic#4790 (comment).

I'm really not that convinced that excessive effort to support the small monitory using weird and wonderful setups is a good use of our time.

[abravalheri]

Hi @samuelcolvin, thank you for the response. I understand, projects have decide how to invest their resources.

Just for the sake of replying to your question (not trying to argue in favour of the request): yes I do (not together though 😅). I use BSD on servers (there is some market share for that, I believe for example that Netflix uses FreeBSD on its CDN), and Cygwin on machines that are required to have Windows installed (e.g. at work). One of the reasons why I opened the issue is because recently we started to observe errors in the pypa/setuptools test suite, as one of the test dependencies had an indirect dependency on pydantic which halts pip resolution algorithm (unfortunately it will not backtrack and use an older version of pydantic). The approach I took so far was to disable the specific tests on the platforms that are not supported.

[jaraco]

I think the more important concern here is that if Pydantic doesn't run everywhere Python runs, then it's in a different class of library than others. For example, the inflect library now has to weigh whether it wants to run on Python or only on the subset of Pythons where rust is available, and if it can't accept the latter and impose that on its dependencies, it cannot use Pydantic.

Other projects like sqlalchemy found themselves in a similar situation, but back in a time before wheels where a thing, so basically every system had to have a full build suite in order to install libraries with C extensions. In those times, libraries like sqlalchemy would distribute the non-pure functionality (speedups) as a separate, optional dependency. These days, the preferred alternative is for non-pure libraries to provide wheels so end users don't have to have a build suite.

Pydantic has it particularly hard because it's attempting to provide low-level, foundational behavior that might be found deep in the dependency tree.

I was considering building more general programming abstractions on Pydantic, but now that I realize that employing Pydantic basically limits the distribution and viability of its usage, I'll need to avoid employing it unless I can know in advance that the limited distribution will be acceptable.

I genuinely appreciate the tradeoff here. Pydantic is providing high-speed functionality to the majority of users. I only wish there were a fallback for the minority that are unsupported.

[samuelcolvin]

I get that, I too would love to offer a fallback for those few who can't compile rust.

There's a world where we write a pure python version of pydantic-core (in fact that's one of the reasons I built it as a separate library), but the effort to do so is so huge that I doubt it will happen. (Still, if you wanted only a very small amount of the functionality, it might be feasible)

Apart from that, there's really not that much we can do if pypi doesn't accept the wheels.

[davidhewitt]

Sorry fat finger error, had not meant to reopen.

[davidhewitt]

What I was going to add is that as far as I'm aware FreeBSD and Cygwin can both compile rust, so pydantic is still usable in those platforms for those who are prepared to compile from source. This is the same as cryptography implemented in Rust or even numpy as you'll need to compile the C.

It's possible there is an alternative wheel hosting for these platforms (e.g. piwheels for raspberry pi) and you can always raise a pep to create a wheel tag for these platforms, as was done for musllinux.

So the proportion of users where pydantic is unusable is the tiny fraction who cannot build from source of the tiny fraction who cannot download precompiled wheels.

[jaraco]

That's fair. In the case of Setuptools, probably the request should be to the cygwin environment maintainers to support compiling to rust. Maybe all that's needed is to make sure a rust compiler is installed in that environment. I do agree that it's untenable for all projects to maintain wheels for any possible environment.