bundix doesn't like architecture-specific gems #71
Comments
|
I have the same issue. It's trying to download from a url that doesn't exist (missing This is the correct path: It's trying to fetch this: |
|
I found an open PR for this. |
|
In the meantime, for those who would like to use Sorbet with Nix and Linux, the following works: let
# Import Nixpkgs however you like, be it <nixpkgs> or with Niv
pkgs = ...;
rubyGems = pkgs.bundlerEnv {
name = "gems-for-my-project";
ruby = pkgs.ruby_3_1;
gemdir = ./path/to/gemfile/directory;
gemConfig = pkgs.defaultGemConfig // {
# Poor man's solution: `sorbet-static` has a specific version per system, and Bundix currently does not support that.
# Additionally, the `sorbet` executable needs ELF patching.
# If the Sorbet version ever changes (after re-running `bundle update` then `bundix`) then the SHA256 must be updated accordingly.
# The simplest way to do so is to leave it empty and look at the error message to get the right hash.
# See https://nixos.org/manual/nixpkgs/stable/#developing-with-ruby
# See https://github.com/nix-community/bundix/issues/71
sorbet-static = attrs:
# Bundix picks the Darwin version by default, so we only need to patch on Linux
if pkgs.stdenv.isLinux then {
# Append the OS-specific version suffix
version = attrs.version + "-x86_64-linux";
# Update the SHA256 accordingly
source = attrs.source // { sha256 = "sha256-nuvwK1pQSJ4EsYdkT42q5NsG4QlZZuYT9Zxzz6839rk="; };
# Post-fix the gem by fixing the interpreter with the correct Nix one (default interpreter is `/lib64/ld-linux-x86-64.so.2` which usually does not exist on Nix machines)
nativeBuildInputs = with pkgs; [ autoPatchelfHook ];
} else attrs;
};
};
in
pkgs.mkShell {
buildInputs = [
# Bundler must be before `rubyGems` because the `bundle` executable provided by `rubyGems` does not allow updating the `Gemfile.lock` files
bundler
rubyGems.wrappedRuby
rubyGems
bundix
];
}The SHA256 part is not ideal, but otherwise it seems to work just fine.
Also, I think the code above can be easily adapted to other gems than Sorbet with a similar architecture-specific issue. I hope this is helpful to someone! |
|
This issue has been mentioned on NixOS Discourse. There might be relevant details there: |
jdelStrother
referenced
this issue
in jdelStrother/nixpkgs
Nov 26, 2022
|
Thanks @sir4ur0n this helped me a lot to figure out how to use autoPatchelfHook on the sorbet gem. I vendored a forked version of bundix @jdelStrother's patches to generate separate gemsets as well as support ruby 3, and this is my current thing that is so far working great for linux and arm mac sorbet let
overlays = [
(self: super: {
ruby = pkgs.ruby_3_1;
})
];
pkgs = import nixpkgs { inherit overlays system; };
bundlerPlatform = if pkgs.ruby.system == utils.system.aarch64-darwin then "arm64-darwin" else pkgs.ruby.system;
rubyEnv = pkgs.bundlerEnv {
name = "ruby-env";
inherit (pkgs) ruby;
gemdir = ./.;
gemset = .nix/gemset + ".${bundlerPlatform}.nix";
gemConfig = pkgs.defaultGemConfig // {
sorbet-static = attrs:
if pkgs.stdenv.isLinux then {
nativeBuildInputs = with pkgs; [ autoPatchelfHook ];
} else attrs;
};
}; |
sangster
added a commit
to sangster/bundix
that referenced
this issue
Mar 12, 2023
This commit is an attempt to make Bundix more "platform-aware."
Every rubygem has a "platform" field which names the ruby-platform that
it can run on. For the vast majority of gems, this will be `"ruby"`,
indicating that it's a gem written entirely in ruby and should be able
to run on any ruby interpreter; however, there are some gems that rely
on native extensions that might only work on certain systems.
The previous `gemset.nix` format is generated exactly from the contents
of the inputted `Gemfile.lock`. So if the `Gemfile.lock` included a
reference to `nokogiri (1.14.2)`, Bundix would render a `gemset.nix`
that included the sha256 hash for the ruby-platform version of that gem.
The trouble with this behaviour is that both RubyGems and Bundler, at
install time, will want to install the platform-specific version of the
gem most suitable for the host system. So even though the `Gemfile.lock`
references `nokogiri (1.14.2)`, if your host system's platform were
`x86_64-linux`, they will try to install `nokogiri (1.14.2-x86-linux)`
instead. This leads to issues like [1] and [2].
The new format of `gemfile.nix` is:
```nix
{
dependencies = ["list" "of" "top-level" "dependencies"];
platforms = {
ruby = {
gem-name = {
dependencies = ["list" "of" 'transitive" "dependencies"]
groups = [ ... ];
version = "...";
source = {
# attrs describing the git, path, or RubyGems source
};
};
};
other-platform = {
gem-name = { ... };
};
};
}
```
The gemset's `dependencies` entry copies the `DEPENDENCIES` section of
the `Gemfile.lock` and names all the top-level gem dependencies from the
`Gemfile`.
The gemset's `platforms` entry is an attrset where every key is a
ruby-platform and its value is an attrset of the gems particular to that
platform. This last attrset is essentially the same as the previous
`gemset.nix` format.
This commit also introduces a new nix function: `bundixEnv` (like
`bundlerEnv`, but for Bundix). This function accepts the same arguments
as `bundlerEnv`, with the addition of a `platform`. `bundixEnv` then
converts the given gemset into a format suitable for `bundlerEnv` by
selecting the gems appropriate for the given `platform`. Finally, it
delegates to `bundlerEnv`, with the platform-specific gemset.
`bundix --init` will generate an example `flake.nix` with an example
package that demonstrates how `bundixEnv` works.
See `gem help platforms` for more info about ruby platforms. The Bundler
Guide [3] provides a few examples of rubygems that support multiple
platforms.
[1] nix-community#71
[2] https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019
[3] https://guides.rubygems.org/gems-with-extensions/
sangster
added a commit
to sangster/bundix
that referenced
this issue
Mar 18, 2023
This commit is an attempt to make Bundix more "platform-aware."
Every rubygem has a "platform" field which names the ruby-platform that
it can run on. For the vast majority of gems, this will be `"ruby"`,
indicating that it's a gem written entirely in ruby and should be able
to run on any ruby interpreter; however, there are some gems that rely
on native extensions that might only work on certain systems.
The previous `gemset.nix` format only supported one version of each
dependency. If the `Gemfile.lock` included a reference to
`nokogiri (1.14.2)` and also `nokogiri (1.14.2-x86-linux)`, Bundix would
render a `gemset.nix` that included only one of them. The trouble with
this behaviour is that, at runtime, Bundler, will see has multiple
nokogiri gems to choose from and may choose that one that was never
built. This leads to issues like [1] and [2].
The new format of `gemfile.nix` is:
```nix
{
dependencies = ["list" "of" "top-level" "dependencies"];
platforms = {
ruby = {
gem-name = {
dependencies = ["list" "of" 'transitive" "dependencies"]
groups = [ ... ];
version = "...";
source = {
# attrs describing the git, path, or RubyGems source
};
};
};
other-platform = {
gem-name = { ... };
};
};
}
```
The gemset's `dependencies` entry copies the `DEPENDENCIES` section of
the `Gemfile.lock` and names all the top-level gem dependencies from the
`Gemfile`.
The gemset's `platforms` entry is an attrset where every key is a
ruby-platform and its value is an attrset of the gems particular to that
platform. This last attrset is essentially the same as the previous
`gemset.nix` format.
This commit also introduces a new nix function: `bundixEnv` (like
`bundlerEnv`, but for Bundix). This function accepts the same arguments
as `bundlerEnv`, with the addition of a `platform`. `bundixEnv` then
converts the given gemset into a format suitable for `bundlerEnv` by
selecting the gems appropriate for the given `platform`. Finally, it
delegates to `bundlerEnv`, with the platform-specific gemset.
`bundix --init` will generate an example `flake.nix` with an example
package that demonstrates how `bundixEnv` works.
See `gem help platforms` for more info about ruby platforms. The Bundler
Guide [3] provides a few examples of rubygems that support multiple
platforms.
This commit also adds two guides, `guides/getting-started.md` and
`guides/motivations.md` that describe this format in more detail.
[1] nix-community#71
[2] https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019
[3] https://guides.rubygems.org/gems-with-extensions/
sangster
added a commit
to sangster/bundix
that referenced
this issue
Mar 18, 2023
This commit is an attempt to make Bundix more "platform-aware."
Every rubygem has a "platform" field which names the ruby-platform that
it can run on. For the vast majority of gems, this will be `"ruby"`,
indicating that it's a gem written entirely in ruby and should be able
to run on any ruby interpreter; however, there are some gems that rely
on native extensions that might only work on certain systems.
The previous `gemset.nix` format only supported one version of each
dependency. If the `Gemfile.lock` included a reference to
`nokogiri (1.14.2)` and also `nokogiri (1.14.2-x86-linux)`, Bundix would
render a `gemset.nix` that included only one of them. The trouble with
this behaviour is that, at runtime, Bundler, will see has multiple
nokogiri gems to choose from and may choose that one that was never
built. This leads to issues like [1] and [2].
The new format of `gemfile.nix` is:
```nix
{
dependencies = ["list" "of" "top-level" "dependencies"];
platforms = {
ruby = {
gem-name = {
dependencies = ["list" "of" 'transitive" "dependencies"]
groups = [ ... ];
version = "...";
source = {
# attrs describing the git, path, or RubyGems source
};
};
};
other-platform = {
gem-name = { ... };
};
};
}
```
The gemset's `dependencies` entry copies the `DEPENDENCIES` section of
the `Gemfile.lock` and names all the top-level gem dependencies from the
`Gemfile`.
The gemset's `platforms` entry is an attrset where every key is a
ruby-platform and its value is an attrset of the gems particular to that
platform. This last attrset is essentially the same as the previous
`gemset.nix` format.
This commit also introduces a new nix function: `bundixEnv` (like
`bundlerEnv`, but for Bundix). This function accepts the same arguments
as `bundlerEnv`, with the addition of a `platform`. `bundixEnv` then
converts the given gemset into a format suitable for `bundlerEnv` by
selecting the gems appropriate for the given `platform`. Finally, it
delegates to `bundlerEnv`, with the platform-specific gemset.
`bundix --init` will generate an example `flake.nix` with an example
package that demonstrates how `bundixEnv` works.
See `gem help platforms` for more info about ruby platforms. The Bundler
Guide [3] provides a few examples of rubygems that support multiple
platforms.
This commit also adds two guides, `guides/getting-started.md` and
`guides/motivations.md` that describe this format in more detail.
[1] nix-community#71
[2] https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019
[3] https://guides.rubygems.org/gems-with-extensions/
sangster
added a commit
to sangster/bundix
that referenced
this issue
Mar 18, 2023
This commit is an attempt to make Bundix more "platform-aware."
Every rubygem has a "platform" field which names the ruby-platform that
it can run on. For the vast majority of gems, this will be `"ruby"`,
indicating that it's a gem written entirely in ruby and should be able
to run on any ruby interpreter; however, there are some gems that rely
on native extensions that might only work on certain systems.
The previous `gemset.nix` format only supported one version of each
dependency. If the `Gemfile.lock` included a reference to
`nokogiri (1.14.2)` and also `nokogiri (1.14.2-x86-linux)`, Bundix would
render a `gemset.nix` that included only one of them. The trouble with
this behaviour is that, at runtime, Bundler, will see has multiple
nokogiri gems to choose from and may choose that one that was never
built. This leads to issues like [1] and [2].
The new format of `gemfile.nix` is:
```nix
{
dependencies = ["list" "of" "top-level" "dependencies"];
platforms = {
ruby = {
gem-name = {
dependencies = ["list" "of" 'transitive" "dependencies"]
groups = [ ... ];
version = "...";
source = {
# attrs describing the git, path, or RubyGems source
};
};
};
other-platform = {
gem-name = { ... };
};
};
}
```
The gemset's `dependencies` entry copies the `DEPENDENCIES` section of
the `Gemfile.lock` and names all the top-level gem dependencies from the
`Gemfile`.
The gemset's `platforms` entry is an attrset where every key is a
ruby-platform and its value is an attrset of the gems particular to that
platform. This last attrset is essentially the same as the previous
`gemset.nix` format.
This commit also introduces a new nix function: `bundixEnv` (like
`bundlerEnv`, but for Bundix). This function accepts the same arguments
as `bundlerEnv`, with the addition of a `platform`. `bundixEnv` then
converts the given gemset into a format suitable for `bundlerEnv` by
selecting the gems appropriate for the given `platform`. Finally, it
delegates to `bundlerEnv`, with the platform-specific gemset.
`bundix --init` will generate an example `flake.nix` with an example
package that demonstrates how `bundixEnv` works.
See `gem help platforms` for more info about ruby platforms. The Bundler
Guide [3] provides a few examples of rubygems that support multiple
platforms.
This commit also adds two guides, `guides/getting-started.md` and
`guides/motivations.md` that describe this format in more detail.
[1] nix-community#71
[2] https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019
[3] https://guides.rubygems.org/gems-with-extensions/
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I am trying to bundix the sorbet typechecker.
Gemfile:default.nix(I usecallPackageon this):Because of #70 , I am trying to generate
gemset.nixusingbundle update && bundix. Here is what happens:First, bundler correctly downloads the gem that matches my arch (
x86_64-linux):Then bundix downloads a Darwin gem when constructing
gemset.nix?Attempting to build this fails:
Note that it's trying to download a gem file with no architecture. If you visit the gem page for
sorbet-static, you'll see a download link that includes the arch. This seems to be getting lost somewhere. Not sure if that is abundlerAppbug in nix code, or a problem with the generatedgemset.nix. But I'm pretty sure that bundix looking at the Darwin gem when constructinggemset.nixis wrong.The text was updated successfully, but these errors were encountered: