Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mkYarnPackage: fix uncopied resolutions field #214952

Conversation

GuillaumeDesforges
Copy link
Contributor

@GuillaumeDesforges GuillaumeDesforges commented Feb 6, 2023

Description of changes

Yarn only uses the "resolutions" field in the top-level package.json file of a workspace. While it has been taken into account for mkYarnWorkspace (9801e6e), it has not been fixed for mkYarnPackage yet, which also uses the yarn workspace mechanism under the hood.

This change should fix some issues such as this.

Things done
  • Tested compilation of all packages that depend on this change using nix shell github:nixos/nixpkgs#nixpkgs-review --command nixpkgs-review rev HEAD
  • Tested evaluation against some previously broken builds
  • Fits CONTRIBUTING.md.

@GuillaumeDesforges
Copy link
Contributor Author

This breaks Sharedown

@olynch
Copy link
Contributor

olynch commented Feb 6, 2023

I ran into this problem independently literally 10 minutes ago, and this fixes it! Thanks @GuillaumeDesforges.

@GuillaumeDesforges GuillaumeDesforges force-pushed the GuillaumeDesforges@fix-mkyarnpackage-resolutions branch from c914aa7 to 65a492f Compare February 7, 2023 08:33
@GuillaumeDesforges
Copy link
Contributor Author

Rebased onto master (7479275)

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/mkyarnpackage-lockfile-has-incorrect-entry/21586/6

@SuperSandro2000
Copy link
Member

This uses IFD which is not allowed in nixpkgs.

@GuillaumeDesforges
Copy link
Contributor Author

@SuperSandro2000 thanks for alerting me. This package was not added by me, nor mkYarnPackage, I'm merely fixing a bug there. Or am I missing something and my changes add IFD?

What's the impact for my PR?
I'm all for fixing the IFD issue, but I'd be sad if it blocked this PR, which fixes a bug that is hard to fix on your own.

@Radvendii
Copy link
Contributor

Radvendii commented Feb 21, 2023

This doesn't use IFD on its own, but if something calls these functions with a packageJSON that's from a derivation (for instance, if it's from a fetchurl-like source), it will be IFD. Presumably SuperSandro2000 is saying this is something that happens in NIxpkgs.

The way I see around this is to add in the resolutions field at build time, probably using jq.

@SuperSandro2000
Copy link
Member

What's the impact for my PR?

As long as it triggers IFD in any form while ofborg evals it cannot be merged.

You could give Radvendii idea a try, I didn't try to come up with one.

@aidalgol
Copy link
Contributor

aidalgol commented Apr 6, 2023

A package I maintain (heroic) added a resolutions field to its package.json in its last release, and it will not build on master, but it does with the changes from this PR.

@GuillaumeDesforges
Copy link
Contributor Author

I used @lilyinstarlight 's snippet and it works. I can build packages with a resolutions field in the yarn workspace (e.g. marp-cli).

Copy link
Member

@lilyinstarlight lilyinstarlight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think you could commit this without all of the reformatting changes?

It makes it a bit difficult to review and we still don't yet have a blessed formatter for nixpkgs

@GuillaumeDesforges
Copy link
Contributor Author

My bad 🤦‍♂️ I still forget that nixpkgs is not always formatted with nixpkgs-fmt... will do

@GuillaumeDesforges GuillaumeDesforges force-pushed the GuillaumeDesforges@fix-mkyarnpackage-resolutions branch 2 times, most recently from 5e4283a to 3ca8b5c Compare April 7, 2023 15:26
@GuillaumeDesforges
Copy link
Contributor Author

GuillaumeDesforges commented Apr 7, 2023

@lilyinstarlight how about now? I tested and it still seems to work.

$ nix repl
Welcome to Nix 2.14.1. Type :? for help.

nix-repl> nixpkgs = builtins.getFlake (builtins.toString ./.)

nix-repl> pkgs = nixpkgs.legacyPackages.x86_64-linux

nix-repl> marp-cli = pkgs.mkYarnPackage { pname = "marp-cli"; version = "2.4.0"; src = pkgs.fetchFromGitHub { owner = "marp-team"; repo = "marp-cli"; rev = "v2.4.0"; sha256 = "vZ1EKjuicFxkzcT/akV3U7+1adhMkJUgC3mALsAw/WM="; }; }

nix-repl> :b marp-cli

This derivation produced the following outputs:
  out -> /nix/store/acszdx7jxpvq693w1p5lklbazni7d0bs-marp-team-marp-cli-2.4.0

Copy link
Member

@lilyinstarlight lilyinstarlight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! No regressions against master either


Result of nixpkgs-review pr 214952 run on x86_64-linux 1

1 package marked as broken and skipped:
  • micropad
6 packages failed to build: (all broken in master)
  • apache-airflow (python310Packages.apache-airflow) (dependency marked broken on master)
  • apache-airflow.dist (python310Packages.apache-airflow.dist) (dependency marked broken on master)
  • plausible (broken on master - Hydra failure)
  • python311Packages.apache-airflow (dependency marked broken on master)
  • python311Packages.apache-airflow.dist (dependency marked broken on master)
  • zammad (broken on master - Hydra failure)
31 packages built:
  • gotify-server
  • grafana-image-renderer
  • hedgedoc
  • heroic
  • heroic-unwrapped
  • jellyseerr
  • lemmy-ui
  • listmonk
  • matrix-alertmanager
  • matrix-appservice-discord
  • matrix-appservice-slack
  • matrix-hookshot
  • meshcentral
  • mirakurun
  • pgadmin4
  • pgadmin4-desktopmode
  • pgadmin4-desktopmode.dist
  • pgadmin4.dist
  • pomerium
  • powerdns-admin
  • puppeteer-cli
  • rstudio
  • rstudio-server
  • rstudioServerWrapper
  • rstudioWrapper
  • sharedown
  • synapse-admin
  • uivonim
  • v2raya
  • woodpecker-server
  • yarn2nix

Yarn only uses the "resolutions" field in the top-level package.json
file of a workspace. While it has been taken into account for
mkYarnWorkspace (9801e6e), it has not
been fixed for mkYarnPackage yet, which also uses the yarn workspace
mechanism under the hood.

A bit of care was needed because we don't want to introduce any IFD.
@GuillaumeDesforges GuillaumeDesforges force-pushed the GuillaumeDesforges@fix-mkyarnpackage-resolutions branch from 3ca8b5c to 6bf78d8 Compare April 11, 2023 10:58
@lilyinstarlight
Copy link
Member

@SuperSandro2000, any objection to me going ahead and merging this? It looks good to me now and nixpkgs-review still looks good

@lilyinstarlight
Copy link
Member

I'm going to go ahead and merge so this doesn't just bitrot. If there are any more review comments, they can be fixed in a follow-up

Thanks @GuillaumeDesforges!

@lilyinstarlight lilyinstarlight merged commit bc9a078 into NixOS:master Apr 30, 2023
@GuillaumeDesforges GuillaumeDesforges mentioned this pull request Jul 21, 2023
12 tasks
@GuillaumeDesforges GuillaumeDesforges deleted the GuillaumeDesforges@fix-mkyarnpackage-resolutions branch July 21, 2023 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants