Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modular overrides #162

Merged
merged 41 commits into from
May 30, 2023
Merged

Modular overrides #162

merged 41 commits into from
May 30, 2023

Conversation

srid
Copy link
Owner

@srid srid commented May 17, 2023

Resolves #101 (and resolves #82 by design)

What this PR does

Retire source-overrides and overrides in favour of a more friendly/ modular design, viz.: packages (configure both source overrides & local packages) and settings (modular replacement for overrides and use pkgs.haskell.lib). The outputs.finalOverlay value is computed from these two options. There are other associated changes; see ChangeLog.

Example: srid/emanote#437

Tasks:

  • Initial implementation
  • Evaluate the design, and finalize API
  • Add the rest of settings.<name>.* options, mirroring pkgs.haskell.lib.
  • Docs
  • ...

@srid srid changed the title WIP: Friendly overrides WIP: Improved overrides May 17, 2023
@srid srid changed the title WIP: Improved overrides WIP: Modular overrides May 17, 2023
Also, add package.<name>.cabal.executables and projectFlakeName options.
@srid srid marked this pull request as ready for review May 22, 2023 20:30
@srid srid requested a review from roberth May 22, 2023 20:32
@srid
Copy link
Owner Author

srid commented May 22, 2023

@roberth What do you think (re: "Evaluate the design, and finalize API" in the PR description)?

For context, here's your previous comment (I might be misunderstanding some of that): srid/emanote#437 (comment)

@srid
Copy link
Owner Author

srid commented May 22, 2023

Not sure why, after ed06b2d, the CI fails around doc tests:

       … while evaluating the option `perSystem.haskellProjects.<name>.projectRoot':

       error: A definition for option `perSystem.haskellProjects.<name>.projectRoot' is not of type `path'. Definition values:
       - In `/nix/store/fl5jf3qnhndz7lb50l92s5rgdyljaj7v-source/nix/modules/project.nix'

@srid

This comment was marked as resolved.

@srid
Copy link
Owner Author

srid commented May 25, 2023

Another example: nammayatri/nammayatri@8c640d6

image

srid added a commit to nammayatri/clickhouse-haskell that referenced this pull request May 25, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for
  dependencies and local packages alike.
srid added a commit to nammayatri/shared-kernel that referenced this pull request May 25, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/beckn-gateway that referenced this pull request May 25, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/nammayatri that referenced this pull request May 25, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- upstream the default overrides to common repo
srid added a commit to nammayatri/euler-hs that referenced this pull request May 25, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.
@srid
Copy link
Owner Author

srid commented May 30, 2023

I think it is time to (squash) merge this now. But I'm still receptive to any or all feedback (regarding implementation or API). We can certainly change things before next release (0.4).

@srid srid merged commit 996f5c2 into master May 30, 2023
srid added a commit to nammayatri/clickhouse-haskell that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for
  dependencies and local packages alike.
srid added a commit to nammayatri/clickhouse-haskell that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for
  dependencies and local packages alike.
srid added a commit to nammayatri/euler-hs that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.
srid added a commit to nammayatri/shared-kernel that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/shared-kernel that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/shared-kernel that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/beckn-gateway that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/shared-kernel that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/beckn-gateway that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/beckn-gateway that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/beckn-gateway that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- add default overrides for all local packages. This should also reduce build times across repos by reusing the same package from dependency builds.
srid added a commit to nammayatri/nammayatri that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- upstream the default overrides to common repo
srid added a commit to nammayatri/nammayatri that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- upstream the default overrides to common repo
srid added a commit to nammayatri/nammayatri that referenced this pull request May 30, 2023
Update haskell-flake to use [srid/haskell-flake#162](srid/haskell-flake#162) which gets us

- a nicer API for overriding Haskell packages.
- reduced duplicate builds (due to use `build-haskell-package.nix` for dependencies and local packages alike.

Also,

- upstream the default overrides to common repo
@roberth
Copy link
Collaborator

roberth commented Mar 12, 2024

The option documentation at perSystem.haskellProjects..packages is incomplete, it doesn't tell me what's valid to put there. An example would be very helpful, but sub-options would be even better, if feasible.

I think part of the problem is deferredModule, which only renders documentation if you use deferredModuleWith { staticModules = ...; }, but that's only usable if you have modules should always be imported, which may not actually be the case here.

@roberth
Copy link
Collaborator

roberth commented Mar 12, 2024

I would like to be able to still specify an overlay of haskell packages, because that helps me when sending changes to Nixpkgs. Judging from the list of options and the changelog, that's not an option anymore?

@srid
Copy link
Owner Author

srid commented Mar 12, 2024

Judging from the list of options and the changelog, that's not an option anymore?

Yes, but you can use basePackages = pkgs.haskellPackages.extend ....

We could add an overlays option (that cannot be merged), but is there a reason for it?

@roberth
Copy link
Collaborator

roberth commented Mar 12, 2024

I just lacked the creativity to do it in basePackages.

is there a reason for it?

It's more efficient to provide all overlays in one go, to avoid computing the fairly expensive fixpoint of the otherwise unnecessary package set with this overlay, but without the built-in overlays. Or to put it simply, we should try to invoke extend only once.

Actually we could work with Nixpkgs to provide a function that behaves like pkgs.haskellPackages.extend without having to evaluate the pkgs.haskellPackages attrset first, but that's an unnecessary scope creep for now.

@srid srid mentioned this pull request Mar 13, 2024
@srid
Copy link
Owner Author

srid commented Mar 13, 2024

Let's do this -- #273. But I trust that you can use extend until that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

A simple interface to overrides Facilitate predictable overlay merging
2 participants