-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deprecate builtins.filterSource in favor of something that takes a a few regexes (perhaps different regexes for files/symlinks/directories) #885
Comments
As an alternative to regexes, glob patterns would also be nice. Would be cool if both could be supported. |
Or even |
Is this still a thing? I was going to make an issue that the That said, I would LOVE |
Also, from examining |
I'd also be interested in this - some low level function that's easier to use than Willing to put in the work if there is consensus around the details. I.e. what name to pick, exact API. Longer term I'd like to see standard |
Please… 💔 😭 |
@michalrus can you try |
Oh, that’s cool! I wonder if it works with negative lookahead. OK, so how hard would it be to implement honoring
I remember once trying to run |
Hey, so I finally looked again at this issue and a basic version doesn’t look very hard (a fragment of let
# TODO: How to fully re-use recursive .gitignore’s? https://git.io/vSo80
gitignoreToRegexes = gitignorePath:
builtins.map (line: if lib.hasPrefix "/" line then line else ".*/" + line + "($|/.*)")
(builtins.map (builtins.replaceStrings ["." "**" "*"] ["\\." ".*" "[^/]*"] )
(builtins.map (lib.removeSuffix "/")
(lib.filter (line: line != "" && !(lib.hasPrefix "#" line))
(lib.splitString "\n" (builtins.readFile gitignorePath)))));
sourceByNegativeRegex = regexes: src:
builtins.filterSource (path: type:
let relPath = lib.removePrefix (toString src) (toString path);
in lib.all (re: builtins.match re relPath == null) regexes) src;
build = let
src = sourceByNegativeRegex (gitignoreToRegexes ./.gitignore ++ ["/config" "/default.nix"]) ./.;
in haskell.lib.overrideCabal (haskellPackagesWithOverrides.callCabal2nix "backend" src {}) (drv: {
…
});
in … It seems to be working correctly, at least for this
Now we’d have to have some clever way of running |
@michalrus that is a great step in the right direction but I think that doesn't handle a few |
IMO we should close this, as the core Nix bit of things can be done with |
Well the goal here wasn't so much to enable .gitignore, but to make a serializable version of the builtin 😄 |
@copumpkin Ah. If you do end up doing that, I'd prefer it be on top of |
After struggling with nix-build errors, which turned out to be caused by .ghc.environemnt.* files autogenerated by cabal new-build within nix-shell, and trying to find a solution with builtins.filterSource without success (namely, .ghc.environment.* files still picked up by nix-build even if filtered out by filterSource), I ended up with the following hack to emulate .gitignore let
pkgs = import <nixpkgs> { };
this-src = builtins.fetchGit { url = ../.; }; # url corresponds to git root, './.' in most cases
this-nix = builtins.toPath (this-src + "/project1/default.nix");
in { project1 = pkgs.haskellPackages.callPackage this-nix { }; } which behaved as expected/desired so far (i.e. no nix-build errors following cabal new-build in nix-shell + no rebuilds if any git-ignored or untracked files are created or modified + uncommitted changes in tracked files ARE picked up). An additional benefit is that the local build above is more consistent with the one where the source is fetched from a remote url. |
By the way, here is another implementation for gitignore support https://github.com/numtide/nix-gitignore (haven't tried it myself) |
Here's another: https://github.com/siers/nix-gitignore (for nix 2.0) It has tests against the real git, though it doesn't require it or (By the way, I've just realized this might not've been the best place for such a comment, so sorry.) |
Unless |
Something that I find works very well is simply write a derivation that outputs the source files. This way you can do any simple or complicated logic that you want using the magic of Bash. For example: buildRakuPackage {
name = "infobox";
src = stdenvNoCC.mkDerivation {
name = "infobox-src";
phases = [ "installPhase" ];
installPhase = ''
mkdir "$out"
ln --symbolic ${./bin} "$out/bin"
ln --symbolic ${./lib} "$out/lib"
ln --symbolic ${./META6.json} "$out/META6.json"
'';
};
# ...
} Substitute cp, find, rsync, or anything else for ln as needed. |
@chloekek Filtering sources in a derivation causes unnecessary rebuilds when "ignored" files change, because the hash of your filtered source depends on the hash of the unfiltered input. Content-addressable outputs solve this, but will still require putting a potentially large unfilter file tree in the store for the derivation to work. It also requires that all the unfiltered files are accessible by the user, which is not the case with You can definitely use a derivation to post-process sources, but it's recommended to use one of the non-derivation-based methods first, to avoid unnecessary rebuilds. |
I marked this as stale due to inactivity. → More info |
I closed this issue due to inactivity. → More info |
I developed a safer and easier-to-use abstraction on top of |
It'll likely be faster (no evaluation of nix expressions), easier to use, and we can serialize its invocations for #709.
Perhaps we could keep the current one, but I know that my uses of it would be covered by the regex version, and be easier to use.
The text was updated successfully, but these errors were encountered: