[perf] Add flag to fix missing store paths

[mikeland73]

Summary

Adds new --tidy-lockfile flag that can be used with install. It fills in any missing store paths in lockfile, improving future performance.

I'm not 100% convinced this works in all cases. My two biggest concerns would be:

• I'm setting all outputs to default.
• re derivation name parsing: Not 100% sure we can rely on versions never having dashes.

How was it tested?

• unit test
• Manually removed store paths form lockfile and tried to install. Got warning, ran with flag.

[mikeland73]

@Lagoja @gcurtis made updates we talked about:

• Better copy when missing store paths.
• Only show message for devbox packages (i.e. pkg@version)
• Fix store paths on update

[Lagoja]

Running this, I get:

Warning: Outputs for libiconv@latest are not in lockfile. Fetching store paths from nix
To fix this issue and improve performance, please run `devbox install --fix-missing-store-paths

Are we fetching the store paths at the time that I get this message? Or do I need to run the command with the flag

[Lagoja]

Suggested change

	"Outputs for %s are not in lockfile. Fetching store paths from nix\n"+
	"Outputs for %s are not in lockfile.+

[mikeland73]

@Lagoja I need to hide the message when using the flag. Will do.

[gcurtis]

Should we name the flag --tidy-lockfile? Are missing store paths in the lockfile a problem? I thought it just made things slower.

[gcurtis]

What's the downside if this ends up being wrong?

[mikeland73]

not much. We may install a bit more than we're supposed to (on the other hand, if we forget to add this value, that's bad).

[mikeland73]

Should we name the flag --tidy-lockfile? Are missing store paths in the lockfile a problem? I thought it just made things slower.

I like that name. Will change. It gives us flexibility to do more stuff.

missing store paths from lockfile can be a big perf drag, especially if all packages are in cache. If the store paths are missing we end up having to download nixpkgs every time (which adds 40s for each nixpkgs). Particularly painful on CICD because env is fresh every time.

[sentry-io]

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

• ‼️ Generic Error: GET https://search.devbox.sh/v2/resolve?name=go&version=latest: unexpected status code 403 Forbid... go.jetpack.io/devbox/internal/searcher in execG... View Issue
• ‼️ **Generic Error: <redacted errors.withStack>: <redacted errors.fundamental> go.jetpack.io/devbox/internal/devpkg in (*Packa... View Issue
• ‼️ *net.DNSError: <redacted errors.withStack>: GET https://search.devbox.sh/v2/resolve?name=jdk17&version=latest: ... go.jetpack.io/devbox/internal/searcher in execG... View Issue
• ‼️ syscall.Errno: GET https://search.devbox.sh/v2/resolve?name=k9s&version=latest: Get "https://search.devbox.sh/v2... go.jetpack.io/devbox/internal/searcher in execG... View Issue

_{Did you find this useful? React with a 👍 or 👎}