fix(npm): stale metadata cache issue

[arcanis]

What's the problem this PR addresses?

We now keep the package metadata in cache. To avoid missing new packages being released we have a check so that we only accept the cached metadata if 1/ the request asks for a semver version (not a range), and 2/ the requested version is found inside the cached metadata. In theory this means that whenever a dependency asks for a version we didn't cache, we assume something new got published, and we refetch it.

However, to prevent fetching the package metadata many times for many different versions or ranges, we also have an in-memory metadata cache where we store the cached metadata once we extracted them from either the disk or the network.

This may lead to memory cache corruption issues when two versions from the same package are resolved if one exists in the cached metadata but the other doesn't. In that case, the first package will pass the check for "is this version inside the cached metadata", get stored in the in-memory cache, and be reused for further resolutions (even if those resolutions would have failed this check). This is because the disk cache and the memory cache are the same.

Fixes #5989

How did you fix it?

I separated the in-memory cache into two buckets: the disk cache, and the network cache. This ensures that the disk cache gets properly ignored when retrieving versions we don't know, rather than be mistakenly assumed to be what the network fetched.

Checklist

• I have read the Contributing Guide.

• I have set the packages that need to be released for my changes to be effective.

• I will check that all automated PR checks pass before the PR gets reviewed.

[derHodrig]

This should be included in 4.3.1, correct?
The issue still persists.

1. Publish package to self hosted gitlab
2. try to install this package >> no candidates found
3. delete the metadata folder
4. try to install again > works

so from that perspective, it doesnt seems to be included or does not work.

[arcanis]

Please open an issue with a reproduction. We can't fix what we can't see.

[ds1371dani]

I am using version 4.4.1 and still have this issue

[merceyz]

@ds1371dani See the comment above yours.

[ds1371dani]

#6477 here you go