Add option to remove unaffected ports from ci action

[dg0yt]

Implementing an idea for how to reduce the set of packages to build during vcpkg PR CI, in particular after significant merges to master. Originating in #201 (comment).

ATM CI rebuilds all packages which don't have a matching cache entry, including unrelated changes from master. This can include a significant number of ports which are not affected by a PR.
In addition, the install list is created at the beginning of the CI command, possibly hours before building leaf ports. Different PR CI runs will build the same ports in parallel.
In case of broken artifact caching, as on 2021-10-03 for x64-osx, all PRs will build all ports (>1500, >20 hrs).
Building unrelated ports also increase the vulnerability for unrelated issues such as download problems.

This PR adds an optional step which removes the packages from the installation plan which have identical ABI hashes in a json file provided by the user and are not required as a dependency.
This json file can be created in a dry run, even with binary caching disabled, for the "known" state before the PR, typically the master parent of the PR merge commit (i.e. HEAD~1).

Simulating the impact for 2a31089 (port gsl-lite change), worst case (nothing cached):

$ git checkout 2a31089~1 -- .
$ ./vcpkg.test ci --dry-run x64-linux --output-hashes=parent_hashes.json --no-binarycaching | grep ' ->' | wc -l
1725
$ git checkout 2a31089 -- .
$ ./vcpkg.test ci --dry-run x64-linux --no-binarycaching | grep ' ->' | wc -l
1725
$ ./vcpkg.test ci --dry-run x64-linux --parent-hashes=parent_hashes.json --no-binarycaching | grep ' ->' | wc -l
52

So instead of building (or considering) 1725 ports, we are down to 52 ports. And it is possible to inspect that these are indeed the ports which are needed to build, or depend on, port gsl-lite.

Disclaimer:
This started as an attempt into the feasibility of an implementation. In particular the code for loading the json data into a list of strings (hashes) doesn't do much error handling.

[strega-nil-ms]

What's the idea here? We still need to do a full test of all of the ports, so where would this make a difference for our CI?

[dg0yt]

What's the idea here? We still need to do a full test of all of the ports, so where would this make a difference for our CI?

Not sure if you really took the time to read. Let's turn the question around:
What is the point of rebuilding qt5-base after changing cpr ?

[strega-nil-ms]

@dg0yt because that's part of the SLA/value-add of vcpkg; we build all of these ports together so you can trust that, if you use a specific baseline, you'll get a universe that works.

[dg0yt]

because that's part of the SLA/value-add of vcpkg; we build all of these ports together so you can trust that, if you use a specific baseline, you'll get a universe that works.

@strega-nil-ms No, this is neither observed nor implemented for pull requests.
(This is implemented in AZP microsoft.vcpkg.ci, with binary caching disabled.)

In my example, on a PR changing cpr,

• AZP microsoft.vcpkg.pr builds qt5-base when it is not in the binary cache,
• AZP microsoft.vcpkg.pr doesn't build qt5-base when it is already in the binary cache.
• building qt5-base, or having a particular build in the cache, doesn't say anything about how the ports interact, given the random build order.

Whether qt5-base is in the binary cache depends on:

• changes in the PR affecting the ABI hash of qt5-base (none in case of cpr)
• state of master when pushing to the PR.

(Note that in most cases, PR CI never tests the state which results from the GH merge action: When the merge is done, the state of master is already different from the state when the PR was tested.)

So this PR is just adds the opportunity to reliably isolate the installation plan from changes which are caused by the state of master (provided as parent hashes). What is actually build still depends on the state of the cache.

[dg0yt]

@strega-nil-ms Note how all PRs are rebuilding all ports for osx now, because the cache was empty when determining the installation plan. A pointless exercise. The PRs scheduled first will run into the timeout again. And https://github.com/microsoft/vcpkg/pull/19034/files doesn't affect a single other port.

[dg0yt]

@ras0219-msft Note that there was an extra condition here: Don't keep specs just for being dependencies of an affected port if the affected port is excluded. I think this can be safely taken to the combined reduce_actions, even if it may sometimes reduce the installation plan when no parent hashes are given.

[BillyONeal]

@dg0yt because that's part of the SLA/value-add of vcpkg; we build all of these ports together so you can trust that, if you use a specific baseline, you'll get a universe that works.

That's not exactly what our existing PR validation does though: it is affected by edits to master. Sometimes the cone of destruction we calculate currently contains unrelated stuff, for example if a "world rebuild" commit has landed before something has actually built those bits.

The approach described here ( exclude ABI hashes already in the master branch for purposes of determining the cone of destruction since they aren't affected by the PR being validated ) seems a reasonable tradeoff to make to me, and might let us add another triplet or similar in exchange without blowing the Azure budget.

[cenit]

@dg0yt you are buying our own release-only triplet with this work, no need to wait for extra budget! or do you have anything better to invest this azure time into? you deserve it man if you need it!

[dg0yt]

What a pity that this is not in today's release. In its original formal, it would have been easy to put the behavioural variation behind an x- parameter. This is blocking other improvements.

[BillyONeal]

What a pity that this is not in today's release. In its original formal, it would have been easy to put the behavioural variation behind an x- parameter. This is blocking other improvements.

We can turn the release crank whenever we think it's important to do so, don't worry :). This is important enough that IMO we would do a new release just for it.

[BillyONeal]

Also pushed a merge with main here since for some reason this PR was stuck in "waiting" status, that's probably why I missed it when I looked for things ready to land yesterday.

[BillyONeal]

Thanks for your contribution!

[dg0yt]

With the the updated tool used in boostrapping, I moved this forward in microsoft/vcpkg#21078.