Feat: Ensure that we have Windows images for both nanoserver and servercore, on Adoptium's JDK8,JDK11 and JDK17 [INFRA-3047]

[dduportal]

This PR introduces the following changes:

• Ensure that all the existing Windows images are using Adoptium (Eclipse Temurin)

• Add the new JDK17 Windows images

• Ensure that each Windows images exists for all JDKs (8, 11 and 17) with a nanoserver and a windows server core each

• Improve the Windows Build step's performance by using docker-compose (and a build-windows.yaml docker-compose file) for parallel building the Windows images

• Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!

• Ensure that the pull request title represents the desired changelog entry

• Please describe what you did
  - [ ] Link to relevant issues in GitHub or Jira
  - [ ] Link to relevant pull requests, esp. upstream and downstream changes
  - [ ] Ensure you have provided tests - that demonstrates feature works or fixes the issue

[timja]

if they work and don't add too long to the build time.

[dduportal]

One of the modified images is failing to build. I'll check later.

[dduportal]

A few discoveries while working on this:

• All the nanoserver images can be derived from the windowsservercore easily (that would avoid the slow download part where we re-download the same git, git-lfs, remoting.jar...). That could be a nice improvement to ensure the windows images does not diverge over time
• I'm trying to use docker-compose to have a centralized manifest for the windows images (wip). It would also greatly accelerate the build time since docker-compose build accepts a --parallel flag :)
• I need to work on tests

[dduportal]

@timja @MarkEWaite So this PR is ready to be merged.
It should improve the build time, even by adding images:

= Last Master Branch Build (no Docker caching, only focusing on the Windows Parallel Stage)

Ref. https://ci.jenkins.io/blue/organizations/jenkins/Packaging%2Fdocker-agent/detail/master/140/pipeline/9

- Total Windows time is 31min 07s
  - The build step (sequential docker build): 20:15:43 -> 20:34:41 => 18 min 58
  - The test step (sequential test harnesses): 20:34:41 -> 20:46:32 => 11 min 51

== This PR

https://ci.jenkins.io/blue/organizations/jenkins/Packaging%2Fdocker-agent/detail/PR-202/37/pipeline/9 (first success and complete build)

- Total Windows build time is 24min 49s
  - New initial Build Step (Docker Compose build, in parallel, which populates the Docker Engine cache): 13:26:36 -> 13:34:46 => 8 min 10s
  - The (legacy) build step (sequential docker builds, kept to be sure that nothing is broken): 13:34:46 -> 13:36:16 => 1 min 30
  - The test step (sequential test harnesses):13:36:16 -> 13:51:19 => 15min 03

[dduportal]

Last build was 26 min (~ 14 % )

[timja]

Do we need that legacy build step?

[dduportal]

Do we need that legacy build step?

We could remove it, but it would side-track the work here: we would have to update the build.ps1 script to ensure that the docker image name from docker-compose's build are passed to tests.
Since it also manages the tags, I would prefer keeping the existing for now (in favor for a separated change of the script later) to not break builds on trusted.ci

[timja]

Tests are failing on windows on trusted CI: https://trusted.ci.jenkins.io:1443/blue/organizations/jenkins/Containers%2Fdocker%2Fdocker-agent/detail/4.10-7/2/pipeline

[dduportal]

@timja Interesting, the error is not related to the change here, but to the "Pester module" that could not be loaded in Windows. I'm digging to check why.

[dduportal]

@timja I've relaunched a build and it worked (and deployed the tag): https://trusted.ci.jenkins.io:1443/blue/organizations/jenkins/Containers%2Fdocker%2Fdocker-agent/detail/4.10-7/3/pipeline/9/

[timja]

cool thanks

[timja]

I tried twice before pinging here :p

[dduportal]

I tried twice before pinging here :p

Yeah, that is what I saw, don't worry for that. Network issues ;)
I'm thinking about adding Pester and bats in the packer-images template for the future.

[timja]

Weird errors occurring when publishing the next version:
https://trusted.ci.jenkins.io:1443/blue/organizations/jenkins/Containers%2Fdocker%2Fdocker-agent/detail/4.11-1/2/pipeline/9

[2021-10-16T16:56:07.279Z] docker-compose version 1.29.2, build 5becea4c

[2021-10-16T16:56:07.279Z] BUILD: Starting Docker Compose General Build

[2021-10-16T16:56:08.668Z] powershell.exe : Flag '--parallel' is ignored when building with COMPOSE_DOCKER_CLI_BUILD=1

[2021-10-16T16:56:08.668Z] At C:\Jenkins\workspace\iners_docker_docker-agent_4.11-1@tmp\durable-9f6b78b3\powershellWrapper.ps1:3 char:1

[2021-10-16T16:56:08.668Z] + & powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -Comm ...

[2021-10-16T16:56:08.668Z] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[2021-10-16T16:56:08.668Z]     + CategoryInfo          : NotSpecified: (Flag '--paralle...KER_CLI_BUILD=1:String) [], RemoteException

[2021-10-16T16:56:08.668Z]     + FullyQualifiedErrorId : NativeCommandError

and

2021-10-16T17:03:42.911Z] x509: certificate signed by unknown authority

all over the place =/

[timja]

Passed on the third attempt

[dduportal]

The x509 error is weird. The other error is expected: it's a report of the docker-compose command failing, but powershell shows the top level error, and then prints the message from subcommand.