Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade all VMs to AMD64 Ubuntu 22.04 #86194

Merged
merged 19 commits into from
Aug 24, 2023
Merged

Upgrade all VMs to AMD64 Ubuntu 22.04 #86194

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
3f914f8
Upgrade all VMs to Ubuntu 22.04
richlander May 13, 2023
284f535
Update display names
richlander May 13, 2023
ac7d989
Revert wasm and mono back to 18.04
richlander May 13, 2023
185b519
Update eng/pipelines/coreclr/templates/helix-queues-setup.yml
richlander May 16, 2023
1f48b88
Merge branch 'main' into vm-host-upgrade
akoeplinger Jun 21, 2023
a47c5eb
Fix merge conflicts
akoeplinger Jun 21, 2023
895639c
Update Android queue
akoeplinger Jun 21, 2023
84c8065
Merge branch 'main' into vm-host-upgrade
richlander Jun 26, 2023
cd01132
Merge branch 'main' into vm-host-upgrade
akoeplinger Jun 27, 2023
6b5f4a5
Merge branch 'main' into vm-host-upgrade
richlander Jul 27, 2023
9e935d3
Merge branch 'main' into vm-host-upgrade
ivdiazsa Aug 8, 2023
2b5d543
Removed an Ubuntu.1804.Tiger queue I accidentally enabled when mergin…
ivdiazsa Aug 8, 2023
4f95327
Restored the Ubuntu 18.04 OS for the crossgen tiger perf queue due to…
ivdiazsa Aug 8, 2023
6b29522
Restored Ubuntu 18.04 in source-build.yml because those changes come …
ivdiazsa Aug 8, 2023
ba79284
Merge branch 'main' into vm-host-upgrade
richlander Aug 9, 2023
f232388
Merge branch 'main' into vm-host-upgrade
richlander Aug 15, 2023
6724888
Merge remote-tracking branch 'origin/main' into vm-host-upgrade
richlander Aug 21, 2023
8a03a11
Merge branch 'main' into vm-host-upgrade
richlander Aug 21, 2023
9ed75e6
Fix bad merge
richlander Aug 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions eng/common/templates/job/source-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,11 @@ jobs:
pool:
${{ if eq(variables['System.TeamProject'], 'public') }}:
name: $[replace(replace(eq(contains(coalesce(variables['System.PullRequest.TargetBranch'], variables['Build.SourceBranch'], 'refs/heads/main'), 'release'), 'true'), True, 'NetCore-Svc-Public' ), False, 'NetCore-Public')]
demands: ImageOverride -equals Build.Ubuntu.1804.Amd64.Open
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is eng/common - this change needs to happen in arcade.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've a quick question. How do you mean "in arcade"? The arcade repo?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, anything in eng/common is updated by arcade on each update from that repo, so it will always clobber this change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, we DON'T have to get all the 1804 instances in this PR. We can do a update PR. We do, however, need to make this PR run clean (obviously).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Since we are fine with gradually upgrading our lab machines, I can restore this specific change back to 18.04 and in the meantime, I'll work on the arcade part

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry. I should have mentioned that initially.

demands: ImageOverride -equals Build.Ubuntu.2204.Amd64.Open

${{ if eq(variables['System.TeamProject'], 'internal') }}:
name: $[replace(replace(eq(contains(coalesce(variables['System.PullRequest.TargetBranch'], variables['Build.SourceBranch'], 'refs/heads/main'), 'release'), 'true'), True, 'NetCore1ESPool-Svc-Internal'), False, 'NetCore1ESPool-Internal')]
demands: ImageOverride -equals Build.Ubuntu.1804.Amd64
demands: ImageOverride -equals Build.Ubuntu.2204.Amd64

${{ if ne(parameters.platform.pool, '') }}:
pool: ${{ parameters.platform.pool }}
Expand Down
4 changes: 2 additions & 2 deletions eng/pipelines/common/xplat-setup.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,12 +154,12 @@ jobs:
# Public Linux Build Pool
${{ if and(or(in(parameters.osGroup, 'linux', 'freebsd', 'android', 'tizen'), eq(parameters.jobParameters.hostedOs, 'linux')), eq(variables['System.TeamProject'], 'public')) }}:
name: $(DncEngPublicBuildPool)
demands: ImageOverride -equals Build.Ubuntu.1804.Amd64.Open
demands: ImageOverride -equals Build.Ubuntu.2204.Amd64.Open

# Official Build Linux Pool
${{ if and(or(in(parameters.osGroup, 'linux', 'freebsd', 'android', 'tizen'), eq(parameters.jobParameters.hostedOs, 'linux')), ne(variables['System.TeamProject'], 'public')) }}:
name: $(DncEngInternalBuildPool)
demands: ImageOverride -equals Build.Ubuntu.1804.Amd64
demands: ImageOverride -equals Build.Ubuntu.2204.Amd64

# OSX Build Pool (we don't have on-prem OSX BuildPool).
${{ if in(parameters.osGroup, 'osx', 'maccatalyst', 'ios', 'iossimulator', 'tvos', 'tvossimulator') }}:
Expand Down
10 changes: 5 additions & 5 deletions eng/pipelines/coreclr/templates/helix-queues-setup.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:

# Android x64
- ${{ if in(parameters.platform, 'android_x64') }}:
- Ubuntu.1804.Amd64.Android.29.Open
- Ubuntu.2204.Amd64.Android.29.Open
richlander marked this conversation as resolved.
Show resolved Hide resolved

# Browser wasm
- ${{ if eq(parameters.platform, 'browser_wasm') }}:
Expand Down Expand Up @@ -79,9 +79,9 @@ jobs:
# Linux musl x64
- ${{ if eq(parameters.platform, 'linux_musl_x64') }}:
- ${{ if eq(variables['System.TeamProject'], 'public') }}:
- (Alpine.315.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-amd64
- (Alpine.315.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-amd64
- ${{ if eq(variables['System.TeamProject'], 'internal') }}:
- (Alpine.315.Amd64)ubuntu.1804.amd64@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-amd64
- (Alpine.315.Amd64)Ubuntu.2204.Amd64@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-amd64

# Linux musl arm32
- ${{ if eq(parameters.platform, 'linux_musl_arm') }}:
Expand All @@ -100,9 +100,9 @@ jobs:
# Linux x64
- ${{ if eq(parameters.platform, 'linux_x64') }}:
- ${{ if eq(variables['System.TeamProject'], 'public') }}:
- Ubuntu.1804.Amd64.Open
- Ubuntu.2204.Amd64.Open
- ${{ if eq(variables['System.TeamProject'], 'internal') }}:
- Ubuntu.1804.Amd64
- Ubuntu.2204.Amd64

# OSX arm64
- ${{ if eq(parameters.platform, 'osx_arm64') }}:
Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/libraries/enterprise/linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ extends:
timeoutInMinutes: 120
pool:
name: $(DncEngPublicBuildPool)
demands: ImageOverride -equals Build.Ubuntu.1804.Amd64.Open
demands: ImageOverride -equals Build.Ubuntu.2204.Amd64.Open
steps:
- bash: |
cd $(enterpriseTestsSetup)
Expand Down
28 changes: 14 additions & 14 deletions eng/pipelines/libraries/helix-queues-setup.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ jobs:
# Linux musl x64
- ${{ if eq(parameters.platform, 'linux_musl_x64') }}:
- ${{ if or(ne(parameters.jobParameters.isExtraPlatforms, true), eq(parameters.jobParameters.includeAllPlatforms, true)) }}:
- (Alpine.315.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-amd64
- (Alpine.315.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-amd64
- ${{ if or(eq(parameters.jobParameters.isExtraPlatforms, true), eq(parameters.jobParameters.includeAllPlatforms, true)) }}:
- (Alpine.317.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.17-helix-amd64
- (Alpine.317.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.17-helix-amd64

# Linux musl arm64
- ${{ if and(eq(parameters.platform, 'linux_musl_arm64'), or(eq(parameters.jobParameters.isExtraPlatforms, true), eq(parameters.jobParameters.includeAllPlatforms, true))) }}:
Expand All @@ -61,25 +61,25 @@ jobs:
- ${{ if and(eq(parameters.jobParameters.interpreter, ''), ne(parameters.jobParameters.isSingleFile, true)) }}:
- ${{ if and(eq(parameters.jobParameters.testScope, 'outerloop'), eq(parameters.jobParameters.runtimeFlavor, 'mono')) }}:
- SLES.15.Amd64.Open
- (Centos.8.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-stream8-helix
- (Fedora.36.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-36-helix
- (Ubuntu.2204.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-22.04-helix-amd64
- (Centos.8.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-stream8-helix
- (Fedora.36.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-36-helix
- (Ubuntu.2204.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-22.04-helix-amd64
- (Debian.11.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64
- ${{ if or(ne(parameters.jobParameters.testScope, 'outerloop'), ne(parameters.jobParameters.runtimeFlavor, 'mono')) }}:
- ${{ if or(eq(parameters.jobParameters.isExtraPlatforms, true), eq(parameters.jobParameters.includeAllPlatforms, true)) }}:
- SLES.15.Amd64.Open
- (Fedora.36.Amd64.Open)ubuntu.1804.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-36-helix
- (Fedora.36.Amd64.Open)Ubuntu.2204.Amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-36-helix
- Ubuntu.2204.Amd64.Open
- (Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64
- (Mariner.2.0.Amd64.Open)ubuntu.1804.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-helix-amd64
- (openSUSE.15.2.Amd64.Open)ubuntu.1804.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:opensuse-15.2-helix-amd64
- (Debian.11.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64
- (Mariner.2.0.Amd64.Open)Ubuntu.2204.Amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-helix-amd64
- (openSUSE.15.2.Amd64.Open)Ubuntu.2204.Amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:opensuse-15.2-helix-amd64
- ${{ if or(ne(parameters.jobParameters.isExtraPlatforms, true), eq(parameters.jobParameters.includeAllPlatforms, true)) }}:
- (Centos.8.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-stream8-helix
- (Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64
- Ubuntu.1804.Amd64.Open
- (Centos.8.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-stream8-helix
- (Debian.11.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64
- Ubuntu.2204.Amd64.Open
- ${{ if or(eq(parameters.jobParameters.interpreter, 'true'), eq(parameters.jobParameters.isSingleFile, true)) }}:
# Limiting interp runs as we don't need as much coverage.
- (Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64
- (Debian.11.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64

# Linux s390x
- ${{ if eq(parameters.platform, 'linux_s390x') }}:
Expand All @@ -99,7 +99,7 @@ jobs:

# Android
- ${{ if in(parameters.platform, 'android_x86', 'android_x64', 'linux_bionic_x64') }}:
- Ubuntu.1804.Amd64.Android.29.Open
- Ubuntu.2204.Amd64.Android.29.Open
akoeplinger marked this conversation as resolved.
Show resolved Hide resolved
- ${{ if in(parameters.platform, 'android_arm', 'android_arm64', 'linux_bionic_arm64') }}:
- Windows.10.Amd64.Android.Open

Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/libraries/stress/ssl.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ extends:
timeoutInMinutes: 120
pool:
name: $(DncEngPublicBuildPool)
demands: ImageOverride -equals Build.Ubuntu.1804.Amd64.Open
demands: ImageOverride -equals Build.Ubuntu.2204.Amd64.Open

steps:
- checkout: self
Expand Down
10 changes: 4 additions & 6 deletions eng/testing/performance/performance-setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ benchmark_directory=$payload_directory/BenchmarkDotNet
workitem_directory=$source_directory/workitem
extra_benchmark_dotnet_arguments="--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
perflab_arguments=
queue=Ubuntu.1804.Amd64.Open
queue=Ubuntu.2204.Amd64.Open
creator=$BUILD_DEFINITIONNAME
helix_source_prefix="pr"

Expand All @@ -279,14 +279,12 @@ if [[ "$internal" == true ]]; then
elif [[ "$logical_machine" == "perfampere" ]]; then
queue=Ubuntu.2004.Arm64.Perf
elif [[ "$logical_machine" == "cloudvm" ]]; then
queue=Ubuntu.1804.Amd64
queue=Ubuntu.2204.Amd64
elif [[ "$architecture" == "arm64" ]]; then
queue=Ubuntu.1804.Arm64.Perf
else
if [[ "$logical_machine" == "perfowl" ]]; then
queue=Ubuntu.1804.Amd64.Owl.Perf
elif [[ "$logical_machine" == "perftiger_crossgen" ]]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perftiger_crossgen requires using a version of lttng that's not available on 22

queue=Ubuntu.1804.Amd64.Tiger.Perf
queue=Ubuntu.2204.Amd64.Owl.Perf
else
queue=Ubuntu.2204.Amd64.Tiger.Perf
fi
Expand All @@ -299,7 +297,7 @@ else
if [[ "$architecture" == "arm64" ]]; then
queue=ubuntu.1804.armarch.open
else
queue=Ubuntu.1804.Amd64.Open
queue=Ubuntu.2204.Amd64.Open
fi

if [[ "$alpine" == "true" ]]; then
Expand Down
4 changes: 2 additions & 2 deletions src/coreclr/scripts/superpmi_collect_setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
# | Arch | windows | Linux | macOS |
# |-------|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------|----------------|
# | x86 | Windows.10.Amd64.X86.Rt | - | - |
# | x64 | Windows.10.Amd64.X86.Rt | Ubuntu.1804.Amd64 | OSX.1014.Amd64 |
# | x64 | Windows.10.Amd64.X86.Rt | Ubuntu.2204.Amd64 | OSX.1014.Amd64 |
# | arm | - | (Ubuntu.1804.Arm32)Ubuntu.2004.ArmArch@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7 | - |
# | arm64 | Windows.11.Arm64 | (Ubuntu.1804.Arm64)Ubuntu.2004.ArmArch@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8 | OSX.1100.ARM64 |
#
Expand Down Expand Up @@ -463,7 +463,7 @@ def main(main_args):
elif arch == "arm64":
helix_queue = "(Ubuntu.1804.Arm64)Ubuntu.2004.ArmArch@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8"
else:
helix_queue = "Ubuntu.1804.Amd64"
helix_queue = "Ubuntu.2204.Amd64"
elif platform_name == "osx":
helix_queue = "OSX.1100.ARM64" if arch == "arm64" else "OSX.1014.Amd64"

Expand Down
2 changes: 1 addition & 1 deletion src/tests/Common/helixpublishwitharcade.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
<_HelixAccessToken></_HelixAccessToken>
<_HelixBuild></_HelixBuild>
<_HelixSource></_HelixSource>
<_HelixTargetQueues>Ubuntu.1804.Amd64.Open</_HelixTargetQueues>
<_HelixTargetQueues>Ubuntu.2204.Amd64.Open</_HelixTargetQueues>
<_HelixType>test/stuff</_HelixType>
<_PublishTestResults>false</_PublishTestResults>
<_RunCrossGen>false</_RunCrossGen>
Expand Down