Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Packaging] Drop CentOS 7 RPM package #23047

Merged
merged 2 commits into from
Jul 4, 2022
Merged

[Packaging] Drop CentOS 7 RPM package #23047

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e7279d8
drop-centos7
jiasli Jun 28, 2022
b787d8b
test
jiasli Jun 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 5 additions & 32 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -640,34 +640,7 @@ jobs:
ArtifactName: rpm-mariner$(tag)


- job: BuildRpmPackageCentOS7
displayName: Build Rpm Package CentOS 7
# Uncomment this line to disable this job on Pull Requests
# condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI', 'Manual', 'Schedule'))
pool:
vmImage: 'ubuntu-20.04'
steps:
- task: Bash@3
displayName: 'Build Rpm Package'
inputs:
targetType: 'filePath'
filePath: scripts/release/rpm/pipeline.sh
env:
IMAGE: centos7
- task: AzureArtifacts.manifest-generator-task.manifest-generator-task.ManifestGeneratorTask@0
displayName: 'SBOM'
inputs:
BuildDropPath: $(Build.ArtifactStagingDirectory)
- task: PublishPipelineArtifact@0
displayName: 'Publish Artifact: rpm'
inputs:
TargetPath: $(Build.ArtifactStagingDirectory)
# Should be 'rpm-centos7', but we keep 'yum' for backward compatibility
ArtifactName: yum

# TODO: rpmbuild on Red Hat UBI 8 is slow for unknown reason. Still working with Red Hat to investigate.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good news! BuildRpmPackageUbi8 is not as slow as before. Now it only takes ~20min.

image

# We use a separate job for Red Hat UBI 8 instead of strategy.matrix so that TestRpmPackage can start right after
# BuildRpmPackageCentOS7 finishes.
- job: BuildRpmPackageUbi8
displayName: Build Rpm Package Red Hat Universal Base Image 8
# Do not run this job for Pull Requests due to the slowness
Expand Down Expand Up @@ -696,7 +669,7 @@ jobs:
- job: TestRpmPackage
displayName: Test Rpm Package
timeoutInMinutes: 120
dependsOn: BuildRpmPackageCentOS7
dependsOn: BuildRpmPackageUbi8
condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI', 'Manual', 'Schedule'))
pool:
vmImage: 'ubuntu-20.04'
Expand All @@ -712,19 +685,19 @@ jobs:
displayName: 'Download Build Artifacts'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/rpm'
artifactName: yum
artifactName: rpm-ubi8


- bash: |
set -ex
Copy link
Contributor

@bebound bebound Jun 29, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bash's intent is incorrect (it should be 2 whitespace), could you please fix this in this PR?

Copy link
Member Author

@jiasli jiasli Jul 4, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean "indent"? You do have sharp eyes. 😀 There are some other occurrences:

azure-cli/azure-pipelines.yml

Lines 500 to 526 in e7279d8

- bash: |
#!/bin/bash
root=$(cd $(dirname $0); pwd)
set -evx
CLI_VERSION=`cat $BUILD_ARTIFACTSTAGINGDIRECTORY/metadata/version`
HOMEBREW_UPSTREAM_URL=`curl -Ls -o /dev/null -w %{url_effective} https://github.com/gitapi/repos/Azure/azure-cli/tarball/$BUILD_SOURCEVERSION`
docker_files=$(cd $BUILD_SOURCESDIRECTORY/scripts/release/homebrew/docker; pwd)
src_files=$(cd $BUILD_SOURCESDIRECTORY/src; pwd)
echo "Generating formula in docker container ... "
docker run -v $docker_files:/mnt/scripts \
-v $src_files:/mnt/src \
-e CLI_VERSION=$CLI_VERSION \
-e HOMEBREW_UPSTREAM_URL=$HOMEBREW_UPSTREAM_URL \
--name azurecli \
python:3.10 \
/mnt/scripts/run.sh
# clean up
rm -rf $BUILD_ARTIFACTSTAGINGDIRECTORY/metadata
docker cp azurecli:azure-cli.rb $BUILD_ARTIFACTSTAGINGDIRECTORY/azure-cli.rb
docker rm --force azurecli

Actually YAML does support arbitrary number of spaces as indent. Let's do this in another PR.


CLI_VERSION=`cat $SYSTEM_ARTIFACTSDIRECTORY/metadata/version`
RPM_NAME=azure-cli-$CLI_VERSION-1.el7.x86_64.rpm
RPM_NAME=azure-cli-$CLI_VERSION-1.el8.x86_64.rpm
RPM_FILE=$SYSTEM_ARTIFACTSDIRECTORY/rpm/$RPM_NAME

echo "== Test rpm package on CentOS =="
echo "== Test rpm package on Red Hat UBI 8 =="

IMAGE=centos:centos7
IMAGE=registry.access.redhat.com/ubi8/ubi:8.4
docker pull $IMAGE
docker run --rm -e RPM_NAME=$RPM_NAME -v $SYSTEM_ARTIFACTSDIRECTORY/rpm:/mnt/rpm -v $(pwd):/azure-cli $IMAGE /bin/bash "/azure-cli/scripts/release/rpm/test_rpm_in_docker.sh"

Expand Down
24 changes: 14 additions & 10 deletions scripts/release/rpm/test_rpm_in_docker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,26 @@ set -exv

export USERNAME=azureuser

yum --nogpgcheck localinstall /mnt/rpm/$RPM_NAME -y
PYTHON_PACKAGE=python39
PYTHON_CMD=python3.9
PIP_CMD=pip3.9

yum install git gcc python3-devel -y
dnf --nogpgcheck install /mnt/rpm/$RPM_NAME -y
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yum is deprecated. We use the latest dnf. Note localinstall is now a deprecated alias of install: https://dnf.readthedocs.io/en/latest/command_ref.html#install-command


ln -s -f /usr/bin/python3 /usr/bin/python
ln -s -f /usr/bin/pip3 /usr/bin/pip
dnf install git gcc $PYTHON_PACKAGE-devel -y

ln -s -f /usr/bin/$PYTHON_CMD /usr/bin/python
ln -s -f /usr/bin/$PIP_CMD /usr/bin/pip
time az self-test
time az --version

cd /azure-cli/
pip3 install wheel
pip install wheel
./scripts/ci/build.sh
pip3 install pytest --prefix /usr/lib64/az
pip3 install pytest-xdist --prefix /usr/lib64/az
pip install pytest --prefix /usr/lib64/az
pip install pytest-xdist --prefix /usr/lib64/az

find /azure-cli/artifacts/build -name "azure_cli_testsdk*" | xargs pip3 install --prefix /usr/lib64/az --upgrade --ignore-installed
find /azure-cli/artifacts/build -name "azure_cli_fulltest*" | xargs pip3 install --prefix /usr/lib64/az --upgrade --ignore-installed --no-deps
find /azure-cli/artifacts/build -name "azure_cli_testsdk*" | xargs pip install --prefix /usr/lib64/az --upgrade --ignore-installed
find /azure-cli/artifacts/build -name "azure_cli_fulltest*" | xargs pip install --prefix /usr/lib64/az --upgrade --ignore-installed --no-deps

python3 /azure-cli/scripts/release/rpm/test_rpm_package.py
python /azure-cli/scripts/release/rpm/test_rpm_package.py
4 changes: 2 additions & 2 deletions scripts/release/rpm/test_rpm_package.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@
import sys
import subprocess

root_dir = '/usr/lib64/az/lib/python3.6/site-packages/azure/cli/command_modules'
root_dir = '/usr/lib64/az/lib/python3.9/site-packages/azure/cli/command_modules'
mod_list = [mod for mod in sorted(os.listdir(root_dir)) if os.path.isdir(os.path.join(root_dir, mod)) and mod != '__pycache__']

pytest_base_cmd = 'PYTHONPATH=/usr/lib64/az/lib/python3.6/site-packages python3 -m pytest -x -v --boxed -p no:warnings --log-level=WARN'
pytest_base_cmd = 'PYTHONPATH=/usr/lib64/az/lib/python3.9/site-packages python3 -m pytest -x -v --boxed -p no:warnings --log-level=WARN'
pytest_parallel_cmd = '{} -n auto'.format(pytest_base_cmd)
serial_test_modules = ['botservice', 'network', 'cloud', 'appservice']

Expand Down