ci: buildkite release pipeline #509
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| #!/usr/bin/env bash | ||
| ## This script prepares the secret context | ||
| ## | ||
| ## NOTE: *_SECRET or *_TOKEN env variables are masked, hence if you'd like to avoid any | ||
| ## surprises please use the suffix _SECRET or _TOKEN for those values that contain | ||
| ## any sensitive data. Buildkite can mask those values automatically | ||
|
|
||
| set -eo pipefail | ||
|
|
||
| # To help with testing the GPG signing | ||
| BUILDKITE_TOKEN_SECRET=$(vault kv get -field=buildkite_token kv/ci-shared/observability-ci/buildkite-read-build-access) | ||
| export BUILDKITE_TOKEN_SECRET | ||
|
|
||
| # If not Git tag release then DRY_RUN=true | ||
| DRY_RUN=true | ||
| if [ -n "${BUILDKITE_TAG}" ] ; then | ||
| DRY_RUN=false | ||
| fi | ||
| export DRY_RUN | ||
|
|
||
| # Upload should only allow configuring the Buildkite Token ih the pre-command. | ||
| if [[ "$BUILDKITE_COMMAND" =~ .*"upload".* ]]; then | ||
| echo "Skipped pre-command when running the Upload pipeline" | ||
| # NOTE: exit 0 with the pre-command does not work! | ||
| else | ||
| echo "~~~ Configure ephemeral GitHub token" | ||
| GITHUB_TOKEN=$VAULT_GITHUB_TOKEN | ||
| GH_TOKEN=$VAULT_GITHUB_TOKEN | ||
| export GH_TOKEN GITHUB_TOKEN | ||
| set +x | ||
| set +e | ||
| echo "~~~ Install nvm" | ||
| touch ~/.zshrc # See https://github.com/nvm-sh/nvm?tab=readme-ov-file#troubleshooting-on-macos | ||
| curl -o- https://github.com/raw/nvm-sh/nvm/v0.39.7/install.sh | bash | ||
| fi | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| #!/usr/bin/env bash | ||
| # Download the signed artifacts and run the GitHub release using the GH cli | ||
| # | ||
| # Required environment variables: | ||
| # - BUILDKITE_TAG | ||
| # | ||
| set -eox pipefail | ||
|
|
||
| DIST_LOCATION=artifacts-to-upload | ||
|
|
||
| echo "--- Download signed artifacts" | ||
| mkdir -p "$DIST_LOCATION" | ||
| buildkite-agent artifact download --step gpg "*.*" "$DIST_LOCATION"/ | ||
| buildkite-agent artifact download --step macos "*.*" "$DIST_LOCATION"/ | ||
| buildkite-agent artifact download --step windows "*.*" "$DIST_LOCATION"/ | ||
| ls -l "$DIST_LOCATION"/ | ||
|
|
||
| echo "--- Install gh :github:" | ||
| if ! gh --version &>/dev/null ; then | ||
| wget -q https://github.com/cli/cli/releases/download/v2.50.0/gh_2.50.0_linux_amd64.tar.gz -O gh.tar.gz | ||
| tar -xpf gh.tar.gz --strip-components=2 | ||
| PATH="$(pwd):${PATH}" | ||
| export PATH | ||
| gh --version | ||
| fi | ||
|
|
||
| echo "--- Run GitHub release" | ||
| if [ -n "${BUILDKITE_TAG}" ] ; then | ||
| gh release \ | ||
| create \ | ||
| "${BUILDKITE_TAG}" \ | ||
| --draft \ | ||
| --title "${BUILDKITE_TAG}" \ | ||
| --repo elastic/synthetics-recorder \ | ||
| "${DIST_LOCATION}/*.*" | ||
| else | ||
| echo "gh release won't be triggered this is not a Git tag release" | ||
| fi | ||
reakaleek marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Create a dynamic buildkite step with the artifacts to be downloaded | ||
| # | ||
| # Required environment variables: | ||
| # - BUILDKITE_TOKEN_SECRET | ||
| # | ||
|
|
||
| STEP=$1 | ||
| DOWNLOAD_STEP_NAME=$2 | ||
|
|
||
| ## Support main pipeline and downstream pipelines | ||
| if [ -n "$BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG" ] ; then | ||
| BUILDKITE_PIPELINE_SLUG=$BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG | ||
| BUILDKITE_BUILD_NUMBER=$BUILDKITE_TRIGGERED_FROM_BUILD_NUMBER | ||
| fi | ||
|
|
||
| ## Fail if no token | ||
| if [ -z "$BUILDKITE_TOKEN_SECRET" ] ; then | ||
| echo "Token could not be loaded from vault. Please review .buildkite/hooks/pre-command" | ||
| exit 1 | ||
| fi | ||
|
|
||
| BUILDS_URL="https://api.buildkite.com/v2/organizations/elastic/pipelines/$BUILDKITE_PIPELINE_SLUG/builds" | ||
| build_json=$(curl -sH "Authorization: Bearer $BUILDKITE_TOKEN_SECRET" "$BUILDS_URL/$BUILDKITE_BUILD_NUMBER") | ||
| # sign-service is the pipeline step in .buildkite/release.yml | ||
| SIGN_BUILD_ID=$(jq -r ".jobs[] | select(.step_key == \"$STEP\").triggered_build.id" <<< "$build_json") | ||
|
|
||
| ## Fail if no build id | ||
| if [ -z "$SIGN_BUILD_ID" ] ; then | ||
| echo "Sign build id could not be found. Please review $BUILDS_URL/$BUILDKITE_BUILD_NUMBER and the below json output:" | ||
| echo "$build_json" | ||
| curl -sH "Authorization: Bearer $BUILDKITE_TOKEN_SECRET" "$BUILDS_URL" | ||
| exit 1 | ||
| fi | ||
|
|
||
| cat << EOF | ||
| - label: ":pipeline: Download signed artifacts" | ||
| key: "$DOWNLOAD_STEP_NAME" | ||
| commands: | ||
| - mkdir -p signed-artifacts | ||
| - buildkite-agent artifact download --build "$SIGN_BUILD_ID" "*.*" signed-artifacts/ | ||
| - cd signed-artifacts | ||
| - ls -ltra *.* | ||
| - buildkite-agent artifact upload "*.*" | ||
| agents: | ||
| image: docker.elastic.co/ci-agent-images/ubuntu-build-essential | ||
| EOF |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| #!/usr/bin/env bash | ||
| set -exo pipefail | ||
|
|
||
| echo "~~~ Load nvm" | ||
| if [ -n "$BUILDKITE" ] ; then | ||
| set +xe | ||
| # Need to figure out what's the reason NVM is not explictly loaded | ||
| source "$HOME/.zshrc" | ||
| fi | ||
|
|
||
| echo "--- Install node and gather dependencies" | ||
| nvm install "$(cat .nvmrc)" | ||
| npm ci | ||
|
|
||
| echo "--- run release-ci" | ||
| # Disable signing | ||
| # see https://www.electron.build/code-signing#how-to-disable-code-signing-during-the-build-process-on-macos | ||
| export CSC_IDENTITY_AUTO_DISCOVERY=false | ||
| # Disable notarize, see scripts/notarize.js | ||
| export SKIP_NOTARIZATION=true | ||
| npm run release-ci | ||
|
|
||
| # Store unsigned artifacts | ||
| if [ -n "$BUILDKITE" ] ; then | ||
| echo "--- Upload artifacts" | ||
| mv dist artifacts-to-sign | ||
| # (only *nix) | ||
| buildkite-agent artifact upload "artifacts-to-sign/*.deb;artifacts-to-sign/*.dmg;artifacts-to-sign/*.zip" | ||
|
|
||
| # (only windows) | ||
| buildkite-agent artifact upload "artifacts-to-sign/*.exe" | ||
|
|
||
| # (only mac) | ||
| buildkite-agent artifact upload "artifacts-to-sign/*.dmg" | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -41,7 +41,7 @@ npm test | |
| ``` | ||
|
|
||
| This will start the release process. Wait for an email/slack to confirm the | ||
| release is done. You can track the progress of the release job in [Buildkite](https://buildkite.com/elastic/synthetics-recorder-release/). | ||
|
|
||
| ### Criteria for publishing | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,7 +26,10 @@ spec: | |
| cancel_intermediate_builds: false | ||
| skip_intermediate_builds: false | ||
| provider_settings: | ||
| build_branches: false | ||
| build_tags: true | ||
| filter_condition: 'build.tag =~ /^v[0-9.]+$/' | ||
| filter_enabled: true | ||
| teams: | ||
| observablt-robots: | ||
| access_level: MANAGE_BUILD_AND_READ | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is an ongoing task to move away from BK API Tokens and use some opinionated contracts so we can skip this API call for the sake of simplicity. Release team is working on it