From 3e9b4ea0865294848f2eea956a562a6e30e9564e Mon Sep 17 00:00:00 2001 From: Alejandro Alvarez Date: Thu, 14 Dec 2023 12:30:05 +0100 Subject: [PATCH] DAT-13175 Full automation of OSS extension release (#134) * DAT-13175 * chore(release-drafter.yml): update Full Changelog link to include version prefix for better readability feat(extension-attach-artifact-release.yml): set draft release title to include the artifact id and version for better clarity and organization * fix(extension-attach-artifact-release.yml): update release title to use the version number from Maven The release title was previously set to "Support for Liquibase ${{ github.event.repository.name }} Extension v${{ env.artifact_id }}". This commit updates the release title to use the version number obtained from Maven. The version number is retrieved using the command "mvn help:evaluate -Dexpression=project.version -q -DforceStdout". The new release title format is "v$version", where $version is the version number obtained from Maven. * feat: add automated release workflow for OS extensions This commit adds a new workflow file `os-extension-automated-release.yml` that automates the release process for OS extensions. The workflow is triggered by a workflow call and requires two inputs: `version` (the version to release) and `repositories` (a comma-separated list of repositories). The workflow consists of several jobs: 1. `check-security-vulnerabilities`: This job checks for open Dependabot security alerts in the specified repositories. It loops through each repository, checks for open alerts, and sets a flag if any security vulnerabilities are found. If security vulnerabilities are found, the job prints an error message and exits with an error code. 2. `run-liquibase-dependabot`: This job installs the Dependabot CLI and runs Dependabot to check for and update dependencies in the Liquibase/Liquibase repository. 3. `release-draft-releases`: This job checks for draft releases in the specified repositories. If draft releases are found, it checks if they contain the specified version to release. If the version is found, it publishes the draft release as the latest release. 4. `create-and-close-nexus-stagging`: This job pauses the workflow for 60 seconds to allow time for Sonatype Nexus staging repositories to be ready. It then uses the Sonatype Nexus API to retrieve information about staging repositories and extracts the IDs of repositories containing the string "liquibase." For each repository, it checks if it is an extension by examining its relative path. If an extension is detected, the repository is closed using the Sonatype Nexus API. The purpose of this workflow is to automate the release process for OS extensions, including checking for security vulnerabilities, updating dependencies, publishing draft releases, and closing Nexus staging repositories for extensions. Note: The release step in the `create-and-close-nexus-stagging` job is currently commented out for safety. * chore(os-extension-automated-release.yml): update description for repositories input parameter to clarify its purpose * fix(os-extension-automated-release.yml): add support for archiving published_drafts.txt and closed_nexus_repos.txt files to preserve release information feat(os-extension-automated-release.yml): add logic to store the name and version of published drafts in published_drafts.txt file feat(os-extension-automated-release.yml): add logic to store the name and path of closed Nexus repositories in closed_nexus_repos.txt file * fix(os-extension-automated-release.yml): remove verbose flag from curl command to reduce output noise * fix(os-extension-automated-release.yml): change curl command to use silent mode (-s) to reduce output noise feat(os-extension-automated-release.yml): retrieve relative path of closed repository and store it in a file for reference * chore(os-extension-automated-release.yml): update dependabot command to run on multiple repositories fix(.DS_Store): delete .DS_Store file * chore(os-extension-automated-release.yml): update job name to accurately reflect the repositories being targeted by dependabot * fix(os-extension-automated-release.yml): fix typo in comment feat(os-extension-automated-release.yml): add update-pom job to update pom.xml with specified version * chore(os-extension-automated-release.yml): update default value of repositories input to be a JSON array instead of a comma separated string to improve readability and maintainability * fix(os-extension-automated-release.yml): remove unnecessary loop and fix variable references feat(os-extension-automated-release.yml): add support for matrix strategy to run jobs for multiple repositories refactor(os-extension-automated-release.yml): simplify code by removing unnecessary loop and fix variable references refactor(os-extension-automated-release.yml): improve code readability by removing unnecessary loop and fix variable references fix(os-extension-automated-release.yml): fix typo in comment fix(os-extension-automated-release.yml): fix typo in comment and remove unnecessary loop fix(os-extension-automated-release.yml): fix typo in comment and simplify code by removing unnecessary loop fix(os-extension-automated-release.yml): fix typo in comment and improve code readability by removing unnecessary loop fix(os-extension-automated-release.yml): fix typo in comment and improve code readability by removing unnecessary loop and fix variable references fix(os-extension-automated-release.yml): fix typo in comment and improve code readability by removing unnecessary loop, fix variable references, and remove unnecessary steps * fix(os-extension-automated-release.yml): rename step name from "Check Security Vulnerabilities" to "Security" for clarity fix(os-extension-automated-release.yml): rename step name from "Dependabot" to "Update pom.xml" for clarity fix(os-extension-automated-release.yml): rename step name from "Release Draft" to "Nexus" for clarity * chore(os-extension-automated-release.yml): add steps to set up Git and checkout repository to improve workflow reliability chore(os-extension-automated-release.yml): remove redundant git config commands from Update pom.xml step * chore(os-extension-automated-release.yml): remove unnecessary step to clone repository chore(os-extension-automated-release.yml): remove unused git clone command to improve workflow efficiency * fix(os-extension-automated-release.yml): remove unnecessary 'cd' command to improve script efficiency * chore(os-extension-automated-release.yml): update pom.xml with the specified version using sed command fix(os-extension-automated-release.yml): set GH_TOKEN environment variable to the value of BOT_TOKEN secret for updating pom.xml * fix(workflows): unset extraheader configuration for GitHub repository to fix authentication issue fix(workflows): set up Git user email and name to use "liquibot" for automated releases * fix(os-extension-automated-release.yml): fix indentation and remove unnecessary empty line fix(os-extension-automated-release.yml): fix token variable indentation and remove unnecessary whitespace fix(os-extension-automated-release.yml): set remote URL with token for authentication * chore(os-extension-automated-release.yml): add a 2-minute sleep before checking for artifacts in draft releases to allow time for the release to be published * chore(extension-attach-artifact-release.yml): remove unnecessary condition for running the job * chore(os-extension-automated-release.yml): update liquibase.version and project version to ${{ inputs.version }} to reflect changes in the project * chore(os-extension-automated-release.yml): increase sleep time to 180 seconds to allow more time for artifact to be published in draft releases feat(os-extension-automated-release.yml): add maven settings for liquibase and liquibase-pro repositories to enable fetching dependencies from them * chore(os-extension-automated-release.yml): reduce sleep time from 180 seconds to 120 seconds for checking artifact in draft releases to improve workflow efficiency * add module docs --- .github/workflows/create-release.yml | 2 +- .../extension-attach-artifact-release.yml | 9 +- .../workflows/extension-release-prepare.yml | 2 +- .../workflows/extension-release-published.yml | 2 +- .../os-extension-automated-release.yml | 244 ++++++++++++++++++ .github/workflows/os-extension-test.yml | 2 +- .github/workflows/package-linux.yml | 10 +- .github/workflows/pom-release-published.yml | 2 +- .github/workflows/pro-extension-test.yml | 2 +- README.md | 54 ++-- doc/.DS_Store | Bin 6148 -> 0 bytes doc/os-extension-automated-release.md | 103 ++++++++ 12 files changed, 395 insertions(+), 37 deletions(-) create mode 100644 .github/workflows/os-extension-automated-release.yml delete mode 100644 doc/.DS_Store create mode 100644 doc/os-extension-automated-release.md diff --git a/.github/workflows/create-release.yml b/.github/workflows/create-release.yml index ea76df7a..3398460a 100644 --- a/.github/workflows/create-release.yml +++ b/.github/workflows/create-release.yml @@ -11,7 +11,7 @@ on: jobs: sonar: - uses: liquibase/build-logic/.github/workflows/sonar-push.yml@v0.5.8 + uses: liquibase/build-logic/.github/workflows/sonar-push.yml@v0.5.9 secrets: inherit with: extraCommand: ${{ inputs.extraCommand }} diff --git a/.github/workflows/extension-attach-artifact-release.yml b/.github/workflows/extension-attach-artifact-release.yml index db4c2d43..1cba1ede 100644 --- a/.github/workflows/extension-attach-artifact-release.yml +++ b/.github/workflows/extension-attach-artifact-release.yml @@ -27,7 +27,6 @@ on: jobs: attach-to-release: name: Attach Artifact to Release - if: github.event.pull_request.merged == true runs-on: ubuntu-latest steps: - run: sleep 30 @@ -85,9 +84,9 @@ jobs: - name: Get Reusable Script Files run: | - curl -o $PWD/.github/get_draft_release.sh https://github.com/raw/liquibase/build-logic/v0.5.8/.github/get_draft_release.sh - curl -o $PWD/.github/sign_artifact.sh https://github.com/raw/liquibase/build-logic/v0.5.8/.github/sign_artifact.sh - curl -o $PWD/.github/upload_asset.sh https://github.com/raw/liquibase/build-logic/v0.5.8/.github/upload_asset.sh + curl -o $PWD/.github/get_draft_release.sh https://github.com/raw/liquibase/build-logic/v0.5.9/.github/get_draft_release.sh + curl -o $PWD/.github/sign_artifact.sh https://github.com/raw/liquibase/build-logic/v0.5.9/.github/sign_artifact.sh + curl -o $PWD/.github/upload_asset.sh https://github.com/raw/liquibase/build-logic/v0.5.9/.github/upload_asset.sh chmod +x $PWD/.github/get_draft_release.sh chmod +x $PWD/.github/sign_artifact.sh chmod +x $PWD/.github/upload_asset.sh @@ -176,7 +175,7 @@ jobs: - name: Get upload_zip.sh Script File if: inputs.zip == 'true' run: | - curl -o $PWD/.github/upload_zip.sh https://github.com/raw/liquibase/build-logic/v0.5.8/.github/upload_zip.sh + curl -o $PWD/.github/upload_zip.sh https://github.com/raw/liquibase/build-logic/v0.5.9/.github/upload_zip.sh chmod +x $PWD/.github/upload_zip.sh - name: Attach Zip File to Draft Release diff --git a/.github/workflows/extension-release-prepare.yml b/.github/workflows/extension-release-prepare.yml index f84371b1..cc7065b7 100644 --- a/.github/workflows/extension-release-prepare.yml +++ b/.github/workflows/extension-release-prepare.yml @@ -107,7 +107,7 @@ jobs: release-rollback: needs: prepare-release if: ${{ always() && contains(needs.*.result, 'failure') }} - uses: liquibase/build-logic/.github/workflows/extension-release-rollback.yml@v0.5.8 + uses: liquibase/build-logic/.github/workflows/extension-release-rollback.yml@v0.5.9 secrets: inherit with: extraCommand: ${{ inputs.extraCommand }} diff --git a/.github/workflows/extension-release-published.yml b/.github/workflows/extension-release-published.yml index 09ec4701..215c791f 100644 --- a/.github/workflows/extension-release-published.yml +++ b/.github/workflows/extension-release-published.yml @@ -19,7 +19,7 @@ on: jobs: maven-release: needs: release - uses: liquibase/build-logic/.github/workflows/extension-release-prepare.yml@v0.5.8 + uses: liquibase/build-logic/.github/workflows/extension-release-prepare.yml@v0.5.9 secrets: inherit with: extraCommand: ${{ inputs.extraCommand }} diff --git a/.github/workflows/os-extension-automated-release.yml b/.github/workflows/os-extension-automated-release.yml new file mode 100644 index 00000000..248f9382 --- /dev/null +++ b/.github/workflows/os-extension-automated-release.yml @@ -0,0 +1,244 @@ +name: Automated OS Extension Release + +on: + workflow_call: + inputs: + version: + description: 'Version to release (4.26.0, 4.26.1, etc.))' + required: true + type: string + repositories: + description: 'Comma separated list of repositories to release' + required: false + default: '["liquibase-bigquery","liquibase-cache","liquibase-cassandra","liquibase-cosmosdb","liquibase-db2i","liquibase-filechangelog","liquibase-nochangeloglock","liquibase-hanadb","liquibase-maxdb","liquibase-modify-column","liquibase-mssql","liquibase-oracle","liquibase-postgresql","liquibase-redshift","liquibase-sqlfire","liquibase-teradata","liquibase-vertica","liquibase-yugabytedb","liquibase-hibernate"]' + type: string + +jobs: + check-security-vulnerabilities: + # 1. Parse the input repositories into an array. + # 2. Initialize a flag to track if there are security vulnerabilities. + # 3. Check for open Dependabot security alerts. + # 4. Print the status of security vulnerabilities for each repository. + # 5. Set the flag if security vulnerabilities are found in any repository. + # 6. If security vulnerabilities are found in any repository, print an error message and exit with an error code. + runs-on: ubuntu-latest + name: Check Security Vulnerabilities + strategy: + matrix: + repository: ${{ fromJson(inputs.repositories) }} + steps: + - name: Security + run: | + security_fail=false + echo "Checking repository: ${{ matrix.repository }}" + security_url="https://github.com/gitapi/repos/liquibase/${{ matrix.repository }}/dependabot/alerts?state=open" + response=$(curl -s -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" -H "Authorization: Bearer ${{ secrets.BOT_TOKEN }}" $security_url | jq length) + echo "Open Alerts: $response" + if [[ $response == "0" ]]; then + echo "Security vulnerabilities for ${{ matrix.repository }} are addressed." + else + echo "Security vulnerabilities for ${{ matrix.repository }} are not addressed." + security_fail=true + fi + if [[ $security_fail == true ]]; then + echo "Security vulnerabilities are not addressed for ${{ matrix.repository }}" + exit 1 + fi + + run-extensions-dependabot: + needs: check-security-vulnerabilities + runs-on: ubuntu-latest + name: Dependabot + strategy: + matrix: + repository: ${{ fromJson(inputs.repositories) }} + steps: + - name: Install Dependabot CLI + run: | + #https://github.com/dependabot/cli + wget https://github.com/dependabot/cli/releases/download/v1.39.0/dependabot-v1.39.0-linux-amd64.tar.gz + tar xvzf dependabot-v1.39.0-linux-amd64.tar.gz + sudo mv dependabot /usr/local/bin/ + + - name: Run dependabot on extension + run: | + echo "Running Dependabot on repository: ${{ matrix.repository }}" + dependabot update maven "liquibase/${{ matrix.repository }}" + + update-pom: + needs: check-security-vulnerabilities + runs-on: ubuntu-latest + name: Update pom.xml + strategy: + matrix: + repository: ${{ fromJson(inputs.repositories) }} + + steps: + - name: Checkout repository + uses: actions/checkout@v3 + with: + repository: "liquibase/${{ matrix.repository }}" + token: ${{ secrets.BOT_TOKEN }} + + - name: Set up Git + run: | + git config --unset-all http.https://github.com/.extraheader + git config --local user.email "64099989+liquibot@users.noreply.github.com" + git config --local user.name "liquibot" + + # look for dependencies in maven + - name: maven-settings-xml-action + uses: whelk-io/maven-settings-xml-action@v21 + with: + repositories: | + [ + { + "id": "liquibase", + "url": "https://maven.pkg.github.com/liquibase/liquibase", + "releases": { + "enabled": "true" + }, + "snapshots": { + "enabled": "true", + "updatePolicy": "always" + } + }, + { + "id": "liquibase-pro", + "url": "https://maven.pkg.github.com/liquibase/liquibase-pro", + "releases": { + "enabled": "true" + }, + "snapshots": { + "enabled": "true", + "updatePolicy": "always" + } + } + ] + servers: | + [ + { + "id": "liquibase-pro", + "username": "liquibot", + "password": "${{ secrets.LIQUIBOT_PAT }}" + }, + { + "id": "liquibase", + "username": "liquibot", + "password": "${{ secrets.LIQUIBOT_PAT }}" + } + ] + + - name: Update pom.xml + env: + GH_TOKEN: ${{ secrets.BOT_TOKEN }} + run: | + mvn versions:set -DnewVersion=${{ inputs.version }}-SNAPSHOT + sed -i "s/.*<\/liquibase.version>/${{ inputs.version }}<\/liquibase.version>/" pom.xml + git add pom.xml + git commit -m "Update liquibase.version to ${{ inputs.version }}" + git remote set-url origin https://liquibot:${{ secrets.BOT_TOKEN }}@github.com/liquibase/${{ matrix.repository }}.git + git push + + release-draft-releases: + # 1. Initialize a flag to track if the specified version is found. + # 2. If draft releases are found, check if they contain the specified version to release. + # 3. If the version is found publish it as the latest release. + # 4. Print relevant information about the process. + needs: update-pom + runs-on: ubuntu-latest + name: Release Draft + strategy: + matrix: + repository: ${{ fromJson(inputs.repositories) }} + steps: + - name: Check for Artifact in Draft Releases + run: | + sleep 120 + published_drafts_file=published_drafts.txt + found=false + echo "Checking repository: ${{ matrix.repository }}" + assets=$(curl -s -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" -H "Authorization: Bearer ${{ secrets.BOT_TOKEN }}" "https://github.com/gitapi/repos/liquibase/${{ matrix.repository }}/releases" | jq -r '.[] | select(.draft == true)' | jq -r '.assets[]') + echo "Assets: $assets" + # check if assests are empty + if [ -z "$assets" ]; then + echo "No assets found" + else + if echo "$assets" | jq -e '.name | contains("${{ inputs.version }}")' > /dev/null; then + found=true + echo "Found an asset containing '${{ inputs.version }}'" + else + echo "No asset containing '${{ inputs.version }}' found" + fi + if [ "$found" = true ] ; then + # Get the draft release ID + RELEASE_ID=$(curl -s -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" -H "Authorization: Bearer ${{ secrets.BOT_TOKEN }}" "https://github.com/gitapi/repos/liquibase/${{ matrix.repository }}/releases" | jq -r '.[] | select(.draft == true) | .id') + echo "Draft release ID: $RELEASE_ID" + RELEASE_TITLE="v${{ inputs.version }}" + # Update the release title + # echo "Updating release title to $RELEASE_TITLE... for ${{ matrix.repository }}" + # curl -s -X PATCH -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" -H "Authorization: Bearer ${{ secrets.BOT_TOKEN }}" -d '{"name": "'"$RELEASE_TITLE"'"}' "https://github.com/gitapi/repos/liquibase/${{ matrix.repository }}/releases/$RELEASE_ID" + # Publish the draft release as the latest release + echo "Publishing the draft release as the latest release to https://github.com/gitapi/repos/liquibase/${{ matrix.repository }}/releases/$RELEASE_ID" + curl -s -X PATCH -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" -H "Authorization: Bearer ${{ secrets.BOT_TOKEN }}" -d '{"draft": false}' "https://github.com/gitapi/repos/liquibase/${{ matrix.repository }}/releases/$RELEASE_ID" + echo "Draft release published as the latest release for ${{ matrix.repository }}" + echo "${{ matrix.repository }}: v${{ inputs.version }}" >> $published_drafts_file + else + echo "No artifact containing '${{ inputs.version }}' found in the last draft release." + fi + fi + + + - name: Archive published_drafts.txt + uses: actions/upload-artifact@v3 + with: + name: published_drafts + path: published_drafts.txt + + create-and-close-nexus-stagging: + # 1. This job pauses the workflow for 60 seconds, allowing time for Sonatype Nexus stagging repos to be ready. + # 2. The job uses the Sonatype Nexus API to retrieve information about staging repositories. + # 3. It extracts the IDs of repositories containing the string "liquibase." + # 4. For each repository, it checks if it is an extension by examining its relative path. + # 5. If an extension is detected, the repository is closed using the Sonatype Nexus API. + # 6. A sleep of 120 seconds follows the closure before the final release step (commented out for safety). + needs: release-draft-releases + runs-on: ubuntu-latest + name: Nexus + steps: + - name: Wait for Sonatype Nexus + run: sleep 60 + + - name: Publish OSS Extensions + run: | + closed_nexus_repos_file=closed_nexus_repos.txt + api_output=$(curl -s -u "${{ secrets.SONATYPE_USERNAME }}:${{ secrets.SONATYPE_TOKEN }}" "https://oss.sonatype.org/service/local/staging/profile_repositories") + # Extract ids of repositories-item containing the string "liquibase" + repositories=$(echo "$api_output" | grep -B 8 "liquibase" | grep "" | awk -F"<|>" '{print $3}') + echo "Repository IDs containing 'liquibase': $repositories" + # Loop through each repository and close it + for repo_id in $repositories; do + echo "Check if $repo_id repository is an extension" + api_output=$(curl -s -u "${{ secrets.SONATYPE_USERNAME }}:${{ secrets.SONATYPE_TOKEN }}" "https://oss.sonatype.org/service/local/repositories/$repo_id/content/org/liquibase/") + relative_path=$(echo "$api_output" | grep -oP '\K[^<]+' | awk 'NR==1') + echo "Relative path: $relative_path" + if [[ "$relative_path" == *"/org/liquibase/ext/"* ]]; then + echo "Relative path contains '/org/liquibase/ext/'. It is an extension." + curl -s -u "${{ secrets.SONATYPE_USERNAME }}:${{ secrets.SONATYPE_TOKEN }}" -H "Accept: application/json" -H "Content-Type: application/json" -d '{"data":{"stagedRepositoryIds":["'"$repo_id"'"],"description":"Closing '"$repo_id"'."}}' https://oss.sonatype.org/service/local/staging/bulk/close + api_output=$(curl -s -u "${{ secrets.SONATYPE_USERNAME }}:${{ secrets.SONATYPE_TOKEN }}" "https://oss.sonatype.org/service/local/repositories/$repo_id/content/org/liquibase/ext/") + relative_path=$(echo "$api_output" | grep -oP '\K[^<]+' | awk 'NR==1') + echo "Closed repository: $repo_id - $relative_path" >> $closed_nexus_repos_file + #sleep 120 + #curl -s -u "${{ secrets.SONATYPE_USERNAME }}:${{ secrets.SONATYPE_TOKEN }}" -H "Accept: application/json" -H "Content-Type: application/json" -d '{"data":{"stagedRepositoryIds":["'"$repo_id"'"],"description":"Releasing '"$repo_id"'."}}' https://oss.sonatype.org/service/local/staging/bulk/promote + #echo "Released repository: $repo_id" + else + echo "Relative path does not contain '/org/liquibase/ext/'. It is not an extension." + fi + done + + - name: Archive closed_nexus_repos.txt + uses: actions/upload-artifact@v3 + with: + name: closed_nexus_repos + path: closed_nexus_repos.txt + diff --git a/.github/workflows/os-extension-test.yml b/.github/workflows/os-extension-test.yml index 58c6232b..aea6eee5 100644 --- a/.github/workflows/os-extension-test.yml +++ b/.github/workflows/os-extension-test.yml @@ -203,7 +203,7 @@ jobs: sonar-pr: if: ${{ !inputs.nightly }} needs: [ unit-test ] - uses: liquibase/build-logic/.github/workflows/sonar-pull-request.yml@v0.5.8 + uses: liquibase/build-logic/.github/workflows/sonar-pull-request.yml@v0.5.9 secrets: inherit with: extraCommand: ${{ inputs.extraCommand }} diff --git a/.github/workflows/package-linux.yml b/.github/workflows/package-linux.yml index 7858ff09..b8bc4841 100644 --- a/.github/workflows/package-linux.yml +++ b/.github/workflows/package-linux.yml @@ -64,11 +64,11 @@ jobs: # Under the src folder is where specific packages files live. The GitHub action inputs will modify the universal package-deb-pom.xml to tell the process which assets to use during the packaging step mkdir -p $PWD/.github/src/${{ inputs.artifactId }}/deb/control mkdir -p $PWD/.github/src/${{ inputs.artifactId }}/main/archive - curl -o $PWD/.github/src/${{ inputs.artifactId }}/deb/control/control https://github.com/raw/liquibase/build-logic/v0.5.8/src/${{ inputs.artifactId }}/deb/control/control - curl -o $PWD/.github/src/${{ inputs.artifactId }}/deb/control/postinst https://github.com/raw/liquibase/build-logic/v0.5.8/src/${{ inputs.artifactId }}/deb/control/postinst - curl -o $PWD/.github/src/${{ inputs.artifactId }}/main/archive/${{ inputs.artifactId }}-env.sh https://github.com/raw/liquibase/build-logic/v0.5.8/src/${{ inputs.artifactId }}/main/archive/${{ inputs.artifactId }}-env.sh - curl -o $PWD/.github/package-deb-pom.xml https://github.com/raw/liquibase/build-logic/v0.5.8/.github/package-deb-pom.xml - curl -o $PWD/.github/sign_artifact.sh https://github.com/raw/liquibase/build-logic/v0.5.8/.github/sign_artifact.sh + curl -o $PWD/.github/src/${{ inputs.artifactId }}/deb/control/control https://github.com/raw/liquibase/build-logic/v0.5.9/src/${{ inputs.artifactId }}/deb/control/control + curl -o $PWD/.github/src/${{ inputs.artifactId }}/deb/control/postinst https://github.com/raw/liquibase/build-logic/v0.5.9/src/${{ inputs.artifactId }}/deb/control/postinst + curl -o $PWD/.github/src/${{ inputs.artifactId }}/main/archive/${{ inputs.artifactId }}-env.sh https://github.com/raw/liquibase/build-logic/v0.5.9/src/${{ inputs.artifactId }}/main/archive/${{ inputs.artifactId }}-env.sh + curl -o $PWD/.github/package-deb-pom.xml https://github.com/raw/liquibase/build-logic/v0.5.9/.github/package-deb-pom.xml + curl -o $PWD/.github/sign_artifact.sh https://github.com/raw/liquibase/build-logic/v0.5.9/.github/sign_artifact.sh chmod +x $PWD/.github/sign_artifact.sh - name: Import GPG key diff --git a/.github/workflows/pom-release-published.yml b/.github/workflows/pom-release-published.yml index 18ac51cc..d37007ee 100644 --- a/.github/workflows/pom-release-published.yml +++ b/.github/workflows/pom-release-published.yml @@ -68,5 +68,5 @@ jobs: maven-release: needs: release - uses: liquibase/build-logic/.github/workflows/extension-release-prepare.yml@v0.5.8 + uses: liquibase/build-logic/.github/workflows/extension-release-prepare.yml@v0.5.9 secrets: inherit \ No newline at end of file diff --git a/.github/workflows/pro-extension-test.yml b/.github/workflows/pro-extension-test.yml index 263a2160..f9ba5a97 100644 --- a/.github/workflows/pro-extension-test.yml +++ b/.github/workflows/pro-extension-test.yml @@ -263,7 +263,7 @@ jobs: sonar-pr: if: ${{ !inputs.nightly }} needs: [ unit-test ] - uses: liquibase/build-logic/.github/workflows/sonar-pull-request.yml@v0.5.8 + uses: liquibase/build-logic/.github/workflows/sonar-pull-request.yml@v0.5.9 secrets: inherit with: extraCommand: ${{ inputs.extraCommand }} diff --git a/README.md b/README.md index 2a0fc99a..72a3dac1 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ software development lifecycle directly in your Github repository. These workflo different tasks, called actions, that can be executed automatically when certain events occur. At Liquibase, we use GitHub Actions for a wide range of tasks involved in the build, test, and release of extensions. + ## About reusable workflows To avoid code duplication of GitHub Actions workflow files across thousands of repositories, we @@ -64,30 +65,35 @@ graph LR Please review the below table of reusable workflows and their descriptions: -| Workflow | Description | -|-----------------------------------------|-------------------------------------------------------------------------------------------------------| -| `build-artifact.yml` | Runs maven build and saves artifacts | -| `codeql.yml` | Runs CodeQL scanning | -| `create-release.yml` | Runs Release Drafter to auto create draft release notes | -| `extension-attach-artifact-release.yml` | Attaches a tested artifact to the draft release. Receives a `zip` input to upload generated zip files | -| `extension-release-published.yml` | Publishes a release to Maven Central | -| `extension-update-version.yml` | Updates release and development `pom.xml` versions | -| `os-extension-test.yml` | Unit tests across build matrix on previously built artifact | -| `package-deb.yml` | Creates and uploads deb packages | -| `pom-release-published.yml` | Publishes a release pom to Maven Central | -| `pro-extension-test.yml` | Same as OS job, but with additional Pro-only vars such as License Key | -| `sonar-pull-request.yml` | Code Coverage Scan for PRs. Requires branch name parameter | -| `sonar-test-scan.yml` | Code Coverage Scan for unit and integration tests | -| `sonar-push.yml` | Same as PR job, but for pushes to main. Does not require branch name parameter | -| `snyk-nightly.yml` | Nightly Security Scans | -| various shell scripts | helper scripts for getting the draft release, signing artifacts, and uploading assets | +| Workflow | Description | +|-----------------------------------------|-------------------------------------------------------------------------------------------------------------------------| +| `build-artifact.yml` | Runs maven build and saves artifacts | +| `codeql.yml` | Runs CodeQL scanning | +| `create-release.yml` | Runs Release Drafter to auto create draft release notes | +| `extension-attach-artifact-release.yml` | Attaches a tested artifact to the draft release. Receives a `zip` input to upload generated zip files | +| `extension-release-published.yml` | Publishes a release to Maven Central | +| `extension-update-version.yml` | Updates release and development `pom.xml` versions | +| `os-extension-automated-release.yml` | Publishes draft releases and closes Nexus staging repositories. Details [here](./doc/os-extension-automated-release.md) | +| `os-extension-test.yml` | Unit tests across build matrix on previously built artifact | +| `package-deb.yml` | Creates and uploads deb packages | +| `pom-release-published.yml` | Publishes a release pom to Maven Central | +| `pro-extension-test.yml` | Same as OS job, but with additional Pro-only vars such as License Key | +| `sonar-pull-request.yml` | Code Coverage Scan for PRs. Requires branch name parameter | +| `sonar-test-scan.yml` | Code Coverage Scan for unit and integration tests | +| `sonar-push.yml` | Same as PR job, but for pushes to main. Does not require branch name parameter | +| `snyk-nightly.yml` | Nightly Security Scans | +| various shell scripts | helper scripts for getting the draft release, signing artifacts, and uploading assets | ## Requirements + ### pom.xml + The pom must meet all the requirements from sonatype: https://central.sonatype.org/publish/requirements/#a-complete-example-pom #### Jacoco + Jacoco must be configured and exporting test results. + ```xml org.jacoco @@ -105,7 +111,9 @@ Jacoco must be configured and exporting test results. ``` + #### Surefire + All unit tests must run and pass with `surefire:test`. If any test require additional setup, such as docker, they will need to run separately from the reusable build logic. ```xml @@ -120,12 +128,13 @@ All unit tests must run and pass with `surefire:test`. If any test require addit ``` #### Artifacts + The following artifacts must be created `mvn clean package`. If the javadoc and sources should not be public, please copy the contents of the readme for those files. This is based on the recommendation from sonatype: https://central.sonatype.org/publish/requirements/#supply-javadoc-and-sources. -* {artifactId}-{version}.jar -* {artifactId}-{version}.pom -* {artifactId}-{version}-javadoc.jar -* {artifactId}-{version}-sources.jar +- {artifactId}-{version}.jar +- {artifactId}-{version}.pom +- {artifactId}-{version}-javadoc.jar +- {artifactId}-{version}-sources.jar ```xml @@ -161,6 +170,7 @@ The following artifacts must be created `mvn clean package`. If the javadoc and ``` #### Maven release + The Maven release plugin must be configured to allow extensions update `pom.xml` versions: ```xml @@ -185,7 +195,9 @@ The Maven release plugin must be configured to allow extensions update `pom.xml` | `lth-docker.yml` | Runs Liquibase Test Harness against a docker container | ### Docker Databases + #### Requirements + - Docker Compose file must be located in `src/test/resources/docker-compose.yml` ## Liquibase test (unit & integration tests) + Sonar diff --git a/doc/.DS_Store b/doc/.DS_Store deleted file mode 100644 index 89066db13b73aa2938c87d67db4a9893d09b9b0c..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 6148 zcmeHK%}T>S5Z-O8Z7D(y3VI88E!dXQB3?qRFJMFuDm5{s2GeY5QgbMU-1UWg5ueAI z-3eIi!IOxcf!%L*em1)wWOqjhAqR_~Ercb6s6j)NsB924Z*-L?xC|(9j%4OX%7@Xr zKO^5?2V8qugkmZte)e_!QJAEy)`u*WD?7VYkVf_1eNvg5dg&q=c=KBvT`3j$h3@&+ z;W!&v`{yc7y)YinbwU`7A>{5Rj02So)FKWNo$DG$9b{x3Etl<1uhq2so&Kt6FHbw5 zcYEE{s$M%dJ~_J_Jx9}6y&66x(5+>Uuz)vEXgfjj82PX~3?Hlah9 z8w3xCG^v0lRZ@u|O*+_ZigO5agC<=_Dn2BAxhP(w{yNNW>u@0*gWNL%%)mMWSo?xl zc>iC(FSD`8*CE_71I)mGWk6Pj?r;FRqr3InKJl)#p*=uD!MGe15YX2i0WiRG