Post GCS links in CI summary (#13273)

Post useful artifact links in the summary. They should help people find the GCS links to download artifacts from CI, instead of looking into the build log.
iree-org · May 1, 2023 · c00c88c · c00c88c
1 parent 5c93c67
commit c00c88c
Show file tree

Hide file tree

Showing 2 changed files with 71 additions and 28 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -1089,3 +1089,51 @@ jobs:
         with:
           webhook: ${{ secrets.DISCORD_WEBHOOK }}
           description: "The following jobs failed: ${{ steps.failed_jobs.outputs.failed-jobs }}"
+      - name: Show useful artifact links
+        if: always()
+        env:
+          BUILD_DIR_GCS_ARTIFACT: ${{ needs.build_all.outputs.build-dir-gcs-artifact }}
+          E2E_TEST_ARTIFACTS_GCS_ARTIFACT_DIR: ${{ needs.build_e2e_test_artifacts.outputs.e2e-test-artifacts-gcs-artifact-dir }}
+          BENCHMARK_TOOLS_GCS_ARTIFACT_DIR: ${{ needs.build_benchmark_tools.outputs.benchmark-tools-gcs-artifact-dir }}
+          EXECUTION_BENCHMARK_RESULTS_GCS_ARTIFACT_DIR: ${{ needs.execution_benchmarks.outputs.benchmark-results-gcs-artifact-dir }}
+          COMPILATION_BENCHMARK_RESULTS_GCS_ARTIFACT: ${{ needs.compilation_benchmarks.outputs.compile-stats-results-gcs-artifact }}
+        # If the job of an artifact is skipped or failed, we show "Not present".
+        run: |
+          cat <<EOF >> "${GITHUB_STEP_SUMMARY}"
+          ### :link: Artifact Links
+
+          \`\`\`sh
+          # Compiler and host tools archive:
+          export BUILD_DIR_URL="${BUILD_DIR_GCS_ARTIFACT:-Not present}"
+          # Directory of e2e test artifacts:
+          export E2E_TEST_ARTIFACTS_DIR_URL="${E2E_TEST_ARTIFACTS_GCS_ARTIFACT_DIR:-Not present}"
+          # Directory of benchmark tool binaries:
+          export BENCHMARK_TOOLS_DIR_URL="${BENCHMARK_TOOLS_GCS_ARTIFACT_DIR:-Not present}"
+          # Directory of execution benchmark results and traces:
+          export EXECUTION_BENCHMARK_RESULTS_DIR_URL="${EXECUTION_BENCHMARK_RESULTS_GCS_ARTIFACT_DIR:-Not present}"
+          # Compilation benchmark results:
+          export COMPILATION_BENCHMARK_RESULTS_URL="${COMPILATION_BENCHMARK_RESULTS_GCS_ARTIFACT:-Not present}"
+          \`\`\`
+
+          You can list \`gs://\` directories and download files with [gcloud cli](
+          https://cloud.google.com/sdk/gcloud/reference/storage).
+
+          <details>
+          <summary>Click to show common usages</summary>
+
+          \`\`\`sh
+          # Download all artifacts
+          mkdir /tmp/iree_e2e_test_artifacts
+          gcloud storage cp -r "\${E2E_TEST_ARTIFACTS_DIR_URL?}/*" /tmp/iree_e2e_test_artifacts
+          \`\`\`
+
+          \`\`\`sh
+          # Download execution benchmark results and traces
+          mkdir /tmp/iree_benchmark_results
+          gcloud storage cp -r "\${EXECUTION_BENCHMARK_RESULTS_DIR_URL?}/*" /tmp/iree_benchmark_results
+          \`\`\`
+          </details>
+
+          To run benchmarks locally with the CI-built e2e test artifacts, see [IREE Benchmark Suites](
+          https://github.com/${GITHUB_REPOSITORY}/blob/main/docs/developers/developing_iree/benchmark_suites.md#3-fetch-the-benchmark-artifacts).
+          EOF
diff --git a/docs/developers/developing_iree/benchmark_suites.md b/docs/developers/developing_iree/benchmark_suites.md
@@ -152,12 +152,12 @@ If you don't have artifacts locally, see
 find the GCS directory of the CI artifacts. Then fetch the needed files:
 
 ```sh
-export GCS_URL="gs://iree-github-actions-<presubmit|postsubmit>-artifacts/<run_id>/<run_attempt>"
+# Get ${E2E_TEST_ARTIFACTS_DIR_URL} from "Fetching Benchmark Artifacts from CI".
 export E2E_TEST_ARTIFACTS_DIR="e2e_test_artifacts"
 
 # Download all artifacts
 mkdir "${E2E_TEST_ARTIFACTS_DIR?}"
-gcloud storage cp -r "${GCS_URL?}/e2e-test-artifacts" "${E2E_TEST_ARTIFACTS_DIR?}"
+gcloud storage cp -r "${E2E_TEST_ARTIFACTS_DIR_URL?}" "${E2E_TEST_ARTIFACTS_DIR?}"
 ```
 
 Run the helper tool to dump benchmark commands from benchmark configs:
@@ -188,22 +188,18 @@ build_tools/benchmarks/benchmark_helper.py dump-cmds \
 #### 1. Find the corresponding CI workflow run
 
 On the commit of the benchmark run, you can find the list of the workflow jobs
-by clicking the green check mark. Click the `Details` of job
-`build_e2e_test_artifacts`:
+by clicking the green check mark. Click any job starts with `CI /`:
 
-![image](https://user-images.githubusercontent.com/2104162/223781032-c22e2922-2bd7-422d-abc2-d6ef0d31b0f8.png)
+![image](https://user-images.githubusercontent.com/2104162/234647960-3df9d0f0-a34a-47ad-bda8-095ae44de865.png)
 
-#### 2. Get the GCS directory of the built artifacts
+#### 2. Get URLs of GCS artifacts
 
-On the job detail page, expand the step `Uploading e2 test artifacts`, you will
-see a bunch of lines like below. The
-URL`gs://iree-github-actions-<presubmit|postsubmit>-artifacts/<run_id>/<run_attempt>/`
-is the directory of artifacts:
+On the CI page, click `Summary` on the top-left to open the summary page. Scroll
+down and the links to artifacts are listed in a section titled "Artifact Links".
+Paste the content in your shell to define all needed variables for the following
+steps:
 
-```
-Copying file://build-e2e-test-artifacts/e2e_test_artifacts/iree_MobileBertSquad_fp32_module_fdff4caa105318036534bd28b76a6fe34e6e2412752c1a000f50fafe7f01ef07/module.vmfb to gs://iree-github-actions-postsubmit-artifacts/4360950546/1/e2e-test-artifacts/iree_MobileBertSquad_fp32_module_fdff4caa105318036534bd28b76a6fe34e6e2412752c1a000f50fafe7f01ef07/module.vmfb
-...
-```
+![image](https://user-images.githubusercontent.com/2104162/234716421-3a69b6ad-211d-4e39-8f9e-a4f22f91739d.png)
 
 #### 3. Fetch the benchmark artifacts
 
@@ -214,43 +210,42 @@ more usages). If you want to use CI artifacts to reproduce benchmarks locally,
 see
 [Find Compile and Run Commands to Reproduce Benchmarks](#find-compile-and-run-commands-to-reproduce-benchmarks).
 
-Set the GCS directory URL from the step
-[2. Get the GCS directory of the built artifacts](#2-get-the-gcs-directory-of-the-built-artifacts):
-
-```sh
-export GCS_URL="gs://iree-github-actions-<presubmit|postsubmit>-artifacts/<run_id>/<run_attempt>"
-```
+Assume you get the GCS URL variables from
+[Get URLs of GCS artifacts](#2-get-urls-of-gcs-artifacts).
 
 Download artifacts:
 
 ```sh
 # The GCS directory has the same structure as your local ${IREE_BUILD_DIR?}/e2e_test_artifacts.
-gcloud storage ls "${GCS_URL?}/e2e-test-artifacts"
+gcloud storage ls "${E2E_TEST_ARTIFACTS_DIR_URL?}"
 
 # Download all source and imported MLIR files:
-gcloud storage cp "${GCS_URL?}/e2e-test-artifacts/*.mlir" "<target_dir>"
+gcloud storage cp "${E2E_TEST_ARTIFACTS_DIR_URL?}/*.mlir" "<target_dir>"
 ```
 
 Execution and compilation benchmark configs can be downloaded at:
 
 ```sh
 # Execution benchmark config:
 gcloud storage cp \
-  "${GCS_URL?}/e2e-test-artifacts/execution-benchmark-config.json" \
+  "${E2E_TEST_ARTIFACTS_DIR_URL?}/execution-benchmark-config.json" \
   "${E2E_TEST_ARTIFACTS_DIR?}/exec_config.json"
 
 # Compilation benchmark config:
 gcloud storage cp \
-  "${GCS_URL?}/e2e-test-artifacts/compilation-benchmark-config.json" \
+  "${E2E_TEST_ARTIFACTS_DIR_URL?}/compilation-benchmark-config.json" \
   "${E2E_TEST_ARTIFACTS_DIR?}/comp_config.json"
 ```
 
 Benchmark raw results and traces can be downloaded at:
 
 ```sh
-# Benchmark raw results
-gcloud storage cp "${GCS_URL?}/benchmark-results/benchmark-results-*.json" .
+# Execution benchmark raw results
+gcloud storage cp "${EXECUTION_BENCHMARK_RESULTS_DIR_URL?}/benchmark-results-*.json" .
+
+# Execution benchmark traces
+gcloud storage cp "${EXECUTION_BENCHMARK_RESULTS_DIR_URL?}/benchmark-traces-*.tar.gz" .
 
-# Benchmark traces
-gcloud storage cp "${GCS_URL?}/benchmark-results/benchmark-traces-*.tar.gz" .
+# Compilation benchmark results
+gcloud storage cp "${COMPILATION_BENCHMARK_RESULTS_URL?}" .
 ```