Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: set job status of e2e tests to passed after 15m timeout #6017

Merged
merged 4 commits into from
Oct 3, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 32 additions & 28 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ on:
branches: [unstable, stable]
pull_request:
workflow_dispatch:

env:
GETH_DOCKER_IMAGE: ethereum/client-go:v1.11.6
NETHERMIND_DOCKER_IMAGE: nethermind/nethermind:1.18.0

jobs:
build:
name: Build
Expand Down Expand Up @@ -53,9 +53,9 @@ jobs:
if: steps.cache-build-restore.outputs.cache-hit == 'true'
run: yarn build
- name: Check Build
run: yarn check-build
run: yarn check-build
- name: Test root binary exists
run: ./lodestar --version
run: ./lodestar --version
- name: Reject yarn.lock changes
run: .github/workflows/scripts/reject_yarn_lock_changes.sh
# Run only on forks
Expand All @@ -69,17 +69,17 @@ jobs:
packages/*/node_modules
lib/
packages/*/lib
packages/*/.git-data.json
packages/*/.git-data.json
key: ${{ runner.os }}-node-${{ matrix.node }}-${{ github.sha }}

lint:
name: Lint
needs: build
runs-on: 'ubuntu-latest'
runs-on: "ubuntu-latest"
strategy:
fail-fast: false
matrix:
node: [20]
node: [20]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
Expand All @@ -96,28 +96,28 @@ jobs:
packages/*/node_modules
lib/
packages/*/lib
packages/*/.git-data.json
packages/*/.git-data.json
key: ${{ runner.os }}-node-${{ matrix.node }}-${{ github.sha }}
fail-on-cache-miss: true
- name: Assert yarn prints no warnings
run: scripts/assert_no_yarn_warnings.sh
run: scripts/assert_no_yarn_warnings.sh
- name: Lint Code
run: yarn lint
run: yarn lint
- name: Lint Grafana dashboards
run: scripts/validate-grafana-dashboards.sh
run: scripts/validate-grafana-dashboards.sh
- name: Assert ESM module exports
run: node scripts/assert_exports.mjs
- name: Assert eslintrc rules sorted
run: scripts/assert_eslintrc_sorted.mjs

type-checks:
type-checks:
name: Type Checks
needs: build
runs-on: 'ubuntu-latest'
runs-on: "ubuntu-latest"
strategy:
fail-fast: false
matrix:
node: [20]
node: [20]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
Expand All @@ -134,16 +134,16 @@ jobs:
packages/*/node_modules
lib/
packages/*/lib
packages/*/.git-data.json
packages/*/.git-data.json
key: ${{ runner.os }}-node-${{ matrix.node }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Check Types
run: yarn check-types

- name: README check
run: yarn check-readme

unit-tests:
name: Unit Tests
needs: type-checks
Expand All @@ -168,7 +168,7 @@ jobs:
packages/*/node_modules
lib/
packages/*/lib
packages/*/.git-data.json
packages/*/.git-data.json
key: ${{ runner.os }}-node-${{ matrix.node }}-${{ github.sha }}
fail-on-cache-miss: true

Expand Down Expand Up @@ -199,7 +199,7 @@ jobs:
with:
node-version: ${{matrix.node}}
check-latest: true
cache: yarn
cache: yarn
- name: Restore build cache
id: cache-primes-restore
uses: actions/cache/restore@v3
Expand All @@ -209,15 +209,19 @@ jobs:
packages/*/node_modules
lib/
packages/*/lib
packages/*/.git-data.json
packages/*/.git-data.json
key: ${{ runner.os }}-node-${{ matrix.node }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Run the e2e test environment
run: scripts/run_e2e_env.sh start

- name: E2E tests
run: yarn test:e2e
# E2E tests are sometimes stalling until timeout is reached but we know that
# after 15 minutes those should have passed already if there are no failed test cases.
# In this case, just set the job status to passed as there was likely no actual issue.
# See https://github.com/ChainSafe/lodestar/issues/5913
run: timeout 15m yarn test:e2e || { test $? -eq 124 || exit 1; }
env:
GOERLI_RPC_URL: ${{ secrets.GOERLI_RPC_URL!=0 && secrets.GOERLI_RPC_URL || env.GOERLI_RPC_DEFAULT_URL }}

Expand All @@ -230,7 +234,7 @@ jobs:
with:
name: debug-e2e-test-logs-node-${{matrix.node}}
path: test-logs/e2e-test-env

browser-tests:
name: Browser Tests
runs-on: buildjet-4vcpu-ubuntu-2204
Expand All @@ -246,7 +250,7 @@ jobs:
with:
node-version: ${{matrix.node}}
check-latest: true
cache: yarn
cache: yarn
- name: Restore build cache
id: cache-primes-restore
uses: actions/cache/restore@v3
Expand All @@ -256,7 +260,7 @@ jobs:
packages/*/node_modules
lib/
packages/*/lib
packages/*/.git-data.json
packages/*/.git-data.json
key: ${{ runner.os }}-node-${{ matrix.node }}-${{ github.sha }}
fail-on-cache-miss: true

Expand Down Expand Up @@ -290,18 +294,18 @@ jobs:
packages/*/node_modules
lib/
packages/*/lib
packages/*/.git-data.json
packages/*/.git-data.json
key: ${{ runner.os }}-node-${{ matrix.node }}-${{ github.sha }}
fail-on-cache-miss: true

# Download spec tests with cache
- name: Restore spec tests cache
uses: actions/cache@master
with:
path: packages/beacon-node/spec-tests
key: spec-test-data-${{ hashFiles('packages/beacon-node/test/spec/specTestVersioning.ts') }}
- name: Download spec tests
run: yarn download-spec-tests
- name: Download spec tests
run: yarn download-spec-tests
working-directory: packages/beacon-node
# Run them in different steps to quickly identifying which command failed
# Otherwise just doing `yarn test:spec` you can't tell which specific suite failed
Expand Down