Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: centralized job for rocksdb libaries cache (backport #21657) #21665

Merged
merged 2 commits into from
Sep 11, 2024
Merged

ci: centralized job for rocksdb libaries cache (backport #21657) #21665

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dfa6e88
ci: centralized job for rocksdb libaries cache (#21657)
auricom Sep 11, 2024
0e2d7e7
conflicts
julienrbrt Sep 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/scripts/install-rocksdb-deps.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -Eeuo pipefail

sudo apt update && sudo apt-get install -y libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev build-essential
3 changes: 0 additions & 3 deletions .github/scripts/install-rocksdb.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@ if [ -z "$ROCKSDB_VERSION" ]; then
exit 1
fi

# Update and install dependencies
sudo apt update && sudo apt-get install -y libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev build-essential

# Clone RocksDB repository
git clone https://github.com/facebook/rocksdb.git /home/runner/rocksdb
cd /home/runner/rocksdb || exit 1
Expand Down
12 changes: 3 additions & 9 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,18 +38,12 @@ jobs:
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-${{ matrix.go-arch }}
- name: Install rocksdb deps
if: matrix.go-arch == 'amd64'
run: ./.github/scripts/install-rocksdb-deps.sh
- name: Install rocksdb
if: matrix.go-arch == 'amd64' && steps.cache-rocksdb.outputs.cache-hit != 'true'
id: install_rocksdb
run: ./.github/scripts/install-rocksdb.sh
- name: Saves rocksdb libraries cache
if: matrix.go-arch == 'amd64' && steps.install_rocksdb.outcome == 'success'
uses: actions/cache/save@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-${{ matrix.go-arch }}
###################
#### Build App ####
###################
Expand Down
62 changes: 62 additions & 0 deletions .github/workflows/cache-rocksdb.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: Cache rocksdb libraries
on:
push:
paths:
- build.mk
schedule:
- cron: "*/15 * * * *" # Every 15 minutes
workflow_dispatch:

permissions:
contents: read

jobs:

check-cache-rocksdb:
name: Check existing cache
runs-on: ubuntu-latest
outputs:
cache-hit: ${{ steps.cache-rocksdb.outputs.cache-hit }}

steps:
- uses: actions/checkout@v4

- name: Get rocksdb version
run: ./.github/scripts/get-rocksdb-version.sh

- name: Fix permissions for cache
run: sudo chown $(whoami) /usr/local/lib /usr/local/include

- name: Restore rocksdb libraries cache
id: cache-rocksdb
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64


save-cache-rocksdb:
name: Build rocksdb libraries and save cache
runs-on: ubuntu-latest
needs: check-cache-rocksdb
if: needs.check-cache-rocksdb.outputs.cache-hit != 'true'
steps:
- uses: actions/checkout@v4

- name: Get rocksdb version
run: ./.github/scripts/get-rocksdb-version.sh

- name: Install rocksdb deps
run: ./.github/scripts/install-rocksdb-deps.sh
- name: Install rocksdb
run: ./.github/scripts/install-rocksdb.sh

- name: Saves rocksdb libraries cache
uses: actions/cache/save@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
13 changes: 3 additions & 10 deletions .github/workflows/lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,18 +39,11 @@ jobs:
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: Install rocksdb deps
run: ./.github/scripts/install-rocksdb-deps.sh
- name: Install rocksdb
if: env.GIT_DIFF && steps.cache-rocksdb.outputs.cache-hit != 'true'
id: install_rocksdb
if: steps.cache-rocksdb.outputs.cache-hit != 'true'
run: ./.github/scripts/install-rocksdb.sh
- name: Saves rocksdb libraries cache
if: steps.install_rocksdb.outcome == 'success'
uses: actions/cache/save@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: run linting (long)
if: env.GIT_DIFF
id: lint_long
Expand Down
26 changes: 6 additions & 20 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -799,18 +799,11 @@ jobs:
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: Install rocksdb deps
run: ./.github/scripts/install-rocksdb-deps.sh
- name: Install rocksdb
if: env.GIT_DIFF && steps.cache-rocksdb.outputs.cache-hit != 'true'
id: install_rocksdb
if: steps.cache-rocksdb.outputs.cache-hit != 'true'
run: ./.github/scripts/install-rocksdb.sh
- name: Saves rocksdb libraries cache
if: steps.install_rocksdb.outcome == 'success'
uses: actions/cache/save@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: tests
if: env.GIT_DIFF
run: |
Expand Down Expand Up @@ -856,18 +849,11 @@ jobs:
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: Install rocksdb deps
run: ./.github/scripts/install-rocksdb-deps.sh
- name: Install rocksdb
if: env.GIT_DIFF && steps.cache-rocksdb.outputs.cache-hit != 'true'
id: install_rocksdb
if: steps.cache-rocksdb.outputs.cache-hit != 'true'
run: ./.github/scripts/install-rocksdb.sh
- name: Saves rocksdb libraries cache
if: steps.install_rocksdb.outcome == 'success'
uses: actions/cache/save@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: test & coverage report creation
if: env.GIT_DIFF
run: |
Expand Down
Loading