Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add version.txt add checksum for llamacpp engine #229

Merged
merged 4 commits into from
Sep 18, 2024
Merged

Add version.txt add checksum for llamacpp engine #229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ab25e21
Add version.txt to tar.gz file
hientominh Sep 15, 2024
bd35fa8
Add checksum.yml to release
hientominh Sep 15, 2024
15c52b8
Add checksum.yml to release
hientominh Sep 15, 2024
e24ed60
Merge branch 'feat/checksum-and-version' of github.com:janhq/cortex.l…
hientominh Sep 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 95 additions & 24 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
name: "${{ env.VERSION }}"
draft: true
generate_release_notes: true
prerelease: false

build-and-test:
Expand Down Expand Up @@ -293,6 +294,11 @@ jobs:
with:
submodules: recursive

- name: Install coreutils macos
if: runner.os == 'macOS'
run: |
brew install coreutils

- name: Prepare Vulkan SDK Linux
if: ${{ matrix.vulkan && (matrix.os == 'linux') }}
run: |
Expand Down Expand Up @@ -371,13 +377,23 @@ jobs:
SCCACHE_IDLE_TIMEOUT: "0"

- name: Build
id: build-and-test
run: |
make build-example-server CMAKE_EXTRA_FLAGS="${{ matrix.cmake-flags }}"

- name: Pre Package
run: |
make pre-package

- uses: 1arp/create-a-file-action@0.4.5
with:
path: 'cortex.llamacpp'
isAbsolutePath: false
file: 'version.txt'
content: |
name: ${{ matrix.os }}-${{ matrix.name }}
version: ${{needs.create-draft-release.outputs.version}}

- name: Code Signing macOS
if: runner.os == 'macOS'
run: |
Expand All @@ -392,6 +408,7 @@ jobs:

- name: Package
run: |
cat cortex.llamacpp/version.txt
make package

- name: Run e2e testing
Expand All @@ -405,6 +422,41 @@ jobs:
name: cortex.llamacpp-${{ matrix.os }}-${{ matrix.name }}
path: ./cortex.llamacpp

- name: Calculate SHA512 Checksum (macOS)
if: runner.os == 'macOS'
run: |
sha512sum ./cortex.llamacpp.tar.gz | awk '{ print $1 }' > sha512.txt
size=$(stat -f%z ./cortex.llamacpp.tar.gz) # Sử dụng -f%z cho macOS
echo "checksum=$(cat sha512.txt)" >> $GITHUB_ENV
echo "size=$size" >> $GITHUB_ENV

- name: Calculate SHA512 Checksum (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
CertUtil -hashfile ./cortex.llamacpp.tar.gz SHA512 | Select-String -Pattern "^[0-9a-fA-F]+$" | Out-File sha512.txt
$size = (Get-Item ./cortex.llamacpp.tar.gz).length
echo "checksum=$(Get-Content sha512.txt)" >> $env:GITHUB_ENV
echo "size=$size" >> $env:GITHUB_ENV

- name: Calculate SHA512 Checksum (Linux)
if: runner.os == 'Linux'
run: |
sha512sum ./cortex.llamacpp.tar.gz | awk '{ print $1 }' > sha512.txt
size=$(stat -c%s ./cortex.llamacpp.tar.gz)
echo "checksum=$(cat sha512.txt)" >> $GITHUB_ENV
echo "size=$size" >> $GITHUB_ENV

## Write for matrix outputs workaround
- uses: cloudposse/github-action-matrix-outputs-write@v1
id: out
with:
matrix-step-name: ${{ github.job }}
matrix-key: ${{ matrix.os }}-${{ matrix.name }}
outputs: |-
sha512: ${{ env.checksum }}
size: ${{ env.size }}

- uses: actions/upload-release-asset@v1.0.1
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
env:
Expand All @@ -421,33 +473,52 @@ jobs:
run: |
sccache --stop-server
rm ${{ matrix.sccache-conf-path }}
update_release_draft:


## Read matrix outputs
read:
runs-on: ubuntu-latest
needs: [build-and-test]
steps:
- uses: cloudposse/github-action-matrix-outputs-read@v1
id: read
with:
matrix-step-name: build-and-test
outputs:
result: "${{ steps.read.outputs.result }}"

create-checksum-file:
runs-on: ubuntu-latest
needs: [read, create-draft-release]
permissions:
# write permission is required to create a github release
contents: write
# write permission is required for autolabeler
# otherwise, read permission is required at least
pull-requests: write
runs-on: ubuntu-latest
steps:
# (Optional) GitHub Enterprise requires GHE_HOST variable set
#- name: Set GHE_HOST
# run: |
# echo "GHE_HOST=${GITHUB_SERVER_URL##https:\/\/}" >> $GITHUB_ENV

# Drafts your next Release notes as Pull Requests are merged into "master"
- uses: release-drafter/release-drafter@v5
# (Optional) specify config name to use, relative to .github/. Default: release-drafter.yml
# with:
# config-name: my-config.yml
# disable-autolabeler: true
- name: Create checksum.yml
run: |
version=${{ needs.create-draft-release.outputs.version }}
outputs=${{ toJson(needs.read.outputs.result) }}

echo $outputs

echo "version: $version" > checksum.yml
echo "files:" >> checksum.yml

echo "$outputs" | jq -r --arg version "$version" '
.sha512 as $sha512 |
.size as $size |
(.sha512 | keys[]) as $key |
"- url: cortex.llamacpp-\($version)-\($key).tar.gz\n sha512: >-\n \($sha512[$key])\n size: \($size[$key])"
' >> checksum.yml

cat checksum.yml

- name: Upload checksum.yml to GitHub Release
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Clean
if: always()
continue-on-error: true
run: |
sccache --stop-server
rm ${{ matrix.sccache-conf-path }}
uses: actions/upload-release-asset@v1
with:
upload_url: ${{ needs.create-draft-release.outputs.upload_url }}
asset_path: ./checksum.yml
asset_name: checksum.yml
asset_content_type: text/yaml
98 changes: 98 additions & 0 deletions .github/workflows/nightly-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,11 @@ jobs:
with:
submodules: recursive

- name: Install coreutils macos
if: runner.os == 'macOS'
run: |
brew install coreutils

- name: Prepare Vulkan SDK Linux
if: ${{ matrix.vulkan && (matrix.os == 'linux') }}
run: |
Expand Down Expand Up @@ -371,13 +376,23 @@ jobs:
SCCACHE_IDLE_TIMEOUT: "0"

- name: Build
id: build-and-test
run: |
make build-example-server CMAKE_EXTRA_FLAGS="${{ matrix.cmake-flags }}"

- name: Pre Package
run: |
make pre-package

- uses: 1arp/create-a-file-action@0.4.5
with:
path: 'cortex.llamacpp'
isAbsolutePath: false
file: 'version.txt'
content: |
name: ${{ matrix.os }}-${{ matrix.name }}
version: ${{needs.create-draft-release.outputs.version}}

- name: Code Signing macOS
if: runner.os == 'macOS'
run: |
Expand Down Expand Up @@ -405,6 +420,41 @@ jobs:
name: cortex.llamacpp-${{ matrix.os }}-${{ matrix.name }}
path: ./cortex.llamacpp

- name: Calculate SHA512 Checksum (macOS)
if: runner.os == 'macOS'
run: |
sha512sum ./cortex.llamacpp.tar.gz | awk '{ print $1 }' > sha512.txt
size=$(stat -f%z ./cortex.llamacpp.tar.gz) # Sử dụng -f%z cho macOS
echo "checksum=$(cat sha512.txt)" >> $GITHUB_ENV
echo "size=$size" >> $GITHUB_ENV

- name: Calculate SHA512 Checksum (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
CertUtil -hashfile ./cortex.llamacpp.tar.gz SHA512 | Select-String -Pattern "^[0-9a-fA-F]+$" | Out-File sha512.txt
$size = (Get-Item ./cortex.llamacpp.tar.gz).length
echo "checksum=$(Get-Content sha512.txt)" >> $env:GITHUB_ENV
echo "size=$size" >> $env:GITHUB_ENV

- name: Calculate SHA512 Checksum (Linux)
if: runner.os == 'Linux'
run: |
sha512sum ./cortex.llamacpp.tar.gz | awk '{ print $1 }' > sha512.txt
size=$(stat -c%s ./cortex.llamacpp.tar.gz)
echo "checksum=$(cat sha512.txt)" >> $GITHUB_ENV
echo "size=$size" >> $GITHUB_ENV

## Write for matrix outputs workaround
- uses: cloudposse/github-action-matrix-outputs-write@v1
id: out
with:
matrix-step-name: ${{ github.job }}
matrix-key: ${{ matrix.os }}-${{ matrix.name }}
outputs: |-
sha512: ${{ env.checksum }}
size: ${{ env.size }}

- uses: actions/upload-release-asset@v1.0.1
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
env:
Expand All @@ -421,3 +471,51 @@ jobs:
run: |
sccache --stop-server
rm ${{ matrix.sccache-conf-path }}

## Read matrix outputs
read:
runs-on: ubuntu-latest
needs: [build-and-test]
steps:
- uses: cloudposse/github-action-matrix-outputs-read@v1
id: read
with:
matrix-step-name: build-and-test
outputs:
result: "${{ steps.read.outputs.result }}"

create-checksum-file:
runs-on: ubuntu-latest
permissions:
contents: write
needs: [read, create-draft-release]
steps:
- name: Create checksum.yml
run: |
version=${{ needs.create-draft-release.outputs.version }}
outputs=${{ toJson(needs.read.outputs.result) }}

echo $outputs

echo "version: $version" > checksum.yml
echo "files:" >> checksum.yml

echo "$outputs" | jq -r --arg version "$version" '
.sha512 as $sha512 |
.size as $size |
(.sha512 | keys[]) as $key |
"- url: cortex.llamacpp-\($version)-\($key).tar.gz\n sha512: >-\n \($sha512[$key])\n size: \($size[$key])"
' >> checksum.yml

cat checksum.yml

- name: Upload checksum.yml to GitHub Release
uses: actions/upload-release-asset@v1
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ needs.create-draft-release.outputs.upload_url }}
asset_path: ./checksum.yml
asset_name: checksum.yml
asset_content_type: text/yaml
Loading
Loading