Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validate release actions support mulit os #197

Merged
merged 24 commits into from
Apr 7, 2023
Merged
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 33 additions & 12 deletions .github/workflows/validate-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ on:
release_version:
required: true
default: '1.0.0'
gpg_user:
required: true
default: 'imbajin'


push:
branches:
Expand All @@ -16,11 +20,12 @@ on:

jobs:
build:
runs-on: ubuntu-latest
name: "Build On ${{ matrix.os }}(jdk-${{ matrix.java_version }})"
runs-on: ${{ matrix.os }}
env:
SCRIPT_PATH: hugegraph-dist/scripts/
URL_PREFIX: https://dist.apache.org/repos/dist/dev/incubator/hugegraph/
USER: 'imbajin'
USER: ${{ inputs.gpg_user }}
# TODO: parse version from the running branch name & also adapt the input version
RELEASE_VERSION: ''
steps:
Expand Down Expand Up @@ -104,27 +109,34 @@ jobs:
if [[ ! -f "DISCLAIMER" ]]; then
echo "The package should include DISCLAIMER file" && exit 1
fi
# 4.2 ensure doesn't contains *GPL/BCL/JSR-275/RSAL/QPL/SSPL/CPOL/NPL1.*/CC-BY
# dependency in LICENSE and NOTICE file
COUNT=$(grep -E "GPL|BCL|JSR-275|RSAL|QPL|SSPL|CPOL|NPL1|CC-BY" LICENSE NOTICE | wc -l)
if [[ $COUNT -ne 0 ]]; then
grep -E "GPL|BCL|JSR-275|RSAL|QPL|SSPL|CPOL|NPL1.0|CC-BY" LICENSE NOTICE
echo "The package shouldn't include GPL* invalid dependency, but get $COUNT" && exit 1
fi

# 4.2 ensure doesn't contains empty directory or file
# 4.3 ensure doesn't contains empty directory or file
COUNT=$(find . -type d -empty | wc -l)
if [[ $COUNT -ne 0 ]]; then
find . -type d -empty
echo "The package shouldn't include empty directory, but get $COUNT" # TODO: && exit 1
echo "The package shouldn't include empty directory, but get $COUNT" && exit 1
fi

# 4.3 ensure any file should less than 900kb & not include binary file
# 4.4 ensure any file should less than 900kb & not include binary file
COUNT=$(find . -type f -size +900k | wc -l)
if [[ $COUNT -ne 0 ]]; then
find . -type f -size +900k
echo "The package shouldn't include file larger than 900kb, but get $COUNT"
echo "The package shouldn't include file larger than 900kb, but get $COUNT" && exit 1
fi
COUNT=$(find . -type f | perl -lne 'print if -B' | grep -v *.txt | wc -l)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need a better way to search binary files(and exclude some files marked in properties)

no we can't exit here,but we should search the key word in action log manually

if [[ $COUNT -ne 0 ]]; then
find . -type f | perl -lne 'print if -B'
echo "The package shouldn't include binary file, but get $COUNT"
Copy link
Member

@imbajin imbajin Feb 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Find a critical problem here, we need exclude the files with a white-list & exit here:

  1. *.txt should be allowed, but why grep -v *.txt doesn't work?
  2. exclude the file /hugegraph-hubble/hubble-fe/src/assets/imgs/logo.png
  3. exclude the file ./hugegraph-hubble/hubble-fe/public/favicon.ico
  4. exclude the file yarn.lock

This is the 1st priority we need to fix and enhance it

Copy link
Member Author

@z7658329 z7658329 Feb 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Find a critical problem here, we need exclude the files with a white-list & exit here:

  1. *.txt should be allowed, but why grep -v *.txt doesn't work?
  2. exclude the file /hugegraph-hubble/hubble-fe/src/assets/imgs/logo.png
  3. exclude the file ./hugegraph-hubble/hubble-fe/public/favicon.ico
  4. exclude the file yarn.lock

This is the 1st priority we need to fix and enhance it

1.should exit while check failed ?
2.use 'grep --exclude=*.{txt}' to exclude *.txt , and similar to others ?
exclude works in my repo test,see: https://github.com/z7658329/incubator-hugegraph-doc/actions/runs/4168898449/jobs/7216220550

fi

# 4.4 test compile the packages
# 4.5 test compile the packages
if [[ $JAVA_VERSION == 8 && "$i" =~ "computer" ]]; then
cd .. && echo "skip computer module in java8"
continue
Expand Down Expand Up @@ -182,7 +194,7 @@ jobs:
fi
tar xzvf "$i" || exit

# 7.1 check root dir include "NOTICE"/"LICENSE"/"DISCLAIMER" files & "release-docs" dir
# 7.1 check root dir include "NOTICE"/"LICENSE"/"DISCLAIMER" files & "licenses" dir
cd "$(basename "$i" .tar.gz)" && ls -lh || exit
if [[ ! -f "LICENSE" ]]; then
echo "The package should include LICENSE file" && exit 1
Expand All @@ -193,15 +205,23 @@ jobs:
if [[ ! -f "DISCLAIMER" ]]; then
echo "The package should include DISCLAIMER file" && exit 1
fi
if [[ ! -d "release-docs" ]]; then
echo "The package should include release-docs dir" && exit 1
if [[ ! -d "licenses" ]]; then
echo "The package should include licenses dir" && exit 1
fi

# 7.2 ensure doesn't contains *GPL/BCL/JSR-275/RSAL/QPL/SSPL/CPOL/NPL1.*/CC-BY
# dependency in LICENSE/NOTICE and licenses/* files
COUNT=$(grep -r -E "GPL|BCL|JSR-275|RSAL|QPL|SSPL|CPOL|NPL1|CC-BY" LICENSE NOTICE licenses | wc -l)
if [[ $COUNT -ne 0 ]]; then
grep -r -E "GPL|BCL|JSR-275|RSAL|QPL|SSPL|CPQL|NPL1|CC-BY" LICENSE NOTICE licenses
echo "The package shouldn't include GPL* invalid dependency, but get $COUNT" && exit 1
fi

# 7.2 ensure doesn't contains empty directory or file
# 7.3 ensure doesn't contains empty directory or file
COUNT=$(find . -type d -empty | wc -l)
if [[ $COUNT -ne 0 ]]; then
find . -type d -empty
echo "The package shouldn't include empty directory, but get $COUNT" # TODO: && exit 1
echo "The package shouldn't include empty directory, but get $COUNT" && exit 1
fi

cd - || exit
Expand Down Expand Up @@ -250,3 +270,4 @@ jobs:
fail-fast: false
matrix:
java_version: [ '8','11' ]
os: [ubuntu-latest, macos-latest] # TODO: support windows-latest or other os in future
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perfer not to use comment after the line(use a separate line)

and seems miss centos here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i try to find centos, It doesn't seem to be found

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems github does not support CentOS, the supported runners: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources

I think we can mark "TODO: support CentOS" now.
how to support CentOS: https://zhuanlan.zhihu.com/p/344367359