Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ability to specify commit of avalanchego in ci #589

Merged
merged 1 commit into from
Mar 24, 2023
Merged

Add ability to specify commit of avalanchego in ci #589

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 98 additions & 21 deletions scripts/install_avalanchego_release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,41 +11,118 @@ source "$SUBNET_EVM_PATH"/scripts/versions.sh
# Load the constants
source "$SUBNET_EVM_PATH"/scripts/constants.sh

VERSION=$AVALANCHEGO_VERSION

############################
# download avalanchego
# https://github.com/ava-labs/avalanchego/releases
GOARCH=$(go env GOARCH)
GOOS=$(go env GOOS)
BASEDIR=${BASE_DIR:-"/tmp/avalanchego-release"}
BASEDIR=${BASEDIR:-"/tmp/avalanchego-release"}
AVALANCHEGO_BUILD_PATH=${AVALANCHEGO_BUILD_PATH:-${BASEDIR}/avalanchego}

mkdir -p ${BASEDIR}
AVAGO_DOWNLOAD_URL=https://github.com/ava-labs/avalanchego/releases/download/${VERSION}/avalanchego-linux-${GOARCH}-${VERSION}.tar.gz
AVAGO_DOWNLOAD_PATH=${BASEDIR}/avalanchego-linux-${GOARCH}-${VERSION}.tar.gz

AVAGO_DOWNLOAD_URL=https://github.com/ava-labs/avalanchego/releases/download/${AVALANCHEGO_VERSION}/avalanchego-linux-${GOARCH}-${AVALANCHEGO_VERSION}.tar.gz
AVAGO_DOWNLOAD_PATH=${BASEDIR}/avalanchego-linux-${GOARCH}-${AVALANCHEGO_VERSION}.tar.gz

if [[ ${GOOS} == "darwin" ]]; then
AVAGO_DOWNLOAD_URL=https://github.com/ava-labs/avalanchego/releases/download/${VERSION}/avalanchego-macos-${VERSION}.zip
AVAGO_DOWNLOAD_PATH=${BASEDIR}/avalanchego-macos-${VERSION}.zip
AVAGO_DOWNLOAD_URL=https://github.com/ava-labs/avalanchego/releases/download/${AVALANCHEGO_VERSION}/avalanchego-macos-${AVALANCHEGO_VERSION}.zip
AVAGO_DOWNLOAD_PATH=${BASEDIR}/avalanchego-macos-${AVALANCHEGO_VERSION}.zip
fi

AVALANCHEGO_BUILD_PATH=${AVALANCHEGO_BUILD_PATH:-${BASEDIR}/avalanchego-${VERSION}}
mkdir -p $AVALANCHEGO_BUILD_PATH
BUILD_DIR=${AVALANCHEGO_BUILD_PATH}-${AVALANCHEGO_VERSION}
aaronbuchwald marked this conversation as resolved.
Show resolved Hide resolved

if [[ ! -f ${AVAGO_DOWNLOAD_PATH} ]]; then
echo "downloading avalanchego ${VERSION} at ${AVAGO_DOWNLOAD_URL} to ${AVAGO_DOWNLOAD_PATH}"
curl -L ${AVAGO_DOWNLOAD_URL} -o ${AVAGO_DOWNLOAD_PATH}
fi
echo "extracting downloaded avalanchego to ${AVALANCHEGO_BUILD_PATH}"
if [[ ${GOOS} == "linux" ]]; then
mkdir -p ${AVALANCHEGO_BUILD_PATH} && tar xzvf ${AVAGO_DOWNLOAD_PATH} --directory ${AVALANCHEGO_BUILD_PATH} --strip-components 1
elif [[ ${GOOS} == "darwin" ]]; then
unzip ${AVAGO_DOWNLOAD_PATH} -d ${AVALANCHEGO_BUILD_PATH}
mv ${AVALANCHEGO_BUILD_PATH}/build/* ${AVALANCHEGO_BUILD_PATH}
rm -rf ${AVALANCHEGO_BUILD_PATH}/build/
extract_archive() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😎 (why I can't do this as a reaction is beyond me)

mkdir -p ${BUILD_DIR}

if [[ ${AVAGO_DOWNLOAD_PATH} == *.tar.gz ]]; then
tar xzvf ${AVAGO_DOWNLOAD_PATH} --directory ${BUILD_DIR} --strip-components 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we drop v?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's worth the change. Every time I make a change, I have to run CI three different times to make sure everything is working as expected (one for tagged version, one for branch, and one for commit). While I know that dropping a "verbose" option won't change any functionality, if I don't run those tests again, the screen shots and linked actions won't actually contain a commit in this branch.

Long winded way of saying, "if we want to make that change, it should happen in another PR".

elif [[ ${AVAGO_DOWNLOAD_PATH} == *.zip ]]; then
unzip ${AVAGO_DOWNLOAD_PATH} -d ${BUILD_DIR}
mv ${BUILD_DIR}/build/* ${BUILD_DIR}
rm -rf ${BUILD_DIR}/build/
fi
}

# first check if we already have the archive
if [[ -f ${AVAGO_DOWNLOAD_PATH} ]]; then
# if the download path already exists, extract and exit
echo "found avalanchego ${AVALANCHEGO_VERSION} at ${AVAGO_DOWNLOAD_PATH}"

extract_archive
else
# try to download the archive if it exists
if curl -s --head --request GET ${AVAGO_DOWNLOAD_URL} | grep "302" > /dev/null; then
echo "${AVAGO_DOWNLOAD_URL} found"
echo "downloading to ${AVAGO_DOWNLOAD_PATH}"
curl -L ${AVAGO_DOWNLOAD_URL} -o ${AVAGO_DOWNLOAD_PATH}

extract_archive
else
# else the version is a git commitish (or it's invalid)
GIT_CLONE_URL=https://github.com/ava-labs/avalanchego.git
GIT_CLONE_PATH=${BASEDIR}/avalanchego-repo/

# check to see if the repo already exists, if not clone it
if [[ ! -d ${GIT_CLONE_PATH} ]]; then
echo "cloning ${GIT_CLONE_URL} to ${GIT_CLONE_PATH}"
git clone --no-checkout ${GIT_CLONE_URL} ${GIT_CLONE_PATH}
fi

# check to see if the commitish exists in the repo
WORKDIR=$(pwd)

cd ${GIT_CLONE_PATH}

git fetch
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we shouldn't fetch or clone the entire repository to get a specific branch or commit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "fetch or clone the entire repository"? To my knowledge, you can't partially fetch a repository (unless you have git submodules or you're using LFS or something like that).

From the git docs:

When no remote is specified, by default the origin remote will be used, unless there’s an upstream branch configured for the current branch.

Regardless, fetch will grab everything we need here, then we always checkout a detached-head by prefixing branch names with origin/<branch_name> or a commit directly.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, however it also fetches other branches that are not needed for this run.
I think this is fine for now, in the longer term I think we should use github actions to cache or create artifacts


echo "checking out ${AVALANCHEGO_VERSION}"

set +e
# try to checkout the branch
git checkout origin/${AVALANCHEGO_VERSION} > /dev/null 2>&1
CHECKOUT_STATUS=$?
set -e

# if it's not a branch, try to checkout the commit
if [[ $CHECKOUT_STATUS -ne 0 ]]; then
set +e
git checkout ${AVALANCHEGO_VERSION} > /dev/null 2>&1
CHECKOUT_STATUS=$?
set -e

if [[ $CHECKOUT_STATUS -ne 0 ]]; then
echo
echo "'${VERSION}' is not a valid release tag, commit hash, or branch name"
exit 1
fi
fi

COMMIT=$(git rev-parse HEAD)

# use the commit hash instead of the branch name or tag
BUILD_DIR=${AVALANCHEGO_BUILD_PATH}-${COMMIT}
Comment on lines +102 to +103
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm at first I thought that the below code where it builds iff the build dir does not exist may run into issues for branches if there were newly pushed changes, but I see that this ensures that the build dir is based off of the commit, so that's not an issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm brilliant, what can I say, 😉

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤝


# if the build-directory doesn't exist, build avalanchego
if [[ ! -d ${BUILD_DIR} ]]; then
echo "building avalanchego ${COMMIT} to ${BUILD_DIR}"
./scripts/build.sh
mkdir -p ${BUILD_DIR}

mv ${GIT_CLONE_PATH}/build/* ${BUILD_DIR}/
fi

cd $WORKDIR
fi
fi

AVALANCHEGO_PATH=${AVALANCHEGO_BUILD_PATH}/avalanchego
AVALANCHEGO_PLUGIN_DIR=${AVALANCHEGO_BUILD_PATH}/plugins

echo "Installed AvalancheGo release ${VERSION}"
mkdir -p ${AVALANCHEGO_BUILD_PATH}

cp ${BUILD_DIR}/avalanchego ${AVALANCHEGO_PATH}


echo "Installed AvalancheGo release ${AVALANCHEGO_VERSION}"
echo "AvalancheGo Path: ${AVALANCHEGO_PATH}"
echo "Plugin Dir: ${AVALANCHEGO_PLUGIN_DIR}"
3 changes: 2 additions & 1 deletion scripts/versions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
# Set up the versions to be used - populate ENV variables only if they are not already populated
SUBNET_EVM_VERSION=${SUBNET_EVM_VERSION:-'v0.4.12'}
# Don't export them as they're used in the context of other calls
AVALANCHEGO_VERSION=${AVALANCHE_VERSION:-'v1.9.11'}
AVALANCHE_VERSION=${AVALANCHE_VERSION:-'v1.9.11'}
AVALANCHEGO_VERSION=${AVALANCHEGO_VERSION:-$AVALANCHE_VERSION}
Comment on lines +6 to +7
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
AVALANCHE_VERSION=${AVALANCHE_VERSION:-'v1.9.11'}
AVALANCHEGO_VERSION=${AVALANCHEGO_VERSION:-$AVALANCHE_VERSION}
AVALANCHEGO_VERSION=${AVALANCHEGO_VERSION:-'v1'.9.11'}

These are the same thing, can we make them one variable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was a little worried about AVALANCHE_VERSION being used in other scripts though. My change makes this backwards compatible, so you can change the other scripts one at a time and make sure you don't break anything. I think that would be be better as a follow up PR so the context is a little more clear.

So how about I do that immediately after this is merged?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me

GINKGO_VERSION=${GINKGO_VERSION:-'v2.2.0'}

# This won't be used, but it's here to make code syncs easier
Expand Down