Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

misc: fix gcp-collection scripts #14625

Merged
merged 2 commits into from
Jan 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions core/scripts/gcp-collection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

By default, the instance will be called `lighthouse-collection-$gcloudusername-instance0` but you can replace `instance0` by supplying an arg to `run.sh`.

1. Googlers: [set up ssh to gcloud](go/lighthouse-ssh-gcloud-readme).
1. Googlers: [set up ssh to gcloud](go/common-fw-policy-flows).
1. Spin up machines and start the collection.
1. (Prerequiste) Install GCloud SDK and authenticate locally, Run `brew cask install google-cloud-sdk && gcloud auth login`
1. (Prerequiste Googlers only) Ensure you're on the corp VPN or you won't be able to SSH into a Google-owned instance.
1. (Prerequiste) Install GCloud SDK and authenticate locally, Run `brew install google-cloud-sdk && gcloud auth login`
1. (Prerequiste Googlers only) Ensure you're on the corp VPN or you won't be able to SSH into a Google-owned instance (run `gcert`).
1. (Optional) Run `export TARGET_GIT_REF=<a lighthouse git ref that has been pushed> ` if wanting to run on anything but main.
1. (Optional) Run `export TARGET_RUNS=<a number> ` if needing more than 1 run per URL.
1. (Optional) Run `export LIGHTHOUSE_FLAGS=...` to configure Lighthouse CLI.
Expand Down
2 changes: 2 additions & 0 deletions core/scripts/gcp-collection/fleet-create.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ node fleet-create-directories.js $@

cd "$LH_ROOT/.tmp/gcp-instances"

trap "trap - SIGTERM && kill -- -$$" SIGINT SIGTERM EXIT

for instance in $(ls)
do
cd "$instance"
Expand Down
1 change: 1 addition & 0 deletions core/scripts/gcp-collection/gcp-run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ cd ./lighthouse
git checkout -f "$LIGHTHOUSE_GIT_REF"

sudo yarn --frozen-lockfile
sudo yarn build-report
sudo yarn link

cd /home/lighthouse
Expand Down
2 changes: 1 addition & 1 deletion core/scripts/gcp-collection/gcp-setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" | sud
wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | sudo apt-key add -

# Node apt-key
curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -

# Install dependencies
sudo apt-get update
Expand Down
6 changes: 3 additions & 3 deletions core/scripts/gcp-collection/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ INSTANCE_NAME="lighthouse-collection-$GCLOUD_USER-$INSTANCE_SUFFIX"
CLOUDSDK_CORE_PROJECT=${LIGHTHOUSE_COLLECTION_GCLOUD_PROJECT:-lighthouse-lantern-collect}
LIGHTHOUSE_GIT_REF=${TARGET_GIT_REF:-main}
NUMBER_OF_RUNS=${TARGET_RUNS:-1}
BASE_LIGHTHOUSE_FLAGS="--max-wait-for-load=90000 $LIGHTHOUSE_FLAGS"
BASE_LIGHTHOUSE_FLAGS="--max-wait-for-load=90000 ${LIGHTHOUSE_FLAGS:-}"
ZONE=us-central1-a

gcloud --project="$CLOUDSDK_CORE_PROJECT" compute instances create $INSTANCE_NAME \
Expand All @@ -36,7 +36,7 @@ EOF
# Instance needs time to start up.
until gcloud --project="$CLOUDSDK_CORE_PROJECT" compute scp ./.tmp_env $INSTANCE_NAME:/tmp/lhenv --zone="$ZONE" --scp-flag="-o ConnectTimeout=5"
do
echo "Waiting for start up ..."
echo "Waiting for start up (scp connection errors are expected for ~30s) ..."
sleep 10
done
rm .tmp_env
Expand Down Expand Up @@ -65,7 +65,7 @@ echo " $ bash .tmp/gcp/$INSTANCE_NAME-copy-traces.sh"
echo " For LHR data for smaller transfer sizes replication"
echo " $ bash .tmp/gcp/$INSTANCE_NAME-copy-lhrs.sh"
echo " To delete the instance"
echo " $ bash .tmp/gcp/$INSTANCE_NAME-delete-instance.sh"
echo " $ bash .tmp/gcp/$INSTANCE_NAME-delete.sh"

cd $LH_ROOT
mkdir -p .tmp/gcp/
Expand Down
2 changes: 2 additions & 0 deletions core/test/devtools-tests/download-devtools.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ set -euxo pipefail
# Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
##

BUILD_FOLDER="${BUILD_FOLDER:-LighthouseIntegration}"

if [ -d "$DEVTOOLS_PATH" ]
then
echo "Directory $DEVTOOLS_PATH already exists."
Expand Down