Investigate workflow job failing on main: e2ePerformanceTests / Run E2E tests in AWS device farm

[github-actions]

🚨 Failure Summary 🚨:

• 📋 Job Name: e2ePerformanceTests / Run E2E tests in AWS device farm
• 🔧 Failure in Workflow: Process new code merged to main
• 🔗 Triggered by PR: PR Link
• 👤 PR Author: @marcaaron
• 🤝 Merged by: @rafecolton
• 🐛 Error Message:
  warning: The following actions uses Node.js version which is deprecated and will be forced to run on node20: realm/aws-devicefarm@7b9a912. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
  failure: An HTTP protocol error occurred: statusCode = 404
  warning: No matching artifact of type LOG found for debug.log, skipping

⚠️ Action Required ⚠️:

🛠️ A recent merge appears to have caused a failure in the job named e2ePerformanceTests / Run E2E tests in AWS device farm.
This issue has been automatically created and labeled with Workflow Failure for investigation.

👀 Please look into the following:

1. Why the PR caused the job to fail?
2. Address any underlying issues.

🐛 We appreciate your help in squashing this bug!

[rafecolton]

@marcaaron can you look at this one with me? I'm not really sure what to do here. The changes in performance it's showing don't look at all related to your change

We also failed to notify in Slack, I think because the room title changed from #newdot-performance to #newdot-quality. For that one, maybe we should change to the room ID if that's possible?

I'm not sure if this should actually be a deploy blocker, so I'm going to change this to Daily for now. LMK!

[marcaaron]

I'm not too sure on either account. Agree that the failure is unrelated - it's a performance test so there are probably some circumstances that can cause false positives. Looks like the PR did not get deployed to staging though so we can try to re-run the job.

[marcaaron]

Tried running them again but they failed. I would guess it's some problem with AWS or that something else caused a regression (other people are reporting the same thing)

[rafecolton]

So we can probably close this and remove DeployBlockerCash from the PR? Does having that label on the PR even do anything?

[marcaaron]

That label is applied to issues. QA applies it while doing staging regression and PR testing.

Not sure I've seen it applied to a PR before. But perhaps someone chose to add this as a preventative measure e.g. after the PR moves to staging we could possibly deploy without resolving the problem and we want to avoid that?

Tbh I'm not well versed in the e2e tests or these "Investigate workflow" situations so your guess is as good as mine.

[rafecolton]

The GH action is what added it to the PR. @roryabraham do you know if this was intentional? It seems like a bug. Regardless, it seems we can close this issue.