fix: gracefully handle missing test cases between two apps #40693
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Details
Sometimes we may add new test case or rename a test. In this case the e2e pipeline will be broken, because we'll try to run not existing tests on baseline app version (which leads to timeout error now, because test is not getting executed and after 300s app gets killed by our test runner).
In this PR I fixed this problem by gracefully handling the case - when we receive non critical error (currently this is the only one error indicating that test is not found) we'll call
forceTestCompletion
on a server side to clearly indicate that test finished its execution and we can go further.After these change in
compareResults
we'll have next test data:Delta VS Main (report)
Fixed Issues
$ #40544
PROPOSAL: N/A
Tests
Offline tests
N/A
QA Steps
QA can skip testing this PR because it's e2e stuff.
PR Author Checklist
### Fixed Issues
section aboveTests
sectionOffline steps
sectionQA steps
sectiontoggleReport
and notonIconClick
)myBool && <MyComponent />
.src/languages/*
files and using the translation methodSTYLE.md
) were followedAvatar
, I verified the components usingAvatar
are working as expected)StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)
)Avatar
is modified, I verified thatAvatar
is working as expected in all cases)Design
label and/or tagged@Expensify/design
so the design team can review the changes.ScrollView
component to make it scrollable when more elements are added to the page.main
branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTest
steps.Screenshots/Videos
Android: Native
Android: mWeb Chrome
E2E tests are available only on Android.
iOS: Native
E2E tests are available only on Android.
iOS: mWeb Safari
E2E tests are available only on Android.
MacOS: Chrome / Safari
E2E tests are available only on Android.
MacOS: Desktop
E2E tests are available only on Android.