[HOLD for payment 2024-08-07] [$500] Onyx updates are not being queued in the correct order

[blimpich]

Problem

When batching update operations (e.g. when coming online), Onyx executes the merge and mergeCollection operations in an incorrect order which leads to data inconsistency. See this comment for context.

Why do we care?

This has caused multiple issues: this, this, and is currently holding this.

What's expected?

The updates should happen in the same exact order as they've been enqueued to guarantee the eventual consistency of the data.

Note: this will probably involve working with the react-native-onyx repository

Issue reported by: @paultsimura
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1709229991663269?thread_ts=1705484273.717249&cid=C01GTK53T8Q

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01fec4289917eb6d71
• Upwork Job ID: 1763289951731056640
• Last Price Increase: 2024-02-29
• Automatic offers:
• jjcoffee | Reviewer | 0
• paultsimura | Contributor | 0

Issue Owner

Current Issue Owner: @Christinadobrzyn

[melvin-bot]

Triggered auto assignment to @Christinadobrzyn (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~01fec4289917eb6d71

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @jjcoffee (External)

[melvin-bot]

📣 @jjcoffee 🎉 An offer has been automatically sent to your Upwork account for the Reviewer role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job

[melvin-bot]

📣 @paultsimura 🎉 An offer has been automatically sent to your Upwork account for the Contributor role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job
Please accept the offer and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻
Keep in mind: Code of Conduct | Contributing 📖

[blimpich]

@jjcoffee I assigned @paultsimura without proposal based off this conversation. @paultsimura is going to work on this with help from @tgolen.

[paultsimura]

Hey @tgolen, I'd like to discuss potential ways of fixing this (actually, only one of them worked for me).

This simple change fixes the issue:

- return clearPromise.then(() => Promise.all(_.map(promises, (p) => p())));
+ return clearPromise.then(() => _.reduce(promises, (p, fn) => p.then(fn), Promise.resolve()));

https://github.com/Expensify/react-native-onyx/blob/8cca556b08bf858c17a7d4f5ddd2879ac3f456d4/lib/Onyx.js#L1518

However, it makes all the batched changes execute in sequence, not in parallel, and I guess it's not something we would like, right?

Screen.Recording.2024-03-01.at.00.08.10-compressed.mp4

[melvin-bot]

@tgolen, @paultsimura, @jjcoffee, @Christinadobrzyn Whoops! This issue is 2 days overdue. Let's get this updated quick!

[paultsimura]

Melvin, I'm doing my best with the PR 🙂

[Christinadobrzyn]

PR in the works!

[paultsimura]

@tgolen should I open the PR, or are there some kind of regular centralized Onyx bumps?

[tgolen]

I think you need to open a PR. There are no regular bumps for that. @mountiny might know if there are any currently open PRs that bump the version, but I am not aware of any.

[mountiny]

We have just merged a PR from @hannojg but I dont think he started a bump PR yet. So we should make sure we will test for all the merged PRs in onyx compared to the one used in App now

[paultsimura]

Ok @tgolen @mountiny - I will open a bump PR today. Will we look for C+ volunteer like usual for bump PRs, or should @jjcoffee review since he's the C+ for this issue?

[tgolen]

I don't think we need to have a C+ if it's just a version bump.

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 9.0.14-6 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Bump Onyx to 2.0.57 #46297

If no regressions arise, payment will be issued on 2024-08-07. 🎊

For reference, here are some details about the assignees on this issue:

• @paultsimura requires payment automatic offer (Contributor)
• @jjcoffee requires payment automatic offer (Reviewer)
• @paultsimura requires payment automatic offer (Contributor)

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@paultsimura] The PR that introduced the bug has been identified. Link to the PR:
• [@paultsimura] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@paultsimura] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@paultsimura] Determine if we should create a regression test for this bug.
• [@paultsimura] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@Christinadobrzyn] Link the GH issue for creating/updating the regression test once above steps have been agreed upon: https://github.com/Expensify/Expensify/issues/418518

[Christinadobrzyn]

Payouts due:

• Contributor: $500 @paultsimura (Paid in Upwork - https://www.upwork.com/nx/wm/offer/101149714)

Do we need a regression test @paultsimura?

[paultsimura]

Sorry @Christinadobrzyn, this one is $500, not $250 as mentioned in the payment summary.

[paultsimura]

I was not a C+ here, but would suggest the regression test:

1. Submit a distance request with 3 waypoints
2. Navigate to the distance request details (transaction thread)
3. Disable the internet connection
4. Click on the distance field to edit
5. Click on a waypoint
6. Click on the 3 dot menu > Delete waypoint
7. Hit Save
8. Switch chat
9. Go back online
10. Go back to the transaction thread
11. Click "distance" again
12. Verify the deleted waypoint is not present (i.e. it's "indeed" deleted)

[jjcoffee]

@Christinadobrzyn No payment due to me as I never reviewed anything in the end 🙏

[Christinadobrzyn]

Thanks @paultsimura and @jjcoffee - are we paying someone else? @sobitneupane I see you on the PR, were you the C+ for this?

[paultsimura]

Technically, there was no C+ here, only me and @tgolen mostly.

[Christinadobrzyn]

Ah okay! Thanks @paultsimura!

I paid this out based on this payment summary. The regression test is created so I think this is good to close. Let me know if I'm missing anything!