[HOLD for payment 2023-12-07] [HOLD for payment 2023-10-13] [HOLD for payment 2023-07-14] Onyx migration for personalDetails changes

[puneetlath]

Part of "Secure Logins in NewDot" Project

• Main tracking issue
• Re-keying tracking issue

As part of this project, we started sending new data to Onyx that was accountID-based (e.g. personalDetailsList, participantAccountIDs, etc). We did not stop sending the old data, but we will be soon.

As a cleanup item, we should add an Onyx migration for two reasons:

1. To remove stale data from clients (e.g. to remove the personalDetails object)
2. To fix stale local data that will cause the app to break (e.g. old reportActions that don't have participantAccountIDs)

We should have this ready to go-live after https://github.com/Expensify/Expensify/issues/294647 is done.

This migration will be handled in stages:

Before updating the back-end:

• Migrate reportActions that don't have accountID-based data to include it (without removing email-based data)
  • Note: reportActions are the only pieces of data that can be stale in Onyx and might not be overwritten by the API) (Report actions Onyx migration for accountIDs #21509)

Once back-end has been updated to stop sending email-based data:

• Update the reportActions migration to clear out any email-based data Update PersonalDetails migration of reportActions to remove email-based data #22432
• Update migration to remove email-based data from reports
  • [HOLD E/E#294647] Update PersonalDetails migration of reports to remove email-based data #22811
  • Update migration for report.participants and report.lastActorEmail #28555
  • [No QA] Remove managerEmail and ownerEmail from Onyx #31664
• Update migration to remove the old policyMembersList Remove policyMemberList as part of personalDetails migration #21694
• Update migration to remove the old personalDetails Remove personalDetails as part of personalDetails migration #21844
• Update migration not to run when it isn't needed

[puneetlath]

@s77rt if you want, you can C+ all of them. There will probably be at least 3 or 4 PRs.

[s77rt]

@puneetlath That would be great

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.35-5 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Remove policyMemberList as part of personalDetails migration #21694

If no regressions arise, payment will be issued on 2023-07-10. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.37-7 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Remove personalDetails as part of personalDetails migration #21844

If no regressions arise, payment will be issued on 2023-07-14. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[Beamanator]

Planning to help with Update migration to remove email-based data from reports once we have all of the emails out of reports 👍

[s77rt]

@Beamanator We have PR for that already #22432

[puneetlath]

@Beamanator is working on the back-end that should unblock the last couple pieces. Hopefully we're very close to the finish line here 🤞🏾

[puneetlath]

Getting closer!

[puneetlath]

Next (last?) PR is here: #31664

[puneetlath]

I believe we're officially done with this now that the last PR was merged 🎉

@Beamanator @s77rt do you agree that we're done? Or are we still missing something?

[Beamanator]

I thinkkkk yes we can finally close this one out! (if payment is complete)

[s77rt]

• Update migration not to run when it isn't needed

What about this one?

[puneetlath]

Hmm, I can't remember what I meant by that. Is there a way we can know that the migration doesn't need to run and skip it that y'all can think of?

[Beamanator]

haha fair point - i don't think we'd have any way of knowing that it already ran unless we set some kind of NVP, but we'd have to have it run for every device where you had logged in to your account - so I don't think we need to do that now, we may need to if we start running too many migrations that end up taking a long time or something?

[s77rt]

Agree ^ I think we are good to close for now

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.4.5-7 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• [No QA] Remove managerEmail and ownerEmail from Onyx #31664

If no regressions arise, payment will be issued on 2023-12-07. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

For reference, here are some details about the assignees on this issue:

• @s77rt requires payment

[puneetlath]

Ok, I count a total of 7 PRs which would be $3500. Does that sound right?

[s77rt]

• Report actions Onyx migration for accountIDs #21509
• Remove policyMemberList as part of personalDetails migration #21694
• Remove personalDetails as part of personalDetails migration #21844
• Update PersonalDetails migration of reportActions to remove email-based data #22432
• Update migration for report.participants and report.lastActorEmail #28555
• [No QA] Remove managerEmail and ownerEmail from Onyx #31664
• [HOLD E/E#294647] Update PersonalDetails migration of reports to remove email-based data #22811 <-- This does not count since it's a dupe / closed

I found 6 PRs. All are pretty minimal, it should be $250 each. Total: $1500

[puneetlath]

Ok that sounds good. Thank you for being so honest and straightforward. Offer here: https://www.upwork.com/nx/wm/offer/27918877

[s77rt]

Accepted! Thanks!

[puneetlath]

Alright paid. Closing this out!