Improve device user account creation during MDM IdP enrollment flow

[gillespi314]

Issue #19185

TODO (follow up in separate issue)

• Handle case where host_emails is deleted when host is deleted in UI then fleetd reenrolls. We're covering this somewhat for macOS hosts via DirectIngestMDMMac where we're upserting host_emails from mdm_idp_accounts, but we don't have anything similar for iOS. Consider adding a cron or hooking into the MDM Checkin for iOS hosts.
• Handle case where MDM IdP is disabled and a device that previously enrolled to MDM via IdP reenrolls to MDM without IdP (e.g., device is wiped or profiles renew). We currently never delete from mdm_idp_accounts and the IdP email from the prior enrollment will get re-associated to the device UUID.
• Look out for other edge cases around deletions and re-enrollments.
• Expand test coverage

Checklist for submitter

If some of the following don't apply, delete the relevant line.

• Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
  See Changes files for more information.
• Input data is properly validated, SELECT * is avoided, SQL injection is prevented (using placeholders for values in statements)
• Added/updated tests
• If database migrations are included, checked table schema to confirm autoupdate
• For database migrations:
  • Checked schema for all modified table for columns that will auto-update timestamps during migration.
  • Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects.
  • Ensured the correct collation is explicitly set for character columns (COLLATE utf8mb4_unicode_ci).
• Manual QA for all new/changed functionality

[roperzh]

looking amazing, Ieft a few questions. I was imagining we could get rid of the enroll reference. The main purpose of that was to create a "link" between a device visiting the webview, and an enrollment.

because we can use device info for that now (🎉 ) we could get rid of it. Feel free to hit me up if you want to chat

[roperzh]

LGTM, left some very minor notes

[mna]

Backend looks good to me, left just some minor comments and regarding the dependency, I agree that we should extract just what we need, there's very little added value for us in this package AFAICT, it's a think wrapper to parse from the request's header, but the rest of the logic uses the pkcs7 and plist repos that we already depend on (and the logic in verifyPKCS7SHA1RSA which may be the more involved part but otherwise just depends on Go's crypto).

[mna]

I'm not super familiar with the DEP/IdP flow, but backend code LGTM!

[codecov]

Codecov Report

Attention: Patch coverage is 33.98058% with 204 lines in your changes missing coverage. Please review.

Project coverage is 55.14%. Comparing base (d54ac3a) to head (2f4b6f9).
Report is 8 commits behind head on main.

Files	Patch %	Lines
server/mdm/apple/deviceinfo.go	4.54%	62 Missing and 1 partial ⚠️
server/datastore/mysql/hosts.go	32.07%	33 Missing and 3 partials ⚠️
server/service/apple_mdm.go	9.09%	28 Missing and 2 partials ⚠️
server/service/handler.go	0.00%	29 Missing ⚠️
server/mock/datastore_mock.go	25.80%	23 Missing ⚠️
...ables/20240709175341_AddColumnsToMdmIdpAccounts.go	71.42%	6 Missing and 2 partials ⚠️
server/worker/apple_mdm.go	66.66%	5 Missing and 2 partials ⚠️
server/datastore/mysql/apple_mdm.go	85.29%	4 Missing and 1 partial ⚠️
cmd/fleet/serve.go	0.00%	2 Missing ⚠️
server/service/osquery_utils/queries.go	91.66%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #20162      +/-   ##
==========================================
- Coverage   55.27%   55.14%   -0.14%     
==========================================
  Files        1416     1420       +4     
  Lines      133123   133764     +641     
  Branches     3201     3200       -1     
==========================================
+ Hits        73589    73759     +170     
- Misses      53543    53992     +449     
- Partials     5991     6013      +22     

Flag	Coverage Δ
backend	55.25% <33.98%> (-0.16%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.