Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sdk): Mark tracked users as dirty when the SS connection is reset. #3965

Merged

Conversation

Hywan
Copy link
Member

@Hywan Hywan commented Sep 9, 2024

Closes #3959.

There is a non-negligible difference MSC3575 and MSC4186 in how the
e2ee extension works. When the client sends a request with no pos:

  • MSC3575 returns all device lists updates since the last request
    from the device that asked for device lists (this works similarly to
    to-device message handling),

  • MSC4186 returns no device lists updates, as it only returns changes
    since the provided pos (which is null in this case); this is in
    line with sync v2.

Therefore, with MSC4186, the device list cache must be marked as to be
re-downloaded if the since token is None, otherwise it's easy to
miss device lists updates that happened between the previous request and
the new “initial” request.

This patch first off implements OlmMachine::mark_all_tracked_users_as_dirty.
Next, this patch uses this method inside sliding sync when pos is None and
e2ee extension is enabled.

@Hywan Hywan requested review from a team as code owners September 9, 2024 13:14
@Hywan Hywan requested review from jmartinesp, andybalaam and poljar and removed request for a team, jmartinesp and andybalaam September 9, 2024 13:14
Copy link

codecov bot commented Sep 9, 2024

Codecov Report

Attention: Patch coverage is 97.67442% with 1 line in your changes missing coverage. Please review.

Project coverage is 84.31%. Comparing base (cb82586) to head (943c46f).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
crates/matrix-sdk-crypto/src/machine/mod.rs 96.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3965      +/-   ##
==========================================
+ Coverage   84.27%   84.31%   +0.03%     
==========================================
  Files         267      267              
  Lines       28296    28323      +27     
==========================================
+ Hits        23846    23880      +34     
+ Misses       4450     4443       -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@poljar poljar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some nits, I think it generally does what it should do.

crates/matrix-sdk-crypto/src/machine/mod.rs Show resolved Hide resolved
crates/matrix-sdk-crypto/src/machine/mod.rs Outdated Show resolved Hide resolved
let account = vodozemac::olm::Account::new();

// Put some tracked users
let damir = user_id!("@damir:localhost");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

crates/matrix-sdk/src/sliding_sync/error.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/src/sliding_sync/mod.rs Outdated Show resolved Hide resolved
@kegsay
Copy link
Member

kegsay commented Sep 10, 2024

It's worth emphasising here that this will negatively impact performance and bandwidth, as the rust SDK will be doing many more /keys/query requests now because the client itself resets the E2EE connection more frequently due to not remembering ?pos=. The SDK needs to remember the ?pos= in the same way it remembers to_device.since to not be hit with this performance penalty.

@Hywan Hywan force-pushed the fix-sdk-sliding-sync-no-since-must-reset-devices branch from dc004b4 to 62cf64c Compare September 10, 2024 13:07
This patch adds the `OldMachine::mark_all_tracked_users_as_dirty`.

This patch rewrites a bit `OlmMachine::new_helper` by extracting some
piece of it inside `OlmMachine::new_helper_prelude`. With that, we
can rewrite `OlmMachine::migration_post_verified_latch_support` to use
`IdentityManager::mark_all_tracked_users_as_dirty`.
This latter is the shared implementation with
`OlmMachine::mark_all_tracked_users_as_dirty`.

This patch adds a test for `OlmMachine:mark_all_tracked_users_as_dirty`.
@Hywan Hywan force-pushed the fix-sdk-sliding-sync-no-since-must-reset-devices branch from 62cf64c to c0bb278 Compare September 10, 2024 13:18
@Hywan Hywan requested a review from poljar September 10, 2024 13:19
There is a non-negligible difference MSC3575 and MSC4186 in how the
`e2ee` extension works. When the client sends a request with no `pos`:

* MSC3575 returns all device lists updates since the last request
  from the device that asked for device lists (this works similarly to
  to-device message handling),

* MSC4186 returns no device lists updates, as it only returns changes
  since the provided `pos` (which is `null` in this case); this is in
  line with sync v2.

Therefore, with MSC4186, the device list cache must be marked as to be
re-downloaded if the `since` token is `None`, otherwise it's easy to
miss device lists updates that happened between the previous request and
the new “initial” request.
@Hywan Hywan force-pushed the fix-sdk-sliding-sync-no-since-must-reset-devices branch from c0bb278 to 781d3d0 Compare September 10, 2024 13:36
Copy link
Contributor

@poljar poljar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still some nits left, but they are trivial so gonna approve ahead of time.

Thanks for taking the time to write a test for this.

crates/matrix-sdk/src/sliding_sync/mod.rs Show resolved Hide resolved
crates/matrix-sdk-crypto/src/machine/mod.rs Outdated Show resolved Hide resolved
@Hywan Hywan enabled auto-merge (rebase) September 11, 2024 07:45
@Hywan Hywan merged commit 2576042 into matrix-org:main Sep 11, 2024
40 checks passed
kegsay added a commit to matrix-org/matrix-js-sdk that referenced this pull request Sep 18, 2024
See matrix-org/matrix-rust-sdk#3965 for
more information. Requires `Extension.onRequest` to be `async`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sliding sync: Clear the device list cache when the connection is reset.
3 participants