Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Peeking over federation via MSC2444 #1391

Merged
merged 92 commits into from
Jan 22, 2021
Merged

Peeking over federation via MSC2444 #1391

merged 92 commits into from
Jan 22, 2021

Conversation

ara4n
Copy link
Member

@ara4n ara4n commented Sep 4, 2020

First wave of peeking over federation a la MSC2444 to see how it feels.

I think this is now good to be merged as an initial wave of work, to stop it bitrotting, even though all the housekeeping is missing.

  • make outbound /peek over SS API
  • track outbound peeks in the federationsender db
  • handle inbound /peek requests over SS API
  • track inbound peeks in the federationsender db
  • make it build
  • make it work
  • support postgres
  • validate the state received by /peek correctly
  • fix the race between /peek and /send (which is causing the tests to be flakey)
  • don't re-peek over federation if we're already peeking
  • handle initial history visibility
  • handle history visibility if it transitions mid-peek
  • handle inbound peek renewals
  • expire stale inbound peeks
  • handle the transition from peeking into a remote room to joining a remote room nicely
  • housekeeping to renew outbound peeks automatically if there is a client syncing them
  • remove remote peeks if all the local clients /unpeek
  • decide if/how to unify the 4 near-identical {in,out}bound-peeks-tables
  • sytest - [WIP] Testing peeking over federation via MSC2444 sytest#953

Implements matrix-org/matrix-spec-proposals#2444
Builds on #1370
Requires matrix-org/gomatrixserverlib#220

ara4n and others added 30 commits August 30, 2020 17:46
doesn't yet compile or work.
needs to actually add the peeking block into the sync response.
checking in now before it gets any bigger, and to gather any initial feedback on the vague shape of it.
To use: set `DENDRITE_TRACE_SQL=1` then grep for `unsafe`
@kegsay kegsay self-assigned this Oct 30, 2020
return resErr
}

if !response.RoomExists {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would've failed earlier as QueryRoomVersionForRoom would return a failure.

}

type PerformOutboundPeekResponse struct {
LastError *gomatrix.HTTPError
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we be getting gomatrixserverlib.RespPeek somewhere?


// PerformInboundPeek handles peeking into matrix rooms, including over
// federation by talking to the federationsender. called when a remote server
// initiates a /peek over federation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels bad that we don't call PerformInboundPeek when local clients call /peek like we do with all the other Perform stuff.

// for now we just use the room ID again. In future, if we ever
// support concurrent peeks to the same room with different filters
// then we would need to disambiguate further.
peekID := roomID
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Annoyingly the MSC forbids this because:

should be a string consisting of the characters [0-9a-zA-Z.=_-]. Its length must not exceed 8 characters and it should not be empty.

RoomID string
ServerName gomatrixserverlib.ServerName
CreationTimestamp int64
RenewedTimestamp int64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As N points out, it would be nicer for this to be gomatrixserverlib.Timestamp types (which are just ints)

// possible candidate for finding the room via federation. Add
// it to the list of servers to try.
// handle federated peeks
// FIXME: don't create an outbound peek if we already have one going.
if domain != r.Cfg.Matrix.ServerName {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work in cases where you create the room, other servers join then you leave the room then try to peek into it, as this will resolve to false. We shouldn't ever be switching on the domain of the room ID like this.


For peeking into other server's rooms ("outbound peeks"):
* The `roomserver` will kick the `federationsender` much as it does for a federated `/join` in order to trigger a federated outbound `/peek`
* The `federationsender` tracks the existence of the outbound peek in in its federationsender_outbound_peeks table.
* The `federationsender` regularly renews the remote peek as long as there are still peeking devices syncing for it.
* TBD: how do we tell if there are no devices currently syncing for a given peeked room? The syncserver needs to tell the roomserver
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we care? We time out the peek anyway, and really just because there are no devices currently syncing doesn't mean we should stop peeking. Network interruptions are a reasonable reason, we should still be honouring the renewal period.

@kegsay kegsay merged commit 0571d39 into master Jan 22, 2021
@kegsay kegsay deleted the matthew/peeking-over-fed branch January 22, 2021 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants