From 23af87e9fc898daffd3823490578f9c46442b85e Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 20 Jun 2019 17:41:19 +0100 Subject: [PATCH 01/62] Proposal for IS & IM TOS API --- proposals/2140-terms-of-service-2.md | 203 +++++++++++++++++++++++++++ 1 file changed, 203 insertions(+) create mode 100644 proposals/2140-terms-of-service-2.md diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md new file mode 100644 index 0000000000..8c2ee75be4 --- /dev/null +++ b/proposals/2140-terms-of-service-2.md @@ -0,0 +1,203 @@ +# MSCXXXX: Terms of Service API for Identity Servers and Integration Managers + +MSC1692 introduces a method for homeservers to require that users read and +agree to certain documents before being permitted to use the service. This +proposal introduces a corresponding method that can be used with Identity +Servers and Integration Managers. + +The challenge here is that Identity Servers do not require any kind of user +login to access the service and so are unable to track what users have agreed +to what terms in the way that Homeservers do. We thereforce cannot re-use the +same method for Identity Servers without fundamentally changing the Identity +Service API. + +Requirements for this proposal are: + * ISs and IMs should be able to give multiple documents a user must agree to + abide by + * Each document shoud be versioned + * ISs and IMs must be able to prevent users from using the service if they + have not provided agreement. + * A user should only have to agree to each version of each document once for + their Matrix ID, ie. having agreed to a set of terms in one client, they + should not have to agree to them again when using a different client. + +## Proposal + +Throuhgout this proposal, $prefix will be used to refer to the prefix of the +API in question, ie. `/_matrix/identity/api/v1` for the IS API and +`/_matrix/integrations/v1` for the IM API. + +This proposal introduces: + * The `$prefix/terms` endpoint + * The `m.third_party_terms` section in account data + * The `X-TERMS-TOKEN` HTTP header + +### Terms API + +New API endpoints will be introduced: + +#### `GET $prefix/terms`: +This returns a set of documents that the user must agree to abide by in order +to use the service. Its response is similar to the structure used in the +`m.terms` UI auth flow of the Client/Server API: + +```json +{ + "policies": { + "terms_of_service": { + "version": "2.0", + "en": { + "name": "Terms of Service", + "url": "https://example.org/somewhere/terms-2.0-en.html" + }, + "fr": { + "name": "Conditions d'utilisation", + "url": "https://example.org/somewhere/terms-2.0-fr.html" + } + } + } +} +``` + +Each document (ie. key/value pair in the 'policies' object) MUST be +uniquely identified by its URL. It is therefore strongly recommended +that the URL contains the version number of the document. The name +and version keys, however, are used only to provide a human-readable +description of the document to the user. + +In the IM API, the client should provide authentication for this endpoint. + +#### `POST $prefix/terms`: +Requests to this endpoint have a single key, `user_accepts` whose value is +a list of URLs (given by the `url` field in the GET response) of documents that +the user has agreed to: + +```json +{ + "user_accepts": ["https://example.org/somewhere/terms-2.0-en.html"] +} +``` + +In the IM API, the client should provide authentication for this endpoint. + +The clients MUST include the correct URL for the language of the document that +was presented to the user and they agreed to. How servers store or serialise +acceptance into the `acceptance_token` is not defined, eg. they may internally +transform all URLs to the URL of the English-language version of each document +if the server deems it appropriate to do so. Servers should accept agreement of +any one language of each document as sufficient, regardless of what language a +client is operating in: users should not have to re-consent to documents if +they change their client to a different language. + +The response MAY contain a `acceptance_token` which, if given, is an +opaque string that the client must store for user in subsequent requests +to any endpoint to the same server. + +If the server has stored the fact that the user has agreed to these terms, +(which implies the user is authenticated) it can supply no `acceptance_token`. +The server may instead choose to supply an `acceptance_token`, for example if, +as in the IS API, the user is unauthenticated and therefore the server is +unable to store the fact a user has agreed to a set of terms. + +The `acceptance_token` is opaque and it is up to the server how it computes it, +but the server must be able to given an `acceptance_token`, compute whether it +constitutes agreement to a given set of terms. For example, the simplest (but +most verbose) implemenation would be to make the `acceptance_token` the JSON +array of documents as provided in the request. A smarter implementation may be +a simple hash, or even cryptograhic hash if desired. + +### Third-Party Terms Account Data + +This proposal also defines the `m.third_party_terms` section in User Account +Data in the client/server API that clients SHOULD use to track what sets of +terms the user has consented to. This has an array of URLs under the 'accepted' +key to which the user has agreed to. + +An `m.third_party_terms` section therefore resembles the following: + +```json +{ + "accepted": [ + "https://example.org/somewhere/terms-1.2-en.html", + "https://example.org/somewhere/privacy-1.2-en.html" + ] +} +``` + +Whenever a client submits a `POST $prefix/terms` request to an IS or IM, it +SHOULD update this account data section adding any the URLs of any additional +documents that the user agreed to to this list. + +### Terms Acceptance in the API + +Any request to any endpoint in the IS and IM APIs, with the exception of +`/_matrix/identity/api/v1` may return a `M_TERMS_NOT_SIGNED` errcode. This +indicates that the user must agree to (new) terms in order to use or continue +to use the service. + +The client uses the `GET $prefix/terms` endpoint to get the latest set of terms +that must be agreed to. It then cross-references this set of documents against +the `m.third_party_terms` account data and presents to the user any documents +that they have not already agreed to, along with UI for them to indicate their +agreement. Once the user has indicated their agreement, then, and only then, +must the client use the `POST $prefix/terms` API to signal to the server the +set of documents that the user has agreed to. + +If the server returns an `acceptance_token`, the client should include this +token in the `X-TERMS-TOKEN` HTTP header in all subsequent requests to an +endpoint on the API with the exception of `/_matrix/identity/api/v1`. + +Both making the `POST $prefix/terms` request and providing an `X-TERMS-TOKEN` +header signal that the user consents to the terms contained within the +corresponding documents. That is to say, if a client or user obtains an +acceptance token via means other than a response to the `POST $perfix/terms` +API, inclusion of the acceptance token in an `X-TERMS-TOKEN` header in a +request still constitutes agreement to the terms in the corresponding +documents. + +## Tradeoffs + +This introduces a different way of accepting terms from the client/server API +which uses User-Interactive Authentication. In the client/server API, the use +of UI auth allows terms acceptance to be integrated into the registration flow +in a simple and backwards-compatible way. Indtroducing the UI Auth mechanism +into these other APIs would add significant complexity, so this functionality +has been provided with simpler, dedicated endpoints. + +## Potential issues + +If the server does not authentcate users, some mechanism is required to track +users agreement to terms. The introduction of an extra HTTP header on all +requests adds overhead to every request and complexity to the client to add a +custom header. + + +## Security considerations + +The `acceptance_token` is, in effect, a cookie and could be used to identify +users of the service. Users of the Integration manager must be authenticated +anyway, so this is irrelevant for the IM API. It could allow an Identity Server +to identify users where it may otherwise not be able to do so (if a client was +careful to mask other identifying HTTP headers). Given most requests to the IS +API, by their nature, include 3pids which, even if hashed, will make users +easily identifiable, this probably does not add any significant concern. + +It is assumed that once servers publish a given version of a document at a +given URL, the contents of that URL will not change. This could be mitigated by +identifying documents based on a hash of their contents rather than their URLs. +Agreement to terms in the client/server API makes this assumption, so this +proposal aims to be consistent. + + +## Conclusion + +This proposal adds an error response to all endpoints on the API and a custom +HTTP header on all requests that is used to signal agreement to a set of terms +and conditions. The use of the header is only necessary if the server has no +other means of tracking acceptance of terms per-user. The IS API is not +authenticated so ISes will have no choice but to use the header. The IM API is +authenticated so IMs may either use the header or store acceptance per-user. + +A separate endpoint is specified with a GET request for retrieving the set +of terms required and a POST to indicate that the user consents to those +terms. From 32c7fc638dfb31eb05f484fc0c933cef7b604dc5 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 20 Jun 2019 17:44:28 +0100 Subject: [PATCH 02/62] you have a number now --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 8c2ee75be4..77d44e60e0 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -1,4 +1,4 @@ -# MSCXXXX: Terms of Service API for Identity Servers and Integration Managers +# MSC2140: Terms of Service API for Identity Servers and Integration Managers MSC1692 introduces a method for homeservers to require that users read and agree to certain documents before being permitted to use the service. This From cf48030d1fe7c3e180cd5a5617261fa508dbad59 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 20 Jun 2019 17:48:21 +0100 Subject: [PATCH 03/62] One more tradeoff --- proposals/2140-terms-of-service-2.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 77d44e60e0..bf3226767d 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -164,6 +164,14 @@ in a simple and backwards-compatible way. Indtroducing the UI Auth mechanism into these other APIs would add significant complexity, so this functionality has been provided with simpler, dedicated endpoints. +The `m.third_party_terms` section contains only URLs of the documents that +have been agreed to. This loses information like the name and version of +the document, but: + * It would be up to the clients to copy this information correctly into + account data. + * Having just the URLs makes it much easier for clients to make a list + of URLs and find documents not already agreed to. + ## Potential issues If the server does not authentcate users, some mechanism is required to track From 276e2b6843b9d27e7782222cf66d2c34f6c528f9 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 21 Jun 2019 09:14:24 +0100 Subject: [PATCH 04/62] Typo Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index bf3226767d..b678881a91 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -90,7 +90,7 @@ client is operating in: users should not have to re-consent to documents if they change their client to a different language. The response MAY contain a `acceptance_token` which, if given, is an -opaque string that the client must store for user in subsequent requests +opaque string that the client must store for use in subsequent requests to any endpoint to the same server. If the server has stored the fact that the user has agreed to these terms, From d4ca0c237a7b744ffc05003b3a84b31f0b064367 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 21 Jun 2019 09:25:16 +0100 Subject: [PATCH 05/62] Specify ID grammar and add comma --- proposals/2140-terms-of-service-2.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index b678881a91..dbdad33432 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -99,12 +99,13 @@ The server may instead choose to supply an `acceptance_token`, for example if, as in the IS API, the user is unauthenticated and therefore the server is unable to store the fact a user has agreed to a set of terms. -The `acceptance_token` is opaque and it is up to the server how it computes it, -but the server must be able to given an `acceptance_token`, compute whether it -constitutes agreement to a given set of terms. For example, the simplest (but -most verbose) implemenation would be to make the `acceptance_token` the JSON -array of documents as provided in the request. A smarter implementation may be -a simple hash, or even cryptograhic hash if desired. +The `acceptance_token` is an opaque string contining characters +`[a-zA-Z0-9._-]`. It is up to the server how it computes it, but the server +must be able to, given an `acceptance_token`, compute whether it constitutes +agreement to a given set of terms. For example, the simplest (but most verbose) +implemenation would be to make the `acceptance_token` the JSON array of +documents as provided in the request. A smarter implementation may be a simple +hash, or even cryptograhic hash if desired. ### Third-Party Terms Account Data From 9ca3ccc81ce0ddfcf529b6c9445a2d11887707c4 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 21 Jun 2019 09:35:26 +0100 Subject: [PATCH 06/62] Add requirments section for de-duping between services. --- proposals/2140-terms-of-service-2.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index dbdad33432..066a1184af 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -20,6 +20,11 @@ Requirements for this proposal are: * A user should only have to agree to each version of each document once for their Matrix ID, ie. having agreed to a set of terms in one client, they should not have to agree to them again when using a different client. + * Documents should be de-duplicated between services. If two or more services + are hosted by the same organisation, the organistation should have the + option to give their users a single document that encompasses both services + (bearing in mind that the user must be able to opt-out of components of a + service whilst still being able to use the service without that component). ## Proposal From a63e4420eb9030d4afb5ac036c9617e5247e466e Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 24 Jun 2019 13:17:58 +0100 Subject: [PATCH 07/62] Linkify Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 066a1184af..66efa70810 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -1,6 +1,6 @@ # MSC2140: Terms of Service API for Identity Servers and Integration Managers -MSC1692 introduces a method for homeservers to require that users read and +[MSC1692](https://github.com/matrix-org/matrix-doc/issues/1692) introduces a method for homeservers to require that users read and agree to certain documents before being permitted to use the service. This proposal introduces a corresponding method that can be used with Identity Servers and Integration Managers. From 4ba9b2a59910ac14d69f92be34e8b123011ea6d5 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 24 Jun 2019 13:18:41 +0100 Subject: [PATCH 08/62] perfix --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 66efa70810..b43cadf2af 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -156,7 +156,7 @@ endpoint on the API with the exception of `/_matrix/identity/api/v1`. Both making the `POST $prefix/terms` request and providing an `X-TERMS-TOKEN` header signal that the user consents to the terms contained within the corresponding documents. That is to say, if a client or user obtains an -acceptance token via means other than a response to the `POST $perfix/terms` +acceptance token via means other than a response to the `POST $prefix/terms` API, inclusion of the acceptance token in an `X-TERMS-TOKEN` header in a request still constitutes agreement to the terms in the corresponding documents. From 25558014580b411249ae3699d798dc9408997978 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 24 Jun 2019 14:56:48 +0100 Subject: [PATCH 09/62] m.third_party_terms -> m.accepted_terms as it will have the HS's terms too --- proposals/2140-terms-of-service-2.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index b43cadf2af..8851eda394 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -34,7 +34,7 @@ API in question, ie. `/_matrix/identity/api/v1` for the IS API and This proposal introduces: * The `$prefix/terms` endpoint - * The `m.third_party_terms` section in account data + * The `m.accepted_terms` section in account data * The `X-TERMS-TOKEN` HTTP header ### Terms API @@ -114,12 +114,12 @@ hash, or even cryptograhic hash if desired. ### Third-Party Terms Account Data -This proposal also defines the `m.third_party_terms` section in User Account +This proposal also defines the `m.accepted_terms` section in User Account Data in the client/server API that clients SHOULD use to track what sets of terms the user has consented to. This has an array of URLs under the 'accepted' key to which the user has agreed to. -An `m.third_party_terms` section therefore resembles the following: +An `m.accepted_terms` section therefore resembles the following: ```json { @@ -143,7 +143,7 @@ to use the service. The client uses the `GET $prefix/terms` endpoint to get the latest set of terms that must be agreed to. It then cross-references this set of documents against -the `m.third_party_terms` account data and presents to the user any documents +the `m.accepted_terms` account data and presents to the user any documents that they have not already agreed to, along with UI for them to indicate their agreement. Once the user has indicated their agreement, then, and only then, must the client use the `POST $prefix/terms` API to signal to the server the @@ -170,7 +170,7 @@ in a simple and backwards-compatible way. Indtroducing the UI Auth mechanism into these other APIs would add significant complexity, so this functionality has been provided with simpler, dedicated endpoints. -The `m.third_party_terms` section contains only URLs of the documents that +The `m.accepted_terms` section contains only URLs of the documents that have been agreed to. This loses information like the name and version of the document, but: * It would be up to the clients to copy this information correctly into From 8ae47557c9a9e82e2333894be04583b7d968ca47 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 24 Jun 2019 15:22:19 +0100 Subject: [PATCH 10/62] s/Third Party/Accepted/ --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 8851eda394..f4d78a9d41 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -112,7 +112,7 @@ implemenation would be to make the `acceptance_token` the JSON array of documents as provided in the request. A smarter implementation may be a simple hash, or even cryptograhic hash if desired. -### Third-Party Terms Account Data +### Accepted Terms Account Data This proposal also defines the `m.accepted_terms` section in User Account Data in the client/server API that clients SHOULD use to track what sets of From abb407145abe1a51f1ba4d4b7c07e000039791e3 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 24 Jun 2019 15:30:19 +0100 Subject: [PATCH 11/62] HS docs must be added too also, unbind must not error when called by HSes and proxy terms token --- proposals/2140-terms-of-service-2.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index f4d78a9d41..6c5bb6a5e4 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -130,16 +130,19 @@ An `m.accepted_terms` section therefore resembles the following: } ``` -Whenever a client submits a `POST $prefix/terms` request to an IS or IM, it -SHOULD update this account data section adding any the URLs of any additional -documents that the user agreed to to this list. +Whenever a client submits a `POST $prefix/terms` request to an IS or IM or +completes an `m.terms` flow on the HS, it SHOULD update this account data +section adding any the URLs of any additional documents that the user agreed to +to this list. ### Terms Acceptance in the API Any request to any endpoint in the IS and IM APIs, with the exception of `/_matrix/identity/api/v1` may return a `M_TERMS_NOT_SIGNED` errcode. This indicates that the user must agree to (new) terms in order to use or continue -to use the service. +to use the service. The `_matrix/identity/api/v1/3pid/unbind` must also not +return the `M_TERMS_NOT_SIGNED` if the request has a valid signature from a +Homeserver. The client uses the `GET $prefix/terms` endpoint to get the latest set of terms that must be agreed to. It then cross-references this set of documents against @@ -153,6 +156,11 @@ If the server returns an `acceptance_token`, the client should include this token in the `X-TERMS-TOKEN` HTTP header in all subsequent requests to an endpoint on the API with the exception of `/_matrix/identity/api/v1`. +The client must also include the X-TERMS-TOKEN on any request to the Homeserver +where it specifies an Identity Server to be used by the Homeserver. Homeservers +must read this header from the request headers of any such endpoint and add it +to the request headers of any request it makes to the Identity Server. + Both making the `POST $prefix/terms` request and providing an `X-TERMS-TOKEN` header signal that the user consents to the terms contained within the corresponding documents. That is to say, if a client or user obtains an From 2c09580e2798f0cd51590174040df6bde2b996a4 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 25 Jun 2019 11:14:35 +0100 Subject: [PATCH 12/62] line wrap --- proposals/2140-terms-of-service-2.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 6c5bb6a5e4..42a1ea386e 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -1,9 +1,10 @@ # MSC2140: Terms of Service API for Identity Servers and Integration Managers -[MSC1692](https://github.com/matrix-org/matrix-doc/issues/1692) introduces a method for homeservers to require that users read and -agree to certain documents before being permitted to use the service. This -proposal introduces a corresponding method that can be used with Identity -Servers and Integration Managers. +[MSC1692](https://github.com/matrix-org/matrix-doc/issues/1692) introduces a +method for homeservers to require that users read and agree to certain +documents before being permitted to use the service. This proposal introduces a +corresponding method that can be used with Identity Servers and Integration +Managers. The challenge here is that Identity Servers do not require any kind of user login to access the service and so are unable to track what users have agreed From 6f374dc981b6330d59cf5f45022f0176ce19cf16 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 25 Jun 2019 14:58:15 +0100 Subject: [PATCH 13/62] Re-write for OpenID auth --- proposals/2140-terms-of-service-2.md | 143 +++++++++++++++------------ 1 file changed, 80 insertions(+), 63 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 42a1ea386e..7b6dd4dc45 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -6,16 +6,15 @@ documents before being permitted to use the service. This proposal introduces a corresponding method that can be used with Identity Servers and Integration Managers. -The challenge here is that Identity Servers do not require any kind of user -login to access the service and so are unable to track what users have agreed -to what terms in the way that Homeservers do. We thereforce cannot re-use the -same method for Identity Servers without fundamentally changing the Identity -Service API. - Requirements for this proposal are: * ISs and IMs should be able to give multiple documents a user must agree to abide by * Each document shoud be versioned + * ISes and IMs must, for each request that they handle, know that the user + making the request has agreed to their data being used. This need not be + absolute proof (we will always have to trust that the client actually + showed the document to the user) but it must be reasonably demonstrable that + the user has given informed consent for the client to use that service. * ISs and IMs must be able to prevent users from using the service if they have not provided agreement. * A user should only have to agree to each version of each document once for @@ -27,16 +26,61 @@ Requirements for this proposal are: (bearing in mind that the user must be able to opt-out of components of a service whilst still being able to use the service without that component). +Identity Servers do not currently require any kind of user login to access the +service and so are unable to track what users have agreed to what terms in the +way that Homeservers do. + ## Proposal Throuhgout this proposal, $prefix will be used to refer to the prefix of the -API in question, ie. `/_matrix/identity/api/v1` for the IS API and +API in question, ie. `/_matrix/identity/v2` for the IS API and `/_matrix/integrations/v1` for the IM API. +Note the removal of the `/api` prefix and migration to v2 in the IS API +following convention from +[MSC2134](https://github.com/matrix-org/matrix-doc/issues/2134). + This proposal introduces: * The `$prefix/terms` endpoint * The `m.accepted_terms` section in account data - * The `X-TERMS-TOKEN` HTTP header + +This proposal relies on both Integration Managers and Identity Servers being +able to identity users by their mxid and store the fact that a given mxid has +indicated that they accept the terms given. Integration Managers already +identity users in this way by authenticating them using the OpenID endpoint on +the Homeserver. This proposal introduces the same mechanism to Identity Servers +and adds authentication to accross the Identity Service API. + +### IS API Authentication + +All current endpoints within `/_matrix/identity/api/v1/` will be duplicated +into `/_matrix/identity/v2`. + +Any request to any endpoint within `/_matrix/identity/v2`, with the exception of +`/_matrix/identity/v2` and the new `/_matrix/identity/v2/account/register` may +return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. This +indicates that the user must authenticate with OpenID and supply a valid +`access_token`. + +The existing endpoints under `/_matrix/identity/api/v1/` continue to be unauthenticated. +ISes may support the old v1 API for as long as they wish. Clients must update to use +the v2 API as soon as possible. + +OpenID authentication in the IS API will work the same as in the Integration Manager +API, as specified in [MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961). + +### IS Register API + +The following new APIs will be introduced to support OpenID auth as per +[MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961): + + * `/_matrix/identity/v2/account/register` + * `/_matrix/identity/v2/account` + * `/_matrix/identity/v2/account/logout` + +Note again the removal of the `/api` prefix and migration to v2 following +convention from +[MSC2134](https://github.com/matrix-org/matrix-doc/issues/2134). ### Terms API @@ -71,7 +115,7 @@ that the URL contains the version number of the document. The name and version keys, however, are used only to provide a human-readable description of the document to the user. -In the IM API, the client should provide authentication for this endpoint. +The client should provide authentication for this endpoint. #### `POST $prefix/terms`: Requests to this endpoint have a single key, `user_accepts` whose value is @@ -84,7 +128,7 @@ the user has agreed to: } ``` -In the IM API, the client should provide authentication for this endpoint. +The client should provide authentication for this endpoint. The clients MUST include the correct URL for the language of the document that was presented to the user and they agreed to. How servers store or serialise @@ -95,24 +139,6 @@ any one language of each document as sufficient, regardless of what language a client is operating in: users should not have to re-consent to documents if they change their client to a different language. -The response MAY contain a `acceptance_token` which, if given, is an -opaque string that the client must store for use in subsequent requests -to any endpoint to the same server. - -If the server has stored the fact that the user has agreed to these terms, -(which implies the user is authenticated) it can supply no `acceptance_token`. -The server may instead choose to supply an `acceptance_token`, for example if, -as in the IS API, the user is unauthenticated and therefore the server is -unable to store the fact a user has agreed to a set of terms. - -The `acceptance_token` is an opaque string contining characters -`[a-zA-Z0-9._-]`. It is up to the server how it computes it, but the server -must be able to, given an `acceptance_token`, compute whether it constitutes -agreement to a given set of terms. For example, the simplest (but most verbose) -implemenation would be to make the `acceptance_token` the JSON array of -documents as provided in the request. A smarter implementation may be a simple -hash, or even cryptograhic hash if desired. - ### Accepted Terms Account Data This proposal also defines the `m.accepted_terms` section in User Account @@ -138,12 +164,17 @@ to this list. ### Terms Acceptance in the API -Any request to any endpoint in the IS and IM APIs, with the exception of -`/_matrix/identity/api/v1` may return a `M_TERMS_NOT_SIGNED` errcode. This -indicates that the user must agree to (new) terms in order to use or continue -to use the service. The `_matrix/identity/api/v1/3pid/unbind` must also not -return the `M_TERMS_NOT_SIGNED` if the request has a valid signature from a -Homeserver. +Any request to any endpoint in the IM API, and the `_matrix/identity/v2/` +namespace of the IS API, with the exception of `/_matrix/identity/v2` itself, +may return: + + * `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that + the user must authenticate with OpenID and supply a valid `access_token`. + * `M_TERMS_NOT_SIGNED` errcode. This indicates that the user must agree to + (new) terms in order to use or continue to use the service. + +The `_matrix/identity/v2/3pid/unbind` must not return either of these +errors if the request has a valid signature from a Homeserver. The client uses the `GET $prefix/terms` endpoint to get the latest set of terms that must be agreed to. It then cross-references this set of documents against @@ -153,25 +184,17 @@ agreement. Once the user has indicated their agreement, then, and only then, must the client use the `POST $prefix/terms` API to signal to the server the set of documents that the user has agreed to. -If the server returns an `acceptance_token`, the client should include this -token in the `X-TERMS-TOKEN` HTTP header in all subsequent requests to an -endpoint on the API with the exception of `/_matrix/identity/api/v1`. - -The client must also include the X-TERMS-TOKEN on any request to the Homeserver -where it specifies an Identity Server to be used by the Homeserver. Homeservers -must read this header from the request headers of any such endpoint and add it -to the request headers of any request it makes to the Identity Server. - -Both making the `POST $prefix/terms` request and providing an `X-TERMS-TOKEN` -header signal that the user consents to the terms contained within the -corresponding documents. That is to say, if a client or user obtains an -acceptance token via means other than a response to the `POST $prefix/terms` -API, inclusion of the acceptance token in an `X-TERMS-TOKEN` header in a -request still constitutes agreement to the terms in the corresponding -documents. - ## Tradeoffs +The Identity Service API previously did not require authentication, and OpenID +is reasonably complex, adding a significant burden to both clients and servers. +A custom HTTP Header was also considered that could be added to assert that the +client agrees to a particular set of terms. We decided against this in favour +of re-using existing primitives that already exist in the Matrix ecosystem. +Custom HTTP Headers are not used anywhere else within Matrix. This also gives a +very simple and natural way for ISes to enforce that users may only bind 3pids +to their own mxids. + This introduces a different way of accepting terms from the client/server API which uses User-Interactive Authentication. In the client/server API, the use of UI auth allows terms acceptance to be integrated into the registration flow @@ -189,21 +212,15 @@ the document, but: ## Potential issues -If the server does not authentcate users, some mechanism is required to track -users agreement to terms. The introduction of an extra HTTP header on all -requests adds overhead to every request and complexity to the client to add a -custom header. - +This change is not backwards compatible: clients implementing older versions of +the specification will expect to be able to access all IS API endpoints without +authentication. Care should be taken to manage the rollout of authentication +on IS APIs. ## Security considerations -The `acceptance_token` is, in effect, a cookie and could be used to identify -users of the service. Users of the Integration manager must be authenticated -anyway, so this is irrelevant for the IM API. It could allow an Identity Server -to identify users where it may otherwise not be able to do so (if a client was -careful to mask other identifying HTTP headers). Given most requests to the IS -API, by their nature, include 3pids which, even if hashed, will make users -easily identifiable, this probably does not add any significant concern. +Requiring authentication on the IS API means it will no longer be possible to +use it anonymously. It is assumed that once servers publish a given version of a document at a given URL, the contents of that URL will not change. This could be mitigated by From 0dae2d5812d0d22350c9237b84634e81e0306b24 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 25 Jun 2019 15:52:55 +0100 Subject: [PATCH 14/62] GET terms must be unauthed. Detail process for new auth (don't register until consent given). Specifically mention the authentication header. --- proposals/2140-terms-of-service-2.md | 55 +++++++++++++++++++++------- 1 file changed, 41 insertions(+), 14 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 7b6dd4dc45..67e6a2eb03 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -56,11 +56,15 @@ and adds authentication to accross the Identity Service API. All current endpoints within `/_matrix/identity/api/v1/` will be duplicated into `/_matrix/identity/v2`. -Any request to any endpoint within `/_matrix/identity/v2`, with the exception of -`/_matrix/identity/v2` and the new `/_matrix/identity/v2/account/register` may -return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. This -indicates that the user must authenticate with OpenID and supply a valid -`access_token`. +Any request to any endpoint within `/_matrix/identity/v2`, with the exception +of `/_matrix/identity/v2` and the new `/_matrix/identity/v2/account/register` +and `GET /_matrix/identity/v2/terms` may return an error with `M_UNAUTHORIZED` +errcode with HTTP status code 401. This indicates that the user must +authenticate with OpenID and supply a valid `access_token`. + +These endpoints require authentication by the client supplying an access token +either via an `Authorization` header with a `Bearer` token or an `access_token` +query parameter. The existing endpoints under `/_matrix/identity/api/v1/` continue to be unauthenticated. ISes may support the old v1 API for as long as they wish. Clients must update to use @@ -115,7 +119,7 @@ that the URL contains the version number of the document. The name and version keys, however, are used only to provide a human-readable description of the document to the user. -The client should provide authentication for this endpoint. +This endpoint does *not* require authentication. #### `POST $prefix/terms`: Requests to this endpoint have a single key, `user_accepts` whose value is @@ -128,7 +132,7 @@ the user has agreed to: } ``` -The client should provide authentication for this endpoint. +This endpoint requires authentication. The clients MUST include the correct URL for the language of the document that was presented to the user and they agreed to. How servers store or serialise @@ -164,6 +168,22 @@ to this list. ### Terms Acceptance in the API +Before any requests are made to an Identity Server or Integration Manager, +the client must use the `GET $prefix/terms` endpoint to fetch the set of +documents that the user must agree to in order to use the service. + +It then cross-references this set of documents against the `m.accepted_terms` +account data and presents to the user any documents that they have not already +agreed to, along with UI for them to indicate their agreement. Once the user +has indicated their agreement, it adds these URLs to `m.accepted_terms` account +data. Once this has succeeded, then, and only then, must the client perform +OpenID authentication, getting a token from the Homeserver and submitting this +to the service using the `register` endpoint. + +Having done this, if the user agreed to any new documents, it performs a `POST +$prefix/terms` request to signal to the server the set of documents that the +user has agreed to. + Any request to any endpoint in the IM API, and the `_matrix/identity/v2/` namespace of the IS API, with the exception of `/_matrix/identity/v2` itself, may return: @@ -176,13 +196,20 @@ may return: The `_matrix/identity/v2/3pid/unbind` must not return either of these errors if the request has a valid signature from a Homeserver. -The client uses the `GET $prefix/terms` endpoint to get the latest set of terms -that must be agreed to. It then cross-references this set of documents against -the `m.accepted_terms` account data and presents to the user any documents -that they have not already agreed to, along with UI for them to indicate their -agreement. Once the user has indicated their agreement, then, and only then, -must the client use the `POST $prefix/terms` API to signal to the server the -set of documents that the user has agreed to. +In summary, the process for using a service that has not previously been used +in the current login sessions is: + + * `GET $prefix/terms` + * Compare result with `m.accepted_terms` account data, get set of documents + pending agreement + * If non-empty, show this set of documents to the user and wait for the user + to indicate their agreement. + * Add the newly agreed documents to `m.accepted_terms` + * On success, or if there were no documents pending agreement, get an OpenID + token from the Homeserver and submit this token to the `register` endpoint. + Store the resulting access token. + * If the set of documents pending agreement was non-empty, Perform a + `POST $prefix/terms` request to the servcie with these documents. ## Tradeoffs From 9e0d8b9cb29807d2e16b2332dc59f7b0aaad1fb2 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 25 Jun 2019 17:17:39 +0100 Subject: [PATCH 15/62] Use M_CONSENT_NOT_GIVEN No idea where I got the other one from: we already have one in the spec, so use it. --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 67e6a2eb03..b0b15b071c 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -190,7 +190,7 @@ may return: * `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that the user must authenticate with OpenID and supply a valid `access_token`. - * `M_TERMS_NOT_SIGNED` errcode. This indicates that the user must agree to + * `M_CONSENT_NOT_GIVEN` errcode. This indicates that the user must agree to (new) terms in order to use or continue to use the service. The `_matrix/identity/v2/3pid/unbind` must not return either of these From 57094276ce01e324b174a74c40b63ea8afdc0ba8 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 14:36:15 +0100 Subject: [PATCH 16/62] Typing hard is Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index b0b15b071c..a483607b44 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -21,7 +21,7 @@ Requirements for this proposal are: their Matrix ID, ie. having agreed to a set of terms in one client, they should not have to agree to them again when using a different client. * Documents should be de-duplicated between services. If two or more services - are hosted by the same organisation, the organistation should have the + are hosted by the same organisation, the organisation should have the option to give their users a single document that encompasses both services (bearing in mind that the user must be able to opt-out of components of a service whilst still being able to use the service without that component). From af691b5a8ad09ea82cfc49f596eae90448aa4725 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 14:37:06 +0100 Subject: [PATCH 17/62] Clarify this applies to 2134 Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index a483607b44..b508a9e2a8 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -54,7 +54,7 @@ and adds authentication to accross the Identity Service API. ### IS API Authentication All current endpoints within `/_matrix/identity/api/v1/` will be duplicated -into `/_matrix/identity/v2`. +into `/_matrix/identity/v2`, noting that MSC2134 changes the behaviour of lookups. Authentication is still expected on MSC2134's proposed endpoints. Any request to any endpoint within `/_matrix/identity/v2`, with the exception of `/_matrix/identity/v2` and the new `/_matrix/identity/v2/account/register` From 1d75828c71bb442ceed735c4ee6593171a08319c Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 14:45:25 +0100 Subject: [PATCH 18/62] Clarify what to do if no (new) docs --- proposals/2140-terms-of-service-2.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index b508a9e2a8..ad6175081f 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -174,11 +174,13 @@ documents that the user must agree to in order to use the service. It then cross-references this set of documents against the `m.accepted_terms` account data and presents to the user any documents that they have not already -agreed to, along with UI for them to indicate their agreement. Once the user -has indicated their agreement, it adds these URLs to `m.accepted_terms` account -data. Once this has succeeded, then, and only then, must the client perform -OpenID authentication, getting a token from the Homeserver and submitting this -to the service using the `register` endpoint. +agreed to, along with UI for them to indicate their agreement. If there are no +such documents (ie. if the `policies` dict is empty or the user has already +agreed to all documents) the client proceeds to perform the OpenID +registration. Once the user has indicated their agreement, it adds these URLs +to `m.accepted_terms` account data. Once this has succeeded, then, and only +then, must the client perform OpenID authentication, getting a token from the +Homeserver and submitting this to the service using the `register` endpoint. Having done this, if the user agreed to any new documents, it performs a `POST $prefix/terms` request to signal to the server the set of documents that the From ba7047ce7746733f77dbf4d6163767304690a2c7 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 14:51:11 +0100 Subject: [PATCH 19/62] Clarify we must be accepting HS auth Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index ad6175081f..631ff5e817 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -196,7 +196,7 @@ may return: (new) terms in order to use or continue to use the service. The `_matrix/identity/v2/3pid/unbind` must not return either of these -errors if the request has a valid signature from a Homeserver. +errors if the request has a valid signature from a Homeserver, and is being authenticated as such. In summary, the process for using a service that has not previously been used in the current login sessions is: From 4edf826c9371aad7c5c22fec7434a1889d66b74c Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 14:52:19 +0100 Subject: [PATCH 20/62] Capitalise on our identifiers --- proposals/2140-terms-of-service-2.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 631ff5e817..fc7d33b2a2 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -45,7 +45,7 @@ This proposal introduces: * The `m.accepted_terms` section in account data This proposal relies on both Integration Managers and Identity Servers being -able to identity users by their mxid and store the fact that a given mxid has +able to identity users by their MXID and store the fact that a given MXID has indicated that they accept the terms given. Integration Managers already identity users in this way by authenticating them using the OpenID endpoint on the Homeserver. This proposal introduces the same mechanism to Identity Servers @@ -221,8 +221,8 @@ A custom HTTP Header was also considered that could be added to assert that the client agrees to a particular set of terms. We decided against this in favour of re-using existing primitives that already exist in the Matrix ecosystem. Custom HTTP Headers are not used anywhere else within Matrix. This also gives a -very simple and natural way for ISes to enforce that users may only bind 3pids -to their own mxids. +very simple and natural way for ISes to enforce that users may only bind 3PIDs +to their own MXIDs. This introduces a different way of accepting terms from the client/server API which uses User-Interactive Authentication. In the client/server API, the use From 6273868323ab19ed42edaaa7d8d4ac5f738e718f Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 15:05:43 +0100 Subject: [PATCH 21/62] Clarify v1 API deprecation --- proposals/2140-terms-of-service-2.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index fc7d33b2a2..6963c96512 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -66,9 +66,11 @@ These endpoints require authentication by the client supplying an access token either via an `Authorization` header with a `Bearer` token or an `access_token` query parameter. -The existing endpoints under `/_matrix/identity/api/v1/` continue to be unauthenticated. -ISes may support the old v1 API for as long as they wish. Clients must update to use -the v2 API as soon as possible. +The existing endpoints under `/_matrix/identity/api/v1/` continue to be +unauthenticated but will be deprecated. ISes may support the old v1 API for as +long as they wish. Once ISes remove support for the old APIs, those endpoints +must return HTTP Status 404. Clients must update to use the v2 API as soon as +possible. OpenID authentication in the IS API will work the same as in the Integration Manager API, as specified in [MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961). From 58cf083a6a93e273385cad5a1621895ca15272f5 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 15:31:11 +0100 Subject: [PATCH 22/62] backwards compat --- proposals/2140-terms-of-service-2.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 6963c96512..7018e364fd 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -243,10 +243,8 @@ the document, but: ## Potential issues -This change is not backwards compatible: clients implementing older versions of -the specification will expect to be able to access all IS API endpoints without -authentication. Care should be taken to manage the rollout of authentication -on IS APIs. +This change deprecates all v1 endpoints and so will require clients to update +to continue working. ## Security considerations From 2694bb1090d565a10ef934a45ce01ae06a009804 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 17:41:21 +0100 Subject: [PATCH 23/62] Add really horrible custom HTTP header for giving the IS token to the HS --- proposals/2140-terms-of-service-2.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 7018e364fd..ed1269a918 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -75,6 +75,12 @@ possible. OpenID authentication in the IS API will work the same as in the Integration Manager API, as specified in [MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961). +When clients supply an identity server to the Homeserver in order for the Homeserver +to make calls to the IS on its behalf, it must also supply its access token for the +Identity Server either as the `is_token` query parameter or a bearer token in the +`X-Identity-Authorization` HTTP header with the same syntax as an `Authorizationn` +header. + ### IS Register API The following new APIs will be introduced to support OpenID auth as per From 21b9eaf8de06d2562c6326dedda66e334f03d6b0 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 Jun 2019 17:56:41 +0100 Subject: [PATCH 24/62] No custom HTTP headers Use the obvious way: in the same place as the ID server address --- proposals/2140-terms-of-service-2.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index ed1269a918..00a6ba5252 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -77,9 +77,13 @@ API, as specified in [MSC1961](https://github.com/matrix-org/matrix-doc/issues/1 When clients supply an identity server to the Homeserver in order for the Homeserver to make calls to the IS on its behalf, it must also supply its access token for the -Identity Server either as the `is_token` query parameter or a bearer token in the -`X-Identity-Authorization` HTTP header with the same syntax as an `Authorizationn` -header. +Identity Server alongside in the `is_token` key of the same JSON object. That is, +in the main request object for a `requestToken` request and in the `threepidCreds` +object when supplying 3PID credentials (eg. in the `m.email.identity` UI auth stage). +Exxceptions to this are any requests where the only IS operation the Homeserver may +perform is unbinding, ie. `/_matrix/client/r0/account/deactivate` and +`/_matrix/client/r0/account/3pid/delete`, in which case the unbind is authenticated +by a signed request from the Homeserver. ### IS Register API From b5326de1c4c9d894d668cc4e5ec031226b7ad442 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 27 Jun 2019 16:34:46 +0100 Subject: [PATCH 25/62] Exclude requestToken endpoints from auth requirement --- proposals/2140-terms-of-service-2.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 00a6ba5252..22bdb1f4ed 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -57,12 +57,20 @@ All current endpoints within `/_matrix/identity/api/v1/` will be duplicated into `/_matrix/identity/v2`, noting that MSC2134 changes the behaviour of lookups. Authentication is still expected on MSC2134's proposed endpoints. Any request to any endpoint within `/_matrix/identity/v2`, with the exception -of `/_matrix/identity/v2` and the new `/_matrix/identity/v2/account/register` -and `GET /_matrix/identity/v2/terms` may return an error with `M_UNAUTHORIZED` -errcode with HTTP status code 401. This indicates that the user must -authenticate with OpenID and supply a valid `access_token`. - -These endpoints require authentication by the client supplying an access token +of `/_matrix/identity/v2`, any `requestToken` endpoint and the new +`/_matrix/identity/v2/account/register` and `GET /_matrix/identity/v2/terms` +may return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. +This indicates that the user must authenticate with OpenID and supply a valid +`access_token`. + +`requestToken` endpoints are excluded from the auth check because they are used +in the registration process before the user has an MXID and therefore cannot +log in with OpenID. It is up to the IS to manage its privacy obligations +appropriately when fulfilling these requests, bearing in mind that the user has +not explictly incicated their agreement to any documents, and may abort the +registration process without doing so. + +All other endpoints require authentication by the client supplying an access token either via an `Authorization` header with a `Bearer` token or an `access_token` query parameter. From 10a6a59a12924243c2a321df111eec64d01f0234 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 27 Jun 2019 16:58:22 +0100 Subject: [PATCH 26/62] Deprecate `bind_email` / `bind_msisdn` --- proposals/2140-terms-of-service-2.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 22bdb1f4ed..74923e6415 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -93,6 +93,13 @@ perform is unbinding, ie. `/_matrix/client/r0/account/deactivate` and `/_matrix/client/r0/account/3pid/delete`, in which case the unbind is authenticated by a signed request from the Homeserver. +### HS Register API + +The `bind_email` and `bind_msisdn` options to `/_matrix/client/r0/register` in +the client/server API will be deprecated. Due to the fact that +`/_matrix/identity/v2/3pid/bind` requires authentication, it will no longer be +possible for the Homeserver to bind 3PIDs as part of the registration process. + ### IS Register API The following new APIs will be introduced to support OpenID auth as per From f95197b4228aec45111c75923bca8f499a38eff7 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 27 Jun 2019 17:30:36 +0100 Subject: [PATCH 27/62] make the many-anded sentence a list --- proposals/2140-terms-of-service-2.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 74923e6415..b06b38eb97 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -57,9 +57,13 @@ All current endpoints within `/_matrix/identity/api/v1/` will be duplicated into `/_matrix/identity/v2`, noting that MSC2134 changes the behaviour of lookups. Authentication is still expected on MSC2134's proposed endpoints. Any request to any endpoint within `/_matrix/identity/v2`, with the exception -of `/_matrix/identity/v2`, any `requestToken` endpoint and the new -`/_matrix/identity/v2/account/register` and `GET /_matrix/identity/v2/terms` -may return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. +of: + * `/_matrix/identity/v2` + * any `requestToken` endpoint + * The new `/_matrix/identity/v2/account/register` + * The new `GET /_matrix/identity/v2/terms` + +...may return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that the user must authenticate with OpenID and supply a valid `access_token`. From 4be283ccb3c25008e53f987c94c3749c40baea14 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 27 Jun 2019 17:31:15 +0100 Subject: [PATCH 28/62] Typing Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index b06b38eb97..cd6be68bff 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -71,7 +71,7 @@ This indicates that the user must authenticate with OpenID and supply a valid in the registration process before the user has an MXID and therefore cannot log in with OpenID. It is up to the IS to manage its privacy obligations appropriately when fulfilling these requests, bearing in mind that the user has -not explictly incicated their agreement to any documents, and may abort the +not explicitly indicated their agreement to any documents, and may abort the registration process without doing so. All other endpoints require authentication by the client supplying an access token From 83bb3861ba2926bd711930d1d0e89ff08087383e Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 28 Jun 2019 09:31:53 +0100 Subject: [PATCH 29/62] line wrap --- proposals/2140-terms-of-service-2.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index cd6be68bff..69696946ec 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -54,7 +54,8 @@ and adds authentication to accross the Identity Service API. ### IS API Authentication All current endpoints within `/_matrix/identity/api/v1/` will be duplicated -into `/_matrix/identity/v2`, noting that MSC2134 changes the behaviour of lookups. Authentication is still expected on MSC2134's proposed endpoints. +into `/_matrix/identity/v2`, noting that MSC2134 changes the behaviour of +lookups. Authentication is still expected on MSC2134's proposed endpoints. Any request to any endpoint within `/_matrix/identity/v2`, with the exception of: From 45d630951c6c81aaa7796c08e348e71ac6290f72 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 28 Jun 2019 09:32:15 +0100 Subject: [PATCH 30/62] back to M_TERMS_NOT_SIGNED --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 69696946ec..5bcb58a761 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -224,7 +224,7 @@ may return: * `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that the user must authenticate with OpenID and supply a valid `access_token`. - * `M_CONSENT_NOT_GIVEN` errcode. This indicates that the user must agree to + * `M_TERMS_NOT_SIGNED` errcode. This indicates that the user must agree to (new) terms in order to use or continue to use the service. The `_matrix/identity/v2/3pid/unbind` must not return either of these From 786d5bc281baf8ecbb79e4bf27ead5ab7adbde58 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 28 Jun 2019 10:25:24 +0100 Subject: [PATCH 31/62] rewrite UI auth tradeoffs --- proposals/2140-terms-of-service-2.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 5bcb58a761..f84c026c4e 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -259,9 +259,10 @@ to their own MXIDs. This introduces a different way of accepting terms from the client/server API which uses User-Interactive Authentication. In the client/server API, the use of UI auth allows terms acceptance to be integrated into the registration flow -in a simple and backwards-compatible way. Indtroducing the UI Auth mechanism -into these other APIs would add significant complexity, so this functionality -has been provided with simpler, dedicated endpoints. +in a simple and backwards-compatible way. Another option here would be to use +UI Auth on the register endpoint. This would also not allow users to register +before accepting the terms. However, this would then make the OpenID +registration process different and non-standard. The `m.accepted_terms` section contains only URLs of the documents that have been agreed to. This loses information like the name and version of From fe14d3c9f0928c037784f029f68af9fa7cd15757 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 28 Jun 2019 18:07:24 +0100 Subject: [PATCH 32/62] Spec terms response --- proposals/2140-terms-of-service-2.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index f84c026c4e..befa655d7d 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -175,6 +175,9 @@ any one language of each document as sufficient, regardless of what language a client is operating in: users should not have to re-consent to documents if they change their client to a different language. +The server responds with an empty JSON object. The server must not assume that +the client will agree to all documents in a single request. + ### Accepted Terms Account Data This proposal also defines the `m.accepted_terms` section in User Account From 8af35be13faff18010407f135d9b894fdded5768 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 10:56:12 +0100 Subject: [PATCH 33/62] Typo Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index befa655d7d..e47584bb08 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -47,7 +47,7 @@ This proposal introduces: This proposal relies on both Integration Managers and Identity Servers being able to identity users by their MXID and store the fact that a given MXID has indicated that they accept the terms given. Integration Managers already -identity users in this way by authenticating them using the OpenID endpoint on +identify users in this way by authenticating them using the OpenID endpoint on the Homeserver. This proposal introduces the same mechanism to Identity Servers and adds authentication to accross the Identity Service API. From 2d11217d4ed1282d4c80e97faffd544de2b34bcb Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 10:56:32 +0100 Subject: [PATCH 34/62] Typo Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index e47584bb08..2beeafbcb7 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -49,7 +49,7 @@ able to identity users by their MXID and store the fact that a given MXID has indicated that they accept the terms given. Integration Managers already identify users in this way by authenticating them using the OpenID endpoint on the Homeserver. This proposal introduces the same mechanism to Identity Servers -and adds authentication to accross the Identity Service API. +and adds authentication across the Identity Service API. ### IS API Authentication From 5374030cc0644dadb40bb53a7cde63cb58353376 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 10:59:08 +0100 Subject: [PATCH 35/62] Drop application/x-form-www-urlencoded in v2 --- proposals/2140-terms-of-service-2.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 2beeafbcb7..0d4a9f7c15 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -56,6 +56,8 @@ and adds authentication across the Identity Service API. All current endpoints within `/_matrix/identity/api/v1/` will be duplicated into `/_matrix/identity/v2`, noting that MSC2134 changes the behaviour of lookups. Authentication is still expected on MSC2134's proposed endpoints. +Support for `application/x-form-www-urlencoded` parameters in requests will +be dropped from all endpoints. Any request to any endpoint within `/_matrix/identity/v2`, with the exception of: From f02e4c2e9cae69d6cb2a7ffe0dc08edffde30a13 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:00:39 +0100 Subject: [PATCH 36/62] both registers are excluded from auth Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 0d4a9f7c15..e4a3083be2 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -63,7 +63,7 @@ Any request to any endpoint within `/_matrix/identity/v2`, with the exception of: * `/_matrix/identity/v2` * any `requestToken` endpoint - * The new `/_matrix/identity/v2/account/register` + * The new `$prefix/account/register` endpoint * The new `GET /_matrix/identity/v2/terms` ...may return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. From d00dfb78226b2015633aac4b612f8ea28b9fefdd Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:04:06 +0100 Subject: [PATCH 37/62] exclude submittoken too --- proposals/2140-terms-of-service-2.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index e4a3083be2..568a50df19 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -62,20 +62,21 @@ be dropped from all endpoints. Any request to any endpoint within `/_matrix/identity/v2`, with the exception of: * `/_matrix/identity/v2` - * any `requestToken` endpoint + * any `requestToken` or `submitToken` endpoint * The new `$prefix/account/register` endpoint * The new `GET /_matrix/identity/v2/terms` + * `$prefix/logout` ...may return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that the user must authenticate with OpenID and supply a valid `access_token`. -`requestToken` endpoints are excluded from the auth check because they are used -in the registration process before the user has an MXID and therefore cannot -log in with OpenID. It is up to the IS to manage its privacy obligations -appropriately when fulfilling these requests, bearing in mind that the user has -not explicitly indicated their agreement to any documents, and may abort the -registration process without doing so. +`requestToken` and `submitToken` endpoints are excluded from the auth check +because they are used in the registration process before the user has an MXID +and therefore cannot log in with OpenID. It is up to the IS to manage its +privacy obligations appropriately when fulfilling these requests, bearing in +mind that the user has not explicitly indicated their agreement to any +documents, and may abort the registration process without doing so. All other endpoints require authentication by the client supplying an access token either via an `Authorization` header with a `Bearer` token or an `access_token` From 03e6ab0103fad0cf80d31ef0d4003f577bd130d3 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:11:50 +0100 Subject: [PATCH 38/62] re-word double openid --- proposals/2140-terms-of-service-2.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 568a50df19..2e874ee8ce 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -215,10 +215,11 @@ account data and presents to the user any documents that they have not already agreed to, along with UI for them to indicate their agreement. If there are no such documents (ie. if the `policies` dict is empty or the user has already agreed to all documents) the client proceeds to perform the OpenID -registration. Once the user has indicated their agreement, it adds these URLs -to `m.accepted_terms` account data. Once this has succeeded, then, and only -then, must the client perform OpenID authentication, getting a token from the -Homeserver and submitting this to the service using the `register` endpoint. +registration. If there are new terms documents, the client prompts the user for +agreement, then once the user has indicated their agreement, it adds these URLs +to `m.accepted_terms` account data and then proceeds with OpenID +authentication, getting a token from the Homeserver and submitting this to the +service using the `register` endpoint. Having done this, if the user agreed to any new documents, it performs a `POST $prefix/terms` request to signal to the server the set of documents that the From 7f653648045fded42b8e2321de52ed996dc81da2 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:13:12 +0100 Subject: [PATCH 39/62] Typo Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 2e874ee8ce..f7b4337662 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -96,7 +96,7 @@ to make calls to the IS on its behalf, it must also supply its access token for Identity Server alongside in the `is_token` key of the same JSON object. That is, in the main request object for a `requestToken` request and in the `threepidCreds` object when supplying 3PID credentials (eg. in the `m.email.identity` UI auth stage). -Exxceptions to this are any requests where the only IS operation the Homeserver may +Exceptions to this are any requests where the only IS operation the Homeserver may perform is unbinding, ie. `/_matrix/client/r0/account/deactivate` and `/_matrix/client/r0/account/3pid/delete`, in which case the unbind is authenticated by a signed request from the Homeserver. From ac6b9bdb7ccf316ee77f65c3bd07edaed4a927f2 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:16:25 +0100 Subject: [PATCH 40/62] s/deprecate/remove/ --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index f7b4337662..152f28d36e 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -104,7 +104,7 @@ by a signed request from the Homeserver. ### HS Register API The `bind_email` and `bind_msisdn` options to `/_matrix/client/r0/register` in -the client/server API will be deprecated. Due to the fact that +the client/server API will be removed. Due to the fact that `/_matrix/identity/v2/3pid/bind` requires authentication, it will no longer be possible for the Homeserver to bind 3PIDs as part of the registration process. From 79dbad2914b1feeff57a908556fc95254c4de112 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:17:27 +0100 Subject: [PATCH 41/62] remove acceptance token mention --- proposals/2140-terms-of-service-2.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 152f28d36e..cb3736789a 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -170,12 +170,9 @@ the user has agreed to: This endpoint requires authentication. The clients MUST include the correct URL for the language of the document that -was presented to the user and they agreed to. How servers store or serialise -acceptance into the `acceptance_token` is not defined, eg. they may internally -transform all URLs to the URL of the English-language version of each document -if the server deems it appropriate to do so. Servers should accept agreement of -any one language of each document as sufficient, regardless of what language a -client is operating in: users should not have to re-consent to documents if +was presented to the user and they agreed to. Servers should accept agreement +of any one language of each document as sufficient, regardless of what language +a client is operating in: users should not have to re-consent to documents if they change their client to a different language. The server responds with an empty JSON object. The server must not assume that From 10858bf83b6c01cf8962d63d20f4db3c57e5fde6 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:22:41 +0100 Subject: [PATCH 42/62] set account data after registration --- proposals/2140-terms-of-service-2.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index cb3736789a..0ee4ae20b0 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -197,9 +197,10 @@ An `m.accepted_terms` section therefore resembles the following: ``` Whenever a client submits a `POST $prefix/terms` request to an IS or IM or -completes an `m.terms` flow on the HS, it SHOULD update this account data -section adding any the URLs of any additional documents that the user agreed to -to this list. +completes an `m.terms` flow on the HS (or as soon as possible afterwards, ie. +after registration is complete), it SHOULD update this account data section +adding any the URLs of any additional documents that the user agreed to to this +list. ### Terms Acceptance in the API From 4c72c37b80aefef94c2ab1bc30719351e2f88ece Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:24:51 +0100 Subject: [PATCH 43/62] slash Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 0ee4ae20b0..40278b37bf 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -223,7 +223,7 @@ Having done this, if the user agreed to any new documents, it performs a `POST $prefix/terms` request to signal to the server the set of documents that the user has agreed to. -Any request to any endpoint in the IM API, and the `_matrix/identity/v2/` +Any request to any endpoint in the IM API, and the `/_matrix/identity/v2/` namespace of the IS API, with the exception of `/_matrix/identity/v2` itself, may return: From e28f7aad7203fab769ea0c18638fdfd10275e6f3 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:25:12 +0100 Subject: [PATCH 44/62] slash Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 40278b37bf..e5fd21d65f 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -232,7 +232,7 @@ may return: * `M_TERMS_NOT_SIGNED` errcode. This indicates that the user must agree to (new) terms in order to use or continue to use the service. -The `_matrix/identity/v2/3pid/unbind` must not return either of these +The `/_matrix/identity/v2/3pid/unbind` endpoint must not return either of these errors if the request has a valid signature from a Homeserver, and is being authenticated as such. In summary, the process for using a service that has not previously been used From d15c9df115a09b02cee265063cc328072a4ddaf2 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:25:26 +0100 Subject: [PATCH 45/62] fullstop Co-Authored-By: Travis Ralston --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index e5fd21d65f..4a69ee035d 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -240,7 +240,7 @@ in the current login sessions is: * `GET $prefix/terms` * Compare result with `m.accepted_terms` account data, get set of documents - pending agreement + pending agreement. * If non-empty, show this set of documents to the user and wait for the user to indicate their agreement. * Add the newly agreed documents to `m.accepted_terms` From 1a669348d8de4e83675e4847abbc67595ac0863b Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 2 Jul 2019 11:27:46 +0100 Subject: [PATCH 46/62] http status code --- proposals/2140-terms-of-service-2.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 4a69ee035d..dbdb19f689 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -229,8 +229,9 @@ may return: * `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that the user must authenticate with OpenID and supply a valid `access_token`. - * `M_TERMS_NOT_SIGNED` errcode. This indicates that the user must agree to - (new) terms in order to use or continue to use the service. + * `M_TERMS_NOT_SIGNED` errcode with HTTP status code 403. This indicates + that the user must agree to (new) terms in order to use or continue to + use the service. The `/_matrix/identity/v2/3pid/unbind` endpoint must not return either of these errors if the request has a valid signature from a Homeserver, and is being authenticated as such. From 30dcc28f9b7280f88cc6898182ffa3891c1b18e7 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 4 Jul 2019 18:38:31 +0100 Subject: [PATCH 47/62] try & clarify that HS signature isn't the only acceptable auth for unbind --- proposals/2140-terms-of-service-2.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index dbdb19f689..2f6c88d8b7 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -91,15 +91,16 @@ possible. OpenID authentication in the IS API will work the same as in the Integration Manager API, as specified in [MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961). -When clients supply an identity server to the Homeserver in order for the Homeserver -to make calls to the IS on its behalf, it must also supply its access token for the -Identity Server alongside in the `is_token` key of the same JSON object. That is, -in the main request object for a `requestToken` request and in the `threepidCreds` -object when supplying 3PID credentials (eg. in the `m.email.identity` UI auth stage). -Exceptions to this are any requests where the only IS operation the Homeserver may -perform is unbinding, ie. `/_matrix/client/r0/account/deactivate` and -`/_matrix/client/r0/account/3pid/delete`, in which case the unbind is authenticated -by a signed request from the Homeserver. +When clients supply an identity server to the Homeserver in order for the +Homeserver to make calls to the IS on its behalf, it must also supply its +access token for the Identity Server alongside in the `is_token` key of the +same JSON object. That is, in the main request object for a `requestToken` +request and in the `threepidCreds` object when supplying 3PID credentials (eg. +in the `m.email.identity` UI auth stage). Exceptions to this are any requests +where the only IS operation the Homeserver may perform is unbinding, ie. +`/_matrix/client/r0/account/deactivate` and +`/_matrix/client/r0/account/3pid/delete`, in which case the unbind will be +authenticated by a signed request from the Homeserver. ### HS Register API From bf8a1e5d5f14855470ba607976ed4ae4f1c1898c Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 5 Jul 2019 18:11:42 +0100 Subject: [PATCH 48/62] Add way to get the HS to bind/unbind existing 3pids --- proposals/2140-terms-of-service-2.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 2f6c88d8b7..e203ffcf07 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -41,8 +41,14 @@ following convention from [MSC2134](https://github.com/matrix-org/matrix-doc/issues/2134). This proposal introduces: + * A v2 API prefix, with authentication, for the Identity Service * The `$prefix/terms` endpoint * The `m.accepted_terms` section in account data + * `POST /_matrix/client/r0/account/3pid/unbind` endpoints on the client/server + API + +This proposal removes: + * The `bind_email` and `bind_msisdn` on the Homeserver `/register` endpoint This proposal relies on both Integration Managers and Identity Servers being able to identity users by their MXID and store the fact that a given MXID has @@ -252,6 +258,20 @@ in the current login sessions is: * If the set of documents pending agreement was non-empty, Perform a `POST $prefix/terms` request to the servcie with these documents. +### `POST /_matrix/client/r0/account/3pid/unbind` + +A client uses this client/server API endpoint to request that the Homeserver +removes the given 3PID from the given Identity Server, or all Identity Servers. +Takes parameters the same parameters as +`POST /_matrix/client/r0/account/3pid/delete`, ie. `id_server`, `medium`, +`address` and the newly added `is_token`. + +Returns the same as `POST /_matrix/client/r0/account/3pid/delete`. + +Clients may add IS bindings for 3PIDs that already exist on the user's +Homeserver account by using the `POST /_matrix/client/r0/account/3pid` +to re-add the 3PID. + ## Tradeoffs The Identity Service API previously did not require authentication, and OpenID From 701d340da18b9fbfdc98f4f5aff1f8e6ea2e7f54 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 5 Jul 2019 19:00:15 +0100 Subject: [PATCH 49/62] Remove exception for request/submitToken --- proposals/2140-terms-of-service-2.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index e203ffcf07..9608093a7c 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -68,7 +68,6 @@ be dropped from all endpoints. Any request to any endpoint within `/_matrix/identity/v2`, with the exception of: * `/_matrix/identity/v2` - * any `requestToken` or `submitToken` endpoint * The new `$prefix/account/register` endpoint * The new `GET /_matrix/identity/v2/terms` * `$prefix/logout` @@ -77,13 +76,6 @@ of: This indicates that the user must authenticate with OpenID and supply a valid `access_token`. -`requestToken` and `submitToken` endpoints are excluded from the auth check -because they are used in the registration process before the user has an MXID -and therefore cannot log in with OpenID. It is up to the IS to manage its -privacy obligations appropriately when fulfilling these requests, bearing in -mind that the user has not explicitly indicated their agreement to any -documents, and may abort the registration process without doing so. - All other endpoints require authentication by the client supplying an access token either via an `Authorization` header with a `Bearer` token or an `access_token` query parameter. From 9bb6ad80d1c01e6d1588cf77f05fc84af1260e98 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 10 Jul 2019 16:13:38 +0100 Subject: [PATCH 50/62] typo --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 9608093a7c..fa89f60394 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -248,7 +248,7 @@ in the current login sessions is: token from the Homeserver and submit this token to the `register` endpoint. Store the resulting access token. * If the set of documents pending agreement was non-empty, Perform a - `POST $prefix/terms` request to the servcie with these documents. + `POST $prefix/terms` request to the service with these documents. ### `POST /_matrix/client/r0/account/3pid/unbind` From f474b31f5f8b74604ae652ec62eb3090615a80ee Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 12 Jul 2019 11:54:45 +0100 Subject: [PATCH 51/62] typo Co-Authored-By: J. Ryan Stinnett --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index fa89f60394..b840932cd9 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -51,7 +51,7 @@ This proposal removes: * The `bind_email` and `bind_msisdn` on the Homeserver `/register` endpoint This proposal relies on both Integration Managers and Identity Servers being -able to identity users by their MXID and store the fact that a given MXID has +able to identify users by their MXID and store the fact that a given MXID has indicated that they accept the terms given. Integration Managers already identify users in this way by authenticating them using the OpenID endpoint on the Homeserver. This proposal introduces the same mechanism to Identity Servers From 6e061b1baf8e03de4b01525b9aa39d4eb7e82c8b Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 12 Jul 2019 11:55:11 +0100 Subject: [PATCH 52/62] unnecessary capital Co-Authored-By: J. Ryan Stinnett --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index b840932cd9..21caa305f3 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -268,7 +268,7 @@ to re-add the 3PID. The Identity Service API previously did not require authentication, and OpenID is reasonably complex, adding a significant burden to both clients and servers. -A custom HTTP Header was also considered that could be added to assert that the +A custom HTTP header was also considered that could be added to assert that the client agrees to a particular set of terms. We decided against this in favour of re-using existing primitives that already exist in the Matrix ecosystem. Custom HTTP Headers are not used anywhere else within Matrix. This also gives a From 25a47afa329d39538b0856a1b6df3b4bec293197 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 12 Jul 2019 11:55:40 +0100 Subject: [PATCH 53/62] unnecessary capital mk. 2 Co-Authored-By: J. Ryan Stinnett --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 21caa305f3..eb4fd75c48 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -271,7 +271,7 @@ is reasonably complex, adding a significant burden to both clients and servers. A custom HTTP header was also considered that could be added to assert that the client agrees to a particular set of terms. We decided against this in favour of re-using existing primitives that already exist in the Matrix ecosystem. -Custom HTTP Headers are not used anywhere else within Matrix. This also gives a +Custom HTTP headers are not used anywhere else within Matrix. This also gives a very simple and natural way for ISes to enforce that users may only bind 3PIDs to their own MXIDs. From a1de6ff63445c0e59233875a9c53eee6efc9a9e5 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 15 Jul 2019 10:20:25 +0100 Subject: [PATCH 54/62] Hopefully clarify some bits --- proposals/2140-terms-of-service-2.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index eb4fd75c48..1120373fd5 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -76,9 +76,8 @@ of: This indicates that the user must authenticate with OpenID and supply a valid `access_token`. -All other endpoints require authentication by the client supplying an access token -either via an `Authorization` header with a `Bearer` token or an `access_token` -query parameter. +Clients authenticate either via an `Authorization` header with a `Bearer` token +or an `access_token` query parameter. The existing endpoints under `/_matrix/identity/api/v1/` continue to be unauthenticated but will be deprecated. ISes may support the old v1 API for as @@ -142,6 +141,17 @@ to use the service. Its response is similar to the structure used in the "name": "Conditions d'utilisation", "url": "https://example.org/somewhere/terms-2.0-fr.html" } + }, + "privacy_policy": { + "version": "1.2", + "en": { + "name": "Privacy Policy", + "url": "https://example.org/somewhere/privaacy-1.2-en.html" + }, + "fr": { + "name": "Politique de confidentialité", + "url": "https://example.org/somewhere/privacy-1.2-fr.html" + } } } } From d9269b084f6fbb6e95364c9bb919596919f3a3f3 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 15 Jul 2019 16:58:24 +0100 Subject: [PATCH 55/62] Exclude pubkey endpoints from auth --- proposals/2140-terms-of-service-2.md | 1 + 1 file changed, 1 insertion(+) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 1120373fd5..557376cc69 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -68,6 +68,7 @@ be dropped from all endpoints. Any request to any endpoint within `/_matrix/identity/v2`, with the exception of: * `/_matrix/identity/v2` + * `/_matrix/identity/v2/pubkey/*` * The new `$prefix/account/register` endpoint * The new `GET /_matrix/identity/v2/terms` * `$prefix/logout` From e4bdc283fd6cdeb2bf90b5a79a180fb19a8e9c63 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 19 Aug 2019 11:45:46 +0100 Subject: [PATCH 56/62] Apply suggestions from code review Typos / spelling Co-Authored-By: Hubert Chathi --- proposals/2140-terms-of-service-2.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 557376cc69..78a682d7cb 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -7,7 +7,7 @@ corresponding method that can be used with Identity Servers and Integration Managers. Requirements for this proposal are: - * ISs and IMs should be able to give multiple documents a user must agree to + * ISes and IMs should be able to give multiple documents a user must agree to abide by * Each document shoud be versioned * ISes and IMs must, for each request that they handle, know that the user @@ -15,7 +15,7 @@ Requirements for this proposal are: absolute proof (we will always have to trust that the client actually showed the document to the user) but it must be reasonably demonstrable that the user has given informed consent for the client to use that service. - * ISs and IMs must be able to prevent users from using the service if they + * ISes and IMs must be able to prevent users from using the service if they have not provided agreement. * A user should only have to agree to each version of each document once for their Matrix ID, ie. having agreed to a set of terms in one client, they @@ -265,7 +265,7 @@ in the current login sessions is: A client uses this client/server API endpoint to request that the Homeserver removes the given 3PID from the given Identity Server, or all Identity Servers. -Takes parameters the same parameters as +Takes the same parameters as `POST /_matrix/client/r0/account/3pid/delete`, ie. `id_server`, `medium`, `address` and the newly added `is_token`. From 12377fbf50515989004bc133071ca56c34ae300f Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 19 Aug 2019 11:53:41 +0100 Subject: [PATCH 57/62] /account/logout not /logout Co-Authored-By: Hubert Chathi --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 78a682d7cb..c19a1ea9b1 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -71,7 +71,7 @@ of: * `/_matrix/identity/v2/pubkey/*` * The new `$prefix/account/register` endpoint * The new `GET /_matrix/identity/v2/terms` - * `$prefix/logout` + * `$prefix/account/logout` ...may return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that the user must authenticate with OpenID and supply a valid From 6d0067320c5703d3408c19952d68fb5b41c588f0 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 19 Aug 2019 13:31:20 +0100 Subject: [PATCH 58/62] clarify error proxying --- proposals/2140-terms-of-service-2.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index c19a1ea9b1..ed8e740e90 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -94,7 +94,8 @@ Homeserver to make calls to the IS on its behalf, it must also supply its access token for the Identity Server alongside in the `is_token` key of the same JSON object. That is, in the main request object for a `requestToken` request and in the `threepidCreds` object when supplying 3PID credentials (eg. -in the `m.email.identity` UI auth stage). Exceptions to this are any requests +in the `m.email.identity` UI auth stage). The server must also relay +`M_TERMS_NOT_SIGNED` errors back to the client. Exceptions to this are any requests where the only IS operation the Homeserver may perform is unbinding, ie. `/_matrix/client/r0/account/deactivate` and `/_matrix/client/r0/account/3pid/delete`, in which case the unbind will be From 4073d940da7d0a3ed25dd30f820dd83692389a47 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 21 Aug 2019 15:11:10 +0100 Subject: [PATCH 59/62] Typo Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index ed8e740e90..3e2456bd34 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -148,7 +148,7 @@ to use the service. Its response is similar to the structure used in the "version": "1.2", "en": { "name": "Privacy Policy", - "url": "https://example.org/somewhere/privaacy-1.2-en.html" + "url": "https://example.org/somewhere/privacy-1.2-en.html" }, "fr": { "name": "Politique de confidentialité", From 69315417b5c576e4e7966fda4f40503f2c2d3ad4 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 21 Aug 2019 15:12:25 +0100 Subject: [PATCH 60/62] Typo Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 3e2456bd34..d256fa88b3 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -248,7 +248,7 @@ The `/_matrix/identity/v2/3pid/unbind` endpoint must not return either of these errors if the request has a valid signature from a Homeserver, and is being authenticated as such. In summary, the process for using a service that has not previously been used -in the current login sessions is: +in the current login session is: * `GET $prefix/terms` * Compare result with `m.accepted_terms` account data, get set of documents From 8bd9d7caeb2b5185878482eda0c81f50bc873468 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 21 Aug 2019 15:12:45 +0100 Subject: [PATCH 61/62] Add full stop Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/2140-terms-of-service-2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index d256fa88b3..49042aaf87 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -255,7 +255,7 @@ in the current login session is: pending agreement. * If non-empty, show this set of documents to the user and wait for the user to indicate their agreement. - * Add the newly agreed documents to `m.accepted_terms` + * Add the newly agreed documents to `m.accepted_terms`. * On success, or if there were no documents pending agreement, get an OpenID token from the Homeserver and submit this token to the `register` endpoint. Store the resulting access token. From 4ea8f645d6a6f134a137f86e81c4599e1cd057f7 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 21 Aug 2019 18:13:50 +0100 Subject: [PATCH 62/62] is_token -> id_access_token and add invite to proxy list --- proposals/2140-terms-of-service-2.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index 49042aaf87..9f96a00b5e 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -91,13 +91,14 @@ API, as specified in [MSC1961](https://github.com/matrix-org/matrix-doc/issues/1 When clients supply an identity server to the Homeserver in order for the Homeserver to make calls to the IS on its behalf, it must also supply its -access token for the Identity Server alongside in the `is_token` key of the -same JSON object. That is, in the main request object for a `requestToken` -request and in the `threepidCreds` object when supplying 3PID credentials (eg. -in the `m.email.identity` UI auth stage). The server must also relay -`M_TERMS_NOT_SIGNED` errors back to the client. Exceptions to this are any requests -where the only IS operation the Homeserver may perform is unbinding, ie. -`/_matrix/client/r0/account/deactivate` and +access token for the Identity Server alongside in the `id_access_token` key of +the same JSON object. That is, in the main request object for `requestToken` +and `/_matrix/client/r0/rooms/{roomId}/invite` requests and in the +`threepidCreds` object when supplying 3PID credentials (eg. in the +`m.email.identity` UI auth stage). The server must also relay +`M_TERMS_NOT_SIGNED` errors back to the client. Exceptions to this are any +requests where the only IS operation the Homeserver may perform is unbinding, +ie. `/_matrix/client/r0/account/deactivate` and `/_matrix/client/r0/account/3pid/delete`, in which case the unbind will be authenticated by a signed request from the Homeserver.