.well-known lookup isn't playing nicely with custom servers which don't support .well-known

[peter-scholtens]

Description

Trying to login with riot-web via https://riot.im/app/#/login using a alternative home server generates an error message "Cannot find homeserver"

Steps to reproduce

• Browse to https://riot.im/app/#/login
• Click 'custom server'
• Fill the field home server URL with https://matrix4me.nl
• Fill user field with @test.user2:matrix4me.nl (is an existing user)
  Immediately, the webpage repsonses with "Cannot find homeserver", even before typing a password.

Till last week the server was recognized, and users could login with credentials. Since 13 ... 14 december (or maybe earlier) it worked correctly: user could login.

Last week I did a server upgrade, mostly on tooling (I guess) using:
pip install --upgrade pip (didn't log the version update number)
pip install --upgrade setuptools (didn't log the version update number)
pip install https://github.com/matrix-org/synapse/tarball/master (was already 0.33.9)

Version information

For the web app:

• Client: Ubuntu Linux 18.04
• Client: Firefox, 64.0+build3-0ubuntu0.18.04.1, amd64.

Used synapse server:

• Server: Synapse 0.33.9
• Server: Debian 9 (stretch), 4.9.0-8-amd64 Leave room button #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64

Javascript errors

• Failed to open log database: InvalidStateError
• Failed to initialise rageshake: Error: Failed to open log database: InvalidStateError
• XML Parsing Error: syntax error Location: https://riot.im/app/version?cachebuster=1544996746660 Line Number 1, Column 1:
• Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://matrix4me.nl/.well-known/matrix/client. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).
• Object { err: Error, response: XMLHttpRequest, wkAction: "FAIL_ERROR" }

Observations

• Other users reported error when starting a new session, while some of the existing sessions kept working.

[turt2live]

This is almost certainly the new .well-known (auto discovery) interfering with your login. You could set up a .well-known file, although instructions are relatively limited here.

Instead, it may be easier to use your localpart (the bit between the @ and :, so @alice:example.org's localpart would be alice) and ensure the custom server option points to the right place.

[peter-scholtens]

When I fill in only the localpart (test.user2), then I get the error "Failed to perform homeserver discovery". Still with correct server name and protocol "https://matrix4me.nl".

[turt2live]

Well that's not supposed to happen :(

Does refreshing and trying from scratch help?

[peter-scholtens]

You mean the server? I've rebooted it. Didn't help. And the tarball does not have memory of previous versions...

[turt2live]

I meant riot-web. It shouldn't be complaining about homeserver discovery if you're using your localpart.

[peter-scholtens]

I don't own https://riot.im/app/#/login so I cannot do much. Currently I've not my own server running for the riot-web, only for synapse.

[lampholder]

@peter-scholtens I suspect you'd be able to log in if you:

• go to https://riot.im/app/#/login
• change to 'custom server'
• enter 'https://matrix4me.nl'
• enter just 'test.user2' in the 'User name' field
• enter your password and click 'login'

This is almost certainly the new .well-known (auto discovery) interfering with your login. You could set up a .well-known file, although instructions are relatively limited here.

@turt2live I'm going to rename this bug to reflect this likely cause

[turt2live]

I think the stuck error problem is #7845. The lack of documentation is more or less #7760. The interference is something that we might not be able to solve without design.

[peter-scholtens]

@lampholder I tried from several networks, with and without "@" and ":matrix4me.nl". All fail with "Cannot find homeserver".

[lampholder]

#7977 is another report of the same thing with some potentially-useful discussion

[schildbach]

I've got the same issue. The "only use localpart" workaround did not help for me either. I'm still using matrix-synapse 0.24.0+dfsg-1, that's what comes with Ubuntu 18.04 LTS.

[MTRNord]

@schildbach I have the same issue. It especially breaks when the browser auto inputs fields with non user data :/ It seems like to try to use it as mxid but than locks up in a wrong state. I had to disable browser auto input, prevent to focus the localpart field and first type the new custom server ip and after that my localpart and password to get it to login at all

[turt2live]

The stuck error is #7845 and a different issue

[julienschmidt]

We started using the .well-known/matrix/server file for our server recently and one user reported the same problem with our riot-web setup (however, somehow just on Windows, regardless of the used browser).

I think there are two problems here:

1. riot-web very likely makes a cross-origin request when it tries to load the .well-known/matrix/client file (unless riot-web is running on the exact same domain), which means that the server has to send a Access-Control-Allow-Origin: * or similar header, as a CORS request requires to make a HEAD request first and check if access is allowed. This should be documented (see Document .well-known setup, etc (including ACAO headers) #7760).
2. Web servers (e.g. nginx) usually don't send such headers on errors. E.g. if one would set add_header Access-Control-Allow-Origin *; in the nginx config, it might still not work. In our case the problem was that .well-known/matrix/client did not exist (while .well-known/matrix/server DID exist), thus causing a 404 error which then does not send the required header. One can fix that either by using add_header Access-Control-Allow-Origin * always; or by creating the missing file. It would however probably be a good idea to catch failed requests for .well-known/matrix/client (which then have the status code 0) in riot-web and treat them like the file does not exist.

[aaronraimist]

@turt2live so it seems like the problem is

if (response.statusCode === 404) {
    action = "IGNORE";
    reason = "No .well-known JSON file found";
}

https://github.com/matrix-org/matrix-js-sdk/blob/03a54353be98941c7e8c124393b386d0b6affc6f/src/autodiscovery.js#L366

If I comment out the if statement then it won't display an error message if the homeserver has no .well-known/matrix/client file. The reason it can't see the 404 is because of CORS.

With the check for 404:

Error message to user: Invalid homeserver discovery response

Without the check for 404:

No error message to user

The downside of commenting this out is no user facing error message will appear if the file does exist but it can't be accessed due to CORS. Maybe that's an OK compromise?