logout when uploading a big folder with a lot of subfolders and files

[individual-it]

Steps to reproduce

1. have a folder with a lot of folders and files
2. upload the folder through the browser

Expected behaviour

folder should be uploaded

Actual behaviour

after a while the uploads fail with 401 and the user gets logged out

Environment general

Operating system:
Ubuntu
Backend (ownCloud Core or Infinite Scale):
ocis

Environment ownCloud Infinite Scale

Version:
3.0.0 rc2

Updated from an older Infinite Scale or fresh install:
updated from 3.0.0 rc1

Where did you install Infinite Scale from:
download.owncloud.org

Client configuration

Browser:
Firefox 112
Operating system:
Ubuntu

Logs

Web server error log

2023-05-04T13:38:07+05:45 ERR failed to authenticate the request error="failed to verify access token: token contains an invalid number of segments" authenticator=oidc path=/remote.php/dav/spaces/ecabb8de-991a-426d-9a58-82fae8d6bcf4$56499ea2-bd76-47d1-bf97-4453d5ca98a5/connectiq-sdk-lin-4.1.7-2022-11-21-562b8a195/bin/templates/datafield/complex service=proxy
2023-05-04T13:38:07+05:45 ERR failed to authenticate the request error="failed to verify access token: token contains an invalid number of segments" authenticator=oidc path=/remote.php/dav/spaces/ecabb8de-991a-426d-9a58-82fae8d6bcf4$56499ea2-bd76-47d1-bf97-4453d5ca98a5/connectiq-sdk-lin-4.1.7-2022-11-21-562b8a195/bin/templates/datafield/complex service=proxy
2023-05-04T13:38:07+05:45 ERR failed to authenticate the request error="failed to verify access token: token contains an invalid number of segments" authenticator=oidc path=/remote.php/dav/spaces/ecabb8de-991a-426d-9a58-82fae8d6bcf4$56499ea2-bd76-47d1-bf97-4453d5ca98a5/connectiq-sdk-lin-4.1.7-2022-11-21-562b8a195/bin/templates/barrel/simple service=proxy

Browser log

CC @dragonchaser

[JammingBen]

Oof that is going to be a nasty one I believe. It looks like the access token can't be renewed during an upload with many files because the network is already crowded with pending & failing requests that eventually lead to a timeout. The timeout comes from the oidc-client-ts lib, needs more debugging.

A simple solution would be to decrease the limit of concurrent tus requests. Currently it is set to 20 (the default described in the uppy docs is wrong). Setting it to 5 should work for most cases and is maybe a good idea either way.

[individual-it]

Sometimes the upload just works fine, I suspect (and if you like I can try to prove it) that its about how busy the system and/or the browser is

[JammingBen]

Sometimes the upload just works fine, I suspect (and if you like I can try to prove it) that its about how busy the system and/or the browser is

Yes I agree, seems to be heavily related to the system/browser load (and of course the token expiring during the upload). No need to prove it, I can reproduce it quite well on my machine 🙂

Next step for me is to debug the oidc-client-ts lib and try to find out what exactly happens during the timeout. Then we can look for solutions. Maybe we need to think of a queue that prioritizes the token requests over other pending requests to assure a new token can be fetched when needed.

[JammingBen]

Next step for me is to debug the oidc-client-ts lib and try to find out what exactly happens during the timeout.

Okay so the timeout can be increased actually via a config. Unfortunately this just shifts the problem the bootstrap process afterwards, which then fails. The main problem stays the same: Too many concurrent request blocking the proper handling of a token renewal.

I think limiting the amount of concurrent requests is a quick fix as well as a good thing either way (see #8987). It should solve this issue here (hopefully).

We should still keep the option in mind to introduce some kind of request queue. But that would be a bigger task and something for the future.

[JammingBen]

Fixed via #8987 -> closing here.