Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for chunked upload v2 - S3 multipart upload #924

Closed
juliusknorr opened this issue Sep 1, 2023 · 6 comments
Closed

Support for chunked upload v2 - S3 multipart upload #924

juliusknorr opened this issue Sep 1, 2023 · 6 comments
Assignees
Labels
1. to develop Accepted and waiting to be taken care of bug Something isn't working regression Regression of a previous working feature

Comments

@juliusknorr
Copy link
Contributor

We need to adapt the chunked upload bits to also work with nextcloud/server#27034

documented in nextcloud/documentation#10740

@juliusknorr juliusknorr added the bug Something isn't working label Sep 1, 2023
@juliusknorr
Copy link
Contributor Author

Would be a regression for 28 if we don't do that, I can check this somewhen next week.

@juliusknorr juliusknorr self-assigned this Sep 1, 2023
@skjnldsv
Copy link
Contributor

skjnldsv commented Sep 1, 2023

Interesting!
Looking at this, it shouldn't be too complicated.

Though, shall we ever support the v1 ?
Like is there some limitations to it?

@skjnldsv skjnldsv added 1. to develop Accepted and waiting to be taken care of regression Regression of a previous working feature labels Sep 1, 2023
@juliusknorr
Copy link
Contributor Author

There is a Limitation as the maximum possible upload size got a file would be 5TB and the chunk size needs to be increased dynamically in case the filesize doesn't fit in 10.000 chunks of the default 10MB chunk size (but that would be for files >100gb). So overall nothing I would consider worth keeping two implementations, especially since this would then still only help for local storage as s3 always had those limitations. Also uploading 100gb in 10 mb chunks doesn't make much sense at all in terms of performance.

Hope that makes it a bit clearer :)

@skjnldsv
Copy link
Contributor

skjnldsv commented Sep 1, 2023

How does the front decides which to use then?

@juliusknorr
Copy link
Contributor Author

It does not need to, new api calls are backward compatible with the old backend.

@juliusknorr
Copy link
Contributor Author

Fixed by #925

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1. to develop Accepted and waiting to be taken care of bug Something isn't working regression Regression of a previous working feature
Projects
None yet
Development

No branches or pull requests

2 participants