Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3NG: EOF if blob is not accessible #4785

Closed
wkloucek opened this issue Oct 11, 2022 · 3 comments
Closed

S3NG: EOF if blob is not accessible #4785

wkloucek opened this issue Oct 11, 2022 · 3 comments
Assignees
Labels
Priority:p1-urgent Consider a hotfix release with only that fix Storage:S3NG Type:Bug

Comments

@wkloucek
Copy link
Contributor

Describe the bug

When using S3NG and a file's blob is not accessible, we get no real error code but an EOF on downloading the file.

Steps to reproduce

Steps to reproduce the behavior:

  1. run the oCIS S3 example
  2. upload a file
  3. delete the blob via the minio GUI
  4. download the file inside ownCloud Web

Expected behavior

We get a Webdav spec compliant status code that the file can not be downloaded

Actual behavior

EOF errors throughout all service that participate in that request

ocis_s3-ocis-1  | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"00000000000000000000000000000000","svc":"datatx","handler":"download","error":"The specified key does not exist.","time":"2022-10-11T15:25:04.942148689Z","message":"error copying data to response"}
ocis_s3-ocis-1  | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"00000000000000000000000000000000","error":"unexpected EOF","time":"2022-10-11T15:25:04.942366289Z","message":"error writing body after headers were sent"}
ocis_s3-ocis-1  | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"00000000000000000000000000000000","content-length":75416,"transferred-bytes":0,"time":"2022-10-11T15:25:04.94238269Z","message":"content length vs transferred bytes mismatch"}
ocis_s3-ocis-1  | 2022/10/11 15:25:04 httputil: ReverseProxy read error during body copy: unexpected EOF
ocis_s3-ocis-1  | 2022/10/11 15:25:04 httputil: ReverseProxy read error during body copy: unexpected EOF
ocis_s3-ocis-1  | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"00000000000000000000000000000000","svc":"datatx","handler":"download","error":"The specified key does not exist.","time":"2022-10-11T15:25:04.986053716Z","message":"error copying data to response"}
ocis_s3-ocis-1  | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"00000000000000000000000000000000","error":"unexpected EOF","time":"2022-10-11T15:25:04.986354823Z","message":"error writing body after headers were sent"}
ocis_s3-ocis-1  | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"00000000000000000000000000000000","content-length":75416,"transferred-bytes":0,"time":"2022-10-11T15:25:04.986369841Z","message":"content length vs transferred bytes mismatch"}
ocis_s3-ocis-1  | 2022/10/11 15:25:04 httputil: ReverseProxy read error during body copy: unexpected EOF
ocis_s3-ocis-1  | 2022/10/11 15:25:04 httputil: ReverseProxy read error during body copy: unexpected EOF

Additional context

We didn't have this problem by deleting an actual blob. Somehow our S3 client / S3 bucket caused a "The specified key does not exist." error (which is coming from the minio-go lib).

We don't know if the client (connection) or the S3 server was stuck, but a restart of the storage-users service resolved the issue.

During that error state, we discovered the EOF bug described above.

@kobergj
Copy link
Collaborator

kobergj commented Oct 11, 2022

Mmh. We do return the error from minio-go client correctly: https://github.com/cs3org/reva/blob/experimental/pkg/storage/fs/s3ng/blobstore/blobstore.go#L130-L133

So somehow minio-go is just logging the error, but not returning it.

@wkloucek
Copy link
Contributor Author

It was a reva log, that contained the error description from minio-go.

I think it was here:

https://github.com/cs3org/reva/blob/edge/pkg/rhttp/datatx/utils/download/download.go#L110-L115

@micbar micbar added Priority:p1-urgent Consider a hotfix release with only that fix GA-Blocker labels Oct 11, 2022
@micbar micbar added this to the 2.0.0 General Availability milestone Oct 11, 2022
@wkloucek
Copy link
Contributor Author

will be fixed by cs3org/reva#3341

@wkloucek wkloucek self-assigned this Oct 12, 2022
@wkloucek wkloucek reopened this Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority:p1-urgent Consider a hotfix release with only that fix Storage:S3NG Type:Bug
Projects
None yet
Development

No branches or pull requests

3 participants