Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[spaces 1/2] initial ListStorageSpaces implementation #1802

Merged
merged 5 commits into from
Jul 8, 2021

Conversation

butonic
Copy link
Contributor

@butonic butonic commented Jun 18, 2021

I split #1776 into two PRs:

  1. the initial list spaces implementation
  2. the new /dav/spaces endpoint to access storage with relative references

This is the first PR about spaces. It

  • adds logic to the gateway to find or query the known storage providers and deduplicate the results
    • we currently have a storage provider mounted at /home and /users, both will return the the same spaces
    • the registry currently does not set the ProviderId, only the ProviderPath, which we use in a hacky way to ignore all storage providers that start with a /, eg. /public, /home, etc. We then query only the remaining ones, which have a storage id set and can be routed to when using the ResourceId of a relative Reference.
  • adds a ListStorageSpaces implementation for the decomposedfs
  • adds ListStorageSpaces stubs for the other storage providers

@butonic butonic requested a review from labkode as a code owner June 18, 2021 09:16
@update-docs
Copy link

update-docs bot commented Jun 18, 2021

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

C0rby
C0rby previously approved these changes Jun 24, 2021
Copy link
Contributor

@C0rby C0rby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just some small remarks.

internal/grpc/services/gateway/storageprovider.go Outdated Show resolved Hide resolved
internal/grpc/services/gateway/storageprovider.go Outdated Show resolved Hide resolved
pkg/storage/fs/owncloudsql/owncloudsql.go Outdated Show resolved Hide resolved
pkg/storage/utils/decomposedfs/decomposedfs.go Outdated Show resolved Hide resolved
pkg/storage/utils/decomposedfs/decomposedfs.go Outdated Show resolved Hide resolved
C0rby
C0rby previously approved these changes Jun 24, 2021
@C0rby
Copy link
Contributor

C0rby commented Jun 25, 2021

@butonic, can you restart the CI build or do I need git push something?

@butonic
Copy link
Contributor Author

butonic commented Jun 25, 2021

@C0rby I can't, but pushing should trigger it automatically.

Copy link
Member

@refs refs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like for the Reva code base to be more approachable and part of it is to start being consistent. During this review I had this in mind:

  • focus in consistency (i.e var blocks vs inline declarations)
  • special emphasis on reducing the cyclomatic complexity of certain functions
    • this includes:
      • refactoring
      • abstracting function bodies away (not refactoring per-se...)
  • clarity of language
    • this refers to comments as well as blocks of logic

While I was probably not 100% enforcing all this points, the areas I reviewed I believe serve as a good example where most of our logic goes awry. Just because we have a verbose API doesn't mean our blocks have to be verbose 🚀.

This review is by no means a blocker, but I would find value if the points are discussed collectively in the public chat.

internal/grpc/services/gateway/storageprovider.go Outdated Show resolved Hide resolved
Comment on lines +134 to +141
parts := strings.SplitN(id.OpaqueId, "!", 2)
if len(parts) != 2 {
return &provider.ListStorageSpacesResponse{
Status: status.NewInvalidArg(ctx, "space id must be separated by !"),
}, nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd refactor this to something with more semantic meaning. All we see here is "something has to be split in 2 using ! as a delimiter, and this will probably be used wherever we use the spaces API.

internal/grpc/services/gateway/storageprovider.go Outdated Show resolved Hide resolved

spacesFromProviders := make([][]*provider.StorageSpace, len(providers))
errors := make([]error, len(providers))
var wg sync.WaitGroup
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var wg sync.WaitGroup


uniqueSpaces := map[string]*provider.StorageSpace{}
for i := range providers {
if errors[i] != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would an error condition mean skip the provider? Should we check the error type? Could it be a "mission critical" error and halt this operation? Maybe i'm overthinking 🦖

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when a provider retuns an unsupported error we should continue with the others. expect failure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's one only provider which matches the request, we won't raise any errors and return an empty slice. Maybe we can handle that case separately?

pkg/storage/utils/decomposedfs/decomposedfs.go Outdated Show resolved Hide resolved
pkg/storage/utils/decomposedfs/decomposedfs.go Outdated Show resolved Hide resolved
pkg/storage/utils/decomposedfs/decomposedfs.go Outdated Show resolved Hide resolved
pkg/storage/utils/decomposedfs/tree/tree.go Show resolved Hide resolved
refs
refs previously approved these changes Jun 28, 2021
@butonic butonic requested a review from C0rby July 2, 2021 12:26
@butonic
Copy link
Contributor Author

butonic commented Jul 2, 2021

@labkode @ishank011 this is ready for review

@butonic butonic requested a review from refs July 5, 2021 10:25
refs
refs previously approved these changes Jul 5, 2021

uniqueSpaces := map[string]*provider.StorageSpace{}
for i := range providers {
if errors[i] != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's one only provider which matches the request, we won't raise any errors and return an empty slice. Maybe we can handle that case separately?

Comment on lines +163 to +164
// TODO if the storage id is not set but node id is set we could poll all storage providers to check if the node is known there
// for now, say the reference is invalid
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this one 😅

pkg/storage/utils/decomposedfs/decomposedfs.go Outdated Show resolved Hide resolved
butonic and others added 5 commits July 7, 2021 16:45
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
address small review items

Co-authored-by: Alex Unger <6905948+refs@users.noreply.github.com>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@ishank011 ishank011 merged commit d2f72e0 into cs3org:master Jul 8, 2021
@butonic butonic deleted the list-spaces branch July 8, 2021 20:36
thmour pushed a commit to thmour/reva that referenced this pull request Jul 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants