Spaces registry (#2234)

* Do not allow cross-storage restores * Fix expected failures for s3ng * Be more robust when handling error cases * WIP: Fix restoring recycle items * Fix unit tests * Fix license headers * Fix linter issues * move unwrapping and wrapping of paths to the gateway temporary change to check CI * minimal space provider and registry prototypes Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add missing go.sum entry Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix import Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add embedded mounts when listing /home Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * remove unexpected passes Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * use space registry to manage aliases Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add changelog * add old logic as comment as we want to bring parts of it back for OCM Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * further work Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix access vi space & path Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * register /home as path alies for the users personal space Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * update toml files Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * rewrite gateway Stat() Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * rewrite ListContainer() Co-authored-by: David Christofas <C0rby@users.noreply.github.com> Co-authored-by: jkoberg <jkoberg@owncloud.com> Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * return not found Status instead of error Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add comment Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix create container Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * introduce findAndUnwrap Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * internally use spaces instead of simple download protocol Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix drone pipeline Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix litmus on spaces Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * check if parent exists in decomposedfs Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * align local frontend config with drone Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix move Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix copy Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * spaces provider is no longer needed Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * don't discards absolute pathes on providers * add DeleteStorageSpace to internal storage interface Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * sharesstorageprovider: implement ListStorageSpaces * tune ListContainer Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fine tune spaces registry Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * drop mount_id config, fix decomposedfs shares spaces Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * try fixing shares Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix shares Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * also wipe spaces when deleting users Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * try with empty quota Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add NewErrtypeFromStatus() helper Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make decomposedfs use nodeid as spaceid, hide spaces of type share Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * align and refactor Stat and ListContainer Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make sharesstorageprovider list each share as a space Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make spaces registry stat resource id instead of calling list spaces Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * sharesstorageprovider align stat & list Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * register shares spaces as /home/Shares/{name} Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * refactor sharesstorageprovider Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add TODO Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make gateway storageprovider return correct type on stat Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix path when listing unacceptaed shares Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add public function commend fort hound Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make gateway handle absolute id based stats correctly Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make spaces registry use configured provider path in id based requests Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * update combined.toml Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * drop gateway storage_rules Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix stat to root of space and absolute id Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * better handle relative references Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * revert path mangling for unaccepted shares Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * cleanup toml config Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * remove stale TODO Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * avoid caching sideeffects for now Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * try cleaner responsibilities Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * always send spaceid!nodeid in storagespaceid Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix spaces litmus tests Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make hound happy Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix spaces GET Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix up & download into shares Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * send space type when resolving id based request Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix public share listing Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix create container on public share Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * use reference for internal recycle api Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix shares Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * allow CurrentUser and Space in space registry template Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * ignore error when unsetting a not existing favorite flag Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * update path in dav report comment Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * only list spaces as nodes if no matches were found Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * create a copy of the reference to prevent losing the original Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * get rid of /home mountpoint to make paths unique we may have two providers: /home and /users/u-u-i-d for an id based request which one do we pick? we may end up with /dav/files/marie/f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c/PARENT/ in a REPORT https://cloud.ocis.test/dav/files/marie depending on the order we may pick f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c/PARENT/ over just /PARENT the /dav/files andpoint is configured to use the use id: "/users/{{.Id.OpaqueId}}" but the REPORT does not truncate it? how can the report search properly? with a relative request? no that would cut off the mount point the problem is there may be two mount points -> how can we get rid of that? no /home provider gateway GetHome should return "/users/{{.Id.OpaqueId}}" frontend uses "/users/{{.Id.OpaqueId}}" instead of "/home" for "/webdav" Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add template to ocs home namespace Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * id based requests must be forwarded as is * decomposedfs: fix spaces root * fix public path_template * correct config Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * decomposedfs: add missing error check * allow sending uuid in createspace Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix drone config * remove unexpected passes Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add xattrs.CopyMetadata() function Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * when constructing a folder wipe checksums from metadata Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * align sharesstorageprovider up & download Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * drop unused local toml config Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * decomposedfs: copy metadata from old to new revision Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix loop when downloading from a share Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix npe when no checksim is set Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix renaming share Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make resharing possible * work on ocs share paths Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * cut off path of received shares Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * work on ocs shares paths Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * use space id when listing trash Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * set name when err == nil Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * use spaceid instead of owner id when deleting a resource Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix pending share path Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * unmount share on delete Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * wrap trash item paths correctly in the gateway * the trash key contains root node and relative path * unprefix trash item path in ocdav * fix listing recycle in decomposedfs * add docs to public CopyMetadata() Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix renaming shares * wrap path in share response Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * wrap path in more share responses Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix public storage spaces path_template Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * omit virtual spaces config for now Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * omit PUT request for empty files * Do not try to walk an empty path * publicstorageprovider: do not prefix relative path to ListContainer items Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * publicstorageprovider: fix id based stat Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * return expected xml error on public link auth errors Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix sharesstorageprovider unit tests * fix file_target Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * unwrap permission error from scope Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix sharing unit tests * forward recycle item key when purging Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * work on decomposedfs tree tests Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix chunked upload Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * refactor gateway RestoreRecycleItem Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix trash restore Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * return ocs body on error when stating during create share Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * remove unexpected passes Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * render xml when resoucre not found on /dav/meta Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * Add grpc integration tests for the gateway * skip spaces tests and fix tree_test * when resource has same storageid delegate MOVE decision Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make content disposition reflect request base path Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * remove EnableHome from decomposedfs * make unit tests presumably green * remove home references from decomposedfs * use exact path match to find responsible ListRecycle provider Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix log typo Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix finding correct recycle source provider Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix restore ref unwrapping Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * align expected failures Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix decomposedfs/node unit tests * fix decomposedfs/tree unit tests * fix decomposedfs unit tests * fix decomposedfs concurrency unit tests * look up home provider from gateway Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make the linter happy * make build work again * make linter even more happy * more linting fixes * add even more linting fixes * fix trash item lookup Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * cache spaceid -> provider lookup Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * actually cache providers Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * first round of integration test fixes * get rid of old cache in spaces registry Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * don't use custom tempdir for integration tests * fix linter, add mount point caching thoughts Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix log typo Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * update spaces Readme Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * comment unused mount cache for now Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * integration test fixes part II * fix restore logic in decomposedfs * fix ocis integration tests * Add gateway integration tests for sharded directories * Start refactoring spaces registry, bring back unit tests * fix spaces registry tests Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make storageregistry service use new internal interface Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * commant internal storage registry interface Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * Bring back support for id based requests * adjust gateway to new registry API Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make static registry compatible with new gateway Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix findAndUnwrap Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * make hound happy again Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * send mount path in GetProvider response Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add naming comment Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix path in stat response for id based requests Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * experimental fix for integration tests * sharpen unwrap logic * attempt to fix the problem + probdescription * more sophisticated way of determining the root * add same logic to recycle operations * fix stat file type mount points Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * Streamline tests, increase coverage * Increase test coverage * comment cleanup Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix ListStorageSpace to contain correct spaceid Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix linting & hound * fix linting issues * log correct call Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * extract ref from opaque for public link scope Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix publiclink corner cases and scope Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * Clarify spaces registry configuration The new structure uses a providers map instead of a rules map with the provider address being the key. That makes things more clear and prevents bad configuration with the same provider being mounted at diffferent locations. * update expected failures Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fixing gateway.RestoreTrashItem Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * Remove programmatic test focus * refactor utils.SplitStorageSpaceID Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * forbid cross provider RestoreRecycleItem Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix relative references in space registry Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * refactor integration tests * fallback to CreateHome, deprecate GetHome Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * Fix CreateHome/GetHome when using the static registry * update cache notes Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * rename spaces registry rule struct to provider Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add a Stat cache to the gateway * also activate the cache * fix build after master merge * fix integration tests (again) * Allow read access to the root node * Add grpc integration tests for the virtual views case * disable cache for now Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * add comment to PublicStorageProviderID Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix sharing Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix unit tests (again) * fix 0 byte uploads Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix concurrency tests (again) * add license header for storageprovidercache * no longer wipe storage after every test Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> * fix linting (again) * Fix integration tests * make codacy happy * Revert "no longer wipe storage after every test" This reverts commit 00f4942. * make codacy happy (part II) * make codacy happy (part III) * virtual views pipeline is covered by tests/integration/grpc/gateway_storageprovider_static_test.go Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de> Co-authored-by: André Duffeck <andre.duffeck@firondu.de> Co-authored-by: David Christofas <dchristofas@owncloud.com> Co-authored-by: jkoberg <jkoberg@owncloud.com>
cs3org · Dec 6, 2021 · b2a9a7e · b2a9a7e
1 parent 255730c
commit b2a9a7e
Show file tree

Hide file tree

Showing 161 changed files with 9,795 additions and 4,287 deletions.
diff --git a/.drone.star b/.drone.star
@@ -105,7 +105,6 @@ def main(ctx):
     litmusOcisOldWebdav(),
     litmusOcisNewWebdav(),
     litmusOcisSpacesDav(),
-    virtualViews(),
   ] + ocisIntegrationTests(6) + s3ngIntegrationTests(12)
 
 
@@ -485,64 +484,6 @@ def release():
     "depends_on": ['changelog'],
   }
 
-def virtualViews():
-  return {
-    "kind": "pipeline",
-    "type": "docker",
-    "name": "virtual-views",
-    "platform": {
-      "os": "linux",
-      "arch": "amd64",
-    },
-    "trigger": {
-      "event": {
-        "include": [
-          "pull_request",
-          "tag",
-        ],
-      },
-    },
-    "steps": [
-      makeStep("build-ci"),
-      {
-        "name": "revad-services",
-        "image": "registry.cern.ch/docker.io/library/golang:1.17",
-        "detach": True,
-        "commands": [
-          "cd /drone/src/tests/oc-integration-tests/drone/",
-          "/drone/src/cmd/revad/revad -c frontend-global.toml &",
-          "/drone/src/cmd/revad/revad -c gateway.toml &",
-          "/drone/src/cmd/revad/revad -c storage-home-ocis.toml &",
-          "/drone/src/cmd/revad/revad -c storage-local-1.toml &",
-          "/drone/src/cmd/revad/revad -c storage-local-2.toml &",
-          "/drone/src/cmd/revad/revad -c users.toml",
-        ],
-      },
-      cloneOc10TestReposStep(),
-      {
-        "name": "oC10APIAcceptanceTestsOcisStorage",
-        "image": "registry.cern.ch/docker.io/owncloudci/php:7.4",
-        "commands": [
-          "cd /drone/src",
-          "make test-acceptance-api",
-        ],
-        "environment": {
-          "PATH_TO_CORE": "/drone/src/tmp/testrunner",
-          "TEST_SERVER_URL": "http://revad-services:20180",
-          "OCIS_REVA_DATA_ROOT": "/drone/src/tmp/reva/data/",
-          "DELETE_USER_DATA_CMD": "rm -rf /drone/src/tmp/reva/data/nodes/root/* /drone/src/tmp/reva/data/nodes/*-*-*-* /drone/src/tmp/reva/data/blobs/*",
-          "STORAGE_DRIVER": "OCIS",
-          "SKELETON_DIR": "/drone/src/tmp/testing/data/apiSkeleton",
-          "TEST_REVA": "true",
-          "REGULAR_USER_PASSWORD": "relativity",
-          "SEND_SCENARIO_LINE_REFERENCES": "true",
-          "BEHAT_SUITE": "apiVirtualViews",
-        },
-      },
-    ],
-    "depends_on": ['changelog'],
-  }
-
 def litmusOcisOldWebdav():
   return {
     "kind": "pipeline",
@@ -570,7 +511,6 @@ def litmusOcisOldWebdav():
           "cd /drone/src/tests/oc-integration-tests/drone/",
           "/drone/src/cmd/revad/revad -c frontend.toml &",
           "/drone/src/cmd/revad/revad -c gateway.toml &",
-          "/drone/src/cmd/revad/revad -c storage-home-ocis.toml &",
           "/drone/src/cmd/revad/revad -c storage-users-ocis.toml &",
           "/drone/src/cmd/revad/revad -c users.toml",
         ],
@@ -623,7 +563,6 @@ def litmusOcisNewWebdav():
           "cd /drone/src/tests/oc-integration-tests/drone/",
           "/drone/src/cmd/revad/revad -c frontend.toml &",
           "/drone/src/cmd/revad/revad -c gateway.toml &",
-          "/drone/src/cmd/revad/revad -c storage-home-ocis.toml &",
           "/drone/src/cmd/revad/revad -c storage-users-ocis.toml &",
           "/drone/src/cmd/revad/revad -c users.toml",
         ]
@@ -677,7 +616,6 @@ def litmusOcisSpacesDav():
           "cd /drone/src/tests/oc-integration-tests/drone/",
           "/drone/src/cmd/revad/revad -c frontend.toml &",
           "/drone/src/cmd/revad/revad -c gateway.toml &",
-          "/drone/src/cmd/revad/revad -c storage-home-ocis.toml &",
           "/drone/src/cmd/revad/revad -c storage-users-ocis.toml &",
           "/drone/src/cmd/revad/revad -c users.toml",
         ]
@@ -700,7 +638,7 @@ def litmusOcisSpacesDav():
         "commands": [
           # The spaceid is randomly generated during the first login so we need this hack to construct the correct url.
           "curl -s -k -u einstein:relativity -I http://revad-services:20080/remote.php/dav/files/einstein",
-          "export LITMUS_URL=http://revad-services:20080/remote.php/dav/spaces/123e4567-e89b-12d3-a456-426655440000!$(ls /drone/src/tmp/reva/data/spaces/personal/)",
+          "export LITMUS_URL=http://revad-services:20080/remote.php/dav/spaces/$(ls /drone/src/tmp/reva/data/spaces/personal/)",
           "/usr/local/bin/litmus-wrapper",
         ]
       },
@@ -743,7 +681,8 @@ def ocisIntegrationTests(parallelRuns, skipExceptParts = []):
               "/drone/src/cmd/revad/revad -c frontend.toml &",
               "/drone/src/cmd/revad/revad -c gateway.toml &",
               "/drone/src/cmd/revad/revad -c shares.toml &",
-              "/drone/src/cmd/revad/revad -c storage-home-ocis.toml &",
+              "/drone/src/cmd/revad/revad -c storage-shares.toml &",
+              "/drone/src/cmd/revad/revad -c machine-auth.toml &",
               "/drone/src/cmd/revad/revad -c storage-users-ocis.toml &",
               "/drone/src/cmd/revad/revad -c storage-publiclink.toml &",
               "/drone/src/cmd/revad/revad -c ldap-users.toml",
@@ -760,7 +699,7 @@ def ocisIntegrationTests(parallelRuns, skipExceptParts = []):
             "environment": {
               "TEST_SERVER_URL": "http://revad-services:20080",
               "OCIS_REVA_DATA_ROOT": "/drone/src/tmp/reva/data/",
-              "DELETE_USER_DATA_CMD": "rm -rf /drone/src/tmp/reva/data/nodes/root/* /drone/src/tmp/reva/data/nodes/*-*-*-* /drone/src/tmp/reva/data/blobs/*",
+              "DELETE_USER_DATA_CMD": "rm -rf /drone/src/tmp/reva/data/nodes/root/* /drone/src/tmp/reva/data/nodes/*-*-*-* /drone/src/tmp/reva/data/blobs/* /drone/src/tmp/reva/data/spaces/*/*",
               "STORAGE_DRIVER": "OCIS",
               "SKELETON_DIR": "/drone/src/tmp/testing/data/apiSkeleton",
               "TEST_WITH_LDAP": "true",
@@ -818,10 +757,11 @@ def s3ngIntegrationTests(parallelRuns, skipExceptParts = []):
               "/drone/src/cmd/revad/revad -c frontend.toml &",
               "/drone/src/cmd/revad/revad -c gateway.toml &",
               "/drone/src/cmd/revad/revad -c shares.toml &",
-              "/drone/src/cmd/revad/revad -c storage-home-s3ng.toml &",
               "/drone/src/cmd/revad/revad -c storage-users-s3ng.toml &",
               "/drone/src/cmd/revad/revad -c storage-publiclink.toml &",
-              "/drone/src/cmd/revad/revad -c ldap-users.toml",
+              "/drone/src/cmd/revad/revad -c storage-shares.toml &",
+              "/drone/src/cmd/revad/revad -c ldap-users.toml &",
+              "/drone/src/cmd/revad/revad -c machine-auth.toml",
             ],
           },
           cloneOc10TestReposStep(),
@@ -835,7 +775,7 @@ def s3ngIntegrationTests(parallelRuns, skipExceptParts = []):
             "environment": {
               "TEST_SERVER_URL": "http://revad-services:20080",
               "OCIS_REVA_DATA_ROOT": "/drone/src/tmp/reva/data/",
-              "DELETE_USER_DATA_CMD": "rm -rf /drone/src/tmp/reva/data/nodes/root/* /drone/src/tmp/reva/data/nodes/*-*-*-* /drone/src/tmp/reva/data/blobs/*",
+              "DELETE_USER_DATA_CMD": "rm -rf /drone/src/tmp/reva/data/nodes/root/* /drone/src/tmp/reva/data/nodes/*-*-*-* /drone/src/tmp/reva/data/blobs/* /drone/src/tmp/reva/data/spaces/*/*",
               "STORAGE_DRIVER": "S3NG",
               "SKELETON_DIR": "/drone/src/tmp/testing/data/apiSkeleton",
               "TEST_WITH_LDAP": "true",

diff --git a/changelog/unreleased/sharestorageprovider.md b/changelog/unreleased/sharestorageprovider.md
@@ -0,0 +1,5 @@
+Change: Add a sharestorageprovider
+
+This PR adds a ShareStorageProvider which enables us to get rid of a lot of special casing in other parts of the code. It also fixes several issues regarding shares and group shares.
+
+https://github.com/cs3org/reva/pull/2023
diff --git a/changelog/unreleased/spaces-registry.md b/changelog/unreleased/spaces-registry.md
@@ -0,0 +1,5 @@
+Change: Add a spaces registry
+
+Spaces registry is supposed to manage spaces. Read `pkg/storage/registry/spaces/Readme.md` for full details
+
+https://github.com/cs3org/reva/pull/2234
diff --git a/changelog/unreleased/wrap-unwrap-in-gateway.md b/changelog/unreleased/wrap-unwrap-in-gateway.md
@@ -0,0 +1,5 @@
+Change: move wrapping and unwrapping of paths to the storage gateway
+
+We've moved the wrapping and unwrapping of reference paths to the storage gateway so that the storageprovider doesn't have to know its mount path.
+
+https://github.com/cs3org/reva/pull/2016
diff --git a/docs/content/en/docs/config/grpc/services/storageprovider/_index.md b/docs/content/en/docs/config/grpc/services/storageprovider/_index.md
@@ -8,32 +8,16 @@ description: >
 
 # _struct: config_
 
-{{% dir name="mount_path" type="string" default="/" %}}
-The path where the file system would be mounted. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L57)
-{{< highlight toml >}}
-[grpc.services.storageprovider]
-mount_path = "/"
-{{< /highlight >}}
-{{% /dir %}}
-
-{{% dir name="mount_id" type="string" default="-" %}}
-The ID of the mounted file system. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L58)
-{{< highlight toml >}}
-[grpc.services.storageprovider]
-mount_id = "-"
-{{< /highlight >}}
-{{% /dir %}}
-
 {{% dir name="driver" type="string" default="localhome" %}}
-The storage driver to be used. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L59)
+The storage driver to be used. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L54)
 {{< highlight toml >}}
 [grpc.services.storageprovider]
 driver = "localhome"
 {{< /highlight >}}
 {{% /dir %}}
 
 {{% dir name="drivers" type="map[string]map[string]interface{}" default="localhome" %}}
- [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L60)
+ [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L55)
 {{< highlight toml >}}
 [grpc.services.storageprovider.drivers.localhome]
 root = "/var/tmp/reva/"
@@ -44,39 +28,39 @@ user_layout = "{{.Username}}"
 {{% /dir %}}
 
 {{% dir name="tmp_folder" type="string" default="/var/tmp" %}}
-Path to temporary folder. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L61)
+Path to temporary folder. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L56)
 {{< highlight toml >}}
 [grpc.services.storageprovider]
 tmp_folder = "/var/tmp"
 {{< /highlight >}}
 {{% /dir %}}
 
 {{% dir name="data_server_url" type="string" default="http://localhost/data" %}}
-The URL for the data server. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L62)
+The URL for the data server. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L57)
 {{< highlight toml >}}
 [grpc.services.storageprovider]
 data_server_url = "http://localhost/data"
 {{< /highlight >}}
 {{% /dir %}}
 
 {{% dir name="expose_data_server" type="bool" default=false %}}
-Whether to expose data server. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L63)
+Whether to expose data server. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L58)
 {{< highlight toml >}}
 [grpc.services.storageprovider]
 expose_data_server = false
 {{< /highlight >}}
 {{% /dir %}}
 
 {{% dir name="available_checksums" type="map[string]uint32" default=nil %}}
-List of available checksums. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L64)
+List of available checksums. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L59)
 {{< highlight toml >}}
 [grpc.services.storageprovider]
 available_checksums = nil
 {{< /highlight >}}
 {{% /dir %}}
 
 {{% dir name="mimetypes" type="map[string]string" default=nil %}}
-List of supported mime types and corresponding file extensions. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L65)
+List of supported mime types and corresponding file extensions. [[Ref]](https://github.com/cs3org/reva/tree/master/internal/grpc/services/storageprovider/storageprovider.go#L60)
 {{< highlight toml >}}
 [grpc.services.storageprovider]
 mimetypes = nil

diff --git a/examples/nextcloud-integration/revad.toml b/examples/nextcloud-integration/revad.toml
@@ -84,8 +84,6 @@ driver = "static"
 
 [grpc.services.storageprovider]
 driver = "nextcloud"
-mount_path = "/home"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 expose_data_server = true
 data_server_url = "http://127.0.0.1:19001/data"
 enable_home_creation = true
@@ -110,8 +108,6 @@ driver = "nextcloud"
 end_point = "http://localhost/apps/sciencemesh/"
 
 [http]
-enabled_services = ["ocmd"]
-enabled_middlewares = ["providerauthorizer", "cors"]
 address = "0.0.0.0:19001"
 
 [http.services.dataprovider]

diff --git a/examples/oc-phoenix/ocmd.toml b/examples/oc-phoenix/ocmd.toml
@@ -28,8 +28,6 @@ driver = "json"
 providers = "providers.demo.json"
 
 [http]
-enabled_services = ["ocmd"]
-enabled_middlewares = ["providerauthorizer", "cors"]
 address = "0.0.0.0:13001"
 
 [http.services.ocmd]

diff --git a/examples/oc-phoenix/storage-home.toml b/examples/oc-phoenix/storage-home.toml
@@ -23,8 +23,6 @@ address = "0.0.0.0:12000"
 # the context path wrapper reads tho username from the context and prefixes the relative storage path with it
 [grpc.services.storageprovider]
 driver = "owncloud"
-mount_path = "/home"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 expose_data_server = true
 data_server_url = "http://localhost:12001/data"
 enable_home_creation = true

diff --git a/examples/oc-phoenix/storage-oc.toml b/examples/oc-phoenix/storage-oc.toml
@@ -15,8 +15,6 @@ address = "0.0.0.0:11000"
 # we have a locally running dataprovider
 [grpc.services.storageprovider]
 driver = "owncloud"
-mount_path = "/oc"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 expose_data_server = true
 data_server_url = "http://localhost:11001/data"
 

diff --git a/examples/ocmd/ocmd-server-1.toml b/examples/ocmd/ocmd-server-1.toml
@@ -91,8 +91,6 @@ app_url = "https://your-collabora-server.org:9980"
 
 [grpc.services.storageprovider]
 driver = "localhome"
-mount_path = "/home"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 expose_data_server = true
 data_server_url = "http://localhost:19001/data"
 enable_home_creation = true

diff --git a/examples/ocmd/ocmd-server-2.toml b/examples/ocmd/ocmd-server-2.toml
@@ -65,8 +65,6 @@ driver = "memory"
 
 [grpc.services.storageprovider]
 driver = "localhome"
-mount_path = "/home"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 expose_data_server = true
 data_server_url = "http://localhost:17001/data"
 enable_home_creation = true

diff --git a/examples/storage-references/storage-home.toml b/examples/storage-references/storage-home.toml
@@ -3,8 +3,6 @@ address = "0.0.0.0:17000"
 
 [grpc.services.storageprovider]
 driver = "localhome"
-mount_path = "/home"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 data_server_url = "http://localhost:17001/data"
 
 [http]

diff --git a/examples/storage-references/storage-public.toml b/examples/storage-references/storage-public.toml
@@ -3,8 +3,6 @@ address = "0.0.0.0:16000"
 
 [grpc.services.publicstorageprovider]
 driver = "localhome"
-mount_path = "/public"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 data_server_url = "http://localhost:16001/data"
 gateway_addr = "localhost:19000"
 

diff --git a/examples/storage-references/storage-reva.toml b/examples/storage-references/storage-reva.toml
@@ -3,8 +3,6 @@ address = "0.0.0.0:18000"
 
 [grpc.services.storageprovider]
 driver = "local"
-mount_path = "/reva"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 data_server_url = "http://localhost:18001/data"
 
 [http]

diff --git a/examples/two-server-setup/storage-home-1.toml b/examples/two-server-setup/storage-home-1.toml
@@ -3,8 +3,6 @@ address = "0.0.0.0:17000"
 
 [grpc.services.storageprovider]
 driver = "localhome"
-mount_path = "/home"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 data_server_url = "http://localhost:17001/data"
 
 [http]

diff --git a/examples/two-server-setup/storage-home-2.toml b/examples/two-server-setup/storage-home-2.toml
@@ -3,8 +3,6 @@ address = "0.0.0.0:27000"
 
 [grpc.services.storageprovider]
 driver = "localhome"
-mount_path = "/home"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 data_server_url = "http://localhost:27001/data"
 
 [http]

diff --git a/examples/two-server-setup/storage-reva-1.toml b/examples/two-server-setup/storage-reva-1.toml
@@ -3,8 +3,6 @@ address = "0.0.0.0:18000"
 
 [grpc.services.storageprovider]
 driver = "local"
-mount_path = "/reva"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 data_server_url = "http://localhost:18001/data"
 
 [http]

diff --git a/examples/two-server-setup/storage-reva-2.toml b/examples/two-server-setup/storage-reva-2.toml
@@ -3,8 +3,6 @@ address = "0.0.0.0:28000"
 
 [grpc.services.storageprovider]
 driver = "local"
-mount_path = "/reva"
-mount_id = "123e4567-e89b-12d3-a456-426655440000"
 data_server_url = "http://localhost:28001/data"
 
 [http]

diff --git a/internal/grpc/interceptors/auth/scope.go b/internal/grpc/interceptors/auth/scope.go
@@ -210,6 +210,26 @@ func checkIfNestedResource(ctx context.Context, ref *provider.Reference, parent
 
 	return strings.HasPrefix(childPath, parentPath), nil
 
+	// resourcePath := statResponse.Info.Path
+
+	// if strings.HasPrefix(ref.GetPath(), resourcePath) {
+	// 	// The path corresponds to the resource to which the token has access.
+	// 	// We allow access to it.
+	// 	return true, nil
+	// }
+
+	// // If we arrived here that could mean that ref.GetPath is not prefixed with the storage mount path but resourcePath is
+	// // because it was returned by the gateway which will prefix it. To fix that we remove the mount path from the resourcePath.
+	// // resourcePath = "/users/<name>/some/path"
+	// // After the split we have [" ", "users", "<name>/some/path"].
+	// trimmedPath := "/" + strings.SplitN(resourcePath, "/", 3)[2]
+	// if strings.HasPrefix(ref.GetPath(), trimmedPath) {
+	// 	// The path corresponds to the resource to which the token has access.
+	// 	// We allow access to it.
+	// 	return true, nil
+	// }
+
+	// return false, nil
 }
 
 func extractRef(req interface{}, hasEditorRole bool) (*provider.Reference, bool) {