Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update reva and refactor config #657

Merged
merged 26 commits into from
Oct 21, 2020
+645 −2,514
Merged

update reva and refactor config #657

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
b64b324
update reva and refactor config
butonic Oct 6, 2020
5a9a9ce
set ocis storage owner on startup, account fixes
butonic Oct 6, 2020
28657f8
use /users endpoint for dav/files
butonic Oct 6, 2020
ece9c55
update reva
Oct 7, 2020
5b2bbb3
update reva
Oct 12, 2020
05dbb5f
update the environment variables in the drone file
Oct 13, 2020
12be19e
set user storage driver env variable in drone file
Oct 13, 2020
9617f91
add metadata root config
Oct 14, 2020
5257e0d
rename command
Oct 14, 2020
1084d45
update expected failures
Oct 14, 2020
001fb86
change mount id of home storage
Oct 15, 2020
dd4c320
change ocis log level to default and change public link storage mount…
Oct 15, 2020
4e548da
clean up storage configuration and update storage documentation
Oct 19, 2020
655c7a1
clean up user data in ocis storage
Oct 19, 2020
4a20f14
update expected failures
Oct 19, 2020
a6a6354
fix OCIS_REVA_DATA_ROOT
Oct 19, 2020
ee7260c
update reva
Oct 19, 2020
f51da17
update expected failures
Oct 19, 2020
9ae1891
fix OCIS_REVA_DATA_ROOT
Oct 19, 2020
916c984
update expected failures
Oct 19, 2020
ec692d3
update reva
Oct 21, 2020
63eede3
update expected failures
Oct 21, 2020
7094375
Don't expose data servers
kulmann Oct 21, 2020
91977ab
Bring back data server URLs..........
kulmann Oct 21, 2020
1da23d1
Bring back exposing user storage
kulmann Oct 21, 2020
8445996
Confirmed that both the home storage and users storage need to be
kulmann Oct 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 14 additions & 22 deletions .drone.star
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,8 +343,7 @@ def localApiTests(ctx, coreBranch = 'master', coreCommit = '', storage = 'ownclo
'environment' : {
'TEST_SERVER_URL': 'https://ocis-server:9200',
'OCIS_REVA_DATA_ROOT': '%s' % ('/srv/app/tmp/ocis/owncloud/data/' if storage == 'owncloud' else ''),
'STORAGE_DRIVER': '%s' % ('OWNCLOUD' if storage == 'owncloud' else 'OCIS'),
'DELETE_USER_DATA_CMD': '%s' % ('' if storage == 'owncloud' else 'rm -rf /srv/app/tmp/ocis/storage/users/nodes/root/*'),
'DELETE_USER_DATA_CMD': '%s' % ('' if storage == 'owncloud' else 'rm -rf /srv/app/tmp/ocis/storage/users/nodes/root/* /srv/app/tmp/ocis/storage/users/nodes/*-*-*-*'),
'SKELETON_DIR': '/srv/app/tmp/testing/data/apiSkeleton',
'OCIS_SKELETON_STRATEGY': '%s' % ('copy' if storage == 'owncloud' else 'upload'),
'TEST_OCIS':'true',
Expand Down Expand Up @@ -398,8 +397,7 @@ def coreApiTests(ctx, coreBranch = 'master', coreCommit = '', part_number = 1, n
'environment' : {
'TEST_SERVER_URL': 'https://ocis-server:9200',
'OCIS_REVA_DATA_ROOT': '%s' % ('/srv/app/tmp/ocis/owncloud/data/' if storage == 'owncloud' else ''),
'STORAGE_DRIVER': '%s' % ('OWNCLOUD' if storage == 'owncloud' else 'OCIS'),
'DELETE_USER_DATA_CMD': '%s' % ('' if storage == 'owncloud' else 'rm -rf /srv/app/tmp/ocis/storage/users/nodes/root/*'),
'DELETE_USER_DATA_CMD': '%s' % ('' if storage == 'owncloud' else 'rm -rf /srv/app/tmp/ocis/storage/users/nodes/root/* /srv/app/tmp/ocis/storage/users/nodes/*-*-*-*'),
'SKELETON_DIR': '/srv/app/tmp/testing/data/apiSkeleton',
'OCIS_SKELETON_STRATEGY': '%s' % ('copy' if storage == 'owncloud' else 'upload'),
'TEST_OCIS':'true',
Expand Down Expand Up @@ -548,22 +546,18 @@ def accountsUITests(ctx, phoenixBranch, phoenixCommitId, storage):
'detach': True,
'environment' : {
#'OCIS_LOG_LEVEL': 'debug',
'STORAGE_STORAGE_HOME_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_HOME_DATA_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_OC_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_OC_DATA_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_HOME_DATA_TEMP_FOLDER': '/srv/app/tmp/',
'STORAGE_HOME_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_OCIS_ROOT': '/srv/app/tmp/ocis/storage/users',
'STORAGE_STORAGE_LOCAL_ROOT': '/srv/app/tmp/ocis/reva/root',
'STORAGE_STORAGE_OWNCLOUD_DATADIR': '/srv/app/tmp/ocis/owncloud/data',
'STORAGE_STORAGE_OC_DATA_TEMP_FOLDER': '/srv/app/tmp/',
'STORAGE_STORAGE_OWNCLOUD_REDIS_ADDR': 'redis:6379',
'STORAGE_OIDC_ISSUER': 'https://ocis-server:9200',
'STORAGE_LDAP_IDP': 'https://ocis-server:9200',
'PROXY_OIDC_ISSUER': 'https://ocis-server:9200',
'STORAGE_STORAGE_OC_DATA_SERVER_URL': 'http://ocis-server:9164/data',
'STORAGE_DATAGATEWAY_URL': 'https://ocis-server:9200/data',
'STORAGE_FRONTEND_URL': 'https://ocis-server:9200',
'STORAGE_HOME_EXPOSE_DATA_SERVER': 'true',
'STORAGE_HOME_DATA_SERVER_URL': 'http://ocis-server:9155/data',
'STORAGE_DATAGATEWAY_PUBLIC_URL': 'https://ocis-server:9200/data',
'STORAGE_FRONTEND_PUBLIC_URL': 'https://ocis-server:9200',
'PHOENIX_WEB_CONFIG': '/drone/src/accounts/ui/tests/config/drone/ocis-config.json',
'KONNECTD_IDENTIFIER_REGISTRATION_CONF': '/drone/src/accounts/ui/tests/config/drone/identifier-registration.yml',
'KONNECTD_ISS': 'https://ocis-server:9200',
Expand Down Expand Up @@ -1407,22 +1401,20 @@ def ocisServer(storage):
'detach': True,
'environment' : {
#'OCIS_LOG_LEVEL': 'debug',
'STORAGE_STORAGE_HOME_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_HOME_DATA_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_OC_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_OC_DATA_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_HOME_DATA_TEMP_FOLDER': '/srv/app/tmp/',
'STORAGE_HOME_DRIVER': '%s' % (storage),
'STORAGE_USERS_DRIVER': '%s' % (storage),
'STORAGE_STORAGE_OCIS_ROOT': '/srv/app/tmp/ocis/storage/users',
'STORAGE_STORAGE_LOCAL_ROOT': '/srv/app/tmp/ocis/local/root',
'STORAGE_STORAGE_OWNCLOUD_DATADIR': '/srv/app/tmp/ocis/owncloud/data',
'STORAGE_STORAGE_OC_DATA_TEMP_FOLDER': '/srv/app/tmp/',
'STORAGE_METADATA_ROOT': '/srv/app/tmp/ocis/metadata',
'STORAGE_STORAGE_OWNCLOUD_REDIS_ADDR': 'redis:6379',
'STORAGE_LDAP_IDP': 'https://ocis-server:9200',
'STORAGE_OIDC_ISSUER': 'https://ocis-server:9200',
'PROXY_OIDC_ISSUER': 'https://ocis-server:9200',
'STORAGE_STORAGE_OC_DATA_SERVER_URL': 'http://ocis-server:9164/data',
'STORAGE_DATAGATEWAY_URL': 'https://ocis-server:9200/data',
'STORAGE_FRONTEND_URL': 'https://ocis-server:9200',
'STORAGE_HOME_DATA_SERVER_URL': 'http://ocis-server:9155/data',
'STORAGE_DATAGATEWAY_PUBLIC_URL': 'https://ocis-server:9200/data',
'STORAGE_USERS_DATA_SERVER_URL': 'http://ocis-server:9158/data',
'STORAGE_FRONTEND_PUBLIC_URL': 'https://ocis-server:9200',
'PHOENIX_WEB_CONFIG': '/drone/src/ocis/tests/config/drone/ocis-config.json',
'KONNECTD_IDENTIFIER_REGISTRATION_CONF': '/drone/src/ocis/tests/config/drone/identifier-registration.yml',
'KONNECTD_ISS': 'https://ocis-server:9200',
Expand Down
7 changes: 3 additions & 4 deletions accounts/pkg/storage/cs3.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,12 +206,11 @@ func (r CS3Repo) DeleteGroup(ctx context.Context, id string) (err error) {

func (r CS3Repo) authenticate(ctx context.Context) (token string, err error) {
u := &user.User{
Id: &user.UserId{},
Id: &user.UserId{
OpaqueId: r.cfg.ServiceUser.UUID,
Copy link
Member Author

@butonic butonic Oct 6, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kulmann we can always set the uuid ... if it is "" that is the same as this if magic ;-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not empty by default. It has a default UUID value in the flatset, so that it just works as soon as the username is provided. You're right, but it needs more change than this. ;-) Clear the default value for the UUID in the flagset and replace the checks on the username flag with checks on the uuid flag.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it has a uuid, but before this change it would only set it if the username was also set ... but it is empty by default: https://github.com/owncloud/ocis/pull/657/files#diff-69d2327c70e84f0e2b037204de3ef8e3L212-L214

},
Groups: []string{},
}
if r.cfg.ServiceUser.Username != "" {
u.Id.OpaqueId = r.cfg.ServiceUser.UUID
}
return r.tm.MintToken(ctx, u)
}

Expand Down
20 changes: 6 additions & 14 deletions docs/extensions/storage/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,12 @@ It uses the port range 9140-9179 to preconfigure several services.
| 9149 | authbearer debug |
| 9150 | sharing |
| 9151 | sharing debug |
| 9152 | storage root |
| 9153 | storage root debug |
| 9154 | storage home |
| 9155 | storage home debug |
| 9156 | storage home data |
| 9157 | storage home data debug |
| 9158 | storage eos |
| 9159 | storage eos debug |
| 9160 | storage eos data |
| 9161 | storage eos data debug |
| 9162 | storage oc |
| 9163 | storage oc debug |
| 9164 | storage oc data |
| 9165 | storage oc data debug |
| 9155 | storage home data |
| 9156 | storage home debug |
| 9157 | storage users |
| 9158 | storage users data |
| 9159 | storage users debug |
| 9166-9177 | reserved for s3, wnd, custom + data providers |
| 9178 | storage public link |
| 9179 | storage public link data |
| 9179 | storage public link debug |
Loading