Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Spaces] Space-Aware Saved Objects #18862

Merged
merged 52 commits into from
Jul 26, 2018
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
f208d3c
Crude and incomplete impl of Space-Aware Saved Objects Client
legrego May 7, 2018
7538bf3
Code review updates
legrego May 8, 2018
bb3e511
Missed one - move extraBodyProperties to the top
legrego May 8, 2018
0e2e4e8
Remove documentFilter from bulkGet
legrego May 9, 2018
a6287cc
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego May 21, 2018
7db0a4a
Make config document id independent of Kibana version
legrego May 21, 2018
192d9c2
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jun 18, 2018
1c4afd8
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jun 20, 2018
d573457
cleanup and fixes following initial rbac phase 1 merge
legrego Jun 20, 2018
d99cec7
remove unused/migrated files
legrego Jun 20, 2018
7e2d1e3
remove unused code
legrego Jun 20, 2018
862752b
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jun 20, 2018
a63128c
partial updates for space aware saved objects and tests
legrego Jun 22, 2018
858eff0
working get & find functional tests
legrego Jun 22, 2018
1b95aa0
added bulk_get tests
legrego Jun 22, 2018
fc663d9
refactor query params into dedicated module
legrego Jun 25, 2018
fc61594
additional tests and bugfixes for space aware saved objects
legrego Jun 25, 2018
776da8a
revert changes to ui settings service
legrego Jun 25, 2018
d2545d4
additional tests for space-aware saved objects
legrego Jun 25, 2018
1fd7699
Fix navigating to the default space
legrego Jun 25, 2018
9469742
additional unit tests
legrego Jun 25, 2018
e24578f
Create default space on startup, *after* ES has gone green
legrego Jun 25, 2018
5fe4bfd
support & testing for bulk_create for space-enabled installations
legrego Jun 26, 2018
c6e8925
cleanup and docs
legrego Jun 26, 2018
f4a19ab
undo formatting changes
legrego Jun 27, 2018
dee335b
only allow filters to be passed to getQueryParams
legrego Jun 29, 2018
452de10
don't add space id when updating within the default space
legrego Jun 29, 2018
6bf3515
renaming files
legrego Jun 29, 2018
8cb871a
additional SOC and repository tests
legrego Jul 5, 2018
53bb020
remove default context from utility functions
legrego Jul 5, 2018
90892ca
rename spacesSavedObjectsClientWrapper => spacesSavedObjectsClientWra…
legrego Jul 5, 2018
4181c9e
don't mutate passed options for SOC create method
legrego Jul 5, 2018
a35d15f
allow options to be passed for get and bulkGet
legrego Jul 5, 2018
093dd47
additional review updates
legrego Jul 5, 2018
6e1c4c4
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jul 5, 2018
2195ee0
consolidate init logic
legrego Jul 5, 2018
48c5f23
Add error handling when switching spaces
legrego Jul 6, 2018
3a832e9
rename single character variables
legrego Jul 6, 2018
71f0634
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jul 9, 2018
00bd94c
fix merge
legrego Jul 16, 2018
415fa09
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jul 16, 2018
8d48c80
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jul 19, 2018
bc8aaef
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jul 20, 2018
839a0cf
Merge branch 'spaces-phase-1' into space-aware-saved-objects
legrego Jul 23, 2018
8b1ff6b
address PR feedback
legrego Jul 23, 2018
299efb8
Consolidate space url parsing on the server
legrego Jul 12, 2018
3d0f6c9
fix base path parsing
legrego Jul 23, 2018
1747a74
refactor spaces saved objects client tests
legrego Jul 25, 2018
78a54d1
fix bug routing to the correct space
legrego Jul 25, 2018
2149f87
address PR feedback
legrego Jul 25, 2018
85b315f
remove stray console.log
legrego Jul 26, 2018
9c69568
augment find tests to include global types
legrego Jul 26, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions src/server/saved_objects/client/lib/search_dsl/query_params.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,19 +27,27 @@ function getFieldsForTypes(searchFields, types) {
* @param {Object} type
* @param {String} search
* @param {Array<string>} searchFields
* @param {Array<object>} extraFilters
* @return {Object}
*/
export function getQueryParams(mappings, type, search, searchFields) {
export function getQueryParams(mappings, type, search, searchFields, extraFilters) {
if (!type && !search) {
return {};
}

const bool = {};

const filters = [];
if (type) {
bool.filter = [
{ term: { type } }
];
filters.push({ term: { type } });
}

if (extraFilters) {
filters.push(...extraFilters);
}

if (filters.length > 0) {
bool.filter = filters;
}

if (search) {
Expand Down
9 changes: 7 additions & 2 deletions src/server/saved_objects/client/lib/search_dsl/search_dsl.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ export function getSearchDsl(mappings, options = {}) {
search,
searchFields,
sortField,
sortOrder
sortOrder,
extraFilters,
} = options;

if (!type && sortField) {
Expand All @@ -20,8 +21,12 @@ export function getSearchDsl(mappings, options = {}) {
throw Boom.notAcceptable('sortOrder requires a sortField');
}

if (extraFilters && !Array.isArray(extraFilters)) {
throw Boom.notAcceptable('extraFilters must be an array');
}

return {
...getQueryParams(mappings, type, search, searchFields),
...getQueryParams(mappings, type, search, searchFields, extraFilters),
...getSortingParams(mappings, type, sortField, sortOrder),
};
}
55 changes: 45 additions & 10 deletions src/server/saved_objects/client/saved_objects_client.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ export class SavedObjectsClient {
async create(type, attributes = {}, options = {}) {
const {
id,
extraBodyProperties = {},
overwrite = false
} = options;

Expand All @@ -120,9 +121,10 @@ export class SavedObjectsClient {
index: this._index,
refresh: 'wait_for',
body: {
...extraBodyProperties,
type,
updated_at: time,
[type]: attributes
[type]: attributes,
},
});

Expand Down Expand Up @@ -167,9 +169,10 @@ export class SavedObjectsClient {
}
},
{
...object.extraBodyProperties,
type: object.type,
updated_at: time,
[object.type]: object.attributes
[object.type]: object.attributes,
}
];
};
Expand Down Expand Up @@ -272,6 +275,7 @@ export class SavedObjectsClient {
sortField,
sortOrder,
fields,
extraFilters,
} = options;

if (searchFields && !Array.isArray(searchFields)) {
Expand All @@ -282,6 +286,10 @@ export class SavedObjectsClient {
throw new TypeError('options.searchFields must be an array');
}

if (extraFilters && !Array.isArray(extraFilters)) {
throw new TypeError('options.extraFilters must be an array');
}

const esOptions = {
index: this._index,
size: perPage,
Expand All @@ -295,7 +303,8 @@ export class SavedObjectsClient {
searchFields,
type,
sortField,
sortOrder
sortOrder,
extraFilters
})
}
};
Expand Down Expand Up @@ -342,7 +351,7 @@ export class SavedObjectsClient {
* { id: 'foo', type: 'index-pattern' }
* ])
*/
async bulkGet(objects = []) {
async bulkGet(objects = [], options = {}) {
if (objects.length === 0) {
return { saved_objects: [] };
}
Expand All @@ -357,8 +366,17 @@ export class SavedObjectsClient {
}
});

const { docs } = response;

let docsToReturn = docs;
if (typeof options.documentFilter === 'function') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should be able to pull this logic up into the spaces_saved_object_client now that we've introduced extraSourceProperties

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome, thanks!

docsToReturn = docs.filter(options.documentFilter);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filters ES results to only show documents for the active Space

}

const { extraSourceProperties = [] } = options;

return {
saved_objects: response.docs.map((doc, i) => {
saved_objects: docsToReturn.map((doc, i) => {
const { id, type } = objects[i];

if (!doc.found) {
Expand All @@ -370,13 +388,21 @@ export class SavedObjectsClient {
}

const time = doc._source.updated_at;
return {
const savedObject = {
id,
type,
...time && { updated_at: time },
version: doc._version,
attributes: doc._source[type]
attributes: {
...extraSourceProperties
.map(s => doc._source[s])
.reduce((acc, prop) => ({ ...acc, ...prop }), {}),

...doc._source[type],
}
};

return savedObject;
})
};
}
Expand All @@ -388,7 +414,7 @@ export class SavedObjectsClient {
* @param {string} id
* @returns {promise} - { id, type, version, attributes }
*/
async get(type, id) {
async get(type, id, options = {}) {
const response = await this._callCluster('get', {
id: this._generateEsId(type, id),
type: this._type,
Expand All @@ -403,14 +429,22 @@ export class SavedObjectsClient {
throw errors.createGenericNotFoundError();
}

const { extraSourceProperties = [] } = options;

const { updated_at: updatedAt } = response._source;

return {
id,
type,
...updatedAt && { updated_at: updatedAt },
version: response._version,
attributes: response._source[type]
attributes: {
...extraSourceProperties
.map(s => response._source[s])
.reduce((acc, prop) => ({ ...acc, ...prop }), {}),

...response._source[type],
}
};
}

Expand All @@ -434,8 +468,9 @@ export class SavedObjectsClient {
ignore: [404],
body: {
doc: {
...options.extraBodyProperties,
updated_at: time,
[type]: attributes
[type]: attributes,
}
},
});
Expand Down
4 changes: 4 additions & 0 deletions x-pack/plugins/spaces/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { initSpacesApi } from './server/routes/api/v1/spaces';
import { initSpacesRequestInterceptors } from './server/lib/space_request_interceptors';
import { mirrorPluginStatus } from '../../server/lib/mirror_plugin_status';
import mappings from './mappings.json';
import { spacesSavedObjectsClientWrapper } from './server/lib/saved_objects_client/saved_objects_client_wrapper';

export const spaces = (kibana) => new kibana.Plugin({
id: 'spaces',
Expand Down Expand Up @@ -55,6 +56,9 @@ export const spaces = (kibana) => new kibana.Plugin({
const config = server.config();
validateConfig(config, message => server.log(['spaces', 'warning'], message));

const savedObjectsClientProvider = server.getSavedObjectsClientProvider();
savedObjectsClientProvider.addClientWrapper(spacesSavedObjectsClientWrapper);

initSpacesApi(server);

initSpacesRequestInterceptors(server);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { SpacesSavedObjectsClient } from './spaces_saved_objects_client';

export function spacesSavedObjectsClientWrapper(baseClient, options) {
return new SpacesSavedObjectsClient({
baseClient,
...options
});
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export class SpacesSavedObjectsClient {
constructor(options) {
const {
request,
baseClient,
spaceUrlContext,
} = options;

this.errors = baseClient.errors;

this._client = baseClient;
this._request = request;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we aren't using this._request elsewhere, do we need to keep a reference to it?


this._spaceUrlContext = spaceUrlContext;
}

async create(type, attributes = {}, options = {}) {

if (this._isTypeSpaceAware(type)) {
options.extraBodyProperties = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this method, and some others, mutate the options that are passed into them, which might not be what the consumers are expecting.

...options.extraBodyProperties,
spaceId: await this._getSpaceId()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we already have the spaceId, we shouldn't need to get it again.

};
}

return await this._client.create(type, attributes, options);
}

async bulkCreate(objects, options = {}) {
options.extraBodyProperties = {
...options.extraBodyProperties,
spaceId: await this._getSpaceId()
};

return await this._client.bulkCreate(objects, options);
}

async delete(type, id) {
return await this._client.delete(type, id);
}

async find(options = {}) {
const spaceOptions = {};

if (this._isTypeSpaceAware(options.type)) {
const spaceId = await this._getSpaceId();
if (spaceId) {
spaceOptions.extraFilters = [{
term: {
spaceId
}
}];
}
}

return await this._client.find({ ...options, ...spaceOptions });
}

async bulkGet(objects = []) {
// ES 'mget' does not support queries, so we have to filter results after the fact.
const thisSpaceId = await this._getSpaceId();

return await this._client.bulkGet(objects, {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a similar fashion to the above, how do you feel about passing in an extraSourceProperties array that we use to copy the additional properties from the document? Then we could just do the filter on the result without adding the extension point.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it! Done!

extraSourceProperties: ['spaceId'],
documentFilter: (doc) => {
if (!doc.found) return true;

const { type, spaceId } = doc._source;

if (this._isTypeSpaceAware(type)) {
return spaceId === thisSpaceId;
}

return true;
}
});
}

async get(type, id) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this also take the options parameter and add the spaceId to the extraSourceProperties if they're specified?

// ES 'get' does not support queries, so we have to filter results after the fact.
let thisSpaceId;

if (this._isTypeSpaceAware(type)) {
thisSpaceId = await this._getSpaceId();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, if we added the extraSourceProperties then we could by default assign that property from the _source to the object and then just filter the response.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it! Done!

}

const response = await this._client.get(type, id, {
extraSourceProperties: ['spaceId']
});

const { spaceId: objectSpaceId } = response.attributes;

if (objectSpaceId !== thisSpaceId) {
throw this._client.errors.createGenericNotFoundError();
}

return response;
}

async update(type, id, attributes, options = {}) {
attributes.spaceId = await this._getSpaceId();
return await this._client.update(type, id, attributes, options);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be safe, let's specify the spaceId here always so that they don't accidentally write the document without a spaceId or to the wrong spaceId.

}

_isTypeSpaceAware(type) {
return type !== 'space';
}

async _getSpaceId() {
if (!this._spaceId) {
const {
saved_objects: spaces = []
} = await this.find({
type: 'space',
search: `"${this._spaceUrlContext}"`,
search_fields: ['urlContext'],
});

if (spaces.length > 0) {
this._spaceId = spaces[0].id;
}
}

return this._spaceId;
}
}