Add `created_by` to saved objects #179344

Dosant · 2024-03-25T13:26:52Z

~~Blocked by #179258~~

Summary

This PR adds an optional created_by field to root saved object fields.
We're doing for adding a filter by an author to the dashboard listing page (and then other listings)

Implementation

`created_by: {type: keyword}`

In this implementation, I added created_by as a simple keyword field assuming we will store profile_uid only

created_by: {
  type: 'keyword',
},

The profile_uid is not always available, as @azasypkin described here
It is not available for anonymous users, for users authenticated via proxy, and, in some cases, for API users authenticated with API keys.
But this is the best way to globally identify users and longer term we might get profile_uid for all the users

Accessing `getCurrentUser` from saved object repo

After exploring different options and discussing with @pgayvallet we decided to provide getCurrentUser through existing security extensions as it's a better isolation of concern, and we avoid leaking the request down to the SOR.

Dosant · 2024-03-25T13:26:59Z

/ci

Dosant · 2024-03-25T13:50:28Z

/ci

…eated-by # Conflicts: # packages/core/saved-objects/core-saved-objects-migration-server-internal/src/__snapshots__/kibana_migrator.test.ts.snap # packages/core/saved-objects/core-saved-objects-migration-server-internal/src/core/__snapshots__/build_active_mappings.test.ts.snap

…eated-by

Dosant · 2024-03-27T12:25:07Z

/ci

…fix'

…to d/2024-03-25-created-by # Conflicts: # packages/core/saved-objects/core-saved-objects-server/tsconfig.json

Dosant · 2024-03-27T12:53:55Z

/ci

Dosant · 2024-03-27T13:39:18Z

/ci

Dosant · 2024-03-27T14:35:45Z

/ci

Dosant · 2024-03-27T15:31:09Z

/ci

Dosant · 2024-03-27T16:17:34Z

/ci

pgayvallet

Overall the PR looks fine. The changes at Core / the SOR level are what I expected, and test coverage was properly added.

~~My only real question is regarding the way we're passing down the getCurrentUser API down to the SOR (see comment)~~

pgayvallet · 2024-04-02T12:43:32Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/apis/helpers/user.ts

+  getCurrentUserId(): string | null {
+    return this.securityExtension?.getCurrentUser()?.profile_uid ?? null;
+  }


Question to @azasypkin regarding the naming. Are userId and profile_uid the same thing? Are we fine with getCurrentUserId here?

We don't have such a thing as userId yet, and I'm somewhat hesitant to introduce a new concept at this point. Considering userProfiles and uids are already exposed through our public APIs, would getCurrentUserProfileUid be that bad? If a larger API surface within SOR/SOC isn't an issue, we can also consider just re-exposing getCurrentUser...

I'll update to getCurrentUserProfileUid

pgayvallet

LGTM

azasypkin · 2024-04-03T09:11:40Z

ACK: will review today

elasticmachine · 2024-04-03T14:08:24Z

Pinging @elastic/kibana-security (Team:Security)

Dosant · 2024-04-03T14:11:19Z

x-pack/plugins/cases/server/services/user_actions/test_utils.ts

@@ -52,7 +52,7 @@ export const createUserActionFindSO = (
 });

 export const createConnectorUserAction = (
-  overrides?: Partial<CaseUserActionWithoutReferenceIds>
+  overrides?: Partial<SavedObject<CaseUserActionWithoutReferenceIds>>


@elastic/response-ops-cases, the overrides from these utils are unused, but adding a created_by on a root saved object found a type issue with the types

cnasikas

Cases changes LGTM!

Dosant · 2024-04-04T10:34:25Z

@elasticmachine merge upstream

…eated-by

jloleysens

Re-approving for core to unblock progress, read through the code and LGTM.

Dosant · 2024-04-04T13:59:26Z

@elasticmachine merge upstream

Dosant · 2024-04-04T14:02:17Z

buildkite test this

…eated-by

…eated-by # Conflicts: # x-pack/plugins/security/tsconfig.json

…eated-by

kibana-ci · 2024-04-04T15:35:06Z

💚 Build Succeeded

Buildkite Build
Commit: 75e9f68

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/core-saved-objects-server`	130	132	+2

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`core`	405.7KB	405.8KB	+59.0B

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/saved-objects-service.html#_mappings

id	before	after	diff
`created_by`	-	1	+1

Unknown metric groups

API count

id	before	after	diff
`@kbn/core-saved-objects-api-browser`	111	112	+1
`@kbn/core-saved-objects-api-server`	353	354	+1
`@kbn/core-saved-objects-server`	555	559	+4
`data`	3283	3285	+2
`dataViews`	1069	1070	+1
total			+9

ESLint disabled line counts

id	before	after	diff
`@kbn/core-saved-objects-api-server-internal`	6	7	+1

References to deprecated APIs

id	before	after	diff
`@kbn/core-saved-objects-api-browser`	25	26	+1
`@kbn/core-saved-objects-browser-internal`	173	174	+1
total			+2

Total ESLint disabled count

id	before	after	diff
`@kbn/core-saved-objects-api-server-internal`	17	18	+1

History

💔 Build #201115 failed 1b194fe
💔 Build #201049 failed 1db38fd
💚 Build #200822 succeeded 4ca1e4a
💚 Build #200775 succeeded fe83ea6
💚 Build #199793 succeeded 4c19cdc
💔 Build #199766 failed fbca670

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @Dosant

) ## Summary Align V2 behavior with ZDT after #179595 Under the assumption that whenever we want to add new root fields (e.g. [created_by](#179344)), we will systematically add mappings for them, we can skip the `updateAndPickupMappings` operation iif ONLY root fields' mappings have changed during an upgrade (aka if none of the SO types have updated their mappings). Up until now, the logic was updating ALL SO types whenever a `root` field was added / updated. This is expensive and unnecessary, and can cause a noticeable impact to large customers during migrations.

Part of elastic/kibana-team#769 ## Summary This PR adds a "created by" filter to the dashboard listing page. We've added the created_by field recently [here](#179344) so only newly created by interactive users dashboards will be populated with `created_by` ### Demo <img width="1510" alt="Screenshot 2024-04-25 at 16 32 03" src="https://github.com/elastic/kibana/assets/7784120/ac70ffea-05be-4d70-bbfa-c1cc9d5dd747"> <img width="1510" alt="Screenshot 2024-04-25 at 16 32 11" src="https://github.com/elastic/kibana/assets/7784120/9889ccae-08d0-4fb6-977e-c42e85337e0e"> ### Implementation notes #### Suggesting users We suggest only the users that have at least one dashboard. We do this by aggregating dashboard saved objects by `created_by`. Because we don't have server-side pagination or sorting yet, in this initial version, I decided to do this client-side to keep it as simple as possible for now. Initially, I did this server-side inside a dashboard plugin as an internal API route. In the long term, this should probably be moved to content management in some form, but it is not clear how it should look yet. This is the commit where I reverted and moved it to client-side 06316ee #### Clint side filtering The user filter is simply a client-side filter because we anyway currently have an in-memory table with client-side sorting and pagination. There is no much practical value in server-side filter atm The filter allows to filter-in by multiple users. It also allows to filter items without creators #### `created_by` filter isn't part of the query string I introduced a separate state for this filter, similar to `tableSort` and `pagination`. The reason why the `created_by` filter isn't a part of the `searchQuery` is because we use `profile_id` which looks ugly for the user, e.g. `u_mGBROF_q5bmFCATLXAcCwKa0k8JvONAwSruelyKA5E_0` so we can't just add it to a user-facing search query like we do with tags. `profile_id` is the correct identified we should use, `username` might not be enough in some cases + `username` in cloud is unreadble account number. The best alternative could be email, but it isn't always available. This is why I decided to keep the `created_by` filter hidden from the user behind the user picker popover and not expose it to the search bar. The `created_by` filter is synced into the URL just like the rest of the table state. #### No distinguishing the current user There is no logic in place that somehow highlights or renames the current user in the filter. This can be added, but to limit the scope of the MVP I didn't do it. #### User filter component We use the `UsersProfilePopover` provided by the security team. We use it without any overrides, so the component decides how to show users in a popover. I noticed a small visual issue with an email truncation in a smaller popover. Might be worth a separate issue. ### Testing 1. Create multiple new users 2. Login for each of them 3. Created a dashboard for each of them 4. Play with the filter #### Next Steps The next step would be to add a column with the user: ![image (5)](https://github.com/elastic/kibana/assets/7784120/9a3fe30e-6a41-4718-a7aa-53fbe0affd22)

Part of https://github.com/elastic/kibana-team/issues/769 ## Summary This PR adds a "created by" filter to the dashboard listing page. We've added the created_by field recently [here](elastic#179344) so only newly created by interactive users dashboards will be populated with `created_by` ### Demo <img width="1510" alt="Screenshot 2024-04-25 at 16 32 03" src="https://github.com/elastic/kibana/assets/7784120/ac70ffea-05be-4d70-bbfa-c1cc9d5dd747"> <img width="1510" alt="Screenshot 2024-04-25 at 16 32 11" src="https://github.com/elastic/kibana/assets/7784120/9889ccae-08d0-4fb6-977e-c42e85337e0e"> ### Implementation notes #### Suggesting users We suggest only the users that have at least one dashboard. We do this by aggregating dashboard saved objects by `created_by`. Because we don't have server-side pagination or sorting yet, in this initial version, I decided to do this client-side to keep it as simple as possible for now. Initially, I did this server-side inside a dashboard plugin as an internal API route. In the long term, this should probably be moved to content management in some form, but it is not clear how it should look yet. This is the commit where I reverted and moved it to client-side elastic@06316ee #### Clint side filtering The user filter is simply a client-side filter because we anyway currently have an in-memory table with client-side sorting and pagination. There is no much practical value in server-side filter atm The filter allows to filter-in by multiple users. It also allows to filter items without creators #### `created_by` filter isn't part of the query string I introduced a separate state for this filter, similar to `tableSort` and `pagination`. The reason why the `created_by` filter isn't a part of the `searchQuery` is because we use `profile_id` which looks ugly for the user, e.g. `u_mGBROF_q5bmFCATLXAcCwKa0k8JvONAwSruelyKA5E_0` so we can't just add it to a user-facing search query like we do with tags. `profile_id` is the correct identified we should use, `username` might not be enough in some cases + `username` in cloud is unreadble account number. The best alternative could be email, but it isn't always available. This is why I decided to keep the `created_by` filter hidden from the user behind the user picker popover and not expose it to the search bar. The `created_by` filter is synced into the URL just like the rest of the table state. #### No distinguishing the current user There is no logic in place that somehow highlights or renames the current user in the filter. This can be added, but to limit the scope of the MVP I didn't do it. #### User filter component We use the `UsersProfilePopover` provided by the security team. We use it without any overrides, so the component decides how to show users in a popover. I noticed a small visual issue with an email truncation in a smaller popover. Might be worth a separate issue. ### Testing 1. Create multiple new users 2. Login for each of them 3. Created a dashboard for each of them 4. Play with the filter #### Next Steps The next step would be to add a column with the user: ![image (5)](https://github.com/elastic/kibana/assets/7784120/9a3fe30e-6a41-4718-a7aa-53fbe0affd22)

## Summary close elastic/kibana-team#899 - Adds `updated_by` to saved object, similar to recently added `created_by` #179344 - Fixes `created_by` / `created_at` should be set during upsert - Improves functional tests coverage

## Summary close elastic/kibana-team#899 - Adds `updated_by` to saved object, similar to recently added `created_by` elastic#179344 - Fixes `created_by` / `created_at` should be set during upsert - Improves functional tests coverage

wip add created_by to saved object

75256fe

Dosant added 4 commits March 26, 2024 11:04

wip created by to so

2ede62f

wip set current user id

1b86b65

Merge branch 'main' of github.com:elastic/kibana into d/2024-03-25-cr…

d09afac

…eated-by

kibanamachine and others added 5 commits March 27, 2024 12:32

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

c85ef73

…fix'

wip

0895088

wip

bfaffa5

Merge branch 'd/2024-03-25-created-by' of github.com:Dosant/kibana in…

2bcb52d

…to d/2024-03-25-created-by # Conflicts: # packages/core/saved-objects/core-saved-objects-server/tsconfig.json

wip

372a679

fix mock

618a33c

fix mock

7db8692

fix types

fbca670

Dosant added 2 commits March 27, 2024 17:14

fix cases overrides

08d980c

fix overrides

4c19cdc

Dosant changed the title ~~wip add created_by to saved object~~ [wip] Add created_by to saved objects Mar 28, 2024

Dosant changed the title ~~[wip] Add created_by to saved objects~~ Add created_by to saved objects Mar 28, 2024

Dosant requested review from pgayvallet and azasypkin March 28, 2024 15:01

pgayvallet reviewed Apr 2, 2024

View reviewed changes

pgayvallet approved these changes Apr 2, 2024

View reviewed changes

Dosant added the Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! label Apr 3, 2024

gsoldevila mentioned this pull request Apr 3, 2024

Do not update+pickup docs if SO types mappings have not changed #179937

Merged

Dosant commented Apr 3, 2024

View reviewed changes

cnasikas approved these changes Apr 4, 2024

View reviewed changes

Merge branch 'main' of github.com:elastic/kibana into d/2024-03-25-cr…

1db38fd

…eated-by

jloleysens approved these changes Apr 4, 2024

View reviewed changes

Dosant added 3 commits April 4, 2024 16:02

Merge branch 'main' of github.com:elastic/kibana into d/2024-03-25-cr…

8ea275e

…eated-by

Merge branch 'main' of github.com:elastic/kibana into d/2024-03-25-cr…

1b194fe

…eated-by # Conflicts: # x-pack/plugins/security/tsconfig.json

Merge branch 'main' of github.com:elastic/kibana into d/2024-03-25-cr…

75e9f68

…eated-by

Dosant merged commit 7d80e4f into elastic:main Apr 4, 2024
18 checks passed

kibanamachine added v8.14.0 backport:skip This commit does not require backporting labels Apr 4, 2024

thomheymann mentioned this pull request Apr 8, 2024

[Observability Onboarding] Removes hover effect from use case selection #180303

Merged

Dosant mentioned this pull request Apr 9, 2024

Dashboard filter by creator #180147

Merged

cnasikas mentioned this pull request Apr 10, 2024

[Cases] Leverage the built in attributes provided by the SO client instead using custom ones #180473

Open

pgayvallet mentioned this pull request Apr 16, 2024

Kibana: Show rule creation date and author #141011

Closed

Heenawter mentioned this pull request May 2, 2024

[Dashboard] Add "Creator" column to dashboard listing page #182256

Merged

Dosant mentioned this pull request May 24, 2024

Add updated_by to saved objects #182687

Merged

azasypkin mentioned this pull request Jun 5, 2024

Consider including the Saved Object's top-level created_by and updated_by properties in the AAD automatically. #184800

Open

nickpeihl mentioned this pull request Oct 16, 2024

Non-interactive sessions leveraging the http auth provider should have their profiles activated #158277

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `created_by` to saved objects #179344

Add `created_by` to saved objects #179344

Dosant commented Mar 25, 2024 •

edited

Loading

Dosant commented Mar 25, 2024

Dosant commented Mar 25, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

pgayvallet left a comment •

edited

Loading

pgayvallet Apr 2, 2024

azasypkin Apr 3, 2024

Dosant Apr 3, 2024

pgayvallet left a comment

azasypkin commented Apr 3, 2024

elasticmachine commented Apr 3, 2024

Dosant Apr 3, 2024

cnasikas left a comment

Dosant commented Apr 4, 2024

jloleysens left a comment

Dosant commented Apr 4, 2024

Dosant commented Apr 4, 2024

kibana-ci commented Apr 4, 2024

API count

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

Add created_by to saved objects #179344

Add created_by to saved objects #179344

Conversation

Dosant commented Mar 25, 2024 • edited Loading

Summary

Implementation

created_by: {type: keyword}

Accessing getCurrentUser from saved object repo

Dosant commented Mar 25, 2024

Dosant commented Mar 25, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

Dosant commented Mar 27, 2024

pgayvallet left a comment • edited Loading

Choose a reason for hiding this comment

pgayvallet Apr 2, 2024

Choose a reason for hiding this comment

azasypkin Apr 3, 2024

Choose a reason for hiding this comment

Dosant Apr 3, 2024

Choose a reason for hiding this comment

pgayvallet left a comment

Choose a reason for hiding this comment

azasypkin commented Apr 3, 2024

elasticmachine commented Apr 3, 2024

Dosant Apr 3, 2024

Choose a reason for hiding this comment

cnasikas left a comment

Choose a reason for hiding this comment

Dosant commented Apr 4, 2024

jloleysens left a comment

Choose a reason for hiding this comment

Dosant commented Apr 4, 2024

Dosant commented Apr 4, 2024

kibana-ci commented Apr 4, 2024

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Page load bundle

Saved Objects .kibana field count

API count

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

History

Add `created_by` to saved objects #179344

Add `created_by` to saved objects #179344

Dosant commented Mar 25, 2024 •

edited

Loading

`created_by: {type: keyword}`

Accessing `getCurrentUser` from saved object repo

pgayvallet left a comment •

edited

Loading