Add privilege for script actions

[spinscale]

Describe the feature: The manage_watcher privilege right now only contains the "cluster:admin/xpack/watcher/*", "cluster:monitor/xpack/watcher/*" actions.

I do think it makes sense to also add cluster:admin/script/* (get/put/delete scripts) in there as well, as a watcher admin may need to refer to external scripts.

If you think that makes sense, I'm happy to open a PR for this.

[elasticmachine]

Pinging @elastic/es-security (:Security/Authorization)

[albertzaharovits]

Rather than adding new permitted actions to the manage_watcher privilege, I think a new privilege such as manage_scripts is more consistent with the rest of our privileges namespace.
In this case the role for the watcher user should have both manage_watcher and manage_scripts privilege.

[P1llus]

@albertzaharovits

Do you mean that manage_scripts should be a global permission, and not restricted to watcher?

Would you then also want to support more granular permission, so scripts can only be executed in single or multiple places (like watches)?

[albertzaharovits]

@P1llus I originally meant to create a new cluster privilege to be able to use the GET/PUT/DELETE script APIs.

What you are suggesting, being able to confine the usage of scripts to certain contexts, is a different thing. But I think this is an interesting proposal. I envisage being able to define a list of context names when PUTing the script. I wonder what folks from @elastic/es-core-features think about this?

[P1llus]

@albertzaharovits

The reason the idea came up, is just like @spinscale mentioned, when the case is external scripts, just to minimize the possibility of some sort of vulnerability.
Now I have not fully set myself into the capabilities of the external scripts engine, what painless allows and not etc, but If there is any possibilities that the script can be any sort of malicious, being able to have more granular control would be beneficial.

While context level would be great, if that is a really big task, then we should at least be able to give access based on script type maybe?

[albertzaharovits]

We discussed this in our weekly team meeting.

We have decided on adding a manage_scripts cluster privilege that would permit usage of the GET/PUT/DELETE script APIs. This means the manage_watcher privilege is not changed. It also means APIs can use all the scripts internally, without requiring any extra privilege (nothing changes in this regard).

What @P1llus is suggesting, to have fine-grained privilege on scripts, and enforce those privileges whenever a script is used, is difficult to achieve at the moment and would have a broad impact (it would be a breaking change). However, this is an interesting proposal that should be explored as part of the work to add the new manage_scripts privilege.

[albertzaharovits]

@bytebilly offered to take the temperature of the requirement for fine-grained script privileges, as we currently don't have a clear idea about the solutions benefiting from such a privilege.

[P1llus]

@albertzaharovits Thanks for following this up! :)

[bytebilly]

Sorry for the late update here: as of today, I got no evidence that fine-grained access is required. I'll keep an eye on that and will update here in case something will come out.

[spinscale]

@bytebilly thanks a ton for digging into this! Just to be sure I understand your answer correctly in the context of my initial question: Your answer only refers to the requirement for fine grained privileges, but not for the ask to be able to manage scripts as an watcher admin (which still is an open issue then to have a manage_scripts privilege), correct?

[bytebilly]

@spinscale yes that's the case.

We don't have scheduled it yet, given the long list of current priorities.
Let us know if you feel that there are compelling reasons to push it now, happy to evaluate again.

[jimczi]

We plan to use the execute painless API when creating a Runtime Fields in the Index Pattern Editor. Most of the users that will interact with Runtime Fields at this stage are not eligible for the manage privilege so we'll need another solution. manage_scripts seems a better fit.
We should also add this privilege to the Kibana internal user to make the usage of this API transparent for users in this context.
@bytebilly Can we re-evaluate the priority regarding the need for Runtime Fields ?

[bytebilly]

@jimczi do we need full control over scripts (add, remove, etc) or just execute? I'm not sure we want to grant full control over every stored script (as it would be all or nothing) to any user.

Do we need to add it just to kibana_system, and use it after performing sanity checks on user input? In this case, we could just grant cluster:admin/scripts/painless/execute without waiting for a new "public" role.

What do you think?

[jimczi]

Do we need to add it just to kibana_system, and use it after performing sanity checks on user input? In this case, we could just grant cluster:admin/scripts/painless/execute without waiting for a new "public" role.

+1, @javanna @jdconrad do you think it's ok to expect Kibana to always use the kibana_system to call the execute API ?

[javanna]

Runtime fields don't support stored scripts, so no need for add and remove privilege on our end. The plan of granting painless/execute role to kibana_system sounds good to me.

[jdconrad]

+1 to what @javanna has already said - there shouldn't be anything dangerous about running these scripts through the execute API

[tvernum]

We plan to use the execute painless API when creating a Runtime Fields in the Index Pattern Editor.

Can you clarify how we intend to use this?
I take it that Kibana will construct the execute request in order to verify the accuracy (*) of a runtime field expression that the user enters in the editor.

(*) Would that checking be just for compilation, or will it submit a sample document, or something else?

My concern in the more general case, is that the execute API allow access to specific index metadata (existence of indices, mappings, etc) which Kibana system can access, but the user themselves may not have access to. We need to be sure that we're not opening a new path for the user to escalate their access.

[jdconrad]

@tvernum The execute API takes in an example document created only for this request, but it also takes in an existing index name and pulls mappings from live indexes. If this is a security risk we need to re-think how we take in mappings. Painless Lab currently also uses this same strategy for score and filter contexts.

I know in the future we would like to provide a doc id and pull that data so the user doesn't have to create an entire example doc -- but maybe this would be easier/better from the Kibana side as a separate request to fill in that data.

Edit: I wonder if Kibana should pull in mappings as a separate request and then the execute api should process them in the same way as the strategy for grabbing an existing document.

[LeeDr]

FYI, Kibana now has runtime field preview in 7.15.0 but I get an error with my Okta account;

{"values":[],"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:admin/scripts/painless/execute]
 is unauthorized for user [xxxxxxxxxxxxxxxxxxx] with roles 
[watcher_admin,monitoring_user,kibana_user,reporting_user,kibana_admin,everyone-read], 
this action is granted by the cluster privileges [manage,all]"}],
"type":"security_exception","reason":"action [cluster:admin/scripts/painless/execute]
 is unauthorized for user [xxxxxxxxxxxxxxxxxxx] with roles 
[watcher_admin,monitoring_user,kibana_user,reporting_user,kibana_admin,everyone-read], 
this action is granted by the cluster privileges [manage,all]"},"status":403}

Runtime fields don't support stored scripts, so no need for add and remove privilege on our end. The plan of granting painless/execute role to kibana_system sounds good to me.

Did we do this? Or still need to?

[LeeDr]

btw, this is kind of blocking me on one of our internal clusters. It would be great if we could resolve it in 7.15.1.

[bytebilly]

@LeeDr probably the simplest short-term fix to your problem is adding cluster:admin/scripts/painless/execute to your permissions.
A different approach would require changing the way Kibana performs the query, I'm not sure if this is something that they are still evaluating.

[jimczi]

@bytebilly @LeeDr , we have a plan for this but it felt through the cracks :(.
Search requests are allowed to execute a script on an index so we'd like to change the painless execute API to also require the read permission if an index is provided, and nothing else. I'll open a PR soon to expose the approach.

[LeeDr]

Thanks @jimczi. I'm not sure if the change will be something appropriate for a patch release, but 7.15.1 FF is about a week away. If we can't get it in there, or don't feel the change is appropriate for a patch release, I'll ask Infra to add cluster:admin/scripts/painless/execute to the role that all normal users (like me) get.

Also, in case anybody wonders if there's a work-around for adding or editing a runtime field without this privilege, I did look into that. I figured I could use examples from https://www.elastic.co/guide/en/elasticsearch/reference/master/runtime-mapping-fields.html in the Dev Tools Console to add or replace the runtime field (without hitting the Preview path which I don't have privs for). But the Elasticsearch examples are adding runtime fields to an index mapping and that's not how Kibana does it (AFAIK).

I think Kibana saves the runtime field in the Kibana index-pattern saved object and sends the query in searches each time.
So it seems I could use the Kibana saved object API to write the index pattern. But I can't do that in the Dev Tools Console (which only accesses Elasticsearch). So I would have to use curl or a Cloud console...

Or I could execute a command in the Dev Tools Console to modify the index-pattern in the .kibana index.
While I'm sure that's possible, it seems a bit more complicated with risk of breaking something. I could probably create a temp Space in Kibana and try modifying the index-pattern in that space to avoid breaking anybody else...

[jimczi]

@LeeDr let's move the discussion to Kibana. It seems that the creation of runtime field is broken so we should have an issue there. @sebelga must know more about a possible workaround for 7.15.

[mattkime]

I'd like to revisit this as kibana is still unable to use the execute api. Granting privileges to the kibana_system user would cause problems with field level security. I'm not completely certain how access and roles align on the ES side with kibana needs, but if a kibana user can use runtime fields then they should have access to the execute api.

There will be another api created which with take a painless script and return the a field list - part of the UX for creating composite runtime fields. This new api will have similar needs.

[bytebilly]

but if a kibana user can use runtime fields then they should have access to the execute api.

Could you please expand this a bit more? What do you mean by "can use"? I think that a user should be able to execute scripts when adding runtime fields, but not when just "using" them (e.g. getting results), as they are transparent. Is that what you meant?

[sixstringcode]

We have the need for admins to disable runtime field authoring in Kibana. Does it make sense to set that up as a privilege all users have by default which allows both execute for that user, and the authoring UX in Kibana. Admins can disable the privilege globally for Kibana, or by role.

[mattkime]

Could you please expand this a bit more?

Sorry, perhaps 'add runtime fields' makes more sense

[bytebilly]

We have the need for admins to disable runtime field authoring in Kibana.

Which is the main reason for this access control? It is about adding runtime fields to the mapping, or more broadly using them in search queries too? Since these features are available at the Elasticsearch API level, blocking at Kibana level may not be the best approach.

We also don't provide privileges that are automatically granted to all users, or "negative" privileges. This would make it more complex to leverage the current privilege model to forbid the use of runtime fields.

[mattkime]

It might be best to take a step back and have a more thorough conversation about this.

• Users have requested the ability to disable runtime fields - I'm not sure if you're addressing that and how it should be addressed in concert with kibana. This is in line with goals to make ES / kibana more stable.
• Users who can create runtime fields within kibana - which are run at query time - need access to execute api. This is mostly managed on the kibana side as the privileged is really regarding whether data view saved objects can have runtime field definitions appended to them but the UI to determine if a painless script is valid relies on the execute api.

[bytebilly]

I think that we should split this issue into separate — even if related — discussions:

1. ability to manage stored scripts (e.g. a manage_script new privilege)
2. define per-context permissions to execute scripts (allow on runtime fields, forbid on watcher)
3. how to use the execute API to test scripts before using them (e.g. during runtime fields authoring)
4. disable runtime fields

We agreed to implement 1 (even if it has not been prioritized), and that 2 is not necessary today.

For the execute call (3), AFAIK it only impacts creating a runtime field via the Data Views UI, is it correct? Can we use the Kibana service account to call the API, while fetching the test doc with user credentials to guarantee proper access? Do we have security concerns that the script can leverage Kibana privileges to do something potentially not needed (since the script is user-provided)?

For 4, do we want to prevent new runtime fields to be added to the Data View, to the index, or to be executed during searches?

[sixstringcode]

@bytebilly for #4, the customer concern centers around the performance impact of users who create Runtime Fields in Kibana without understanding the implications. I took that as concern for users creating them in the UX without knowing the impact. Runtime Fields fields living within the data view potentially increases the overhead for everyone. I agree with separating the issues, but that is why I was conflating them. I don't think we want a kill switch for any runtime field coming from Kibana, we want to give administrators the power to control who can author persisted runtime fields.

[mattkime]

I don't think we want a kill switch for any runtime field coming from Kibana, we want to give administrators the power to control who can author persisted runtime fields.

If that the case then there's no reason to involve ES.

[mattkime]

Can we use the Kibana service account to call the API, while fetching the test doc with user credentials to guarantee proper access?

I can't take responsibility for the implications of a design decision like that on the ES side. If you tell me that it can't inadvertently result in privileged escalation then I'll believe you.

[fludo]

Would need this specific privilege to allow user to add script without having 'manage,all' privilege.

[daveneeley]

Would need this specific privilege to allow user to add script without having 'manage,all' privilege.

Yeah, we're interested in the same thing. We want to give engineers access to dev tools and the painless lab but only for the indices that they can already access. In our case they use painless scripts for building and testing watchers.