Merge branch 'main' into issue_138402

elastic · Aug 9, 2022 · be526d0 · be526d0
2 parents a7ce28c + 287e3b8
commit be526d0
Show file tree

Hide file tree

Showing 73 changed files with 2,000 additions and 1,055 deletions.
diff --git a/docs/api/osquery-manager.asciidoc b/docs/api/osquery-manager.asciidoc
@@ -0,0 +1,41 @@
+[[osquery-manager-api]]
+== Osquery manager API
+
+experimental[] Run live queries, manage packs and saved queries
+
+WARNING: Use the osquery manager APIs for managing packs and saved queries instead of lower-level <<saved-objects-api, saved objects API>>.
+
+The following osquery manager APIs are available: 
+
+* Live queries
+    ** <<osquery-manager-live-queries-api-get-all, Get all live queries API>> to retrieve a list of live queries
+    ** <<osquery-manager-live-queries-api-get, Get live query API>> to retrieve a single live query
+    ** <<osquery-manager-live-queries-api-create, Create live query API>> to create a live query
+    ** <<osquery-manager-live-queries-api-get-results, Get live query results API>> to retrieve the results of a single live query
+* Packs
+    ** <<osquery-manager-packs-api-get-all, Get all packs API>> to retrieve a list of packs
+    ** <<osquery-manager-packs-api-get, Get pack API>> to retrieve a pack
+    ** <<osquery-manager-packs-api-create, Create pack API>> to create a pack
+    ** <<osquery-manager-packs-api-update, Update pack API>> to partially update an existing pack
+    ** <<osquery-manager-packs-api-delete, Delete pack API>> to delete a pack
+* Saved queries
+    ** <<osquery-manager-saved-queries-api-get-all, Get all saved queries API>> to retrieve a list of saved queries
+    ** <<osquery-manager-saved-queries-api-get, Get saved query API>> to retrieve a saved query
+    ** <<osquery-manager-saved-queries-api-create, Create saved query API>> to create a saved query
+    ** <<osquery-manager-saved-queries-api-update, Update saved query API>> to partially update an existing saved query
+    ** <<osquery-manager-saved-queries-api-delete, Delete saved query API>> to delete a saved query
+
+include::osquery-manager/live-queries/get.asciidoc[]
+include::osquery-manager/live-queries/get-all.asciidoc[]
+include::osquery-manager/live-queries/get-results.asciidoc[]
+include::osquery-manager/live-queries/create.asciidoc[]
+include::osquery-manager/packs/get.asciidoc[]
+include::osquery-manager/packs/get-all.asciidoc[]
+include::osquery-manager/packs/create.asciidoc[]
+include::osquery-manager/packs/update.asciidoc[]
+include::osquery-manager/packs/delete.asciidoc[]
+include::osquery-manager/saved-queries/get.asciidoc[]
+include::osquery-manager/saved-queries/get-all.asciidoc[]
+include::osquery-manager/saved-queries/create.asciidoc[]
+include::osquery-manager/saved-queries/update.asciidoc[]
+include::osquery-manager/saved-queries/delete.asciidoc[]
diff --git a/docs/api/osquery-manager/live-queries/create.asciidoc b/docs/api/osquery-manager/live-queries/create.asciidoc
@@ -0,0 +1,184 @@
+[[osquery-manager-live-queries-api-create]]
+=== Create live query API
+++++
+<titleabbrev>Create live query</titleabbrev>
+++++
+
+experimental[] Create live queries.
+
+
+[[osquery-manager-live-queries-api-create-request]]
+==== Request
+
+`POST <kibana host>:<port>/api/osquery/live_queries`
+
+`POST <kibana host>:<port>/s/<space_id>/api/osquery/live_queries`
+
+
+[[osquery-manager-live-queries-api-create-path-params]]
+==== Path parameters
+
+`space_id`::
+  (Optional, string) An identifier for the space. When `space_id` is not provided in the URL, the default space is used.
+
+
+[[osquery-manager-live-queries-api-create-body-params]]
+==== Request body
+
+`agent_ids`:: (Optional, array) A list of agent IDs to run the query on.
+
+`agent_all`:: (Optional, boolean) When `true`, the query runs on all agents.
+
+`agent_platforms`:: (Optional, array) A list of agent platforms to run the query on.
+
+`agent_policy_ids`:: (Optional, array) A list of agent policy IDs to run the query on.
+
+`query`:: (Optional, string) The SQL query you want to run.
+
+`saved_query_id`:: (Optional, string) The ID of a saved query.
+
+`ecs_mapping`:: (Optional, object) Map osquery results columns or static values to Elastic Common Schema (ECS) fields.
+
+`pack_id`:: (Optional, string) The ID of the pack you want to run.
+
+`alert_ids`:: (Optional, array) A list of alert IDs associated to the live query.
+
+`case_ids`:: (Optional, array) A list of case IDs associated to the live query.
+
+`event_ids`:: (Optional, array) A list of event IDs associated to the live query.
+
+`metadata`:: (Optional, object) Custom metadata object associated to the live query.
+
+
+[[osquery-manager-live-queries-api-create-request-codes]]
+==== Response code
+
+`200`::
+    Indicates a successful call.
+
+
+[[osquery-manager-live-queries-api-create-example]]
+==== Examples
+
+Run a live query on all supported agents:
+
+ TIP: `osquery_manager` integration has to be added to the agent policy.
+
+[source,sh]
+--------------------------------------------------
+$ curl -X POST api/osquery/live_queries \
+{
+  "query": "select * from uptime;",
+
+  "ecs_mapping": {
+    "host.uptime": {
+      "field": "total_seconds"
+    }
+  },
+  "agent_all": true,
+}
+
+--------------------------------------------------
+// KIBANA
+
+
+The API returns the live query object:
+
+[source,sh]
+--------------------------------------------------
+{
+  "data": {
+    "action_id": "3c42c847-eb30-4452-80e0-728584042334",
+    "@timestamp": "2022-07-26T09:59:32.220Z",
+    "expiration": "2022-07-26T10:04:32.220Z", # after this time no more agents will run the query
+    "type": "INPUT_ACTION",
+    "input_type": "osquery",
+    "agent_ids": [],
+    "agent_all": true,
+    "agent_platforms": [],
+    "agent_policy_ids": [],
+    "agents": ["16d7caf5-efd2-4212-9b62-73dafc91fa13"], # stores the actual queried agent IDs 
+    "user_id": "elastic",
+    "metadata": {
+      "execution_context": {
+        "name": "osquery",
+        "url": "/app/osquery/live_queries/new"
+      }
+    },
+    "queries": [
+      {
+        "action_id": "609c4c66-ba3d-43fa-afdd-53e244577aa0", # unique ID of the query, use it when querying the live query API to get the single query results
+        "id": "6724a474-cbba-41ef-a1aa-66aebf0879e2", # ID of the query, doesn't have to be unique
+        "query": "select * from uptime;",
+        "ecs_mapping": {
+          "host.uptime": {
+            "field": "total_seconds"
+          }
+        },
+        "agents": [
+          "16d7caf5-efd2-4212-9b62-73dafc91fa13" # stores the actual queried agent IDs 
+        ]
+      }
+    ]
+  }
+}
+--------------------------------------------------
+
+
+Run a pack on Darwin-supported agents: 
+
+[source,sh]
+--------------------------------------------------
+$ curl -X POST api/osquery/live_queries \
+{
+  "pack_id": "bbe5b070-0c51-11ed-b0f8-ad31b008e832"
+  "agent_platforms": ["darwin"]
+}
+
+--------------------------------------------------
+// KIBANA
+
+The API returns the live query object:
+
+[source,sh]
+--------------------------------------------------
+{
+  "data": {
+    "action_id": "3c42c847-eb30-4452-80e0-728584042334",
+    "@timestamp": "2022-07-26T09:59:32.220Z",
+    "expiration": "2022-07-26T10:04:32.220Z", # after this time no more agents will run the query
+    "type": "INPUT_ACTION",
+    "input_type": "osquery",
+    "agent_ids": [],
+    "agent_all": false,
+    "agent_platforms": ["darwin"],
+    "agent_policy_ids": [],
+    "agents": ["16d7caf5-efd2-4212-9b62-73dafc91fa13"], # stores the actual queried agent IDs 
+    "user_id": "elastic",
+    "pack_id": "bbe5b070-0c51-11ed-b0f8-ad31b008e832",
+    "pack_name": "test_pack",
+    "pack_prebuilt": false,
+    "metadata": {
+      "execution_context": {
+        "name": "osquery",
+        "url": "/app/osquery/live_queries/new"
+      }
+    },
+    "queries": [
+      {
+        "action_id": "609c4c66-ba3d-43fa-afdd-53e244577aa0", # unique ID of the query, use it when querying the live query API to get the single query results
+        "id": "uptime", # ID of the query, doesn't have to be unique
+        "query": "select * from uptime;",
+        "ecs_mapping": {
+          "host.uptime": {
+            "field": "total_seconds"
+          }
+        },
+        "agents": [
+          "16d7caf5-efd2-4212-9b62-73dafc91fa13" # stores the actual queried agent IDs 
+        ]
+      }
+    ]
+  }
+}
+--------------------------------------------------
diff --git a/docs/api/osquery-manager/live-queries/get-all.asciidoc b/docs/api/osquery-manager/live-queries/get-all.asciidoc
@@ -0,0 +1,97 @@
+[[osquery-manager-live-queries-api-get-all]]
+=== Get live queries API
+++++
+<titleabbrev>Get live queries</titleabbrev>
+++++
+
+experimental[] Get live queries.
+
+
+[[osquery-manager-live-queries-api-get-all-request]]
+==== Request
+
+`GET <kibana host>:<port>/api/osquery/live_queries`
+
+`GET <kibana host>:<port>/s/<space_id>/api/osquery/live_queries`
+
+
+[[osquery-manager-live-queries-api-get-all-params]]
+==== Path parameters
+
+`space_id`::
+(Optional, string) An identifier for the space. When `space_id` is not provided in the URL, the default space is used.
+
+
+=== Query parameters
+
+`page`::
+(Optional, integer) The page number to return. The default is `1`.
+
+`perPage`::
+(Optional, integer) The number of rules to return per page. The default is `20`.
+
+`sortField`::
+(Optional, string) The field that is used to sort the results. Options include `createdAt` or `updatedAt`.
+The default is `createdAt`.
++
+NOTE: Even though the JSON case object uses `created_at` and `updated_at`
+fields, you must use `createdAt` and `updatedAt` fields in the URL
+query.
+
+`sortOrder`::
+(Optional, string) Specified the sort order. Options include `desc` or `asc`.
+The defaults is `desc`.
+
+
+[[osquery-manager-live-queries-api-get-all-codes]]
+==== Response code
+
+`200`::
+Indicates a successful call.
+
+
+[[osquery-manager-live-queries-api-get-all-example]]
+==== Example
+
+Retrieve the last 10 live queries :
+
+[source,sh]
+--------------------------------------------------
+$ curl -X GET api/osquery/live_queries?page=1&perPage=10
+--------------------------------------------------
+// KIBANA
+
+The API returns a JSON object of the retrieved live queries:
+
+[source,sh]
+--------------------------------------------------
+{
+  "page": 1,
+  "per_page": 10,
+  "total": 11,
+  "data": [
+    {
+      "action_id": "3c42c847-eb30-4452-80e0-728584042334",
+      "expiration": "2022-07-26T10:04:32.220Z",
+      "@timestamp": "2022-07-26T09:59:32.220Z",
+      "agents": ["16d7caf5-efd2-4212-9b62-73dafc91fa13"],
+      "user_id": "elastic",
+      "queries": [
+        {
+          "action_id": "609c4c66-ba3d-43fa-afdd-53e244577aa0",
+          "id": "6724a474-cbba-41ef-a1aa-66aebf0879e2",
+          "query": "select * from uptime;",
+          "saved_query_id": "42ba9c50-0cc5-11ed-aa1d-2b27890bc90d",
+          "ecs_mapping": {
+            "host.uptime": {
+              "field": "total_seconds"
+            }
+          },
+          "agents": ["16d7caf5-efd2-4212-9b62-73dafc91fa13"],
+        }
+      ],
+    },
+    {...}
+  ]
+}
+--------------------------------------------------
diff --git a/docs/api/osquery-manager/live-queries/get-results.asciidoc b/docs/api/osquery-manager/live-queries/get-results.asciidoc
@@ -0,0 +1,64 @@
+[[osquery-manager-live-queries-api-get-results]]
+=== Get live query results API
+++++
+<titleabbrev>Get live query results</titleabbrev>
+++++
+
+experimental[] Retrieve a single live query result by ID.
+
+
+[[osquery-manager-live-queries-api-get-results-request]]
+==== Request
+
+`GET <kibana host>:<port>/api/osquery/live_queries/<id>/results/<query_action_id>`
+
+`GET <kibana host>:<port>/s/<space_id>/api/osquery/live_queries/<query_action_id>`
+
+
+[[osquery-manager-live-queries-api-get-results-params]]
+==== Path parameters
+
+`space_id`::
+(Optional, string) The space identifier. When `space_id` is not provided in the URL, the default space is used.
+
+`id`::
+(Required, string) The ID of the live query result you want to retrieve.
+
+`query_action_id`::
+(Required, string) The ID of the query action that generated the live query results.
+
+
+
+[[osquery-manager-live-queries-api-get-results-codes]]
+==== Response code
+
+`200`::
+Indicates a successful call.
+
+`404`::
+The specified live query or <query_action_id> doesn't exist.
+
+
+[[osquery-manager-live-queries-api-get-results-example]]
+==== Example
+
+Retrieve the live query results for `3c42c847-eb30-4452-80e0-728584042334` ID and `609c4c66-ba3d-43fa-afdd-53e244577aa0` query action ID:
+
+
+[source,sh]
+--------------------------------------------------
+$ curl -X GET api/osquery/live_queries/3c42c847-eb30-4452-80e0-728584042334/results/609c4c66-ba3d-43fa-afdd-53e244577aa0
+--------------------------------------------------
+// KIBANA
+
+The API returns a live query action single query result:
+
+[source,sh]
+--------------------------------------------------
+{
+  "data": {
+    "total": 2,
+    "edges": [{...}, {...}],
+  }
+}
+--------------------------------------------------