[docs] API key documentation from libbeat (#3227) (#3263)

docs/copied-from-beats/docs/monitoring/monitoring-internal-collection-legacy.asciidoc

@@ -0,0 +1,38 @@
+//////////////////////////////////////////////////////////////////////////
+//// This content is shared by all Elastic Beats. Make sure you keep the
+//// descriptions here generic enough to work for all Beats that include
+//// this file. When using cross references, make sure that the cross
+//// references resolve correctly for any files that include this one.
+//// Use the appropriate variables defined in the index.asciidoc file to
+//// resolve Beat names: beatname_uc and beatname_lc.
+//// Use the following include to pull this content into a doc file:
+//// include::../../libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc[]
+//////////////////////////////////////////////////////////////////////////
+
+[role="xpack"]
+[[monitoring-internal-collection-legacy]]
+== Use legacy internal collection to send monitoring data
+++++
+<titleabbrev>Legacy internal collection (deprecated)</titleabbrev>
+++++
+
+deprecated[7.2.0]
+
+In {beatname_uc} version 7.1 and earlier, you configured internal collectors
+that sent monitoring data to the production cluster, which would either index
+the data locally, or forward the data to a dedicated monitoring cluster via HTTP
+exporters.
+
+Starting in {beatname_uc} version 7.2, the legacy settings for internal
+collection are deprecated and will be removed in version 8.0.0. Instead of
+sending monitoring data to your production cluster, it's recommended that you
+use the configuration described under
+<<monitoring-internal-collection,internal collection>> to route
+monitoring data directly to your monitoring cluster.
+
+ifndef::serverless[]
+Or as an alternative to internal collection, use
+<<monitoring-metricbeat-collection>>.
+endif::[]
+
+include::shared-monitor-config-legacy.asciidoc[]

docs/copied-from-beats/docs/monitoring/monitoring-internal-collection.asciidoc

@@ -11,29 +11,30 @@
 
 [role="xpack"]
 [[monitoring-internal-collection]]
-== Collect {beatname_uc} monitoring data with internal collectors
+== Use internal collection to send monitoring data
 ++++
 <titleabbrev>Internal collection</titleabbrev>
 ++++
 
-The following method involves sending the metrics to the production cluster, 
-which ultimately routes them to the monitoring cluster.
+Use internal collectors to send {beats} monitoring data directly to your
+monitoring cluster.
 ifndef::serverless[]
-For an alternative method, see <<monitoring-metricbeat-collection>>.
+Or as an alternative to internal collection, use
+<<monitoring-metricbeat-collection>>. The benefit of using internal collection
+instead of {metricbeat} is that you have fewer pieces of software to install
+and maintain.
 endif::[]
 
-To learn about monitoring in general, see 
-{ref}/monitor-elasticsearch-cluster.html[Monitoring a cluster]. 
+//Commenting out this link temporarily until the general monitoring docs can be
+//updated. 
+//To learn about monitoring in general, see 
+//{ref}/monitor-elasticsearch-cluster.html[Monitor a cluster]. 
 
-//TODO: Not sure if these docs need to be updated to be parallel with other
-//stack components since this is the old way of configuring monitoring. 
-
-. Create a user that has appropriate authority to send system-level monitoring
+. Create an API key or user that has appropriate authority to send system-level monitoring
 data to {es}. For example, you can use the built-in +{beat_monitoring_user}+ user or
 assign the built-in +{beat_monitoring_user}+ role to another user. For more
-information, see
-{ref}/setting-up-authentication.html[User authentication] and
-{ref}/built-in-roles.html[Built-in roles].
+information on the required privileges, see <<privileges-to-publish-monitoring>>.
+For more information on how to use API keys, see <<beats-api-keys>>.
 
 . Add the `monitoring` settings in the {beatname_uc} configuration file. If you
 configured the {es} output and want to send {beatname_uc} monitoring events to
@@ -44,9 +45,23 @@ the same {es} cluster, specify the following minimal configuration:
 monitoring:
   enabled: true
   elasticsearch:
+    api_key:  id:api_key <1>
     username: {beat_monitoring_user}
     password: somepassword
 --------------------
+<1> Specify one of `api_key` or `username`/`password`.
++
+If you want to send monitoring events to an https://cloud.elastic.co/[{ecloud}]
+monitoring cluster, you can use two simpler settings. When defined, these settings
+overwrite settings from other parts in the configuration. For example:
++
+[source,yaml]
+--------------------
+monitoring:
+  enabled: true
+  cloud.id: 'staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWM2ZjI2MWE3NGJmMjRjZTMzYmI4ODExYjg0Mjk0ZiRjNmMyY2E2ZDA0MjI0OWFmMGNjN2Q3YTllOTYyNTc0Mw=='
+  cloud.auth: 'elastic:{pwd}'
+--------------------
 +
 If you
 ifndef::no-output-logstash[]
@@ -63,6 +78,7 @@ monitoring:
   cluster_uuid: PRODUCTION_ES_CLUSTER_UUID <1>
   elasticsearch:
     hosts: ["https://example.com:9200", "https://example2.com:9200"] <2>
+    api_key:  id:api_key <3>
     username: {beat_monitoring_user}
     password: somepassword
 --------------------
@@ -72,6 +88,29 @@ Stack Monitoring UI. To get a cluster's `cluster_uuid`,
 call the `GET /` API against that cluster.
 <2> This setting identifies the hosts and port numbers of {es} nodes
 that are part of the monitoring cluster.
+<3> Specify one of `api_key` or `username`/`password`.
++
+If you want to use PKI authentication to send monitoring events to
+{es}, you must specify a different set of configuration options. For
+example:
++
+[source,yaml]
+--------------------
+monitoring:
+  enabled: true
+  cluster_uuid: PRODUCTION_ES_CLUSTER_UUID
+  elasticsearch:
+    hosts: ["https://example.com:9200", "https://example2.com:9200"]
+    username: ""
+    ssl:
+      ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
+      ssl.certificate: "/etc/pki/client/cert.pem"
+      ssl.key: "/etc/pki/client/cert.key"
+--------------------
++
+You must specify the `username` as `""` explicitly so that
+the username from the client certificate (`CN`) is used. See
+<<configuration-ssl>> for more information about SSL settings.
 
 ifndef::serverless[]
 . Start {beatname_uc}.

docs/copied-from-beats/docs/monitoring/monitoring-metricbeat.asciidoc

@@ -1,14 +1,15 @@
 [role="xpack"]
 [[monitoring-metricbeat-collection]]
-== Collect {beatname_uc} monitoring data with {metricbeat}
+== Use {metricbeat} to send monitoring data
 [subs="attributes"]
 ++++
 <titleabbrev>{metricbeat} collection</titleabbrev>
 ++++
 
 In 7.3 and later, you can use {metricbeat} to collect data about {beatname_uc} 
-and ship it to the monitoring cluster, rather than routing it through the 
-production cluster as described in <<monitoring-internal-collection>>.
+and ship it to the monitoring cluster. The benefit of using {metricbeat} instead
+of internal collection is that the monitoring agent remains active even if the
+{beatname_uc} instance dies.
 
 ifeval::["{beatname_lc}"=="metricbeat"]
 Because you'll be using {metricbeat} to _monitor_ {beatname_uc}, you'll need to
@@ -25,8 +26,10 @@ concurrently. If you don't want to run two instances concurrently, use
 {metricbeat}.
 endif::[]
 
-To learn about monitoring in general, see 
-{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
+//Commenting out this link temporarily until the general monitoring docs can be
+//updated. 
+//To learn about monitoring in general, see 
+//{ref}/monitor-elasticsearch-cluster.html[Monitor a cluster].
 
 //NOTE: The tagged regions are re-used in the Stack Overview.
 
@@ -174,7 +177,7 @@ it via HTTPS. For example, use a `hosts` setting like `https://localhost:5066`.
 If the Elastic {security-features} are enabled, you must also provide a user 
 ID and password so that {metricbeat} can collect metrics successfully: 
 
-.. Create a user on the production cluster that has the 
+.. Create a user on the {es} cluster that has the 
 `remote_monitoring_collector` {ref}/built-in-roles.html[built-in role]. 
 Alternatively, if it's available in your environment, use the
 `remote_monitoring_user` {ref}/built-in-users.html[built-in user].

docs/copied-from-beats/docs/monitoring/shared-monitor-config.asciidoc

@@ -6,13 +6,13 @@
 //// Use the appropriate variables defined in the index.asciidoc file to
 //// resolve Beat names: beatname_uc and beatname_lc.
 //// Use the following include to pull this content into a doc file:
-//// include::../../libbeat/docs/monitoring/configuring.asciidoc[]
+//// include::../../libbeat/docs/monitoring/shared-monitor-config.asciidoc[]
 //// Make sure this content appears below a level 2 heading.
 //////////////////////////////////////////////////////////////////////////
 
 [role="xpack"]
 [[configuration-monitor]]
-=== Settings for internal monitoring collection
+=== Settings for internal collection
 
 Use the following settings to configure internal collection when you are not
 using {metricbeat} to collect monitoring data.
@@ -32,92 +32,97 @@ The default value is `false`.
 The {es} instances that you want to ship your {beatname_uc} metrics to. This
 configuration option contains the following fields:
 
-==== `bulk_max_size`
+===== `api_key`
+
+The detail of the API key to be used to send monitoring information to {es}.
+See <<beats-api-keys>> for more information.
+
+===== `bulk_max_size`
 
 The maximum number of metrics to bulk in a single {es} bulk API index request.
 The default is `50`. For more information, see <<elasticsearch-output>>.
 
-==== `backoff.init`
+===== `backoff.init`
 
 The number of seconds to wait before trying to reconnect to Elasticsearch after
 a network error. After waiting `backoff.init` seconds, {beatname_uc} tries to
 reconnect. If the attempt fails, the backoff timer is increased exponentially up
 to `backoff.max`. After a successful connection, the backoff timer is reset. The
 default is 1s.
 
-==== `backoff.max`
+===== `backoff.max`
 
 The maximum number of seconds to wait before attempting to connect to
 Elasticsearch after a network error. The default is 60s.
 
-==== `compression_level`
+===== `compression_level`
 
 The gzip compression level. Setting this value to `0` disables compression. The
 compression level must be in the range of `1` (best speed) to `9` (best
 compression). The default value is `0`. Increasing the compression level
 reduces the network usage but increases the CPU usage.
 
-==== `headers`
+===== `headers`
 
 Custom HTTP headers to add to each request. For more information, see
 <<elasticsearch-output>>.
 
-==== `hosts`
+===== `hosts`
 
 The list of {es} nodes to connect to. Monitoring metrics are distributed to
 these nodes in round robin order. For more information, see
 <<elasticsearch-output>>.
 
-==== `max_retries`
+===== `max_retries`
 
 The number of times to retry sending the monitoring metrics after a failure.
 After the specified number of retries, the metrics are typically dropped. The
 default value is `3`. For more information, see <<elasticsearch-output>>.
 
-==== `parameters`
+===== `parameters`
 
 Dictionary of HTTP parameters to pass within the url with index operations.
 
-==== `password`
+===== `password`
 
 The password that {beatname_uc} uses to authenticate with the {es} instances for
 shipping monitoring data.
 
-==== `metrics.period`
+===== `metrics.period`
 
 The time interval (in seconds) when metrics are sent to the {es} cluster. A new
 snapshot of {beatname_uc} metrics is generated and scheduled for publishing each
 period. The default value is 10 * time.Second.
 
-==== `state.period`
+===== `state.period`
 
 The time interval (in seconds) when state information are sent to the {es} cluster. A new
 snapshot of {beatname_uc} state is generated and scheduled for publishing each
 period. The default value is 60 * time.Second.
 
-==== `protocol`
+===== `protocol`
 
 The name of the protocol to use when connecting to the {es} cluster. The options
 are: `http` or `https`. The default is `http`. If you specify a URL for `hosts`,
 however, the value of protocol is overridden by the scheme you specify in the URL.
 
-==== `proxy_url`
+===== `proxy_url`
 
 The URL of the proxy to use when connecting to the {es} cluster. For more
 information, see <<elasticsearch-output>>.
 
-==== `timeout`
+===== `timeout`
 
 The HTTP request timeout in seconds for the {es} request. The default is `90`.
 
-==== `ssl`
+===== `ssl`
 
 Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer
 (SSL) parameters like the certificate authority (CA) to use for HTTPS-based
 connections. If the `ssl` section is missing, the host CAs are used for
 HTTPS connections to {es}. For more information, see <<configuration-ssl>>.
 
-==== `username`
+===== `username`
 
 The user ID that {beatname_uc} uses to authenticate with the {es} instances for
 shipping monitoring data.