Refactor Prometheus metric mappings #9948
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| FROM prom/prometheus:v2.6.0 | ||
| HEALTHCHECK --interval=1s --retries=90 CMD nc -w 1 localhost 9090 </dev/null | ||
| EXPOSE 9090 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,2 @@ | ||
| This module periodically scrapes metrics from | ||
| https://prometheus.io/docs/[Prometheus]. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,20 @@ | ||
| - key: prometheus | ||
| title: "Prometheus" | ||
| description: > | ||
| Stats scraped from a Prometheus endpoint. | ||
| short_config: false | ||
| release: ga | ||
| settings: ["ssl", "http"] | ||
| fields: | ||
| # Order is important here, labels will match first, the rest are double | ||
|
There was a problem hiding this comment. What about placing labels in the base level to avoid possible issues with this? This is also the recommendation in ECS. There was a problem hiding this comment. I would prefer to keep it like this for now, main reasons are:
I want to make sure this module won't end on mapping issues and we can use these metrics reliably, not against this change once the problem is gone (for instance, if/when index per module is implemented). WDYT? There was a problem hiding this comment. LGTM, but I wonder some things (not blocking this PR)
Why would we want to do it here?
I guess this is no different with any other module, maybe we shouldn't have a field for labels in ECS till we don't have a complete solution for these mapping issues. There was a problem hiding this comment.
That could make sense, I still have some doubts on how ECS will play out in cases like this, I would prefer to play it safe on this module and learn from the process with others 😇 |
||
| - name: prometheus.labels.* | ||
| type: object | ||
| object_type: keyword | ||
| description: > | ||
| Prometheus metric labels | ||
| - name: prometheus.* | ||
| type: object | ||
| object_type: double | ||
|
|
||
| object_type_mapping_type: "*" | ||
| description: > | ||
| Prometheus metric | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,25 +1,30 @@ | ||
| { | ||
| "@timestamp": "2017-10-12T08:05:34.853Z", | ||
| "agent": { | ||
| "hostname": "host.example.com", | ||
| "name": "host.example.com" | ||
| }, | ||
| "event": { | ||
| "dataset": "prometheus.collector", | ||
| "duration": 115000, | ||
| "module": "prometheus" | ||
| }, | ||
| "metricset": { | ||
| "name": "collector" | ||
| }, | ||
| "prometheus": { | ||
| "http_request_duration_microseconds_count": 0, | ||
|
There was a problem hiding this comment. An alternative here would have been |
||
| "http_request_duration_microseconds_sum": 0, | ||
| "http_request_size_bytes_count": 0, | ||
| "http_request_size_bytes_sum": 0, | ||
| "http_response_size_bytes_count": 0, | ||
| "http_response_size_bytes_sum": 0, | ||
| "labels": { | ||
| "handler": "drop_series" | ||
| } | ||
| }, | ||
| "service": { | ||
| "address": "172.29.0.2:9090", | ||
| "type": "prometheus" | ||
|
There was a problem hiding this comment. This is an interesting one. We should probably make it configurable so the user can set service.type and service.name. |
||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,43 @@ | ||
| The Prometheus `collector` metricset scrapes data from https://prometheus.io/docs/instrumenting/exporters/[prometheus exporters]. | ||
|
|
||
|
|
||
| [float] | ||
| === Scraping from a Prometheus exporter | ||
|
|
||
| To scrape metrics from a Prometheus exporter, configure the `hosts` field to it. The path | ||
| to retrieve the metrics from (`/metrics` by default) can be configured with `metrics_path`. | ||
|
|
||
| [source,yaml] | ||
| ------------------------------------------------------------------------------------- | ||
| - module: prometheus | ||
| period: 10s | ||
| hosts: ["localhost:9090"] | ||
| metrics_path: /metrics | ||
|
|
||
| #username: "user" | ||
| #password: "secret" | ||
|
|
||
| # This can be used for service account based authorization: | ||
| #bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token | ||
| #ssl.certificate_authorities: | ||
| # - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt | ||
| ------------------------------------------------------------------------------------- | ||
|
|
||
|
|
||
| [float] | ||
| === Scraping all metrics from a Prometheus server | ||
|
|
||
| This module can scrape all metrics stored in a Prometheus server, by using the | ||
| https://prometheus.io/docs/prometheus/latest/federation/[federation API]. By pointing this | ||
| config to the Prometheus server: | ||
|
|
||
| [source,yaml] | ||
| ------------------------------------------------------------------------------------- | ||
| metricbeat.modules: | ||
| - module: prometheus | ||
|
There was a problem hiding this comment. This is kind of like a second "flawor" of the metricset. In the future we potentially could add a simple config There was a problem hiding this comment. I like it, it would be a nice addition, I agree it can be added later |
||
| period: 10s | ||
| hosts: ["localhost:9090"] | ||
| metrics_path: '/federate' | ||
| query: | ||
| 'match[]': '{__name__!=""}' | ||
| ------------------------------------------------------------------------------------- | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This also needs a note, that the stat metricset was removed.