From 435c422579a257f808ee93896674c58c6f7fe656 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Wed, 25 Aug 2021 23:53:42 -0700 Subject: [PATCH 01/20] add MetricReader interface --- specification/metrics/sdk.md | 91 ++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index edc8ac41867..908f325f0fc 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -25,6 +25,8 @@ Table of Contents * [MeterProvider](#meterprovider) * [Attribute Limits](#attribute-limits) * [MeasurementProcessor](#measurementprocessor) +* [MetricReader](#metricreader) + * [Periodic exporting MetricReader](#periodic-exporting-metricreader) * [MetricExporter](#metricexporter) * [Push Metric Exporter](#push-metric-exporter) * [Pull Metric Exporter](#pull-metric-exporter) @@ -411,6 +413,95 @@ active span](../trace/api.md#context-interaction)). +------------------+ ``` +## MetricReader + +`MetricReader` is an interface which provides the following capabilities: + +* Collecting metrics from the SDK. +* Handling the [ForceFlush](#forceflush) and [Shutdown](#shutdown) signals from + the SDK. + +```text ++-----------------+ +--------------+ +| | Metrics... | | +| In-memory state +------------> MetricReader | +| | | | ++-----------------+ +--------------+ + ++-----------------+ +--------------+ +| | Metrics... | | +| In-memory state +------------> MetricReader | +| | | | ++-----------------+ +--------------+ +``` + +### MetricReader operations + +#### Collect + +Collects the metrics from the SDK. If there are [asynchronous +Instruments](./api.md#asynchronous-instrument) involved, their callback +functions will be triggered. Then the [OnCollect callback](#oncollect) will be +called with the collected metrics from the in-memory state. + +`Collect` SHOULD provide a way to let the caller know whether it succeeded, +failed or timed out. + +`Collect` does not have any required parameters, however, individual language +client MAY choose to support optional parameters (e.g. filter, timeout). + +#### OnCollect(batch) + +Callback function which will be triggered by [Collect](#collect). + +`OnCollect` SHOULD provide a way to indicate whether it succeeded, failed or +timed out. + +`OnCollect` MUST accept the following parameters: + +* A `batch` of metrics. The exact data type of the batch is language specific, + typically it is some kind of list, e.g. it could be an enumerator that travels + through the in-memory state. + +Individual language client MAY choose to add optional parameters (e.g. timeout). + +#### OnForceFlush + +Callback function which will be triggered by [ForceFlush](#forceflush). + +`OnForceFlush` SHOULD provide a way to let the caller know whether it succeeded, +failed or timed out. + +`OnForceFlush` SHOULD complete or abort within some timeout. `OnForceFlush` can +be implemented as a blocking API or an asynchronous API which notifies the +caller via a callback or an event. OpenTelemetry client authors can decide if +they want to make the flush timeout configurable. + +#### OnShutdown + +Callback function which will be triggered by [Shutdown](#shutdown). This is an +opportunity for `MetricReader` to perform any required cleanup. + +`OnShutdown` should be called only once for each `MetricReader` instance. + +`OnShutdown` should not block indefinitely (e.g. if it attempts to flush the +data and the destination is unavailable). OpenTelemetry client authors can +decide if they want to make the shutdown timeout configurable. + +### Periodic exporting MetricReader + +This is an implementation of the `MetricReader` which collects metrics based on +a user-configurable time interval, and passes the metrics to the configured +[Push Metric Exporter](#push-metric-exporter). + +Configurable parameters: + +* `exporter` - the push exporter where the metrics are sent to. +* `exportIntervalMillis` - the time interval in milliseconds between two + consecutive exports. The default value is 5000 (milliseconds). +* `exportTimeoutMillis` - how long the export can run before it is cancelled. + The default value is 30000 (milliseconds). + ## MetricExporter `MetricExporter` defines the interface that protocol-specific exporters MUST From f50922d2b804fee82170fb5e037721d91be1a97a Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Thu, 26 Aug 2021 08:57:00 -0700 Subject: [PATCH 02/20] fix link --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 908f325f0fc..49be9bad340 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -450,7 +450,7 @@ failed or timed out. `Collect` does not have any required parameters, however, individual language client MAY choose to support optional parameters (e.g. filter, timeout). -#### OnCollect(batch) +#### OnCollect Callback function which will be triggered by [Collect](#collect). From 76230039b592b5abc5aabeef1cc63369676fa761 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Thu, 26 Aug 2021 09:30:01 -0700 Subject: [PATCH 03/20] add the base exporting metricreader --- specification/metrics/sdk.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 49be9bad340..2e9a04e62b2 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -488,6 +488,25 @@ opportunity for `MetricReader` to perform any required cleanup. data and the destination is unavailable). OpenTelemetry client authors can decide if they want to make the shutdown timeout configurable. +### Base exporting MetricReader + +This is an implementation of the `MetricReader` which collects metrics when +[OnCollect](#oncollect) is called. and passes the metrics to the configured +[MetricExporter](#metricexporter). + +Configurable parameters: + +* `exporter` - the exporter where the metrics are sent to. + +If the configured exporter only supports [pull mode](#pull-metric-exporter): + +* [Collect](#collect) SHOULD only be called when the [Pull Metric + Exporter](#pull-metric-exporter) triggers the scraping, otherwise it SHOULD be + treated as an error. +* Individual language client MAY decide if [OnForceFlush](#onforceflush) will + return immediately (with an indication of failure) or wait for the next + [OnCollect](#oncollect) call to complete. + ### Periodic exporting MetricReader This is an implementation of the `MetricReader` which collects metrics based on From b3f0fe0e5dcf08999d4b4801e4d4a255e48abb64 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Thu, 26 Aug 2021 09:38:07 -0700 Subject: [PATCH 04/20] update diagram --- specification/metrics/sdk.md | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 2e9a04e62b2..c635c63bd06 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -535,17 +535,29 @@ The following diagram shows `MetricExporter`'s relationship to other components in the SDK: ```text -+-----------------+ +-----------------------+ -| | Metrics... | | -| In-memory state +------------> MetricExporter (push) +--> Another process -| | | | -+-----------------+ +-----------------------+ - -+-----------------+ +-----------------------+ -| | Metrics... | | -| In-memory state +------------> MetricExporter (pull) +--> Another process (scraper) -| | | | -+-----------------+ +-----------------------+ ++-----------------+ +-----------------------------+ +| | Metrics... | | +| In-memory state +------------> Base exporting MetricReader | +| | | | ++-----------------+ | +-----------------------+ | + | | | | + | | MetricExporter (push) +-----> Another process + | | | | + | +-----------------------+ | + | | + +-----------------------------+ + ++-----------------+ +-----------------------------+ +| | Metrics... | | +| In-memory state +------------> Base exporting MetricReader | +| | | | ++-----------------+ | +-----------------------+ | + | | | | + | | MetricExporter (pull) +-----> Another process (scraper) + | | | | + | +-----------------------+ | + | | + +-----------------------------+ ``` Metric Exporter has access to the [pre-aggregated metrics From 0c2fa70808363c08b96dd5c5e324bec6bbe40a80 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Fri, 27 Aug 2021 10:32:07 -0700 Subject: [PATCH 05/20] clarify reentrancy for OnCollect --- specification/metrics/sdk.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index c635c63bd06..af4a872c124 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -457,6 +457,9 @@ Callback function which will be triggered by [Collect](#collect). `OnCollect` SHOULD provide a way to indicate whether it succeeded, failed or timed out. +`OnCollect` SHOULD never be called concurrently for the same `MetricReader` +instance. `OnCollect` can be called again only after the current call returns. + `OnCollect` MUST accept the following parameters: * A `batch` of metrics. The exact data type of the batch is language specific, From 4e003a81b8a5a425756eb388fcca2c9822bff6c1 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Fri, 27 Aug 2021 10:54:58 -0700 Subject: [PATCH 06/20] describe how MetricReader is registered to MeterProvider --- specification/metrics/sdk.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index af4a872c124..1845c02c835 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -51,10 +51,10 @@ to create an instance which is stored on the created `Meter`. Configuration (i.e., [MeasurementProcessors](#measurementprocessor), -[MetricExporters](#metricexporter) and [`Views`](#view)) MUST be managed solely -by the `MeterProvider` and the SDK MUST provide a way to configure all options -that are implemented by the SDK. This MAY be done at the time of MeterProvider -creation if appropriate. +[MetricExporters](#metricexporter), [MetricReaders](#metricreader) and +[Views](#view)) MUST be managed solely by the `MeterProvider` and the SDK MUST +provide a way to configure all options that are implemented by the SDK. This MAY +be done at the time of MeterProvider creation if appropriate. The `MeterProvider` MAY provide methods to update the configuration. If configuration is updated (e.g., adding a `MeasurementProcessor`), the updated @@ -184,26 +184,26 @@ meter_provider "Bar", instrument_name="Y", aggregation=HistogramAggregation(buckets=[5.0, 10.0, 25.0, 50.0, 100.0])) - .set_exporter(PrometheusExporter()) + .add_metric_reader(BaseExportingMetricReader(PrometheusExporter())) ``` ```python # all the metrics will be exported using the default configuration -meter_provider.set_exporter(ConsoleExporter()) +meter_provider.add_metric_reader(PeriodicExportingMetricReader(ConsoleExporter())) ``` ```python # all the metrics will be exported using the default configuration meter_provider .add_view("*") # a wildcard view that matches everything - .set_exporter(ConsoleExporter()) + .add_metric_reader(PeriodicExportingMetricReader(ConsoleExporter())) ``` ```python # Counter X will be exported as cumulative sum meter_provider .add_view("X", aggregation=SumAggregation(CUMULATIVE)) - .set_exporter(ConsoleExporter()) + .add_metric_reader(PeriodicExportingMetricReader(ConsoleExporter())) ``` ```python @@ -212,7 +212,7 @@ meter_provider meter_provider .add_view("X", aggregation=SumAggregation(DELTA)) .add_view("*", attribute_keys=["a", "b"]) - .set_exporter(ConsoleExporter()) + .add_metric_reader(PeriodicExportingMetricReader(ConsoleExporter())) ``` ### Aggregation From 6716bd7c525b47d56b9f662321bca3e9419ae651 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Mon, 30 Aug 2021 16:24:43 -0700 Subject: [PATCH 07/20] Update specification/metrics/sdk.md Co-authored-by: John Watson --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 66723a96348..dea92896761 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -581,7 +581,7 @@ instance. `OnCollect` can be called again only after the current call returns. typically it is some kind of list, e.g. it could be an enumerator that travels through the in-memory state. -Individual language client MAY choose to add optional parameters (e.g. timeout). +Individual language clients MAY choose to add optional parameters (e.g. timeout). #### OnForceFlush From c42cbc677ac7e3678434235989d7741af1733c1d Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Mon, 30 Aug 2021 16:24:49 -0700 Subject: [PATCH 08/20] Update specification/metrics/sdk.md Co-authored-by: John Watson --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index dea92896761..e5e5c27fa05 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -621,7 +621,7 @@ If the configured exporter only supports [pull mode](#pull-metric-exporter): * [Collect](#collect) SHOULD only be called when the [Pull Metric Exporter](#pull-metric-exporter) triggers the scraping, otherwise it SHOULD be treated as an error. -* Individual language client MAY decide if [OnForceFlush](#onforceflush) will +* Individual language clients MAY decide if [OnForceFlush](#onforceflush) will return immediately (with an indication of failure) or wait for the next [OnCollect](#oncollect) call to complete. From b1b5b2c1edc6f2a76b4edfeab71c103d5bdda0b0 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Mon, 30 Aug 2021 16:25:53 -0700 Subject: [PATCH 09/20] update toc --- specification/metrics/sdk.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index e5e5c27fa05..7d21c766ef1 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -26,6 +26,7 @@ Table of Contents * [Attribute Limits](#attribute-limits) * [MeasurementProcessor](#measurementprocessor) * [MetricReader](#metricreader) + * [Base exporting MetricReader](#base-exporting-metricreader) * [Periodic exporting MetricReader](#periodic-exporting-metricreader) * [MetricExporter](#metricexporter) * [Push Metric Exporter](#push-metric-exporter) From 13054537ca0232957abd093c65e468845d1a3589 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Mon, 30 Aug 2021 20:45:03 -0700 Subject: [PATCH 10/20] address review comments --- specification/metrics/sdk.md | 80 ++++++++++++++++++++++++++++++++---- 1 file changed, 73 insertions(+), 7 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 7d21c766ef1..0674ac634fe 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -679,6 +679,46 @@ in the SDK: +-----------------------------+ ``` +The following sequence diagram shows how `MetricExporter` interacts with other +components in the SDK. Note that the _SIGNAL_ in the diagram could come from: + +* A periodic timer from the `MetricReader`, if the MetricReader is a [Periodic + exporting MetricReader](#periodic-exporting-metricreader). +* A scraping call received by the `MetricExporter`, if the MetricExporter is a + [Pull Metric Exporter](#pull-metric-exporter). +* An explicit call to the `MetricReader.Collect()` from the user. +* An implicit call to the `MetricReader.Collect()` (e.g. + `MeterProvider.ForceFlush()`). + +```text ++---------------+ +--------------+ +----------------+ +| | | | | | +| MeterProvider | | MetricReader | | MetricExporter | +| | | | | | ++---------------+ +--------------+ +----------------+ + | +--------+ | | + | | | | | + | | SIGNAL | | | + | | | | | + | +---+----+ | | + | | | | + | | MetricReader.Collect() | | + | +------------------------> | + | | | + | invoke callbacks from async instuments | | + <----------------------------------------+ | + | | | + | MetricReader.OnCollect(batch) callback | | + +----------------------------------------> | + | | | + | | MetricExporter.Export(batch) | + | +------------------------------> + | | | + | | | send metrics + | | +--------------> + | | | +``` + Metric Exporter has access to the [pre-aggregated metrics data](./datamodel.md#timeseries-model). @@ -693,13 +733,6 @@ example: * Exporter D is a pull exporter which reacts to another scraper over a named pipe. -### Push Metric Exporter - -Push Metric Exporter sends the data on its own schedule. Here are some examples: - -* Sends the data based on a user configured schedule, e.g. every 1 minute. -* Sends the data when there is a severe error. - #### Interface Definition A Push Metric Exporter MUST support the following functions: @@ -770,12 +803,45 @@ return a Failure result. and the destination is unavailable). OpenTelemetry client authors can decide if they want to make the shutdown timeout configurable. +### Push Metric Exporter + +Push Metric Exporter sends the data on its own schedule. Here are some examples: + +* Sends the data based on a user configured schedule, e.g. every 1 minute. +* Sends the data when there is a severe error. + ### Pull Metric Exporter Pull Metric Exporter reacts to the metrics scrapers and reports the data passively. This pattern has been widely adopted by [Prometheus](https://prometheus.io/). +```text ++---------------+ +--------------+ +-----------------------+ +| | | | | | +| MeterProvider | | MetricReader | | MetricExporter (pull) | +| | | | | | ++---------------+ +--------------+ +-----------------------+ + | | | +---------+ + | | | scraping | | + | | <----------+ Scraper | + | | MetricReader.Collect() | | | + | <------------------------------+ +---------+ + | | | + | invoke callbacks from async instuments | | + <----------------------------------------+ | + | | | + | MetricReader.OnCollect(batch) callback | | + +----------------------------------------> | + | | | + | | MetricExporter.Export(batch) | + | +------------------------------> + | | | + | | | respond to scraper + | | +--------------------> + | | | +``` + ## Defaults and Configuration The SDK MUST provide the following configuration parameters for Exemplar From ee5f5b85d5b06e0de1bf76b2bac60f879c73c8ef Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Tue, 31 Aug 2021 08:05:29 -0700 Subject: [PATCH 11/20] fix heading --- specification/metrics/sdk.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 0674ac634fe..b5181e524b1 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -733,11 +733,11 @@ example: * Exporter D is a pull exporter which reacts to another scraper over a named pipe. -#### Interface Definition +### Interface Definition A Push Metric Exporter MUST support the following functions: -##### Export(batch) +#### Export(batch) Exports a batch of `Metrics`. Protocol exporters that will implement this function are typically expected to serialize and transmit the data to the @@ -772,7 +772,7 @@ Returns: `ExportResult` Note: this result may be returned via an async mechanism or a callback, if that is idiomatic for the language implementation. -##### ForceFlush() +#### ForceFlush() This is a hint to ensure that the export of any `Metrics` the exporter has received prior to the call to `ForceFlush` SHOULD be completed as soon as @@ -790,7 +790,7 @@ implemented as a blocking API or an asynchronous API which notifies the caller via a callback or an event. OpenTelemetry client authors can decide if they want to make the flush timeout configurable. -##### Shutdown() +#### Shutdown() Shuts down the exporter. Called when SDK is shut down. This is an opportunity for exporter to do any cleanup required. From 104d65a7e652e797466cf5fa67222f0d073898d3 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Tue, 31 Aug 2021 08:28:23 -0700 Subject: [PATCH 12/20] fix typo --- specification/metrics/sdk.md | 98 ++++++++++++++++++------------------ 1 file changed, 49 insertions(+), 49 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index b5181e524b1..7f1943c6c7e 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -691,32 +691,32 @@ components in the SDK. Note that the _SIGNAL_ in the diagram could come from: `MeterProvider.ForceFlush()`). ```text -+---------------+ +--------------+ +----------------+ -| | | | | | -| MeterProvider | | MetricReader | | MetricExporter | -| | | | | | -+---------------+ +--------------+ +----------------+ - | +--------+ | | - | | | | | - | | SIGNAL | | | - | | | | | - | +---+----+ | | - | | | | - | | MetricReader.Collect() | | - | +------------------------> | - | | | - | invoke callbacks from async instuments | | - <----------------------------------------+ | - | | | - | MetricReader.OnCollect(batch) callback | | - +----------------------------------------> | - | | | - | | MetricExporter.Export(batch) | - | +------------------------------> - | | | - | | | send metrics - | | +--------------> - | | | ++---------------+ +--------------+ +----------------+ +| | | | | | +| MeterProvider | | MetricReader | | MetricExporter | +| | | | | | ++---------------+ +--------------+ +----------------+ + | +--------+ | | + | | | | | + | | SIGNAL | | | + | | | | | + | +---+----+ | | + | | | | + | | MetricReader.Collect() | | + | +-------------------------> | + | | | + | invoke callbacks from async instruments | | + <-----------------------------------------+ | + | | | + | MetricReader.OnCollect(batch) callback | | + +-----------------------------------------> | + | | | + | | MetricExporter.Export(batch) | + | +------------------------------> + | | | + | | | send metrics + | | +--------------> + | | | ``` Metric Exporter has access to the [pre-aggregated metrics @@ -817,29 +817,29 @@ passively. This pattern has been widely adopted by [Prometheus](https://prometheus.io/). ```text -+---------------+ +--------------+ +-----------------------+ -| | | | | | -| MeterProvider | | MetricReader | | MetricExporter (pull) | -| | | | | | -+---------------+ +--------------+ +-----------------------+ - | | | +---------+ - | | | scraping | | - | | <----------+ Scraper | - | | MetricReader.Collect() | | | - | <------------------------------+ +---------+ - | | | - | invoke callbacks from async instuments | | - <----------------------------------------+ | - | | | - | MetricReader.OnCollect(batch) callback | | - +----------------------------------------> | - | | | - | | MetricExporter.Export(batch) | - | +------------------------------> - | | | - | | | respond to scraper - | | +--------------------> - | | | ++---------------+ +--------------+ +-----------------------+ +| | | | | | +| MeterProvider | | MetricReader | | MetricExporter (pull) | +| | | | | | ++---------------+ +--------------+ +-----------------------+ + | | | +---------+ + | | | scraping | | + | | <----------+ Scraper | + | | MetricReader.Collect() | | | + | <------------------------------+ +---------+ + | | | + | invoke callbacks from async instruments | | + <-----------------------------------------+ | + | | | + | MetricReader.OnCollect(batch) callback | | + +-----------------------------------------> | + | | | + | | MetricExporter.Export(batch) | + | +------------------------------> + | | | + | | | respond to scraper + | | +--------------------> + | | | ``` ## Defaults and Configuration From fd9dfc45c33c0895c0d9d7f62d767d9ed4ce6323 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Tue, 31 Aug 2021 18:07:13 -0700 Subject: [PATCH 13/20] add clarification and example for multiple readers --- specification/metrics/sdk.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 7f1943c6c7e..828b7db1e45 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -537,6 +537,16 @@ measurements using the equivalent of the following naive algorithm: * Handling the [ForceFlush](#forceflush) and [Shutdown](#shutdown) signals from the SDK. +The SDK MUST support multiple `MetricReader` instances to be registered on the +same `MeterProvider`, and the [MetricReader.Collect](#collect) invocation on one +`MetricReader` instance MUST NOT introduce side-effects to other `MetricReader` +instances. For example, if a `MetricReader` instance is receiving metric data +points that have [delta temporality](./datamodel.md#temporality) via the +[MetricReader.OnCollect](#oncollect) callback, it is expected that SDK will +update the time range - e.g. from (Tn, Tn+1] to +(Tn+1, Tn+2] - **ONLY** for this particular `MetricReader` +instance, so the next `OnCollect` will get a new start time. + ```text +-----------------+ +--------------+ | | Metrics... | | From 502913169873d3878a6d4160141ef1955dfe8a49 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Fri, 3 Sep 2021 16:36:14 -0700 Subject: [PATCH 14/20] update the PR based on discussion during the SIG meeting --- specification/metrics/sdk.md | 149 ++++++----------------------------- 1 file changed, 25 insertions(+), 124 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 828b7db1e45..3c3d3e74a7d 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -561,66 +561,31 @@ instance, so the next `OnCollect` will get a new start time. +-----------------+ +--------------+ ``` +The SDK SHOULD provide a way to allow `MetricReader` to respond to +[MeterProvider.ForceFlush](#forceflush) and [MeterProvider.Shutdown](#shutdown). +Individual language clients can decide the language idiomatic approach, for +example, as `OnForceFlush` and `OnShutdown` callback functions. + ### MetricReader operations #### Collect Collects the metrics from the SDK. If there are [asynchronous Instruments](./api.md#asynchronous-instrument) involved, their callback -functions will be triggered. Then the [OnCollect callback](#oncollect) will be -called with the collected metrics from the in-memory state. +functions will be triggered. `Collect` SHOULD provide a way to let the caller know whether it succeeded, failed or timed out. `Collect` does not have any required parameters, however, individual language -client MAY choose to support optional parameters (e.g. filter, timeout). - -#### OnCollect - -Callback function which will be triggered by [Collect](#collect). - -`OnCollect` SHOULD provide a way to indicate whether it succeeded, failed or -timed out. - -`OnCollect` SHOULD never be called concurrently for the same `MetricReader` -instance. `OnCollect` can be called again only after the current call returns. - -`OnCollect` MUST accept the following parameters: - -* A `batch` of metrics. The exact data type of the batch is language specific, - typically it is some kind of list, e.g. it could be an enumerator that travels - through the in-memory state. - -Individual language clients MAY choose to add optional parameters (e.g. timeout). - -#### OnForceFlush - -Callback function which will be triggered by [ForceFlush](#forceflush). - -`OnForceFlush` SHOULD provide a way to let the caller know whether it succeeded, -failed or timed out. - -`OnForceFlush` SHOULD complete or abort within some timeout. `OnForceFlush` can -be implemented as a blocking API or an asynchronous API which notifies the -caller via a callback or an event. OpenTelemetry client authors can decide if -they want to make the flush timeout configurable. - -#### OnShutdown - -Callback function which will be triggered by [Shutdown](#shutdown). This is an -opportunity for `MetricReader` to perform any required cleanup. - -`OnShutdown` should be called only once for each `MetricReader` instance. - -`OnShutdown` should not block indefinitely (e.g. if it attempts to flush the -data and the destination is unavailable). OpenTelemetry client authors can -decide if they want to make the shutdown timeout configurable. +clients MAY choose to support optional parameters (e.g. callback, filter, +timeout). Individual language clients MAY choose the return value type, or do +not return anything. ### Base exporting MetricReader This is an implementation of the `MetricReader` which collects metrics when -[OnCollect](#oncollect) is called. and passes the metrics to the configured +[Collect](#collect) is called. and passes the metrics to the configured [MetricExporter](#metricexporter). Configurable parameters: @@ -632,9 +597,6 @@ If the configured exporter only supports [pull mode](#pull-metric-exporter): * [Collect](#collect) SHOULD only be called when the [Pull Metric Exporter](#pull-metric-exporter) triggers the scraping, otherwise it SHOULD be treated as an error. -* Individual language clients MAY decide if [OnForceFlush](#onforceflush) will - return immediately (with an indication of failure) or wait for the next - [OnCollect](#oncollect) call to complete. ### Periodic exporting MetricReader @@ -689,46 +651,6 @@ in the SDK: +-----------------------------+ ``` -The following sequence diagram shows how `MetricExporter` interacts with other -components in the SDK. Note that the _SIGNAL_ in the diagram could come from: - -* A periodic timer from the `MetricReader`, if the MetricReader is a [Periodic - exporting MetricReader](#periodic-exporting-metricreader). -* A scraping call received by the `MetricExporter`, if the MetricExporter is a - [Pull Metric Exporter](#pull-metric-exporter). -* An explicit call to the `MetricReader.Collect()` from the user. -* An implicit call to the `MetricReader.Collect()` (e.g. - `MeterProvider.ForceFlush()`). - -```text -+---------------+ +--------------+ +----------------+ -| | | | | | -| MeterProvider | | MetricReader | | MetricExporter | -| | | | | | -+---------------+ +--------------+ +----------------+ - | +--------+ | | - | | | | | - | | SIGNAL | | | - | | | | | - | +---+----+ | | - | | | | - | | MetricReader.Collect() | | - | +-------------------------> | - | | | - | invoke callbacks from async instruments | | - <-----------------------------------------+ | - | | | - | MetricReader.OnCollect(batch) callback | | - +-----------------------------------------> | - | | | - | | MetricExporter.Export(batch) | - | +------------------------------> - | | | - | | | send metrics - | | +--------------> - | | | -``` - Metric Exporter has access to the [pre-aggregated metrics data](./datamodel.md#timeseries-model). @@ -743,11 +665,18 @@ example: * Exporter D is a pull exporter which reacts to another scraper over a named pipe. -### Interface Definition +### Push Metric Exporter + +Push Metric Exporter sends the data on its own schedule. Here are some examples: + +* Sends the data based on a user configured schedule, e.g. every 1 minute. +* Sends the data when there is a severe error. + +#### Interface Definition A Push Metric Exporter MUST support the following functions: -#### Export(batch) +##### Export(batch) Exports a batch of `Metrics`. Protocol exporters that will implement this function are typically expected to serialize and transmit the data to the @@ -782,7 +711,7 @@ Returns: `ExportResult` Note: this result may be returned via an async mechanism or a callback, if that is idiomatic for the language implementation. -#### ForceFlush() +##### ForceFlush() This is a hint to ensure that the export of any `Metrics` the exporter has received prior to the call to `ForceFlush` SHOULD be completed as soon as @@ -800,7 +729,7 @@ implemented as a blocking API or an asynchronous API which notifies the caller via a callback or an event. OpenTelemetry client authors can decide if they want to make the flush timeout configurable. -#### Shutdown() +##### Shutdown() Shuts down the exporter. Called when SDK is shut down. This is an opportunity for exporter to do any cleanup required. @@ -813,44 +742,16 @@ return a Failure result. and the destination is unavailable). OpenTelemetry client authors can decide if they want to make the shutdown timeout configurable. -### Push Metric Exporter - -Push Metric Exporter sends the data on its own schedule. Here are some examples: - -* Sends the data based on a user configured schedule, e.g. every 1 minute. -* Sends the data when there is a severe error. - ### Pull Metric Exporter Pull Metric Exporter reacts to the metrics scrapers and reports the data passively. This pattern has been widely adopted by [Prometheus](https://prometheus.io/). -```text -+---------------+ +--------------+ +-----------------------+ -| | | | | | -| MeterProvider | | MetricReader | | MetricExporter (pull) | -| | | | | | -+---------------+ +--------------+ +-----------------------+ - | | | +---------+ - | | | scraping | | - | | <----------+ Scraper | - | | MetricReader.Collect() | | | - | <------------------------------+ +---------+ - | | | - | invoke callbacks from async instruments | | - <-----------------------------------------+ | - | | | - | MetricReader.OnCollect(batch) callback | | - +-----------------------------------------> | - | | | - | | MetricExporter.Export(batch) | - | +------------------------------> - | | | - | | | respond to scraper - | | +--------------------> - | | | -``` +Implementors MAY choose the best idiomatic design for their language. For +example, they could apply the [Push Metric Exporter +interface](#push-metric-exporter) design for consistency, or they could design a +completely different pull exporter interface. ## Defaults and Configuration From ee0419024e937d7bb8f341abeb9e84917fbd009f Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Fri, 3 Sep 2021 16:39:42 -0700 Subject: [PATCH 15/20] cleanup --- specification/metrics/sdk.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 3c3d3e74a7d..20b3a7ea4b3 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -541,11 +541,10 @@ The SDK MUST support multiple `MetricReader` instances to be registered on the same `MeterProvider`, and the [MetricReader.Collect](#collect) invocation on one `MetricReader` instance MUST NOT introduce side-effects to other `MetricReader` instances. For example, if a `MetricReader` instance is receiving metric data -points that have [delta temporality](./datamodel.md#temporality) via the -[MetricReader.OnCollect](#oncollect) callback, it is expected that SDK will -update the time range - e.g. from (Tn, Tn+1] to -(Tn+1, Tn+2] - **ONLY** for this particular `MetricReader` -instance, so the next `OnCollect` will get a new start time. +points that have [delta temporality](./datamodel.md#temporality), it is expected +that SDK will update the time range - e.g. from (Tn, Tn+1] +to (Tn+1, Tn+2] - **ONLY** for this particular +`MetricReader` instance. ```text +-----------------+ +--------------+ From b2e2fad942f098901231c8dd263ea8678fa53704 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Fri, 3 Sep 2021 16:41:57 -0700 Subject: [PATCH 16/20] cleanup --- specification/metrics/sdk.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 20b3a7ea4b3..913927c60d2 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -539,7 +539,7 @@ measurements using the equivalent of the following naive algorithm: The SDK MUST support multiple `MetricReader` instances to be registered on the same `MeterProvider`, and the [MetricReader.Collect](#collect) invocation on one -`MetricReader` instance MUST NOT introduce side-effects to other `MetricReader` +`MetricReader` instance SHOULD NOT introduce side-effects to other `MetricReader` instances. For example, if a `MetricReader` instance is receiving metric data points that have [delta temporality](./datamodel.md#temporality), it is expected that SDK will update the time range - e.g. from (Tn, Tn+1] @@ -577,9 +577,9 @@ functions will be triggered. failed or timed out. `Collect` does not have any required parameters, however, individual language -clients MAY choose to support optional parameters (e.g. callback, filter, -timeout). Individual language clients MAY choose the return value type, or do -not return anything. +clients MAY choose to add parameters (e.g. callback, filter, timeout). +Individual language clients MAY choose the return value type, or do not return +anything. ### Base exporting MetricReader From 86a2c6dbd23bbfe15f6c5adcf18b0b75bec3eadf Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Fri, 3 Sep 2021 16:49:23 -0700 Subject: [PATCH 17/20] clarify the wording --- specification/metrics/sdk.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 913927c60d2..f9735d7b744 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -587,6 +587,11 @@ This is an implementation of the `MetricReader` which collects metrics when [Collect](#collect) is called. and passes the metrics to the configured [MetricExporter](#metricexporter). +**Note:** The name **"Base" exporting MetricReader** has nothing to do with +object oriented programming or class inheritance / hierarchy. In case it might +introduce confusion in certain programming languages, individual language client +CAN choose a different wording. + Configurable parameters: * `exporter` - the exporter where the metrics are sent to. From 47f74e98bb00a17efa6ea51302f13a076a426e5a Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Tue, 7 Sep 2021 21:47:32 -0700 Subject: [PATCH 18/20] adjust wording for pull exporter --- specification/metrics/sdk.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index f9735d7b744..3063cabc5c4 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -752,10 +752,15 @@ Pull Metric Exporter reacts to the metrics scrapers and reports the data passively. This pattern has been widely adopted by [Prometheus](https://prometheus.io/). +Unlike [Push Metric Exporter](#push-metric-exporter) which can send data on its +own schedule, pull exporter can only send the data when it is being asked by the +scraper, and `ForceFlush` would not make sense. + Implementors MAY choose the best idiomatic design for their language. For -example, they could apply the [Push Metric Exporter -interface](#push-metric-exporter) design for consistency, or they could design a -completely different pull exporter interface. +example, they could generalize the [Push Metric Exporter +interface](#push-metric-exporter) design and use that for consistency, they +could model the pull exporter as [MetricReader](#metricreader), or they could +design a completely different pull exporter interface. ## Defaults and Configuration From 7977fbfe721a8e5681d18b15da8fc7e8a145ea80 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Wed, 8 Sep 2021 10:30:27 -0700 Subject: [PATCH 19/20] remove base exporting metric reader, mention that it can be an option but not required --- specification/metrics/sdk.md | 100 +++++++++++++++++------------------ 1 file changed, 48 insertions(+), 52 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 3063cabc5c4..b9cae11c029 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -26,7 +26,6 @@ Table of Contents * [Attribute Limits](#attribute-limits) * [MeasurementProcessor](#measurementprocessor) * [MetricReader](#metricreader) - * [Base exporting MetricReader](#base-exporting-metricreader) * [Periodic exporting MetricReader](#periodic-exporting-metricreader) * [MetricExporter](#metricexporter) * [Push Metric Exporter](#push-metric-exporter) @@ -188,7 +187,7 @@ meter_provider "Bar", instrument_name="Y", aggregation=HistogramAggregation(buckets=[5.0, 10.0, 25.0, 50.0, 100.0])) - .add_metric_reader(BaseExportingMetricReader(PrometheusExporter())) + .add_metric_reader(PeriodicExportingMetricReader(ConsoleExporter())) ``` ```python @@ -581,27 +580,6 @@ clients MAY choose to add parameters (e.g. callback, filter, timeout). Individual language clients MAY choose the return value type, or do not return anything. -### Base exporting MetricReader - -This is an implementation of the `MetricReader` which collects metrics when -[Collect](#collect) is called. and passes the metrics to the configured -[MetricExporter](#metricexporter). - -**Note:** The name **"Base" exporting MetricReader** has nothing to do with -object oriented programming or class inheritance / hierarchy. In case it might -introduce confusion in certain programming languages, individual language client -CAN choose a different wording. - -Configurable parameters: - -* `exporter` - the exporter where the metrics are sent to. - -If the configured exporter only supports [pull mode](#pull-metric-exporter): - -* [Collect](#collect) SHOULD only be called when the [Pull Metric - Exporter](#pull-metric-exporter) triggers the scraping, otherwise it SHOULD be - treated as an error. - ### Periodic exporting MetricReader This is an implementation of the `MetricReader` which collects metrics based on @@ -626,35 +604,6 @@ The goal of the interface is to minimize burden of implementation for protocol-dependent telemetry exporters. The protocol exporter is expected to be primarily a simple telemetry data encoder and transmitter. -The following diagram shows `MetricExporter`'s relationship to other components -in the SDK: - -```text -+-----------------+ +-----------------------------+ -| | Metrics... | | -| In-memory state +------------> Base exporting MetricReader | -| | | | -+-----------------+ | +-----------------------+ | - | | | | - | | MetricExporter (push) +-----> Another process - | | | | - | +-----------------------+ | - | | - +-----------------------------+ - -+-----------------+ +-----------------------------+ -| | Metrics... | | -| In-memory state +------------> Base exporting MetricReader | -| | | | -+-----------------+ | +-----------------------+ | - | | | | - | | MetricExporter (pull) +-----> Another process (scraper) - | | | | - | +-----------------------+ | - | | - +-----------------------------+ -``` - Metric Exporter has access to the [pre-aggregated metrics data](./datamodel.md#timeseries-model). @@ -676,6 +625,23 @@ Push Metric Exporter sends the data on its own schedule. Here are some examples: * Sends the data based on a user configured schedule, e.g. every 1 minute. * Sends the data when there is a severe error. +The following diagram shows `Push Metric Exporter`'s relationship to other +components in the SDK: + +```text ++-----------------+ +---------------------------------+ +| | Metrics... | | +| In-memory state +------------> Periodic exporting MetricReader | +| | | | ++-----------------+ | +-----------------------+ | + | | | | + | | MetricExporter (push) +-------> Another process + | | | | + | +-----------------------+ | + | | + +---------------------------------+ +``` + #### Interface Definition A Push Metric Exporter MUST support the following functions: @@ -762,6 +728,36 @@ interface](#push-metric-exporter) design and use that for consistency, they could model the pull exporter as [MetricReader](#metricreader), or they could design a completely different pull exporter interface. +The following diagram gives some examples on how `Pull Metric Exporter` can be +modeled to interact with other components in the SDK: + +* Model the pull exporter as MetricReader + + ```text + +-----------------+ +-----------------------------+ + | | Metrics... | | + | In-memory state +------------> PrometheusExporter (pull) +---> Another process (scraper) + | | | (modeled as a MetricReader) | + +-----------------+ | | + +-----------------------------+ + ``` + +* Use the same MetricExporter design for both push and pull exporters + + ```text + +-----------------+ +-----------------------------+ + | | Metrics... | | + | In-memory state +------------> Base exporting MetricReader | + | | | | + +-----------------+ | +-----------------------+ | + | | | | + | | MetricExporter (pull) +------> Another process (scraper) + | | | | + | +-----------------------+ | + | | + +-----------------------------+ + ``` + ## Defaults and Configuration The SDK MUST provide the following configuration parameters for Exemplar From 68f4fdc1156ae3a8eed3332006040b8648db32e1 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Wed, 8 Sep 2021 12:21:23 -0700 Subject: [PATCH 20/20] change default exporting interval to 60sec --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index b9cae11c029..78ab4e94b8d 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -590,7 +590,7 @@ Configurable parameters: * `exporter` - the push exporter where the metrics are sent to. * `exportIntervalMillis` - the time interval in milliseconds between two - consecutive exports. The default value is 5000 (milliseconds). + consecutive exports. The default value is 60000 (milliseconds). * `exportTimeoutMillis` - how long the export can run before it is cancelled. The default value is 30000 (milliseconds).