Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MinMaxSumCount data type #122

Closed
wants to merge 1 commit into from

Conversation

MrAlias
Copy link
Contributor

@MrAlias MrAlias commented Mar 10, 2020

The specification includes this as a default aggregation type, however, the proto has no way to transfer this data natively. This adds a new data type and datapoints to be used in this transfer.

The specification includes this as a default aggregation type, however,
the proto has no way to transfer this data natively. This adds a new
data type and datapoints to be used in this transfer.
@bogdandrutu
Copy link
Member

bogdandrutu commented Mar 11, 2020

We can use Summary with 0 percentile min and 1 percentile max

@jmacd
Copy link
Contributor

jmacd commented Mar 11, 2020

@MrAlias
Copy link
Contributor Author

MrAlias commented Mar 11, 2020

@jmacd & @bogdandrutu: further down in the conversation that @jmacd linked both @c24t and @jkwatson continued to discuss the fact that the OpenTelemetry proto definition did not match the OpenTelemetry specified aggregations, and there was desire there to have them match.

It seems like this proto definition is a loud echo (or a direct renaming) of the OpenCensus proto. As such, it seems like this kind of hold over is going to provide future confusion and friction. Parts of the OpenTelemetry project will be bound by a previous specification's design choices.

The solution @bogdandrutu proposed indeed can be used to solve this transport mismatch, but should it? I think for the same reasons we decided to include more than just Measurement instruments in the metric API we should include message types that match OpenTelemetry specified aggregations. It makes the project a cohesive set of ideas instead of a patch work one.

Are there other reasons besides an opposition to changing what was inherited that are leading you to oppose this inclusion? I'm interested to understand both of your reasons on this issue, and guessing I've overlooked something.

@bogdandrutu
Copy link
Member

Your comment makes me think that we actually misdefined what is the difference between Aggregation and Aggregator.

The Aggregation represents the result of an Instrument + Aggregator, and is defined in a generic way so that different combinations of Instruments and Aggregators can produce the same Aggregation. As an example DDSketch and MinMaxSumCount (or any other algorithm that allows to calculate quantiles) aggregators applied to a Measure instrument will both produce summary aggregation (with different level of confidence and .

Another reason is that the cardinality of the combinations Instruments + Aggregators will be very high and make it impossible to define an object for every Aggregation.

I would be happy to add a new Aggregation in the proto, if that gives us a clear benefit, but for this specific request I do not see what is the advantage to know that this is MinMaxSumCount vs Summary.

It seems like this proto definition is a loud echo (or a direct renaming) of the OpenCensus proto. As such, it seems like this kind of hold over is going to provide future confusion and friction. Parts of the OpenTelemetry project will be bound by a previous specification's design choices.

We are trying to also stay compatible with OpenMetrics protocol which does not define a MinMaxCountSum representation.

@jkwatson
Copy link
Contributor

What if we add an optional minimum to the Summary? The thing that bothers me the most is that a percentile value of 0.0 is mathematical nonsense. We're shoehorning the minimum into that percentile slot because we don't have a proper place for it, rather than it being a good idea.

@bogdandrutu
Copy link
Member

@jkwatson that is an interesting reason and I would I would like to document a bit about that. I personally didn't know that is the case, will investigate more. But hope that my explanation makes sense why we don't need an aggregations for every possible aggregators.

@jkwatson
Copy link
Contributor

To quote wikipedia:

A percentile (or a centile) is a measure used in statistics indicating the value below which a given percentage of observations in a group of observations falls. For example, the 20th percentile is the value (or score) below which 20% of the observations may be found. Similarly, 80% of the observations are found above the 20th percentile.

so, there is, by definition, nothing in the "0th" percentile. :)

@MrAlias
Copy link
Contributor Author

MrAlias commented Mar 13, 2020

I've realized I probably should have opened an issue prior to this to have the discussion there. Going to close in favor of #125 where I hope we can come to a solution to better support OpenTelemetry aggregations.

@MrAlias MrAlias closed this Mar 13, 2020
@MrAlias MrAlias deleted the add-min-max-sum-count branch March 13, 2020 19:46
MrAlias added a commit to MrAlias/opentelemetry-proto that referenced this pull request Apr 15, 2020
Update metric descriptor to contain information on the data qualities.
This includes:

 - The meaning of the time interval measurements were made over is now
   described by the `interval` value.
 - The monotonic nature of the data is captured if known.
 - If the data values are restricted to a subset of numbers.

The `SummaryDataPoint` and `HistogramDataPoint` were merged into a
unified `Distribution` message that contains statistics about the
observed population of values.

Adds support for multiple histogram bucket definitions and adds a linear
bucket definition message.

Includes an explicit minimum and maximum for a `Distribution` (open-telemetry#122).

Removes the exemplar code (open-telemetry#81). This can be added back in as a more
generalized case now given the new `Data` structure of the Metrics. But,
it has been left for a separate PR.

Unifies the common parts of the previous `*DataPoints` into a unified
`Data` message.

This is not complete. It likely needs better names and it certainly
needs comments.

Related to open-telemetry#125
Fixes to open-telemetry#81
@MrAlias MrAlias mentioned this pull request Apr 15, 2020
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants