Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to limit length of values of attributes and metric values #1130

Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
4507656
Add an option to limit span attribute length
jtmalinowski Oct 22, 2020
deca25b
Change default value of OTEL_SPAN_ATTRIBUTE_SIZE_LIMIT to empty
jtmalinowski Oct 22, 2020
748eae0
Fix wording in Span attribute limiting section
jtmalinowski Oct 22, 2020
e7e54b3
Introduce metric label value size limit
jtmalinowski Oct 23, 2020
10af7ba
Spec compliance for truncating metric label values
jtmalinowski Oct 23, 2020
ed9bc5f
Limits docs improvements
jtmalinowski Oct 23, 2020
dabb292
Fix traces limiting requirement
jtmalinowski Oct 26, 2020
4102464
Expand span attribute limits to all attributes
jtmalinowski Oct 27, 2020
a6da2b9
Merge remote-tracking branch 'upstream/master' into feature/span-attr…
jtmalinowski Oct 27, 2020
c175613
Wording improvements
jtmalinowski Oct 27, 2020
5259377
More accurate naming for attribute truncation env variables
jtmalinowski Oct 28, 2020
d4c28eb
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jun 15, 2021
df01ff4
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jun 15, 2021
5ea1f00
fix: typos
jtmalinowski Jun 15, 2021
47c86a3
fix: merge mistakes
jtmalinowski Jun 15, 2021
8bd3a9b
fix: PR feedback
jtmalinowski Jun 16, 2021
c2ed17c
fix: typo in link
jtmalinowski Jun 16, 2021
05173b9
fix: PR feedback
jtmalinowski Jun 21, 2021
5b763d3
fix: PR feedback
jtmalinowski Jun 21, 2021
b180054
chore: changelog
jtmalinowski Jun 21, 2021
1434084
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jun 28, 2021
77a0688
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jul 5, 2021
5bde7ba
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jul 8, 2021
cf73a3b
Update specification/sdk-environment-variables.md
jtmalinowski Jul 12, 2021
50556d1
Update specification/common/common.md
jtmalinowski Jul 12, 2021
f3c5243
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jul 13, 2021
89f4b3a
fix: clarify priority of limits
jtmalinowski Jul 13, 2021
992b132
Merge branch 'main' into feature/span-attribute-size-limit
jtmalinowski Jul 16, 2021
f428739
fix: nerf logging requirements
jtmalinowski Jul 17, 2021
6f9992a
fix: grammar
jtmalinowski Jul 20, 2021
dd0b245
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jul 20, 2021
e3d7b73
Fixes from Yuri
jtmalinowski Jul 27, 2021
e526784
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jul 27, 2021
243b07d
fix: clarify metrics exemption
jtmalinowski Jul 27, 2021
cd51664
fix: link
jtmalinowski Jul 27, 2021
ff664a2
fix: language
jtmalinowski Jul 27, 2021
53522b6
Merge branch 'main' into feature/span-attribute-size-limit
jtmalinowski Jul 28, 2021
684a0a6
Merge remote-tracking branch 'upstream/main' into feature/span-attrib…
jtmalinowski Jul 29, 2021
d33eadf
Merge branch 'main' into feature/span-attribute-size-limit
jtmalinowski Jul 30, 2021
49b7ec9
Merge branch 'main' into feature/span-attribute-size-limit
jtmalinowski Aug 3, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion spec-compliance-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ status of the feature is not known.
|links collection size limit | | | + | | | | | | | |
|[Span attributes](https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/api.md#set-attributes)|
|SetAttribute | + | + | + | + | + | + | + | + | + | + |
|Span attribute size limit | | + | | | | | | | | |
|Set order preserved | + | - | + | + | + | + | + | + | + | + |
|String type | + | + | + | + | + | + | + | + | + | + |
|Boolean type | + | + | + | + | + | + | + | + | + | + |
Expand Down Expand Up @@ -81,7 +82,7 @@ status of the feature is not known.

|Feature |Go|Java|JS |Python|Ruby|Erlang|PHP|Rust|C++|.Net|
|----------------------------------------------|--|----|---|------|----|------|---|----|---|----|
|TBD|
|Metric label value size limit | | | | | | | | | | |

## Resource

Expand Down Expand Up @@ -128,6 +129,8 @@ status of the feature is not known.
|OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | | | | | | | | | | |
|OTEL_SPAN_EVENT_COUNT_LIMIT | | | | | | | | | | |
|OTEL_SPAN_LINK_COUNT_LIMIT | | | | | | | | | | |
|OTEL_SPAN_ATTRIBUTE_SIZE_LIMIT | | | | | | | | | | |
|OTEL_METRIC_LABEL_SIZE_LIMIT | | | | | | | | | | |

## Exporters

Expand Down
19 changes: 19 additions & 0 deletions specification/metrics/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -494,3 +494,22 @@ and Sketch.

TODO: T.B.D.: DDSketch considered a good choice for ValueRecorder
instrument default aggregation.

## Limits on Metrics

Erroneous code can add unintended labels to a span. If they are unbounded, they
jtmalinowski marked this conversation as resolved.
Show resolved Hide resolved
can quickly exhaust available memory, resulting in crashes that are difficult to
recover from safely.

To protect against such errors, SDKs MAY be configured to truncate label values.
By default, label values SHOULD NOT be truncated. If an SDK provides a way to set
this limit and the limit is set, then for each Metric label value, if its length
exceeds that limit, SDK Spans SHOULD truncate that value, so that its length is
jtmalinowski marked this conversation as resolved.
Show resolved Hide resolved
at most equal to the limit.

If there is a configurable limit, the SDK SHOULD honor the environment variables
specified in [SDK environment variables](../sdk-environment-variables.md#metric-limits).

There SHOULD be a log emitted to indicate to the user that any value was discarded
or truncated due to such a limit. To prevent excessive logging, the log should
not be emitted more than once per metric or value.
jtmalinowski marked this conversation as resolved.
Show resolved Hide resolved
21 changes: 16 additions & 5 deletions specification/sdk-environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,23 @@ Additional values can be specified in the respective SDK's documentation, in cas
| OTEL_BSP_MAX_EXPORT_BATCH_SIZE | Maximum batch size | 512 | Must be less than or equal to OTEL_BSP_MAX_QUEUE_SIZE |

## Span Collection Limits
Moved [here](#span-limits). This section serves as
a permalink.
jtmalinowski marked this conversation as resolved.
Show resolved Hide resolved

| Name | Description | Default | Notes |
| ------------------------------- | ------------------------------------ | ------- | ----- |
| OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | Maximum allowed span attribute count | 1000 | |
| OTEL_SPAN_EVENT_COUNT_LIMIT | Maximum allowed span event count | 1000 | |
| OTEL_SPAN_LINK_COUNT_LIMIT | Maximum allowed span link count | 1000 | |
## Span Limits
jtmalinowski marked this conversation as resolved.
Show resolved Hide resolved

| Name | Description | Default | Notes |
| ------------------------------- | ------------------------------------- | ------- | ----- |
| OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | Maximum allowed span attribute count | 1000 | |
| OTEL_SPAN_EVENT_COUNT_LIMIT | Maximum allowed span event count | 1000 | |
| OTEL_SPAN_LINK_COUNT_LIMIT | Maximum allowed span link count | 1000 | |
| OTEL_SPAN_ATTRIBUTE_SIZE_LIMIT | Maximum allowed span attribute length | | Empty value is treated as infinity. Non-integer and negative values are invalid. |

## Metric Limits

| Name | Description | Default | Notes |
| ---------------------------- | ----------------------------------- | ------- | ----- |
| OTEL_METRIC_LABEL_SIZE_LIMIT | Maximum allowed metric value length | | Empty value is treated as infinity. Non-integer and negative values are invalid. |

## OTLP Exporter

Expand Down
15 changes: 13 additions & 2 deletions specification/trace/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
* [Tracer Provider](#tracer-provider)
* [Additional Span Interfaces](#additional-span-interfaces)
* [Limits on Span Collections](#limits-on-span-collections)
jtmalinowski marked this conversation as resolved.
Show resolved Hide resolved
* [Limits on Span Collections and Attributes](#limits-on-span-collections-and-attributes)
* [Span Processor](#span-processor)
* [Span Exporter](#span-exporter)

Expand Down Expand Up @@ -265,6 +266,10 @@ Thus, the SDK specification defines sets of possible requirements for
or a getter could be provided).

## Limits on Span Collections
Moved [here](#limits-on-span-collections-and-attributes). This section serves as
a permalink.
jtmalinowski marked this conversation as resolved.
Show resolved Hide resolved

## Limits on Span Collections and Attributes

Erroneous code can add unintended attributes, events, and links to a span. If
these collections are unbounded, they can quickly exhaust available memory,
Expand All @@ -274,12 +279,18 @@ To protect against such errors, SDK Spans MAY discard attributes, links, and
events that would increase the number of elements of each collection beyond
the recommended limit of 1000 elements. SDKs MAY provide a way to change this limit.

Similarly, SDKs MAY be configured to truncate attribute values. By default, attribute
values SHOULD NOT be truncated. If an SDK provides a way to set this limit and the
limit is set, then for each span attribute value, serialized into a string,
if it exceeds that limit, SDK Spans SHOULD truncate that value, so that its length
jtmalinowski marked this conversation as resolved.
Show resolved Hide resolved
is at most equal to the limit.

If there is a configurable limit, the SDK SHOULD honor the environment variables
specified in [SDK environment variables](../sdk-environment-variables.md#span-collection-limits).
specified in [SDK environment variables](../sdk-environment-variables.md#span-limits).

There SHOULD be a log emitted to indicate to the user that an attribute, event,
or link was discarded due to such a limit. To prevent excessive logging, the log
should not be emitted once per span, or per discarded attribute, event, or links.
should not be emitted more than once per span, attribute, event, or link.

## Span processor

Expand Down