Refactor trace semantic convention yaml structure

[jamesmoessis]

Changes

I've refactored the semantic convention yaml so it lines up with @tedsuo's draft PR #1977. This is an initial piece of work which precedes refactoring the markdown into a similar structure.

The structure is as follows:

├── trace
│   ├── database
│   │   ├── database.yaml
│   │   ├── implementations
│   │   │   ├── cassandra.yaml
│   │   │   ├── mongodb.yaml
│   │   │   ├── mssql.yaml
│   │   │   └── redis.yaml
│   │   └── sql.yaml
│   ├── exception
│   │   └── exception.yaml
│   ├── faas
│   │   ├── faas.yaml
│   │   └── implementations
│   │       └── lambda.yaml
│   ├── general.yaml
│   ├── http
│   │   └── http.yaml
│   ├── instrumentation
│   │   └── aws-sdk.yml
│   ├── messaging
│   │   ├── implementations
│   │   │   ├── kafka.yaml
│   │   │   ├── rabbitmq.yaml
│   │   │   └── rocketmq.yaml
│   │   └── messaging.yaml
│   └── rpc
│       ├── implementations
│       │   ├── grpc.yaml
│       │   └── jsonrpc.yaml
│       └── rpc.yaml

I deleted the aws folder as it had one file lambda.yaml and moved it under faas/implementations.

Additionally, make table-generation produces no changes to the markdown, so it looks like it isn't breaking anything.

Why restructure?

It has been raised on several occasions that the semantic convention is becoming bloated with implementation-specific conventions. This is the first step to separating them out.

This will help the semantic convention towards stability because we will be able to declare the "core" conventions as stable, while the "implementations" can remain experimental.

Related issues #

• User feedback for the semantic convention #1908
• Refactor semantic conventions #1977

[tigrannajaryan]

I've refactored the semantic convention yaml so it lines up with @tedsuo's draft PR #1977. This is an initial piece of work which precedes refactoring the markdown into a similar structure.

I think this PR deviates from what @tedsuo proposed:

Create one folder per topic (HTTP, faas, messaging, etc). All relevant tracing, metric, and resource conventions are combined into these topics.

An important different to me is that trace in this proposal is a separate branch of the tree, which I think would be good to avoid since many (most?) of the conventions that are applicable to traces are also applicable to logs. I think @tedsuo's proposal correctly addressed that need, but this PR foregoes that.

[jamesmoessis]

@tigrannajaryan Only traces and resources have YAML definitions, and I can't think of a way to combine them that would make sense. Restructuring to unify metrics/traces would need to happen in the markdown, since the metrics semconv is defined directly in the markdown.

many (most?) of the conventions that are applicable to traces are also applicable to logs.

Do logs have a semantic convention? I can't find it in the spec, so I haven't taken logs into consideration at all.

Ted's PR says:

Centralize the location of all semantic conventions, rather than split them across many subsections of the spec.

Create one folder per topic (HTTP, faas, messaging, etc). All relevant tracing, metric, and resource conventions are combined into these topics.

I'm trying to think of how to tackle these in terms of the YAML. In the name of centralization, we could:

• Rename the trace folder to something more general, and keep the YAML organised by "topic" like is mentioned. But currently there are only YAML definitions for traces, so I kept the trace folder name.
• Move the YAML and markdown nearer to each other in the spec.

If anyone has ideas about how to structure the yaml, would be interesting to hear.

[jamesmoessis]

As a separate issue, I would argue that metrics and traces convention should be separate, due to use case and cardinality concerns. However, that doesn't mean that we can't make the semantic convention easier to navigate. Would like to hear people's thoughts on that too.

[tigrannajaryan]

Do logs have a semantic convention? I can't find it in the spec, so I haven't taken logs into consideration at all.

Not right now in the spec, but yes, virtually every semantic convention that is defined for traces is also applicable to logs. We need to refactor the spec in a way that takes this into account, we just did not have time to do this. Since you are now doing this refactoring I think it is now a good time to address this problem.

[tigrannajaryan]

As a separate issue, I would argue that metrics and traces convention should be separate, due to use case and cardinality concerns.

There is some overlap between traces and metrics as well (although not as much as between traces and logs). For example HTTP attributes:
Metrics: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/semantic_conventions/http-metrics.md#attributes
Traces: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/http.md#common-attributes

Things like http.method or http.scheme have the same name and values when they are recorded as a trace attribute or as a metric attribute.

There may be nuanced variations though (see e.g. http.status_code), so whatever approach we choose there may be a need to express these variations.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[jamesmoessis]

@tigrannajaryan I would argue that metric attributes should be a subset of trace attributes (where applicable). @tedsuo also mentioned this recently in the instrumentation SIG.

Perhaps somewhere in the metrics spec there should be some specification of this subsetting? It would help the metrics convention towards stability. I wonder how this could be achieved.