Shim: OpenCensus Shim

[rghetia]

Create an OpenCensus shim that bridges opencensus to opentelemetry.

[joivo]

Is that still active?

[MrAlias]

Is that still active?

It is still a goal across OpenTelemetry to provide backwards compatible support with OpenCensus. Though, this is not prioritized work here currently.

[dashpole]

I'd like to tackle this. I recently made it possible to swap out the OpenCensus "SDK" for go: census-instrumentation/opencensus-go#1238, and would like to contribute a shim library that implements the OpenCensus APIs using opentelemetry. I have a working prototype here: https://github.com/dashpole/opencensus-migration-go/blob/master/migration/tracer.go

If that direction sounds good, let me know where the shim should live, and i'll open a PR.

[MrAlias]

If that direction sounds good, let me know where the shim should live, and i'll open a PR.

@dashpole that sounds good to me. Including a new opencensus package in the bridge package seems appropriate.

[dashpole]

I forgot one piece in opencensus-go needed to accomplish this, so I haven't opened the PR yet.

In the mean time, I wanted to raise a few incompatibilities between OpenCensus and OpenTelemetry:

1. OpenCensus supports calling StartSpan with a Sampler. OpenTelemetry does not. The current migration would ignore any OpenCensus Samplers that are specified during span creation.
2. OpenTelemetry supports "Debug" and "Deferred" as trace flags in the SpanContext. OpenCensus does not. The current migration would drop those flags when propagating context from an OpenTelemetry library to an OpenCensus-migration library.
3. Somewhat related, OpenCensus GRPC mandates the use of a binary propagation format. That was removed from the Specification, and adding it back is tracked by Update Binary format in the Specification opentelemetry-specification#437. However, it might be useful to provide an implementation of an OpenCensus-compatible binary propagation format as part of the opencensus bridge in the meantime. I have a working implementation of this here: https://github.com/dashpole/opencensus-migration-go/blob/master/migration/binary.go

[yegle]

Is the current focus of the shim on the tracing side of OpenCensus?

While looking into migrating from OpenCensus to OpenTelemetry with the focus on our metrics, I realized OpenTelemetry don't have full metrics support in packages like sql, grpc etc. It would be great if those can be supported by the bridge/shim.

[MrAlias]

@yegle the tracing API is more mature than the metric API in OpenTelemetry. The ultimate goal is to have a complete migration shim. However, something is better than nothing and it seems like we are hoping to iterate a into a full solution, starting with what we can currently do in tracing.

[MrAlias]

1. OpenCensus supports calling StartSpan with a Sampler. OpenTelemetry does not. The current migration would ignore any OpenCensus Samplers that are specified during span creation.
2. OpenTelemetry supports "Debug" and "Deferred" as trace flags in the SpanContext. OpenCensus does not. The current migration would drop those flags when propagating context from an OpenTelemetry library to an OpenCensus-migration library.

SGTM

3. Somewhat related, OpenCensus GRPC mandates the use of a binary propagation format. That was removed from the Specification, and adding it back is tracked by open-telemetry/opentelemetry-specification#437. However, it might be useful to provide an implementation of an OpenCensus-compatible binary propagation format as part of the opencensus bridge in the meantime. I have a working implementation of this here: https://github.com/dashpole/opencensus-migration-go/blob/master/migration/binary.go

I think this is a good way to at least start.

[dashpole]

I wanted to add an example for using the bridge, but otelgrpc is in the contrib repo. Using it in the example in this repo would produce an import cycle, or pin the example to a released version of this repo (v0.13.0) instead of using HEAD. I could:

1. Just add the example somewhere in contrib (where would it go?)
2. Add the example in the bridge/opencensus/examples directory, and pin it to a released version (v0.13.0)
3. Move the opencensus binary propagation bridge to contrib entirely, and add the example there as well. (e.g. under /bridge/opencensus/binary)

[dashpole]

We chose option 3 in the meeting today. We can put it in the propagators package in contrib. The example will live within the opencensus binary propagation package.

[dashpole]

For metrics, this is the approach java is taking: open-telemetry/opentelemetry-java#2085

They implemented a metrics exporter bridge (i.e. wrap an OpenTelemetry exporter so that it implements the OpenCensus exporter interface). If you have a mix of OpenTelemetry and OpenCensus libraries, you can use a single OpenTelemetry exporter with the OpenTelemetry SDK and the OpenCensus SDK.

Q: Why didn't we take that approach with tracing? It seems simpler...
A: This approach, applied to tracing, would have broken context propagation between OpenTelemetry and OpenCensus spans since they store spans in different context.Context keys.

Q: Why not take the approach tracing took by implementing the OpenCensus monitoring API using OpenTelemetry?
A: The APIs are quite different. For example, OpenTelemetry doesn't have a notion of Views, so we wouldn't be able to provide a complete bridge.

Q: Why do we need a bridge at all if we don't need to worry about the context propagation problem?
A: There are some OpenTelemetry exporters which don't have an OpenCensus equivalent. If a project wants to adopt OpenTelemetry, but some dependencies use OpenCensus, they won't be able to get complete monitoring data without an exporter bridge.
A: There are some features (views and context-based labels come to mind) which OpenTelemetry does not support yet. For libraries that need these features, a bridge will allow them to delay moving to OpenTelemetry until OpenTelemetry adds (or chooses not to add) these features. During that time period, their users can still have a single export pipeline even though the library still uses OpenCensus APIs.

Questions I have:

• Does the above make sense as an approach for Metrics?
• Should we release a metric exporter bridge with the Tracing GA? That would encourage the use of OpenTelemetry's non-GA components, such as OTLP's metric definition, or the exporter APIs. Should we wait until Metrics is closer to GA?

[dashpole]

AIs from 12/10/20 sig meeting:

• What are the features people want from OpenCensus monitoring?
  • Note: We expect that many of the features OpenCensus has will exist in OpenTelemetry soon. Views and context-based labels should both have rough feature parity in the future.
• The exporter approach will work now, and means we don't need to worry about API compatibility. Plan to proceed with the exporter-bridge approach laid out above.

[punya]

Can we split this into two issues (for trace and metric), such that we can close out the trace-related one which blocks RC and put the metrics-related into a future milestone?