POC remotetap extension

[wildum]

Description: THIS DRAFT IS A PROOF OF CONCEPT to show what advanced debugging capabilities the remotetap extension could offer.

The intent of this draft is to start discussions and gather feedback on the current approach to this feature.

Link to tracking Issue: #34096

Demo video of the POC:

Screen.Recording.2024-07-31.at.11.mp4

Description of the current approach:

• the remotetap extension exposes a very simple UI on the endpoint "localhost:11000" (configurable)
• components can register themselves on start to the extension (if available)
• users can see the list of registered components in the UI and "tap" to them
• when a user "tap" to a component, it allows the component to publish data to the extension

Support has been added for metrics to the attributes and metricstransform processors as examples.

Pros:

• no need to modify the pipeline to tap into the components
• when no "tap" is open, the performance cost is negligible
• every component can decide what data should be sent to the extension
• supports metrics, logs, traces, and string (if the component wants to send specific data)

Cons:

• support must be added to all relevant components (receivers, processors, connectors) one by one
• the otel collector does not have an official UI where this can be integrated. The extension has its own UI.

Potential improvements of the current solution:

• the UI can be improved to have more controls (for example sampling or filtering)
• all signals are currently mixed together, we could have three tabs "metrics, traces, logs"
• the current formatting is quite verbose (especially for the histograms). A better formatting would make the debugging experience much nicer.

Alternative approaches:

• add the extension to core instead of contrib and connect it in the framework directly to have it supported by all components by default. Not sure how feasible this is. Pros: no need to add support to components one by one. Cons: components cannot choose the data that they want to send

Please comment on this draft if you have feedback on this approach.
If you want to discuss other solutions or if you have a different opinion on what the remotetap extension should do, please comment on the linked issue to keep the discussions organized.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.