feat: More detailed tracing for distributors #14504
Open
+59
−34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def buckets Swap metrics for subspans in Distributor.push
benclive
changed the title
| @@ -304,6 +305,12 @@ func New( | |||
| Help: "The number of records a single per-partition write request has been split into.", | |||
| Buckets: prometheus.ExponentialBuckets(1, 2, 8), | |||
| }), | |||
| kafkaBytesPerRecord: promauto.With(registerer).NewHistogram(prometheus.HistogramOpts{ | |||
| Namespace: constants.Loki, | |||
| Name: "distributor_kafka_bytes_per_record", | |||
There was a problem hiding this comment.
Perhaps I misunderstood, but it seems like either Name or Help is incorrect. Name suggests this is record size, but Help suggests this is the number of records, not the record size.
| @@ -594,36 +601,40 @@ func (d *Distributor) Push(ctx context.Context, req *logproto.PushRequest) (*log | |||
| const maxExpectedReplicationSet = 5 // typical replication factor 3 plus one for inactive plus one for luck | |||
| var descs [maxExpectedReplicationSet]ring.InstanceDesc | |||
|
|
|||
| tracker := pushTracker{ | |||
| kafkaTracker := pushTracker{ | |||
There was a problem hiding this comment.
Something that is unclear to me is what was the behavior before when sharing a common tracker, and what is the behavior now having separate trackers?
|
In general LGTM, but a couple questions before I ✅ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Improves the tracing in the distributor. Currently we just get a single span for the whole request.