fix(query engine): Include lines with ts equal to end timestamp of the query range when executing range aggregations

[chaudum]

What this PR does / Why we need it

Background

When performing range vector aggregations, such as count_over_time({env="dev"}[1h]), the query range is divided into multiple steps at which the aggregation operation (e.g. counting the log lines) is evaluated.
Each step starts at current step - step interval and ends at current step, as depicted in the following chart. The select range for the logs is extended by the step interval into the past, in order to select logs for calculating the first step.

However, the select range for logs is start inclusive and end exclusive (written as [start, end)), but the evaluation of the steps for the range aggregation is start exclusive and end inclusive (written as (start, end]).

This leads to the problem that the very first timestamp at the beginning of the select range and the very last timestamp at the end of the select range are not included in the range aggregation. The "missing" last timestamp is not a problem, because a) in an instant query it is not supposed to be included anyway because of the [start, end) inclusivity of the query range and b) in a range query the last point of the previous step will be part of the next step evaluation.

Issue

The missing first timestamp, however, gets problematic when executing an instant query and the log timestamps are exactly at the start of the query range. This can happen when the query is split in the query frontend into multiple smaller time ranges, e.g. 1h, 30m, ...
Since the sub queries are executed independently on the queriers, all logs that have a timestamp exactly a multiple of the split interval, e.g. 00:00, 01:00, 02:00, ... for a 1h interval, are dismissed and therefore missing in the query result over the full time range of the original query.

Fix

In order to avoid the missing logs that have a timestamp a multiple of the split interval in instant queries, we need to adjust the query range for logs to also include the end timestamp (written as [start, end]). This is done by adding a "leap nanosecond" to the end timestamp of the log select range. This ensures that the included end timestamp of the step evaluation is also included in the log selection.

Checklist

• Reviewed the CONTRIBUTING.md guide (required)
• Documentation added
• Tests updated
• Title matches the required conventional commits format, see here
  • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
• Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
• For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
• If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

[trevorwhitney]

Should this be noted as a breaking change, and/or require a call out in the upgrading guide?

[chaudum]

Should this be noted as a breaking change, and/or require a call out in the upgrading guide?

This is not a breaking change, but a fix to include chunks that were otherwise dismissed because the overlap check failed.

[kavirajk]

LGTM 👍

[kavirajk]

q: wondering if any other places of Overlap usage need this swap of arguments as well?

[chaudum]

I checked, but all other places already use the Overlap function with the correct argument order.

[cyriltovena]

LGTM