chore: merge with origin/main

Squashed commit of the following: commit 15dc2ba Author: Christian Haudum <christian.haudum@gmail.com> Date: Wed Mar 27 19:55:13 2024 +0100 feat(bloomstore): Support for sharding blocks across multiple different directories (#12375) Signed-off-by: Christian Haudum <christian.haudum@gmail.com> commit c1edb82 Author: J Stickler <julie.stickler@grafana.com> Date: Wed Mar 27 14:46:00 2024 -0400 docs: fix formatting in structured metadata topic (#12382) commit c9e5c7f Author: Owen Diehl <ow.diehl@gmail.com> Date: Wed Mar 27 10:27:44 2024 -0700 chore(blooms): removes bloom-gw & bloom-compactor from all non-microservice targets (#12381) commit 2b8db8b Author: Travis Patterson <travis.patterson@grafana.com> Date: Wed Mar 27 08:48:04 2024 -0600 fix: attempt non-string label filtering when errors are present (#12378) commit f99e6ac Author: Dylan Guedes <djmgguedes@gmail.com> Date: Wed Mar 27 10:51:21 2024 -0300 docs: Add missing changelog entry for mTLS memcached addition (#12377) commit 016daa5 Author: Karsten Jeschkies <karsten.jeschkies@grafana.com> Date: Wed Mar 27 14:28:32 2024 +0100 fix: Track all OTLP metadata bytes in usage tracker. (#12376) Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com> commit aca2a38 Author: Paul Rogers <129207811+paul1r@users.noreply.github.com> Date: Wed Mar 27 07:50:25 2024 -0400 test: gcs object client test data race (#12324) commit bdad1d3 Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Date: Wed Mar 27 10:52:30 2024 +0000 chore(deps): update module google.golang.org/protobuf to v1.33.0 [security] (main) (#12137) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> commit ca667aa Author: Christian Haudum <christian.haudum@gmail.com> Date: Wed Mar 27 09:53:33 2024 +0100 chore(bloom-gw): Cleanup tracing implementation (#12368) The huge amount of `bloomgateway.ProcessTask` spans caused problem with Tempo's span limit, resulting in dropped/dangling spans. Since the processing time is now recorded also in a metrics.go-like stats log line, we can remove the spans and only add a log event with the summary to the main span of the request handler. This PR also unifies the event logging in the index gateway, and adds the real processing time (not aggregated processing time) to the request stats. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com> commit 120b76d Author: Owen Diehl <ow.diehl@gmail.com> Date: Wed Mar 27 00:16:07 2024 -0700 chore(blooms): misc fixes (#12369) unbounds bloom-gw-client concurrency removes unused log_gateway_requests param adds note wrt bloom-gw results cache keygen issue add msg field to stats log sets bloomshipper download workers count to 16 default commit 19c046f Author: Christian Haudum <christian.haudum@gmail.com> Date: Tue Mar 26 23:07:02 2024 +0100 chore(blooms): Make max bloom page size for querying configurable (#12337) Signed-off-by: Christian Haudum <christian.haudum@gmail.com> commit 9810e8e Author: Owen Diehl <ow.diehl@gmail.com> Date: Tue Mar 26 11:57:27 2024 -0700 chore(blooms): computed fields for bloomgw stats logging (#12367) commit 6b3bbb3 Author: J Stickler <julie.stickler@grafana.com> Date: Tue Mar 26 14:22:48 2024 -0400 docs: Update release notes for 2.9.6 (#12365) commit 30ac88b Author: Owen Diehl <ow.diehl@gmail.com> Date: Tue Mar 26 11:00:47 2024 -0700 chore(blooms): increase blockpool by factor-of-2 (#12363) commit d3266a1 Author: Pangidoan Butar <38452094+doanbutar@users.noreply.github.com> Date: Wed Mar 27 01:53:05 2024 +0800 docs: Update structured_metadata.md (#12355) Co-authored-by: J Stickler <julie.stickler@grafana.com> commit c0c7a19 Author: J Stickler <julie.stickler@grafana.com> Date: Tue Mar 26 11:30:00 2024 -0400 docs: Update release notes for recent 2.9 releases (#12349) commit 9af191f Author: Christian Haudum <christian.haudum@gmail.com> Date: Tue Mar 26 15:09:40 2024 +0100 feat(bloom-gw): Add `metrics.go` style log line to bloom gateway `FilterChunks` call (#12354) This PR adds a "metrics.go" style log line to the bloom gateway that contains latencies for queueing, processing, post-processing, as well as number of chunks/series requested/filtered. This log line should give operators a better understanding where time is spent for individual requests. Example log line: ``` ts=2024-03-26T13:16:11.869619964Z caller=spanlogger.go:109 component=bloom-gateway tenant=XXX method=bloomgateway.FilterChunkRefs user=XXX traceID=6239d49e1e88f88645bd7f1f6f1b85c8 level=info status=success tasks=1 series_requested=35 series_filtered=31 chunks_requested=103 chunks_filtered=93 queue_time=4.108128936s metas_fetch_time=22.040408ms blocks_fetch_time=1.284575ms processing_time=30.215863002s post_processing_time=16.084µs ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com> commit 5c6a031 Author: Salva Corts <salva.corts@grafana.com> Date: Tue Mar 26 14:59:26 2024 +0100 fix(blooms): Not filters should always match (#12358) commit 64c7812 Author: Ashwanth <iamashwanth@gmail.com> Date: Tue Mar 26 18:55:12 2024 +0530 fix(codec): inject disable wrappers in ctx (#12352) commit a36483b Author: Owen Diehl <ow.diehl@gmail.com> Date: Tue Mar 26 03:36:49 2024 -0700 feat(blooms): bloom integration in query planning (#12208) Signed-off-by: Owen Diehl <ow.diehl@gmail.com> commit d5ecf9a Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 26 10:21:48 2024 +0000 chore(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /operator (#12207) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Periklis Tsirakidis <periklis@redhat.com> commit 774b01d Author: Ashwanth <iamashwanth@gmail.com> Date: Tue Mar 26 13:15:54 2024 +0530 fix(ruler): pass noop analyseRules to rules manager (#12353) commit 03982c2 Author: arukiidou <arukiidou@yahoo.co.jp> Date: Tue Mar 26 16:10:31 2024 +0900 docs: fix otlp guide - collector config example (#12328) Signed-off-by: junya koyama <arukiidou@yahoo.co.jp> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> commit 664e569 Author: Trevor Whitney <trevorjwhitney@gmail.com> Date: Mon Mar 25 16:12:58 2024 -0600 feat: prepare 3.0.0 release candidate (#12348) Release-As: 3.0.0-alpha.1 commit 40f695e Author: Trevor Whitney <trevorjwhitney@gmail.com> Date: Mon Mar 25 15:29:12 2024 -0600 ci: prepare 3.0 release workflow (#12344) Release-As: 3.0.0-alpha.1 commit 04398a1 Author: Travis Patterson <travis.patterson@grafana.com> Date: Mon Mar 25 14:52:24 2024 -0600 fix: log state of 'disable pipeline wrappers' (#12345)
grafana · Mar 27, 2024 · 99d57d2 · 99d57d2
1 parent 2f8cc43
commit 99d57d2
Show file tree

Hide file tree

Showing 164 changed files with 6,700 additions and 1,112 deletions.
diff --git a/.github/jsonnetfile.lock.json b/.github/jsonnetfile.lock.json
@@ -8,8 +8,8 @@
           "subdir": "workflows"
         }
       },
-      "version": "d9bfe17ed706fdfbd02445a576eb820dea4811a3",
-      "sum": "RCFmsb2FD3ZQmjSVfAo+5/GNi2mit/T9BZ474iE2u2o="
+      "version": "d3fa90c124d13a4e0359a46c8708704e92ee8a50",
+      "sum": "TQ2X5sm7o+BCrytzSbJ7Th2YqNZ2ZPx9Wg62x5mEVJ0="
     }
   ],
   "legacyImports": false

diff --git a/.github/release-workflows.jsonnet b/.github/release-workflows.jsonnet
@@ -46,7 +46,7 @@ local imagePrefix = 'grafana';
   ),
   'minor-release-pr.yml': std.manifestYamlDoc(
     lokiRelease.releasePRWorkflow(
-      branches=['k[0-9]+'],
+      branches=['k[0-9]+', 'main'],
       buildImage=buildImage,
       checkTemplate=checkTemplate,
       golangCiLintVersion=golangCiLintVersion,
@@ -63,7 +63,7 @@ local imagePrefix = 'grafana';
   ),
   'release.yml': std.manifestYamlDoc(
     lokiRelease.releaseWorkflow(
-      branches=['release-[0-9]+.[0-9]+.x', 'k[0-9]+'],
+      branches=['release-[0-9]+.[0-9]+.x', 'k[0-9]+', 'main'],
       getDockerCredsFromVault=true,
       imagePrefix='grafana',
       releaseLibRef=releaseLibRef,

diff --git a/.github/vendor/github.com/grafana/loki-release/workflows/build.libsonnet b/.github/vendor/github.com/grafana/loki-release/workflows/build.libsonnet
diff --git a/.github/vendor/github.com/grafana/loki-release/workflows/main.jsonnet b/.github/vendor/github.com/grafana/loki-release/workflows/main.jsonnet
diff --git a/.github/vendor/github.com/grafana/loki-release/workflows/release.libsonnet b/.github/vendor/github.com/grafana/loki-release/workflows/release.libsonnet
diff --git a/.github/vendor/github.com/grafana/loki-release/workflows/workflows.jsonnet b/.github/vendor/github.com/grafana/loki-release/workflows/workflows.jsonnet
diff --git a/.github/workflows/minor-release-pr.yml b/.github/workflows/minor-release-pr.yml
@@ -1,9 +1,11 @@
 concurrency:
   group: "create-release-pr-${{ github.sha }}"
 env:
+  BUILD_ARTIFACTS_BUCKET: "loki-build-artifacts"
   BUILD_TIMEOUT: 40
   CHANGELOG_PATH: "CHANGELOG.md"
   DOCKER_USERNAME: "grafana"
+  DRY_RUN: false
   IMAGE_PREFIX: "grafana"
   RELEASE_LIB_REF: "main"
   RELEASE_REPO: "grafana/loki"
@@ -81,14 +83,15 @@ jobs:
           --group-pull-request-title-pattern "chore\${scope}: release\${component} \${version}" \
           --label "backport main,autorelease: pending,product-approved" \
           --manifest-file .release-please-manifest.json \
-          --pull-request-footer "Merging this PR will release the [artifacts](https://console.cloud.google.com/storage/browser/loki-build-artifacts/${SHA}) of ${SHA}" \
+          --pull-request-footer "Merging this PR will release the [artifacts](https://console.cloud.google.com/storage/browser/${BUILD_ARTIFACTS_BUCKET}/${SHA}) of ${SHA}" \
           --pull-request-title-pattern "chore\${scope}: release\${component} \${version}" \
           --release-type simple \
           --repo-url "${{ env.RELEASE_REPO }}" \
           --separate-pull-requests false \
           --target-branch "${{ steps.extract_branch.outputs.branch }}" \
           --token "${{ steps.github_app_token.outputs.token }}" \
-          --versioning-strategy "${{ env.VERSIONING_STRATEGY }}"
+          --versioning-strategy "${{ env.VERSIONING_STRATEGY }}" \
+          --dry-run ${{ fromJSON(env.DRY_RUN) }}
         
       working-directory: "lib"
   dist:
@@ -146,7 +149,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}"
         path: "release/dist"
         process_gcloudignore: false
   fluent-bit:
@@ -203,7 +206,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/fluent-bit-plugin-loki-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -265,7 +268,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/fluent-plugin-loki-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -327,7 +330,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/logcli-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -391,7 +394,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/logstash-output-loki-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -453,7 +456,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/loki-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -517,7 +520,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/loki-canary-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -581,7 +584,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/loki-canary-boringcrypto-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -645,7 +648,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/promtail-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -709,7 +712,7 @@ jobs:
       name: "upload artifacts"
       uses: "google-github-actions/upload-cloud-storage@v2"
       with:
-        destination: "loki-build-artifacts/${{ github.sha }}/images"
+        destination: "${{ env.BUILD_ARTIFACTS_BUCKET }}/${{ github.sha }}/images"
         path: "release/images/loki-query-tee-${{ needs.version.outputs.version}}-${{ steps.platform.outputs.platform }}.tar"
         process_gcloudignore: false
     strategy:
@@ -801,6 +804,7 @@ name: "create release PR"
   push:
     branches:
     - "k[0-9]+"
+    - "main"
 permissions:
   contents: "write"
   id-token: "write"