From 9af8a5d4155dc58670ca3b2c5d53e65c8e1a808e Mon Sep 17 00:00:00 2001 From: Owen Diehl Date: Thu, 27 Aug 2020 12:30:38 -0400 Subject: [PATCH 1/5] index file links --- docs/README.md | 34 +++++++++---------- docs/sources/_index.md | 2 +- .../best-practices/current-best-practices.md | 2 +- docs/sources/clients/aws/_index.md | 2 +- .../clients/docker-driver/configuration.md | 2 +- docs/sources/clients/fluentbit/_index.md | 6 ++-- docs/sources/clients/promtail/stages/drop.md | 2 +- docs/sources/design-documents/_index.md | 2 +- 8 files changed, 26 insertions(+), 26 deletions(-) diff --git a/docs/README.md b/docs/README.md index ac42ff0a8076..3e17ca31afe7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,51 +14,51 @@ simplifies the operation and significantly lowers the cost of Loki. ## Table of Contents -1. [Overview](sources/overview/_index.md) +1. [Overview](sources/overview) 1. [Comparison to other Log Systems](sources/overview/comparisons.md) -2. [Installation](sources/installation/_index.md) +2. [Installation](sources/installation) 1. [Installing with Tanka](sources/installation/tanka.md) 2. [Installing with Helm](sources/installation/helm.md) 3. [Installing with Docker](sources/installation/docker.md) 4. [Installing locally](sources/installation/local.md) -3. [Getting Started](sources/getting-started/_index.md) +3. [Getting Started](sources/getting-started) 1. [Grafana](sources/getting-started/grafana.md) 2. [LogCLI](sources/getting-started/logcli.md) 3. [Labels](sources/getting-started/labels.md) 4. [Troubleshooting](sources/getting-started/troubleshooting.md) -4. [Best Practices](sources/best-practices/_index.md) +4. [Best Practices](sources/best-practices) 1. [Current Best Practices](sources/best-practices/current-best-practices.md) -5. [Configuration](sources/configuration/_index.md) +5. [Configuration](sources/configuration) 1. [Examples](sources/configuration/examples.md) -6. [Clients](sources/clients/_index.md) - 1. [Promtail](sources/clients/promtail/_index.md) +6. [Clients](sources/clients) + 1. [Promtail](sources/clients/promtail) 1. [Installation](sources/clients/promtail/installation.md) 2. [Configuration](sources/clients/promtail/configuration.md) 3. [Scraping](sources/clients/promtail/scraping.md) 4. [Pipelines](sources/clients/promtail/pipelines.md) 5. [Troubleshooting](sources/clients/promtail/troubleshooting.md) - 2. [Docker Driver](sources/clients/docker-driver/_index.md) + 2. [Docker Driver](sources/clients/docker-driver) 1. [Configuration](sources/clients/docker-driver/configuration.md) - 4. [Fluent Bit](sources/clients/fluentbit/_index.md) - 3. [Fluentd](sources/clients/fluentd/_index.md) -7. [LogQL](sources/logql/_index.md) -8. [Operations](sources/operations/_index.md) + 4. [Fluent Bit](sources/clients/fluentbit) + 3. [Fluentd](sources/clients/fluentd) +7. [LogQL](sources/logql) +8. [Operations](sources/operations) 1. [Authentication](sources/operations/authentication.md) 2. [Observability](sources/operations/observability.md) 3. [Scalability](sources/operations/scalability.md) - 4. [Storage](sources/operations/storage/_index.md) + 4. [Storage](sources/operations/storage) 1. [Table Manager](sources/operations/storage/table-manager.md) 2. [Retention](sources/operations/storage/retention.md) 3. [BoltDB Shipper](sources/operations/storage/boltdb-shipper.md) 5. [Multi-tenancy](sources/operations/multi-tenancy.md) 6. [Loki Canary](sources/operations/loki-canary.md) -9. [HTTP API](sources/api/_index.md) -10. [Architecture](sources/architecture/_index.md) -11. [Community](sources/community/_index.md) +9. [HTTP API](sources/api) +10. [Architecture](sources/architecture) +11. [Community](sources/community) 1. [Governance](sources/community/governance.md) 2. [Getting in Touch](sources/community/getting-in-touch.md) 3. [Contributing to Loki](sources/community/contributing.md) -12. [Loki Maintainers Guide](sources/./maintaining/_index.md) +12. [Loki Maintainers Guide](sources/./maintaining) 1. [Releasing Loki](sources/./maintaining/release.md) ## Updating docs diff --git a/docs/sources/_index.md b/docs/sources/_index.md index d26dc4a4fc69..df95eaeee171 100644 --- a/docs/sources/_index.md +++ b/docs/sources/_index.md @@ -16,4 +16,4 @@ Unlike other logging systems, Loki is built around the idea of only indexing metadata about your logs: labels (just like Prometheus labels). Log data itself is then compressed and stored in chunks in object stores such as S3 or GCS, or even locally on the filesystem. A small index and highly compressed chunks -simplifies the operation and significantly lowers the cost of Loki. \ No newline at end of file +simplifies the operation and significantly lowers the cost of Loki. diff --git a/docs/sources/best-practices/current-best-practices.md b/docs/sources/best-practices/current-best-practices.md index 1db8bcee8e76..ad5457595b2f 100644 --- a/docs/sources/best-practices/current-best-practices.md +++ b/docs/sources/best-practices/current-best-practices.md @@ -121,4 +121,4 @@ Starting in version 1.6.0 Loki and Promtail have flags which will dump the entir `-print-config-stderr` is nice when running loki directly e.g. `./loki ` as you can get a quick output of the entire Loki config. -`-log-config-reverse-order` is the flag we run Loki with in all our environments, the config entries are reversed so that the order of configs reads correctly top to bottom when viewed in Grafana's Explore. \ No newline at end of file +`-log-config-reverse-order` is the flag we run Loki with in all our environments, the config entries are reversed so that the order of configs reads correctly top to bottom when viewed in Grafana's Explore. diff --git a/docs/sources/clients/aws/_index.md b/docs/sources/clients/aws/_index.md index 287b80355f51..d048a475c2e1 100644 --- a/docs/sources/clients/aws/_index.md +++ b/docs/sources/clients/aws/_index.md @@ -1,3 +1,3 @@ --- title: AWS ---- \ No newline at end of file +--- diff --git a/docs/sources/clients/docker-driver/configuration.md b/docs/sources/clients/docker-driver/configuration.md index 9335a1ee18dd..9cc4d1eb5ec7 100644 --- a/docs/sources/clients/docker-driver/configuration.md +++ b/docs/sources/clients/docker-driver/configuration.md @@ -99,7 +99,7 @@ Once deployed, the Grafana service will send its logs to Loki. ## Labels -Loki can received a set of labels along with log line. These labels are used to index log entries and query back logs using [LogQL stream selector](../../logql/_index.md#log-stream-selector). +Loki can received a set of labels along with log line. These labels are used to index log entries and query back logs using [LogQL stream selector](../../logql#log-stream-selector). By default, the Docker driver will add the following labels to each log line: diff --git a/docs/sources/clients/fluentbit/_index.md b/docs/sources/clients/fluentbit/_index.md index 59c608f6e230..b8285fd82464 100644 --- a/docs/sources/clients/fluentbit/_index.md +++ b/docs/sources/clients/fluentbit/_index.md @@ -44,7 +44,7 @@ helm upgrade --install loki-stack loki/loki-stack \ ### AWS Elastic Container Service (ECS) You can use fluent-bit Loki Docker image as a Firelens log router in AWS ECS. -For more information about this see our [AWS documentation](../aws/ecs/_index.md) +For more information about this see our [AWS documentation](../aws/ecs) ### Local @@ -92,7 +92,7 @@ You can also adapt your plugins.conf, removing the need to change the command li ### Labels -Labels are used to [query logs](../../logql/_index.md) `{container_name="nginx", cluster="us-west1"}`, they are usually metadata about the workload producing the log stream (`instance`, `container_name`, `region`, `cluster`, `level`). In Loki labels are indexed consequently you should be cautious when choosing them (high cardinality label values can have performance drastic impact). +Labels are used to [query logs](../../logql) `{container_name="nginx", cluster="us-west1"}`, they are usually metadata about the workload producing the log stream (`instance`, `container_name`, `region`, `cluster`, `level`). In Loki labels are indexed consequently you should be cautious when choosing them (high cardinality label values can have performance drastic impact). You can use `Labels`, `RemoveKeys` , `LabelKeys` and `LabelMapPath` to how the output plugin will perform labels extraction. @@ -103,7 +103,7 @@ If set to true, it will add all Kubernetes labels to Loki labels automatically a ### LabelMapPath When using the `Parser` and `Filter` plugins Fluent Bit can extract and add data to the current record/log data. While Loki labels are key value pair, record data can be nested structures. -You can pass a json file that defines how to extract [labels](../../docs/sources/overview/_index.md#overview-of-loki) from each record. Each json key from the file will be matched with the log record to find label values. Values from the configuration are used as label names. +You can pass a json file that defines how to extract [labels](../../docs/sources/overview#overview-of-loki) from each record. Each json key from the file will be matched with the log record to find label values. Values from the configuration are used as label names. Considering the record below : diff --git a/docs/sources/clients/promtail/stages/drop.md b/docs/sources/clients/promtail/stages/drop.md index 0a8b0e656291..113771420939 100644 --- a/docs/sources/clients/promtail/stages/drop.md +++ b/docs/sources/clients/promtail/stages/drop.md @@ -190,4 +190,4 @@ Given the pipeline: regex: ".*trace.*" ``` -Would drop all logs older than 24h OR longer than 8kb bytes OR have a json `msg` field containing the word _trace_ \ No newline at end of file +Would drop all logs older than 24h OR longer than 8kb bytes OR have a json `msg` field containing the word _trace_ diff --git a/docs/sources/design-documents/_index.md b/docs/sources/design-documents/_index.md index de4bb396a9ff..3e58ebee3a80 100644 --- a/docs/sources/design-documents/_index.md +++ b/docs/sources/design-documents/_index.md @@ -1,4 +1,4 @@ --- title: Design documents menu_exclude: true ---- \ No newline at end of file +--- From d8b908ce3eff3d08d1149e19fe3e794c24bdf21c Mon Sep 17 00:00:00 2001 From: Owen Diehl Date: Thu, 27 Aug 2020 12:34:53 -0400 Subject: [PATCH 2/5] non _index.md links --- docs/README.md | 58 +++++++++---------- docs/sources/clients/aws/ec2/_index.md | 8 +-- docs/sources/clients/aws/ecs/_index.md | 2 +- docs/sources/clients/aws/eks/_index.md | 2 +- docs/sources/clients/docker-driver/_index.md | 2 +- docs/sources/clients/fluentbit/_index.md | 4 +- docs/sources/community/governance.md | 4 +- .../operations/storage/boltdb-shipper.md | 2 +- docs/sources/operations/upgrade.md | 2 +- docs/sources/storage/_index.md | 12 ++-- 10 files changed, 48 insertions(+), 48 deletions(-) diff --git a/docs/README.md b/docs/README.md index 3e17ca31afe7..ada2c716c734 100644 --- a/docs/README.md +++ b/docs/README.md @@ -15,51 +15,51 @@ simplifies the operation and significantly lowers the cost of Loki. ## Table of Contents 1. [Overview](sources/overview) - 1. [Comparison to other Log Systems](sources/overview/comparisons.md) + 1. [Comparison to other Log Systems](sources/overview/comparisons) 2. [Installation](sources/installation) - 1. [Installing with Tanka](sources/installation/tanka.md) - 2. [Installing with Helm](sources/installation/helm.md) - 3. [Installing with Docker](sources/installation/docker.md) - 4. [Installing locally](sources/installation/local.md) + 1. [Installing with Tanka](sources/installation/tanka) + 2. [Installing with Helm](sources/installation/helm) + 3. [Installing with Docker](sources/installation/docker) + 4. [Installing locally](sources/installation/local) 3. [Getting Started](sources/getting-started) - 1. [Grafana](sources/getting-started/grafana.md) - 2. [LogCLI](sources/getting-started/logcli.md) - 3. [Labels](sources/getting-started/labels.md) - 4. [Troubleshooting](sources/getting-started/troubleshooting.md) + 1. [Grafana](sources/getting-started/grafana) + 2. [LogCLI](sources/getting-started/logcli) + 3. [Labels](sources/getting-started/labels) + 4. [Troubleshooting](sources/getting-started/troubleshooting) 4. [Best Practices](sources/best-practices) - 1. [Current Best Practices](sources/best-practices/current-best-practices.md) + 1. [Current Best Practices](sources/best-practices/current-best-practices) 5. [Configuration](sources/configuration) - 1. [Examples](sources/configuration/examples.md) + 1. [Examples](sources/configuration/examples) 6. [Clients](sources/clients) 1. [Promtail](sources/clients/promtail) - 1. [Installation](sources/clients/promtail/installation.md) - 2. [Configuration](sources/clients/promtail/configuration.md) - 3. [Scraping](sources/clients/promtail/scraping.md) - 4. [Pipelines](sources/clients/promtail/pipelines.md) - 5. [Troubleshooting](sources/clients/promtail/troubleshooting.md) + 1. [Installation](sources/clients/promtail/installation) + 2. [Configuration](sources/clients/promtail/configuration) + 3. [Scraping](sources/clients/promtail/scraping) + 4. [Pipelines](sources/clients/promtail/pipelines) + 5. [Troubleshooting](sources/clients/promtail/troubleshooting) 2. [Docker Driver](sources/clients/docker-driver) - 1. [Configuration](sources/clients/docker-driver/configuration.md) + 1. [Configuration](sources/clients/docker-driver/configuration) 4. [Fluent Bit](sources/clients/fluentbit) 3. [Fluentd](sources/clients/fluentd) 7. [LogQL](sources/logql) 8. [Operations](sources/operations) - 1. [Authentication](sources/operations/authentication.md) - 2. [Observability](sources/operations/observability.md) - 3. [Scalability](sources/operations/scalability.md) + 1. [Authentication](sources/operations/authentication) + 2. [Observability](sources/operations/observability) + 3. [Scalability](sources/operations/scalability) 4. [Storage](sources/operations/storage) - 1. [Table Manager](sources/operations/storage/table-manager.md) - 2. [Retention](sources/operations/storage/retention.md) - 3. [BoltDB Shipper](sources/operations/storage/boltdb-shipper.md) - 5. [Multi-tenancy](sources/operations/multi-tenancy.md) - 6. [Loki Canary](sources/operations/loki-canary.md) + 1. [Table Manager](sources/operations/storage/table-manager) + 2. [Retention](sources/operations/storage/retention) + 3. [BoltDB Shipper](sources/operations/storage/boltdb-shipper) + 5. [Multi-tenancy](sources/operations/multi-tenancy) + 6. [Loki Canary](sources/operations/loki-canary) 9. [HTTP API](sources/api) 10. [Architecture](sources/architecture) 11. [Community](sources/community) - 1. [Governance](sources/community/governance.md) - 2. [Getting in Touch](sources/community/getting-in-touch.md) - 3. [Contributing to Loki](sources/community/contributing.md) + 1. [Governance](sources/community/governance) + 2. [Getting in Touch](sources/community/getting-in-touch) + 3. [Contributing to Loki](sources/community/contributing) 12. [Loki Maintainers Guide](sources/./maintaining) - 1. [Releasing Loki](sources/./maintaining/release.md) + 1. [Releasing Loki](sources/./maintaining/release) ## Updating docs diff --git a/docs/sources/clients/aws/ec2/_index.md b/docs/sources/clients/aws/ec2/_index.md index 89fd9ecc751f..8feffdbcb7d1 100644 --- a/docs/sources/clients/aws/ec2/_index.md +++ b/docs/sources/clients/aws/ec2/_index.md @@ -265,7 +265,7 @@ That's it, save the config and you can `reboot` the machine (or simply restart t Let's head back to Grafana and verify that your Promtail logs are available in Grafana by using the [LogQL](../../../logql/) query `{unit="promtail.service"}` in Explore. Finally make sure to checkout [live tailing][live tailing] to see logs appearing as they are ingested in Loki. -[promtail]: ../../promtail/README.md +[promtail]: ../../promtail/README [aws cli]: https://aws.amazon.com/cli/ [create an vpc]: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-subnets-commands-example.html [create an ssh key]: https://confluence.atlassian.com/bitbucketserver/creating-ssh-keys-776639788.html @@ -276,18 +276,18 @@ Let's head back to Grafana and verify that your Promtail logs are available in G [chef]: https://www.chef.io/ [security group]: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html [Amazon Linux 2]: https://aws.amazon.com/amazon-linux-2/ -[promtail configuration]: ../../promtail/configuration.md +[promtail configuration]: ../../promtail/configuration [prometheus scrape config]: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config [ec2_sd_config]: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#ec2_sd_config [role]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html [discovery page]: ./promtail-ec2-discovery.png "Service discovery" [relabel]: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config [systemd]: https://www.freedesktop.org/software/systemd/man/systemd.service.html -[logql]: ../../../logql.md +[logql]: ../../../logql [ec2 logs]: ./promtail-ec2-logs.png "Grafana Loki logs" [config gist]: https://gist.github.com/cyriltovena/d0881cc717757db951b642be48c01445 [labels]: https://grafana.com/blog/2020/04/21/how-labels-in-loki-can-make-log-queries-faster-and-easier/ -[troubleshooting loki]: ../../../getting-started/troubleshooting.md#troubleshooting-targets +[troubleshooting loki]: ../../../getting-started/troubleshooting#troubleshooting-targets [live tailing]: https://grafana.com/docs/grafana/latest/features/datasources/loki/#live-tailing [systemd]: https://github.com/grafana/loki/tree/master/production/helm/promtail#run-promtail-with-systemd-journal-support [journald]: https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html diff --git a/docs/sources/clients/aws/ecs/_index.md b/docs/sources/clients/aws/ecs/_index.md index 8c3fc41d7207..bf7b25075bc4 100644 --- a/docs/sources/clients/aws/ecs/_index.md +++ b/docs/sources/clients/aws/ecs/_index.md @@ -239,4 +239,4 @@ That's it ! Make sure to checkout the [LogQL][logql] to learn more about Loki po [routing]: https://fluentbit.io/documentation/0.13/getting_started/routing.html [grafanacloud account]: https://grafana.com/login [grafana logs firelens]: ./ecs-grafana.png -[logql]: ../../../logql.md +[logql]: ../../../logql diff --git a/docs/sources/clients/aws/eks/_index.md b/docs/sources/clients/aws/eks/_index.md index 0207fdd9cd5a..6a8fe2bb06e9 100644 --- a/docs/sources/clients/aws/eks/_index.md +++ b/docs/sources/clients/aws/eks/_index.md @@ -261,7 +261,7 @@ If you want to push this further you can check out [Joe's blog post][blog annota [blog annotations]: https://grafana.com/blog/2019/12/09/how-to-do-automatic-annotations-with-grafana-and-loki/ [kubectl]: https://kubernetes.io/docs/tasks/tools/install-kubectl/ [eksctl]: https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html -[Promtail]: ./../../promtail/README.md +[Promtail]: ./../../promtail/README [prometheus conf]: https://prometheus.io/docs/prometheus/latest/configuration/configuration/ [eventrouter]: https://github.com/heptiolabs/eventrouter [explore]: https://grafana.com/docs/grafana/latest/features/explore/ diff --git a/docs/sources/clients/docker-driver/_index.md b/docs/sources/clients/docker-driver/_index.md index c99e2a9d126c..b483fce804f6 100644 --- a/docs/sources/clients/docker-driver/_index.md +++ b/docs/sources/clients/docker-driver/_index.md @@ -11,7 +11,7 @@ to a private Loki instance or [Grafana Cloud](https://grafana.com/oss/loki). > [Docker docs](https://docs.docker.com/engine/extend) for more information. Documentation on configuring the Loki Docker Driver can be found on the -[configuration page](./configuration.md). +[configuration page](./configuration). If you have any questions or issues using the Docker plugin feel free to open an issue in this [repository](https://github.com/grafana/loki/issues). diff --git a/docs/sources/clients/fluentbit/_index.md b/docs/sources/clients/fluentbit/_index.md index b8285fd82464..bd04b147528d 100644 --- a/docs/sources/clients/fluentbit/_index.md +++ b/docs/sources/clients/fluentbit/_index.md @@ -21,7 +21,7 @@ docker run -v /var/log:/var/log \ You can run Fluent Bit as a [Daemonset](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) to collect all your Kubernetes workload logs. -To do so you can use our [Fluent Bit helm chart](../../production/helm/fluent-bit/README.md): +To do so you can use our [Fluent Bit helm chart](../../production/helm/fluent-bit/README): > Make sure [tiller](https://helm.sh/docs/install/) is installed correctly in your cluster @@ -48,7 +48,7 @@ For more information about this see our [AWS documentation](../aws/ecs) ### Local -First you need to follow those [instructions](https://github.com/grafana/loki/blob/master/cmd/fluent-bit/README.md) to build the plugin dynamic library. +First you need to follow those [instructions](https://github.com/grafana/loki/blob/master/cmd/fluent-bit/README) to build the plugin dynamic library. The assuming you have Fluent Bit installed in your `$PATH` you can run the plugin using: diff --git a/docs/sources/community/governance.md b/docs/sources/community/governance.md index fde230248f13..9c1b3f03fe89 100644 --- a/docs/sources/community/governance.md +++ b/docs/sources/community/governance.md @@ -156,9 +156,9 @@ The ex-member is If needed, we reserve the right to publicly announce removal. [announce]: https://groups.google.com/forum/#!forum/loki-announce -[coc]: https://github.com/grafana/loki/blob/master/CODE_OF_CONDUCT.md +[coc]: https://github.com/grafana/loki/blob/master/CODE_OF_CONDUCT [devs]: https://groups.google.com/forum/#!forum/loki-developers -[maintainers]: https://github.com/grafana/loki/blob/master/MAINTAINERS.md +[maintainers]: https://github.com/grafana/loki/blob/master/MAINTAINERS [rough]: https://tools.ietf.org/html/rfc7282 [team]: https://groups.google.com/forum/#!forum/loki-team [users]: https://groups.google.com/forum/#!forum/loki-users diff --git a/docs/sources/operations/storage/boltdb-shipper.md b/docs/sources/operations/storage/boltdb-shipper.md index cf75fb95c7f7..67ea34bc229c 100644 --- a/docs/sources/operations/storage/boltdb-shipper.md +++ b/docs/sources/operations/storage/boltdb-shipper.md @@ -91,7 +91,7 @@ ttl can be configured using `cache_ttl` config. ### Write Deduplication disabled -Loki does write deduplication of chunks and index using Chunks and WriteDedupe cache respectively, configured with [ChunkStoreConfig](../../configuration/README.md#chunk_store_config). +Loki does write deduplication of chunks and index using Chunks and WriteDedupe cache respectively, configured with [ChunkStoreConfig](../../configuration/README#chunk_store_config). The problem with write deduplication when using `boltdb-shipper` though is ingesters only keep uploading boltdb files periodically to make them available to all the other services which means there would be a brief period where some of the services would not have received updated index yet. The problem due to that is if an ingester which first wrote the chunks and index goes down and all the other ingesters which were part of replication scheme skipped writing those chunks and index due to deduplication, we would end up missing those logs from query responses since only the ingester which had the index went down. This problem would be faced even during rollouts which is quite common. diff --git a/docs/sources/operations/upgrade.md b/docs/sources/operations/upgrade.md index 2e731807c2ec..8657462f7d02 100644 --- a/docs/sources/operations/upgrade.md +++ b/docs/sources/operations/upgrade.md @@ -297,7 +297,7 @@ max_retries: ## 1.4.0 -Loki 1.4.0 vendors Cortex v0.7.0-rc.0 which contains [several breaking config changes](https://github.com/cortexproject/cortex/blob/v0.7.0-rc.0/CHANGELOG.md). +Loki 1.4.0 vendors Cortex v0.7.0-rc.0 which contains [several breaking config changes](https://github.com/cortexproject/cortex/blob/v0.7.0-rc.0/CHANGELOG). One such config change which will affect Loki users: diff --git a/docs/sources/storage/_index.md b/docs/sources/storage/_index.md index c42e555a8c75..9c8f3b14c885 100644 --- a/docs/sources/storage/_index.md +++ b/docs/sources/storage/_index.md @@ -57,7 +57,7 @@ You may use any subsitutable services, such as those that implement the S3 API l ### Cassandra -Cassandra can also be utilized for the index store and asides from the experimental [boltdb-shipper](./storage/boltdb-shipper.md), it's the only non-cloud offering that can be used for the index that's horizontally scalable and has configurable replication. It's a good candidate when you already run Cassandra, are running on-prem, or do not wish to use a managed cloud offering. +Cassandra can also be utilized for the index store and asides from the experimental [boltdb-shipper](./storage/boltdb-shipper), it's the only non-cloud offering that can be used for the index that's horizontally scalable and has configurable replication. It's a good candidate when you already run Cassandra, are running on-prem, or do not wish to use a managed cloud offering. ### BigTable @@ -73,7 +73,7 @@ DynamoDB is susceptible to rate limiting, particularly due to overconsuming what ### BoltDB -BoltDB is an embedded database on disk. It is not replicated and thus cannot be used for high availability or clustered Loki deployments, but is commonly paired with a `filesystem` chunk store for proof of concept deployments, trying out Loki, and development. There is also an experimental mode, the [boltdb-shipper](./operations/storage/boltdb-shipper.md), which aims to support clustered deployments using `boltdb` as an index. +BoltDB is an embedded database on disk. It is not replicated and thus cannot be used for high availability or clustered Loki deployments, but is commonly paired with a `filesystem` chunk store for proof of concept deployments, trying out Loki, and development. There is also an experimental mode, the [boltdb-shipper](./operations/storage/boltdb-shipper), which aims to support clustered deployments using `boltdb` as an index. ## Period Configs @@ -113,7 +113,7 @@ table_manager: retention_period: 2520h ``` -For more information, see the table manager [doc](./operations/storage/table-manager.md). +For more information, see the table manager [doc](./operations/storage/table-manager). ### Provisioning @@ -132,13 +132,13 @@ table_manager: inactive_read_throughput: | Default = 300 ``` -Note, there are a few other DynamoDB provisioning options including DynamoDB autoscaling and on-demand capacity. See the [docs](./configuration/README.md#provision_config) for more information. +Note, there are a few other DynamoDB provisioning options including DynamoDB autoscaling and on-demand capacity. See the [docs](./configuration/README#provision_config) for more information. ## Upgrading Schemas When a new schema is released and you want to gain the advantages it provides, you can! Loki can transparently query & merge data from across schema boundaries so there is no disruption of service and upgrading is easy. -First, you'll want to create a new [period_config](./configuration/README.md#period_config) entry in your [schema_config](./configuration/README.md#schema_config). The important thing to remember here is to set this at some point in the _future_ and then roll out the config file changes to Loki. This allows the table manager to create the required table in advance of writes and will ensure that existing data isn't queried as if it adheres to the new schema. +First, you'll want to create a new [period_config](./configuration/README.md#period_config) entry in your [schema_config](./configuration/README#schema_config). The important thing to remember here is to set this at some point in the _future_ and then roll out the config file changes to Loki. This allows the table manager to create the required table in advance of writes and will ensure that existing data isn't queried as if it adheres to the new schema. As an example, let's say it's 2020-07-14 and we want to start using the `v11` schema on the 20th: ```yaml @@ -168,7 +168,7 @@ With the exception of the `filesystem` chunk store, Loki will not delete old chu We're interested in adding targeted deletion in future Loki releases (think tenant or stream level granularity) and may include other strategies as well. -For more information, see the configuration [docs](./operations/storage/retention.md). +For more information, see the configuration [docs](./operations/storage/retention). ## Examples From 21f848654fd378d468179e0daaa496fb0336cf39 Mon Sep 17 00:00:00 2001 From: Owen Diehl Date: Thu, 27 Aug 2020 12:53:47 -0400 Subject: [PATCH 3/5] path fix --- docs/sources/clients/lambda-promtail/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/clients/lambda-promtail/_index.md b/docs/sources/clients/lambda-promtail/_index.md index c683cebb61b0..68a5b6be0e46 100644 --- a/docs/sources/clients/lambda-promtail/_index.md +++ b/docs/sources/clients/lambda-promtail/_index.md @@ -1,6 +1,6 @@ # Lambda Promtail -Loki includes an [AWS SAM](https://aws.amazon.com/serverless/sam/) package template for shipping Cloudwatch logs to Loki via a set of promtails [here](../../../tools/lambda-promtail/). This is done via an intermediary [lambda function](https://aws.amazon.com/lambda/) which processes cloudwatch events and propagates them to a promtail instance (or set of instances behind a load balancer) via the push-api [scrape config](docs/clients/promtail/configuration#loki_push_api_config). +Loki includes an [AWS SAM](https://aws.amazon.com/serverless/sam/) package template for shipping Cloudwatch logs to Loki via a set of promtails [here](../../../tools/lambda-promtail/). This is done via an intermediary [lambda function](https://aws.amazon.com/lambda/) which processes cloudwatch events and propagates them to a promtail instance (or set of instances behind a load balancer) via the push-api [scrape config](../promtail/configuration#loki_push_api_config). ## Uses From c59ba6d1bad333ed5f032ec2fcc3ce51ca9cd357 Mon Sep 17 00:00:00 2001 From: Owen Diehl Date: Thu, 27 Aug 2020 13:28:44 -0400 Subject: [PATCH 4/5] base readme links --- README.md | 35 +++++++++--------- docs/README.md | 97 ++------------------------------------------------ 2 files changed, 19 insertions(+), 113 deletions(-) diff --git a/README.md b/README.md index b2406ff0d4e5..1e2a5f7115ec 100644 --- a/README.md +++ b/README.md @@ -29,18 +29,17 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver ## Getting started -* [Installing Loki](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/installation/_index.md) -* [Installing Promtail](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/installation.md) -* [Getting Started Guide](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/_index.md) +* [Installing Loki](https://grafana.com/docs/loki/latest/installation/) +* [Installing Promtail](https://grafana.com/docs/loki/latest/clients/promtail/installation/) +* [Getting Started Guide](https://grafana.com/docs/loki/latest/getting-started/) ## Upgrading -* [Upgrading Loki](https://github.com/grafana/loki/blob/master/docs/sources/operations/upgrade.md) +* [Upgrading Loki](https://grafana.com/docs/loki/latest/operations/upgrade/) ### Documentation -* [master](./docs/README.md) -* [v1.6.0](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/_index.md) +* [master](https://grafana.com/docs/loki/latest/) * [v1.5.0](https://github.com/grafana/loki/tree/v1.5.0/docs/README.md) * [v1.4.1](https://github.com/grafana/loki/tree/v1.4.1/docs/README.md) * [v1.4.0](https://github.com/grafana/loki/tree/v1.4.0/docs/README.md) @@ -49,18 +48,18 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver * [v1.1.0](https://github.com/grafana/loki/tree/v1.1.0/docs/README.md) * [v1.0.0](https://github.com/grafana/loki/tree/v1.0.0/docs/README.md) -Commonly used sections (from the latest release v1.6.0): - -- [API documentation](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/api/_index.md) for alternative ways of getting logs into Loki. -- [Labels](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/labels.md) -- [Operations](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/operations/_index.md) for important aspects of running Loki. -- [Promtail](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/_index.md) is an agent which can tail your log files and push them to Loki. -- [Pipelines](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/pipelines.md) for detailed log processing pipeline documentation -- [Docker Logging Driver](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/docker-driver/_index.md) is a docker plugin to send logs directly to Loki from Docker containers. -- [LogCLI](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/logcli.md) on how to query your logs without Grafana. -- [Loki Canary](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/operations/loki-canary.md) for monitoring your Loki installation for missing logs. -- [Troubleshooting](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/troubleshooting.md) for help around frequent error messages. -- [Loki in Grafana](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/grafana.md) for how to set up a Loki datasource in Grafana and query your logs. +Commonly used sections: + +- [API documentation](https://grafana.com/docs/loki/latest/api/) for alternative ways of getting logs into Loki. +- [Labels](https://grafana.com/docs/loki/latest/getting-started/labels/) +- [Operations](https://grafana.com/docs/loki/latest/operations/) for important aspects of running Loki. +- [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/) is an agent which can tail your log files and push them to Loki. +- [Pipelines](https://grafana.com/docs/loki/latest/clients/promtail/pipelines/) for detailed log processing pipeline documentation +- [Docker Logging Driver](https://grafana.com/docs/loki/latest/clients/docker-driver/) is a docker plugin to send logs directly to Loki from Docker containers. +- [LogCLI](https://grafana.com/docs/loki/latest/getting-started/logcli/) on how to query your logs without Grafana. +- [Loki Canary](https://grafana.com/docs/loki/latest/operations/loki-canary/) for monitoring your Loki installation for missing logs. +- [Troubleshooting](https://grafana.com/docs/loki/latest/getting-started/troubleshooting/) for help around frequent error messages. +- [Loki in Grafana](https://grafana.com/docs/loki/latest/getting-started/grafana/) for how to set up a Loki datasource in Grafana and query your logs. ## Getting Help diff --git a/docs/README.md b/docs/README.md index ada2c716c734..5201cda98a28 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,98 +2,5 @@

Loki Logo
Like Prometheus, but for logs!

- -Grafana Loki is a set of components that can be composed into a fully featured -logging stack. - -Unlike other logging systems, Loki is built around the idea of only indexing -metadata about your logs: labels (just like Prometheus labels). Log data itself -is then compressed and stored in chunks in object stores such as S3 or GCS, or -even locally on the filesystem. A small index and highly compressed chunks -simplifies the operation and significantly lowers the cost of Loki. - -## Table of Contents - -1. [Overview](sources/overview) - 1. [Comparison to other Log Systems](sources/overview/comparisons) -2. [Installation](sources/installation) - 1. [Installing with Tanka](sources/installation/tanka) - 2. [Installing with Helm](sources/installation/helm) - 3. [Installing with Docker](sources/installation/docker) - 4. [Installing locally](sources/installation/local) -3. [Getting Started](sources/getting-started) - 1. [Grafana](sources/getting-started/grafana) - 2. [LogCLI](sources/getting-started/logcli) - 3. [Labels](sources/getting-started/labels) - 4. [Troubleshooting](sources/getting-started/troubleshooting) -4. [Best Practices](sources/best-practices) - 1. [Current Best Practices](sources/best-practices/current-best-practices) -5. [Configuration](sources/configuration) - 1. [Examples](sources/configuration/examples) -6. [Clients](sources/clients) - 1. [Promtail](sources/clients/promtail) - 1. [Installation](sources/clients/promtail/installation) - 2. [Configuration](sources/clients/promtail/configuration) - 3. [Scraping](sources/clients/promtail/scraping) - 4. [Pipelines](sources/clients/promtail/pipelines) - 5. [Troubleshooting](sources/clients/promtail/troubleshooting) - 2. [Docker Driver](sources/clients/docker-driver) - 1. [Configuration](sources/clients/docker-driver/configuration) - 4. [Fluent Bit](sources/clients/fluentbit) - 3. [Fluentd](sources/clients/fluentd) -7. [LogQL](sources/logql) -8. [Operations](sources/operations) - 1. [Authentication](sources/operations/authentication) - 2. [Observability](sources/operations/observability) - 3. [Scalability](sources/operations/scalability) - 4. [Storage](sources/operations/storage) - 1. [Table Manager](sources/operations/storage/table-manager) - 2. [Retention](sources/operations/storage/retention) - 3. [BoltDB Shipper](sources/operations/storage/boltdb-shipper) - 5. [Multi-tenancy](sources/operations/multi-tenancy) - 6. [Loki Canary](sources/operations/loki-canary) -9. [HTTP API](sources/api) -10. [Architecture](sources/architecture) -11. [Community](sources/community) - 1. [Governance](sources/community/governance) - 2. [Getting in Touch](sources/community/getting-in-touch) - 3. [Contributing to Loki](sources/community/contributing) -12. [Loki Maintainers Guide](sources/./maintaining) - 1. [Releasing Loki](sources/./maintaining/release) - -## Updating docs - -### Building the docs locally - -When you contribute to documentation, it is a good practice to build the docs on your local machine to make sure your changes appear as you expect. This README explains the process for doing that. - -#### Requirements - -Docker >= 2.1.0.3 - -### Build the doc site - -1. In the command line, make sure you are in the docs folder: `cd docs`. -2. Run `make docs`. This launches a preview of the docs website at `http://localhost:3002/docs/loki/latest/` which will refresh automatically when changes to content in the `sources` directory are made. - ---- - -### Content guidelines - -Edit content in the `sources` directory. - -### Edit the side menu - -The side menu is built automatically from the folder structure. Ordering is done with the `weight` front matter param. - -### Add images - -Images are sourced in this repo alonside content. They will sync to the website repo just like markdown files. - ---- - -## Deploy changes to grafana.com - -When a PR is merged to master with changes in the `docs/sources` directory, those changes are automatically synched to the grafana/website repo and published to the staging site. - -Generally, someone from marketing will publish to production each day, so as long as the sync is successful your docs edits will be published. + +Please see the [Documentation Site](https://grafana.com/docs/loki/latest/) The files in this directory are used to generate it but consequently the links don't work in Github. From 08fcbadf37d7ad00a1e58243153f062690cf6af3 Mon Sep 17 00:00:00 2001 From: Owen Diehl Date: Thu, 27 Aug 2020 13:38:56 -0400 Subject: [PATCH 5/5] fixes drone badge --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1e2a5f7115ec..c01fbe188f64 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@

Loki Logo

-Drone CI +Drone CI CircleCI Go Report Card Slack