From b16ddee3a1f8733f7308d11e1135312bb2458b10 Mon Sep 17 00:00:00 2001 From: Robert Fratto Date: Thu, 5 Mar 2020 10:55:33 -0500 Subject: [PATCH] address review feedback This commit leaves logcli.md unmodified; that will be done in a later commit if the changes are approved. --- cmd/logcli/main.go | 30 +++++++++++++++++++++--------- docs/logql.md | 44 +++++++++++++++++++++++++++++--------------- 2 files changed, 50 insertions(+), 24 deletions(-) diff --git a/cmd/logcli/main.go b/cmd/logcli/main.go index f0d43d1c15c9..869449a36318 100644 --- a/cmd/logcli/main.go +++ b/cmd/logcli/main.go @@ -29,13 +29,20 @@ var ( queryCmd = app.Command("query", `Run a LogQL query. -The "query" command is useful for querying for log lines. The default -output of this command are log entries (a combination of timestamp, -labels, and a log line) along with various extra information about -the performed query and its results. Raw log lines (i.e., without a -label and timestamp) can be retrieved by passing the "-o raw" flag. -The extra information about the query (API URL, set of common labels, -excluded labels) can be suppressed with the --query flag. +The "query" command is useful for querying for logs. Logs can be +returned in a few output modes: + + raw: log line + default: log timestamp + log labels + log line + jsonl: JSON response from Loki API of log line + +The output of the log can be specified with the "-o" flag, for +example, "-o raw" for the raw output format. + +The "query" command will output extra information about the query +and its results, such as the API URL, set of common labels, and set +of excluded labels. This extra information can be suppressed with the +--quiet flag. While "query" does support metrics queries, its output contains multiple data points between the start and end query time. This output is used to @@ -55,7 +62,12 @@ view; if you want a metrics query that is used to build a Grafana graph, you should use the "query" command instead. This command does not produce useful output when querying for log lines; -you should always use the "query" command when you are running log queries.`) +you should always use the "query" command when you are running log queries. + +For more information about log queries and metric queries, refer to the +LogQL documentation: + +https://github.com/grafana/loki/blob/master/docs/logql.md`) instantQuery = newQuery(true, instantQueryCmd) labelsCmd = app.Command("labels", "Find values for a given label.") @@ -141,7 +153,7 @@ func newQueryClient(app *kingpin.Application) *client.Client { app.Flag("tls-skip-verify", "Server certificate TLS skip verify.").Default("false").BoolVar(&client.TLSConfig.InsecureSkipVerify) app.Flag("cert", "Path to the client certificate. Can also be set using LOKI_CLIENT_CERT_PATH env var.").Default("").Envar("LOKI_CLIENT_CERT_PATH").StringVar(&client.TLSConfig.CertFile) app.Flag("key", "Path to the client certificate key. Can also be set using LOKI_CLIENT_KEY_PATH env var.").Default("").Envar("LOKI_CLIENT_KEY_PATH").StringVar(&client.TLSConfig.KeyFile) - app.Flag("org-id", "org ID header to be substituted for auth").StringVar(&client.OrgID) + app.Flag("org-id", "adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when bypassing an auth gateway.").StringVar(&client.OrgID) return client } diff --git a/docs/logql.md b/docs/logql.md index 935a02830c1f..8c0c28db9c69 100644 --- a/docs/logql.md +++ b/docs/logql.md @@ -1,25 +1,32 @@ # LogQL: Log Query Language -Loki comes with its very own language for querying logs called *LogQL*. LogQL -can be considered a distributed `grep` with labels for filtering. +Loki comes with its own PromQL-inspired language for queries called *LogQL*. +LogQL can be considered a distributed `grep` that aggregates log sources and +using labels and operators for filtering. -A basic LogQL query consists of two parts: the **log stream selector** and a +There are two types of LogQL queries: *log queries* which return the contents of +log lines, and *metric queries* which extend log queries and calculates values +based on the counts of logs from a log query. + +A basic log query consists of two parts: the **log stream selector** and a **filter expression**. Due to Loki's design, all LogQL queries are required to contain a log stream selector. -The log stream selector will reduce the number of log streams to a manageable -volume. Depending how many labels you use to filter down the log streams will -affect the relative performance of the query's execution. The filter expression -is then used to do a distributed `grep` over the retrieved log streams. +The log stream selector determines how many log streams (unique sources of log +content, such as files) will be searched. A more granular log stream selector +reduces the number of searched streams to a manageable volume. This means that +the labels passed to the log stream selector will affect the relative +performance of the query's execution. The filter expression is then used to do a +distributed `grep` over the aggregated logs from the matching log streams. ### Log Stream Selector The log stream selector determines which log streams should be included in your -query. The stream selector is comprised of one or more key-value pairs, where -each key is a **log label** and the value is that label's value. +query results. The stream selector is comprised of one or more key-value pairs, +where each key is a **log label** and the value is that label's value. -The log stream selector is written by wrapping the key-value pairs in a -pair of curly braces: +The log stream selector is written by wrapping the key-value pairs in a pair of +curly braces: ``` {app="mysql",name="mysql-backup"} @@ -27,7 +34,10 @@ pair of curly braces: In this example, log streams that have a label of `app` whose value is `mysql` _and_ a label of `name` whose value is `mysql-backup` will be included in the -query results. +query results. Note that this will match any log stream whose labels _at least_ +contain `mysql-backup` for their name label; if there are multiple streams that +contain that label, logs from all of the matching streams will be shown in the +results. The `=` operator after the label name is a **label matching operator**. The following label matching operators are supported: @@ -74,10 +84,14 @@ When using `|~` and `!~`, matching is case-sensitive by default and can be switched to case-insensitive prefixing the regex with `(?i)`. -## Counting logs +## Metric Queries + +LogQL also supports wrapping a log query with functions that allows for counting +entries per stream. -LogQL also supports functions that wrap a query and allow for counting entries -per stream. +Metric queries can be used to calculate things such as the rate of error +messages, or the top N log sources with the most amount of logs over the last 3 +hours. ### Range Vector aggregation