Skip to content

Commit

Permalink
Include statement attributes in EXPLAIN PLAN output (#14074)
Browse files Browse the repository at this point in the history
This commit adds attributes that contain metadata information about the query
in the EXPLAIN PLAN output. The attributes currently contain two items:
- `statementTyp`: SELECT, INSERT or REPLACE
- `targetDataSource`: provides the target datasource name for DML statements

It is added to both the legacy and native query plan outputs.
  • Loading branch information
abhishekrb19 authored Apr 17, 2023
1 parent be6745f commit c98c665
Show file tree
Hide file tree
Showing 16 changed files with 388 additions and 151 deletions.
220 changes: 118 additions & 102 deletions docs/querying/sql-translation.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,11 @@ appreciated.

The [EXPLAIN PLAN](sql.md#explain-plan) functionality can help you understand how a given SQL query will
be translated to native.
EXPLAIN PLAN statements return a `RESOURCES` column that describes the resource being queried as well as a `PLAN` column that contains a JSON array of native queries that Druid will run.
EXPLAIN PLAN statements return:
- a `PLAN` column that contains a JSON array of native queries that Druid will run
- a `RESOURCES` column that describes the resource being queried as well as a `PLAN` column that contains a JSON array of native queries that Druid will run
- a `ATTRIBUTES` column that describes the attributes of a query, such as the statement type and target data source

For example, consider the following query:

```sql
Expand All @@ -77,120 +81,132 @@ WHERE channel IN (SELECT page FROM wikipedia GROUP BY page ORDER BY COUNT(*) DES
GROUP BY channel
```

The EXPLAIN PLAN statement returns the following plan:
The EXPLAIN PLAN statement returns the following result with plan, resources, and attributes information in it:

```json
[
{
"query": {
"queryType": "topN",
"dataSource": {
"type": "join",
"left": {
"type": "table",
"name": "wikipedia"
},
"right": {
"type": "query",
"query": {
"queryType": "groupBy",
"dataSource": {
"type": "table",
"name": "wikipedia"
},
"intervals": {
"type": "intervals",
"intervals": [
"-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z"
]
},
"granularity": {
"type": "all"
},
"dimensions": [
{
"type": "default",
"dimension": "page",
"outputName": "d0",
"outputType": "STRING"
}
],
"aggregations": [
{
"type": "count",
"name": "a0"
}
],
"limitSpec": {
"type": "default",
"columns": [
[
{
"query": {
"queryType": "topN",
"dataSource": {
"type": "join",
"left": {
"type": "table",
"name": "wikipedia"
},
"right": {
"type": "query",
"query": {
"queryType": "groupBy",
"dataSource": {
"type": "table",
"name": "wikipedia"
},
"intervals": {
"type": "intervals",
"intervals": [
"-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z"
]
},
"granularity": {
"type": "all"
},
"dimensions": [
{
"dimension": "a0",
"direction": "descending",
"dimensionOrder": {
"type": "numeric"
}
"type": "default",
"dimension": "page",
"outputName": "d0",
"outputType": "STRING"
}
],
"limit": 10
},
"context": {
"sqlOuterLimit": 101,
"sqlQueryId": "ee616a36-c30c-4eae-af00-245127956e42",
"useApproximateCountDistinct": false,
"useApproximateTopN": false
"aggregations": [
{
"type": "count",
"name": "a0"
}
],
"limitSpec": {
"type": "default",
"columns": [
{
"dimension": "a0",
"direction": "descending",
"dimensionOrder": {
"type": "numeric"
}
}
],
"limit": 10
},
"context": {
"sqlOuterLimit": 101,
"sqlQueryId": "ee616a36-c30c-4eae-af00-245127956e42",
"useApproximateCountDistinct": false,
"useApproximateTopN": false
}
}
},
"rightPrefix": "j0.",
"condition": "(\"channel\" == \"j0.d0\")",
"joinType": "INNER"
},
"dimension": {
"type": "default",
"dimension": "channel",
"outputName": "d0",
"outputType": "STRING"
},
"metric": {
"type": "dimension",
"ordering": {
"type": "lexicographic"
}
},
"rightPrefix": "j0.",
"condition": "(\"channel\" == \"j0.d0\")",
"joinType": "INNER"
},
"dimension": {
"type": "default",
"dimension": "channel",
"outputName": "d0",
"outputType": "STRING"
},
"metric": {
"type": "dimension",
"ordering": {
"type": "lexicographic"
"threshold": 101,
"intervals": {
"type": "intervals",
"intervals": [
"-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z"
]
},
"granularity": {
"type": "all"
},
"aggregations": [
{
"type": "count",
"name": "a0"
}
],
"context": {
"sqlOuterLimit": 101,
"sqlQueryId": "ee616a36-c30c-4eae-af00-245127956e42",
"useApproximateCountDistinct": false,
"useApproximateTopN": false
}
},
"threshold": 101,
"intervals": {
"type": "intervals",
"intervals": [
"-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z"
]
},
"granularity": {
"type": "all"
},
"aggregations": [
"signature": [
{
"type": "count",
"name": "a0"
"name": "d0",
"type": "STRING"
},
{
"name": "a0",
"type": "LONG"
}
],
"context": {
"sqlOuterLimit": 101,
"sqlQueryId": "ee616a36-c30c-4eae-af00-245127956e42",
"useApproximateCountDistinct": false,
"useApproximateTopN": false
}
},
"signature": [
{
"name": "d0",
"type": "STRING"
},
{
"name": "a0",
"type": "LONG"
}
]
]
}
],
[
{
"name": "wikipedia",
"type": "DATASOURCE"
}
],
{
"statementType": "SELECT",
"targetDataSource": null
}
]
```
Expand Down
4 changes: 2 additions & 2 deletions docs/querying/sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,8 +250,8 @@ Add "EXPLAIN PLAN FOR" to the beginning of any query to get information about ho
the query will not actually be executed. Refer to the [Query translation](sql-translation.md#interpreting-explain-plan-output)
documentation for more information on the output of EXPLAIN PLAN.

> Be careful when interpreting EXPLAIN PLAN output, and use [request logging](../configuration/index.md#request-logging) if in doubt.
Request logs show the exact native query that will be run.
> For the legacy plan, be careful when interpreting EXPLAIN PLAN output, and use [request logging](../configuration/index.md#request-logging) if in doubt.
Request logs show the exact native query that will be run. Alternatively, to see the native query plan, set `useNativeQueryExplain` to true in the query context.

## Identifiers and literals

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ public void validate() throws SqlParseException, ValidationException
try {
handler.validate();
plannerContext.setResourceActions(handler.resourceActions());
plannerContext.setExplainAttributes(handler.explainAttributes());
}
catch (RuntimeException e) {
throw new ValidationException(e);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.druid.sql.calcite.planner;

import com.fasterxml.jackson.annotation.JsonProperty;
import org.apache.calcite.sql.SqlNode;

import javax.annotation.Nullable;

/**
* ExplainAttributes holds the attributes of a SQL statement that is used in the EXPLAIN PLAN result.
*/
public final class ExplainAttributes
{
private final String statementType;

@Nullable
private final SqlNode targetDataSource;

public ExplainAttributes(
@JsonProperty("statementType") final String statementType,
@JsonProperty("targetDataSource") @Nullable final SqlNode targetDataSource)
{
this.statementType = statementType;
this.targetDataSource = targetDataSource;
}

/**
* @return the statement kind of a SQL statement. For example, SELECT, INSERT, or REPLACE.
*/
@JsonProperty
public String getStatementType()
{
return statementType;
}

/**
* @return the target datasource in a SQL statement. Returns null
* for SELECT/non-DML statements where there is no target datasource.
*/
@Nullable
@JsonProperty
public String getTargetDataSource()
{
return targetDataSource == null ? null : targetDataSource.toString();
}

@Override
public String toString()
{
return "ExplainAttributes{" +
"statementType='" + statementType + '\'' +
", targetDataSource=" + targetDataSource +
'}';
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,15 @@ public void validate() throws ValidationException
}
super.validate();
}

@Override
public ExplainAttributes explainAttributes()
{
return new ExplainAttributes(
DruidSqlInsert.OPERATOR.getName(),
sqlNode.getTargetTable()
);
}
}

/**
Expand Down Expand Up @@ -331,5 +340,14 @@ public void validate() throws ValidationException
);
}
}

@Override
public ExplainAttributes explainAttributes()
{
return new ExplainAttributes(
DruidSqlReplace.OPERATOR.getName(),
sqlNode.getTargetTable()
);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,8 @@ public class PlannerContext
private String planningError;
private QueryMaker queryMaker;
private VirtualColumnRegistry joinExpressionVirtualColumnRegistry;
// set of attributes for a SQL statement used in the EXPLAIN PLAN output
private ExplainAttributes explainAttributes;

private PlannerContext(
final PlannerToolbox plannerToolbox,
Expand Down Expand Up @@ -502,4 +504,18 @@ public void setJoinExpressionVirtualColumnRegistry(VirtualColumnRegistry joinExp
{
this.joinExpressionVirtualColumnRegistry = joinExpressionVirtualColumnRegistry;
}

public ExplainAttributes getExplainAttributes()
{
return this.explainAttributes;
}

public void setExplainAttributes(ExplainAttributes explainAttributes)
{
if (this.explainAttributes != null) {
throw new ISE("ExplainAttributes has already been set");
}
this.explainAttributes = explainAttributes;
}

}
Loading

0 comments on commit c98c665

Please sign in to comment.