document expression aggregator #14497
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
|
|
||
| Query time only aggregator that can aggregate results using [Druid expressions](./math-expr.md) functions to facilitate building custom functions. | ||
|
|
||
| | property | description | required | |
There was a problem hiding this comment.
Suggested change
| | property | description | required | | |
| | Property | Description | Required | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
|
|
||
| | property | description | required | | ||
| | --- | --- | --- | | ||
| | `type` | must be `expression` | true | |
There was a problem hiding this comment.
Suggested change
| | `type` | must be `expression` | true | | |
| | `type` | Must be `expression`. | true | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | property | description | required | | ||
| | --- | --- | --- | | ||
| | `type` | must be `expression` | true | | ||
| | `name` | aggregator output name | true | |
There was a problem hiding this comment.
Suggested change
| | `name` | aggregator output name | true | | |
| | `name` | The aggregator output name. | true | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | --- | --- | --- | | ||
| | `type` | must be `expression` | true | | ||
| | `name` | aggregator output name | true | | ||
| | `fields` | list of aggregator input columns | true | |
There was a problem hiding this comment.
Suggested change
| | `fields` | list of aggregator input columns | true | | |
| | `fields` | The list of aggregator input columns. | true | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `type` | must be `expression` | true | | ||
| | `name` | aggregator output name | true | | ||
| | `fields` | list of aggregator input columns | true | | ||
| | `accumulatorIdentifier` | variable which identifies the accumulator value in the `fold` and `combine` expressions | false (default `__acc`)| |
There was a problem hiding this comment.
Suggested change
| | `accumulatorIdentifier` | variable which identifies the accumulator value in the `fold` and `combine` expressions | false (default `__acc`)| | |
| | `accumulatorIdentifier` | The variable which identifies the accumulator value in the `fold` and `combine` expressions. | false (default `__acc`)| |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `name` | aggregator output name | true | | ||
| | `fields` | list of aggregator input columns | true | | ||
| | `accumulatorIdentifier` | variable which identifies the accumulator value in the `fold` and `combine` expressions | false (default `__acc`)| | ||
| | `fold` | expression to accumulate values from `fields`. The result of the expression will be stored in `accumulatorIdentifier` and available to the next computation. | true | |
There was a problem hiding this comment.
Suggested change
| | `fold` | expression to accumulate values from `fields`. The result of the expression will be stored in `accumulatorIdentifier` and available to the next computation. | true | | |
| | `fold` | The expression to accumulate values from `fields`. The result of the expression is stored in `accumulatorIdentifier` and available to the next computation. | true | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `fields` | list of aggregator input columns | true | | ||
| | `accumulatorIdentifier` | variable which identifies the accumulator value in the `fold` and `combine` expressions | false (default `__acc`)| | ||
| | `fold` | expression to accumulate values from `fields`. The result of the expression will be stored in `accumulatorIdentifier` and available to the next computation. | true | | ||
| | `combine` | expression to combine the results of various `fold` expressions of each segment when merging results. If not defined and `fold` has a single input column in `fields`, then the `fold` expression may be used, otherwise the input is available to the expression as the `name`| false (default to `fold` expression if and only if the expression has a single input in `fields`)| |
There was a problem hiding this comment.
Suggested change
| | `combine` | expression to combine the results of various `fold` expressions of each segment when merging results. If not defined and `fold` has a single input column in `fields`, then the `fold` expression may be used, otherwise the input is available to the expression as the `name`| false (default to `fold` expression if and only if the expression has a single input in `fields`)| | |
| | `combine` | The expression to combine the results of various `fold` expressions of each segment when merging results. You can use the `fold` expression if `combine` is not defined and `fold` has a single input column in `fields`. Otherwise, the input is available to the expression as `name`.| false (Defaults to `fold` expression if the expression has a single input in `fields`.)| |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `accumulatorIdentifier` | variable which identifies the accumulator value in the `fold` and `combine` expressions | false (default `__acc`)| | ||
| | `fold` | expression to accumulate values from `fields`. The result of the expression will be stored in `accumulatorIdentifier` and available to the next computation. | true | | ||
| | `combine` | expression to combine the results of various `fold` expressions of each segment when merging results. If not defined and `fold` has a single input column in `fields`, then the `fold` expression may be used, otherwise the input is available to the expression as the `name`| false (default to `fold` expression if and only if the expression has a single input in `fields`)| | ||
| | `compare` | comparator expression which can only refer to 2 input variables, `o1` and `o2`, where `o1` and `o2` are the output of `fold` or `combine` expressions, and must adhere to the Java comparator contract. If not set, this will try to fall back to an output type appropriate comparator | false | |
There was a problem hiding this comment.
Suggested change
| | `compare` | comparator expression which can only refer to 2 input variables, `o1` and `o2`, where `o1` and `o2` are the output of `fold` or `combine` expressions, and must adhere to the Java comparator contract. If not set, this will try to fall back to an output type appropriate comparator | false | | |
| | `compare` | The comparator expression which can only refer to two input variables, `o1` and `o2`. `o1` and `o2` represent the output of `fold` or `combine` expressions and must adhere to the Java comparator contract. If not set, `compare` will try to fall back to an output type appropriate comparator. | false | |
What do you mean by "try to fall back"? What happens if it doesn't fall back?
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `fold` | expression to accumulate values from `fields`. The result of the expression will be stored in `accumulatorIdentifier` and available to the next computation. | true | | ||
| | `combine` | expression to combine the results of various `fold` expressions of each segment when merging results. If not defined and `fold` has a single input column in `fields`, then the `fold` expression may be used, otherwise the input is available to the expression as the `name`| false (default to `fold` expression if and only if the expression has a single input in `fields`)| | ||
| | `compare` | comparator expression which can only refer to 2 input variables, `o1` and `o2`, where `o1` and `o2` are the output of `fold` or `combine` expressions, and must adhere to the Java comparator contract. If not set, this will try to fall back to an output type appropriate comparator | false | | ||
| | `finalize` | finalize expression which can only refer to a single input variable, `o`, and is used to perform any final transformation of the output of `fold` or `combine` expressions. If not set, then the value is not transformed | false | |
There was a problem hiding this comment.
Suggested change
| | `finalize` | finalize expression which can only refer to a single input variable, `o`, and is used to perform any final transformation of the output of `fold` or `combine` expressions. If not set, then the value is not transformed | false | | |
| | `finalize` | The finalize expression which can only refer to a single input variable, `o`. You use `finalize` to perform final transformation of the output of `fold` or `combine` expressions. If not set, the value is not transformed. | false | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `combine` | expression to combine the results of various `fold` expressions of each segment when merging results. If not defined and `fold` has a single input column in `fields`, then the `fold` expression may be used, otherwise the input is available to the expression as the `name`| false (default to `fold` expression if and only if the expression has a single input in `fields`)| | ||
| | `compare` | comparator expression which can only refer to 2 input variables, `o1` and `o2`, where `o1` and `o2` are the output of `fold` or `combine` expressions, and must adhere to the Java comparator contract. If not set, this will try to fall back to an output type appropriate comparator | false | | ||
| | `finalize` | finalize expression which can only refer to a single input variable, `o`, and is used to perform any final transformation of the output of `fold` or `combine` expressions. If not set, then the value is not transformed | false | | ||
| | `initialValue` | initial value of the accumulator for `fold` (and `combine`, if `InitialCombineValue` is null) expression | true | |
There was a problem hiding this comment.
Suggested change
| | `initialValue` | initial value of the accumulator for `fold` (and `combine`, if `InitialCombineValue` is null) expression | true | | |
| | `initialValue` | The initial value of the accumulator for `fold` (and `combine`, if `InitialCombineValue` is null) expression. | true | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `compare` | comparator expression which can only refer to 2 input variables, `o1` and `o2`, where `o1` and `o2` are the output of `fold` or `combine` expressions, and must adhere to the Java comparator contract. If not set, this will try to fall back to an output type appropriate comparator | false | | ||
| | `finalize` | finalize expression which can only refer to a single input variable, `o`, and is used to perform any final transformation of the output of `fold` or `combine` expressions. If not set, then the value is not transformed | false | | ||
| | `initialValue` | initial value of the accumulator for `fold` (and `combine`, if `InitialCombineValue` is null) expression | true | | ||
| | `initialCombineValue` | initial value of the accumulator for `combine` expression | false (default `initialValue`) | |
There was a problem hiding this comment.
Suggested change
| | `initialCombineValue` | initial value of the accumulator for `combine` expression | false (default `initialValue`) | | |
| | `initialCombineValue` | The initial value of the accumulator for the `combine` expression. | false (defaults to `initialValue`) | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `combine` | expression to combine the results of various `fold` expressions of each segment when merging results. If not defined and `fold` has a single input column in `fields`, then the `fold` expression may be used, otherwise the input is available to the expression as the `name`| false (default to `fold` expression if and only if the expression has a single input in `fields`)| | ||
| | `compare` | comparator expression which can only refer to 2 input variables, `o1` and `o2`, where `o1` and `o2` are the output of `fold` or `combine` expressions, and must adhere to the Java comparator contract. If not set, this will try to fall back to an output type appropriate comparator | false | | ||
| | `finalize` | finalize expression which can only refer to a single input variable, `o`, and is used to perform any final transformation of the output of `fold` or `combine` expressions. If not set, then the value is not transformed | false | | ||
| | `initialValue` | initial value of the accumulator for `fold` (and `combine`, if `InitialCombineValue` is null) expression | true | |
There was a problem hiding this comment.
Suggested change
| | `initialValue` | initial value of the accumulator for `fold` (and `combine`, if `InitialCombineValue` is null) expression | true | | |
| | `initialValue` | initial value of the accumulator for the `fold` (and `combine`, if `InitialCombineValue` is null) expression | true | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `finalize` | finalize expression which can only refer to a single input variable, `o`, and is used to perform any final transformation of the output of `fold` or `combine` expressions. If not set, then the value is not transformed | false | | ||
| | `initialValue` | initial value of the accumulator for `fold` (and `combine`, if `InitialCombineValue` is null) expression | true | | ||
| | `initialCombineValue` | initial value of the accumulator for `combine` expression | false (default `initialValue`) | | ||
| | `isNullUnlessAggregated` | indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | false (defaults to value of `druid.generic.useDefaultValueForNull`)| |
There was a problem hiding this comment.
Suggested change
| | `isNullUnlessAggregated` | indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | false (defaults to value of `druid.generic.useDefaultValueForNull`)| | |
| | `isNullUnlessAggregated` | Indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | false (defaults to the value of `druid.generic.useDefaultValueForNull`)| |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `initialValue` | initial value of the accumulator for `fold` (and `combine`, if `InitialCombineValue` is null) expression | true | | ||
| | `initialCombineValue` | initial value of the accumulator for `combine` expression | false (default `initialValue`) | | ||
| | `isNullUnlessAggregated` | indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | false (defaults to value of `druid.generic.useDefaultValueForNull`)| | ||
| | `shouldAggregateNullInputs` | indicates if the `fold` expression should operate on any `null` input values | false (default value is `true`) | |
There was a problem hiding this comment.
Suggested change
| | `shouldAggregateNullInputs` | indicates if the `fold` expression should operate on any `null` input values | false (default value is `true`) | | |
| | `shouldAggregateNullInputs` | Indicates that the `fold` expression should operate on any `null` input values. | false (defaults to `true`) | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `initialCombineValue` | initial value of the accumulator for `combine` expression | false (default `initialValue`) | | ||
| | `isNullUnlessAggregated` | indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | false (defaults to value of `druid.generic.useDefaultValueForNull`)| | ||
| | `shouldAggregateNullInputs` | indicates if the `fold` expression should operate on any `null` input values | false (default value is `true`) | | ||
| | `shouldCombineAggregateNullInputs` | indicates if the `combine` expression should operate on any `null` input values | false (default value is `shouldAggregateNullInputs`) | |
There was a problem hiding this comment.
Suggested change
| | `shouldCombineAggregateNullInputs` | indicates if the `combine` expression should operate on any `null` input values | false (default value is `shouldAggregateNullInputs`) | | |
| | `shouldCombineAggregateNullInputs` | Indicates if the `combine` expression should operate on any `null` input values. | false (defaults to the value of `shouldAggregateNullInputs`) | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| | `isNullUnlessAggregated` | indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | false (defaults to value of `druid.generic.useDefaultValueForNull`)| | ||
| | `shouldAggregateNullInputs` | indicates if the `fold` expression should operate on any `null` input values | false (default value is `true`) | | ||
| | `shouldCombineAggregateNullInputs` | indicates if the `combine` expression should operate on any `null` input values | false (default value is `shouldAggregateNullInputs`) | | ||
| | `maxSizeBytes` | maximum size in bytes that variably sized aggregator output types such as strings and arrays are allowed to grow before the aggregation will fail. | false (8192 bytes) | |
There was a problem hiding this comment.
Suggested change
| | `maxSizeBytes` | maximum size in bytes that variably sized aggregator output types such as strings and arrays are allowed to grow before the aggregation will fail. | false (8192 bytes) | | |
| | `maxSizeBytes` | The maximum size in bytes that variably sized aggregator output types such as strings and arrays are allowed to grow to before the aggregation fails. | false (8192 bytes) | |
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| } | ||
| ``` | ||
|
|
||
| > JavaScript-based functionality is disabled by default. Please refer to the Druid [JavaScript programming guide](../development/javascript.md) for guidelines about using Druid's JavaScript functionality, including instructions on how to enable it. |
There was a problem hiding this comment.
Suggested change
| > JavaScript-based functionality is disabled by default. Please refer to the Druid [JavaScript programming guide](../development/javascript.md) for guidelines about using Druid's JavaScript functionality, including instructions on how to enable it. | |
| > JavaScript-based functionality is disabled by default. Refer to the Druid [JavaScript programming guide](../development/javascript.md) for guidelines about using Druid's JavaScript functionality, including instructions on how to enable it. |
There was a problem hiding this comment.
this isn't new, i just moved it, but i can adjust it...
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
|
|
||
| ### Expression aggregator | ||
|
|
||
| Query time only aggregator that can aggregate results using [Druid expressions](./math-expr.md) functions to facilitate building custom functions. |
There was a problem hiding this comment.
I think it would be helpful to show the expression aggregator here. Similar to what you have for the JavaScript aggregator on line 478.
There was a problem hiding this comment.
i don't really understand, there are several real examples of the expression aggregator after the table, and the table seems a more suitable place to explain the syntax rather than a pseudo json example like the javascript aggregator uses...
There was a problem hiding this comment.
I think it would probably be better if all of the aggregators on this page had a table to explain all of the parameters and then saved the json for realistic examples, but .. i wasn't very motivated to fix the whole page 😅
There was a problem hiding this comment.
Works for me. Fixing the whole page is quite an undertaking :)
There was a problem hiding this comment.
eh, i think maybe i will just try to fix the whole page once i get around to fixing up this PR based on review, seems like it won't be that bad
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
|
|
||
| Query time only aggregator that can aggregate results using [Druid expressions](./math-expr.md) functions to facilitate building custom functions. | ||
|
|
||
| | property | description | required | |
There was a problem hiding this comment.
The rows of the Required column should say Yes or No instead of true or false.
For example:
| type | Must be expression. | Yes |
clintropolis
commented
Jun 29, 2023
|
|
||
| ## Expression aggregators | ||
|
|
||
| ### Expression aggregator |
There was a problem hiding this comment.
i considered both this aggregator and the javascript aggregator as free form "expression" aggregators since you can just write whatever functions you want, which is why they are both under the category. can you think of a better category name than "Expression aggregators"? The native expression aggregator and javascript aggregator are totally separate, just similar in spirit...
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| @@ -422,6 +389,121 @@ It is not possible to determine a priori how well this aggregator will behave fo | |||
|
|
|||
| For these reasons, we have deprecated this aggregator and recommend using the DataSketches Quantiles aggregator instead for new and existing use cases, although we will continue to support Approximate Histogram for backwards compatibility. | |||
|
|
|||
|
|
|||
| ## Expression aggregators | |||
There was a problem hiding this comment.
Do we need to list both the expression aggregator and the JavaScript aggregator under Expression aggregators? Can we remove line 393 and change line 395 to H2 (## Expression aggregator)? If so, you can add an H3 section called Examples and list all of the expression aggregator examples there.
For example:
## Expression aggregator
### Examples
## JavaScript aggregator
There was a problem hiding this comment.
I'm just wondering is the JavaScript agg worth it's own section? It's not enabled by default due to security issues
There was a problem hiding this comment.
What about "Expression aggregations" for H2 and then "Expression aggregator" and "JavaScript aggregator"? So something like this:
H2 Expression aggregations
H3 Expression aggregator
H4 Examples
H3 JavaScript aggregator
H4 Example
ektravel
reviewed
Jun 29, 2023
docs/querying/aggregations.md
Outdated
| } | ||
| ``` | ||
|
|
||
| **Example** |
There was a problem hiding this comment.
Suggested change
| **Example** | |
| #### Example |
…into document-expression-aggregator
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
|
|
||
| ### Expression aggregator | ||
|
|
||
| Query time only aggregator that can aggregate results using [Druid expressions](./math-expr.md) functions to facilitate building custom functions. |
There was a problem hiding this comment.
Suggested change
| Query time only aggregator that can aggregate results using [Druid expressions](./math-expr.md) functions to facilitate building custom functions. | |
| Aggregator applicable only at query time. Aggregates results using [Druid expressions](./math-expr.md) to facilitate building custom functions. |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| | `maxSizeBytes` | Maximum size in bytes that variably sized aggregator output types such as strings and arrays are allowed to grow to before the aggregation fails. | No (8192 bytes) | | ||
|
|
||
| #### Example: a "count" aggregator | ||
| The initial value is `0` and adds `1` for each row processed. |
There was a problem hiding this comment.
Suggested change
| The initial value is `0` and adds `1` for each row processed. | |
| The initial value is `0`. `fold` adds `1` for each row processed. |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| ``` | ||
|
|
||
| #### Example: a "sum" aggregator | ||
| The initial value is `0`, adds the numeric value `column_a` for each row processed. |
There was a problem hiding this comment.
Suggested change
| The initial value is `0`, adds the numeric value `column_a` for each row processed. | |
| The initial value is `0`. `fold` adds the numeric value `column_a` for each row processed. |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| ``` | ||
|
|
||
| #### Example: a "distinct array element" aggregator, sorted by array_length | ||
| The initial value is an empty array, `fold` adds the elements of `column_a` to the accumulator using set semantics, `combine` merges the sets, and `compare` orders the values by `array_length`. |
There was a problem hiding this comment.
Suggested change
| The initial value is an empty array, `fold` adds the elements of `column_a` to the accumulator using set semantics, `combine` merges the sets, and `compare` orders the values by `array_length`. | |
| The initial value is an empty array. `fold` adds the elements of `column_a` to the accumulator using set semantics, `combine` merges the sets, and `compare` orders the values by `array_length`. |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| ``` | ||
|
|
||
| #### Example: an "approximate count" aggregator using the built-in hyper-unique | ||
| Similar to the 'cardinality' aggregator, the default value is an empty hyper-unique sketch, `fold` adds the value of `column_a` to the sketch, `combine` merges the sketches, and `finalize` gets the estimated count from the accumulated sketch. |
There was a problem hiding this comment.
Suggested change
| Similar to the 'cardinality' aggregator, the default value is an empty hyper-unique sketch, `fold` adds the value of `column_a` to the sketch, `combine` merges the sketches, and `finalize` gets the estimated count from the accumulated sketch. | |
| Similar to the cardinality aggregator, the default value is an empty hyper-unique sketch, `fold` adds the value of `column_a` to the sketch, `combine` merges the sketches, and `finalize` gets the estimated count from the accumulated sketch. |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| | `type` | Must be "javascript". | Yes | | ||
| | `name` | The aggregator output name. | Yes | | ||
| | `fieldNames` | The list of aggregator input columns. | Yes | | ||
| | `fnAggregate` | Javascript function that updates partial aggregate based on the current row values, and returns the updated partial aggregate. | Yes | |
There was a problem hiding this comment.
Suggested change
| | `fnAggregate` | Javascript function that updates partial aggregate based on the current row values, and returns the updated partial aggregate. | Yes | | |
| | `fnAggregate` | JavaScript function that updates partial aggregate based on the current row values, and returns the updated partial aggregate. | Yes | |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| | `name` | The aggregator output name. | Yes | | ||
| | `fieldNames` | The list of aggregator input columns. | Yes | | ||
| | `fnAggregate` | Javascript function that updates partial aggregate based on the current row values, and returns the updated partial aggregate. | Yes | | ||
| | `fnCombine` | Javascript function to combine partial aggregates and return the combined result. | Yes | |
There was a problem hiding this comment.
Suggested change
| | `fnCombine` | Javascript function to combine partial aggregates and return the combined result. | Yes | | |
| | `fnCombine` | JavaScript function to combine partial aggregates and return the combined result. | Yes | |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| | `fieldNames` | The list of aggregator input columns. | Yes | | ||
| | `fnAggregate` | Javascript function that updates partial aggregate based on the current row values, and returns the updated partial aggregate. | Yes | | ||
| | `fnCombine` | Javascript function to combine partial aggregates and return the combined result. | Yes | | ||
| | `fnReset` | Javascript function that returns the 'initial' value. | Yes | |
There was a problem hiding this comment.
Suggested change
| | `fnReset` | Javascript function that returns the 'initial' value. | Yes | | |
| | `fnReset` | JavaScript function that returns the initial value. | Yes | |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| } | ||
| ``` | ||
|
|
||
| > JavaScript-based functionality is disabled by default. Refer to the Druid [JavaScript programming guide](../development/javascript.md) for guidelines about using Druid's JavaScript functionality, including instructions on how to enable it. |
There was a problem hiding this comment.
Suggested change
| > JavaScript-based functionality is disabled by default. Refer to the Druid [JavaScript programming guide](../development/javascript.md) for guidelines about using Druid's JavaScript functionality, including instructions on how to enable it. | |
| > JavaScript functionality is disabled by default. Refer to the Druid [JavaScript programming guide](../development/javascript.md) for guidelines about using Druid's JavaScript functionality, including instructions on how to enable it. |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| @@ -432,15 +581,28 @@ This makes it possible to compute the results of a filtered and an unfiltered ag | |||
|
|
|||
| *Note:* If only the filtered results are required, consider putting the filter on the query itself, which will be much faster since it does not require scanning all the data. | |||
There was a problem hiding this comment.
Suggested change
| *Note:* If only the filtered results are required, consider putting the filter on the query itself, which will be much faster since it does not require scanning all the data. | |
| If only the filtered results are required, consider putting the filter on the query itself, which will be much faster since it does not require scanning all the data. |
ektravel
reviewed
Aug 2, 2023
docs/querying/aggregations.md
Outdated
| | `groupings` | The list of columns to use in the grouping set. | Yes | | ||
|
|
||
|
|
||
| For example, if the aggregator has `["dim1", "dim2"]` as input dimensions: |
There was a problem hiding this comment.
Suggested change
| For example, if the aggregator has `["dim1", "dim2"]` as input dimensions: | |
| For example, the following aggregator has `["dim1", "dim2"]` as input dimensions: | |
| { "type" : "grouping", "name" : "someGrouping", "groupings" : ["dim1", "dim2"] } | |
| If you use this aggregator in a query with `[["dim1", "dim2"], ["dim1"], ["dim2"], []]` as subtotals, the aggregator produces the following output: |
Updated the text of the example to make it easier to read and comprehend.
ektravel
approved these changes
Aug 2, 2023
writer-jill
suggested changes
Aug 7, 2023
docs/querying/aggregations.md
Outdated
| | --- | --- | --- | | ||
| | `type` | Must be "longSum", "doubleSum", or "floatSum". | Yes | | ||
| | `name` | Output name for the summed value. | Yes | | ||
| | `fieldName` | Name of the input column to sum over. At most one of `fieldName` or `expression` must be defined. | No | |
There was a problem hiding this comment.
Suggested change
| | `fieldName` | Name of the input column to sum over. At most one of `fieldName` or `expression` must be defined. | No | | |
| | `fieldName` | Name of the input column to sum over. You must define `fieldName` or `expression`. | No | |
There was a problem hiding this comment.
is that clear enough that it is an error to define both?
docs/querying/aggregations.md
Outdated
| | `type` | Must be "longSum", "doubleSum", or "floatSum". | Yes | | ||
| | `name` | Output name for the summed value. | Yes | | ||
| | `fieldName` | Name of the input column to sum over. At most one of `fieldName` or `expression` must be defined. | No | | ||
| | `expression` | Alternative to `fieldName`, an inline [expression](./math-expr.md) can be specified instead. At most one of `fieldName` or `expression` can be defined. | No | |
There was a problem hiding this comment.
Suggested change
| | `expression` | Alternative to `fieldName`, an inline [expression](./math-expr.md) can be specified instead. At most one of `fieldName` or `expression` can be defined. | No | | |
| | `expression` | You can specify an inline [expression](./math-expr.md) as an alternative to `fieldName`. You must define `fieldName` or `expression`. | No | |
docs/querying/aggregations.md
Outdated
| | --- | --- | --- | | ||
| | `type` | Must be "doubleMin", "doubleMax", "floatMin", "floatMax", "longMin", or "longMax". | Yes | | ||
| | `name` | Output name for the min or max value. | Yes | | ||
| | `fieldName` | Name of the input column to compute the minimum or maximum value over. At most one of `fieldName` or `expression` can be defined. | No | |
There was a problem hiding this comment.
Suggested change
| | `fieldName` | Name of the input column to compute the minimum or maximum value over. At most one of `fieldName` or `expression` can be defined. | No | | |
| | `fieldName` | Name of the input column to compute the minimum or maximum value over. You must specify `fieldName` or `expression`. | No | |
docs/querying/aggregations.md
Outdated
| | `type` | Must be "doubleMin", "doubleMax", "floatMin", "floatMax", "longMin", or "longMax". | Yes | | ||
| | `name` | Output name for the min or max value. | Yes | | ||
| | `fieldName` | Name of the input column to compute the minimum or maximum value over. At most one of `fieldName` or `expression` can be defined. | No | | ||
| | `expression` | Alternative to `fieldName`, an inline [expression](./math-expr.md) can be specified instead. At most one of `fieldName` or `expression` can be defined. | No | |
There was a problem hiding this comment.
Suggested change
| | `expression` | Alternative to `fieldName`, an inline [expression](./math-expr.md) can be specified instead. At most one of `fieldName` or `expression` can be defined. | No | | |
| | `expression` | You can specify an inline [expression](./math-expr.md) as an alternative to `fieldName`. You must define `fieldName` or `expression`. | No | |
docs/querying/aggregations.md
Outdated
| | `type` | Must be "stringFirst", "stringLast". | Yes | | ||
| | `name` | Output name for the first or last value. | Yes | | ||
| | `fieldName` | Name of the input column to compute the first or last value over. | Yes | | ||
| | `timeColumn` | Name of the input column to use for time values. Must be a LONG typed column. | No, defaults to `__time` | |
There was a problem hiding this comment.
Suggested change
| | `timeColumn` | Name of the input column to use for time values. Must be a LONG typed column. | No, defaults to `__time` | | |
| | `timeColumn` | Name of the input column to use for time values. Must be a LONG typed column. | No. Defaults to `__time` | |
docs/querying/aggregations.md
Outdated
| | `type` | Must be "expression". | Yes | | ||
| | `name` | The aggregator output name. | Yes | | ||
| | `fields` | The list of aggregator input columns. | Yes | | ||
| | `accumulatorIdentifier` | The variable which identifies the accumulator value in the `fold` and `combine` expressions. | No (default `__acc`)| |
There was a problem hiding this comment.
Suggested change
| | `accumulatorIdentifier` | The variable which identifies the accumulator value in the `fold` and `combine` expressions. | No (default `__acc`)| | |
| | `accumulatorIdentifier` | The variable which identifies the accumulator value in the `fold` and `combine` expressions. | No. Default is `__acc`| |
docs/querying/aggregations.md
Outdated
| | `name` | The aggregator output name. | Yes | | ||
| | `fields` | The list of aggregator input columns. | Yes | | ||
| | `accumulatorIdentifier` | The variable which identifies the accumulator value in the `fold` and `combine` expressions. | No (default `__acc`)| | ||
| | `fold` | The expression to accumulate values from `fields`. The result of the expression will be stored in `accumulatorIdentifier` and available to the next computation. | Yes | |
There was a problem hiding this comment.
Suggested change
| | `fold` | The expression to accumulate values from `fields`. The result of the expression will be stored in `accumulatorIdentifier` and available to the next computation. | Yes | | |
| | `fold` | The expression to accumulate values from `fields`. The result of the expression is stored in `accumulatorIdentifier` and available to the next computation. | Yes | |
docs/querying/aggregations.md
Outdated
| | `fields` | The list of aggregator input columns. | Yes | | ||
| | `accumulatorIdentifier` | The variable which identifies the accumulator value in the `fold` and `combine` expressions. | No (default `__acc`)| | ||
| | `fold` | The expression to accumulate values from `fields`. The result of the expression will be stored in `accumulatorIdentifier` and available to the next computation. | Yes | | ||
| | `combine` | The expression to combine the results of various `fold` expressions of each segment when merging results. The input is available to the expression as a variable identified by the `name`. | No (default to `fold` expression if and only if the expression has a single input in `fields`)| |
There was a problem hiding this comment.
Suggested change
| | `combine` | The expression to combine the results of various `fold` expressions of each segment when merging results. The input is available to the expression as a variable identified by the `name`. | No (default to `fold` expression if and only if the expression has a single input in `fields`)| | |
| | `combine` | The expression to combine the results of various `fold` expressions of each segment when merging results. The input is available to the expression as a variable identified by the `name`. | No. Default is `fold` expression if the expression has a single input in `fields`)| |
docs/querying/aggregations.md
Outdated
| | `isNullUnlessAggregated` | Indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | No (defaults to the value of `druid.generic.useDefaultValueForNull`)| | ||
| | `shouldAggregateNullInputs` | Indicates if the `fold` expression should operate on any `null` input values. | No (defaults to `true`) | | ||
| | `shouldCombineAggregateNullInputs` | Indicates if the `combine` expression should operate on any `null` input values. | No (defaults to the value of `shouldAggregateNullInputs`) | | ||
| | `maxSizeBytes` | Maximum size in bytes that variably sized aggregator output types such as strings and arrays are allowed to grow to before the aggregation fails. | No (8192 bytes) | |
There was a problem hiding this comment.
Suggested change
| | `maxSizeBytes` | Maximum size in bytes that variably sized aggregator output types such as strings and arrays are allowed to grow to before the aggregation fails. | No (8192 bytes) | | |
| | `maxSizeBytes` | Maximum size in bytes that variably sized aggregator output types such as strings and arrays are allowed to grow to before the aggregation fails. | No. Default is 8192 bytes. | |
docs/querying/aggregations.md
Outdated
| @@ -430,17 +579,30 @@ A filtered aggregator wraps any given aggregator, but only aggregates the values | |||
|
|
|||
| This makes it possible to compute the results of a filtered and an unfiltered aggregation simultaneously, without having to issue multiple queries, and use both results as part of post-aggregations. | |||
|
|
|||
| *Note:* If only the filtered results are required, consider putting the filter on the query itself, which will be much faster since it does not require scanning all the data. | |||
| If only the filtered results are required, consider putting the filter on the query itself, which will be much faster since it does not require scanning all the data. | |||
There was a problem hiding this comment.
Suggested change
| If only the filtered results are required, consider putting the filter on the query itself, which will be much faster since it does not require scanning all the data. | |
| If only the filtered results are required, consider putting the filter on the query itself. This will be much faster because it does not require scanning all the data. |
317brian
approved these changes
Aug 8, 2023
There was a problem hiding this comment.
LGTM with the caveat that custom.css needs to be restored. Those CSS classes are part of the API refactor/clean-up @demo-kratia is working on.
This comment was marked as duplicate.
This comment was marked as duplicate.
Sorry, something went wrong.
vtlim
approved these changes
Aug 8, 2023
clintropolis
added a commit
to clintropolis/druid
that referenced
this pull request
Aug 8, 2023
AmatyaAvadhanula
pushed a commit
that referenced
this pull request
Aug 11, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Documents the expression aggregator added in #11104, since it seems to have stabilized. I also provided a bunch of examples to hopefully make it easier to understand.
Finally, I updated the rest of the aggregators to put the parameter descriptions in tables so that things are more consistently formatted, and transitioned the JSON snippets into examples.
I made a new 'Expression aggregators' section and moved the Javascript aggregator into this section. I imagine there is maybe a better categorization of these things, though I'm not entirely sure what it is...