Address missing processor JsonPropertyDescriptions and validations #4837
Conversation
graytaylor0
requested review from
chenqi0805,
engechas,
dinujoh,
kkondaka,
KarstenSchnitter,
dlvenable and
oeyh
as code owners
graytaylor0
force-pushed
the
ProcessorReviews
branch
5 times, most recently
from
3582ebd
to
a35679b
Compare
|
@graytaylor0 looks like tests are failing. |
| @@ -78,6 +79,10 @@ public AggregateProcessor(final AggregateProcessorConfig aggregateProcessorConfi | |||
| this.localMode = aggregateProcessorConfig.getLocalMode(); | |||
|
|
|||
| pluginMetrics.gauge(CURRENT_AGGREGATE_GROUPS, aggregateGroupManager, AggregateGroupManager::getAllGroupsSize); | |||
|
|
|||
| if (aggregateProcessorConfig.getWhenCondition() != null && (!expressionEvaluator.isValidExpressionStatement(aggregateProcessorConfig.getWhenCondition()))) { | |||
There was a problem hiding this comment.
This makes sense to me. Are we adding this validation to all processors now to fail fast on invalid expressions?
| @@ -57,7 +57,7 @@ public class CsvProcessorConfig { | |||
| private List<String> columnNames; | |||
|
|
|||
| @JsonProperty("csv_when") | |||
| @JsonPropertyDescription("Allows you to specify a [conditional expression](https://opensearch.org/docs/latest/data-prepper/pipelines/expression-syntax/), " + | |||
There was a problem hiding this comment.
I might be wrong but I thought we want to keep this markdown format here so it will be rendered in the documentation website. @chenqi0805
There was a problem hiding this comment.
We aren't rendering the documentation website from this.
There was a problem hiding this comment.
Not today, but we are planning to do that, see this: opensearch-project/documentation-website#7651
There was a problem hiding this comment.
Right. We plan to generate the config table from those annotations
Thanks for reviewing. Yeah looks like I missed the GeoIP integration tests I'll fix them |
graytaylor0
force-pushed
the
ProcessorReviews
branch
2 times, most recently
from
d5b5d8d
to
1747c31
Compare
graytaylor0
force-pushed
the
ProcessorReviews
branch
from
1747c31
to
b86fc7f
Compare
Signed-off-by: Taylor Gray <tylgry@amazon.com>
…n, add @Valid annotation for DDB source config models Signed-off-by: Taylor Gray <tylgry@amazon.com>
graytaylor0
force-pushed
the
ProcessorReviews
branch
from
b86fc7f
to
52aef81
Compare
Signed-off-by: Taylor Gray <tylgry@amazon.com>
Description
Adds JsonPropertyDescriptions to all remaining processors that were missing them, and adds validation on construction of the processors for their
whencondition in the processors that were missing this validationCheck List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.