add DataSchema.Builder to tidy stuff up a bit #17065
Conversation
github-actions
bot
added
Area - Batch Ingestion
Area - Streaming Ingestion
Kubernetes
Area - Ingestion
Area - MSQ
![github-advanced-security[bot]](https://github.com/avatars/in/57789?s=60&v=4){kind=small}
| new StringInputRowParser( | ||
| new JSONParseSpec( | ||
| new TimestampSpec(null, null, null), | ||
| DimensionsSpec.EMPTY, | ||
| null, | ||
| null, | ||
| null | ||
| ) | ||
| ), |
Check notice
Code scanning / CodeQL
Deprecated method or constructor invocation Note test
| new StringInputRowParser( | ||
| new JSONParseSpec( | ||
| new TimestampSpec("timestamp", "iso", null), | ||
| new DimensionsSpec( | ||
| Arrays.asList( | ||
| new StringDimensionSchema("dim1"), | ||
| new StringDimensionSchema("dim1t"), | ||
| new StringDimensionSchema("dim2"), | ||
| new LongDimensionSchema("dimLong"), | ||
| new FloatDimensionSchema("dimFloat") | ||
| ) | ||
| ), | ||
| new JSONPathSpec(true, ImmutableList.of()), | ||
| ImmutableMap.of(), | ||
| false | ||
| ) | ||
| ), |
Check notice
Code scanning / CodeQL
Deprecated method or constructor invocation Note test
| DataSchema schema = DataSchema.builder() | ||
| .withDataSource(IdUtilsTest.VALID_ID_CHARS) | ||
| .withParserMap(parser) | ||
| .withAggregators( | ||
| new DoubleSumAggregatorFactory("metric1", "col1"), | ||
| new DoubleSumAggregatorFactory("metric2", "col2"), | ||
| new DoubleSumAggregatorFactory("metric1", "col3"), | ||
| new DoubleSumAggregatorFactory("metric3", "col4"), | ||
| new DoubleSumAggregatorFactory("metric3", "col5") | ||
| ) | ||
| .withGranularity(ARBITRARY_GRANULARITY) | ||
| .withObjectMapper(jsonMapper) | ||
| .build(); |
Check notice
Code scanning / CodeQL
Unread local variable Note test
| .withDimensions(dimensionsAndAggregators.lhs) | ||
| .withAggregators(dimensionsAndAggregators.rhs.toArray(new AggregatorFactory[0])) | ||
| .withGranularity(makeGranularitySpecForIngestion(querySpec.getQuery(), querySpec.getColumnMappings(), isRollupQuery, jsonMapper)) | ||
| .withTransform(new TransformSpec(null, Collections.emptyList())) |
There was a problem hiding this comment.
Could leave this line out, if the default is a no-op TransformSpec.
| return this; | ||
| } | ||
|
|
||
| public Builder withParserMap(Map<String, Object> parserMap) |
There was a problem hiding this comment.
Include @Deprecated to match the annotation in the constructor?
| return this; | ||
| } | ||
|
|
||
| public Builder withObjectMapper(ObjectMapper objectMapper) |
There was a problem hiding this comment.
Include @Deprecated, since this is only used with the parserMap, which is itself deprecated?
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
* add DataSchema.Builder to tidy stuff up a bit * fixes * fixes * more style fixes * review stuff
* add DataSchema.Builder to tidy stuff up a bit * fixes * fixes * more style fixes * review stuff
* abstract `IncrementalIndex` cursor stuff to prepare for using different "views" of the data based on the cursor build spec (#17064) * abstract `IncrementalIndex` cursor stuff to prepare to allow for possibility of using different "views" of the data based on the cursor build spec changes: * introduce `IncrementalIndexRowSelector` interface to capture how `IncrementalIndexCursor` and `IncrementalIndexColumnSelectorFactory` read data * `IncrementalIndex` implements `IncrementalIndexRowSelector` * move `FactsHolder` interface to separate file * other minor refactorings * add DataSchema.Builder to tidy stuff up a bit (#17065) * add DataSchema.Builder to tidy stuff up a bit * fixes * fixes * more style fixes * review stuff * Projections prototype (#17214)
|
This has already been backported in #17257 , so added it to the milestone. |
changes:
DataSchemanow only has a single constructor (the JSON creator)DataSchema.builder()to simplify things a bit, as well as insulate against changes to the JSON schema