Remove Decoder stage (breaking change) #225
Conversation
|
(draft because not tested yet) |
Codecov Report
@@ Coverage Diff @@
## main #225 +/- ##
==========================================
- Coverage 61.33% 60.94% -0.39%
==========================================
Files 67 67
Lines 3843 3795 -48
==========================================
- Hits 2357 2313 -44
+ Misses 1342 1333 -9
- Partials 144 149 +5
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
| func toMap(t *testing.T, in string) config.GenericMap { | ||
| var m config.GenericMap | ||
| err := json.Unmarshal([]byte(in), &m) | ||
| require.NoError(t, err) | ||
| return m |
There was a problem hiding this comment.
Perhaps move this function to test/utils.go
| description: "middle node decode1 does not have outputs", | ||
| config: baseConfig + `- name: decode2 | ||
| decode: | ||
| description: "middle node transform1 does not have outputs", |
There was a problem hiding this comment.
"does not have outputs"? or "does not have inputs"?
There was a problem hiding this comment.
this is really "outputs" not "inputs" here. I guess you were looking at the pipeline defined above, but this description is for the one below:
- { follows: ingest1, name: transform1 }
- { follows: ingest1, name: transform2 }
- { follows: transform2, name: write1 }
| pipeline: | ||
| - { follows: ingest1, name: decode1 } | ||
| - { follows: ingest1, name: decode2 } | ||
| - { follows: decode2, name: write1 } | ||
| - { follows: ingest1, name: transform1 } | ||
| - { follows: ingest1, name: transform2 } | ||
| - { follows: transform2, name: write1 } | ||
| `, | ||
| failingNodeName: "decode1", | ||
| failingNodeName: "transform1", | ||
| }, { | ||
| description: "terminal node write1 does not have inputs", | ||
| config: baseConfig + `- transform: | ||
| type: none |
There was a problem hiding this comment.
It looks like transform1 does not have outputs.
There was a problem hiding this comment.
here again I think you're looking at the wrong pipeline, it's the one defined below
Decoder is removed as a stage, and added wherever it makes sense as Ingest config: relevant only for File ingester and Kafka ingester.
Following netobserv/flowlogs-pipeline#225 , decode stages should not be set after ingesting ipfix or grpc Also rename stages to make them more explicit
Decoder is removed as a stage, and added wherever it makes sense as
Ingest config: relevant only for File ingester and Kafka ingester.
Also remove AWS decoder (after discussion)