You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I can override deliveryId in ProducerFactory according to the param label, e.g. override def deliveryId = s"delivery_id_$label" for sure.
I am wondering that, the ConsumerFactory is generic enough, If I have Producers with deliveryId: "delivery_id_a", "delivery_id_b", how can I using ConsumerFactory to consume data from these Producers? As I cannot set deliveryId in annotation according to the param label
@Delivery(id =s"delivery_id_$label")
privatevallabel:String=""// ERROR raised: annotation argument needs to be a constant;
right now, I am creating new pipeline to use ProducerFactory and ConsumerFactory with fixed deliveryId; but how to clean the pipeline if there are some cached/persisted data.
Hello @conderls,
In order to generate the workflow based on relationships between factories and steps, the predictability of dependencies is mandatory, so dynamic delivery ids will broke this requirement.
I don't think this is something that we can acheive, at least not with the current design of the framework. @qxzzxq if you want to add something ? if not you can close the ticket
Hi @conderls,
As @glc-mfelja says, the workflow must be deterministic at runtime. SETL does not support dynamic delivery. This is also due to the fact that Java/Scala's annotation does not support dynamically generated strings.
If I have a producer Factory whose result depends on param
label
,I may use the data from ProducerFactory in another Factory by
I can override
deliveryId
in ProducerFactory according to the paramlabel
, e.g.override def deliveryId = s"delivery_id_$label"
for sure.deliveryId
:"delivery_id_a", "delivery_id_b"
, how can I using ConsumerFactory to consume data from these Producers? As I cannot set deliveryId in annotation according to the paramlabel
deliveryId
; but how to clean the pipeline if there are some cached/persisted data.Originally posted by @conderls in #263
The text was updated successfully, but these errors were encountered: