duplicate some ops to enable more fusion opportunities and reduce memory footprint #433
Conversation
| // Modern NN networks are usually composed of multiple similar layers. Thus the | ||
| // above patterns are very common especailly when we enable shape constraint ir | ||
| // optimization (if enabled, we will do shape prpagation egaerly, and may | ||
| // further enable cross layer CSE, which in turn increases the change of the |
| let options = [ | ||
| Option<"gpu_enabled_", "gpu-enabled", "bool", | ||
| /*default=*/"true", "whether gpu is available.">, | ||
| Option<"fusion_strategy_", "fusion-strategy", "std::string", |
There was a problem hiding this comment.
Do we actually need the fusion-strategy option? If it is always base, we can remove it in this PR, and add this option if it's required.
There was a problem hiding this comment.
I prefer to leave the option here even though we do not use it now actually. It's better if we make use of such config. Currently implementation is just a conservative strategy. Furthermore, we only duplicate scalar-bcast pattern in this PR, while eventually we will need a general "duplicate fusion" pass like XLA.
|
BTW, any benchmark data on a model about this pass? |
In my test case, it reduces around ~1.5ms (e2e is ~6.5ms). I haven't test this feature on other models. Thus I do not enable this feature by default (guarded by shape-constraint-ir flag). I'll evaluate |
No description provided.