Enhance backward process #18700
Enhance backward process #18700
Conversation
chengduoZH
force-pushed
the
fix_prun_backward
branch
from
2c1c92a
to
b619b11
Compare
| feature = fluid.layers.fc(input=x, size=10, act=None) | ||
| label = fluid.layers.cast(label, dtype="float32") | ||
| label = fluid.layers.cast(label, dtype='int64') | ||
| loss = fluid.layers.cross_entropy(input=feature, label=label) |
There was a problem hiding this comment.
Note that the label is not persistable in fluid.layers.cross_entropy.
There was a problem hiding this comment.
这个解释https://github.com/PaddlePaddle/Paddle/pull/18700/files#r305269149 可以加入单测代码中么？
这个单测的意思是，如果有并不需要的label，也能保持正确么？
test=develop
chengduoZH
force-pushed
the
fix_prun_backward
branch
from
b619b11
to
a59069f
Compare
test=develop
| feature = fluid.layers.fc(input=x, size=10, act=None) | ||
| label = fluid.layers.cast(label, dtype="float32") | ||
| label = fluid.layers.cast(label, dtype='int64') | ||
| loss = fluid.layers.cross_entropy(input=feature, label=label) |
There was a problem hiding this comment.
这个解释https://github.com/PaddlePaddle/Paddle/pull/18700/files#r305269149 可以加入单测代码中么？
这个单测的意思是，如果有并不需要的label，也能保持正确么？
python/paddle/fluid/backward.py
Outdated
| @@ -247,6 +247,104 @@ def _op_can_be_removed_(op_desc, no_grad_set): | |||
| return op_descs | |||
|
|
|||
|
|
|||
| def __find_not_need_ops(grad_op_descs, ops, input_grad_names_set): | |||
There was a problem hiding this comment.
- 可以多加一些注释，说一下整体剪枝的逻辑么，比如是先转成图再剪？
- 会有性能上的影响么？
There was a problem hiding this comment.
同@luotao02
虽然issue里面有详情，也建议在PR里面描述一下。
There was a problem hiding this comment.
可以多加一些注释，说一下整体剪枝的逻辑么，比如是先转成图再剪？
Done
会有性能上的影响么？
时间复杂度是O(N)，与之前构建backward过程的时间复杂度是一样的。
test=develop
chengduoZH
force-pushed
the
fix_prun_backward
branch
from
ab1e51c
to
23a8872
Compare
test=develop
test=develop
| Pruning Program with Structural Analysis Method of Computational Graph. | ||
| The nodes of the computational graph composed of backward OPS should be | ||
| interconnected. If there are unconnected sub-graphs in the computational graph, | ||
| these sub-graphs should be cut off. |
There was a problem hiding this comment.
Maybe better to add comments for grad_op_descs, ops, input_grad_names_set
test=develop
Prune the unnecessary op in backward.
Related PR #15955
Fix #15283