[Zero-Dim] distributed all_gather/scatter/all_to_all support input 0D tensor #53186
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
❌ The PR is not created using PR's template. You can refer to this Demo. |
zhwesky2010
changed the title
zhwesky2010
force-pushed
the
0d_dist_scatter
branch
3 times, most recently
from
4d58d16
to
69ee1cf
Compare
zhwesky2010
force-pushed
the
0d_dist_scatter
branch
from
69ee1cf
to
33f009c
Compare
zhwesky2010
changed the title
LiYuRio
reviewed
Apr 26, 2023
| int64_t send_numel = in->numel(); | ||
| const T* send_buff = in->data<T>(); | ||
| T* recv_buff = out->data<T>(); | ||
| T* recv_buff = out->mutable_data<T>(place); |
There was a problem hiding this comment.
这样out的大小对吗，allgather的out是in的nranks倍
There was a problem hiding this comment.
是的，原来的nranks倍，在infershape里已经做了out的dim设置。目前是op计算时又重复设置了一次，所以去掉了
From00
approved these changes
Apr 26, 2023
|
你的PR已合入Paddle库，请关注后续测试结果。 |
zhwesky2010
added a commit
to zhwesky2010/Paddle
that referenced
this pull request
May 8, 2023
XiaoguangHu01
pushed a commit
that referenced
this pull request
May 9, 2023
#53601) * [Zero-Dim] fix functool.reduce more safe with intial value, to support empty list (#53182) * [Zero-Dim] support 0d tensor for shape and squeeze onednn kernel (#52832) * support 0d tensor for shape and squeeze onednn kernel * set python api for shape op ut * [Zero-Dim] distributed scatter/all_to_all support input 0D tensor (#53186) * [Zero-Dim] Support paddle.sum/mean/loss api output 0D,test=allcase (#52739) * [CINN Support 0D-Tensor] CINN supports 0D-Tensor with trick temporarily (#53382) * [CINN Support 0D-Tensor] CINN supports 0D-Tensor with trick temporarily * Add unittest * [CINN Support 0D-Tensor] CINN hack squeeze2 with trick temporarily (#53454) * fix test_autograd_dynamic (#53473) Co-authored-by: zhwesky2010 <zhouwei25@baidu.com> --------- Co-authored-by: YangQun <qun.yang@intel.com> Co-authored-by: HongyuJia <jiahongyu@baidu.com> Co-authored-by: HydrogenSulfate <490868991@qq.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
APIs
Description
Pcard-66984
为distributed all_gather/scatter/all_to_all 支持0D Tennsor输入
相当于 #49279 的静态图分支实现。