Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Eager] publish python c api for eager #37550

Merged
merged 21 commits into from
Dec 3, 2021
Merged

[Eager] publish python c api for eager #37550

merged 21 commits into from
Dec 3, 2021

Conversation

wanghuancoder
Copy link
Contributor

@wanghuancoder wanghuancoder commented Nov 25, 2021

PR types

New features

PR changes

Others

Describe

本PR是Eager(动态图重构项目)从C++端向Python端暴露接口的首个PR,主要提交内容如下:

  1. 暴露EagerTensor,作为未来动态图用户使用的新Tensor,将来将替换现有的VarBase
  2. 暴露了EagerTensor的必要成员变量和成员函数
  3. 支持了to_tensor、tensor.numpy()、print(tensor)
  4. 开发了诸多基础函数,如:
  • Python变量类型向C++变量类型的转换
  • C++变量类型向Python变量类型的转换
  • Python EagerTensor向C++EagerTensor的转换
  • C++ EagerTensor向Python EagerTensor的转换
  1. 支持了报错信息的正常向外抛出
  2. 在Python端支持了with eager_guard(),用于目前过渡期,从默认动态图向Eager动态图的切换

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

#include "paddle/pten/core/convert_utils.h"
#include "paddle/pten/core/dense_tensor.h"
#include "paddle/pten/include/core.h"
#pragma GCC diagnostic ignored "-Wwrite-strings"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是因为CI编译会报编译错误:
eager_properties.cc:143:50: error: ISO C++ forbids converting a string constant to 'char*' [-Werror=write-strings]

@@ -141,6 +141,7 @@ set(COMMON_FLAGS
-Wno-unused-parameter
-Wno-unused-function
-Wno-error=literal-suffix
-Wno-error=missing-field-initializers
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this, remove it if not necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, thx!

#include "paddle/pten/include/core.h"

namespace paddle {
namespace pybind {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about change these to a new namespace, since we will have serval api as same as imperative.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我觉得还是这个namespace更合理,所有向python暴露的代码都放在pybind文件夹下,所有代码均用pybind namespace。

#include "pybind11/stl.h"

namespace paddle {
namespace pybind {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same namespace issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上

PyObject* arr_;
};

static inline PyObject* eager_api_numpy_to_tensor(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why inline here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已删除

return res;
}

class EagerNumpyAllocation : public paddle::memory::allocation::Allocation {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plz rearrange all helper class and function to above and list all api implements below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, thx!


setattr(core.eager.EagerTensor, "__str__", __str__)

# for method_name, method in (("__str__", __str__), ("__name__", "Tensor")):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, thx!

self.check_to_tesnsor_and_numpy('complex128')


class EagerTensorPropertiesTestCase(unittest.TestCase):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add test for Grad

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, thx!

Copy link
Contributor

@JiabinYang JiabinYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@JiabinYang JiabinYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@TCChenlong TCChenlong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@raindrops2sea
Copy link
Collaborator

PR types

New features

PR changes

Others

Describe

提交python c api相关代码

翔实地描述一下。

@wanghuancoder wanghuancoder merged commit 07b4fe9 into PaddlePaddle:develop Dec 3, 2021
Zjq9409 pushed a commit to Zjq9409/Paddle that referenced this pull request Dec 10, 2021
* refine a test case, test=develop

* publish python c api for eager, test=develop

* revert modify about test_allclose_layer.py, test=develop

* refine, test=develop

* refine, test=develop

* refine, test=develop

* refine, test=develop

* refine, test=develop

* refine, test=develop

* delete numpy includes, use pybind11 numpy.h, test=develop

* refine, test=develop

* refine, test=develop

* refine, test=develop

* suport eager error msg, and add grad test case, test=develop

* refine, test=develop

* refine, test=develop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants