-
Notifications
You must be signed in to change notification settings - Fork 651
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FEAT-#7139: Use ray-core instead of ray-default #6955
Conversation
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
e975a7c
to
4a9aabe
Compare
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
4a9aabe
to
243a7dd
Compare
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
62e6723
to
bd37115
Compare
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
10c1eb8
to
ba12d22
Compare
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
0f8c253
to
cf1aff5
Compare
…try-another-ray-target Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
cf1aff5
to
4887443
Compare
I think there's no package that only includes cluster launcher. ray[client] is not for the cluster launcher (it is for ray client mode https://docs.ray.io/en/master/cluster/running-applications/job-submission/ray-client.html, which is not recommended to use) |
@rkooo567 thanks! Now it’s more clear how to deal with this. |
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py
Fixed
Show fixed
Hide fixed
modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py
Fixed
Show fixed
Hide fixed
…y-another-ray-target
modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py
Fixed
Show fixed
Hide fixed
This reverts commit a2c48c3.
It looks like we can safely remove |
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py
Outdated
Show resolved
Hide resolved
Co-authored-by: Iaroslav Igoshev <Poolliver868@mail.ru>
README.md
Outdated
@@ -76,6 +76,10 @@ Otherwise, installation of `modin[mpi]` may fail. Refer to | |||
[Installing with pip](https://unidist.readthedocs.io/en/latest/installation.html#installing-with-pip) | |||
section of the unidist documentation for more details about installation. | |||
|
|||
**Note:** Since Modin 0.30.0 we use a reduced set of Ray dependencies: `ray` instead of `ray[default]`. | |||
This means that the dashboard and cluster launcher are no longer installed by default. | |||
If you need those, consider installing `ray[default]` (through pip) along with `modin[ray]` or `ray-default` (through conda). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you need those, consider installing `ray[default]` (through pip) along with `modin[ray]` or `ray-default` (through conda). | |
If you need those, consider installing `ray[default]` (through pip) or `ray-default` (through conda) along with `modin[ray]`. |
The first suggestion above.
The second is maybe we should also add this note to installation with conda below like.
**Note:** Since Modin 0.30.0 we use a reduced set of Ray dependencies: `ray-core` instead of `ray-default`.
This means that the dashboard and cluster launcher are no longer installed by default.
If you need those, consider installing `ray-default` along with `modin-ray`.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would go with the second one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
0e33048
to
d99f198
Compare
What do these changes do?
These changes are aimed at reducing the number of dependencies and thereby simplifying the installation of Modin with the Ray engine in the user environment. For example, we might not install dependencies for the Ray dashboard.
To do this, we could use either packageray[client]
, but it appears only from versionRay>=2.6.0
(there is no desire to increase the minimally supported version of Ray so much for now), or switch to using the basic package itself (ray
), but this will not work for fairly new versions of Ray, so how they don't includegrpcio
dependency.As a transitional option, we can use the basic package with an explicit indication ofgrpcio
package . This option will work, in this pull request, I tried it with different versions of Ray. However, there is a chance thatgrpcio
package of some version will be installed, with which Ray has problems. Ray can explicitly exclude any versions in such cases (note: the exceptions are not made for the basic package, but forray[client]
package, so they cannot be reused on our side), but using the proposed approach, we most likely will not know about this until the problem arises for users.If we decide to switch to such a transitional option, then I believe we should also consider switching toray[client]
package forModin 1.0
release.For example, dependencies for
ray[default]==2.9.3
:flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
git commit -s
added andpassingdocs/development/architecture.rst
is up-to-date