Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pandas-ta doesnt work with numpy 2.0 #799

Closed
Chuck321123 opened this issue Jun 17, 2024 · 27 comments · May be fixed by #800
Closed

Pandas-ta doesnt work with numpy 2.0 #799

Chuck321123 opened this issue Jun 17, 2024 · 27 comments · May be fixed by #800
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@Chuck321123
Copy link

Chuck321123 commented Jun 17, 2024

So pandas-ta doesnt work with the newest numpy 2.0.

Reproducible example:
import pandas_ta as ta

Log:

ImportError: cannot import name 'NaN' from 'numpy' (C:\Users\User\anaconda3\envs\py312Test\Lib\site-packages\numpy\__init__.py)

Would be nice to get support for the latest numpy version.

Pandas-ta version 0.3.14b
Python: 3.12.2

@Chuck321123 Chuck321123 added the bug Something isn't working label Jun 17, 2024
@twopirllc twopirllc added enhancement New feature or request help wanted Extra attention is needed and removed bug Something isn't working labels Jun 17, 2024
@twopirllc twopirllc removed their assignment Jun 17, 2024
@twopirllc
Copy link
Owner

So pandas-ta doesnt work with the newest numpy 2.0.

That's crazy @Chuck321123! How is that possible? 🤷🏼‍♂️
It was released.... yesterday! 😮

Please inform the following packages that they are not up to date with the newest numpy 2.0. I don't want them to miss out on the action.

Screenshot 2024-06-17 at 12 13 35 PM

Note: streamlit is not necessary for pandas-ta

Would be nice to get support for the latest numpy version.

Let's do it! ✅
Fork the repo, checkout the development branch, edit (including tests), and submit a PR. 😎

FYI, I'm now running:
Python: 3.11.9 & 3.12.4, no longer supporting <=3.9
Pandas TA (development) 0.4.17b

@Chuck321123
Copy link
Author

@twopirllc Unfortunately, I don't know how to do that. However, If anyone knows how to do it I would be very happy

@XotEmBotZ
Copy link

I have created the pull request. Have tested with server so it should work. Though haven't touched the package requirements though should also be compatible with 1.22.
Kindly check

@danielverissimo
Copy link

danielverissimo commented Jun 20, 2024

Until the next release, on momentum/squeeze_pro.py change:

from numpy import NaN as npNaN

to

import numpy as np

@twopirllc
Copy link
Owner

Import the whole numpy namespace for a single item? 🤔

@danielverissimo
Copy link

danielverissimo commented Jun 20, 2024

Temporary bro, just to wait the final fix release.

@seraph-wing
Copy link

instead, just rename to

from numpy import nan as npNaN

@XotEmBotZ
Copy link

I believe this has already been done, the NaN is already renamed to nan and in the #801 I have fixed the imports regarding infinity (to inf). Let all these get merged then we can close this issue i guess.

@ironhak
Copy link

ironhak commented Jun 22, 2024

Temporary bro, just to wait the final fix release.

Sorry but there are no new releases since 2021, why should a new release come out any soon?

@danielverissimo
Copy link

I don't know, the comment was just try to help, feel free to improve and send another one. This is not a PR.

@twopirllc
Copy link
Owner

Sorry but there are no new releases since 2021, why should a new release come out any soon?

You shouldn't expect a main release soon as the development (wih 360+ commits beyond main) is still ironing out main's issues of numerous bugs, new indicators, speed and documentation improvements. If you want to help speed up this process, contributions to developement or answering basic Issues have always been welcome. 😎

However if you are as busy as I am, perhaps consider Buying Me a Coffee or Sponsoring my work instead. While it is Open Source, it is not free. I find it rather disheartening that only 25 people out of 4900 ⭐'s (or 0.5% of people/companies/downstream projects have bought a ☕ and one is a sponsor). 🤷🏼‍♂️

Kind Regards!

@XotEmBotZ
Copy link

XotEmBotZ commented Jun 22, 2024

Was going to the log files of the server, noticed that the current version of pandas_ta asks for numpy>=1.26 which is a problem as numpy2 (latest) one is not supported in the main version. This is resulting in an error while trying to use pandas_ta.
Though since it can be mitigated with replacing numpy2 with numpy1.26, it gives the message that this project is abandoned and dosent work.
Thus I kindly request that the numpy version should fixed to 1.26.4. thus preventing further confusion and errors.

@ironhak
Copy link

ironhak commented Jun 23, 2024

Sorry but there are no new releases since 2021, why should a new release come out any soon?

You shouldn't expect a main release soon as the development (wih 360+ commits beyond main) is still ironing out main's issues of numerous bugs, new indicators, speed and documentation improvements. If you want to help speed up this process, contributions to developement or answering basic Issues have always been welcome. 😎

However if you are as busy as I am, perhaps consider Buying Me a Coffee or Sponsoring my work instead. While it is Open Source, it is not free. I find it rather disheartening that only 25 people out of 4900 ⭐'s (or 0.5% of people/companies/downstream projects have bought a ☕ and one is a sponsor). 🤷🏼‍♂️

Kind Regards!

Sorry, I thought this was discontinued.

Also, I'm sorry there are not many donations but you're not pushing a new version since 2021, naturally end users will avoid a project that looks like discontinued, and less users = less donations. You can't complain about users not donating when they're not receiving an update since 2021, since there's also TA-Lib which is more active.

@kolyarice
Copy link

kolyarice commented Jun 23, 2024

I went ahead and manually updated the import as suggested by @seraph-wing

The import succeeded but when using RSI another compat issue arose:
_ = df.ta.rsi(cumulative=True, append=True)
resulted in the error:

File ~/code/pylibs/lib/python3.12/site-packages/pandas_ta/core.py:1058, in AnalysisIndicators.rsi(self, length, scalar, drift, offset, **kwargs)
   1056 def rsi(self, length=None, scalar=None, drift=None, offset=None, **kwargs):
   1057     close = self._get_column(kwargs.pop("close", "close"))
-> 1058     result = rsi(close=close, length=length, scalar=scalar, drift=drift, offset=offset, **kwargs)
   1059     return self._post_process(result, **kwargs)

File ~/code/pylibs/lib/python3.12/site-packages/pandas_ta/momentum/rsi.py:22, in rsi(close, length, scalar, talib, drift, offset, **kwargs)
     20 # Calculate Result
     21 if Imports["talib"] and mode_tal:
---> 22     from talib import RSI
     23     rsi = RSI(close, length)
     24 else:

File ~/code/pylibs/lib/python3.12/site-packages/talib/__init__.py:93
     89 else:
     90     _wrapper = lambda x: x
---> 93 from ._ta_lib import (
     94     _ta_initialize, _ta_shutdown, MA_Type, __ta_version__,
     95     _ta_set_unstable_period as set_unstable_period,
     96     _ta_get_unstable_period as get_unstable_period,
     97     _ta_set_compatibility as set_compatibility,
     98     _ta_get_compatibility as get_compatibility,
     99     __TA_FUNCTION_NAMES__
    100 )
    102 # import all the func and stream functions
    103 from ._ta_lib import *

File talib/_ta_lib.pyx:1, in init talib._ta_lib()

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

@XotEmBotZ
Copy link

I went ahead and manually updated the import as suggested by @seraph-wing

The import succeeded but when using RSI another compat issue arose: _ = df.ta.rsi(cumulative=True, append=True) resulted in the error:

File ~/code/pylibs/lib/python3.12/site-packages/pandas_ta/core.py:1058, in AnalysisIndicators.rsi(self, length, scalar, drift, offset, **kwargs)
   1056 def rsi(self, length=None, scalar=None, drift=None, offset=None, **kwargs):
   1057     close = self._get_column(kwargs.pop("close", "close"))
-> 1058     result = rsi(close=close, length=length, scalar=scalar, drift=drift, offset=offset, **kwargs)
   1059     return self._post_process(result, **kwargs)

File ~/code/pylibs/lib/python3.12/site-packages/pandas_ta/momentum/rsi.py:22, in rsi(close, length, scalar, talib, drift, offset, **kwargs)
     20 # Calculate Result
     21 if Imports["talib"] and mode_tal:
---> 22     from talib import RSI
     23     rsi = RSI(close, length)
     24 else:

File ~/code/pylibs/lib/python3.12/site-packages/talib/__init__.py:93
     89 else:
     90     _wrapper = lambda x: x
---> 93 from ._ta_lib import (
     94     _ta_initialize, _ta_shutdown, MA_Type, __ta_version__,
     95     _ta_set_unstable_period as set_unstable_period,
     96     _ta_get_unstable_period as get_unstable_period,
     97     _ta_set_compatibility as set_compatibility,
     98     _ta_get_compatibility as get_compatibility,
     99     __TA_FUNCTION_NAMES__
    100 )
    102 # import all the func and stream functions
    103 from ._ta_lib import *

File talib/_ta_lib.pyx:1, in init talib._ta_lib()

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

if we are talking about the current main branch, why not just limit the numpy to 1.26.4 in the setup.py.
I guess this solves almost all issues. since no numpy 2 no numpy errors.
In my opinion that is the most logical change that can be done without incurring the huge burden of dev in main branch. should just be few lines of code insead of many hours of debugging
please inform me if I am wrong

@twopirllc
Copy link
Owner

twopirllc commented Jun 23, 2024

Sorry, I thought this was discontinued. Also, I'm sorry there are not many donations but you're not pushing a new version since 2021, naturally end users will avoid a project that looks like discontinued, and less users = less donations. You can't complain about users not donating when they're not receiving an update since 2021, since there's also TA-Lib which is more active.

Understandable. However, this is not a high churn library nor is it archived (which is considered discontinued).

Thanks for the feedback

@twopirllc
Copy link
Owner

In my opinion that is the most logical change that can be done without incurring the huge burden of dev in main branch. should just be few lines of code insead of many hours of debugging

Even better, let not upgrade prematurely to a new release especially when downstream library and applications haven't done so either... ie pandas, pyarrow, & numba., as numpy states. 😅

Screenshot 2024-06-18 at 1 31 58 PM

and

Screenshot 2024-06-18 at 1 32 52 PM

Using pip to manage dependencies is the easiest solution if right now is crucial. In the mean time, with @XotEmBotZ's help 😎 , a future development version will support this urgent demand for numpy >= 2.

@XotEmBotZ
Copy link

XotEmBotZ commented Jun 23, 2024

In my opinion that is the most logical change that can be done without incurring the huge burden of dev in main branch. should just be few lines of code insead of many hours of debugging

Even better, let not upgrade prematurely to a new release especially when downstream library and applications haven't done so either... ie pandas, pyarrow, & numba., as numpy states. 😅

Screenshot 2024-06-18 at 1 31 58 PM and Screenshot 2024-06-18 at 1 32 52 PM Using `pip` to manage dependencies is the easiest solution if **right now** is _crucial_. In the mean time, with @XotEmBotZ's help 😎 , a future _development_ version will support this **urgent demand** for _numpy_ >= 2.

Yup, that's what I suggested in the last comment. In the mean time lets change the requirements mentioned in setpu.py to numpy==1.26.4 from numpy>=1.26.4 thus preventing pip from installing numpy2 when installing pandas_ta. Should be a 2 or 3 lines of change so opening PR is just a time waste for you. you can just change > to = to make it work

@XotEmBotZ
Copy link

In my opinion that is the most logical change that can be done without incurring the huge burden of dev in main branch. should just be few lines of code insead of many hours of debugging

Even better, let not upgrade prematurely to a new release especially when downstream library and applications haven't done so either... ie pandas, pyarrow, & numba., as numpy states. 😅
Screenshot 2024-06-18 at 1 31 58 PM
and
Screenshot 2024-06-18 at 1 32 52 PM
Using pip to manage dependencies is the easiest solution if right now is crucial. In the mean time, with @XotEmBotZ's help 😎 , a future development version will support this urgent demand for numpy >= 2.

Yup, that's what I suggested in the last comment. In the mean time lets change the requirements mentioned in setpu.py to numpy==1.26.4 from numpy>=1.26.4 thus preventing pip from installing numpy2 when installing pandas_ta. Should be a 2 or 3 lines of change so opening PR is just a time waste for you. you can just change > to = to make it work

since this is relatively small change ,by which date can we expect to get the update pushed to main branch?

@twopirllc
Copy link
Owner

Fixed numpy version to 1.26.4 in setup.py for main. Not updating pip for a single change.

@ironhak
Copy link

ironhak commented Jun 24, 2024

Fixed numpy version to 1.26.4 in setup.py for main. Not updating pip for a single change.

Why not? It's stalled since 2021...

@eervin123
Copy link
Sponsor

Sorry but there are no new releases since 2021, why should a new release come out any soon?

You shouldn't expect a main release soon as the development (wih 360+ commits beyond main) is still ironing out main's issues of numerous bugs, new indicators, speed and documentation improvements. If you want to help speed up this process, contributions to developement or answering basic Issues have always been welcome. 😎

However if you are as busy as I am, perhaps consider Buying Me a Coffee or Sponsoring my work instead. While it is Open Source, it is not free. I find it rather disheartening that only 25 people out of 4900 ⭐'s (or 0.5% of people/companies/downstream projects have bought a ☕ and one is a sponsor). 🤷🏼‍♂️

Kind Regards!

@twopirllc Thank you for all you do. I went to sponsor you just now and forgot I already was (not much but a small show of gratitude.) Come on folks let's help this fella retire early so he can build more cool Sh%^ for us.

@seraph-wing
Copy link

seraph-wing commented Jun 25, 2024

Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Installing collected packages: widgetsnbextension, tzlocal, tqdm, threadpoolctl, schedule, regex, numpy, mypy-extensions, llvmlite, jupyterlab-widgets, joblib, dill, scipy, numba, imageio, dateparser, scikit-learn, ipywidgets, vectorbt
  Attempting uninstall: numpy
    Found existing installation: numpy 1.26.4
    Uninstalling numpy-1.26.4:
      Successfully uninstalled numpy-1.26.4
  WARNING: Failed to remove contents in a temporary directory 'C:\Users\vishw\AppData\Local\Temp\pip-uninstall-f04ies4l'.
  You can safely remove it manually.
  WARNING: Failed to remove contents in a temporary directory 'C:\Users\vishw\.conda\envs\crypto-bot\Lib\site-packages\~umpy.libs'.
  You can safely remove it manually.
  WARNING: Failed to remove contents in a temporary directory 'C:\Users\vishw\.conda\envs\crypto-bot\Lib\site-packages\numpy\~ft'.
  You can safely remove it manually.
  WARNING: Failed to remove contents in a temporary directory 'C:\Users\vishw\.conda\envs\crypto-bot\Lib\site-packages\numpy\~inalg'.
  You can safely remove it manually.
  WARNING: Failed to remove contents in a temporary directory 'C:\Users\vishw\.conda\envs\crypto-bot\Lib\site-packages\numpy\~andom'.
  You can safely remove it manually.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pandas-ta 0.3.14b0 requires numpy==1.26.4, but you have numpy 1.23.5 which is incompatible.
Successfully installed dateparser-1.2.0 dill-0.3.8 imageio-2.34.2 ipywidgets-8.1.3 joblib-1.4.2 jupyterlab-widgets-3.0.11 llvmlite-0.39.1 mypy-extensions-1.0.0 numba-0.56.4 numpy-1.23.5 regex-2024.5.15 schedule-1.2.2 scikit-learn-1.5.0 scipy-1.13.1 threadpoolctl-3.5.0 tqdm-4.66.4 tzlocal-5.2 vectorbt-0.26.1 widgetsnbextension-4.0.11

this change resulted in above message when i installed vectorbt. I'm not sure whether to raaise this on the vectorbt repo, or here, since a large part of the people using pandas-ta would be using vectorbt. As @twopirllc mentioned, it might be best to keep the numpy version as '''numpy<2''' after all...

@twopirllc
Copy link
Owner

Why not? It's stalled since 2021...

Yes, only the main branch has been on hold.

"This is not because it is dead, but rather because it is (mostly) right: the basic idea is right, and the implementation at least right enough.

This means there will not be constant activity and churn on the project, but rather a stewardship relationship: the main goal now is to not screw it up. The documentation will be improved, tests will be added, features/indicators will be added/removed as needed, but there will be no massive rewrite or constant updating. This is in contrast with the software industry in general and the front end world in particular, which often has comical levels of churn."

  • Adapted from Intercooler JS

Unfortunately, the development version still has more improvements left before merging into main. These improvements have been discussed recently in Issue #771 and below is the updated requirements from the post that @Rossco8 has helped with.

Required Improvements

  • Numpy/Numba implementations of core moving averages, especially loop based indicators
  • Rewrite slow indicators like qqe, alligator, etc...
  • Complete remaining Hilbert Transform indicators from TA Lib
  • Clean up or make new examples
  • Necessary PRs

Main Branch: pip install git+https://github.com/twopirllc/pandas-ta
Dev Branch: pip install -U git+https://github.com/twopirllc/pandas-ta.git@development

If neither work for you, fork and carry on or use another library.

🍀

@twopirllc
Copy link
Owner

@twopirllc Thank you for all you do. I went to sponsor you just now and forgot I already was (not much but a small show of gratitude.) Come on folks let's help this fella retire early so he can build more cool Sh%^ for us.

@eervin123: Thank you for the recent sponsorship upgrade and continued support! It means a lot. 😎

@sbg0099
Copy link

sbg0099 commented Jul 25, 2024

Until the next release, on momentum/squeeze_pro.py change:

from numpy import NaN as npNaN

to

import numpy as np

Hi all, I am just at the very beginning of working with python. So please forgive the dumb questions. Where is the renaming done at? I tried just using that in the IDE but I am still getting the same error.

I still get 'ImportError: cannot import name 'NaN' from 'numpy'

@Royd12386
Copy link

Anyone has already a fix to this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants