Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

freeze_support error when creating a strategy #181

Closed
tudorelu opened this issue Jan 1, 2021 · 3 comments
Closed

freeze_support error when creating a strategy #181

tudorelu opened this issue Jan 1, 2021 · 3 comments
Labels
info Informational

Comments

@tudorelu
Copy link

tudorelu commented Jan 1, 2021

Which version are you running?
0.2.28b0

Describe the bug

I am getting some OHLCV data from the ccxt library, saving it into a dataframe with the 'open', 'high', 'low', 'close' and 'volume' columns. The dataframe loks like this:

              time      open     high       low     close        volume       date
0    1557705600000   6968.24   8100.0   6870.00   7790.71  85804.735333 2019-05-13
1    1557792000000   7795.62   8366.0   7599.56   7947.56  76583.722603 2019-05-14
2    1557878400000   7945.26   8249.0   7850.00   8169.87  37884.327211 2019-05-15
3    1557964800000   8169.08   8320.0   7705.00   7866.59  69630.513996 2019-05-16
4    1558051200000   7868.67   7925.0   6913.00   7355.26  88752.008159 2019-05-17
..             ...       ...      ...       ...       ...           ...        ...
595  1609113600000  26281.54  27500.0  26101.00  27079.41  79721.742496 2020-12-28
596  1609200000000  27079.42  27410.0  25880.00  27385.00  69411.592606 2020-12-29
597  1609286400000  27385.00  28996.0  27320.00  28875.54  95356.057826 2020-12-30
598  1609372800000  28875.55  29300.0  27850.00  28923.63  75508.505152 2020-12-31
599  1609459200000  28923.63  29470.0  28690.17  29296.28  25362.458601 2021-01-01

I then create a strategy with a few indicators and try to run it on the dataframe.

Running the following code works perfectly from the python command line, however saving it into a file and running it from there enters an infinite loop, where an error is printed.

To Reproduce

import ccxt
from pprint import pprint
import pandas as pd
import pandas_ta as ta

def ccxt_ohlcv_to_dataframe(ohlcv):
    """ Converts cctx ohlcv data from list of lists to dataframe. """
    df = pd.DataFrame(ohlcv)
    df.columns = ['time', 'open', 'high', 'low', 'close', 'volume']
    df['date'] = pd.to_datetime(df['time'] * 1000000, infer_datetime_format=True)
    return df

exchange = ccxt.binance()
ohlcv = exchange.fetch_ohlcv('BTC/USDT', '1d', limit=600)
df = ccxt_ohlcv_to_dataframe(ohlcv)

# Create your own Custom Strategy
CustomStrategy = ta.Strategy(
    name="Momo and Volatility",
    description="SMA 50, 200, BBANDS, RSI, MACD and Volume SMA 20",
    ta=[
        {"kind": "sma", "length": 50},
        {"kind": "sma", "length": 200},
        {"kind": "bbands", "length": 20},
        {"kind": "rsi"},
        {"kind": "macd", "fast": 8, "slow": 21},
        {"kind": "sma", "close": "volume", "length": 20, "prefix": "VOLUME"},
    ]
)

# To run your "Custom Strategy"
df.ta.strategy(CustomStrategy)

print(df.columns)
pprint(df.tail(20))

Error Message

# This gets printed over and over again, every few seconds

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/tudor/Developer/My Tutorials/udemy_domingo_101/03 Defining Strategies/Strategies.py", line 34, in <module>
    df.ta.strategy(CustomStrategy)
  File "/Users/tudor/Developer/My Tutorials/udemy_domingo_101/venv/lib/python3.9/site-packages/pandas_ta/core.py", line 633, in strategy
    pool = Pool(self.cores)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

Expected behavior
I would expect it to work fine when caling from a file, as it does from the python command line.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
I am using a virtual environment (venv).

Note: If I add "col_names" to even one indicator, the code prints the error once or twice but then returns the correct dataframe:

# Change Strategy to include col_names in at least one indicator
CustomStrategy = ta.Strategy(
    name="Momo and Volatility",
    description="SMA 50, 200, BBANDS, RSI, MACD and Volume SMA 20",
    ta=[
        # ...
        {"kind": "macd", "fast": 8, "slow": 21, "col_names": ("MACD", "MACD_H", "MACD_S")}, # Include col_names here
        {"kind": "sma", "close": "volume", "length": 20, "prefix": "VOLUME"},
    ]
)

I also tried importing freeze_support from multiprocessing and calling the function at the beginning of the file, to no avail.

@tudorelu tudorelu added the bug Something isn't working label Jan 1, 2021
@tudorelu
Copy link
Author

tudorelu commented Jan 1, 2021

Found fix immediately before posting this, but decided to post as it might help someone else.

Placing the code under the python main() function makes it work.

Correct Code (in my case)

import ccxt
from pprint import pprint
import pandas as pd
import pandas_ta as ta

def ccxt_ohlcv_to_dataframe(ohlcv):
    """ Converts cctx ohlcv data from list of lists to dataframe. """
    df = pd.DataFrame(ohlcv)
    df.columns = ['time', 'open', 'high', 'low', 'close', 'volume']
    df['date'] = pd.to_datetime(df['time'] * 1000000, infer_datetime_format=True)
    return df


if __name__ == '__main__':

    exchange = ccxt.binance()
    ohlcv = exchange.fetch_ohlcv('BTC/USDT', '1d', limit=600)
    df = ccxt_ohlcv_to_dataframe(ohlcv)

    pprint(df)

    # Create your own Custom Strategy
    CustomStrategy = ta.Strategy(
        name="Momo and Volatility",
        description="SMA 50, 200, BBANDS, RSI, MACD and Volume SMA 20",
        ta=[
            {"kind": "sma", "length": 50},
            {"kind": "sma", "length": 200},
            {"kind": "bbands", "length": 20},
            {"kind": "rsi"},
            {"kind": "macd", "fast": 8, "slow": 21},
            {"kind": "sma", "close": "volume", "length": 20, "prefix": "VOLUME"},
        ]
    )


    # To run your "Custom Strategy"
    df.ta.strategy(CustomStrategy)

    print(df.columns)
    pprint(df.tail(20))

@tudorelu tudorelu closed this as completed Jan 1, 2021
@twopirllc twopirllc added info Informational and removed bug Something isn't working labels Jan 1, 2021
@twopirllc twopirllc removed their assignment Jan 1, 2021
@twopirllc
Copy link
Owner

Hello @tudorelu,

Happy New Year! Thanks for checking out Pandas TA!

Glad you were able to quickly find the solution to freeze_support() issue. This was also brought up in Issue #176 recently as well. Luckily the solution is trivial. Thanks for sharing pre and post code of this issue with others. It helps a lot.

Thanks,
KJ

@greenscar
Copy link

Thank you!!!!!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
info Informational
Projects
None yet
Development

No branches or pull requests

3 participants