Corrupt unified Kraken CSV pair-converter

[rapus95]

This is the error I get. Any ideas how to fix it? :/

INFO: Building manifest to optimize price calculation with the pair converters.
INFO: Resolving transactions
 20% |###############################################                                                                                                                                                                                            | Elapsed Time: 0:00:00 ETA:  00:00:00Do you want to download the file now?[yn]y
INFO: Attempting to retrieve USDTUSD pair from the unified Kraken CSV file.
INFO: Corrupt unified CSV file found, deleting and trying again.
INFO: .dali_cache/kraken/csv/Kraken_OHLCVT.zip has been safely deleted.
INFO:
In order to provide accurate pricing from Kraken, a large (3.9+ gb) zipfile needs to be downloaded.
Downloading: |#                                                                                                                                                                                                                                                 |   1.6 KiB   1.6 MiB/s
Downloading: |#                                                                                                                                                                                                                                                 |   1.6 KiB   1.6 MiB/sINFO: .dali_cache/kraken/csv/Kraken_OHLCVT.zip has been safely deleted.
INFO: Downloaded file is invalid, trying to download again.
Downloading: |#                                                                                                                                                                                                                                                   |   1.6 KiB   0.0 s/B
Downloading: |#                                                                                                                                                                                                                                                 |   1.6 KiB   1.0 MiB/sINFO: .dali_cache/kraken/csv/Kraken_OHLCVT.zip has been safely deleted.
INFO: Downloaded file is invalid, trying to download again.
Downloading: |#                                                                                                                                                                                                                                                   |   1.6 KiB   0.0 s/B
Downloading: |#                                                                                                                                                                                                                                                   |   1.6 KiB   0.0 s/B
ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "C:\Python312\Lib\site-packages\dali\plugin\pair_converter\csv\kraken.py", line 446, in _unzip_and_chunk
    with ZipFile(self.__UNIFIED_CSV_FILE, "r") as zip_ref:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\zipfile\__init__.py", line 1349, in __init__
    self._RealGetContents()
  File "C:\Python312\Lib\zipfile\__init__.py", line 1416, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python312\Lib\site-packages\dali\dali_main.py", line 193, in _dali_main_internal
    resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\transaction_resolver.py", line 286, in resolve_transactions
    transaction = _update_spot_price_from_web(transaction, global_configuration)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\transaction_resolver.py", line 138, in _update_spot_price_from_web
    conversion: RateAndPairConverter = _get_pair_conversion_rate(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\transaction_resolver.py", line 110, in _get_pair_conversion_rate
    rate = cast(AbstractPairConverterPlugin, pair_converter).get_conversion_rate(timestamp, from_asset, to_asset, exchange)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\abstract_pair_converter_plugin.py", line 172, in get_conversion_rate
    historical_bar = self.get_historic_bar_from_native_source(timestamp, from_asset, to_asset, exchange)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\plugin\pair_converter\ccxt.py", line 285, in get_historic_bar_from_native_source
    self._cache_graph_snapshots(exchange)
  File "C:\Python312\Lib\site-packages\dali\plugin\pair_converter\ccxt.py", line 659, in _cache_graph_snapshots
    optimizations = self._optimize_assets_for_exchange(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\plugin\pair_converter\ccxt.py", line 804, in _optimize_assets_for_exchange
    bar_check = self.find_historical_bars(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\plugin\pair_converter\ccxt.py", line 444, in find_historical_bars
    csv_bar = csv_reader.find_historical_bars(from_asset, to_asset, timestamp, True, _ONE_WEEK)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\plugin\pair_converter\csv\kraken.py", line 431, in find_historical_bars
    if self._unzip_and_chunk(base_asset, quote_asset, all_bars):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\dali\plugin\pair_converter\csv\kraken.py", line 471, in _unzip_and_chunk
    self.__download_unified_csv()
  File "C:\Python312\Lib\site-packages\dali\plugin\pair_converter\csv\kraken.py", line 256, in __download_unified_csv
    raise RP2RuntimeError("Invalid zipfile. Giving up. Try again later.")
rp2.rp2_error.RP2RuntimeError: Invalid zipfile. Giving up. Try again later.
INFO: Log file: ./log/rp2_2024_06_25_13_32_56_298150.log
INFO: Generated output directory: output
INFO: Done

and that's my config:

[dali.plugin.input.rest.binance_com]
account_holder = myname
api_key = <censored>
api_secret = <censored>
native_fiat = EUR

[dali.plugin.pair_converter.ccxt]
historical_price_type = nearest
default_exchange = "Binance.com"
fiat_access_key = crapkey

[macanudo527]

The Kraken file got updated. The latest merge has the updated file if you can pull it directly from github.

Alternatively, you can just download the file yourself and place it in .dali_cache/kraken/csv

[rapus95]

closing in favor or #252