Read Parquet file with fastparquet instead of Arrow

[trgiangdo]

Hello Modin team,

I'm trying to read Parquet file using

modin.pandas.read_parquet("example.parquet", engine="fastparquet")

On my environment, only fastparquet is install.
The read_parquet will raise:

ImportError: Missing optional dependency 'pyarrow'. pyarrow is required to read parquet files.

However, if I remove Line 606-609 from parquet_dispatcher.py, the read function works fine since I believe Modin support both fastparquet and pyarrow.

Can we also check for fastparquet here?
Or is there any specific reason to not do so?

Thank you in advance for answering my question,

[pyrito]

Hi @trgiangdo thank you for raising this issue! I was able to replicate the issue on latest master. I think you're right, we should be adding fastparquet to that check as well! I think that eluded our tests since we have pyarrow installed in our CI runners.

@trgiangdo would you like to contribute the change to Modin? We can help you with your PR! 😄

[trgiangdo]

Yes, this should be a simple fix.

My main concern is that we have to wait until the next realease of Modin for this to work.

Is there any possible way to add a hotfix to the current version of Modin?

[pyrito]

We could merge in the fix to master and you could work off the latest master if that would work for your codebase? I also think the next minor release should be coming soon.

[trgiangdo]

@pyrito @mvashishtha Can you share when will the next minor version be released? It would help us so much. Thank you.

[mvashishtha]

@trgiangdo the next planned release is 0.18.0, scheduled for December 7. After #5285 is merged and before the next release comes out, you can install from master or from a specific commit to get your fix.