FIX-#6793: use pandas.api.types.pandas_dtype instead of np.dtype for some more places

[anmyachev]

What do these changes do?

• first commit message and PR title follow format outlined here

  NOTE: If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

• passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
• passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
• signed commit with git commit -s
• Resolves Use pandas.api.types.pandas_dtype instead of np.dtype in all Modin codebase to allow work with pandas dtypes #6793
• tests added and passing
• module layout described at docs/development/architecture.rst is up-to-date

[anmyachev]

In this case dtype cannot be serialized by Dask.

[dchigarev]

maybe like this? pandas_dtype returns exactly the same object if a categorical is passed

>>> cat = pd.CategoricalDtype([1, 2])
>>> pd.api.types.pandas_dtype(cat) is cat
True

Suggested change

	new_dtype = pandas.api.types.pandas_dtype(dtype)
	if hasattr(dtype, "_is_materialized"):
	_ = dtype._materialize_categories()
	if isinstance(dtype, LazyCategoricalProxy):
	new_dtype = dtype
	else:
	new_dtype = pandas.api.types.pandas_dtype(dtype)

[anmyachev]

In LazyCategoricalProxy case, new_dtype will remain with non-materialized categories, which will lead to a problem with Dask.

[dchigarev]

can we then add a dask specific branching here and label it as a hack? Otherwise, materializing lazy categories without a need, breaks the whole sense of them being lazy

[dchigarev]

Mostly looks good, left a few comments

[dchigarev]

can we then add a dask specific branching here and label it as a hack? Otherwise, materializing lazy categories without a need, breaks the whole sense of them being lazy

[anmyachev]

@dchigarev ready for review

[anmyachev]

@dchigarev ping, I think all your comments have been answered.