Guidance on how to write a mypy plugin to workaround limitations related to MetaPathFinder / PathEntryFinder instances

[abravalheri]

Feature

It would be nice if mypy docs contains guidance/examples on how to write a mypy plugin that works around the limitations related to the use of MetaPathFinder/PathEntryFinders.

I suppose this FR also entails adding hooks that allows plugins to "redirect imports" to different files.
Maybe something like get_module_path_hook? Or even better, something that allows MetaPathFinder / PathEntryFinder instances themselves to be re-used?

If we can at least tell mypy where to find the files for certain module names, that would be a huge improvement.

Pitch

Understandably MetaPathFinder/PathEntryFinder are very dynamic aspects of the Python machinery implementation that cannot be easily reconciled with static analysis tools.

However, these are still very important parts of the Python programming language. Therefore, it would be nice if we can write plugins that workaround some of the limitations of mypy on handling these finders.

The following are a couple of examples of finders used in the wild:

1. setuptools.extern/pkg_resources.extern
   • This is the motivation that brought me here today (there has been a lot of effort recently on type checking setuptools, so eventually we could retire the typeshed stub), and it is been a challenge to work with setuptools.extern.
2. More broadly, MetaPathFinder/PathEntryFinder are mentioned in PEP 660 as a viable solution for editable installs¹.

Would it be possible to have something like mypy.plugin.Plugin.register_finder(finder: MetaPathFinder) or mypy.plugin.Plugin.get_module_path_hook(fullname: str) -> str | os.PathLike[str] | None?

Footnotes

1. Indeed this strategy is somewhat praised in the text²:

   The proxy strategy can achieve a higher level of fidelity than path-based methods.

   ↩

2. Note that the use of the term "proxy" is associated with the strategy used by the editables project, which is a MetaPathFinder: https://github.com/pfmoore/editables/blob/c8cf40778c829ec434537c131fc866c262bd6bbd/src/editables/redirector.py#L11. So I think it is safe to assume that the mention/praise in PEP 660 refers to use of finders. ↩

[pelson]

I too would love the ability to handle meta_hooks. For me, I would be fine writing a specific mypy plugin which emulates my meta hook, and returns the code that should be checked.

In short: I would like a mypy hook that allows me to transform imported code before it is analysed.