Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support overloaded functions #182

Closed
JelleZijlstra opened this issue May 12, 2021 · 3 comments · Fixed by #357
Closed

Support overloaded functions #182

JelleZijlstra opened this issue May 12, 2021 · 3 comments · Fixed by #357

Comments

@JelleZijlstra
Copy link
Contributor

Pyanalyze currently doesn't support overloaded functions at all. It should. Some thoughts in no particular order:

  • Supporting overloads for runtime code is hard because the overloads aren't stored in the module dict. The best solution I can think of is to add a pyanalyze.extensions.overload decorator that somehow does register the overloads at runtime. It could be an alias of from typing import overload under if TYPE_CHECKING for compatibility with other type checkers. We could add an @overload_implementation decorator that collects the other overloads and stores them in the function dict or something.
  • But it's much easier to support them in stubs and implementation.py. It might require changes to typeshed-client though.
  • Internally an overloaded function could be represented as a new sister class of pyanalyze.signature.Signature. It would hold all of the overloaded signatures in an attribute.
  • Checking a call to an overload should try them one by one and succeed if any overload succeeds. But consider this example:
@overload
def f(x: int) -> Literal[1]: ...
@overload
def f(x: str) -> Literal[2]: ...
y: Union[int, str]
f(y)

I would want this to return Literal[1, 2], but naively looping over the overloads would not work for that; it would require special handling for Union. Interestingly, though, TypeScript doesn't let you do this. You have to write a separate overload for the Union instead.

  • Overloads could be useful internally for representing the generic mutators (list.append etc.), for which there are currently complicated impl functions. These overloads would require being able to dispatch on whether a value is annotated with WeakExtension.
@JelleZijlstra
Copy link
Contributor Author

#317 adds an internal OverloadedSignature class, but no way to create an overloaded function from user code. Some notes on future steps:

  • typeshed.py should be enhanced to infer overloaded types for overloads in stubs. float.__round__ might be a good one to start testing with. (Many overloads are on __init__, which adds another level of complexity.)
  • We could provide a runtime @overload implementation in pyanalyze.extensions. It could store overloads keyed by (func.module, func.qualname), and then we can look those up when we compute the signature for the implementation function. We could monkeypatch typing.overload with our implementation to support some runtime use cases, but only if the monkeypatch happens before the module defining the runtime overload is imported.

@JelleZijlstra
Copy link
Contributor Author

More notes:

  • There is no real specification of how overload matching should work. The closest is Clarify what @overload means python/typing#253 (comment). As far as I can tell pyright has no documentation on how it matches overloads. PEP 484 is vague about it. The current overload resolution mechanism from Internal support for overloads #317 is too simple and should be changed to something like the mechanism in Michael's comment:
    • Any can match multiple overloads
    • If the arguments include Unions, we can decompose them to get a match
  • typeshed-client needs work to support overloads in methods: Recognize overloaded methods JelleZijlstra/typeshed_client#33. A test case for an overloaded top-level function in stubs is builtins.print (with print(flush=True), it wants a file object that also supports .flush()).
  • I think it's going to be better to implement runtime @overload first so it's easier to write unit tests for overload resolution.

@JelleZijlstra
Copy link
Contributor Author

#321 improved the overload matching algorithm to one I'm happy with.

Remaining work:

  • Infer overloads from stubs
  • Support monkeypatching typing.overload at runtime
  • Think more about what happens if we hit both of the edge cases (Any and Union matching) in overload resolution at once
  • Add more tests for overload matching

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant