Overloads with None don't work

[bdarnell]

This example (straight out of the PEP, in a stub file instead of a source file) doesn't pass mypy's type checking:

from typing import overload

unicode = str

@overload
def utf8(value: None) -> None:
    pass
@overload
def utf8(value: bytes) -> bytes:
    pass
@overload
def utf8(value: unicode) -> bytes:
    pass

/tmp/overloadtest.pyi:6: error: Overloaded function signatures 1 and 2 overlap with incompatible return types
/tmp/overloadtest.pyi:6: error: Overloaded function signatures 1 and 3 overlap with incompatible return types

It looks like None is being interpreted as something broader, when at least for overload resolution it just means the exact value None and shouldn't overlap with any non-optional type.

Related to #885.

[bdarnell]

As an ugly workaround, I can define a class Dummy(object): pass and make the None variant def utf8(value: Optional[Dummy]) -> None:. That gets past the overlap error, but subsequent type checks aren't happening as expected. What's the best way to see what type mypy has inferred for the expression utf8(None)?

from typing import overload, Optional, Union

unicode = str

class Dummy(object): pass

@overload
def utf8(value: Optional[Dummy]) -> None:
    pass
@overload
def utf8(value: Union[bytes, unicode]) -> bytes:
    pass

def optional_bytes(x: Optional[bytes]): pass
def required_bytes(x: bytes): pass
def optional_int(x: Optional[int]): pass
def required_int(x: int): pass

optional_bytes(utf8(None))
optional_bytes(utf8(u""))
optional_bytes(utf8(b""))
required_bytes(utf8(None))  # should fail, but doesn't
required_bytes(utf8(u""))
required_bytes(utf8(b""))

optional_int(utf8(None))
optional_int(utf8(u""))  # should fail, and does
optional_int(utf8(b""))  # should fail, and does
required_int(utf8(None))  # should fail, but doesn't
required_int(utf8(u""))  # should fail, and does
required_int(utf8(b""))  # should fail, and does

[gvanrossum]

Yeah, mypy currently treats None special in a way that's very different from what the PEP intended. We plan to overhaul this once we implement proper Optional checking. (I think it's scheduled for release 0.5.)

The easiest way to get mypy to tell you what type it has inferred for a given variable is usually to use it in a blatantly incorrect way -- then it will show you the type in the error message. (It would be nice if there was a better idiom for this. Maybe a special annotation like # type: reveal could force a message with the type?)

[bdarnell]

Yeah, a "blatantly incorrect" usage is exactly what I was hoping for with required_int(utf8(None)), but that didn't do anything. Does that mean that it's getting turned into Any?

[gvanrossum]

I used pdb to look into this and it does seem to be Any.

[JukkaL]

Related issue: #1322 (Any inferred for calls to overloaded functions)

[gvanrossum]

We should address this when we do the work on strict Optional checking.

[ddfisher]

Would be fixed by #1278, which is likely to be part of strict Optional checking.

[JukkaL]

This seems to be fixed now (there is no error).

[smsteel]

Seems still an issue with classmethod's overloads at least:

class classmethod(Generic[_T, _P, _R_co]):
    ...
    def __get__(self, __instance: _T, __owner: type[_T] | None = None) -> Callable[_P, _R_co]: ...
    @overload
    def __get__(self, __instance: None, __owner: type[_T]) -> Callable[_P, _R_co]: ...`
    ...

If i use it:

def trace_class_method(_classmethod: classmethod[T, P, R_co]) -> classmethod[T, P, R_co]:
    @wraps(_classmethod.__func__)
    def wrapper(_cls: type[T], *args: P.args, **kwargs: P.kwargs) -> R_co:
        method = _classmethod.__get__(None, _cls) # type: ignore[arg-type]
        return _log_trace(method, *args, **kwargs)

    return classmethod(wrapper) # type: ignore[arg-type]

The _classmethod.__get__ doesn't seem to use correct overload.
On top of that there is another issue with return classmethod(wrapper) (could be connected to #3482)

[JelleZijlstra]

Please open a new issue if you think something is wrong, rather than posting on a vaguely related long-closed one.

Try putting the None overload first.