Accurate overloads for ZipFile.__init__

[max-muoto]

Add overload cases based on each mode to eliminate false positives for types that don't fully implement IO[bytes], per #10880

[max-muoto]

Run-through of MyPy primer here:

Pip:

Removes false positive as the object they're passing in is readable, but doesn't implement IO[bytes]. They put a typing ignore, which is why this adds an additional warning.

werkzeug:

Removes false positive in test where a writable ResponseStream object is being passed into ZipFile.__init__, as the mode is set to "w".

[sobolevn]

I would consider adding test cases for this overload. It is rather complex.

[max-muoto]

I would consider adding test cases for this overload. It is rather complex.

Sounds good, can handle that tomorrow!

[max-muoto]

I would consider adding test cases for this overload. It is rather complex.

Added tests, found some minor issues which I fixed as a result: 4cf07ab

[max-muoto]

@JelleZijlstra Just wanted to follow up on this one!

[JelleZijlstra]

I'll periodically look over open typeshed PRs when I have some time, but can't guarantee I'll come back to a specific one (and other maintainers may also pick up this PR if they're confident enough in the change to merge it).

[max-muoto]

I'll periodically look over open typeshed PRs when I have some time, but can't guarantee I'll come back to a specific one (and other maintainers may also pick up this PR if they're confident enough in the change to merge it).

Sounds good, thanks for the initial look nonetheless.

[hauntsaninja]

Is this right? I'm not sure _ZipSeekTellable is enough, e.g. _RealGetContents requires read

[max-muoto]

You're right, we want a protocol that is seekable/readable if it's a valid zip file, otherwise it should implement tell: 0b52c6a

[max-muoto]

I think likely misread this, and assumed if an attribute error occurred it just needed to be tellable.

[srittau]

Aren't all following overloads shadowed by this one?

[max-muoto]

No - for example, IO[bytes] is broader than _ZipReadable, which will lead to the second overload getting chosen if you just an object that is readable.

[max-muoto]

As for _ZipFileMode, if file doesn't match we'll end up skipping over the overload. The tests show that this is working properly.

[srittau]

IO[bytes] is broader than _ZipReadable

But that's exactly the problem. It's my understanding that overloads are processed in order (although the typing spec doesn't mention that), so IO[bytes] always matches before _ZipReadable can match.

[srittau]

I've opened python/typing#1803 for clarification.

[max-muoto]

How is that a problem though? _ZipReadable will get matched if IO[bytes] isn't fully fulfilled, or am I misunderstanding something?

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

pip (https://github.com/pypa/pip)
+ src/pip/_internal/network/lazy_wheel.py:162: error: Unused "type: ignore" comment  [unused-ignore]

werkzeug (https://github.com/pallets/werkzeug)
- tests/test_wrappers.py:1216: error: No overload variant of "ZipFile" matches argument types "ResponseStream", "str"  [call-overload]
- tests/test_wrappers.py:1216: note: Possible overload variants:
- tests/test_wrappers.py:1216: note:     def __init__(self, file: str | PathLike[str] | IO[bytes], mode: Literal['r'] = ..., compression: int = ..., allowZip64: bool = ..., compresslevel: int | None = ..., *, strict_timestamps: bool = ..., metadata_encoding: str | None) -> ZipFile
- tests/test_wrappers.py:1216: note:     def __init__(self, file: str | PathLike[str] | IO[bytes], mode: Literal['r', 'w', 'x', 'a'] = ..., compression: int = ..., allowZip64: bool = ..., compresslevel: int | None = ..., *, strict_timestamps: bool = ..., metadata_encoding: None = ...) -> ZipFile