Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add flag to warn about unreachable branches and exprs #7050

Merged
merged 7 commits into from
Jun 28, 2019

Conversation

Michael0x2a
Copy link
Collaborator

@Michael0x2a Michael0x2a commented Jun 24, 2019

This diff adds a --warn-unreachable flag that reports an error if:

  1. Any statement is inferred to be unreachable
  2. Any subexpression in boolean expressions, inline if statements, and list/set/generator/dict comprehensions are always unreachable or redundant.

This only takes into account the results of type analysis. We do not report errors for statements that are inferred to be unreachable in the semantic analysis phase (e.g. due to sys.platform checks and the like).

Those checks are all intentional/deliberately flag certain branches as unreachable, so error messages would be annoying in those cases.

A few additional notes:

  1. I didn't spend a huge amount of time trying to come up with error messages. They could probably do with some polishing/rewording.

  2. I thought about enabling this flag for default with mypy, but unfortunately that ended up producing ~40 to 50 (apparently legitimate) error messages.

    About a fourth of them were due to runtime checks checking to see if some type is None (despite not being declared to be Optional), about a fourth seemed to be due to mypy not quite understanding how to handle things like traits and Bogus[...], and about a fourth seemed to be legitimately unnecessary checks we could safely remove.

    The final fourth was a mixture of typeshed bugs and misc errors with the reachability checks. (e.g. see Fix reachability inference with 'isinstance(Any, Any)' #7048)

  3. For these reasons, I decided against adding this flag to --strict and against documenting it yet. We'll probably need to flush out a few bugs in mypy first/get mypy to the state where we can at least honestly dogfood this.

Resolves #2395; partially addresses #7008.

This diff adds a `--disallow-inferred-unreachable` flag that
reports an error if:

1. Any branch is inferred to be unreachable
2. Any subexpression in boolean expressions, inline if statements,
   and list/set/generator/dict comprehensions are always unreachable
   or redundant.

This only takes into account the results of *type* analysis. We do not
report errors for branches that are inferred to be unreachable in the
semantic analysis phase (e.g. due to `sys.platform` checks and the
like).

Those checks are all intentional/deliberately flag certain branches
as unreachable, so error messages would be annoying in those cases.

A few additional notes:

1. I didn't spend a huge amount of time trying to come up with error
   messages. They could probably do with some polishing/rewording.

2. I thought about enabling this flag for default with mypy, but
   unfortunately that ended up producing ~40 to 50 (apparently
   legitimate) error messages.

   About a fourth of them were due to runtime checks checking to see
   if some type is None (despite not being declared to be Optional),
   about a fourth seemed to be due to mypy not quite understanding how
   to handle things like traits and Bogus[...], and about a fourth
   seemed to be legitimately unnecessary checks we could safely remove.

   The final checks were a mixture of typeshed bugs and misc errors with
   the reachability checks. (e.g. see python#7048)

3. For these reasons, I decided against adding this flag to `--strict`
   and against documenting it yet. We'll probably need to flush out
   a few bugs in mypy first/get mypy to the state where we can at least
   honestly dogfood this.

Resolves python#2395; partially
addresses python#7008.
Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this -- I've wanted something like this for a long time. Looks mostly good, I mostly had a few nits and ideas about improving wording of error messages.

mypy/main.py Outdated Show resolved Hide resolved
test-data/unit/check-unreachable-code.test Outdated Show resolved Hide resolved
test-data/unit/check-unreachable-code.test Outdated Show resolved Hide resolved
test-data/unit/check-unreachable-code.test Outdated Show resolved Hide resolved
test-data/unit/check-unreachable-code.test Outdated Show resolved Hide resolved
test-data/unit/check-unreachable-code.test Show resolved Hide resolved
mypy/checker.py Show resolved Hide resolved
mypy/messages.py Outdated
@@ -1228,6 +1228,30 @@ def note_call(self, subtype: Type, call: Type, context: Context) -> None:
self.note('"{}.__call__" has type {}'.format(self.format_bare(subtype),
self.format(call, verbosity=1)), context)

def unreachable_branch(self, context: Context) -> None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about renaming this to unreachable_statement? Would it be clearer (and correct)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea! I think it'd be more correct, especially since we're also reporting these errors after things like asserts and returns. (I also added a test case for the latter).

mypy/messages.py Outdated Show resolved Hide resolved
@@ -715,3 +715,179 @@ if sys.version_info[0] >= 2:
reveal_type('') # N: Revealed type is 'builtins.str'
reveal_type('') # N: Revealed type is 'builtins.str'
[builtins fixtures/ops.pyi]

[case testUnreachableFlagWithBadControlFlow]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about something like DEBUG: Final = False followed by if DEBUG: print('foo')? Is there a test for this use case?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our reachability checks currently don't special-case Literal bools (or Finals that are bools) at all -- we'd actually just end up checking both branches in the if DEBUG: ... else: ... block.

I'm not sure exactly what behavior we want though, which is why I refrained from adding a test case.

(On one hand, it'd be nice to make mypy smarter about reachability in general; on the other improving mypy's understanding would lead to actual regressions in type inference if users don't opt-in to using this new flag.)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current behavior seems reasonable to me.

@Michael0x2a
Copy link
Collaborator Author

@JukkaL -- Thanks for the CR! I updated the PR; this should be ready for a second look.

@bluetech
Copy link
Contributor

bluetech commented Jun 28, 2019

I tried this PR on some of our code. It found a bunch of useless is None checks (and a typeshed bug), which is nice.

[Edit: this is discussed above already - didn't notice.] I was interested to check whether this would "break" the assert_never trick from #5818, which is like the opposite of this feature ("warn if this is not unreachable"), but the warning is not triggered in that case, which is good. I used this:

from __future__ import annotations
from typing import Union, NoReturn


def assert_never(value: NoReturn) -> NoReturn:
    assert False, f'Unhandled value: {value} ({type(value).__name__})'


def f(x: Union[int, str]) -> None:
    if isinstance(x, int):
        pass
    elif isinstance(x, str):
        pass
    else:
        # Triggers "Statement is unreachable".
        # print('xx')
        # Doesn't trigger.
        assert_never(x)

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates! Looks great!

@JukkaL JukkaL merged commit 8030804 into python:master Jun 28, 2019
@ilevkivskyi
Copy link
Member

Nice feature! Would be great to add docs for this.

@Michael0x2a Michael0x2a deleted the unreachability-warnings branch June 28, 2019 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Warn about always true/false isinstance tests
4 participants