How to declare type variables

[gvanrossum]

There are two issues around type variables:

• What function to call a type variable
• How to specify constraints (and what do the constraints mean)

Currently, mymy uses

T = typevar('T', values=[t1, t2, t3])

while the PEP currently proposes

T = Var('T', t1, t2, t3)

In any case it should be noted in the PEP that this is not the same as

T = Union[t1, t2, t3]

The reason is quite subtle, and the best example is currently the predefined type variable

AnyStr = Var('AnyStr', bytes, str)

Consider a polymorphic function that does something to filenames and works on both bytes and str, e.g.

def extension(file: AnyStr) ->AnyStr:
   return file.rsplit(b'.' if isinstance(file, bytes) else '.', 1)[1]

We really need AnyStr to be a type variable here, because we want to express that if the argument is a bytes, so is the return value, and if the argument is a str, the return value is too.

But that's not all! Such a type variable is constrained to exactly the given types. Consider the case where the argument is an instance of a user-defined subclass of bytes. We don't want the declaration to mean that the return value is then also an instance of that same subclass -- we want it to mean that the return value is a bytes (not a str).

I believe this makes the use of such a type variable equivalent to a collection of overloaded functions; in the above example

def extension(file: bytes) -> bytes:
    ...
def extension(file: str) -> str:
    ...

Open issues:

• What should the rules be if the type variable is unconstrained?
• Other languages often have a syntax for specifying a different type of constraint on type variables, e.g. the presence of a certain method; and then the implied type usually does vary with the actually specified type (I think). Do we need a way to do this?
• Should we name this Var(), typevar(), or something else? (Var() is a PEP 8 violation.)
• Should the constraints be additional positional arguments, or a keyword argument? If the latter, what should it be named?

[gvanrossum]

Also, should Var (or typevar) be added to collections.abc, or only to typing?

[ambv]

Renamed Var to TypeVar in b7d16de to leave space for variable types in the future.

[ambv]

As for the name not being PEP8-compliant, neither is Union, which is also a factory. That's minor.

As for accepting subclasses of constrained types for arguments but limiting return values to explicitly listed classes, that's reasonable but subtle and thus confusing. Seems like it's the covariance discussion in disguise. It's problematic because of this example in the PEP:

  X = TypeVar('X')
  Y = TypeVar('Y', Iterable[X])

  def filter(rule: Callable[[X], bool], input: Y) -> Y:
      ...

1. Since Iterable is abstract, both input and the return value need to be subclasses.
2. Do they have to be the same subclasses? IMHO no, as long as they're both subclassing Iterable.
3. More generally, does it have to be true that `issubclass(type(input), type(return_value))? IMHO no, your comment suggests otherwise.

[gvanrossum]

I explained the naming/syntax issue in #5 (I hope).

mypy currently doesn't like something like Iterable[X] as a typevar constraint (the 'values=' kwarg). This may be because it doesn't support parametrized types as type aliases; e.g. this fails:

X = typevar('X')
Y = Iterable[X]

I propose that in the first published draft of the PEP we use a simpler example that actually works when translated to mypy syntax, and think more deeply about constraints on type variables a bit later. E.g. this works as an example using Callable and generic types:

X = TypeVar('X')
def filter(pred: Callable[[X], bool], input: Iterable[X]) -> Iterable[X]:
      return [x for x in input if pred(x)]

For constrained types it would be better to discuss the AnyStr example in the PEP.

[vlasovskikh]

TypeVar('X', Constraint1) is reducible to Constraint1 as Guido pointed out. A type variable with an upper bound makes sense only if the upper bound consists of several constraints: TypeVar('X', Constraint1, Constraint2, ..., ConstraintN).

FYI, we have bounded type variables in PyCharm. There are 46 bounded parameterized types out of 1186 typed parameters in our stubs. Most of them are our variant of AnyStr except for several parameters of the functions of the datetime module and the filter built-in function for Python 2.

[gvanrossum]

We're agreed on X = TypeVar('X'). For the constraints, see #2.