[Python-Dev] Status of PEP 484 and the typing module

Guido van Rossum guido at python.org
Thu May 21 22:27:50 CEST 2015


Things are looking up. I think we're down to a very small number of issues
where we still disagree -- hopefully you'll allow me some leeway. :-)

On Thu, May 21, 2015 at 8:45 AM, Mark Shannon <mark at hotpy.org> wrote:

>
>
> On 21/05/15 16:01, Guido van Rossum wrote:
>
>> Hi Mark,
>>
>> We're down to the last few items here. I'm CC'ing python-dev so folks
>> can see how close we are. I'll answer point by point.
>>
>> On Thu, May 21, 2015 at 6:24 AM, Mark Shannon <mark at hotpy.org
>> <mailto:mark at hotpy.org>> wrote:
>>
>>     Hi,
>>
>>     The PEP itself is looking fairly good.
>>
>>
>> I hope you'll accept it at least provisionally so we can iterate over
>> the finer points while a prototype of typing.py in in beta 1.
>>
>>     However, I don't think that typing.py is ready yet, for a number of
>>     reasons:
>>
>>     1.
>>     As I've said before, there needs to be a distinction between classes
>>     and types.
>>     They is no need for Any, Generic, Generic's subtypes, or Union to
>>     subclass builtins.type.
>>
>>
>> I strongly disagree. They can appear in many positions where real
>> classes are acceptable, in particular annotations can have classes (e.g.
>> int) or types (e.g. Union[int, str]).
>>
>
> Why does this mean that they have to be classes? Annotations can be any
> object.
>

I want to encourage users to think about annotations as types, and for most
users the distinction between type and class is too subtle, so a simpler
rule is to say they are classes. This works out nicely when the annotations
are simple types such as 'int' or 'str' or user-defined classes (e.g.
'Employee').


> It might to help to think, not in terms of types being classes, but
> classes being shorthand for the nominal type for that class (from the point
> of view of the checker and type geeks)
> So when the checker sees 'int' it treats it as Type(int).
>

I'm fine with that being the formal interpretation (except that I don't
want to introduce a function named Type()). But it's too subtle for most
users.


> Subtyping is distinct from subclassing;
> Type(int) <: Union[Type(int), Type(str)]
> has no parallel in subclassing.
> There is no class that corresponds to a Union, Any or a Generic.
>

Again, for most people te distinction is too subtle. People expect to be
able to play around with things interactively. I think it will be helpful
if they can experiment with the objects exported by typing too:
experimenting with things like isinstance(42, Union[int, str]) or
issubclass(Any, Employee) and issubclass(Employee, Any) is a useful thing
to explore how these things work (always with the caveat that when Any is
involved, issubclass is not transitive). Of course it won't work when they
advance to type variables -- at that point you just *have* to understand
the theory and switch from using the interactive interpreter to writing
small test programs and seeing how mypy (or some other checker) responds to
them.


> In order to support the
> class C(ParameterType[T]): pass
>

I presume you mean class C(Generic[T])?


> syntax, parametric types do indeed need to be classes, but Python has
> multiple inheritance, so thats not a problem:
> class ParameterType(type, Type): ...
> Otherwise typing.Types shouldn't be builtin.types and vice versa.
>

There's one thing here that Jukka has convinced me of. While I really want
Union[...] to act like a class (though not subclassable!), plain Union
(without the [...]) needn't. The same is true for Callable and Tuple
without [...]. I've filed https://github.com/ambv/typehinting/issues/133
for this. I'm not sure how much work it will be to fix this but I don't
think it absolutely needs to be done in beta 1 -- there's not much you can
do with them anyway.


> I think a lot of this issues on the tracker would not have been issues had
> the distinction been more clearly enforced.
>
>
>>     Playing around with typing.py, it has also become clear to me that it
>>     is also important to distinguish type constructors from types.
>>
>>     What do I mean by a type constructor?
>>     A type constructor makes types.
>>     "List" is an example of a type constructor. It constructs types such
>>     as List[T] and List[int].
>>     Saying that something is a List (as opposed to a list) should be
>>     rejected.
>>
>>
>> The PEP actually says that plain List (etc.) is equivalent to List[Any].
>> (Well, at least that's the intention; it's implied by the section about
>> the equivalence between Node() and Node[Any]().
>>
>
> Perhaps we should change that. Using 'List', rather than 'list' or
> 'List[Any]' suggests an error, or misunderstanding, to me.
>
> Is there a use case where 'List' is needed, and 'list' will not suffice?
> I'm assuming that the type checker knows that 'list' is a MutableSequence.
>

I think it's easier if we ask people to always write 'List' rather than
'list' when they are talking about types, and 'List[Any]' will probably be
a popular type (lots of people don't want to think about exactly what the
item type is, but they are sure that the container is a list).

There's also an argument from consistency with the collection ABCs. As you
know, typing defines a bunch of types that act as "stand ins" for the
corresponding ABCs defined in collections.abc (Iterable, Sequence, Sized,
etc.). The intention here is that anywhere one of the collection ABCs is
valid it should be okay to use the corresponding class imported from typing
-- so that if you have code that currently uses "from collections.abc
import Sequence, Mapping" you can just replace that with "from typing
import Sequence, Mapping" and your code will still work. (You can then
iterate at leisure on parametrizing the types.)

So we can use e.g. Sequence as a base class and it means the same as
Sequence[Any]. Given this rule, it would be somewhat surprising if you
couldn't use List but were forced to write List[Any] in other places.
(Neither Sequence[Any] nor List[Any] can be instantiated so that's not a
concern.)


>
>
>>     2.
>>     Usability of typing as it stands:
>>
>>     Let's try to make a class that implements a mutable mapping.
>>
>>      >>> import typing as tp
>>     #Make some variables.
>>      >>> T = tp.TypeVar('T')
>>      >>> K = tp.TypeVar('K')
>>      >>> V = tp.TypeVar('V')
>>
>>     #Then make our class:
>>
>>      >>> class MM(tp.MutableMapping): pass
>>     ...
>>     #Oh that worked, but it shouldn't. MutableMapping is a type
>> constructor.
>>
>>
>> It means MutableMapping[Any].
>>
>>     #Let's make one
>>      >>> MM()
>>     Traceback (most recent call last):
>>        File "<stdin>", line 1, in <module>
>>        File "/home/mark/repositories/typehinting/prototyping/typing.py",
>>     line 1095, in __new__
>>          if _gorg(c) is Generic:
>>        File "/home/mark/repositories/typehinting/prototyping/typing.py",
>>     line 887, in _gorg
>>          while a.__origin__ is not None:
>>     AttributeError: type object 'Sized' has no attribute '__origin__'
>>
>>     # ???
>>
>>
>> Sorry, that's a bug I introduced in literally the last change to
>> typing.py. I will fix it. The expected behavior is
>>
>> TypeError: Can't instantiate abstract class MM with abstract methods
>> __len__
>>
>>     #Well let's try using type variables.
>>     class MM2(tp.MutableMapping[K, V]): pass
>>     ...
>>      >>> MM2()
>>     Traceback (most recent call last):
>>        File "<stdin>", line 1, in <module>
>>        File "/home/mark/repositories/typehinting/prototyping/typing.py",
>>     line 1095, in __new__
>>          if _gorg(c) is Generic:
>>        File "/home/mark/repositories/typehinting/prototyping/typing.py",
>>     line 887, in _gorg
>>          while a.__origin__ is not None:
>>     AttributeError: type object 'Sized' has no attribute '__origin__'
>>
>>
>> Ditto, and sorry.
>>
> No need to apologise, I'm just a bit worried about how easy it was for me
> to expose this sort of bug.
>

Well, I'm just glad you exposed it so soon after I introduced it. :-)

>
>
>>
>>     At this point, we have to resort to using 'Dict', which forces us to
>>     subclass 'dict' which may not be what we want as it may cause
>>     metaclass conflicts.
>>
>>     3.
>>     Memory consumption is also a worry. There is no caching, which means
>>     every time I use "List[int]" as an annotation, a new class object is
>>     created. Each class may only be a few KB, but collectively this
>>     could easily add up to several MBs.
>>     This should be easy to fix.
>>
>>
>> I can work on this after the beta-1 release. Until then, type aliases
>> can be used to avoid redundant type creation (and often they are clearer
>> anyway :-).
>>
> Sure.
>

OK. Tracking this in https://github.com/ambv/typehinting/issues/130

[...]

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150521/d9db4af5/attachment-0001.html>


More information about the Python-Dev mailing list