On Wed, Jan 29, 2020 at 10:55 AM Serhiy Storchaka <storchaka@gmail.com> wrote:
29.01.20 20:11, Guido van Rossum пише:

With the help of Ethan Smith (and some code review by Serhiy) I have started a prototype of PEP585. You can see my code at https://github.com/python/cpython/pull/18239 (a PR to CPython) or directly in my branch (where I take PRs too): https://github.com/gvanrossum/cpython/tree/pep585. I am cautiously optimistic that we can get this integrated into Python 3.9.

It is an excellent work. It is much simpler than I expected.

Well, it's not over till it's over. :-)
 


Based on this work in progress I have a bunch of questions and suggestions for PEP 585.

1. The PEP says that e.g. list[str][str] should be forbidden. But there's at least one example where we currently allow chained subscript operations. This currently works (in mypy as well as at runtime):

import typing
T = typing.TypeVar("T")
D = typing.Dict[str, T]
# Now D is a generic type
x: D[int]
# Now x has type dict[str, int]

But if, in my branch, we replace Dict with dict, we get an error on the last line:

import typing
T = typing.TypeVar("T")
D = dict[str, T]
x: D[int]  # E: TypeError: 'GenericAlias' object is not subscriptable

The last line raises a TypeError.

We can add `__getitem__` which replaces type vars with actual parameters.

I tried to write the implementation here, but it turned out to be quite complicated, taking to account that the same type variable can be used multiple times and that they can be nested. It is not an easy problem, but neither very hard.

Yeah, my main question is whether we should do this or not -- if we decide we need to do this it won't be that hard to do, but it would mean the C code is more complicated, and we couldn't change it as easily if we found problems.
 


2. The PEP says that isinstance([1, 2, 3], list[int]) should be True. I think this is wrong -- with typing.List[int] it currently raises a TypeError, and I think we should keep forbidding this. Similar with issubclass.

Agree.


3. The PEP says that list == list[str]. This currently returns false in my branch (apparently this doesn't use __getattribute__ to get the __eq__ operation), and I think I like that better. It also matches the behavior of typing.List.

Do not have opinion.


I hope Łukasz will either agree to change the PEP, or have a convincing argument why we should make this work.


4. The PEP doesn't provide a name for the proxy type. I propose to name it GenericAlias and to add it to the types.py module, so it can be imported as types.GenericAlias (note: types; not typing).

Great! It can be used for implementing `__class_getitem__` for many Python classes (which already have trivial `__class_getitem__`) without importing the typing module.

Right. This part is very simple.


5. I think the following classes should also be made generic: re.Pattern, re.Match, io.IO.

But it would be nice to not make generic the BaseIO subclasses BufferedIOBase, RawIOBase and TextIOBase (if it is not too hard).

I have to admit that I'm lukewarm about this because I don't know the code any more... If it's easy to do we should do this, I agree.
 


6. How far should we go with replacing everything in typing.py that shadows something that has become generic due to the PEP's changes? (collections, collections.abc, contextlib, re, io). I expect this to be somewhat controversial (Ivan may have some ideas) and difficult to keep compatible with 3.8.

Is the goal of the PEP to make typing.List an alias to list? Sorry, it's hard for me to read, so I could miss it.

Heh, good question. I had assumed that typing.py would be changed so that typing.List is builtins.list, but the PEP does not specify that -- it merely states that using typing.List etc. will be deprecated without generating runtime warnings (instead the static checkers should warn), and that these will be removed 5 years after 3.9.0 is released. Although the PEP doesn't *exclude* that typing.List becomes a mere alias for builtins.list -- it just doesn't require it. (If I read the PEP correctly.)


7. I haven't figured out yet how to support type[x] (which should have the same meaning as typing.Type[x]) without it also enabling int[x], which we would like to prevent. (We don't want to just add object.__class_getitem__!)

We can add type.__class_getitem__ and check that "cls is type". There is a precedence in type.__call__: type('1')  returns a type, but int('1') returns an int. Using the same type.__call__.

IIRC that's done by overriding int.__new__. (Though type() does have a special case -- type(x) returns the type of x, while type(a, b, c) creates a new class with name a, bases b, and namespace c.)
 

8. I have a tricky bug remaining where inheriting from Tuple[something] causes problems. Take this example:

class C(Tuple[int]): pass

If we now examine C.__mro__, we see that it is (C, tuple, typing.Generic, object). IOW, inheriting from typing.Tuple causes builtins.tuple to be inserted into the MRO. The problem is that in my branch, tuple.__class_getitem__ overrides Generic.__class_getitem__, whereas the latter is needed to satisfy some tests for typing.py (and presumably to enable certain special behavior). I'm mostly looking for suggestions on how to make those failing tests work (see https://github.com/python/cpython/commit/35cd5f477c9da985880130d72297335e010450e4/checks?check_suite_id=427090531).
Interesting problem.

There are actually several different issues that are revealed by these failures.

A. In typing.py, the dunders of a generic or parameterized class are different!

- `__origin__` same thing
- `__args__` what PEP 585 calls `__parameters__`
- `__parameters__` only parameters that are *free* (i.e. are typevars)

For example:
```
>>> import typing
>>> T = typing.TypeVar("T")
>>> D = typing.Dict[str, T]
>>> D.__origin__
<class 'dict'>
>>> D.__args__
(<class 'str'>, ~T)
>>> D.__parameters__
(~T,)
>>>
```

B. Generic classes as defined in typing.py only allow `__class_getitem__` calls with the same number of values as there are in `__parameters__`.

C. `__orig_class__` of an instantiated generic class with parameters
  (e.g. MyList[int]()) points to the generic class that was
  instantiated.  (Only for pure Python classes, since this is added to
  the `__dict__`, and not if there's no `__dict__`.)

For example:
```
>>> class C(typing.Generic[T]): pass
...
>>> x = C[int]()
>>> x.__class__
<class '__main__.C'>
>>> x.__orig_class__
__main__.C[int]
>>>
```

The assignment to `__orig_class__` is ignored if it fails (e.g. if class `C` has `__slots__`). It would not be very hard to implement this in the C version of GenericAlias. (Alternatively, we might be able to implement Saul Shanabrook's feature request directly -- i.e. pass the proxy/GenericAlias instance to class methods instead of the origin class.)

--
--Guido van Rossum (python.org/~guido)