I agree with Nick, this is indeed big, but not impossible. If you are not sure yet whether you will work on implementation, you can focus on polishing the PEP text, and then if it is accepted and you will decide to give implementation to someone else, we will find a volunteer.

Unless I'm misunderstanding something, it seems that the OP is under the impression that because they authored the PEP, they're obligated and responsible for the entire implementation. Instead, we should be able to relatively easily split up the implementation into multiple self-contained PRs. The OP can still work on some of them if they want to and are able to, but other contributors can work on other parts of it. The easiest way of doing this would likely be separating each self-contained PR into a bullet point in a bpo issue with a brief description (similar to the layout posted above), and then the OP can work through them as they have time to do so. Any other contributor can simply pick up where the OP left off, or on other parts of it while the OP is working on something else. This would of course be after/if the PEP is accepted. On Mon, Nov 4, 2019 at 7:33 AM Ivan Levkivskyi wrote:

On Thu, 31 Oct 2019 at 23:20, Nick Coghlan wrote:

...

[...snip...]

...

It's not enough to move the typing classes, I must move functools.lru_cache() and dependencies, collections.abs.Mapping and dependencies, and track the frame level.

*It's too big for me.*

It's certainly not an easy project to tackle.

For some of the specific points you raise though, you wouldn't translate the existing Python code directly to C.

I agree with Nick, this is indeed big, but not impossible. If you are not sure yet whether you will work on implementation, you can focus on polishing the PEP text, and then if it is accepted and you will decide to give implementation to someone else, we will find a volunteer.

-- Ivan

_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AKXBKSBY... Code of Conduct: http://python.org/psf/codeofconduct/

Cool ! I'm very happy you take time to do this job. Your idea to use a shadowobject objet to promote just-in-time is very good for me, and can be used for others extensions. Thanks Philippe Prados Le sam. 9 nov. 2019 à 00:12, Richard Eames a écrit :

Hi Philippe and list,

Long time lurker here, I've had a few spare cycles at $work this week that I spent exploring an alternate implementation to PEP604 starting with the implementation from Philippe. I don't say "reference implementation", because it doesn't implement PEP604 faithfully, but it does try to implement it's goal. My thinking was that the "runtime" component to it only needs to support the isinstance/issubclass methods and so can be pretty minimal without need to import all the typing.py machinery. At which point, when actual type checking was done (anything in typing.py or mypy), the `union` type can be special cased, or promoted to a `typing.Union`.

I started by creating a built-in "union" type, that was a thin wrapper around a tuple, such that type.__or__ will return `union(lhs, rhs)` which stored the two items in a 2-length tuple. I also changed the methods in `typing.Union` to return a `union`. However, I hit my first issue: `int | int | str` would then return `union(int, union(int, str))`, whereas it should deduplicate the types. So, I implemented logic to detect and merge union | union, but changed the underlying data to a plain flat tuple. And the next issue I hit was that some of the other `typing.py` types weren't being detected as valid types (probably due to my limited understanding of the PyObject datastructure/flags), specifically, `int | "forwardref"`, I needed a way to signal to typing.py that I had a forwardref without importing it. This is where I changed my approach to the problem.

I wanted to find a way to have these metatypes (Union/TypeVar/ForwardRef) without re-implementing them in c, or importing them. My Idea was to change the builtin type to what I called a "shadow type". In the C layer it's just holds a tuple of objects/parameters, and has a type field to say what it is, then when in the type checking/typing.py layer, if it detects an object is a "shadowobject", then it promotes it to a full type. What I liked about this approach, is that for isinstance/issubclass, when it see a "shadowobject" it can just grab the underlying types from the shadowobject->params tuple and no import magic needs to happen. All of the type magic can be left to mypy/typing.py without much knowledge of the shadowobject. I've mostly got through this implementation with all but one test passing.

At this point, I've got as far as I can go without help, so I'm posting here, looking for feedback or discussion. I expect that maybe the next step should be me commenting on the PEP604 itself for implementation concerns regarding `_GenericAlias`?

Implementation is located at https://github.com/Naddiseo/cpython/tree/PEP604

Richard _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/P56ZE6US... Code of Conduct: http://python.org/psf/codeofconduct/

To add some more input about forward references, here are couple of thoughts: 1. I think ideally we should support forward references to the same extent they are currently supported: i.e. Union[None, "ForwardRef"], Union["OtherForwardRef", None], and Union["ForwardRef", "OtherForwardRef"] should all be supported. The last one looks the hardest, in the sense that it looks like we will need to add `str.__or__()` to support these forms. 2. Forward references to type variables were never supported and we shouldn't start supporting this (even if it accidentally worked in some situations). Type variables should always be defined before use. So I think we should not worry about point 3 in Richard's e-mail. Philippe, I think it makes sense to reflect these points to the PEP draft. -- Ivan On Thu, 14 Nov 2019 at 02:23, Richard Eames wrote:

Thanks for the encouragement!

I've been working some more on it since I had some more free cycles in the last few days and I think I've got to the limit of my capabilities. I think I've got it to a point where it needs more eyes because my experience level writing code in the python interpreter is pretty low, so I know that I have holes in there, especially around using INCREF/DECREF. I'm currently stuck on 3 things:

1. repr( pickle.loads( pickle.dumps(int | str) ) ) causes an infinite recursion that I can't figure out. I might end up re-specializing `repr()` again since it isn't defined in the PEP.

2. - `None | "forwardref"` and `None | None` - `"forwardref" | None` and `"forwardRefA" | "forwardRefB"` I've had a go at adding a `type.__ror__` as suggested by GvR, but I think I'm missing something.

3. type parameters - I'm not sure how to handle this: ``` MaybeT = None | "T" # creates shadow union T = TypeVar('T') maybe_int = MaybeT[int] # should this stay in shadow land, or go back to typing.py isinstance(1, maybe_int) isinstance(1, MaybeT[int]) ``` I have a couple options for this: - implement the type substitution in c; which can be done, but feels like overkill? - store the type parameters on the shadow object and vivify when needed (which will end up being in the isinstance call) - implement the __getitem__ as a call into the typing.py library and promote as it's requested. This is how I've currently implemented it.

I'm happy to keep going on this if/when needed,

Richard _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RYH3TYK5... Code of Conduct: http://python.org/psf/codeofconduct/

On Mon., 18 Nov. 2019, 6:52 am MRAB, wrote:

On 2019-11-17 20:09, Ivan Levkivskyi wrote:

...

To add some more input about forward references, here are couple of thoughts:

1. I think ideally we should support forward references to the same extent they are currently supported: i.e. Union[None, "ForwardRef"], Union["OtherForwardRef", None], and Union["ForwardRef", "OtherForwardRef"] should all be supported. The last one looks the hardest, in the sense that it looks like we will need to add `str.__or__()` to support these forms.

I'm not sure I like the thought of adding `str.__or__()` because it would mean that "Foo" | "Bar" becomes valid and returns a type, which is unexpected.

Perhaps an alternative would be to add 'Forward' to make it explicit:

Forward("ForwardRef") | Forward("OtherForwardRef")

or just:

Forward("ForwardRef") | "OtherForwardRef"

Or require that the entire expression be quoted in that case: "ForwardRef | OtherForwardRef" I wouldn't personally object to an explicit "typing.Forward" construct, though. Cheers, Nick.