On Tue, Sep 17, 2019 at 4:55 PM Philippe Prados wrote:

Hello,

I would like to publish a new PEP (see here). I discussed this idea here. The PEP1 explains that I must have a sponsor.

Who can help me ?

To clarify: This is the proposal to allow type|type to represent a union, and ~type to represent type|None. (Please, provide more information in the email, rather than forcing people to click to elsewhere to know what proposal you're even talking about.) I'm not a core dev and can't actually sponsor it, but once you get a name on it, I'll be happy to help you merge that in and assign it a PEP number! ChrisA

Since this proposal touches upon type checking I recommend that you take the proposal to typing-sig. That's where most typing-related proposals are discussed. On Mon, Sep 16, 2019 at 11:57 PM Philippe Prados wrote:

Hello,

I would like to publish a new PEP (see here https://github.com/pprados/peps/blob/master/pep-9999.rst). I discussed this idea here https://mail.python.org/archives/list/python-ideas@python.org/thread/FCTXGDT... . The PEP1 https://www.python.org/dev/peps/pep-0001/ explains that I must have a sponsor.

Who can help me ?

Thanks

Philippe Prados _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WWSOVK... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On Sep 16, 2019, at 23:54, Philippe Prados wrote:

Hello,

I would like to publish a new PEP (see here). I discussed this idea here.

Your PEP is missing any mention of some key issues that came up during the discussion, and I suspect you want to make it more complete before getting a sponsor. Some issues, like figuring out what if anything needs to be benchmarked, maybe you can wait until you have a sponsor who can help you work them out. But things where you know exactly what needs to be done, you’re probably better off doing that now. I don’t know how PEP sponsorship works, but I suspect something that’s 90% of the way to ready for final discussion will attract a sponsor more easily than something that’s only halfway there. * You need __ror__ as well as __or__. * The static types in typing are not instances of type, so you need to work out what to do with them. * Making isinstance work isn’t a matter of accepting new syntax as you suggest, but making the values already created by the existing syntax work—and, since this appear to have been deliberated removed in 3.6, you need to explain why this was a mistake and should be undone. (Have you uncovered the reason for this change?) And whether it should affect just Union or other types. * What about except clauses? Shouldn’t they take unions if isinstance does? How does that work? * You also need unions to work on the first argument of issubclass for one of your previous examples, which is different from the rules on the second argument of isinstance and issubclass, and was also apparently deliberately removed in 3.6. Is that still part of the proposal? Why, or why not? * Should you be able to test whether List or List[int] is a subclass if List|Tuple or List[int]|Tuple[int]? If so, that reverses even more of the 3.6 change, and then you have to explain why you can’t use issubclass(List[int], Iterable[int]) or issubclass(List[Integral], List[int]) but can use this. If not, what’s the use case for issubclass with unions in the first place? * You will probably want to create a new builtin type for unions, rather than having a bunch of different parts of the Python core import from typing. * In addition to other benefits, someone (Stephen?) pointed out that builtin support could mean that, e.g., isinstance(3, int|str) could be just as efficient as isinstance(3, (int,str)), which alleviated multiple people’s concerns. Is that’s part of the proposal you should make that point; if not, explain why not. * The match with the normal meaning of the | operator is a strong argument for your proposal, so it’s probably worth mentioning. (Whether you read it as union or as or, it does what you expect.) But you also need to address ~ being potentially confusing. (Whether you read it as not or complement or invert, it doesn’t do anything near what you expect, but there are only so many operators and Optional is so useful that it’s worth getting used to this. At least you quickly sold Guido with that argument, so I think it would work just as well in the PEP, if it’s there.) * There’s usually a section on rejected alternatives, where you can explain why just making static type tools handle a tuple or set or frozenset of types isn’t better than making isinstance work with unions and creating them with |. * Mentioning the wide variety of other languages’ typing systems that use | for related features would probably make it more compelling.

On Tue, Sep 17, 2019 at 4:55 PM Philippe Prados wrote:

Hello,

I would like to publish a new PEP (see here). I discussed this idea here. The PEP1 explains that I must have a sponsor.

Who can help me ?

With the blessing of the Python Steering Council, I am now your PEP's sponsor. Your PEP number - unless there's a merge conflict - will be 604. Feel free to open a pull request against the PEPs repository to create the .rst file! I'll be happy to help out with any issues you have regarding formatting etc. Let's do this! ChrisA

On Thu, Sep 19, 2019 at 09:03:06AM +0200, Philippe Prados wrote:

* You need __ror__ as well as __or__. No, in this situation, Python auto invoke ``__or__`` in case of ``__ror__``.

I don't think that Python will invoke ``__or__`` if the ``__ror__`` method is missing. Testing in Python 3.8: py> class Test(object): ... def __or__(self, other): ... return "Called __or__ method" ... py> Test() | None # works 'Called __or__ method' py> None | Test() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for |: 'NoneType' and 'Test' Unless I have missed something, you need both ``__ror__`` as well as ``__or__`` to make it work correctly.

* The static types in typing are not instances of type, so you need to work out what to do with them. I do not understand the remark. My patch of 'mypy' accept this new syntax.

This is not just about mypy static checks, it is also about Python runtime checks. [...]

Where I can find the argument to explain why this have been deliberated removed in 3.6 ?

Try the What's New file for 3.6, and the bug tracker.

In my implementation, they affect just Union. isinstance() can now accept type, Tuple or Union.

What about issubclass?

* What about except clauses? Shouldn’t they take unions if isinstance does? How does that work? Good question. To accept `except TypeError | ZeroDivisionError:` in place of `except (TypeError, ZeroDivisionError):`, the impact is bigger, but why not ?

I don't think this will be a problem. The except clause will accept any expression: py> try: ... 1/0 ... except None or [] or 0 or ZeroDivisionError: ... print("caught divide by zero") ... caught divide by zero So long as the expression ``TypeError | ZeroDivisionError`` returns a subclass of BaseException, everything should Just Work with no syntax changes. -- Steven

Response inline Le jeu. 19 sept. 2019 à 19:18, Andrew Barnert a écrit :

Steven already answered many of these, so I’ll just snip the ones he didn’t.

On Sep 19, 2019, at 00:03, Philippe Prados wrote:

* The static types in typing are not instances of type, so you need to work out what to do with them. I do not understand the remark. My patch of 'mypy' accept this new syntax.

If you’re only adding __or__ to type, then List[int] | Tuple[int] is still an exception.

You are right. You find a bug in my current implementation. I add __or__, __ror__ and __invert__ in _GenericAlias and published the new version. Now, assert List[int] | Tuple[int] == Union[List[int], Tuple[int]]

* Making isinstance work isn’t a matter of accepting new syntax as you suggest, but making the values already created by the existing syntax work—and, since this appear to have been deliberated removed in 3.6, you need to explain why this was a mistake and should be undone. (Have you uncovered the reason for this change?) And whether it should affect just Union or other types. Where I can find the argument to explain why this have been deliberated removed in 3.6 ? In my implementation, they affect just Union. isinstance() can now accept type, Tuple or Union.

If isinstahce(2, List) is still a TypeError, then what happens with isinstance(2, List|Tuple)? Probably it should raise a TypeError complaining that List can’t be used in type checks, like isinstance(2, List) does?

Now, assert isinstance(2, List|Tuple) == False assert isinstance(2, List|int) == True assert isinstance(2, List) == False assert isinstance(2, ~int) == True assert isinstance(None, ~int) == True

* What about except clauses? Shouldn’t they take unions if isinstance does? How does that work? Good question. To accept `except TypeError | ZeroDivisionError:` in place of `except (TypeError, ZeroDivisionError):`, the impact is bigger, but why not ?

As Steven pointed out, the syntax will already handle this, but you do need to change the implementation of how exceptions check for a matching except spec while looking for a handler. I’m guessing it’ll be easy, but I haven’t looked at that code in a long time.

I think so too

* Should you be able to test whether List or List[int] is a subclass if List|Tuple or List[int]|Tuple[int]? If so, that reverses even more of the 3.6 change, and then you have to explain why you can’t use issubclass(List[int], Iterable[int]) or issubclass(List[Integral], List[int]) but can use this. If not, what’s the use case for issubclass with unions in the first place? It's a question for "typing". My proposition change nothing about that.

No, typing avoids having to make this decision, because these aren’t types, and can’t be used in issubclass at all. If you’re changing that some these are now legal calls, you have to decide which ones, and what those calls return.

I fix this

* You will probably want to create a new builtin type for unions, rather than having a bunch of different parts of the Python core import from typing. May be.

* In addition to other benefits, someone (Stephen?) pointed out that builtin support could mean that, e.g., isinstance(3, int|str) could be just as efficient as isinstance(3, (int,str)), which alleviated multiple people’s concerns. Is that’s part of the proposal you should make that point; if not, explain why not. The implementation use the tuple present in the Union type. The impact is just to check the type of the second parameter and replace it with the tuple from the Union.

If you have to do an instance check against a type with a subclass hook, and that you have to import first, on every call to find out whether the second parameter is a Union, that will slow down every isinstance call. And if you then have to get the tuple out of the instance dict, that could make unions significantly slower than tuples.

It's why at this time, I use a trick if (!strcmp(Py_TYPE(cls)->tp_name,"_GenericAlias")) I am sure, the implementation may be better. With using the macro PyType_FastSubclass() ?

Checking a value against a built in type and accessing a member out of a C struct are a lot faster

It’s possible that neither of those will affect performance enough to matter, but you’d have to benchmark it to prove that.

* Mentioning the wide variety of other languages’ typing systems that use | for related features would probably make it more compelling. I find only Scala now.

The way you build sum types in ML and most languages derived or inspired by it (from Haskell to Swift) is with the | operator.

Most of those languages sum types give optional or mandatory constructor names to the alternatives, so it’s not quite the same thing as anonymous unions:

IntStr = Int int | Str str

… this is a type whose values are an int or a str, but the way you construct one is with Int(2) or Str("a"), not with IntStr(2). And the way you extract the values is not by explicitly type-checking, but by pattern matching using | again, against the constructor names, something like this Int(n): Int(n*2) | Str(s): Str(f"{s} doubled")

For a more anonymous union, you usually use an explicit union or either or similar type: Either[int, str] gives you a type whose values are either an int or a str, and you access them with the Left and Right constructors from Either rather than specific named ones.

They could have syntactic sugar so that int|str means Either[int, str], it just doesn’t come up often enough—except the special case of Either[Error, X], which in some languages is the main way to do error handling. So if they do add any sugar, it’s usually for that special case, not the general one.

So you can see how this is all related to anonymous union types but not identical.

Scala is the exception here, but that’s because Scala has anonymous Union and Either as separate types, where Union means least upper bound on the lattice while Either is the direct one-or-the-other-and-nothing-else like Python’s Union. (And it’s the former rather than the latter that gets | for shorthand.(

The similarity is a lot like Python’s implicit Optional vs. the explicit Optional in other languages, where you construct an Optional[int] as a constructor call to Nothing() or Just(3) and then extract the value with pattern matching. Most of these languages have added a variety of kinds of syntactic sugar because it’s so common, so, e.g., something like x? means Optional[x], and ?x gives you a bool that says whether x is a non-empty value and !x means a pattern match on Just(x) and throw if it fails, and if let val = x: stuff is further shorthand for if ?x: val = !x; stuff and so on. None of this is identical to proposing ~int to mean Optional[int], but it’s all clearly similar, and shows how many languages have found it worth having shorthand like this.

So it’s the same with |. No language does something identical to what you’re proposing, but lots of languages do something similar and related.

One more thing:

In fact, having the two versions both ready could help clarify the

...

discussion about PEP 585, and more generally about the advantages and disadvantages of going more toward a “two-kinded” type system vs. leaning more into “everything is first-class”.

I don’t think this was very clear, so apologies.

In lots of languages (Scala, Haskell, Swift, even C++) there’s syntax for operating on types, but it’s separate from the syntax for operating on values. PEP 585 is a step in that direction—the syntax of annotations is still the syntax of expressions, but the semantics are different (they just aren’t fully evaluated at all). That means you can put forward references in an annotation without getting a NameError. And it means you could define int|str as valid in annotations, without making it valid at runtime. This would avoid all the hard decisions and optimization issues, and it would be a lot closer to what Scala does, and it could even open the door for things like using the | operator for related but different meanings in type expressions and value expressions (e.g., the way ? means Optional in a type expression but optional-chaining in a value expression in Swift).

On the other hand, in Python, everything is a first-class value, including types, and everything that types do (constructing objects, type checks, even metaclass stuff) is determined by the normal language syntax and semantics operating on types as values. There are a lot of benefits to that. Which is probably why everyone who uses Python naturally expects that if int|str is a thing, it’ll be a thing you can pass around at runtime and use in isinstance and so on, even if that makes the language and the implementation a little more complicated and they don’t have a compelling use in mind for it.

I haven’t followed the discussion around PEP 585, but I suspect that when you take this PEP to the typing-sig people, these questions are going to be more relevant to them than the bikeshedding stuff about how to spell ~ and how other languages spell it. But I could be wrong.

Thanks, it more clear for me