Mailman 3 September 2019 - Typing-sig

Explicit Type Aliases
by Shannon Zhu 17 Aug '22

17 Aug '22

Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon

7 10

Bay Area Typing Meetup 9/20
by Dominik Gabi 01 Jun '20

01 Jun '20

Jukka and Ivan will be visiting the Bay Area in September and we'd like to organize another meetup. Here are the proposed (tentative) logistics: Date: Friday, Sept. 20 Time: 1pm - 5pm Location: Facebook in Menlo Park (building and room TBD) So far Ivan, Jukka, Guido, Rebecca, Martin (?), and myself have committed to attending. I'm also volunteering the rest of the Pyre team. Please let us know in advance if you plan on attending so that we can book an adequately sized room. Also now's the time to think about topics to discuss or presentations you want to give. – Dominik

4 3

PEP 593: Flexible function and variable annotations
by Till 04 Mar '20

04 Mar '20

Hi, We've just landed pep-593 (https://www.python.org/dev/peps/pep-0593/); it proposes adding support to embed metadata inside type annotations. I'm looking forward to hearing people's opinion. Best, Till

9 45

Discussions-To PEP-0604: Complementary syntax for Union[] and Optional[]
by Philippe Prados 01 Oct '19

01 Oct '19

Hello everybody, I just publish the PEP-0604 <https://www.python.org/dev/peps/pep-0604/> for discussions (https://www.python.org/dev/peps/pep-0604/). Scala 3 propose a new syntax for Union type. See here <https://dotty.epfl.ch/docs/reference/new-types/union-types.html>. I propose to add a similar syntax in Python. # Operator for Union assert( int | str == Union[int,str]) assert( int | str | float == Union[int,str,float]) # Operator for Optional assert( ~int == Optional[int]) Now, it's possible to write: def fn(bag:List[int | str], option: ~int = None) -> float | str: ... in place of def fn(bag:List[Option[int,str]], option: Optional[int] = None) -> Union[float,str]: ... The discussion on python-new(a)python.org here <https://mail.python.org/archives/list/python-ideas@python.org/thread/FCTXGD…> have emerged the idea of extending also issubclass() and isinstance(). isinstance(int, int | str) I think these syntaxes are more clear, and can help with the adoption of typing. I test and implement these ideas in a two fork : One for CPython <https://github.com/pprados/cpython/tree/updage_isinstance> and one for MyPy <https://github.com/pprados/mypy/tree/add_INVERT_to_types>. See the branches add_OR_to_types (for Union syntax), add_INVERT_to_types (for Union and Optional syntax) or updage_isinstance (for new isinstance() and issubclass()). How I implement that ? I add the operators __or__ and __revert__ to PyType_Type. The C code is similar of : from typing import * def type_or(self,right): return Union[self,right] type(type).__or__ = type_or with some check before return Union. Actually, the accepted BNF syntax for typing is : annotation: name_type name_type: NAME (args)? args: '[' paramslist ']' paramslist: annotation (',' annotation)* [','] I propose to extend the BNF syntax to : annotation: ( name_type | or_type | invert_type ) name_type: NAME (args)? args: '[' paramslist ']' paramslist: annotation (',' annotation)* [','] or_type: name_type '|' annotation invert_type: '~' annotation Your remarks are welcome. Philippe Prados

10 27

Re: Discussions-To PEP-0604: Complementary syntax for Union[] and Optional[]
by Philippe Prados 25 Sep '19

25 Sep '19

My current implementation <https://github.com/pprados/cpython/blob/add_INVERT_to_types/Objects/typeobj…> is not perfect. static PyObject * type_or(PyTypeObject* self, PyObject* param) { // Check param is a PyType or GenericAlias if ((param == NULL) || (strcmp("_GenericAlias", Py_TYPE(param)->tp_name) != 0 && (param != Py_None) && (PyObject_IsInstance(param, (PyObject *) &PyType_Type) != 1) ) ) { PyErr_SetString(PyExc_TypeError, "'type' expected"); return NULL; } // 1. Create a tuple with types PyObject *tuple=PyTuple_Pack(2,self, param); // 2. Create Union with tuple PyObject* typing=PyImport_ImportModule("typing"); PyObject* unionType = PyObject_GetAttrString(typing,"Union"); PyObject *newUnion=PyObject_GetItem(unionType, tuple); // 3. Clean memory Py_DECREF(typing); Py_DECREF(unionType); Py_DECREF(tuple); // 4. Return instance return newUnion; } PyObject * type_invert(PyTypeObject* self) { return type_or(self,Py_None); } static PyNumberMethods type_as_number = { .nb_or = (binaryfunc)type_or, // Add __or__ function .nb_invert = (unaryfunc)type_invert, // Add __invert__ function }; PyTypeObject PyType_Type = { ... &type_as_number, /* tp_as_number */ ... }; Things not good: - I compare the string of Py_TYPE(param)->tp_name with "_GenericAlias". - I import module typing in the function type_or(). What is the best practice ? - static import the module typing ? - lazy loading the module typing ? - Make the class _GenericAlias a first class type like tuple, list, etc to have a direct access of this type ? The current implementation <https://github.com/pprados/cpython/blob/update_isinstance/Objects/abstract.…> of isinstance() and issubclass() use this function: static PyObject* union_to_tuple(PyObject* cls) { if (!strcmp(Py_TYPE(cls)->tp_name,"_GenericAlias")) { PyObject* origin = PyObject_GetAttrString(cls, "__origin__"); //printf("origin = %p\n",origin); if (origin == NULL) { printf("origin == NULL, return cls"); return cls; } if (PyObject_HasAttrString(origin, "_name")) { PyObject* name = PyObject_GetAttrString(origin, "_name"); if (name==NULL) { printf("_name = NULL\n"); Py_DECREF(origin); return cls; } const char* data = (char*)PyUnicode_1BYTE_DATA(name); if (data != NULL && !strcmp(data,"Union")) { PyObject* new_cls = PyObject_GetAttrString(cls, "__args__"); if (new_cls != NULL) { cls = new_cls; } } Py_DECREF(name); } Py_DECREF(origin); } return cls; } I have the same problem with a comparison of "_GenericAlias". I must do that, because it's impossible to import the module due to infinite recursion. I add just one line in PyObject_IsInstance(PyObject *inst, PyObject *cls) and PyObject_IsSubclass(PyObject *derived, PyObject *cls) to extract the inner tuple. cls = union_to_tuple(cls); I think it's be cool to add the class _GenericAlias to be compatible with PyType_FastSubclass() Any idea ? Philippe Prados Le mer. 25 sept. 2019 à 10:25, Philippe Prados <philippe.prados(a)gmail.com> a écrit : > Ok. I agree. > > I will split the PEP. The next version I'm going to submit propose only > int | str, int | None, and isinstance(x str | int). > What do you think about that ? > except int | str : > > Philippe Prados > > > Le mar. 24 sept. 2019 à 22:42, Ivan Levkivskyi <levkivskyi(a)gmail.com> a > écrit : > >> On Tue, 24 Sep 2019 at 13:13, Guido van Rossum <guido(a)python.org> wrote: >> >>> Since you mentioned my name, let me explain my current position on how >>> to spell "optional". >>> >>> - We've had a lot of feedback from mypy users who were confused by >>> Optional[T] and thought this was needed for "optional" function arguments >>> (i.e. those that have a default). >>> >>> - Another situation where people have been confused Optional is in >>> TypedDict, where we wanted to have a way to say "this key may be present or >>> absent" (in the end we couldn't find a good keyword for this so we went >>> with "total=<bool>" for the entire dict). >>> >>> - It would probably have been less confusing if it had been called >>> "nullable" (or "noneable"? Ruby's Sorbet type system calls it "nilable"). >>> >>> - In TypeScript you just write "int | None", and that looks readable >>> enough to me -- and perfectly unambiguous. >>> >>> So my preference would be to drop the "~" proposal (or aspirations for >>> "?") and instead just write "int | None". >>> >>> IIRC at last Friday's Bay Area typing meetup there was general agreement >>> with this sentiment. >>> >> >> Yes, most people agreed that `~` is not a good idea, and we should just >> go ahead with `int | None`. Also most people agreed we may reconsider >> `int?` in a separate PEP later (possibly in conjunction with PEP 505). >> Btw, majority of people also agreed that allowing `isinstance(x, int | >> str)` is a good idea (mostly for type aliases that can be used both at >> runtime and in an annotation) assuming `isinstance(x, str | List[str])` is >> still an error. >> >> If the PEP authors don't have problems with this, then I would propose to >> move towards discussing details of the proposed implementation. >> >> -- >> Ivan >> >>

1 0

Fwd: Re: Discussions-To PEP-0604: Complementary syntax for Union[] and Optional[]
by Markus Unterwaditzer 20 Sep '19

20 Sep '19

Forgot to reply-all. > Begin forwarded message: > > From: Markus Unterwaditzer <markus(a)unterwaditzer.net> > Subject: Re: [Typing-sig] Re: Discussions-To PEP-0604: Complementary syntax for Union[] and Optional[] > Date: 20. September 2019 at 22:37:33 CEST > To: Anthony Sottile <asottile(a)umich.edu> > > In a hypothetical A[T] I am not thinking of the T. I am thinking of A. > > Let's say you have some old library that defines an Array class. For some reason that library implements getitem on its metaclass. Now you want to define typesheds that are generic over the Array's items (Array[T]). You're screwed. > > Luckily the workaround is easy (I believe) using type hint comments or the string-based syntax. > >> On 20.09.2019, at 22:11, Anthony Sottile <asottile(a)umich.edu> wrote: >> >> generic of T where T defines __getitem__ is not a problem, I don't see your point >> >> On Fri, Sep 20, 2019, 10:07 PM Markus Unterwaditzer <markus(a)unterwaditzer.net> wrote: >> I am thinking of code that has been written long before typing existed, and overloads __getitem__ with something else. Now if one had to define stub/typeshed files for that code and had to use generics, it would conflict. >> >>> On 20.09.2019, at 21:51, Anthony Sottile <asottile(a)umich.edu> wrote: >>> >>> how so? I don't see how this interferes at all unless the metaclass is trying to implement genericism itself >>> >>> Anthony >>> >>> On Fri, Sep 20, 2019, 9:48 PM Markus Unterwaditzer <markus(a)unterwaditzer.net> wrote: >>> Code that defines dunder methods on meta classes is likely already broken due to use of __getitem__ for generics. So there exists a precedent for not caring about such code. >>> >>> On September 20, 2019 6:04:04 PM GMT+02:00, Anthony Sottile <asottile(a)umich.edu> wrote: >>> I'd like to see a backward compatibility concern highlighted in the PEP -- given it's possible (and likely?) that some metaclass in the wild already defines `__or__` or `__invert__` semantics >>> >>>>>> class M(type): >>> ... def __or__(self, other): return self >>> ... >>>>>> class C(metaclass=M): pass >>> ... >>>>>> C | int >>> <class '__main__.C'> >>> >>> Anthony >>> >>> On Fri, Sep 20, 2019 at 7:16 AM Philippe Prados <philippe.prados(a)gmail.com> wrote: >>> The Resolving type hints at runtime says: “For code which uses annotations for other purposes, a regular eval(ann, globals, locals) call is enough to resolve the annotation.”. Without add a new operator `__or__` in type `type`, it’s not possible to resolve type hints at runtime. >>>>>> from __future__ import annotations >>>>>> def foo() -> int | str: pass >>> ... >>>>>> eval(foo.__annotations__['return']) >>> Traceback (most recent call last): >>> File "<stdin>", line 1, in <module> >>> File "<string>", line 1, in <module> >>> TypeError: unsupported operand type(s) for |: 'type' and 'type' >>> >>> >>> >>> Le ven. 20 sept. 2019 à 15:35, Sebastian Rittau <srittau(a)rittau.biz> a écrit : >>> Am 20.09.19 um 14:49 schrieb Chris Angelico: >>>> Both "int or str" and "str?" have the problem that they don't work >>>> under existing Python syntax, so they would be far greater changes. >>>> The definition of the "or" operator is baked into the language, but >>>> the "|" operator can have its semantics defined by the objects on >>>> either side. >>> >>> "or" works with "from __future__ import annotations", which is a fine >>> limitation in my opinion. >>> >>> - Sebastian >>> _______________________________________________ >>> Typing-sig mailing list -- typing-sig(a)python.org >>> To unsubscribe send an email to typing-sig-leave(a)python.org >>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>> _______________________________________________ >>> Typing-sig mailing list -- typing-sig(a)python.org >>> To unsubscribe send an email to typing-sig-leave(a)python.org >>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>> _______________________________________________ >>> Typing-sig mailing list -- typing-sig(a)python.org >>> To unsubscribe send an email to typing-sig-leave(a)python.org >>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >> >> _______________________________________________ >> Typing-sig mailing list -- typing-sig(a)python.org >> To unsubscribe send an email to typing-sig-leave(a)python.org >> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >

2 3

Bay Area Typing Meetup 9/20
by Dominik Gabi 19 Sep '19

19 Sep '19

Reminder that the meetup is happening this Friday. Thank you all for RSVPing and proposing talks and topics for discussions! Logistics Date: 9/20 Location: Facebook, MPK 28 (https://goo.gl/maps/qZbDQaVVh7jwbZYM9) Time: 1pm – 5pm Please make sure you arrive early so that we can check you in. If you are late for any reason you can call my cell: 650 464 21 41. Attendees - Jelle Zijlstra - Ethan Smith Dropbox: - Ivan Levkivskyi - Guido van Rossum - Michael “Sully” Sullivan - Jukka Lehtosalo - Michael Lee Google - Rebecca Chen - Martin DeMello - Teddy Sudol Facebook/Instagram - Sinan Cepel - Maggie Moss - Mark Mendoza - Shannon Zhu - Arun Sharma - Jia Chen - Shachee Mishra - Vincent Siles* - Benjamin Woodruff - Anirudh Padmarao - Carl Meyer* - Dominik Gabi We do have a big room available and space for more for people who want to join last minute (please RSVP). Tentative Schedule We should have plenty of time. I’ve put together a tentative schedule trying to roughly group by related topics. 1:00: Settling in 1:10: Typed HTTP APIs in Instagram (Ani, 20min) 1:30: Security Engineers and Types (Sinan, 20min) (Break) 2:00: Using mypy to aid Python 3 migration (Jukka, 20min) 2:20: Refactoring with PyType (Rebecca, 20min) (Break) 2:50: Variadic Types: demo, feedback & discussion (Mark, Vincent, 40min) (Break) 3:40: Error codes (Jukka, 10min) 3:50: Explicit Aliases (Shannon, 15min) 4:05: Improved Syntax (Ivan, 25min) (Break) 4:35: Incremental consistency (Jia, 10min) If there’s anything I missed or any other topic you would like to add, please let me know. Looking forward to see you all on Friday! Dominik

1 0

Bay Area Typing Meetup 9/20
by Dominik Gabi 06 Sep '19

06 Sep '19

Jukka and Ivan will be visiting the Bay Area in September and we'd like to organize another meetup. Here are the proposed (tentative) logistics: Date: Friday, Sept. 20 Time: 1pm - 5pm Location: Facebook in Menlo Park (building and room TBD) So far Ivan, Jukka, Guido, Rebecca, Martin (?), and myself have committed to attending. I'm also volunteering the rest of the Pyre team. Please let us know in advance if you plan on attending so that we can book an adequately sized room. Also now's the time to think about topics to discuss or presentations you want to give. – Dominik

1 0