Mailman 3 May 2021 - Typing-sig

Towards statically checked ORM projections
by Tin Tvrtković May 18, 2021

May 18, 2021

Dear typing-sig, I've noticed recently that there is practically no support for type-safe ORM/ODM projections in the broader Python ecosystem. A little context: ORMs (object-relational mapper) and ODM (object-document mapper/mapping) are tools or libraries for interacting with databases. SQLAlchemy and the Django ORM are probably the most famous examples. Basically, you model a database table as a class, and rows in the table are instances of that class. ODMs are a little simpler than ORMs, they practically only map rows/documents to instances of a class, ORMs are more complex. Writing a type-safe ODM nowadays is super simple, and there are loads out there. The idea is this: ``` @dataclass class User: id: int username: str user = await fetch(User, id=1) ``` This can be made to work with the proper type annotations without much fuss. Now, the issue is doing projections in a type-safe manner. A projection basically means loading only a subset of the fields, usually for performance. Let's say that we're only interested in the user username (imagine there are 30 other fields in the class that we don't care about). That would look kinda like: ``` from dataclasses import fields user_projection: tuple[str] = await fetch_projection(User, id=1, fields(User)[1]) ``` This can't really be very type-safe since Mypy treats `fields(User)[1]` as `dataclasses.Field*[Any]`. Now, if Mypy treated it as `dataclasses.Field*[str]`, I assume that would be a different story, and the function could be annotated to return a 1-tuple of `str`. As mentioned, I've looked at a bunch of ORM/ODM libraries (and written a few internally) and as far as I can tell this use-case is very much unsupported as of yet. (I would appreciate counter-examples, obviously!) This made me sad so I came to the list to see if there's anything to be done. To be super honest I'm more interested in getting attrs support for this, but attrs and dataclasses are so similar I figured if someone did the work in the dataclass plugin, the logic could also be ported over to the attrs plugin. attrs also has a nicer API for actually getting the fields, so the equivalent attrs example would be: ``` from attrs import fields as f user_projection = await fetch_projection(User, id=1, f(User).username) ``` Wouldn't it be super cool if Mypy (or other type checkers) could check this statically?

6 11

Supporting dataclass-like semantics
by Eric Traut May 15, 2021

May 15, 2021

PEP 557 introduced the dataclass to the Python stdlib. Several popular libraries (including attrs, pydantic, and various libraries that support database ORMs such as django and edgedb) have behaviors that are similar to dataclass, but these behaviors cannot be described using standard type annotations. To work around this limitation, mypy custom plugins have been developed for many of these libraries, but these plugins don't work with other type checkers, linters or language servers. They are also costly to maintain for library authors, and they require that Python developers know about the existence of these plugins and download and configure them within their environment. I've been exploring ways to extend support for dataclass semantics to other other libraries in a standardized manner. I've posted a [proposal in the form of a proto-PEP](https://github.com/microsoft/pyright/blob/master/specs/dataclass…. The latest versions of pylance and pyright implement this proposal as a proof of concept. It was relatively easy to implement (on the order of a couple hundred lines of code), and I assume that it would be similarly easy for other type checkers to add this support. I've created a [github discussion and feedback forum](https://github.com/microsoft/pyright/discussions/1782), and I welcome feedback on the proposal. I've already been in contact with the maintainers of attrs and pydantic. They are generally supportive, and I've incorporated their initial input into the latest draft of the proposal. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp.

9 19

Type Arithmetic presentation at Typing Summit
by Alfonso L. Castaño May 12, 2021

May 12, 2021

Hi everyone! Tomorrow at the Typing Summit I will make a presentation on Type Arithmetic for Tensor Typing. In order to facilitate the discussion, I share the slides of the presentation in advance (as well as from the previous presentation during the Tensor Typing Meetings) in case that someone would find them useful to understand better the proposal and contribute to the discussion. Typing Summit slides: https://docs.google.com/presentation/d/1NZzZfbUCXi78LDDe8C3hAaK9on8ZiX3Z2k4… Tensor Typing Meeting slides: https://docs.google.com/presentation/d/16VGNfVo6Kxk9lBhm1cx1-Jr19dqESyy10Wk… See you tomorrow! Alfonso.

1 0

Re: Supporting dataclass-like semantics
by Semyon Proshev May 6, 2021

May 6, 2021

Hello from the PyCharm team! While reading your suggestion, I figured out that there is one technical difficulty for PyCharm. To have a quick code insight (inspections, completion and so on) PyCharm has to build reduced syntax trees. Access to them is faster because some information is omitted (e.g. assigned values, function bodies) and they don't require any parsing as original sources do. This is done during the well-known indexing stage that processes every file independently with no ability to run resolve to other files. With the current proposal, it would be difficult to process dataclasses correctly in one case, I'll comment several examples to shed some light on PyCharm internals: ``` # everything is fine here, # `id` and `name` are saved as a class level attributes; # after indexing is finished, PyCharm is able to figure out that `create_model` describes a dataclass # because resolve becomes available; # same for the example with ModelBase @create_model class CustomerModel: id: int name: str ``` Next sample: ``` # since resolve is not available during indexing, # there is no way to determine if `CustomerModel` is a dataclass # and hence no way to protect `ModelField(default=0)` from being skipped (I've mentioned above that assigned values are omitted) @create_model(init=False) class CustomerModel: id: int = ModelField(default=0) name: str ``` Possible solution for PyCharm would be to save assigned values for all class level attributes (and make indexes more heavy) or to have some predefined names/heuristics to filter out class attributes that can not describe dataclass field. I'd not say it is a major issue, please keep it in mind, I'll be glad to discuss. -- Semyon Proshev, PyCharm --

3 2

Supporting sys.platform.startswith in type checkers
by talhayon1＠gmail.com May 5, 2021

May 5, 2021

Hello, I've been directed here from the typeshed community. Currently none of the type checkers and pyi linters support startswith with regards to sys.platform. It is however necessary for some more precise definitions for platform dependent constants - the sys.platform value for openbsd is openbsd{version} so comparison alone is not enough. Note that this is the recommended way to check for these other operating systems in the docs. Examples for such constants: mmap.ADV_* time.CLOCK_PROF time.CLOCK_UPTIME While obviously not a high priority issue, I'd still like to hear opinions. Thanks, Tal.

3 3

Re: Supporting dataclass-like semantics
by Sebastián Ramírez May 1, 2021

May 1, 2021

For what it's worth, I (representing FastAPI) think this is a great idea. 🎉 More details about my support for this are below (feel free to skip them): Even if the proposed change requires modifications (or not), the current state is already very helpful, and it would be a huge improvement in the development experience using FastAPI/pydantic and other similar libraries. By being part of standard Python it would be more straightforward for different editors to implement support for this in a standard way. And by having this simple way to declare that "dataclass-like" behavior, more libraries could take advantage of these features from type annotations similar to dataclasses, and much better support from editors (constructor autocompletion and type checks). I could imagine several libraries that would benefit from this, even some that are way older than dataclasses, like SQLAlchemy, or also other future possible tools. As this is already available and usable with the "alternate form" in Pyright (VS Code/Pylance) it can be implemented right away in pydantic and others: https://github.com/samuelcolvin/pydantic/pull/2721 The benefits are immediately evident and would help a lot of users (see that PR for a screenshot of autocompletion in a constructor with type info). All the best, Sebastián Ramírez

1 0

typing_extensions 3.10.0.0 has been released
by Jelle Zijlstra May 1, 2021

May 1, 2021

I just released typing_extensions 3.10.0.0 and typing 3.10.0.0 to PyPI. typing_extensions now supports the new type system features introduced in Python 3.10. Notable additions include TypeGuard, ParamSpec, Concatenate, and OrderedDict. The typing backport library now explicitly disallows installation on Python 3.5 and later. Thanks to all the people who contributed to this release: Dominic Davis-Foster, Dawid Kraczkowski, Ken Jin, James Morris, Daniel Ciborowski, Nils K, Antonio Valentino, Noel Evans, and Guido van Rossum. If something is wrong with the release, please open an issue at https://github.com/python/typing/issues.

1 0