[Python-ideas] PEP 561: Distributing Type Information V3

Tue Oct 10 07:34:49 EDT 2017

Thanks Ethan!

The PEP draft now looks good to me. I think it makes sense to make
a PoC implementation of the PEP at this point to see if everything
works smoothly in practice.

(You could also link few examples with your PoC implementation in the PEP)

--
Ivan

On 6 October 2017 at 22:00, Ethan Smith <ethan at ethanhs.me> wrote:

> Hello,
>
> I have made some changes to my PEP on distributing type information. A
> summary of the changes:
>
>    - Move to adding a new metadata specifier so that more packaging tools
>    can participate
>    - Clarify version matching between third party stubs and runtime
>    packages.
>    - various other fixes for clarity, readability, and removal of
>    repetition
>
> As usual I have replicated a copy below.
>
> Cheers,
> Ethan
>
>
> PEP: 561
> Title: Distributing and Packaging Type Information
> Author: Ethan Smith <ethan at ethanhs.me>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 09-Sep-2017
> Python-Version: 3.7
> Post-History:
>
>
> Abstract
> ========
>
> PEP 484 introduced type hinting to Python, with goals of making typing
> gradual and easy to adopt. Currently, typing information must be distributed
> manually. This PEP provides a standardized means to package and distribute
> type information and an ordering for type checkers to resolve modules and
> collect this information for type checking using existing packaging
> architecture.
>
>
> Rationale
> =========
>
> Currently, package authors wish to distribute code that has
> inline type information. However, there is no standard method to distribute
> packages with inline type annotations or syntax that can simultaneously
> be used at runtime and in type checking. Additionally, if one wished to
> ship typing information privately the only method would be via setting
> ``MYPYPATH`` or the equivalent to manually point to stubs. If the package
> can be released publicly, it can be added to typeshed [1]_. However, this
> does not scale and becomes a burden on the maintainers of typeshed.
> Additionally, it ties bugfixes to releases of the tool using typeshed.
>
> PEP 484 has a brief section on distributing typing information. In this
> section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
> shipping stub files. However, manually adding a path to stub files for each
> third party library does not scale. The simplest approach people have taken
> is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
> checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
> and Django).
>
>
> Specification
> =============
>
> There are several motivations and methods of supporting typing in a package.
> This PEP recognizes three (3) types of packages that may be created:
>
> 1. The package maintainer would like to add type information inline.
>
> 2. The package maintainer would like to add type information via stubs.
>
> 3. A third party would like to share stub files for a package, but the
>    maintainer does not want to include them in the source of the package.
>
> This PEP aims to support these scenarios and make them simple to add to
> packaging and deployment.
>
> The two major parts of this specification are the packaging specifications
> and the resolution order for resolving module type information. The packaging
> spec is based on and extends PEP 345 metadata. The type checking spec is
> meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [2]_.
>
> New third party stub libraries are encouraged to distribute stubs via the
> third party packaging proposed in this PEP in place of being added to
> typeshed. Typeshed will remain in use, but if maintainers are found, third
> party stubs in typeshed are encouraged to be split into their own package.
>
> Packaging Type Information
> --------------------------
> In order to make packaging and distributing type information as simple and
> easy as possible, the distribution of type information, and typed Python code
> is done through existing packaging frameworks. This PEP adds a new item to the
> ``*.distinfo/METADATA`` file to contain metadata about a package's support for
> typing. The new item is optional, but must have a name of ``Typed`` and have a
> value of either ``inline`` or ``stubs``, if present.
>
> Metadata Examples::
>
>     Typed: inline
>     Typed: stubs
>
>
> Stub Only Packages
> ''''''''''''''''''
>
> For package maintainers wishing to ship stub files containing all of their
> type information, it is prefered that the ``*.pyi`` stubs are alongside the
> corresponding ``*.py`` files. However, the stubs may be put in a sub-folder
> of the Python sources, with the same name the ``*.py`` files are in. For
> example, the ``flyingcircus`` package would have its stubs in the folder
> ``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are
> not found in ``flyingcircus/`` the type checker may treat the subdirectory as
> a normal package. The normal resolution order of checking ``*.pyi`` before
> ``*.py`` will be maintained.
>
> Third Party Stub Packages
> '''''''''''''''''''''''''
>
> Third parties seeking to distribute stub files are encouraged to contact the
> maintainer of the package about distribution alongside the package. If the
> maintainer does not wish to maintain or package stub files or type information
> inline, then a "third party stub package" should be created. The structure is
> similar, but slightly different from that of stub only packages. If the stubs
> are for the library ``flyingcircus`` then the package should be named
> ``flyingcircus-stubs`` and the stub files should be put in a sub-directory
> named ``flyingcircus``. This allows the stubs to be checked as if they were in
> a regular package.
>
> In addition, the third party stub package should indicate which version(s)
> of the runtime package are supported by indicating the runtime package's
> version(s) through the normal dependency data. For example, if there was a
> stub package ``flyingcircus-stubs``, it can indicate the versions of the
> runtime ``flyingcircus`` package supported through ``install_requires``
> in distutils based tools, or the equivalent in other packaging tools.
>
> Type Checker Module Resolution Order
> ------------------------------------
>
> The following is the order that type checkers supporting this PEP should
> resolve modules containing type information:
>
> 1. User code - the files the type checker is running on.
>
> 2. Stubs or Python source manually put in the beginning of the path. Type
>    checkers should provide this to allow the user complete control of which
>    stubs to use, and patch broken stubs/inline types from packages.
>
> 3. Third party stub packages - these packages can supersede the installed
>    untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``,
>    however it is encouraged to check the package's metadata using packaging
>    query APIs such as ``pkg_resources`` to assure that the package is meant
>    for type checking, and is compatible with the installed version.
>
> 4. Inline packages - finally, if there is nothing overriding the installed
>    package, and it opts into type checking.
>
> 5. Typeshed (if used) - Provides the stdlib types and several third party
>    libraries
>
> Type checkers that check a different Python version than the version they run
> on must find the type information in the ``site-packages``/``dist-packages``
> of that Python version. This can be queried e.g.
> ``pythonX.Y -c 'import site; print(site.getsitepackages())'``. It is also recommended
> that the type checker allow for the user to point to a particular Python
> binary, in case it is not in the path.
>
> To check if a package has opted into type checking, type checkers are
> recommended to use the ``pkg_resources`` module to query the package
> metadata. If the ``typed`` package metadata has ``None`` as its value, the
> package has not opted into type checking, and the type checker should skip
> that package.
>
>
> References
> ==========
> .. [1] Typeshed (https://github.com/python/typeshed)
>
> .. [2] PEP 484, Storing and Distributing Stub Files
>    (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files)
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
>
> 

> ..
>    Local Variables:
>    mode: indented-text
>    indent-tabs-mode: nil
>    sentence-end-double-space: t
>    fill-column: 70
>    coding: utf-8
>    End:
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20171010/cab322a4/attachment-0001.html>