[Python-Dev] PEP 561 rework
Jelle Zijlstra
jelle.zijlstra at gmail.com
Sun Nov 12 12:53:34 EST 2017
2017-11-12 3:40 GMT-08:00 Ethan Smith <ethan at ethanhs.me>:
> Hello,
>
> I re-wrote my PEP to have typing opt-in be per-package rather than
> per-distribution. This greatly simplifies things, and thanks to the
> feedback and suggestions of Nick Coghlan, it is entirely compatible with
> older packaging tooling.
>
> The main idea is there are two types of packages:
> - types are packaged with runtime code (inline or stubs in the same
> package)
> - types are in a separate package (a third party or maintainer wants to
> ship type information, but not with runtime code).
>
> The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/
>
> And as always, duplicated below.
>
> Cheers,
>
> Ethan Smith
>
> ---------------------------------------------------
>
> PEP: 561
> Title: Distributing and Packaging Type Information
> Author: Ethan Smith <ethan at ethanhs.me>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 09-Sep-2017
> Python-Version: 3.7
> Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017
>
>
> Abstract
> ========
>
> PEP 484 introduced type hinting to Python, with goals of making typing
> gradual and easy to adopt. Currently, typing information must be distributed
> manually. This PEP provides a standardized means to leverage existing tooling
> to package and distribute type information with minimal work and an ordering
> for type checkers to resolve modules and collect this information for type
> checking.
>
>
> Rationale
> =========
>
> Currently, package authors wish to distribute code that has inline type
> information. Additionally, maintainers would like to distribute stub files
> to keep Python 2 compatibility while using newer annotation syntax. However,
> there is no standard method to distribute packages with type information.
> Also, if one wished to ship stub files privately the only method available
> would be via setting ``MYPYPATH`` or the equivalent to manually point to
> stubs. If the package can be released publicly, it can be added to
> typeshed [1]_. However, this does not scale and becomes a burden on the
> maintainers of typeshed. In addition, it ties bug fixes in stubs to releases
> of the tool using typeshed.
>
> PEP 484 has a brief section on distributing typing information. In this
> section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
> shipping stub files. However, manually adding a path to stub files for each
> third party library does not scale. The simplest approach people have taken
> is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
> checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
> and Django).
>
>
> Definition of Terms
> ===================
>
> The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are
> to be interpreted as described in RFC 2119.
>
> "inline" - the types are part of the runtime code using PEP 526 and 3107
> syntax.
>
> "stubs" - files containing only type information, empty of runtime code.
>
> "Distributions" are the packaged files which are used to publish and distribute
> a release. [3]_
>
> "Module" a file containing Python runtime code or stubbed type information.
>
> "Package" a directory or directories that namespace Python modules.
>
>
> Specification
> =============
>
> There are several motivations and methods of supporting typing in a package.
> This PEP recognizes three (3) types of packages that users of typing wish to
> create:
>
> 1. The package maintainer would like to add type information inline.
>
> 2. The package maintainer would like to add type information via stubs.
>
> 3. A third party or package maintainer would like to share stub files for
> a package, but the maintainer does not want to include them in the source
> of the package.
>
> This PEP aims to support these scenarios and make them simple to add to
> packaging and deployment.
>
> The two major parts of this specification are the packaging specifications
> and the resolution order for resolving module type information. The type
> checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec
> of PEP 484 [2]_.
>
> New third party stub libraries SHOULD distribute stubs via the third party
> packaging methods proposed in this PEP in place of being added to typeshed.
> Typeshed will remain in use, but if maintainers are found, third party stubs
> in typeshed MAY be split into their own package.
>
>
> Packaging Type Information
> --------------------------
>
> In order to make packaging and distributing type information as simple and
> easy as possible, packaging and distribution is done through existing
> frameworks.
>
> Package maintainers who wish to support type checking of their code MUST add
> a ``py.typed`` file to their package supporting typing. This marker is
> recursive, if a top-level package includes it, all sub-packages MUST support
> type checking as well. To have this file installed with the package,
> maintainers can use existing packaging options such as ``package_data`` in
> distutils, shown below.
>
> Distutils option example::
>
> ...
> package_data = {
> 'pkg': ['py.typed'],
> },
> ...
>
> For namespace packages, the ``py.typed`` file should be in the submodules of
> the namespace, to avoid conflicts and for clarity.
>
> Stub Only Packages
> ''''''''''''''''''
>
> For package maintainers wishing to ship stub files containing all of their
> type information, it is preferred that the ``*.pyi`` stubs are alongside the
> corresponding ``*.py`` files. However, the stubs can also be put in a separate
> package and distributed separately. Third parties can also find this method
> useful if they wish to distribute stub files. The name of the stub package
> MUST follow the scheme ``pkg_stubs`` for type stubs for the package named
> ``pkg``. The normal resolution order of checking ``*.pyi`` before ``*.py``
> will be maintained.
>
> This is very minor, but what do you think of using "pkg-stubs" instead of
"pkg_stubs" (using a hyphen rather than an underscore)? This would make the
name illegal to import as a normal Python package, which makes it clear
that it's not a normal package. Also, there could be real packages named
"_stubs".
>
> Third parties seeking to distribute stub files are encouraged to contact the
> maintainer of the package about distribution alongside the package. If the
> maintainer does not wish to maintain or package stub files or type information
> inline, then a third party stub only package can be created.
>
> In addition, stub only distributions SHOULD indicate which version(s)
> of the runtime package are supported by indicating the runtime distribution's
> version(s) through normal dependency data. For example, if there was a
> stub package ``flyingcircus_stubs``, it can indicate the versions of the
> runtime ``Flyingcircus`` distribution supported through ``install_requires``
> in distutils based tools, or the equivalent in other packaging tools.
>
>
> Type Checker Module Resolution Order
> ------------------------------------
>
> The following is the order that type checkers supporting this PEP SHOULD
> resolve modules containing type information:
>
> 1. User code - the files the type checker is running on.
>
> 2. Stubs or Python source manually put in the beginning of the path. Type
> checkers SHOULD provide this to allow the user complete control of which
> stubs to use, and patch broken stubs/inline types from packages.
>
> 3. Stub packages - these packages can supersede the installed packages.
> They can be found at ``pkg_stubs`` for package ``pkg``.
>
> 4. Inline packages - if there is nothing overriding the installed
> package, and it opts into type checking, inline types SHOULD be used.
>
> 5. Typeshed (if used) - Provides the stdlib types and several third party
> libraries.
>
> Type checkers that check a different Python version than the version they run
> on MUST find the type information in the ``site-packages``/``dist-packages``
> of that Python version. This can be queried e.g.
> ``pythonX.Y -c 'import site; print(site.getsitepackages())'``. It is also recommended
> that the type checker allow for the user to point to a particular Python
> binary, in case it is not in the path.
>
>
> Implementation
> ==============
>
> The proposed scheme of indicating support for typing is completely backwards
> compatible, and requires no modification to tooling. A sample package with
> inline types is available [typed_pkg]_, as well as a sample package checker
> [pkg_checker]_ which reads the metadata of installed packages and reports on
> their status as either not typed, inline typed, or a stub package.
>
>
> Acknowledgements
> ================
>
> This PEP would not have been possible without the ideas, feedback, and support
> of Ivan Levkivskyi, Jelle Zijlstra, Nick Coghlan, Daniel F Moisset, Nathaniel
> Smith, and Guido van Rossum.
>
>
> Version History
> ===============
>
> * 2017-11-12
>
> * Rewritten to use existing tooling only
> * No need to indicate kind of type information in metadata
> * Name of marker file changed from ``.typeinfo`` to ``py.typed``
>
> * 2017-11-10
>
> * Specification re-written to use package metadata instead of distribution
> metadata.
> * Removed stub only packages and merged into third party packages spec.
> * Removed suggestion for typecheckers to consider checking runtime versions
> * Implementations updated to reflect PEP changes.
>
> * 2017-10-26
>
> * Added implementation references.
> * Added acknowledgements and version history.
>
> * 2017-10-06
>
> * Rewritten to use .distinfo/METADATA over a distutils specific command.
> * Clarify versioning of third party stub packages.
>
> * 2017-09-11
>
> * Added information about current solutions and typeshed.
> * Clarify rationale.
>
>
> References
> ==========
> .. [1] Typeshed (https://github.com/python/typeshed)
>
> .. [2] PEP 484, Storing and Distributing Stub Files
> (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files)
>
> .. [3] PEP 426 definitions
> (https://www.python.org/dev/peps/pep-0426/)
>
> .. [typed_pkg] Sample typed package
> (https://github.com/ethanhs/sample-typed-package)
>
> .. [pkg_checker] Sample package checker
> (https://github.com/ethanhs/check_typedpkg)
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
>
>
> ..
> Local Variables:
> mode: indented-text
> indent-tabs-mode: nil
> sentence-end-double-space: t
> fill-column: 70
> coding: utf-8
> End:
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> jelle.zijlstra%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171112/d1ed4da2/attachment.html>
More information about the Python-Dev
mailing list