<div dir="ltr"><div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Sep 10, 2017 at 6:21 PM, Jelle Zijlstra <span dir="ltr"><<a href="mailto:jelle.zijlstra@gmail.com" target="_blank">jelle.zijlstra@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-h5">2017-09-10 18:10 GMT-07:00 Ethan Smith <span dir="ltr"><<a href="mailto:ethan@ethanhs.me" target="_blank">ethan@ethanhs.me</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Sun, Sep 10, 2017 at 5:39 PM, Jelle Zijlstra <span dir="ltr"><<a href="mailto:jelle.zijlstra@gmail.com" target="_blank">jelle.zijlstra@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Congratulations on your first PEP! This is solving an important problem for typing in Python, so I'm glad we're tackling it.</div></blockquote><div><br></div></span><div>Thanks! <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-m_-9023867689572438628m_-6581699733987741501gmail-h5"><snip><span></span></div></div></div></div></div></blockquote><span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-m_-9023867689572438628m_-6581699733987741501gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><pre>Packaging Type Information
--------------------------
Packages must opt into supporting typing. This will be done though a distutils
extension [2]_, providing a ``typed`` keyword argument to the distutils
``setup()`` command. The argument value will depend on the kind of type
information the package provides. The distutils extension will be added to the
``typing`` package. Therefore a package maintainer may write</pre></div></div></blockquote></span><div>Is the addition to the `typing` package just a legacy feature for Python versions without typing in the standard library? This should be made explicit. <br></div></div></div></div></blockquote><div><br></div></span><div>The intent here is that the typing package would be required for the extra setup keyword to work, otherwise it would fail.<br></div></div></div></div></blockquote></div></div><div>Then I would have to install the `typing` PyPI package even if I am only using Python 3.7+? That seems suboptimal. Perhaps the new keyword can be part of Python core in 3.7 and added to `typing_extensions` for 3.5 and 3.6.</div></div></div></div></blockquote><div><br></div><div>That would be acceptable.<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><span class="gmail-m_-9023867689572438628m_-6581699733987741501gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><pre>::
setup(
...
setup_requires=["typing"],
typed="inline",
...
)
Inline Typed Packages
'''''''''''''''''''''
Packages that have inline type annotations simply have to pass the value
``"inline"`` to the ``typed`` argument in ``setup()``.
Stub Only Packages
''''''''''''''''''
For package maintainers wishing to ship stub files containing all of their
type information, it is prefered that the ``*.pyi`` stubs are alongside the
corresponding ``*.py`` files. However, the stubs may be put in a sub-folder
of the Python sources, with the same name the ``*.py`` files are in. For
example, the ``flyingcircus`` package would have its stubs in the folder
``flyingcircus/flyingcircus/``<wbr>. This path is chosen so that if stubs are</pre></div></div></blockquote></span><div>What if `flyingcircus` already contains a subpackage called `flyingcircus`? This might be theoretical but there's probably a package out there that does this. </div></div></div></div></blockquote><div><br></div></span><div>I considered this. I considered it as worrying too much. The alternative would be to special case the name and have type checkers follow that. </div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> </div></div></div></div></blockquote></span><div>That's fair. </div><div><div class="gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-m_-9023867689572438628m_-6581699733987741501gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><pre>not found in ``flyingcircus/`` the type checker may treat the subdirectory as
a normal package. The normal resolution order of checking ``*.pyi`` before
``*.py`` will be maintained. The value of the ``typed`` argument to
``setup()`` is ``"stubs"`` for this type of distribution. The author of the
package is suggested to use ``package_data`` to assure the stub files are
installed alongside the runtime Python code.</pre></div></div></blockquote></span><div>It would be helpful to have an example of what this looks like. This PEP will likely end up being the reference document for people looking to add typing support to their packages. <br></div></div></div></div></blockquote><div><br></div></span><div>I plan on writing examples of the distutils plugin and a sample package tomorrow if I have time. <br></div><div><div class="gmail-m_-9023867689572438628h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><span class="gmail-m_-9023867689572438628m_-6581699733987741501gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><pre>Third Party Stub Packages
'''''''''''''''''''''''''
Third parties seeking to distribute stub files are encouraged to contact the
maintainer of the package about distribution alongside the package. If the
maintainer does not wish to maintain or package stub files or type information
inline, then a "third party stub package" should be created. The structure is
similar, but slightly different from that of stub only packages. If the stubs
are for the library ``flyingcircus`` then the package should be named
``flyingcircus-stubs`` and the stub files should be put in a sub-directory
named ``flyingcircus``. This allows the stubs to be checked as if they were in
a regular package. These packages should also pass ``"stubs"`` as the value
of ``typed`` argument in ``setup()``. These packages are suggested to use
``package_data`` to package stub files.
The version of the ``flyingcircus-stubs`` package should match the version of
the ``flyingcircus`` package it is providing types for.</pre></div></div></blockquote></span><div>What if I made stubs for flyingcircus 1.0 in my flyingcircus-stubs package, but then realized that I made a terrible mistake in the stubs? I can't upload a new version of flyingcircus-stubs 1.0 to PyPI, and I can't make flyingcircus-stubs have some other version than 1.0 because of this requirement.</div><div><br></div><div>Another option is as follows:</div><div>- Stub packages are versioned independently of the packages they provide stubs for.</div><div>- There is a special global (say __version__) that can be used in stub packages and gets resolved to the version of the package that is being checked.</div><div><br></div><div>The contents of flyingcircus-stubs might look like:</div><div><br></div><div>import enum</div><div><br></div><div>class Swallow(enum.Enum):</div><div> african = 0</div><div> european = 1</div><div> if __version__ >= (2, 0):</div><div> asian = 2</div><div><br></div><div>This option doesn't have the problem I described above, but it requires this magical __version__ variable.</div></div></div></div></blockquote><div><br></div></div></div><div>Guido has said he doesn't like this idea (<a href="https://github.com/python/typing/issues/84#issuecomment-318256377" target="_blank">https://github.com/python/typ<wbr>ing/issues/84#issuecomment-318<wbr>256377</a>), and I'm not convinced it is worth the complications it involves <br></div></div></div></div></blockquote></div></div><div>But Guido's comment implies that people could use an independent versioning scheme in their stubs package (his "django-1.1-stubs" example). The PEP makes the situation worse because the version is fixed.</div><div><br></div><div>I agree that my proposal introduces a lot of complication, but I don't think what the PEP proposes is workable (if django-stubs 1.1 got the stubs for django 1.1 wrong, there is no second chance). We get enough bugs in typeshed to know that stub packages will need updates independent from the package they're providing stubs for.</div><div><br></div><div>Hopefully we can come up with something that allows stub packages to be versioned separately. Guido's comment actually suggests a way forward: What if stub packages can declare (in their package metadata or something) what version of the package they are type checking? That way, django1.1-stubs 0.1 can provide buggy stubs for django 1.1, and then fix them in django1.1-stubs 0.2.</div></div></div></div></blockquote><div><br></div><div>After thinking about this for a while, I have come to believe metadata is the solution. Keeping the versioning in the name becomes a burden on both the maintainer and PyPI. I think the best solution is to leverage existing metadata and have the "stub only" packages list the package versions they support via the install_requires keyword, and I believe type checkers can verify that rather easily. So django-stubs would list e.g. django>=1.1.0,django<1.2.0 or whatever the stub maintainer wishes to support. Then the package of stubs can use normal versioning.</div><div><br></div><div>I think I have reached the point where I should go back and work on some changes to the PEP barring objection to your feedback and/or my suggested amendments.<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-m_-9023867689572438628m_-6581699733987741501gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><pre>Type Checker Module Resolution Order
------------------------------<wbr>------
The following is the order that type checkers supporting this PEP should
resolve modules containing type information:
1. User code - the files the type checker is running on.
2. Stubs or Python source in ``PYTHONPATH``. This is to allow the user
complete control of which stubs to use, and patch broken stubs/inline
types from packages.</pre></div></div></blockquote></span><div>Current type checkers don't use PYTHONPATH as far as I know, because they may not run under the same Python version or environment as the code to be type checked. I don't think we should require type checkers to listen to PYTHONPATH; perhaps we should just say that type checkers should provide a way for users to put code at the beginning of the search path.</div></div></div></div></blockquote><div><br></div></span><div>I agree. <br></div><span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-m_-9023867689572438628m_-6581699733987741501gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><pre>3. Third party stub packages - these packages can supersede the installed
untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``,
however it is encouraged to check their metadata to confirm that they opt
into type checking.</pre></div></div></blockquote></span><div>The metadata of the stubs package? I'm not sure why that makes much sense here. </div></div></div></div></blockquote><div><br></div></span><div>Essentially, a package opts into being checked via a setup() keyword. That keyword is put in the packages metadata. <br></div><div><div class="gmail-m_-9023867689572438628h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-m_-9023867689572438628m_-6581699733987741501gmail-h5"><div dir="ltr"><div><pre>4. Inline packages - finally, if there is nothing overriding the installed
package, and it opts into type checking.
5. Typeshed (if used) - Provides the stdlib types and several third party libraries
When resolving step (3) type checkers should assure the version of the stubs
match the installed runtime package.
Type checkers that check a different Python version than the version they run
on must find the type information in the ``site-packages``/``dist-packa<wbr>ges``
of that Python version. This can be queried e.g.
``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is also recommended
that the type checker allow for the user to point to a particular Python
binary, in case it is not in the path.
To check if a package has opted into type checking, type checkers are
recommended to use the ``pkg_resources`` module to query the package
metadata. If the ``typed`` package metadata has ``None`` as its value, the
package has not opted into type checking, and the type checker should skip that
package.
References
==========
.. [1] PEP 484, Storing and Distributing Stub Files
(<a href="https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files" target="_blank">https://www.python.org/dev/pe<wbr>ps/pep-0484/#storing-and-distr<wbr>ibuting-stub-files</a>)
.. [2] Distutils Extensions, Adding setup() arguments
(<a href="http://setuptools.readthedocs.io/en/latest/setuptools.html#adding-setup-arguments" target="_blank">http://setuptools.readthedocs<wbr>.io/en/latest/setuptools.html#<wbr>adding-setup-arguments</a>)
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
</pre></div></div>
<br></div></div>______________________________<wbr>_________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" target="_blank">https://mail.python.org/mailma<wbr>n/listinfo/python-ideas</a><br>
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">http://python.org/psf/codeofco<wbr>nduct/</a><br>
<br></blockquote></div><br></div></div>
</blockquote></div></div></div><br></div></div>
</blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div></div>