[Python-Dev] PEP 561: Distributing and Packaging Type Information

[Nathaniel Smith]

On Thu, Oct 26, 2017 at 3:42 PM, Ethan Smith <ethan at ethanhs.me> wrote:
> However, the stubs may be put in a sub-folder
> of the Python sources, with the same name the ``*.py`` files are in. For
> example, the ``flyingcircus`` package would have its stubs in the folder
> ``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are
> not found in ``flyingcircus/`` the type checker may treat the subdirectory as
> a normal package.

I admit that I find this aesthetically unpleasant. Wouldn't something
like __typestubs__/ be a more Pythonic name? (And also avoid potential
name clashes, e.g. my async_generator package has a top-level export
called async_generator; normally you do 'from async_generator import
async_generator'. I think that might cause problems if I created an
async_generator/async_generator/ directory, especially post-PEP 420.)

I also don't understand the given rationale -- it sounds like you want
to be able say well, if ${SOME_DIR_ON_PYTHONPATH}/flyingcircus/
doesn't contain stubs, then just stick the
${SOME_DIR_ON_PYTHONPATH}/flyingcircus/ directory *itself* onto
PYTHONPATH, and then try again. But that's clearly the wrong thing,
because then you'll also be adding a bunch of other random junk into
that directory into the top-level namespace. For example, suddenly the
flyingcircus.summarise_proust module has become a top-level
summarise_proust package. I must be misunderstanding something?

> Type Checker Module Resolution Order
> ------------------------------------
>
> The following is the order that type checkers supporting this PEP should
> resolve modules containing type information:
>
> 1. User code - the files the type checker is running on.
>
> 2. Stubs or Python source manually put in the beginning of the path. Type
>    checkers should provide this to allow the user complete control of which
>    stubs to use, and patch broken stubs/inline types from packages.
>
> 3. Third party stub packages - these packages can supersede the installed
>    untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``,
>    however it is encouraged to check the package's metadata using packaging
>    query APIs such as ``pkg_resources`` to assure that the package is meant
>    for type checking, and is compatible with the installed version.

Am I right that this means you need to be able to map from import
names to distribution names? I.e., if you see 'import foo', you need
to figure out which *.dist-info directory contains metadata for the
'foo' package? How do you plan to do this?

The problem is that technically, import names and distribution names
are totally unrelated namespaces -- for example, the '_pytest' package
comes from the 'pytest' distribution, the 'pylab' package comes from
'matplotlib', and 'pip install scikit-learn' gives you a package
imported as 'sklearn'. Namespace packages are also challenging,
because a single top-level package might actually be spread across
multiple distributions.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org