On Tue, Oct 29, 2019 at 10:50 AM Brett Cannon <brett@python.org> wrote:
Jukka Lehtosalo wrote:
> On Tue, Oct 22, 2019 at 9:59 AM Ivan Levkivskyi levkivskyi@gmail.com
> wrote:
> > Also IIUC Jukka (cc'ed) is going to spend a week or
> > two early November
> > working 100% on kick-starting
> > the transformation of typeshed to a more modular model.

That's great to hear! Is any of this outlined/planned somewhere? Or is this all in your head, Jukka? ;)

> > I'm probably going to work at least on improving stubgen so that it can
> more reliably generate draft stubs for most third-party packages. Another
> small project would be to add support for writing tests for stubs in
> typeshed. These should make it easier to create baseline stubs for many
> additional third-party packages and to ensure that they are not quite
> broken.

Yeah, when I thought about the dream scenario the one that came to mind would be to check projects for `py.typed` and if they lacked it and a `-stubs` project to then generate the stubs for the project as best effort. And then if someone came along and contributed a better, tighter set of stubs to then test that they would be valid to the best of everyone's knowledge somehow in an automated fashion. That way short of needing to step in for spam purposes, things could more-or-less be optimized.

> I may also look at automated uploading of stub packages.
> Perhaps we can work around the lack of pypi namespaces by writing some
> additional tooling. A strawman proposal would be to embed a cryptographic
> signature in packages uploaded from typeshed and have a tool that downloads
> the stubs and verifies the signature in the stub package. This could be
> partially integrated to tools like mypy so that it could suggest how to
> safely download stubs if some stub seems to be missing.

Probably whatever PyPI has set up would work.

One thing I have always wondered about is how typeshed handles package versions? It's currently a completely flat namespace with no inherent versioning, so when a major version for a package comes out that changes the types of things in a non-backwards-compatible fashion, how are projects supposed to express that for type stubs on typeshed (or in their `-stubs` packages on PyPI for that matter)?

PEP 561 has the following on this topic:

In addition, stub-only distributions SHOULD indicate which version(s)
of the runtime package are supported by indicating the runtime distribution's
version(s) through normal dependency data. For example, the
stub package ``flyingcircus-stubs`` can indicate the versions of the
runtime ``flyingcircus`` distribution it supports through ``install_requires``
in distutils-based tools, or the equivalent in other packaging tools. [...]
 
So if we start having stub-only packages in typeshed (like DefinitelyTyped, and per Jukka's plans) we should do it the same way.

--
--Guido van Rossum (python.org/~guido)