[Distutils] distlib updated - comments sought

[Daniel Holth]

On Fri, Oct 5, 2012 at 1:24 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 5 October 2012 17:04, Daniel Holth <dholth at gmail.com> wrote:
>
>> ~1300 of the ~20000 packages on pypi have trouble using setup.py as
>> their build system / metadata source format.
>
> That's interesting information. Do you know in what way they have
> trouble with setup.py? Do they not use it at all, do they need
> features it doesn't provide, or what?

I'm basing this only on Vinay's numbers of
how-many-packages-can-generate-a-yaml. It's probably mostly packages
that import something he didn't have installed inside setup.py, but I
don't have a good way to find out exactly what is wrong with each one.
An awful lot of packages do work fine; I don't have an automated way
to detect that either.

>> People have to install setuptools against their will because there is
>> only one implementation of the pkg_resources API and 75% of the
>> packages on pypi require setuptools.
>
> I wish we could separate pkg_resources and setuptools. I'd love to
> know which packages needed each (but I suspect that's not a question
> that can be answered without looking at the actual code). Ignoring the
> egg support aspects, pkg_resources is something that could be replaced
> - a reasonable proportion of distlib offers alternatives to the
> pkg_resources code, and more could be added. On the other hand,
> setuptools per se is almost entirely a build time facility, so
> shouldn't be needed at runtime (and so using it for build should be
> relatively unimportant).

And there is a good bit of pkg_resources.py that is only needed at
install time, if you don't mind giving up [console_scripts] dependency
loading. It would probably be a win to only populate working_set
lazily at least. Whatever you did would probably mean the loss of a
feature for some users, but that might be OK if they knew what to
expect.