[Distutils] Comments on PEP 426

[Paul Moore]

On 7 September 2013 23:36, Carl Meyer <carl at oddbird.net> wrote:
>> But if a distribution uses only distutils, it won't be declaring any
>> dependency metadata through install_requires=, setup_requires= etc. or any
>> other kwarg recognised only by setuptools - so where would the dependency
>> information come from?
>
> It wouldn't have it, of course. I wasn't implying that it would, just
> that the egg-info command gives pip a uniform way to get all the
> available metadata from any distribution, without trying to detect what
> its setup.py uses.

It might well have dependency information. The point here is that it
is perfectly possible for a setup.py to write its own custom cmdclass
that implements an egg-info command which writes out dependency
metadata that it gets from anywhere it likes. Obviously, to do this,
it needs to follow the spec for the egg-info command, in terms of what
it should produce, and TBH I don't know without looking how clearly
that is defined by the setuptools documentation (for example, it may
be too mixed up with the rest of the documentation to be useful for an
independent implementation, or it may be defined purely by the
implementation).

The *other* problem is that a custom implementation of an egg-info
command is pretty much certain to be incompatible with pip injecting
setuptools. And that's the big issue, injecting setuptools actively
prevents people writing their own implementations of the relevant
command line APIs.

>> It seems like the egg_info command is the sole
>> reason, or did I miss something?
>
> Also the installation command (not having to detect what the setup.py
> uses and decide accordingly whether to supply
> --single-version-externally-managed)

That to me is a major issue with setuptools, as it *behaves
differently* than distutils does for the same command line. But
setting that aside, again a setup.py could implement a custom cmdclass
that simply ignores the --single-version-externally-managed flag on
install. And again, doing so could easily be incompatible with
setuptools injection.

> and the installed format, which I
> already mentioned but you snipped in your reply. (Setuptools with
> --single-version-externally-managed installs metadata in a .egg-info
> dir, plain distutils just installs a single file of metadata, not a
> directory; the directory gives pip a place to put the results of --record).

That is an issue which hasn't been picked up on yet. But I'd argue
that pip could easily check what version the setup.py created and
adapt accordingly (upgrading the single-file format to the directory
format). Sure, it doesn't, because the setuptools injection makes it
unnecessary. But that's getting cause and effect backwards...

> Also there's "setup.py develop"; because of the setuptools-import hack,
> pip is able to provide this feature uniformly for all distributions (as
> "pip install -e"), not only setuptools-using distributions.

Again, distributions could write their own develop command, but that's
probably not practical. The develop functionality is probably the one
case here where I'd agree that setuptools injection is the only
realistic way to go. But it would be possible to *only* inject
setuptools if -e was used, so it's still not true that always
injecting setuptools is necessary.

> There may be other things pip relies on the setuptools-import hack for
> that aren't crossing my mind at the moment.

Regardless of all the above, I do still understand the reasoning for
the hack and accept that it's probably the most practical option right
now. It's been used in pip forever, and has caused far less problems
than it has solved. All I want to do here is to make sure people are
clear that it's still a trade-off:

1. It stifles certain types of innovation (experimenting with other
ways of implementing egg-info, for example).
2. It encourages continuing the bad habit of "specification by
implementation" (see my point about not knowing the spec for the
egg-info command)

Paul