[Python-Dev] Fwd: PEP 426 is now the draft spec fordistribution metadata 2.0

[Nick Coghlan]

On Tue, Feb 19, 2013 at 9:57 PM,  <a.cavallo at cavallinux.eu> wrote:
>> I've never seen environment markers being used or supported
>> in the wild.
>>
>> I'm not against modernizing the format, but given that version 1.2
>> has been out for around 8 years now, without much following,
>> I think we need to make the implementation bit a requirement
>> before accepting the PEP.

PEP 345 has NOT been an approved specification for 8 years. It was
first drafted in 2005, but not Accepted until 2010:
http://hg.python.org/peps/rev/7d78653a0e79 (as Daniel noted, what
became PEP 426 actually started as *another* round of edits to PEP
345)

PEP 345 was a first attempt at defining a formal standard that
adequately described an *existing* implementation defined software
distribution ecosystem built around the behaviour of setuptools. It
failed, because it did *not* achieve that goal, and because there were
associated declarations along the lines of "to migrate to this
standard, you must completely change the way you build and distribute
your Python code" (by conflating the setup.py -> setup.cfg migration
of distutils2 with the use of the new metadata standard). The response
was an overwhelming meh, as people willing migrated to distribute and
pip (based on the de facto setuptools standards), while the distutils2
project never made it to production readiness because it was trying to
solve too many problems at once.

By contrast, the PEP 376 metadata format created at the same time has
been adopted quite well.

PEPs 345 and 386 were also accepted at a time when python-dev had
neither interest nor extensive experience in packaging systems (or if
anyone did have such experience, they weren't volunteering that
information).

So what's changed since 2010 that makes me think the time is right for
a new metadata version?

1. We've recognised that without setuptools and/or distribute on board
to generate it, and pip to consume it, any new metadata version is
dead in the water. That groundwork has been done (mostly by Daniel as
part of his work on the wheel format)

2. We've systematically gone through and resolved the significant
discrepancies between the formal versioning scheme and the
implementation defined behaviour of setuptools and distribute. This
means that standards compliant versions will sort the same way in
those tools, even if they are not updated.

3. We've addressed other objections PJ Eby (the author of setuptools)
had to the previous version of the format (notably the handling of
project obsolescence).

4. Various other missing features from setuptools are now supported
directly (such as extras)

5. The defined interaction of version specifiers and pre- and post-
releases was not useful. The new version fixes that to be something
developers are more likely to want (i.e. they won't get pre-releases
unless they explicitly reference one).

6. We've enhanced the format so it can handle extensions in a
systematic fashion with clearly defined lines of authority based on
PyPI distribution names, rather than adding arbitrarily named files to
a distribution's info directory without any clear indication of where
to find that specification for the file's meaning.

7. I've updated the metadata specification itself, including the
addition of the Private-Version field, to more clearly explain to
developers that the public Version in their metadata is primarily a
tool for communicating the relative ordering of versions to
installation tools, but they're still free to use their own preferred
version labels internally, and certainly are not obliged to actually
use the full complexity of the standard versioning scheme that the
installers understand.

It's OK if people don't want to read the detailed rationale provided
for each of the major changes as part of the PEP, or if they want to
dispute a particular piece of that rationale. But merely going "it's
too complicated!" or "metadata 1.2 failed, so 2.0 will fail as well!"
is not a reasonable response. Software distribution is complicated -
trying to oversimplify it is one of the reasons setuptools became
necessary.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia