[Distutils] PEP 426 updated (with more than you ever wanted to know about version schemes)

[Nick Coghlan]

On Tue, Feb 12, 2013 at 6:03 PM, Chris Jerdonek
<chris.jerdonek at gmail.com> wrote:
> On Mon, Feb 11, 2013 at 11:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Making
>> sure the necessary metadata for the build step is provided as part of
>> the sdist is the responsibility of the Archiver.
>>
>> We don't really *care* what format the Archiver uses to gather info
>> from the user, so long as it uses PEPs 426 and 427 to communicate
>> those details to later steps in the chain.
>>
>> So, to my mind, the next PEP we're missing is actually one for the
>> *sdist* format itself, including a definition for how the
>> meta-packaging system should invoke the sdist->wheel build step,
>> rather than one for the Archiver/Builder configuration data (which is
>> what PEP 390 tries to be, and which I'm not convinced needs to be
>> standardised at all, so long as the Archiver takes care of translating
>> it to the standard formats).
>
> Can you explain your rationale here?  Instinctively, it seems like it
> would benefit project authors and Archiver authors to have a
> standardized (or at least "blessed") way of providing data to
> Archivers.  For example, there would be less for project authors to
> learn if they need to use different Archivers for different projects.
> I think I understand why it's not *necessary* (because only the
> project owners/authors would be using the Archiver), but naively it
> seems like it would help to standardize this.

As an *Archiver* API, I actually don't see anything wrong with the
existing approach of "./setup.py sdist". The use of arbitrary Python
code is primarily a problem at dependency analysis and install time,
rather than during the creation of the sdist. At build time, I'd to
get away from setup.py as well, to make it clear that builders only
need to be able to take an unpacked sdist and turn it into an
appropriate wheel, rather than implementing any of the other setup.py
commands.

However, bento and distutils2 have already proposed different
competing approaches to changing the archiver UI, and
setuptools/distribute offer a superset of the standard distutils UI.

Basically, the front end archiver API is where the greatest divergence
between projects is, and is also the one that most directly affects
the developers on individual projects. Therefore, I see the chance of
rapid widespread adoption of a new interface on that front to be
effectively nil and a leading cause of the burnout seen in previous
attempts to better standardise Python's distribution toolchain.

By contrast, I think it's *definitely* possible for all of
setuptools/distribute, distutils (in Python 3.4), distutils2 and bento
to support emitting an sdist with the appropriate v1.3 metadata in
PKG-INFO that:
- correctly identifies the project's dependencies
- correctly uses Setup-Requires-Dist to indicate the required build chain
- correctly uses Extension, extension fields and potentially
additional metadata files to convey any supplementary metadata
- supports either "./setup.py bdist_wheel" or a TBD build hook to
create a wheel directly from the sdist with no additional information

Individual projects shouldn't have to do anything except upgrade to
newer versions of their tools, but we'll get all the benefits that I'm
after in terms of static metadata (with dependencies recorded in
PKG-INFO), the ability for a wheel to be built from an sdist given
only the sdist as a starting point, *and* for deployment to happen
from a wheel without invoking any project-specific code.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia