[Distutils] PEP 426 updated (with more than you ever wanted to know about version schemes)

[Chris Jerdonek]

[In this e-mail I'm combining comments to a few of the e-mails in this thread.]

On Mon, Feb 11, 2013 at 11:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> My overarching ambition is to encourage migration to a model where
> Archivers create sdists from source control, Builders create wheels
> from sdists, and Installers are able to retrieve wheels and deploy
> them onto a system. That is:
>
> source checkout -> Archiver -> sdist -> Builder -> wheel -> Installer
> -> deployed

Where does the "??? -> PyPI" step fit into the above, and which of the
above tools if any should take responsibility for that step?  Related
to this, is it necessary to spell out minimum requirements that PyPI
or alternate PyPI implementations should satisfy?


On Tue, Feb 12, 2013 at 12:33 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Feb 12, 2013 at 6:25 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> How far do you want to standardize the sdist format? For the goals you
>> mentioned earlier you'd only have to specify that an sdist contains
>> a configuration file that says which builder should be used. It might be
>> nice if the full PKG-INFO contents were also available to extract some
>> metadata without invoking the builder.
>
> The sdist format already requires PKG-INFO - it just needs to be
> documented *as* a standard format, rather than implied by "what
> distutils creates when you call 'setup.py sdist'". PEP 426 actually
> takes a fairly big step in that direction already by declaring that
> sdist archives should contain a PKG-INFO file that follows PEP 426
> (see http://www.python.org/dev/peps/pep-0426/#metadata-files)

Something else that would help is guidance on which source files
should be included in an sdist and when.  For example, as a project
author, it wasn't obvious to me whether files like the following
should be included (in my case via MANIFEST.in): textual documentation
files, VCS-related files like .gitignore, unittest files, license
files, test-support files like .travis.yml and tox.ini, *.pyc files,
etc.  Because sdist stands for "source distribution," I took this
literally to mean everything under source control.

A side note related to PyPI and sdists: at one time I was confused as
to why setup.py's upload command didn't update/register a package's
metadata on PyPI even though the sdist contained a proper PKG-INFO.
For reasons like this, it may also be worth discussing/documenting how
distribution to servers like PyPI should be done and what the
expectations are there, if any.


On Tue, Feb 12, 2013 at 12:52 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Basically, the front end archiver API is where the greatest divergence
> between projects is, and is also the one that most directly affects
> the developers on individual projects. Therefore, I see the chance of
> rapid widespread adoption of a new interface on that front to be
> effectively nil and a leading cause of the burnout seen in previous
> attempts to better standardise Python's distribution toolchain.

As an analogy, it may be useful to compare this to the variety of VCS
systems used (Git, Mercurial, etc), which are even closer to the
developers on individual projects (and so may have even greater
divergence).

--Chris