[Python-Dev] packaging location ?

[Nick Coghlan]

On Thu, Sep 13, 2012 at 8:43 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> When the removal was being pondered, the possibility of keeping certain
> bits that were more ready than others was discussed.  Perhaps the best
> way forward is to put it back in bits, with the most finished (and PEP
> relevant) stuff going in first.  That might also give non-packaging
> people bite-sized-enough chunks to actually digest and help with.

This is the plan I'm going to propose. The previous approach was to
just throw the entirety of distutils2 in there, but there are some
hard questions that doesn't address, and some use cases it doesn't
handle. So, rather than importing it wholesale and making the stdlib
the upstream for distutils2, I believe it makes more sense for
distutils2 to remain an independent project, and we cherry pick bits
and pieces for the standard library's new packaging module as they
stabilise.

In particular, Tarek was focused on being able to create *binary* RPMs
automatically. That isn't enough for my purposes, I need to be able to
create *source* RPMs, which can then be fed to the koji build service
for conversion to binary RPMs in accordance with the (ideally)
autogenerated spec file. A binary RPM that isn't built from a source
RPM is no good to me, and the distutils2 support for this approach is
awful, because setup.cfg inherits all the command model cruft from
distutils which is stupidly hard to integrate with other build
systems. I also want to be able to automate most dependency
management, so people can write "Requires: python(pypi-dist-name)" in
their RPM spec files and have it work, just as they can already write
things like "Requires: perl(File::Rules)"

I'm currently working on creating a coherent map of the status quo,
that describes the overall process of software distribution across
various phases (from development -> source archive -> building ->
binary archive -> installation -> import) and looks at the tools and
formats which exist at each step, both legacy
(distutils/setuptools/distribute) and proposed (e.g. distutils2,
bento, wheel), and the kinds of tasks which need to be automated.

Part of the problem with distutils is that the phases of software
distribution are not clearly documented, instead being implicit in the
behaviour of setuptools. The distutils2 project, to date, has not
remedied that deficiency, instead retaining the implicit overall
workflow and just hacking on various pieces in order to "fix Python
packaging". If we're going to get support from the scientific
community (which has some of the more exotic build requirements going
around) and the existing community that wants the full setuptools
feature set rather than the subset currently standardised (primarily
non-Django web developers in my experience), then we need to address
*their* concerns as well, not just the concerns of those of us that
don't like the opaque nature of setuptools and its preference for
guessing in the presence of ambiguity.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia