[Python-Dev] Python Language Summit at PyCon: Agenda

[Nick Coghlan]

On Fri, Mar 1, 2013 at 9:39 AM, Doug Hellmann <doug.hellmann at gmail.com> wrote:
>
> On Feb 27, 2013, at 11:51 AM, Michael Foord wrote:
>
>> Hello all,
>>
>> PyCon, and the Python Language Summit, is nearly upon us. We have a good number of people confirmed to attend. If you are intending to come to the language summit but haven't let me know please do so.
>>
>> The agenda of topics for discussion so far includes the following:
>>
>> * A report on pypy status - Maciej and Armin
>> * Jython and IronPython status reports - Dino / Frank
>> * Packaging (Doug Hellmann and Monty Taylor at least)
>
> Since the time I suggested we add packaging to the agenda, Nick has set up a separate summit meeting for Friday evening. I don't know if it makes sense to leave this on the agenda for Wednesday or not.
>
> Nick, what do you think?

I think it's definitely worth my taking some time to explain my goals
for the Friday night session, and some broader things in terms of
where I'd like to see packaging going, but a lot of the key packaging
people aren't involved in Python language development *per se*, and
hence won't be at the language summit.

There's also one controversial point that *does* need to be raised at
the summit: I would like to make distutils-sig the true authority for
packaging standards, so we can stop cross-posting PEPs intended to
apply to packaging in *current* versions of Python to python-dev. The
split discussions suck, and most of the people that need to be
convinced in order for packaging standards to be supported in current
versions of Python aren't on python-dev, since it's a tooling issue
rather than a language design issue. Standard lib support is necessary
in the long run to provide a good "batteries included" experience, but
it's *not* the way to create the batteries in the first place. Until
these standards have been endorsed by the authors of *existing*
packaging tools, proposing them for stdlib addition is premature, but
has been perceived as necessary in the past due to the confused power
structure.

This means that those core developers that want a say in the future
direction of packaging and distribution of Python software would need
to be actively involved in the ongoing discussions on distutils-sig,
rather than relying on being given an explicit invitation to weigh in
at the last minute through a thread (or threads) on python-dev. The
requirement that BDFL-delegates for packaging and distribution related
PEPs also be experienced core developers will remain, however, as
"suitable for future stdlib inclusion" is an important overarching
requirement for packaging and distribution standards. Such delegates
will just be expected to participate actively in distutils-sig *as
well as* python-dev.

Proposals for *actual* standard library updates (to bring it into line
with updated packaging standards) would still be subject to python-dev
discussion and authority (and would *not* have their Discussions-To
header set). Such discussions aren't particularly relevant to most of
the packaging tool developers, since the standard library version
isn't updated frequently enough to be useful to them, and also isn't
available on older Python releases, so python-dev is a more
appropriate venue from both perspectives.

At the moment, python-dev, catalog-sig and distutils-sig create an
awkward trinity where decision making authority and the appropriate
venues for discussion are grossly unclear. I consider this to be one
of the key reasons that working on packaging issues has quite a high
incidence of developer burnout - it's hard to figure out who needs to
be convinced of what, so it's easy for the frustration levels to reach
the "this just isn't worth the hassle" stage (especially when trying
to bring python-dev members up to speed on discussions that may have
taken months on distutils-sig, and when many of the details are
awkward compromises forced by the need to support *existing* tools and
development processes on older versions of Python). Under my proposal,
the breakdown would be slightly clearer:

distutils-sig: overall authority for packaging and distribution
related standards, *including* the interfaces between index servers
(such as PyPI) and automated tools. If a PEP has "Discussions-To" set
to distutils-sig, announcements of new PEPs, new versions of those
PEPs, *and* their acceptance or rejection should be announced there,
and *not* on python-dev. The "Resolution" header will thus point to a
distutils-sig post rather than a python-dev one. distutils-sig will
focus on solutions that work for *current* versions of Python, while
keeping in mind the need for future stdlib support.

python-dev: authority over stdlib support for packaging and
distribution standards, and the "batteries included" experience of
interacting with those standards. Until a next generation distribution
infrastructure is firmly established (which may involve years of
running the legacy infrastructure and the next generation metadata 2.x
based infrastructure in parallel), the stdlib will typically trail the
upstream standards substantially, since many upstream enhancements
will run afoul of the standard library's "no new features" rule,
preventing their inclusion in maintenance releases of old Python
versions.

catalog-sig: authority over the PyPI index server (supported by the
infrastructure SIG for actual operation of the service). Key people
are expected to also participate in distutils-sig, as that is where
the expected interface exposed to automated tools will be defined. How
PyPI exposes the packaging and distribution standards to *users*
through its web UI will be up to catalog-sig.

The vehicle for *documenting* such a change in policy would be an
update to PEP 1 to indicate that the "Discussions-To" header should
always point to the list where any formal acceptance of rejection of
the PEP would be announced. If preliminary discussions take place on a
feeder list like python-ideas, or import-sig, or somewhere else, that
will be declared as explicitly irrelevant from the PEP metadata point
of view: the Discussions-To field would be documented as referring to
the list that any Resolution field will eventually reference.

Regards,
NIck.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia