[Python-Dev] Re: [PythonLabs] PEP 2

[Martijn Faassen]

[Barry]

 [snip] 
 I was going to point David at PEP 2 as the guidelines for getting modules 
 added to the standard library, but I don't think PEP 2 really describes 
 current practice.

 PEP 2 says:

    When developers wish to have a contribution accepted into the
    standard library, they will first form a group of maintainers
    (normally initially consisting of themselves).

    Then, this group shall produce a PEP called a library PEP. A
    library PEP is a special form of standards track PEP.  The library
    PEP gives an overview of the proposed contribution, along with the
    proposed contribution as the reference implementation.  This PEP
    should also contain a motivation on why this contribution should
    be part of the standard library.

 I think only in rare situations do we need a PEP for a library
 module.  If you agree then I think we should rewrite PEP 2 to describe
 current practice.

 [Barry's description of current practice later in this post]

[Guido]

 Sounds like a good idea.

 It really depends a lot on circumstances.  PEP 282 was written to
 propose a logging module, and then a logging module was proposed that
 will soon go into the std lib (I hope).  But Optik will end up being
 adapted without a PEP.

[back to me]

PEP 2 indeed does not describe current practice; it tries to introduce new
procedures for the development of the standard library.

PEP 2 was written as informed by this post by Tim Peters:

http://groups.google.com/groups?q=g:thl2977775070d&dq=&hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=mailman.993958922.4025.python-list%40python.org

which in turn is referring to this post by Guido:

http://groups.google.com/groups?q=g:thl717774049d&dq=&hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=cpvglejfl8.fsf%40cj20424-a.reston1.va.home.com

Tim said the following:

> All the core
> developers have done major work on the libraries, so that's not a hangup.
> What is a hangup is that people also want stuff the current group of core
> developers has no competence (and/or sometimes interest) to write.  Like SSL
> support on Windows, or IPv6 support, etc.  Expert-level work in a field
> requires experts in that field to contribute.  We also need a plan to keep
> their stuff running after they go away again, the lack of which is one
> strong reason Guido resists adding stuff to the library.

And he suggested I look at the then empty PEP 2.

I took the point of these postings to be that many core library modules
cannot be developed and maintained by the core developers, as they lack
the knowledge and expertise to do so. Therefore the community needs
to develop and importantly also maintain those. In particular,
having explicit and active maintainers was held to be a particularly 
important precondition for library inclusion by the core developers.

Since I thought development of the standard library was important I
tried to make sure that these requirements were to be fulfilled by
people wanting to add a new module.

Barry describes these steps as the way new modules get added now:

- develop the library as an independent project, outside the Python project

- make the library available to the Python community, usually in the
  form of a distutils package

- get feedback and experience from the Python community at large

- if the module becomes popular, is widely backed, and/or fills a   
  niche in the standard library, propose it for inclusion in the core
  distro via discussion and consensus building on python-dev
   
And quoting from PEP 2:
  
    The library PEP differs from a normal standard track PEP in that
    the reference
    implementation should in this case always already have been
    written before the PEP is to be reviewed for inclusion by the
    integrators and to be commented upon by the community; the
    reference implementation _is_ the proposed contribution.

and Barry quotes PEP 2 himself:

    When developers wish to have a contribution accepted into the
    standard library, they will first form a group of maintainers
    (normally initially consisting of themselves).

This is not incompatible at all with the above procedure, if you 
read it carefully. :) The idea of PEP 2 is that there is *already* 
a (potential) contribution, and people want it accepted into the
standard library. The PEP does not describe where this contribution
is coming from; it may indeed be a popular module or it may fill
a niche or whatnot, or it may simply have a stunning design and
implementation extremely useful to everybody. I can make this more
explicit in the PEP.

What PEP 2 tries to supply is a procedure to follow if people
have already decided they would like to try to get a module or set of
modules accepted into the standard library. They can decide this before
or after they write the module; the PEP doesn't care -- as long as the
module is there when they submit the library PEP. At least they know
there'll be Integrators that will review things, and they know they had
better come up with some maintainers before submitting the PEP.

I can add a phrase to make initial community participation more important,
though the PEP process itself already indicates to ask for community input
when submitting a PEP.

Right now the Integrators are described to be PythonLabs, but this could be a
separate group dedicated to maintaining the library later on if desired.

Anyway, what I was aiming for with the PEP was not to codify existing
procedure but to improve on it, informed by what I thought were
the requirements. I thought it was a problem that the standard library is 
apparently not the central focus of the core developers, while the standard
library *is* a very important part of what makes Python appealing. If
the community is to develop it further, I thought the procedures to do
so needed a bit more framework than there is now.

Perhaps I misunderstood something and perhaps PEP 2 won't help anyway,
but that's the background behind it.

I cc-ed my reply to Python-dev as this may need a bit more input. There was
only little (Aazh I recall, though shamefully I think I forgot to integrate
one of his suggestions on backporting bugfixes) when I first posted it both 
on comp.lang.python and on python-dev, but perhaps the reasoning behind it
was unclear.

PEP 2 is here:

http://www.python.org/peps/pep-0002.html

Regards,

Martijn