[Python-ideas] Moving development out of the standard library

Ian Bicking ianb at colorstudy.com
Mon Jun 7 23:22:56 CEST 2010

On Mon, Jun 7, 2010 at 3:40 PM, Tarek Ziadé <ziade.tarek at gmail.com> wrote:

> On Mon, Jun 7, 2010 at 10:00 PM, Ian Bicking <ianb at colorstudy.com> wrote:
> > On Mon, Jun 7, 2010 at 2:57 PM, Tarek Ziadé <ziade.tarek at gmail.com>
> wrote:
> >>
> >> Now for older versions of Python, I will provide a backport at PyPI,
> >> so people can use
> >> it under Python 2.x. This backport will probably be made with the
> >> trunk so the 2.x line has the latest
> >> code. IOW the latest 2.7 release might be more advanced than the one
> >> provided in 3.3 for example,
> >> but I don't see this as a problem.
> >
> > It means that, at least for pip, distutils2 3.3 would be effectively the
> > last version.
> To make sure it's clear: The latest would be 3.4 here, with its
> backport in 2.7 and an older version
> in 3.3.

The latest version pip could *depend on* would be 3.3, meaning all
subsequent releases would be relatively unimportant.  If you have to work
around a known-bad version of something, then all later versions become
liabilities instead of improvements (unless the workaround is egregiously

I'd be okay saying that pip could require 3.4+, but not if it means Python
3.3 users would be excluded.

 > If there are important bugs we'll have to work around them.
> > If there are added features we'll have to ignore them.
> Not for the bug fixes because they will likely to be backported in all
> versions. (3.3 and 2.7)

In minor releases, which are new Python releases and take a long time to be
widely enough distributed to depend on the bug fix.  Though admittedly less
time than waiting for a major release to die.

Now for new features, if pip uses the latest 2.x and the latest 3.x
> versions, you will get them.
> I am not sure why you would have to ignore them.  You would probably want
> to
> use the new features when they are released, and still make your code
> work with older versions.

A feature that may or may not be available is not a useful feature.  We'll
just have to backport distutils2 features in an ad hoc way using conditional
imports and other nuisances.

This is not a new problem btw: if you want to support several versions
> of Python, you have to
> work around the differences.

Yes, but because the standard library changes so little it's not too bad,
and in some cases we can rely on backports, and otherwise we simply ignore
new functionality.

Example: There's a big bug in tarfile in Python 2.4, and I had to
> backport part of the 2.5 version for a while
> in my 2.4 projects. That's doesn't mean I don't want tarfile to be in
> the stdlib.

Instead of conditionally monkeypatching tarfile, I'd rather I just had a
known-good version of tarfile.  And maybe in a sense that is a solution; why
try to patch tarfile at all, why not just include swaths of the standard
library inline with libraries?  Right now typically in projects I've noticed
we carefully tease apart a libraries bugs when monkeypatching in an
upgrade... but that's probably not worth it.  OTOH, I don't think people
would be happy if I just included all of distutils2 in pip with some
sys.path magic to "upgrade" distutils2 as needed.  But then... it might be
the most reasonable approach.

Or another option, allow versioning of portions of the standard library as
need demands.  Versioned portions of the standard library may still be quite
constrained with respect to backward compatibility, but at least there would
be an orderly way to handle backports and for libraries to require bugfixes
instead of monkeypatching them in.  Maybe an additional constraint would be
that all new features have to be in new modules or via new names, and so
upgrades would be additive and less likely to affect backward
compatibility.  And we just keep bad stuff out of the standard library
(since perhaps the PyXML lessons is conflate a namespace issue with what was
simply a lot of bad code).

As an example of how this might have worked, unittest enhancements would
have been in a separate module or with as a TestCase subclass (TestCase2) or
something else highly unintrusive, and the result could be installed in any
version of Python with little danger of conflicts.  I.e., API versioning
(for the standard library only) gets pushed into module and class names and
isn't separate metadata.

Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100607/23275974/attachment.html>

More information about the Python-ideas mailing list