On Mon, Jun 7, 2010 at 3:40 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
On Mon, Jun 7, 2010 at 10:00 PM, Ian Bicking <ianb@colorstudy.com> wrote:
> On Mon, Jun 7, 2010 at 2:57 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
>>
>> Now for older versions of Python, I will provide a backport at PyPI,
>> so people can use
>> it under Python 2.x. This backport will probably be made with the
>> trunk so the 2.x line has the latest
>> code. IOW the latest 2.7 release might be more advanced than the one
>> provided in 3.3 for example,
>> but I don't see this as a problem.
>
> It means that, at least for pip, distutils2 3.3 would be effectively the
> last version.

To make sure it's clear: The latest would be 3.4 here, with its
backport in 2.7 and an older version
in 3.3.

The latest version pip could *depend on* would be 3.3, meaning all subsequent releases would be relatively unimportant.  If you have to work around a known-bad version of something, then all later versions become liabilities instead of improvements (unless the workaround is egregiously ugly).

I'd be okay saying that pip could require 3.4+, but not if it means Python 3.3 users would be excluded.

> If there are important bugs we'll have to work around them.
> If there are added features we'll have to ignore them.

Not for the bug fixes because they will likely to be backported in all
versions. (3.3 and 2.7)

In minor releases, which are new Python releases and take a long time to be widely enough distributed to depend on the bug fix.  Though admittedly less time than waiting for a major release to die.

Now for new features, if pip uses the latest 2.x and the latest 3.x
versions, you will get them.
I am not sure why you would have to ignore them.  You would probably want to
use the new features when they are released, and still make your code
work with older versions.

A feature that may or may not be available is not a useful feature.  We'll just have to backport distutils2 features in an ad hoc way using conditional imports and other nuisances.

This is not a new problem btw: if you want to support several versions
of Python, you have to
work around the differences.

Yes, but because the standard library changes so little it's not too bad, and in some cases we can rely on backports, and otherwise we simply ignore new functionality.

Example: There's a big bug in tarfile in Python 2.4, and I had to
backport part of the 2.5 version for a while
in my 2.4 projects. That's doesn't mean I don't want tarfile to be in
the stdlib.

Instead of conditionally monkeypatching tarfile, I'd rather I just had a known-good version of tarfile.  And maybe in a sense that is a solution; why try to patch tarfile at all, why not just include swaths of the standard library inline with libraries?  Right now typically in projects I've noticed we carefully tease apart a libraries bugs when monkeypatching in an upgrade... but that's probably not worth it.  OTOH, I don't think people would be happy if I just included all of distutils2 in pip with some sys.path magic to "upgrade" distutils2 as needed.  But then... it might be the most reasonable approach.

Or another option, allow versioning of portions of the standard library as need demands.  Versioned portions of the standard library may still be quite constrained with respect to backward compatibility, but at least there would be an orderly way to handle backports and for libraries to require bugfixes instead of monkeypatching them in.  Maybe an additional constraint would be that all new features have to be in new modules or via new names, and so upgrades would be additive and less likely to affect backward compatibility.  And we just keep bad stuff out of the standard library (since perhaps the PyXML lessons is conflate a namespace issue with what was simply a lot of bad code).

As an example of how this might have worked, unittest enhancements would have been in a separate module or with as a TestCase subclass (TestCase2) or something else highly unintrusive, and the result could be installed in any version of Python with little danger of conflicts.  I.e., API versioning (for the standard library only) gets pushed into module and class names and isn't separate metadata.

--
Ian Bicking  |  http://blog.ianbicking.org