[Python-ideas] stdlib upgrades

[Jesse Noller]

On Tue, Jun 1, 2010 at 5:40 PM, Tarek Ziadé <ziade.tarek at gmail.com> wrote:
> On Tue, Jun 1, 2010 at 4:46 PM, Jesse Noller <jnoller at gmail.com> wrote:
> [..]
>> I dislike this more than I thought I would - I would rather have the
>> stdlib broken out from core and have it have more releases than the
>> whole of python then allowing for piecemeal "blessed" upgrades.
>> Allowing piecemeal upgrades of the stdlib means you have to say
>> something akin to:
>>
>> "I support Python 2.6, with the upgraded unittest (2.6.1.3), socket
>> (2.6.1.2) and multiprocessing modules"
>>
>> And so on. Sure, API compatibility should be "fine" - but we all know
>> that there are exceptions to the rule all the time, and that alone is
>> enough to put the nix on allowing arbitrary upgrades of individual
>> modules within the standard lib. For package authors, and users, the
>> simple "I support 2.6" statement is key. For corporations with strict
>> upgrade checks and verifications, the same applies.
>
>
> What I expect would be for some projects to state :
>
>  "I support Python 2.6, with the upgraded unittest (2.6.1.3), or Python 3.2"
>
> Instead of:
>
>   "I support Python 2.6, with unittest2 or Python 3.2 with its own unittest"
>
>
> Because the latter makes more work in the project itself, (and no
> difference on the
> corporation/end user side) where it has to deal with two different
> unittest versions. Well, the
> same, but with a different namespace that is used to be able to
> install it on previous
> Python versions besides the stdlib one.

Either fight is a losing one. In the first, you're requiring that
someone *fundamentally alter* their standard library install to monkey
patch in something with the same name, which means it may or may not
break something else - which makes it a non zero risk, and therefore,
unacceptable to a lot of people.

The second requires that the user install an external package; which;
until we include something as a standard, is a fools errand, only to
be taken on by the bravest of people (that might be hyperbole ;))

In all seriousness - the second you ask people to alter one tiny
sliver of the stdlib for the sake of your unique-snowflake project or
app, you've lost. The stdlib is a critical piece of Python - and it's
relative integrity is assumed when people download it from the
python.org website. Asking them to download it from the site, and then
possibly install piecemeal upgrades seems like a bad idea.

Imagine a future where project dependencies look like this:

Python 2.7.1
Python 2.7.1 with upgraded unittest
Python 2.7.1 with upgraded unittest, socket, multiprocessing
Python 2.7.1 with upgraded unittest, socket, multiprocessing, httplib

And so on - sure, eventually (say, 6 months later) there might be a
2.7.2 with all of those changes rolled in, but that begs the question
- why release them individually when you know there's another release
coming shortly, and avoid the confusion?

> At some point, if a package or module in the stdlib evolve in a
> backward compatible
> way, it would be nice to be able to upgrade an existing Python installation.

Yes, but wouldn't it also be nice to simply have a built in package
installation script, and a shorter (say, 6 month) release cycle for
patch releases which maintain the backwards compatibility guarantee?
This way, bug fixes can move more quickly.

We're really discussing the window from a release, to the next - which
could easily be shortened lacking language changes (stdlib bugfixes
only).

> And this is going to be more and more true with the moratorium I guess: what
> people are creating now for Python should work in a wider range of Pythons.
>
> Now, releasing the stdlib on its own and shortening its cycle would also resolve
> the problem we have. But then, while there will be less combinations,
> the problems you have mentioned will remain the same.
> Just replace in your example "I support Python 2.6, with the upgraded unittest
> (2.6.1.3), socket (2.6.1.2) and multiprocessing modules" by  "I
> support Python 2.6,
> with the upgraded stdlib 2.6.1.2".

Then don't fragment it - just release everything more rapidly. And
then suffer from the fact that OS vendors don't pick up your releases
quickly, and so on and so forth.

All I'm trying to say, is that allowing piecemeal upgrades of stdlib
modules is a risky prospect - I know plenty of people (myself
included) who write code which is ONLY dependent on the standard
library intentionally, to save ourselves from the
packaging/installation/etc heartache.

This isn't just because of the pain of installing, or dependency
management issues - it's because the stdlib is known, stable and the
one requirement we can rely on, other than the language itself. The
stdlib doesn't require anyone to install from github, or bitbucket, or
figure out distribute vs. distutils2 - it's just there, despite all
it's warts and dusty corners.

jesse