Threading will probably break here as I wasn't on the list for the first email...

My concern with the standard library is that there's a couple things going on:

1. The standard library represents "accepted" functionality, kind of best practice, kind of just conventional.  Everyone (roughly) knows what you are talking about when you use things from the standard library.
2. The standard library has some firm backward compatibility guarantees.  It also has some firm stability guarantees, especially within releases (though in practice, nearly for eternity).
3. The standard library is kind of collectively owned; it's not up to the whims of one person, and can't be abandoned.
4. The standard library is one big chunk of functionality, upgraded all under one version number, and specifically works together (though in practice cross-module refactorings are uncommon).

There's positive things about these features, but 4 really drives me nuts, and I think is a strong disincentive to putting stuff into the standard library.  For packaging I think 4 actively damages maintainability.

Packaging is at the intersection of several systems:

* Python versions
* Forward and backward compatibility with distributed libraries
* System policies (e.g., Debian has changed things around a lot in the last few years)
* A whole other ecosystem of libraries outside of Python (e.g., binding to C libraries)
* Various developer toolkits, some Python specific (e.g., Cython) some not (gcc)

I don't think it's practical to think that we can determine some scope of packaging where it will be stable in the long term, all these things are changing and many are changing without any particular concern for how it affects Python (i.e., packaging must be reactive).  And frankly we clearly do not have packaging figured out, we're still circling in on something... and I think the circling will be more like a Strange Attractor than a sink drain.

The issues exist for other libraries that aren't packaging-related, of course, it's just worse for packaging.  argparse for instance is not "done"... it has bugs that won't be fixed before release, and functionality that it should reasonably include.  But there's no path for it to get better.  Will it have new and better features in Python 3.3?  Who seriously wants to write code that is only compatible with Python 3.3+ just because of some feature in argparse?  Instead everyone will work around argparse as it currently exists.  In the process they'll probably use undocumented APIs, further calcifying the library and making future improvements disruptive.

It's not very specific to argparse, I think ElementTree has similar issues.  The json library is fairly unique in that it has a scope that can be "done".  I don't know what to say about wsgiref... it's completely irrelevant in Python 3 because it was upgraded along the Python schedule despite being unready to be released (this is relatively harmless as I don't think anyone is using wsgiref in Python 3).

So, this is the tension I see.  I think aspects of the standard library process and its guarantees are useful, but the current process means releasing code that isn't ready or not releasing code that should be released, and neither is good practice and both compromise those guarantees.  Lots of moving versions can indeed be difficult to manage... though it can be made a lot easier with good practices.  Though even then distutils2 (and pip) does not even fit into that... they both enter into the workflow before you start working with libraries and versions, making them somewhat unique (though also giving them some more flexibility as they are not so strongly tied to the Python runtime, which is where stability requirements are most needed).

Ian Bicking  |