On Mon, 7 Jun 2010 13:33:48 -0500 Ian Bicking wrote:

On Mon, Jun 7, 2010 at 1:14 PM, Antoine Pitrou wrote:

...

On Mon, 7 Jun 2010 11:35:16 -0500 Ian Bicking wrote:

...

I think there is a general consensus that functionality should not be

tied

...

to a Python release, but the results are ad hoc.

I disagree with this. Tying new functionality to a Python release vastly simplifies dependency management (instead of having to track the versions of N external libraries, sometimes with inter-dependencies).

I say there is consensus because as far as I know anything substantial has a maintained version outside the standard library; argparse is implicitly, unittest is unittest2, ElementTree always has maintained a separate existence, simplejson implicitly.

"Anything substantial" is more than exagerated. The modules you are mentioning are exceptions, two of which may even be temporary (argparse and unittest2). Most sdtlib modules don't have external releases, and many of them are still "substantial".

1. How will distutils2 updates be made available between Python releases? 2. How will distutils2 features be made available in older Python releases?

Why are you expecting any of these to happen? I don't know what Tarek intends to do in that respect, but he certainly doesn't have any moral obligation to do external releases. Regards Antoine.

On Mon, Jun 7, 2010 at 9:20 PM, Ian Bicking wrote: [..]

...

...

1. How will distutils2 updates be made available between Python releases? 2. How will distutils2 features be made available in older Python releases?

Why are you expecting any of these to happen? I don't know what Tarek intends to do in that respect, but he certainly doesn't have any moral obligation to do external releases.

distutils2 won't be in 2.7 at least, and any packaging system not available for Python 2 would be irrelevant.

Distutils2 will be provided for Python 2.4 to 3.x with frequent releases. Then, once it's added in 3.2 (maybe 3.3 if not ready by then), its release cycle will be driven by Python's one, with backports released at the same pace for Python versions that didn't have it in the stdlib. Regards Tarek -- Tarek Ziadé | http://ziade.org

On Mon, Jun 7, 2010 at 9:39 PM, Ian Bicking wrote:

On Mon, Jun 7, 2010 at 2:36 PM, Tarek Ziadé wrote:

...

On Mon, Jun 7, 2010 at 9:20 PM, Ian Bicking wrote: [..]

...

...

...

1. How will distutils2 updates be made available between Python releases? 2. How will distutils2 features be made available in older Python releases?

Why are you expecting any of these to happen? I don't know what Tarek intends to do in that respect, but he certainly doesn't have any moral obligation to do external releases.

distutils2 won't be in 2.7 at least, and any packaging system not available for Python 2 would be irrelevant.

Distutils2 will be provided for Python 2.4 to 3.x with frequent releases.

Then, once it's added in 3.2 (maybe 3.3 if not ready by then), its release cycle will be driven by Python's one, with backports released at the same pace for Python versions that didn't have it in the stdlib.

So let's say distutils2 gets version parity with Python,

yes, in other word, it's part of the stdlib

so distutils2 3.3 is released with Python 3.3.  Then distutils2 3.4 is released with Python 3.4, and is backported to all previous versions of Python... except for Python 3.3, which will always have distutils2 3.3?

Distutils2 will behave exactly like other packages in the standard library. That is: - the development is going on in trunk - bug fixes are backported to older python releases that have this package present - some feature or small refactoring are also backported when it makes sense So Python 3.3.2 will have Distutils 3.3.2, but also os.path 3.3.2, and shutil 3.3.2 Like Python 3.4 its own versions. Now for older versions of Python, I will provide a backport at PyPI, so people can use it under Python 2.x. This backport will probably be made with the trunk so the 2.x line has the latest code. IOW the latest 2.7 release might be more advanced than the one provided in 3.3 for example, but I don't see this as a problem. Tarek -- Tarek Ziadé | http://ziade.org

On Mon, Jun 7, 2010 at 10:00 PM, Ian Bicking wrote:

On Mon, Jun 7, 2010 at 2:57 PM, Tarek Ziadé wrote:

...

Now for older versions of Python, I will provide a backport at PyPI, so people can use it under Python 2.x. This backport will probably be made with the trunk so the 2.x line has the latest code. IOW the latest 2.7 release might be more advanced than the one provided in 3.3 for example, but I don't see this as a problem.

It means that, at least for pip, distutils2 3.3 would be effectively the last version.

To make sure it's clear: The latest would be 3.4 here, with its backport in 2.7 and an older version in 3.3.

If there are important bugs we'll have to work around them. If there are added features we'll have to ignore them.

Not for the bug fixes because they will likely to be backported in all versions. (3.3 and 2.7) Now for new features, if pip uses the latest 2.x and the latest 3.x versions, you will get them. I am not sure why you would have to ignore them. You would probably want to use the new features when they are released, and still make your code work with older versions. This is not a new problem btw: if you want to support several versions of Python, you have to work around the differences. Example: There's a big bug in tarfile in Python 2.4, and I had to backport part of the 2.5 version for a while in my 2.4 projects. That's doesn't mean I don't want tarfile to be in the stdlib. Tarek

On 7 June 2010 22:20, Eric Smith wrote:

Tarek Ziadé wrote:

...

On Mon, Jun 7, 2010 at 10:00 PM, Ian Bicking wrote:

...

On Mon, Jun 7, 2010 at 2:57 PM, Tarek Ziadé wrote: If there are important bugs we'll have to work around them. If there are added features we'll have to ignore them.

Not for the bug fixes because they will likely to be backported in all versions. (3.3 and 2.7)

Now for new features, if pip uses the latest 2.x and the latest 3.x versions, you will get them. I am not sure why you would have to ignore them. You would probably want to use the new features when they are released, and still make your code work with older versions.

There's no way for the new features to show up in 3.3, is there? You can't add them to a micro release, and you can't replace a module in the standard library. I think that's Ian's point.

But that's no different to pip using *any* standard library module. If you want to support Python 2.4 you can't use os.path.relpath (or you have to provide it yourself anyway) for example. Michael

pip could use the new features in 3.4, and it could get the new features in 2.x if the users were willing to install the updated library, since it's not in the stdlib. But for 3.3 you'd be stuck.

-- Eric.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- http://www.voidspace.org.uk

On Tue, Jun 8, 2010 at 3:49 AM, Ian Bicking wrote:

On Mon, Jun 7, 2010 at 5:45 PM, Michael Foord wrote:

...

On 7 June 2010 22:20, Eric Smith wrote:

...

Tarek Ziadé wrote:

...

On Mon, Jun 7, 2010 at 10:00 PM, Ian Bicking wrote:

...

On Mon, Jun 7, 2010 at 2:57 PM, Tarek Ziadé wrote: If there are important bugs we'll have to work around them. If there are added features we'll have to ignore them.

Not for the bug fixes because they will likely to be backported in all versions. (3.3 and 2.7)

Now for new features, if pip uses the latest 2.x and the latest 3.x versions, you will get them. I am not sure why you would have to ignore them.  You would probably want to use the new features when they are released, and still make your code work with older versions.

There's no way for the new features to show up in 3.3, is there? You can't add them to a micro release, and you can't replace a module in the standard library. I think that's Ian's point.

But that's no different to pip using *any* standard library module. If you want to support Python 2.4 you can't use os.path.relpath (or you have to provide it yourself anyway) for example.

This is part of why I don't care about reforming or modifying what's in the standard library now -- I know the constraints well, and they can't be changed.  I'm solely concerned about new functionality which need not repeat this pattern.

Are you suggesting to freeze the stdlib development ? So you don't have to deal with different Python version at your level ? If so, that doesn't sound right because making the "batteries included" evolve is part (imho) of the Python spirit, and the constraints we are talking about right now is not a huge problem as you seem to think in my opinion. I don't find it extremely hard to cope with various Python version.

-- Ian Bicking  |  http://blog.ianbicking.org

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- Tarek Ziadé | http://ziade.org

On 6/7/2010 5:20 PM, Eric Smith wrote:

pip could use the new features in 3.4, and it could get the new features in 2.x if the users were willing to install the updated library, since it's not in the stdlib. But for 3.3 you'd be stuck.

I see no reason why 3.3 users could not also download the 3.4 version of something just as well as user of earlier versions (who well might already have the 3.3 version).

On Tue, Jun 8, 2010 at 3:56 PM, Nick Coghlan wrote:

On 09/06/10 05:33, Terry Reedy wrote:

...

On 6/7/2010 5:20 PM, Eric Smith wrote:

pip could use the new features in 3.4, and it could get the new features

...

in 2.x if the users were willing to install the updated library, since it's not in the stdlib. But for 3.3 you'd be stuck.

I see no reason why 3.3 users could not also download the 3.4 version of something just as well as user of earlier versions (who well might already have the 3.3 version).

I believe Eric was pointing out the fact that, by default, there is no directory on sys.path that will override the standard library versions of a module or package for all applications using that interpreter installation.

So you're forced to either resort to destructive replacement (actually overwriting the standard library module on disk) or else tinkering with sys.path in each app or a library to insert an "override" directory before the normal standard library paths.

It sometimes seems to me that, for the advocates of a more granular standard library, proposing standardisation of such an override directory would be an interesting way to test the waters (since it would make it much easier to drop in backported updates to standard library modules).

Setuptools uses .pth file hackery to handle this case (not to intentionally override the stdlib, but it would also do that), and I think it could apply here as well. Also the way Setuptools installs eggs might be useful in this case, as it makes it very obvious in tracebacks what version of a module is being used (and if it is from the standard library). Even if the particular mechanics were revisited, I think these basic ideas would be helpful. -- Ian Bicking | http://blog.ianbicking.org

Tarek Ziadé writes:

The problem is, any project would start overriding the stdlib to fix things or change some behavior, unless this is somehow controlled.

But in the end, that's precisely what you propose to do yourself with "partial stdlib upgrades"! It's just that you trust yourself more than you trust "any project". But that just doesn't fly from the point of the third party clients of the stdlib. Either stability of any particular version's stdlib applies to the stdlib developers too, or it doesn't really apply at all.

Tarek Ziadé writes:

If the maintainer of unittest for example, provides an upgrade for this package, don't you think we can trust that he will provide a more stable upgrade for the unittest package in the stdlib than another project that would implement a unittest package ?

For the users who really care about this, it's not a question of relative stability. Either the only changes in documented behavior involve successful completion of jobs that used to fail, or instability has been introduced. For many people (though a small fraction) that is *very bad*, and they have complained vociferously in the past. I really don't understand where the big benefits are to having minor improvements introduced in bugfix releases. People who want those benefits should upgrade to a more recent series. The people who really need them but must stick to an older series for Python itself can get the most recent version of the few packages that have "must-have" improvements from PyPI. "No behavior changes in micro releases" is an easily understood, reasonably easily followed policy. The policy you propose requires judgment calls that will differ from module maintainer to module maintainer, and every upgrade will involve discussion on python-dev. Yuck.

On 09/06/10 19:05, Tarek Ziadé wrote:

And "not putting distutils2 in the stdlib" is not the solution because this is a problem for all packages in there.

That's exactly what unitest currently do (but with a new name "unittest2") and as soon as Python 2.7 final will be out, unittest will have the same problem: it won't be able to backport new features anymore under the same namespace.

Something we may want to seriously consider is maintaining parallel releases of packages indefinitely when the benefits are deemed to justify the additional overheads. So, for example, many of the current features of unittest2 will be available in unittest in 2.7 and 3.2. This makes those features available to those that rely almost entirely on standard library functionality rather than third party packages. In the meantime, users of previous versions of Python can already use unittest2, and that package will work *without name conflicts* in both 2.7 and 3.2, even though many of its features have been added to the standard library. unittest2 may then even go through a few external releases before being synced up again with the standard library's unittest when 3.3 comes around. Something similar may turn out to be a good idea for distutils2: rather than consider the anticipated merge back into distutils prior to 3.3 the end of the road, instead continue to use distutils2 to release faster updates while evolving the API design towards 3.4. Users then have the choice - the solid, stable standard library version, or the distinctly named, more rapidly updated PyPI version. As others have suggested, this namespace separation approach could be standardised through the use of a PEP 382 namespace package so that users could choose between (e.g.) "unittest" and "distutils" and "cutting_edge.unittest" and "cutting_edge.distutils" (with the latter being the regularly updated, new features and all, PyPI versions, and the former the traditional stable API, bugfix-only standard library versions). That would probably be an improvement over the current ad hoc approach to naming separation for standard library updates. I don't see any way to ever resolve the two competing goal sets (stability vs latest features) without permanently maintaining separate namespaces. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Wed, 9 Jun 2010 07:34:56 -0400 David Stanek wrote:

I had a very similar thought. Why not have all the real development of those packages happen outside of the standard lib and just grab the latest stable version when cutting a new version of Python.

-1. We have had too much trouble with externally-maintained modules such as elementtree and json. The Python SVN (or hg) tree should be the primary place where development takes place. Regards Antoine.

On Wed, Jun 9, 2010 at 08:17, Ian Bicking wrote:

On Wed, Jun 9, 2010 at 6:38 AM, Antoine Pitrou wrote:

...

On Wed, 9 Jun 2010 07:34:56 -0400 David Stanek wrote:

...

I had a very similar thought. Why not have all the real development of those packages happen outside of the standard lib and just grab the latest stable version when cutting a new version of Python.

-1.  We have had too much trouble with externally-maintained modules such as elementtree and json.  The Python SVN (or hg) tree should be the primary place where development takes place.

New releases could also be cut from the Python tree.  I believe everyone here agrees that entering the standard library in any form should imply a greater sense of collective ownership of a package.

But this makes the assumption that core developers are going to choose to develop modules such that they can be released before the next release of Python goes out. There seems to be a lot of extrapolation from the fact that Michael takes the time to do unittest2 (which, now that I think about it, should probably have been named unittest_external or something as it really isn't a sequel to unittest, just an external release) and that Tarek is doing the initial development of distutils2 externally. Out of all the core developers that is not that many. Sure you could maybe toss in ElementTree, but that probably is it (e.g. simplejson only gets new releases when Bob fixes stuff for a new Python release or simply makes an external release of stdlib fixes). For me, importlib will never work in this environment. When new features in modules or the language come in I try to update code to use it when I can. That means that importlib is not going to have a stable release cycle outside of Python's. Nor am I going to be willing to change that practice as that makes development harder for me -- I don't want to have to check if some new feature has been made available externally -- and honestly more boring -- I like using the new features of the language as that helps make core development fun. The only way I see this working is for individual core developers to decide they want to keep a module dependent only on the last minor release of Python (or older). At that point you can try to convince the developers to use a common package name (stdlib, py, stdlib_ext, etc.) and then use namespace packages or pkgutil.extend_path to pull them in under the common package name to signify that they are "early" releases of the modules from the stdlib. That gives you an easier way to gauge usage as you can look at use of the package as a whole to see what kind of community pickup there is. If you can do that *and* show that there was a clear benefit to the community then you might have a chance to get more developers to participate in this development scheme. But in terms of a general python-dev policy, it simply won't happen as it is just too much extra work to force upon everyone (Nick seems to be the only one I can think of still throwing ideas out there, but even with that I don't know how much work he wants to put in).

On 09/06/10 21:34, David Stanek wrote:

On Wed, Jun 9, 2010 at 7:08 AM, Nick Coghlan wrote:

...

Something we may want to seriously consider is maintaining parallel releases of packages indefinitely when the benefits are deemed to justify the additional overheads.

I had a very similar thought. Why not have all the real development of those packages happen outside of the standard lib and just grab the latest stable version when cutting a new version of Python.

Namespaces are fine, but I'd be happy enough with a way for these packages to show up on the Python path before the stdlib.

Another parallel that occurs to me is the stable/testing/unstable distinction in Debian - if you want rock solid (but old) you stick with stable, if you want comparatively cutting edge you go with unstable and testing splits the difference. Ubuntu's "normal release" vs "Long Term Support release" distinction is also worth thinking about. Most home desktop users will upgrade every 6 months (modulo rocky transitions like the upgrade to KDE 4), while corporate users will wait for the next LTS to arrive. To bring that back to a Python context and revisit the proposal that Guido rejected for biennial Python releases with annual standard library releases, suppose that, instead of the standard library itself having annual releases, there was a python-dev maintained (or even third-party maintained, python-dev blessed) "pybackports" project. The idea being, that package would not only make many new standard library features of the current Python release available on widely used previous versions, but also make upcoming features of the *next* Python release available on the current version. The periods of support would be shorter to reflect the faster release Third party libraries and applications could then either target the "rock solid" market and depend solely on the standard library, or go for the "latest and greatest" crowd and require pybackports. Make no mistake, something like this would be quite a bit of work, but it would be far from impossible and could go a long way towards fighting the impression that the standard library is the place where modules go to die. TLDR version: Year A: Python 3.x release, pybackports 3.x.0 release Year B: No Python release, pybackports 3.x.5 release (with 3.y features) Year C: Python 3.y release, pybackports 3.y.0 release Year D: No Python release, pybackports 3.y.5 release (with 3.z features) ...etc Unlike Python itself, Pybackports would not provide bugfix support for prior releases (since it would only be for the latest-and-greatest crowd, those that want the longer support should stick with the standard library). Something to think about, anyway. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On 09/06/10 22:40, Antoine Pitrou wrote:

Isn't that "impression" largely constructed, and propagated by a limited number of people who apparently don't like the very idea of a "batteries included" stdlib? There has been an amount of anti-stdlib activism (including in this thread) that I find both antagonizing and unconstructive. Outside of that vocal minority, there doesn't seem to be that much criticism against the stdlib.

The "where modules go to die" version of it is overstated, but the standard library definitely evolves more slowly than many third party packages. To use numpy as an example (just going off their SF file dates): Dec 2007: 1.0.4 May 2008: 1.1.0 Sep 2008: 1.2.0 Jul 2009: 1.3.0 Apr 2010: 1.4.1 Faster cycle times allow developers to be much more responsive to feedback when changes don't turn out as well as was hoped. The comparatively slow evolution of the standard library is the grain of truth that underlies the exaggerated form. The trick is to explore avenues that make these faster cycle times available to those that want them, while still providing a stable foundation for those that need it. It is exactly this situation that the Ubuntu release cycle is designed around: regular 6-monthly releases for most people, less frequent Long Term Support releases for those that need stability. That said, the status quo, with ad hoc porting to PyPI by module maintainers that consider doing so to be worthwhile is certainly a viable option going forward as well. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Le mercredi 09 juin 2010 à 09:19 -0400, Fred Drake a écrit :

I don't think so. I've no particular dislike of "batteries included" per se. What I don't like is dealing with packages that may or may not be in the standard library (that affects the requirements for my software),

I don't understand what you mean. Are these packages in the stdlib or aren't they? It can't be both.

Writing cross-Python-version code that deals with the differences between the stdlib and 3rd-party versions of packages is certainly unconstructive, but that's an argument to avoid moving packages into the standard library.

I don't see how, really. You make it sound like you have to deal with several versions of the stdlib, but not with several versions of external packages. I wonder why: do you force your users to install version X of module Y? More generally, it seems that people are reproaching things to the stdlib that are equally true for non-stdlib modules. This is quite bewildering to me.

On Wed, Jun 9, 2010 at 3:10 PM, Terry Reedy wrote:

Yes, as N grows, any constant fraction f times N will grow. In fact, f*N can grow even if f shrinks. but just more slowly than N grows.

True. Since there's no empirical measure of either N (number of Python users) or VM (the Vocal Minority), it's hard to tell if f is a constant fraction or something more (or less) interesting. I also suspect that VM is less likely to be readily measurable than N. For those in the VM, the severity of their objections only increases over time, until they exit both N and VM. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller

On Wed, Jun 9, 2010 at 7:08 AM, Nick Coghlan wrote:

Something we may want to seriously consider is maintaining parallel releases of packages indefinitely when the benefits are deemed to justify the additional overheads.

And this provides an opportunity to bring the workload back under control, as well: if we continue the non-stdlib releases for these packages, all we really need to do is remember not to add them to the stdlib in the first place. The one case where we clearly get some win by having the newer package rolled in is distutils2, since that's really about a core service that needs to be available for everyone. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller

On 6/9/2010 5:05 AM, Tarek Ziadé wrote:

To try to find a solution to the problems described in this thread earlier.

If I summarize so far the threads as I understood them, people don't want to rely on stdlib packages because their release cycles are too slow for what they want/need to do with it *today*. A package that enters the stdlib suffers from being slowed down. That's also a huge benefit for many reasons: stability, blessing, etc.

The initial reason is that Ian doesn't want Pip do depend on distutils2 if it's in the stdlib, because he will have to cope with various versions of Python to make sure his users will have the same set of features I guess.

So he needs to provide his own backports of any new distutils2 features.

If we can find a way to facilitate this work, that would be great. IOW, if we can provide somehow a backport of these features so some projects can use it no matter what the python version is...

And "not putting distutils2 in the stdlib" is not the solution because this is a problem for all packages in there.

That's exactly what unitest currently do (but with a new name "unittest2") and as soon as Python 2.7 final will be out, unittest will have the same problem: it won't be able to backport new features anymore under the same namespace.

I do not see that changing the 'no new features in micro (bugfix) releases' policy would solve Ian's problem at all. Suppose unittest2 were sufficiently done in September to go into 3.2 as unittest. Suppose the policy were changed and unittest2 were also backported into (3.1.final) and 2.7.1. That still would not help Ian with respect to 2.6, 2.5, 2.4, and however far back he wants to support. Since whatever solution he uses for 2.6- should also work for 2.7 (and 3.1) what is the use? Perhaps the reason for the policy needs to be restated: if new features are introduced in every x.y.z release, then 'Python x.y' has no particular meaning. This *was* the case somewhat during Python 1 and early Python 2 days, when the choice between moving from x.y.z to either x.y.(z+1) and x.(y+1) was somewhat arbitrary. (The move from '1.6' to '2.0' rather than '1.7' was for legal reasons only. Python2 really began with 2.2) The precipitating event for the new policy was the introduction of bool() and prebound True and False late in 2.2 series. People who download the latest '2.2' release and who used the new feature found that their code would not run on the great majority of '2.2' installations. It caused enough problems for enough people that Guido decided that he should have waited for 2.3 for official introduction, and perhaps first released bool as a separate module for earlier use. People who propose to change the policy back (are part way back) to what it used to be should at least be aware that it is a reversion and that there are reasons, and not just arbitrary whim or accidental happenstance, for the change. Terry Jan Reedy

On 7 June 2010 20:20, Ian Bicking wrote:

distutils2 won't be in 2.7 at least, and any packaging system not available for Python 2 would be irrelevant.

I find it hard to interpret this in any way that doesn't pretty much imply "Python 3 is irrelevant". While I disagree with that, I suspect it isn't what you mean. So can you clarify? Paul.

On Mon, 7 Jun 2010 14:20:41 -0500 Ian Bicking wrote:

On Mon, Jun 7, 2010 at 1:52 PM, Antoine Pitrou wrote:

...

...

I say there is consensus because as far as I know anything substantial has a maintained version outside the standard library; argparse is implicitly, unittest is unittest2, ElementTree always has maintained a separate existence, simplejson implicitly.

"Anything substantial" is more than exagerated. The modules you are mentioning are exceptions, two of which may even be temporary (argparse and unittest2). Most sdtlib modules don't have external releases, and many of them are still "substantial".

Most other modules are very old.

Well, even if that's true (I haven't checked and I guess we wouldn't agree on the meaning of "old"), so what? I guess what I'm asking is: what is your line of reasoning? You started with a contention that: “There is no reason any new library or functionality should be tied to a Python release” and, in my humble opinion, you failed to demonstrate that. In particular, you haven't replied to my argument that it dramatically eases dependency management.

distutils2 won't be in 2.7 at least, and any packaging system not available for Python 2 would be irrelevant.

That's your opinion and I guess some people would disagree. Besides, decreeing that distutils2 be kept out of the stdlib won't make it its code magically compatible with Python 2.x. Regards Antoine.

On Mon, Jun 7, 2010 at 10:09 PM, Ian Bicking wrote: [..]

Also we have implicit dependency management and versioning already because a few libraries in the standard library have .egg-info files installed.

If you are talking about wsgiref, that's the only one that has the egg-info in the stdlib, and it will go away soonish (probably in 3.2) since: - this format will be replaced by the new dist-info (PEP 376). - this was a mistake imho to add it since wsgiref was not a distribution in that case

-- Ian Bicking  |  http://blog.ianbicking.org

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- Tarek Ziadé | http://ziade.org

On Mon, Jun 7, 2010 at 4:27 PM, Antoine Pitrou wrote:

...

I am not aware of anything added since 2.5 that isn't backported or previously available as a separate library (I'm sure there's *something*, just nothing I can think of).

Really, I'm too lazy to go and read the changelogs, but there definitely are many improvements that are not available in Python 2.5 and older.

...

There is no clear policy about how backports are managed.

That's because, contrary to what you seem to think, external backports are the exception and not the rule.

I offered examples, you were too lazy to read the changelogs, your sweeping declaration does not seem justified.

From What's New in Python 2.6:

contextlib: attached to a language feature. multiprocessing: backported new string formatting: though a method, it'd be easy to produce in a module form. I'm not aware of a backport. abstract base classes: I don't think this can be backported wouldn't all kinds of contortions ast: associated with the implementation/language json: backported (implicitly, as it is simplejson) plistlib: I'm guessing this was added to support packaging. It exists separately. What's New in Python 2.7: argparse: implicitly backported changes to logging: not sure what will happen with this; the module has been backported in the past memoryview: not aware of it being backported importlib: technically including it in 2.7 is a backport, but otherwise no ttk: appears to be backported (http://pypi.python.org/pypi/pyttk/) unittest: backported ElementTree: backported Digging deeper into 2.5: functools: apparently backported at one time, now defunct ctypes: the backport appears to be dead sqlite3: actively developed (different name?) wsgiref: backported hashlib: backported Every release there's some additions to collections, which have not been backported. So in summary, of 17 additions which seemed "backportable" to me (not counting 3 modules that seemed tied to language features): * 4 were not backported * 3 have defunct or ambiguous backports * 10 were backported -- Ian Bicking | http://blog.ianbicking.org

On 7 June 2010 22:09, Ian Bicking wrote:

On Mon, Jun 7, 2010 at 2:56 PM, Antoine Pitrou wrote:

...

On Mon, 7 Jun 2010 14:20:41 -0500 Ian Bicking wrote:

...

On Mon, Jun 7, 2010 at 1:52 PM, Antoine Pitrou wrote:

...

 > I say there is consensus because as far as I know anything substantial has a

...

maintained version outside the standard library; argparse is implicitly, unittest is unittest2, ElementTree always has maintained a separate existence, simplejson implicitly.

"Anything substantial" is more than exagerated. The modules you are mentioning are exceptions, two of which may even be temporary (argparse and unittest2). Most sdtlib modules don't have external releases, and many of them are still "substantial".

Most other modules are very old.

Well, even if that's true (I haven't checked and I guess we wouldn't agree on the meaning of "old"), so what? I guess what I'm asking is: what is your line of reasoning? You started with a contention that:

“There is no reason any new library or functionality should be tied to a Python release”

and, in my humble opinion, you failed to demonstrate that. In particular, you haven't replied to my argument that it dramatically eases dependency management.

It only eases dependency management in closed systems with a fixed Python version.  If you support more than one Python version than you still have dependency management, it is just tied to ad hoc workarounds when there are compatibility problems, or ignoring new functionality.

The importance of "old" vs. "new" modules is that people tend to have a lowest version of Python they support, as simply a hard stop.  This is currently Python 2.5 for most people, 2.4 for some groups (and just a very few stragglers with 2.3).  So long as you never use anything beyond what 2.5 provides then it's okay, which is most of the standard library.  I am not aware of anything added since 2.5 that isn't backported or previously available as a separate library (I'm sure there's *something*, just nothing I can think of).

There is no clear policy about how backports are managed.

Which gives me an idea: What if all the backports were managed within a single PyPI package, (e.g. backport26, backport27) with clear policy on older Python versions supported. then I could write in my py2.4 script: from backport26.os.path import relpath Konrad

On 9 June 2010 10:20, Konrad Delong wrote:

Which gives me an idea:

What if all the backports were managed within a single PyPI package, (e.g. backport26, backport27) with clear policy on older Python versions supported.

then I could write in my py2.4 script:

from backport26.os.path import relpath

It's not a bad idea. A key benefit seems to be that it can be done by anyone, whether or not they are a core developer. So it can be set up right now, without taking up any of the limited core-dev resource. Of course, conversely, the disadvantage is that nobody's done this already, implying that either nobody's thought of it before or there's actually little motivation for someone to put the work into such a solution :-) If nobody's thought of it, you may be lucky and someone will pick up the idea and run with it. Paul.

On Mon, Jun 7, 2010 at 8:50 PM, Ian Bicking wrote:

On Mon, Jun 7, 2010 at 1:42 PM, Brett Cannon wrote:

...

On Mon, Jun 7, 2010 at 09:35, Ian Bicking wrote: [SNIP]

...

Another alternative is to simply not make improvements to the standard library beyond a very well-defined set of appropriate functionality. This would be much closer to the status quo.  Defining what categories would be "appropriate" would be contentious, I am sure, but would sharply focus future discussions.

I personally would love to see this happen. Having a more clear focus for the stdlib would be a good thing in my opinion since as of right now it's just what the group thinks it is at that point.

Indeed, each person projects different ideas and motivations onto the standard library and I don't see a great deal of shared understanding about what it is.

There's one thing that is clear though: Distutils is in the standard library, and it would be a non sense not to have Distutils2 included to replace it. While I understand your motivations not to see Pip included in the standard library, I will strongly object the exclusion Distutils2 from the standard library. I have agreed to temporarily develop Distutils2 outside the stdlib with the sole condition that it will be included back as soon as it is ready, because we badly need such a system in a vanilla Python. Distutils2 is the place where we are implementing the PEP that where accepted lately, and its inclusion to the standard library will provide a working packaging system for Python (that is *batteries included*) and a blessed playground for third party packaging tools. Regards Tarek -- Tarek Ziadé | http://ziade.org