Re: Stability and change

On c.p.py, Alex Martelli wrote:
I'm moving the discussion to python-dev, since that's where I think it belongs. Personally, I hate the way Linux releases are numbered (I can never tell which one is stable and which one isn't). But I could get used to it if we used the micro version number to indicate stability -- in particular, 2.2 would be experimental, and 2.2.1 and following would be stable; 2.3 would be experimental, and 2.3.1 stable. This would require only a very slight change of policy: PEP 6 would have to be renamed from "Bug Fix Releases" to "Stable Releases". It wouldn't even have to loosen up its prohibitions; quoting: """ Prohibitions Patch releases are required to adhere to the following restrictions: 1. There must be zero syntax changes. All .pyc and .pyo files must work (no regeneration needed) with all patch releases forked off from a feature release. 2. There must be zero pickle changes. 3. There must be no incompatible C API changes. All extensions must continue to work without recompiling in all patch releases in the same fork as a feature release. Breaking any of these prohibitions requires a BDFL proclamation (and a prominent warning in the release notes). """ This doesn't prevent us to evolve the standard library. It doesn't prohibit adding new built-in functions, although I'd be reluctant to do so. It would, however, require a big change in how we present new releases to the world. Currently, Once 2.x is out, anything running 2.(x-1) is labeled obsolete, or at least oldfashioned. That needs to change! The website needs to present at least two options: - Bleeding edge (e.g. 2.3 alpha, or 2.3 once it's out) - Stable (e.g. 2.2.1) There may be more options: Zope 2 currently requires 2.1.2 and will require 2.1.3 as soon as it's out; other people still swear by 1.5.2. (I don't think that the 1.6 and 2.0 lines are still popular.) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Saturday 06 April 2002 05:48 pm, Guido van Rossum wrote:
Right.
Sounds quite workable, as long as we never want to do "bug-fix" point releases for experimental/bleeding-edge versions. The Linux folks do several experimental A.B (with B being odd, meaning experimental).N, but then Linux kernel/modules/drivers ARE an order of magnitude larger than Python, so we may not have analogous needs -- I dunno.
Yes, library changes should be OK, as long as they don't alter behavior of code which used to run (without specific BDFL proclamation and prominent notice). I personally wouldn't mind if some good-intention wording about not breaking existing doctests (e.g. no change in wording of error messages) snuck in, but then I'm a doctest junky.
Yes, public perception issues are to some extent paramount to ensure this slight variation has the intended effect. Your freedom to experiment as well as public perception might be enhanced by some explicit mention that the first (.1) stable release does not guarantee 100% compatibility with the immediately previous bleeding-edge/experimental release, so you can (within some common-sense bounds) try putting something in 2.X and then taking it away in 2.X.1 if it didn't work. I.e., stability being guaranteed 2.X.1 -> 2.X.2 and so on, but not necessarily 2.X -> 2.X.1, just as not necessarily 2.(X-1).N -> 2..X.1. Other folks more adept at marketing than me may come up with other ways to help public perception of 2.X.Y as "good, solid, stable, NOT old, dusty, junky" and thus helping this variant on "dual-track" fulfil its role. Alex

Or we could stay longer in beta.
I don't think so. Doctest is extremely picky, on purpose, and a breaking doctest doesn't mean that regular apps will break.
Good point.
Well, we can probably ride some of the common fear of X.0 releases. (Though we'd continuously have to explain that for Python, 2.X corresponds to X.0 in other vendors' numbering. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido]
How are "experimental" and "stable" defined? 2.2 was as stable as we could make it with 7(!) full pre-releases strung out over half a year. 2.2.1 has fewer bugs, but it would be extremely optimistic to believe more surprises aren't lurking in, e.g., the type/class dark corners. So is 2.2.1 "stable"? Whatever that means, intuitively I doubt it will be *as* stable as 2.1.2. So long as we don't have people testing Python full-time (i.e., quite possibly forever), Python history says relatively few people will bother to try a pre-release, so lots of bugs have no hope of getting caught before an i.j.0 release.
... Or we could stay longer in beta.
I don't think it would help much -- just a few days after initial release of an alpha or beta, downloads go way down. In practice, Python releases get a "street rep" that's not hard to pick up from c.l.py traffic. For example, several people independently recommend 2.1.2 as "the most stable" version of Python currently available, and I expect 2.2.1 will still be viewed as bleeding edge. Since there's no effective way to get wide testing of pre-releases (btw, I don't believe Linux shares this problem), there's no real way to judge a release's perceived stability until after it's been released. This makes a priori stability-number schemes "a problem" for us. Well, instead of calling 2.3 "2.3", we could call it 2.3.1. Then release 2.3.2, 2.3.3, ..., until consensus appears that 2.3.k is the most stable version of Python available. At that point we could re-release 2.3.k under the name 2.3 <0.9 wink>. microsoft-is-still-the-most-successful-software-vendor-in-the-solar- system-ly y'rs - tim

Alex Martelli <aleax@aleax.it> writes:
Is that actually a problem (or, *the* problem)? I.e. if somebody finds that 2.2 is "unstable", does it ever happen that, to solve a problem, something has to be taken back? The typical problems seem more to be in the area of bugs: a bug was fixed, and this fix broke existing code, or a feature was introduced, and that broke existing code. In either case, it is desirable to keep the existing fix, and the new feature, but to find a way to let the old application continue to work. The need to phase out new features that turn out to be a mistake is relatively rare, IMO; instead, there is a stronger desire to phase out old features that have been superceded by newer ones. Or are you just hoping that booleans will be removed in 2.3.1 again :-? Regards, Martin

All that matters is that you have a numbering system and follow it consistently. The python users will figure out what the numbers mean to them. And I guess that 2.2 is not consistent. The changes from 1.5.2 to 2.0 seem to be closer to what happend with 2.1 to 2.2. One measure of the oddness of 2.2 is the lack of 3rd party support. BArry

On Sat, Apr 06, 2002, Guido van Rossum wrote:
I'm kind of in agreement with Tim; I don't think this buys much. Seems more important to me to specify process than stability per se.
What I suggest is that instead of major.minor.bugfix we do major.minor.enhanced.bugfix This way, only major.minor releases can introduce actual language changes, but enhancement releases can pick up library changes. If this sounds at all workable, I recommend issuing a quick BDFL pronouncement changing 2.2.1 to 2.2.0.1. Then what we might try for a 2.2.1 release is making the new boolean type a module that people can start using. I am also a bit amused because an early draft of PEP 6 did try to address this issue, but you and Tim dissuaded me. I think both of you were right back then, but it's still kind of funny to see it picked up a year later. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

"A" == Aahz <aahz@pythoncraft.com> writes:
A> What I suggest is that instead of A> major.minor.bugfix A> we do A> major.minor.enhanced.bugfix A> This way, only major.minor releases can introduce actual A> language changes, but enhancement releases can pick up library A> changes. If this sounds at all workable, I recommend issuing a A> quick BDFL pronouncement changing 2.2.1 to 2.2.0.1. Ug, do we really need more dots? If we're really going to do that then maybe we should give up our biases about changing the first number.

On Sat, Apr 06, 2002, Barry A. Warsaw wrote:
Hmph. Why are you winking? Between new-style classes and the division change, that's actually a sensible suggestion. How about we change "2.3" to "3.1", making 3.0 a retroactive symlink to 2.2. (Okay, so I'm mostly joking here, but I really do think there'd be fewer complaints about 2.2 if it had been billed as a major change.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

[Aahz]
I'm kind of in agreement with Tim; I don't think this buys much. Seems more important to me to specify process than stability per se.
I think our process is impeccable. What's missing is clear guidelines for the users. Maybe Alex is right and the Linux odd/even distinction is clearer than using micro release numbers to indicate bugfix/stable releases.
Ah, yuck. I really don't want to start using more dots. Too confusing.
This way, only major.minor releases can introduce actual language changes, but enhancement releases can pick up library changes.
I don't see how this helps.
If this sounds at all workable, I recommend issuing a quick BDFL pronouncement changing 2.2.1 to 2.2.0.1.
No. :-)
Then what we might try for a 2.2.1 release is making the new boolean type a module that people can start using.
No again. (Banning the new bool type to a separate module is a waste of time.)
We still seem to be disagreeing on how to do it. :-( [Barry]
From my own perspective it seems that 2.1.x is viewed as the stable release family, and each micro release reaffirms its stability.
That's about right. Maybe we should continue to promote 2.1.x and relegate 2.2 to the bleeding edge? A simple rearrangement of the website might be sufficient.
Or maybe 2.3 should become 2.2.3. <0.5 wink> [Martin]
Usually the latter. When fixing a bug breaks code, it can mean two things: (a) the fix was to introduce a feature (sometimes the right thing to do); or (b) the bug was so pervasive that work-arounds became widespread, and the fix breaks the work-around. Both are relatively rare (but have happened). But new features regularly break existing code: either by introducing a new keyword (e.g. yield), or by changing results in certain situations (e.g. str(1==1), or nested scopes), or by taking away old crutches (e.g. import * inside a function). There are many more examples, despite the fact that in each of these cases we've tried to do our best to minimize breakage.
I would think that the best way to keep old code running is to keep the old interpreter around. Maybe one problem is that we've been installing each new version as "python" by default? Even on Windows that can be made to work, as long as you find a way to start your program with a specific Python version: you can't use #!/usr/local/bin/python2.2 and double clicking .py will invoke the most recently installed version. (Hm, would it be totally outrageous to let Python itself on Windows check for a #! line, and when there is one that names an existing Python interpreter that's not the current one, simply exec that one on top of the current one???)
IMO the most common breakage is when a new feature uses up a tiny portion of some namespace that used to belong to the application.
Or are you just hoping that booleans will be removed in 2.3.1 again :-?
Fat chance. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

I just thought of another issue. James Logajan (possibly our most outspoken conservative developer) brought up the problem that whenever you install a new version, you start developing for that version, and your code is no longer portable to previous versions. Even if we had a perfect track record of keeping old code working, the addition of new, enticing features has a "lock-in" effect. (We may sometimes have done this on purpose, at least semi-consciously.) This would suggest designating each series 2.X, 2.X.1, 2.X.2, 2.X.3, ... as an asymptotic series approaching perfection for a given feature set, where the feature set is pretty much frozen the moment 2.X is released. Then user code portability within the 2.X series is close to guaranteed, making it easy to move a script to a previous version, and if that doesn't work because the script relies on some bug being fixed, there shouldn't be much resistance to upgrading 2.X.Y to 2.X.(Y+1). We have to commit to maintaining a 2.X series as long as it is in use. I don't think I want to chop off the leading "2." -- since as we're releasing new versions every 6-12 months, this would move us to 9.0 way too quickly. It shouldn't be too hard for users to get used to the fact that the jump from 2.X to 2.(X+1) is usually a big one (despite comments that the jump from 2.1 to 2.2 was as big as the one from 1.5.2 to 2.0 -- after all, so was the jump from 1.5.1 to 1.5.2, so we're already doing better. :-) I'm not sure what exactly we need to *do* now. Be clearer to the user community that we're still maintaining 2.1.*? Delay the 2.3 release and spend more time on 2.2.2 (assuming that 2.2.1 is about to go out)? Change the Makefile and the RPMs to install each 2.X version under a different name, e.g. only install as python2.2?[*] Rearrange the website to emphasize the stable version? While I agree that 2.1.x is the most stable release, there's also a significant user population who want to use 2.2 (e.g. the new Boost library wrappers use new-style classes). We should maintain the 2.2.x line for their benefit. And then maybe we could take our time releasing 2.3 while experimenting with new features to our heart's content. To counteract the fact (as Tim noted) that few users bother to download betas, maybe we could release 2.3 relatively rough, clearly mark it as an experimental release, and work on improvements and stability in 2.3.1, 2.3.2 and so on, until we're happy to call it stable and start experimenting with 2.4. While we're working the kinks out of 2.3.x, the most stable release advertised would be 2.1.x, and adventurous users could basically choose between 2.2.x (advanced but stable) or 2.3.x (bleeding edge). Compared to the Linux numbering scheme, we'd lose the "mnemonic" value (if you can call it that) of even-stable / odd-experimental; instead, we'd have to give each release a color, maybe starting with red (bleeding edge) going via yellow (advanced but stable) to green (most stable), and finally to blue (behind the pack) and black (dead). ----- [*] Here's an idea. Maybe "make install" should check if there's already a $(prefix)/bin/python that's got a different major.minor version number than the one it's about to install, and then *ask* whether to make a link named "python" to the new version rather than just doing it (unless you specified "make altinstall", which probably too few people know about). --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sat, Apr 06, 2002, Guido van Rossum wrote:
First of all, I agree with the idea of better management of multiple Python versions. But before we can start being clearer to the community, we need to be clearer to ourselves. Historically, there has been an attitude of, um, antipathetic neutrality toward the idea of maintaining old versions of Python. Until today, I was under the impression that your official attitude (as usually channeled by Tim) was that creating patch releases was something that PythonLabs would assist volunteers to accomplish, but would not push for in any way. It sounds to me like your attitude has in fact changed. If so, then I think the first step is to say so explicitly. After that, I'd like a clear statement of the new goalset for Python development. Once we have that, I think it should be relatively straightforward to create a process to support the new goals. Perhaps the whole PythonLabs crew should have an in-person brainstorming session to kickstart the goalmaking process? What's Zope doing to handle this issue? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

Perhaps the whole PythonLabs crew should have an in-person brainstorming session to kickstart the goalmaking process?
Why should this be PythonLabs' internal brainstorm? We may be the only ones who get paid directly to maintain Python (part time, the rest is Zope work!), but we're not its only maintainers. This concerns us (active developers) all, because we're all limited resources.
What's Zope doing to handle this issue?
Assuming you mean "how is change in the Zope software handled" (rather than "how does Zope Corporation handle change in Python"), they have a rather different approach. There's an effort to create a Zope 3 which is a total rewrite, (almost) completely from scratch. There's an explicit compatibility requirement for the end result, but that's only needed in the final phase -- while it's being created (with lots of help from the Zope community) it can do whatever it wants. In the mean time, various previous Zope releases are still being maintained: I believe that both 2.4 and 2.5 are under active maintenance, meaning that micro releases are occasionally issued, and there's an effort underway to create a 2.6 release (again with community help). While there's a striving for easy upgrading from 2.x to 2.(x+1), each new one (2.(x+1)) adds significant new features; I believe strict compatibility between 2.x and 2.(x+1) is not a goal (except at the storage level -- the database contents are sacred). I believe that it's very common for 3rd party products written against 2.x to require significant porting effort to work with 2.(x+1). The www.zope.org site for example runs a very old version of Zope (I believe 2.3) because it's encrusted with much custom code that make an upgrade difficult. Instead, there's a completely new version in the works (new.zope.org). --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido]
I just thought of another issue. James Logajan (possibly our most outspoken conservative developer)
Ex-developer: he's made it quite clear that he no longer intends to use Python at all (after some brief current period of "compatibilizing" his old code).
I must be exceptional -- this wasn't news to me <wink>.
I've done it with full awareness in my own code.
If this differs from what we're doing today, I don't understand how: "no incompatibilities and new no features!" has been strongly observed in the bugfix releases.
Again this *appears* to be arguing for what we already do. Maybe with one difference: I assure people on c.l.py that PLabs will continue distributing I.J.K for increasing K for as long as the community contributes the work for bugfix releases. Perhaps you're saying that PLabs should take that over? Note too that it wouldn't make James Logajan one whit happier, if we're to take him at his word. He developed some scripts for 1.5.2, and that's it. He wants them to work forever under all future Python releases and all platforms, and got particularly burned by Unicode's interaction with how Tk deals with UTF-8 (and a lot of people got burned by that).
Are we? We've made no effort to backport any bugfixes into 2.1.3, and to the contrary declared that we wouldn't do anything with 2.1.3 except fix the single gc-vs-trashcan disaster. We're not even building a Windows installer for 2.1.3 (last I heard). I don't think anyone is going to buy that as "maintaining" the 2.1.x series; I sure don't.
Delay the 2.3 release and spend more time on 2.2.2 (assuming that 2.2.1 is about to go out)?
2.2 did go out the door unfinished <wink>. But, again, we could have delayed it two years and it still would have been relatively unstable: there's just no substitute for getting thousands of users *using* a release.
You can fiddle what you *call* what we're already doing anything you want, but if a long-lived two-branch comes out of this, I quit <wink>. I do believe we should direct more effort into bugfix releases; that can't be done without taking away from something else, though (that's a trivial deduction: everyone is already overloaded).

No, this is for all active developers. I agree that we don't have to change what we do very much, but I'm looking for a way to "spin" it to satisfy the more conservative user community (the ones who still like Python but are worried about the future). Since there are different "use cases", this may be hard or impossible, or we may have to explain it differently for different folks. For example, developers who distribute useful Python programs want to have a way to ensure that code they write gets run on a compatible Python version. We could make it easier to have different X.Y releases installed on the same system; this is already pretty easy on Unix if you know about "make altinstall". If we invent a standard way to specify the required version (or range of versions?) then that might be sufficient. You shot down my #! proposal for Windows, but maybe we can do something else. (Obviously this only works for programs, not for modules or packages. Package authors pretty much know the drill.) Beginning programmers have a different concern: they don't know much about software evolution, Python may be the first or second language they learn, and they hear that there's a concern about the rate of change. How can they know what that means? I've seen plenty of posts from such people worrying that in a few years from now they might have to learn a whole new language. They needn't worry, but it needs to be addressed with more explanation than "don't worry, the BDFL is watching over you."
Maybe we need to use a different word? Barry noted that 2.1.x is the most popular "stable" release and will probably remain that. So maybe we should say this is the "recommended" release. And then maybe we need to revise the decision not to make a Windows installer. Or maybe we *do* need to spend more effort backporting patches (not necessarily PLabs, but *somebody*). Another way to keep adding value to older releases is by backporting the build procedure enhancements. Once we've figured out how to build Python 2.3 on, say, some ancient version of AIX, maybe somebody should port that back to 2.2 and 2.1. That's not a language change, but still improves the usability of the release. (For some folks. Maybe the AIX user community is small enough that we shouldn't care, but there are other platforms with similar issues. I recall that you backported the new 32 bit Windows installer to 2.1.2; maybe the recent fixes could be backported to 2.1.3.) OK, so I *am* proposing to do something different. Maybe it's not very attractive to work on old releases. But Zope Corp does it all the time (and the fact that Zope 2.x requires Python 2.1.x may be an additional motivator).
Indeed.
Would you settle for a separate CVS repository? :-)
Yup. That's why we need to put some effort in communication to the users, and some in a real change of priorities. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Tim]
Perhaps you're saying that PLabs should take that over?
[Guido]
No, this is for all active developers.
How does that differ from PLabs? There are only a few active developers outside of PLabs who aren't "one issue" developers, and I wouldn't consider even *asking* Martin or Michael to take on more than they already do. If a new plan involves docs, it necessarily sucks Fred in; if a new plan involves the PLabs Windows installer, it necessarily sucks me in; and new plan or old, you're hopelessly sucked in <wink>. The sum of all active developers isn't enough to tackle anything truly new.
If they write a Python script for 1.5.2, and it doesn't work in Python 58.7.9, they're unhappy. I'm thinking of Logajan, Cave and Rubin there, as they're complaining the loudest these days. They can't be spun into accepting that any backward incompatibility is defensible. I assume almost all of that would go away if incompatible language changes also changed the extensions used for Python files (like, say, .py3 for Python 3).
Since there are different "use cases", this may be hard or impossible, or we may have to explain it differently for different folks.
A possibility I haven't rejected is give up spin and just acknowledge that Python can't be all things to all people here: some people are going to lose. Unfortunate, but that's life if so.
That's not what the people above are saying, though: they want all future Pythons to be compatible with everything they wrote 4 years ago. If a reasonable <wink> developer appeared who asked for what you're suggesting, I'd like to try to help them.
I have at least 4 versions of Python installed on each of my Windows boxes, but there are limits to how deep we can push that (e.g., a given user can associate .py with at most one interpreter, and before Win2K that was at most one association per machine; *I* don't care, because I never use any Windows gimmick that looks at the .py association; if I did care, I'd be screwed, and because of how Windows works, not because of how Python works).
I didn't see that it addressed a real problem.
There are lots of ways, but I don't have practical experience with any of them as I'm not change-aversive. Possibilities include specifying minimal version numbers required of the language and/or of specific modules at import sites, relying on a strict backward compatibility guarantee to ensure that any later version will also work OK; and keeping all old versions of all modules around in all future releases, viewing specified version numbers as upper bounds instead of lower bounds. Another possibility is to work in terms of versioned interfaces, and rather than import a module directly, ask the system for any module object that's compatible with a specified version of a specified interface. ...
I don't think anyone is going to buy that as "maintaining" the 2.1.x series; I sure don't.
Who? Anthony Baxter withdrew from the 2.1.x line after 2.1.2 -- he was tired of doing it. He popped up again for 2.1.3 under the assurance that it was going to be a trivial effort. I have a real problem when a group of people demands something but won't volunteer anything to make it happen except their complaints; if they were paying us, fine, but they're not offering funding either.
Another way to keep adding value to older releases is by backporting the build procedure enhancements.
Isn't this dangerous? Build config seems very delicate to me.
Once we've figured out how to build Python 2.3 on, say, some ancient version of AIX, maybe somebody should port that back to 2.2 and 2.1.
If they can swear it's not going to break some other Unix mutant, maybe.
I was actually reluctant to check those into 2.2.1: the complaint came from an XP user, and I don't have an XP box to test it on. I believe I fixed a *similar* complaint on my Win2K box, but who knows whether that fixes it for XP too? I don't; I can only guess. Ditto for its effects on ME -- I'm using installer settings I never tried before, and I can't know whether there may be a glitch on ME. It seems to work OK on Win98SE too, so I took a calculated risk and checked it in to 2.2.1. But I won't be shocked if it breaks Windows installation on some version of Windows I don't have.
If you haven't noticed yet <wink>, you don't control a fraction of the resource a project of Python's complexity requires. It's harder to maintain old Python releases than it is to maintain old Zope releases, because Python is much closer to the metal: testing on Linux doesn't say much about what will happen on HP-UX or Solaris; testing on Windows doesn't say much about what will happen on Macs; and so on. Testing a x-platform product requires a lot of warm bodies and a lot of time. Without more volunteers, I don't see it happening. That said, I'll be happy to build a Windows installer for 2.1.3. It is, quite literally, the least I could do <wink>.

GvR> Maybe we need to use a different word? Barry noted that GvR> 2.1.x is the most popular "stable" release and will probably GvR> remain that. So maybe we should say this is the GvR> "recommended" release. And then maybe we need to revise the GvR> decision not to make a Windows installer. Or maybe we *do* GvR> need to spend more effort backporting patches (not GvR> necessarily PLabs, but *somebody*). I think those are all good ideas that help promote the idea of a long-lived stable baseline release. GvR> OK, so I *am* proposing to do something different. Maybe GvR> it's not very attractive to work on old releases. But Zope GvR> Corp does it all the time (and the fact that Zope 2.x GvR> requires Python 2.1.x may be an additional motivator). Note that Jim often brings up his opinion that the /only/ way to guarantee Zope Z.Y.Z works on Python X.Y.Z is to distribute Python with Zope. (This harkens back to JWZ's much earlier opinion that Netscape could never ship using dynamic libraries, because you really can't trust anything about the end user's system). We're really touching on much deeper flaws in software development, with no good answers, IMO. we-fear-change-ly y'rs, -Barry

[Barry A. Warsaw]
And note that we ship Microsoft's C libraries with Python on Windows, to insulate users from x-release incompatibilities in MS's C libraries. Whether it actually helps Python's users varies across Windows flavors, and also across vagaries of the user's installation and configuration settings. But I think it's important to note that in recent MS OSes, they're trying to make it *easier* for apps to force use of the specific DLLs they ship with (although our Python installer isn't yet doing any of the things necessary to exploit these new capabilities). Don't think Linux is immune either. Upgrading gcc or glibc can "suddenly break" programs that used to work fine. God has a sense of humor about this, though: He usually arranges that breakage show up first in obscure libm endcases <wink>.
Indeed so! Python's not the only thing that changes in incompatible ways over time: *everything* does, including the compilers, system libraries, system tools, and OS policies. The first textbook I had on mathematical logic was Stephen Kleene's (yes, that Kleene) cheerfully named "Mathematical Logic". The first chapter contained a warning to the student: It will be very important as we proceed to keep in mind this distinction between the logic we are studying (the object logic) and our use of logic in studying it (the observer's logic). To any student who is not ready to do so, we suggest that he close the book now, and pick some other subject instead, such as acrostics or beekeeping. For this reason, I suggest naming the hypothetical "forward and backward compatible forever more" Python branch the "Python Acrostics and Beekeeping" branch. you-can't-spell-pablum-without-pab-ly y'rs - tim

"TP" == Tim Peters <tim.one@comcast.net> writes:
TP> Don't think Linux is immune either. Upgrading gcc or glibc TP> can "suddenly break" programs that used to work fine. God has TP> a sense of humor about this, though: He usually arranges that TP> breakage show up first in obscure libm endcases <wink>. Or upgrade a supposedly backwards compatible library, like, oh I dunno, libz? >> ... We're really touching on much deeper flaws in software >> development, with no good answers, IMO. TP> Indeed so! Python's not the only thing that changes in TP> incompatible ways over time: *everything* does, including the TP> compilers, system libraries, system tools, and OS policies. Heck, I wash my pillow and I can't sleep right for weeks. TP> For this reason, I suggest naming the hypothetical "forward TP> and backward compatible forever more" Python branch the TP> "Python Acrostics and Beekeeping" branch. TP> you-can't-spell-pablum-without-pab-ly y'rs - tim Or how about the "Garth Releases" (named after character in Wayne's World who utters the immortal truth "We fear change"). -Barry

But I don't think that's what we do any more. We just tell them to use 2.1.2.
I've started to call it the Logajan paradox: he wants his cake (zero backwards incompatibilities) and eat it too (releases with new features). It can't be done. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido]
I don't believe I ever heard James ask for a new feature, although he has allowed that new features might be OK provided that the Python version introducing them continues to run all older Python scripts without change. Rubin's view is more complicated, and I'm not sure it's been coherently expressed. Note that Logajan's view is hard to distinguish from what most ANSI/ISO language standard committees accept as their burden most of the time: no breakage of programs that conformed under the last standard, period. That's why, e.g., the C99 bool type was added via a pile of macro tricks, and *requires* #include'ing a new header file in any program that wants to use it as intended (the *actual* name of the new type is "_Bool", which is of a syntactic form C89 reserved to the language implementation; indeed, all the symbols Python uses internally of the form _Pxyz are forbidden to us by the std (Python's implementation is non-conforming all over the place); C99 couldn't add "bool" as a new keyword, but they could add _Bool, and then "#define bool _Bool" in a new standard header file; that can't break any existing code, because no existing code could possibly have #include'd the previously non-existent <stdbool.h>; note too that older header files (like <stdio.h>) are forbidden from #include'ing stdbool.h on their own -- or, if they do, they have to hide all evidence of that from users). The ANSI/ISO committees pour person-decades into dreaming up ways to break nothing. It's not a crazy view on its own, but it's extremely expensive to do things that way (I've been on one ANSI committee, and worked closely with two others in previous lives -- it's excruciatingly slow going, and costs participants millions of dollars before it's over). It also requires extreme care in defining what "a conforming program" is, exactly.

There seem to be two groups: 1) Early adopters: release early, release often; more features the better 2) The masses: easy does it, don't break anything, don't release so often I believe we can satisfy both groups--not perfectly, but good enough. This is partly summary and partly suggestion for terminology and schedule. I propose doing: Release name/type Version Schedule ----------------- ------- -------- Major releases 2.4 1.5 - 3 years Bugfix releases 2.2.1 ~ 3 months or as needed Development releases 2.3a0 ~ 3 months Don't get hung up on the version numbers, it doesn't matter. If we agree on the concept, we can name it later. 'Major' releases (roughly corresponding to Linux kernel even releases) would occur every ~ 18-36 months. These releases would be full executable, doc, etc. This seems to be the crux of what many people want. They want a vibrant changing language. But they don't want to have to deal with that change. They want longer cycles. We are talking about users of the language, not hard-core developers. These releases would still go through the alpha, beta, gamma releases. The last development release in a cycle would become the first alpha. Bugfix or point releases (eg, 2.x.y) would be made as needed, ~ 3 months. These releases would still go through the alpha, beta, gamma releases. This release would be concurrent with the development release. Since 2.2 is out there, I would suggest putting the bug fix effort only into it. There doesn't seem to be enough people to share the load for keeping 2.1 in good shape. Development releases, which are source only, could be released approximately every 3 months. These are basically alpha releases. They roughly correspond to Linux kernel odd releases. This would be to satisfy those that want new features or to test compatibility and are willing to build from source. These should be able to be released with very little effort. The benefits of this scheme is that I think it will appeal to a large majority of people regardless of whether they are an early-adopters or not. The only real drawback is that it will take more time to keep up the previous released version. However, I think this can be minimized by having everyone try to backport their work if it's a bug fix and have several people handle the patches for bug fix releases. I would volunteer to help Michael (or whoever rolls 2.2.2), although I don't have enough time to be the One. I don't think the numbering/labeling scheme is important. Does it really matter if we call the next release 2.4, 3.0, 2.3-RELEASE or whatever. I personally think the Linux versioning is the most well known and a bit more normal for most people over BSD. Neal

On Tuesday 09 April 2002 03:45, Neal Norwitz wrote:
I think this identifies the issues very precisely and proposes a good solution strategy.
I fully agree on your concepts. I would suggest some renaming, for clearer/more incisive communication, but you're right, let's not get hang up on that: the substance of your proposal is _excellent_.
Perfect.
Not sure about the source-only part -- maybe there's some hacker wannabe on some Windows box out there that can't afford to buy VC++6. But I could volunteer to remedy that by compiling and making the win32 binaries available (not an installer: I wouldn't know how to write that, so this would have to be arranged) -- so it's not a killer.
OK. I opine there's a little bit of importance in such "naming", but not all that much -- the substance matters more, and I love the substance of your proposal. Alex

Guido van Rossum wrote:
Maybe Alex is right and the Linux odd/even distinction is clearer than using micro release numbers to indicate bugfix/stable releases.
That has not been my impression. I have seen quite a few people confused by the Linux odd/even system. I think the current major.minor.micro system works just fine. I think the discussion should focus on whether more effort should be spent on micro releases. Perhaps 2.2.1 should have been finished before work was started on 2.3. Personally, I don't think so. Neil

[Guido]
No, but how to name an existing interpreter is a problem: people don't install Python in the same locations, so an absolute path wouldn't be portable. At least for the PythonLabs and ActiveState distros, we could use the version number to look up the installation path in the Windows registry (the PLabs and AS installers create registry entries to record this info, under a key path with major.minor in its name, although the PLabs distro has never *used* this info; I believe the PythonWare distro doesn't set any info in the registry).

Absolute paths aren't portable on Unix either, although #! /usr/bin/env python2.2 comes close. I wouldn't mind if users who need this feature would be required to know where their interpreter lives. But I wouldn't mind a registry-based solution either: #! python2.2 could mean "find Python 2.2 in the registry". I believe the PythonWare distro always installs in the same place, so if Fredrik thinks this will be useful, we could provide a hack that looks in certain places on the filesystem if no registry key is found. Still, I'd prefer to do the simplest thing that could possibly work, which IMO would be just an absolute pathname. It makes it *possible* for a specific script to request a specific interpreter. If you want fancy, you can create an installer that figures out the right path and sticks it in the #! line of scripts that need it at install time. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido]
I don't believe any user needs this feature! Not on their own box. Let's be clear about "the problem" (or at least about Logajan's): you want to distribute your Python scripts all over the world. You're targeting Windows, and Unix flavors, and Macs (you hope). Freezing an interpreter into your distro isn't an option, as you have no idea how to even go about finding out how to do that on all the platforms your users run on, and you don't have access to all of them anyway. What then can you do to ensure that your scripts won't break on your *users'* boxes when they happen to upgrade their own Python installations? The only way I see that a "#!" line could do a lick of good for you then is if it were a x-platform way to identify version number, and that's all. It would also need to abort with "This app requires Python I.J" if it couldn't find Python I.J on the box. As a matter of fact, it wouldn't do Logajan any good even if Python 2.3 did a perfect job of that, as he only cares about 1.5.2. Going forward, maybe it would do someone some good. I'm skeptical of the cost/benefit ratio, though (James is extreme, IMO).
If it didn't also mean "find Python 2.2" on Linux and Macs, it wouldn't do anything to help the people who are unhappy in James's way.
It can't possibly work, except to solve a problem I haven't heard anyone bring up. I expect the simplest thing that could possibly work for James's problem would be to add a pragma "#requires 2.3", and teach Python to complain if its own version doesn't match what the pragma says. It would also need some way to do this in .pyc and .pyo files. But that's not really want James wants either: he wants his scripts to run forever without change, under all Pythons that will ever be released, and on all platforms.

On Sunday 07 April 2002 03:18, Guido van Rossum wrote: ...
I agree with Guido. I think the issue is first one of perception by users and potential users of Python. I evangelize for Python on a regular basis, and I notice the issue of _perception_ of stability and change is a significant block for a certain important category of prospective users. Basically, users like my previous employer, think3, inc -- "previous" because, despite it being a great job on many other scores, I grew frustrated enough with being unable to evangelize Python there, that I left in February in order to be able to do Python full time. I had no problem firing up the enthusiasm of programmers and system administrators for using Python in scripting, testing and throw-away tasks -- Python "sells itself" in that way. But management wants to see a strategic picture, one with more perspective -- and it wants to see one of change and evolution within a framework of stability, before it starts OK'ing a given platform for programs with hundreds or thousands of function-points. We had a great opportunity when Microsoft announced the switch of Visual Basic from VB6 to VB.NET, with lots of enhancements that would break just about every existing program. The break with stability was so large, that even managers with a strong pro-MS bias were suddenly quite willing to consider alternatives. I think that if, at that point, I had been able to point at some implementation that was both stable _and_ evolving, I might have gotten the OK to do a pilot project. "All scripts written 2 years ago still run fine, see, but at the same time there is active ongoing maintenance -- bug fixes, new internal speed-ups, new stuff in the library, tools, etc, etc". I couldn't, and I couldn't even indicate such a perspective for the future -- each 2.N release would keep breaking some (little, but some) code of 2.(N-1), while 2.N.M would be constrained to absolutely no new functionality vs 2.N.(M-1), bug-fixes only, assuming there were any such 2.N.M. I'm not sure what they are considering now (besides C++, which dominates there but IS seen as hard and unproductive enough that the search for higher-productivity languages continues) -- maybe Java (that got a horrid reputation there due to its instability in '95/'96, but enough years have passed that they might look at it again now). "Keep the old interpreter around" is not the solution such prospective users want to hear. The choice between being locked off from _all_ enhancements OR having existing code break is what they have with Visual Basic, say -- exactly the unpleasant choice that makes them willing to look around for alternatives. "If you want the bright, new email package, or to say 'for key in somedict:', you also have to accept X and Y risks of existing code breakage -- oh and if you don't so accept it, we'd have to do our own port to the Sharp Zaurus because the existing one only works for 2.2, and ...", etc, etc. To some extent this reminds me of another similar frustration of mine, which was longstanding enough that I had more or less gotten used to it. We used to be a multi-platform shop, then switched to releasing our (mechanical CAD) apps only for Windows OS's, as the wide market started demanding the ability to use cheap, newly-powerful PCs instead of costly proprietary workstations. I kept proposing we do a Linux release as well, but each time the perception of "the platform changing under us" (even for the allegedly-stable 2.0.* series etc) was a big factori in blocking it. More recently I had had better luck with OpenBSD -- far stabler than Windows AND actively maintained -- to the point of doing some minor pilot projects for it. But in this case the problem was that our prospective user base had never heard of it (important when choosing an OS to target, though probably not a language for development) and would not be well placed to install and maintain it (probably true, it does require more skill than for Windows or one of the new, friendly Linux releases such as Mandrake or Suse). Still, the "stability _and_ change" paradigm did once again prove quite attractive to management -- they were at least willing to give OpenBSD a try, as they had not been for Linux.
I'm not sure it's clearer. Clarity may not even matter all that much, as long as the message is heard as "there are two parallel lines of development: one along stable lines, that will break no previously correct code; one experimental, blazing new trails, for innovation and higher-risk changes" or something like that. What I could present today with integrity is more like "All core developers' interest focuses on the mildly-experimental track, constrained but only up to a point; oh yes, there _are_ sometimes bug fix releases to old and stabler versions, but absolutely no new stuff there -- all new things, even those which would break no previously correct code, always go into the mildly-experimental track". That's not the message the prospective users want to hear, I think. Not those I have in mind -- substantial software development shops considering whether to go ahead with Python for very substantial projects or to relegate it to testing/scripting/throw-away stuff but never functionality to be delivered to the customer.
It would help to some degree if the "stable" releases could be shown to have SOME kind of enhancement. Not as much as if language changes which did not break existing correct code could also be there, but some. Modules such as email are a big part of the draw of 2.2, for example (not quite as big as language-level enhancements, but big).
It would let people "return Boolean.True" where they want to do so in new parts of their programs while at the same time not worrying about str(x>y) changing under their feet. I.e., it would provide, or be perceived to provide, change within a framework of stability. Thus it would help us send the message I think prospective users want to hear -- that Python stands for stability AND change, that there is not necessarily a contradiction between the two goals. In a very minor way, to be sure.
More relevantly, I'm not sure I had managed to explain exactly why I think "stability and change" IS important to Python adoption in a certain category of software shops. What numbering or naming scheme we adopt has importance in determining what we communicate, but before that, deciding what we DO want to communicate is probably more important.
I think such a "retroactive" rebranding of 2.2 as "bleeding-edge" would be a serious blow to those of us who have accepted it as stable enough as our base over the last 3 months -- in books, magazine articles, advocacy within our firms and outside of them, etc. For example, AB Strakt has decided 2.2 was stable enough for us (and it was being presented as such) -- our new code now uses "for x in somedict:", "if x in somedict:", etc, for example -- now hearing that it wasn't, and 2.2.X micro-releases might introduce backwards-incompatible changes, would be a real problem indeed.
It would take away my ability to check if old code still works with a new release in a simple way -- by explicitly naming the new interpreter on the command line, whatever is in the shebang line in the script itself.
Indeed -- they won't go. So, stabilizing on 2.2.* (maybe with Booleans in a module, but surely no breakage of e.g. str(x>y)) might be one possibility, if the 2.2.* line was "stable and changing" - introducing enhancements but not ones that break previously correct code. I perceive Booleans as small (albeit negatively), but both our CTO and principal investor have vehemently posted against them, so it would be hard to 'sell' them even if I were quite favorable. Still, if that was the one hurdle to jump to get to a "stable and changing" release, who knows. But if jumping that small hurdle only gets to more of the same, either unstability or feature-frozen versions, the carrot may be too small. Alex

Alex, please try brevity. :-)
Now there's a good proposal. But it costs time and effort that takes away from our main mission (and PythonLabs is already under a lot of pressure). I wonder if there's a commercial market for this? Maybe ActiveState could back-port important packages or language changes and insert them into ActivePython? [proposal to semi-backport bool snipped]
Given the requirements you give elsewhere in your message, I'm not sure how to satisfy all these constraints. Let's be concrete. Should we backport stuff to 2.1 (the earliest release I'm willing to consider)? If so, what? Look at the "What's New?" section in http://www.python.org/2.2/. The new Windows installer has already been backported to 2.1. Type-class unification is obviously out. So are interators and generators (too much effort). So is // division (I think). Returning longs instead of raising OverflowError on int operations would probably break too much code, so it's better not to backport. That leaves the email package and xmlrpclib (both available as separate distros!), and large file support -- which I believe *did* get ported to 2.1.2.
Unfortunately, it's not clear what adaptation in that (apparently very change-resistant) category buys *Python*. IOW why should we, mostly volunteers working for the community, care?
Indeed. But I'd love it if all we need to do is change our communication patterns, because that takes much less effort than fully maintaining two separate release tracks.
I'm happy to hear that 2.2 is stable for you -- exactly as I intended and expected. But what confuses me is that while you're happy with 2.2's stability, you don't want to testify of this fact to the more conservative shops. Rather than telling them "honestly, I cannot recommend 2.2", why don't you tell them "I've used 2.2 for three months and it's as solid as a rock"?
If I didn't suspect that you, the CTI and the P.I. together are 75% of the company, I would say that they are micromanaging you. :-)
I suspect that the people who complain that Python's rate of change is too fast are misguided. There's a lot of fuss on the newsgroup about potentially breaking code and undesired changes, and before you know it, everybody just *believes* that Python is unstable and changes too fast. But if you try it, you find very little incompatibility, and what's there is easily dealt with. The people on the newsgroup who complain that the introduction of bool is such a major change that it would only be acceptable in a major release (i.e. 3.0) vastly overestimate its impact, which is minuscule (zero in most cases). Logajan's main gripe (and also McMillan's biggest effort in keeping his codebase cross-version portable) was about things that changed between 1.5.2 and 2.0 -- that *was* a major release. And at the time we didn't have PEPs 4-6, there was no warning module, and so on. So maybe there's only (or mostly) a problem of *perception*? --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido van Rossum]
I use Python on a few systems and flavours, and Python versions are not the same on all machines. At places, people ask me to limit myself to 1.5.2. At home, I have 2.2, but 2.0 is currently the common denominator for the set of machines on which I do my own projects (I physically travel between sites). For one, I have no real problem aiming 1.5.2 or 2.0 instead of 2.2, when projects or teams require it. Despite I much enjoy most of the new features, I still have to be careful about various development contexts. I would presume that for a newcomer to Python having to ping and pong between different Python versions, not firmly aware of the differences, might feel irritated by things working here, but not there. This is just an hypothesis, as I did not experience such an irritation myself: because when one knows which version has which features, there is no real problem. I guess the difficulty comes from the time evolution takes for new versions of Python to get installed universally, so new features are just there, dependably, blindly, everywhere. This means a few years at least. The difficulty does _not_ come from forward incompatibilities (there were a few around sockets, say, but these were simple things, not really worth complaining about in my opinion), in my experience, forward migration has been either very easy or just fully transparent. The current pace of change does not look unreasonable to me. I wonder if the perception would not improve, if there was a bit more documentation or publicity about what users should do about version inter-operability. If users understand what they choose, they will less likely feel changes as imposed to them. Maybe! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard

On Sunday 07 April 2002 09:38 pm, François Pinard wrote:
I also use 2.0 as the lowest common denominator. Its hard to avoid 1.5.2 (because Red Hat has been shipping with 1.5.x). I can't complain about the differences in Python versions. For the most part my experience has been hassle free. PHP version issues have been more troublesome to me than Python version issues.
There are some who think preserving familiarity between versions is way more important than incremental improvement. In my opinion, sticking to that very idea is a good way to kill and stagnate a language. I believe that maintaing and improving the language *usually* is more important than maintaining consistency between versions. That being said, going overboard with changes isn't good either. I think documenting these changes would be a step in the right direction. My two cents. Good day, ~Mark

[Mark J. Nenadov]
I also use 2.0 as the lowest common denominator.
Linguistic problem :-). Should we say "greatest" instead of "lowest"? Granted that the greatest common denominator is not "greatest" in any other way, but it is lower or equal than any of the things we consider. The real "lowest" common denominator might be very close to nothing, might it not?
Exactly. The changes _are_ documented in detail, indeed, but in special documents which only serious users read when they about to migrate from one version to another. I'm thinking about users who use the language occasionally or even regularly, but not fanatic about following everything about versions -- they mainly rely on the Python Library Reference, or even the Python Language Reference. These references describe some Python, but not necessarily the Python which happens to run on a given machine, and I guess (without having really experienced this myself) it might be frustrating to read and study, for discovering soon after that the feature is unavailable in this version. Or even, for someone, to have handy information about how to write for the common denominator, without having to compare many printings of the references at various Python levels. I guess that notes or footnotes, about Python levels in which described features have been implemented, might help users having to cope with release lags between Linux releases. Such lags are unavoidable whenever Python evolves. I understand the effort it would require to add and maintain such notes in the references (especially for the Library Reference which has myriad of details, the Language Reference is more clearly cut), my hope is that this might remove some of the irritation people seem to show. -- François Pinard http://www.iro.umontreal.ca/~pinard

"Martin v. Loewis" wrote:
GCD = greatest common divisor LCD = lowest common denominator http://www.dictionary.com/cgi-bin/dict.pl?term=gcd&r=67 http://www.dictionary.com/cgi-bin/dict.pl?term=lcd&r=67 Paul Prescod

From: "Paul Prescod" <paul@prescod.net>
See your own link. nitpicking-ly yr's dave

And you believed him? What bullshit. --Guido van Rossum (home page: http://www.python.org/~guido/)

david wrote:
did you check the link? the first entry only expands the abbreviation, the third entry provides the full explanation: least common denominator n. Abbr. lcd The least common multiple of the denominators of a set of fractions: The least common denominator of 1/3 and 1/4 is 12. Also called lowest common denominator.
nitpicking-ly yr's
if you're gonna nitpick, you could at least do it right ;-) </F>

[Martin v. Loewis]
pinard@iro.umontreal.ca (François Pinard) writes:
So, besides arithmetic, what would be the correct expression to replace: "I also use 2.0 as the lowest common denominator." ? P.S. - It's amusing how fast off-topic matters slide even more off topic! :-) The real thread is about how documentation could be amended to (maybe) help many people at being comfortable with Python, despite the rate of change. It is probably worth staying concentrated on the real thread... -- François Pinard http://www.iro.umontreal.ca/~pinard

So, besides arithmetic, what would be the correct expression to replace: "I also use 2.0 as the lowest common denominator." ?
If you want a more accurate mathematical analogy, you should be saying "greatest common factor". Better yet, avoid the cliche altogether and think of a more direct way of saying what you mean. e.g. "I use 2.0 because it supports the largest common feature set." Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

[Greg Ewing]
"I use 2.0 because it supports the largest common feature set."
Nicely put! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard

----- Original Message ----- From: "Martin v. Loewis" <martin@v.loewis.de> To: "François Pinard" <pinard@iro.umontreal.ca> Cc: "Mark J. Nenadov" <mark@freelance-developer.com>; <python-dev@python.org> Sent: Monday, May 27, 2002 2:21 AM Subject: Re: [Python-Dev] Re: Stability and change pinard@iro.umontreal.ca (François Pinard) writes: > > I also use 2.0 as the lowest common denominator. > > Linguistic problem :-). Should we say "greatest" instead of "lowest"? > Granted that the greatest common denominator is not "greatest" in any other > way, but it is lower or equal than any of the things we consider. The real > "lowest" common denominator might be very close to nothing, might it not? For any two natural numbers, the lowest common denominator is 1. Finding the greatest (largest?) common denominator is indeed what involves an algorithm. This terminology came from fractions. The canonical form of a fraction uses the lowest possible number as the denominator. When performing additive operations on fractions, the easiest way to proceed is to convert them to a form with the same number as a denominator. The usual choice is the lowest common denominator, which is the smallest number that can be used as the denominator in both fractions. This is the LCM (least common multiple) of the two denominators. e.g.: 2/3 + 3/8 = 16/24 + 9/24 [24 being the LCD] = 25/24 = 1 1/24 It doesn't therefore (IMO) make much sense to talk about the lowest common denominator of any two natural numbers. The LCM, yes. The LCD, no. It makes even less sense to talk about the lowest common denominator of Python implementations, except in the metaphorical sense Mark (I believe) intended. but-since-we're-being-literal-please-read-this-with-a-straight-face-ly 'rs - steve ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ -----------------------------------------------------------------------

Hm. People who don't read the detailed documents shouldn't expect to rely on details.
Have you read the library manual recently? We are very careful in adding notes about which version added a particular feature or even detail. So I think there is no reason to complain about this preemptively unless your *own* experience indicates there's a lack of documentation. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
People do read the detailed documents, but not again and again every 6-8 months... heck, I work in the Python development team and don't even feel like I am up-to-date with all the changes going on in the core. If not even developers can follow the rate of change, how should a typical Python user feel ? Conclusion: I think that we need a migration guide for Python. This would solve a whole lot of these "rate of change" problems. The guide should complement the additions and changes to the other documentation and provide a single source of knowledge in that area. Currently, the only source we have in this area is the Misc/NEWS file and this doesn't provide any upgrade path hints or porting details. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

Conclusion: I think that we need a migration guide for Python.
I think Andrew's "What's new in Python x.x" documents are a good start. What's missing from them? --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
The migration path, including hints on how to port Python applications from one version to the next, how to work around deprecations, tips on quick ways to find problem areas in the code, etc. Andrew usually puts more focus on new features than changes which could cause older applications to fail -- which is what you'd expect in a document titled "What's new in ..." :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

OK. The question is, who's going to write the migration guide? One problem is that often specific migration issues don't surface until after the release is made. --Guido van Rossum (home page: http://www.python.org/~guido/)

I have fixed minor incompatibilities many times. Adrew's work was always my reference for that changes. Eventualy some changes weren't there, and it was always a pleasure to mention them to Andrew.
I'd rather help Andrew appending additive porting information in his already excellent work. I'm sure he will appreciate any construtive comments about the paper. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]

On Tue, May 28, 2002 at 06:20:54PM +0200, M.-A. Lemburg wrote:
Part of the problem is that I work from the CVS logs and from the Misc/NEWS file, and people don't always indicate when a potential incompatibility is introduced. (That may not be clear to the person making the change, of course, if they think they're changing an obscure bit of code.) I'd certainly like to point out required changes. The 2.0 document had a "Porting to 2.0" section, but now I generally try to cover such issues in the section for the relevant PEP. --amk

akuchlin@mems-exchange.org wrote:
You're doing a great hob here, Andrew ! What should we do to make your work easier ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

On Wed, May 29, 2002 at 03:03:27PM +0200, M.-A. Lemburg wrote:
What should we do to make your work easier ?
Just let me know about incompatibilities that you run into with 2.3, and I'll add brief notes about them. (I'm on python-dev and won't miss issues raised here, so don't bother e-mailing me about them, but might easily miss a python-list thread.) Given all the recent flamage over this issue, probably I should have a separate "Porting to 2.3" section that summarizes everything to watch out for. --amk

That section might start by noting that anything that prints Boolean results (e.g. ``print a==b'') or converts them to strings might be affected by the new bool values that are now often returned as Boolean results. Skimming Misc/NEWS might be the most efficient way to find out other things that could affect old code; e.g. I read that xrange's deprecated features are now removed, assert no longer tests for __debug__, an obscure change to __new__ and __init__, a change to pickling objects with __slots__, and a restriction of sys.exit() to a single argument (sure to bite someone). Library: pwd, grp and resource return enhanced tuples; a change in ftplib.retrlines. At the C API level (do you cover that?) the pymalloc changes probably warrant mention. And more, probably. One question is how spin this. Listing a long series of incompatibilities presents the image of a language that has many gratuitous incompatible changes -- which is not true and the opposite of the message we want to convey. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Please do (maybe as separate section only for the C API changes).
-- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

On Wed, May 29, 2002 at 03:52:02PM -0400, Guido van Rossum wrote:
Yes, C-level changes are covered.
This way madness lies. Document nothing and there will certainly be complaints. Why worry about losers who are in essence complaining that the documentation is *too* complete? --amk

All I'm asking is a leading paragraph something like this: """While the Python developers do their best to make every subsequent release backwards compatible with previous releases, in a system of this complexity, sometimes incompatibilities in odd corners of the language are unavoidable. While we expect that these will affect only a small minority of Python users, we document them here for completeness.""" --Guido van Rossum (home page: http://www.python.org/~guido/)

On Wed, May 29, 2002 at 04:44:20PM -0400, Guido van Rossum wrote:
I don't like this text, because the section on porting should also mention changes such as 'yield' becoming a keyword. Changes like that are hardly accidental, and hardly in an obscure corner. I'll try to come up with a different leading paragraph. --amk

Good point. Though yield has been foreshadowed by a warning and a future statement in 2.2, so it's only news for people porting from 2.1 or before. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
When I read complete change lists, I always think "don't they have more important things to worry about", with the implication that the author of the list did a good job, and unless I spot something that I know will affect me, it gives me the warm feeling that I don't have to worry about unexpected breakages. So I'm all for producing a as-complete-as-possible list, and let readers judge the relevance of this list themselves. Regards, Martin

On 29 May 2002 at 15:52, Guido van Rossum wrote:
Hopefully, Python-in-a-tie changes the equilibrium somewhat. You can point the "pace of change" whiners to it (once there's something to point to <wink>); conversely, you needn't worry much about spinning a non-PyTie release. Because of the longer release cycle, a PyTie release *will* involve a lot of language changes. MAL's idea of a migration guide is great - just make sure it's presented as an "aid", not a "warning". -- Gordon http://www.mcmillan-inc.com/

On Wed, 29 May 2002, Guido van Rossum wrote: ...
Maybe an Anti-News document like the one that accompanies Emacs. This explains all the 'benefits' of downgrading to the previous version. It could be useful for porting between versions while at the same time poking fun in a good-natured way at those living in the past... I have rudely interrupted here before without introducing myself. I'm a graduate student in astronomy. I started using python in 1998 because of the Monty Python jokes. I use Numerical Python a lot when I'm working, and lurk around here when I should be. -- Greg Ball

Andrew Kuchling wrote:
Ok.
Right. Maybe even as separate document, so that a casual visitor doesn't get blown away by the rate of change ;-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

[Guido van Rossum]
Have you read the library manual recently?
No. I read manuals a few times quite carefully, but now am only using them as references. As a user, I cannot really afford reading everything at each release.
Checking random sections of the library reference for actual needs, I remember having had to cross-check with previous printing of the same to make sure that the code I was writing could be ported to other machines running older Python versions. I would probably not have done this if I stumbled on that information in the first place, so it may lack here or there. It is very nice that you are careful about adding notes as you explain above, thanks for all of us. If I meet other unclear cases, I'll try reporting them precisely in the future (yet I'm surprised by the amount of email which is sometimes needed to get a little point through.) My own experience has no frustration, because I took the time to check. I only guess that other's experience may yield frustration for them, witnessing the reluctance or anger of some of my co-workers to any kind of change. Surely, I do not feel real problems myself. Maybe I should shut up. I only thought useful, at least a bit, that I try to explain what I see around, because I know my co-workers are not going to step in the public discussion arena (they also have to go over the English barrier), and presume most users are just silent alike.
Hm. People who don't read the detailed documents shouldn't expect to rely on details.
You know, everything is a detail of some sort, and at least for newcomers, the language and the library are mere accumulation of details. Users just cannot program without relying on these. One needs a lot of perspective for sorting out the relative importance of various specifications. So, in your argument above (which I find a bit harsh), it might not be very realistic expecting that people read everything in every release. It is reasonable to expect users to read everything once, however -- and users may then have to wave between sites, distributions, and Python releases. If you try putting perspective and historical notes in the documentation, this is very good, and I guess it will be helpful for many people. -- François Pinard http://www.iro.umontreal.ca/~pinard

Now consider my frustration. We go through a *lot* of efforts to make consecutive releases backwards compatible, to document changes, to introduce warnings about future incompatible changes, etc. And here you are referring to second-hand frustration with the pace of change, which cannot be substantiated. Is it not fair that I ask you to provide more details or shut up? After all it is a fact of life that many people like to complain without reason, will not admit to their own mistakes, and blame them on others.
Actually, in my experience, most users don't read the manuals even once (what I know of your personality, you are an exception). Example: the possibility to write list.append(a, b, c, ...) was never documented, yet it caused widespread complaints when we disallowed it. My expectation is that what is really going on here is that people write code that uses undocumented features, because they found experimentally (or accidentally) that certain things work in one version, and do not bother to check the reference manual if the language guarantees the observed behavior. If your code only uses documented features, you won't have a problem upgrading, and if upgrading breaks old promises, this is mentioned very prominently in the release notes. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido van Rossum]
Yes, I quite understand how frustrating it may be for you, and I do witness the results of all these efforts. The Python documentation is of high quality, rather complete, and very abundant for whoever looks around a tiny bit. In my own experience, migrating has been a breeze all over.
Is it not fair that I ask you to provide more details or shut up?
Not only fair, but also easier: providing details is natural for me! I sometimes fear being perceived as a nit-picker. On one hand, I met many maintainers who like detailed reports on little things! :-). On the other hand, I've seen a few maintainers getting furious, and this is no fun.
[...] most users don't read the manuals even once
You may be right. But then, they miss something! :-)
Example: the possibility to write list.append(a, b, c, ...) was never documented, yet it caused widespread complaints when we disallowed it.
*This* is frustrating, indeed. Wandering outside specs is a capital sin[1]. (This is why, for example, I find that most current HTML usage is horrible.) I wonder if, in your place, I would be so soft with users :-). -------------------- [1] Don't throw the first stone, they say! For one, I routinely abuse of immediate automatic finalisation in C-Python, but _only_ after Tim told me it will never go away. Still, there is no promise of this in the manual :-). -- François Pinard http://www.iro.umontreal.ca/~pinard

[François Pinard]
That's because there isn't a CPython manual: *the* reference manual is telling the truth about Python-the-language. You'll regret your sinful ways if you ever move to JPython or Vyper, but you won't, so sleep easy. If Guido ever gets rid of refcounts in CPython-the-implementation, the language will get renamed to Orlijnthon (or something like that <wink>).

[Tim Peters]
You'll regret your sinful ways if you ever move to JPython or Vyper, but you won't, so sleep easy.
I might, who knows. Then, I'll have no regret and gladly assume my choices. However, if I ever move, I will have the impression of writing like another language, very familiar of course, but a bit less concise, less clean. :-) -- François Pinard http://www.iro.umontreal.ca/~pinard

On dinsdag, mei 28, 2002, at 07:00 , Guido van Rossum wrote:
As an aside, note that this backward compatibility is actually a mixed blessing, because it means you don't have to update your modules now, but there will come a time when it is going to bite you. As a personal example: the MacPython toolbox modules haven't been updated to make use of the GC stuff yet (and that's been there since 2.0, no?), let alone the new type system. And these are almost all generated, so it would probably only take a few dozen lines of code to fix them. And the new type system would be a real boon for some of the modules (such as the windowing and dialog stuff), but because there's no real push (i.e. everything still works) nothing has happened yet... -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

When new releases take features away, we will issue warnings as a gentle push. When they add features, I don't know why you *should* use the new features, unless you need them -- and then you have your motivation in your needs.
But the API was totally changed for 2.2, so you're actually lucky that you didn't do it for 2.0. ;-)
I don't think *anything* can be done to force you to start using new optional features... Eventually classic classes will go away, but that will be a long time. --Guido van Rossum (home page: http://www.python.org/~guido/)

We seem to see this issue a lot here and perhaps the problem stems from the fact that there's no really experimental branch (or at least I don't know about it :) ) to python where the cutting edge people can mess with the cutting edge and won't feel too bad if they get burnt. Perhaps Python should adopt a versioning number such that major changes occur during odd numbers, etc. Just so users have an idea what's coming. Add a minor version number and release more minor versions. That way we can always expect major changes between them and at least everybody was *always* warned, as opposed to downloading the latest version and praying that something you liked wasn't depreciated, a warning was received or that something might break. Maybe that might help identify how to document the major changes, or rather when users should look and see what has changed/what has broken, etc. I think the backwards compatibility is pretty damn good though. I had no problems writing an app and backporting it to 1.5.x (I basically just had to use the string module). I did use functional stuff over list comprehension though... My next app, however, will require python 2.x and that's an audience decision. But not every one has that luxury. Almost an after thought, the last thing we want is to take the Java route though and have 'classes' disappear and new ones replace them with new interfaces. python's uniformity was one of the biggest sellers to me. Ugh. -- Mike On Tue, May 28 @ 16:49, Guido van Rossum wrote:
-- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html

On Wednesday 29 May 2002 05:11 pm, Michael Gilfix wrote: ...
get burnt. Perhaps Python should adopt a versioning number such that major changes occur during odd numbers, etc. Just so users
Yep. I proposed this in the very first message of the thread with this subject, about 2 months ago I believe. Linux (and perhaps more relevantly Ruby, a language very close to Python in many respects) use even minor versions for the releases of the stable-track (slow, prudent and backwards-compatible change -- in theory, at least; not sure how good is Ruby at satisfying this promise... Linux is so-so), odd minor versions for the releases of the experimental-track (fast, frequently-released, not necessarily backwards compatible changes). However, after bouncing the idea back and forth a bit, in several variations suggested by various people, Guido ended up rejecting it. It may be satisfactory for Ruby, but he decided that it would not be for Python, no matter how close the two cases appear to others. And he _is_ the BDFL, after all. Some help on this score may hopefully come from the recently formed Python Business Forum. Among other tasks the PBF plans to choose and exhaustively test certain Python releases, proposing to Guido to designate those releases "Python in a tie" (I think GvR came up with this specific monicker), then backporting fixes to them &c to ensure the long period of stability typically desired by commercial end-users (I think the target is 18-24 months vs the 6-8 months target of Python minor releases). I apologize if this summary is in any way imprecise -- http://pbf.nuxeo.org/ has authoritative info. Alex

Jack Jansen wrote:
I don't see the argument here ? You don't seriously expect all extensions, add-ons, existing applications, etc. to be rewritten, patched or changed in other ways with every new Python release, or do you ? Python builds a lot of its popularity on the huge number of add-ons you can download from the web. We should be *very* careful not to break these in ways which make them unusable, because otherwise, we'll have frustrated users and these would be seriously bad for continuing the ride on the wave we're currently seeing. The developer side of things is a little different, since there changes cost time which is usually a rare resource. For small companies it also costs money which they usually don't have. (A migration guide would help both.) In summary, rapid change is not a good approach to a stable product, rapid bug fixing and addition of useful enhancements in backward compatible ways after longer trial phases is, at least IMHO. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

On Monday 08 April 2002 00:08, Guido van Rossum wrote:
You know PythonLabs's missions and goals better than I do. If such goals weigh "making a new and better Python" far more highly than helping spread and preserve the use of Python, then you may well be right that the extra effort for (e.g.) making email part of the standard distribution vs the current "maintaining a separate distro" would be a bad allocation of scarce resources.
If I understand correctly, ActiveState has backed off (because of not sufficient revenue returns) from "owning" Hammond's win32all extensions, which are truly indispensable to firms using Python for Windows-only development. If win32all did not "have a commercial market" (sufficient to justify ActiveState's investments), then I don't see what other packages could.
Something requiring much effort is not necessarily "obviously out": it depends on how important you judge the results of those effort versus the results you could have by spending the effort elsewhere. In terms of "how it looks to actual and prospective users of Python" rather than "how much effort would it be to implement", iterators (not generators) and // division look good -- enabling "for x in dict" and "if x in dict" are two features of instant appeal -- language enhancements that do not break previously correct code. It's of course possible to construct a program that would break due to any such change, because it expects and relies upon a specific exception being raised when such usage is attempted -- when the usage becomes legal and meaningful, crack. But in most cases that would be a "sea-lawyer" level issue. You're probably right that, alas, overflow on int operations wouldn't be feasible to remove (I don't know). Syntax changes that make some source code legal where it didn't use to be, e.g. // , are probably at the other extreme -- in practice one would not rely on e.g. compile('a//b') throwing a SyntaxException and thus break if it doesn't. To "be concrete" as you ask, I think that targeting 2.1 as the stable release would not be the best allocation of effort, even though I see others disagree. Basically, the argument against that would be that 2.2 was _announced_ as the new stable release, not as a bleeding edge experimental one, and making it "retroactively experimental" would send exactly the wrong message. So, I think effort would be better spent ensuring that enhancements for 2.3 are well divided between [a] those which are safe to have in some 2.2.* as well, vs [b] those which ARE experimental (break previously correct code) and thus should only go into 2.3.
The fact that email and xmlrpclib are already installable as separate distros on 2.1.* suggests the amount of effort to put them into the standard library of 2.1.something might be modest; therefore, even if the advantage is also modest, it might still be worth doing.
Assuming one cares about Python being widely used, "capturing" a new "customer" that is highly change-resistant has better returns than getting one who's very change-prone. Exactly because of that user's inertia, they'll stick with Python once they've decided for it -- some other shop who'll gladly have a carefree fling with Python is just as liable to leave it for Ruby, or whatever, tomorrow. Apparently we DO want to promote the spread of Python (otherwise the purpose of the python marketing list and the PSF becomes murky to me). We believe the world will be a better place if more development uses Python rather than different languages. That goal is well served by helping Python acceptance by development shops that do care about stability, because such shops are a substantial fraction of our industry. Over the decades, the people who are now the technical managers and senior developers in such shops have seen too much of their work laid waste by incompatible changes in platforms, and have thus become quite sensitive to that specific technological risk; they've seen some other fraction of their efforts similarly wasted by having been spent on platforms who became "really stable" (i.e., dead) and are thus also sensitized to that antisymmetric risk. These are survival traits in the software jungle. Our industry suffers from so many "bandwagons" (in platforms, methodologies, tools, etc) that learning when NOT to jump on one -- developing a nose for the symptoms of the two opposed risks, excessive/churning rate of change and the "sunset boulevard" syndrome of a slowly dying technology -- make a shop's survival more likely. So, I think we need to avoid triggering either reflex (in sensible cases: we can't really make Logajan happy, I think). That is, IF we want to promote Python's spread -- if we do believe we'll be happier if Python is widely accepted rather than less used than it deserves to be. I'm reasonably sure of that, myself. The more Python is widespread, the more of a "Python market" commercial ventures perceive as existing, the better my chances to find a good Python port to some weird new platform, the better some other enthusiast's chances to find jobs based on Python (or mine, should my current arrangements fail or terminate), the better tools I'm likely to have my pick amongst, and so forth. I didn't think this needed to be re-examined on this thread, but I'm quite game if it does.
I understand that doing nothing except changing communication patterns entails less effort than other changes. Exactly because of that, the other changes send a stronger message: they bespeak real commitment to "stability AND change" _because_ they don't come for free. Economy Nobel prizes last year all went to economists whose major work has been on this subject: asymmetric-information situations and message-sending. For example, that's how advertising, sponsoring sports, etc, mostly work: a firm sends a message "we really believe in this product" by splurging a lot of money in promoting it, _because_ it's substantial effort -- effort it would make no sense for them to spend if they (who have more info on the product than the prospective buyer) really knew it was a dud. I'm summarizing and oversimplifying, but that's the gist.
I _do_ testify that all new code I've written or ported to 2.2 has run without a hitch so far. That's quite different from saying that existing code written for 2.1 ran unchanged with 2.2, because it didn't -- nor do I think it reasonably COULD have, mind you. E.g., any time I introspected an instance via dir() I only used to get per-instance attributes -- now I get attributes it gets from its class too; it's an enhancement, but it does fool any such introspection that I had. Unit-tests might help, but unfortunately I relied on doctest for those, so I get a zillion "breakage" reports that aren't all that helpful. Guess I misused doctest (but it's SO much handier than unittest that it's an "attractive nuisance", consarn it!-). Once you do manage to identify the real breakage points, it's not (generally) all that hard to turn 2.1 code into 2.2 code. People who maintain both 2.1 and 2.2 versions of their packages are no doubt better placed to evaluate that. But the message that "only little broke, and fixing it was not too terrible" is not strong enough for the shop looking for stability -- which I guess help explains 2.1's relative longevity, no?
No: there's 12 of us, but we also have a strong commitment to *no "code ownership"*. We need to agree on coding standards that are strong enough that we _can't tell_ who wrote what piece -- at least, that's the ideal (and part of XP). And the standards must get consensus, which is no picnic (but the alternatives are worse).
The impact is small (that's my evaluation) in terms of maintenance of existing code (not necessarily all that small in teaching the language or writing about it, but that's another issue). Total breakage of doctests is probably the worst part (maybe it's just what we needed to stop using doctests and go for other approaches instead). Still, "zero in most cases" is NOT the situation with doctest usage today: I do believe I was not the only one to fall into its attractive-nuisance trap.
Mostly, yes, and I do agree that Logajan's impossible to satisfy -- I see no amount of effort that could have kept him happy, given his expressed complaints. But if you're implying that said perception (by prospective users) is unfounded, I disagree. It has a solid basis in reality. Some previously correct code does break at each minor release (doctests first -- as apparently is their "canary in a coal mine" intended role -- but that is quite visible and contributes highly to perception). To some extent ("sea-lawyering" level) that may be inevitable. Moreover, the "status" perceived for releases such as 2.1.2, etc, seems correct to me: you do NOT want to invest substantial effort on them -- isn't this what you're saying throughout this? So, the perception "not quite dead but close, BDFL doesn't care much at all about them" is correct, no? It may be possible to change perception without changing reality, but I have some doubt. "You can fool some of the people all of the time", etc, etc:-). Alex

Actually, in the minds of the people who pay my salary, "making a new and better Python" is one of the best ways to "help spread and preserve the use of Python".
I very muich doubt that the corporate users who are currently worried about the fast pace of change are interested in any particular feature, and I don't think that any backported feature is going to make that previous version more popular amongst managers with decision power. The question in front of them is, "should we use Python or not", not "should we use Python 2.1". To answer that question, they want to perceive "Python" as stable, not "Python 2.1". --Guido van Rossum (home page: http://www.python.org/~guido/)

On Monday 08 April 2002 20:18, Guido van Rossum wrote:
Surely having a better Python (by definition of 'better') is in the long run important. I'm not sure of how that importance compares with perceived stability in determining how widespread Python becomes. Judging by languages such as C or C++, stability seems paramount; yet Java churned a lot and still managed to spread a lot too (not without a lot of help from high-$$$ marketing efforts, though).
True, and yet such decision makers DO want to perceive that the specific software they use IS actively supported. It's a reasonable desire indeed, as I've tried to explain quite a few times. If they perceive that choosing "Python in general" means they have to choose between an "old, not actively supported any more" version of the language, and one that breaks previously working code every six months, then that will weigh on their mind as a big minus for Python. If they perceived they could choose a "stable but actively supported" version (the existence of an experimental one too would not worry them, I believe -- many popular languages sprout experimental ones based on them too) then that worry would be out of the way, and I'd have a better chance to get them to LOOK at the huge productivity improvements Python has in wait for them... Alex

[Alex, getting better at brevity]
Another example of the Logajan paradox.
Yet it is close to Logajan's position.
So it's purely a matter of spin. Because new Python releases every 6 months do *not* mean that code breaks every 6 months. Yet some people continue to believe this.
Maybe all we need to do is make the micro releases a bit more visible... --Guido van Rossum (home page: http://www.python.org/~guido/)

On Monday 08 April 2002 21:26, Guido van Rossum wrote: ...
I dispute that, even though he himself appears to be judging from one post he made to c.l.p -- which I answered trying to dispell his false hopes in the matter. He wants to ensure that software that runs on (e.g.) 2.(N+1) must also have run on 2.N : this is unreasonable, and I've never heard anybody else ask for that. He also seems to want this to hold across major release number changes, 1.* to 2.* -- and again that does not seem to be a widespread priority.
I may be wrong, but I think that the fact that doctests DO break with every release has something to do with this perception. doctest is SO nice, easy and natural to use (within a single release) that it's hard to accept it as "unusable between minor releases", even though I have to reluctantly accept that decision.
I may be wrong, but I think the concept of dual, stable and experimental, tracks, already familiar and accepted from other pieces of software, would facilitate prospective user's perception of "stable but actively supported software". Alex

[Alex Martelli]
Not to be cynical <wink>, but what those all have in common is the visible backing of multi-billion dollar companies. Ditto Visual Basic from MS. Python will enjoy that status too just as soon as people start posting in outraged droves about Zope Corp's monopoly over the half-trillion dollar web content market.

On Tuesday 09 April 2002 01:51, Tim Peters wrote:
Not an issue of being cynical -- it's just a fact that C enjoyed no "visible backing" from AT&T (or did you have some other corp in mind?) to help it gain mindshare, and neither did C++. If "backing from multi-billion dollar corps" mattered as much as you seem to think we'd all be using PL/? or maybe some APL. Or do you forget who DID back _those_, and how much Big Blue dominated the computer scene until not-all-that-many years ago? Alex

[Alex Martelli]
No, I don't think backing matters much at all to how a language *becomes* popular. C and C++ and Java have both popularity and the backing of billion-dollar companies *now*, and big money wants to play with big money. This becomes very relevant when considering how big shops make adoption decisions: a small money player has an uphill battle even to get noticed. Programming languages, databases, web content management systems -- I don't care, it's all the same. Big shops want their suppliers to have deep pockets. Little shops can be more flexible, but a lot of them just copy what the big shops do. The only reason C got into big shops is that American management is so incompetent it didn't notice what kinds of risk the hippies in the IT department were taking <0.9 wink>. give-the-psf-a-billion-dollars-and-we'll-put-it-to-the-test-ly y'rs - tim

On Tuesday 09 April 2002 09:41, Tim Peters wrote: ...
No, I don't think backing matters much at all to how a language *becomes* popular.
Ah, OK, we agree on that. Though it helped Java and VB.
C and C++ and Java have both popularity and the backing of billion-dollar companies *now*, and big money wants to play with big
Once something does become that popular, money flows more (again I see Java as different -- big money from the start).
This is possible. It's not how it got into IBM (I remember the early times of it at IBM Research), but other managements might have been less careful or competent.
give-the-psf-a-billion-dollars-and-we'll-put-it-to-the-test-ly y'rs -
IBM invested about that much in Linux technologies _after_ Linux had started earning huge popularity -- surely helped that popularity grow (IBM claims they're already roughly even in return from that investment through mainframe sales and consulting services -- Palmisano headed that effort, and he's IBM's #1 now, having been recently promoted). I may be wrong, but I perceive a slowly mounting group of entrepreneurs trying to make a billion (or whatever:-) from Python -- and quite ready to feed some of that moolah back, once the moolah IS there. We'll see... Alex

Quoting Guido van Rossum (guido@python.org):
I think, as this discussion slowly seems to be winding down and my list subscription goes through, that I can finally contribute something. One of the points I've seen brought across is that "stable python" needs to be more widely acclaimed as such. The example seems to be the 2.1.3 release, or the 2.1.* series in general. I agree with this, wholeheartedly. The remains of the discussion seems to circle around how to number or manage the changes so that it's easier to know how things are developing. To this, I think that picking a scheme is a lot less important than documenting what that scheme means to the people who are using it. Maybe we can cash in on some of the ideas put forward by other people, maybe we can't, maybe we shouldn't. What's important is not how we do it, but that we're explicit about what we do. The remains of this note are going to be long-winded, so if you're prone to start skimming, now would be a good time <wink>. BACKGROUND (where I'm coming from) A little background -- I'm part of a company (at my other email address) that views Python as part of their competitive advantage. We engage in weekly rolls into production, with product cycles that last between two and four weeks. Three of us write python regularly; four of us write DTML and HTML. Management duties, release scheduling, release management, deployment, and site maintenance responsibilities are scattered liberally among this crew of seven. We do a lot of business at this site, which makes those scattered responsibilities fairly time-consuming for some of us. Despite this, we manage to maintain the aggressive schedule, which has in turn enabled us to bring in big partnerships and close substantial deals with a very small core of engineers holding it all together. In many ways, we are the sort of story that serves as an advertisement for Python (though you should stop well short of holding up our code as examples, if you know what I mean). On the other hand, a full overhaul of the site to advance to a new version of Python is a substantial project. Between the dependencies on third-party products, the extensive customizations we've had to make, the QA overhead that goes into a full site regression, and the management overhead that goes into tracking down all the bits and pieces -- that's about a month-long project, though we might be able to fit it into 3 weeks. This puts us into the "conservative" camp. For some time now, we have regarded it easier to steer around, work around, or hack around bugs in 1.5.2 than to upgrade. We've backported a fix or two. It's still not enough. Recently, we've agreed to upgrade, and managed to secure executive permission to take a 3 week cycle (scattered over the next couple months) to do the updating work. My situation may be atypical, but I think that we're the sort of operation that the python community doesn't really want to alienate. THOUGHTS ON STABILITY When I use python for myself, I prefer to use 2.2, because I prefer to stay current. When I develop python for a wider audience, I plan for 1.5.2, because that's what I'm pretty sure I can count on people to have available. I can speculate why this is so, but it would be speculation. When I looked at upgrading to the 2.* series of python interpreters, I wasn't sure where to target. 2.0 is for all intents and purposes a dead end. 2.1 is seeing some active bugfix development, but there's no visible commitment from the community to continue that trend. In fact, when the topic gets broached, it more often than not gets shot down with "not likely, who would want to waste their time on doing that?" or some variation therof. 2.2 appears stable, but there's a bugfix release in the works that hasn't (as far as I know) been made official yet. In other words, none of the 2.* series appeared like particularly attractive targets to shoot at, because all of them appeared to be unsuitable for the needs of our organization. I decided to shoot for 2.2, because it's hard to tell when we'll be able to do the upgrade again, and I wanted to solve as many problems as could be solved, now. It appears that may have been a bad choice, given progress with 2.1.3. People talk about a 6 month gap between releases, but I'm not sure anyone recognizes that timeframe as "average" yet. You can have an average with only two measurements, but it really takes three to make a series, and the gap between 1.5.2 and 2.0 was considerably longer than 6 months. The point here is that when you adopt any given minor version of python, you have no idea how long you have before you're obsolescent. Six months is reasonable, if that is in fact what we continue to hit. But being predictable is more important to "the conservative crowd" than any particular interval could be. In conclusion, I think that what "the conservatives" are looking for is really quite easy to provide. I summarize them as: - Be predictable. If we're putting off pain until 3.0, make a guess when we're going to need to deal with that pain, so we can start preparing. If 3.0 is 9 months off, that's a lot scarier and more concerning than if it's 2 years off. - Be explicit about the process. If the process is to stick a fork in the ham and call it "stable" just before moving on to the next minor release, then be explicit about that process. - Be explicit about obsolescence. If things in 2.3 or 3.0 or X.Y are shaping up to be so different that bugfixes won't be back-portable, then identify that fact early -- and make an effort to provide a reasonable target for people to upgrade to, as soon as possible. The perceived problem with change being too rapid may be just that, a perceived problem. With adequate and careful communication, the problem may disappear entirely. It may not, but we can always dream. :) Anyway, that's about all I can contribute, after thinking long and seriously about what I feel like our company needs, and trying to ignore the specific details of implementation. I hope it's helpful. Thanks, --G. -- Geoff Gerrietts <geoff at gerrietts dot net> http://www.gerrietts.net/ "Politics, as a practice, whatever its professions, has always been the systematic organization of hatreds." --Henry Adams

[Geoff Gerrietts]
If you're coming from 1.5.2, picking 2.2, the most recent "final" release, is the right choice. Most breakage of 1.5.2 code was done in 2.0, when we didn't have warnings in place, and added Unicode. 2.1.3 is good for people who ar already on 2.1 and who want to see some bugs fixed. We fixed TONS of bugs in 2.1.1 and 2.1.2; 2.1.3 will be released tonight with only a very small number of additional fixes, but including one that happens to be important for some Zope users (Zope can do some weird shit that you're not likely to run into yourself :-). 2.2.1 will *also* be released tonight. It fixes most known bugs in 2.2, adding almost no new features (depending on how the size of your yardstick). The idea is that anybody who would want to run 2.2 would be even happier with 2.2.1, and that people who were afraid that 2.2 would crash an burn can now try out 2.2 without that fear.
Before 1.5.2, we *also* hit 6 months with some regularity. The only irregularity was really the 18 month gap between 1.5.2 and 2.0. We're seeing the gap slowly increase, to maybe 8 months for 2.3, but I expect that 6-8 months is probably what we'll continue to see.
3.0 will always be 2 years off. :-) At least for now, it really is.
We are already pretty explicit; there are several PEPs about the process.
It depends on the bugfix. Some modules and subsystems are being refactored in order to provide new features, and then bugfixes won't port back. Other modules and subsystems are very stable (some could be called stagnant :-) and will have no problem getting bugfixes backported.
Yes, maybe it can be made to disappear. Does that mean we need to throw the reactionaries on c.l.py a bone? Is this really all politics? In that case I'm out of here.
Thanks! --Guido van Rossum (home page: http://www.python.org/~guido/)

"AM" == Alex Martelli <aleax@aleax.it> writes:
AM> It would help to some degree if the "stable" releases could be AM> shown to have SOME kind of enhancement. Not as much as if AM> language changes which did not break existing correct code AM> could also be there, but some. Modules such as email are a AM> big part of the draw of 2.2, for example (not quite as big as AM> language-level enhancements, but big). And of course, I spend a non-trivial amount of effort to maintain a separate email package that is compatible with Python 2.1 and Python 2.2. But I consciously gave up Python 2.0 (or earlier) support because it became obvious to me that I wanted /some/ of the newer features. But that's really difficult, considering it means I have to keep two cvs repositories in sync (and yes, I've got a bunch of changes that need merging back into Python 2.3). -Barry

On Monday 08 April 2002 07:54, Barry A. Warsaw wrote:
How does this amount of effort (keeping two CVS repositories in sync) compare to maintaining just one tree but with "feature-test macros" as one might try to do in C? In other words, how much help would it be if (for example) Python's compiler optimized away tests of the form, e.g., if __version__ > ( 2, 2 ) : as it does now with -O for "if __debug__:" but even more deeply (so even some "syntax errors" in the optimized-away conditional would be OK -- catering for future syntax changes at the add-a-keyword level, e.g.)? Maybe the obfuscated-with-version-tests merged source would be too hard to maintain -- and maybe not, so I'm quite curious about what your experience suggests here. Intuitively, it might seem a version test could perhaps help two or more versions be alive at the same time, but I know of so much controversy over C's #if (mostly, IMHO, because of its over-use) that I wonder. Alex

"AM" == Alex Martelli <aleax@aleax.it> writes:
AM> How does this amount of effort (keeping two CVS repositories AM> in sync) compare to maintaining just one tree but with AM> "feature-test macros" as one might try to do in C? In other AM> words, how much help would it be if (for example) Python's AM> compiler optimized away tests of the form, e.g., if AM> __version__ > ( 2, 2 ) : as it does now with -O for "if AM> __debug__:" but even more deeply (so even some "syntax errors" AM> in the optimized-away conditional would be OK -- catering for AM> future syntax changes at the add-a-keyword level, e.g.)? AM> Maybe the obfuscated-with-version-tests merged source would be AM> too hard to maintain -- and maybe not, so I'm quite curious AM> about what your experience suggests here. Intuitively, it AM> might seem a version test could perhaps help two or more AM> versions be alive at the same time, but I know of so much AM> controversy over C's #if (mostly, IMHO, because of its AM> over-use) that I wonder. Part of the problem is implementation details (i.e. avoiding the use of Python 2.2 features in standalone, but not in the Python cvs tree). Part of it is due to the nature of distutils package layout (i.e. you need to put setup.py, README, etc. someplace, and distutils seems to strongly discourage it being in th epackage). Part of it's also that I feel I have more leeway to change things in the standalone package. But at some point it will stop making sense to distribute a separate package. -Barry

On Mon, Apr 08, 2002, Barry A. Warsaw wrote:
Hrm. But until it's no longer a separate package, why use 2.2-specific features in the builtin package? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

"A" == Aahz <aahz@pythoncraft.com> writes:
A> Hrm. But until it's no longer a separate package, why use A> 2.2-specific features in the builtin package? Because it's convenient, and the semantics are subtly different (and IMO suboptimal) without them. E.g. the builtin version uses a generator in a few places, while the standalone version uses an iterator. The generator version is much more intuitive, but to write cross-version code that uses that interface you need to do something like call list() on a return value. -Barry

On Mon, Apr 08, 2002, Barry A. Warsaw wrote:
Well, sure. But you have to do that suboptimal stuff *anyway* in order to maintain the separate package. How is the overall development process more convenient to maintain two codebases? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

"A" == Aahz <aahz@pythoncraft.com> writes:
A> Well, sure. But you have to do that suboptimal stuff *anyway* A> in order to maintain the separate package. How is the overall A> development process more convenient to maintain two codebases? It isn't! And now that I've announced email 2.0, I'm going to merge it all into Python 2.3 cvs (but not the suboptimal implementation. ;). I suspect updating the docs will be the most work. -Barry

I expect that most of the incompatibilities are the use of some tiny new shortcut in the language (think of +=, or list comprehensions, except that these were 2.0 features) where it really doesn't pay to do a version test -- instead, you end up coding in the old dialect. --Guido van Rossum (home page: http://www.python.org/~guido/)

Barry A. Warsaw writes:
PyXML is specifically maintaining compatibility, which means I have to work around the lack of weakrefs when removing reference cycles. The differences between the Python standard library and PyXML, for modules present in both, is mostly trivial still, but the differences are growing. ;-( (It's enough to make me think that Andrew may have been right about not integrating the XML support the way we did.) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation

PyXML is specifically maintaining compatibility, which means I have to work around the lack of weakrefs when removing reference cycles.
Weren't weakrefs an external module in 2.1? Can't you distribute that separately with PyXML (under a different name)?
(It's enough to make me think that Andrew may have been right about not integrating the XML support the way we did.)
Yeah, but there were some very loud supporters who mostly wanted it for political reasons at the time. I think I may have fallen for their arguments a little too fast. ;-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum writes:
Weren't weakrefs an external module in 2.1? Can't you distribute that separately with PyXML (under a different name)?
They were an external module in that the types as well as the Python-level access was in Modules/_weakref.c, but invalidating references required cooperation from the core, since action is taken when the refcount goes to 0. It cannot work for 2.0 under any name.
Maybe. I *still* think providing XML support out of the box is a good thing, the questions for me are related to how that should be done rather than whether it should be. But that's a separate discussion. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation

"Fred" == Fred L Drake, Jr <fdrake@acm.org> writes:
Fred> (It's enough to make me think that Andrew may have been Fred> right about not integrating the XML support the way we did.) I wonder if decoupling more of the standard library isn't a good idea, although it flies in the face of "batteries included". One thing I like about XEmacs's model is that I kind of get both for little effort. I want all the latest packages, but I don't want to re-install XEmacs to get them. So I use the package manager, which isn't exactly one-click-updates, but it's pretty simple to use. We'd still have to deal with incompatibilities in core modules, and in the language spec, but those probably require different tools. -Barry

I wonder if decoupling more of the standard library isn't a good idea, although it flies in the face of "batteries included".
I think it will be a *bad* thing, unless we can automate it to perfection (more about that below).
For most mortals, the XEmacs package manager simply *sucks*. Maybe that is because it's broken out of the box and you have to find the right webpage that tells you how to manually fix some things. But it still sucks. The only thing that *might* work IMO would be something that pulls down the latest (compatible) library modules when you say "make install", or in a post-install script when using RPMs or some such thing. But even then, the people distributing Python on CD with their books lose -- those CDs are there for folks who have a slow or non-existent network connection. --Guido van Rossum (home page: http://www.python.org/~guido/)

"GvR" == Guido van Rossum <guido@python.org> writes:
GvR> For most mortals, the XEmacs package manager simply *sucks*. GvR> Maybe that is because it's broken out of the box and you have GvR> to find the right webpage that tells you how to manually fix GvR> some things. But it still sucks. I agree totally, but that's a quality of implementation problem. Once it works, it's (almost) great. But you're right, it would have been nearly completely automatic, and bug free out of the box. I guess it's no surprise we've talked about this for years but it still hasn't materialized. -Barry

"GvR" == Guido van Rossum <guido@python.org> writes:
GvR> For most mortals, the XEmacs package manager simply *sucks*. GvR> Maybe that is because it's broken out of the box and you have GvR> to find the right webpage that tells you how to manually fix GvR> some things. But it still sucks.
"BAW" == Barry A Warsaw <barry@zope.com> writes:
BAW> I agree totally, but that's a quality of implementation BAW> problem. Once it works, it's (almost) great. I wish. There are real problems waiting to bite us, besides the gratuitous menu changes and implementation sillinesses, such as its dependence on EFS. The main one that is Python-planning-relevant is that you guys have standard libraries nested several deep all over the place, no? We have about five packages that other packages consider "a lot closer to core than I am": APEL (used by many), EFS (by the package manager itself), semantic/eieio (by JDE), and widget.el (used by just about everything via the Customize facility). JDE, the Java Development Environment, is one of our big problem children because it insists on being distributed with its own versions of semantic and eieio. Fortunately, nothing else much uses them. EFS (Extended File System) is well-done and well-maintained, but unfortunately by design depends on external FTP implementations, and every few months Red Hat or SuSE "does something" to FTP (kerberize, whatever) and EFS loses on that platform---as does our package manager. APEL is "A Portable Emacs Library", and what it does is smooth over backwards compatibility issues. Unfortunately, it does it in ways that (a) tend to introduce forward compatibility issues and (b) not really visible to debuggers. This leads to embarrassments where A Leading Developer says on Usenet that XEmacs supports such-and-so API from GNU Emacs 21, and then discovers that it ain't so, it's actually subtly different code from APEL---only the docstrings are the same. The widgets are pretty stable, but they're only loosely coupled with anything besides Custom. In other words, the main reason our package system has been so successful is that we simply don't have to worry about the issues this thread is about. But in a few cases where we do, we tend to lose. So be cautious. I won't say our experience proves it can't work; I'd actually rather say the glass is half-full, and I think that you guys can fill it better than we have so far. But it won't come "for free", there's more conceptual work to do than just "do it better than XEmacs." And Andrew Kuchling mentions Debian apt, well, that's the direction a number of the library developers are heading in terms of wishlists. For all that, it's been a roaring success with our users. Fortunately-for-them-the-BDFL-is-not-our-BDFL-ly y'rs, -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Don't ask how you can "do" free software business; ask what your business can "do for" free software.

On Mon, Apr 08, 2002 at 11:38:19AM -0400, Barry A. Warsaw wrote:
I wonder if decoupling more of the standard library isn't a good idea, although it flies in the face of "batteries included". One thing I
I'd like this approach, though perhaps the "Batteries Included" distribution rather than the standard library should be handled like this. It would also require finalizing a catalog implementation (at last!), An open question is how to avoid the CPAN trap of having your modules require you to upgrade your Python installation. Perhaps each module version would specify the Python version it's compatible with. Module version compat. with 0.0.1 Py2.1 0.0.2 Py2.1 0.0.5 Py2.2 So when you're on Python 2.1, you can only get upgraded to 0.0.2. The problem is, if the developer of version 0.0.5 didn't realize that it was incompatible with 2.1, you'll get upgraded to it and your system will be broken. (Take package management far enough and you begin to reinvent Debian's APT. I wonder if we could write a dpkg-like tool in Python and then just use APT on top of that?) --amk (www.amk.ca) Nothing I've ever written has reached 1.0. -- Greg Ward at IPC7, on using small version numbers

[Barry]
I wonder if decoupling more of the standard library isn't a good idea, although it flies in the face of "batteries included". One thing I
[AMK]
For a batteries included release, this would be great.
It would also require finalizing a catalog implementation (at last!)
I believe Kapil is working on it; he's presenting a paper at OSCON2002.
Ideally it should list a range of versions (or a set?). Of course, the upper end of the range may need to be adjusted when new releases come out -- nobody knows whether Module 0.0.5 will be compatible with Py2.3 until the latter is released.
I think version dependency management is subject to the Logajan paradox. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido> I think version dependency management is subject to the Logajan Guido> paradox. :-( Can you state that succintly for us? :-) S

On Monday 08 April 2002 22:45, Skip Montanaro wrote:
"Python keeps being maintained yet it *never* happens that release X (for any X and Y) can run some code that release Y cannot run". It's unachievable (as well as unreasonable) and thus I don't think it should be considered. A desire that I think more reasonable (still unachievable, but one can come closer) is a similar wish stated only for Y > X. In practice we do need break-points where such backwards compatibility cannot be guaranteed ("major releases") -- further, I think that for the wish to be both achievable AND compatible with other important goals it must be nuanced ('some code' becoming 'CORRECT code', for example -- broken code that HAPPENS to run by accident is quite another thing: although for some users the difference can be hard to appreciate, the concept of 'undefined behavior' is, I believe, known widely enough to rely on it a bit, at least where the docs mention it). Alex

the concept of 'undefined behavior' is, I believe, known widely enough to rely on it a bit, at least where the docs mention it).
Tell that to the people who write L.append(1, 2, 3) despite that it was never documented. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

On Monday 08 April 2002 23:00, Guido van Rossum wrote:
I do -- if and when I see that (haven't seen it in ages). Technical managers at development shops know about stuff "accidentally working" and are more tolerant of bug-fixes against that than of other backwards-compatibility breaks, in my experience. Alex

Hm. This was widely complained about, probably because many examples (including in books) got it wrong. Maybe technical manager are tolerant of that stuff, but not if it means they have to fix their won code. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido> I think version dependency management is subject to the Logajan Guido> Guido> paradox. :-( Skip> Can you state that succintly for us? :-) Alex> "Python keeps being maintained yet it *never* happens that release Alex> X (for any X and Y) can run some code that release Y cannot run". Alex> It's unachievable (as well as unreasonable) and thus I don't think Alex> it should be considered. The only consideration I was giving it was to maybe submit it to the global fortune cookie file... ;-) Skip

Quoting Skip Montanaro (skip@pobox.com):
I think that while I agree that there's a paradox there, what we're seeing is Logajan failing to express himself, not Logajan asking for a dead language. I think Logajan's idea was that backwards-compatibility breaks were bad, where you (for instance) write code for Tkinter under 1.5.2 and in 2.whatever it breaks because of Unicode changes. But it's worse when you can't write that code such that it works in 2.whatever and also works in 1.5.2 without resorting to testing version numbers and special-casing. Maybe that translates to the same thing for you, but for me, it's a very different thing. Further, I think that the issue would be quashed if it were simple to work around, rather than complex. If you could set a global variable in sys or tkinter or __main__ and be assured that you could write cross-version compatible code, that would be much preferable to having two write two different sets of modules, one that works under 1.5 and one that works under 2.0, and one that works under 2.1. I could be mistken, but I think that's what I've heard. Thanks, --G. -- Geoff Gerrietts "Democracy is a form of government that substitutes geoff at gerrietts dot net election by the incompetant many for appointment http://www.gerrietts.net/ by the corrupt few." --George Bernard Shaw

Logajan wants all code he wrote for 1.5.2 to work in 2.0, 2.1, and so on. He also complains that a script written for 2.1 doesn't work in 2.0 (and no, that's not reversed!). I conclude that he doesn't want the language to change at all. --Guido van Rossum (home page: http://www.python.org/~guido/)

"Fred L. Drake, Jr." wrote:
I don't see the relationship between the first paragraph and the last. PyXML maintains compatibility because its maintainers care about old versions of Python. That causes your headache, whether XML support is in Python itself or not. The cure for the pain is the same as for any other third-party module: tie newer releases to newer Python versions. That said, I think that the relationship between PyXML and Python is weird too. As I said at the last conference I'd clean it up by making PyXML a package unrelated to Python XML, but I was roundly voted down. Paul Prescod

The cure for the pain is the same as for any other third-party module: tie newer releases to newer Python versions.
But that's exactly what's unacceptable in theis case: newer versions of PyXML provide new PyXML features, and Python users who wish to use the new PyXML features should *not* have to upgrade to the latest and greatest Python. That would cause the kind of dependency nightmares we just heard about with CPAN (and which are also common with RPMs in my experience -- I recently had to upgrade a printer driver and it was a true nightmare). --Guido van Rossum (home page: http://www.python.org/~guido/)

I didn't say that. NumPy has exactly the same issues. You proposed to tie newer versions of 3rd party modules to newer Python releases. I think it's wise to stay at least 1 or 2 releases behind. You don't want to lose your users because you forced them into a bad upgrade of the rest of their Python installation. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
...
I didn't say that. NumPy has exactly the same issues.
We've lost the thread of discussion and are now in the land of wasted breath, but just in case you are curious how we got here... Fred said: "Keeping PyXML compatible with old versions of Python is a pain." Then he said: "It's enough to make me think that Andrew was right about including XML in Python." To me, that's a non sequiter. PyXML has the same backwards compatibility headache as any other third party module. If I had a module called PyGraphs that needed weak references, I would have a problem if I wanted it to work on Python 2.0 or 1.5.2. Then you said: "Yeah, we only included XML because some people were very vocal about including it for political reasons." Of course I was one of those people so I still wanted to get to the bottom of this non sequiter. PyXML would have exactly the same backwards compatibility headaches if there were no XML support in Python whatsoever. It's just a problem that third party modules have. You either endure the pain or you make your users upgrade to a new version of Python. As far as I can tell, those are the only two choices. Paul Prescod

On 6 Apr 2002 at 20:18, Guido van Rossum wrote:
I keep a fairly large body of code working with 1.5.2 onwards. If you broaden (b) to include cases where code that *accidentally* worked in an earlier release fails in a later release, then it covers 100% of the changes I've had to make. (1.5.2 to 2.0 was painful; the others have been easy). (FWIW, the hardest post 1.5.2 feature for me to do without is augmented assignment.) -- Gordon http://www.mcmillan-inc.com/

[Gordon]
Interesting. Two questions. (1) Got any details on which changes caused the most pain? (2) Was the pain worth it, or would you prefer we'd spent more time on being more backwards compatible?
(FWIW, the hardest post 1.5.2 feature for me to do without is augmented assignment.)
Since you're also a C programmer (I believe), I'm not surprised. --Guido van Rossum (home page: http://www.python.org/~guido/)

guido wrote:
(2) Was the pain worth it, or would you prefer we'd spent more time on being more backwards compatible?
fwiw, I think it would be a really good idea to decouple (large portions of) the standard library from the interpreter core, and keep the same code base running on multiple versions. if not else, it would force the core developers to "share the pain" ;-) </F>

I thought Gordon was saying that most of the pain was to fix code that *accidentally* worked in 1.5.2 but broke in 2.0. Whether or not you fix that in a backwards-compatible way, we've all shared that pain already. Other than sharing the pain, I don't see much of an advantage. --Guido van Rossum (home page: http://www.python.org/~guido/)

On 7 Apr 2002 at 10:20, Guido van Rossum wrote:
[Guido]
Tightening up functions which allowed two params where a tuple was correct. The changes to ConfigParser bit me hard; I think a couple other std lib changes got me, too. In many cases my 1.5.2 code ended up better, so the urge to whine is over pretty quickly. In a broader sense, Unicode is by far the most disruptive change. My excuses for ignoring the damn stuff are disappearing.
I don't have more than a muted grumble about backwards compatibility. Where I end up with checking the version, it's to make use of a new feature, not keep old code working. Recompiling all those extensions is the biggest pain.
Well, the other side of that coin is that I'm still only +0 on list comprehensions and -0 on lexical scoping :-). -- Gordon http://www.mcmillan-inc.com/

Tightening up functions which allowed two params where a tuple was correct.
Yeah, we should have used warnings for that first.
The changes to ConfigParser bit me hard;
I think we were too careless with ConfigParser.
Yeah, Unicode will continue to bite where you least expect it. :-(
Distutils to the rescue?
That's OK. List comprehensions didn't incur any incompatibilities, and lexical scoping incurred only very rare ones (I think). --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum writes:
I think we were too careless with ConfigParser.
I think I was on vacation when ConfigParser was added, and I still remember being quite surprised that it landed in the Python CVS. I'd love to see that module frozen (in the sense of "let's not even *try* to fix it), and come up with a more reasonable API that takes a single syntax (.ini file syntax is fine; let's not invent one). It might be worth looking at the GConf API used in Gnome; I'm not sure how it compares. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation

[Barry]
From my own perspective it seems that 2.1.x is viewed as the stable release family, and each micro release reaffirms its stability.
GvR> That's about right. Maybe we should continue to promote GvR> 2.1.x and relegate 2.2 to the bleeding edge? A simple GvR> rearrangement of the website might be sufficient. That might not be a bad idea. 2.2's got some neat stuff in it, but until it's well documented, and really beat upon, I don't think we can rightly call it stable. From my own perspective, while it's gotten pretty hard to continue to support Python 2.0 (nested scopes, warnings, weak refs, the improved build process, and lib fixes), Python 2.1 seems like a good baseline.
GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> I think the new bool type has already prevented that. -Barry

BAW> [Barry] >> From my own perspective it seems that 2.1.x is viewed as the stable >> release family, and each micro release reaffirms its stability. GvR> That's about right. Maybe we should continue to promote GvR> 2.1.x and relegate 2.2 to the bleeding edge? A simple GvR> rearrangement of the website might be sufficient. BAW> That might not be a bad idea. 2.2's got some neat stuff in it, but BAW> until it's well documented, and really beat upon, I don't think we BAW> can rightly call it stable. This sounds more or less like the Linux kernel, except the meaning of the odd and even minor revisions are reversed. (Guido: You mentioned before that you have trouble keeping the Linux stable/experimental kernel numbering straight. Just execute "uname -a" and look at the kernel's minor version number. I can almost guarantee you aren't running an experimental kernel. If so, you don't have enough work to do. ;-) Personally, I think the way it's done in the Linux world is fine and other than the initial turmoil caused by switching between the current release scheme and a Linux-like release scheme I think it would probably play well in Peoria (that is, c.l.py). You'd dispense with alphas and betas as they are currently done and use the micro releases for that. Let's assume even-numbered minor releases are experimental, and odd-numbered ones are stable. The early micro releases of experimental versions would be where most of the turmoil takes place. New stuff is added, and maybe ripped back out if it doesn't work. As the micro release numbers increase, the amount of turmoil reduces, the feature set is frozen, and eventually 2.2.N becomes both 2.3.0 and 2.4.0. The 2.3.x branch gets essentially nothing but bug fixes and new development then pours into 2.4.x. The advantage over the current scheme in my mind is that we have a lot of turmoil in the release train near to a release. I think the ideal scenario would be the exact opposite. Today, little activity takes place during the time between one release and the first alpha of the next release. After 2.x.alpha1 is announced, the repository churns a lot. The closer and closer you get to the first beta, the more agitated things seem to become. I think this is because you try to set and adhere to specific release dates and because each release is meant to be "stable". What's Fred Brooks' aphorism about software? "Plan to throw one away." That can be every other set of minor releases. GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> BAW> I think the new bool type has already prevented that. Why? If you postulate that 2.even.x become the experimental release branches, then 2.2.3 with a bool type makes perfect sense. Once the bool type is in and you're satisfied that you've accounted for most of the necessary changes, you make a micro release. No big deal. Create tgz and zip files, maybe a Windows installer. On to the next one. release-early-release-often-ly, y'rs, Skip

"SM" == Skip Montanaro <skip@pobox.com> writes:
GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> BAW> I think the new bool type has already prevented that. SM> Why? If you postulate that 2.even.x become the experimental SM> release branches, then 2.2.3 with a bool type makes perfect SM> sense. Given that interpretation, I'd agree. -Barry

BAW> Given that interpretation, I'd agree. *If* we're going for an even/odd scheme, I wouldn't want to swap the Linux convention, so 2.2 would have to be stable, and 2.3 experimental. Given Alex's comments on 2.2, I don't think it's too far of a stretch to label 2.2 stable after the fact. It could make it easier to start experimenting with stuff in 2.3. I also like the idea of doing away with alpha/beta releases and use micro releases instead. I guess we'd have to issue a press release (we may already be doing one for 2.2.1, we may have to add some words to it). Maybe I should PEP it first? :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Monday 08 April 2002 03:44 pm, Guido van Rossum wrote:
Personally, I'd just _love_ this arrangement. 2.2.* stable, 2.3.* experimental -- and once you have a 2.3.x, for some value of x, that you judge IS worth becoming the next Stable branch, you can make it 2.4.0, or 2.4, or 3.0 -- whatever fits best.
I also like the idea of doing away with alpha/beta releases and use micro releases instead.
Yay for the Experimental branch -- not sure about the Stable branch, but we can think about at at the time 2.4 or whatever is ready; part of the arrangement should IMHO be to change the current commitment to twice-yearly-minor-releases into a "release early, release often" idea for Experimental, "release when needed to get important bug-fixes or backported stuff" for 2.Stable.microrelease -- but 2.(NextStable).0 should be solid when it happens (break compatibility with 2.PreviousStable.* only in specified ways, for example). Maybe that comes for free from 2.4.0 being == 2.3.something -- that depends on how popular the 2.3.* branch becomes, I guess. Hard to predict (offhand, it would seem to me that 2.3.* will be very popular, but I wouldn't be astonished if that proved to not be the case).
Seems an opportunity (to send a nice-sounding message to a certain segment of the development community).
Maybe I should PEP it first? :-)
Why not -- if you don't announce the PEP on c.l.p there shouldn't be too much troublesome bickering about it anyway:-). Alex

Guido> *If* we're going for an even/odd scheme, I wouldn't want to swap Guido> the Linux convention, so 2.2 would have to be stable, and 2.3 Guido> experimental. Yeah, I'd tend to agree. For one thing, all the Linux distributions are used to the Linux even-odd convention. Not confusing them would probably be a good thing. (Hey, how about we skip 2.3 altogether and get in complete version sync with Linux? Maybe we could even convince Linus to do our releases for us. :-) Guido> Maybe I should PEP it first? :-) You've got a smiley there, but some things will obviously have to change about the way releases are managed, so a PEP might not be a bad idea. Has Jack chimed in on this thread from the Mac quarter? Tim, can Windows installers be created by other people? Skip

[Skip]
I think Linus has once said that Python alone was not a reason to reject a certain piece of software; that's a far cry from helping us out. :-)
Guido> Maybe I should PEP it first? :-)
You've got a smiley there,
Because any PEP I write appears to mostly raise controversy once brought up on c.l.py.
but some things will obviously have to change about the way releases are managed, so a PEP might not be a bad idea.
An informational PEP explaining the procedure, yes. A PEP asking the community what they want, heck no. The ones that are still believing that python-dev is a closed list should *definitely* continue to believe so. The fake toll booth is doing its work admirably. :-)
Has Jack chimed in on this thread from the Mac quarter? Tim, can Windows installers be created by other people?
--Guido van Rossum (home page: http://www.python.org/~guido/)

I very much doubt it. If the language is too slick, we'll get complaints about a "marketing PEP" (see PEP 287's fate). --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, Apr 08, 2002, Guido van Rossum wrote:
You may be right, but you misunderstood what I was suggesting: I'm simply talking about looking at the PEP from the perspective of Joe Stupid Pythonista, figuring out where the screams are likely to come from, and clarify the language or write pre-emptive rebuttals. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

And I'm saying that with the current user base, *anything* from me will cause outrage. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido, I would not worry so much about the current user base and their "feelings". People that want to use Python will. You have always led the direction of Python, and your choices have always been good for Python. That is why you are the BDFL. With your guidance and vision for Python, you have kept from turning into another Perl with all the "special case" language "features." There have been many changes people have wanted, that would have damaged the usability of the language even though they think it would have improved it. Most of them are still using Python even though they complained. I would not worry if people are outraged. Let them be. Most of the people that are outraged do not know the history of Python, the previous decisions that have been made. They just want it to be "their language." Continue to guide the development of Python and all will end up well. Lance Ellinghaus

[Lance Ellinghaus, to Guido]
No they haven't. In particular, his decision to accept your goofy rotormodule.c has gotten the PSF listed as a terrorist front by the US Government <wink>.
... Continue to guide the development of Python and all will end up well.
Well, now that I think about it, I'm forced to agree after all. persuaded-against-his-will-ly y'rs - tim

guido wrote:
And I'm saying that with the current user base, *anything* from me will cause outrage.
I don't think the comp.lang.python crowd represents a large part of the current user base... (most python programmers I know don't read any python mailing list or newsgroup -- they have better things to do ;-) </F>

I don't think the comp.lang.python crowd represents a large part of the current user base...
Maybe I should stop posting PEPs to c.l.py, and tell them that if they want a say in the future of the language, they should subscribe to python-dev? --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, Apr 09, 2002, Guido van Rossum wrote:
Do you really want Paul Rubin on this mailing list? How about at least trying my suggestion that wording a PEP differently might make a difference? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

Andrew Kuchling wrote:
PPS: NOOOOOO!!!!! I'd suggest to either leave the discussions on c.l.p or setup a new mailing list (e.g. peps@python.org) for PEP postings. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/

mal> I'd suggest to either leave the discussions on c.l.p ... Hey, there's an idea! Let's only post the PEP announcements to comp.lang.perl. That should solve the nasty problem of negative feedback from comp.lang.python... ;-) Skip

I wasn't serious, but AFAICT /F was serious in his suggestion not to discuss my PEPs on c.l.py any more. I think the consequence of that would be that my PEPs only get discussed on python-dev. Then I see two possibilities: simply ignore the c.l.py crowd, or inviting them. I see serious downsides to each. So that means I'll have to face c.l.py every time I have a PEP, or stop writing PEPs. The latter sounds the most attractive -- I'll just check things in after a brief discussion on python-dev. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, Apr 09, 2002, Guido van Rossum wrote:
Seems to me that you're being unreasonably, er, boolean in your thinking here. From my POV, the fact that the BDFL follows the same PEP process as everyone else is a powerful point in favor of stability. There may not be a way to do it, but I'd sure like to try some things to lessen the difficulty of using c.l.py. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

Seems to me that you're being unreasonably, er, boolean in your thinking here.
Yes, as a way to help move the discussion forward. :-)
From my POV, the fact that the BDFL follows the same PEP process as everyone else is a powerful point in favor of stability.
Even if in practice it causes the peasants to show up at the gates armed with pitchforks?
There may not be a way to do it, but I'd sure like to try some things to lessen the difficulty of using c.l.py.
I'd be happy to run my PEPs by the Aahz PR machine. You might want to remind me of that next time I float a PEP in python-dev, i.e. well before it hits c.l.py. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, Apr 09, 2002, Guido van Rossum wrote:
Perhaps, but let's see if we can dissuade the peasants first.
Added to my Todo list. ;-0) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

[Aahz]
From my POV, the fact that the BDFL follows the same PEP process as everyone else is a powerful point in favor of stability.
[Guido]
Even if in practice it causes the peasants to show up at the gates armed with pitchforks?
Probably, yes: nobody pitching a PEP on c.l.py gets off easy. By the time someone has posted a few revisions of any PEP, they've been through fire. You just get more of everything (both pro and con). Keep that in mind next time you reject a PEP -- the author went through a minor suburb of hell to carry it that far.

guido wrote:
I wasn't serious, but AFAICT /F was serious in his suggestion not to discuss my PEPs on c.l.py any more.
not sure that was what I suggested, really -- my point is that you cannot treat c.l.py as a representative subset of the current user base. most people are elsewhere, and most users won't see the PEP at all, no matter where you post it. </F>

But what am I to do? c.l.py may not be representative, but neither is python-dev, and I'd like to get feedback from as many folks as possible. Maybe it's just a matter of wording -- make it clear that the feedback won't necessarily affect the PEP unless I like what's being said. But that will still cause certain folks to bitch. Sigh. --Guido van Rossum (home page: http://www.python.org/~guido/)

>> most people are elsewhere, and most users won't see the >> PEP at all, no matter where you post it. Guido> But what am I to do? I propose: 1. Do most/all of the serious discussion on python-dev. 2. When http://www.python.org/peps/ changes, post a simple announcement just to c.l.py.announce that such-and-such a PEP was updated or is new. If people are that interested in tracking PEPs, there is the above URL after all. They can register with one of those URL notifier services if they are that intent. Since python.org doesn't host ads, there should be no problem with spurious notifications. I sure a little watcher script for #2 wouldn't be hard to write. Skip

From: Guido van Rossum <guido@python.org>
I have the impression (at least for 285) that you got enough feeback on the technical points from python-dev. For the rest the important aspects are: - breakage you typically know what a change will breaks, OTOH you might not know how much code will be concretely affected, for that you could use the experimental track to get feedback from people, admitting that they will try things out and report honestly. But my very personal impression is that you prefer a bit asymptotic language purity over avoiding breakage, so you should expect some over-violent reactions. - newbie friendliness and teaching: I'm happy I am not the BDFL of a self-declared newbie-friendly language <wink>. regards.

Guido van Rossum <guido@python.org> spake:
There are always going to be people who will bitch, because they enjoy bitching. There are always going to be people who will take an instant dislike to any change, because they fear change. There are always going to be people who will automatically reject any PEP you propose, *because* you created Python - I don't understand this behavior at all, but you see it in many fields where the "fans" hate anything new from an artist or writer or inventor. There is nothing you can do to fix them. Don't sweat it, and please keep posting your PEPs where as many people as possible can see and discuss them - most of us have a serious need to know what's coming down the line, and at least some voice in that process. -- <a href="http://kuoi.asui.uidaho.edu/~kamikaze/"> Mark Hughes </a>

On Tue, Apr 09, 2002, Gustavo Niemeyer wrote:
Yup, that's the next step IMO, if editing the PEPs doesn't work. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

Guido van Rossum wrote:
The latter pretty much defeats the whole purpose of PEPs. I wouldn't want to get c.l.p style discussions on python-dev about PEPs, though. Why not setup a special PEP discussion list and then invite people from c.l.p over to it for additional insights ? Regarding the subject line: don't know if it's just me, but I would like to see some of the conservative development style we had established a few years ago return in Python's development process. Some of the recent developments left me under the impression of the need to rush changes with no apparent reason (for rushing them). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/

"MAL" == M <mal@lemburg.com> writes:
MAL> I wouldn't want to get c.l.p style discussions on python-dev MAL> about PEPs, though. Why not setup a special PEP discussion MAL> list and then invite people from c.l.p over to it for MAL> additional insights ? I thought we rejected this a week or so ago. :) But if that's what people want, it's easy to do. -Barry

On Tue, Apr 09, 2002, M.-A. Lemburg wrote:
Or, to put it another way, perhaps before development work for each release starts (aside from bug fixes), we draw up a list of the feature goals for that release and project a target date for finishing those goals, but the goals get the focus rather than the date. Changing goals after development starts would require official BDFL prounouncement. If a great feature comes up after development starts, too bad -- the next development cycle will usually be less than nine months away. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

"A" == aahz <aahz@pythoncraft.com> writes:
A> On Tue, Apr 09, 2002, M.-A. Lemburg wrote:
A> Or, to put it another way, perhaps before development work for A> each release starts (aside from bug fixes), we draw up a list of A> the feature goals for that release and project a target date for A> finishing those goals, but the goals get the focus rather than A> the date. Changing goals after development starts would require A> official BDFL prounouncement. That's pretty much what we do now, isn't it? We consider both a set of functional goals and a release schedule. We don't want to make these decisions in isolation. If there are N features in the release, and all but 1 can be finished in six months, we don't want to hold them all up for one feature that takes two years. So we say the next release will have N-1 features, and the other feature will go in a future release. A> If a great feature comes up after development starts, too bad -- A> the next development cycle will usually be less than nine months A> away. The only person sneaking in new features is Guido <0.2 wink>. Jeremy

Jeremy Hylton writes:
We consider both a set of functional goals and a release schedule. We don't want to make these decisions in isolation. If there are N
That's right. But we definately determine the release schedule without any regard for what else is on our schedule, and we don't make changes to the schedule based on what gets added to our schedule before we're done. Or that's been my perception at least. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation

"FLD" == Fred L Drake, <fdrake@acm.org> writes:
FLD> Jeremy Hylton writes:
FLD> That's right. But we definately determine the release schedule FLD> without any regard for what else is on our schedule, and we FLD> don't make changes to the schedule based on what gets added to FLD> our schedule before we're done. FLD> Or that's been my perception at least. We chose a release schedule a few weeks or months after the previous release, and it always slides. So I think we do adjust the schedule to some extent. Did we add an extra alpha or beta release of the type-class unification? On the other hand, we tend to resist too much slippage after the schedule has been set. There was definitely a lot of pressure to get the __future__ stuff for nested_scopes into 2.1 without causing the schedule to be delayed :-O. Jeremy

[David Ascher]
Or at least deadlines. Note that this ties in with frequent releases: if we said we won't release something "until it's ready", and the time between releases slobbers out to over a year again, then people will fight like hell to get "just one more" change in -- over and over and over again. Almost nothing is ever really finished. Frequent releases make it much easier to sell "too bad, you're too late for this release, but you'll get another chance in 6 months".

A> Or, to put it another way, perhaps before development work for each A> release starts (aside from bug fixes), we draw up a list of the A> feature goals for that release ... Jeremy> That's pretty much what we do now, isn't it? Hmm... I must have missed that PEP. There is a section in the 2.3 Release Schedule entitled "Planned features for 2.3". Is that what you're referring to? I would toss the time information altogether and rename the PEP "Planned Features for 2.x". I just haven't seen enough justification for identifying specific release dates, particularly early on, before new code has been added to the repository and had some testing. We may well be talking more about perception than reality, but it seems to me that placing fairly artificial time limits on releases can only serve to reinforce the perception of instability. Skip

If a great feature comes up after development starts, too bad -- the next development cycle will usually be less than nine months away.
Bah. For most features, 9 months is an eternity compared to the time it takes to code it. About the only exceptions I recall are new-style classes and Unicode; major packages like email or xml also are exceptions, but they are usually developed outside the Python CVS tree first anyway. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, Apr 09, 2002, Guido van Rossum wrote:
It's not the time for coding that's the issue, it's the time for testing, documentation, integration, and so on. But even that's beside the point; this suggestion is being brought up in the context of the stability/change thread, and this suggestion would IMO go a long way toward changing perceptions. For example, what's the cost in postponing the bool() change to 2.4? (I'm not talking about adding the builtin the way you just did for 2.2.1 (which I agree with), but changing the type of "not x".) Note that I'm not pushing this suggestion, but I think your "Bah" is overstating the opposition. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

[Aahz]
If a great feature comes up after development starts, too bad -- the next development cycle will usually be less than nine months away.
[me]
[Aahz]
It depends a lot on the impact of the feature. If something like list comprehensions came up at the last moment, I would agree to put it off. But something like sys._getframe() shouldn't need to wait 9 months, no matter *how* conservative you want to be. I don't think you can give hard and fast rules here.
I disagree with your suggestion to put off bool() to 2.4. I see no technical reason to do so, and I don't think politics should influence release decisions to this extent. --Guido van Rossum (home page: http://www.python.org/~guido/)

mal> Regarding the subject line: don't know if it's just me, but I would mal> like to see some of the conservative development style we had mal> established a few years ago return in Python's development process. mal> Some of the recent developments left me under the impression of the mal> need to rush changes with no apparent reason (for rushing them). I think this is a side-effect of having a largely time-based instead of goal-based release process. With a goal-based release process you can effectively say: We will sell no wine before its time. - Orson Welles, in a Paul Masson ad from my childhood Skip

[Guido]
Because any PEP I write appears to mostly raise controversy once brought up on c.l.py.
This one was partly my fault: when I announced your PEP Parade essay, I added Note that Guido wants to keep 2.3 free of new core language features. at the start of the last paragraph. I did so because that's what I believed <wink> I had heard the day before. The next thing the community saw was a PEP from you introducing a new core language feature for 2.3. Some of the early responses specifically mentioned "wtf?! didn't Tim just say Guido wants yadda yadda?". BTW, the reaction to the sentence quoted above was crushingly favorable when it was first posted. The idea that 2.3 was to be a "consolidation" release was very popular. it's-not-bool's-fault-but-the-timing-was-perhaps-sub-optimal-ly y'rs - tim

Sounds like you need to be careful with the channeling there... :-)
Maybe my mistake was thinking that adding bool was a small change. Maybe you could have brought this up when PEP 285 was first circulated on python-dev? I guess you didn't foresee the outcyr either. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

[Tim
I did so because that's what I believed <wink> I had heard the day before.
[Guido]
Sounds like you need to be careful with the channeling there... :-)
It wasn't channeling at the time -- it was very clear the day before, at a PLabs meeting, when I asked what our plans were for 2.3, and all the "new feature" PEPs (at the time) were explicitly ruled out, in favor of consolidation. That you then introduced a "new feature" PEP of your own, and within 9 hours of the PEP Parade post, was utterly unexpected.
... Maybe my mistake was thinking that adding bool was a small change.
It *is* a small change; I think the problem, at least at the start of the brouhaha, was merely that it was *a* change. More substantial complaints surfaced later, so it would have been a tempest regardless.
Maybe you could have brought this up when PEP 285 was first circulated on python-dev? I guess you didn't foresee the outcyr either. :-)
I posted my PEP Parade announcement to Python-Dev: http://mail.python.org/pipermail/python-dev/2002-March/020735.html and tried to tweak you about it before 24 hours had passed: http://mail.python.org/pipermail/python-dev/2002-March/020805.html As I've said before, I was raised in Wisconsin. If someone doesn't get the message the first time, it's too damn cold to stay outside all night long screaming -- and they're probably drunk anyway <wink>.

Guido,
Maybe my mistake was thinking that adding bool was a small change.
I think sometimes you're excessively careful when publishing PEPs. When I first read your post about PEP 285, and read you asking if this should be implemented at all, I knew this would happen. This makes people which have no idea what they're talking about feel like they can influence something. Have a look at the ACCEPTED thread, and compare it to the first thread. In the second thread, you were decided to implement it. I'm not saying you shouldn't care about anyone's opinion, I just think you should publish PEPs as accepted, even if for some good reason, you fallback your idea (that's what really happens, anyway). -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]

[me]
Maybe my mistake was thinking that adding bool was a small change.
[Gustavo]
I think sometimes you're excessively careful when publishing PEPs.
:-)
Good idea. Though given the history, I don't think the outrage will be any worse next time. Some people just like to rally against "dictators". :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

[Skip Montanaro]
... Tim, can Windows installers be created by other people?
By anyone with MSVC 6 and Wise 8.something. Few people have the latter. They need also to be careful that the specific versions of the MS components they ship (like msvcrt.dll) are officially blessed by MS for redistribution (e.g., the system's msvcrt.dll on my Win98SE box is, but the one I picked up on my Win2K box via a Win2K-specific service pack is not). I called for volunteers on c.l.py more than once to complete the start at an InnoSetup installer I checked in (InnoSetup is free-as-in-beer), but got no responses.

Guido, As I see it:- - you're not going to please everybody. - you're not even going to please most people. - you'll be lucky to please a good proportion (say a quarter to a third) of your active user (as in developers writing Python apps) base. All I think you'll be able to do:- - try and make your development plans and release functionality goals more explicit and widely known; - use your limited development resources in much the same way as you currently do; - make a decision as to whether fixing warts at the cost of backward compatibility is better than preserving warts in perpetuity. [long-winded rambling attempt to support above statements...] It all comes down to perception. Release version numbers are both a blessing and a curse, which is why Microsoft has tried desperately to find some other code to identify releases without implying functional performance or API compatibility. There is this perception that a "minor" release number has a specific meaning, and a "patch" or "fix" release number has an even more specific meaning. We even have a PEP spelling it out. In the scheme of things, you may have had a somewhat different response had the following version numbers been used instead: 2.0 => 2.0 2.0.1 => 2.1 2.1 => 2.5 2.1.1 => 2.6 2.1.2 => 2.7 2.1.3 => 2.8 2.2 => 3.0 2.2.1 => 3.1 2.3 => 4.0 (or perhaps 3.5) I'm not saying the response would be more favourable, just different, as different people would read different things from this different set of tea-leaves. Part of the perception problem that I think Paul Rubin and others are exhibiting symptoms of is the implied development road-map associated with version numbering. A corporate software development effort has to take a longish time to consolidate its development plans, and match them up with code developed and bugs/misfeatures/limitations resolved. Hence a 2-3 year major release cycle, with the odd bug-fix release to pacify those on maintenance contracts. People steeped in this sort of corporate ecology find this stability comforting, and the apparently well defined development roadmap reassuring. Because Python isn't the centre of a commercial development effort, its development isn't as well defined as some might expect. Indeed, as with most open source projects, its developers scratch their own itches because they aren't being paid to scratch someone elses. Python is also subject to much tighter resource constraints than I would expect of any commercial effort to do the same task. Consequently, the development plans are more fluid and the delivery schedule more variable. There is also the sense that completed work should be released for use, rather than being hoarded until it fits in with the "grand plan" of the major release cycle. This is in keeping with the "release early, release often" mantra espoused by ESR as an ethic of open source development - the Bazaar. But its antithetical to a corporate ecology - the Cathedral. On the resourcing topic, despite your offers of the keys to the "stable" branches in CVS to anyone who might wish take them on, the only result has been individuals stepping up and doing one "fix" release and then retiring burnt out. This points to the extreme cost of managing "bug fixes" only releases in the midst of active development. Something those who demand such releases (especially those without a maintenance contract to fund the work) don't and probably won't acknowledge. While Alex's "stable" and "development" branch thinking is attractive, I don't believe that the current or likely future resources can sustain it. I also believe that it would be counterproductive to even try it out, in that the resources expended in the effort are IMHO must unlikely to produce any usable return. There is also the matter of backward compatibility. My experience has been that beyond a certain point, the burden of maintaining backward compatibility raises the cost of further enhancement to unacceptable levels. As I prefer to keep making progress, rather than be forever looking back into the past, I expect that backward compatibility should be balanced against the need for progress, and is therefore not sacrosanct though it should be retained where possible/practical. I've already wasted too much of my own resources on this (not to mention yours or those of anyone who has read all the above), so I'll just close by saying that I admire your juggling act, and have faith in what you're doing with Python, even though there are small (in the scheme of things) quibbles I have with some of your decisions. Regards, Andrew. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia

On Saturday 06 April 2002 05:48 pm, Guido van Rossum wrote:
Right.
Sounds quite workable, as long as we never want to do "bug-fix" point releases for experimental/bleeding-edge versions. The Linux folks do several experimental A.B (with B being odd, meaning experimental).N, but then Linux kernel/modules/drivers ARE an order of magnitude larger than Python, so we may not have analogous needs -- I dunno.
Yes, library changes should be OK, as long as they don't alter behavior of code which used to run (without specific BDFL proclamation and prominent notice). I personally wouldn't mind if some good-intention wording about not breaking existing doctests (e.g. no change in wording of error messages) snuck in, but then I'm a doctest junky.
Yes, public perception issues are to some extent paramount to ensure this slight variation has the intended effect. Your freedom to experiment as well as public perception might be enhanced by some explicit mention that the first (.1) stable release does not guarantee 100% compatibility with the immediately previous bleeding-edge/experimental release, so you can (within some common-sense bounds) try putting something in 2.X and then taking it away in 2.X.1 if it didn't work. I.e., stability being guaranteed 2.X.1 -> 2.X.2 and so on, but not necessarily 2.X -> 2.X.1, just as not necessarily 2.(X-1).N -> 2..X.1. Other folks more adept at marketing than me may come up with other ways to help public perception of 2.X.Y as "good, solid, stable, NOT old, dusty, junky" and thus helping this variant on "dual-track" fulfil its role. Alex

Or we could stay longer in beta.
I don't think so. Doctest is extremely picky, on purpose, and a breaking doctest doesn't mean that regular apps will break.
Good point.
Well, we can probably ride some of the common fear of X.0 releases. (Though we'd continuously have to explain that for Python, 2.X corresponds to X.0 in other vendors' numbering. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido]
How are "experimental" and "stable" defined? 2.2 was as stable as we could make it with 7(!) full pre-releases strung out over half a year. 2.2.1 has fewer bugs, but it would be extremely optimistic to believe more surprises aren't lurking in, e.g., the type/class dark corners. So is 2.2.1 "stable"? Whatever that means, intuitively I doubt it will be *as* stable as 2.1.2. So long as we don't have people testing Python full-time (i.e., quite possibly forever), Python history says relatively few people will bother to try a pre-release, so lots of bugs have no hope of getting caught before an i.j.0 release.
... Or we could stay longer in beta.
I don't think it would help much -- just a few days after initial release of an alpha or beta, downloads go way down. In practice, Python releases get a "street rep" that's not hard to pick up from c.l.py traffic. For example, several people independently recommend 2.1.2 as "the most stable" version of Python currently available, and I expect 2.2.1 will still be viewed as bleeding edge. Since there's no effective way to get wide testing of pre-releases (btw, I don't believe Linux shares this problem), there's no real way to judge a release's perceived stability until after it's been released. This makes a priori stability-number schemes "a problem" for us. Well, instead of calling 2.3 "2.3", we could call it 2.3.1. Then release 2.3.2, 2.3.3, ..., until consensus appears that 2.3.k is the most stable version of Python available. At that point we could re-release 2.3.k under the name 2.3 <0.9 wink>. microsoft-is-still-the-most-successful-software-vendor-in-the-solar- system-ly y'rs - tim

Alex Martelli <aleax@aleax.it> writes:
Is that actually a problem (or, *the* problem)? I.e. if somebody finds that 2.2 is "unstable", does it ever happen that, to solve a problem, something has to be taken back? The typical problems seem more to be in the area of bugs: a bug was fixed, and this fix broke existing code, or a feature was introduced, and that broke existing code. In either case, it is desirable to keep the existing fix, and the new feature, but to find a way to let the old application continue to work. The need to phase out new features that turn out to be a mistake is relatively rare, IMO; instead, there is a stronger desire to phase out old features that have been superceded by newer ones. Or are you just hoping that booleans will be removed in 2.3.1 again :-? Regards, Martin

All that matters is that you have a numbering system and follow it consistently. The python users will figure out what the numbers mean to them. And I guess that 2.2 is not consistent. The changes from 1.5.2 to 2.0 seem to be closer to what happend with 2.1 to 2.2. One measure of the oddness of 2.2 is the lack of 3rd party support. BArry

On Sat, Apr 06, 2002, Guido van Rossum wrote:
I'm kind of in agreement with Tim; I don't think this buys much. Seems more important to me to specify process than stability per se.
What I suggest is that instead of major.minor.bugfix we do major.minor.enhanced.bugfix This way, only major.minor releases can introduce actual language changes, but enhancement releases can pick up library changes. If this sounds at all workable, I recommend issuing a quick BDFL pronouncement changing 2.2.1 to 2.2.0.1. Then what we might try for a 2.2.1 release is making the new boolean type a module that people can start using. I am also a bit amused because an early draft of PEP 6 did try to address this issue, but you and Tim dissuaded me. I think both of you were right back then, but it's still kind of funny to see it picked up a year later. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

"A" == Aahz <aahz@pythoncraft.com> writes:
A> What I suggest is that instead of A> major.minor.bugfix A> we do A> major.minor.enhanced.bugfix A> This way, only major.minor releases can introduce actual A> language changes, but enhancement releases can pick up library A> changes. If this sounds at all workable, I recommend issuing a A> quick BDFL pronouncement changing 2.2.1 to 2.2.0.1. Ug, do we really need more dots? If we're really going to do that then maybe we should give up our biases about changing the first number.

On Sat, Apr 06, 2002, Barry A. Warsaw wrote:
Hmph. Why are you winking? Between new-style classes and the division change, that's actually a sensible suggestion. How about we change "2.3" to "3.1", making 3.0 a retroactive symlink to 2.2. (Okay, so I'm mostly joking here, but I really do think there'd be fewer complaints about 2.2 if it had been billed as a major change.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

[Aahz]
I'm kind of in agreement with Tim; I don't think this buys much. Seems more important to me to specify process than stability per se.
I think our process is impeccable. What's missing is clear guidelines for the users. Maybe Alex is right and the Linux odd/even distinction is clearer than using micro release numbers to indicate bugfix/stable releases.
Ah, yuck. I really don't want to start using more dots. Too confusing.
This way, only major.minor releases can introduce actual language changes, but enhancement releases can pick up library changes.
I don't see how this helps.
If this sounds at all workable, I recommend issuing a quick BDFL pronouncement changing 2.2.1 to 2.2.0.1.
No. :-)
Then what we might try for a 2.2.1 release is making the new boolean type a module that people can start using.
No again. (Banning the new bool type to a separate module is a waste of time.)
We still seem to be disagreeing on how to do it. :-( [Barry]
From my own perspective it seems that 2.1.x is viewed as the stable release family, and each micro release reaffirms its stability.
That's about right. Maybe we should continue to promote 2.1.x and relegate 2.2 to the bleeding edge? A simple rearrangement of the website might be sufficient.
Or maybe 2.3 should become 2.2.3. <0.5 wink> [Martin]
Usually the latter. When fixing a bug breaks code, it can mean two things: (a) the fix was to introduce a feature (sometimes the right thing to do); or (b) the bug was so pervasive that work-arounds became widespread, and the fix breaks the work-around. Both are relatively rare (but have happened). But new features regularly break existing code: either by introducing a new keyword (e.g. yield), or by changing results in certain situations (e.g. str(1==1), or nested scopes), or by taking away old crutches (e.g. import * inside a function). There are many more examples, despite the fact that in each of these cases we've tried to do our best to minimize breakage.
I would think that the best way to keep old code running is to keep the old interpreter around. Maybe one problem is that we've been installing each new version as "python" by default? Even on Windows that can be made to work, as long as you find a way to start your program with a specific Python version: you can't use #!/usr/local/bin/python2.2 and double clicking .py will invoke the most recently installed version. (Hm, would it be totally outrageous to let Python itself on Windows check for a #! line, and when there is one that names an existing Python interpreter that's not the current one, simply exec that one on top of the current one???)
IMO the most common breakage is when a new feature uses up a tiny portion of some namespace that used to belong to the application.
Or are you just hoping that booleans will be removed in 2.3.1 again :-?
Fat chance. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

I just thought of another issue. James Logajan (possibly our most outspoken conservative developer) brought up the problem that whenever you install a new version, you start developing for that version, and your code is no longer portable to previous versions. Even if we had a perfect track record of keeping old code working, the addition of new, enticing features has a "lock-in" effect. (We may sometimes have done this on purpose, at least semi-consciously.) This would suggest designating each series 2.X, 2.X.1, 2.X.2, 2.X.3, ... as an asymptotic series approaching perfection for a given feature set, where the feature set is pretty much frozen the moment 2.X is released. Then user code portability within the 2.X series is close to guaranteed, making it easy to move a script to a previous version, and if that doesn't work because the script relies on some bug being fixed, there shouldn't be much resistance to upgrading 2.X.Y to 2.X.(Y+1). We have to commit to maintaining a 2.X series as long as it is in use. I don't think I want to chop off the leading "2." -- since as we're releasing new versions every 6-12 months, this would move us to 9.0 way too quickly. It shouldn't be too hard for users to get used to the fact that the jump from 2.X to 2.(X+1) is usually a big one (despite comments that the jump from 2.1 to 2.2 was as big as the one from 1.5.2 to 2.0 -- after all, so was the jump from 1.5.1 to 1.5.2, so we're already doing better. :-) I'm not sure what exactly we need to *do* now. Be clearer to the user community that we're still maintaining 2.1.*? Delay the 2.3 release and spend more time on 2.2.2 (assuming that 2.2.1 is about to go out)? Change the Makefile and the RPMs to install each 2.X version under a different name, e.g. only install as python2.2?[*] Rearrange the website to emphasize the stable version? While I agree that 2.1.x is the most stable release, there's also a significant user population who want to use 2.2 (e.g. the new Boost library wrappers use new-style classes). We should maintain the 2.2.x line for their benefit. And then maybe we could take our time releasing 2.3 while experimenting with new features to our heart's content. To counteract the fact (as Tim noted) that few users bother to download betas, maybe we could release 2.3 relatively rough, clearly mark it as an experimental release, and work on improvements and stability in 2.3.1, 2.3.2 and so on, until we're happy to call it stable and start experimenting with 2.4. While we're working the kinks out of 2.3.x, the most stable release advertised would be 2.1.x, and adventurous users could basically choose between 2.2.x (advanced but stable) or 2.3.x (bleeding edge). Compared to the Linux numbering scheme, we'd lose the "mnemonic" value (if you can call it that) of even-stable / odd-experimental; instead, we'd have to give each release a color, maybe starting with red (bleeding edge) going via yellow (advanced but stable) to green (most stable), and finally to blue (behind the pack) and black (dead). ----- [*] Here's an idea. Maybe "make install" should check if there's already a $(prefix)/bin/python that's got a different major.minor version number than the one it's about to install, and then *ask* whether to make a link named "python" to the new version rather than just doing it (unless you specified "make altinstall", which probably too few people know about). --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sat, Apr 06, 2002, Guido van Rossum wrote:
First of all, I agree with the idea of better management of multiple Python versions. But before we can start being clearer to the community, we need to be clearer to ourselves. Historically, there has been an attitude of, um, antipathetic neutrality toward the idea of maintaining old versions of Python. Until today, I was under the impression that your official attitude (as usually channeled by Tim) was that creating patch releases was something that PythonLabs would assist volunteers to accomplish, but would not push for in any way. It sounds to me like your attitude has in fact changed. If so, then I think the first step is to say so explicitly. After that, I'd like a clear statement of the new goalset for Python development. Once we have that, I think it should be relatively straightforward to create a process to support the new goals. Perhaps the whole PythonLabs crew should have an in-person brainstorming session to kickstart the goalmaking process? What's Zope doing to handle this issue? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

Perhaps the whole PythonLabs crew should have an in-person brainstorming session to kickstart the goalmaking process?
Why should this be PythonLabs' internal brainstorm? We may be the only ones who get paid directly to maintain Python (part time, the rest is Zope work!), but we're not its only maintainers. This concerns us (active developers) all, because we're all limited resources.
What's Zope doing to handle this issue?
Assuming you mean "how is change in the Zope software handled" (rather than "how does Zope Corporation handle change in Python"), they have a rather different approach. There's an effort to create a Zope 3 which is a total rewrite, (almost) completely from scratch. There's an explicit compatibility requirement for the end result, but that's only needed in the final phase -- while it's being created (with lots of help from the Zope community) it can do whatever it wants. In the mean time, various previous Zope releases are still being maintained: I believe that both 2.4 and 2.5 are under active maintenance, meaning that micro releases are occasionally issued, and there's an effort underway to create a 2.6 release (again with community help). While there's a striving for easy upgrading from 2.x to 2.(x+1), each new one (2.(x+1)) adds significant new features; I believe strict compatibility between 2.x and 2.(x+1) is not a goal (except at the storage level -- the database contents are sacred). I believe that it's very common for 3rd party products written against 2.x to require significant porting effort to work with 2.(x+1). The www.zope.org site for example runs a very old version of Zope (I believe 2.3) because it's encrusted with much custom code that make an upgrade difficult. Instead, there's a completely new version in the works (new.zope.org). --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido]
I just thought of another issue. James Logajan (possibly our most outspoken conservative developer)
Ex-developer: he's made it quite clear that he no longer intends to use Python at all (after some brief current period of "compatibilizing" his old code).
I must be exceptional -- this wasn't news to me <wink>.
I've done it with full awareness in my own code.
If this differs from what we're doing today, I don't understand how: "no incompatibilities and new no features!" has been strongly observed in the bugfix releases.
Again this *appears* to be arguing for what we already do. Maybe with one difference: I assure people on c.l.py that PLabs will continue distributing I.J.K for increasing K for as long as the community contributes the work for bugfix releases. Perhaps you're saying that PLabs should take that over? Note too that it wouldn't make James Logajan one whit happier, if we're to take him at his word. He developed some scripts for 1.5.2, and that's it. He wants them to work forever under all future Python releases and all platforms, and got particularly burned by Unicode's interaction with how Tk deals with UTF-8 (and a lot of people got burned by that).
Are we? We've made no effort to backport any bugfixes into 2.1.3, and to the contrary declared that we wouldn't do anything with 2.1.3 except fix the single gc-vs-trashcan disaster. We're not even building a Windows installer for 2.1.3 (last I heard). I don't think anyone is going to buy that as "maintaining" the 2.1.x series; I sure don't.
Delay the 2.3 release and spend more time on 2.2.2 (assuming that 2.2.1 is about to go out)?
2.2 did go out the door unfinished <wink>. But, again, we could have delayed it two years and it still would have been relatively unstable: there's just no substitute for getting thousands of users *using* a release.
You can fiddle what you *call* what we're already doing anything you want, but if a long-lived two-branch comes out of this, I quit <wink>. I do believe we should direct more effort into bugfix releases; that can't be done without taking away from something else, though (that's a trivial deduction: everyone is already overloaded).

No, this is for all active developers. I agree that we don't have to change what we do very much, but I'm looking for a way to "spin" it to satisfy the more conservative user community (the ones who still like Python but are worried about the future). Since there are different "use cases", this may be hard or impossible, or we may have to explain it differently for different folks. For example, developers who distribute useful Python programs want to have a way to ensure that code they write gets run on a compatible Python version. We could make it easier to have different X.Y releases installed on the same system; this is already pretty easy on Unix if you know about "make altinstall". If we invent a standard way to specify the required version (or range of versions?) then that might be sufficient. You shot down my #! proposal for Windows, but maybe we can do something else. (Obviously this only works for programs, not for modules or packages. Package authors pretty much know the drill.) Beginning programmers have a different concern: they don't know much about software evolution, Python may be the first or second language they learn, and they hear that there's a concern about the rate of change. How can they know what that means? I've seen plenty of posts from such people worrying that in a few years from now they might have to learn a whole new language. They needn't worry, but it needs to be addressed with more explanation than "don't worry, the BDFL is watching over you."
Maybe we need to use a different word? Barry noted that 2.1.x is the most popular "stable" release and will probably remain that. So maybe we should say this is the "recommended" release. And then maybe we need to revise the decision not to make a Windows installer. Or maybe we *do* need to spend more effort backporting patches (not necessarily PLabs, but *somebody*). Another way to keep adding value to older releases is by backporting the build procedure enhancements. Once we've figured out how to build Python 2.3 on, say, some ancient version of AIX, maybe somebody should port that back to 2.2 and 2.1. That's not a language change, but still improves the usability of the release. (For some folks. Maybe the AIX user community is small enough that we shouldn't care, but there are other platforms with similar issues. I recall that you backported the new 32 bit Windows installer to 2.1.2; maybe the recent fixes could be backported to 2.1.3.) OK, so I *am* proposing to do something different. Maybe it's not very attractive to work on old releases. But Zope Corp does it all the time (and the fact that Zope 2.x requires Python 2.1.x may be an additional motivator).
Indeed.
Would you settle for a separate CVS repository? :-)
Yup. That's why we need to put some effort in communication to the users, and some in a real change of priorities. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Tim]
Perhaps you're saying that PLabs should take that over?
[Guido]
No, this is for all active developers.
How does that differ from PLabs? There are only a few active developers outside of PLabs who aren't "one issue" developers, and I wouldn't consider even *asking* Martin or Michael to take on more than they already do. If a new plan involves docs, it necessarily sucks Fred in; if a new plan involves the PLabs Windows installer, it necessarily sucks me in; and new plan or old, you're hopelessly sucked in <wink>. The sum of all active developers isn't enough to tackle anything truly new.
If they write a Python script for 1.5.2, and it doesn't work in Python 58.7.9, they're unhappy. I'm thinking of Logajan, Cave and Rubin there, as they're complaining the loudest these days. They can't be spun into accepting that any backward incompatibility is defensible. I assume almost all of that would go away if incompatible language changes also changed the extensions used for Python files (like, say, .py3 for Python 3).
Since there are different "use cases", this may be hard or impossible, or we may have to explain it differently for different folks.
A possibility I haven't rejected is give up spin and just acknowledge that Python can't be all things to all people here: some people are going to lose. Unfortunate, but that's life if so.
That's not what the people above are saying, though: they want all future Pythons to be compatible with everything they wrote 4 years ago. If a reasonable <wink> developer appeared who asked for what you're suggesting, I'd like to try to help them.
I have at least 4 versions of Python installed on each of my Windows boxes, but there are limits to how deep we can push that (e.g., a given user can associate .py with at most one interpreter, and before Win2K that was at most one association per machine; *I* don't care, because I never use any Windows gimmick that looks at the .py association; if I did care, I'd be screwed, and because of how Windows works, not because of how Python works).
I didn't see that it addressed a real problem.
There are lots of ways, but I don't have practical experience with any of them as I'm not change-aversive. Possibilities include specifying minimal version numbers required of the language and/or of specific modules at import sites, relying on a strict backward compatibility guarantee to ensure that any later version will also work OK; and keeping all old versions of all modules around in all future releases, viewing specified version numbers as upper bounds instead of lower bounds. Another possibility is to work in terms of versioned interfaces, and rather than import a module directly, ask the system for any module object that's compatible with a specified version of a specified interface. ...
I don't think anyone is going to buy that as "maintaining" the 2.1.x series; I sure don't.
Who? Anthony Baxter withdrew from the 2.1.x line after 2.1.2 -- he was tired of doing it. He popped up again for 2.1.3 under the assurance that it was going to be a trivial effort. I have a real problem when a group of people demands something but won't volunteer anything to make it happen except their complaints; if they were paying us, fine, but they're not offering funding either.
Another way to keep adding value to older releases is by backporting the build procedure enhancements.
Isn't this dangerous? Build config seems very delicate to me.
Once we've figured out how to build Python 2.3 on, say, some ancient version of AIX, maybe somebody should port that back to 2.2 and 2.1.
If they can swear it's not going to break some other Unix mutant, maybe.
I was actually reluctant to check those into 2.2.1: the complaint came from an XP user, and I don't have an XP box to test it on. I believe I fixed a *similar* complaint on my Win2K box, but who knows whether that fixes it for XP too? I don't; I can only guess. Ditto for its effects on ME -- I'm using installer settings I never tried before, and I can't know whether there may be a glitch on ME. It seems to work OK on Win98SE too, so I took a calculated risk and checked it in to 2.2.1. But I won't be shocked if it breaks Windows installation on some version of Windows I don't have.
If you haven't noticed yet <wink>, you don't control a fraction of the resource a project of Python's complexity requires. It's harder to maintain old Python releases than it is to maintain old Zope releases, because Python is much closer to the metal: testing on Linux doesn't say much about what will happen on HP-UX or Solaris; testing on Windows doesn't say much about what will happen on Macs; and so on. Testing a x-platform product requires a lot of warm bodies and a lot of time. Without more volunteers, I don't see it happening. That said, I'll be happy to build a Windows installer for 2.1.3. It is, quite literally, the least I could do <wink>.

GvR> Maybe we need to use a different word? Barry noted that GvR> 2.1.x is the most popular "stable" release and will probably GvR> remain that. So maybe we should say this is the GvR> "recommended" release. And then maybe we need to revise the GvR> decision not to make a Windows installer. Or maybe we *do* GvR> need to spend more effort backporting patches (not GvR> necessarily PLabs, but *somebody*). I think those are all good ideas that help promote the idea of a long-lived stable baseline release. GvR> OK, so I *am* proposing to do something different. Maybe GvR> it's not very attractive to work on old releases. But Zope GvR> Corp does it all the time (and the fact that Zope 2.x GvR> requires Python 2.1.x may be an additional motivator). Note that Jim often brings up his opinion that the /only/ way to guarantee Zope Z.Y.Z works on Python X.Y.Z is to distribute Python with Zope. (This harkens back to JWZ's much earlier opinion that Netscape could never ship using dynamic libraries, because you really can't trust anything about the end user's system). We're really touching on much deeper flaws in software development, with no good answers, IMO. we-fear-change-ly y'rs, -Barry

[Barry A. Warsaw]
And note that we ship Microsoft's C libraries with Python on Windows, to insulate users from x-release incompatibilities in MS's C libraries. Whether it actually helps Python's users varies across Windows flavors, and also across vagaries of the user's installation and configuration settings. But I think it's important to note that in recent MS OSes, they're trying to make it *easier* for apps to force use of the specific DLLs they ship with (although our Python installer isn't yet doing any of the things necessary to exploit these new capabilities). Don't think Linux is immune either. Upgrading gcc or glibc can "suddenly break" programs that used to work fine. God has a sense of humor about this, though: He usually arranges that breakage show up first in obscure libm endcases <wink>.
Indeed so! Python's not the only thing that changes in incompatible ways over time: *everything* does, including the compilers, system libraries, system tools, and OS policies. The first textbook I had on mathematical logic was Stephen Kleene's (yes, that Kleene) cheerfully named "Mathematical Logic". The first chapter contained a warning to the student: It will be very important as we proceed to keep in mind this distinction between the logic we are studying (the object logic) and our use of logic in studying it (the observer's logic). To any student who is not ready to do so, we suggest that he close the book now, and pick some other subject instead, such as acrostics or beekeeping. For this reason, I suggest naming the hypothetical "forward and backward compatible forever more" Python branch the "Python Acrostics and Beekeeping" branch. you-can't-spell-pablum-without-pab-ly y'rs - tim

"TP" == Tim Peters <tim.one@comcast.net> writes:
TP> Don't think Linux is immune either. Upgrading gcc or glibc TP> can "suddenly break" programs that used to work fine. God has TP> a sense of humor about this, though: He usually arranges that TP> breakage show up first in obscure libm endcases <wink>. Or upgrade a supposedly backwards compatible library, like, oh I dunno, libz? >> ... We're really touching on much deeper flaws in software >> development, with no good answers, IMO. TP> Indeed so! Python's not the only thing that changes in TP> incompatible ways over time: *everything* does, including the TP> compilers, system libraries, system tools, and OS policies. Heck, I wash my pillow and I can't sleep right for weeks. TP> For this reason, I suggest naming the hypothetical "forward TP> and backward compatible forever more" Python branch the TP> "Python Acrostics and Beekeeping" branch. TP> you-can't-spell-pablum-without-pab-ly y'rs - tim Or how about the "Garth Releases" (named after character in Wayne's World who utters the immortal truth "We fear change"). -Barry

But I don't think that's what we do any more. We just tell them to use 2.1.2.
I've started to call it the Logajan paradox: he wants his cake (zero backwards incompatibilities) and eat it too (releases with new features). It can't be done. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido]
I don't believe I ever heard James ask for a new feature, although he has allowed that new features might be OK provided that the Python version introducing them continues to run all older Python scripts without change. Rubin's view is more complicated, and I'm not sure it's been coherently expressed. Note that Logajan's view is hard to distinguish from what most ANSI/ISO language standard committees accept as their burden most of the time: no breakage of programs that conformed under the last standard, period. That's why, e.g., the C99 bool type was added via a pile of macro tricks, and *requires* #include'ing a new header file in any program that wants to use it as intended (the *actual* name of the new type is "_Bool", which is of a syntactic form C89 reserved to the language implementation; indeed, all the symbols Python uses internally of the form _Pxyz are forbidden to us by the std (Python's implementation is non-conforming all over the place); C99 couldn't add "bool" as a new keyword, but they could add _Bool, and then "#define bool _Bool" in a new standard header file; that can't break any existing code, because no existing code could possibly have #include'd the previously non-existent <stdbool.h>; note too that older header files (like <stdio.h>) are forbidden from #include'ing stdbool.h on their own -- or, if they do, they have to hide all evidence of that from users). The ANSI/ISO committees pour person-decades into dreaming up ways to break nothing. It's not a crazy view on its own, but it's extremely expensive to do things that way (I've been on one ANSI committee, and worked closely with two others in previous lives -- it's excruciatingly slow going, and costs participants millions of dollars before it's over). It also requires extreme care in defining what "a conforming program" is, exactly.

There seem to be two groups: 1) Early adopters: release early, release often; more features the better 2) The masses: easy does it, don't break anything, don't release so often I believe we can satisfy both groups--not perfectly, but good enough. This is partly summary and partly suggestion for terminology and schedule. I propose doing: Release name/type Version Schedule ----------------- ------- -------- Major releases 2.4 1.5 - 3 years Bugfix releases 2.2.1 ~ 3 months or as needed Development releases 2.3a0 ~ 3 months Don't get hung up on the version numbers, it doesn't matter. If we agree on the concept, we can name it later. 'Major' releases (roughly corresponding to Linux kernel even releases) would occur every ~ 18-36 months. These releases would be full executable, doc, etc. This seems to be the crux of what many people want. They want a vibrant changing language. But they don't want to have to deal with that change. They want longer cycles. We are talking about users of the language, not hard-core developers. These releases would still go through the alpha, beta, gamma releases. The last development release in a cycle would become the first alpha. Bugfix or point releases (eg, 2.x.y) would be made as needed, ~ 3 months. These releases would still go through the alpha, beta, gamma releases. This release would be concurrent with the development release. Since 2.2 is out there, I would suggest putting the bug fix effort only into it. There doesn't seem to be enough people to share the load for keeping 2.1 in good shape. Development releases, which are source only, could be released approximately every 3 months. These are basically alpha releases. They roughly correspond to Linux kernel odd releases. This would be to satisfy those that want new features or to test compatibility and are willing to build from source. These should be able to be released with very little effort. The benefits of this scheme is that I think it will appeal to a large majority of people regardless of whether they are an early-adopters or not. The only real drawback is that it will take more time to keep up the previous released version. However, I think this can be minimized by having everyone try to backport their work if it's a bug fix and have several people handle the patches for bug fix releases. I would volunteer to help Michael (or whoever rolls 2.2.2), although I don't have enough time to be the One. I don't think the numbering/labeling scheme is important. Does it really matter if we call the next release 2.4, 3.0, 2.3-RELEASE or whatever. I personally think the Linux versioning is the most well known and a bit more normal for most people over BSD. Neal

On Tuesday 09 April 2002 03:45, Neal Norwitz wrote:
I think this identifies the issues very precisely and proposes a good solution strategy.
I fully agree on your concepts. I would suggest some renaming, for clearer/more incisive communication, but you're right, let's not get hang up on that: the substance of your proposal is _excellent_.
Perfect.
Not sure about the source-only part -- maybe there's some hacker wannabe on some Windows box out there that can't afford to buy VC++6. But I could volunteer to remedy that by compiling and making the win32 binaries available (not an installer: I wouldn't know how to write that, so this would have to be arranged) -- so it's not a killer.
OK. I opine there's a little bit of importance in such "naming", but not all that much -- the substance matters more, and I love the substance of your proposal. Alex

Guido van Rossum wrote:
Maybe Alex is right and the Linux odd/even distinction is clearer than using micro release numbers to indicate bugfix/stable releases.
That has not been my impression. I have seen quite a few people confused by the Linux odd/even system. I think the current major.minor.micro system works just fine. I think the discussion should focus on whether more effort should be spent on micro releases. Perhaps 2.2.1 should have been finished before work was started on 2.3. Personally, I don't think so. Neil

[Guido]
No, but how to name an existing interpreter is a problem: people don't install Python in the same locations, so an absolute path wouldn't be portable. At least for the PythonLabs and ActiveState distros, we could use the version number to look up the installation path in the Windows registry (the PLabs and AS installers create registry entries to record this info, under a key path with major.minor in its name, although the PLabs distro has never *used* this info; I believe the PythonWare distro doesn't set any info in the registry).

Absolute paths aren't portable on Unix either, although #! /usr/bin/env python2.2 comes close. I wouldn't mind if users who need this feature would be required to know where their interpreter lives. But I wouldn't mind a registry-based solution either: #! python2.2 could mean "find Python 2.2 in the registry". I believe the PythonWare distro always installs in the same place, so if Fredrik thinks this will be useful, we could provide a hack that looks in certain places on the filesystem if no registry key is found. Still, I'd prefer to do the simplest thing that could possibly work, which IMO would be just an absolute pathname. It makes it *possible* for a specific script to request a specific interpreter. If you want fancy, you can create an installer that figures out the right path and sticks it in the #! line of scripts that need it at install time. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido]
I don't believe any user needs this feature! Not on their own box. Let's be clear about "the problem" (or at least about Logajan's): you want to distribute your Python scripts all over the world. You're targeting Windows, and Unix flavors, and Macs (you hope). Freezing an interpreter into your distro isn't an option, as you have no idea how to even go about finding out how to do that on all the platforms your users run on, and you don't have access to all of them anyway. What then can you do to ensure that your scripts won't break on your *users'* boxes when they happen to upgrade their own Python installations? The only way I see that a "#!" line could do a lick of good for you then is if it were a x-platform way to identify version number, and that's all. It would also need to abort with "This app requires Python I.J" if it couldn't find Python I.J on the box. As a matter of fact, it wouldn't do Logajan any good even if Python 2.3 did a perfect job of that, as he only cares about 1.5.2. Going forward, maybe it would do someone some good. I'm skeptical of the cost/benefit ratio, though (James is extreme, IMO).
If it didn't also mean "find Python 2.2" on Linux and Macs, it wouldn't do anything to help the people who are unhappy in James's way.
It can't possibly work, except to solve a problem I haven't heard anyone bring up. I expect the simplest thing that could possibly work for James's problem would be to add a pragma "#requires 2.3", and teach Python to complain if its own version doesn't match what the pragma says. It would also need some way to do this in .pyc and .pyo files. But that's not really want James wants either: he wants his scripts to run forever without change, under all Pythons that will ever be released, and on all platforms.

On Sunday 07 April 2002 03:18, Guido van Rossum wrote: ...
I agree with Guido. I think the issue is first one of perception by users and potential users of Python. I evangelize for Python on a regular basis, and I notice the issue of _perception_ of stability and change is a significant block for a certain important category of prospective users. Basically, users like my previous employer, think3, inc -- "previous" because, despite it being a great job on many other scores, I grew frustrated enough with being unable to evangelize Python there, that I left in February in order to be able to do Python full time. I had no problem firing up the enthusiasm of programmers and system administrators for using Python in scripting, testing and throw-away tasks -- Python "sells itself" in that way. But management wants to see a strategic picture, one with more perspective -- and it wants to see one of change and evolution within a framework of stability, before it starts OK'ing a given platform for programs with hundreds or thousands of function-points. We had a great opportunity when Microsoft announced the switch of Visual Basic from VB6 to VB.NET, with lots of enhancements that would break just about every existing program. The break with stability was so large, that even managers with a strong pro-MS bias were suddenly quite willing to consider alternatives. I think that if, at that point, I had been able to point at some implementation that was both stable _and_ evolving, I might have gotten the OK to do a pilot project. "All scripts written 2 years ago still run fine, see, but at the same time there is active ongoing maintenance -- bug fixes, new internal speed-ups, new stuff in the library, tools, etc, etc". I couldn't, and I couldn't even indicate such a perspective for the future -- each 2.N release would keep breaking some (little, but some) code of 2.(N-1), while 2.N.M would be constrained to absolutely no new functionality vs 2.N.(M-1), bug-fixes only, assuming there were any such 2.N.M. I'm not sure what they are considering now (besides C++, which dominates there but IS seen as hard and unproductive enough that the search for higher-productivity languages continues) -- maybe Java (that got a horrid reputation there due to its instability in '95/'96, but enough years have passed that they might look at it again now). "Keep the old interpreter around" is not the solution such prospective users want to hear. The choice between being locked off from _all_ enhancements OR having existing code break is what they have with Visual Basic, say -- exactly the unpleasant choice that makes them willing to look around for alternatives. "If you want the bright, new email package, or to say 'for key in somedict:', you also have to accept X and Y risks of existing code breakage -- oh and if you don't so accept it, we'd have to do our own port to the Sharp Zaurus because the existing one only works for 2.2, and ...", etc, etc. To some extent this reminds me of another similar frustration of mine, which was longstanding enough that I had more or less gotten used to it. We used to be a multi-platform shop, then switched to releasing our (mechanical CAD) apps only for Windows OS's, as the wide market started demanding the ability to use cheap, newly-powerful PCs instead of costly proprietary workstations. I kept proposing we do a Linux release as well, but each time the perception of "the platform changing under us" (even for the allegedly-stable 2.0.* series etc) was a big factori in blocking it. More recently I had had better luck with OpenBSD -- far stabler than Windows AND actively maintained -- to the point of doing some minor pilot projects for it. But in this case the problem was that our prospective user base had never heard of it (important when choosing an OS to target, though probably not a language for development) and would not be well placed to install and maintain it (probably true, it does require more skill than for Windows or one of the new, friendly Linux releases such as Mandrake or Suse). Still, the "stability _and_ change" paradigm did once again prove quite attractive to management -- they were at least willing to give OpenBSD a try, as they had not been for Linux.
I'm not sure it's clearer. Clarity may not even matter all that much, as long as the message is heard as "there are two parallel lines of development: one along stable lines, that will break no previously correct code; one experimental, blazing new trails, for innovation and higher-risk changes" or something like that. What I could present today with integrity is more like "All core developers' interest focuses on the mildly-experimental track, constrained but only up to a point; oh yes, there _are_ sometimes bug fix releases to old and stabler versions, but absolutely no new stuff there -- all new things, even those which would break no previously correct code, always go into the mildly-experimental track". That's not the message the prospective users want to hear, I think. Not those I have in mind -- substantial software development shops considering whether to go ahead with Python for very substantial projects or to relegate it to testing/scripting/throw-away stuff but never functionality to be delivered to the customer.
It would help to some degree if the "stable" releases could be shown to have SOME kind of enhancement. Not as much as if language changes which did not break existing correct code could also be there, but some. Modules such as email are a big part of the draw of 2.2, for example (not quite as big as language-level enhancements, but big).
It would let people "return Boolean.True" where they want to do so in new parts of their programs while at the same time not worrying about str(x>y) changing under their feet. I.e., it would provide, or be perceived to provide, change within a framework of stability. Thus it would help us send the message I think prospective users want to hear -- that Python stands for stability AND change, that there is not necessarily a contradiction between the two goals. In a very minor way, to be sure.
More relevantly, I'm not sure I had managed to explain exactly why I think "stability and change" IS important to Python adoption in a certain category of software shops. What numbering or naming scheme we adopt has importance in determining what we communicate, but before that, deciding what we DO want to communicate is probably more important.
I think such a "retroactive" rebranding of 2.2 as "bleeding-edge" would be a serious blow to those of us who have accepted it as stable enough as our base over the last 3 months -- in books, magazine articles, advocacy within our firms and outside of them, etc. For example, AB Strakt has decided 2.2 was stable enough for us (and it was being presented as such) -- our new code now uses "for x in somedict:", "if x in somedict:", etc, for example -- now hearing that it wasn't, and 2.2.X micro-releases might introduce backwards-incompatible changes, would be a real problem indeed.
It would take away my ability to check if old code still works with a new release in a simple way -- by explicitly naming the new interpreter on the command line, whatever is in the shebang line in the script itself.
Indeed -- they won't go. So, stabilizing on 2.2.* (maybe with Booleans in a module, but surely no breakage of e.g. str(x>y)) might be one possibility, if the 2.2.* line was "stable and changing" - introducing enhancements but not ones that break previously correct code. I perceive Booleans as small (albeit negatively), but both our CTO and principal investor have vehemently posted against them, so it would be hard to 'sell' them even if I were quite favorable. Still, if that was the one hurdle to jump to get to a "stable and changing" release, who knows. But if jumping that small hurdle only gets to more of the same, either unstability or feature-frozen versions, the carrot may be too small. Alex

Alex, please try brevity. :-)
Now there's a good proposal. But it costs time and effort that takes away from our main mission (and PythonLabs is already under a lot of pressure). I wonder if there's a commercial market for this? Maybe ActiveState could back-port important packages or language changes and insert them into ActivePython? [proposal to semi-backport bool snipped]
Given the requirements you give elsewhere in your message, I'm not sure how to satisfy all these constraints. Let's be concrete. Should we backport stuff to 2.1 (the earliest release I'm willing to consider)? If so, what? Look at the "What's New?" section in http://www.python.org/2.2/. The new Windows installer has already been backported to 2.1. Type-class unification is obviously out. So are interators and generators (too much effort). So is // division (I think). Returning longs instead of raising OverflowError on int operations would probably break too much code, so it's better not to backport. That leaves the email package and xmlrpclib (both available as separate distros!), and large file support -- which I believe *did* get ported to 2.1.2.
Unfortunately, it's not clear what adaptation in that (apparently very change-resistant) category buys *Python*. IOW why should we, mostly volunteers working for the community, care?
Indeed. But I'd love it if all we need to do is change our communication patterns, because that takes much less effort than fully maintaining two separate release tracks.
I'm happy to hear that 2.2 is stable for you -- exactly as I intended and expected. But what confuses me is that while you're happy with 2.2's stability, you don't want to testify of this fact to the more conservative shops. Rather than telling them "honestly, I cannot recommend 2.2", why don't you tell them "I've used 2.2 for three months and it's as solid as a rock"?
If I didn't suspect that you, the CTI and the P.I. together are 75% of the company, I would say that they are micromanaging you. :-)
I suspect that the people who complain that Python's rate of change is too fast are misguided. There's a lot of fuss on the newsgroup about potentially breaking code and undesired changes, and before you know it, everybody just *believes* that Python is unstable and changes too fast. But if you try it, you find very little incompatibility, and what's there is easily dealt with. The people on the newsgroup who complain that the introduction of bool is such a major change that it would only be acceptable in a major release (i.e. 3.0) vastly overestimate its impact, which is minuscule (zero in most cases). Logajan's main gripe (and also McMillan's biggest effort in keeping his codebase cross-version portable) was about things that changed between 1.5.2 and 2.0 -- that *was* a major release. And at the time we didn't have PEPs 4-6, there was no warning module, and so on. So maybe there's only (or mostly) a problem of *perception*? --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido van Rossum]
I use Python on a few systems and flavours, and Python versions are not the same on all machines. At places, people ask me to limit myself to 1.5.2. At home, I have 2.2, but 2.0 is currently the common denominator for the set of machines on which I do my own projects (I physically travel between sites). For one, I have no real problem aiming 1.5.2 or 2.0 instead of 2.2, when projects or teams require it. Despite I much enjoy most of the new features, I still have to be careful about various development contexts. I would presume that for a newcomer to Python having to ping and pong between different Python versions, not firmly aware of the differences, might feel irritated by things working here, but not there. This is just an hypothesis, as I did not experience such an irritation myself: because when one knows which version has which features, there is no real problem. I guess the difficulty comes from the time evolution takes for new versions of Python to get installed universally, so new features are just there, dependably, blindly, everywhere. This means a few years at least. The difficulty does _not_ come from forward incompatibilities (there were a few around sockets, say, but these were simple things, not really worth complaining about in my opinion), in my experience, forward migration has been either very easy or just fully transparent. The current pace of change does not look unreasonable to me. I wonder if the perception would not improve, if there was a bit more documentation or publicity about what users should do about version inter-operability. If users understand what they choose, they will less likely feel changes as imposed to them. Maybe! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard

On Sunday 07 April 2002 09:38 pm, François Pinard wrote:
I also use 2.0 as the lowest common denominator. Its hard to avoid 1.5.2 (because Red Hat has been shipping with 1.5.x). I can't complain about the differences in Python versions. For the most part my experience has been hassle free. PHP version issues have been more troublesome to me than Python version issues.
There are some who think preserving familiarity between versions is way more important than incremental improvement. In my opinion, sticking to that very idea is a good way to kill and stagnate a language. I believe that maintaing and improving the language *usually* is more important than maintaining consistency between versions. That being said, going overboard with changes isn't good either. I think documenting these changes would be a step in the right direction. My two cents. Good day, ~Mark

[Mark J. Nenadov]
I also use 2.0 as the lowest common denominator.
Linguistic problem :-). Should we say "greatest" instead of "lowest"? Granted that the greatest common denominator is not "greatest" in any other way, but it is lower or equal than any of the things we consider. The real "lowest" common denominator might be very close to nothing, might it not?
Exactly. The changes _are_ documented in detail, indeed, but in special documents which only serious users read when they about to migrate from one version to another. I'm thinking about users who use the language occasionally or even regularly, but not fanatic about following everything about versions -- they mainly rely on the Python Library Reference, or even the Python Language Reference. These references describe some Python, but not necessarily the Python which happens to run on a given machine, and I guess (without having really experienced this myself) it might be frustrating to read and study, for discovering soon after that the feature is unavailable in this version. Or even, for someone, to have handy information about how to write for the common denominator, without having to compare many printings of the references at various Python levels. I guess that notes or footnotes, about Python levels in which described features have been implemented, might help users having to cope with release lags between Linux releases. Such lags are unavoidable whenever Python evolves. I understand the effort it would require to add and maintain such notes in the references (especially for the Library Reference which has myriad of details, the Language Reference is more clearly cut), my hope is that this might remove some of the irritation people seem to show. -- François Pinard http://www.iro.umontreal.ca/~pinard

"Martin v. Loewis" wrote:
GCD = greatest common divisor LCD = lowest common denominator http://www.dictionary.com/cgi-bin/dict.pl?term=gcd&r=67 http://www.dictionary.com/cgi-bin/dict.pl?term=lcd&r=67 Paul Prescod

From: "Paul Prescod" <paul@prescod.net>
See your own link. nitpicking-ly yr's dave

And you believed him? What bullshit. --Guido van Rossum (home page: http://www.python.org/~guido/)

david wrote:
did you check the link? the first entry only expands the abbreviation, the third entry provides the full explanation: least common denominator n. Abbr. lcd The least common multiple of the denominators of a set of fractions: The least common denominator of 1/3 and 1/4 is 12. Also called lowest common denominator.
nitpicking-ly yr's
if you're gonna nitpick, you could at least do it right ;-) </F>

[Martin v. Loewis]
pinard@iro.umontreal.ca (François Pinard) writes:
So, besides arithmetic, what would be the correct expression to replace: "I also use 2.0 as the lowest common denominator." ? P.S. - It's amusing how fast off-topic matters slide even more off topic! :-) The real thread is about how documentation could be amended to (maybe) help many people at being comfortable with Python, despite the rate of change. It is probably worth staying concentrated on the real thread... -- François Pinard http://www.iro.umontreal.ca/~pinard

So, besides arithmetic, what would be the correct expression to replace: "I also use 2.0 as the lowest common denominator." ?
If you want a more accurate mathematical analogy, you should be saying "greatest common factor". Better yet, avoid the cliche altogether and think of a more direct way of saying what you mean. e.g. "I use 2.0 because it supports the largest common feature set." Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

[Greg Ewing]
"I use 2.0 because it supports the largest common feature set."
Nicely put! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard

----- Original Message ----- From: "Martin v. Loewis" <martin@v.loewis.de> To: "François Pinard" <pinard@iro.umontreal.ca> Cc: "Mark J. Nenadov" <mark@freelance-developer.com>; <python-dev@python.org> Sent: Monday, May 27, 2002 2:21 AM Subject: Re: [Python-Dev] Re: Stability and change pinard@iro.umontreal.ca (François Pinard) writes: > > I also use 2.0 as the lowest common denominator. > > Linguistic problem :-). Should we say "greatest" instead of "lowest"? > Granted that the greatest common denominator is not "greatest" in any other > way, but it is lower or equal than any of the things we consider. The real > "lowest" common denominator might be very close to nothing, might it not? For any two natural numbers, the lowest common denominator is 1. Finding the greatest (largest?) common denominator is indeed what involves an algorithm. This terminology came from fractions. The canonical form of a fraction uses the lowest possible number as the denominator. When performing additive operations on fractions, the easiest way to proceed is to convert them to a form with the same number as a denominator. The usual choice is the lowest common denominator, which is the smallest number that can be used as the denominator in both fractions. This is the LCM (least common multiple) of the two denominators. e.g.: 2/3 + 3/8 = 16/24 + 9/24 [24 being the LCD] = 25/24 = 1 1/24 It doesn't therefore (IMO) make much sense to talk about the lowest common denominator of any two natural numbers. The LCM, yes. The LCD, no. It makes even less sense to talk about the lowest common denominator of Python implementations, except in the metaphorical sense Mark (I believe) intended. but-since-we're-being-literal-please-read-this-with-a-straight-face-ly 'rs - steve ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ -----------------------------------------------------------------------

Hm. People who don't read the detailed documents shouldn't expect to rely on details.
Have you read the library manual recently? We are very careful in adding notes about which version added a particular feature or even detail. So I think there is no reason to complain about this preemptively unless your *own* experience indicates there's a lack of documentation. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
People do read the detailed documents, but not again and again every 6-8 months... heck, I work in the Python development team and don't even feel like I am up-to-date with all the changes going on in the core. If not even developers can follow the rate of change, how should a typical Python user feel ? Conclusion: I think that we need a migration guide for Python. This would solve a whole lot of these "rate of change" problems. The guide should complement the additions and changes to the other documentation and provide a single source of knowledge in that area. Currently, the only source we have in this area is the Misc/NEWS file and this doesn't provide any upgrade path hints or porting details. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

Conclusion: I think that we need a migration guide for Python.
I think Andrew's "What's new in Python x.x" documents are a good start. What's missing from them? --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
The migration path, including hints on how to port Python applications from one version to the next, how to work around deprecations, tips on quick ways to find problem areas in the code, etc. Andrew usually puts more focus on new features than changes which could cause older applications to fail -- which is what you'd expect in a document titled "What's new in ..." :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

OK. The question is, who's going to write the migration guide? One problem is that often specific migration issues don't surface until after the release is made. --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (38)
-
Aahz
-
akuchlin@mems-exchange.org
-
Alex Martelli
-
Andrew Kuchling
-
Andrew MacIntyre
-
Anthony Baxter
-
Barry Scott
-
barry@zope.com
-
David Abrahams
-
David Ascher
-
Fred L. Drake, Jr.
-
Fredrik Lundh
-
Geoff Gerrietts
-
Gordon McMillan
-
Greg Ball
-
Greg Ewing
-
Guido van Rossum
-
Gustavo Niemeyer
-
Jack Jansen
-
Jeremy Hylton
-
jeremy@zope.com
-
kamikaze@kuoi.asui.uidaho.edu
-
Lance Ellinghaus
-
M.-A. Lemburg
-
Mark J. Nenadov
-
martin@v.loewis.de
-
Michael Gilfix
-
Neal Norwitz
-
Neil Schemenauer
-
Paul Hughett
-
Paul Prescod
-
pinard@iro.umontreal.ca
-
Samuele Pedroni
-
Skip Montanaro
-
Stephen J. Turnbull
-
Steve Holden
-
Tim Peters
-
Tim Peters