Python-3000 upgrade path

I'm sending this to python-dev instead of python-3000 for two reasons: It's not about features to be added to Python 3.0, and it's not really about 3.0at all -- it's about 2.6 and later. It's about how we get Python 2.x to 3.0, and howmuch of 3.0we put into 2.6 and later. So here at PyCon, Guido did his usual talk about Py3K and how it's going to look. He also covered the tool he's writing to do the necessary syntactic translations, and short bits about how to write code to accommodate the other changes. One thing Guido didn't cover -- because it isn't his area of expertise and (correctly) expects others to step up and do it -- is the upgrade path for third party Python applications and libraries. I happen to care a lot about that, and so do a few other core Python developers, and we *will* make it happen in the smoothest way possible. We could use some community guidance on this, though. Here's my current thinking on the subject (and I talked about this with a couple of different people at the conference and in email): 1) Anything we can backport from Python 3.0 to Python 2.6 without breaking existing code, will be backported. So far, this means all the new features. 2) A few of the things going away in 3.0 will be warned about in 2.6 by default: things that have had better alternative for ages and are generally bad: backticks and input() are the only ones that come to mind right now. 3) The rest of the things that will change or go away in 3.0 *and* have alternatives in 2.6 will be warned about optionally. Neal is rewriting the warnings module to C, which should speed up warning generation a lot, and if necessary we'll use a global 'warn-for-py3k' flag to reduce the overhead for these warnings to the absolute minimum. 4) The new features of 3.0 that we can't backport (things changing semantics, instead of being added or going away) will be made available in 2.6 (as much as possible), using future statements. Right now, I cannot think of a single thing that cannot be made available that way. They will be, in essence, backward-compatibility hacks, but they'll work and the performance impact will be minimal. Of course, 4 is somewhat of a sweeping statement, even with the parenthesised reservation, considering some of the 3.0 features haven't been implemented yet. I'm pretty confident we can do this without comprimising 3.0, though. If we cannot, we might need to add an extra 'migration feature' in Python 2.7, or 2.8 or even 2.9 (but as it is now, I'm not sure that's specifically necessary. We may still want to release Python 2.7 and later for other reasons, of course.) The end result will be that you can write your Python code so it works in 2.6-or-later and 3.0, or -- just like now -- in 2.x-or-later (where 'x' depends on the features you use) but not 3.0. If you write careful code, you may get away with writing 2.2-or-later code that can be run in 3.0 with a single translation run. It will be fairly unlikely you can write code for 2.5-or-earlier *and* 3.0 without using the translation step, unless you live in a world blissfully unaware of anything non-ASCII. In any case, while we aren't working to make that possible, we certainly won't go out of the way to prevent it (but there still won't be any compromises in the 3.0 language just for code compatibility.) As I said, I've talked with a few people about this, both python core developers and people who develop their own software. I would like to hear from people who have concrete doubts about this upgrade path. I don't mean doubts about Python 3.0 -- this mail isn't about 3.0 at all, and I can only suggest you try out the py3k branch when the dubious feature(s) are implemented. I also don't mean doubts about how we're going to keep the performance impact minimal or how we're going to handle dict views and what not. I mean doubts about this procedure of having transitional releases between 2.5 and 3.0, and the *community* impact of 2.6+ and 3.0 this way. One thing in particular I wonder about is the warning about mixing tabs and spaces. Should it be in category 2) (on by default) or 3) (still off by default, but part of -Wpy3k)? -- Thomas Wouters <thomas@python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On 2/25/07, Thomas Wouters <thomas@python.org> wrote:
It's about how we get Python 2.x to 3.0, and howmuch of 3.0 we put into 2.6 and later.
I've also talked to a bunch of people at PyCon, including Thomas. There seems to be much concern (rightfully so!) about the upgrade path from 2.x to 3.x. I don't think it will be easy to navigate, but I have confidence we can do a great job. My goal is to rip out as much cruft from the code base for 3k to make it easier to maintain. In order for users to adopt 3k, it must provide real benefit. In addition, it as painless as possible for third party developers to support both 2.x and 3.x. All the developers I talked to expressed relief that we weren't forgetting about them. There's still unease and will be until we demonstrate what we plan to do. They were satisfied with our general approach. The time schedules in PEP 361 (2.6 release schedule) and what Guido has said for 3k (from what I remember) are roughly: April 2007 - 3.0 PEPs and features accepted/decided June 2007 - 3.0a1 - basic (most) features implemented Dec 2007 - 2.6a1 Apr 2008 - 2.6 final July 2008 - 3.0 final Even if we slip these schedules, as long as we keep them in relative order, we can keep 2.6 robust, while also offering many of the 3k features. n

On 2/25/07, Neil Schemenauer <nas@arctrix.com> wrote:
AFAIK, there hasn't been any work on the new IO system or str/unicode unification. Guido asked for help on these, but I don't know if there were any takers. Sprints will be held over the next several days, so hopefully there will be some work in these areas. n

On 2/25/07, Neal Norwitz <nnorwitz@gmail.com> wrote:
Right. To be honest, I consider the str/unicode unification a much bigger project than the new I/O library. I plan to do the new I/O library first (mostly in Python) first, hopefully at the PyCon sprint, since it can be done with the bytes and unicode objects; the old I/O library will break once we do the unification so the I/O library must be replaced before the unification can be started. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sun, Feb 25, 2007 at 05:37:08PM -0600, Guido van Rossum wrote:
Right. To be honest, I consider the str/unicode unification a much bigger project than the new I/O library.
I was more concerned about IO because it would seem to require your time for design work. The str/unicode work could be farmed out to a bunch of workers. Neil

On 2/25/07, Neil Schemenauer <nas@arctrix.com> wrote:
I was more thinking of pulling a few all-nighters -- seriously, since it needs to be done as quickly as possible once it is started. The number of volunteers who want to do C code is also dwindling... -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Just the "it's not there yet" part :) There's some prototype code and email conversations archived, but no recent work that I'm aware of. On 2/25/07, Neil Schemenauer <nas@arctrix.com> wrote:
-- Thomas Wouters <thomas@python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 25, 2007, at 3:49 PM, Neal Norwitz wrote:
I said semi-jokingly that the first release of Py3k should be / literally/ called Python 3000. Then each successive stabilizing release should drop a zero -- e.g. Python 3000, then Python 300, then Python 30. By the time we get to Python 3 all we should basically have to do is change the defaults of whatever Python 2.x version is out to complete the transition. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBReJgp3EjvBPtnXfVAQIZQgP+K5iWRTYHYvqb3cAUFJw/yDDiz5JPG94x XdMEnCUXwJVyU30q3FGSNeaz3hwQq7xgJuH5DBRHnGkxp7H/K42ft/KXnJVGnkt4 Kai8e26+zBXmCSCTVdJyKhpAiiFAqKTw26+L+a1jyJdSbUnly3coBAvRaOS9oQn6 QcVfx5vCOsM= =o7Ux -----END PGP SIGNATURE-----

Stephen J. Turnbull wrote:
RKN = reverse Knuth numbering?<wink>
No, for RKN you'd have to start with 3141.5926... (an infinite number of digits following) and drop one off the end each time... although it would take rather a long time to get to the final release then. :-( -- Greg

On Sun, 25 Feb 2007 13:12:51 -0800, Thomas Wouters <thomas@python.org> wrote:
I'm pretty tired at the moment, fried from PyCon, and have a lot of work to catch up on, so I'll have to spend a while collecting my thoughts to respond in more depth. However, especially since I've been a vocal proponent of a strategy like this for a while, I would like to say that I'm basically happy with this. Not only that, I'm significantly relieved after PyCon about the difficulty of the migration, and confident that, if my current understanding is correct, Twisted _will_ support 3.0 and beyond. Effectively, what this plan means is that the most *basic* aspects of transitioning to Python 3.0 should work like the transition to any other new Python release. More changes may be necessary, but it should be possible to add a few restrictions to your codebase, and continue to supporting older releases with little trouble. Transitioning to 2.5 was a fairly rocky upgrade for Twisted as well, though, and we made that one reasonably well. The one concern I have is that there are things the source translator won't do correctly. That's fine, but I think there are two important technical features it really needs to make the social aspects of this upgrade work well. 2to3 should take great care _tell_ you when it fails. One concern I have is that the source translation may subtly alter the *semantics* of unit test code, so that the tests are no longer effective and do not provide adequate coverage. I think this is an extremely unlikely edge case, but a very dangerous one in the event that it does happen. I don't know of any bugs in 2to3 that will do this at the moment, but then it's hardly the final release. Also, the limitations of 2to3 should be very well documented, so that when it does tell you how it has failed, it's clear to the developer what changes to make to the *2.6 source* to cause it to translate correctly, not how to fix up the translated 3.0 code. Thanks *very* much to all the python core developers and community members who have been working to make this happen, especially to Neal, Thomas, and Guido for listening intently to all of our concerns at PyCon.

On 2/28/07, glyph@divmod.com <glyph@divmod.com> wrote: [snip]
We're trying hard to avoid semantic changes, and anywhere that semantic changes might be introduced, 2to3 will throw up all kinds of warning messages. 2to3 generally tries to limit itself to safe, syntax-only changes ("apply(f, v, k)" -> "(f)(*v, **k)"), though there are pathological cases where even that could go astray ("d.has_key(k)" -> "k in d", where d.has_key() and d.__contains__ differ). 2to3 should be approached as a conversion guide, not as a converter itself.
This is something I'm working on at the moment, compiling a list of "things 2to3 can't catch". Some are obvious (m = d.has_key; if m(k): ...), others less so ("raise E, V" where V is an exception instance). We're hoping that 2to3 will be able to convert 90% of Python 2.x automatically and offer a solid, step-by-step plan for manually recoding the remaining 10%. Collin Winter

On 2/27/07, glyph@divmod.com <glyph@divmod.com> wrote:
2to3 should take great care _tell_ you when it fails. One concern I have is that the source translation may subtly alter the *semantics* of unit test code, so that the tests are no longer effective and do not provide adequate coverage. I think this is an extremely unlikely edge case, but a very dangerous one in the event that it does happen. I don't know of any bugs in 2to3 that will do this at the moment, but then it's hardly the final release.
Actually, this did happen with the iterkeys -> keys translation. A few unittests were reduced to pure nonsense by this. Most of those failed, but it was definitely a hassle finding out what went wrong (in part because I was too greedy and checked in a bunch of converted code without any review at all). Lesson learned I would say. Fortunately, this is a case where 2.6's Py3k warning mode should compensate.
Also, the limitations of 2to3 should be very well documented, so that when it does tell you how it has failed, it's clear to the developer what changes to make to the *2.6 source* to cause it to translate correctly, not how to fix up the translated 3.0 code.
I'm hoping Collin will continue his excellent work on 2to3. Hopefully he'll get help from others in writing docs aimed at teaching the c.l.py crowd how to use it and what to expect.
Thanks *very* much to all the python core developers and community members who have been working to make this happen, especially to Neal, Thomas, and Guido for listening intently to all of our concerns at PyCon.
You're welcome. I believe I even got a beer out of it. ;-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 10:22 pm, guido@python.org wrote:
Was this due to a bug that can be fixed, or an inherent problem? I could imagine, thanks to Python's dynamism, there would be edge cases that are impossible to detect completely reliably. If that's the case it would be good to at least have pedantic warnings from 2to3 alerting the user to places where translation could possibly be doing something semantically dodgy. Ideally before jumping the chasm with Twisted I'd like to make sure that all of that sort of warning was gone, in _addition_ to a clean test run.
I'm sure he'll hear from me if anything goes wrong with it :).
Well deserved!

On 3/6/07, glyph@divmod.com <glyph@divmod.com> wrote:
Uh-huh. ;-) I wrote two versions of the dict views refactoring. One that turns d.keys() into list(d.keys()) and d.iterkeys() into iter(d.keys()). This one is pretty robust except if you have classes that emulate 2.x-style dicts. But it generates ugly code. So I have a "light-weight" version that leaves d.keys() alone, while turning d.iterkeys() into d.keys(). This generates prettier code but more buggy. I probably should have used the heavy-duty one instead.
Right, this is my hope for 2.6.
Ideally before jumping the chasm with Twisted I'd like to make sure that all of that sort of warning was gone, in _addition_ to a clean test run.
Right. You need both. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

The ugliness is a virtue in this case as it stands out enough to motivate developers to review each case. The pretty/efficient version is tantamount to guessing, and effectively discards information in the transformation ("here be dragons"). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

Absolutely right. I'll withdraw the lightweight version. It's done enough damage. On 3/11/07, Andrew McNamara <andrewm@object-craft.com.au> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 3/6/07, glyph@divmod.com <glyph@divmod.com> wrote:
I've started work on "Capabilities" and "Limitations" sections for the 2to3 README: http://svn.python.org/view/sandbox/trunk/2to3/README?view=markup. I intend for them to provide a comprehensive look at what 2to3 can and can't do. Collin Winter

Thomas Wouters wrote:
developers and people who develop their own software. I would like to hear from people who have concrete doubts about this upgrade path. I don't mean
Disclaimer: I'm not involved in Py3k, and not even tried it once. And don't know the details of the tool to "transform" Py2 to Py3. Having said that, the *only* fear I have, is how safe it will be. And here comes the explanation. I'm now in an enviroment where we rely on Python, and I advocates on it everyday. We use 2.3, and 2.4, some servers we're about to deploy have 2.5, and we'll use the lastest Py2.x everytime we deploy a new machine. But, as we have always running behind our ToDo, we don't have time to say "Ok, this server runs 2.3, how can I start using 2.5?". Too many servers, too many applications, too many undocumented-and-nobody-knows-about-it applications. And they all are applications running 7x24. And Py3k is around the corner, and I even heard some guys saying "If I'd spent time taking care that this app runs ok after changing a bit of it, I'll wait to Python 3000". So, the enviroment is explained, now I go with "how safe it will be". I'd love to know that there'll be a tool that tells me, after running it over my application, Your app is "ready for migration"'. I do *not* care if it takes a lot of work to make the proper changes, but I need to feel confident that after running the last changed version during a week in, say, Py2.7, and no warnings appear, and the "verification tool" say green light, I can start running it in Py3k. And look at it a week more. And say "life is great", :) Anyway, I know how hard is it, and I regret not having the time I'd love to have to help you. All I can do is thank you. Thank you very much!
For me, I'd love to 2.6 start warning "you're mixing tab and spaces, shame yourself!". Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

On 3/5/07, Facundo Batista <facundo@taniquetil.com.ar> wrote:
I know the feeling. Google still uses Python 2.2 for most apps. We're supporting 2.4 as well, and hope to have the last apps migrated to 2.4 in a year or so. *Then* we'll start supporting a higher version -- probably 2.6 by then.
In your kind of env it's clearly too early to think about 3.0. You shouldn't expect to be switching until a year after 3.0 is released, probably.
You need comperhensive unit tests too. Then: *If* you get the green light from 2.6's py3k warning mode, *and* the conversion tool produces syntactically correct code without issuing warnings, *and* your unit tests all pass, *then* I expect you'll be in about the same situation as for any version upgrade in such an environment (e.g. 2.4 -> 2.5). I.e. you need to do a big integration test and once that passes you should be set for a stress-free transition. I think that's the best we can hope for. I plan to do it this way at Google too.
You're welcome!
(The English say "shame on you" :-) While I'd like to do this too I don't know how many folks there are out there who have no way to suppress the warning (because they're end users of a script someone else wrote for them). I think it may be more of a development warning, so I think category is appropriate. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 2/25/07, Thomas Wouters <thomas@python.org> wrote:
It's about how we get Python 2.x to 3.0, and howmuch of 3.0 we put into 2.6 and later.
I've also talked to a bunch of people at PyCon, including Thomas. There seems to be much concern (rightfully so!) about the upgrade path from 2.x to 3.x. I don't think it will be easy to navigate, but I have confidence we can do a great job. My goal is to rip out as much cruft from the code base for 3k to make it easier to maintain. In order for users to adopt 3k, it must provide real benefit. In addition, it as painless as possible for third party developers to support both 2.x and 3.x. All the developers I talked to expressed relief that we weren't forgetting about them. There's still unease and will be until we demonstrate what we plan to do. They were satisfied with our general approach. The time schedules in PEP 361 (2.6 release schedule) and what Guido has said for 3k (from what I remember) are roughly: April 2007 - 3.0 PEPs and features accepted/decided June 2007 - 3.0a1 - basic (most) features implemented Dec 2007 - 2.6a1 Apr 2008 - 2.6 final July 2008 - 3.0 final Even if we slip these schedules, as long as we keep them in relative order, we can keep 2.6 robust, while also offering many of the 3k features. n

On 2/25/07, Neil Schemenauer <nas@arctrix.com> wrote:
AFAIK, there hasn't been any work on the new IO system or str/unicode unification. Guido asked for help on these, but I don't know if there were any takers. Sprints will be held over the next several days, so hopefully there will be some work in these areas. n

On 2/25/07, Neal Norwitz <nnorwitz@gmail.com> wrote:
Right. To be honest, I consider the str/unicode unification a much bigger project than the new I/O library. I plan to do the new I/O library first (mostly in Python) first, hopefully at the PyCon sprint, since it can be done with the bytes and unicode objects; the old I/O library will break once we do the unification so the I/O library must be replaced before the unification can be started. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sun, Feb 25, 2007 at 05:37:08PM -0600, Guido van Rossum wrote:
Right. To be honest, I consider the str/unicode unification a much bigger project than the new I/O library.
I was more concerned about IO because it would seem to require your time for design work. The str/unicode work could be farmed out to a bunch of workers. Neil

On 2/25/07, Neil Schemenauer <nas@arctrix.com> wrote:
I was more thinking of pulling a few all-nighters -- seriously, since it needs to be done as quickly as possible once it is started. The number of volunteers who want to do C code is also dwindling... -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Just the "it's not there yet" part :) There's some prototype code and email conversations archived, but no recent work that I'm aware of. On 2/25/07, Neil Schemenauer <nas@arctrix.com> wrote:
-- Thomas Wouters <thomas@python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 25, 2007, at 3:49 PM, Neal Norwitz wrote:
I said semi-jokingly that the first release of Py3k should be / literally/ called Python 3000. Then each successive stabilizing release should drop a zero -- e.g. Python 3000, then Python 300, then Python 30. By the time we get to Python 3 all we should basically have to do is change the defaults of whatever Python 2.x version is out to complete the transition. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBReJgp3EjvBPtnXfVAQIZQgP+K5iWRTYHYvqb3cAUFJw/yDDiz5JPG94x XdMEnCUXwJVyU30q3FGSNeaz3hwQq7xgJuH5DBRHnGkxp7H/K42ft/KXnJVGnkt4 Kai8e26+zBXmCSCTVdJyKhpAiiFAqKTw26+L+a1jyJdSbUnly3coBAvRaOS9oQn6 QcVfx5vCOsM= =o7Ux -----END PGP SIGNATURE-----

Stephen J. Turnbull wrote:
RKN = reverse Knuth numbering?<wink>
No, for RKN you'd have to start with 3141.5926... (an infinite number of digits following) and drop one off the end each time... although it would take rather a long time to get to the final release then. :-( -- Greg

On Sun, 25 Feb 2007 13:12:51 -0800, Thomas Wouters <thomas@python.org> wrote:
I'm pretty tired at the moment, fried from PyCon, and have a lot of work to catch up on, so I'll have to spend a while collecting my thoughts to respond in more depth. However, especially since I've been a vocal proponent of a strategy like this for a while, I would like to say that I'm basically happy with this. Not only that, I'm significantly relieved after PyCon about the difficulty of the migration, and confident that, if my current understanding is correct, Twisted _will_ support 3.0 and beyond. Effectively, what this plan means is that the most *basic* aspects of transitioning to Python 3.0 should work like the transition to any other new Python release. More changes may be necessary, but it should be possible to add a few restrictions to your codebase, and continue to supporting older releases with little trouble. Transitioning to 2.5 was a fairly rocky upgrade for Twisted as well, though, and we made that one reasonably well. The one concern I have is that there are things the source translator won't do correctly. That's fine, but I think there are two important technical features it really needs to make the social aspects of this upgrade work well. 2to3 should take great care _tell_ you when it fails. One concern I have is that the source translation may subtly alter the *semantics* of unit test code, so that the tests are no longer effective and do not provide adequate coverage. I think this is an extremely unlikely edge case, but a very dangerous one in the event that it does happen. I don't know of any bugs in 2to3 that will do this at the moment, but then it's hardly the final release. Also, the limitations of 2to3 should be very well documented, so that when it does tell you how it has failed, it's clear to the developer what changes to make to the *2.6 source* to cause it to translate correctly, not how to fix up the translated 3.0 code. Thanks *very* much to all the python core developers and community members who have been working to make this happen, especially to Neal, Thomas, and Guido for listening intently to all of our concerns at PyCon.

On 2/28/07, glyph@divmod.com <glyph@divmod.com> wrote: [snip]
We're trying hard to avoid semantic changes, and anywhere that semantic changes might be introduced, 2to3 will throw up all kinds of warning messages. 2to3 generally tries to limit itself to safe, syntax-only changes ("apply(f, v, k)" -> "(f)(*v, **k)"), though there are pathological cases where even that could go astray ("d.has_key(k)" -> "k in d", where d.has_key() and d.__contains__ differ). 2to3 should be approached as a conversion guide, not as a converter itself.
This is something I'm working on at the moment, compiling a list of "things 2to3 can't catch". Some are obvious (m = d.has_key; if m(k): ...), others less so ("raise E, V" where V is an exception instance). We're hoping that 2to3 will be able to convert 90% of Python 2.x automatically and offer a solid, step-by-step plan for manually recoding the remaining 10%. Collin Winter

On 2/27/07, glyph@divmod.com <glyph@divmod.com> wrote:
2to3 should take great care _tell_ you when it fails. One concern I have is that the source translation may subtly alter the *semantics* of unit test code, so that the tests are no longer effective and do not provide adequate coverage. I think this is an extremely unlikely edge case, but a very dangerous one in the event that it does happen. I don't know of any bugs in 2to3 that will do this at the moment, but then it's hardly the final release.
Actually, this did happen with the iterkeys -> keys translation. A few unittests were reduced to pure nonsense by this. Most of those failed, but it was definitely a hassle finding out what went wrong (in part because I was too greedy and checked in a bunch of converted code without any review at all). Lesson learned I would say. Fortunately, this is a case where 2.6's Py3k warning mode should compensate.
Also, the limitations of 2to3 should be very well documented, so that when it does tell you how it has failed, it's clear to the developer what changes to make to the *2.6 source* to cause it to translate correctly, not how to fix up the translated 3.0 code.
I'm hoping Collin will continue his excellent work on 2to3. Hopefully he'll get help from others in writing docs aimed at teaching the c.l.py crowd how to use it and what to expect.
Thanks *very* much to all the python core developers and community members who have been working to make this happen, especially to Neal, Thomas, and Guido for listening intently to all of our concerns at PyCon.
You're welcome. I believe I even got a beer out of it. ;-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 10:22 pm, guido@python.org wrote:
Was this due to a bug that can be fixed, or an inherent problem? I could imagine, thanks to Python's dynamism, there would be edge cases that are impossible to detect completely reliably. If that's the case it would be good to at least have pedantic warnings from 2to3 alerting the user to places where translation could possibly be doing something semantically dodgy. Ideally before jumping the chasm with Twisted I'd like to make sure that all of that sort of warning was gone, in _addition_ to a clean test run.
I'm sure he'll hear from me if anything goes wrong with it :).
Well deserved!

On 3/6/07, glyph@divmod.com <glyph@divmod.com> wrote:
Uh-huh. ;-) I wrote two versions of the dict views refactoring. One that turns d.keys() into list(d.keys()) and d.iterkeys() into iter(d.keys()). This one is pretty robust except if you have classes that emulate 2.x-style dicts. But it generates ugly code. So I have a "light-weight" version that leaves d.keys() alone, while turning d.iterkeys() into d.keys(). This generates prettier code but more buggy. I probably should have used the heavy-duty one instead.
Right, this is my hope for 2.6.
Ideally before jumping the chasm with Twisted I'd like to make sure that all of that sort of warning was gone, in _addition_ to a clean test run.
Right. You need both. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

The ugliness is a virtue in this case as it stands out enough to motivate developers to review each case. The pretty/efficient version is tantamount to guessing, and effectively discards information in the transformation ("here be dragons"). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

Absolutely right. I'll withdraw the lightweight version. It's done enough damage. On 3/11/07, Andrew McNamara <andrewm@object-craft.com.au> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 3/6/07, glyph@divmod.com <glyph@divmod.com> wrote:
I've started work on "Capabilities" and "Limitations" sections for the 2to3 README: http://svn.python.org/view/sandbox/trunk/2to3/README?view=markup. I intend for them to provide a comprehensive look at what 2to3 can and can't do. Collin Winter

Thomas Wouters wrote:
developers and people who develop their own software. I would like to hear from people who have concrete doubts about this upgrade path. I don't mean
Disclaimer: I'm not involved in Py3k, and not even tried it once. And don't know the details of the tool to "transform" Py2 to Py3. Having said that, the *only* fear I have, is how safe it will be. And here comes the explanation. I'm now in an enviroment where we rely on Python, and I advocates on it everyday. We use 2.3, and 2.4, some servers we're about to deploy have 2.5, and we'll use the lastest Py2.x everytime we deploy a new machine. But, as we have always running behind our ToDo, we don't have time to say "Ok, this server runs 2.3, how can I start using 2.5?". Too many servers, too many applications, too many undocumented-and-nobody-knows-about-it applications. And they all are applications running 7x24. And Py3k is around the corner, and I even heard some guys saying "If I'd spent time taking care that this app runs ok after changing a bit of it, I'll wait to Python 3000". So, the enviroment is explained, now I go with "how safe it will be". I'd love to know that there'll be a tool that tells me, after running it over my application, Your app is "ready for migration"'. I do *not* care if it takes a lot of work to make the proper changes, but I need to feel confident that after running the last changed version during a week in, say, Py2.7, and no warnings appear, and the "verification tool" say green light, I can start running it in Py3k. And look at it a week more. And say "life is great", :) Anyway, I know how hard is it, and I regret not having the time I'd love to have to help you. All I can do is thank you. Thank you very much!
For me, I'd love to 2.6 start warning "you're mixing tab and spaces, shame yourself!". Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

On 3/5/07, Facundo Batista <facundo@taniquetil.com.ar> wrote:
I know the feeling. Google still uses Python 2.2 for most apps. We're supporting 2.4 as well, and hope to have the last apps migrated to 2.4 in a year or so. *Then* we'll start supporting a higher version -- probably 2.6 by then.
In your kind of env it's clearly too early to think about 3.0. You shouldn't expect to be switching until a year after 3.0 is released, probably.
You need comperhensive unit tests too. Then: *If* you get the green light from 2.6's py3k warning mode, *and* the conversion tool produces syntactically correct code without issuing warnings, *and* your unit tests all pass, *then* I expect you'll be in about the same situation as for any version upgrade in such an environment (e.g. 2.4 -> 2.5). I.e. you need to do a big integration test and once that passes you should be set for a stress-free transition. I think that's the best we can hope for. I plan to do it this way at Google too.
You're welcome!
(The English say "shame on you" :-) While I'd like to do this too I don't know how many folks there are out there who have no way to suppress the warning (because they're end users of a script someone else wrote for them). I think it may be more of a development warning, so I think category is appropriate. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (11)
-
Andrew McNamara
-
Barry Warsaw
-
Collin Winter
-
Facundo Batista
-
glyph@divmod.com
-
Greg Ewing
-
Guido van Rossum
-
Neal Norwitz
-
Neil Schemenauer
-
Stephen J. Turnbull
-
Thomas Wouters