Mailman 3 Proposal: from __future__ import unicode_string_literals - Python-Dev

newer
Two questions about jump opcodes

Proposal: from future import unicode_string_literals

older
Re: [Python-Dev] [Python-checkins]...

Lennart Regebro

23 Mar 2008 23 Mar '08

1:37 a.m.

Eric Smith wrote:

...

It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change.

Oh, <swear words>!

...

Pretty much. And even if it were possible, I don't see the point in doing it.

The point is this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-w... Basically, the 2to3 strategy on large codebases supported by a wide set of developers isn't likely to be maintanable, and without a gradual path forward, 3.0 isn't likely to be adopted in some parts of the Python community. Worst case this splits the efforts of the community into two groups, which would be damaging to Python as a whole. Howver: If 2.6 doesn't support this it's not the end of the world. Because if it turn out to be necessary, a 2.7 that does could always be released. I don't know about other large codebases like Twisted, but the Zope/Plone world isn't likely to even try moving to 3.0 any time soon after it's final release, so since it's complicated you can probably wait with this support until it turns out to be needed. M.-A. Lemburg wrote:

...

Could we point them to a special byte-code compiler such as Andrew Dalke's python4ply:

??? I'm not sure what this means... :) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

Show replies by date

Thomas Wouters

23 Mar 23 Mar

7:13 a.m.

On Sun, Mar 23, 2008 at 8:37 AM, Lennart Regebro wrote:

...

Eric Smith wrote:

...
It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change.

Oh, <swear words>!

I don't believe this to be true (we can simply store the source encoding information (which it might already be) and translate the *use* of string literals into unicode(literal, encoding).) But I still think this is a bad idea. Using the same codebase for 3.0 and 2.6 will leave you with inefficient and fragile code in one of the two codebases -- quite likely both. I know, I've tried. I don't see how it improves maintainability to leave your source full of forward- and backward-compatibility hacks. 3.0 was meant to be a clean break. Please treat it as such. Yes, that means you can't run the same source tree in two different Python versions -- too bad. It means adding a compilation-style step to one of the two Python versions -- too bad. It's quite easily automated, as proven by the vast majority of programming languages out there, which need such a step anyway.

...

Howver: If 2.6 doesn't support this it's not the end of the world. Because if it turn out to be necessary, a 2.7 that does could always be released. I don't know about other large codebases like Twisted, but the Zope/Plone world isn't likely to even try moving to 3.0 any time soon after it's final release, so since it's complicated you can probably wait with this support until it turns out to be needed.

That was always the assumption, and also the idea for 2.6 and 2.7. I would much rather 3.0 isn't widely accepted for a few years longer than that it be cluttered with backward compatibility crap, and even rather than that widely used code be riddled with such. But it's up to Guido in the end. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

Lennart Regebro

2:11 p.m.

On Sun, Mar 23, 2008 at 2:13 PM, Thomas Wouters wrote:

...

That was always the assumption, and also the idea for 2.6 and 2.7. I would much rather 3.0 isn't widely accepted for a few years longer than that it be cluttered with backward compatibility crap, and even rather than that widely used code be riddled with such. But it's up to Guido in the end.

Sure but this is a bit misleading. The risk isn't that 3.0 is not widely accepted quickly. The risk is that the community splits in two. And the suggestion is not for backwards compatibility in 3.0, but forwards compatibility in 2.6. So the question is rather: Do you want to disk a community split, or add forwards compatibility? But as I noted, if it turns out to be necessary to add that forwards compatibility, it's always possible to do a 2.7 that has it. I have loads of experience in writing code for evolving APIs, this is how things have been done in the Zope community for years. It's not a problem. It does not lead to fragile code. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

"Martin v. Löwis"

3:45 p.m.

New subject: Proposal: from __future__ import unicode_string_literals

...

So the question is rather: Do you want to disk a community split, or add forwards compatibility?

I don't think the risk is big. As far as people start saying "I will only support Python 3", or saying "I will not support Python 3" - that's fine. In the former case, people can still continue to use the old versions of the software (assuming we are talking about open source here), and continue to use those with 2.x. They won't get all the new features, and perhaps that is a reason for them to move to 3.x. In the latter case, people relying on the library either have to stay with 2.x until all their dependencies get ported, or they will have to contribute 3.x ports themselves to the developers. In some cases, this may cause a fork of the project, but I guess these cases are rare (and occur only if the maintainer is not cooperative in at least incorporating patches even if its for stuff he doesn't care about). So in short: no, the risk that the community splits is very small. When people contribute code, it's not because of care about the community, but because of their own needs. That's how open-source software works. Regards, Martin

Lennart Regebro

4:17 p.m.

On Sun, Mar 23, 2008 at 10:45 PM, "Martin v. Löwis" wrote:

...

...
So the question is rather: Do you want to disk a community split, or add forwards compatibility?

I don't think the risk is big. As far as people start saying "I will only support Python 3", or saying "I will not support Python 3" - that's fine.

No, it isn't. That's the whole thing. It isn't fine.

...

In the latter case, people relying on the library either have to stay with 2.x until all their dependencies get ported, or they will have to contribute 3.x ports themselves to the developers.

You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. I repeat: I have no doubt the 2to3 approach works well in that case, if you want to support both 2.6 and 3.0.

...

So in short: no, the risk that the community splits is very small.

No, it is a significant risk. Don't brush it away. We do NOT end up having a 2.x python world and a 3.x python world. The community doesn't have the resources to maintain momentum in a language if the energy is divided in half. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

"Martin v. Löwis"

5:14 p.m.

New subject: Proposal: from __future__ import unicode_string_literals

...

You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks.

Can you give me examples of such software? Are you perhaps talking about closed source software? Regards, Martin

Jean-Paul Calderone

7:30 p.m.

On Mon, 24 Mar 2008 00:14:13 +0100, "\"Martin v. Löwis\"" wrote:

...

...
You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks.

Can you give me examples of such software? Are you perhaps talking about closed source software?

I'm not sure what software he was talking about. I can say that for the work I do on both Twisted and Divmod software, I'd be quite happy to see this feature. As either part of a migration path towards 3k _or_ as a feature entirely on its own merits, this would be very useful to me. I'm a bit curious about why Thomas said this sort of thing results in fragile code. Twisted has been using __future__ imports for years and they've never been a problem. Twisted currently supports Python 2.3 through Python 2.5, and the only thing that's really difficult about that is subtle changes in library behavior, not syntax. I'm also curious about why Lennart thinks that this would only be relevant for large projects with lots of developers making regular releases. Sure, I'd use it in that scenario, but that's because it's a subset of "all development". ;) Jean-Paul

Lennart Regebro

24 Mar 24 Mar

2:22 a.m.

On Mon, Mar 24, 2008 at 2:30 AM, Jean-Paul Calderone wrote:

...

I'm also curious about why Lennart thinks that this would only be relevant for large projects with lots of developers making regular releases.

No, you misunderstood, or I miswrote. I think 2to3 is a procedure that will work well for library type projects with a reasonably small set of developers that make regular releases. There you can release both a python 2 and a python 3 version of the module, for example. I think more future compatibility is relevant for everybody, but I think it's really *necessary* for large projects with lots of developers that do NOT make regular releases, but where releases are done when the project as a whole is done. I.e, the developers themselves work on a project, and use trunk of most modules. Many modules are updated in parallell, and developed on in parallell, and (if the project is reasonably well-managed) releases are made when new releases are sent to the customer. Nuxeo CPS worked like this, but we can ignore them as that project is all but dead will never move to Python 3 in any case. Zope/CMF/Plone works like this. The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk. I know Django people both strive to support multiple versions of Python and regularily run their production sites on trunk. Insane, perhaps, but that's how things are. :) This is often how big in-house projects work as well, although I don't know of any of those in Python, reasonably because I'm an independent contractor in open source. :) So, in short: Large projects with interconnected modules where the developers and users of module are the same people will have big difficulties with the 2to3 approach and would be the people who are most likely to not be able to in practice go forward to Python 3 unless they have some sort of smooth path forward. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

"Martin v. Löwis"

4:27 a.m.

New subject: Proposal: from __future__ import unicode_string_literals

...

Nuxeo CPS worked like this, but we can ignore them as that project is all but dead will never move to Python 3 in any case. Zope/CMF/Plone works like this.

I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda? It's fine that all people develop on the same subversion repository. Doing so integrates nicely with 2to3.

...

The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk.

And that's fine. You still can integrate 2to3 with that transparently.

...

I know Django people both strive to support multiple versions of Python and regularily run their production sites on trunk. Insane, perhaps, but that's how things are.

And no need to change that, see http://wiki.python.org/moin/PortingDjangoTo3k Just to repeat myself: With that patch to Django, you can a) support all versions of Python simultaneously, from 2.x to 3.0 b) work from the subversion trunk all the time c) transparently invoke 2to3 on the trunk; you won't even notice that you do (except for all the diffs it currently prints to stdout also :-) FWIW, your approach (of adding __future__ imports to 2.6) wouldn't help Django at all, as they would need to give up 2.5 and earlier.

...

So, in short: Large projects with interconnected modules where the developers and users of module are the same people will have big difficulties with the 2to3 approach and would be the people who are most likely to not be able to in practice go forward to Python 3 unless they have some sort of smooth path forward.

I still don't see why that is. In the examples you gave, no such difficulties are apparent. Regards, Martin

Lennart Regebro

4:39 a.m.

On Mon, Mar 24, 2008 at 11:27 AM, "Martin v. Löwis" wrote:

...

...
Nuxeo CPS worked like this, but we can ignore them as that project is all but dead will never move to Python 3 in any case. Zope/CMF/Plone works like this.

I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda?

No, Zope is not a library, it's an application. No, you typically do not setup packages, although most (but not all) parts of Zope 3 is setup if you run Zope in a buildout configuration. Zope 2 does not.

...

...
The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk.

And that's fine. You still can integrate 2to3 with that transparently.

I can not see how that would work.

...

I still don't see why that is. In the examples you gave, no such difficulties are apparent.

Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. But that's just the way it is. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

"Martin v. Löwis"

5:26 a.m.

...

...
I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda?

No, Zope is not a library, it's an application. No, you typically do not setup packages, although most (but not all) parts of Zope 3 is setup if you run Zope in a buildout configuration. Zope 2 does not.

I was talking about http://svn.zope.org/zope.psycopgda/trunk/ Is that not the right source? In any case, using 2to3 for applications is even easier than using it for libraries, assuming there is an installation procedure for the application.

...

...
...
The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk.

And that's fine. You still can integrate 2to3 with that transparently.

I can not see how that would work.

To me "run against trunk" means that I check out some source, then run "setup.py install" (or buildout, or make). This will invoke 2to3 behind the scenes. I don't know exactly how plone works, but I could imagine that it's also possible to run 2to3 when plone "loads" a "component" (and I'm sure I'm using the wrong words here).

...

Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. But that's just the way it is.

I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. "The Plone collective" is not a specific example, as it doesn't allow me to reproduce your problems. What is the specific thing I would want to do with it, and what specific source repository do I need to check out to do that? Regards, Martin

Lennart Regebro

5:38 a.m.

On Mon, Mar 24, 2008 at 12:26 PM, "Martin v. Löwis" wrote:

...

http://svn.zope.org/zope.psycopgda/trunk/

Is that not the right source?

No, and yes. Many of the zope3 modules are installable as separate modules. Zope 3 in general has been "eggified". This is not however how everybody installs Zope3. Zope2 is not installed this way. Also, most Zope2 products are not modules installed by setup.py.

...

To me "run against trunk" means that I check out some source, then run "setup.py install" (or buildout, or make). This will invoke 2to3 behind the scenes.

I repeat: There is no setup.py install to run.

...

I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. "The Plone collective" is not a specific example

It is a specific example. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

Nick Coghlan

9:34 a.m.

New subject: Proposal: from __future__ import unicode_string_literals

Lennart Regebro wrote:

...

On Mon, Mar 24, 2008 at 12:26 PM, "Martin v. Löwis" wrote:

...
I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. "The Plone collective" is not a specific example

It is a specific example.

No it isn't. A specific example would be "I have environment X setup on a machine. I go to website/SVN repository Y and retrieve source code Z and start using it". Without the step by step process, it is impossible to identify if there is any point in the procedure where an invocation of 2to3 could be inserted relatively painlessly. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Lennart Regebro

9:43 a.m.

On Mon, Mar 24, 2008 at 4:34 PM, Nick Coghlan wrote:

...

No it isn't. A specific example would be "I have environment X setup on a machine. I go to website/SVN repository Y and retrieve source code Z and start using it".

"I have environment Plone setup on a machine. I also have several products from the Plone collective which I use from the custom product that contains the custom site code". Thats it. It is a specific example. I can't get more specific than that without you learning Plone.

...

Without the step by step process, it is impossible to identify if there is any point in the procedure where an invocation of 2to3 could be inserted relatively painlessly.

It can't. That's the whole point. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

"Martin v. Löwis"

1:30 p.m.

...

"I have environment Plone setup on a machine. I also have several products from the Plone collective which I use from the custom product that contains the custom site code".

Thats it. It is a specific example. I can't get more specific than that without you learning Plone.

Ok, let me try to guess, then. You use https://svn.plone.org/svn/collective/LatexTool/ and you perform an svn checkout into /var/lib/zope2.10/instance/plone-site/Products You start zope, edit the source, or perform a svn update, and then restart or refresh the product. Correct? If so, there is a fairly simple way to integrate 2to3: In OFS.Application.import_product, run 2to3 first before importing the product, if a file "run2to3.txt" exists in the product's root directory. This will perform a copy of the product into /var/lib/zope2.10/instance/plone-site/Products.2to3 Voila, transparent invocation of 2to3. Now, if you want pdb to display the original source rather than the one being converted, subclass pdb and strip out .2to3 from the source filename. Regards, Martin

Lennart Regebro

1:51 p.m.

On Mon, Mar 24, 2008 at 8:30 PM, "Martin v. Löwis" wrote:

...

....

Ah, I see. Your point was that with enough magic there is some way to run 2to3 automatically. Sure, I never doubted that. I don't see how that changes anything I said really. I still think the forwards compatibility that exists in 2.6 is a good idea, and the more of it the better. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

Terry Reedy

5:39 p.m.

New subject: Proposal: from __future__import unicode_string_literals

""Martin v. Löwis"" wrote in message news:47E78226.1070607@v.loewis.de... I think we all can agree that we would like Plone to migrate to 3.x without too much pain and that their collective situation may be more difficult for migration than some others. | http://wiki.python.org/moin/PortingDjangoTo3k | | Just to repeat myself: With that patch to Django, you can | a) support all versions of Python simultaneously, from 2.x to 3.0 I find this surprising for two reasons. 1. I had the impression from discussions over the past year that fully automatic use of 2to3 would presume use of 2.6 and its backported 3.0 features and __future__ imports. If it really works with ealier 2.x code, great, but please pardon any initial surprise or scepticism. 2. You report has caveats such as * there are certainly large parts of the code base that I haven't touched, so more issues are likely to show up *This port attempts to maintain compatibility with older Python versions from a single code base; the goal is to support all versions that Django supports, plus 3.0. The current patch fails to do so in certain details, due to bugs in the 2to3 tool. *This approach mostly works, and has the following flaws: some of the fixers work incorrectly (bugs 2453, 2446, 2468) *I have worked with sqlite3 only; all the other databases have not been tested. So your unqualified assertion seems more like an anticipated future (certain to you but maybe not to others) than present reality. 3. Also, you said you worked around some 2to3 failings with conditional blocks like, I presume, the following. if sys.version < (3,0,0): else: Do I assume correctly that you, rather than 2to3 had to do such? Will 2to3 remove the wrapper to leave just the 3.0 code? Or would someone have to go thru by hand to get clean 3.0 code? I understand that this is a standard method for multiple release code, but it seems a bit like cheating since the point of 2to3 is to not to have to do this. Or is converting 'types.ClassType' to 'types' a future fixer item? Terry

"Martin v. Löwis"

8:37 p.m.

New subject: Proposal: from __future__import unicode_string_literals

...

| Just to repeat myself: With that patch to Django, you can | a) support all versions of Python simultaneously, from 2.x to 3.0

I find this surprising for two reasons.

1. I had the impression from discussions over the past year that fully automatic use of 2to3 would presume use of 2.6 and its backported 3.0 features and __future__ imports. If it really works with ealier 2.x code, great, but please pardon any initial surprise or scepticism.

This is precisely why I started this porting experiment. If you are still skeptic, please substantiate your skepticism with facts: run my patch, and tell me why it doesn't work, or couldn't be completed. If you are now merely surprised, but not skeptic anymore: my pleasure. The believe that you must port to 2.6 first is wide-spread. It probably originates from the statement that the official porting strategy involves porting to 2.6 first. That strategy does so, however, to enable you to run the -3 option, so you can find porting problems more easily. If you can find the porting problems the hard way, i.e. by running the software and testing it, you don't need the -3 warnings. When I started a week ago, a few essential 2to3 fixers did not exist (in particular the one David Wolever wrote to make implicit relative imports explicit). That fixer really falls into the 2to2.5 category; it would have been possible to change the code to use relative imports everywhere, thereby breaking 2.3 compatibility. It is possible that other examples like this still exist (i.e. 2to3 doesn't fix it, but doesn't have to if you can assume 2.5), but I'm not aware of any (actually, that's not entirely true - the email module renaming is of the same kind. However, this can be dealt with by ImportError guards. Still, having a fixer for that might be useful)

...

2. You report has caveats such as

* there are certainly large parts of the code base that I haven't touched, so more issues are likely to show up

True.

...

*This port attempts to maintain compatibility with older Python versions from a single code base; the goal is to support all versions that Django supports, plus 3.0. The current patch fails to do so in certain details, due to bugs in the 2to3 tool.

*This approach mostly works, and has the following flaws: some of the fixers work incorrectly (bugs 2453, 2446, 2468)

These bugs are really shallow, and some have been fixed already.

...

*I have worked with sqlite3 only; all the other databases have not been tested.

True.

...

So your unqualified assertion seems more like an anticipated future (certain to you but maybe not to others) than present reality.

Likewise, the statement that you *can't* possibly use the same code base from 2.1 to 3.0 is unfounded, and, unlike my claim, doesn't have any kind of proof behind it.

...

3. Also, you said you worked around some 2to3 failings with conditional blocks like, I presume, the following.

if sys.version < (3,0,0): else:

Do I assume correctly that you, rather than 2to3 had to do such?

Indeed.

...

Will 2to3 remove the wrapper to leave just the 3.0 code?

Currently, it leaves the code unchanged. It could fairly easily remove it, but doing so might shift line numbers, which in turn is undesirable. 2to3 has support for optional fixers, and that might be a use case.

...

Or would someone have to go thru by hand to get clean 3.0 code?

See above. Writing a fixer for it is fairly easy, and somebody will certainly do that, but I would only run it when using a "burn the bridges" run.

...

I understand that this is a standard method for multiple release code, but it seems a bit like cheating since the point of 2to3 is to not to have to do this. Or is converting 'types.ClassType' to 'types' a future fixer item?

No. This is the sort of change that 2to3 can't do right. If Django would require Python 2.5, the conditional could go away, as this appears in a context of creating exception classes on-the-fly; they can be new-style classes in 2.5. However, I consider conditional code blocks not as cheating at all. If you want to provide backwards compatibility, you *always* have to compromise readability for portability. This was even the case within 2.x, where you can't use True and False if you want to support Python versions that didn't have it, or where you can use generators, or iterators, or ..., when older Python versions need to be supported. The main point of 2to3 is not to have the 3.x code "look nice", in fact, in many cases, it won't, since 2to3 must make conservative assumptions in some cases, so to not break the semantics of the program. Instead, the main point of 2to3 is to replace syntax that is going away with the 3.x equivalent syntax. In the cases where I conventionalize on the Python version, it's not syntax that has changed. Regards, Martin

Barry Warsaw

7:50 a.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've missed most of this thread, but let me just put my two cents in. I'd still like a future import to turn on unicode string literals (or more accurately, treat unadorned string literals as unicodes). As someone who is striving mightily to get various libraries and applications unicode clean, it's simply a matter of training my brain to correctly think, "this is a bytes" or "this is a string", and training my fingers to type the right thing. I'd like to be able to start retraining the muscle memory so that by the time 3.0 comes around, it'll will be a much smoother transition, for me and other coders. Now, if it's not feasible to add such a future import, that's one thing. If it's feasible then I think we should do it. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+exl3EjvBPtnXfVAQLa0QQAl07BSwokgspNoIT0s2nn3kcWDn//PBmM ARgUCwd2fwZhHkiFsx5YgfzHJaBOuQjPNM4jOwUVy8vZpwUEVZNWmWE7rh+AHxQD FFLyier6/O1PkIe4US1RwuE3/53viP2qWo2Fr0z4zwbJbI6QOQvRVZeZ6OhU02jn GsNFuhuBz58= =1hYN -----END PGP SIGNATURE-----

Christian Heimes

10:58 a.m.

Barry Warsaw schrieb:

...

I've missed most of this thread, but let me just put my two cents in. I'd still like a future import to turn on unicode string literals (or more accurately, treat unadorned string literals as unicodes). As someone who is striving mightily to get various libraries and applications unicode clean, it's simply a matter of training my brain to correctly think, "this is a bytes" or "this is a string", and training my fingers to type the right thing.

I'd like to be able to start retraining the muscle memory so that by the time 3.0 comes around, it'll will be a much smoother transition, for me and other coders.

Now, if it's not feasible to add such a future import, that's one thing. If it's feasible then I think we should do it.

I'm working on a node based __future__ parser based on the Python 2.3 code as MvL suggested. I'm making good progress. If I get it working it should be trivial to implement a __future__ unicode_string_literals feature. Christian

M.-A. Lemburg

25 Mar 25 Mar

3:43 a.m.

New subject: Proposal: from __future__ import unicode_string_literals

On 2008-03-24 09:22, Lennart Regebro wrote:

...

I think 2to3 is a procedure that will work well for library type projects with a reasonably small set of developers that make regular releases. There you can release both a python 2 and a python 3 version of the module, for example. ... So, in short: Large projects with interconnected modules where the developers and users of module are the same people will have big difficulties with the 2to3 approach and would be the people who are most likely to not be able to in practice go forward to Python 3 unless they have some sort of smooth path forward.

I don't think there's a lot to worry about: Companies using Python for applications typically have a completely different life-cycle of releases and applications compared to the Python release schedule, i.e. they often still run Python 2.3 or 2.4 and wait for major releases to settle before deciding to port to them. Every now and then, they make the decision to port to the next release (for the next version of their software) and this change is then managed accordingly - sometimes skipping a complete major release of Python. In such projects, 2to3 will get applied to the sources once and then all development continues on the Python 3.0 version of the code. In reality, I don't think that 2to3 will get used for continuous porting between a 2.x code base and a 3.0 one all that much. The transition from 2.x to 3.0 will happen during a longer period of time (probably a few years) and depend a lot on the release cycle of the applications using Python, whether or not the 3.0 version provides better features, more performance, etc. and whether the 2.x branches of Python and the used 3rd party modules are still supported or not. New applications will likely choose 3.0 right away - provided that the needed 3rd party modules are available and stable enough. In summary: 2to3 is a very useful tool to have. Whether or not it is used for continuous porting between the two worlds is really secondary. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 25 2008)

...

...
...
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611

Thomas Wouters

24 Mar 24 Mar

4:29 a.m.

On Mon, Mar 24, 2008 at 2:30 AM, Jean-Paul Calderone wrote:

...

On Mon, 24 Mar 2008 00:14:13 +0100, "\"Martin v. Löwis\"" < martin@v.loewis.de> wrote:

...
...
You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks.

Can you give me examples of such software? Are you perhaps talking about closed source software?

I'm not sure what software he was talking about. I can say that for the work I do on both Twisted and Divmod software, I'd be quite happy to see this feature. As either part of a migration path towards 3k _or_ as a feature entirely on its own merits, this would be very useful to me.

I'm a bit curious about why Thomas said this sort of thing results in fragile code. Twisted has been using __future__ imports for years and they've never been a problem. Twisted currently supports Python 2.3 through Python 2.5, and the only thing that's really difficult about that is subtle changes in library behavior, not syntax.

I wasn't talking about future imports. I was talking about writing Python 2.6 code and expecting it to work *the same way* in 3.0 without modification. It requires all programmers working on the project to have knowledge of how 3.0 and 2.6 differ. *I* can't even look at code and tell you how 2.6 and 3.0 will differ. Since Lennart was talking specifically about projects with a large number of developers, I do not believe this is a safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. Or, alternatively, a preprocessor step involving 3to2, which I think will result in better code. Unfortunately I don't have time to work on either anytime soon. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

Lennart Regebro

4:41 a.m.

On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters wrote:

...

safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge.

As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

Paul Moore

6:03 a.m.

On 24/03/2008, Lennart Regebro wrote:

...

On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters wrote:

...
safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge.

As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference.

Do you not test the code you distribute under each version of Python that you (claim to) support, in any case? If not, what does "supported under Python 2.4" mean to you? Paul.

Lennart Regebro

6:08 a.m.

On Mon, Mar 24, 2008 at 1:03 PM, Paul Moore wrote:

...

On 24/03/2008, Lennart Regebro wrote:

...
On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters wrote:

...
safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge.

As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference.

Do you not test the code you distribute under each version of Python that you (claim to) support, in any case?

Yes I do. I don't think my text above was unclear on the issue or could be misunderstood in this way. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

Paul Moore

6:26 a.m.

On 24/03/2008, Lennart Regebro wrote:

...

On Mon, Mar 24, 2008 at 1:03 PM, Paul Moore wrote:

...
On 24/03/2008, Lennart Regebro wrote:

...

...
...
As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference.

Do you not test the code you distribute under each version of Python that you (claim to) support, in any case?

Yes I do. I don't think my text above was unclear on the issue or could be misunderstood in this way.

Your statement "using 2to3 will also require you to test the code in both environments" seemed to me to say that *not* having to use 2to3 would save you from doing this (as if this were either desirable, or your current practice). My apologies if I misread your comment. What *are* you trying to say, then? It seems that you're saying that using 2to3 doesn't make things any more complex, but that contradicts your previous argument. Paul.

Lennart Regebro

6:31 a.m.

On Mon, Mar 24, 2008 at 1:26 PM, Paul Moore wrote:

...

Your statement "using 2to3 will also require you to test the code in both environments" seemed to me to say that *not* having to use 2to3 would save you from doing this (as if this were either desirable, or your current practice).

I think maybe you missed the statement I responded to, claiming that 2to3 would require no knowledge about the differences between Python 2.6 and 3.0, implying that you could just run it, and it would always work, which I don't believe. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

Lennart Regebro

2:10 a.m.

On Mon, Mar 24, 2008 at 12:14 AM, "Martin v. Löwis" wrote:

...

...
You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks.

Can you give me examples of such software?

I'll repeat the link where I explained my point on this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-w... -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

"Martin v. Löwis"

3:23 a.m.

Lennart Regebro wrote:

...

On Mon, Mar 24, 2008 at 12:14 AM, "Martin v. Löwis" wrote:

...
...
You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks.

Can you give me examples of such software?

I'll repeat the link where I explained my point on this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-w...

Yet, there you merely claim that such software exists ("within larger organizations", and "within large communities like Zope and Plone"), without giving specific examples. This is not very convincing. Regards, Martin

Lennart Regebro

25 Mar 25 Mar

5:10 a.m.

On Mon, Mar 24, 2008 at 10:23 AM, "Martin v. Löwis" wrote:

...

Yet, there you merely claim that such software exists ("within larger organizations", and "within large communities like Zope and Plone"), without giving specific examples.

No I don't. Here is what I said: "In many other cases, this is not how code is developed. Both within larger organisations and within large communities like Zope and Plone". I don't think chewing through this issue once more is meaningful, so I'll stop now. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

5875

Age (days ago)

5877

Last active (days ago)

List overview

Download

29 comments

10 participants

participants (10)

"Martin v. Löwis"
Barry Warsaw
Christian Heimes
Jean-Paul Calderone
Lennart Regebro
M.-A. Lemburg
Nick Coghlan
Paul Moore
Terry Reedy
Thomas Wouters

Proposal: from __future__ import unicode_string_literals

tags

participants (10)

Proposal: from future import unicode_string_literals