Mailman 3 nonlocal keyword in 2.x? - Python-Dev

nonlocal keyword in 2.x?

older
Re: [Python-Dev] 2.7 Release? 2.7...

Mike Krell

21 Oct 2009 21 Oct '09

9:56 p.m.

Is there any possibility of backporting support for the nonlocal keyword into a 2.x release? I see it's not in 2.6, but I don't know if that was an intentional design choice or due to a lack of demand / round tuits. I'm also not sure if this would fall under the scope of the proposed moratorium on new language features (although my first impression was that it could be allowed since it already exists in python 3. One of my motivations for asking is a recent blog post by Fernando Perez of IPython fame that describes an interesting decorator-based idiom inspired by Apple's Grand Central Dispatch which would allow many interesting possibilities for expressing parallelization and other manipulations of execution context for blocks of python code. Unfortunately, using the technique to its fullest extent requires the nonlocal keyword. The blog post is here: https://cirl.berkeley.edu/fperez/py4science/decorators.html Mike

Attachments:

attachment.htm (text/html — 1.0 KB)

Show replies by thread

Neal Norwitz

22 Oct 22 Oct

12:46 a.m.

On Wed, Oct 21, 2009 at 6:56 PM, Mike Krell wrote:

...

Is there any possibility of backporting support for the nonlocal keyword into a 2.x release? I see it's not in 2.6, but I don't know if that was an intentional design choice or due to a lack of demand / round tuits. I'm also not sure if this would fall under the scope of the proposed moratorium on new language features (although my first impression was that it could be allowed since it already exists in python 3.

IIRC, Jeremy Hylton tried to backport it during a Pycon sprint a few years back. I think it was difficult and he dropped it. I don't know if there were any showstoppers or if Jeremy saved his work...or if my memory is even correct. :-) n

Fabio Zadrozny

11:32 a.m.

On Wed, Oct 21, 2009 at 10:56 PM, Mike Krell wrote:

...

Is there any possibility of backporting support for the nonlocal keyword into a 2.x release? I see it's not in 2.6, but I don't know if that was an intentional design choice or due to a lack of demand / round tuits. I'm also not sure if this would fall under the scope of the proposed moratorium on new language features (although my first impression was that it could be allowed since it already exists in python 3.

One of my motivations for asking is a recent blog post by Fernando Perez of IPython fame that describes an interesting decorator-based idiom inspired by Apple's Grand Central Dispatch which would allow many interesting possibilities for expressing parallelization and other manipulations of execution context for blocks of python code. Unfortunately, using the technique to its fullest extent requires the nonlocal keyword.

The blog post is here: https://cirl.berkeley.edu/fperez/py4science/decorators.html

Just as a note, the nonlocal there is not a requirement... You can just create a mutable object there and change that object (so, you don't need to actually rebind the object in the outer scope). E.g.: instead of creating a float in the context, create a list with a single float and change the float in the list (maybe the nonlocal would be nicer, but it's certainly still usable) Cheers, Fabio

"Martin v. Löwis"

4:24 p.m.

Mike Krell wrote:

...

Is there any possibility of backporting support for the nonlocal keyword into a 2.x release?

If so, only into 2.7. Can you please explain why it would be desirable to do that? 2.7 will likely be the last 2.x release, so only a fairly small portion of the applications would be actually able to use this (or any other new feature added to 2.7): most code supporting 2.x will also have to support 2.6, so the keyword won't be available to such code, anyway. Regards, Martin

exarkun＠twistedmatrix.com

4:46 p.m.

On 08:24 pm, martin@v.loewis.de wrote:

...

Mike Krell wrote:

...
Is there any possibility of backporting support for the nonlocal keyword into a 2.x release?

If so, only into 2.7. Can you please explain why it would be desirable to do that? 2.7 will likely be the last 2.x release, so only a fairly small portion of the applications would be actually able to use this (or any other new feature added to 2.7): most code supporting 2.x will also have to support 2.6, so the keyword won't be available to such code, anyway.

For the same reason that it is desirable to backport all of the other changes from 3.x - because it makes the 2.x to 3.x transition easier. If Python 2.7 supports the nonlocal keyword, then 2.7 becomes that much better of a stepping stone towards 3.x. You've suggested that most 2.x code will have to support 2.6 and so won't be able to use the nonlocal keyword even if it is added to 2.7. This precise argument could be applied to all of the features in 2.6 which aim to bring it closer to 3.x. Any program which must retain Python 2.5 compatibility will not be able to use them. Yet 2.6 is a more useful stepping stone towards 3.x than 2.5 is. So yes, it would be quite desirable to see nonlocal and as many other 3.x features as possible backported for 2.7. And depending on how close 2.7 manages to get, it may make sense to backport anything that doesn't make it into 2.7 for a 2.8 release. The 3.x transition is *hard*. Anything that makes it easier is good. Jean-Paul

"Martin v. Löwis"

4:57 p.m.

exarkun@twistedmatrix.com wrote:

...

On 08:24 pm, martin@v.loewis.de wrote:

...
Mike Krell wrote:

...
Is there any possibility of backporting support for the nonlocal keyword into a 2.x release?

If so, only into 2.7. Can you please explain why it would be desirable to do that? 2.7 will likely be the last 2.x release, so only a fairly small portion of the applications would be actually able to use this (or any other new feature added to 2.7): most code supporting 2.x will also have to support 2.6, so the keyword won't be available to such code, anyway.

For the same reason that it is desirable to backport all of the other changes from 3.x - because it makes the 2.x to 3.x transition easier.

Hmm. Really?

...

If Python 2.7 supports the nonlocal keyword, then 2.7 becomes that much better of a stepping stone towards 3.x.

What use has such a stepping stone? Why, and (more importantly) when would anybody currently supporting 2.x give up 2.6 and earlier, and only support 2.7? And, if they chose to do so, why would they not move the code base to 3.x right away?

...

You've suggested that most 2.x code will have to support 2.6 and so won't be able to use the nonlocal keyword even if it is added to 2.7.

Correct.

...

This precise argument could be applied to all of the features in 2.6 which aim to bring it closer to 3.x.

Not so. One of the features added to 2.6 was the 3k warning. This warning can be used without any modification to the code. So code can run on 2.6 and use the feature, while running unmodified on 2.5 and earlier (not using it). As for actual language and library changes (such as any new future import): there was indeed little point adding them. However, given that the possible migration paths weren't as clear back then as they are now, it is understandable that people considered this a viable path. In addition, for 2.6, it's a bit more realistic to assume that people might drop 2.5 support and still support 2.x for some more time (in particular as people wouldn't rule out a 2.8 release back then, either).

...

Any program which must retain Python 2.5 compatibility will not be able to use them. Yet 2.6 is a more useful stepping stone towards 3.x than 2.5 is.

I disagree fairly much (except that the 3k warnings probably *are* useful - even though I haven't ever used them myself).

...

So yes, it would be quite desirable to see nonlocal and as many other 3.x features as possible backported for 2.7. And depending on how close 2.7 manages to get, it may make sense to backport anything that doesn't make it into 2.7 for a 2.8 release.

There might not be a 2.8 release at all, though.

...

The 3.x transition is *hard*. Anything that makes it easier is good.

I agree. I question whether backporting features actually makes the transition easier. In addition, in the *specific* case: the nonlocal keyword isn't necessary for a transition *at all*. Code that currently works without it won't need it when ported to 3.x. You may not be able to use it while maintaining 2.x and 3.x simultaneously, but you can certainly do the transition just fine without it. Regards, Martin

Michael Foord

5:08 p.m.

Martin v. Löwis wrote:

...

exarkun@twistedmatrix.com wrote:

...
On 08:24 pm, martin@v.loewis.de wrote:

...
Mike Krell wrote:

...
Is there any possibility of backporting support for the nonlocal keyword into a 2.x release?

If so, only into 2.7. Can you please explain why it would be desirable to do that? 2.7 will likely be the last 2.x release, so only a fairly small portion of the applications would be actually able to use this (or any other new feature added to 2.7): most code supporting 2.x will also have to support 2.6, so the keyword won't be available to such code, anyway.

For the same reason that it is desirable to backport all of the other changes from 3.x - because it makes the 2.x to 3.x transition easier.

Hmm. Really?

...
If Python 2.7 supports the nonlocal keyword, then 2.7 becomes that much better of a stepping stone towards 3.x.

What use has such a stepping stone? Why, and (more importantly) when would anybody currently supporting 2.x give up 2.6 and earlier, and only support 2.7? And, if they chose to do so, why would they not move the code base to 3.x right away?

From the Django roadmap for supporting 3.0, using 2.6 as a stepping stone (and if 2.7 was a *better* stepping stone then it would make it easier): http://groups.google.com/group/django-developers/msg/0888b1c8f2518059? * First half of 2009: Django 1.1 is released, with a notification that it will be the final release supporting Python 2.3. * Second half of 2009: Django 1.2 is released, drops Python 2.3 support and is the final release supporting Python 2.4. * First half of 2010: Django 1.3 is released, drops Python 2.4 support and is the final release supporting Python 2.5. * Second half of 2010: Django 1.4 is released and drops Python 2.5 support. This gets us to a situation where, about a year after the release of Python 3.0, Django will be ready to make the transition -- the only 2.x Python we'll be supporting is Python 2.6, and 2to3 plus manual effort and available supporting libraries should make it possible to also run Django on Python 3.0 either at that point or not long after. From there, 2.6 support can be dropped whenever convenient, and Django can move to running only on Python 3.x at whatever time is judged appropriate. All the best, Michael Foord

...

...
You've suggested that most 2.x code will have to support 2.6 and so won't be able to use the nonlocal keyword even if it is added to 2.7.

Correct.

...
This precise argument could be applied to all of the features in 2.6 which aim to bring it closer to 3.x.

Not so. One of the features added to 2.6 was the 3k warning. This warning can be used without any modification to the code. So code can run on 2.6 and use the feature, while running unmodified on 2.5 and earlier (not using it).

As for actual language and library changes (such as any new future import): there was indeed little point adding them. However, given that the possible migration paths weren't as clear back then as they are now, it is understandable that people considered this a viable path.

In addition, for 2.6, it's a bit more realistic to assume that people might drop 2.5 support and still support 2.x for some more time (in particular as people wouldn't rule out a 2.8 release back then, either).

...
Any program which must retain Python 2.5 compatibility will not be able to use them. Yet 2.6 is a more useful stepping stone towards 3.x than 2.5 is.

I disagree fairly much (except that the 3k warnings probably *are* useful - even though I haven't ever used them myself).

...
So yes, it would be quite desirable to see nonlocal and as many other 3.x features as possible backported for 2.7. And depending on how close 2.7 manages to get, it may make sense to backport anything that doesn't make it into 2.7 for a 2.8 release.

There might not be a 2.8 release at all, though.

...
The 3.x transition is *hard*. Anything that makes it easier is good.

I agree. I question whether backporting features actually makes the transition easier.

In addition, in the *specific* case: the nonlocal keyword isn't necessary for a transition *at all*. Code that currently works without it won't need it when ported to 3.x. You may not be able to use it while maintaining 2.x and 3.x simultaneously, but you can certainly do the transition just fine without it.

Regards, Martin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog

"Martin v. Löwis"

5:51 p.m.

...

From the Django roadmap for supporting 3.0, using 2.6 as a stepping stone (and if 2.7 was a *better* stepping stone then it would make it easier):

http://groups.google.com/group/django-developers/msg/0888b1c8f2518059?

Is that still a current plan? It's from November 2008.

...

This gets us to a situation where, about a year after the release of Python 3.0, Django will be ready to make the transition -- the only 2.x Python we'll be supporting is Python 2.6, and 2to3 plus manual effort and available supporting libraries should make it possible to also run Django on Python 3.0 either at that point or not long after.

From there, 2.6 support can be dropped whenever convenient, and Django can move to running only on Python 3.x at whatever time is judged appropriate.

I would claim that this specific plan was ignoring opportunities for migrating to Python 3.0. My Django port demonstrates that you can very well support 2.3 and 3.0 simultaneously: http://wiki.python.org/moin/PortingDjangoTo3k Regards, Martin

Fernando Perez

23 Oct 23 Oct

2:07 a.m.

On Thu, 22 Oct 2009 12:32:37 -0300, Fabio Zadrozny wrote:

...

Just as a note, the nonlocal there is not a requirement...

You can just create a mutable object there and change that object (so, you don't need to actually rebind the object in the outer scope).

E.g.: instead of creating a float in the context, create a list with a single float and change the float in the list (maybe the nonlocal would be nicer, but it's certainly still usable)

Yup, that's what I meant by 'some slightly ugly solutions' in this note: http://mail.scipy.org/pipermail/ipython-dev/2009-September/005529.html in the thread that spawned those notes. nonlocal allows for this pattern to work without the ugliness of writing code like: s = [s] @somedeco def foo(): s[0] += 1 s = s[0] just to be able to 'change s' inside the foo() scope. I felt this was both obvious and ugly enough not to warrant too much explicit mention, but I probably should have left it there for the sake of completeness. Thanks for the feedback. Cheers, f ps - the above shouldn't be taken as either pro or con on the idea of nonlocal in 2.x, just a clarification on why I didn't add the mutable container trick to the original notes.

Mike Krell

3:30 a.m.

On Thu, Oct 22, 2009 at 1:24 PM, "Martin v. Löwis" wrote:

...

Can you please explain why it would be desirable to [backport nonlocal]? 2.7 will likely be the last 2.x release, so only a fairly small portion of the applications would be actually able to use this (or any other new feature added to 2.7): most code supporting 2.x will also have to support 2.6, so the keyword won't be available to such code, anyway.

Others have explained the rationale for the backport, so I won't bother repeating those arguments. I understand your point about code supporting 2.6, but as you say, that applies to any new features being added in 2.7. I'm therefore confused as to what the rationale for a 2.7 release is. It's a bump in minor release number, so it's purpose is to add new features, correct? Could someone please explain what new features are currently envisioned for 2.7? Why would those make the cut but a not backport of nonlocal? Mike

"Martin v. Löwis"

4:15 a.m.

...

Others have explained the rationale for the backport, so I won't bother repeating those arguments.

I understand your point about code supporting 2.6, but as you say, that applies to any new features being added in 2.7. I'm therefore confused as to what the rationale for a 2.7 release is. It's a bump in minor release number, so it's purpose is to add new features, correct?

That, but not only - it's purpose is also to allow for certain incompatible changes, and to arrive at a version that will be maintained until 2015 (wrt. security fixes).

...

Could someone please explain what new features are currently envisioned for 2.7?

Read through Misc/NEWS for details of what has been added already.

...

Why would those make the cut but a not backport of nonlocal?

Just to name a few changes: - Issue #1655: Make imaplib IPv6-capable. Patch by Derek Morr. - Issue #4915: Port sysmodule to Windows CE. - Issue #6101: A new opcode, SETUP_WITH, has been added to speed up the with statement and correctly lookup the __enter__ and __exit__ special methods. - Issue #1616979: Added the cp720 (Arabic DOS) encoding. In all of these cases, in order to use the new feature, no change to Python code will be necessary: the feature just becomes available transparently. This is the kind of new features I like to see in 2.7. The other kind (i.e. features that do require application changes in order to use them) I find questionable. Regards, Martin

Karen Tracey

2:05 p.m.

On Thu, Oct 22, 2009 at 5:51 PM, "Martin v. Löwis" wrote:

...

...
From the Django roadmap for supporting 3.0, using 2.6 as a stepping stone (and if 2.7 was a *better* stepping stone then it would make it easier):

http://groups.google.com/group/django-developers/msg/0888b1c8f2518059 ?

Is that still a current plan? It's from November 2008.

Django 1.1 came out in 2H09, not 1H09, and Django 1.2 is now looking to come out in 1H10, not 2H09, so the dates in that note have slipped out by 3-6 months. (The labeling of some features as must-have for a release has been dropped for 1.2, so as to hopefully prevent the kind of slip seen with 1.1.) Django 1.2 is still scheduled to drop Python 2.3 support. I think it's too early to say whether Python 2.4 support will be dropped with 1.3, nor do I have any good idea when supporting 3.x will become a priority. Karen

Lennart Regebro

28 Oct 28 Oct

2:30 a.m.

2009/10/22 "Martin v. Löwis" :

...

What use has such a stepping stone? Why, and (more importantly) when would anybody currently supporting 2.x give up 2.6 and earlier, and only support 2.7? And, if they chose to do so, why would they not move the code base to 3.x right away?

Python 2 support is not only about supporting old versions of Python, but also supporting users of Python2-only modules. If you have a module that runs only on Python 2.7, the people who for some reason can not move to Python 3 yet, can still use it, by running Python 2.7. For that your module doesn't have to run on 2.6 or 2.5. So 2.7 support will for the most part be a case not of supporting Python versions, but Python *users*. Which is a good thing. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Antoine Pitrou

5:12 a.m.

Lennart Regebro writes:

...

So 2.7 support will for the most part be a case not of supporting Python versions, but Python *users*.

That's still not a good reason to backport nonlocal. The same reasoning could be used to backport new features to the 2.6 branch after all. Regards Antoine.

skip＠pobox.com

9:35 a.m.

>> So 2.7 support will for the most part be a case not of supporting >> Python versions, but Python *users*. Antoine> That's still not a good reason to backport nonlocal. The same Antoine> reasoning could be used to backport new features to the 2.6 Antoine> branch after all. No, because 2.6 is in feature freeze (bug fixes only). 2.7 is the current version of 2.x where new features are allowed to be added. Skip

Antoine Pitrou

10:49 a.m.

writes:

...

>> So 2.7 support will for the most part be a case not of supporting >> Python versions, but Python *users*.

Antoine> That's still not a good reason to backport nonlocal. The same Antoine> reasoning could be used to backport new features to the 2.6 Antoine> branch after all.

No, because 2.6 is in feature freeze (bug fixes only). 2.7 is the current version of 2.x where new features are allowed to be added.

That was precisely my point. There are development practices which mitigate the idea that backporting is always helpful to the user. Regards Antoine.

Lennart Regebro

29 Oct 29 Oct

4:15 a.m.

2009/10/28 Antoine Pitrou :

...

writes:

...
>> So 2.7 support will for the most part be a case not of supporting >> Python versions, but Python *users*.

Antoine> That's still not a good reason to backport nonlocal. The same Antoine> reasoning could be used to backport new features to the 2.6 Antoine> branch after all.

No, because 2.6 is in feature freeze (bug fixes only). 2.7 is the current version of 2.x where new features are allowed to be added.

That was precisely my point.

Then I don't understand what you are saying. Obviously we shouldn't backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I don't see what 2.6 has to do with it.

...

There are development practices which mitigate the idea that backporting is always helpful to the user.

And those are? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Nick Coghlan

5:23 a.m.

Lennart Regebro wrote:

...

2009/10/28 Antoine Pitrou :

...
writes:

...
>> So 2.7 support will for the most part be a case not of supporting >> Python versions, but Python *users*.

Antoine> That's still not a good reason to backport nonlocal. The same Antoine> reasoning could be used to backport new features to the 2.6 Antoine> branch after all.

No, because 2.6 is in feature freeze (bug fixes only). 2.7 is the current version of 2.x where new features are allowed to be added.

That was precisely my point.

Then I don't understand what you are saying. Obviously we shouldn't backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I don't see what 2.6 has to do with it.

...
There are development practices which mitigate the idea that backporting is always helpful to the user.

And those are?

You said it above yourself: "bugfix mode" That's all Antoine's point was - backporting of new features to previous branches is not automatically a good idea. In the case of 3.2 -> 2.7 backports, there are issues with the initial development time investment to do the backport, future double-keying of additional maintenance issues, consideration of possible poor interaction with legacy features in the 2.x series. It's a bunch of additional work that isn't going to happen without someone volunteering to do it. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Lennart Regebro

1 Nov 1 Nov

4:26 p.m.

2009/10/29 Nick Coghlan :

...

Lennart Regebro wrote:

...
2009/10/28 Antoine Pitrou :

...
writes:

...
>> So 2.7 support will for the most part be a case not of supporting >> Python versions, but Python *users*.

Antoine> That's still not a good reason to backport nonlocal. The same Antoine> reasoning could be used to backport new features to the 2.6 Antoine> branch after all.

No, because 2.6 is in feature freeze (bug fixes only). 2.7 is the current version of 2.x where new features are allowed to be added. That was precisely my point.

Then I don't understand what you are saying. Obviously we shouldn't backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I don't see what 2.6 has to do with it.

...
There are development practices which mitigate the idea that backporting is always helpful to the user.

And those are?

You said it above yourself: "bugfix mode"

Again: Then I don't understand what you are saying. Obviously we shouldn't backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I don't see what 2.6 has to do with it.

...

That's all Antoine's point was - backporting of new features to previous branches is not automatically a good idea.

Obviously we shouldn't backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I don't see what 2.6 has to do with it.

...

In the case of 3.2 -> 2.7 backports, there are issues with the initial development time investment to do the backport, future double-keying of additional maintenance issues, consideration of possible poor interaction with legacy features in the 2.x series. It's a bunch of additional work that isn't going to happen without someone volunteering to do it.

Yes? I feel that you are saying things that are obvious, and that you then expect me to draw a conclusion from this, that you probably find obvious too, but I don't. So I still don't understand what you are saying, or how this in any way contradicts what I said, or clarified or expanded on the matter. As I said: Python 2 support is not only about supporting old versions of Python, but also supporting users of Python2-only modules. So 2.7 support will for the most part be a case not of supporting Python versions, but Python *users*. And contrary to what Antoine said, that *is* a good reason to backport it. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Raymond Hettinger

4:39 p.m.

...

As I said: Python 2 support is not only about supporting old versions of Python, but also supporting users of Python2-only modules. So 2.7 support will for the most part be a case not of supporting Python versions, but Python *users*. And contrary to what Antoine said, that *is* a good reason to backport it.

FWIW, I support backporting the nonlocal-keyword in 2.7. All of the reasons for introducting nonlocal to 3.x also apply to 2.x. Using the nonlocal keyword in clear and explicit, especially when compared to the existing workarounds which are not pretty. Raymond

Brett Cannon

6:57 p.m.

On Sun, Nov 1, 2009 at 13:39, Raymond Hettinger wrote:

...

...
As I said: Python 2 support is not only about supporting old versions of Python, but also supporting users of Python2-only modules. So 2.7 support will for the most part be a case not of supporting Python versions, but Python *users*. And contrary to what Antoine said, that *is* a good reason to backport it.

FWIW, I support backporting the nonlocal-keyword in 2.7. All of the reasons for introducting nonlocal to 3.x also apply to 2.x. Using the nonlocal keyword in clear and explicit, especially when compared to the existing workarounds which are not pretty.

We are voting, I'm -0. -Brett

Michael Foord

2 Nov 2 Nov

3:29 a.m.

Brett Cannon wrote:

...

On Sun, Nov 1, 2009 at 13:39, Raymond Hettinger wrote:

...
...
As I said: Python 2 support is not only about supporting old versions of Python, but also supporting users of Python2-only modules. So 2.7 support will for the most part be a case not of supporting Python versions, but Python *users*. And contrary to what Antoine said, that *is* a good reason to backport it.

FWIW, I support backporting the nonlocal-keyword in 2.7. All of the reasons for introducting nonlocal to 3.x also apply to 2.x. Using the nonlocal keyword in clear and explicit, especially when compared to the existing workarounds which are not pretty.

We are voting, I'm -0.

nonlocal is really useful for tests so I'd love to have this in 2.7, but obviously someone has to do the work so I'm only +0. Michael

...

-Brett _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.ironpythoninaction.com/

Guido van Rossum

1:01 p.m.

I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from __future__ import nonlocal_keyword" (or some such phrasing) to enable it. -- --Guido van Rossum (python.org/~guido)

Raymond Hettinger

1:06 p.m.

[Guido van Rossum]

...

I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from __future__ import nonlocal_keyword" (or some such phrasing) to enable it.

With the "from __future__" option, what keeps you from being a full +1 on nonlocal? Is there something that makes it a better solution for 3.x than 2.x? Just curious about the pros and cons from your point of view. Raymond

Guido van Rossum

1:09 p.m.

On Mon, Nov 2, 2009 at 10:06 AM, Raymond Hettinger wrote:

...

[Guido van Rossum]

...
I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from __future__ import nonlocal_keyword" (or some such phrasing) to enable it.

With the "from __future__" option, what keeps you from being a full +1 on nonlocal? Is there something that makes it a better solution for 3.x than 2.x? Just curious about the pros and cons from your point of view.

I think the number of projects that can afford to drop support for 2.6 is pretty limited, so I think the utility of the feature is thereby also limited. -- --Guido van Rossum (python.org/~guido)

R. David Murray

2:18 p.m.

On Mon, 2 Nov 2009 at 10:09, Guido van Rossum wrote:

...

On Mon, Nov 2, 2009 at 10:06 AM, Raymond Hettinger wrote:

...
[Guido van Rossum]

...
I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from __future__ import nonlocal_keyword" (or some such phrasing) to enable it.

With the "from __future__" option, what keeps you from being a full +1 on nonlocal? � Is there something that makes it a better solution for 3.x than 2.x? �Just curious about the pros and cons from your point of view.

I think the number of projects that can afford to drop support for 2.6 is pretty limited, so I think the utility of the feature is thereby also limited.

I don't currently have an opinion on this backport proposal, but in regard to this argument: if we do not do any 2.x releases after 2.7, then over time the number of packages that can afford to drop 2.6 support will grow, yet many will need to retain 2.7 support for much longer. --David

"Martin v. Löwis"

4:17 p.m.

...

I don't currently have an opinion on this backport proposal, but in regard to this argument: if we do not do any 2.x releases after 2.7, then over time the number of packages that can afford to drop 2.6 support will grow, yet many will need to retain 2.7 support for much longer.

I don't think the argument applies to 2.7 as much as it applied to earlier releases: 2.7 will have a life time of 18 months perhaps (I think we still need to decide formally against 2.8, and also decide when to make the last 2.7 bug fix release). There is some likelihood that by the time people can agree to drop 2.6 support, 2.7 will be out of bug fix maintenance already (*). I'm fairly skeptical that people will be interested in adding new code to specifically clean up their 2.x codebase. Regards, Martin (*) just as it happened with any other release: people are now dropping support for 2.3, when this got out of security fixes a year ago. By that measure, people will drop 2.6 support in 2015. I think there are *really* good chances that many packages will drop 2.6 support along with dropping 2.x altogether.

R. David Murray

4:27 p.m.

On Mon, 2 Nov 2009 at 22:17, "Martin v. L�wis" wrote:

...

...
I don't currently have an opinion on this backport proposal, but in regard to this argument: if we do not do any 2.x releases after 2.7, then over time the number of packages that can afford to drop 2.6 support will grow, yet many will need to retain 2.7 support for much longer.

I don't think the argument applies to 2.7 as much as it applied to earlier releases: 2.7 will have a life time of 18 months perhaps (I think we still need to decide formally against 2.8, and also decide when to make the last 2.7 bug fix release). There is some likelihood

I was under the impression that if 2.7 was the last release that it would be maintained (ie: bugfixed) until we decided 3.x uptake was "sufficient", and that that might be considerably longer than 18 months. If that is not the case, then what you say is true. --David

Guido van Rossum

4:33 p.m.

On Mon, Nov 2, 2009 at 1:27 PM, R. David Murray wrote:

...

On Mon, 2 Nov 2009 at 22:17, "Martin v. Löwis" wrote:

...
...
I don't currently have an opinion on this backport proposal, but in regard to this argument: if we do not do any 2.x releases after 2.7, then over time the number of packages that can afford to drop 2.6 support will grow, yet many will need to retain 2.7 support for much longer.

I don't think the argument applies to 2.7 as much as it applied to earlier releases: 2.7 will have a life time of 18 months perhaps (I think we still need to decide formally against 2.8, and also decide when to make the last 2.7 bug fix release). There is some likelihood

I was under the impression that if 2.7 was the last release that it would be maintained (ie: bugfixed) until we decided 3.x uptake was "sufficient", and that that might be considerably longer than 18 months. If that is not the case, then what you say is true.

Is it even wort doing a 2.7 release? Isn't the effort better spent on 3.2 alone? (Note, these aren't rhetorical questions. It's well possible that there are good reasons for pushing along with 2.7. Maybe considering those reasons will also help answering questions about whether to backport things like nonlocal.) -- --Guido van Rossum (python.org/~guido)

"Martin v. Löwis"

4:33 p.m.

R. David Murray wrote:

...

On Mon, 2 Nov 2009 at 22:17, "Martin v. Löwis" wrote:

...
...
I don't currently have an opinion on this backport proposal, but in regard to this argument: if we do not do any 2.x releases after 2.7, then over time the number of packages that can afford to drop 2.6 support will grow, yet many will need to retain 2.7 support for much longer.

I don't think the argument applies to 2.7 as much as it applied to earlier releases: 2.7 will have a life time of 18 months perhaps (I think we still need to decide formally against 2.8, and also decide when to make the last 2.7 bug fix release). There is some likelihood

I was under the impression that if 2.7 was the last release that it would be maintained (ie: bugfixed) until we decided 3.x uptake was "sufficient", and that that might be considerably longer than 18 months. If that is not the case, then what you say is true.

I think that's as-yet undecided. My understanding was that we would decide, at some point, whether to create a 2.8 release or not, and if not, that 2.7 would be the final release. To me, this always implied that there wouldn't be any bug fix releases after 18 months, and no security releases after five years. If we would decide to continue doing 2.x releases, the we definitely wouldn't go beyond 2.9 (because there can't be a 2.10 release, numerically). At a rate of a release every 18 month, this would mean we would make bug fix releases for 2.x for no more than 5 years (from now). But I would really hope that we stop before 2.9. Regards, Martin

Antoine Pitrou

4:41 p.m.

Guido van Rossum writes:

...

Is it even wort doing a 2.7 release? Isn't the effort better spent on 3.2 alone? (Note, these aren't rhetorical questions. It's well possible that there are good reasons for pushing along with 2.7. Maybe considering those reasons will also help answering questions about whether to backport things like nonlocal.)

2.7 has an up-to-date backport of the C IO lib (as well as the memoryview object), which means it is better for people wanting to ease transition to 3.x. But of course, as Martin said, few people will want to support 2.7 only and not 2.6.

"Martin v. Löwis"

4:42 p.m.

...

Is it even wort doing a 2.7 release? Isn't the effort better spent on 3.2 alone? (Note, these aren't rhetorical questions. It's well possible that there are good reasons for pushing along with 2.7. Maybe considering those reasons will also help answering questions about whether to backport things like nonlocal.)

If no 2.7 is produced, people will ask for the life time of 2.6. I think users will appreciate not being forced to 3.x as long as possible, which essentially means as long as we create 2.x releases. It may be that users could also "live" with 2.6 bug fix releases only - in this case, we would have to decide for how long these will have to be created. There is also the issue of committers who already added stuff to the trunk after the 2.6 release - some may be unhappy to learn that their stuff is not going to be released (at least not on that branch). In the end, the EOL for 2.x will need to be decided by BDFL pronouncement. Users would continue to ask for 2.x releases until 2020 and beyond if all they have to do is ask. Some committers would likely also want to continue creating 2.x releases for a few more years (not sure whether that's because they see themselves also as users, or because they sympathize with the users, or for other reasons). In any case, a decision not to release 2.7 should be made before Benjamin produces the first alpha release. Regards, Martin

M.-A. Lemburg

3 Nov 3 Nov

4:10 a.m.

Antoine Pitrou wrote:

...

Guido van Rossum writes:

...
Is it even wort doing a 2.7 release? Isn't the effort better spent on 3.2 alone? (Note, these aren't rhetorical questions. It's well possible that there are good reasons for pushing along with 2.7. Maybe considering those reasons will also help answering questions about whether to backport things like nonlocal.)

2.7 has an up-to-date backport of the C IO lib (as well as the memoryview object), which means it is better for people wanting to ease transition to 3.x.

But of course, as Martin said, few people will want to support 2.7 only and not 2.6.

Since 2.7 will be closer to 3.2 than 2.6, the result will more likely be: many people will want to support 2.7 and 3.x - which is really what we should be after. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 03 2009)

...

...
...
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

"Martin v. Löwis"

12:53 p.m.

...

...
2.7 has an up-to-date backport of the C IO lib (as well as the memoryview object), which means it is better for people wanting to ease transition to 3.x.

But of course, as Martin said, few people will want to support 2.7 only and not 2.6.

Since 2.7 will be closer to 3.2 than 2.6, the result will more likely be: many people will want to support 2.7 and 3.x - which is really what we should be after.

Supporting 2.7 doesn't mean that you have to use the nonlocal keyword (nor does support for 3.2 require that keyword). So I'm not sure how the backport of a feature to 2.x could encourage people to forward-port to 3.x. Regards, Martin

M.-A. Lemburg

2:15 p.m.

"Martin v. Löwis" wrote:

...

...
...
2.7 has an up-to-date backport of the C IO lib (as well as the memoryview object), which means it is better for people wanting to ease transition to 3.x.

But of course, as Martin said, few people will want to support 2.7 only and not 2.6.

Since 2.7 will be closer to 3.2 than 2.6, the result will more likely be: many people will want to support 2.7 and 3.x - which is really what we should be after.

Supporting 2.7 doesn't mean that you have to use the nonlocal keyword (nor does support for 3.2 require that keyword). So I'm not sure how the backport of a feature to 2.x could encourage people to forward-port to 3.x.

True. This particular backport is not all that useful - it is well possible to write code that doesn't need the feature, both in 2.7 and 3.2. However, if a 3.2 feature is required for code to work out of the box without changes, then I'd regard such a feature as potential backport candidate. memoryviews are a good example. In 3.x the buffer object is gone and memoryviews are the new replacement for it. Code written to support both 2.7 and 3.2 would need to have access to these memoryviews in order to avoid having to write special support for buffer objects (for 2.7) and memoryviews (for 3.x). By backporting memoryviews to 2.7, users no longer have to write support code for buffer objects and that makes things easier for them. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 03 2009)

...

...
...
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

Lennart Regebro

4 Nov 4 Nov

2:06 a.m.

I should maybe point out that although I'm generally +1 on backporting, I'm not specifically anything on backporting the nonlocal keyword. There are probably things that would help more from a compatibility standpoint than that. For example, from __future__ import unicode_literals doesn't switch the types. So this code will rais an assertion error in 2.6.

...

...
...
from __future__ import unicode_literals assert isinstance("fghjkl", str)

While it of course will run fine in 3.0. Now this has a fairly trivial workaround or two, but still. I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Nick Coghlan

7 a.m.

New subject: Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)

Lennart Regebro wrote:

...

I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work.

Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard library needed to be tweaked to cope with bytes objects in 3.x. Generally speaking, the "bytes = str" trick represents a reasonable compromise as the APIs that you would pass a bytes object to in 3.x expect an 8-bit str instance in 2.x. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

M.-A. Lemburg

7:21 a.m.

New subject: Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)

Nick Coghlan wrote:

...

Lennart Regebro wrote:

...
I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work.

Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard library needed to be tweaked to cope with bytes objects in 3.x.

Generally speaking, the "bytes = str" trick represents a reasonable compromise as the APIs that you would pass a bytes object to in 3.x expect an 8-bit str instance in 2.x.

Could you please add such hints, tricks and tips to the wiki page: http://wiki.python.org/moin/PortingToPy3k Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 04 2009)

...

...
...
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

Nick Coghlan

8:20 a.m.

New subject: Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)

M.-A. Lemburg wrote:

...

Nick Coghlan wrote:

...
...
I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work. Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard

Lennart Regebro wrote: library needed to be tweaked to cope with bytes objects in 3.x.

Generally speaking, the "bytes = str" trick represents a reasonable compromise as the APIs that you would pass a bytes object to in 3.x expect an 8-bit str instance in 2.x.

Could you please add such hints, tricks and tips to the wiki page:

http://wiki.python.org/moin/PortingToPy3k

Done (although the task of figuring out how to get the wiki to display code properly defeated me... the normal Python documentation syntax for it seemed to give the wiki's ReST interpreter fits). I also mentioned the trick someone mentioned about "from __future__ import unicode_literals" not changing the meaning of 'str' since it only alters the parser but not the builtins. In writing it up, it occurred to me that having that kind of thing in a "py3_compat" compatibility module (to be used as, e.g., "from py3_compat import str, bytes") would not only make it easier to use in multiple modules, but also easier for 2to3 to remove when forward porting. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Guido van Rossum

11:53 a.m.

On Tue, Nov 3, 2009 at 11:06 PM, Lennart Regebro wrote:

...

I should maybe point out that although I'm generally +1 on backporting, I'm not specifically anything on backporting the nonlocal keyword. There are probably things that would help more from a compatibility standpoint than that.

For example, from __future__ import unicode_literals doesn't switch the types. So this code will rais an assertion error in 2.6.

...
...
...
from __future__ import unicode_literals assert isinstance("fghjkl", str)

While it of course will run fine in 3.0. Now this has a fairly trivial workaround or two, but still.

I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work.

There are many interfaces that return a str instance which should become a bytes instance for code that is bytes-aware, but should stay a str instance (which after all has the same representation) for code that is not bytes-aware. In 3.x it's easy, such interfaces always return bytes instances. But in 2.x, it would have to be context-aware, and there are many reasons why that can't work. (E.g. an object could be shared between two modules, one of which is bytes-aware, while the other is not. Or a call could be made by a module that isn't bytes-aware and the return value passed to a module that is bytes-aware. Or vice versa. It's a mess unless *everyone* is bytes-aware, which is the 3.x world.) -- --Guido van Rossum (python.org/~guido)

"Martin v. Löwis"

1:37 p.m.

New subject: Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)

Nick Coghlan wrote:

...

Lennart Regebro wrote:

...
I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work.

Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard library needed to be tweaked to cope with bytes objects in 3.x.

I see the problem differently: if a bytes type was added, nothing would use it. In particular, IO wouldn't start returning bytes (although it could accept them); IO would continue to return str. Therefore, I'm skeptical that adding a *third* string type to 3.x would do any good. Regards, Martin

M.-A. Lemburg

5 Nov 5 Nov

5:56 a.m.

New subject: Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)

Nick Coghlan wrote:

...

M.-A. Lemburg wrote:

...
Nick Coghlan wrote:

...
...
I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work. Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard

Lennart Regebro wrote: library needed to be tweaked to cope with bytes objects in 3.x.

Generally speaking, the "bytes = str" trick represents a reasonable compromise as the APIs that you would pass a bytes object to in 3.x expect an 8-bit str instance in 2.x.

Could you please add such hints, tricks and tips to the wiki page:

http://wiki.python.org/moin/PortingToPy3k

Done (although the task of figuring out how to get the wiki to display code properly defeated me... the normal Python documentation syntax for it seemed to give the wiki's ReST interpreter fits).

Thanks.

...

I also mentioned the trick someone mentioned about "from __future__ import unicode_literals" not changing the meaning of 'str' since it only alters the parser but not the builtins.

In writing it up, it occurred to me that having that kind of thing in a "py3_compat" compatibility module (to be used as, e.g., "from py3_compat import str, bytes") would not only make it easier to use in multiple modules, but also easier for 2to3 to remove when forward porting.

That would certainly be useful to have, yes, since all developers wanting to support both 2.7 and 3.2 using the same code base will have a need for these things. It would also help to have a package that contains proxy modules using the Python 3.x module/package names which map the functionality back to the Python 2.x modules. This avoids problems with freezing tools that scan the source code for imports as well as pickles which store the full dotted path name to classes. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 05 2009)

...

...
...
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

Nick Coghlan

7:19 a.m.

New subject: Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)

Martin v. Löwis wrote:

...

Nick Coghlan wrote:

...
...
I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work. Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard

Lennart Regebro wrote: library needed to be tweaked to cope with bytes objects in 3.x.

I see the problem differently: if a bytes type was added, nothing would use it. In particular, IO wouldn't start returning bytes (although it could accept them); IO would continue to return str. Therefore, I'm skeptical that adding a *third* string type to 3.x would do any good.

I'm guessing you meant '2.x' in that last sentence, in which case we agree (just emphasising different parts of the "binary data" ecosystem that would be necessary to make a backported bytes type effective). Although I think Guido did a better job than either of us in explaining why backporting the full bytes type to 2.x really wouldn't help all that much. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Lennart Regebro

3:36 p.m.

New subject: Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)

2009/11/4 Nick Coghlan :

...

In writing it up, it occurred to me that having that kind of thing in a "py3_compat" compatibility module (to be used as, e.g., "from py3_compat import str, bytes") would not only make it easier to use in multiple modules, but also easier for 2to3 to remove when forward porting.

Well, when using 2to3 it already handles that stuff. But a module like that would be very handy if you want to support both 2.6 and 3.x without 2to3. With such a module it would be quite simple. In fact, I think the module should be called "timemachine". ;-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Lennart Regebro

3:38 p.m.

New subject: Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)

2009/11/4 Nick Coghlan :

...

Lennart Regebro wrote:

...
I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work.

Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard library needed to be tweaked to cope with bytes objects in 3.x.

Ah, right, that was the problem, the standard library. I knew I heard a good reason against it.

...

Generally speaking, the "bytes = str" trick represents a reasonable compromise as the APIs that you would pass a bytes object to in 3.x expect an 8-bit str instance in 2.x.

Yeah, the problem being that bytes and str has quite different API's. Ah well. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

5283

Age (days ago)

5297

Last active (days ago)

List overview

Download

44 comments

17 participants

participants (17)

"Martin v. Löwis"
Antoine Pitrou
Brett Cannon
exarkun＠twistedmatrix.com
Fabio Zadrozny
Fernando Perez
Guido van Rossum
Karen Tracey
Lennart Regebro
M.-A. Lemburg
Michael Foord
Mike Krell
Neal Norwitz
Nick Coghlan
R. David Murray
Raymond Hettinger
skip＠pobox.com

nonlocal keyword in 2.x?

tags

participants (17)