Nick Coghlan <ncoghlan <at> gmail.com> writes:
It reduces the problem (compared to omitting the import and using a u() function), but it's still ugly and still involves the "action at a distance" of the unicode literals import.
I agree about the action-at-a-distance leading to non-obvious bugs and wasted head-scratching time caused by such. It could be mitigated somewhat by project-level conventions, e.g. that all string literals are Unicode on that project. Then, if you put yourself in the relevant mindset when working on that project, there are fewer surprises. It's probably a matter of choosing the lesser among evils, since the proposal seems to allow mixing of literals with and without u prefixes in 3.x code - doesn't that also seem ugly? When this came up earlier (when I think Chris McDonough raised it) the issue of what to do on 3.2 came up, and though it has been addressed somewhat in the PEP, it would be nice to see the suggested on-installation hook fleshed out a little more. Regards, Vinay Sajip
Hi, On 2/26/12 12:42 PM, Vinay Sajip wrote:
When this came up earlier (when I think Chris McDonough raised it) the issue of what to do on 3.2 came up, and though it has been addressed somewhat in the PEP, it would be nice to see the suggested on-installation hook fleshed out a little more. I wanted to do that but the tokenizer module is quite ugly to customize in order to allow "u" prefixes to strings which is why I postponed that. It would work similar to how 2to3 is invoked however.
In case this PEP gets approved I will refactor the tokenize module while adding support for "u" prefixes and use that as the basis for a installation hook for older Python 3 versions. Regards, Armin
On 2/26/2012 7:46 AM, Armin Ronacher wrote: I am not enthusiastic about adding duplication that is useless for writing Python 3 code, but like others, I do want to encourage more porting of libraries to run with Python 3. I understand that the unicode transition seems the be the biggest barrier, especially for some applications. It is OK with me if ported code only runs on 3.3+, with its improved unicode. If u'' is added, I would like it to be added as deprecated in the doc with a note that it is only intended for multi-version Python 2/3 code.
In case this PEP gets approved I will refactor the tokenize module while adding support for "u" prefixes and use that as the basis for a installation hook for older Python 3 versions.
I presume such a hook would simply remove 'u' prefixes and would run *much* faster than 2to3. If such a hook is satisfactory for 3.2, why would it not be satisfactory for 3.3? -- Terry Jan Reedy
On Mon, Feb 27, 2012 at 11:55 AM, Terry Reedy <tjreedy@udel.edu> wrote:
I presume such a hook would simply remove 'u' prefixes and would run *much* faster than 2to3. If such a hook is satisfactory for 3.2, why would it not be satisfactory for 3.3?
Because an import hook is still a lot more complicated than "Write modern code that runs on 2.6+ and follows certain guidelines and it will also just run on 3.3+". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi, On 2/27/12 1:55 AM, Terry Reedy wrote:
I presume such a hook would simply remove 'u' prefixes and would run *much* faster than 2to3. If such a hook is satisfactory for 3.2, why would it not be satisfactory for 3.3? Agile development and unittests. An installation hook means that you need to install the package before running the tests. Which is fine for CI but horrible during development. "python3 run-tests.py" beats "make venv; install library; run testsuite" anytime in terms of development speed.
Regards, Armin
On Mon, Feb 27, 2012 at 5:44 PM, Armin Ronacher <armin.ronacher@active-4.com> wrote:
Agile development and unittests. An installation hook means that you need to install the package before running the tests. Which is fine for CI but horrible during development. "python3 run-tests.py" beats "make venv; install library; run testsuite" anytime in terms of development speed.
"python3 setup.py test" works already with 2to3 (it builds the code and runs the tests under build/). It is however slow and it can be a bit annoying to have to debug things by looking at the generated code under build/lib.linux-i686-3.2/ (or similar).
On 2/27/2012 10:44 AM, Armin Ronacher wrote:
On 2/27/12 1:55 AM, Terry Reedy wrote:
I presume such a hook would simply remove 'u' prefixes and would run *much* faster than 2to3. If such a hook is satisfactory for 3.2, why would it not be satisfactory for 3.3?
Agile development and unittests.
Which I am all for. So the issue is not 3.3 versus 3.1/2, but development versus installation. I somehow did not get that reading the PEP but it seems a crucial point in its favor.
An installation hook means that you need to install the package before running the tests. Which is fine for CI but horrible during development. "python3 run-tests.py" beats "make venv; install library; run testsuite" anytime in terms of development speed.
That I can appreciate. It makes programming more fun. I presume you are saying that you would run the 'Python 3' tests quickly with 3.3 in your normal development cycle. Then, if you want your library to also run under 3.1/2, only occasionally (daily?) check that they also run under a 3.1/2 installation. That *does* make sense to me. -- Terry Jan Reedy
Terry Reedy <tjreedy <at> udel.edu> writes:
An installation hook means that you need to install the package before running the tests. Which is fine for CI but horrible during development. "python3 run-tests.py" beats "make venv; install library; run testsuite" anytime in terms of development speed.
That I can appreciate. It makes programming more fun. I presume you are saying that you would run the 'Python 3' tests quickly with 3.3 in your normal development cycle. Then, if you want your library to also run under 3.1/2, only occasionally (daily?) check that they also run under a 3.1/2 installation. That *does* make sense to me.
Right, but Armin, while arguing against an installation hook for 2to3, is ISTM arguing for an analogous hook for use with 3.2 (and earlier 3.x), which does a smaller amount of work than 2to3 but is the same kind of beast. The "programming fun" part is really an argument about a single codebase, which I am completely in agreement with. But (summarising for my own benefit, but someone please tell me if I've missed or misunderstood something) there are (at least) three ways to get there: 1. Support only 2.6+ code, use from __future__ import unicode_literals, do away with u'xxx' in favour of 'xxx'. This has been opposed because of action-at-a-distance, but can be mitigated by strongly applied discipline on a given project (so that everyone knows that all string literals are Unicode, period). Of course, the same discipline needs to be applied to depended-upon projects, too. 2. Support 2.5 or earlier, where you would have to use u('xxx') in place of u'xxx', unless PEP 414 is accepted - but you would still have the exception syntax hacks to upset you. This has been opposed because of performance and productivity concerns, but I don't think these are yet proven in practice (for performance, microbenchmarks notwithstanding - there's no data on more representative workloads. For productivity I call shenanigans, since if we can trust 2to3 to work automatically, we should be able to trust a 2to3 fixer to do the work on u'xxx' -> u('xxx') or u('xxx') -> 'xxx' automatically). 3. Do one of the above, but approve this PEP and keep u'xxx' literals around for some yet-to-be-determined time, but perhaps the life of Python 3. This has been called a retrograde step, and one can see why; ISTM the main reason for accepting this path is that some fairly vocal and respected developers don't want to (as opposed to can't) take one of the other paths, and are basically saying they're not porting their work to 3.x unless this path is taken. They're saying between the lines that their views are representative of a fair number of other less vocal developers, who are also not porting their code for the same reason. (ISTM they're glossing over the other issues which come up in a 2.x -> 3.x port, which require more time to diagnose and fix than problems caused by string literals.) But never mind that - if we're worried about the pace of the 2.x -> 3.x transition, we can appease these views fairly easily, so why not do it? And while we're at it, we can perhaps also look at those pesky exception clauses and see if we can't get 3.x to support 2.x exception syntax, to make that porting job even easier ;-) Regards, Vinay Sajip
On 2/27/2012 10:44 AM, Armin Ronacher wrote:
On 2/27/12 1:55 AM, Terry Reedy wrote:
I presume such a hook would simply remove 'u' prefixes and would run *much* faster than 2to3. If such a hook is satisfactory for 3.2, why would it not be satisfactory for 3.3?
Agile development and unittests.
Given that last December you wrote "And in my absolutely personal opinion Python 3.3/3.4 should be more like Python 2* and Python 2.8 should happen and be a bit more like Python 3." * you wrote '3' but obviously must have meant '2'. http://lucumr.pocoo.org/2011/12/7/thoughts-on-python3/ I would like to know if you think that this one change is enough to do agile development and testing, etc, or whether, as Chris McDonough hopes, this is just the first of a series of proposals you have planned. -- Terry Jan Reedy
Hi, On 2/27/12 10:18 PM, Terry Reedy wrote:
I would like to know if you think that this one change is enough to do agile development and testing, etc, or whether, as Chris McDonough hopes, this is just the first of a series of proposals you have planned. Indeed I have three other PEPs in the work. The reintroduction of "except (((ExceptionType),),)", the "<>" comparision operator and the removal of "nonlocal", the latter to make Python 2.x developers feel better about themselves. :-)
Regards, Armin
On Feb 27, 2012, at 10:38 PM, Armin Ronacher wrote:
Indeed I have three other PEPs in the work. The reintroduction of "except (((ExceptionType),),)", the "<>" comparision operator and the removal of "nonlocal", the latter to make Python 2.x developers feel better about themselves. :-)
One of them's a winner in my book, but I'll let you guess which one. OTOH, the time machine can bring you back to the future again. -Barry
28.02.12 00:52, Barry Warsaw написав(ла):
On Feb 27, 2012, at 10:38 PM, Armin Ronacher wrote:
Indeed I have three other PEPs in the work. The reintroduction of "except (((ExceptionType),),)", the"<>" comparision operator and the removal of "nonlocal", the latter to make Python 2.x developers feel better about themselves. :-)
One of them's a winner in my book, but I'll let you guess which one. OTOH, the time machine can bring you back to the future again.
Hi Armin, Could you please remove from the PEP the following statement: """As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3.""" While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org. The python ecosystem is not just limited to WSGI apps, Django and Flask. Yes, we don't have all the packages on pypi support python 3, but many of those are portable within 10 minutes to couple of hours of work (and I did many of such ports for our internal systems.) And many of the essential packages do exist for python 3, like numpy, zeromq etc. I know several sturt-ups, including mine that develop huge commercial applications entirely on python 3. Thanks, -Yury On 2012-02-27, at 5:38 PM, Armin Ronacher wrote:
Hi,
On 2/27/12 10:18 PM, Terry Reedy wrote:
I would like to know if you think that this one change is enough to do agile development and testing, etc, or whether, as Chris McDonough hopes, this is just the first of a series of proposals you have planned. Indeed I have three other PEPs in the work. The reintroduction of "except (((ExceptionType),),)", the "<>" comparision operator and the removal of "nonlocal", the latter to make Python 2.x developers feel better about themselves. :-)
Regards, Armin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com
On Feb 28, 2012, at 11:29 AM, Yury Selivanov wrote:
Could you please remove from the PEP the following statement:
"""As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3."""
While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org.
+∞ -Barry
Armin, I see you've (or somebody) changed: """As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3.""" to: """As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3.""" Could you just remove the statement completely? Again, my understanding of what is the best choice for certain *long-term* investments is drastically different from yours. In my opinion, python 3 is much more suitable for anything *long-term* than python 2. I don't think that PEPs are the right place to put such polemic and biased statements. Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. There are blogs for that. Thank you. - Yury On 2012-02-28, at 11:29 AM, Yury Selivanov wrote:
Hi Armin,
Could you please remove from the PEP the following statement:
"""As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3."""
While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org.
The python ecosystem is not just limited to WSGI apps, Django and Flask. Yes, we don't have all the packages on pypi support python 3, but many of those are portable within 10 minutes to couple of hours of work (and I did many of such ports for our internal systems.) And many of the essential packages do exist for python 3, like numpy, zeromq etc.
I know several sturt-ups, including mine that develop huge commercial applications entirely on python 3.
Thanks, -Yury
On 2012-02-27, at 5:38 PM, Armin Ronacher wrote:
Hi,
On 2/27/12 10:18 PM, Terry Reedy wrote:
I would like to know if you think that this one change is enough to do agile development and testing, etc, or whether, as Chris McDonough hopes, this is just the first of a series of proposals you have planned. Indeed I have three other PEPs in the work. The reintroduction of "except (((ExceptionType),),)", the "<>" comparision operator and the removal of "nonlocal", the latter to make Python 2.x developers feel better about themselves. :-)
Regards, Armin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com
On Feb 29, 2012, at 07:30 AM, Yury Selivanov wrote:
"""As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3."""
to:
"""As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3."""
Could you just remove the statement completely?
If I read correctly, Nick is undertaking a rewrite of PEP 414, which should help a lot. -Barry
I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction? -- --Guido van Rossum (python.org/~guido)
On Mar 01, 2012, at 09:42 AM, Guido van Rossum wrote:
I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction?
Not yet, but I believe Nick volunteered to do a rewrite. -Barry
Guido van Rossum <guido <at> python.org> writes:
I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction?
I'm not sure if Nick has finished his updates, but I for one would like to see some improvements in a few places: "Many thought that the unicode_literals future import might make a common source possible, but it turns out that it's doing more harm than good." Rather than talking about it doing more harm than good, it would be better to say that unicode_literals is not the best solution in some scenarios (specifically, WSGI, but any other scenarios can also be mentioned). The "more harm than good" is not true in all scenarios, but as it's worded now, it seems like it is always a bad approach. "(either by having a u function that marks things as unicode without future imports or the inverse by having a n function that marks strings as native). Unfortunately, this has the side effect of slowing down the runtime performance of Python and makes for less beautiful code." The use of u() and n() are not equivalent in the sense that n() only has to be used when unicode_literals are in effect, and the incidence of n() calls in an application would be much lower than using u() in the absence of unicode_literals. In at least some cases, it is possible that some of the APIs which fail unless native strings are provided may be broken (e.g. some database adapters expect datetimes in ISO format as native strings, where there is no apparent reason why they couldn't accept them as text). As far as "less beautiful" code is concerned, it's subjective: I see nothing especially ugly about 'xxx' for text, and certainly don't find u'xxx' "more" beautiful - and I doubt if I'm the only person with that view. The point about the added cognitive burden of semantic-changing __future__ imports is, however, quite valid. "As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3." This looks to become a self-fulfilling prophecy, if you take it seriously. You would expect that, if Python 3 is the future of Python, then Python 3 is *precisely* the choice for *long*-term investments. The ecosystem is not yet fully developed, true: but that is because some people aren't ready to grasp the nettle and undergo the short-term pain required to get things in order. By "things", I mean places in existing 2.x code where no distinction was made between bytes and text, which you could get away with because of 2.x's forgiving nature. Whether you're using unicode_literals and 'xxx' or u'xxx', these things will need to be sorted out, and the syntax element is only one possible focus. If that entire sentence is removed, it does the PEP no harm, and the PEP will antagonise fewer people. "A valid point is that this would encourage people to become dependent on Python 3.3 for their ports. Fortunately that is not a big problem since that could be fixed at installation time similar to how many projects are currently invoking 2to3 as part of their installation process." Yes, but avoiding the very pain of running 2to3 is what (at least in part) motivates the PEP in the first place. This appears to be moving the pain that 2.x developers feel when trying to move to 3.x, to people who want to support 3.2 and 3.3 and 2.6+ in the same codebase. "For Python 3.1 and Python 3.2 (even 3.0 if necessary) a simple on-installation hook could be provided that tokenizes all source files and strips away the otherwise unnecessary u prefix at installation time." There's some confusion about this hook - The PEP calls it an on-installation hook (like 2to3) but Nick said it was an import-time hook. I'm more comfortable with the latter - it has a chance of providing an acceptable performance for a large codebase, as it will only kick in when .py files are newer than their .pyc. A 2to3 like hook, when working with a large codebase like Django, is likely to be about as painful as people are finding 2to3 now (when used in an edit-test-edit-test workflow). "Possible Downsides" does not mention any possible adverse impact on single codebase for 3.2/3.3, which I mention only because it's still not clear how the hook which is to make 3.2 development easier will work (in terms of its impact on development workflow). In the section on "Modernizing code", "but to make strings cheap for both 2.x and 3.x it is nearly impossible. The way it currently works is by abusing the unicode-escape codec on Python 2.x native strings." IIUC, the unicode-escape codec is only needed if you don't use unicode_literals - am I wrong about that? How are strings not equally cheap (near enough) on 2.x and 3.x if you use unicode_literals? In the "Runtime overhead of wrappers", the times may be valid, but a rider should be added to the effect that in a realistic workload, the wrapper overhead will be somewhat diluted where wrapper calls are fairly infrequent (i.e. the unicode_literals and n() case). Of course, if the PEP is targeting Python 2.5 and earlier where unicode_literals is not available, then it should say so. I would say that the overall impression given by the PEP is that "the unicode_literals approach is not worth bothering with", and that I do not find to be true based on my own experience. Regards, Vinay Sajip
Vinay, Thank you for the comprehensive summary. Big +1. I really do hope that Nick and Armin will rectify the PEP. Otherwise, many of its points are moot, and we need to raise a question of rejecting it somehow. On 2012-03-01, at 2:00 PM, Vinay Sajip wrote:
Guido van Rossum <guido <at> python.org> writes:
I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction?
I'm not sure if Nick has finished his updates, but I for one would like to see some improvements in a few places:
"Many thought that the unicode_literals future import might make a common source possible, but it turns out that it's doing more harm than good."
Rather than talking about it doing more harm than good, it would be better to say that unicode_literals is not the best solution in some scenarios (specifically, WSGI, but any other scenarios can also be mentioned). The "more harm than good" is not true in all scenarios, but as it's worded now, it seems like it is always a bad approach.
"(either by having a u function that marks things as unicode without future imports or the inverse by having a n function that marks strings as native). Unfortunately, this has the side effect of slowing down the runtime performance of Python and makes for less beautiful code."
The use of u() and n() are not equivalent in the sense that n() only has to be used when unicode_literals are in effect, and the incidence of n() calls in an application would be much lower than using u() in the absence of unicode_literals. In at least some cases, it is possible that some of the APIs which fail unless native strings are provided may be broken (e.g. some database adapters expect datetimes in ISO format as native strings, where there is no apparent reason why they couldn't accept them as text).
As far as "less beautiful" code is concerned, it's subjective: I see nothing especially ugly about 'xxx' for text, and certainly don't find u'xxx' "more" beautiful - and I doubt if I'm the only person with that view. The point about the added cognitive burden of semantic-changing __future__ imports is, however, quite valid.
"As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3."
This looks to become a self-fulfilling prophecy, if you take it seriously. You would expect that, if Python 3 is the future of Python, then Python 3 is *precisely* the choice for *long*-term investments. The ecosystem is not yet fully developed, true: but that is because some people aren't ready to grasp the nettle and undergo the short-term pain required to get things in order. By "things", I mean places in existing 2.x code where no distinction was made between bytes and text, which you could get away with because of 2.x's forgiving nature. Whether you're using unicode_literals and 'xxx' or u'xxx', these things will need to be sorted out, and the syntax element is only one possible focus.
If that entire sentence is removed, it does the PEP no harm, and the PEP will antagonise fewer people.
"A valid point is that this would encourage people to become dependent on Python 3.3 for their ports. Fortunately that is not a big problem since that could be fixed at installation time similar to how many projects are currently invoking 2to3 as part of their installation process."
Yes, but avoiding the very pain of running 2to3 is what (at least in part) motivates the PEP in the first place. This appears to be moving the pain that 2.x developers feel when trying to move to 3.x, to people who want to support 3.2 and 3.3 and 2.6+ in the same codebase.
"For Python 3.1 and Python 3.2 (even 3.0 if necessary) a simple on-installation hook could be provided that tokenizes all source files and strips away the otherwise unnecessary u prefix at installation time."
There's some confusion about this hook - The PEP calls it an on-installation hook (like 2to3) but Nick said it was an import-time hook. I'm more comfortable with the latter - it has a chance of providing an acceptable performance for a large codebase, as it will only kick in when .py files are newer than their .pyc. A 2to3 like hook, when working with a large codebase like Django, is likely to be about as painful as people are finding 2to3 now (when used in an edit-test-edit-test workflow).
"Possible Downsides" does not mention any possible adverse impact on single codebase for 3.2/3.3, which I mention only because it's still not clear how the hook which is to make 3.2 development easier will work (in terms of its impact on development workflow).
In the section on "Modernizing code",
"but to make strings cheap for both 2.x and 3.x it is nearly impossible. The way it currently works is by abusing the unicode-escape codec on Python 2.x native strings."
IIUC, the unicode-escape codec is only needed if you don't use unicode_literals - am I wrong about that? How are strings not equally cheap (near enough) on 2.x and 3.x if you use unicode_literals?
In the "Runtime overhead of wrappers", the times may be valid, but a rider should be added to the effect that in a realistic workload, the wrapper overhead will be somewhat diluted where wrapper calls are fairly infrequent (i.e. the unicode_literals and n() case).
Of course, if the PEP is targeting Python 2.5 and earlier where unicode_literals is not available, then it should say so. I would say that the overall impression given by the PEP is that "the unicode_literals approach is not worth bothering with", and that I do not find to be true based on my own experience.
Regards,
Vinay Sajip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com
Yury Selivanov writes:
Otherwise, many of its points are moot, and we need to raise a question of rejecting it somehow.
Yury, that's not going to happen. Guido made it quite clear that he agrees with those who consider this PEP useful, obvious, and safe, and the PEP *is* approved. There has been no hint of second thoughts, and AFAICS no reason why there would be. Please be patient, as Nick has taken on the next revision of this PEP with Armin's approval, and has indicated multiple times that he may take some time to actually do it because of other personal commitments. Regards,
Hi, On 2/29/12 12:30 PM, Yury Selivanov wrote:
I see you've (or somebody) changed: Yes, I reworded that.
Could you just remove the statement completely? I will let Nick handle the PEP wording.
I don't think that PEPs are the right place to put such polemic and biased statements. Why call it polemic? If you want to use ubuntu LTS you're forcing yourself to stick to a particular Python version for a longer time. Which means you don't want to have to adjust your code. Which again means that you're better of with the Python 2.x ecosystem which is proven, does not change nearly as quickly as the Python 3 one (hopefully) so if you have the choice between those two you would chose 2.x over 3.x. That's what this sentence is supposed to say. That's not polemic, that's just a fact.
Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. That is not a personal-feeling-PEP. If people would be 100% happy with Python 3 we would not have these discussions, would we.
Why is it that I'm getting "attacked" on this mailinglist for writing this PEP, or the wording etc. Regards, Armin
On Thu, 01 Mar 2012 21:12:48 +0000, Armin Ronacher <armin.ronacher@active-4.com> wrote:
Hi,
On 2/29/12 12:30 PM, Yury Selivanov wrote:
I see you've (or somebody) changed: Yes, I reworded that.
Could you just remove the statement completely? I will let Nick handle the PEP wording.
I don't think that PEPs are the right place to put such polemic and biased statements. Why call it polemic? If you want to use ubuntu LTS you're forcing
Presumably because it comes across to him that way. Perception aside, I do think it matches the dictionary meaning of the term ("One who writes in support of one opinion, doctrine, or system, in opposition to another"), which Nick's edits will presumably fix (by addressing all sides of the argument, as a finished PEP should).
yourself to stick to a particular Python version for a longer time. Which means you don't want to have to adjust your code. Which again means that you're better of with the Python 2.x ecosystem which is proven, does not change nearly as quickly as the Python 3 one (hopefully) so if you have the choice between those two you would chose 2.x over 3.x. That's what this sentence is supposed to say. That's not polemic, that's just a fact.
Wow. I never would have guessed that from the sentence in question. I don't think I agree with your "that means" statement either, I can imagine other motivations for using an LTS. But I don't think that discussion is worth getting in to or matters for the PEP.
Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. That is not a personal-feeling-PEP. If people would be 100% happy with Python 3 we would not have these discussions, would we.
Why is it that I'm getting "attacked" on this mailinglist for writing this PEP, or the wording etc.
I think it is because people are *perceiving* that you are attacking Python3 and arguing (out of your personal experience) that porting is harder than other people (out of their personal experience) have found it to be. This presumably reflects the different problem domains people are working in. --David
Hopefully, I can say the following in a constructive way. I certainly don't mean to attack anyone personally for their closely held beliefs, though I might disagree with them. And you have the right to those beliefs and to express them in a respectful and constructive manner on this mailing list, which I think you've done. No criticisms there. However, PEPs *are* official documents from the Python developer community, so I think it's required of us to present technical issues in an honest light, yet devoid of negative connotations which harm Python. On Mar 01, 2012, at 09:12 PM, Armin Ronacher wrote:
Why call it polemic? If you want to use ubuntu LTS you're forcing yourself to stick to a particular Python version for a longer time.
Not just a particular Python 3 version, but a particular Python 2 version too. And a particular kernel version, and version of Perl, Ruby, Java, gcc, etc. etc. That's kind of the whole point of an LTS. :)
Which means you don't want to have to adjust your code. Which again means that you're better of with the Python 2.x ecosystem which is proven, does not change nearly as quickly as the Python 3 one (hopefully) so if you have the choice between those two you would chose 2.x over 3.x. That's what this sentence is supposed to say. That's not polemic, that's just a fact.
I don't agree with the conclusion. But none of that is germane to the PEP anyway. The PEP could simply say that for some domains, the ability to port code from Python 2 to Python 3 would be enhanced by the reintroduction of the u-prefix. It could even explain why WSGI applications in particular would benefit from this. That would be enough to justify Guido's acceptance of the PEP. Cheers, -Barry
Hi Armin, Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all porting scenarios with well-thought set of benchmarks, nobody would ever call your PEP "polemic". Thanks, - Yury On 2012-03-01, at 4:12 PM, Armin Ronacher wrote:
Hi,
On 2/29/12 12:30 PM, Yury Selivanov wrote:
I see you've (or somebody) changed: Yes, I reworded that.
Could you just remove the statement completely? I will let Nick handle the PEP wording.
I don't think that PEPs are the right place to put such polemic and biased statements. Why call it polemic? If you want to use ubuntu LTS you're forcing yourself to stick to a particular Python version for a longer time. Which means you don't want to have to adjust your code. Which again means that you're better of with the Python 2.x ecosystem which is proven, does not change nearly as quickly as the Python 3 one (hopefully) so if you have the choice between those two you would chose 2.x over 3.x. That's what this sentence is supposed to say. That's not polemic, that's just a fact.
Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. That is not a personal-feeling-PEP. If people would be 100% happy with Python 3 we would not have these discussions, would we.
Why is it that I'm getting "attacked" on this mailinglist for writing this PEP, or the wording etc.
Regards, Armin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com
Hi, On 3/1/12 10:38 PM, Yury Selivanov wrote:
Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that impression from the responses here (which are greatly different from the responses I got on other communication channels and by peers). You were just the unlucky mail I responded to :-)
But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all porting scenarios with well-thought set of benchmarks, nobody would ever call your PEP "polemic". I tried my best but obviously it was not good enough to please everybody. In all honesty I did not expect that such a small change would spawn such a great discussion. After all what we're discussing here is the introduction of one letter to literals :-)
Regards, Armin
On 2012-03-01, at 5:52 PM, Armin Ronacher wrote:
Hi,
On 3/1/12 10:38 PM, Yury Selivanov wrote:
Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that impression from the responses here (which are greatly different from the responses I got on other communication channels and by peers). You were just the unlucky mail I responded to :-)
It's OK ;)
But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all porting scenarios with well-thought set of benchmarks, nobody would ever call your PEP "polemic". I tried my best but obviously it was not good enough to please everybody. In all honesty I did not expect that such a small change would spawn such a great discussion. After all what we're discussing here is the introduction of one letter to literals :-)
Well, unfortunately it's not that simple from the standpoint of how this change will be perceived by the community. If we have u'' syntax in python 3, will people even understand what is the key difference from python 2? Will the internet be polluted with weird source-code targeted only for python3, but with the wide use of u''? When to deprecate it, and will it ever be deprecated (as everybody is already tired with all this)? Will it further strengthen the common misbelief the porting is hard (as for the many of the projects it is not), or that the right way it to have one code-base for all versions? And that's just the beginning of such questions. And when this PEP was suddenly approved, many of us felt that all those questions are not answered and were not even discussed. - Yury
Hello, Le 02/03/2012 00:15, Yury Selivanov a écrit :
And that's just the beginning of such questions. And when this PEP was suddenly approved, many of us felt that all those questions are not answered and were not even discussed.
Let me comment on that “suddenly”. We joke about Guido being the dictator for Python, but it’s actually not a joke. The point of the PEP process is to help Guido make an informed decision on a proposed change. (There are also side benefits like providing a record of design or implementation choices, or documenting once and for all why some idea will never be accepted, but let’s ignore them here.) I can’t read Guido’s mind, but I think that here he pronounced somewhat quickly because he was convinced by the arguments in the PEP, while choosing to ignore the problems therein, knowing that they could be fixed later. Regards
On Fri, Mar 2, 2012 at 2:08 PM, Éric Araujo <merwok@netwok.org> wrote:
I can’t read Guido’s mind, but I think that here he pronounced somewhat quickly because he was convinced by the arguments in the PEP, while choosing to ignore the problems therein, knowing that they could be fixed later.
It's also the case that this particular point has been the subject of debate for a *long* time. When the decision was first made to offer the unicode_literals future import, one of the other contenders was just to allow the u/U prefix in Python 3 and not worry about it, and while the "purity" side carried the day back then, it was a close run thing. While that approach turned out to work pretty well for many users that didn't use unicode literals all that much, the web community understandably feel like they're being actively *punished* for doing Unicode right in Python 2. Consider: an application that uses 8-bit strings everywhere and blows up on non-ASCII data in Python 2 has at least a fighting chance to run unmodified *and* handle Unicode properly on Python 3. Because unicode literals are gone, a Unicode-aware Python 2 application currently has *no* chance to run unmodified on Python 3. So even though the PEP doesn't currently do a good job of *presenting* that history to new readers, it's still very much a factor in the decision making process. Accordingly, I'd like to ask folks not to stress too much about the precise wording until I get a chance to update it over the weekend :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mar 02, 2012, at 02:48 PM, Nick Coghlan wrote:
Consider: an application that uses 8-bit strings everywhere and blows up on non-ASCII data in Python 2 has at least a fighting chance to run unmodified *and* handle Unicode properly on Python 3. Because unicode literals are gone, a Unicode-aware Python 2 application currently has *no* chance to run unmodified on Python 3.
On its face, this statement is incorrect. It *might* be accurate if qualified by saying "a Unicode-aware Python 2 *web* application". I say "might" because I'm not an expert on web frameworks so I defer to those who are. It certainly can't be applied to the entire universe of Unicode-aware Python 2 applications.
Accordingly, I'd like to ask folks not to stress too much about the precise wording until I get a chance to update it over the weekend :)
/me takes a deep breath. :) -Barry
On Fri, 2012-03-02 at 14:41 -0500, Barry Warsaw wrote:
On Mar 02, 2012, at 02:48 PM, Nick Coghlan wrote:
Consider: an application that uses 8-bit strings everywhere and blows up on non-ASCII data in Python 2 has at least a fighting chance to run unmodified *and* handle Unicode properly on Python 3. Because unicode literals are gone, a Unicode-aware Python 2 application currently has *no* chance to run unmodified on Python 3.
On its face, this statement is incorrect.
It *might* be accurate if qualified by saying "a Unicode-aware Python 2 *web* application". I say "might" because I'm not an expert on web frameworks so I defer to those who are. It certainly can't be applied to the entire universe of Unicode-aware Python 2 applications.
FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue. If e.g. wxPython had already been ported, I think you'd be hearing the same sorts of things from folks that had investments in existing Python-2-compatible code when trying to port stuff to Py3 (at least if they wanted to run on both Python 2 and Python 3 within the same codebase). - C
On Mar 02, 2012, at 03:13 PM, Chris McDonough wrote:
FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue. If e.g. wxPython had already been ported, I think you'd be hearing the same sorts of things from folks that had investments in existing Python-2-compatible code when trying to port stuff to Py3 (at least if they wanted to run on both Python 2 and Python 3 within the same codebase).
Okay, I just want to be very careful about the message we're sending here, because I think many libraries and applications will work fine with the facilities available in today's stable releases, i.e. unicode_literals and b-prefixes. For these, there's no need to define "native strings", nor do they require language constructs above what's already available. -Barry
On Fri, 2012-03-02 at 15:39 -0500, Barry Warsaw wrote:
On Mar 02, 2012, at 03:13 PM, Chris McDonough wrote:
FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue. If e.g. wxPython had already been ported, I think you'd be hearing the same sorts of things from folks that had investments in existing Python-2-compatible code when trying to port stuff to Py3 (at least if they wanted to run on both Python 2 and Python 3 within the same codebase).
Okay, I just want to be very careful about the message we're sending here, because I think many libraries and applications will work fine with the facilities available in today's stable releases, i.e. unicode_literals and b-prefixes. For these, there's no need to define "native strings", nor do they require language constructs above what's already available.
Although the change makes it possible, and it is very useful for very low level WSGI apps, the issue this change addresses really isn't really 100% about "needing to define native strings". It's also just preservation of a resource in pretty short supply: developer energy. You will probably need to modify less code when taking piece of software that currently runs on Python 2 and changing it so that it runs on both Python 2 and Python 3, without needing to worry over the unintended consequences of using a unicode_literals future import or replacing existing u'' with a function call. This, IMO, can only be a good thing, because the nominal impact of some future user who must now understand u'' syntax is (again IMO) not as consequential as that user having less software to choose from because porting to Python 3 was just that much harder for existing Python 2 developers. - C
Chris McDonough <chrism <at> plope.com> writes:
Although the change makes it possible, and it is very useful for very low level WSGI apps, the issue this change addresses really isn't really 100% about "needing to define native strings". It's also just preservation of a resource in pretty short supply: developer energy.
Apparently developer energy is a limitless resource when it comes to arguing over PEPs ;-)
This, IMO, can only be a good thing, because the nominal impact of some
It can also have some downsides, at least according to some points of view. For example, I regard elevating "native strings" to undue prominence, and the continued use of u'xxx' in Python 3 code, as unfortunate consequences. For example, with PEP 414, it will be possible to mix Unicode with and without prefix - how would that not be at least a little confusing for users new to Python? Remember, "native strings" are a Python-only concept.
future user who must now understand u'' syntax is (again IMO) not as consequential as that user having less software to choose from because porting to Python 3 was just that much harder for existing Python 2 developers.
I don't believe it's because porting to Python 3 is especially hard. I'm not saying it's trivial, but it isn't rocket surgery ;-) Even if porting were trivially easy to do technically at the level the PEP addresses, there would still be additional tests, and perhaps documentation, and perhaps release-related work to be done. Since Python 2.x is a very good platform for software development, where's the incentive to move over to 3.x? It's the chicken and egg effect. Many people are waiting for other people to move over (perhaps projects they depend upon), and while the transition is happening, it's not as quick as it could be. I think a lot of it is down to inertia. Possibly another factor was the "just use 2to3 message", which we now know doesn't work well in all scenarios. However, I don't believe that the "use a single codebase, use six or six-like techniques, use unicode_literals, use the 2to3 fixer to remove unicode prefixes, and use native string markers where you need to" message has received anything like the same level of airplay. If you talk to people who have *actually tried* this approach (say Barry, or me) you'll hear that it's not been all that rough a ride. Regards, Vinay Sajip
On Fri, Mar 2, 2012 at 7:22 PM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Apparently developer energy is a limitless resource when it comes to arguing over PEPs ;-)
Aren't *you* the one who keeps kicking this dead horse? -- --Guido van Rossum (python.org/~guido)
Guido van Rossum <guido <at> python.org> writes:
Aren't *you* the one who keeps kicking this dead horse?
From looking at the overall thread, I'm just one of many people posting on it. Which dead horse am I kicking? It's not as if I'm opposing anything or anyone - just putting my point of view forward about porting from 2.x -> 3.x, as others have done - that's not OT, is it?
Regards, Vinay Sajip
On Sat, Mar 3, 2012 at 04:22, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
It can also have some downsides, at least according to some points of view. For example, I regard elevating "native strings" to undue prominence, and the continued use of u'xxx' in Python 3 code, as unfortunate consequences. For example, with PEP 414, it will be possible to mix Unicode with and without prefix - how would that not be at least a little confusing for users new to Python? Remember, "native strings" are a Python-only concept.
This is true, new users will see 'foo', r'foo', b'foo', and will naturally assume u'foo' is something special too, and will have to be told it is not. But that's an unfortunate effect of Python 3 making the change to Unicode strings, a change that *removed* a lot of other much more confusing things. So the question is if you have any proposal that is *less* confusing while still being practical. Because we do need to distinguish between binary, Unicode and "native" strings. Isn't this the least confusing solution? The only way we could have avoided this "three strings" situation is by actually removing native strings from Python for at least five years, and only used b'' or u''. That would not have been any less confusing. //Lennart
Lennart Regebro <regebro <at> gmail.com> writes:
So the question is if you have any proposal that is *less* confusing while still being practical. Because we do need to distinguish between binary, Unicode and "native" strings. Isn't this the least confusing solution?
It's a matter of the degree of confusion caused (hard to assess) and also a question of taste, so there will be differing views on this. Considering use of unicode_literals, 'xxx' for text, b'yyy' for bytes and with a function wrapper to mark native strings, it becomes clear that the native strings are special cases - much less encountered when looking at code compared to 'xxx' / b'yyy', so there are fewer opportunities for confusion. Where native strings need to be discussed, then it is not unexceptional, nor I believe incorrect, to explain that they are there to suit the requirements of legacy APIs which pre-date Python 3 and the latest versions of Python 2. In terms of practicality, it is IMO quite practical (assuming 2.5 / earlier support can be dropped) to move to a 2.6+/3.x-friendly codebase, e.g. by using Armin's python-modernize. Regards, Vinay Sajip
On Sat, Mar 3, 2012 at 10:26, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Lennart Regebro <regebro <at> gmail.com> writes:
So the question is if you have any proposal that is *less* confusing while still being practical. Because we do need to distinguish between binary, Unicode and "native" strings. Isn't this the least confusing solution?
It's a matter of the degree of confusion caused (hard to assess) and also a question of taste, so there will be differing views on this. Considering use of unicode_literals, 'xxx' for text, b'yyy' for bytes and with a function wrapper to mark native strings, it becomes clear that the native strings are special cases - much less encountered when looking at code compared to 'xxx' / b'yyy',
I'm not sure that's true at all. In most cases where you support both Python 2 and Python 3, most strings will be "native", ie, without prefix in either Python 2 or Python 3. The native case is the most common case.
In terms of practicality, it is IMO quite practical (assuming 2.5 / earlier support can be dropped) to move to a 2.6+/3.x-friendly codebase, e.g. by using Armin's python-modernize.
I think there is some misunderstanding here. The binary/unicode/native separation is only possible on Python 2.6 and 2.7 at the moment, unless you use function wrappers like b(). //Lennart
Lennart Regebro <regebro <at> gmail.com> writes:
I'm not sure that's true at all. In most cases where you support both Python 2 and Python 3, most strings will be "native", ie, without prefix in either Python 2 or Python 3. The native case is the most common case.
Sorry, I didn't make myself clear. If you import unicode_literals, then in both 2.x and 3.x code, an unadorned literal string is text, and a b-adorned literal string is bytes. My assertion was based on that assumption - the text (Unicode) case then becomes the most common case.
In terms of practicality, it is IMO quite practical (assuming 2.5 / earlier support can be dropped) to move to a 2.6+/3.x-friendly codebase, e.g. by using Armin's python-modernize.
I think there is some misunderstanding here. The binary/unicode/native separation is only possible on Python 2.6 and 2.7 at the moment, unless you use function wrappers like b().
Right, and that is a possible option for 2.5 and earlier: though obviously not a desirable one from an aesthetic point of view! What I meant (and should have said) was: if you can drop support for 2.5 / earlier, a lib2to3 fixer-based approach brings your 2.x code into the 3-friendly region of 2.x - 2.6 and 2.7. You can then, using the unicode_literals approach, arrive at a common codebase for 2.6+ and 3.x which is not slow to run (see my other post on ported Django test run performance), and clean (looks just like 3 code, pretty much, and means the same, as far as string literals are concerned). Where you hit native string requirements, apply the wrapper. I don't actually use python-modernize, as I independently developed fixers when doing the Django port late last year. I initially wrote a fixer to transform u'xxx' to u('xxx') (as I was assuming 2.5 support was needed), and then, when it appeared likely that Django would drop 2.5 support after 1.4, I wrote a fixer to go from u('xxx') to 'xxx'. Once I learned to use lib2to3, with a few pointers from Benjamin, it worked like a charm for me. Regards, Vinay Sajip
On Sat, Mar 3, 2012 at 11:39, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Sorry, I didn't make myself clear. If you import unicode_literals, then in both 2.x and 3.x code, an unadorned literal string is text, and a b-adorned literal string is bytes. My assertion was based on that assumption - the text (Unicode) case then becomes the most common case.
Absolutely.
I think there is some misunderstanding here. The binary/unicode/native separation is only possible on Python 2.6 and 2.7 at the moment, unless you use function wrappers like b().
Right, and that is a possible option for 2.5 and earlier: though obviously not a desirable one from an aesthetic point of view!
What I meant (and should have said) was: if you can drop support for 2.5 / earlier, a lib2to3 fixer-based approach brings your 2.x code into the 3-friendly region of 2.x - 2.6 and 2.7. You can then, using the unicode_literals approach, arrive at a common codebase for 2.6+ and 3.x which is not slow to run (see my other post on ported Django test run performance), and clean (looks just like 3 code, pretty much, and means the same, as far as string literals are concerned). Where you hit native string requirements, apply the wrapper.
Yes, that's a doable solution. Just as the common solution of using b() and u() wrappers. But these are still more confusing and less aesthetically pleasing (and insignificantly slower) than supporting u'' in Python 3. //Lennart
On Sat, Mar 3, 2012 at 5:02 AM, Lennart Regebro <regebro@gmail.com> wrote:
I'm not sure that's true at all. In most cases where you support both Python 2 and Python 3, most strings will be "native", ie, without prefix in either Python 2 or Python 3. The native case is the most common case.
Exactly. The reason "native strings" even exist as a concept in WSGI was to make it so that the idiomatic manipulation of header data in both Python 2 and 3 would use plain old string constants with no special wrappers or markings. What's thrown the monkey wrench in here for the WSGI case is the use of unicode_literals. If you simply skip using unicode_literals for WSGI code, you should be fine with a single 2/3 codebase. But then you need some way to mark some things as unicode... which is how we end up back at this PEP. I suppose WSGI could have gone the route of using byte strings for headers instead, but I'm not sure it would have helped. The design goals for PEP 3333 were to sanely support both 2to3 and 2+3 single codebases, and WSGI does actually do that... for the code that's actually doing WSGI stuff. Ironically enough, the effect of the WSGI API is that it's all the *non* WSGI-specific code in the same module that ends up needing to mark its strings as unicode... or else it has to use unicode_literals and mark all the WSGI code with str(). There's really no good way to deal with a *mixed* WSGI/non-WSGI module, except to use explicit markers on one side or the other. Perhaps the simplest solution of all might be to just isolate direct WSGI code in modules that don't import unicode_literals. Web frameworks usually hide WSGI stuff away from the user anyway, and many are already natively unicode in their app-facing APIs. So, if a framework or library encapsulates WSGI in a str-safe/unicode-friendly API, this really shouldn't be an issue for the library's users. But I suppose somebody's got to port the libraries first. ;-) If anyone's updating porting strategy stuff, a mention of this in the tips regarding unicode_literals would be a good idea. i.e., something like: "If you have 2.x modules which work with WSGI and also contain explicit u'' strings, you should not use unicode_literals unless you are willing to explicitly mark all WSGI environment and header strings as native strings using 'str()'. This is necessary because WSGI headers and environment keys/values are defined as byte strings in Python 2.x, and unicode strings in 3.x. Alternatively, you may continue to use u'' strings if you are targeting Python 3.3+ only, or can use the import or install hooks provided for Python 3.2, or if you are using 2to3... but in this case you should not use unicode_literals." That could probably be written a lot more clearly. ;-)
Chris McDonough <chrism <at> plope.com> writes:
FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue. If e.g. wxPython had already been ported, I think you'd be hearing the same sorts of things from folks that had investments in existing Python-2-compatible code when trying to port stuff to Py3 (at least if they wanted to run on both Python 2 and Python 3 within the same codebase).
As I understand it, WSGI happens to explicitly expect str in certain places, even places where conceptually text should be acceptable. The perception of webness seems to be substantiated by Nick's comment about endorsement from you, Armin, Jacob Kaplan-Moss, and Kenneth Reitz for this change. Not that webness is a bad thing, of course - it's a very important part of the ecosystem. It would be good to hear from other constituencies about where else (apart from WSGI and the other uses mentioned in the "APIs and Concepts Using Native Strings" section of the PEP) native strings are needed. I have encountered such needs sometimes, but not uncommonly, they appear to be broken APIs that just expect str even though text should be OK (e.g. cookie APIs, or the sqlite adapter's insisting on accepting datetimes in text format, but only as native strings). It would be a shame to leave these APIs as they are indefinitely, and perhaps using a marker like n('xxx') for native strings would help to remind us that these areas need addressing at some point. Regards, Vinay Sajip
Chris McDonough writes:
FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue.
If we generalize "web" to "wire protocols", I would say that nonweb code that has the same issue is poorly coded. I suppose there may be some similar issues in say XML handling, because XML can be used in binary applications as well as for structuring text (ie, XML is really a wire protocol too). But pure user interface modules like wxPython? Text should be handled as text, not as bytes that "probably" are ASCII-encoded or locale-specifically-encoded (or are magic numbers that happen to be mnemonic when interpreted as ASCII). I don't say that we should ignore the pain of the nonweb users -- but it is a different issue, with different solutions. In particular, using "native strings" (and distinguishing them by the absence of u'') is usually a non-solution for non-web applications, because it propagates the bad practice of pretending that unknown encodings can be assumed to be well-behaved into an environment where good practice is designed in. This is quite different from the case for webby usage, where it often makes sense to handle many low-level operations without ever converting to text, while the same literal strings may be useful in both wire and text contexts (and so should be present only once according to DRY). (N.B. I suspect that it is probably also generally possible for webby applications to avoid native strings without much cost, as Nick showed in urlparse. But at least manipulations of the wire protocol without conversion to text are a plausible optimization.)
Armin Ronacher <armin.ronacher <at> active-4.com> writes:
I tried my best but obviously it was not good enough to please everybody. In all honesty I did not expect that such a small change would spawn such a great discussion. After all what we're discussing here is the introduction of one letter to literals
The objections are not to the introduction of one letter to literals. It is the extent to which, in your presentation of the PEP, a narrow set of concerns and a specific addressing of those concerns has been represented as if it is the only possible view of all right-thinking people in the Python community. What is "obvious" to you may not be so to others - au contraire. A PEP author obviously will promote their specific views - it is an instrument of advocacy - but an author should not, in my view, arrogate to themselves the presumption of speaking for everyone in the community; rather, they should respect that others may have different sensibilities. The PEP comes across as being primarily motivated by WSGI concerns: Nick mentioned that he would update the PEP to "name drop" and indicate support from you, Jacob Kaplan-Moss and Chris McDonough - all authors of Web frameworks. While I completely acknowledge the importance and ubiquity of these web frameworks in the overall ecosystem, I think Python the language is about more than just Web development. There is a bit of a sense of the tail wagging the dog. Let's remember, it's possible to do Web development without the concept of "native" strings - this doesn't exist AFAIK in many other languages which allow Web applications to be developed - the concept is a sort of historical accident arising in part out of how the WSGI spec was written and evolved, interacting with how 3.x differs from 2.x, and how some legacy APIs expect native strings because they are broken. There are a number of possible ways of addressing the concerns which motivated the PEP, but in my view you have given some of them short shrift because of what come across as personal prejudices. An example - on a Reddit thread about PEP 414, I commented: "The PEP does not (for example) consider the possibility of leaving literals as they are and using a n('xxx') callable for native strings. Since there are very few places where native strings are needed, this approach is potentially less obtrusive than either u'xxx' or u('xxx')." Your response to this was: "Because in all honesty, because string wrappers make a codebase horrible to work with. I will have to continue maintaining 2.x versions for at least another four or five years. The idea if having to use string wrappers for that long makes me puke." I know that this was just a comment on Reddit and was not in the PEP, but it smacks of you throwing all your toys out of the pram. It certainly wasn't a reasoned response to my point. And some of that toys-pram attitude bleeds through into the language of the PEP, leading others to make the criticisms that they have. A PEP is supposed to be balanced, reasonable and thought through. It's not supposed to gloss over things in a hand-wavy sort of way - there's still uncertainty in my mind, for example, whether the 3.2 hook will be a 2to3-style tool that runs over a chunk of the whole project's codebase between editing and running a test, or whether it's an import-time hook which only kicks in on files that have just been edited in a development session. Which of these it is might affect crucially the experience of someone wanting to support 3.2 and 3.3 and 2.6+ - but that clearly isn't you, and you don't seem to have much sympathy or patience with that constituency - we're all stick-in-the-muds who want to work with Ubuntu LTS, rather than people subject to constraints imposed by employers, clients, projects we depend on etc. In contrast, Nick made a more reasonable case when commenting ion my preference for unicode_literals (on this list, not on Reddit), by reminding me about how unicode_literals changes the semantics of string literals, and this increases the cognitive burden on developers. I'm not whinging about the PEP in this post - I've said elsewhere that I wasn't opposed to it. I'm just trying to respond to your apparent bewilderment at some of the reaction to the PEP. I have confidence that with your continued input and Nick's input, the wording of the PEP can be made such that it doesn't ruffle quite so many feathers. I'm looking forward to seeing the updates. Regards, Vinay Sajip
[quoting Armin from Reddit] "Because in all honesty, because string wrappers make a codebase horrible to work with. I will have to continue maintaining 2.x versions for at least another four or five years. The idea if having to use string wrappers for that long makes me puke."
Reading this led me to think the following: * 2.5 is now available basically everywhere, and it was released almost 5 years ago (Sep 2006); * if it takes the same time for 3.3, it will be widespread after 4-5 years (i.e. 2016-2017) [0]; * if you want to target a Python 3 version that is widespread [1], you will want to support 3.1/3.2 too in the meanwhile; * therefore you will have to use the hook on 3.1/3.2; * in 2016-2017 you'll finally be able to drop 3.1/3.2 and use only 3.3 without hooks; * in 2016-2017 you'll also stop maintaining the 2.x version (according to that quote); * if you are not maintaining 2.x anymore, you won't need u'' -- right when you could finally stop using the hook; Now, if the hook doesn't get in the way (AFAIU you just have to "install" it and it will do its work automatically), wouldn't it be better to use it in 3.3 too (especially considering that you will probably have to use it already for 3.1/3.2)? If my reasoning is correct, by the time you will be able to use u without problems you will have to start phasing it out because you won't need to support 2.x anymore. Is this hook available somewhere? How difficult is the installation? Does it strip the u automatically or is there a further step that developers should do before testing on 3.1/3.2? Best Regards, Ezio Melotti [0]: ISTM that people think "once you decide to switch to 3.x, there's really no reason to pick an older release, just pick the latest (3.3)". While this might be true for single developers that install it by hand, I don't think it's the same for distros and I expect for 3.x the same time span between release and widespread availability that we have with 2.x (i.e. 4-5 years). However this is just an assumption, if you have more accurate information that can show that the time span will indeed be shorter for 3.x (e.g. 2-3 year), feel free to prove me wrong. [1]: I think most projects still support 2.5, some support even older versions, some support only newer ones, but 2.5 as minimum support version seems a good average to me. Targeting the same use base seems reasonable to me (albeit not strictly necessary).
I know that this was just a comment on Reddit and was not in the PEP, but it smacks of you throwing all your toys out of the pram. It certainly wasn't a reasoned response to my point. And some of that toys-pram attitude bleeds through into the language of the PEP, leading others to make the criticisms that they have. A PEP is supposed to be balanced, reasonable and thought through. It's not supposed to gloss over things in a hand-wavy sort of way - there's still uncertainty in my mind, for example, whether the 3.2 hook will be a 2to3-style tool that runs over a chunk of the whole project's codebase between editing and running a test, or whether it's an import-time hook which only kicks in on files that have just been edited in a development session. Which of these it is might affect crucially the experience of someone wanting to support 3.2 and 3.3 and 2.6+ - but that clearly isn't you, and you don't seem to have much sympathy or patience with that constituency - we're all stick-in-the-muds who want to work with Ubuntu LTS, rather than people subject to constraints imposed by employers, clients, projects we depend on etc.
In contrast, Nick made a more reasonable case when commenting ion my preference for unicode_literals (on this list, not on Reddit), by reminding me about how unicode_literals changes the semantics of string literals, and this increases the cognitive burden on developers.
I'm not whinging about the PEP in this post - I've said elsewhere that I wasn't opposed to it. I'm just trying to respond to your apparent bewilderment at some of the reaction to the PEP.
I have confidence that with your continued input and Nick's input, the wording of the PEP can be made such that it doesn't ruffle quite so many feathers. I'm looking forward to seeing the updates.
Regards,
Vinay Sajip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ezio.melotti%40gmail.com
Wiadomość napisana przez Ezio Melotti w dniu 2 mar 2012, o godz. 10:33:
Now, if the hook doesn't get in the way (AFAIU you just have to "install" it and it will do its work automatically), wouldn't it be better to use it in 3.3 too (especially considering that you will probably have to use it already for 3.1/3.2)?
+1 -- Best regards, Łukasz Langa Senior Systems Architecture Engineer IT Infrastructure Department Grupa Allegro Sp. z o.o.
Hi Ezio, Am 02.03.2012 um 10:33 schrieb Ezio Melotti: >> [quoting Armin from Reddit] >> "Because in all honesty, because string wrappers make a codebase horrible to >> work with. I will have to continue maintaining 2.x versions for at least another >> four or five years. The idea if having to use string wrappers for that long >> makes me puke." > Reading this led me to think the following: > * 2.5 is now available basically everywhere, and it was released almost 5 years ago (Sep 2006); > * if it takes the same time for 3.3, it will be widespread after 4-5 years (i.e. 2016-2017) [0]; > * if you want to target a Python 3 version that is widespread [1], you will want to support 3.1/3.2 too in the meanwhile; > * therefore you will have to use the hook on 3.1/3.2; > * in 2016-2017 you'll finally be able to drop 3.1/3.2 and use only 3.3 without hooks; > * in 2016-2017 you'll also stop maintaining the 2.x version (according to that quote); > * if you are not maintaining 2.x anymore, you won't need u'' -- right when you could finally stop using the hook; I don't think you can compare 2.5 and 3.2 like that. Although 3.2 is/will be shipped with some distributions, it never has, and never will have, the adoption of 2.5 that was "mainstream" for a quite long time. 3.3 is the IMHO the first 3.x release that brings really cool stuff to the table and might be the tipping point for people to start embracing Python 3 – despite the fact that Ubuntu LTS will alas ship 3.2 for the next 10 years. I hope for some half-official back port there. :) Re the language thingie (not directed towards Ezio): It's true that Armin tends to be opinionated – maybe even polemic. However I can't recall a case where he personally attacked people like it happened here. Regards, Hynek
Hi Ezio,
Am 02.03.2012 um 10:33 schrieb Ezio Melotti:
Reading this led me to think the following: * 2.5 is now available basically everywhere, and it was released almost 5 years ago (Sep 2006); * if it takes the same time for 3.3, it will be widespread after 4-5 years (i.e. 2016-2017) [0]; * if you want to target a Python 3 version that is widespread [1], you will want to support 3.1/3.2 too in the meanwhile; * therefore you will have to use the hook on 3.1/3.2; * in 2016-2017 you'll finally be able to drop 3.1/3.2 and use only 3.3 without hooks; * in 2016-2017 you'll also stop maintaining the 2.x version (according to that quote); * if you are not maintaining 2.x anymore, you won't need u'' -- right when you could finally stop using the hook; I don't think you can compare 2.5 and 3.2 like that. Although 3.2 is/will be shipped with some distributions, it never has, and never will have, the adoption of 2.5 that was "mainstream" for a quite long time.
But I don't think the adoption of 3.2 will affect the decisions that distros will take about 3.3. Even in the unlikely case that e.g. Debian/RHEL make Python 3.3 available as soon as it's released, not everyone will immediately upload to the latest Debian or RHEL version. The point is that regardless the current Python 3 situation, it will take a few years before 3.3 will be widely available on most of the machine. For example I work on a server where I have 3.1. When/if it will be updated it will probably get 3.2, not 3.3 -- and this might happen in a couple of years. If I want 3.3 I will probably have to wait another couple of years. Other people might have to wait less time, others more.
3.3 is the IMHO the first 3.x release that brings really cool stuff to the table and might be the tipping point for people to start embracing Python 3 – despite the fact that Ubuntu LTS will alas ship 3.2 for the next 10 years. I hope for some half-official back port there. :)
I heard this about 3.1 and 3.2 too, and indeed they are both perfectly valid releases. The fact that 3.3 is even cooler doesn't mean that 3.1/3.2 are not cool. (I'm perfectly fine with the aforementioned server and 3.1, and currently I don't miss anything that is new in 3.2/3.3.) Best Regards, Ezio Melotti
Just my 2 cents on the PEP rewrite: u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or "native". //Lennart
02.03.12 15:49, Lennart Regebro написав(ла):
Just my 2 cents on the PEP rewrite:
u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or "native".
What "native" means in context Python 3 only? "Native" strings only have meaning if we consider Python 2 and Python 3 together. "Native" string is a text string, which was binary in Python 2. There is a flag for such strings -- str().
On Fri, Mar 2, 2012 at 15:26, Serhiy Storchaka <storchaka@gmail.com> wrote:
02.03.12 15:49, Lennart Regebro написав(ла):
Just my 2 cents on the PEP rewrite:
u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or "native".
What "native" means in context Python 3 only?
I don't understand your question.
"Native" strings only have meaning if we consider Python 2 and Python 3 together. "Native" string is a text string, which was binary in Python 2. There is a flag for such strings -- str().
Yes.
Zitat von Lennart Regebro <regebro@gmail.com>:
Just my 2 cents on the PEP rewrite:
u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or "native".
How so? In the Python 3 code, the u"" prefix would not appear, even if appears in the original source, as 2to3 eliminates it. So you surely need the u"" prefix to distinguish binary, unicode, or native strings in your source - but with 2to3, the PEP 414 change is unnecessary. Regards, Martin
On Sat, Mar 3, 2012 at 01:49, <martin@v.loewis.de> wrote:
Zitat von Lennart Regebro <regebro@gmail.com>:
Just my 2 cents on the PEP rewrite:
u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or "native".
How so? In the Python 3 code, the u"" prefix would not appear, even if appears in the original source, as 2to3 eliminates it.
Well, not if you disable that fixer. ;-) But you are right, it isn't necessary. I was thinking of 3to2, actually. That was one of the objections I had to the usefulness of 3to2, there is no way to make the distinction between unicode and native strings. (The u'' prefix hence actually makes 3to2 a realistic option, and that's good.) So everyone can ignore this, I mixed up two issues. :-) //Lennart
03.03.12 08:20, Lennart Regebro написав(ла):
But you are right, it isn't necessary. I was thinking of 3to2, actually. That was one of the objections I had to the usefulness of 3to2, there is no way to make the distinction between unicode and native strings. (The u'' prefix hence actually makes 3to2 a realistic option, and that's good.)
2to3 should recognize the str(string_literal) (or nstr(), or native(), etc) as a native string and does not add prefix "u" to it. And you have to explicitly specify these tips.
2to3 should recognize the str(string_literal) (or nstr(), or native(), etc) as a native string and does not add prefix "u" to it. And you have to explicitly specify these tips.
That is already implemented. 2to3 *never* adds a u prefix anywhere, including not for str(string_literal). Regards, Martin
04.03.12 00:12, "Martin v. Löwis" написав(ла):
2to3 should recognize the str(string_literal) (or nstr(), or native(), etc) as a native string and does not add prefix "u" to it. And you have to explicitly specify these tips.
That is already implemented. 2to3 *never* adds a u prefix anywhere, including not for str(string_literal).
Sorry, I mean *3to2*.
On Mar 02, 2012, at 12:58 PM, Hynek Schlawack wrote:
3.3 is the IMHO the first 3.x release that brings really cool stuff to the table and might be the tipping point for people to start embracing Python 3 – despite the fact that Ubuntu LTS will alas ship 3.2 for the next 10 years. I hope for some half-official back port there. :)
Although I disagree with the premise (I think Python 3.2 is a fine platform to build many applications on) it's probably likely what we'll have backports of stable Python 3 releases to 12.04, at the very least in semi-official PPAs. Just like today we're trying to provide a smoother path for LTS->LTS upgrades where 10.04 had only Python 2.6 but 12.04 has only Python 2.7. We have a semi-official Lucid PPA providing Python 2.7, though afaict very few people have actually used or tested it. Cheers, -Barry
Am 02.03.2012 um 20:44 schrieb Barry Warsaw:
3.3 is the IMHO the first 3.x release that brings really cool stuff to the table and might be the tipping point for people to start embracing Python 3 – despite the fact that Ubuntu LTS will alas ship 3.2 for the next 10 years. I hope for some half-official back port there. :) Although I disagree with the premise (I think Python 3.2 is a fine platform to build many applications on)
Just to be clear: I didn't say 3.2 is “bad” or “not fine”. It's just the fact that people need more than “fine” to feel urged to switch to Python 3. I sincerely hope 3.3 fulfills that and if PEP 414 even makes porting easier we might have a perfect storm. :)
it's probably likely what we'll have backports of stable Python 3 releases to 12.04, at the very least in semi-official PPAs.
That's what I've been hoping for. Maybe it will work the other way around too: People like 3.3, target it first and port back later to reach more users. It's all about encouraging people to try the nectar of Python 3 – once they're caught it's sticky sweetness[1]… ;) Cheers, Hynek [1] disclaimer: sticky sweetness only applies if you're not a maintainer of wsgi-related middleware/framework
On Mar 02, 2012, at 09:23 PM, Hynek Schlawack wrote:
Just to be clear: I didn't say 3.2 is “bad” or “not fine”. It's just the fact that people need more than “fine” to feel urged to switch to Python 3. I sincerely hope 3.3 fulfills that and if PEP 414 even makes porting easier we might have a perfect storm. :)
Cool, and yes reaching that tipping point is what it's all about. :)
it's probably likely what we'll have backports of stable Python 3 releases to 12.04, at the very least in semi-official PPAs.
That's what I've been hoping for. Maybe it will work the other way around too: People like 3.3, target it first and port back later to reach more users. It's all about encouraging people to try the nectar of Python 3 – once they're caught it's sticky sweetness[1]… ;)
Indeed! -Barry
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/01/2012 05:52 PM, Armin Ronacher wrote:
Hi,
On 3/1/12 10:38 PM, Yury Selivanov wrote:
Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that impression from the responses here (which are greatly different from the responses I got on other communication channels and by peers). You were just the unlucky mail I responded to :-)
Several responses on the list *have* been offensive, not criticizing the PEP on its own merits but on your (presumed) motives for introducing it. Such attacks are wildly off-base.
But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all porting scenarios with well-thought set of benchmarks, nobody would ever call your PEP "polemic". I tried my best but obviously it was not good enough to please everybody. In all honesty I did not expect that such a small change would spawn such a great discussion. After all what we're discussing here is the introduction of one letter to literals :-)
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk9QE5AACgkQ+gerLs4ltQ6m7wCdHufuDMrplgg0+MQr4M10Y4Oy N74AoJW5UKbUjPOdrreeTT38C9Cl2eFk =DkBX -----END PGP SIGNATURE-----
Armin, I see you've (or somebody) changed: """As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3.""" to: """As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3.""" Could you just remove the statement completely? Again, my understanding of what is the best choice for certain *long-term* investments is drastically different from yours. In my opinion, python 3 is much more suitable for anything *long-term* than python 2. I don't think that PEPs are the right place to put such polemic and biased statements. Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. There are blogs for that. Thank you. - Yury On 2012-02-28, at 11:29 AM, Yury Selivanov wrote:
Hi Armin,
Could you please remove from the PEP the following statement:
"""As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3."""
While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org.
The python ecosystem is not just limited to WSGI apps, Django and Flask. Yes, we don't have all the packages on pypi support python 3, but many of those are portable within 10 minutes to couple of hours of work (and I did many of such ports for our internal systems.) And many of the essential packages do exist for python 3, like numpy, zeromq etc.
I know several sturt-ups, including mine that develop huge commercial applications entirely on python 3.
Thanks, -Yury
On 2012-02-27, at 5:38 PM, Armin Ronacher wrote:
Hi,
On 2/27/12 10:18 PM, Terry Reedy wrote:
I would like to know if you think that this one change is enough to do agile development and testing, etc, or whether, as Chris McDonough hopes, this is just the first of a series of proposals you have planned. Indeed I have three other PEPs in the work. The reintroduction of "except (((ExceptionType),),)", the "<>" comparision operator and the removal of "nonlocal", the latter to make Python 2.x developers feel better about themselves. :-)
Regards, Armin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com
participants (22)
-
"Martin v. Löwis" -
Armin Ronacher -
Barry Warsaw -
Chris McDonough -
Ezio Melotti -
Guido van Rossum -
Hynek Schlawack -
Lennart Regebro -
martin＠v.loewis.de
-
Nick Coghlan -
PJ Eby -
R. David Murray -
Serhiy Storchaka -
Simon Cross -
Stephen J. Turnbull -
Terry Reedy -
Tres Seaver -
Vinay Sajip -
Yury Selivanov -
Yury Selivanov -
Éric Araujo -
Łukasz Langa