Emit SyntaxWarning on unrecognized backslash escapes?
See topic "Unrecognized backslash escapes in string literals" in Python list [1]. I agree that this is a problem, especially for novices (but even experience users can make a typo). May be emit SyntaxWarning on unrecognized backslash escapes? An exception is already raised on invalid octal or hexadecimal escapes. '\x' is syntax error, not two characters '\\' and 'x'. [1] http://comments.gmane.org/gmane.comp.python.general/772455
I think that's a bit too strong. This has been unquestionably valid, correct Python -- it was an intentional feature from the start. It may not have turned out great, but I think that before warning loudly about every instance of this we should have a silent deprecation (which you can turn into a visible warning with a command-line flag or a warnings filter). And we should have agreement that we're eventually going to make it a syntax error. On Mon, Feb 23, 2015 at 1:57 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
See topic "Unrecognized backslash escapes in string literals" in Python list [1]. I agree that this is a problem, especially for novices (but even experience users can make a typo). May be emit SyntaxWarning on unrecognized backslash escapes? An exception is already raised on invalid octal or hexadecimal escapes. '\x' is syntax error, not two characters '\\' and 'x'.
[1] http://comments.gmane.org/gmane.comp.python.general/772455
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ guido%40python.org
-- --Guido van Rossum (python.org/~guido)
On Tue, Feb 24, 2015 at 2:44 AM, Guido van Rossum <guido@python.org> wrote:
I think that's a bit too strong. This has been unquestionably valid, correct Python -- it was an intentional feature from the start. It may not have turned out great, but I think that before warning loudly about every instance of this we should have a silent deprecation (which you can turn into a visible warning with a command-line flag or a warnings filter). And we should have agreement that we're eventually going to make it a syntax error.
Is it at all possible for this to be introduced in the 2.x line, or is the entire concept of a deprecation period one that has to start with a minor version? If it's never going to happen in 2.x, I'll raise this as yet another reason to get the course and all our students migrated to 3.x, but on the flip side, it means that we absolutely can't get the benefit until that jump is made. ChrisA
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Feb 24, 2015 at 2:44 AM, Guido van Rossum <guido@python.org> wrote:
I think that's a bit too strong. This has been unquestionably valid, correct Python -- it was an intentional feature from the start. It may not have turned out great, but I think that before warning loudly about every instance of this we should have a silent deprecation (which you can turn into a visible warning with a command-line flag or a warnings filter). And we should have agreement that we're eventually going to make it a syntax error.
Is it at all possible for this to be introduced in the 2.x line, or is the entire concept of a deprecation period one that has to start with a minor version?
Starts with a minor version.
If it's never going to happen in 2.x, I'll raise this as yet another
reason to get the course and all our students migrated to 3.x, but on
the flip side, it means that we absolutely can't get the benefit until that jump is made.
Never going to happen in 2.x..
On Tue, Feb 24, 2015 at 3:12 AM, Brett Cannon <brett@python.org> wrote:
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Feb 24, 2015 at 2:44 AM, Guido van Rossum <guido@python.org> wrote:
I think that's a bit too strong. This has been unquestionably valid, correct Python -- it was an intentional feature from the start. It may not have turned out great, but I think that before warning loudly about every instance of this we should have a silent deprecation (which you can turn into a visible warning with a command-line flag or a warnings filter). And we should have agreement that we're eventually going to make it a syntax error.
Is it at all possible for this to be introduced in the 2.x line, or is the entire concept of a deprecation period one that has to start with a minor version?
Starts with a minor version.
If it's never going to happen in 2.x, I'll raise this as yet another
reason to get the course and all our students migrated to 3.x, but on the flip side, it means that we absolutely can't get the benefit until that jump is made.
Never going to happen in 2.x..
Thanks, thought that'd be the case but figured I may as well ask. ChrisA
On 02/23/2015 08:12 AM, Brett Cannon wrote:
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico wrote:
Is it at all possible for this to be introduced in the 2.x line [...]
Starts with a minor version.
Isn't there a -3 switch or something like that in 2.7.x to trigger warnings/errors to help port to 3.x? Seems like this kind of warning could go there. -- ~Ethan~
On Mon, Feb 23, 2015 at 9:01 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/23/2015 08:12 AM, Brett Cannon wrote:
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico wrote:
Is it at all possible for this to be introduced in the 2.x line [...]
Starts with a minor version.
Isn't there a -3 switch or something like that in 2.7.x to trigger warnings/errors to help port to 3.x? Seems like this kind of warning could go there.
If we agree it will be a syntax error in 3.x eventually. -- --Guido van Rossum (python.org/~guido)
On Feb 23, 2015 9:30 AM, "Guido van Rossum" <guido@python.org> wrote:
On Mon, Feb 23, 2015 at 9:01 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/23/2015 08:12 AM, Brett Cannon wrote:
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico wrote:
Is it at all possible for this to be introduced in the 2.x line [...]
Starts with a minor version.
Isn't there a -3 switch or something like that in 2.7.x to trigger
warnings/errors to help port to 3.x? Seems like this
kind of warning could go there.
If we agree it will be a syntax error in 3.x eventually.
+1 to that FWIW. -n
On Mon, 23 Feb 2015 09:29:09 -0800 Guido van Rossum <guido@python.org> wrote:
On Mon, Feb 23, 2015 at 9:01 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/23/2015 08:12 AM, Brett Cannon wrote:
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico wrote:
Is it at all possible for this to be introduced in the 2.x line [...]
Starts with a minor version.
Isn't there a -3 switch or something like that in 2.7.x to trigger warnings/errors to help port to 3.x? Seems like this kind of warning could go there.
If we agree it will be a syntax error in 3.x eventually.
That sounds frankly like a pedantic change. Regards Antoine.
On 23 February 2015 at 16:47, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Mon, 23 Feb 2015 09:29:09 -0800 Guido van Rossum <guido@python.org> wrote:
On Mon, Feb 23, 2015 at 9:01 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/23/2015 08:12 AM, Brett Cannon wrote:
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico wrote:
Is it at all possible for this to be introduced in the 2.x line [...]
Starts with a minor version.
Isn't there a -3 switch or something like that in 2.7.x to trigger warnings/errors to help port to 3.x? Seems like this kind of warning could go there.
If we agree it will be a syntax error in 3.x eventually.
That sounds frankly like a pedantic change.
The real problem there, even motivating the request, seems to be new users in Windows platforms using "\" as file path separator. That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
Regards
Antoine.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br
On Mon, 23 Feb 2015 16:58:29 -0300 "Joao S. O. Bueno" <jsbueno@python.org.br> wrote:
On 23 February 2015 at 16:47, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Mon, 23 Feb 2015 09:29:09 -0800 Guido van Rossum <guido@python.org> wrote:
On Mon, Feb 23, 2015 at 9:01 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/23/2015 08:12 AM, Brett Cannon wrote:
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico wrote:
Is it at all possible for this to be introduced in the 2.x line [...]
Starts with a minor version.
Isn't there a -3 switch or something like that in 2.7.x to trigger warnings/errors to help port to 3.x? Seems like this kind of warning could go there.
If we agree it will be a syntax error in 3.x eventually.
That sounds frankly like a pedantic change.
The real problem there, even motivating the request, seems to be new users in Windows platforms using "\" as file path separator.
Thanks. That sounds like a good reason indeed. Regards Antoine.
On 23.02.15 21:58, Joao S. O. Bueno wrote:
That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Serhiy Storchaka wrote:
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Even with the syntax error, there are filenames that will mysteriously fail, e.g. "C:\ninjamoves.txt". -- Greg
On Tue, Feb 24, 2015 at 8:39 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Serhiy Storchaka wrote:
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Even with the syntax error, there are filenames that will mysteriously fail, e.g. "C:\ninjamoves.txt".
Yes, there are; but they will fail, rather than randomly happening to work. With the proposed syntax warning, there will be no cases which silently succeed; everything will either be interpreted as an escape sequence (as in your example), fail with a hard error ("C:\Users" in a Unicode string, or "C:\x-do-not-track"), or produce a warning (everything else). The most tricky escape sequence that I can envision is \134, so you might find that an all-numeric path gets beheaded:
print("C:\134689") C:\689
The chances that someone will accidentally get something like this are very low, and the main thing is, "\X" for any X will never be interpreted the same way as "\\X", so the backslash will _always_ be seen as special. ChrisA
On 23 February 2015 at 18:39, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Serhiy Storchaka wrote:
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Even with the syntax error, there are filenames that will mysteriously fail, e.g. "C:\ninjamoves.txt".
That is the reason I urge folks to think about the root cause, and check if this proposal is the best way to tackle it: The failing cases are the ones that won't be affected by this change at all - as they are valid escaped strings! There will be no error for c:\tmp\myfile.txt - it will ever be just "file not found" - unless the warning comes to using "\" as file path separator (and no, I don think just doing that would properly address the issue as well). Could we just use Guido's time machine and go back to the point in time where some MS head decided to go with "\" instead of "/", and deliver a well placed punch? :-) Or maybe have IOError for "file not found" to get some fuzzy logic and display a more verbose error message if there are "\n"s and "\t"s on the filename and platform is Windows? I think that could go along the proposal for deprecating non-escaping "\<x>" sequences . By the way, although I am writing to get the root issue covered, I think deprecating them would be a good thing, yes. Despite my previous comment on deprecating code that works, on a deeper though it _is_ worth the issue. js -><-
-- Greg
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br
On Tue, Feb 24, 2015 at 1:00 PM, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
On 23 February 2015 at 18:39, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Serhiy Storchaka wrote:
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt",
but
doesn't work with "D:\test.txt". He has no ideas what happen.
Even with the syntax error, there are filenames that will mysteriously fail, e.g. "C:\ninjamoves.txt".
That is the reason I urge folks to think about the root cause, and check if this proposal is the best way to tackle it:
The failing cases are the ones that won't be affected by this change at all
- as they are valid escaped strings!
I would argue that some cases working is the root cause of the confusion. If *none* of the cases work, it would be a lot less confusing, even if the error message is different. Trying to make the error messages more similar, or more identifying, may be a good idea (as long as they aren't misleading when people *meant* to use escape sequences in a string) but fortunately that's separate from making the cases that current silently and misleadingly work, emit a warning (or error.) Emitting a warning does not address the question 'why does "C:\test" not exist when I see it in Explorer.' It does address the question 'why does my program break when I change "C:\Test" to "C:\test". It also addresses 'why does my regexp not work when I change "\d+" to "\b\d+", and unlike changing the error message for opening files it is much closer, in time and space, to the actual problem. It's a helpful thing regardless of what other changes are made. I do expect there to need to be a much longer deprecation period than one or even two releases, though: there is a lot of code that uses the misfeature, mostly by accident (not realizing they should be using r'', etc.)
There will be no error for c:\tmp\myfile.txt - it will ever be just "file not found" - unless the warning comes to using "\" as file path separator (and no, I don think just doing that would properly address the issue as well).
Could we just use Guido's time machine and go back to the point in time where some MS head decided to go with "\" instead of "/", and deliver a well placed punch? :-)
Or maybe have IOError for "file not found" to get some fuzzy logic and display a more verbose error message if there are "\n"s and "\t"s on the filename and platform is Windows? I think that could go along the proposal for deprecating non-escaping "\<x>" sequences .
By the way, although I am writing to get the root issue covered, I think deprecating them would be a good thing, yes. Despite my previous comment on deprecating code that works, on a deeper though it _is_ worth the issue.
js -><-
-- Greg
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/thomas%40python.org
-- Thomas Wouters <thomas@python.org> Hi! I'm an email virus! Think twice before sending your email to help me spread!
On Wed, Feb 25, 2015 at 12:00 AM, Thomas Wouters <thomas@python.org> wrote:
I do expect there to need to be a much longer deprecation period than one or even two releases, though: there is a lot of code that uses the misfeature, mostly by accident (not realizing they should be using r'', etc.)
I don't mind how long the deprecation period is, as long as there can be an option to Python that makes a noisy warning. The mixing of tabs and spaces needed Python 3.0 to become an error, but "python -tt" is my go-to solution for anyone using Python 2, tabs set to four spaces, and an editor that doesn't highlight indentation mismatches. Now, maybe we can't get this option backported to 2.7 (I'd like to, but I don't think this is on par with hash randomization!), but we could at least have Python 3.5 or 3.6 accept an option "python3 -e" to warn on unrecognized escapes, and (paralleling the others) "-ee" to raise SyntaxErrors. Eventually, even if the deprecation lasts until Python 4.0 or beyond, we could at least depend on being able to call up the warnings. ChrisA
Chris Angelico writes:
I don't mind how long the deprecation period is, as long as there can be an option to Python that makes a noisy warning.
If that's OK with you and for the use case you explicitly described, though, a .bat file that runs a linter first might be the better option since (a) you don't have to wait and (b) you can put any bugaboo you like in there and (c) it can warn about syntacticly correct strings that "just fail" and may be hard to find. I think the Zen is right on, here: Now is better than never. Although never is often better than *right* now. I don't have a good sense for which it is, though, partly because (a) I don't program on Windows, but more importantly, (b) many of the dangerous strings actually used won't generate warnings or errors ever.
On 25 Feb 2015 01:07, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:
Chris Angelico writes:
I don't mind how long the deprecation period is, as long as there can be an option to Python that makes a noisy warning.
If that's OK with you and for the use case you explicitly described, though, a .bat file that runs a linter first might be the better option since (a) you don't have to wait and (b) you can put any bugaboo you like in there and (c) it can warn about syntacticly correct strings that "just fail" and may be hard to find.
The linter developers don't have a decision making process that lets them pursue things like this on their own. By contrast, if we went with a deprecation warning (and potentially a command line option) then the linter developers can backport the already defined check comparatively easily. Regards, Nick.
I think the Zen is right on, here:
Now is better than never. Although never is often better than *right* now.
I don't have a good sense for which it is, though, partly because (a) I don't program on Windows, but more importantly, (b) many of the dangerous strings actually used won't generate warnings or errors ever. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
Nick Coghlan writes:
The linter developers don't have a decision making process that lets them pursue things like this on their own.
I'm not suggesting that the linter developers do any such thing. I'm suggesting that users (specifically Chris) customize their preferred linters (most permit that). What bothers me is that the hardest to understand failures cannot possibly be helped, because they are legal, meaningful syntax that conflicts with syntax common in external contexts. Replacing mnemonics like '\t' with '\x09' (or even '\u0009' in Python 3) in debugging output (including error messages) seems like a much better idea to me.
"Stephen J. Turnbull" <stephen@xemacs.org> writes:
What bothers me is that the hardest to understand failures cannot possibly be helped, because they are legal, meaningful syntax that conflicts with syntax common in external contexts.
There is a wider context here, too: semantics of the backslash escape <URL:https://en.wikipedia.org/wiki/Backslash#Usage> commonly include “backslash followed by a character not otherwise mentioned will produce that character, verbatim”. Proposals to change that for Python's string literals must account for the fact this will make Python's rules for backslash escape surprisingly different in this regard from many other usages of backslash escape.
Replacing mnemonics like '\t' with '\x09' (or even '\u0009' in Python 3) in debugging output (including error messages) seems like a much better idea to me.
+1. This will not cause any valid behaviour today to fail, whereas it will make the common problems much more likely to be noticed by newcomers. -- \ “I went to a fancy French restaurant called ‘Déjà Vu’. The head | `\ waiter said, ‘Don't I know you?’” —Steven Wright | _o__) | Ben Finney
On Wed, Feb 25, 2015 at 5:00 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
There is a wider context here, too: semantics of the backslash escape <URL:https://en.wikipedia.org/wiki/Backslash#Usage> commonly include “backslash followed by a character not otherwise mentioned will produce that character, verbatim”.
Proposals to change that for Python's string literals must account for the fact this will make Python's rules for backslash escape surprisingly different in this regard from many other usages of backslash escape.
That's different from Python's rule, which is that "backslash followed by a character not otherwise mentioned will produce *a backslash followed by* that character, verbatim". ChrisA
Thomas Wouters wrote:
Trying to make the error messages more similar, or more identifying, may be a good idea (as long as they aren't misleading when people *meant* to use escape sequences in a string)
It seems that Windows won't let you use control characters in filenames, so there is room for a more pointed error message from functions that take pathnames on Windows. -- Greg
On 23/02/2015 21:27, Serhiy Storchaka wrote:
On 23.02.15 21:58, Joao S. O. Bueno wrote:
That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Isn't this why users have help desks? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence
On 24 February 2015 at 07:39, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
On 23/02/2015 21:27, Serhiy Storchaka wrote:
On 23.02.15 21:58, Joao S. O. Bueno wrote:
That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Isn't this why users have help desks?
Most don't, and cases like "\n" or "\t" in a Windows path name being converted to whitespace are utterly impossible to look up in an internet search when they fail, so a user learning on their own gets left with a broken program and no particularly effective ways to ask for help figuring it out. Like Unicode encoding errors they may appear a long way from the source of the offending data value (in this case, likely to be a file name copy and pasted from elsewhere on their system), and they don't give a particularly helpful error message (especially when the escape sequences are for whitespace). While I originally disliked the idea, I think this is a genuine usability issue on Windows that would be worth addressing. However, it's a significant enough change that I believe it needs a PEP and a reasonably long transition period before anything actually breaks. For example: - pep8 and pylint warnings as soon as a patch can be accepted - Py3kWarning in Python 2.7.x - DeprecationWarning in Python 3.5 - SyntaxWarning in Python 3.6 - SyntaxError in Python 3.7 Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 24 February 2015 at 08:40, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 24 February 2015 at 07:39, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
On 23/02/2015 21:27, Serhiy Storchaka wrote:
On 23.02.15 21:58, Joao S. O. Bueno wrote:
That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Isn't this why users have help desks?
Most don't, and cases like "\n" or "\t" in a Windows path name being converted to whitespace are utterly impossible to look up in an internet search when they fail, so a user learning on their own gets left with a broken program and no particularly effective ways to ask for help figuring it out.
Like Unicode encoding errors they may appear a long way from the source of the offending data value (in this case, likely to be a file name copy and pasted from elsewhere on their system), and they don't give a particularly helpful error message (especially when the escape sequences are for whitespace).
While I originally disliked the idea, I think this is a genuine usability issue on Windows that would be worth addressing. However, it's a significant enough change that I believe it needs a PEP and a reasonably long transition period before anything actually breaks. For example:
- pep8 and pylint warnings as soon as a patch can be accepted - Py3kWarning in Python 2.7.x - DeprecationWarning in Python 3.5 - SyntaxWarning in Python 3.6 - SyntaxError in Python 3.7
Another suggestion: we may want to turn this particular deprecation warning on by default in the interactive interpreter, and recommend that other interactive interpreter developers (such as the IPython folks) consider doing the same. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 24.02.15 00:50, Nick Coghlan wrote:
- SyntaxError in Python 3.7
I think that SyntaxError could wait to 4.0.
Another suggestion: we may want to turn this particular deprecation warning on by default in the interactive interpreter, and recommend that other interactive interpreter developers (such as the IPython folks) consider doing the same.
Why not turn all deprecation warning on by default in the interactive interpreter?
On 23/02/2015 23:32, Serhiy Storchaka wrote:
On 24.02.15 00:50, Nick Coghlan wrote:
- SyntaxError in Python 3.7
I think that SyntaxError could wait to 4.0.
Another suggestion: we may want to turn this particular deprecation warning on by default in the interactive interpreter, and recommend that other interactive interpreter developers (such as the IPython folks) consider doing the same.
Why not turn all deprecation warning on by default in the interactive interpreter?
In any given release how many of these would you see? I wouldn't mind seeing a few lines whenever I ran the II, but if the number started getting into tens or hundreds, at best I'd find it extremely irritating. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence
On 2015-02-23 22:50, Nick Coghlan wrote:
On 24 February 2015 at 08:40, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 24 February 2015 at 07:39, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
On 23/02/2015 21:27, Serhiy Storchaka wrote:
On 23.02.15 21:58, Joao S. O. Bueno wrote:
That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Isn't this why users have help desks?
Most don't, and cases like "\n" or "\t" in a Windows path name being converted to whitespace are utterly impossible to look up in an internet search when they fail, so a user learning on their own gets left with a broken program and no particularly effective ways to ask for help figuring it out.
Like Unicode encoding errors they may appear a long way from the source of the offending data value (in this case, likely to be a file name copy and pasted from elsewhere on their system), and they don't give a particularly helpful error message (especially when the escape sequences are for whitespace).
While I originally disliked the idea, I think this is a genuine usability issue on Windows that would be worth addressing. However, it's a significant enough change that I believe it needs a PEP and a reasonably long transition period before anything actually breaks. For example:
- pep8 and pylint warnings as soon as a patch can be accepted - Py3kWarning in Python 2.7.x - DeprecationWarning in Python 3.5 - SyntaxWarning in Python 3.6 - SyntaxError in Python 3.7
Another suggestion: we may want to turn this particular deprecation warning on by default in the interactive interpreter, and recommend that other interactive interpreter developers (such as the IPython folks) consider doing the same.
While we're at it, could something similar also be done to re? In that instance, unknown escapes with ASCII-range letters _drop_ the backslash:
re.match(r'\q', 'q') <_sre.SRE_Match object; span=(0, 1), match='q'>
On 23.02.2015 23:50, Nick Coghlan wrote:
On 24 February 2015 at 08:40, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 24 February 2015 at 07:39, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
On 23/02/2015 21:27, Serhiy Storchaka wrote:
On 23.02.15 21:58, Joao S. O. Bueno wrote:
That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Isn't this why users have help desks?
Most don't, and cases like "\n" or "\t" in a Windows path name being converted to whitespace are utterly impossible to look up in an internet search when they fail, so a user learning on their own gets left with a broken program and no particularly effective ways to ask for help figuring it out.
I think the easiest way would be to tweak the error message output to indicate the real problem. At the moment, you get:
open('c:\test.txt') Traceback (most recent call last): File "<stdin>", line 1, in <module> FileNotFoundError: [Errno 2] No such file or directory: 'c:\test.txt'
which isn't helpful. It would be better to display the escapes as what they are: FileNotFoundError: [Errno 2] No such file or directory: 'c: est.txt' or use a special representation function to highlight the possible problem: FileNotFoundError: [Errno 2] No such file or directory: 'c:[\t]est.txt' - warning: embedded escapes with the representation being: "'c:[\t]est.txt' - warning: embedded escapes" Such a change doesn't require a deprecation phase or a PEP. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 24 2015)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 24 February 2015 at 18:40, M.-A. Lemburg <mal@egenix.com> wrote:
On 23.02.2015 23:50, Nick Coghlan wrote:
On 24 February 2015 at 08:40, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 24 February 2015 at 07:39, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
On 23/02/2015 21:27, Serhiy Storchaka wrote:
On 23.02.15 21:58, Joao S. O. Bueno wrote:
That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
The problem is that the user don't know that he should read the documentation. It just find that his script works with "C:\sample.txt", but doesn't work with "D:\test.txt". He has no ideas what happen.
Isn't this why users have help desks?
Most don't, and cases like "\n" or "\t" in a Windows path name being converted to whitespace are utterly impossible to look up in an internet search when they fail, so a user learning on their own gets left with a broken program and no particularly effective ways to ask for help figuring it out.
I think the easiest way would be to tweak the error message output to indicate the real problem.
At the moment, you get:
open('c:\test.txt') Traceback (most recent call last): File "<stdin>", line 1, in <module> FileNotFoundError: [Errno 2] No such file or directory: 'c:\test.txt'
which isn't helpful.
It would be better to display the escapes as what they are:
FileNotFoundError: [Errno 2] No such file or directory: 'c: est.txt'
or use a special representation function to highlight the possible problem:
FileNotFoundError: [Errno 2] No such file or directory: 'c:[\t]est.txt' - warning: embedded escapes
with the representation being:
"'c:[\t]est.txt' - warning: embedded escapes"
Such a change doesn't require a deprecation phase or a PEP.
Oh, I like that idea, especially the variant that includes a search friendly phrase like "embedded escapes" in the error message. And unlike changing the handling of string literals generally, the impact would be isolated to a few exception __repr__ implementations. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 24.02.15 11:09, Nick Coghlan wrote:
Oh, I like that idea, especially the variant that includes a search friendly phrase like "embedded escapes" in the error message. And unlike changing the handling of string literals generally, the impact would be isolated to a few exception __repr__ implementations.
It can require changing code in tens of places like ``PyErr_Format(FooError, "File %R is not valid FOO", filename)`` or ``raise FooError("File %r is not valid FOO" % (filename,))``. And of course 'c:\test.txt' is valid name on Unix.
On Tue, Feb 24, 2015 at 7:40 PM, M.-A. Lemburg <mal@egenix.com> wrote:
I think the easiest way would be to tweak the error message output to indicate the real problem.
At the moment, you get:
open('c:\test.txt') Traceback (most recent call last): File "<stdin>", line 1, in <module> FileNotFoundError: [Errno 2] No such file or directory: 'c:\test.txt'
which isn't helpful.
The problem isn't the cases where the file can't be found. Those can be dealt with fairly easily, one way or another. The problem is much, much earlier, when the student was using "c:\sample.txt" and everything was working fine. Then he changed the code over to use his own file instead of the provided sample, and at the same time, switched from using open() to using csv.reader(open()), and moved all the code into a function, and fixed three other bugs, and now it isn't working. And he can't figure out why. That's why I'd like "c:\sample.txt" to raise a warning. Should I start writing up a PEP? Is that the way forward with this? ChrisA
On 02/24/2015 02:58 AM, Chris Angelico wrote:
On Tue, Feb 24, 2015 at 7:40 PM, M.-A. Lemburg <mal@egenix.com> wrote:
I think the easiest way would be to tweak the error message output to indicate the real problem.
At the moment, you get:
open('c:\test.txt') Traceback (most recent call last): File "<stdin>", line 1, in <module> FileNotFoundError: [Errno 2] No such file or directory: 'c:\test.txt'
which isn't helpful.
The problem isn't the cases where the file can't be found. Those can be dealt with fairly easily, one way or another. The problem is much, much earlier, when the student was using "c:\sample.txt" and everything was working fine. Then he changed the code over to use his own file instead of the provided sample, and at the same time, switched from using open() to using csv.reader(open()), and moved all the code into a function, and fixed three other bugs, and now it isn't working. And he can't figure out why.
That's why I'd like "c:\sample.txt" to raise a warning.
Should I start writing up a PEP? Is that the way forward with this?
Go for it. I think the solution will be two pieces: - deprecating unknown backslash-escapes, with the goal of eventually being a SyntaxError (this removes a wart from the language, as well as allowing a warning backport to 2.7) - M.A.Lemburg's idea of changing the exception message in key places to make a successful backslash replace obvious (FileNotFoundError: [Errno 2] No such file or directory: 'c:[\t]est.txt' - warning: embedded escapes) -- ~Ethan~
On Tue, Feb 24, 2015 at 5:37 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/24/2015 02:58 AM, Chris Angelico wrote:
On Tue, Feb 24, 2015 at 7:40 PM, M.-A. Lemburg <mal@egenix.com> wrote:
I think the easiest way would be to tweak the error message output to indicate the real problem.
At the moment, you get:
open('c:\test.txt') Traceback (most recent call last): File "<stdin>", line 1, in <module> FileNotFoundError: [Errno 2] No such file or directory: 'c:\test.txt'
which isn't helpful.
The problem isn't the cases where the file can't be found. Those can be dealt with fairly easily, one way or another. The problem is much, much earlier, when the student was using "c:\sample.txt" and everything was working fine. Then he changed the code over to use his own file instead of the provided sample, and at the same time, switched from using open() to using csv.reader(open()), and moved all the code into a function, and fixed three other bugs, and now it isn't working. And he can't figure out why.
That's why I'd like "c:\sample.txt" to raise a warning.
Should I start writing up a PEP? Is that the way forward with this?
Go for it.
I think the solution will be two pieces:
- deprecating unknown backslash-escapes, with the goal of eventually being a SyntaxError (this removes a wart from the language, as well as allowing a warning backport to 2.7)
- M.A.Lemburg's idea of changing the exception message in key places to make a successful backslash replace obvious (FileNotFoundError: [Errno 2] No such file or directory: 'c:[\t]est.txt' - warning: embedded escapes)
Any ideas on how this check could be implemented? How would it distinguish \t from an actual tab in the string literal, or "C:\x64-python" from "C:\d-python"...?
-- ~Ethan~
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/thomas%40python.org
-- Thomas Wouters <thomas@python.org> Hi! I'm an email virus! Think twice before sending your email to help me spread!
TBH I think this will be a tough sell. I worry that an meme will emerge to make all string literals use raw mode, or worse, to deprecate non-raw string literals. I'm not sure how we'll get out of the whole conundrum, and I'm not in a hurry to see this in 3.5. -- --Guido van Rossum (python.org/~guido)
On 02/24/2015 09:44 AM, Guido van Rossum wrote:
TBH I think this will be a tough sell. I worry that an meme will emerge to make all string literals use raw mode, or worse, to deprecate non-raw string literals. I'm not sure how we'll get out of the whole conundrum, and I'm not in a hurry to see this in 3.5.
So should we move forward with just the better error messages? It's a good first step, and even if it ends up being the only step it would be a huge improvement. -- ~Ethan~
On Tue, Feb 24, 2015 at 10:00 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/24/2015 09:44 AM, Guido van Rossum wrote:
TBH I think this will be a tough sell. I worry that an meme will emerge
string literals use raw mode, or worse, to deprecate non-raw string
to make all literals. I'm
not sure how we'll get out of the whole conundrum, and I'm not in a hurry to see this in 3.5.
So should we move forward with just the better error messages? It's a good first step, and even if it ends up being the only step it would be a huge improvement.
This is about messages from failing file-open operations if the filename contains an escaped character? I'd go slow there too: here are a lot of places where files are opened and messages are printed, both in the C code and in the stdlib. I'm not sure I buy the argument that just echoing the repr() of the name back doesn't help -- the escapes in there are actually useful in case a filename containing garbage chars (or even a trailing space) was read from some other source. And I'd weigh the needs of users who know what they are doing somewhat higher than educating newbies through error messages. While newbies are most likely to try out open() with a string literal, in "real" programs that is rare, and filenames are typically taken from the command line or from some other source where the peculiarities of Python string literals are irrelevant. -- --Guido van Rossum (python.org/~guido)
On 02/24/2015 10:14 AM, Guido van Rossum wrote:
This is about messages from failing file-open operations if the filename contains an escaped character? I'd go slow there too: here are a lot of places where files are opened and messages are printed, both in the C code and in the stdlib. I'm not sure I buy the argument that just echoing the repr() of the name back doesn't help -- the escapes in there are actually useful in case a filename containing garbage chars (or even a trailing space) was read from some other source.
I can attest from my impoverished Windows programming days that looking at --> os.listdir('c:\temp') SomeErrorMessage about syntax 'c:\temp' is not very helpful. There is zero visual indication that the \ and the t are one character, not two. Changing that error message to: SomeErrorMessage about syntax 'c:[\t]emp' or SomeErrorMessage about syntax 'c:\x07emp' or something that shouts out, "hey! one character in this location!" would be a good thing. -- ~Ethan~
I like the \x07 solution. On Tue, Feb 24, 2015 at 10:47 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/24/2015 10:14 AM, Guido van Rossum wrote:
This is about messages from failing file-open operations if the filename contains an escaped character? I'd go slow there too: here are a lot of places where files are opened and messages are printed, both in the C code and in the stdlib. I'm not sure I buy the argument that just echoing the repr() of the name back doesn't help -- the escapes in there are actually useful in case a filename containing garbage chars (or even a trailing space) was read from some other source.
I can attest from my impoverished Windows programming days that looking at
--> os.listdir('c:\temp') SomeErrorMessage about syntax 'c:\temp'
is not very helpful. There is zero visual indication that the \ and the t are one character, not two. Changing that error message to:
SomeErrorMessage about syntax 'c:[\t]emp'
or
SomeErrorMessage about syntax 'c:\x07emp'
or something that shouts out, "hey! one character in this location!" would be a good thing.
-- ~Ethan~
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
On 02/24/2015 10:49 AM, Guido van Rossum wrote:
On Tue, Feb 24, 2015 at 10:47 AM, Ethan Furman wrote:
I can attest from my impoverished Windows programming days that looking at
--> os.listdir('c:\temp') SomeErrorMessage about syntax 'c:\temp'
is not very helpful. There is zero visual indication that the \ and the t are one character, not two. Changing that error message to:
SomeErrorMessage about syntax 'c:[\t]emp'
or
SomeErrorMessage about syntax 'c:\x07emp'
or something that shouts out, "hey! one character in this location!" would be a good thing.
I like the \x07 solution.
So final question is do we take the easy road and change the repr for str to always use hex or unicode escapes instead of simple backslash-escapes (and then run for our lives), or do we just hunt down and change the appropriate error messages for files (and possibly re) ? -- ~Ethan~
On Tue, 24 Feb 2015, Ethan Furman wrote:
On 02/24/2015 10:49 AM, Guido van Rossum wrote:
On Tue, Feb 24, 2015 at 10:47 AM, Ethan Furman wrote:
I can attest from my impoverished Windows programming days that looking at
--> os.listdir('c:\temp') SomeErrorMessage about syntax 'c:\temp'
is not very helpful. There is zero visual indication that the \ and the t are one character, not two. Changing that error message to:
SomeErrorMessage about syntax 'c:[\t]emp'
or
SomeErrorMessage about syntax 'c:\x07emp'
or something that shouts out, "hey! one character in this location!" would be a good thing.
I like the \x07 solution.
So final question is do we take the easy road and change the repr for str to always use hex or unicode escapes instead of simple backslash-escapes (and then run for our lives), or do we just hunt down and change the appropriate error messages for files (and possibly re) ?
Just a data point from a random programmer: I like the \x07 solution for the error message as it draws attention to the character at issue, but I also like to see \n, \t etc. in the result of repr because it is more readable for the common cases. Isaac Morland CSCF Web Guru DC 2619, x36650 WWW Software Specialist
On 2/24/2015 1:14 PM, Guido van Rossum wrote:
And I'd weigh the needs of users who know what they are doing somewhat higher than educating newbies through error messages. While newbies are most likely to try out open() with a string literal, in "real" programs that is rare, and filenames are typically taken from the command line or from some other source where the peculiarities of Python string literals are irrelevant.
I have been thinking about proposing a new how-to: Understanding Error Messages, with a section on tracebacks followed by an alphabetical listing of Exceptions that gives people problems, with possible solutions for each. The following begins my first draft. FileNotFoundError: Possible cause [Windows]: You used a normal string literal to create a filename, you used '\' as the path separator, and you forgot that Python (like most languages) treats '\' as a case-sensitive escape character. For example: "C:\Test" is 7 chars and works as a file name, while 'C:\test' is 6 chars, one a literal tab character. The latter does not work as a file name and will raise FileNotFoundError. Possible solutions: 1. Use raw string literals for Windows path names (r'C:\test'). 2 (recommended). Use '/' as the path separator ('C:/test'), just as one does on other systems. This always works when passing file names from Python to Windows, even though it sometimes does not work in Windows Command Prompt console. -- Terry Jan Reedy
On 02/24/2015 09:18 AM, Thomas Wouters wrote:
On Tue, Feb 24, 2015 at 5:37 PM, Ethan Furman wrote:
- M.A.Lemburg's idea of changing the exception message in key places to make a successful backslash replace obvious (FileNotFoundError: [Errno 2] No such file or directory: 'c:[\t]est.txt' - warning: embedded escapes)
Any ideas on how this check could be implemented? How would it distinguish \t from an actual tab in the string literal, or "C:\x64-python" from "C:\d-python"...?
--> 'a tab' 'a\ttab' --> 'a\ttab' 'a\ttab' So the phrase "embedded escapes" may not always be accurate, but will be something to search on, and will help explain what is being seen. -- ~Ethan~
Chris Angelico wrote:
Then he changed the code over to use his own file instead of the provided sample, and at the same time, switched from using open() to using csv.reader(open()), and moved all the code into a function, and fixed three other bugs, and now it isn't working. And he can't figure out why.
There's probably a useful meta-lesson in there: test after making every change! -- Greg
On Wed, Feb 25, 2015 at 6:54 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Chris Angelico wrote:
Then he changed the code over to use his own file instead of the provided sample, and at the same time, switched from using open() to using csv.reader(open()), and moved all the code into a function, and fixed three other bugs, and now it isn't working. And he can't figure out why.
There's probably a useful meta-lesson in there: test after making every change!
Of course! But when you're working with students remotely, you don't necessarily even know that that's what's happened. All you get is "It works with sample.txt, but it doesn't work with my file". You may or may not even get to see the code at first. Firm believer in "commit often" policy... ChrisA
On 02/23/2015 02:40 PM, Nick Coghlan wrote:
- pep8 and pylint warnings as soon as a patch can be accepted - Py3kWarning in Python 2.7.x - DeprecationWarning in Python 3.5 - SyntaxWarning in Python 3.6 - SyntaxError in Python 3.7
+1 -- ~Ethan~
On Tue, Feb 24, 2015 at 9:40 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Most don't, and cases like "\n" or "\t" in a Windows path name being converted to whitespace are utterly impossible to look up in an internet search when they fail, so a user learning on their own gets left with a broken program and no particularly effective ways to ask for help figuring it out.
Like Unicode encoding errors they may appear a long way from the source of the offending data value (in this case, likely to be a file name copy and pasted from elsewhere on their system), and they don't give a particularly helpful error message (especially when the escape sequences are for whitespace).
And on the subject of Unicode errors, imagine a path name of "C:\Users\YourName\Desktop\FileName.txt". Under Python 2, that'll work fine; under Python 3, it'll fail with an error that, in all probability, will be interpreted as "Unicode is hard :(" rather than "loose backslash escapes are a problem". ChrisA
On Tue, Feb 24, 2015 at 9:40 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
While I originally disliked the idea, I think this is a genuine usability issue on Windows that would be worth addressing. However, it's a significant enough change that I believe it needs a PEP and a reasonably long transition period before anything actually breaks.
I'd be happy to write and champion such a PEP if that's the way forward. ChrisA
Hi, On Mon, Feb 23, 2015 at 9:58 PM, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
On 23 February 2015 at 16:47, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Mon, 23 Feb 2015 09:29:09 -0800 Guido van Rossum <guido@python.org> wrote:
On Mon, Feb 23, 2015 at 9:01 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 02/23/2015 08:12 AM, Brett Cannon wrote:
On Mon Feb 23 2015 at 10:55:23 AM Chris Angelico wrote:
Is it at all possible for this to be introduced in the 2.x line [...]
Starts with a minor version.
Isn't there a -3 switch or something like that in 2.7.x to trigger warnings/errors to help port to 3.x? Seems like this kind of warning could go there.
If we agree it will be a syntax error in 3.x eventually.
That sounds frankly like a pedantic change.
The real problem there, even motivating the request, seems to be new users in Windows platforms using "\" as file path separator.
That happens all the time, and is this use case that should possibly be addressed here - maybe something as simple as adding a couple of paragraphs to different places in the documentation could mitigate the issue. (in contrast to make a tons of otherwise valid code to become deprecated in a couple releases).
Note that in the introduction page of the tutorial[0], there is already the following snippet: """ If you don’t want characters prefaced by \ to be interpreted as special characters, you can use raw strings by adding an r before the first quote:
print('C:\some\name') # here \n means newline! C:\some ame print(r'C:\some\name') # note the r before the quote C:\some\name """"
Best Regards, Ezio Melotti [0]: https://docs.python.org/3/tutorial/introduction.html
Regards
Antoine.
On 23.02.15 17:54, Chris Angelico wrote:
If it's never going to happen in 2.x, I'll raise this as yet another reason to get the course and all our students migrated to 3.x, but on the flip side, it means that we absolutely can't get the benefit until that jump is made.
You can use third-party checkers. And if you use an editor with syntax highlighting, it is likely that unrecognized backslash escapes will be marked as suspicious.
participants (19)
-
Antoine Pitrou -
Ben Finney -
Brett Cannon -
Chris Angelico -
Ethan Furman -
Ezio Melotti -
Greg Ewing -
Guido van Rossum -
Isaac Morland -
Joao S. O. Bueno -
M.-A. Lemburg -
Mark Lawrence -
MRAB -
Nathaniel Smith -
Nick Coghlan -
Serhiy Storchaka -
Stephen J. Turnbull -
Terry Reedy -
Thomas Wouters