Line continuations with comments
There were a few people who liked the idea of having comments after a line continuation. I was able to make a small patch that removed the some of the restrictions on the '\' for testing some ideas which does the following. * Allow a line to continue on the same line. * Skips comments before checking for the new line after a back slash. Here are some examples... These are technically the same.
'aaa' \ ... 'bbb' \ ... 'ccc' 'aaabbbccc'
'aaa' \ 'bbb' \ 'ccc' 'aaabbbccc'
Yes there is't much need for this, but I wanted to see if it would work and if the test suit passes. It does. ;-) You can put a comment after a line continuation.
'aaa' \# one ... 'bbb' \# two ... 'ccc' # three 'aaabbbccc'
Works with expressions too.
result = \ ... + 111 \# A ... + 222 \# B ... + 333 \# C ... + 444 # D result 1110
But if it has a space between the \ and the #, the line is continued on the same line instead of the following line.
'aaa' \ #comment 'aaa'
The reason \# works, but not \ #, is when the comment comes directly after the back slash, it's removed and leaves a (backslash + new-line) pair. Removing the white space before the new line check caused some errors in the test suite. I haven't figured out why yet. So this doesn't do that for now. Currently you get this if you try any of these examples.
'abc' \#comment File "<stdin>", line 1 'abc' \#comment ^ SyntaxError: unexpected character after line continuation character
Only one of pythons tests fail, and I don't think it's related. test test_urllib2_localnet failed See the diff below if you want to play with it. It's not big. Cheers, Ron diff -r 155e6fb309f5 Parser/tokenizer.c --- a/Parser/tokenizer.c Tue May 21 21:02:04 2013 +0200 +++ b/Parser/tokenizer.c Tue May 21 22:10:31 2013 -0500 @@ -1391,18 +1391,31 @@ again: tok->start = NULL; + + c = tok_nextc(tok); + + /* Check if continuing line */ + if (tok->cont_line == 1 && c == '\n') { + tok->cont_line = 0; + c = tok_nextc(tok); + } + /* Skip spaces */ - do { + while (c == ' ' || c == '\t' || c == '\014') { c = tok_nextc(tok); - } while (c == ' ' || c == '\t' || c == '\014'); + tok->cont_line = 0; + } /* Set start of current token */ tok->start = tok->cur - 1; /* Skip comment */ - if (c == '#') + if (c == '#') { while (c != EOF && c != '\n') c = tok_nextc(tok); + tok_backup(tok, c); + goto again; + } /* Check for EOF and errors now */ if (c == EOF) { @@ -1641,12 +1654,6 @@ /* Line continuation */ if (c == '\\') { - c = tok_nextc(tok); - if (c != '\n') { - tok->done = E_LINECONT; - tok->cur = tok->inp; - return ERRORTOKEN; - } tok->cont_line = 1; goto again; /* Read next line */ }
Backslash line continuations are mostly to be avoided and making a change like this would seem to a. make them slightly less obvious when they are used, b. increase their use in cases where there aren't even long lines of code involved, and c. seem to encourage their use in general. It seems to me that using parentheses is an already-existing, somewhat-better way to do what you're doing in your examples. Mike On Wed, May 22, 2013 at 12:13 AM, Ron Adam <ron3200@gmail.com> wrote:
There were a few people who liked the idea of having comments after a line continuation.
I was able to make a small patch that removed the some of the restrictions on the '\' for testing some ideas which does the following.
* Allow a line to continue on the same line.
* Skips comments before checking for the new line after a back slash.
Here are some examples...
These are technically the same.
'aaa' \ ... 'bbb' \ ... 'ccc' 'aaabbbccc'
'aaa' \ 'bbb' \ 'ccc' 'aaabbbccc'
Yes there is't much need for this, but I wanted to see if it would work and if the test suit passes. It does. ;-)
You can put a comment after a line continuation.
'aaa' \# one ... 'bbb' \# two ... 'ccc' # three 'aaabbbccc'
Works with expressions too.
result = \ ... + 111 \# A ... + 222 \# B ... + 333 \# C ... + 444 # D result 1110
But if it has a space between the \ and the #, the line is continued on the same line instead of the following line.
'aaa' \ #comment 'aaa'
The reason \# works, but not \ #, is when the comment comes directly after the back slash, it's removed and leaves a (backslash + new-line) pair.
Removing the white space before the new line check caused some errors in the test suite. I haven't figured out why yet. So this doesn't do that for now.
Currently you get this if you try any of these examples.
'abc' \#comment File "<stdin>", line 1 'abc' \#comment ^ SyntaxError: unexpected character after line continuation character
Only one of pythons tests fail, and I don't think it's related.
test test_urllib2_localnet failed
See the diff below if you want to play with it. It's not big.
Cheers, Ron
diff -r 155e6fb309f5 Parser/tokenizer.c --- a/Parser/tokenizer.c Tue May 21 21:02:04 2013 +0200 +++ b/Parser/tokenizer.c Tue May 21 22:10:31 2013 -0500 @@ -1391,18 +1391,31 @@
again: tok->start = NULL; + + c = tok_nextc(tok); + + /* Check if continuing line */ + if (tok->cont_line == 1 && c == '\n') { + tok->cont_line = 0; + c = tok_nextc(tok); + } + /* Skip spaces */ - do { + while (c == ' ' || c == '\t' || c == '\014') { c = tok_nextc(tok); - } while (c == ' ' || c == '\t' || c == '\014'); + tok->cont_line = 0; + }
/* Set start of current token */ tok->start = tok->cur - 1;
/* Skip comment */ - if (c == '#') + if (c == '#') { while (c != EOF && c != '\n') c = tok_nextc(tok); + tok_backup(tok, c); + goto again; + }
/* Check for EOF and errors now */ if (c == EOF) { @@ -1641,12 +1654,6 @@
/* Line continuation */ if (c == '\\') { - c = tok_nextc(tok); - if (c != '\n') { - tok->done = E_LINECONT; - tok->cur = tok->inp; - return ERRORTOKEN; - } tok->cont_line = 1; goto again; /* Read next line */ }
______________________________**_________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
On Wed, May 22, 2013 at 9:41 AM, Mike Graham <mikegraham@gmail.com> wrote:
Backslash line continuations are mostly to be avoided and making a change like this would seem to [snip: mostly make them more usable]
The way I see it, either one believes that backslashes belong in Python -- and therefore they should be made as useful as possible -- or that they do not -- and therefore they should be crippled. But if they don't belong in Python, they shouldn't be crippled, rather, they shouldn't even exist. A compromise should at least be internally consistent. -- Devin
On Wed, May 22, 2013 at 10:11 AM, Devin Jeanpierre <jeanpierreda@gmail.com>wrote:
The way I see it, either one believes that backslashes belong in Python -- and therefore they should be made as useful as possible -- or that they do not -- and therefore they should be crippled. But if they don't belong in Python, they shouldn't be crippled, rather, they shouldn't even exist.
A compromise should at least be internally consistent.
With changes a while back making backslash continuations never strictly necessary, it seems like officially declaring backslash deprecated might be reasonable. They are discouraged by PEP8 and other style guides and at this point they violate the "There should be one-- and preferably only one --obvious way to do it." principle. Mike
On Wed, May 22, 2013, at 10:31, Mike Graham wrote:
With changes a while back making backslash continuations never strictly necessary,
Someone pointed out an example a while back of them being necessary (except for being vacuously unnecessary because you can just make an arbitrarily long physical line): multiple values in a with statement.
On May 22, 2013, at 7:54, random832@fastmail.us wrote:
On Wed, May 22, 2013, at 10:31, Mike Graham wrote:
With changes a while back making backslash continuations never strictly necessary,
Someone pointed out an example a while back of them being necessary (except for being vacuously unnecessary because you can just make an arbitrarily long physical line): multiple values in a with statement.
Well, you can always break within each expression, adding extra parens within the expression if necessary. But often, this looks even worse than backslashes. with closing(NSWhyDoesAppleUseSuchLongNames( NSLongNamedConstant)) as thing1, closing( NSAnotherLongFunction( NSAnotherLongConstant)) as thing2 I may have got this wrong because I'm typing on a phone, but you get the idea. This is legal, and it follows PEP8, and it doesn't require a backslash. But it's horrible. Another alternative is to bind all those silly PyObjC names to shorter names--but then, when you're reading the code, you can't immediately tell what it does. I don't _always_ have to use backslashes in code using ridiculous names like this to make it readable, but sometimes they are the best solution. So, even though I don't know that they're _necessary_, I still don't want backslashes deprecated. But I also don't want them expanded.
On Thu, May 23, 2013 at 12:31 AM, Mike Graham <mikegraham@gmail.com> wrote:
On Wed, May 22, 2013 at 10:11 AM, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
The way I see it, either one believes that backslashes belong in Python -- and therefore they should be made as useful as possible -- or that they do not -- and therefore they should be crippled. But if they don't belong in Python, they shouldn't be crippled, rather, they shouldn't even exist.
A compromise should at least be internally consistent.
With changes a while back making backslash continuations never strictly necessary, it seems like officially declaring backslash deprecated might be reasonable. They are discouraged by PEP8 and other style guides and at this point they violate the "There should be one-- and preferably only one --obvious way to do it." principle.
Maybe the backslash should be considered on par with goto - some use it occasionally and reckon it's vital, others never use it and consider it superfluous. It's not (usually) a problem to have it in the language, it's not deprecated, but style guides advise against its use. ChrisA
On Wed, May 22, 2013 at 6:41 AM, Mike Graham <mikegraham@gmail.com> wrote:
Backslash line continuations are mostly to be avoided and making a change like this would seem to
a. make them slightly less obvious when they are used, b. increase their use in cases where there aren't even long lines of code involved, and c. seem to encourage their use in general.
It seems to me that using parentheses is an already-existing, somewhat-better way to do what you're doing in your examples.
Your last statement seems to be missing the point of the larger discussion. Yes, parenthesis can be used in most cases where someone might use \ continuation. There seems to be strong sentiment to *not* remove \ continuation. Given that, is allowing comments after a continuation a reasonable change? I think so. Notwithstanding that, these discussions are moving away from Guido's original comment about being bitten by implicit continuation of strings and not moving towards consensus. Let me throw in a few facts: 1) There are bugs caused by unintended implicit string concatenation. 2) Using + as it exists now is not a drop-in replacement for implicit string concatenation as it is a run-time operation and has a different precedence than the implicit concatenation. 3) There are programs that use implicit string concatenation that will need to be fixed if the feature is removed. 4) There are programs that use \ continuation that will need to be fixed if the feature is removed. 5) Explicit is better than implicit. Personally, I would endorse deprecating and eventually removing implicit string concatenation and adding a syntax (not an operator) for explicit run-time string concatenation. The use of \ continuation as that syntax seems to me like a reasonable choice if we assume that this feature isn't going away. In particular, it works today so it's easy to start using it and linters can look for it. However, it's pointless to bikeshed the choice of syntax if there's no consensus that there should be an explicit syntax in the first place. On Tue, May 21, 2013 at 9:13 PM, Ron Adam <ron3200@gmail.com> wrote:
I was able to make a small patch that removed the some of the restrictions on the '\' for testing some ideas which does the following.
* Allow a line to continue on the same line.
Aside from serving as a marker for explicit string concatenation, this means that I can freely sprinkle in backslashes anywhere I want, like this: foo \ = \ bar + \ \ 3 That seems like a bad idea.
<snip>
The reason \# works, but not \ #, is when the comment comes directly after the back slash, it's removed and leaves a (backslash + new-line) pair.
Whether we require there be no space between \ and # or, conversely, require there be at least one whitespace character should not be based on the relative ease of patching the current code. Personally, I would prefer the latter as I believe requiring a space after \ will increase readability. --- Bruce Latest blog post: Alice's Puzzle Page http://www.vroospeak.com Learn how hackers think: http://j.mp/gruyere-security
On 05/22/2013 11:02 AM, Bruce Leban wrote:
On Wed, May 22, 2013 at 6:41 AM, Mike Graham <mikegraham@gmail.com <mailto:mikegraham@gmail.com>> wrote:
Backslash line continuations are mostly to be avoided and making a change like this would seem to
a. make them slightly less obvious when they are used, b. increase their use in cases where there aren't even long lines of code involved, and c. seem to encourage their use in general.
It seems to me that using parentheses is an already-existing, somewhat-better way to do what you're doing in your examples.
Your last statement seems to be missing the point of the larger discussion. Yes, parenthesis can be used in most cases where someone might use \ continuation. There seems to be strong sentiment to *not* remove \ continuation. Given that, is allowing comments after a continuation a reasonable change? I think so.
Notwithstanding that, these discussions are moving away from Guido's original comment about being bitten by implicit continuation of strings and not moving towards consensus. Let me throw in a few facts:
1) There are bugs caused by unintended implicit string concatenation.
There was one found in pythons own library recently where a missing comma caused an unintentional implicit concatenation. It's fixed now, but it's not clear how long it's been there.
2) Using + as it exists now is not a drop-in replacement for implicit string concatenation as it is a run-time operation and has a different precedence than the implicit concatenation. 3) There are programs that use implicit string concatenation that will need to be fixed if the feature is removed. 4) There are programs that use \ continuation that will need to be fixed if the feature is removed. 5) Explicit is better than implicit.
Agree on all counts.
Personally, I would endorse deprecating and eventually removing implicit string concatenation and adding a syntax (not an operator) for explicit run-time string concatenation. The use of \ continuation as that syntax seems to me like a reasonable choice if we assume that this feature isn't going away. In particular, it works today so it's easy to start using it and linters can look for it.
I agree here too. I'm still looking for the location in pythons source code where adjacent strings are joined. The docs say this ... ----------- Note that this feature is defined at the syntactical level, but implemented at compile time. The ‘+’ operator must be used to concatenate string expressions at run time. Also note that literal concatenation can use different quoting styles for each component (even mixing raw strings and triple quoted strings). ----------- I haven't found either the syntactic definition, or the compile time implementation yet. <shrug?>
However, it's pointless to bikeshed the choice of syntax if there's no consensus that there should be an explicit syntax in the first place.
It seems the consensus may be dependent on what the syntax might be.
On Tue, May 21, 2013 at 9:13 PM, Ron Adam <ron3200@gmail.com <mailto:ron3200@gmail.com>> wrote:
I was able to make a small patch that removed the some of the restrictions on the '\' for testing some ideas which does the following.
* Allow a line to continue on the same line.
Aside from serving as a marker for explicit string concatenation, this means that I can freely sprinkle in backslashes anywhere I want, like this:
foo \ = \ bar + \ \ 3
That seems like a bad idea.
Yes, But just don't do that. ;-) And don't do this either. (works now) foo \ = \ bar + \ \ 3 Or this. (also works now) foo = ( (bar + ( ( 3)) ) ) I'm not sure why some people dislike the back slash so much as a continuation tool. Python is the language that avoids using {braces around blocks}, so you'd think it would be the other way around. I really don't think the '\' will be over used. New programmers do try a lot of strange things at first for making their own programs easier to read, (I did), but usually they come around to what the community practices and recommends.
<snip>
The reason \# works, but not \ #, is when the comment comes directly after the back slash, it's removed and leaves a (backslash + new-line) pair.
Whether we require there be no space between \ and # or, conversely, require there be at least one whitespace character should not be based on the relative ease of patching the current code. Personally, I would prefer the latter as I believe requiring a space after \ will increase readability.
My preference is to allow any number of spaces before and after the '\'. and that any comments after the slash not change what it means. A built in single space rule would probably frustrate people who don't want to do it that way. Like how you get a character after a continuation error, even if it's only a single space. Yeah, it's how it works, but it's still annoying to get that error in that situation. A comment of course, would still uses up the rest of the line. So a '\' after '#' is just part of the comment. Cheers, Ron
On 23.05.13 01:15, Ron Adam wrote:
I'm not sure why some people dislike the back slash so much as a continuation tool. Python is the language that avoids using {braces around blocks}, so you'd think it would be the other way around.
I really don't think the '\' will be over used. New programmers do try a lot of strange things at first for making their own programs easier to read, (I did), but usually they come around to what the community practices and recommends.
I think the reason that people don't like the '\' is that it opens up so many bad memories: - having to use them inside C macros - having to deal with it on windows - felt unpythonic because it is a low-level trick to produce longer lines Especially the unpythonic feeling comes from the fact that '\' either has special meaning in strings (which is accepted) or has very restricted use outside of strings: there must not be anything but the line end after it. So by lifting the usability of '\' to allow comments after it makes it from an ugly left-over C feature into something useful that people _want_ to use for splitting the continuation of something from the comments. I think after introducing it, the bad prejudice will vanish. The mental change is from "a hack because the line doesn't fit" into "a well-formed structuring tool to split code and comment"... cheers - chris
<snip>
The reason \# works, but not \ #, is when the comment comes directly after the back slash, it's removed and leaves a (backslash + new-line) pair.
Whether we require there be no space between \ and # or, conversely, require there be at least one whitespace character should not be based on the relative ease of patching the current code. Personally, I would prefer the latter as I believe requiring a space after \ will increase readability.
My preference is to allow any number of spaces before and after the '\'. and that any comments after the slash not change what it means.
A built in single space rule would probably frustrate people who don't want to do it that way. Like how you get a character after a continuation error, even if it's only a single space. Yeah, it's how it works, but it's still annoying to get that error in that situation.
A comment of course, would still uses up the rest of the line. So a '\' after '#' is just part of the comment.
Cheers, Ron
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
On 2013-06-09 21:52, Christian Tismer wrote:
On 23.05.13 01:15, Ron Adam wrote:
I'm not sure why some people dislike the back slash so much as a continuation tool. Python is the language that avoids using {braces around blocks}, so you'd think it would be the other way around.
I really don't think the '\' will be over used. New programmers do try a lot of strange things at first for making their own programs easier to read, (I did), but usually they come around to what the community practices and recommends.
I think the reason that people don't like the '\' is that it opens up so many bad memories:
- having to use them inside C macros - having to deal with it on windows - felt unpythonic because it is a low-level trick to produce longer lines
Especially the unpythonic feeling comes from the fact that '\' either has special meaning in strings (which is accepted) or has very restricted use outside of strings:
there must not be anything but the line end after it.
So by lifting the usability of '\' to allow comments after it makes it from an ugly left-over C feature into something useful that people _want_ to use for splitting the continuation of something from the comments.
I think after introducing it, the bad prejudice will vanish.
The mental change is from "a hack because the line doesn't fit" into "a well-formed structuring tool to split code and comment"...
there are two other reasons that jump into sight: - as continuation signal in shell: who has not been bitten by an innocent space following the backslash? ... and it's not the space we dislike ;-) - on my console I use three fingers to produce this one symbol "\" eg. <ALT-SHIFT-7> so I even might visit trigraph-land again ??/ All the best, Stefan.
cheers - chris
<snip>
The reason \# works, but not \ #, is when the comment comes directly after the back slash, it's removed and leaves a (backslash + new-line) pair.
Whether we require there be no space between \ and # or, conversely, require there be at least one whitespace character should not be based on the relative ease of patching the current code. Personally, I would prefer the latter as I believe requiring a space after \ will increase readability.
My preference is to allow any number of spaces before and after the '\'. and that any comments after the slash not change what it means.
A built in single space rule would probably frustrate people who don't want to do it that way. Like how you get a character after a continuation error, even if it's only a single space. Yeah, it's how it works, but it's still annoying to get that error in that situation.
A comment of course, would still uses up the rest of the line. So a '\' after '#' is just part of the comment.
Cheers, Ron
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On Mon, Jun 10, 2013 at 1:33 AM, Stefan Drees <stefan@drees.name> wrote:
- as continuation signal in shell: who has not been bitten by an innocent space following the backslash? ... and it's not the space we dislike ;-)
Right, what we dislike is that \ takes invisible whitespace into account for no good reason.
- on my console I use three fingers to produce this one symbol "\" eg. <ALT-SHIFT-7> so I even might visit trigraph-land again ??/
This isn't a reason to oppose making "\" less painful to use. You can just as easily not use "\" after the change as before. -- Devin
On 2013-06-10 21:13 CEST, Devin Jeanpierre wrote:
On Mon, Jun 10, 2013 at 1:33 AM, Stefan Drees ... wrote:
- as continuation signal in shell: who has not been bitten by an innocent space following the backslash? ... and it's not the space we dislike ;-)
Right, what we dislike is that \ takes invisible whitespace into account for no good reason.
I really thought about not sending the above list item. We are so used to having explicitely visible whitpace character displays in our working environments flanked by linting away all politically incorrect whitespace (we even have a nanny for that ...), but sometimes, I simply enjoy the pure sight of wide empty space separating words and than, invisible WS strikes me after a \\ ... thanks for also blaming the ??/
- on my console I use three fingers to produce this one symbol "\" eg. <ALT-SHIFT-7> so I even might visit trigraph-land again ??/
This isn't a reason to oppose making "\" less painful to use. You can just as easily not use "\" after the change as before.
of course that is right :-) I mean "not for me" and "less painful for the rest" is fully ok. I thought Chris was just exhaustively listing all known sources for "bad fee\ings" and I had one or two I did not spot on his shopping list. Stefan.
participants (9)
-
Andrew Barnert -
Bruce Leban -
Chris Angelico -
Christian Tismer -
Devin Jeanpierre -
Mike Graham -
random832＠fastmail.us -
Ron Adam -
Stefan Drees