On Wed, Mar 14, 2018 at 01:43:53PM +0100, Søren Pilgård wrote: [...]

I have also experienced beginners asking why you can do `x = "abc" "def"` but not `a = "abc"; b = "def"; x = a b` and then you have to either explain them the differences between strings and string literals or just tell them to always use `+`.

If you tell them to "always use `+`", you are teaching them to be cargo-cult coders who write code they don't understand for reasons they don't know. Life is too short to learn *everything*, I know that, and I've certainly copied lots of code I don't understand (and hoped I'd never need to debug it!). If that makes me a cargo-cult coder too, so be it. But never over something as simple as the difference between names a and b, and string literals "abc", "def". -- Steve

While I personally dislike implicit string concatenation, I've worked in projects that use it, and I, more often than I'd like, use them myself. The problem is that there is no "less typing" way of entering a correct multi-line string - due to multiline string literals preserving the indentation characters. It is a bit cumbersome to do something like from textwrap import dedent as D a = D("""\ My code-aligned, multiline text with no spaces to the left goes here """) And people then resort for implicit concatenation instead of that. I am sure this has been discussed before, and decided against, but I'd be all for taking the occasion to add a "D" prefix to multiline strings to apply de-denting to the literals. Regards, js -><- On 14 March 2018 at 10:15, Steven D'Aprano <steve@pearwood.info> wrote:

On Wed, Mar 14, 2018 at 09:18:30AM -0300, Facundo Batista wrote:

...

Hello!

What would you think about formally descouraging the following idiom?

long_string = ( "some part of the string " "with more words, actually is the same " "string that the compiler puts together")

I would hate that.

...

We should write the following, instead:

long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

Should we? I disagree.

Of course you're welcome to specify that in your own style-guide for your own code, but I won't be following that recommendation.

...

I know that "no change to Python itself" is needed, but having a formal discouragement of the idiom will help in avoiding people to fall in mistakes like:

fruits = { "apple", "orange" "banana", "melon", }

People can make all sorts of mistakes through carlessness. I wrote

{y, x*3}

the other day instead of {y: x**3}. (That's *two* errors in one simple expression. I wish I could say it was a record for me.) Should we "discourage" exponentiation and dict displays and insist on writing dict((y, x*x*x)) to avoid the risk of errors? I don't think so.

I think string concatenation falls into the same category. Sometimes even the most careful writer makes a mistake (let alone careless writers). That's life. Not every problem needs a technical "solution". Sometimes the right solution is to proof-read your code, or have code review by a fresh pair of eyes.

And tests, of course.

[...]

...

Note that there's no penalty in adding the '+' between the strings, those are resolved at compilation time.

That is an implementation feature, not a language requirement. Not all Python interpreters will do that, and they are free to put limits on how much they optimize. Here's Python 3.5:

py> import dis py> dis.dis('s = "abcdefghijklmnopqrs" + "t"') 1 0 LOAD_CONST 3 ('abcdefghijklmnopqrst') 3 STORE_NAME 0 (s) 6 LOAD_CONST 2 (None) 9 RETURN_VALUE

But now see this:

py> dis.dis('s = "abcdefghijklmnopqrs" + "tu"') 1 0 LOAD_CONST 0 ('abcdefghijklmnopqrs') 3 LOAD_CONST 1 ('tu') 6 BINARY_ADD 7 STORE_NAME 0 (s) 10 LOAD_CONST 2 (None) 13 RETURN_VALUE

And older versions of CPython didn't optimize this at all, and some day there could be a command-line switch or environment variable to turn these optimizations off.

With string concatentation having potential O(N**2) performance, if you're writing explicit string concatenations with +, they could potentially be *very* expensive at runtime.

-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 14 March 2018 at 13:40, Søren Pilgård <fiskomaten@gmail.com> wrote:

We can't remove all potential pitfalls, but I do think there is value in evaluating whether something has bigger potential to cause harm than the benefits it brings, especially if there are other ways to do the same.

Certainly. However, in this case opinions differ (I, like Steven, have no problem with implicit string concatenation, and in certain cases prefer it over using explicit addition and relying on the compiler optimising that away for me). Also, we need to consider the significant body of code that would break if this construct were prohibited. Even if we were to agree that the harm caused by implicit concatenation outweighed the benefits, that would *still* not justify making it illegal, unless the net gain could be shown to justify the cost of forcing every project that currently uses implicit concatenation to change their code, debug those changes, make new releases, etc. Paul.

2018-03-14 10:54 GMT-03:00 Paul Moore <p.f.moore@gmail.com>:

Also, we need to consider the significant body of code that would break if this construct were prohibited. Even if we were to agree that the harm caused by implicit concatenation outweighed the benefits, that would *still* not justify making it illegal, unless the net gain could be shown to justify the cost of forcing every project that currently uses implicit concatenation to change their code, debug those changes, make new releases, etc.

I never said to prohibit it, just discourage the idiom. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

On Thu, Mar 15, 2018 at 12:40 AM, Søren Pilgård <fiskomaten@gmail.com> wrote:

Of course you can always make error, even in a single letter. But I think there is a big difference between mixing up +-/* and ** where the operator is in "focus" and the implicit concatenation where there is no operator. A common problem is that you have something like foo(["a", "b", "c" ]).bar() but then you remember that there also needs to be a "d" so you just add foo(["a", "b", "c" "d" ]).bar() Causing an error, not with the "d" expression you are working on but due to what you thought was the previous expression but python turns it into one. The , is seen as a delimiter by the programmer not as part of the operation (or the lack of the ,).

You're creating a list. Put a comma at the end of every line; problem solved. Your edit would be from this: foo(["a", "b", "c", ]).bar() to this: foo(["a", "b", "c", "d", ]).bar() and there is no bug. In fact, EVERY time you lay out a list display vertically (or dict or set or any equivalent), ALWAYS put commas. That way, you can reorder the lines freely (you don't special-case the last one), you can append a line without changing the previous one (no noise in the diff), etc, etc, etc. ChrisA

On Thu, Mar 15, 2018 at 8:58 PM, Rob Cliffe via Python-ideas <python-ideas@python.org> wrote:

On 14/03/2018 17:57, Chris Angelico wrote:

On Thu, Mar 15, 2018 at 12:40 AM, Søren Pilgård <fiskomaten@gmail.com> wrote:

Of course you can always make error, even in a single letter. But I think there is a big difference between mixing up +-/* and ** where the operator is in "focus" and the implicit concatenation where there is no operator. A common problem is that you have something like foo(["a", "b", "c" ]).bar() but then you remember that there also needs to be a "d" so you just add foo(["a", "b", "c" "d" ]).bar() Causing an error, not with the "d" expression you are working on but due to what you thought was the previous expression but python turns it into one. The , is seen as a delimiter by the programmer not as part of the operation (or the lack of the ,).

You're creating a list. Put a comma at the end of every line; problem solved. Your edit would be from this:

foo(["a", "b", "c", ]).bar()

to this:

foo(["a", "b", "c", "d", ]).bar()

and there is no bug. In fact, EVERY time you lay out a list display vertically (or dict or set or any equivalent), ALWAYS put commas. That way, you can reorder the lines freely (you don't special-case the last one), you can append a line without changing the previous one (no noise in the diff), etc, etc, etc.

My thoughts exactly. I make it a personal rule to ALWAYS add a comma to every line, including the last one, in this kind of list (/dict/set etc.). Python allows it - take advantage of it! (A perhaps minor-seeming feature of the language which actually is a big benefit if you use it.) Preferably with all the commas vertically aligned to highlight the structure (I'm a great believer BTW in using vertical alignment, even if it means violating Python near-taboos such as more that one statement per line). Also I would automatically put the first string (as well as the last) on a line by itself: foo([ "short string" , "extremely looooooooooooong string" , "some other string" , ]) Then as Chris says (sorry to keep banging the drum), the lines can trivially be reordered, and adding more lines never causes a problem as long as I stick to the rule. Which I do automatically because I think my code looks prettier that way.

From a purist angle, implicit string concatenation is somewhat inelegant (where else in Python can you have two adjacent operands not separated by an operator/comma/whatever? We don't use reverse Polish notation). And I could live without it. But I have found it useful from time to time: constructing SQL queries or error messages or other long strings that I needed for some reason, where triple quotes would be more awkward (and I find line continuation backslashes ugly, *especially* in mid-string). I guess my attitude is: "If you want to read my Python code, 90%+ of the time it will be *obvious* that these strings are *meant* to be concatenated. But if it isn't, then you need to learn some Python basics first (Sorry!).".

I have never as far as I can remember had a bug caused by string concatenation. However, it is possible that I am guilty of selective memory.

+1, though, for linters to point out possible errors from possible accidental omission of a comma (if they don't already). It never hurts to have our code checked.

The linters I know use parsed ast nodes, so if it's not valid grammar, it won't parse. The _linters_ don't check for cases like f(a b) because that's not valid grammar and already caught by the parser. I think that's what you were noting when you said "if they don't already"? As for the issue in general, this is my understanding of linters: The code is loaded immediate into the ast parser. There's a no post-hoc way to know why the node is a string literal. Specifically the nodes for "ab" and "a""b" are identical Str nodes. Reversing from the Str node is impossible (unless a flag/attribute/context/whatever gets added), as the information is destroyed by then. The following might be possible: 1) The line number for a node n1 provides a way select the code between nodes n1 and the next node n2, which contains the offending string. This code needs to be retrieved from somewhere (easily kept from the beginning if not already) 2) A quick reparse of just that chunk will confirm that it contains the target node, so the code retrieval for the target node can be sanity checked (it's not exact retrieval of just the code we want, it's guaranteed to overshoot as resolution is on line numbers.) 3) ? (note below) 4) A quick check shows the tokenizer can differentiate the styles of literal. It gives 2 STRING for 'a''b' and 1 for 'ab'. This is a reliable test if the right code can be found which matches the target string. Hopefully there are others better ways but at least one exists. (better in the sense that the linters i know do not including tokenization currently) 3) The biggest problem for me (hopefully someone just knows the answer) is that some other very reliable parsing is required to know that the literal being _tested_ is the literal we _mean_ to test. I can use ast parser in 2) because im confirming a different piece of information. The problem is that the literal can exist in a very complex section, and coincidentally with 'a''b' and 'ab' in the same expression. The ast node won't tell is if we are looking for 0, 1 or 2 cases of either syntax, we only get 2 Str nodes and 1 line number. I think the parser chosen would pretty much have to be a rewrite of the ast parser due to nesting. Or someone needs to root around in the internals of the ast parser to see if the information can be extracted at the right time. I hope this helps in some way. I don't think it's impossible, but the above will introduce annoying bits into existing linters for this one issue. Of course, a working example would certainly make any case a lot easier. Given there is no technical reason, I agree there's no reason to change anything. This just "feels" ugly to me, so i can tilt at windmills all day on it, but I can see no technical reason. Always amazes me that string are so weird. They are literals and lists, but also multi white space jumping zebras. Are they "multi" "white space" "jumping zebras"? "multi" "white space jumping" "zebras"? "multi white" "space jumping" "zebras? We'll never know! -Matt

Best wishes Rob Cliffe

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Note that this has already been proposed and rejected before: https://www.python.org/dev/peps/pep-3126/ Stephan Op 14 mrt. 2018 15:01 schreef "Oleg Broytman" <phd@phdru.name>:

On Thu, Mar 15, 2018 at 12:15:52AM +1100, Steven D'Aprano < steve@pearwood.info> wrote:

...

On Wed, Mar 14, 2018 at 09:18:30AM -0300, Facundo Batista wrote:

...

We should write the following, instead:

long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

Should we? I disagree.

Of course you're welcome to specify that in your own style-guide for your own code, but I won't be following that recommendation.

...

I know that "no change to Python itself" is needed, but having a formal discouragement of the idiom will help in avoiding people to fall in mistakes like:

fruits = { "apple", "orange" "banana", "melon", }

People can make all sorts of mistakes through carlessness. I wrote

{y, x*3}

the other day instead of {y: x**3}. (That's *two* errors in one simple expression. I wish I could say it was a record for me.) Should we "discourage" exponentiation and dict displays and insist on writing dict((y, x*x*x)) to avoid the risk of errors? I don't think so.

We should fix what causes real problems, not convoluted ones. And this particular misfeature caused problems for me.

...

-- Steve

Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

2018-03-14 11:09 GMT-03:00 Stephan Houben <stephanh42@gmail.com>:

Note that this has already been proposed and rejected before:

https://www.python.org/dev/peps/pep-3126/

What was proposed and reject was the *removal* of the feature/syntax. I propose the discouragement of the idiom. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

+1 on general idea of discouraging it.

[...] mistakes like:

fruits = { "apple", "orange" "banana", "melon", }

+1

(and even making the static analysers, like pyflakes or pylint, to show that as a warning)

+1

I agree that implicit concatenation is a bad feature. I have seen it cause multiple stupid errors while I have never actually used it for anything useful. [...] Zen of Python.

+1. But I think a counter argument would be a real world example.

We should write the following, instead:

long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

Maybe the "+" at the beginnig as PEP8 said, to clearly mark it present and explicit. long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together") or long_string = ("" + "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

People can make all sorts of mistakes through carlessness. I wrote {y, x*3} the other day instead of {y: x**3}

Other comments said this isn't really the same kind of error, I agree to them.

...

Note that there's no penalty in adding the '+' between the strings, those are resolved at compilation time.

That is an implementation feature, not a language requirement. Not all Python interpreters will do that, and they are free to put limits on how much they optimize. Here's Python 3.5:

With string concatentation having potential O(N**2) performance, if you're writing explicit string concatenations with +, they could potentially be *very* expensive at runtime.

I completely agree, by emitting warning if this syntax is used, codes Could be less efficient, but that's also the job of the implementations to be fast. This is the only counter-argument I have to OP, even if I really like the proposal.

Life is too short to learn *everything*, I know that, and I've certainly copied lots of code I don't understand (and hoped I'd never need to debug it!). If that makes me a cargo-cult coder too, so be it.

I find that learning `x = "e" "f"` does indeed compile is also learning and surprising (especially for beginners).

While I personally dislike implicit string concatenation, I've worked in projects that use it, and I, more often than I'd like, use them myself.

We would like to see more real world example vs real world mistakes.

It is a bit cumbersome to do something like from textwrap import dedent as D a = D("""\ My code-aligned, multiline text with no spaces to the left goes here """)

+1. But you should add "\n" to have a = ("hello\n" "world\n" "how") Which would be maybe better written as a = '\n'.join([ "hello", "world", "how"]) (put the spaces where you like) which can easily be switched to ''.join if needed.

A common problem is that you have something like foo(["a", "b", "c" ]).bar() but then you remember that there also needs to be a "d" so you just add

+1

Yes of course the skilled programmer needs to understand the difference. But I am talking about absolute beginners where python in many regards is an excelent first language.

Agreed. 2018-03-14 15:30 GMT+01:00 Stephan Houben <stephanh42@gmail.com>:

Op 14 mrt. 2018 15:23 schreef "Facundo Batista" <facundobatista@gmail.com>:

I propose the discouragement of the idiom.

What does that mean?

Stephan

Regards,

-- . Facundo

Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Hi Facundo, On 14.03.2018 17:47, Facundo Batista wrote:

2018-03-14 11:30 GMT-03:00 Stephan Houben <stephanh42@gmail.com>:

...

Op 14 mrt. 2018 15:23 schreef "Facundo Batista" <facundobatista@gmail.com>:

I propose the discouragement of the idiom.

What does that mean?

That we say "hey, this works but please don't use it, because it tends to a error prone way of writing some code, instead do this".

I believe this falls under coding style and is not something we should actively discourage. It's an idiom that many programming languages share with Python. You may also want to look at this PEP for a longer discussion on the topic: https://www.python.org/dev/peps/pep-3126/ Even pylint rejected a request to have a rule added for this: https://github.com/PyCQA/pylint/issues/1589 FWIW: I use implicit concats a lot and find them very useful for e.g. writing long SQL queries or longer string literals at indented levels where the triple quote approach doesn't work well. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 14 2018)

...

...

Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/

::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/

Facundo Batista wrote:

That we say "hey, this works but please don't use it, because it tends to a error prone way of writing some code, instead do this".

The trouble is that there is nowhere near widespread agreement that it is error prone enough to be worth such a drastic measure as deprecating the whole feature. -- Greg

+1 on general idea of discouraging it.

[...] mistakes like:

fruits = { "apple", "orange" "banana", "melon", }

+1

(and even making the static analysers, like pyflakes or pylint, to show that as a warning)

+1

I agree that implicit concatenation is a bad feature. I have seen it cause multiple stupid errors while I have never actually used it for anything useful. [...] Zen of Python.

+1. But I think a counter argument would be a real world example.

We should write the following, instead:

long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

Maybe the "+" at the beginnig as PEP8 said, to clearly mark it present and explicit. long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together") or long_string = ("" + "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

People can make all sorts of mistakes through carlessness. I wrote {y, x*3} the other day instead of {y: x**3}

Other comments said this isn't really the same kind of error, I agree to them.

...

Note that there's no penalty in adding the '+' between the strings, those are resolved at compilation time.

That is an implementation feature, not a language requirement. Not all Python interpreters will do that, and they are free to put limits on how much they optimize. Here's Python 3.5:

With string concatentation having potential O(N**2) performance, if you're writing explicit string concatenations with +, they could potentially be *very* expensive at runtime.

I completely agree, by emitting warning if this syntax is used, codes Could be less efficient, but that's also the job of the implementations to be fast. This is the only counter-argument I have to OP, even if I really like the proposal.

Life is too short to learn *everything*, I know that, and I've certainly copied lots of code I don't understand (and hoped I'd never need to debug it!). If that makes me a cargo-cult coder too, so be it.

I find that learning `x = "e" "f"` does indeed compile is also learning and surprising (especially for beginners).

While I personally dislike implicit string concatenation, I've worked in projects that use it, and I, more often than I'd like, use them myself.

We would like to see more real world example vs real world mistakes.

It is a bit cumbersome to do something like from textwrap import dedent as D a = D("""\ My code-aligned, multiline text with no spaces to the left goes here """)

+1. But you should add "\n" to have a = ("hello\n" "world\n" "how") Which would be maybe better written as a = '\n'.join([ "hello", "world", "how"]) (put the spaces where you like) which can easily be switched to ''.join if needed.

A common problem is that you have something like foo(["a", "b", "c" ]).bar() but then you remember that there also needs to be a "d" so you just add

+1

Yes of course the skilled programmer needs to understand the difference. But I am talking about absolute beginners where python in many regards is an excelent first language.

Agreed.

On Wed, Mar 14, 2018 at 03:01:07PM +0100, Oleg Broytman wrote:

...

People can make all sorts of mistakes through carlessness. I wrote

...

{y, x*3}

the other day instead of {y: x**3}. (That's *two* errors in one simple expression. I wish I could say it was a record for me.) Should we "discourage" exponentiation and dict displays and insist on writing dict((y, x*x*x)) to avoid the risk of errors? I don't think so.

We should fix what causes real problems, not convoluted ones. And this particular misfeature caused problems for me.

I don't think that using a single asterisk instead of two is a "convoluted" problem. Please Oleg, if you want others to respect your personal experiences, you should return the favour and respect theirs. Don't try to dismiss them as "convoluted" or not "real". I am far more likely to mistype exponentiation with a single asterisk than to accidentally leave out a comma from a list display of strings. But I recognise that this is *my* problem, not the language. You want some sort of "official" recommendation not to use implicit string concatenation. What benefit will that give you, except to nag those who like and use the feature? - It won't give you a error or warning when your code accidentally uses string concatenation; you still need to be alert for the possibility, or use a linter. - It won't teach people not to use this feature; you still need to point them at the recommendation and have them read it and agree to follow it. - It won't automatically remove string concatenation from the code you use, nor prevent it from happening again, or even change your project style-guide. - It won't even add any information to the community that doesn't already exist. I have no doubt there are many blog posts and Stackoverflow discussions about this. I know I've talked about it many times on comp.lang.python. If you know how to google, you can find a post from Guido talking about getting "a mysterious argument count error" due to this. If you want something to point people at, bookmark Guido's comment. But for that matter, what will this statement say? "Don't use implicit concatenation. Even though this is not deprecated and will remain legal, you mustn't use it because you might use it in a list or function call and forget to use a comma between two string literals and get a mysterious error." Um, yeah, okay, I get lots of mysterious errors caused by carelessness. Some of them are easy to fix. Some are much harder to track down and fix than a missing comma. Do they all need official warnings in the docs? "Don't misspell None as 'none' in three dozen places. And if you do, don't spend twenty minutes fixing them one at a time, use your editor's search and replace function." (That really was not my best day.) I don't think this is any worse or more common than any other gotcha or silly error that might trip people up occasionally, and I think that singling it out in the Python docs for special warning is unnecessary. -- Steve

On Thu, Mar 15, 2018 at 01:18:06AM +1100, Steven D'Aprano <steve@pearwood.info> wrote:

On Wed, Mar 14, 2018 at 02:31:58PM +0100, Oleg Broytman wrote:

...

Once I stumbled over a bug caused by this in legacy code in production. Fixing it was quite painful!

Did you mean that *finding* the bug was painful? Fixing it should be trivially easy: add a comma.

The most painful was that the bug destroyed important information that was hard to fix later (enterprise code in production, the code is certificated and hard to replace). The second pain was to understand what was going on. When a program report "The OID in the certificate doesn't belong to the proper root OID" it's simpler to decide that the certificate is wrong, not that the list of root OIDs to check is wrong. The third pain was to find the bug. It's hard to spot a problem in the list oids = [ "1.2.6.254.14." "19.9.91.281", "1.2.6.263.12." "481.7.9.6.3.87" "1.2.7.4.214.7." "9.1.52.12", ] There were about 50 OIDs like these, and the missing comma was in the middle of the list.

-- Steve

Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

Indeed, linters are the place to go, but I think there is no "official" linter (am I wrong ?), are pyflakes and pylint independant projects ? 2018-03-14 16:03 GMT+01:00 Guido van Rossum <gvanrossum@gmail.com>:

I use the feature regularly for long error messages, and when combined with .format() or % it's not so easy to replace it with a + (it would require adding parentheses).

So I am against formally discouraging it or adding something to PEP 8.

However linters could warn about lists of comma-separated strings with a missing comma.

On Wed, Mar 14, 2018, 07:54 Oleg Broytman <phd@phdru.name> wrote:

...

On Thu, Mar 15, 2018 at 01:18:06AM +1100, Steven D'Aprano <steve@pearwood.info> wrote:

...

On Wed, Mar 14, 2018 at 02:31:58PM +0100, Oleg Broytman wrote:

...

Once I stumbled over a bug caused by this in legacy code in production. Fixing it was quite painful!

Did you mean that *finding* the bug was painful? Fixing it should be trivially easy: add a comma.

The most painful was that the bug destroyed important information that was hard to fix later (enterprise code in production, the code is certificated and hard to replace).

The second pain was to understand what was going on. When a program report "The OID in the certificate doesn't belong to the proper root OID" it's simpler to decide that the certificate is wrong, not that the list of root OIDs to check is wrong.

The third pain was to find the bug. It's hard to spot a problem in the list

oids = [ "1.2.6.254.14." "19.9.91.281", "1.2.6.263.12." "481.7.9.6.3.87" "1.2.7.4.214.7." "9.1.52.12", ]

There were about 50 OIDs like these, and the missing comma was in the middle of the list.

...

-- Steve

Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Le mer. 14 mars 2018 à 17:29, Paul Moore <p.f.moore@gmail.com> a écrit :

Ironically, many of the places I see implicit concatenation used are where people need to work around linters complaining about line lengths. I understand the benefits of projects that mandate code passing lint checks, but I foresee sequences of commits:

"Modify error X to be clearer" "Linter fix - split new message onto 2 lines to avoid long line" "Linter fix - explicitly concatenate with + because implicit concatenation is discouraged" "Linter fix - go back to the old message because the stupid linter complains about unnecessary addition of 2 constant values and I can't be bothered any more"

Haha, agreed, didn't think of that. In my humble opinion, long lines are less of a problem with raw strings (personal opinion based on html background).

Le mer. 14 mars 2018 à 18:04, Carl Meyer <carl@oddbird.net> a écrit :

On 3/14/18 8:03 AM, Guido van Rossum wrote:

...

I use the feature regularly for long error messages, and when combined with .format() or % it's not so easy to replace it with a + (it would require adding parentheses).

So I am against formally discouraging it or adding something to PEP 8.

However linters could warn about lists of comma-separated strings with a missing comma.

+1 to all of this.

It's possible to write a linter rule that will reliably catch the potential missing-comma errors, while still allowing implicit string concatenation where it's useful and readable. The rule is that implicit string concatenation should only be disallowed if there are any commas present at the same "parenthesization level". Thus, this is allowed:

raise SomeException( "This is a really long exception message " "which I've split over two lines." )

While both of these are disallowed:

fruits = [ 'orange', 'grape' 'banana', ]

some_str.replace( "This is a long string " "split over two lines.", "And this is another string." )

+1... if this is indeed implemented in a linter !

I'm with Steven D`Aprano here. Implicit string concat, in my experience, is something clear, concise and convenient. I've had a bug because of it only once or twice during my whole career, and not only that "occasional" bug would have occurred either way because of a missing comma or plus sign, but it saves the trouble of having to add more symbols which usually means less surface area for bugs. If you feel it affects you all so often, I would totally understand the addition of that feature in a linter, but not endorsed by the python community in any way, and with defaults probably set to disabled. I wouldn't want to add any superfluous symbols to my otherwise (at least in my opinion) cleaner code. All in all, -1 from me. On Thu, Mar 15, 2018 at 12:41 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Carl Meyer wrote:

...

With this rule the only missing-comma that can slip through is when you've forgotten _all_ the intervening commas in your sequence of strings. That's much less likely.

Not so unlikely when the argument list is of length 2.

-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 3/14/2018 11:03 AM, Guido van Rossum wrote:

I use the feature regularly for long error messages, and when combined with .format() or % it's not so easy to replace it with a + (it would require adding parentheses).

I like using f-strings for this now, and adding '+' stops compile-time concatenation across literal boundaries. See my response to FB's original post. -- Terry Jan Reedy

On Thu, Mar 15, 2018 at 9:06 AM, Kyle Lahnakoski <klahnakoski@mozilla.com> wrote:

I did use implicit concatenation for a long SQL statement, and a few long error messages, but the byte savings is not worth the increased bug density.

...

self.db.execute( "CREATE TABLE files (" + " bucket TEXT," + " key TEXT," + " name TEXT," + " last_modified REAL," + " size INTEGER," + " annotate TEXT, " + " CONSTRAINT pk PRIMARY KEY (bucket, name)" + ")" )

is almost identical to

...

self.db.execute( "CREATE TABLE files (" " bucket TEXT," " key TEXT," " name TEXT," " last_modified REAL," " size INTEGER," " annotate TEXT, " " CONSTRAINT pk PRIMARY KEY (bucket, name)" ")" )

It looks like I am in the market for a linter that prevents implicit string concatenation!

Even better: self.db.execute(""" CREATE TABLE files ( bucket TEXT, key TEXT, name TEXT, last_modified REAL, size INTEGER, annotate TEXT, CONSTRAINT pk PRIMARY KEY (bucket, name) ) """) ChrisA

On Wed, Mar 14, 2018 at 01:43:53PM +0100, Søren Pilgård wrote: [...]

I have also experienced beginners asking why you can do `x = "abc" "def"` but not `a = "abc"; b = "def"; x = a b` and then you have to either explain them the differences between strings and string literals or just tell them to always use `+`.

If you tell them to "always use `+`", you are teaching them to be cargo-cult coders who write code they don't understand for reasons they don't know. Life is too short to learn *everything*, I know that, and I've certainly copied lots of code I don't understand (and hoped I'd never need to debug it!). If that makes me a cargo-cult coder too, so be it. But never over something as simple as the difference between names a and b, and string literals "abc", "def". -- Steve

While I personally dislike implicit string concatenation, I've worked in projects that use it, and I, more often than I'd like, use them myself. The problem is that there is no "less typing" way of entering a correct multi-line string - due to multiline string literals preserving the indentation characters. It is a bit cumbersome to do something like from textwrap import dedent as D a = D("""\ My code-aligned, multiline text with no spaces to the left goes here """) And people then resort for implicit concatenation instead of that. I am sure this has been discussed before, and decided against, but I'd be all for taking the occasion to add a "D" prefix to multiline strings to apply de-denting to the literals. Regards, js -><- On 14 March 2018 at 10:15, Steven D'Aprano <steve@pearwood.info> wrote:

On Wed, Mar 14, 2018 at 09:18:30AM -0300, Facundo Batista wrote:

...

Hello!

What would you think about formally descouraging the following idiom?

long_string = ( "some part of the string " "with more words, actually is the same " "string that the compiler puts together")

I would hate that.

...

We should write the following, instead:

long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

Should we? I disagree.

Of course you're welcome to specify that in your own style-guide for your own code, but I won't be following that recommendation.

...

I know that "no change to Python itself" is needed, but having a formal discouragement of the idiom will help in avoiding people to fall in mistakes like:

fruits = { "apple", "orange" "banana", "melon", }

People can make all sorts of mistakes through carlessness. I wrote

{y, x*3}

the other day instead of {y: x**3}. (That's *two* errors in one simple expression. I wish I could say it was a record for me.) Should we "discourage" exponentiation and dict displays and insist on writing dict((y, x*x*x)) to avoid the risk of errors? I don't think so.

I think string concatenation falls into the same category. Sometimes even the most careful writer makes a mistake (let alone careless writers). That's life. Not every problem needs a technical "solution". Sometimes the right solution is to proof-read your code, or have code review by a fresh pair of eyes.

And tests, of course.

[...]

...

Note that there's no penalty in adding the '+' between the strings, those are resolved at compilation time.

That is an implementation feature, not a language requirement. Not all Python interpreters will do that, and they are free to put limits on how much they optimize. Here's Python 3.5:

py> import dis py> dis.dis('s = "abcdefghijklmnopqrs" + "t"') 1 0 LOAD_CONST 3 ('abcdefghijklmnopqrst') 3 STORE_NAME 0 (s) 6 LOAD_CONST 2 (None) 9 RETURN_VALUE

But now see this:

py> dis.dis('s = "abcdefghijklmnopqrs" + "tu"') 1 0 LOAD_CONST 0 ('abcdefghijklmnopqrs') 3 LOAD_CONST 1 ('tu') 6 BINARY_ADD 7 STORE_NAME 0 (s) 10 LOAD_CONST 2 (None) 13 RETURN_VALUE

And older versions of CPython didn't optimize this at all, and some day there could be a command-line switch or environment variable to turn these optimizations off.

With string concatentation having potential O(N**2) performance, if you're writing explicit string concatenations with +, they could potentially be *very* expensive at runtime.

-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 14 March 2018 at 13:40, Søren Pilgård <fiskomaten@gmail.com> wrote:

We can't remove all potential pitfalls, but I do think there is value in evaluating whether something has bigger potential to cause harm than the benefits it brings, especially if there are other ways to do the same.

Certainly. However, in this case opinions differ (I, like Steven, have no problem with implicit string concatenation, and in certain cases prefer it over using explicit addition and relying on the compiler optimising that away for me). Also, we need to consider the significant body of code that would break if this construct were prohibited. Even if we were to agree that the harm caused by implicit concatenation outweighed the benefits, that would *still* not justify making it illegal, unless the net gain could be shown to justify the cost of forcing every project that currently uses implicit concatenation to change their code, debug those changes, make new releases, etc. Paul.

2018-03-14 10:54 GMT-03:00 Paul Moore <p.f.moore@gmail.com>:

Also, we need to consider the significant body of code that would break if this construct were prohibited. Even if we were to agree that the harm caused by implicit concatenation outweighed the benefits, that would *still* not justify making it illegal, unless the net gain could be shown to justify the cost of forcing every project that currently uses implicit concatenation to change their code, debug those changes, make new releases, etc.

I never said to prohibit it, just discourage the idiom. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

On Thu, Mar 15, 2018 at 12:40 AM, Søren Pilgård <fiskomaten@gmail.com> wrote:

Of course you can always make error, even in a single letter. But I think there is a big difference between mixing up +-/* and ** where the operator is in "focus" and the implicit concatenation where there is no operator. A common problem is that you have something like foo(["a", "b", "c" ]).bar() but then you remember that there also needs to be a "d" so you just add foo(["a", "b", "c" "d" ]).bar() Causing an error, not with the "d" expression you are working on but due to what you thought was the previous expression but python turns it into one. The , is seen as a delimiter by the programmer not as part of the operation (or the lack of the ,).

You're creating a list. Put a comma at the end of every line; problem solved. Your edit would be from this: foo(["a", "b", "c", ]).bar() to this: foo(["a", "b", "c", "d", ]).bar() and there is no bug. In fact, EVERY time you lay out a list display vertically (or dict or set or any equivalent), ALWAYS put commas. That way, you can reorder the lines freely (you don't special-case the last one), you can append a line without changing the previous one (no noise in the diff), etc, etc, etc. ChrisA

On Thu, Mar 15, 2018 at 8:58 PM, Rob Cliffe via Python-ideas <python-ideas@python.org> wrote:

On 14/03/2018 17:57, Chris Angelico wrote:

On Thu, Mar 15, 2018 at 12:40 AM, Søren Pilgård <fiskomaten@gmail.com> wrote:

Of course you can always make error, even in a single letter. But I think there is a big difference between mixing up +-/* and ** where the operator is in "focus" and the implicit concatenation where there is no operator. A common problem is that you have something like foo(["a", "b", "c" ]).bar() but then you remember that there also needs to be a "d" so you just add foo(["a", "b", "c" "d" ]).bar() Causing an error, not with the "d" expression you are working on but due to what you thought was the previous expression but python turns it into one. The , is seen as a delimiter by the programmer not as part of the operation (or the lack of the ,).

You're creating a list. Put a comma at the end of every line; problem solved. Your edit would be from this:

foo(["a", "b", "c", ]).bar()

to this:

foo(["a", "b", "c", "d", ]).bar()

and there is no bug. In fact, EVERY time you lay out a list display vertically (or dict or set or any equivalent), ALWAYS put commas. That way, you can reorder the lines freely (you don't special-case the last one), you can append a line without changing the previous one (no noise in the diff), etc, etc, etc.

My thoughts exactly. I make it a personal rule to ALWAYS add a comma to every line, including the last one, in this kind of list (/dict/set etc.). Python allows it - take advantage of it! (A perhaps minor-seeming feature of the language which actually is a big benefit if you use it.) Preferably with all the commas vertically aligned to highlight the structure (I'm a great believer BTW in using vertical alignment, even if it means violating Python near-taboos such as more that one statement per line). Also I would automatically put the first string (as well as the last) on a line by itself: foo([ "short string" , "extremely looooooooooooong string" , "some other string" , ]) Then as Chris says (sorry to keep banging the drum), the lines can trivially be reordered, and adding more lines never causes a problem as long as I stick to the rule. Which I do automatically because I think my code looks prettier that way.

From a purist angle, implicit string concatenation is somewhat inelegant (where else in Python can you have two adjacent operands not separated by an operator/comma/whatever? We don't use reverse Polish notation). And I could live without it. But I have found it useful from time to time: constructing SQL queries or error messages or other long strings that I needed for some reason, where triple quotes would be more awkward (and I find line continuation backslashes ugly, *especially* in mid-string). I guess my attitude is: "If you want to read my Python code, 90%+ of the time it will be *obvious* that these strings are *meant* to be concatenated. But if it isn't, then you need to learn some Python basics first (Sorry!).".

I have never as far as I can remember had a bug caused by string concatenation. However, it is possible that I am guilty of selective memory.

+1, though, for linters to point out possible errors from possible accidental omission of a comma (if they don't already). It never hurts to have our code checked.

The linters I know use parsed ast nodes, so if it's not valid grammar, it won't parse. The _linters_ don't check for cases like f(a b) because that's not valid grammar and already caught by the parser. I think that's what you were noting when you said "if they don't already"? As for the issue in general, this is my understanding of linters: The code is loaded immediate into the ast parser. There's a no post-hoc way to know why the node is a string literal. Specifically the nodes for "ab" and "a""b" are identical Str nodes. Reversing from the Str node is impossible (unless a flag/attribute/context/whatever gets added), as the information is destroyed by then. The following might be possible: 1) The line number for a node n1 provides a way select the code between nodes n1 and the next node n2, which contains the offending string. This code needs to be retrieved from somewhere (easily kept from the beginning if not already) 2) A quick reparse of just that chunk will confirm that it contains the target node, so the code retrieval for the target node can be sanity checked (it's not exact retrieval of just the code we want, it's guaranteed to overshoot as resolution is on line numbers.) 3) ? (note below) 4) A quick check shows the tokenizer can differentiate the styles of literal. It gives 2 STRING for 'a''b' and 1 for 'ab'. This is a reliable test if the right code can be found which matches the target string. Hopefully there are others better ways but at least one exists. (better in the sense that the linters i know do not including tokenization currently) 3) The biggest problem for me (hopefully someone just knows the answer) is that some other very reliable parsing is required to know that the literal being _tested_ is the literal we _mean_ to test. I can use ast parser in 2) because im confirming a different piece of information. The problem is that the literal can exist in a very complex section, and coincidentally with 'a''b' and 'ab' in the same expression. The ast node won't tell is if we are looking for 0, 1 or 2 cases of either syntax, we only get 2 Str nodes and 1 line number. I think the parser chosen would pretty much have to be a rewrite of the ast parser due to nesting. Or someone needs to root around in the internals of the ast parser to see if the information can be extracted at the right time. I hope this helps in some way. I don't think it's impossible, but the above will introduce annoying bits into existing linters for this one issue. Of course, a working example would certainly make any case a lot easier. Given there is no technical reason, I agree there's no reason to change anything. This just "feels" ugly to me, so i can tilt at windmills all day on it, but I can see no technical reason. Always amazes me that string are so weird. They are literals and lists, but also multi white space jumping zebras. Are they "multi" "white space" "jumping zebras"? "multi" "white space jumping" "zebras"? "multi white" "space jumping" "zebras? We'll never know! -Matt

Best wishes Rob Cliffe

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Note that this has already been proposed and rejected before: https://www.python.org/dev/peps/pep-3126/ Stephan Op 14 mrt. 2018 15:01 schreef "Oleg Broytman" <phd@phdru.name>:

On Thu, Mar 15, 2018 at 12:15:52AM +1100, Steven D'Aprano < steve@pearwood.info> wrote:

...

On Wed, Mar 14, 2018 at 09:18:30AM -0300, Facundo Batista wrote:

...

We should write the following, instead:

long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

Should we? I disagree.

Of course you're welcome to specify that in your own style-guide for your own code, but I won't be following that recommendation.

...

I know that "no change to Python itself" is needed, but having a formal discouragement of the idiom will help in avoiding people to fall in mistakes like:

fruits = { "apple", "orange" "banana", "melon", }

People can make all sorts of mistakes through carlessness. I wrote

{y, x*3}

the other day instead of {y: x**3}. (That's *two* errors in one simple expression. I wish I could say it was a record for me.) Should we "discourage" exponentiation and dict displays and insist on writing dict((y, x*x*x)) to avoid the risk of errors? I don't think so.

We should fix what causes real problems, not convoluted ones. And this particular misfeature caused problems for me.

...

-- Steve

Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

2018-03-14 11:09 GMT-03:00 Stephan Houben <stephanh42@gmail.com>:

Note that this has already been proposed and rejected before:

https://www.python.org/dev/peps/pep-3126/

What was proposed and reject was the *removal* of the feature/syntax. I propose the discouragement of the idiom. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

+1 on general idea of discouraging it.

[...] mistakes like:

fruits = { "apple", "orange" "banana", "melon", }

+1

(and even making the static analysers, like pyflakes or pylint, to show that as a warning)

+1

I agree that implicit concatenation is a bad feature. I have seen it cause multiple stupid errors while I have never actually used it for anything useful. [...] Zen of Python.

+1. But I think a counter argument would be a real world example.

We should write the following, instead:

long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

Maybe the "+" at the beginnig as PEP8 said, to clearly mark it present and explicit. long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together") or long_string = ("" + "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

People can make all sorts of mistakes through carlessness. I wrote {y, x*3} the other day instead of {y: x**3}

Other comments said this isn't really the same kind of error, I agree to them.

...

Note that there's no penalty in adding the '+' between the strings, those are resolved at compilation time.

That is an implementation feature, not a language requirement. Not all Python interpreters will do that, and they are free to put limits on how much they optimize. Here's Python 3.5:

With string concatentation having potential O(N**2) performance, if you're writing explicit string concatenations with +, they could potentially be *very* expensive at runtime.

I completely agree, by emitting warning if this syntax is used, codes Could be less efficient, but that's also the job of the implementations to be fast. This is the only counter-argument I have to OP, even if I really like the proposal.

Life is too short to learn *everything*, I know that, and I've certainly copied lots of code I don't understand (and hoped I'd never need to debug it!). If that makes me a cargo-cult coder too, so be it.

I find that learning `x = "e" "f"` does indeed compile is also learning and surprising (especially for beginners).

While I personally dislike implicit string concatenation, I've worked in projects that use it, and I, more often than I'd like, use them myself.

We would like to see more real world example vs real world mistakes.

It is a bit cumbersome to do something like from textwrap import dedent as D a = D("""\ My code-aligned, multiline text with no spaces to the left goes here """)

+1. But you should add "\n" to have a = ("hello\n" "world\n" "how") Which would be maybe better written as a = '\n'.join([ "hello", "world", "how"]) (put the spaces where you like) which can easily be switched to ''.join if needed.

A common problem is that you have something like foo(["a", "b", "c" ]).bar() but then you remember that there also needs to be a "d" so you just add

+1

Yes of course the skilled programmer needs to understand the difference. But I am talking about absolute beginners where python in many regards is an excelent first language.

Agreed. 2018-03-14 15:30 GMT+01:00 Stephan Houben <stephanh42@gmail.com>:

Op 14 mrt. 2018 15:23 schreef "Facundo Batista" <facundobatista@gmail.com>:

I propose the discouragement of the idiom.

What does that mean?

Stephan

Regards,

-- . Facundo

Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Hi Facundo, On 14.03.2018 17:47, Facundo Batista wrote:

2018-03-14 11:30 GMT-03:00 Stephan Houben <stephanh42@gmail.com>:

...

Op 14 mrt. 2018 15:23 schreef "Facundo Batista" <facundobatista@gmail.com>:

I propose the discouragement of the idiom.

What does that mean?

That we say "hey, this works but please don't use it, because it tends to a error prone way of writing some code, instead do this".

I believe this falls under coding style and is not something we should actively discourage. It's an idiom that many programming languages share with Python. You may also want to look at this PEP for a longer discussion on the topic: https://www.python.org/dev/peps/pep-3126/ Even pylint rejected a request to have a rule added for this: https://github.com/PyCQA/pylint/issues/1589 FWIW: I use implicit concats a lot and find them very useful for e.g. writing long SQL queries or longer string literals at indented levels where the triple quote approach doesn't work well. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Mar 14 2018)

...

...

Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/

::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/

Facundo Batista wrote:

That we say "hey, this works but please don't use it, because it tends to a error prone way of writing some code, instead do this".

The trouble is that there is nowhere near widespread agreement that it is error prone enough to be worth such a drastic measure as deprecating the whole feature. -- Greg

+1 on general idea of discouraging it.

[...] mistakes like:

fruits = { "apple", "orange" "banana", "melon", }

+1

(and even making the static analysers, like pyflakes or pylint, to show that as a warning)

+1

I agree that implicit concatenation is a bad feature. I have seen it cause multiple stupid errors while I have never actually used it for anything useful. [...] Zen of Python.

+1. But I think a counter argument would be a real world example.

We should write the following, instead:

long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

Maybe the "+" at the beginnig as PEP8 said, to clearly mark it present and explicit. long_string = ( "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together") or long_string = ("" + "some part of the string " + "with more words, actually is the same " + "string that the compiler puts together")

People can make all sorts of mistakes through carlessness. I wrote {y, x*3} the other day instead of {y: x**3}

Other comments said this isn't really the same kind of error, I agree to them.

...

Note that there's no penalty in adding the '+' between the strings, those are resolved at compilation time.

That is an implementation feature, not a language requirement. Not all Python interpreters will do that, and they are free to put limits on how much they optimize. Here's Python 3.5:

With string concatentation having potential O(N**2) performance, if you're writing explicit string concatenations with +, they could potentially be *very* expensive at runtime.

I completely agree, by emitting warning if this syntax is used, codes Could be less efficient, but that's also the job of the implementations to be fast. This is the only counter-argument I have to OP, even if I really like the proposal.

Life is too short to learn *everything*, I know that, and I've certainly copied lots of code I don't understand (and hoped I'd never need to debug it!). If that makes me a cargo-cult coder too, so be it.

I find that learning `x = "e" "f"` does indeed compile is also learning and surprising (especially for beginners).

While I personally dislike implicit string concatenation, I've worked in projects that use it, and I, more often than I'd like, use them myself.

We would like to see more real world example vs real world mistakes.

It is a bit cumbersome to do something like from textwrap import dedent as D a = D("""\ My code-aligned, multiline text with no spaces to the left goes here """)

+1. But you should add "\n" to have a = ("hello\n" "world\n" "how") Which would be maybe better written as a = '\n'.join([ "hello", "world", "how"]) (put the spaces where you like) which can easily be switched to ''.join if needed.

A common problem is that you have something like foo(["a", "b", "c" ]).bar() but then you remember that there also needs to be a "d" so you just add

+1

Yes of course the skilled programmer needs to understand the difference. But I am talking about absolute beginners where python in many regards is an excelent first language.

Agreed.

On Wed, Mar 14, 2018 at 03:01:07PM +0100, Oleg Broytman wrote:

...

People can make all sorts of mistakes through carlessness. I wrote

...

{y, x*3}

the other day instead of {y: x**3}. (That's *two* errors in one simple expression. I wish I could say it was a record for me.) Should we "discourage" exponentiation and dict displays and insist on writing dict((y, x*x*x)) to avoid the risk of errors? I don't think so.

We should fix what causes real problems, not convoluted ones. And this particular misfeature caused problems for me.

I don't think that using a single asterisk instead of two is a "convoluted" problem. Please Oleg, if you want others to respect your personal experiences, you should return the favour and respect theirs. Don't try to dismiss them as "convoluted" or not "real". I am far more likely to mistype exponentiation with a single asterisk than to accidentally leave out a comma from a list display of strings. But I recognise that this is *my* problem, not the language. You want some sort of "official" recommendation not to use implicit string concatenation. What benefit will that give you, except to nag those who like and use the feature? - It won't give you a error or warning when your code accidentally uses string concatenation; you still need to be alert for the possibility, or use a linter. - It won't teach people not to use this feature; you still need to point them at the recommendation and have them read it and agree to follow it. - It won't automatically remove string concatenation from the code you use, nor prevent it from happening again, or even change your project style-guide. - It won't even add any information to the community that doesn't already exist. I have no doubt there are many blog posts and Stackoverflow discussions about this. I know I've talked about it many times on comp.lang.python. If you know how to google, you can find a post from Guido talking about getting "a mysterious argument count error" due to this. If you want something to point people at, bookmark Guido's comment. But for that matter, what will this statement say? "Don't use implicit concatenation. Even though this is not deprecated and will remain legal, you mustn't use it because you might use it in a list or function call and forget to use a comma between two string literals and get a mysterious error." Um, yeah, okay, I get lots of mysterious errors caused by carelessness. Some of them are easy to fix. Some are much harder to track down and fix than a missing comma. Do they all need official warnings in the docs? "Don't misspell None as 'none' in three dozen places. And if you do, don't spend twenty minutes fixing them one at a time, use your editor's search and replace function." (That really was not my best day.) I don't think this is any worse or more common than any other gotcha or silly error that might trip people up occasionally, and I think that singling it out in the Python docs for special warning is unnecessary. -- Steve

On Thu, Mar 15, 2018 at 01:18:06AM +1100, Steven D'Aprano <steve@pearwood.info> wrote:

On Wed, Mar 14, 2018 at 02:31:58PM +0100, Oleg Broytman wrote:

...

Once I stumbled over a bug caused by this in legacy code in production. Fixing it was quite painful!

Did you mean that *finding* the bug was painful? Fixing it should be trivially easy: add a comma.

The most painful was that the bug destroyed important information that was hard to fix later (enterprise code in production, the code is certificated and hard to replace). The second pain was to understand what was going on. When a program report "The OID in the certificate doesn't belong to the proper root OID" it's simpler to decide that the certificate is wrong, not that the list of root OIDs to check is wrong. The third pain was to find the bug. It's hard to spot a problem in the list oids = [ "1.2.6.254.14." "19.9.91.281", "1.2.6.263.12." "481.7.9.6.3.87" "1.2.7.4.214.7." "9.1.52.12", ] There were about 50 OIDs like these, and the missing comma was in the middle of the list.

-- Steve

Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

Indeed, linters are the place to go, but I think there is no "official" linter (am I wrong ?), are pyflakes and pylint independant projects ? 2018-03-14 16:03 GMT+01:00 Guido van Rossum <gvanrossum@gmail.com>:

I use the feature regularly for long error messages, and when combined with .format() or % it's not so easy to replace it with a + (it would require adding parentheses).

So I am against formally discouraging it or adding something to PEP 8.

However linters could warn about lists of comma-separated strings with a missing comma.

On Wed, Mar 14, 2018, 07:54 Oleg Broytman <phd@phdru.name> wrote:

...

On Thu, Mar 15, 2018 at 01:18:06AM +1100, Steven D'Aprano <steve@pearwood.info> wrote:

...

On Wed, Mar 14, 2018 at 02:31:58PM +0100, Oleg Broytman wrote:

...

Once I stumbled over a bug caused by this in legacy code in production. Fixing it was quite painful!

Did you mean that *finding* the bug was painful? Fixing it should be trivially easy: add a comma.

The most painful was that the bug destroyed important information that was hard to fix later (enterprise code in production, the code is certificated and hard to replace).

The second pain was to understand what was going on. When a program report "The OID in the certificate doesn't belong to the proper root OID" it's simpler to decide that the certificate is wrong, not that the list of root OIDs to check is wrong.

The third pain was to find the bug. It's hard to spot a problem in the list

oids = [ "1.2.6.254.14." "19.9.91.281", "1.2.6.263.12." "481.7.9.6.3.87" "1.2.7.4.214.7." "9.1.52.12", ]

There were about 50 OIDs like these, and the missing comma was in the middle of the list.

...

-- Steve

Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Le mer. 14 mars 2018 à 17:29, Paul Moore <p.f.moore@gmail.com> a écrit :

Ironically, many of the places I see implicit concatenation used are where people need to work around linters complaining about line lengths. I understand the benefits of projects that mandate code passing lint checks, but I foresee sequences of commits:

"Modify error X to be clearer" "Linter fix - split new message onto 2 lines to avoid long line" "Linter fix - explicitly concatenate with + because implicit concatenation is discouraged" "Linter fix - go back to the old message because the stupid linter complains about unnecessary addition of 2 constant values and I can't be bothered any more"

Haha, agreed, didn't think of that. In my humble opinion, long lines are less of a problem with raw strings (personal opinion based on html background).

Le mer. 14 mars 2018 à 18:04, Carl Meyer <carl@oddbird.net> a écrit :

On 3/14/18 8:03 AM, Guido van Rossum wrote:

...

I use the feature regularly for long error messages, and when combined with .format() or % it's not so easy to replace it with a + (it would require adding parentheses).

So I am against formally discouraging it or adding something to PEP 8.

However linters could warn about lists of comma-separated strings with a missing comma.

+1 to all of this.

It's possible to write a linter rule that will reliably catch the potential missing-comma errors, while still allowing implicit string concatenation where it's useful and readable. The rule is that implicit string concatenation should only be disallowed if there are any commas present at the same "parenthesization level". Thus, this is allowed:

raise SomeException( "This is a really long exception message " "which I've split over two lines." )

While both of these are disallowed:

fruits = [ 'orange', 'grape' 'banana', ]

some_str.replace( "This is a long string " "split over two lines.", "And this is another string." )

+1... if this is indeed implemented in a linter !

I'm with Steven D`Aprano here. Implicit string concat, in my experience, is something clear, concise and convenient. I've had a bug because of it only once or twice during my whole career, and not only that "occasional" bug would have occurred either way because of a missing comma or plus sign, but it saves the trouble of having to add more symbols which usually means less surface area for bugs. If you feel it affects you all so often, I would totally understand the addition of that feature in a linter, but not endorsed by the python community in any way, and with defaults probably set to disabled. I wouldn't want to add any superfluous symbols to my otherwise (at least in my opinion) cleaner code. All in all, -1 from me. On Thu, Mar 15, 2018 at 12:41 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Carl Meyer wrote:

...

With this rule the only missing-comma that can slip through is when you've forgotten _all_ the intervening commas in your sequence of strings. That's much less likely.

Not so unlikely when the argument list is of length 2.

-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 3/14/2018 11:03 AM, Guido van Rossum wrote:

I use the feature regularly for long error messages, and when combined with .format() or % it's not so easy to replace it with a + (it would require adding parentheses).

I like using f-strings for this now, and adding '+' stops compile-time concatenation across literal boundaries. See my response to FB's original post. -- Terry Jan Reedy

On Thu, Mar 15, 2018 at 9:06 AM, Kyle Lahnakoski <klahnakoski@mozilla.com> wrote:

I did use implicit concatenation for a long SQL statement, and a few long error messages, but the byte savings is not worth the increased bug density.

...

self.db.execute( "CREATE TABLE files (" + " bucket TEXT," + " key TEXT," + " name TEXT," + " last_modified REAL," + " size INTEGER," + " annotate TEXT, " + " CONSTRAINT pk PRIMARY KEY (bucket, name)" + ")" )

is almost identical to

...

self.db.execute( "CREATE TABLE files (" " bucket TEXT," " key TEXT," " name TEXT," " last_modified REAL," " size INTEGER," " annotate TEXT, " " CONSTRAINT pk PRIMARY KEY (bucket, name)" ")" )

It looks like I am in the market for a linter that prevents implicit string concatenation!

Even better: self.db.execute(""" CREATE TABLE files ( bucket TEXT, key TEXT, name TEXT, last_modified REAL, size INTEGER, annotate TEXT, CONSTRAINT pk PRIMARY KEY (bucket, name) ) """) ChrisA