Extending expressions using ellipsis

Hi, I write a lot of SQLAlchemy code that looks more or less like this: rows = ( dbsession.query(Table1) .join( Table2, Table2.y = Table1.y) .filter(Table1.x = xx) .all()) The expressions get very long and nearly always need to be spread to multiple lines. I've tried various styles and have chosen the style above as the most tasteful available. Pros of the existing syntax: - It's possible to indent clearly and consistently. - Nested indentation works out OK. - It's flexible; I can combine lines or separate them for emphasis. Cons: - Extra parentheses are required. - The indentation is not enforced by the parser, so I have unnecessary freedom that could let various mistakes slip through. - The closing parenthesis has to move every time I append to or reorder the expression, leading to diff noise in version control. (Alternatively, I could put the closing parenthesis on its own line, but that consumes precious vertical reading space.) I'd like to suggest a small change to the Python parser that would make long expressions read better: rows = dbsession.query(Table1) ... .join( Table2, Table2.y = Table1.y) .filter(Table1.x = xx) .all() The idea is to use an ellipsis at the end of a line to spread an expression over multiple indented lines, terminated by a return to an earlier indentation level. You can still indent more deeply as needed, as shown above by the join() method call. This syntax has all the pros of the existing syntax and resolves all the cons: - No extra parentheses are required. - The indentation is enforced, so my mistakes are more likely to be caught early. - Without a closing parenthesis, there is no diff noise when I append to or reorder an expression. I've thought about using a colon instead of an ellipsis, but in Python, a colon starts a list of statements; that's not my goal. Instead, I'm looking for ways to use parser-enforced indentation to avoid mistakes and help my code read better without changing any semantics. Feedback is welcome! Shane

Guido's time machine strikes again, though using a slash (\) rather than elipse:
This is from Python 2.7.10 (what I have on the machine I am currently on), though I'm fairly sure it has worked for quite a bit longer than that. Chris On Wed, Aug 31, 2016 at 2:46 PM, Shane Hathaway <shane@hathawaymix.org> wrote:

On Aug 31, 2016 7:22 PM, "Chris Kaynor" <ckaynor@zindagigames.com> wrote:
Guido's time machine strikes again,
GAH! We should've just used that for PEPs 484 and 526; instead of trying to prove type hints are useful, Guido could've just: 1. Go 50 years into the future. 2. Make note of the Python's world domination (Perl is overrated). 3. Grab random examples of code using type hints and note that no nuclear missile switches have gone off like last time\b\b\b\b\b\b\b\b\b\b has never happened. 4. Bring them back here and shove it in the rationale. Problem solved! (Unless of course, the time machine accidentally sets off one of those missile switches like last time\b\b\b\b\b\b\b\b\b\b has never happened.) the expression, leading to diff noise in version control. (Alternatively, I could put the closing parenthesis on its own line, but that consumes precious vertical reading space.)
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/

Chris Kaynor writes:
I expect Shane is aware of that. There are three issues. First, the ellipsis is more visible, and because it's syntactic rather than lexical, trailing whitespace (line comments!) isn't a problem. Second, more important, Shane wants indentation enforced by the parser, which requires a syntactic line break. I'll add a third, which Shane may have meant to imply. That is, to get '\' to work with more general expressions (specifically, embedded function calls with arguments that are long enough to themselves require physical line breaks), you need to put in an escaped newline at each such. Shane's idea is that at the *first* physical linebreak, you put in the ellipsis, and after that your expression *must* obey pythonic indentation rules until the end of that expression. N.B. "Pythonic" rather than "Python" because Python currently doesn't have indentation rules for expressions, and the analogy to suite indentation will be imperfect, I suspect. Analogy (very inaccurate): "readable" regexp syntax. As yet, I have no opinion of the proposal itself, but it's clearly more powerful than your reply suggests.

On Wed, Aug 31, 2016 at 2:46 PM, Shane Hathaway <shane@hathawaymix.org> wrote: [...]
(And no, this isn't equivalent to using '\'.) Would this be enforced in the grammar or by the lexer? Since you say you expect the indentation to be enforced, that suggests it would be done by the grammar, but then the question is how you would modify the grammar? You could take the rule that says an expression can be followed by ".NAME" and extended it to also allow "... INDENT xxxxx DEDENT" where the xxxxx is whatever's allowed at ".NAME" (i.e. ".NAME" followed by other tails like "(......)" or "[.......]". But then you could only use this new idea for chaining method calls, and not for spreading other large expressions across multiple lines. -- --Guido van Rossum (python.org/~guido)

On 08/31/2016 07:25 PM, Guido van Rossum wrote:
Exactly.
Yes, I was hoping the enhancement might be useful for more than just chaining method calls; if possible, the ellipsis should be allowed between any expression tokens. I'll study the grammar and see if my idea fits cleanly somehow. Thanks! Shane

On 1 September 2016 at 09:40, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Not sure if this is a good idea but it might also make some sense to in stead have an operator at the beginning of the line for example some languages have a chainging operator for method calls: my_object.m1() ..m2() being equivalent to my_object.m1() my_object.m2() It could also be possible to have a special swallow-the-preceding-newline operator or something of that effect. side note: the chainging operator does not work on the return value of the method thus the method no longer has to choose between returning useful info or `this` for chaining convenience.

On Thu, Sep 01, 2016 at 10:43:17AM +0300, Joonas Liik wrote:
Not sure if I would appreciate this. Most of the cases where I do chaining method calls, it's the return value that's being chained on. Of course that is logical, because that's what's currently possible. Also, would it be called 'call forking' instead of 'call chaining'...? But also, what does this mean: my_object.value.m1() ..m2() is that my_object.value.m1() my_object.value.m2() or my_object.value.m1() my_object.m2() I do think the syntax you suggest is readable, I just think the semantics is confusing and ambiguous in non-trivial examples. And the extra '.' to signify it is chaining is not really a syntactical necessity I think. What I think might be a neat idea is to do the following: if: - we have an 'unexpected indent', and - the line starts with a '.' then: - interpret the physical line as continuation of the previous line. In any current Python version this is a syntax error. In a new Python version it could be *the* obvious way to do method chaining. inactive_admins = User.objects .filter(is_staff=True) .exclude(last_login__gt=three_weeks_ago) In current versions of Python, you have to add a `\` to the end, in a possible future version of Python, you could leave out the `\`. To be fair, I think this only makes sense for when the next line starts with a `.`. In PEP8 we could add a rule about 'aligning the . with the last dot on the previous line' (or an indent of 4 if the previous line has no dot). Even though I think it would make for a good enhancement to the Python language, I can not currently estimate how much of a change to the Python parser this would need to be. Is it a three-hour straight-forward job, or a three-month bug-prone assignment? Regards, Sjoerd Job

On 1 September 2016 at 11:10, Sjoerd Job Postmus <sjoerdjob@sjoerdjob.com> wrote:
its just some syntax i've come across in some other language (can't recall which one unfortunately), the semantics being .. #you have this r = a.b()..c()..d() #is equivalent to t=a a.b() a.c() r=a.d() of yourse if you want to maintain readability you want most calls to be on separate lines so r = a.b() ..c() ..d() # the return value of the last function is the value of the entire expression with some some names that are actually descriptive this could quite nicely represent a sequence of transformations to something for example (is this why jQuery has so much method chaining?) anyway with method chaing you sometimes (often? rarely?) have an issue where you have a meaningful return value so you have to chooze if you want to return meaningful output or return self/this to enable chaining. if you need the return value then you obviously need to capture it but sometimes you just want the side effect (mutating state of `self`) and don't care about what the method returns. if you have this you probably want method chaining to be usable in expressions tho, having subsets of language usable in different contexts rly gets annoying mmighty fast.

On Thu, Sep 01, 2016 at 01:38:19PM +0300, Joonas Liik wrote:
After doing some research, turns out this is called 'method cascading' (not to be confused with 'method chaining'). It is found in Dart, which has exactly this syntax.
After reading up on this, it seems your example is a bit mistaken. It would be t = a.b() a.c() r = a.d() (for the other behaviour, you should do `a..b()..c()..d()`) It turns out that Dart also has assignment cascading: r = Address() ..street = "East Avenue" ..housenumber = 12
Seeing "transformations" makes me think of method chaining more than the method cascading as described above. In jQuery (and also Django querysets), you don't get the same object back, but a new object. So Django again: qs = User.objects.filter(is_staff=True).filter(is_active=True) is not equal to qs = User.objects qs.filter(is_staff=True) qs.filter(is_active=True) but qs = User.objects qs = qs.filter(is_staff=True) qs = qs.filter(is_active=True)
Method chaining is already possible in Python, just sometimes you have to add a bit of extra characters to make it prettily span multiple lines. Inside an expression it already works. Method cascading is not yet possible in Python. To be honest, I think it is off-topic in this thread, but also that it is something that might be a good idea. It is however already covered in the following thread... https://mail.python.org/pipermail//python-ideas/2013-November/024124.html Not to say that the opinions might not have changed since then...

On Wed, Aug 31, 2016 at 03:46:13PM -0600, Shane Hathaway wrote:
Cons:
- Extra parentheses are required.
You have to have extra *something* no matter how you do it. An extra semi-colon at the end of the statement, in semi-colon language. An extra backslash at the end of the line. An extra pair of parens. I don't see this as a meaningful negative.
- The indentation is not enforced by the parser, so I have unnecessary freedom that could let various mistakes slip through.
I don't see how. Its precisely because the indentation is not enforced that you *can't* get errors related to the indentation. Of course you can still get other errors, like dropping a comma between function arguments, or failing to close a pair of brackets, but enforcing indentation doesn't protect you against those. Can you demonstrate an error that can be prevented by enforcing indentation inside an expression? In fact, can you explain what indentation rules you want to enforce? Its not clear to me exactly what you want.
I don't get this argument either. There are two situations: you have the closing paren just after the final token, or you have it on a line of its own. If its on a line of its own, you don't need to change it if you modify the expression (as you state yourself). But if its on the same line as the final token, and you modify that line by appending or re-ordering, there's going to be a diff regardless: result = ( blah blah blah blah blah blah blah + x - y or spam.eggs()) becomes: result = ( blah blah blah blah blah x - y blah blah or spam.eggs() or default) The presence or absence of that outermost pair of brackets doesn't affect the diff in any way. You either change the line, and get a diff, or you don't, and don't. At least that's how I see it. Can you show the sort of diff noise that you're talking about?
Three dots is already syntax for Ellipsis, so this is going to be ambiguous. There will be expressions where ... is valid, and the parser cannot tell if it is intended as a line continuation or not. result = spam + ... (eggs or cheese)(arg) Is that a line continuation, or an accidentally indented line which ought to raise a SyntaxError?
This syntax has all the pros of the existing syntax and resolves all the cons:
I'm not convinced that any of your "cons" actually are cons. See my questions above.
- No extra parentheses are required.
So instead of two extra characters, you need three extra characters. I'm not seeing how this is a benefit.
As above, I'm not seeing how this is supposed to work. -- Steve

On 09/01/2016 09:35 AM, Steven D'Aprano wrote:
I agree that multi-line expressions should always be explicit. Neither of us would want implicit multi-line expressions. I admire the style Python uses to format classes, functions, loops, conditions, and so on. There's always a block start marker, an indented block, and an implicit block end (implied by a reduction in the indentation level). That's a very tidy pattern and I try to use it as much as possible. I currently can't use that pattern for multi-line expressions. Because my SQLAlchemy code has a lot of multi-line expressions, some of my code doesn't look very much like Python. I'd like to keep my style more consistent so it's easier to read.
Sometimes I fix unbalanced parentheses incorrectly. Here's something I might type. There should be another closing parenthesis in the middle: def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value})) Either Python or flake8 will tell me there's some kind of syntax error, so I might fix it by adding a closing parenthesis at the end: def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value}))) This will fix the syntax error but fail at runtime. With my proposed syntax, I would probably never create the error in the first place because I would only need to scan for balanced parentheses on each line, not over multiple lines: def update(names, value): dbsession.query(Table1) ... .filter(Table1.name.in_(names)) .update({'value': value}) dbsession.query(Table2) ... .filter(Table2.name.in_(names)) .update({'value': value})
In fact, can you explain what indentation rules you want to enforce? Its not clear to me exactly what you want.
The indentation must be consistent, and the expression ends when the indentation level drops to the same level as the first line or an earlier line.
Let's say I write this code: rows = ( dbsession.query(Table1) .filter(Table1.name.in_(names))) That code will work, but it will be slow (because it will fetch rows one at a time). I later realize I need to call the all() method to make it fast: rows = ( dbsession.query(Table1) .filter(Table1.name.in_(names)) .all()) One line of code had to change and another had to be added. With my proposed syntax, let's say I write the same code again: rows = dbsession.query(Table1) ... .filter(Table1.name.in_(names)) To make it fast, I add a line: rows = dbsession.query(Table1) ... .filter(Table1.name.in_(names)) .all() In this case, I only added one line of code and didn't change any existing lines, so the signal to noise ratio is better.
Ah, that's very interesting. I had forgotten until now that "..." in Python 3 is a literal value. For anyone else reading along, this now works in Python 3:
I wasn't aware of that complication. Maybe I should limit the idea to method chaining as Guido hinted. Shane

On 1 September 2016 at 19:44, Shane Hathaway <shane@hathawaymix.org> wrote:
Thanks. That's a nice example of how the proposal might help. But you could of course have written your original code as def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value})) That's not going to completely alleviate the problem, but it does make the intent clearer. And it's possible that you could propose a style rule that a dedent in a bracketed expression is not allowed - that might well be something that flake8 could add, and then you'd get a much clearer error message (but only if you ran flake8 - if you just saw a syntax error from Python, you'd probably be just as likely to "fix" it as you said above, without even trying to run flake8). Also, of course, most text editors would highlight matching parentheses - which makes it much easier to spot the "correct" place for the missing parenthesis. One other thing, I'm not at all keen on using "..." for the syntax - it's almost completely invisible when I read this in gmail, and as others have pointed out, it already has a meaning, as Ellipsis. But I don't have a better suggestion to offer, I'm afraid. Overall, though, I'm cautiously in favour of the proposal. I'm not convinced the benefit is huge, and I'm a little concerned that it may be supporting a style of code that isn't ideal (the method chaining from SQLAlchemy you use to illustrate the examples is common enough in languages like JavaScript, but outside of SQLAlchemy I haven't seen it used much in Python). Also, it's very definitely "yet another way to write expressions across multiple lines". But the indented expression format is pretty readable for cases when you *do* have a long expression and no convenient way to bracket it. Paul

On 09/01/2016 02:04 PM, Paul Moore wrote:
Good points. That style does look clearer than my example.
Now that I remember that "..." is a literal value in Python 3 (surprise!), I've been thinking about alternatives. I wonder if a double backslash would work. Double backslash at the end of a line is currently a syntax error. Let's see how it looks: def update(names, value): dbsession.query(Table1) \\ .filter(Table1.name.in_(names)) .update({'value': value}) dbsession.query(Table2) \\ .filter(Table2.name.in_(names)) .update({'value': value}) That looks OK to me right now. Double backslash seems to convey the idea that it's not only a line continuation, but a line continuation with special properties. I would never write it this way, BTW: def update(names, value): dbsession.query(Table1) \ .filter(Table1.name.in_(names)) \ .update({'value': value}) dbsession.query(Table2) \ .filter(Table2.name.in_(names)) \ .update({'value': value}) I never seem to be able to sprinkle those single backslashes in the right places consistently. I forget one, or I leave one in after rearranging lines, leading to mayhem.
Well, you could say the same about triple quoted strings. Sure there are other ways to write strings longer than a single line, but triple quoted strings are clearly worth the extra parser complexity. Shane

On 2016-08-31 14:46, Shane Hathaway wrote:
I sometimes write similar code with a sequence of pandas operations. I think you can get a lot of mileage out of accepting the loss in precious vertical space and just putting the closing parenthesis on its own line, indented to the same level as the beginning of the line where the corresponding open parenthesis was. Your example then becomes: rows = ( dbsession.query(Table1) .join( Table2, Table2.y = Table1.y ) .filter(Table1.x = xx) .all() ) This is essentially the way I write such code. This gives you most of the important advantages you seek. You do need one extra set of parentheses around the whole thing, but you never have to move or alter those parentheses. You can freely add new lines (i.e., add the ".all()" line if it wasn't there, as you suggested in a later post) without interfering with existing lines. Moreover, some of the reasons you cite in your later post for wanting indentation enforcement become less important if you do things this way, because you are less likely to add a parenthesis in the wrong place when your parentheses are clearly set off like this. (It's true, though, that it might help even more if Python would enforce the requirement that, when a line ends with an open parenthesis, its counterpart must appear at the same indentation level.) In my experience, the tradeoff between compactness and clarity is rarely a major issue, because most expressions are either simple enough that clarity is not at risk, or complex enough that compactness is not a real benefit. If your expression is two or three lines long with no long nested parentheses, it will probably be clear enough even if you fudge a bit on the formatting. If your expression is ten lines long, having to add an extra line or two (or even three) for closing parentheses is not such a big deal. Basically, adding extra lines for parentheses is only a pain if the expression was previously "short" and now becomes "long", but not many expressions are very close to that boundary. All that said, I do think it is worth considering some syntax to make this sort of thing easier. I'm not sure the Ellipsis approach is the right one, but I'm interested to see where this thread goes. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

On 09/01/2016 01:24 PM, Brendan Barnwell wrote:
That's a good point. I've been waffling about where to put the closing parenthesis. If flake8 had an option to force the closing parenthesis to be on its own line and at the same indentation level as the line with the opening parenthesis, I would certainly use it.
You may be right, but I'm not sure your experience matches mine. I seem to recall changing single-line expressions into multi-line expressions fairly often. I'll pay more attention in the future.
Thanks for the well considered thoughts. Shane

Guido's time machine strikes again, though using a slash (\) rather than elipse:
This is from Python 2.7.10 (what I have on the machine I am currently on), though I'm fairly sure it has worked for quite a bit longer than that. Chris On Wed, Aug 31, 2016 at 2:46 PM, Shane Hathaway <shane@hathawaymix.org> wrote:

On Aug 31, 2016 7:22 PM, "Chris Kaynor" <ckaynor@zindagigames.com> wrote:
Guido's time machine strikes again,
GAH! We should've just used that for PEPs 484 and 526; instead of trying to prove type hints are useful, Guido could've just: 1. Go 50 years into the future. 2. Make note of the Python's world domination (Perl is overrated). 3. Grab random examples of code using type hints and note that no nuclear missile switches have gone off like last time\b\b\b\b\b\b\b\b\b\b has never happened. 4. Bring them back here and shove it in the rationale. Problem solved! (Unless of course, the time machine accidentally sets off one of those missile switches like last time\b\b\b\b\b\b\b\b\b\b has never happened.) the expression, leading to diff noise in version control. (Alternatively, I could put the closing parenthesis on its own line, but that consumes precious vertical reading space.)
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/

Chris Kaynor writes:
I expect Shane is aware of that. There are three issues. First, the ellipsis is more visible, and because it's syntactic rather than lexical, trailing whitespace (line comments!) isn't a problem. Second, more important, Shane wants indentation enforced by the parser, which requires a syntactic line break. I'll add a third, which Shane may have meant to imply. That is, to get '\' to work with more general expressions (specifically, embedded function calls with arguments that are long enough to themselves require physical line breaks), you need to put in an escaped newline at each such. Shane's idea is that at the *first* physical linebreak, you put in the ellipsis, and after that your expression *must* obey pythonic indentation rules until the end of that expression. N.B. "Pythonic" rather than "Python" because Python currently doesn't have indentation rules for expressions, and the analogy to suite indentation will be imperfect, I suspect. Analogy (very inaccurate): "readable" regexp syntax. As yet, I have no opinion of the proposal itself, but it's clearly more powerful than your reply suggests.

On Wed, Aug 31, 2016 at 2:46 PM, Shane Hathaway <shane@hathawaymix.org> wrote: [...]
(And no, this isn't equivalent to using '\'.) Would this be enforced in the grammar or by the lexer? Since you say you expect the indentation to be enforced, that suggests it would be done by the grammar, but then the question is how you would modify the grammar? You could take the rule that says an expression can be followed by ".NAME" and extended it to also allow "... INDENT xxxxx DEDENT" where the xxxxx is whatever's allowed at ".NAME" (i.e. ".NAME" followed by other tails like "(......)" or "[.......]". But then you could only use this new idea for chaining method calls, and not for spreading other large expressions across multiple lines. -- --Guido van Rossum (python.org/~guido)

On 08/31/2016 07:25 PM, Guido van Rossum wrote:
Exactly.
Yes, I was hoping the enhancement might be useful for more than just chaining method calls; if possible, the ellipsis should be allowed between any expression tokens. I'll study the grammar and see if my idea fits cleanly somehow. Thanks! Shane

On 1 September 2016 at 09:40, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Not sure if this is a good idea but it might also make some sense to in stead have an operator at the beginning of the line for example some languages have a chainging operator for method calls: my_object.m1() ..m2() being equivalent to my_object.m1() my_object.m2() It could also be possible to have a special swallow-the-preceding-newline operator or something of that effect. side note: the chainging operator does not work on the return value of the method thus the method no longer has to choose between returning useful info or `this` for chaining convenience.

On Thu, Sep 01, 2016 at 10:43:17AM +0300, Joonas Liik wrote:
Not sure if I would appreciate this. Most of the cases where I do chaining method calls, it's the return value that's being chained on. Of course that is logical, because that's what's currently possible. Also, would it be called 'call forking' instead of 'call chaining'...? But also, what does this mean: my_object.value.m1() ..m2() is that my_object.value.m1() my_object.value.m2() or my_object.value.m1() my_object.m2() I do think the syntax you suggest is readable, I just think the semantics is confusing and ambiguous in non-trivial examples. And the extra '.' to signify it is chaining is not really a syntactical necessity I think. What I think might be a neat idea is to do the following: if: - we have an 'unexpected indent', and - the line starts with a '.' then: - interpret the physical line as continuation of the previous line. In any current Python version this is a syntax error. In a new Python version it could be *the* obvious way to do method chaining. inactive_admins = User.objects .filter(is_staff=True) .exclude(last_login__gt=three_weeks_ago) In current versions of Python, you have to add a `\` to the end, in a possible future version of Python, you could leave out the `\`. To be fair, I think this only makes sense for when the next line starts with a `.`. In PEP8 we could add a rule about 'aligning the . with the last dot on the previous line' (or an indent of 4 if the previous line has no dot). Even though I think it would make for a good enhancement to the Python language, I can not currently estimate how much of a change to the Python parser this would need to be. Is it a three-hour straight-forward job, or a three-month bug-prone assignment? Regards, Sjoerd Job

On 1 September 2016 at 11:10, Sjoerd Job Postmus <sjoerdjob@sjoerdjob.com> wrote:
its just some syntax i've come across in some other language (can't recall which one unfortunately), the semantics being .. #you have this r = a.b()..c()..d() #is equivalent to t=a a.b() a.c() r=a.d() of yourse if you want to maintain readability you want most calls to be on separate lines so r = a.b() ..c() ..d() # the return value of the last function is the value of the entire expression with some some names that are actually descriptive this could quite nicely represent a sequence of transformations to something for example (is this why jQuery has so much method chaining?) anyway with method chaing you sometimes (often? rarely?) have an issue where you have a meaningful return value so you have to chooze if you want to return meaningful output or return self/this to enable chaining. if you need the return value then you obviously need to capture it but sometimes you just want the side effect (mutating state of `self`) and don't care about what the method returns. if you have this you probably want method chaining to be usable in expressions tho, having subsets of language usable in different contexts rly gets annoying mmighty fast.

On Thu, Sep 01, 2016 at 01:38:19PM +0300, Joonas Liik wrote:
After doing some research, turns out this is called 'method cascading' (not to be confused with 'method chaining'). It is found in Dart, which has exactly this syntax.
After reading up on this, it seems your example is a bit mistaken. It would be t = a.b() a.c() r = a.d() (for the other behaviour, you should do `a..b()..c()..d()`) It turns out that Dart also has assignment cascading: r = Address() ..street = "East Avenue" ..housenumber = 12
Seeing "transformations" makes me think of method chaining more than the method cascading as described above. In jQuery (and also Django querysets), you don't get the same object back, but a new object. So Django again: qs = User.objects.filter(is_staff=True).filter(is_active=True) is not equal to qs = User.objects qs.filter(is_staff=True) qs.filter(is_active=True) but qs = User.objects qs = qs.filter(is_staff=True) qs = qs.filter(is_active=True)
Method chaining is already possible in Python, just sometimes you have to add a bit of extra characters to make it prettily span multiple lines. Inside an expression it already works. Method cascading is not yet possible in Python. To be honest, I think it is off-topic in this thread, but also that it is something that might be a good idea. It is however already covered in the following thread... https://mail.python.org/pipermail//python-ideas/2013-November/024124.html Not to say that the opinions might not have changed since then...

On Wed, Aug 31, 2016 at 03:46:13PM -0600, Shane Hathaway wrote:
Cons:
- Extra parentheses are required.
You have to have extra *something* no matter how you do it. An extra semi-colon at the end of the statement, in semi-colon language. An extra backslash at the end of the line. An extra pair of parens. I don't see this as a meaningful negative.
- The indentation is not enforced by the parser, so I have unnecessary freedom that could let various mistakes slip through.
I don't see how. Its precisely because the indentation is not enforced that you *can't* get errors related to the indentation. Of course you can still get other errors, like dropping a comma between function arguments, or failing to close a pair of brackets, but enforcing indentation doesn't protect you against those. Can you demonstrate an error that can be prevented by enforcing indentation inside an expression? In fact, can you explain what indentation rules you want to enforce? Its not clear to me exactly what you want.
I don't get this argument either. There are two situations: you have the closing paren just after the final token, or you have it on a line of its own. If its on a line of its own, you don't need to change it if you modify the expression (as you state yourself). But if its on the same line as the final token, and you modify that line by appending or re-ordering, there's going to be a diff regardless: result = ( blah blah blah blah blah blah blah + x - y or spam.eggs()) becomes: result = ( blah blah blah blah blah x - y blah blah or spam.eggs() or default) The presence or absence of that outermost pair of brackets doesn't affect the diff in any way. You either change the line, and get a diff, or you don't, and don't. At least that's how I see it. Can you show the sort of diff noise that you're talking about?
Three dots is already syntax for Ellipsis, so this is going to be ambiguous. There will be expressions where ... is valid, and the parser cannot tell if it is intended as a line continuation or not. result = spam + ... (eggs or cheese)(arg) Is that a line continuation, or an accidentally indented line which ought to raise a SyntaxError?
This syntax has all the pros of the existing syntax and resolves all the cons:
I'm not convinced that any of your "cons" actually are cons. See my questions above.
- No extra parentheses are required.
So instead of two extra characters, you need three extra characters. I'm not seeing how this is a benefit.
As above, I'm not seeing how this is supposed to work. -- Steve

On 09/01/2016 09:35 AM, Steven D'Aprano wrote:
I agree that multi-line expressions should always be explicit. Neither of us would want implicit multi-line expressions. I admire the style Python uses to format classes, functions, loops, conditions, and so on. There's always a block start marker, an indented block, and an implicit block end (implied by a reduction in the indentation level). That's a very tidy pattern and I try to use it as much as possible. I currently can't use that pattern for multi-line expressions. Because my SQLAlchemy code has a lot of multi-line expressions, some of my code doesn't look very much like Python. I'd like to keep my style more consistent so it's easier to read.
Sometimes I fix unbalanced parentheses incorrectly. Here's something I might type. There should be another closing parenthesis in the middle: def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value})) Either Python or flake8 will tell me there's some kind of syntax error, so I might fix it by adding a closing parenthesis at the end: def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value}))) This will fix the syntax error but fail at runtime. With my proposed syntax, I would probably never create the error in the first place because I would only need to scan for balanced parentheses on each line, not over multiple lines: def update(names, value): dbsession.query(Table1) ... .filter(Table1.name.in_(names)) .update({'value': value}) dbsession.query(Table2) ... .filter(Table2.name.in_(names)) .update({'value': value})
In fact, can you explain what indentation rules you want to enforce? Its not clear to me exactly what you want.
The indentation must be consistent, and the expression ends when the indentation level drops to the same level as the first line or an earlier line.
Let's say I write this code: rows = ( dbsession.query(Table1) .filter(Table1.name.in_(names))) That code will work, but it will be slow (because it will fetch rows one at a time). I later realize I need to call the all() method to make it fast: rows = ( dbsession.query(Table1) .filter(Table1.name.in_(names)) .all()) One line of code had to change and another had to be added. With my proposed syntax, let's say I write the same code again: rows = dbsession.query(Table1) ... .filter(Table1.name.in_(names)) To make it fast, I add a line: rows = dbsession.query(Table1) ... .filter(Table1.name.in_(names)) .all() In this case, I only added one line of code and didn't change any existing lines, so the signal to noise ratio is better.
Ah, that's very interesting. I had forgotten until now that "..." in Python 3 is a literal value. For anyone else reading along, this now works in Python 3:
I wasn't aware of that complication. Maybe I should limit the idea to method chaining as Guido hinted. Shane

On 1 September 2016 at 19:44, Shane Hathaway <shane@hathawaymix.org> wrote:
Thanks. That's a nice example of how the proposal might help. But you could of course have written your original code as def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value})) That's not going to completely alleviate the problem, but it does make the intent clearer. And it's possible that you could propose a style rule that a dedent in a bracketed expression is not allowed - that might well be something that flake8 could add, and then you'd get a much clearer error message (but only if you ran flake8 - if you just saw a syntax error from Python, you'd probably be just as likely to "fix" it as you said above, without even trying to run flake8). Also, of course, most text editors would highlight matching parentheses - which makes it much easier to spot the "correct" place for the missing parenthesis. One other thing, I'm not at all keen on using "..." for the syntax - it's almost completely invisible when I read this in gmail, and as others have pointed out, it already has a meaning, as Ellipsis. But I don't have a better suggestion to offer, I'm afraid. Overall, though, I'm cautiously in favour of the proposal. I'm not convinced the benefit is huge, and I'm a little concerned that it may be supporting a style of code that isn't ideal (the method chaining from SQLAlchemy you use to illustrate the examples is common enough in languages like JavaScript, but outside of SQLAlchemy I haven't seen it used much in Python). Also, it's very definitely "yet another way to write expressions across multiple lines". But the indented expression format is pretty readable for cases when you *do* have a long expression and no convenient way to bracket it. Paul

On 09/01/2016 02:04 PM, Paul Moore wrote:
Good points. That style does look clearer than my example.
Now that I remember that "..." is a literal value in Python 3 (surprise!), I've been thinking about alternatives. I wonder if a double backslash would work. Double backslash at the end of a line is currently a syntax error. Let's see how it looks: def update(names, value): dbsession.query(Table1) \\ .filter(Table1.name.in_(names)) .update({'value': value}) dbsession.query(Table2) \\ .filter(Table2.name.in_(names)) .update({'value': value}) That looks OK to me right now. Double backslash seems to convey the idea that it's not only a line continuation, but a line continuation with special properties. I would never write it this way, BTW: def update(names, value): dbsession.query(Table1) \ .filter(Table1.name.in_(names)) \ .update({'value': value}) dbsession.query(Table2) \ .filter(Table2.name.in_(names)) \ .update({'value': value}) I never seem to be able to sprinkle those single backslashes in the right places consistently. I forget one, or I leave one in after rearranging lines, leading to mayhem.
Well, you could say the same about triple quoted strings. Sure there are other ways to write strings longer than a single line, but triple quoted strings are clearly worth the extra parser complexity. Shane

On 2016-08-31 14:46, Shane Hathaway wrote:
I sometimes write similar code with a sequence of pandas operations. I think you can get a lot of mileage out of accepting the loss in precious vertical space and just putting the closing parenthesis on its own line, indented to the same level as the beginning of the line where the corresponding open parenthesis was. Your example then becomes: rows = ( dbsession.query(Table1) .join( Table2, Table2.y = Table1.y ) .filter(Table1.x = xx) .all() ) This is essentially the way I write such code. This gives you most of the important advantages you seek. You do need one extra set of parentheses around the whole thing, but you never have to move or alter those parentheses. You can freely add new lines (i.e., add the ".all()" line if it wasn't there, as you suggested in a later post) without interfering with existing lines. Moreover, some of the reasons you cite in your later post for wanting indentation enforcement become less important if you do things this way, because you are less likely to add a parenthesis in the wrong place when your parentheses are clearly set off like this. (It's true, though, that it might help even more if Python would enforce the requirement that, when a line ends with an open parenthesis, its counterpart must appear at the same indentation level.) In my experience, the tradeoff between compactness and clarity is rarely a major issue, because most expressions are either simple enough that clarity is not at risk, or complex enough that compactness is not a real benefit. If your expression is two or three lines long with no long nested parentheses, it will probably be clear enough even if you fudge a bit on the formatting. If your expression is ten lines long, having to add an extra line or two (or even three) for closing parentheses is not such a big deal. Basically, adding extra lines for parentheses is only a pain if the expression was previously "short" and now becomes "long", but not many expressions are very close to that boundary. All that said, I do think it is worth considering some syntax to make this sort of thing easier. I'm not sure the Ellipsis approach is the right one, but I'm interested to see where this thread goes. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

On 09/01/2016 01:24 PM, Brendan Barnwell wrote:
That's a good point. I've been waffling about where to put the closing parenthesis. If flake8 had an option to force the closing parenthesis to be on its own line and at the same indentation level as the line with the opening parenthesis, I would certainly use it.
You may be right, but I'm not sure your experience matches mine. I seem to recall changing single-line expressions into multi-line expressions fairly often. I'll pay more attention in the future.
Thanks for the well considered thoughts. Shane
participants (12)
-
Brendan Barnwell
-
Chris Kaynor
-
Greg Ewing
-
Guido van Rossum
-
Joonas Liik
-
Paul Moore
-
Ryan Gonzalez
-
Ryan Hiebert
-
Shane Hathaway
-
Sjoerd Job Postmus
-
Stephen J. Turnbull
-
Steven D'Aprano