On Aug 31, 2016 7:22 PM, "Chris Kaynor" wrote:

Guido's time machine strikes again,

though using a slash (\) rather than elipse:

...

...

...

'.'\ ... .join( ... ( ... '1', ... '2', ... ) ... ) '1.2'

This is from Python 2.7.10 (what I have on the machine I am currently on), though I'm fairly sure it has worked for quite a bit longer than that.

Chris

On Wed, Aug 31, 2016 at 2:46 PM, Shane Hathaway wrote:

...

Hi,

I write a lot of SQLAlchemy code that looks more or less like this:

rows = ( dbsession.query(Table1) .join( Table2, Table2.y = Table1.y) .filter(Table1.x = xx) .all())

The expressions get very long and nearly always need to be spread to

multiple lines. I've tried various styles and have chosen the style above as the most tasteful available.

...

Pros of the existing syntax:

- It's possible to indent clearly and consistently. - Nested indentation works out OK. - It's flexible; I can combine lines or separate them for emphasis.

Cons:

- Extra parentheses are required. - The indentation is not enforced by the parser, so I have unnecessary

freedom that could let various mistakes slip through.

...

- The closing parenthesis has to move every time I append to or reorder

GAH! We should've just used that for PEPs 484 and 526; instead of trying to prove type hints are useful, Guido could've just: 1. Go 50 years into the future. 2. Make note of the Python's world domination (Perl is overrated). 3. Grab random examples of code using type hints and note that no nuclear missile switches have gone off like last time\b\b\b\b\b\b\b\b\b\b has never happened. 4. Bring them back here and shove it in the rationale. Problem solved! (Unless of course, the time machine accidentally sets off one of those missile switches like last time\b\b\b\b\b\b\b\b\b\b has never happened.) the expression, leading to diff noise in version control. (Alternatively, I could put the closing parenthesis on its own line, but that consumes precious vertical reading space.)

...

I'd like to suggest a small change to the Python parser that would make

long expressions read better:

...

rows = dbsession.query(Table1) ... .join( Table2, Table2.y = Table1.y) .filter(Table1.x = xx) .all()

The idea is to use an ellipsis at the end of a line to spread an

expression over multiple indented lines, terminated by a return to an earlier indentation level. You can still indent more deeply as needed, as shown above by the join() method call.

...

This syntax has all the pros of the existing syntax and resolves all the

cons:

...

- No extra parentheses are required. - The indentation is enforced, so my mistakes are more likely to be

caught early.

...

- Without a closing parenthesis, there is no diff noise when I append to or reorder an expression.

I've thought about using a colon instead of an ellipsis, but in Python, a colon starts a list of statements; that's not my goal. Instead, I'm looking for ways to use parser-enforced indentation to avoid mistakes and help my code read better without changing any semantics.

Feedback is welcome!

Shane _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/

Chris Kaynor writes:

Guido's time machine strikes again, though using a slash (\) rather than elipse:

...

...

...

'.'\ ... .join( ... ( ... '1', ... '2', ... ) ... ) '1.2'

This is from Python 2.7.10 (what I have on the machine I am currently on), though I'm fairly sure it has worked for quite a bit longer than that.

I expect Shane is aware of that. There are three issues. First, the ellipsis is more visible, and because it's syntactic rather than lexical, trailing whitespace (line comments!) isn't a problem. Second, more important, Shane wants indentation enforced by the parser, which requires a syntactic line break. I'll add a third, which Shane may have meant to imply. That is, to get '\' to work with more general expressions (specifically, embedded function calls with arguments that are long enough to themselves require physical line breaks), you need to put in an escaped newline at each such. Shane's idea is that at the *first* physical linebreak, you put in the ellipsis, and after that your expression *must* obey pythonic indentation rules until the end of that expression. N.B. "Pythonic" rather than "Python" because Python currently doesn't have indentation rules for expressions, and the analogy to suite indentation will be imperfect, I suspect. Analogy (very inaccurate): "readable" regexp syntax. As yet, I have no opinion of the proposal itself, but it's clearly more powerful than your reply suggests.

Guido van Rossum wrote:

Would this be enforced in the grammar or by the lexer? Since you say you expect the indentation to be enforced, that suggests it would be done by the grammar,

I think it could be done by having the lexer enter a mode where it swallows a newline that is followed by an indentation to a level greater than the starting level of the construct. Then no change would be needed to the grammar. -- Greg

On Thu, Sep 01, 2016 at 10:43:17AM +0300, Joonas Liik wrote:

Not sure if this is a good idea but it might also make some sense to in stead have an operator at the beginning of the line

for example some languages have a chainging operator for method calls:

my_object.m1() ..m2() being equivalent to my_object.m1() my_object.m2()

It could also be possible to have a special swallow-the-preceding-newline operator or something of that effect.

side note: the chainging operator does not work on the return value of the method thus the method no longer has to choose between returning useful info or `this` for chaining convenience.

Not sure if I would appreciate this. Most of the cases where I do chaining method calls, it's the return value that's being chained on. Of course that is logical, because that's what's currently possible. Also, would it be called 'call forking' instead of 'call chaining'...? But also, what does this mean: my_object.value.m1() ..m2() is that my_object.value.m1() my_object.value.m2() or my_object.value.m1() my_object.m2() I do think the syntax you suggest is readable, I just think the semantics is confusing and ambiguous in non-trivial examples. And the extra '.' to signify it is chaining is not really a syntactical necessity I think. What I think might be a neat idea is to do the following: if: - we have an 'unexpected indent', and - the line starts with a '.' then: - interpret the physical line as continuation of the previous line. In any current Python version this is a syntax error. In a new Python version it could be *the* obvious way to do method chaining. inactive_admins = User.objects .filter(is_staff=True) .exclude(last_login__gt=three_weeks_ago) In current versions of Python, you have to add a `\` to the end, in a possible future version of Python, you could leave out the `\`. To be fair, I think this only makes sense for when the next line starts with a `.`. In PEP8 we could add a rule about 'aligning the . with the last dot on the previous line' (or an indent of 4 if the previous line has no dot). Even though I think it would make for a good enhancement to the Python language, I can not currently estimate how much of a change to the Python parser this would need to be. Is it a three-hour straight-forward job, or a three-month bug-prone assignment? Regards, Sjoerd Job

On Thu, Sep 01, 2016 at 01:38:19PM +0300, Joonas Liik wrote:

On 1 September 2016 at 11:10, Sjoerd Job Postmus wrote:

...

On Thu, Sep 01, 2016 at 10:43:17AM +0300, Joonas Liik wrote:

...

Not sure if this is a good idea but it might also make some sense to in stead have an operator at the beginning of the line

for example some languages have a chainging operator for method calls:

my_object.m1() ..m2() being equivalent to my_object.m1() my_object.m2()

It could also be possible to have a special swallow-the-preceding-newline operator or something of that effect.

side note: the chainging operator does not work on the return value of the method thus the method no longer has to choose between returning useful info or `this` for chaining convenience.

Not sure if I would appreciate this. Most of the cases where I do chaining method calls, it's the return value that's being chained on. Of course that is logical, because that's what's currently possible. Also, would it be called 'call forking' instead of 'call chaining'...?

But also, what does this mean:

my_object.value.m1() ..m2()

is that

my_object.value.m1() my_object.value.m2()

or

my_object.value.m1() my_object.m2()

I do think the syntax you suggest is readable, I just think the semantics is confusing and ambiguous in non-trivial examples. And the extra '.' to signify it is chaining is not really a syntactical necessity I think.

What I think might be a neat idea is to do the following:

if: - we have an 'unexpected indent', and - the line starts with a '.'

then: - interpret the physical line as continuation of the previous line.

In any current Python version this is a syntax error. In a new Python version it could be *the* obvious way to do method chaining.

inactive_admins = User.objects .filter(is_staff=True) .exclude(last_login__gt=three_weeks_ago)

In current versions of Python, you have to add a `\` to the end, in a possible future version of Python, you could leave out the `\`.

To be fair, I think this only makes sense for when the next line starts with a `.`.

In PEP8 we could add a rule about 'aligning the . with the last dot on the previous line' (or an indent of 4 if the previous line has no dot).

Even though I think it would make for a good enhancement to the Python language, I can not currently estimate how much of a change to the Python parser this would need to be. Is it a three-hour straight-forward job, or a three-month bug-prone assignment?

Regards, Sjoerd Job

its just some syntax i've come across in some other language (can't recall which one unfortunately), the semantics being ..

After doing some research, turns out this is called 'method cascading' (not to be confused with 'method chaining'). It is found in Dart, which has exactly this syntax.

#you have this r = a.b()..c()..d()

#is equivalent to t=a a.b() a.c() r=a.d()

After reading up on this, it seems your example is a bit mistaken. It would be t = a.b() a.c() r = a.d() (for the other behaviour, you should do `a..b()..c()..d()`) It turns out that Dart also has assignment cascading: r = Address() ..street = "East Avenue" ..housenumber = 12

of yourse if you want to maintain readability you want most calls to be on separate lines so r = a.b() ..c() ..d() # the return value of the last function is the value of the entire expression

with some some names that are actually descriptive this could quite nicely represent a sequence of transformations to something for example (is this why jQuery has so much method chaining?)

Seeing "transformations" makes me think of method chaining more than the method cascading as described above. In jQuery (and also Django querysets), you don't get the same object back, but a new object. So Django again: qs = User.objects.filter(is_staff=True).filter(is_active=True) is not equal to qs = User.objects qs.filter(is_staff=True) qs.filter(is_active=True) but qs = User.objects qs = qs.filter(is_staff=True) qs = qs.filter(is_active=True)

anyway with method chaing you sometimes (often? rarely?) have an issue where you have a meaningful return value so you have to chooze if you want to return meaningful output or return self/this to enable chaining. if you need the return value then you obviously need to capture it but sometimes you just want the side effect (mutating state of `self`) and don't care about what the method returns.

if you have this you probably want method chaining to be usable in expressions tho, having subsets of language usable in different contexts rly gets annoying mmighty fast.

Method chaining is already possible in Python, just sometimes you have to add a bit of extra characters to make it prettily span multiple lines. Inside an expression it already works. Method cascading is not yet possible in Python. To be honest, I think it is off-topic in this thread, but also that it is something that might be a good idea. It is however already covered in the following thread... https://mail.python.org/pipermail//python-ideas/2013-November/024124.html Not to say that the opinions might not have changed since then...

On Wed, Aug 31, 2016 at 03:46:13PM -0600, Shane Hathaway wrote:

Cons:

- Extra parentheses are required.

You have to have extra *something* no matter how you do it. An extra semi-colon at the end of the statement, in semi-colon language. An extra backslash at the end of the line. An extra pair of parens. I don't see this as a meaningful negative.

- The indentation is not enforced by the parser, so I have unnecessary freedom that could let various mistakes slip through.

I don't see how. Its precisely because the indentation is not enforced that you *can't* get errors related to the indentation. Of course you can still get other errors, like dropping a comma between function arguments, or failing to close a pair of brackets, but enforcing indentation doesn't protect you against those. Can you demonstrate an error that can be prevented by enforcing indentation inside an expression? In fact, can you explain what indentation rules you want to enforce? Its not clear to me exactly what you want.

- The closing parenthesis has to move every time I append to or reorder the expression, leading to diff noise in version control. (Alternatively, I could put the closing parenthesis on its own line, but that consumes precious vertical reading space.)

I don't get this argument either. There are two situations: you have the closing paren just after the final token, or you have it on a line of its own. If its on a line of its own, you don't need to change it if you modify the expression (as you state yourself). But if its on the same line as the final token, and you modify that line by appending or re-ordering, there's going to be a diff regardless: result = ( blah blah blah blah blah blah blah + x - y or spam.eggs()) becomes: result = ( blah blah blah blah blah x - y blah blah or spam.eggs() or default) The presence or absence of that outermost pair of brackets doesn't affect the diff in any way. You either change the line, and get a diff, or you don't, and don't. At least that's how I see it. Can you show the sort of diff noise that you're talking about?

I'd like to suggest a small change to the Python parser that would make long expressions read better:

rows = dbsession.query(Table1) ... .join( Table2, Table2.y = Table1.y) .filter(Table1.x = xx) .all()

The idea is to use an ellipsis at the end of a line to spread an expression over multiple indented lines, terminated by a return to an earlier indentation level. You can still indent more deeply as needed, as shown above by the join() method call.

Three dots is already syntax for Ellipsis, so this is going to be ambiguous. There will be expressions where ... is valid, and the parser cannot tell if it is intended as a line continuation or not. result = spam + ... (eggs or cheese)(arg) Is that a line continuation, or an accidentally indented line which ought to raise a SyntaxError?

This syntax has all the pros of the existing syntax and resolves all the cons:

I'm not convinced that any of your "cons" actually are cons. See my questions above.

- No extra parentheses are required.

So instead of two extra characters, you need three extra characters. I'm not seeing how this is a benefit.

- The indentation is enforced, so my mistakes are more likely to be caught early. - Without a closing parenthesis, there is no diff noise when I append to or reorder an expression.

As above, I'm not seeing how this is supposed to work. -- Steve

On 09/01/2016 09:35 AM, Steven D'Aprano wrote:

On Wed, Aug 31, 2016 at 03:46:13PM -0600, Shane Hathaway wrote:

...

Cons:

- Extra parentheses are required.

You have to have extra *something* no matter how you do it. An extra semi-colon at the end of the statement, in semi-colon language. An extra backslash at the end of the line. An extra pair of parens. I don't see this as a meaningful negative.

I agree that multi-line expressions should always be explicit. Neither of us would want implicit multi-line expressions. I admire the style Python uses to format classes, functions, loops, conditions, and so on. There's always a block start marker, an indented block, and an implicit block end (implied by a reduction in the indentation level). That's a very tidy pattern and I try to use it as much as possible. I currently can't use that pattern for multi-line expressions. Because my SQLAlchemy code has a lot of multi-line expressions, some of my code doesn't look very much like Python. I'd like to keep my style more consistent so it's easier to read.

...

- The indentation is not enforced by the parser, so I have unnecessary freedom that could let various mistakes slip through.

I don't see how. Its precisely because the indentation is not enforced that you *can't* get errors related to the indentation. Of course you can still get other errors, like dropping a comma between function arguments, or failing to close a pair of brackets, but enforcing indentation doesn't protect you against those.

Can you demonstrate an error that can be prevented by enforcing indentation inside an expression?

Sometimes I fix unbalanced parentheses incorrectly. Here's something I might type. There should be another closing parenthesis in the middle: def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value})) Either Python or flake8 will tell me there's some kind of syntax error, so I might fix it by adding a closing parenthesis at the end: def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value}))) This will fix the syntax error but fail at runtime. With my proposed syntax, I would probably never create the error in the first place because I would only need to scan for balanced parentheses on each line, not over multiple lines: def update(names, value): dbsession.query(Table1) ... .filter(Table1.name.in_(names)) .update({'value': value}) dbsession.query(Table2) ... .filter(Table2.name.in_(names)) .update({'value': value})

In fact, can you explain what indentation rules you want to enforce? Its not clear to me exactly what you want.

The indentation must be consistent, and the expression ends when the indentation level drops to the same level as the first line or an earlier line.

...

- The closing parenthesis has to move every time I append to or reorder the expression, leading to diff noise in version control. (Alternatively, I could put the closing parenthesis on its own line, but that consumes precious vertical reading space.)

I don't get this argument either. There are two situations: you have the closing paren just after the final token, or you have it on a line of its own. If its on a line of its own, you don't need to change it if you modify the expression (as you state yourself). But if its on the same line as the final token, and you modify that line by appending or re-ordering, there's going to be a diff regardless:

result = ( blah blah blah blah blah blah blah + x - y or spam.eggs())

becomes:

result = ( blah blah blah blah blah x - y blah blah or spam.eggs() or default)

The presence or absence of that outermost pair of brackets doesn't affect the diff in any way. You either change the line, and get a diff, or you don't, and don't. At least that's how I see it.

Can you show the sort of diff noise that you're talking about?

Let's say I write this code: rows = ( dbsession.query(Table1) .filter(Table1.name.in_(names))) That code will work, but it will be slow (because it will fetch rows one at a time). I later realize I need to call the all() method to make it fast: rows = ( dbsession.query(Table1) .filter(Table1.name.in_(names)) .all()) One line of code had to change and another had to be added. With my proposed syntax, let's say I write the same code again: rows = dbsession.query(Table1) ... .filter(Table1.name.in_(names)) To make it fast, I add a line: rows = dbsession.query(Table1) ... .filter(Table1.name.in_(names)) .all() In this case, I only added one line of code and didn't change any existing lines, so the signal to noise ratio is better.

Three dots is already syntax for Ellipsis, so this is going to be ambiguous. There will be expressions where ... is valid, and the parser cannot tell if it is intended as a line continuation or not.

result = spam + ... (eggs or cheese)(arg)

Is that a line continuation, or an accidentally indented line which ought to raise a SyntaxError?

Ah, that's very interesting. I had forgotten until now that "..." in Python 3 is a literal value. For anyone else reading along, this now works in Python 3:

...

...

x = ... print(x, ...) Ellipsis Ellipsis

I wasn't aware of that complication. Maybe I should limit the idea to method chaining as Guido hinted. Shane

On 1 September 2016 at 19:44, Shane Hathaway wrote:

Sometimes I fix unbalanced parentheses incorrectly. Here's something I might type. There should be another closing parenthesis in the middle:

def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value}))

Either Python or flake8 will tell me there's some kind of syntax error, so I might fix it by adding a closing parenthesis at the end:

def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value})))

This will fix the syntax error but fail at runtime. With my proposed syntax, I would probably never create the error in the first place because I would only need to scan for balanced parentheses on each line, not over multiple lines:

def update(names, value): dbsession.query(Table1) ... .filter(Table1.name.in_(names)) .update({'value': value}) dbsession.query(Table2) ... .filter(Table2.name.in_(names)) .update({'value': value})

Thanks. That's a nice example of how the proposal might help. But you could of course have written your original code as def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value})) That's not going to completely alleviate the problem, but it does make the intent clearer. And it's possible that you could propose a style rule that a dedent in a bracketed expression is not allowed - that might well be something that flake8 could add, and then you'd get a much clearer error message (but only if you ran flake8 - if you just saw a syntax error from Python, you'd probably be just as likely to "fix" it as you said above, without even trying to run flake8). Also, of course, most text editors would highlight matching parentheses - which makes it much easier to spot the "correct" place for the missing parenthesis. One other thing, I'm not at all keen on using "..." for the syntax - it's almost completely invisible when I read this in gmail, and as others have pointed out, it already has a meaning, as Ellipsis. But I don't have a better suggestion to offer, I'm afraid. Overall, though, I'm cautiously in favour of the proposal. I'm not convinced the benefit is huge, and I'm a little concerned that it may be supporting a style of code that isn't ideal (the method chaining from SQLAlchemy you use to illustrate the examples is common enough in languages like JavaScript, but outside of SQLAlchemy I haven't seen it used much in Python). Also, it's very definitely "yet another way to write expressions across multiple lines". But the indented expression format is pretty readable for cases when you *do* have a long expression and no convenient way to bracket it. Paul

On 09/01/2016 02:04 PM, Paul Moore wrote:

Thanks. That's a nice example of how the proposal might help. But you could of course have written your original code as

def update(names, value): (dbsession.query(Table1) .filter(Table1.name.in_(names)) .update({'value': value}) (dbsession.query(Table2) .filter(Table2.name.in_(names)) .update({'value': value}))

That's not going to completely alleviate the problem, but it does make the intent clearer. And it's possible that you could propose a style rule that a dedent in a bracketed expression is not allowed - that might well be something that flake8 could add, and then you'd get a much clearer error message (but only if you ran flake8 - if you just saw a syntax error from Python, you'd probably be just as likely to "fix" it as you said above, without even trying to run flake8). Also, of course, most text editors would highlight matching parentheses - which makes it much easier to spot the "correct" place for the missing parenthesis.

Good points. That style does look clearer than my example.

One other thing, I'm not at all keen on using "..." for the syntax - it's almost completely invisible when I read this in gmail, and as others have pointed out, it already has a meaning, as Ellipsis. But I don't have a better suggestion to offer, I'm afraid.

Now that I remember that "..." is a literal value in Python 3 (surprise!), I've been thinking about alternatives. I wonder if a double backslash would work. Double backslash at the end of a line is currently a syntax error. Let's see how it looks: def update(names, value): dbsession.query(Table1) \\ .filter(Table1.name.in_(names)) .update({'value': value}) dbsession.query(Table2) \\ .filter(Table2.name.in_(names)) .update({'value': value}) That looks OK to me right now. Double backslash seems to convey the idea that it's not only a line continuation, but a line continuation with special properties. I would never write it this way, BTW: def update(names, value): dbsession.query(Table1) \ .filter(Table1.name.in_(names)) \ .update({'value': value}) dbsession.query(Table2) \ .filter(Table2.name.in_(names)) \ .update({'value': value}) I never seem to be able to sprinkle those single backslashes in the right places consistently. I forget one, or I leave one in after rearranging lines, leading to mayhem.

Also, it's very definitely "yet another way to write expressions across multiple lines". But the indented expression format is pretty readable for cases when you *do* have a long expression and no convenient way to bracket it.

Well, you could say the same about triple quoted strings. Sure there are other ways to write strings longer than a single line, but triple quoted strings are clearly worth the extra parser complexity. Shane