Mailman 3 Multiline with statement line continuation - Python-Dev

newer
Re: [Python-Dev] [Python-checkins]...

Multiline with statement line continuation

older
Re: [Python-Dev] [Python-checkins]...

Allen Li

11 Aug 2014 11 Aug '14

6:08 p.m.

This is a problem I sometimes run into when working with a lot of files simultaneously, where I need three or more `with` statements: with open('foo') as foo: with open('bar') as bar: with open('baz') as baz: pass Thankfully, support for multiple items was added in 3.1: with open('foo') as foo, open('bar') as bar, open('baz') as baz: pass However, this begs the need for a multiline form, especially when working with three or more items: with open('foo') as foo, \ open('bar') as bar, \ open('baz') as baz, \ open('spam') as spam \ open('eggs') as eggs: pass Currently, this works with explicit line continuation, but as all style guides favor implicit line continuation over explicit, it would be nice if you could do the following: with (open('foo') as foo, open('bar') as bar, open('baz') as baz, open('spam') as spam, open('eggs') as eggs): pass Currently, this is a syntax error, since the language specification for `with` is with_stmt ::= "with" with_item ("," with_item)* ":" suite with_item ::= expression ["as" target] as opposed to something like with_stmt ::= "with" with_expr ":" suite with_expr ::= with_item ("," with_item)* | '(' with_item ("," with_item)* ')' This is really just a style issue, furthermore a style issue that requires a change to the languagee grammar (probably, someone who knows for sure please confirm), so at first I thought it wasn't worth mentioning, but I'd like to hear what everyone else thinks.

Attachments:

attachment.sig (application/pgp-signature — 473 bytes)

Show replies by date

Ben Finney

11 Aug 11 Aug

6:27 p.m.

New subject: Multiline ‘with’ statement line continuation

Allen Li writes:

...

Currently, this works with explicit line continuation, but as all style guides favor implicit line continuation over explicit, it would be nice if you could do the following:

with (open('foo') as foo, open('bar') as bar, open('baz') as baz, open('spam') as spam, open('eggs') as eggs): pass

Currently, this is a syntax error

Even if it weren't a syntax error, the syntax would be ambiguous. How will you discern the meaning of:: with ( foo, bar, baz): pass Is that three separate context managers? Or is it one tuple with three items? I am definitely sympathetic to the desire for a good solution to multi-line ‘with’ statements, but I also don't want to see a special case to make it even more difficult to understand when a tuple literal is being specified in code. I admit I don't have a good answer to satisfy both those simultaneously. -- \ “We have met the enemy and he is us.” —Walt Kelly, _Pogo_ | `\ 1971-04-22 | _o__) | Ben Finney

Ben Hoyt

7:29 p.m.

New subject: Multiline 'with' statement line continuation

...

Even if it weren't a syntax error, the syntax would be ambiguous. How will you discern the meaning of::

with ( foo, bar, baz): pass

Is that three separate context managers? Or is it one tuple with three items?

Is it meaningful to use "with" with a tuple, though? Because a tuple isn't a context manager with __enter__ and __exit__ methods. For example:

...

...
...
with (1,2,3): pass ... Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: __exit__

So -- although I'm not arguing for it here -- you'd be turning an code (a runtime AttributeError) into valid syntax. -Ben

Ben Finney

10:41 p.m.

New subject: Multiline 'with' statement line continuation

Ben Hoyt writes:

...

So -- although I'm not arguing for it here -- you'd be turning an code (a runtime AttributeError) into valid syntax.

Exactly what I'd want to avoid, especially because it *looks* like a tuple. There are IMO too many pieces of code that look confusingly similar to tuples but actually mean something else. -- \ “I have an answering machine in my car. It says, ‘I'm home now. | `\ But leave a message and I'll call when I'm out.’” —Steven Wright | _o__) | Ben Finney

Nick Coghlan

7:28 p.m.

On 12 Aug 2014 09:09, "Allen Li" wrote:

...

This is a problem I sometimes run into when working with a lot of files simultaneously, where I need three or more `with` statements:

with open('foo') as foo: with open('bar') as bar: with open('baz') as baz: pass

Thankfully, support for multiple items was added in 3.1:

with open('foo') as foo, open('bar') as bar, open('baz') as baz: pass

However, this begs the need for a multiline form, especially when working with three or more items:

with open('foo') as foo, \ open('bar') as bar, \ open('baz') as baz, \ open('spam') as spam \ open('eggs') as eggs: pass

I generally see this kind of construct as a sign that refactoring is needed. For example, contextlib.ExitStack offers a number of ways to manage multiple context managers dynamically rather than statically. Regards, Nick.

Steven D'Aprano

12 Aug 12 Aug

7:15 a.m.

On Tue, Aug 12, 2014 at 10:28:14AM +1000, Nick Coghlan wrote:

...

On 12 Aug 2014 09:09, "Allen Li" wrote:

...
This is a problem I sometimes run into when working with a lot of files simultaneously, where I need three or more `with` statements:

with open('foo') as foo: with open('bar') as bar: with open('baz') as baz: pass

Thankfully, support for multiple items was added in 3.1:

with open('foo') as foo, open('bar') as bar, open('baz') as baz: pass

However, this begs the need for a multiline form, especially when working with three or more items:

with open('foo') as foo, \ open('bar') as bar, \ open('baz') as baz, \ open('spam') as spam \ open('eggs') as eggs: pass

I generally see this kind of construct as a sign that refactoring is needed. For example, contextlib.ExitStack offers a number of ways to manage multiple context managers dynamically rather than statically.

I don't think that ExitStack is the right solution for when you have a small number of context managers known at edit-time. The extra effort of writing your code, and reading it, in a dynamic manner is not justified. Compare the natural way of writing this: with open("spam") as spam, open("eggs", "w") as eggs, frobulate("cheese") as cheese: # do stuff with spam, eggs, cheese versus the dynamic way: with ExitStack() as stack: spam, eggs = [stack.enter_context(open(fname), mode) for fname, mode in zip(("spam", "eggs"), ("r", "w")] cheese = stack.enter_context(frobulate("cheese")) # do stuff with spam, eggs, cheese I prefer the first, even with the long line. -- Steven

Ian Cordasco

8:04 a.m.

On Tue, Aug 12, 2014 at 7:15 AM, Steven D'Aprano wrote:

...

On Tue, Aug 12, 2014 at 10:28:14AM +1000, Nick Coghlan wrote:

...
On 12 Aug 2014 09:09, "Allen Li" wrote:

...
This is a problem I sometimes run into when working with a lot of files simultaneously, where I need three or more `with` statements:

with open('foo') as foo: with open('bar') as bar: with open('baz') as baz: pass

Thankfully, support for multiple items was added in 3.1:

with open('foo') as foo, open('bar') as bar, open('baz') as baz: pass

However, this begs the need for a multiline form, especially when working with three or more items:

with open('foo') as foo, \ open('bar') as bar, \ open('baz') as baz, \ open('spam') as spam \ open('eggs') as eggs: pass

I generally see this kind of construct as a sign that refactoring is needed. For example, contextlib.ExitStack offers a number of ways to manage multiple context managers dynamically rather than statically.

I don't think that ExitStack is the right solution for when you have a small number of context managers known at edit-time. The extra effort of writing your code, and reading it, in a dynamic manner is not justified. Compare the natural way of writing this:

with open("spam") as spam, open("eggs", "w") as eggs, frobulate("cheese") as cheese: # do stuff with spam, eggs, cheese

versus the dynamic way:

with ExitStack() as stack: spam, eggs = [stack.enter_context(open(fname), mode) for fname, mode in zip(("spam", "eggs"), ("r", "w")] cheese = stack.enter_context(frobulate("cheese")) # do stuff with spam, eggs, cheese

I prefer the first, even with the long line.

I agree with Steven for *small* numbers of context managers. Once they become too long though, either refactoring is severely needed or the user should ExitStack. To quote Ben Hoyt:

...

Is it meaningful to use "with" with a tuple, though? Because a tuple isn't a context manager with __enter__ and __exit__ methods. For example:

...
...
...
with (1,2,3): pass ... Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: __exit__

So -- although I'm not arguing for it here -- you'd be turning an code (a runtime AttributeError) into valid syntax.

I think by introducing parentheses we are going to risk seriously confusing users who may then try to write an assignment like a = (open('spam') as spam, open('eggs') as eggs) Because it looks like a tuple but isn't and I think the extra complexity this would add to the language would not be worth the benefit. If we simply look at Ruby for what happens when you have an overloaded syntax that means two different things, you can see why I'm against modifying this syntax. In Ruby, parentheses for method calls are optional and curly braces (i.e, {}) are used for blocks and hash literals. With a method on class that takes a parameter and a block, you get some confusing errors, take for example: class Spam def eggs(ham) puts ham yield if block_present? end end s = Spam.new s.eggs {monty: 'python'} SyntaxError: ... But s.eggs({monty: 'python'}) Will print out the hash. The interpreter isn't intelligent enough to know if you're attempting to pass a hash as a parameter or a block to be executed. This may seem like a stretch to apply to Python, but the concept of muddling the meaning of something already very well defined seems like a bad idea.

Steven D'Aprano

10:38 p.m.

On Tue, Aug 12, 2014 at 08:04:35AM -0500, Ian Cordasco wrote:

...

I think by introducing parentheses we are going to risk seriously confusing users who may then try to write an assignment like

a = (open('spam') as spam, open('eggs') as eggs)

Seriously? If they try it, they will get a syntax error. Now, admittedly Python's syntax error messages tend to be terse and cryptic, but it's still enough to show that you can't do that. py> a = (open('spam') as spam, open('eggs') as eggs) File "<stdin>", line 1 a = (open('spam') as spam, open('eggs') as eggs) ^ SyntaxError: invalid syntax I don't see this as a problem. There's no limit to the things that people *might* do if they don't understand Python semantics: for module in sys, math, os, import module (and yes, I once tried this as a beginner) but they try it once, realise it doesn't work, and never do it again.

...

Because it looks like a tuple but isn't and I think the extra complexity this would add to the language would not be worth the benefit.

Do we have a problem with people thinking that, since tuples are normally interchangable with lists, they can write this? from module import [fe, fi, fo, fum, spam, eggs, cheese] and then being "seriously confused" by the syntax error they receive? Or writing this? from (module import fe, fi, fo, fum, spam, eggs, cheese) It's not sufficient that people might try it, see it fails, and move on. Your claim is that it will cause serious confusion. I just don't see that happening.

...

If we simply look at Ruby for what happens when you have an overloaded syntax that means two different things, you can see why I'm against modifying this syntax.

That ship has sailed in Python, oh, 20+ years ago. Parens are used for grouping, for tuples[1], for function calls, for parameter lists, class base-classes, generator expressions and line continuations. I cannot think of any examples where these multiple uses for parens has cause meaningful confusion, and I don't think this one will either. [1] Technically not, since it's the comma, not the ( ), which makes a tuple, but a lot of people don't know that and treat it as if it the parens were compulsary. -- Steven

Ethan Furman

15 Aug 15 Aug

4:03 p.m.

On 08/12/2014 08:38 PM, Steven D'Aprano wrote:

...

[1] Technically not, since it's the comma, not the ( ), which makes a tuple, but a lot of people don't know that and treat it as if it the parens were compulsary.

It might as well be, because if there can be a non-tuple way to interpret the comma that way takes precedence, and then the parens /are/ required to disambiguate and get the tuple you wanted. -- ~Ethan~

Nick Coghlan

13 Aug 13 Aug

3:34 a.m.

On 12 August 2014 22:15, Steven D'Aprano wrote:

...

Compare the natural way of writing this:

with open("spam") as spam, open("eggs", "w") as eggs, frobulate("cheese") as cheese: # do stuff with spam, eggs, cheese

versus the dynamic way:

with ExitStack() as stack: spam, eggs = [stack.enter_context(open(fname), mode) for fname, mode in zip(("spam", "eggs"), ("r", "w")] cheese = stack.enter_context(frobulate("cheese")) # do stuff with spam, eggs, cheese

You wouldn't necessarily switch at three. At only three, you have lots of options, including multiple nested with statements: with open("spam") as spam: with open("eggs", "w") as eggs: with frobulate("cheese") as cheese: # do stuff with spam, eggs, cheese The "multiple context managers in one with statement" form is there *solely* to save indentation levels, and overuse can often be a sign that you may have a custom context manager trying to get out: @contextlib.contextmanager def dish(spam_file, egg_file, topping): with open(spam_file), open(egg_file, 'w'), frobulate(topping): yield with dish("spam", "eggs", "cheese") as spam, eggs, cheese: # do stuff with spam, eggs & cheese ExitStack is mostly useful as a tool for writing flexible custom context managers, and for dealing with context managers in cases where lexical scoping doesn't necessarily work, rather than being something you'd regularly use for inline code. "Why do I have so many contexts open at once in this function?" is a question developers should ask themselves in the same way its worth asking "why do I have so many local variables in this function?" Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Akira Li

10:47 a.m.

Nick Coghlan writes:

...

On 12 August 2014 22:15, Steven D'Aprano wrote:

...
Compare the natural way of writing this:

with open("spam") as spam, open("eggs", "w") as eggs, frobulate("cheese") as cheese: # do stuff with spam, eggs, cheese

versus the dynamic way:

with ExitStack() as stack: spam, eggs = [stack.enter_context(open(fname), mode) for fname, mode in zip(("spam", "eggs"), ("r", "w")] cheese = stack.enter_context(frobulate("cheese")) # do stuff with spam, eggs, cheese

You wouldn't necessarily switch at three. At only three, you have lots of options, including multiple nested with statements:

with open("spam") as spam: with open("eggs", "w") as eggs: with frobulate("cheese") as cheese: # do stuff with spam, eggs, cheese

The "multiple context managers in one with statement" form is there *solely* to save indentation levels, and overuse can often be a sign that you may have a custom context manager trying to get out:

@contextlib.contextmanager def dish(spam_file, egg_file, topping): with open(spam_file), open(egg_file, 'w'), frobulate(topping): yield

with dish("spam", "eggs", "cheese") as spam, eggs, cheese: # do stuff with spam, eggs & cheese

ExitStack is mostly useful as a tool for writing flexible custom context managers, and for dealing with context managers in cases where lexical scoping doesn't necessarily work, rather than being something you'd regularly use for inline code.

"Why do I have so many contexts open at once in this function?" is a question developers should ask themselves in the same way its worth asking "why do I have so many local variables in this function?"

Multiline with-statement can be useful even with *two* context managers. Two is not many. Saving indentations levels along is a worthy goal. It can affect readability and the perceived complexity of the code. Here's how I'd like the code to look like: with (open('input filename') as input_file, open('output filename', 'w') as output_file): # code with list comprehensions to transform input file into output file Even one additional unnecessary indentation level may force to split list comprehensions into several lines (less readable) and/or use shorter names (less readable). Or it may force to move the inline code into a separate named function prematurely, solely to preserve the indentation level (also may be less readable) i.e., with ... as input_file: with ... as output_file: ... #XXX indentation level is lost for no reason with ... as infile, ... as outfile: #XXX shorter names ... with ... as input_file: with ... as output_file: transform(input_file, output_file) #XXX unnecessary function And (nested() can be implemented using ExitStack): with nested(open(..), open(..)) as (input_file, output_file): ... #XXX less readable Here's an example where nested() won't help: def get_integers(filename): with (open(filename, 'rb', 0) as file, mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mmapped_file): for match in re.finditer(br'\d+', mmapped_file): yield int(match.group()) Here's another: with (open('log'+'some expression that generates filename', 'a') as logfile, redirect_stdout(logfile)): ... -- Akira

yoav glazner

12:08 p.m.

On Aug 13, 2014 7:04 PM, "Akira Li" <4kir4.1i@gmail.com> wrote:

...

Nick Coghlan writes:

...
On 12 August 2014 22:15, Steven D'Aprano wrote:

...
Compare the natural way of writing this:

with open("spam") as spam, open("eggs", "w") as eggs,

frobulate("cheese") as cheese:

...

...
...
# do stuff with spam, eggs, cheese

versus the dynamic way:

with ExitStack() as stack: spam, eggs = [stack.enter_context(open(fname), mode) for fname,

mode in

...
zip(("spam", "eggs"), ("r", "w")] cheese = stack.enter_context(frobulate("cheese")) # do stuff with spam, eggs, cheese

You wouldn't necessarily switch at three. At only three, you have lots of options, including multiple nested with statements:

with open("spam") as spam: with open("eggs", "w") as eggs: with frobulate("cheese") as cheese: # do stuff with spam, eggs, cheese

The "multiple context managers in one with statement" form is there *solely* to save indentation levels, and overuse can often be a sign that you may have a custom context manager trying to get out:

@contextlib.contextmanager def dish(spam_file, egg_file, topping): with open(spam_file), open(egg_file, 'w'), frobulate(topping): yield

with dish("spam", "eggs", "cheese") as spam, eggs, cheese: # do stuff with spam, eggs & cheese

ExitStack is mostly useful as a tool for writing flexible custom context managers, and for dealing with context managers in cases where lexical scoping doesn't necessarily work, rather than being something you'd regularly use for inline code.

"Why do I have so many contexts open at once in this function?" is a question developers should ask themselves in the same way its worth asking "why do I have so many local variables in this function?"

Multiline with-statement can be useful even with *two* context managers. Two is not many.

Saving indentations levels along is a worthy goal. It can affect readability and the perceived complexity of the code.

Here's how I'd like the code to look like:

with (open('input filename') as input_file, open('output filename', 'w') as output_file): # code with list comprehensions to transform input file into output file

Even one additional unnecessary indentation level may force to split list comprehensions into several lines (less readable) and/or use shorter names (less readable). Or it may force to move the inline code into a separate named function prematurely, solely to preserve the indentation level (also may be less readable) i.e.,

with ... as input_file: with ... as output_file: ... #XXX indentation level is lost for no reason

with ... as infile, ... as outfile: #XXX shorter names ...

with ... as input_file: with ... as output_file: transform(input_file, output_file) #XXX unnecessary function

And (nested() can be implemented using ExitStack):

with nested(open(..), open(..)) as (input_file, output_file): ... #XXX less readable

Here's an example where nested() won't help:

def get_integers(filename): with (open(filename, 'rb', 0) as file, mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mmapped_file): for match in re.finditer(br'\d+', mmapped_file): yield int(match.group())

Here's another:

with (open('log'+'some expression that generates filename', 'a') as logfile, redirect_stdout(logfile)): ...

Just a thought, would it bit wierd that: with (a as b, c as d): "works" with (a, c): "boom" with(a as b, c): ?

...

-- Akira

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

https://mail.python.org/mailman/options/python-dev/yoavglazner%40gmail.com

Steven D'Aprano

12:32 p.m.

On Wed, Aug 13, 2014 at 08:08:51PM +0300, yoav glazner wrote: [...]

...

Just a thought, would it bit wierd that: with (a as b, c as d): "works" with (a, c): "boom" with(a as b, c): ?

If this proposal is accepted, there is no need for the "boom". The syntax should allow: # Without parens, limited to a single line. with a [as name], b [as name], c [as name], ...: block # With parens, not limited to a single line. with (a [as name], b [as name], c [as name], ... ): block where the "as name" part is always optional. In both these cases, whether there are parens or not, it will be interpreted as a series of context managers and never as a single tuple. Note two things: (1) this means that even in the unlikely event that tuples become context managers in the future, you won't be able to use a tuple literal: with (1, 2, 3): # won't work as expected t = (1, 2, 3) with t: # will work as expected But I cannot imagine any circumstances where tuples will become context managers. (2) Also note that *this is already the case*, since tuples are made by the commas, not the parentheses. E.g. this succeeds: # Not a tuple, actually two context managers. with open("/tmp/foo"), open("/tmp/bar", "w"): pass -- Steven

Ethan Furman

15 Aug 15 Aug

4:08 p.m.

On 08/13/2014 10:32 AM, Steven D'Aprano wrote:

...

(2) Also note that *this is already the case*, since tuples are made by the commas, not the parentheses. E.g. this succeeds:

# Not a tuple, actually two context managers. with open("/tmp/foo"), open("/tmp/bar", "w"): pass

Thanks for proving my point! A comma, and yet we did *not* get a tuple from it. -- ~Ethan~

Georg Brandl

4:34 p.m.

On 08/15/2014 11:08 PM, Ethan Furman wrote:

...

On 08/13/2014 10:32 AM, Steven D'Aprano wrote:

...
(2) Also note that *this is already the case*, since tuples are made by the commas, not the parentheses. E.g. this succeeds:

# Not a tuple, actually two context managers. with open("/tmp/foo"), open("/tmp/bar", "w"): pass

Thanks for proving my point! A comma, and yet we did *not* get a tuple from it.

Clearly the rule is that the comma makes the tuple, except when it doesn't :) Georg

Steven D'Aprano

10:08 p.m.

On Fri, Aug 15, 2014 at 02:08:42PM -0700, Ethan Furman wrote:

...

On 08/13/2014 10:32 AM, Steven D'Aprano wrote:

...
(2) Also note that *this is already the case*, since tuples are made by the commas, not the parentheses. E.g. this succeeds:

# Not a tuple, actually two context managers. with open("/tmp/foo"), open("/tmp/bar", "w"): pass

Thanks for proving my point! A comma, and yet we did *not* get a tuple from it.

Um, sorry, I don't quite get you. Are you agreeing or disagreeing with me? I spent half of yesterday reading the static typing thread over on Python-ideas and it's possible my brain has melted down *wink* but I'm confused by your response. Normally when people say "Thanks for proving my point", the implication is that the person being thanked (in this case me) has inadvertently undercut their own argument. I don't think I have. I'm suggesting that the argument *against* the proposal: "Multi-line with statements should not be allowed, because: with (spam, eggs, cheese): ... is syntactically a tuple" is a poor argument (that is, I'm disagreeing with it), since *single* line parens-free with statements are already syntactically a tuple: with spam, eggs, cheese: # Commas make a tuple, not parens. ... I think the OP's suggestion is a sound one, and while Nick's point that bulky with-statements *may* be a sign that some re-factoring is needed, there are many things that are a sign that re-factoring is needed and I don't think this particular one warrents rejecting what is otherwise an obvious and clear way of using multiple context managers. -- Steven

Ethan Furman

10:29 p.m.

On 08/15/2014 08:08 PM, Steven D'Aprano wrote:

...

On Fri, Aug 15, 2014 at 02:08:42PM -0700, Ethan Furman wrote:

...
On 08/13/2014 10:32 AM, Steven D'Aprano wrote:

...
(2) Also note that *this is already the case*, since tuples are made by the commas, not the parentheses. E.g. this succeeds:

# Not a tuple, actually two context managers. with open("/tmp/foo"), open("/tmp/bar", "w"): pass

Thanks for proving my point! A comma, and yet we did *not* get a tuple from it.

Um, sorry, I don't quite get you. Are you agreeing or disagreeing with me? I spent half of yesterday reading the static typing thread over on Python-ideas and it's possible my brain has melted down *wink* but I'm confused by your response.

My point is that commas don't always make a tuple, and your example above is a case in point: we have a comma separating two context managers, but we do not have a tuple, and your comment even says so.

...

is a poor argument (that is, I'm disagreeing with it), since *single* line parens-free with statements are already syntactically a tuple:

with spam, eggs, cheese: # Commas make a tuple, not parens.

This point I do not understand -- commas /can/ create a tuple, but don't /necessarily/ create a tuple. So, semantically: no tuple. Syntactically: I don't think there's a tuple there this way either. I suppose one of us should look it up in the lexar. ;) -- ~Ethan~

Steven D'Aprano

16 Aug 16 Aug

12:41 a.m.

On Fri, Aug 15, 2014 at 08:29:09PM -0700, Ethan Furman wrote:

...

On 08/15/2014 08:08 PM, Steven D'Aprano wrote:

[...]

...

...
is a poor argument (that is, I'm disagreeing with it), since *single* line parens-free with statements are already syntactically a tuple:

with spam, eggs, cheese: # Commas make a tuple, not parens.

This point I do not understand -- commas /can/ create a tuple, but don't /necessarily/ create a tuple. So, semantically: no tuple.

Right! I think we are in agreement. It's not that with statements actually generate a tuple, but that they *look* like they include a tuple. That's what I meant by "syntactically a tuple", sorry if that was confusing. I didn't mean to suggest that Python necessarily builds a tuple of context managers. If people were going to be prone to mistake with (a, b, c): ... as including a tuple, they would have already mistaken: with a, b, c: ... the same way. But they haven't. -- Steven

Ben Finney

2:25 a.m.

Steven D'Aprano writes:

...

If people were going to be prone to mistake

with (a, b, c): ...

as including a tuple

… because the parens are a strong signal “this is an expression to be evaluated, resulting in a single value to use in the statement”.

...

they would have already mistaken:

with a, b, c: ...

the same way. But they haven't.

Right. The presence or absence of parens make a big semantic difference. -- \ “The process by which banks create money is so simple that the | `\ mind is repelled.” —John Kenneth Galbraith, _Money: Whence It | _o__) Came, Where It Went_, 1975 | Ben Finney

Devin Jeanpierre

3:04 a.m.

On Sat, Aug 16, 2014 at 12:25 AM, Ben Finney wrote:

...

Steven D'Aprano writes:

...
If people were going to be prone to mistake

with (a, b, c): ...

as including a tuple

… because the parens are a strong signal “this is an expression to be evaluated, resulting in a single value to use in the statement”.

...
they would have already mistaken:

with a, b, c: ...

the same way. But they haven't.

Right. The presence or absence of parens make a big semantic difference.

At least historically so, since "except a, b:" and "except (a, b):" used to be different things (only the latter constructs a tuple in 2.x). OTOH, consider "from .. import (..., ..., ...)". Pretty sure at this point parens can be used for non-expressions quite reasonably -- although I'd still prefer just allowing newlines without requiring extra syntax. -- Devin

Steven D'Aprano

6:16 a.m.

On Sat, Aug 16, 2014 at 05:25:33PM +1000, Ben Finney wrote: [...]

...

...
they would have already mistaken:

with a, b, c: ...

the same way. But they haven't.

Right. The presence or absence of parens make a big semantic difference.

from silly.mistakes.programmers.make import ( hands, up, anyone, who, thinks, this, is_, a, tuple) def function(how, about, this, one): ... But quite frankly, even if there is some person somewhere who gets confused and tries to write: context_managers = (open("a"), open("b", "w"), open("c", "w")) with context_managers as things: text = things[0].read() things[1].write(text) things[2].write(text.upper()) I simply don't care. They will try it, discover that tuples are not context managers, fix their code, and move on. (I've made sillier mistakes, and became a better programmer from it.) We cannot paralyse ourselves out of fear that somebody, somewhere, will make a silly mistake. You can try that "with tuple" code right now, and you will get nice runtime exception. I admit that the error message is not the most descriptive I've ever seen, but I've seen worse, and any half-decent programmer can do what they do for any other unexpected exception: read the Fine Manual, or ask for help, or otherwise debug the problem. Why should this specific exception be treated as so harmful that we have to forgo a useful piece of functionality to avoid it? Some designs are bug-magnets, like the infamous "except A,B" syntax, which fails silently, doing the wrong thing. Unless someone has a convincing rationale for how and why this multi-line with will likewise be a bug-magnet, I don't think that some vague similarity between it and tuples is justification for rejecting the proposal. -- Steven

Marko Rauhamaa

7:47 a.m.

Steven D'Aprano :

...

I simply don't care. They will try it, discover that tuples are not context managers, fix their code, and move on.

*Could* tuples (and lists and sequences) be context managers? *Should* tuples (and lists and sequences) be context managers?

...

I don't think that some vague similarity between it and tuples is justification for rejecting the proposal.

You might be able to have it bothways. You could have: with (open(name) for name in os.listdir("config")) as files: ... Marko

Chris Angelico

4:42 p.m.

On Sat, Aug 16, 2014 at 10:47 PM, Marko Rauhamaa wrote:

...

You might be able to have it bothways. You could have:

with (open(name) for name in os.listdir("config")) as files:

But that's not a tuple, it's a generator. Should generators be context managers? Is anyone seriously suggesting this? I don't think so. Is this solutions looking for problems? ChrisA

Nick Coghlan

8:10 p.m.

On 17 August 2014 07:42, Chris Angelico wrote:

...

On Sat, Aug 16, 2014 at 10:47 PM, Marko Rauhamaa wrote:

...
You might be able to have it bothways. You could have:

with (open(name) for name in os.listdir("config")) as files:

But that's not a tuple, it's a generator. Should generators be context managers? Is anyone seriously suggesting this? I don't think so. Is this solutions looking for problems?

Yes. We have a whole programming language to play with, when "X is hard to read" becomes a problem, it may be time to reach for a better tool. If the context manager line is getting unwieldy, it's often a sign it's time to factor it out to a dedicated helper, or break it up into multiple with statements :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Devin Jeanpierre

12 Aug 12 Aug

5:43 a.m.

I think this thread is probably Python-Ideas territory... On Mon, Aug 11, 2014 at 4:08 PM, Allen Li wrote:

...

Currently, this works with explicit line continuation, but as all style guides favor implicit line continuation over explicit, it would be nice if you could do the following:

with (open('foo') as foo, open('bar') as bar, open('baz') as baz, open('spam') as spam, open('eggs') as eggs): pass

The parentheses seem unnecessary/redundant/weird. Why not allow newlines in-between "with" and the terminating ":"? with open('foo') as foo, open('bar') as bar, open('baz') as baz: pass -- Devin

Guido van Rossum

10:12 a.m.

On Tue, Aug 12, 2014 at 3:43 AM, Devin Jeanpierre wrote:

...

I think this thread is probably Python-Ideas territory...

On Mon, Aug 11, 2014 at 4:08 PM, Allen Li wrote:

...
Currently, this works with explicit line continuation, but as all style guides favor implicit line continuation over explicit, it would be nice if you could do the following:

with (open('foo') as foo, open('bar') as bar, open('baz') as baz, open('spam') as spam, open('eggs') as eggs): pass

The parentheses seem unnecessary/redundant/weird. Why not allow newlines in-between "with" and the terminating ":"?

with open('foo') as foo, open('bar') as bar, open('baz') as baz: pass

That way lies Coffeescript. Too much guessing. -- --Guido van Rossum (python.org/~guido)

Devin Jeanpierre

7:41 p.m.

On Tue, Aug 12, 2014 at 8:12 AM, Guido van Rossum wrote:

...

On Tue, Aug 12, 2014 at 3:43 AM, Devin Jeanpierre wrote:

...
The parentheses seem unnecessary/redundant/weird. Why not allow newlines in-between "with" and the terminating ":"?

with open('foo') as foo, open('bar') as bar, open('baz') as baz: pass

That way lies Coffeescript. Too much guessing.

There's no syntactic ambiguity, so what guessing are you talking about? What *really* requires guessing, is figuring out where in Python's syntax parentheses are allowed vs not allowed ;). For example, "from foo import (bar, baz)" is legal, but "import (bar, baz)" is not. Sometimes it feels like Python is slowly and organically evolving into a parenthesis-delimited language. -- Devin

Armin Rigo

11:57 a.m.

Hi, On 12 August 2014 01:08, Allen Li wrote:

...

with (open('foo') as foo, open('bar') as bar, open('baz') as baz, open('spam') as spam, open('eggs') as eggs): pass

+1. It's exactly the same grammar extension as for "from import" statements, for the same reason. Armin

Georg Brandl

1:52 p.m.

On 08/12/2014 06:57 PM, Armin Rigo wrote:

...

Hi,

On 12 August 2014 01:08, Allen Li wrote:

...
with (open('foo') as foo, open('bar') as bar, open('baz') as baz, open('spam') as spam, open('eggs') as eggs): pass

+1. It's exactly the same grammar extension as for "from import" statements, for the same reason.

Not the same: in import statements it unambiguously replaces a list of (optionally as-renamed) identifiers. Here, it would replace an arbitrary expression, which I think would mean that we couldn't differentiate between e.g. with (expr).meth(): # a line break in "expr" # would make the parens useful and with (expr1, expr2): cheers, Georg

3537

Age (days ago)

3543

Last active (days ago)

List overview

Download

28 comments

15 participants

participants (15)

Akira Li
Allen Li
Armin Rigo
Ben Finney
Ben Hoyt
Chris Angelico
Devin Jeanpierre
Ethan Furman
Georg Brandl
Guido van Rossum
Ian Cordasco
Marko Rauhamaa
Nick Coghlan
Steven D'Aprano
yoav glazner

Multiline with statement line continuation

Allen Li

Marko Rauhamaa

tags

participants (15)