Okay, Nick, thanks for iterating exactly what separation of concerns you meant: it clearly identifies where my thinking was going a bit awry. 

My thinking was more along the lines of, "with" controls deterministic cleanup around a block of code, can't that block be an existing for loop? 
The "with" statement still controls deterministic cleanup; its __enter__() still necessarily precedes evaluation of the for loop, and its __exit__() still immediately follows evaluation of the for loop. 

But, there's not much to gain from the idea, at best, so it's a bit of a waste of time, I'm afraid...



02/02/2013  22:13:02

   



Nick Coghlan
February 2, 2013 8:26 PM
On Sun, Feb 3, 2013 at 1:47 AM, Yuval Greenfield <ubershmekel@gmail.com> wrote:
I really like Golang's solution (defer) which Nick sort of emulates with
ExitStack.

http://docs.python.org/3/library/contextlib.html#contextlib.ExitStack

If ExitStack ever became a language feature

Why would it ever become a language feature? It works just fine as a
context manager, and the need for it isn't frequent enough to justify
special syntax.

we could write stuff like:


    def f():
        fhand = local open(path)
        process(fhand)
        ghand = local open(path2)
        process(ghand)

Why would you leave fhand open longer than necessary? The above would
be better written as:

    def f():
        with open(path) as fhand:
            process(fhand)
        with open(path2) as ghand:
            process(ghand)

If you need both files open at the same time, you can use a nested
context manager:

    def f():
        with open(path) as fhand:
            with open(path2) as ghand:
                process(fhand, ghand)

Or the nesting behaviour built into with statements themselves:

    def f():
        with open(path) as fhand, open(path2) as ghand:
            process(fhand, ghand)

It's only when the number of paths you need to open is dynamic that
ExitStack comes into play (this is actually very close to the example
in ExitStack's docstring, as handling a variable number of
simultaneously open files was the use case that highlighted the fatal
flaw in the way the old contextlib.nested design handled context
managers that acquired the resource in __init__ rather than
__enter__):

    def f(*paths):
        with contextlib.ExitStack() as stack:
            files = [stack.enter_context(open(path)) for path in paths]
            process(files)

Function and class definitions control name scope (amongst other
things), with statements control deterministic cleanup, loops control
iteration. That's what I mean by "separation of concerns" in relation
to these aspects of the language design and it's a *good* thing (and
one of the key reasons with statements behave like PEP 343, rather
than being closer to Guido's original looping idea that is described
in PEP 340).

Cheers,
Nick.

Yuval Greenfield
February 2, 2013 7:47 AM
On Sat, Feb 2, 2013 at 5:16 PM, Shane Green <shane@umbrellacode.com> wrote:
Thanks Nick.  I definitely see your point about iterwith(); have been thinking about that since someone asked where __exit__() would be invoked. 

I meant the following as a more compact way of expressing

for line in file with open(path) as file:
    process(line)



This is an interesting idea, though a bit too dense for my taste.


Indentation levels aren't limited, but flatter is better ;-)


I really like Golang's solution (defer) which Nick sort of emulates with ExitStack.


If ExitStack ever became a language feature, we could write stuff like:


    def f():
        fhand = local open(path)
        process(fhand)
        ghand = local open(path2)
        process(ghand)

# which would be sort of equivalent to

    def g():
        try:
            fhand = None
            ghand = None
            fhand = local open(path)
            process(fhand)
            ghand = local open(path2)
            process(ghand)
        finally:
            if fhand is not None:
                fhand.close()
            if ghand is not None:
                ghand.close()



Yuval
Shane Green
February 2, 2013 7:16 AM
Thanks Nick.  I definitely see your point about iterwith(); have been thinking about that since someone asked where __exit__() would be invoked. 

I meant the following as a more compact way of expressing

for line in file with open(path) as file:
    process(line)

As a more compact way of expressing

with open(path) as file:
    for line in file:
        process(line)

Not a change to the semantics of for-loops; a point my iterwith() function has confuses greatly, I realize now. 
I'm not seeing a loss of separation of concerns there.  

Indentation levels aren't limited, but flatter is better ;-)

I saw a bunch of back and forth regarding iteration and context management in the PEP, but didn't notice anything along these lines in particular . I'll have to go back and take a closer look.



Nick Coghlan
February 2, 2013 6:18 AM

The with statement block is needed to define *when* cleanup happens (unconditionally at the end of the block).

The "iterwith" generator is currently pointless, as it results in nondeterministic cleanup of the context manager, so you may as well not bother and just rely on the underlying iterable's nondeterministic cleanup.

We're never going to add cleanup semantics directly to for loops because:
- separation of concerns is a good design principle
- Indentation levels are not a limited resource (anyone that thinks they are may be forgetting that factoring out context managers, iterators and subfunctions gives you more of them, and that judicious use of early returns and continue statements can avoid wasting them)
- we already considered it when initially designing the with statement and decided it was a bad idea.

I forget where that last part is written up. If it's not in PEP 343, 342, 346 or 340 (the full set of PEPs that led to the current with statement and contextlib.contextmanager designs), it should be in one of the threads they reference.

Cheers,
Nick.

_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
http://mail.python.org/mailman/listinfo/python-ideas
Chris Angelico
February 2, 2013 5:52 AM

According to the OP's posted code, as soon as the iterable runs out.
Not sure what happens if you don't exhaust it but I'm sure generator
functions have already solved that, too.

ChrisA
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
http://mail.python.org/mailman/listinfo/python-ideas