[Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)

[Phillip J. Eby]

At 11:20 AM 5/16/2005 -0700, Guido van Rossum wrote:
>I think the issue here is not implementation details but whether it
>follows a certain protocol. IMO it's totally acceptable to require
>that the expression used in a with-statement support an appropriate
>protocol, just like we require the expression used in a for-loop to be
>an iterable.

Perhaps I didn't explain well.  I mean that whether early release of a 
resource is desirable, depends on the resource.  For example, consider a 
file versus a StringIO.  If files support resource release and StringIO's 
don't, this pollutes client code with implementation knowledge.

Therefore, the penalty for people trying to clean up resources early is 
that they either pollute their client code with checking to see if things 
are 'with'-able (which seems insane), or else adding empty __enter__ and 
__exit__ methods to things so that their consumers can just use "with" 
whenever they want to scope a resource, whether the resource actually needs 
it or not.

I'm suggesting that we simply take Nick's proposal to its logical 
conclusion, and allow any object to be usable under "with", since it does 
not create any problems to do so.  (At least, none that I can see.)  A 
redundant 'with' does no harm; in the worst case it's just a hint to the 
reader about the scope within which an expression is used within the 
current fuction body.


> > This insight may actually be true regardless of what generators do or don't
> > do; the point is that if you change from using a generator to a built-in
> > iterator type, you shouldn't have to change every place you were using the
> > 'with' blocks to work again.
>
>Huh? The with-statement doesn't loop, and its use of generators is
>such that I don't see how you could ever replace it with a built-in
>iterator.

I'm referring here to Nick's proposal of doing things like this:

     with some_function() as items:
         for item in items:
             # etc.

To ensure that a *normal* generator is finalized.  This isn't about 
generator templates.  So, if you were ever to change 'some_function()' to 
return a list instead of being a generator, for example, this code would no 
longer work.


>About deleting VAR I have mixed feelings. I appreciate the observation
>that it's most likely dead after the with-statement is over, but I'm
>not sure that explicit deletion is correct. Remember that VAR can be
>an arbitrary assignment target, which means it could be a global, or a
>tuple, or an indexed list or dict item, or some object's attribute (I
>hesitate to write "etc." after such an exhaustive list :-).

Argh.  I forgot about that.  :(  On the other hand, I haven't seen any 
examples where arbitrary assignment targets were actually used, so perhaps 
it could be limited to local variables (and possibly-nested tuples 
thereof).  But I also partly agree with the false-sense-of-security thing, 
too, so I'm not sure on this either now that you point out the issue.  It 
also makes me wonder about something like this:

     with open("foo") as f:
         def callback():
             return f.read()

which can't behave sanely if 'callback' is used outside the 'with' 
block.  Of course, it won't work right even if 'f' remains bound to the 
file object, but this does seem to open new implementation complexities to 
get all these pieces to work right.  :(


> > should also be short for this (in the case where some_expr() has no
> > __enter__ or __exit__ methods):
> >
> >       foo = some_expr()
> >       try:
> >           # etc.
> >       finally:
> >           del foo
>
>-1 on this. You're trying to pack too much into a single statement.

Note that I was just leaving out the rest of the standard PEP 3XX expansion 
above, just highlighting the proposed 'del' portion of the expansion.  That 
is, I wasn't proposing an alternate expansion, just showing the effect of 
the other parts of the expansion being skipped because of null 
__enter__/__exit__.


> > And that could be a useful thing for many existing object types, without
> > even updating them for PEP 34[0-9].  :)  It wouldn't be *as* useful for
> > non-CPython implementations, but presumably by the time those
> > implementations catch up, more code will be out there with
> > __enter__/__exit__ methods.
>
>Most likely it would just give an implementation-dependent false sense
>of security.

I think I prefer to see the glass half-full here; that is, I think it 
increases the safety during a transitional period.  But the scoping of 
arbitrary VAR expansions makes things trickier, so I'll think about this 
some more.


>-1 on the whole thing (and noting that this would be an easy extension
>if at some point in the future we change our mind).

Your post has made me think that I might have the concept 
backwards.  Instead of guaranteeing that the variable goes out of scope at 
the end of the block, what a 'with' block actually guarantees (implicit in 
all of the PEP 34x variants) is that EXPR will remain *alive* for the 
duration of the block, without needing to hold it in a variable 
anywhere.  That is:

     with foo():
         bar()

Guarantees that the result of 'foo()' will remain alive at least until the 
block is exited.

(I'm not saying this is anything new; it was implicit all along.  I'm just 
musing about ways to summarize or introduce the 'with' block to people.)