[Python-Dev] PEP 340 - possible new name for block-statement

[Nick Coghlan]

Pierre Barbier de Reuille wrote:
> One main reason is a common error could be (using the synchronised 
> iterator introduced in the PEP):
> 
> for l in synchronised(mylock):
>   do_something()
> 
> It will compile, run, never raise any error but the lock will be 
> acquired and never released !

It's better than that. With the code above, CPython is actually likely to 
release the lock when the loop exits. Change the code to the below to ensure the 
lock doesn't get released:

   sync = synchronised(mylock):
   for l in sync:
       do_something()

> Then, I think there is no use case of a generator with __error__ in the 
> for-loop as it is now. So, IMO, it is error-prone and useless to have 
> two different syntaxes for such things.

Hmm. This does make PJE's suggestion of requiring a decorator in order to flag 
generators for finalisation a little more appealing. Existing generators 
(without the flag) would not be cleaned up, preserving backwards compatibility. 
Generators with the flag would allow resource clean up.

In this case of no new statement syntax, it would probably make more sense to 
refer to iterators that get cleaned up as finalised iterators, and a builtin 
with the obvious name would be:

     def finalised(obj):
         obj.__finalise__ = True  # The all important flag!
         return obj

The syntax below would still be horrible:

     for f in opening(filename):
         for line in f:
            # process line

But such ugliness could be fixed by pushing the inner loop inside the block 
iterator:

    for line in opened(filename):
       # process line

    @finalised
    def opened(filename):
        f = open(filename)
        try:
            for line in f:
                yield line
        finally:
            f.close()

Then, in Py3K, finalisation could simply become the default for loop behaviour. 
However, the '__finalise__' flag would result in some impressive code bloat, as 
any for loop would need to expand to:

     itr = iter(EXPR1)
     if getattr(itr, "__finalise__", False):
         # Finalised semantics
         #    I'm trying to channel Guido here.
         #    This would really look like whatever the PEP 340 block statement
         #    semantics end up being
         val = arg = None
         ret = broke = False
         while True:
             try:
                 VAR1 = next(itr, arg)
             except StopIteration:
                 BLOCK2
                 break
             try:
                 val = arg = None
                 ret = False
                 BLOCK1
             except Exception, val:
                 itr.__error__(val)
             if ret:
                 try:
                     itr.__error__(StopIteration())
                 except StopIteration:
                     pass
                 return val
     else:
         # Non-finalised semantics
         arg = None
         while True:
             try:
                 VAR1 = next(itr, arg)
             except StopIteration:
                 BLOCK2
                 break
             arg = None
             BLOCK1

The major danger I see is that you could then write a generator containing a 
yield inside a try/finally, _without_ applying the finalisation decorator. 
Leading to exactly the problem described above - the lock (or whatever) is never 
cleaned up, because the generator is not flagged for finalisation. In this 
scenario, even destruction of the generator object won't help.

Cheers,
Nick.

P.S. I think PEP 340's proposed for loop semantics are currently incorrect, as 
BLOCK2 is unreachable. It should look more like the non-finalised semantics 
above (with BLOCK2 before the break in the except clause)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net