[Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)

[Nick Coghlan]

Ron Adam wrote:
> I agree, re-using or extending 'for' doesn't seem like a good idea to me.

I agree that re-using a straight 'for' loop is out, due to performance and 
compatibility issues with applying finalisation semantics to all such iterative 
loops (there's a reason the PEP redraft doesn't suggest this).

However, it makes sense to me that a "for loop with finalisation" should 
actually *be* a 'for' loop - just with some extra syntax to indicate that the 
iterator is finalised at the end of the loop.

An option other than the one in my PEP draft would be to put 'del' at the end of 
the line instead of before EXPR:

   for [VAR in] EXPR [del]:
       BLOCK1
   else:
       BLOCK2

However, as you say, 'del' isn't great for the purpose, but I was trying to 
avoid introduding yet another keyword. An obvious alternative is to use 
'finally' instead:

   for [finally] [VAR in] EXPR:
       BLOCK1
   else:
       BLOCK2

It still doesn't read all that well, but at least the word more accurately 
reflects the semantics involved.

If a new keyword is used to request iterator finalisation, it should probably 
include the word 'for' since it *is* a for loop:

   foreach [VAR in] EXPR:
       BLOCK1
   else:
       BLOCK2

That is, a basic 'for' loop wouldn't finalise the iterator, but a 'foreach' loop 
would. The other difference is that the iterator in the 'foreach' loop has the 
chance to suppress exceptions other than TerminateBlock/StopIteration (by 
refusing to be finalised in response to the exception).

The last option is to leave finalisation out of the 'for' loop syntax, and 
introduce a user defined statement to handle the finalisation:

   def consuming(iterable):
       itr = iter(iterable)
       try:
           yield itr
       finally:
           itr_exit = getattr(itr, "__exit__", None)
           if itr_exit is not None:
               try:
                   itr_exit(TerminateBlock)
               except TerminateBlock:
                   pass

   stmt consuming(iterable) as itr:
       for item in itr:
           process(item)

With this approach, iterators can't swallow exceptions. This means that 
something like auto_retry() would once again have to be written as a class:

   class auto_retry(3, IOError):
       def __init__(self, times, exc=Exception):
           self.times = xrange(times-1)
           self.exc = exc
           self.succeeded = False

       def __iter__(self):
           attempt = self.attempt
           for i in self.times:
               yield attempt()
               if self.succeeded:
                   break
           else:
               yield self.last_attempt()

       def attempt(self):
           try:
               yield
               self.succeeded = True
           except self.exc:
               pass

       def last_attempt(self):
           yield


   for attempt in auto_retry(3, IOError):
        stmt attempt:
            # Do something!
            # Including break to give up early
            # Or continue to try again without raising IOError

> I wonder how much effect adding, 'for-next' and the 'StopIteration' 
> exception check as proposed in PEP340, will have on 'for''s performance.

I'm not sure what you mean here - 'for' loops already use a StopIteration raised 
by the iterator to indicate that the loop is complete. The code you posted can't 
work, since it also intercepts a StopIteration raised in the body of the loop.

> I think a completely separate looping or non-looping construct would be 
> better for the finalization issue, and maybe can work with class's with 
> __exit__ as well as generators.

The PEP redraft already proposes a non-looping version as a new statement. 
However, since generators are likely to start using the new non-looping 
statement, it's important to be able to ensure timely finalisation of normal 
iterators as well. Tim and Greg's discussion the other day convinced me of this 
- that's why the idea of using 'del' to mark a finalised loop made its way into 
the draft. It can be done using a user defined statement (as shown above), but 
it would be nicer to have something built into the 'for' loop syntax to handle it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com