[Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)

[Nick Coghlan]

Greg Ewing wrote:
> Unless I'm seriously mistaken, all the Python-equivalent
> loop code that's been presented is only for expositional
> purposes -- in real life, the logic would be embedded in
> the ceval code that implements the for-loop control
> bytecodes, so there would be little or no difference in
> the bytecode from what is generated today.

Hmm, that would obviously be more sensible. OK, I'll drop that from my list of 
concerns :)

> It would be better to do it the other way around, and
> have a different form of looping statement for when
> you *don't* want finalization. The programmer knows he's
> doing a partial iteration when he writes the code,
> and is therefore in a position to choose the right
> statement.
> 
> For backwards compatibility, the existing for-loop
> would work for partial iteration of old iterators,
> but this usage would be deprecated.

I actually agree, but the pain comes with generators. Doing it this way means 
that generators can't have an __exit__() method by default - it will need to be 
enabled with a decorator of some description (a future statement won't work, 
since it needs to be selectable on a generator by generator basis). It has to be 
done this way, so that old generators (without the decorator) are not 
inadvertently finalised by unmodified for loops (otherwise old code that isn't 
expecting finalisation could break severely).

Hmm, with that approach, a code inspection tool like pychecker could be used to 
pick up the slack, and flag generators which have a yield inside a try/finally 
or a user defined statement without applying the "needs finalisation" decorator 
(assuming the compiler can't detect this for itself).

OK, given the above, finalising by default definitely seems like the right thing 
to do - there's then the question of how to spell "don't finalise this iterator".

It turns out no keyword is needed for that. Instead, an iterator that iterates 
over a supplied iterable suffices:

   class partial_iter(object):
       def __init__(self, iterable):
           self.itr = iter(iterable)
       def __iter__(self):
           yield self
       def next(self):
           return self.itr.next()

Then, partial iteration over something that defines __exit__ is possible via:

   for item in partial_iter(itr):
         break
   print list(itr) # itr not finalised, even if it defines __exit__()

That reads reasonably well, and it should be possible to reduce the overhead on 
for loops to a single check of a method slot in the various opcodes that can 
exit a for loop.

So, this idea (or something like it) will go into my next draft.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com