[Python-Dev] PEP 343 - next steps

[Nick Coghlan]

Guido van Rossum wrote:
>>It also considers the possibility of using with statements in an RAII
>>style by acquiring the resource in __init__ or __new__ rather than
>>__enter__.
> 
> 
> Python isn't C++; you shouldn't do that.
> 
> 
>>This is a topic that got a brief mention during early discussion here,
>>but doesn't seem to be covered in the current version of PEP 343.
> 
> 
> I haven't been infected with the RAII meme, so I can't help you here.

I expected your reaction to be along these lines, and at one point 
updated the Wiki to say basically this.

Then I went back to PEP 343, and had another look at the examples. The 
one that sprang out at me was the generic 'closing()' template.

Being able to say "ensure method X of object Y is called when the 
block exits" seems like a *really* useful feature for 'with' statements.

However, when using this approach, the resource is always going to be 
acquired during evaluation of EXPR, since it needs to be passed an an 
argument to the 'closing()' (or equivalent) template.

If the guarantee only applies to __enter__ though, then such templates 
would have to be written 'correctly' using deferred evaluation of the 
argument:

   class closing(object):
     def __init__(self, resource_factory)
         self.factory = resource_factory
         self.resource = None

     def __enter__(self):
         if self.resource is not None:
             raise RuntimeException("Cannot nest use of template")
         self.resource = self.factory()
         return self.resource

     def __exit__(self, *exc_info):
       self.resource.close()
       self.resource = None


Then usage gets a bit uglier, because a lambda is needed in order to 
defer evaluation of the argument:

   with closing(lambda: open("somefile.txt", "r")) as f:
     # Do stuff


If, however, two new opcodes were added that blocked and unblocked 
asynchronous exceptions in a given thread, then not only would it be 
possible to implement 'closing()' in the natural way, then resource 
handlers written in Python would 'do the right thing' to.

Specifically, what if we added a new boolean flag 'permit_async_exc' 
to the thread state that is checked at the top of main loop in ceval.c 
where the asynchronous events are handled?

The simplest trick would be to change the line "if (--_Py_Ticker < 0)" 
  to "if ((--_Py_Ticker < 0) && (_Py_Ticker = 0, 
tstate->allow_async_exc))" (using the comma operator to keep 
_Py_Ticker from overflowing in pathological cases).

Then all the two new opcodes (e.g. ALLOW_ASYNC, BLOCK_ASYNC) would 
have to do is set the state of tstate->allow_async_exc appropriately.

Notice that _Py_Ticker keeps decrementing while the async_exceptions 
are blocked, so that any queued events should be handled immediately 
after execution of the BLOCK_ASYNC call (unless the next opcode is 
SETUP_FINALLY, in which case that opcode will be executed first).

The setting of _Py_Ticker to zero, and the check of the tstate field 
only occur when _Py_Ticker goes negative, so they only add to the cost 
of executing the ceval loop when the check interval has expired. 
Compared to the cost of Py_MakePendingCalls, and the check for thread 
context switches, these costs should be negligible.

An alternative implementation would still allow thread switches to 
happen, and prevent just the asynchronous events. Then the ALLOW_ASYNC 
opcode could just force _Py_Ticker to zero in order to ensure prompt 
evaluation of any pending events.

Either way, the code generation for the with statement could simply 
emit BLOCK_ASYNC as the first opcode, then ALLOW_ASYNC as the opcode 
immediate preceding SETUP_FINALLY. Correct behaviour in the face of 
asynchronous events is then guaranteed, even for cases where the 
resource is acquired in EXPR, or when __enter__ is written in Python.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com