contextlib.contextmanager and try/finally

[Robert Kern]

On 1/11/12 3:45 PM, johannh at gmail.com wrote:
> I'm trying to write a context manager to handle database connections, under the principle that I should not rely on CPython's reference-counting semantics to clean up scarce resources, like connections.
>
> I wrote:
>
> @contexlib.contextmanager
> def ensure_connection(con=None):
>      con_created = False
>      if con is None:
>          con_created, con = True, make_connection()
>      try:
>          yield con
>      finally:
>          if con_created:
>              con.close()
>
> However, then I read the following paragraph from PEP-343:
>
>      Note that we're not guaranteeing that the finally-clause is
>      executed immediately after the generator object becomes unused,
>      even though this is how it will work in CPython.  This is similar
>      to auto-closing files: while a reference-counting implementation
>      like CPython deallocates an object as soon as the last reference
>      to it goes away, implementations that use other GC algorithms do
>      not make the same guarantee.  This applies to Jython, IronPython,
>      and probably to Python running on Parrot.
>
> That suggests that I cannot rely on the contextlib.contextmanager decorator to ensure that the connection is closed and would have to write my own object with __enter__ and __exit__ methods to guarantee this.
>
> Is this understanding accurate?  If so, could someone illustrate why this is so?

Looking at the paragraph before this one, it appears that the PEP is talking 
about the .close() method on generators, which is really just a general purpose 
API for closing generators that might not be exhausted yet. It's not really 
related to the context manager stuff except that it came up during the design 
process of the context manager along with the related .send() and .throw() methods.

__enter__() will call .next() once to execute the code up to the yield 
statement. Then __exit__() will call .next() once again to execute the code 
after the yield statement, including the finally: clause. That's the only thing 
you need to rely on. Your connection-closing code will be called if __exit__() 
gets called. That will exhaust your generator, so the .close() method will not 
really do anything helpful or hurtful in such a case.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco