[Python-Dev] 'With' context documentation draft (was Re: Terminology for PEP 343

[Nick Coghlan]

OK, here's some draft documentation using Phillip's context 
terminology. I think it works very well.

"""
With Statements and Context Management

A frequent need in programming is to ensure a particular action is
taken after a specific section of code has been executed (such as 
closing a file or releasing a lock). The tool to achieve this in 
Python is to use the 'with' statement along with the appropriate 
context manager. Context managers ensure a particular action is taken 
to establish the context before the contained suite is entered, and a 
second action to clean up the context when the suite is exited.

The precise behaviour of the 'with' statement is governed by the
supplied context manager - an object which supports the context 
management protocol. This protocol consists of two methods:

     __enter__(self):
       Context managers use this method to create the desired context 
for the execution of the contained suite.
       This method is called without arguments before the contained
suite is entered. If the 'as' clause of the 'with' statement is used,
the value returned from this method is assigned to the identified target.
       Many context managers will return self from this method, but 
returning a different object may make sense for some managers (e.g. 
see the 'closing' suite manager example below).

     __exit__(self, exc_type, exc_value, exc_traceback):
       Context managers use this method to clean up after execution of 
the contained suite.
       This method is called after the contained suite has exited. If
the suite was left due to an exception, the details of that exception
are passed as arguments. Otherwise, all three arguments are set to None.
       If exception details are passed in, and this method returns
without incident, then the original exception continues to propagate.
Otherwise, the exception raised by this method will replace the
original exception.


Using Contexts to Manage Resources

The simplest use of context management is to strictly control the
handling of key resources (such as files, generators, database
connections, synchronisation locks).

These resource managers will generally acquire the resource in their
__enter__ method, although some resource managers may accept the
resource to be managed as an argument to the constructor or acquire it
during construction. Resource managers will then release the resource
in their __exit__ method.

Some resources (such as threading.Lock) support the context management
protocol natively, allowing them to be used directly in 'with' 
statements. The meaning of the established context will depend on the 
specific resource. In the case of threading.Lock, the lock is acquired 
by the __enter__ method, and released by the __exit__ method.


More Context Management Examples

While resource management may be the most obvious use of the context
management protocol, many more uses are possible (otherwise it would 
have been called the resource management protocol!).

For example, when used as a context manager, a decimal.Context object 
will set itself as the current Decimal arithmetic context in the 
__enter__ method, and then automatically revert back to the previous 
Deciaml arithmetic context in the __exit__ method. This allows the 
code in the contained suite to manipulate the Decimal arithmetic 
context freely, without needing to worry about manually undoing any 
changes.

Another example is the use of contexts to handle insertion of the 
appropriate tags when generating HTML:

    with html:
       with body:
          with h1:
             print "Some heading"
          with p:
             print "This is paragraph 1"
          with p:
             print "This is paragraph 2"
          with h2:
             print "Another heading"

Some other possibilities for context management include automatic 
exception logging and handling of database transactions.


Using Generators to Define Context Managers

In conjunction with the 'context' decorator, Python's
generators provide a convenient way to implement the context
management protocol, and share state between the __enter__ and
__exit__ methods.

The generator must yield exactly once during normal execution. The
context manager's __enter__ method executes the generator up to that
point, and the value yielded is returned. The remainder of the
generator is executed by the context manager's __exit__ method. Any
exceptions that occur in the managed context will be injected into the
generator at the location of the yield statement.

For example, the following context manager allows prompt closure of
any resource with a 'close' method (e.g. a generator or file):

     @context
     def closing(resource):
         try:
             yield resource
         finally:
             resource.close()

The operation of the context decorator is described by the following 
Python equivalent (although the exact error messages may differ):

     class ContextManager(object):
         def __init__(self, gen):
             self.gen = gen

         def __enter__(self):
             try:
                 return self.gen.next()
             except StopIteration:
                 raise RuntimeError("generator didn't yield")

         def __exit__(self, type, value, traceback):
             if type is None:
                 try:
                     self.gen.next()
                 except StopIteration:
                     return
                 else:
                     raise RuntimeError("generator didn't stop")
             else:
                 try:
                     self.gen.throw(type, value, traceback)
                 except (type, StopIteration):
                     return
                 else:
                     raise RuntimeError("generator didn't stop")

     def context(func):
         @wraps(func) # [1]
         def helper(*args, **kwds):
             return ContextManager(func(*args, **kwds))
         return helper
"""

[1] Is Python 2.5 going to include a standard decorator for building 
'well-behaved' wrapper functions?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com