[Python-ideas] with *context_managers:

[Nick Coghlan]

On Tue, Apr 3, 2012 at 1:34 AM, Sven Marnach <sven at marnach.net> wrote:
> which only works for a fixed number of context managers.  And there is
> a class 'ContextStack' in Nick Coghlan's 'contextlib2' library [1],
> which might be included in Python 3.3.  With this class, you could
> write your code as
>
>    with ContextStack() as stack:
>        for cm in context_managers:
>            stack.enter_context(cm)
>
> This still leaves the question whether your proposed syntax would be
> preferable, also with regard to issue 2292 [2].

Both "with *(iterable)" and "for cm in iterable: stack.enter(cm)" are
flawed in exactly the same way that contextlib.nested() is flawed:
they encourage creating the iterable of context managers first, which
means that inner __init__ methods are not covered by outer __exit__
methods.

This breaks as soon as you have resources (such as files) where the
acquire/release resource management pairing is actually
__init__/__exit__ with __enter__ just returning self rather than
acquiring the resource. If the iterable of context managers is created
first, then the outer resources *will be leaked* if any of the inner
constructors fail. The only way to write code that handles an
arbitrary number of arbitrary context managers in a robust fashion is
to ensure the initialisation steps are also covered by the outer
context managers:

    with CallbackStack() as stack:
        for make_cm in cm_factories:
            stack.enter(make_cm())

(Note that I'm not particularly happy with the class and method names
for contextlib2.ContextStack, and plan to redesign it a bit before
adding it to the stdlib module:
https://bitbucket.org/ncoghlan/contextlib2/issue/8/rename-contextstack-to-callbackstack-and)

The only time you can get away with a contextlib.nested() style API
where the iterable of context managers is created first is when you
*know* that all of the context managers involved do their resource
acquisition in __enter__ rather than __init__. In the general case,
though, any such API is broken because it doesn't reliably clean up
files and similar acquired-on-initialisation resources.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia