[Python-ideas] A real limitation of contextlib.nested()

March 15, 2009

      I missed the discussion about potentially adding syntactic support for
multiple context managers in the with statement, but figured I should
mention a real limitation of contextlib.nested that *would* be fixed by
adding dedicated syntactic support.

There's a genuine semantic difference between this:

  with cmA():
      with cmB():
          # whatever

and this:

  with nested(cmA(), cmB()):
      # whatever

The latter is actually more accurately translated as:

  mgr1, mgr2 = cmA(), cmB():
  with mgr1:
      with mgr2:
          # whatever

That is, when using nested() the later context managers are created
outside the scope of the earlier context managers.

So, to use Christian's example from the previous discussion:

  with lock:
      with open(infile) as fin:
          with open(outfile, 'w') as fout:
              fout.write(fin.read())

Using contextlib.nested for that would be outright broken:

  with nested(lock, open(infile), open(outfile) as (_, fin, fout):
      fout.write(fin.read())

1. The files are opened without acquiring the lock first
2. If an IOError is raised while opening "outfile", then "infile"
doesn't get closed immediately

I created issue 5491 [1] to point out that the contextlib.nested docs
could do with being tweaked to make this limitation clearer.

Dedicated syntax (such as the form that Christian proposed) would fix
this problem:

  with lock, (open(infile) as fin), (open(outfile, 'w') as fout):
      fout.write(fin.read())

Of course, a custom context manager doesn't suffer any problems either:

    @contextmanager
    def synced_io(lock, infile, outfile):
        with lock:
            with open(infile) as fin:
                with open(outfile) as fout:
                    yield fin, fout

    with synced_io(lock, infile, outfile) as (fin, fout):
       fout.write(fin.read())

Cheers,
Nick.

[1] http://bugs.python.org/issue5491

-- 
Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia
---------------------------------------------------------------