[Python-ideas] A real limitation of contextlib.nested()

[Gregory P. Smith]

On Sat, Mar 14, 2009 at 9:58 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I missed the discussion about potentially adding syntactic support for
> multiple context managers in the with statement, but figured I should
> mention a real limitation of contextlib.nested that *would* be fixed by
> adding dedicated syntactic support.
>
> There's a genuine semantic difference between this:
>
>  with cmA():
>      with cmB():
>          # whatever
>
> and this:
>
>  with nested(cmA(), cmB()):
>      # whatever
>
> The latter is actually more accurately translated as:
>
>  mgr1, mgr2 = cmA(), cmB():
>  with mgr1:
>      with mgr2:
>          # whatever
>
> That is, when using nested() the later context managers are created
> outside the scope of the earlier context managers.
>
> So, to use Christian's example from the previous discussion:
>
>  with lock:
>      with open(infile) as fin:
>          with open(outfile, 'w') as fout:
>              fout.write(fin.read())
>
> Using contextlib.nested for that would be outright broken:
>
>  with nested(lock, open(infile), open(outfile) as (_, fin, fout):
>      fout.write(fin.read())
>
> 1. The files are opened without acquiring the lock first
> 2. If an IOError is raised while opening "outfile", then "infile"
> doesn't get closed immediately
>
> I created issue 5491 [1] to point out that the contextlib.nested docs
> could do with being tweaked to make this limitation clearer.
>
> Dedicated syntax (such as the form that Christian proposed) would fix
> this problem:
>
>  with lock, (open(infile) as fin), (open(outfile, 'w') as fout):
>      fout.write(fin.read())
>
> Of course, a custom context manager doesn't suffer any problems either:
>
>    @contextmanager
>    def synced_io(lock, infile, outfile):
>        with lock:
>            with open(infile) as fin:
>                with open(outfile) as fout:
>                    yield fin, fout
>
>    with synced_io(lock, infile, outfile) as (fin, fout):
>       fout.write(fin.read())


fwiw, I believe you could write a version of nested that generates the above
code based on the parameters but I believe it'd be disgustingly slow...

@contextmanager
def slow_nested(*args):
  code_lines = []
  vars = []
  code_lines.append('@contextmanager')
  code_lines.append('def _nested(*args):')
  for idx in xrange(len(args)):
    vars.append('c%d' % idx)
    code_lines.append('%swith args[%d] as %s:' % (' '*(idx+1), idx,
vars[-1]))
  code_lines.append('%syield %s' % (' '*(len(args)+1), ','.join(vars)))
  code = '\n'.join(code_lines)
  print 'CODE:\n', code
  exec(code)
  yield _nested(*args)



>
>
> Cheers,
> Nick.
>
> [1] http://bugs.python.org/issue5491
>
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090315/d99232e3/attachment.html>