[Tutor] Appending an extra column in a data file

[Oscar Benjamin]

On 11 April 2013 13:25, Albert-Jan Roskam <fomcl at yahoo.com> wrote:
> <snip>
>
>>> Cool. This solves a problem it had with contextlib.nested some time ago.
>>> (sorry for kinda hijacking this thread, but..)
>>> Would it be safe (both __exit__ calls are guaranteed to be made) to use
>>> code like this (if it worked, that is!)?
>>
>> Partly because it doesn't work I really can't figure out what you're
>> trying to do.
>
> You are right, I should have stated the goal: make contextlib.nested work in a way
> that it does not have the shortcoming that caused it become deprecated, namely,
>  __exit__ is not guaranteed to be called if the outer "with" raises an exception.

The problem with contextlib.nested is that it cannot catch errors
raised while its arguments are generated because nested hasn't even
been called yet. For example in the code

with nested(open('foo'), open('bar')) as foo, bar:
    do_stuff()

the expressions used as arguments to the nested function are evaluated
before nested is called. This is always true of function calls:

>>> def f1():
...     print('Calling f1')
...
>>> def f2():
...     print('Calling f2')
...
>>> def g(arg1, arg2):
...     print('Calling g')
...
>>> g(f1(), f2())
Calling f1
Calling f2
Calling g
>>> def f_broken():
...     raise ValueError
...
>>> g(f1(), f_broken())
Calling f1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f_broken
ValueError

So if open raises an error nested is never called and the nested
context manager does not exist in order to call the __exit__ methods
of its arguments.

> Two times "Yaaay properly closed!" means that __exit__ is called twice, right?

Yes it does.

> Anyway, the code below at least runs without errors. Too bad the outcommented
> version with tuple unpacking raises an AttributeError.
>
> import sys
> import contextlib
>
> def someCloseFunc():
>   print "Yaaay properly closed!"
>
> @contextlib.contextmanager
> def funcOne(arg):
>   try:
>     yield arg
>   except:
>     yield sys.exc_info()[1]

Why do you want the two lines below? Attempting to yield twice gives
an error like the one you showed before:

>>> from contextlib import contextmanager
>>> @contextmanager
... def f():
...     try:
...         yield
...     except:
...         yield
...
>>> with f():
...     raise ValueError
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "q:\tools\Python27\lib\contextlib.py", line 36, in __exit__
    raise RuntimeError("generator didn't stop after throw()")
RuntimeError: generator didn't stop after throw()

The contextmanager decorator should be used with a generator function
that hits exactly one yield in its execution. That yield statement
corresponds to the code block in the with statement. There is nothing
for any other yield to correspond to.

>   finally:
>     someCloseFunc()
> funcTwo = funcOne
>
>
> with contextlib.nested(funcOne("one-ish"), funcTwo("two-ish")) as (one, two):
>   print one, two

Since there isn't really any way that funcOne or funcTwo would raise
an error when called there isn't any problem with using nested *in
this case*. The error would have to be raised before the yield
statement. If that's not really possible for any of the context
managers that you want to use then nested is fine and you can use it
without worry.

>
> ##funcs = (funcOne, funcTwo)
> ##with contextlib.nested(*funcs) as (one, two):
> ##  print one("one-ish"), two("two-ish")

That's because you're passing functions into nested rather than
calling the functions and passing the result into nested. Try this:

context_managers = (funcOne('one'), funcTwo('two'))
with contextlib.nested(*context_managers) as cms:
    # blah blah

Note that if you're happy using nested there's no problem with
creating the context managers on the line above.


Oscar