with statement and context managers

[Thomas Rachel]

Am 03.08.2011 04:15 schrieb Steven D'Aprano:

 > I'm not greatly experienced with context managers and the with
 > statement, so I would like to check my logic.
 >
 > Somebody (doesn't matter who, or where) stated that they frequently
 > use this idiom:
 >
 > spam = MyContextManager(*args)
 > for ham in my_iter:
 >      with spam:
 >           # do stuff
 >
 >
 > but to me that looks badly wrong. Surely the spam context manager
 > object will exit after the first iteration, and always raise an
 > exception on the second? But I don't quite understand context
 > managers enough to be sure.

Depends on the implementation. As already stated, a 
contextlib.contextmanager will only run once. But it is easy to turn it 
into a persistent one - which internally initializes as often as needed.

class PersistentCM(object):
     def __init__(self, target):
         self.target = target
         self.current = []
     def __enter__(self):
         cm = self.target()
         cm.__enter__()
	self.current.append(cm)
     def __exit__(self, *e):
         return self.current.pop(-1).__exit__(*e)

(untested)

This would allow for a CM to be used multiple times. It internally 
generates a new one on every __enter__() call and disposes of it on 
every __exit__(). As __enter__() and __exit__() are supposed to be used 
symmetrical, it should(!) work this way.

Many CMs don't allow your shown use, though, partly due to obvious reasons:

* The ...ing named ones do the action they are named after in the 
__exit__() part and do nothing in the __enter__() part. (E.g. closing.)

* The ...ed named ones do the named action in __enter__() and undo it in 
__exit__(). They may or may not work multiple times.

For example, take threading.Lock et al., which you can have locked 
several times.


>>>> spam = open('aaa')
>>>> for ham in range(5):
> ...     with spam:
> ...             print ham
> ...
> 0
> Traceback (most recent call last):
>    File "<stdin>", line 2, in<module>
> ValueError: I/O operation on closed file

To be honest, this one I don't understand as well. __exit__() closes the 
file spam, but even before "print ham" can be executed for a second 
time, there is a check if the file is still open. What does __enter__() 
do here? Just check if it is closed, or is it more? Nevertheless, it 
makes sense.


> # Slightly more complex example.
>
>>>> from contextlib import closing
>>>> import urllib
>>>> spam = closing(urllib.urlopen('http://www.python.org'))
>>>> for ham in range(5):
> ...     with spam as page:
> ...         print ham, sum(len(line) for line in page)
> ...
> 0 18486
> 1
> Traceback (most recent call last):
>    File "<stdin>", line 3, in<module>
>    File "<stdin>", line 3, in<genexpr>
>    File "/usr/local/lib/python2.7/socket.py", line 528, in next
>      line = self.readline()
>    File "/usr/local/lib/python2.7/socket.py", line 424, in readline
>      recv = self._sock.recv
> AttributeError: 'NoneType' object has no attribute 'recv'

Here the above said applies: after closing happens in __exit__() and 
_sock gets set to None, the object is not usable any longer.


> Am I right to expect that the above idiom cannot work? If not, what sort of
> context managers do work as shown?

As said, take threading.Lock as an example: they can be acquire()d and 
release()d as often as you want, and these actions happen in __enter__() 
and __exit__(), respectively.


HTH,

Thomas