[Python-Dev] context manager - generator interaction?
Phillip J. Eby
pje at telecommunity.com
Thu Apr 5 22:38:49 CEST 2007
At 03:54 PM 4/5/2007 -0400, Bob Sidebotham wrote:
>The interaction shown below feels like a bug, or at least very much a
>trap for the unwary. It's some sort of interaction between a context
>manager and a generator, both of which can raise StopIteration. The
>code is excised from a real application, so it's a bit artificial
>looking. Nevertheless, it represents an entirely natural evolution of
>some code (whereby the call to a context manager was added deep within
>the code). Since it seems to work with open as the context manager,
>but not with my own null context manager defined with
>context_manager(), it feels like this is probably a bug in
It actually appears to be a quirk in the "with" machinery itself, that only
affects generator context managers because of their need to handle
StopIteration specially. If you rewrite your context manager as a class
with __enter__ and __exit__, the problem will go away.
Checking what was happening via pdb, I found that the contextmanager's
__exit__ is being called with a "value" of None, rather than with an
exception *instance*. This fouls up GeneratorContextManager.__exit__'s
special handling for StopIteration -- which was apparently never tested by
any unit tests.
I can see a few different ways to work around the problem in
GeneratorContextManager itself, the simplest of which is to add these two
lines right before the self.gen.throw() call:
if type is StopIteration and value is None:
value = type()
It seems to me that this is a bit of a kludge, but trying to make it so
that __exit__ always gets called with an instance is much more of a pain
since it appears that the None a performance enhancement for simple errors
like StopIteration, that avoids creating an instance.
More information about the Python-Dev