Coroutines: unexpected behaviour

[Carl Banks]

On Jun 16, 5:03 am, Jérôme Mainka <jmai... at gmail.com> wrote:
> Hello,
>
> I try to experiment with coroutines and I don't understand why this
> snippet doesn't work as expected... In python 2.5 and python 2.6 I get
> the following output:
>
> 0
> Exception exceptions.TypeError: "'NoneType' object is not callable" in
> <generator object at 0x7e43f8> ignored
>
> The TypeError exception comes from the pprint instruction...
>
> If i replace the main part with:
>
> ==
> p1 = dump()
> p2 = sort(p1)
> for item in my_list: p2.send(item)
> ==
>
> it works as expected.
>
> I don't understand what is goind wrong. Has someone an explanation for
> this issue?
>
> Thanks,
>
> Jérôme
>
> ===
> from functools import wraps
> from pprint import pprint
> import random
>
> def coroutine(f):
>     @wraps(f)
>     def start(*args, **kwargs):
>         res = f(*args, **kwargs)
>         res.next()
>         return res
>     return start
>
> @coroutine
> def sort(target):
>     l = []
>
>     try:
>         while True:
>             l.append((yield))
>     except GeneratorExit:
>         l.sort()
>         for item in l:
>             target.send(item)
>
> @coroutine
> def dump():
>     while True:
>         pprint((yield))
>
> if __name__ == "__main__":
>     my_list = range(100)
>     random.shuffle(my_list)
>
>     p = sort(dump())
>
>     for item in my_list:
>         p.send(item)

Tricky, but pretty simple once you know what happens.

What's happening here is that the GeneratorExit exception isn't raised
until the generator is destroyed (because how does the generator know
there are no more sends coming?).  That only happens when the __main__
module is being destroyed.

Well, when destroying __main__, Python overwrites all its attributes
with None, one-by-one, in arbitrary order.  In the first scenario,
"pprint" was being set to None before "p" was, thus by the time the
generator got about to running, the pprint was None.  In the second
scenario, "p2" was being set to None before "pprint", so that pprint
still pointed at the appropriate function when the generator began
executing.

The solution is to explicity close the generator after the loop, this
signaling it to run before __main__ is destroyed.

p.close()


I suggest, if you intend to use this kind of thing in real code (and I
would not recommend that) that you get in a habit of explicitly
closing the generator after the last send(), even when you don't think
you have to.

IMHO, coroutines are the one time during the PEP-era that Python can
be accused of feature creep.  All other changes seemed driven by
thoughtful analysis, this one seemed like it was, "OMG that would be
totally cool".  PEP 342 doesn't give any compelling use cases (it only
gives examples of "cool things you can do with coroutines"), no
discussion on how in improves the language.  Suffice to say that,
thanks to this example, I'm more averse to using them than I was
before.


Carl Banks