Tue, Oct 9, 2012 at 5:44 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Guido van Rossum wrote:
Indeed, in NDB this works great. However tracebacks don't work so great: If you don't catch the exception right away, it takes work to make the tracebacks look right when you catch it a few generator calls down on the (conceptual) stack. I fixed this to some extent in NDB, by passing the traceback explicitly along when setting an exception on a Future;
Was this before or after the recent change that was supposed to improve tracebacks from yield-fram chains? If there's still a problem after that, maybe exception handling in yield-from requires some more work.
Sadly it was with Python 2.5/2.7...
But so far when thinking about this recently I have found the goal elusive --
Perhaps you can clear things up by
showing some detailed (but still simple enough) example code to handle e.g. a simple web client?
You might like to take a look at this, where I develop a series of examples culminating in a simple multi-threaded server:
http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/yf_current/Exa...
Definitely very enlightening. Though I think you should not use 'thread' since that term is already reserved for OS threads as supported by the threading module. In NDB I chose to use 'tasklet' -- while that also has other meanings, its meaning isn't fixed in core Python. You could also use task, which also doesn't have a core Python meaning. Just don't call it "process", never mind that Erlang uses this (a number of other languages rooted in old traditions do too, I believe). Also I think you can now revisit it and rewrite the code to use Python 3.3.
Code here:
http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/yf_current/Exa...
It does bother me somehow that you're not using .send() and yield arguments at all. I notice that you have a lot ofthree-line code blocks like this: block_for_reading(sock) yield data = sock.recv(1024) The general form seems to be: arrange for a callback when some operation can be done without blocking yield do the operation This seems to be begging to be collapsed into a single line, e.g. data = yield sock.recv_async(1024) (I would also prefer to see the socket wrapped in an object that makes it hard to accidentally block.)
somehow it seems there *has* to be a distinction between an operation you just *yield* (this would be waiting for a specific low-level I/O operation) and something you use with yield-from, which returns a value through StopIteration.
It may be worth noting that nothing in my server example uses 'yield' to send or receive values -- yield is only used without argument as a suspension point. But the functions containing the yields *are* called with yield-from and may return values via StopIteration.
Yeah, but see my remark above...
So I think there are (at least) two distinct ways of using generators, but the distinction isn't quite the one you're making. Rather, we have "coroutines" (don't yield values, do return values) and "iterators" (do yield values, don't return values).
But surely there's still a place for send() and other PEP 342 features?
Moreover, it's *only* the "coroutine" variety that we need to cater for when designing an async event system. Does that help to alleviate any of your monad-induced headaches?
Not entirely, no. I now have a fair amount experience writing an async system and helping users make sense of its error messages, and there are some practical considerations. E.g. my users sometimes want to treat something as a coroutine but they don't have any yields in it (perhaps they are writing skeleton code and plan to fill in the I/O later). Example: def caller(): data = yield from reader() def reader(): return 'dummy' yield works, but if you drop the yield it doesn't work. With a decorator I know how to make it work either way. -- --Guido van Rossum (python.org/~guido)