On 06:24 pm, alex.kirp@gmail.com wrote:
Hi.
I have a question about inline callbacks usage.
Let's assume I have an iterator that works synchronously. I want to wrap this generator to work asynchronously in threads:
def some_generator(): cursor = make_iterator() # cursor's 'next' method uses some nasty blocking I/O while 1: yield deferToThread(cursor.next)
Next, I want to asynchronously iterate over this generator in inlineCallbacks semantics. I should say:
@inlineCallbacks def do_stuff(): for item in some_generator(): try: real_item = yield item # do stuff with real_item except StopIteration: break
It somewhat annoys me that I should always convert item to real_item so inlineCallbacks can properly asynchronize my code. I also need to implement StopIteration myself :( Is there any easier way to do this in Twisted?
You're basically looking for coroutines - ie, context switching across multiple stack frames (from inside some_generator through do_stuff out to the implementation of inlineCallbacks). There are several coroutine libraries for Python. Greenlets is probably the most popular. You can use it with Twisted, if you want. Personally, I'd probably go for something more like: def worker(item): # do stuff with item hook_up(some_generator(), worker) with some kind of hook_up that does the parts of this task that you're tired of repeating over and over again (like turning item into real_item). The result doesn't require figuring out how your coroutines are affecting control flow and probably gives you something more composable too. Jean-Paul