[Twisted-Python] waiting on any deferred in a list?
![](https://secure.gravatar.com/avatar/1006ff3d30f83b384e0f0a2b7e1939ca.jpg?s=120&d=mm&r=g)
Hi all, I've been using a pattern for quite some time to collect a bunch o' deferreds in a list and then yield on them one at a time. Here is some untested pseudocode: --- @defer.inlineCallbacks def foo( ls ): defs = [] # get all the deferreds going for item in ls: d = some_deferred_returning_function( item ) defs.append( ( item, d ) ) # associate item with deferred # process results not as they come in but in the original order for ( item, d ) in defs: result = yield d print 'The result on item', item, 'is', result --- In this way, I can get good concurrency (by spinning up all deferreds at once) but also I can iterate on the results using a straightforward traversal of the list data structure and yield. This is working great for me. But on occasion I'd prefer that I can access the results not in the order that I originally got, but rather in the order of completion of the deferreds, so the faster ones come in first, and the slower ones don't hold up faster ones. The usual case when I'd want this is when I have "slower" deferreds near the front, and "faster" deferreds near the back. I can't help but think that there's some async data structure out there for me that can solve this. I've used DeferredList before, and I can't see that it does what I want: it gives you a choice between getting all results only when they are all done (default behavior), or getting the first one that's ready (fireOnOneCallback=True). I want all results, but I want the processing of the results to unfold as the results come in. Thanks for any ideas. -- Benjamin Rutt
![](https://secure.gravatar.com/avatar/a93db92c60e9d5435d204407264457f2.jpg?s=120&d=mm&r=g)
I think you'll want an idiom kind of like this (also untested ;): @inlineCallbacks def foo(ls): defs = [] def process_an_item(result, item): print("Item '{}' done: {}".format(item, result)) for item in ls: d = something_async(item) d.addCallback(process_an_item, item) defs.append(d) yield DeferredList(defs) Error-handling left as an exercise for the reader :) -- meejah
![](https://secure.gravatar.com/avatar/a93db92c60e9d5435d204407264457f2.jpg?s=120&d=mm&r=g)
I think you'll want an idiom kind of like this (also untested ;): @inlineCallbacks def foo(ls): defs = [] def process_an_item(result, item): print("Item '{}' done: {}".format(item, result)) for item in ls: d = something_async(item) d.addCallback(process_an_item, item) defs.append(d) yield DeferredList(defs) Error-handling left as an exercise for the reader :) -- meejah
participants (2)
-
Benjamin Rutt
-
meejah