[Twisted-Python] getting all results from DeferredList
Hello, I'm trying to process results from several Deferreds (d1, d2) with DeferredList, merging it into one list and sending back to client. While merging works, the client has no results. What is the way to do tasks (db queries in my case) in parallel and then merge results? BTW, are nested runInteraction ok? Thanks for help! Pet class Test: def __init__(self, cursor): self.cursor = cursor def test(db, params): if params.get('query'): params['q'] = self.someFunc3(db, params) def processResults(results, out): #merge results into one for _, r in results: out.extend(r) return out out = [] d1 = self.cursor.runInteraction(self.someFunc, params) d2 = self.cursor.runInteraction(self.someFunc2, params) d = DeferredList([d1,d2]) d.addCallback(processResults, out) d.addErrback(log.err) return d class XMLRPCProtokoll(xmlrpc.XMLRPC): def __init__(self): self.db = adbapi.ConnectionPool() xmlrpc.XMLRPC.__init__(self, True) def xmlrpc_test(self, param): t = Test(self.db) return self.db.runInteraction(t.test, param)
On 02:41 pm, petshmidt@googlemail.com wrote:
Hello,
I'm trying to process results from several Deferreds (d1, d2) with DeferredList, merging it into one list and sending back to client.
You might like twisted.internet.defer.gatherResults.
While merging works, the client has no results. What is the way to do tasks (db queries in my case) in parallel and then merge results?
I don't think I understand this part of the question.
BTW, are nested runInteraction ok?
Not really. Twisted APIs are almost all required to be invoked in the reactor thread. Nested runInteraction calls would mean calling runInteraction in some other thread, and that's not allowed. Jean-Paul
Thanks for help! Pet
class Test: def __init__(self, cursor): self.cursor = cursor
def test(db, params): if params.get('query'): params['q'] = self.someFunc3(db, params)
def processResults(results, out): #merge results into one for _, r in results: out.extend(r) return out
out = [] d1 = self.cursor.runInteraction(self.someFunc, params) d2 = self.cursor.runInteraction(self.someFunc2, params) d = DeferredList([d1,d2]) d.addCallback(processResults, out) d.addErrback(log.err) return d
class XMLRPCProtokoll(xmlrpc.XMLRPC): def __init__(self): self.db = adbapi.ConnectionPool() xmlrpc.XMLRPC.__init__(self, True)
def xmlrpc_test(self, param): t = Test(self.db) return self.db.runInteraction(t.test, param)
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Wed, Jun 30, 2010 at 8:42 PM, <exarkun@twistedmatrix.com> wrote:
On 02:41 pm, petshmidt@googlemail.com wrote:
Hello,
I'm trying to process results from several Deferreds (d1, d2) with DeferredList, merging it into one list and sending back to client.
You might like twisted.internet.defer.gatherResults.
While merging works, the client has no results. What is the way to do tasks (db queries in my case) in parallel and then merge results?
I mean, running functions which performs db queries asynchronously and then merge results
I don't think I understand this part of the question.
BTW, are nested runInteraction ok?
Not really. Twisted APIs are almost all required to be invoked in the reactor thread. Nested runInteraction calls would mean calling runInteraction in some other thread, and that's not allowed.
That is why it not worked. I should redesign my application. Thanks!
Jean-Paul
Thanks for help! Pet
class Test: def __init__(self, cursor): self.cursor = cursor
def test(db, params): if params.get('query'): params['q'] = self.someFunc3(db, params)
def processResults(results, out): #merge results into one for _, r in results: out.extend(r) return out
out = [] d1 = self.cursor.runInteraction(self.someFunc, params) d2 = self.cursor.runInteraction(self.someFunc2, params) d = DeferredList([d1,d2]) d.addCallback(processResults, out) d.addErrback(log.err) return d
class XMLRPCProtokoll(xmlrpc.XMLRPC): def __init__(self): self.db = adbapi.ConnectionPool() xmlrpc.XMLRPC.__init__(self, True)
def xmlrpc_test(self, param): t = Test(self.db) return self.db.runInteraction(t.test, param)
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (2)
-
exarkun＠twistedmatrix.com -
Pet