Twisted 100% event driven or hybrid?

[Mike C. Fletcher]

exarkun at twistedmatrix.com wrote:
> On 05:55 am, jacopo.pecci at gmail.com wrote:
...
>> results to be ready, I collect and process them. From now on I don 19t
>> need a the system to be event drive any more, the processing should
>> occur only on the master machine, following a deterministic flow.
>> As soon as finished I am ready to start again to resubmit the models
>> for recalculation and so on. This should go on forever.
Jean-Paul is obviously far more authoritative on the "twisted way" than
I am, so if he says you can just run your synchronous operation in-situ,
that's probably the way to go, but IIRC there's a
reactor.deferToThread() function which can run your synchronous code
"off to the side", while allowing the twisted code to continue to
process incoming operations.  Thus you'd do something like:

def process( dataset ):
    dl = [ remote_call( x ) for x in dataset]
    dl = defer.DeferredList( dl )
    def on_all_results( results ):
        reactor.deferToThread( sync_process, (results,)).addCallback(
process )
    return dl.addCallback( on_all_results )

(I'm typing all of that from the distance of a few years of memory
decay, so take it as "loosely this, with the proper function names and
the like").  Threads aren't really part of the "twisted way" in my
understanding, but they can be used if necessary AFAIK, and they will
let your application remain responsive to network events during the
processing.

HTH,
Mike

-- 
________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com