On Wed, 27 Apr 2005 05:49:01 -0700 (PDT), Joachim Boomberschloss <boomberschloss@yahoo.com> wrote:

Hi all,

I am just wondering: from the Twisted how-tos, it appears that doing anything that could possibly take arbitrarily long to execute should not be done in the reactor's main thread; i.e. it should be done using an asynchronous library (such as Twisted's network communication facilities), or in a different thread, using the reactor's thread pool.

So it would seem that many things that may be considered "primitive" in Python, such as file IO, require some kind of patching if they are to be immediately usable by a Twisted application. I came up with the following solution, which enables calling any function in a different thread with a deferred interface; I just wanted to make sure that I'm not completely missing some point:

def deferToThread(func, *args, **kargs): """executes the given function in a thread, and passes the return value to the deferred we return""" d = defer.Deferred() reactor.callInThread(_calledInThread, d, func, *args, **kargs) return d

def _calledInThread(d, func, *args, **kargs): try: retval = func(*args, **kargs) except Exception, x: reactor.callFromThread(d.errback, x) else: reactor.callFromThread(d.callback, retval)

Looks like you've got a pretty firm understanding. The one thing you did miss is the twisted.internet.threads module, which provides the deferToThread function. Jp

On Wed, 27 Apr 2005 05:49:01 -0700 (PDT), Joachim Boomberschloss <boomberschloss@yahoo.com> wrote:

Hi all,

I am just wondering: from the Twisted how-tos, it appears that doing anything that could possibly take arbitrarily long to execute should not be done in the reactor's main thread; i.e. it should be done using an asynchronous library (such as Twisted's network communication facilities), or in a different thread, using the reactor's thread pool.

So it would seem that many things that may be considered "primitive" in Python, such as file IO, require some kind of patching if they are to be immediately usable by a Twisted application. I came up with the following solution, which enables calling any function in a different thread with a deferred interface; I just wanted to make sure that I'm not completely missing some point:

def deferToThread(func, *args, **kargs): """executes the given function in a thread, and passes the return value to the deferred we return""" d = defer.Deferred() reactor.callInThread(_calledInThread, d, func, *args, **kargs) return d

def _calledInThread(d, func, *args, **kargs): try: retval = func(*args, **kargs) except Exception, x: reactor.callFromThread(d.errback, x) else: reactor.callFromThread(d.callback, retval)

Looks like you've got a pretty firm understanding. The one thing you did miss is the twisted.internet.threads module, which provides the deferToThread function. Jp