Re: [Twisted-Python] Asynchronous context in Twisted

March 15, 2011

      On Mar 3, 2011, at 2:39 PM, Glyph Lefkowitz wrote:
...
On Mar 3, 2011, at 7:31 AM, Fantix King wrote:
...
Hi,
I tried to make python.context work in asynchronous code between main
loops. Anyone has similar experience to share please?
Not sure if I am rebuilding a wheel :P
http://code.google.com/p/little-site/source/browse/littlesite/custom_reactor...
This is something I've often thought about doing in Twisted itself,
actually :).  But I wasn't sure that chaining context would actually do
anything practically useful most of the time.  Have you found that it's
actually useful?  Have you managed to leverage this to, for example, get
more informative error messages out of Deferred failures?
Doing it as a subclass like this is not optimal, as it limits you to one
reactor (and the Select reactor is not really the best one).  A wrapper
would be slightly more tricky (you'd have to deal with the places that the
reactor passes itself through to things like Process and Port, so you'd have
to create wrappers for those as well) but much more general.
Thanks for replying! :)

Yes! That's a wonderful idea to use this context for asynchronous traceback!
I made
some small changes to the code and wrote a patch for Twisted (as addReader
and
addWriter is quite different from one impl to another, I changed
SelectReactor only.
I haven't got a better idea for this, please advise), please see attachment.

With a simple example of raising exception in deferLater-ed function
(a-b-c-deferLater-d-e-f-g):

from twisted.internet import reactor
from twisted.internet.task import deferLater
reactor.usingAsyncTraceback = True

def g():
    raise Exception('Something happened inside.')

def f():
    return g()

def e():
    return f()

def d():
    return e()

def c():
    deferred = deferLater(reactor, 1, lambda: None)
    deferred.addCallback(lambda x: d())
    return deferred

def b():
    return c()

def a():
    return b()

if __name__ == '__main__':
    deferred = a()
    def errback(failure):
        failure.printTraceback()
    deferred.addErrback(errback)
    deferred.addBoth(lambda x: reactor.stop())
    reactor.run()

I could get this:

Traceback (most recent call last):
  File "test.py", line 31, in <module>
    deferred = a()
  File "test.py", line 28, in a
    return b()
  File "test.py", line 25, in b
    return c()
  File "test.py", line 20, in c
    deferred = deferLater(reactor, 1, lambda: None)
  File "/home/fantix/ac/twisted/internet/task.py", line 751, in deferLater
    delayedCall = clock.callLater(delay, d.callback, None)
  File "/home/fantix/ac/twisted/internet/base.py", line 701, in callLater
    _f, args, kw = self._chainContext(_f, args, kw)
*--- <asynchronous break point> ---*
  File "/home/fantix/ac/twisted/python/context.py", line 59, in
callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/home/fantix/ac/twisted/python/context.py", line 37, in
callWithContext
    return func(*args,**kw)
  File "/home/fantix/ac/twisted/internet/defer.py", line 361, in callback
    self._startRunCallbacks(result)
  File "/home/fantix/ac/twisted/internet/defer.py", line 455, in
_startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/home/fantix/ac/twisted/internet/defer.py", line 542, in
_runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "test.py", line 21, in <lambda>
    deferred.addCallback(lambda x: d())
  File "test.py", line 17, in d
    return e()
  File "test.py", line 14, in e
    return f()
  File "test.py", line 11, in f
    return g()
  File "test.py", line 8, in g
    raise Exception('Something happened inside.')

Additionally, in my scenario of a 5 years old asynchronous Twisted web
application, we
need the "request" object available throughout all code between asynchronous
network
accesses and database accesses because our global configuration system
needs the
request object. It would greatly reduce our manual work to pass through the
request
object here and there to have a context working in the asynchronous way.

BR,
Fantix.