Mailman 3 August 2010 - Twisted

[Twisted-Python] spawn wsgi resource into separate process
by Thomas Johnson Aug. 29, 2010

Aug. 29, 2010

In the following code I am attempting to host several django applications in vhosts - http://pastie.org/1122806 . This code works fine if I only need one settings module. In my case each vhost needs it's own(lines 32-38) DJANGO_SETTINGS_MODULE environment variable. It appears that I need to use reactor.spawnProcess to spawn a new process with the required environment variable. Can anyone give me a few suggestions as to how to accomplish this task? I think that it involves writing a custom protocol that would communicate with the wsgi process but there may be better options available. Any suggestions would be greatly appreciated.

2 2

[Twisted-Python] git clone with spawnProcess() can prevent socket data flow, any clue why?
by Martin Nordholts Aug. 29, 2010

Aug. 29, 2010

Hi I am getting a weird problem where git cloning with reactor.spawnProcess() fails for some repositories. It appears as if Twisted blocks reading from the server socket that git packs should come on. I originally got the problem with buildbot, but I have has isolated the problem further to Twisted. The following program can be used to reproduce the problem: #!/usr/bin/python from twisted.internet import reactor, protocol class ProcessPrinter(protocol.ProcessProtocol): def connectionMade(self): print "::connectionMade" def outReceived(self, data): print "::outReceived" print data def errReceived(self, data): print "::errReceived" print data def processEnded(self, status_object): print "::processEnded" print "exit code %d" % status_object.value.exitCode reactor.stop() def spawn_after_run(): argv = ['/usr/bin/git', 'clone', 'git://git.gnome.org/gimp'] reactor.spawnProcess(ProcessPrinter(), argv[0], argv) reactor.callLater(0, spawn_after_run); reactor.run(); If a debugger is attached to the spawned git process, it can be seen that the process blocks on read() on the socket that is supposed to be fed with git packs from the server. I have done a tcpdump of the traffic between my computer and the git server, and there *is* successful communication going on. More specifically, the git client receives the refs the server has, and sends the refs it wants, which is acknowledged by the server, but for some reason it still eventually blocks on read(). Curiously enough, if the program is changed to clone e.g. 'git://git.gnome.org/gegl' instead of 'git://git.gnome.org/gimp', it works! The GIMP repo is much bigger than the GEGL one, which might be the trigger for the bug, but I don't understand why it would matter. I see this problem on Fedora 13 x86_64, both with the distro version Twisted 0.8.2, and with the latest stable release Twisted 10.1.0. My personal guess is that it has something to do with how spawnProcess() manages the fds or the child process. Does anyone have any guess of what is going on? Can you reproduce it on other systems? Thanks in advance for any input. Regards, Martin

5 13

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Aug. 29, 2010

Aug. 29, 2010

1 0

[Twisted-Python] Check out my photos on Facebook
by Raja Pavan Aug. 28, 2010

Aug. 28, 2010

Hi Twisted, I set up a Facebook profile where I can post my pictures, videos and events and I want to add you as a friend so you can see it. First, you need to join Facebook! Once you join, you can also create your own profile. Thanks, Raja To sign up for Facebook, follow the link below: http://www.facebook.com/p.php?i=100001226497896&k=Z6E3Y6UYSZW1WKBJPB63QTWTP… Already have an account? Add this email address to your account: http://www.facebook.com/n/?merge_accounts.php&e=twisted-python%40twistedmat… ======================================= twisted-python(a)twistedmatrix.com was invited to join Facebook by Raja Pavan. If you do not wish to receive this type of email from Facebook in the future, please follow the link below to unsubscribe. http://www.facebook.com/o.php?k=da0a54&u=100000673873684&mid=2e2c902G5af338… Facebook, Inc. P.O. Box 10005, Palo Alto, CA 94303

1 0

Re: [Twisted-Python] help with Deferreds
by Terry Jones Aug. 26, 2010

Aug. 26, 2010

Hi Sergey > def checkObsolete(item): > if item.obsolete(): > return None > else: > d = item.read() > d.addBoth(release, item) > d.addCallback(renew) > return d > > d = cache.open(key) > d.addCallbacks(checkObsolete, notFound) > return d > > Hi Terry > If item.obsolete() raise exception item remain not released. > Additional try ... except ... safe us but looks ugly Yes, I was expecting you to wrap item.obsolete() and renew(data) in try/except, as needed. If you want a clean but slightly slower way to do that, how about something like: def getCachedResult(cache, key): def release(result, item): item.release() return result def renew(data): return renew(data) def notFound(fail): fail.trap(NotFound) def readItem(item): return item.read() def checkObsolete(item): d = defer.maybeDeferred(item.obsolete) d.addCallback(lambda _: readItem(item)) d.addBoth(release, item) d.addCallback(renew) return d d = cache.open(key) d.addCallbacks(checkObsolete, notFound) return d

2 1

[Twisted-Python] help with Deferreds
by Sergey Magafurov Aug. 26, 2010

Aug. 26, 2010

How to rewrite this regular function with Deferred-style? def getCachedResult(cache, key): try: item = cache.open(key) except NotFound: return None try: if item.obsolete(): return None data = item.read() finally: item.release() result = renew(data) return result Conditions: 1. 'cache.open' and 'item.read' become deffered 2. 'item.obsolete()' and 'renew(data)' are not deferred

2 4

Re: [Twisted-Python] help with Deferreds
by Terry Jones Aug. 26, 2010

Aug. 26, 2010

Hi Sergey > How to rewrite this regular function with Deferred-style? Something like the below (untested) will do the job. You'd have to make sure that cache.open returned a deferred that fails with NotFound in case things don't work out. It would be simpler to translate your code using inlineCallbacks, but it might be to your advantage to first see how to do it without that. If you do take that approach you might want to do some timing tests. Terry --- def getCachedResult(cache, key): def release(result, item): item.release() return result def renew(data): return renew(data) def notFound(fail): fail.trap(NotFound) def checkObsolete(item): if item.obsolete(): return None else: d = item.read() d.addBoth(release, item) d.addCallback(renew) return d d = cache.open(key) d.addCallbacks(checkObsolete, notFound) return d

2 1

[Twisted-Python] Twisted with wxPython in a thread
by Gabriel Rossetti Aug. 24, 2010

Aug. 24, 2010

Hello, I can't find any examples of using Twisted with wxPython, were wx is in a seperate thread. I used to have one somewhere but can' t find it either. I'm looking because for some reason my code doesn't work (the GUI freezes), even though I use reactor.callFromThread() for Twisted calls from wx and wx.CallAfter() for wx calls from Twisted. Can anyone point me to an example? Thank you, Gabriel

4 10

Re: [Twisted-Python] Twisted with wxPython in a thread
by Gabriel Rossetti Aug. 24, 2010

Aug. 24, 2010

On 08/24/2010 04:37 PM, Stephen Waterbury wrote: > On 08/24/2010 10:08 AM, Stephen Waterbury wrote: >> On 08/24/2010 09:24 AM, Stephen Waterbury wrote: >>> On 08/24/2010 09:22 AM, Gabriel Rossetti wrote: >>>> Ok, so you put Twisted in a thread? I read in several posts that this >>>> posed problems, have you had any drawbacks? >>> >>> I haven't observed any. >> >> Reflecting on this a little further (and with the caveat that I >> am a complete ignoramus on threading), it seems to me that >> problems happen when Twisted is not in the main thread. Note >> that in the example, the reactor starts first (i.e., before the >> wx event loop), so I guess it is in the main thread ... I haven't >> tried switching them, but if you do that or anything that breaks >> this method, please let me know -- as I say, I haven't seen it >> break (freeze or otherwise malfunction) yet. > > BTW, Gabriel (sorry about spamming you about this -- this is the > last thing I'll say, I promise! ;), I think what I said above is > completely wrong: the example code *is* running the reactor in a > daemon thread -- it sets it explicitly using 'setDaemon(True)'. > Anyway, if you get it to break, please let me know -- thanks! > > Steve no worries Steve, it's cool of you to look into this. I'll give it a try if I can't get the wx-in-a-thread code to work by tomorrow. I just want to check some things first to make sure it's not another problem hiding. Thanks for your help, I'll keep you up to date! Gabriel

1 0

Re: [Twisted-Python] Twisted with wxPython in a thread
by Andrew Francis Aug. 24, 2010

Aug. 24, 2010

Hi >I can't find any examples of using Twisted with wxPython, were wx is in >a seperate thread. I used to have one somewhere but can' t find it >either. I'm looking because for some reason my code doesn't work (the >GUI freezes), even though I use reactor.callFromThread() for Twisted >calls from wx and wx.CallAfter() for wx calls from Twisted. Can anyone >point me to an example? This may be off topic but isn't there a wxReactor() so that you can avoid a separate thread? Cheers, Andrew

2 1