Mailman 3 February 2005 - Twisted

[Twisted-Python] twisted threading server has to be stopped with KILL
by snacktime 10 Feb '05

10 Feb '05

I have a server that uses threads, and when run standalone or via twistd as an application I have to send it a KILL signal if it has already answered at least one request. Or in some cases if I just send it an INT it will stop if it's currently handling a lot of requests at the time I send it the signal. Seems like there is something I am not doing here that I should be when using threads? Chris

3 9

[Twisted-Python] You are made active
by Stephen 09 Feb '05

09 Feb '05

1 0

[Twisted-Python] HTTP POST with getPage
by snacktime 09 Feb '05

09 Feb '05

Is there a way to get at the HTTPClientFactory instance variables using getPage, or do I need to create my own factory that inherits HTTPClientFactory to get at them? Chris

2 1

[Twisted-Python] re-entrancy question
by apocalypznow 09 Feb '05

09 Feb '05

I've got a Twisted server application that also listens for other types of TCP connections on another port. Upon receiving certain data on that other port, I make calls to objects subclassed from pb.Avatar. Given that Twisted runs on one loop doing select(), I think I have a re-entrancy situation. Is this: 1) dangerous? 2) why is it dangerous? 3) and how would I go about achieving what I want safely?

3 3

Re: [Twisted-Python] 2.0.0 pre-pre-release
by Christopher Armstrong 08 Feb '05

08 Feb '05

On Mon, 7 Feb 2005 18:31:37 +0100, Michal Chruszcz <troll(a)pld-linux.org> wrote: > It seems like on Sunday 06 of February 2005 23:41, Christopher Armstrong > typed: > > Yeah, I might accept the "backwards compatibility with existing > > dependencies" argument, but I don't think it's something that's > > necessary for general use. In a post-split world, deb packages that > > use Twisted stuff will depend on specific twisted subproject packages. > > Anyway, I'm not the one doing the deb packages, but there are my two > > cents. > > You must remember that splitting into several packages from one tarball is > much simpler than doing so using several different .spec files (each for > one tarball) or bearing with many Sources in one big .spec. Moreover one > tarball allows packagers to create -doc and -examples packages containing > such things for whole Twisted. What are you talking about? I am definitely going to be producing a sumo tarball. I was talking about *deb* packages up there. > Personally I am a PLD Linux developer and until now (the one-tarball-distro > version of Twisted 1.3) there are separate packages for docs, examples, ssl > and some other, which I actually don't know what are they for and, believe > me, managing packaging Twisted from one tarball plus splitting different > parts into separate RPMs is really simpler than managing e.g. ten > different .spec's. I know that because Xfce is distributed in dozens of > tarballs, each has its own .spec file and both upgrading and reorganization > of Xfce is a bit annoying. > Summarizing, no, I'm not against changes. I'm only trying to insist on > distributing Twisted also in one "gimme-all" tarball. ;-) As I've already said in both the split FAQ and on the mailing list multiple times I'm going to be producing both big sumo tarballs *and* per-project tarballs. You can package however you want based on these. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | -- http://radix.twistedmatrix.com | Release Manager, Twisted Project \\\V/// | -- http://twistedmatrix.com |o O| | Founding Member, Hobart Hacking Society w----v----w-+ -- http://hackingsociety.org/chapters/hash

1 0

Re: [Twisted-Python] segfaults with deferToThread
by snacktime 08 Feb '05

08 Feb '05

Ok I narrowed this down to the pycrypto package. Specifically the rsa encrypt function. I'll file a bug report with pycrypto. Chris

1 0

[Twisted-Python] segfaults with deferToThread
by snacktime 08 Feb '05

08 Feb '05

This is on Freebsd 5.3-release-p2 with python 2.4. The script below calls a blocking method using deferToThread. When I have 5 or so clients all hitting it at the same time I get a segfault or bus error within minutes. I recompiled python with --with-pydebug and set OPT to -g, but it didn't give me any more information in the backtrace which is also below. Then I ran gdb on the server process and something funny happened. I didn't get a segfault, but the server started catching exceptions and printing them to the screen. As soon as I detached gdb the exceptions stop but it dumps core a minute later (or less). The exception is pasted below also. Python was compiled with pthreads, which is the freebsd kse threads as far as I know. Anyone have an idea what is going on? Server code: ----------------------------------------------------------- from twisted.internet.protocol import Protocol, Factory from twisted.internet import defer, reactor from twisted.python import threadable from twisted.internet import threads threadable.init(1) from otransact.otdirect import process from twisted.python import log class GetData: # this calls my blocking method. THe method blocks when doing database calls with psycopg and when using urllib2 to do an https POST. Other than that everything is non blocking. psycopg is supposed to be thread safe, and I would assume urllib2 is also. def Do(self,data): response = process(data) return response ### Protocol Implementation # This is just about the simplest possible protocol class Echo(Protocol): def dataReceived(self, data): """As soon as any data is received, write it back.""" reactor.callLater(0, self.Start,data) def PrintData(self,data): self.transport.write("%s\r\n" % data) self.transport.loseConnection() #reactor.stop() def Start(self,data): c = GetData() d = threads.deferToThread(c.Do,data) d.addCallback(self.PrintData) d.addErrback(log.err) def main(): f = Factory() f.protocol = Echo reactor.listenTCP(8000, f) reactor.run() if __name__ == '__main__': main() Exception raised by server when gdb was running. ----------------------------------------------------------------------- File "otssl.py", line 42, in ? reactor.run() File "/usr/local/lib/python2.4/site-packages/twisted/internet/default.py", line 126, in run self.mainLoop() File "/usr/local/lib/python2.4/site-packages/twisted/internet/default.py", line 134, in mainLoop self.runUntilCurrent() --- <exception caught here> --- File "/usr/local/lib/python2.4/site-packages/twisted/internet/base.py", line 423, in runUntilCurrent call.func(*call.args, **call.kw) File "/usr/home/otransact/ssl/ots.py", line 72, in Start d = threads.deferToThread(c.Do,data) File "/usr/local/lib/python2.4/site-packages/twisted/internet/threads.py", line 48, in deferToThread reactor.callInThread(_putResultInDeferred, d, f, args, kwargs) File "/usr/local/lib/python2.4/site-packages/twisted/internet/base.py", line 453, in callInThread self.threadpool.callInThread(_callable, *args, **kwargs) File "/usr/local/lib/python2.4/site-packages/twisted/python/threadpool.py", line 130, in callInThread self.q.put(o) File "/usr/local/lib/python2.4/Queue.py", line 91, in put self.not_full.release() thread.error: release unlocked lock Backtrace: ------------------------------------------------ #0 0x28286dbb in pthread_testcancel () from /usr/lib/libpthread.so.1 #1 0x2827eb06 in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 #2 0x00000000 in ?? ()

2 1

[Twisted-Python] 2.0.0 pre-pre-release
by Christopher Armstrong 08 Feb '05

08 Feb '05

I've thrown together some tarballs with my latest release automation code. You can download them here: http://twistedmatrix.com/users/radix/2.0/ Please download the packages relevant to you and give them a test. You'll need to install Twisted-2.0.0a1 before any of the other subproject packages in there, and it depends on zope.interface being installed, which I haven't bothered including yet. If anybody's interested in making packages for their OS, now's the time to start working on it. Thanks! -- Twisted | Christopher Armstrong: International Man of Twistery Radix | -- http://radix.twistedmatrix.com | Release Manager, Twisted Project \\\V/// | -- http://twistedmatrix.com |o O| | Founding Member, Hobart Hacking Society w----v----w-+ -- http://hackingsociety.org/chapters/hash

9 12

[Twisted-Python] Twisted Sprint: Hobart Australia, 1-3 April 2005
by Mary Gardiner 07 Feb '05

07 Feb '05

Hey folks, The dates for the Australian Twisted sprint have been confirmed. Dates: Friday 1 April 2005 - Sunday 3 April 2005 Location: Thanks to Nunatak Systems, Chris Armstrong, Jonathan Lange and Timothy Stebbing we've got locations in Sandy Bay, Hobart, Tasmania, Australia. See the wiki link below for more info. The sprint is going to be fairly informal: no attendance fees, no formal registration system. The flipside is that you'll be responsible for organising travel, accommodation and meals. While we haven't decided on a firm goal or goals for the sprint, we'll be working on code destined for the TM Labs projects (that is, the Twisted subversion respository). I've set up a temporary wiki for notes about the sprint, including topics, venues and accommodation options: http://users.puzzling.org/users/mary/TwistedSprint/moin.cgi/FrontPage If you're going to come along, please let us know ASAP, both by replying to this mail (off list, note that this list has a Reply-To header) and by adding your name to http://users.puzzling.org/users/mary/TwistedSprint/moin.cgi/SprintAttendees (I want people to email me as well so that I have contact details for everyone!). -Mary

1 0

Re: [Twisted-Python] Persistence in browsers?
by Maciej Szumocki 07 Feb '05

07 Feb '05

----- Original Message ----- > Date: Sat, 5 Feb 2005 03:02:58 +0100 > From: "Tibi Dondera" <incoming(a)pronet-romania.com> > Subject: [Twisted-Python] Persistence in browsers? > Short version: > I have 10.000 - 100.000 web browsers that are connected to my site, and > I need to inform them __real-time__ (a max of 3-5 seconds delay) of an > event that happened on the server. Is twisted the right way to go, given > the fact that it promises asynchronous event handling ? > [snip] Why not just use UDP if number of hanging connections is a problem? Maciej Szumocki

1 0