Mailman 3 January 2008 - Twisted

[Twisted-Python] connection pool
by Tjerk.Kusters＠imtech.nl Jan. 16, 2008

Jan. 16, 2008

Hello, I have build a webserver with Twisted 2.4 with imports of the following: from twisted.web import resource, static, server, twcgi from twisted.internet import reactor I'm adding several mods as child to the root with: root = static.File(settings.getSetting("html_directory")) root.putChild(mods.getPath(), mods The mods are classes like class module(resource.Resource): def render_GET(self, request): # do something The reactor is started as: reactor.listenTCP(_port, server.Site(root)) reactor.run( ) Some of the GET request in the mods can take some time to complete. In the current situation other request to the server have to wait on this request. Is it possible to make soms kind of a thread/connection pool so that more then one request can be handled at the same time? Regards, Tjerk

2 1

[Twisted-Python] Re: Twisted-Python Digest, Vol 46, Issue 10
by zechun huang Jan. 15, 2008

Jan. 15, 2008

收到 2008/1/13, twisted-python-request(a)twistedmatrix.com < twisted-python-request(a)twistedmatrix.com>: > > Send Twisted-Python mailing list submissions to > twisted-python(a)twistedmatrix.com > > To subscribe or unsubscribe via the World Wide Web, visit > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > or, via email, send a message with subject or body 'help' to > twisted-python-request(a)twistedmatrix.com > > You can reach the person managing the list at > twisted-python-owner(a)twistedmatrix.com > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Twisted-Python digest..." > > > Today's Topics: > > 1. problem with twcgi.PHPScript (Ottavio Campana) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 12 Jan 2008 17:52:25 +0100 > From: Ottavio Campana <ottavio(a)campana.vi.it> > Subject: [Twisted-Python] problem with twcgi.PHPScript > To: twisted-python(a)twistedmatrix.com > Message-ID: <20080112165225.GA13692(a)campana.vi.it> > Content-Type: text/plain; charset=us-ascii > > Hi, I'm trying to add the possibility of executing some php pages from > an application developed with twisted web. > > I define the VWGPhpScript class as follows, adding the REDIRECT_STATUS > because the debian package complains about it. > > class VWGPhpScript (twcgi.FilteredScript): > filter = '/usr/lib/cgi-bin/php5' > def runProcess(self, env, request, qargs=[]): > p = twcgi.CGIProcessProtocol(request) > env['REDIRECT_STATUS'] = '200' > reactor.spawnProcess(p, self.filename, [self.filename]+qargs, env, > os.path.dirname(self.filename)) > > This is the error I get: > > 2008/01/12 17:34 CET [-] CGI /directory/index.php exited with exit code 1 > 2008/01/12 17:34 CET [-] Errors from CGI /directory/index.php: Upon > execvpe /tmp/index.php ['/tmp/index.php'] in environment id -1217401212 > :Traceback (most recent call last): > File > "/usr/lib/python2.4/site-packages/twisted/internet/process.py", line 377, in > __init__ > command, args, environment) > File > "/usr/lib/python2.4/site-packages/twisted/internet/process.py", line 539, in > _execChild > os.execvpe(command, args, environment) > File "os.py", line 350, in execvpe > _execvpe(file, args, env) > File "os.py", line 367, in _execvpe > func(file, *argrest) > OSError: [Errno 13] Permission denied > > What's the meaning of permission denied? I tried running the application > as root, it didn't change... The php file is in /tmp and this directory > in attached as a child called directory of the root. > > > > ------------------------------ > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > > End of Twisted-Python Digest, Vol 46, Issue 10 > ********************************************** > -- http://sst.oicp.net/

2 1

[Twisted-Python] Where to download twisted for win32 / python 2.5?
by Richard van Paasen Jan. 15, 2008

Jan. 15, 2008

Hi, I tried to download the win32 package of twisted, but the download links on the twisted homepage (http://twistedmatrix.com/trac/) don't work. I tried to download: http://tmrc.mit.edu/mirror/twisted/Twisted/2.5/Twisted_NoDocs-2.5.0.win32-p… Where can I download a win32 version of twisted?

2 1

[Twisted-Python] deferredGenerator or inlineCallbacks
by Justin Warren Jan. 15, 2008

Jan. 15, 2008

Hi folks, I think I have a fundamental misunderstanding of how deferredGenerator (or inlineCallbacks for Python2.5) work. What I *want* to do, is something like this: aValue = MyObject.get_a_value() where the contents of get_a_value() do some asynchronous operations which finally result in some kind of actual value being returned (a string, and integer, whatever) rather than a Deferred. I have some old blocking code that I want to migrate to use some asynchronous, twisted code that does the same thing, but I'd prefer to not have to write a bunch of: def get_a_value_success(retval): ... def get_a_value_failed(retval): ... d = MyObject.get_a_value() d.addCallback(get_a_value_success) d.addErrback(get_a_value_failed) code. Is this even possible, or am I completely misunderstanding what's going on here? I can get the code to do the @defer.deferredGenerator, waitForDeferred(), wfd.getResult() dance.. but how do I actually return a value from my function after doing wfd.getResult() ? -- Justin Warren <daedalus(a)eigenmagic.com>

3 5

[Twisted-Python] Conch and basic.LineReceiver
by Vincent Bernat Jan. 14, 2008

Jan. 14, 2008

Hi ! I have modified sshsimpleserver.py to use a protocol based on basic.LineReceiver. However, I get the following exception: 2008/01/14 19:40 +0200 [SSHChannel session (0) on SSHService ssh-connection on SSHServerTransport,0,127.0.0.1] Unhandled Error Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib/python2.4/site-packages/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "/usr/lib/python2.4/site-packages/twisted/conch/ssh/service.py", line 42, in packetReceived f(packet) File "/usr/lib/python2.4/site-packages/twisted/conch/ssh/connection.py", line 139, in ssh_CHANNEL_DATA log.callWithLogger(channel, channel.dataReceived, data) --- <exception caught here> --- File "/usr/lib/python2.4/site-packages/twisted/python/log.py", line 48, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/lib/python2.4/site-packages/twisted/python/log.py", line 33, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/lib/python2.4/site-packages/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib/python2.4/site-packages/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "/usr/lib/python2.4/site-packages/twisted/conch/ssh/session.py", line 107, in dataReceived self.client.transport.write(data) File "/usr/lib/python2.4/site-packages/twisted/conch/ssh/session.py", line 156, in write self.proto.dataReceived(data) File "/usr/lib/python2.4/site-packages/twisted/protocols/basic.py", line 233, in dataReceived if why or self.transport and self.transport.disconnecting: exceptions.AttributeError: SSHSessionProcessProtocol instance has no attribute 'disconnecting' How may I fix this? I have attached my modified sshsimpleserver.py. Thanks.

1 1

[Twisted-Python] problem with twcgi.PHPScript
by Ottavio Campana Jan. 13, 2008

Jan. 13, 2008

Hi, I'm trying to add the possibility of executing some php pages from an application developed with twisted web. I define the VWGPhpScript class as follows, adding the REDIRECT_STATUS because the debian package complains about it. class VWGPhpScript (twcgi.FilteredScript): filter = '/usr/lib/cgi-bin/php5' def runProcess(self, env, request, qargs=[]): p = twcgi.CGIProcessProtocol(request) env['REDIRECT_STATUS'] = '200' reactor.spawnProcess(p, self.filename, [self.filename]+qargs, env, os.path.dirname(self.filename)) This is the error I get: 2008/01/12 17:34 CET [-] CGI /directory/index.php exited with exit code 1 2008/01/12 17:34 CET [-] Errors from CGI /directory/index.php: Upon execvpe /tmp/index.php ['/tmp/index.php'] in environment id -1217401212 :Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/twisted/internet/process.py", line 377, in __init__ command, args, environment) File "/usr/lib/python2.4/site-packages/twisted/internet/process.py", line 539, in _execChild os.execvpe(command, args, environment) File "os.py", line 350, in execvpe _execvpe(file, args, env) File "os.py", line 367, in _execvpe func(file, *argrest) OSError: [Errno 13] Permission denied What's the meaning of permission denied? I tried running the application as root, it didn't change... The php file is in /tmp and this directory in attached as a child called directory of the root.

2 2

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Jan. 12, 2008

Jan. 12, 2008

1 0

[Twisted-Python] epollreactor for pyicq-0.6-t
by Steven ZHOU Jan. 10, 2008

Jan. 10, 2008

Hi, I hope some one could offer me some help on this. I am quite new to both python and twisted. The thing I do may sounds a bit wierd. I am using a pyicq-0.6-t. As pyicq-0.6-t only works with twisted-2.2 or earlier, so I chose to install twisted-2.2. However, I wished to use epollreactor as the default selectreactor only supports 1024 file descriptors. So I copied the relevant files of epollreactor which is installed on twisted-2.5 to the site-packages/twisted-2.2 directory. I copied 5 files in the total, epollreactor.py, epoll..pyx to the twisted/internet directory, _epoll.py _epoll.pyc and _epoll.so to twisted/python directory. I can't remember the exact file names. I copied it with a hope that it works. I am not sure whether I am doing thing stupid here. correct me if I am. The thing is, after I changed the reactor.py to import epollreactor, pyicq-0.6-t seems to be using epollreactor but it still can only supports 1024 open sockets. I thought the problem is only in selectreactor? I figured this out by doing a netstat -a -n on linux and found that pyicq-0.6-t only opened about 1017 maximum concurrent sockets. I presume that would mean only around 1000 connections can be made. Although strange, but I do have a reason to use pyicq-0.6-t which is an outdated version. thanks a lot. regards, Steven Notice The information in this message is confidential and may be legally privileged. It is intended solely for the addressee. Access to this message by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying or distribution of the message, or any action taken by you in reliance on it, is prohibited and may be unlawful. If you have received this message in error, please delete it and contact the sender immediately. Thank you.

4 5

[Twisted-Python] Performance and select.select
by Nitro Jan. 10, 2008

Jan. 10, 2008

Hello, I profiled my application with cProfiler. Below is an excerpt from the resulting report: Fri Jan 04 01:07:39 2008 profiling_data_cProfile 15123197 function calls (14579262 primitive calls) in 228.517 CPU seconds Ordered by: internal time List reduced from 2933 to 50 due to restriction <50> ncalls tottime percall cumtime percall filename:lineno(function) 138305 135.419 0.001 135.817 0.001 {select.select} 26168 13.159 0.001 14.044 0.001 Sources\Server\Plugins\RKSimulation.py:218(UpdatePhysics) As you can see select.select takes up ~136 of ~229 seconds. This is the most expensive function. The row below shows the next expensive function which is one of my own. Now I am wondering whether a) the select calls are really taking up that much time (more than 50%!) b) if they are taking up that much time I'd like to find out how to reduce their execution time (either by minimizing the number it gets called or the time it takes to execute etc) My application doesn't invoke select.select itself so it is most likely called within twisted. I am on Windows and using the default reactor ("from twisted.internet import reactor"). -Matthias

3 5

[Twisted-Python] twisted gprs
by John Aherne Jan. 10, 2008

Jan. 10, 2008

About a year ago we had a look at twisted to do some work with gprs. I got some useful feedback from that. However, it's only now we have had a chance to move things forward. We have done some simple work with wm5/wm6 devices sending GPS data back to a twisted server. And we have done various tests sending/receiving messages over gprs to see what problems we need to overcome. One problem we have right now to which the solution may be obvious but not to us is :-- We have a problem at the server end when we put the pda into 'flight mode'. Thie means that the wireless is switched off and the gprs connection goes away. However, the server does not see this connection go away since the network routers keep the connection up, as far as we can see. In various tests we have the connection take from 2 - 7 mins before a disconnection happens. As far as I can see using 'ethereal', the server decides itself to drop the connection since I do not see a 'FIN' be sent from anywhere. On the server I have set a loopingCall running every 20 seconds, so that I can switch off various parts to see the results. With the pda not connected, the loopingCall keeps on going and sends between 2 -4 messages without getting an ack back from the pda. Then the server stops trying to send any data. The loopingCall keeps going but no data goes out onto the wire. We get upto 16 - 40 loop messages being sent but only the 2-4 go onto the wire. What I would like to be able to do is to not wait 2-7 minutes before realising the connection is not working. I have been through the docs and the api but can't find anything that I can use. Is there any call I can make use of to be able to detect the failed send before the connection finally is broken. Thanks for any help on this. John Aherne

3 4