Mailman 3 September 2007 - Twisted

[Twisted-Python] Control the number of parallel system calls
by Daniel de la Cuesta 19 Sep '07

19 Sep '07

6 6

Re: [Twisted-Python] cannot access svn repo
by Terry Jones 19 Sep '07

19 Sep '07

Hi Nicholas | [nmattiello@megatron ~]$ svn co svn://svn.twistedmatrix.com/svn/Twisted/trunk This works fine for me too. | svn: Can't connect to host 'svn.twistedmatrix.com': Connection refused | [nmattiello@megatron ~]$ telnet svn.twistedmatrix.com 3690 | Trying 198.49.126.131... | telnet: connect to address 198.49.126.131: Connection refused | telnet: Unable to connect to remote host: Connection refused | | what could be wrong? Firewall? Terry

2 4

[Twisted-Python] cannot access svn repo
by Nicholas Mattiello 19 Sep '07

19 Sep '07

hello, I am trying to check out from the svn repository, but it keeps refusing me, even when i run the svn co command on the downloads page. svn co svn://svn.twistedmatrix.com/svn/Twisted/trunk svn: Can't connect to host 'svn.twistedmatrix.com': Connection refused any help would be appreciated, thanks.

3 4

[Twisted-Python] Transmit queue - how would I go at it?
by Nadav Aharony 16 Sep '07

16 Sep '07

Hi, I am working on an application where clients request files, and since each file is very large, the server ACKs the requests and then queues them to be served later, one at a time. I am relatively new to Twisted and was wondering what would be the best way to approach this. Here is what I came up with so far: 1) Use Deferred callbacks, and string together callbacks to a "file sender" function - once a file is sent, the function is called back with a pointer to the next file in line. If I do so - is there some automatic garbage collection of the callbacks that have already been handled? (there may eventually be hundreds of files to be sent) In this approach, would there be a natural way to check if a new request is actually a repeat of a previous request already queued? 2) Use some type of queue - where each new file request is queued, and some single function handles them one at at time (I guess with asynchronous callbacks to itself every time the transport layer is free to send another file and the queue is not empty). Is there some queue structure like this in Twisted? I saw something called a DefferedQueue but could not find many examples or documentation about it and about its usage. I mainly want to understand, from the "Twisted mindset" which would be the the right way to go? Also, say in the future I would want to implement priority queuing and handling of the requests rather than a FIFO - how would I do that in Twisted? Any thoughts and/or code examples would be greatly appreciated Thanks! Nadav

3 5

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 15 Sep '07

15 Sep '07

Bug summary ______________________ Summary for 2007-09-09 through 2007-09-16 Bugs opened: 6 Bugs closed: 5 Total open bugs: 1037 (+1) |== Type Changes |== Priority Changes |== Component Changes |Defect: +1 |Highest: -2 |Core: +0 |Normal: +3 |Mail: +1 |Names: -1 |Web2: +0 |Words: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2830] string exc_value special case in Failure.__init__ misses cases (opened by exarkun) (CLOSED, fixed) defect core http://twistedmatrix.com/trac/ticket/2830 [#2831] twisted.test.test_tcp.AddressTestCase.testBuildProtocol is buggy and overly complex (opened by exarkun) (CLOSED, fixed) defect core http://twistedmatrix.com/trac/ticket/2831 ===== Normal ===== [#2827] calling IConnector.disconnect() from a connectSSL does not disconnect if remote end is silent (opened by matthubb) defect core http://twistedmatrix.com/trac/ticket/2827 [#2828] xmlstream.IQ shouldn't create timeout deferreds for every IQ (opened by jack) defect words http://twistedmatrix.com/trac/ticket/2828 [#2829] twisted.web2.client.http, fixed (opened by synx) (CLOSED, invalid) defect web2 http://twistedmatrix.com/trac/ticket/2829 [#2832] twisted.mail.test.test_smtp.TimeoutTestCase.testSMTPClient fails intermittently due to timing issues (opened by exarkun) defect mail http://twistedmatrix.com/trac/ticket/2832 Closed Bugs ______________________ ===== Highest ===== [#1730] twisted.test.test_tcp.HalfClose3TestCase.testReadNotificationRaises fails intermittently (opened by radix, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/1730 [#2831] twisted.test.test_tcp.AddressTestCase.testBuildProtocol is buggy and overly complex (opened by exarkun, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/2831 [#2807] twisted.mail.relaymanager.MXCalculator._cbMX raises unhandled IndexError for response with no answers (opened by exarkun, closed by exarkun, fixed) defect names http://twistedmatrix.com/trac/ticket/2807 [#2830] string exc_value special case in Failure.__init__ misses cases (opened by exarkun, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/2830 ===== Normal ===== [#2829] twisted.web2.client.http, fixed (opened by synx, closed by glyph, invalid) defect web2 http://twistedmatrix.com/trac/ticket/2829 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2832] twisted.mail.test.test_smtp.TimeoutTestCase.testSMTPClient fails intermittently due to timing issues (since 2007-09-12 10:55:56). Mean open ticket age: 517 days, 14:21:27.445920. Median: 415 days, 23:27:03.233770. Standard deviation: 401 days, 0:49:47.008297. Interquartile range: 502 days, 16:56:16. Mean time between ticket creation and ticket resolution: 120 days, 22:26:58.101251. Median: 14 days, 8:13:35. Standard deviation is 224 days, 16:16:14.995707. The interquartile range is 127 days, 2:42:46. Mean time spent in review: 64 days, 4:15:36.993554. Median: 4 days, 0:03:10. Standard deviation: 141 days, 14:18:03.262941. Interquartile range: 29 days, 2:08:35. Mean number of times a ticket is reviewed: 1.93882352941. Median: 1 Standard deviation: 1.56762046109. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 20 unique ticket reporters 4 unique ticket reviewers 5 unique ticket resolvers In the last 24 weeks, 82 unique ticket reporters 13 unique ticket reviewers 14 unique ticket resolvers In the last 48 weeks, 157 unique ticket reporters 18 unique ticket reviewers 26 unique ticket resolvers

1 0

[Twisted-Python] XML-RPC Service with Database Access
by Stefan Arentz 15 Sep '07

15 Sep '07

I'm interested in using Twisted for a simple XML-RPC service that needs to talk to either PostgreSQL or MySQL. (I would prefer PG). I've got the XML-RPC part under control, that is fairly simple with Twisted Web. But what would I use for the database part? What is generally the easy way to do database access with Twisted? Are there ORM solutions that are compatible with Twisted? It would be nice if I could use something like SQLObject instead of raw queries. I'm not looking for a complete solution here. Just trying to understand what is usually done in Twisted land. S.

3 2

[Twisted-Python] Cometd?
by Stefan Arentz 15 Sep '07

15 Sep '07

Has anyone done any work in the area of cometd? I'm interested in using twisted for a cometd load tester. If there is already code that I can build on that would be great. S.

2 2

Re: [Twisted-Python] Multiple Twisted Application Plugins in the same directory?
by Beau Hargis 12 Sep '07

12 Sep '07

That indeed worked. It did occur to me for a fleeting moment when I looked at the man page; however, when I ran 'twistd --help-reactors' it spit out nothing so I was not sure of exactly what it wanted or if it worked at all. ----- Original Message ---- From: Christopher Armstrong <radix(a)twistedmatrix.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Wednesday, September 12, 2007 5:15:53 PM Subject: Re: [Twisted-Python] Multiple Twisted Application Plugins in the same directory? On 9/12/07, Beau Hargis <beau(a)subobscur.us> wrote: > For one of the services I want to use pollreactor or epollreactor, so each of the Services has only 4 lines in it: > > from twisted.internet import pollreactor > pollreactor.install() > import Project.Processors.ProcessorX > theService = Project.Processors.ProcessorX.ProcXServiceMaker() Don't install a reactor in the plugin: in fact, don't import the reactor at all. The correct way to choose the reactor is by passing the "-r" or "--reactor" argument to 'twistd', like 'twistd -r poll myplugin' -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/ _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

1 0

Re: [Twisted-Python] Multiple Twisted Application Plugins in the same directory?
by Beau Hargis 12 Sep '07

12 Sep '07

I am using the new twisted/plugin setup. I am not using mktap. So, in my project directory (which is included in the syspath) I have a dir 'twisted/plugins' into which I am going to put each of the services/applications. I do not want them running in the same process because I want to be able to stop one without affecting the others. Some of the services process a lot of data from 80-100 simultaneous connections so I need to be able to use multiple processors on the machine. The directory structure looks something like this: Project\ ...project.py ...Processors\ ......ProcessorX.py ......ProcessorY.py ...twisted\ ......plugins\ .........Service1.py .........Service2.py For one of the services I want to use pollreactor or epollreactor, so each of the Services has only 4 lines in it: from twisted.internet import pollreactor pollreactor.install() import Project.Processors.ProcessorX theService = Project.Processors.ProcessorX.ProcXServiceMaker() This all works great until I want to add a second service in the same manner. When I add the second service I get the error about the reactor already being installed. So 'twistd --help' will only list one of the services and scrolling up the screen one can see the error it spits out when it attempts to evaluate the second Service. If I start the first service, it finds it and then I comment out any reference to importing the reactor, then it will successfully evaluate the first AND second Service and allow me to start the second one. This is the error that it spits out: File "/usr/lib/python2.5/site-packages/twisted/internet/pollreactor.py", line 183, in install main.installReactor(p) File "/usr/lib/python2.5/site-packages/twisted/internet/main.py", line 24, in installReactor "reactor already installed" exceptions.AssertionError: reactor already installed And sometimes it will be a random failure (which is probably a cache interaction). It seems to be the combination of reactors. If I install the pollreactor in one and the epollreactor in the other then it works, but only one time. Then I check again and it fails. It seems to be loading up all the plugins in the same context. ----- Original Message ---- From: Jean-Paul Calderone <exarkun(a)divmod.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Wednesday, September 12, 2007 2:26:29 PM Subject: Re: [Twisted-Python] Multiple Twisted Application Plugins in the same directory? On Wed, 12 Sep 2007 14:06:50 -0700 (PDT), Beau Hargis <beau(a)subobscur.us> wrote: >I started creating my services as twisted application plugins. When adding a second service as a plugin in the same directory as the first I get an error that the reactor was already installed. If I comment out the import of the reactor in one then I can start the other application. Neither the tutorial nor the source indicate what is going on except that it checks to see if a reactor is already installed at some point. What is the nuance here? Should I just go back to .tac files for these apps? Could you describe what you're trying in a little more detail? In particular, I'm not sure if you're using mktap to create a tap file with multiple services, if you're using the original plugin system (using tml files) or if you're using the new plugin system (twisted.plugin). Generally speaking, there's no problem running more than one service in a single process. A service is just an object which receives startup and shutdown notification. Anything beyond that is an application-specific thing. Jean-Paul _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

2 1

[Twisted-Python] Multiple Twisted Application Plugins in the same directory?
by Beau Hargis 12 Sep '07

12 Sep '07

I started creating my services as twisted application plugins. When adding a second service as a plugin in the same directory as the first I get an error that the reactor was already installed. If I comment out the import of the reactor in one then I can start the other application. Neither the tutorial nor the source indicate what is going on except that it checks to see if a reactor is already installed at some point. What is the nuance here? Should I just go back to .tac files for these apps?

2 1