Mailman 3 April 2004 - Twisted

Re: [Twisted-Python] Re: [Twisted-commits] r10475 - Use writeSequence instead of silly for-loop.
by exarkun＠divmod.com 15 Apr '04

15 Apr '04

On Thu, 15 Apr 2004 13:27:56 -0400, James Y Knight <foom(a)fuhm.net> wrote: > > On Apr 5, 2004, at 10:36 AM, Andrew Bennetts wrote: > > > This is of course quite silly, and I suspect a nice performance > enhancement could be achieved by making FileDescriptor.dataBuffer be a > list of strings, rather than a single string. That would both cause > multiple writes in a row to not allocate new strings, and allow > writeSequence to be implemented sanely. Been there :) FileDescriptor.dataBuffer was a list when I first started hacking around in Twisted's internals. The current implementation is the result of an optimization of that code, and is somewhat (not earth-shatteringly) faster. > > Doesn't look very hard really; looks like you'd only need to touch > write/writeSequence/doWrite. This'd also have the advantage of making > it easier to put a Sendfile-struct in there as well. So pahan's > volunteered to fix this! There are a few other places that need to be touched, but it is certainly doable. The main problem is simply that tcp.py is such a mess. Jp

1 0

[Twisted-Python] Re: [Twisted-commits] r10475 - Use writeSequence instead of silly for-loop.
by James Y Knight 15 Apr '04

15 Apr '04

On Apr 5, 2004, at 10:36 AM, Andrew Bennetts wrote: > Author: spiv > Date: Mon Apr 5 08:36:43 2004 > New Revision: 10475 > > Modified: > trunk/twisted/protocols/http.py > Log: > Use writeSequence instead of silly for-loop. You might not have noticed that writeSequence is implemented like so: self.write("".join(iovec)) thus negating the performance enhancement recently added by not copying large data into a new string. This is of course quite silly, and I suspect a nice performance enhancement could be achieved by making FileDescriptor.dataBuffer be a list of strings, rather than a single string. That would both cause multiple writes in a row to not allocate new strings, and allow writeSequence to be implemented sanely. Doesn't look very hard really; looks like you'd only need to touch write/writeSequence/doWrite. This'd also have the advantage of making it easier to put a Sendfile-struct in there as well. So pahan's volunteered to fix this! Unfortunately, python doesn't expose the writev syscall, which means that you depend on Nagling to not coalesce the TCP packets. There *is* a TCP option which you can use to help fix too many small packets: TCP_CORK on linux and TCP_NOPUSH on FreeBSD. This keeps partial packets from being sent at all until you close the connection or turn off the option (except on older freebsd, where it's broken and you need to turn off the option and call write again to wake it up). I suspect it's worthwhile to set TCP_CORK/TCP_NOPUSH while the python-side buffer has any data left in it, or else the kernel may be deciding to send out small packets when it should be waiting for more data from an upcoming write call and wasting time/bandwidth. Python has the TCP_CORK constant, even, yay! FreeBSD will just have to suffer, I suppose, since python doesn't know about TCP_NOPUSH. James

4 4

[Twisted-Python] twisted and interactive/command driven input
by Ellers 15 Apr '04

15 Apr '04

Dear all, I am implementing a pseudo-shell application in python that communicates with a database-like server. I have been tinkering with twisted for a few weeks and am impressed - I could make a client and imitate the server with a small amount of clean python code! However, I'm not sure how to use twisted (or if it can be done) to do the kind of application I want to do. Specifically, the application will take input from the user or from a file, just like a normal shell, then communicate with the server behind this. I'm sure this is a very well known problem, but I'm not sure where to start - threads? twisted? select-loop? In my app the user will see something like this: myshell$ select * from data results: id title 123 Foo 456 Bar myshell$ and so on. But after typing the select statement, THATS when I want to utilize the power of twisted to send a buffer to the server, then wait for the response and unpack it. To complicate things, the server sends async announcments of change events, e.g. new rows in the database. I want to be able to handle these while still receiving user input (eg print them on the console) I've got a basic interactive shell going on its own no probs, and as mentioned I've got a separate basic twisted client going too. Can anyone point me at examples or tips on how to use twisted for handling the socket comms and still allow interactivity with the user? Thanks - and hope this isn't a really silly question! Ellers

1 0

[Twisted-Python] addCallbacks issue
by Sorin C. 14 Apr '04

14 Apr '04

Hi everyone, I'm using PB in my application and I have problems with addCallbacks call, it's not working as expected, at least what I expect. The following snippet: factory = pb.PBClientFactory() reactor.connectTCP('localhost', 2174, factory) d = factory.getRootObject() d.addCallbacks(self.gotRoot, self.netError) when an error occures self.netError doesn't get called, instead I get something like: "Unhandled error in Deferred:" which I assume is an error message telling me that the deffered is not bound to an error handler. Now if I replace the last line like this: factory = pb.PBClientFactory() reactor.connectTCP('localhost', 2174, factory) d = factory.getRootObject() d.addCallback(self.gotRoot) d.addErrback(self.netError) the error handler gets called and everything is ok. Any idea why addCallbacks fails to bound the error handler? I use Twisted 1.2.0. Thanks, Sorin __________________________________ Do you Yahoo!? Yahoo! Tax Center - File online by April 15th http://taxes.yahoo.com/filing.html

5 9

[Twisted-Python] Status of twisted.xish.xpath
by Valentino Volonghi aka Dialtone 14 Apr '04

14 Apr '04

I was wondering what's the status of the xpath parser in Twisted. I need to write a feed aggregator which does not uses the Mark Pilgrim's parser, since that's a dependency and because that's not a really good piece of software IMHO (it's hard to read and debug). Just look at what has been done here: http://xoomer.virgilio.it/dialtone/rsschannel.py # written by mg from Nevow with Nevow and the current xish.xpath, and now look at how this has been done in the feedparser from Pilgrim. I wonder what could be done with a complete implementation of XPath at developers disposal. There are plenty feeds versions to support like Atom, CDF, RDF and 9 rss versions. Also Nevow would be greatly enanched by an updated xpath implementation. I would like to know if it's actively developed, or if it's 'deprecated' in favour of external xpath implementations. I would really like to avoid telling my clients to install 4Suite or libxml2 or anything else. Just using Twisted would be perfect since it's just 1.6Mb of portable python code (and in future it will be split in various packages, so it's even smaller). Thx for your efforts! -- Valentino Volonghi aka Dialtone Linux User #310274, Gentoo Proud User Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/

5 6

[Twisted-Python] PythonCard GUI client for pbecho example
by Stephen C. Waterbury 14 Apr '04

14 Apr '04

Here's a PythonCard client that works with the pbecho.py server (I included pbecho.py so it's a self-contained example, but it's the same pbecho.py as in Twisted/doc/examples). Dependencies: PythonCard (http://pythoncard.sourceforge.net) (requires wxPython) Basically, all the logic of tpcpbechoclient was copied directly from pbechoclient.py -- the only functional difference is the capability to repeat the "Send" function. It's pretty self-explanatory, but anyway: * start up server: python pbecho.py * start up client: python tpcpbechoclient.py * when client comes up, click "Login" button -> "CONNECTED" should appear on lower right side of panel (meaning the client received a perspective) * enter something into the text field at the bottom * click "Send" button -> the text should be echoed in the text area widget above (Note: if you have PythonCard installed as PythonCardPrototype, either change the name of your installation to PythonCard or do the substitution in my example code. PythonCard is about to undergo the name change in its next released version. If you use the CVS version, it's already re-named.) Enjoy! - Steve P.S. Itamar: I couldn't get the wxreactor to work with PythonCard; kept getting "AssertionError: reactor already installed", although I followed the directions in wxreactor's docstring. I tried various permutations but finally got tired of messing with it ... let me know if have time to play with it and find a way that works! For now, I'm using my "twistedpythoncard" module, which encapsulates Uwe's timer recipe so the rest of the code can use normal Twisted and PythonCard idioms. That seems to work pretty well, but I will be trying some more non-trivial examples before I declare success.

2 1

[Twisted-Python] [Fwd: [boston-pig] Possible summer internship]
by Itamar Shtull-Trauring 12 Apr '04

12 Apr '04

-- Itamar Shtull-Trauring http://itamarst.org Looking for a job -- http://itamarst.org/resume.html

2 1

[Twisted-Python] RFC: UDP's transport.write() accepting hostnames
by Itamar Shtull-Trauring 12 Apr '04

12 Apr '04

I'd appreciate comments added to the issue - http://www.twistedmatrix.com/users/roundup.twistd/twisted/issue586 -- Itamar Shtull-Trauring http://itamarst.org Looking for a job -- http://itamarst.org/resume.html

1 0

Re: [Twisted-Python] POP3 Client
by Ofir Reichenberg 12 Apr '04

12 Apr '04

>From: <exarkun(a)divmod.com> >Subject: Re: [Twisted-Python] POP3 Client > > I've been trying to build a simple POP3 client using Twisted and have had no > > luck. > > I'm looking for something that retrieves the list of headers waiting on the > > server. > > There is a much nicer POP3 client class in Quotient (which needs to be folded back into Twisted ... when I get the time). I highly recommend using it instead: >http://divmod.org/cvs/trunk/quotient/proto/pop3client.py?rev=2718&root=Quotient&view=markup This looks exactly like what I need. Thanks a lot. > > > > > It appears that to do that I need to build a POP3 factory based on a domain. > > How can I build a domain ojbect for a "real" internet domain? (as opposed to > > a local server) > > I'm not sure what you mean here. What you want to do is something along these lines: > > from twisted.internet import protocol > from wherever import POP3Client > > class MyPOP3Client(POP3Client): > # Override some callbacks, like connectionMade or perhaps serverGreeting > > f = protocol.ClientFactory() > f.protocol = MyPOP3Client > > from twisted.internet import reactor > reactor.connectTCP('internet.host.name', 110, f) > reactor.run() > > The above can be run just like any other Python program (it won't work, of course, as it is an incomplete example). As alternatives to the last three lines, you can use twisted.application and the utility program "twistd" instead. > Hmmm, I see what you mean. I didn't know of the ClientFactory class. This was the answer to my question, but I'll give Quotient a try first. :) Thanks again, Ofir _________________________________________________________________ STOP MORE SPAM with the new MSN 8 and get 2 months FREE* http://join.msn.com/?page=features/junkmail

1 0

[Twisted-Python] PythonCard GUI client for pbecho example
by Stephen Waterbury 12 Apr '04

12 Apr '04

Here's a gui client made with PythonCard that works with the pbecho.py server. How it works: * start up server: python pbecho.py * start up client: python tpcpbechoclient.py * when client comes up, push "Login" -> "CONNECTED" should appear on lower right side of panel (meaning the client received a perspective) * enter something into the text field at the bottom * click "Send" -> the text should be echoed in the text area widget above (Note: if you have PythonCard installed as PythonCardPrototype, either change the name of your installation to PythonCard or do the substitution in my example code. PythonCard is about to undergo the name change in its next released version.) Enjoy! - Steve P.S. Itamar: I couldn't get the wxreactor to work with PythonCard; kept getting "AssertionError: reactor already installed". I tried a lot of things, but finally got tired of messing with it. For now, I'm using my "twistedpythoncard" module, which encapsulates Uwe's timer recipe so the rest of the code can use normal Twisted and PythonCard idioms.

1 0

Twisted April 2004