Mailman 3 January 2007 - Twisted

Re: [Twisted-Python] client, deferred and/or reactor hung on these 2 URL's
by Rick Graves 14 Jan '07

14 Jan '07

Eric, I ran the same file of almost 850 URL's again today, and again today, in the end the program was hung on the same two URL's. I tried the URL's with a program like WebCat.py from the Twisted book, which I also call WebCat.py. Like the one in the book, it uses client.getPage. It is different from the one in the book as follows: 1) The Errorback in the book did not work for me, so I wrote a simple one, and 2) I do not need to print the text of the page on the screen, my WebCat.py only prints the number of bytes received. If you want to take a look and/or try it yourself, I put it here: http://www.advanced-app.com.hk/MiscJunk/WebCat.py.tar.gz Those two URL's also hang my WebCat.py. WebCat.py does work with other URL's. > (Many ISPs have transparent proxies these days, *grrr*). I am definitely connecting through a proxy at the ISP. Around here, that is what you get. > If you can still reproduce then a traffic capture could be of use. How do I do that? Thanks, Rick

2 1

Re: [Twisted-Python] Ticket Hall of Shame - Current Loser: Brian Warner
by glyph＠divmod.com 14 Jan '07

14 Jan '07

On 04:01 pm, jacob(a)internet24.de wrote: >On Fri, 2007-01-12 at 13:14 +0000, glyph(a)divmod.com wrote: >> Although we should all be happy for the Twisted 2.5 release, we still >> have an unfortunate track-record as far as responsiveness to bugs. >A little bit more flexibility on what patches are added might help. If nobody has said anything about a ticket in 2 years with no patches on it, I don't see how accepting crappy patches is going to help. By "responsiveness" I meant responding to people who report bugs, and being clear on what we are vs. are not working on. The problem I am trying to address is best described here: http://www.jwz.org/doc/cadt.html By highlighting the least responsive developer (who, by the way, is Brian Warner) I hope to encourage people to keep their tickets up to date and respond when users report issues, even if the code in question has been rewritten. >If somebody posts a patch to a problem that is not yet >covered by a unit test (so that would be most patches) >then not absolutely insisting that the patcher provides a unit test as >well >could speed up things considerably. All the evidence I've seen (although I admit, there's not much of it) suggests that the opposite is true. Bugs have been fixed faster and better since the institution of these requirements. Most patches that get submitted without tests address a new ticket that was created specifically to hold the patch. Very few outside contributors seem to be interested in trying to fix _known_ problems with Twisted. Either they want a new feature or they want a new problem they've just discovered fixed. Those outside contributors who do contribute to existing tickets have generally become involved with the project enough to know that they should be developing fixes test-first. >If the developer responsible for >that module thinks the patch is plausible and possibly covers a way to >use twisted which >probably couldn't have worked anyway because of the reported problem, >there >is not all that much risk, is there? After all, the existing tests >should bark if the patch breaks anything else. As Mr. Armstrong already said, this idea doesn't make any sense. If anything, we need to require tests for existing, old functionality that wasn't completely tested before the patch was applied. >Twisted is really a very cool and effective tool, and that it can >actually be used in a stable production environment is definitely not >something I >want to miss, but maybe that is not so much due to a strict development >policy >as it is due to the skills of the core developers in spotting bogus >changes ;) How do you think the core developers got to be skilled in this way? By writing patches that don't have any tests or docs in them? :) >Just a thought... Thanks for trying to help with the suggestion. However, I can guarantee that as long as I am involved with the project, these requirements will continue to stay in place. You can continue talking about it if you like, but it will not change. >Maybe I got the wrong impression on just how things are done >by the twisted team. You seem to have a roughly correct impression. Tests and documentation are required before a patch will be accepted. >BTW, the patch in ticket #1794 is already included in 2.5, it probably >shouldn't be an active ticket anymore. Thanks for the heads up - it looks like Thomas Herve closed it in response.

3 3

[Twisted-Python] Re: trac / website woes
by Matthias Urlichs 14 Jan '07

14 Jan '07

Hi, Matthias Urlichs: > It seems that Deferreds are tricky. > Even more so -- apply on top of the last patch: --- /tmp/pb.py 2007-01-13 18:48:50.508925924 +0100 +++ /usr/lib/python2.4/site-packages/twisted/spread/pb.py 2007-01-13 19:20:15.000000000 +0100 @@ -795,6 +795,9 @@ }) return object + if not self.connected: + return deferred.fail(failure.Failure(DeadReferenceError())) + # XXX This call is NOT REENTRANT and testing for reentrancy is just # crazy, so it likely won't be. Don't ever write methods that call the # broker's serialize() method recursively (e.g. sending a method call -- Matthias Urlichs | {M:U} IT Design @ m-u-it.de | smurf(a)smurf.noris.de Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de - - "Though a program be but three lines long, someday it will have to be maintained." -- The Tao of Programming

1 0

[Twisted-Python] SSL
by Lee Connell 14 Jan '07

14 Jan '07

I am using SSL with LineReceiver on my server and it is working fine, however when I start the server it asks me for the PEM key. Is there a way to automate this so the server can start with out user intervention? Thanks! -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.432 / Virus Database: 268.16.10/624 - Release Date: 1/12/2007 2:04 PM

1 0

[Twisted-Python] Re: client crashed
by Rick Graves 14 Jan '07

14 Jan '07

Andrew, Thanks for the feedback. BTW, I am new to twisted but not to python. > Anyway, there's nothing Twisted specific here that I can see. I disagree -- I was trying to make a suggestion that would make future releases of twisted more bulletproof. Specifically, perhaps _parse() should be changed as follows (4 out of the last 5 lines are different): def _parse(url, defaultPort=None): parsed = urlparse.urlparse(url) scheme = parsed[0] path = urlparse.urlunparse(('','')+parsed[2:]) if defaultPort is None: if scheme == 'https': defaultPort = 443 else: defaultPort = 80 host, port = parsed[1], defaultPort if ':' in host: host, port = host.split(':') try: port = int(port) except ValueError: port = defaultPort return scheme, host, port, path But I will not change the copy of client.py on my computer. I worked around this problem by making sure all the URL's were clean before giving them to twisted. Going at this from the top down (or bottom up, depending on your perspective): lURLs = map( GetCleanURL, lURLs ) def GetCleanURL( sURL ): # return UrlUnSplitMore( UrlSplitMore( sURL ) ) UrlUnSplitMore and UrlSplitMore have longer lines which this mail client would wrap, and the result would look terrible. In case you are interested, they are in this plain text file: http://www.advanced-app.com.hk/MiscJunk/UrlSplitAndUnSplit.txt Maybe I am making a suggestion to change twisted in the wrong place. I will try posting my suggested change as a "bug" -- client.py crashes on a URL that most browsers clean up automatically with no problem. Thanks, Rick

3 2

[Twisted-Python] client, deferred and/or reactor hung on these 2 URL's
by Rick Graves 13 Jan '07

13 Jan '07

Hello, I am new to twisted. I was using client.downloadPage and reactor. Twisted was hung on the following two for a whole hour: http://75i.net/ http://www.theproxy.be/ I had to break out with Ctrl-C. When I did that, they finally fired the Errorback. These two stood out, as I had successfully (otherwise) run through a list of about 850 URL's. Is my reporting this helpful? Should I be reporting this elsewhere? Thanks, Rick Graves

4 3

[Twisted-Python] Twisted lineReceived (sender)
by Lee Connell 13 Jan '07

13 Jan '07

I just started with twisted and want to write a simple chat server. This code here works fine, however I want to know who the data is coming from in the callbacks or events. You can see where I commented code out in def lineReceived as to what I want to do. How can this be done in the easiest way? Thanks! #!/usr/bin/env python from twisted.protocols import basic class MyChat(basic.LineReceiver): def connectionMade(self): print "Got new client!" self.factory.clients.append(self) def connectionLost(self, reason): print "Lost a client!" self.factory.clients.remove(self) def lineReceived(self, line): print "received", repr(line) for c in self.factory.clients: #if c != sender: c.message(line) def message(self, message): if message != '': self.transport.write(message + '\n') from twisted.internet import protocol from twisted.application import service, internet factory = protocol.ServerFactory() factory.protocol = MyChat factory.clients = [] application = service.Application("chatserver") internet.TCPServer(1025, factory).setServiceParent(application) -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.432 / Virus Database: 268.16.10/624 - Release Date: 1/12/2007 2:04 PM

2 4

[Twisted-Python] documentation
by Lee Connell 13 Jan '07

13 Jan '07

Didn’t mean it was scarce, I just am use to MSDN, I’ve managed to find what I’ve needed so far. Thanks! -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.432 / Virus Database: 268.16.10/624 - Release Date: 1/12/2007 2:04 PM

1 0

[Twisted-Python] client crashed
by Rick Graves 13 Jan '07

13 Jan '07

Hello, I am new to twisted. I got one unhandled error (twice) in Deferred: In this code in client.py (you can skip to the 2nd line from the bottom): def _parse(url, defaultPort=None): parsed = urlparse.urlparse(url) scheme = parsed[0] path = urlparse.urlunparse(('','')+parsed[2:]) if defaultPort is None: if scheme == 'https': defaultPort = 443 else: defaultPort = 80 host, port = parsed[1], defaultPort if ':' in host: host, port = host.split(':') port = int(port) return scheme, host, port, path Here is the end of the traceback: File "/usr/lib/python2.4/site-packages/twisted/web/client.py", line 376, in _parse port = int(port) exceptions.ValueError: invalid literal for int(): I was running a script on a big list of URL's that I scraped off a google query. I looked at each URL including a colon, but nothing jumped out at me. Maybe the port = int(port) line should be within a try/except structure. Should I report this somewhere else? Thanks, Rick Graves

2 1

[Twisted-Python] Ticket Hall of Shame - Current Loser: Brian Warner
by glyph＠divmod.com 12 Jan '07

12 Jan '07

Although we should all be happy for the Twisted 2.5 release, we still have an unfortunate track-record as far as responsiveness to bugs. I encourage all readers of this list to have a look at lucky report 13: http://twistedmatrix.com/trac/report/13 Note that there are still high-priority tickets that haven't received even so much as an update since 2004. Paul Swartz has recently removed himself as the ultimate loser of this non-contest, and Brian Warner has replaced him. Don't want to be the biggest loser? Update your tickets before they get to be years old! Please keep in mind that "low" and "lowest" priority tickets don't currently participate, so if you've deemed something truly unimportant feel free to simply drop its priority. There it can safely languish for an eternity. Or at least until somebody changes the rules.

9 12