Mailman 3 March 2008 - Twisted-web

Test
by Philippe Lafoucrière 05 Aug '21

05 Aug '21

Hi all just a test

3 2

Re: [Twisted-Python] disable twisted.web logging
by Phil Christensen 30 Mar '08

30 Mar '08

On Mar 29, 2008, at 1:39 AM, Pavel wrote: > Hi all, > > is there any way to disable twisted.web logging? Or at least > control it somehow, e.g. filter and log only necessary entries. > > Also, is there any way to make http requests be logged into > separate file? FYI, posts like this should probably go to the twisted-web mailing list. I'm not sure what level of control you'd like over the access log, but for simple redirection, it's straightforward. Here's how I build the Site object: myresource = MyResource() if(config['logfile'] != '-'): site = server.Site(myresource, logPath=config['logfile']) else: site = server.Site(myresource) This will let you pass either a log filename or /dev/null as a parameter. If you'd like more control (like logging different kinds of requests to different files), you'll need to get into the LogObserver stuff in twisted.python.log... I've only used t.p.log to create custom loggers, though, I'm not sure what's involved in filtering t.web's log messages.... Hope this helps, -phil

1 0

Bug in twisted.web2.client when receiving non-chunked responses of indeterminate length
by Andrew Warkentin 29 Mar '08

29 Mar '08

twisted.web2.client cuts responses short if they are not in chunked encoding and do not have a Content-Length header. I have attached a patch that fixes this bug. I am posting it to the list because I couldn't register on the site to post it to the bug tracker. diff -r -C 3 TwistedWeb2-svn20071201.orig/twisted/web2/channel/http.py TwistedWeb2-svn20071201/twisted/web2/channel/http.py *** TwistedWeb2-svn20071201.orig/twisted/web2/channel/http.py Sat Dec 1 04:10:51 2007 --- TwistedWeb2-svn20071201/twisted/web2/channel/http.py Wed Mar 26 03:33:09 2008 *************** *** 163,176 **** self.handleContentChunk(data[:self.length]) extraneous = data[self.length:] channel = self.channel # could go away from allContentReceived. ! if not self.chunkedIn: ! self.allContentReceived() ! else: ! # NOTE: in chunked mode, self.length is the size of the current chunk, ! # so we still have more to read. ! self.chunkedIn = 2 # Read next chunksize ! channel.setLineMode(extraneous) def headerReceived(self, line): """Store this header away. Check for too much header data --- 163,177 ---- self.handleContentChunk(data[:self.length]) extraneous = data[self.length:] channel = self.channel # could go away from allContentReceived. ! if not (self.length is None and not self.chunkedIn and self.parseCloseAsEnd): ! if not self.chunkedIn: ! self.allContentReceived() ! else: ! # NOTE: in chunked mode, self.length is the size of the current chunk, ! # so we still have more to read. ! self.chunkedIn = 2 # Read next chunksize ! channel.setLineMode(extraneous) def headerReceived(self, line): """Store this header away. Check for too much header data diff -r -C 3 TwistedWeb2-svn20071201.orig/twisted/web2/client/http.py TwistedWeb2-svn20071201/twisted/web2/client/http.py *** TwistedWeb2-svn20071201.orig/twisted/web2/client/http.py Sat Dec 1 04:11:46 2007 --- TwistedWeb2-svn20071201/twisted/web2/client/http.py Wed Mar 26 03:15:48 2008 *************** *** 8,13 **** --- 8,14 ---- from zope.interface import implements + from twisted.internet.error import ConnectionDone from twisted.internet.defer import Deferred from twisted.protocols.basic import LineReceiver from twisted.protocols.policies import TimeoutMixin *************** *** 165,171 **** self._error(ProtocolError(text)) def connectionLost(self, reason): ! self._error(reason) def gotInitialLine(self, initialLine): parts = initialLine.split(' ', 2) --- 166,177 ---- self._error(ProtocolError(text)) def connectionLost(self, reason): ! # If the response was non-chunked and of indeterminate length, treat the ! # connection being closed cleanly as the end of the response ! if self.length is None and not self.chunkedIn and reason.check(ConnectionDone): ! self.allContentReceived() ! else: ! self._error(reason) def gotInitialLine(self, initialLine): parts = initialLine.split(' ', 2)

3 5

[ANN] Twisted 8.0
by Christopher Armstrong 28 Mar '08

28 Mar '08

http://twistedmatrix.com/ MASSACHUSETTS (DP) -- Version 8.0 of the Twisted networking framework has been released, Twisted Matrix Laboratories announced Wednesday. Enslaved by his new robotic overloads, Master of the Release Christopher Armstrong presented the new package to the Internet on March 26th. Armstrong was unable to comment, because of a device worn around his neck preventing him from doing so, scientists say. Secretary of Defense Jean-Paul Calderone was asked about concerns that French interference may have played a role in the delay of this release. "I find such speculation preposterous. Thomas Hervé is an upstanding member of the Labs and his loyalties lie with us. He is a fine addition to our team." Rumors in the community allege that Secretary Calderone is holding Hervé's cat ransom until the release is successfully distributed. Hervé was unavailable for comment. This release comes shortly after the announcement by Chief of Public Affairs Duncan McGreggor that Twisted had joined the Software Freedom Conservancy. "We're happy to join the SFC, and we are now accepting sponsorship. The fact that we are now ruled by a cabal of robots should not put off potential donors. Our robotic overlords are running us at peak efficiency, so we can most effectively distribute The Love." Asked about the version number jump in this release, Commander-in-Chief Glyph Lefkowitz had the following to say: "Our benefactors have found our previous dice-rolling version number scheme to be inadequate, and have deigned to propose to us a more... logical system of versioning." ===== Twisted is an event-based framework for Internet applications which works on Python 2.3, 2.4, and 2.5. It can be downloaded from http://twistedmatrix.com/ Twisted 8.0 is a major feature release, with several new features and a great number of bug fixes. Some of the highlights follow. - The IOCP reactor is now much improved and many bugs have been resolved. - Twisted is now easy_installable. - Many improvements were made to Trial, Twisted's unit testing system. - A new memcache client protocol implementation was added. - So much more[1]! To see the full list of changes in its fifteen kilobytes of glory, see the release notes[1]. We welcome you to download and enjoy, and please file any bugs you find[2] and send comments to the mailing list[3]. Why the large version number bump? We've decided to switch to a time-based versioning scheme. "8.0" means the first release in 2008. [1] http://twistedmatrix.com/trac/browser/tags/releases/twisted-8.0.1/NEWS?form… [2] Register: http://twistedmatrix.com/trac/register New ticket: http://twistedmatrix.com/trac/newticket [3] http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python Thanks! -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/

8 7

regarding xml elements
by Raj kumar 28 Mar '08

28 Mar '08

Hi, I have an xml file in my application, I have created an element using Example goes like this......... document.createElement("abc") and i appeneded it by using append() method. But how i can reflect this change to my xml file? and one more thing is i want to create element with some other parameters.... like............. <abc m=" " n=" "> </abc> and i have m and n values as strings with me. can anybody help me to create this element and write it to the existing xml file as a child of an existing element? Thanks in advance.. Raj.. Forgot the famous last words? Access your message archive online at http://in.messenger.yahoo.com/webmessengerpromo.php

1 0

formatFailure recursion problems
by Phil Christensen 26 Mar '08

26 Mar '08

I've been using formatFailure recently to provide pretty tracebacks in my webapp, and I'm loving it, but I've run into a scenario where I've got a recursive data structure that occasionally pops up in tracebacks, and causes formatFailure all kinds of problems. Now, I can't really do much about the recursion itself, because it appears to be an unfortunate side effect of embedding Trac into a WSGI app I've written (the app just passes its environ variable to trac, acting as pseudo-transparent middleware). So, the real question is, is there anything that can be done to make formatFailure deal with recursion more gracefully? I'm currently using a extremely quick-and-dirty hack that just caches the id of each non-'primitive' object, and returns '==RECURSION==' when it encounters the id a second time. I guess what would be ideal is if it were possible could set some kind of depth limit for the htmlrepr() stuff, since most of the time I have no need to see that "deeply" into a local or global variable. I'm glad to work on the patch myself, but I was wondering if anyone has any suggestions as to how this should work. The current patch I'm using is attached below, but it's pretty overzealous about preventing recursion; if you output the object once anywhere in the entire traceback, it will never be output again, even in a non-recursive scenario. Anyways, any input would be appreciated... -phil Index: twisted/web/util.py =================================================================== --- twisted/web/util.py (revision 23090) +++ twisted/web/util.py (working copy) @@ -234,8 +234,13 @@ </style> """ - +repr_cache = None def htmlrepr(x): + if(repr_cache is not None and type(x) not in (str, int, long, float, unicode)): + if(id(x) in repr_cache): + return '==RECURSION==' + else: + repr_cache[id(x)] = 1 return htmlReprTypes.get(type(x), htmlUnknown)(x) def saferepr(x): @@ -304,7 +309,6 @@ return ret def formatFailure(myFailure): - exceptionHTML = """ <p class="error">%s: %s</p> """ @@ -325,6 +329,9 @@ <tr class="varRow"><td class="varName">%s</td><td class="varValue"> %s</td></tr> """ + global repr_cache + repr_cache = {} + if not isinstance(myFailure, failure.Failure): return html.PRE(str(myFailure)) io = StringIO() @@ -387,4 +394,5 @@ w(exceptionHTML % (html.escape(str(myFailure.type)), html.escape(str(myFailure.value)))) + repr_cache = None return io.getvalue()

1 0

Athena + formless?
by Harald Blåtand 21 Mar '08

21 Mar '08

Hi, Experimenting with forms in an Athena app. Have a web page with 2 panes (<div>s). Want to avoid page refresh. Started with own test code, like class LeftPane(athena.LiveFragment): def testPlainSubmitForm(self): f = form('formTest', 'Plain submit form', [T.textarea(name='textarea')['testing a plain form']], ['',], 'submit_form') self.callRemote('push_html_to_pane', f, u'rightPane') athena.expose(testPlainSubmitForm) This shows the form ok, but I get in trouble retrieving its data on submit. Trying to benefit from formless felt like a better idea, with the assumption that I could somehow catch the submit by subclassing nevow.rend.FreeformChildMixin, or something. My code would also do request = inevow.IRequest(ctx) request.code = 204 request.code_message = 'No Change' request.finish() to avoid a refresh. So, building on http://divmod.org/trac/attachment/wiki/DivmodNevow/FormHandling/Example1.1.… (q.v.), _except its last 2 lines_. My code now looks like - class LeftPane(athena.LiveFragment): def testFormLess(self): self.callRemote('push_html_to_pane', NewsEditPage(store=store), u'rightPane') athena.expose(testFormLess) Now I get an error in nevow.context.WebContext.locate - exceptions.KeyError: 'Interface formless.iformless.IConfigurableFactory was not remembered.' ..which I think comes from not using nevow.appserver.NevowSite, which would tack on the relevant context. (Of course, NevowSite is used in my "main class" - what I'm trying to do is "sneaking out of my code" - fetching user input - "getting back where I was".) Can't say I understand all that's going on, but I'm trying :) So, questions, por favor - Is this a good idea at all? Is it doable? Can I "attach a proper context" to that form, and how? (Gee, I don't even know how to phrase it correctly :( ) Thanks, k

1 0

proxy and https
by Vasil Vangelovski 19 Mar '08

19 Mar '08

I'm writing a simple content filtering and access restriction proxy using the twisted web proxy package. Here's some sample code: #!/usr/bin/env python from twisted.web import proxy, http import sys from twisted.python import log PROXY_PORT = 80 class FilterProxyClient(proxy.ProxyClient): def handleHeader(self, key, value): #SOME FILTERING CODE proxy.ProxyClient.handleHeader(self, key, value) def handleResponsePart(self, data): #SOME CONTENT RECOGNITION CODE proxy.ProxyClient.handleResponsePart(self, data) def handleResponseEnd(self): proxy.ProxyClient.handleResponseEnd(self) class FilterProxyFactory(proxy.ProxyClientFactory): def buildProtocol(self, addr): client = proxy.ProxyClientFactory.buildProtocol(self, addr) # upgrade proxy.proxyClient object to AdpusherProxyClient client.__class__ = FilterProxyClient return client class FilterProxyRequest(proxy.ProxyRequest): protocols = {'http': AdpusherProxyFactory} def __init__(self,*args): proxy.ProxyRequest.__init__(self, *args) def process(self): #SOME FILTERING CODE proxy.ProxyRequest.process(self) class FilterProxy(proxy.Proxy): def __init__(self): proxy.Proxy.__init__(self) def requestFactory(self, *args): return FilterProxyRequest( *args) class FilterProxyFactory(http.HTTPFactory): def __init__(self): http.HTTPFactory.__init__(self) def buildProtocol(self, addr): protocol = FilterProxy() return protocol if __name__ == "__main__": from twisted.internet import reactor prox = FilterProxyFactory() reactor.listenTCP(PROXY_PORT, prox) reactor.run( ) I'd like to know if there is a way I can set it up so that it also handles https traffic but doesn't do anythin with the content, just so that https traffic passes trough it.

1 0

Nevow-Stan How to save file?
by Suha Onay 13 Mar '08

13 Mar '08

Hi, I am a newbie of nevow. I want to open a save as dialog and enable the user to save files. I tried to use javascript with a href link but with no success. I write: """ ... file = open('fifo.txt', 'w') file.write('smth') file.close() return [ T.a(href="fifo.txt", onclick="saveFile('fifo.txt')")['Save'], webform.renderForms()[FORM_LAYOUT], ] """ How can I achieve in this? How can I give a link to a file in the server without nevow_carryover repling the request? Suha

7 10

Stuck twisted-web2 worker threads after client closes socket after sending data < content length of HTTP Post
by Kamal Choudhary 13 Mar '08

13 Mar '08

On my app server which is built upon twisted+web2 I have seen that at times some of worker threads will get blocked and won't get released ever. Eventually my app server runs out of worker threads and starts throwing message "503 Service not available" On more debugging I have found that I can easily recreate this situation by doing a incomplete HTTP Post with data less then content length specified in POST header. After this incomplete HTTP Post with lesser bytes of data if I close the client socket or kill client process or reboot client machine, in all these three cases it results in a hanged thread on my app server. This hanged threads remains on app server for ever till kill app server process, no time out of any kind. Further debugging into web2 code I found that stream.BufferedStream.readExactly doesn't come out of stream.BufferedStrem._readUntil even if I close client side socket. After putting some debug statements I found that self.stream.read() in stream.BufferedStream._readUntil doesn't return None as a signal for End Of File even after client has closed the socket connection. Since it doesn't get None it will never return with whatever data it has received so far from client. To reproduce this issue, I have written a minimal twisted-web2 server and a client that does a HTTP Post to it with incomplete data. After Post it closes the client socket and sleeps for sometime. Finally it prints the stack of all process threads, which clearly shows one blocked twisted worker thread. In the same program if you change it to do complete HTTP Post, in the thread dump there is no blocked twisted worker thread. Here is the stack traceback of blocked twisted worker thread. File "C:\Python24\lib\threading.py", line 442, in __bootstrap self.run() File "C:\Python24\lib\threading.py", line 422, in run self.__target(*self.__args, **self.__kwargs) File "/home/lotus/Desktop/pywork3.3/lm/foreign-common/src/twisted/python/threadpool.py", line 161, in _worker File "/home/lotus/Desktop/pywork3.3/lm/foreign-common/src/twisted/python/context.py", line 59, in callWithContext File "/home/lotus/Desktop/pywork3.3/lm/foreign-common/src/twisted/python/context.py", line 37, in callWithContext File "/home/lotus/Desktop/pywork3.3/lm/foreign-common/src/twisted/web2/wsgi.py", line 190, in run File "C:\work\twisted-head\twisted_server3.py", line 18, in dummy_app out_content = environ['wsgi.input'].read(int(environ['CONTENT_LENGTH'])) File "/home/lotus/Desktop/pywork3.3/lm/foreign-common/src/twisted/web2/wsgi.py", line 74, in read File "/home/lotus/Desktop/pywork3.3/lm/foreign-common/src/twisted/internet/threads.py", line 81, in blockingCallFromThread File "C:\Python24\lib\Queue.py", line 119, in get self.not_empty.wait() File "C:\Python24\lib\threading.py", line 203, in wait waiter.acquire() To run this program twisted, twisted.web2, zope and threadframe modules are required.

1 1