Mailman 3 September 2004 - Twisted-web

Test
by Philippe Lafoucrière 06 Aug '21

06 Aug '21

Hi all just a test

3 2

How to get rid of web logs?
by Yun Mao 06 Jan '05

06 Jan '05

Right now I'm using the log for debugging, and I don't wan to log those info like below. How should I achieve this? Thanks! 2004/09/30 13:31 Eastern Daylight Time [HTTPChannel,1,127.0.0.1] 127.0.0.1 - - [ 30/Sep/2004:17:31:56 +0000] "GET /?view=all HTTP/1.1" 200 2767 "http://localhost :1083/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3" Yun

3 3

A more Twisted approach to async apps in WSGI
by Phillip J. Eby 06 Oct '04

06 Oct '04

Hi all. I've been away for a few days due to loss of e-mail service when my dedicated server lost a hard drive. Unfortunately my ISP didn't support the OS version any more, so I had to rebuild everything for the new OS version. Anyway, on to the topic of my post. Should 'wsgi.input' become an iterator? Or should we develop a different API for asynchronous applications? On the positive side of the iterator approach, it could make it easier for asynchronous applications to pause waiting for input, and it could in principle support "chunked" transfer encoding of the input stream. However, since we last discussed this, I did some Googling on CGI and chunked encoding. By far and away, the most popular links regarding chunked encoding and CGI, are all about bugs in IIS and Apache leading to various vulnerabilities when chunked encoding is used. :( Once you get past those items (e.g. by adding "-IIS -vulnerability" to your search), you then find *our* discussion here on the Web-SIG! Finally, digging further, I found some 1998 discussion from the IPP (Internet Printing Protocol!) mailing list about what HTTP/1.1 servers support chunked encoding for CGI and which don't. Anyway, the long and short of it is that CGI and chunked encoding are quite simply incompatible, which means that relying on its availability would be nonportable in a WSGI application anyway. That leaves the asynchronous use case, but the benefit is rather strained at that point. Many frameworks reuse the 'cgi' module's 'FieldStorage' class in order to parse browser input, and the 'cgi' module's implementation requires an object with a 'readline()' method. That means that if we switch from an input stream to an iterator, a lot of people are going to be trying to make sensible wrappers to convert the iterator back to an input stream, and that's just getting ridiculous, especially since in many cases the server or gateway has a file-like object to start with. So, I'm thinking we should shift the burden to an async-specific API. But, in this case, "burden" means that we get to give asynchronous apps an API much more suited to their use cases. Suppose that we did something similar to 'wsgi.file_wrapper'? That is, suppose we had an optional extension that a server could provide, to wrap specialized application object(s) in a fashion that then provides backward compatibility to the spec? That is, suppose we had a 'wsgi.async_wrapper', used like this: if 'wsgi.async_wrapper' in environ: controller=environ['wsgi.async_wrapper'](environ) # do stuff with controller, like register its # methods as callbacks return controller The idea is that this would create an iterator that the server/gateway could recognize as "special", similar to the file-wrapper trick. But, the object returned would provide an extra API for use by the asynchronous application, maybe something like: put(data) -- queue data for retrieval when the controller is iterated over finish() -- mark the iterator finished, so it raises StopIteration on_get(length,callback) -- call 'callback(data)' when 'length' bytes are available on 'wsgi.input' (but return immediately from the 'on_get()' call) While this API is an optional extension, it seems it would be closer to what some async fans wanted, and less of a kludge. It won't do away with the possibility that middleware might block waiting for input, of course, but when no middleware is present or the middleware isn't transforming the input stream, it should work out quite well. In any case, the implementation of the methods and the iterator interface are pretty straightforward, either for synchronous or asynchronous servers. What do y'all think? I'd especially like feedback from Twisted folk, as to whether this looks anything like the right kind of API for async apps. (I expect it will need some tweaking and tuning.) But if this is the overall right approach, I'd like to drop the current proposals to make 'wsgi.input' an iterator and add optional 'pause'/'resume' APIs, since they were rather kludgy compared to giving async apps their own mini-API for nonblocking I/O. Comments? Questions?

4 5

nevow/stackless bugs -- patch
by Peter Hunt 04 Oct '04

04 Oct '04

I've been trying to run stan on stackless when I ran into a problem with render methods. Stackless uses "stackless.instancemethod" instead of "__builtin__.instancemethod", and "stackless.function" instead of "__builtin__.function". Attached is a svn diff patch of the changes I made to __init__.py in order for stan to support stackless properly. HTH, Peter Hunt

3 3

need some help
by Bartek Bargiel 30 Sep '04

30 Sep '04

My question may sound weird but I have no ideas about it... If there are users of Internet Explorer and Mozilla Firebird around please use both browsers to enter my website at http://www.itabak.com/start Please pay attention to strange behaviour of IE. It seems to stop downloading site but suddenly it displays the whole page. There's no action like that in Firebird's case. I'm not sure if it's anything related to twisted.web or is it just IE page layouting problem. I'll be glad for all comments. BTW: thanks for Unicode help! Regards, -- Bartek Bargiel

1 0

Serving files from many web-servers thru one central web-server
by Thomas Weholt 30 Sep '04

30 Sep '04

Hi, I got a subnet full of web-servers using twisted and a main web-server based on twisted as well. The main server is available to the outside world. I want to serve files on the other web-servers on the subnet thru the main web-server. There may be many concurrent users on the main server so it has to be able to handle many clients. A few users connect to the main server, requesting files on the subnet web-servers. The main server reads data from several subnet servers and writes the data back to the requesting clients. How can I do this in twisted, without blocking, and handle several clients? We're not talking hardcore P2P here with thousands of clients, most likely 2-5 concurrent users, 10 at the most. Any hints? Or doesn't this make any sense? -- Mvh/Best regards, Thomas Weholt http://www.weholt.org

2 5

adding etag and modified arguments to twisted feedparser
by Selwyn McCracken 29 Sep '04

29 Sep '04

hi, I am having trouble modifying the twisted-based rss aggregator from the python cookbook so that feedparser can make use of the update related arguments of 'etag' and 'modified' to save bandwith. (see http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277099) I realise that the problem is deferred related, but I can't seem to resolve the problem, even after reading the deferred documentation. Anyway, the series of deferred functions that I think are relevant are: 1) def getPage(self, data, args): #args is the rss feed link return client.getPage(args,timeout=TIMEOUT) 2) def parseFeed(self, feed): parsed = feedparser.parse(cStringIO.StringIO(feed)) The problem is that getPage() requests the entire rss feed, and then passes the stream through to feedparser.parse. Normally however, feedparser.parse() takes furthers arguments of 'etag' and 'modified' so that only new feed information is returned, thereby saving bandwidth. I tried modifying getPage() to return feedparser.parse(args), and removing the need for parseFeed(), but it runs substantially slower than the original method, I presume in a synchronous manner. Any assistance in helping to restore the impressive parallel downloading performance, but with the the datetime arguments included, would be greatly appreciated. many thanks, Selwyn

3 5

Remembering things in context from a Realm
by Alex Levy 29 Sep '04

29 Sep '04

I have a Realm class that does something akin to the following: class MyRealm: def requestAvatar(self, id, mind, *interfaces): if IResource is in interfaces: user = figureOutWhoTheUserIs(id) resource = getResourceForUser(user) resource.remember(user, ICurrentUser) return (IResource, resource, lambda:None) ...and this doesn't seem to be working with the ubiquitous context. _Should_ it work, or do I need a new way to do something like this? Help is appreciated; I'm trying hard to get back in the loop. :/ -- Alex Levy WWW: http://mesozoic.geecs.org/ "Never let your sense of morals prevent you from doing what is right." -- Salvor Hardin, Isaac Asimov's _Foundation_

3 5

nevow 0.3 + encoding
by Bartek Bargiel 28 Sep '04

28 Sep '04

Hi! My first comment after having installed Nevow 0.3: my polish cp1250 encoding chars are not displayed: - htmlfile replaces them all with question marks (it works OK when using UTF-8 encoding) - xmlfile works fine when reading the text from file but again it fails to display national charset when it gets it returned from Python code Maybe it's me doing something wrong somewhere, I'm getting more&more confused with that encoding stuff :) Bartek Bargiel

2 1

Re: Nevow status
by Donovan Preston 28 Sep '04

28 Sep '04

On Sep 27, 2004, at 1:57 PM, Marc-Antoine Parent wrote: > I wrote some for the hell of it. Page patterns and children factories > are dead easy; working my way through the forms logic right now. My > only comment so far: I wish adding events did not require writing an > interface. (Or does it? Don't answer if you're busy, I'll figure it > out ;) > > I expected something like > class myPage(rend.Page): > def action_someAction(self, ctx, *kwargs): > .... > > so that I could call, in a simple href or form: > myPage?action=someAction&arg1=1&arg2=b > I know formless and/or liveEvil are better ways to do some of this, > but I like calling a RESTful interface from Javascript to transmit > data to the server. > I may just code it atop what is there, it looks easy enough ;-) > If there are deep reasons _not_ to do it, I'd like to know. I'm ccing this to the list to solicit more general feedback. I agree it should be easier and it's something I have been thinking about for a little while now. The main reason formless requires interfaces is because of the type annotation, coercion and error handling features of formless. If you are just submitting strings and don't care if they are valid or not it would be easy enough to do what you want to do: class ActionPage(rend.Page): def renderHTTP(self, ctx): action = ctx.arg('action') if action: args = IRequest(ctx).args.copy() del args['action'] return getattr(self, 'action_%s' % action)(ctx, **args) return rend.Page.renderHTTP(self, ctx) I might go ahead and add this to rend so people can use it if they want. The thing I like about this approach is that it is simple enough to get started quickly, since formless is a lot to swallow when you are learning the framework. I still think the formless interface approach is valid and useful however, because of all the things you get from it: * Form rendering * Error checking * Error form rendering * Type coercion * Method calling and property setting Really, formless is designed as the ultimate REST architecture, where form POSTs only mutate server side objects and are immediately redirected to a GET for that same object. But it is overkill for simpler tasks. Maybe there should be some sort of middle ground. dp

1 0

Twisted-web September 2004