Mailman 3 February 2007 - Twisted-web

Test
by Philippe Lafoucrière 05 Aug '21

05 Aug '21

Hi all just a test

3 2

https support for proxy server
by Gustavo Rahal 27 Feb '07

27 Feb '07

Hi I would like to know it's possible or simple to add https support to twistes.web proxy server. Thanks Gustavo

1 0

Reg adding etag and modified arguments to twisted feedparser
by mobiledreamers＠gmail.com 26 Feb '07

26 Feb '07

helo I m looking for the twisted rss aggregator with etags I m unable to find the client.getPageCached as this page doesnt exist http://www.twistedmatrix.com/users/roundup.twistd/twisted/issue612 How many concurrent connections can the rss aggregator hav max Thanks a lot for a great rss contrib Mark Valentino Volonghi dialtone at gmail.com Tue Sep 28 18:25:10 MDT 2004 Previous message: [Twisted-web] adding etag and modified arguments to twisted feedparser Next message: [Twisted-web] adding etag and modified arguments to twisted feedparser Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] On Wed, 29 Sep 2004 09:53:31 +1200, Selwyn McCracken <selwyn.mccracken at stonebow.otago.ac.nz> wrote: > hi, > > I am having trouble modifying the twisted-based rss aggregator from the > python cookbook so that feedparser can make use of the update related > arguments of 'etag' and 'modified' to save bandwith. > (see http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277099 ) Glad to see someone found that stuff useful :). I'm the author of that recipe. You will also be glad to know that I already solved your problems months ago and you can find the solution here: http://www.twistedmatrix.com/users/roundup.twistd/twisted/issue612 Unfortunately, when I asked for addition in twisted.web Itamar rejected because twisted.web already was on its way to deprecation. It should work without any problems, and you should use client.getPageCached() instead of client.getPage(). -- Valentino Volonghi aka Dialtone Linux User #310274, Proud Gentoo User Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/

2 1

validating nevow disk templates
by Phil Mayers 24 Feb '07

24 Feb '07

How do people do this? When I attempt to validate pages using the FireFox / Web Developer validator, I get a lot of complaints about tags that should be self-closed e.g.: <input type="text"> <nevow:attr name="name"><nevow:slot name="name"/></nevow:attr> </input> ...generates HTML like this: <input type="text" name="theName"> </input> ...which is not only ugly, it doesn't validate. I like stan, but I dislike the fact that it clutters up .py files and also that it doesn't reload like the disk templates. OTOH I deeply dislike the nevow:attr / nevow:slot pseudo-tags in the disk templates. Thoughts?

4 5

Re: [Twisted-web] Headers dictionary
by glyph＠divmod.com 21 Feb '07

21 Feb '07

On 01:13 pm, jarek.zgoda(a)gmail.com wrote: >Is it only me, or the field names in >twisted.web.http.Request.received_headers (also returned by >getAllHeaders()) are case-sensitive? It's only you. From twisted.web.http.Request: def headerReceived(self, line): """Do pre-processing (for content-length) and store this header away. """ header, data = line.split(':', 1) header = header.lower() ^^^^^ # ... def getHeader(self, key): """Get a header that was sent from the network. """ return self.received_headers.get(key.lower()) ^^^^^ Do you have a more specific example of what's gone wrong?

3 2

Headers dictionary
by Jarek Zgoda 21 Feb '07

21 Feb '07

Is it only me, or the field names in twisted.web.http.Request.received_headers (also returned by getAllHeaders()) are case-sensitive? The RFC states, that they should be case-insensitive (http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2), but if I set "Accept" header in client code (such as using Python's own httplib.HTTPConnection), there's no "accept" key in this dictionary, as available in resource's render_XXX method. Am I supposed to write my own HTTP compatibility layer for twisted.web, or I just overlooked something obvious? -- Jarek Zgoda http://jpa.berlios.de/

2 2

about macro and render
by Manlio Perillo 21 Feb '07

21 Feb '07

Hi. I want to compose a page, using a main template. A simple solution is to use macros class Resource(rend.rend): def macro_main(self, ctx): return loaders.xmlfile(...) The macro_main method is called only once. However it seems that macros are going to be removed, moreover if I change the child template, I have to reload the server. A solution is to use a render (code not tested): class Resource(rend.rend): template = ... # The child template to load def macro_main(self, ctx): return self.state.setdefault( template, loaders.xmlfile(self.template) ) Here I'm storing the loader instance in a dictionary, to optimize template loading. Is this solution as efficient as the one with macros? Thanks Manlio Perillo

2 4

[Nevow] childFactory being called more than once with macros
by Manlio Perillo 21 Feb '07

21 Feb '07

Hi. I'm having strange problems with childFactory and macros. My application uses a template with the main content renderized with a macro. <html> .... <n:invisible n:macro="main" /> ... </html> I have a base class: class Base(rend.rend): template = ... # The template to use as tha main content def macro_main(self, ctx): return loaders.xmlfile(self.template) Every page derives from Base, and defines its template. I do not understand why, but childFactory is called 2 times per request. If I replace the n:macro with a n:render, childFactory is called 3 times? Why is this happening? What is the best solution to compose a page from a main template? Thanks Manlio Perillo

2 4

web client
by Lorenzo 18 Feb '07

18 Feb '07

Hello, I'm using twisted to develop a small crawler. I need to download a large set of HTML pages/RSS feeds and I'd like to use the twisted's Web client. I ì've found some examples on the net, and I'm using mostly the one attached here http://twistedmatrix.com/bugs/issue1079 , but I'm having some problems. first of all I need the reactor to halt after all the files on the queue have been downloaded. Basically the sample uses a counter, that it updates every time it adds a new page to the list. d = getPage(address) d.addBoth(self.decConn) d.setTimeout(4, self.timeout) self.connections += 1 Everytime one of decConn or timeout has been fired I update the counter self.connections -= 1 When it becames <= 0 I stop the reactor. Basically this approach doesn't work, because the two functions are fired many times for some requests (I guess those are redirects...). So, how can I correctly check the number of completed requests and eventually stop the reactor? Is there any sample of a small crawler (also one that doesn't use twisted....) available on the net?? thanks LV ------------------------------------------------------ Passa a Infostrada. ADSL e Telefono senza limiti e senza canone Telecom http://click.libero.it/infostrada18feb07

1 0

Confused about XML-RPC Unicode behaviour (examples attached).
by kgi 09 Feb '07

09 Feb '07

Hi all. I've got a bunch of Twisted XML-RPC servers that serve up a wide range of data types include dates. Up until now I've had some hacky code to protect against bad data types being returned to the serialization layer (Nones, unicodes, etc). However, this code turns out to be difficult to maintain, as well as being a modest performance hit. I thought I see if I couldn't clean up some of the code to make it simpler and less hacky, and I found something rather odd: a Twisted XML-RPC server seemingly can't return a xmlrpc.DateTime object it has only just received from the deserialization layer. I've attached a test client, using Twisted, and two test servers, one using Twisted, one using the standard library's SimpleXMLRPCServer. The standard library version succeeds in returning the received xmlrpclib.DateTime unmolested; the Twisted version complains about the message containing unicode: File "/usr/lib/python2.4/site-packages/twisted/internet/abstract.py",line 173, in write raise TypeError("Data must not be unicode") exceptions.TypeError: Data must not be unicode This seems wrong: surely, at the very least, the serialization and deserialization should allow a round-trip unmodified? Put a slightly different way: I don't have a fundamental problem with the restriction that return types not include unicode, but surely in that case, the same system that enforces this should see to it that I don't receive such a type in the first place (given that the original data type I created in the client was a xmlrpclib.DateTime, which is explicitly designed for serialising over XML-RPC!). Are there technical/historical/political reasons for this of which I know nothing? Regards, Ricky

2 3