Mailman 3 January 2008 - Twisted-web

Test
by Philippe Lafoucrière Aug. 5, 2021

Aug. 5, 2021

Hi all just a test

3 2

How to use the HTTPClientFactory connect one time and get more than 1000 page?
by Cheney Lee Jan. 23, 2008

Jan. 23, 2008

Hi, it is my first time use twisted . i want to use the a function pass a url then get the web page the code as : // *some code call getPage* *while id <= 10000000: getPage("**http://www.mywebsite.com/News.aspx?ID="+str(id*<http://www.mywebsite.com/News.aspx?ID="+str(id> *)) id += 1 *//****************************************** *the getPage is definition in twisted.web.client* *def getPage(url, contextFactory=None, *args, **kwargs): """Download a web page as a string.* * Download a page. Return a deferred, which will callback with a page (as a string) or errback with a description of the error.* * See HTTPClientFactory to see what extra args can be passed. """ scheme, host, port, path = _parse(url) factory = HTTPClientFactory(url, *args, **kwargs) if scheme == 'https': from twisted.internet import ssl if contextFactory is None: contextFactory = ssl.ClientContextFactory() reactor.connectSSL(host, port, factory, contextFactory) else: reactor.connectTCP(host, port, factory) return factory.deferred* -------------------------------------------------------------------------------- *Question: *for the getPage function,if use it to get 10000 page ,it would open/close connection 10000 times,it is a very large cost。 So ,any body can give me some advice?creat a class inherit from HTTPPageGetter(as a protocol class) or HTTPClientFactory?

1 0

connection pool
by Tjerk.Kusters＠imtech.nl Jan. 17, 2008

Jan. 17, 2008

Hello, I have build a webserver with Twisted 2.4 with imports of the following: from twisted.web import resource, static, server, twcgi from twisted.internet import reactor I'm adding several mods as child to the root with: root = static.File(settings.getSetting("html_directory")) root.putChild(mods.getPath(), mods The mods are classes like class module(resource.Resource): def render_GET(self, request): # do something The reactor is started as: reactor.listenTCP(_port, server.Site(root)) reactor.run( ) Some of the GET request in the mods can take some time to complete. In the current situation other request to the server have to wait on this request. Is it possible to make soms kind of a thread/connection pool so that more then one request can be handled at the same time? Regards, Tjerk

2 1

ssl problems
by Phil Christensen Jan. 16, 2008

Jan. 16, 2008

Hello all, I'm trying to configure an SSL-enabled twisted.web service, and I'm having a problem getting it to work with my certificate chain file. I'm trying to replace a similar configuration on an Apache server, so I know my certificates/keys are valid and current, and they work fine under mod_ssl. The issue seems to be with the chain file. If I just include the primary cert and key, the connection works properly, but of course displays an 'unknown root certificate' warning. This is the ContextFactory I'm using: class ContextFactory: isClient = 0 def getContext(self): ctx = SSL.Context(SSL.SSLv23_METHOD) ctx.use_certificate_file('/usr/local/dram/certs/shib.crt') ctx.use_privatekey_file('/usr/local/dram/certs/www.key') ctx.use_certificate_chain_file('/usr/local/dram/certs/ intermediate.crt') return ctx I've attached a minimal test case, but there's really nothing of consequence besides the class above. Using s_client, I got the following debug output: optimus:dram2 phil$ openssl s_client -connect shib.dramonline.org: 443 -debug -state -nbio 2>&1 CONNECTED(00000003) turning on non blocking io SSL_connect:before/connect initialization write to 0020BE30 [00127000] (118 bytes => 118 (0x76)) 0000 - 80 74 01 03 01 00 4b 00-00 00 20 00 00 39 00 00 .t....K... ..9.. 0010 - 38 00 00 35 00 00 16 00-00 13 00 00 0a 07 00 c0 8..5............ 0020 - 00 00 33 00 00 32 00 00-2f 03 00 80 00 00 05 00 .. 3..2../....... 0030 - 00 04 01 00 80 00 00 15-00 00 12 00 00 09 06 00 ................ 0040 - 40 00 00 14 00 00 11 00-00 08 00 00 06 04 00 80 @............... 0050 - 00 00 03 02 00 80 ec 36-f1 ee 1c 4e 29 1e 5d 3c ....... 6...N).]< 0060 - 82 c8 19 76 7d b8 85 94-a0 59 62 67 da 5a 69 7f ...v}....Ybg.Zi. 0070 - 2b 62 68 b3 c7 5e +bh..^ SSL_connect:SSLv2/v3 write client hello A read from 0020BE30 [0012D000] (7 bytes => -1 (0xFFFFFFFF)) SSL_connect:error in SSLv2/v3 read server hello A write R BLOCK read from 0020BE30 [0012D000] (7 bytes => 7 (0x7)) 0000 - 15 03 01 00 02 02 28 ......( SSL3 alert read:fatal:handshake failure SSL_connect:error in SSLv2/v3 read server hello A 5765:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:596: As I said before, I find this strange because I have a working SSL config already in Apache, and it's relatively simple: <VirtualHost 69.60.xxx.xxx:443> ServerName shib.dramonline.org DocumentRoot /var/www/html SSLEngine on SSLProtocol all -TLSv1 SSLCertificateFile /usr/local/dram/certs/shib.crt SSLCertificateKeyFile /usr/local/dram/certs/www.key SSLCACertificateFile /usr/local/dram/certs/intermediate.crt SetEnvIf User-Agent ".*MSIE.*" nokeepalive \ ssl-unclean-shutdown downgrade-1.0 force-response-1.0 </VirtualHost> This may be something I need to take to the py/OpenSSL folks, but I wanted to check here first to make sure I wasn't missing something obvious. I've tried disabling various SSL protocols (using context.set_options()) and sometimes it seems like it gets a little farther in the process, but Thanks in advance for any help, -phil

1 0

return the full (absolute) request URI
by Manlio Perillo Jan. 15, 2008

Jan. 15, 2008

Hi. I need to do a redirect to the same URI the client requested, however it seems that Twisted Web has no support for this. request.prePathURL() is an absolute URI, but is not good and request.uri is not an absolute URI (an absolute URI is required in the Location header http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.30) Do I need to write a fullPathURL function by myself? Thanks Manlio Perillo

4 4

HTTP basic auth
by Vincent Bernat Jan. 8, 2008

Jan. 8, 2008

Hi ! I would like to implement some authentication mechanism on top of an existing nevow site using HTTP basic auth for a web services. I have looked at guard.py from nevow but is seems to be far more complex and I don't quite understand if it uses HTTP basic auth. I am not interested in setting up sessions or other stateful mechanisms. Since authorization may be required for POST or something like that, I would like to stick to simple HTTP auth basic since simple clients may not handle correctly redirection on POST. twisted.web2 authentication seems to be a good start, but I am using Nevow. Has someone already done a HTTP basic auth with nevow? In fact, my case is very similar to the one here: http://twistedmatrix.com/pipermail/twisted-web/2007-January/003270.html But, as I said, Nevow's guard.py seems best suited for web app authentication, not for web services. Can anyone confirm/infirm? Should I look into twisted.web2 and hack authentication from it to Nevow? Thanks.

3 2

Socket timeout in web2 (lost socket)
by Ashish Jha Jan. 7, 2008

Jan. 7, 2008

I have a HTTP connection where the client has died, and twisted.web2 is getting stuck waiting for data. Can anyone tell me how to timeout this case? Details: - client connects to server to perform a POST (XMLRPC in this case) - twisted accepts connection, hands off to twisted.web2 - twisted.web2 wsgi handler runs wsgi app in a thread - wsgi app tries to read the full body, based on Content-Length - client dies before sending all the data (crashes or disconnects without TCP connection being closed) - wsgi app thread is stuck forever in read() Traceback: ....... File "D:\LabManager-Jan-07-2\src\wsgi\XMLRPCServer.py", line 98, in __call__ data = environ['wsgi.input'].read(length) File "D:\LabManager-Jan-07-2\src\twisted\web2\wsgi.py", line 66, in read return callInReactor(self.stream.readExactly, size) File "D:\LabManager-Jan-07-2\src\twisted\web2\wsgi.py", line 48, in callInReactor result = queue.get() File "D:\LabManager-Jan-07-2\python24\lib\Queue.py", line 119, in get self.not_empty.wait() File "D:\LabManager-Jan-07-2\Python24\lib\threading.py", line 203, in wait waiter.acquire() The function self.stream.readExactly is BufferedStream.readExactly (wsgi.py, line 987). However, it's not clear how to fix this up, since there seems to be another class in the middle i.e. _NotifyingProducerStream (and it's base class ProducerStream) Does anyone have any example code for this, or can you give me a rough guide on how to implement this? (I am using Python 2.4.3, Twisted 2.4.0, and Twisted.Web2 0.2.0)

1 0

Divmod Nevow 0.9.29
by Tristan Seligmann Jan. 7, 2008

Jan. 7, 2008

Hi all, I'm pleased to announce the release of Nevow 0.9.29, available here: http://divmod.org/trac/attachment/wiki/SoftwareReleases/Nevow-0.9.29.tar.gz… Changes since 0.9.26 (the last announced release) are listed below. 0.9.29 (2008-01-02): - Prevent NevowSite.handleSegment from raising IndexError in certain situations. - Deprecated wsgi and zomne modules. 0.9.28 (2007-12-10): - Added two APIs to Athena, one for creating the string used as the id attribute of the top node of a widget and one for creating the string used as the id attribute of a node which had an id attribute in the widget's template document. 0.9.27 (2007-11-27): - Unicode URLs now supported.

1 0