Mailman 3 August 2005 - Twisted-web

How to "monitor" parallel executions of Deferreds
by Pedro Sanchez 25 Aug '05

25 Aug '05

Hello, I need to implement a function (say, myfunc) that fires several independent Deferreds (using deferToThread), and that returns a deferred itself. A client of this function (say, client) is expected to do something like this: def client(): def done(success) if success: print 'd1 and d2 are done' d = myfunc() d.addCallback(done) ... myfuct() above does something like this: def myfunc(): def checkFunc1(success): ... def checkFunc2(success): ... d = defer.Deferred() d1 = deferToThread(func1).addCallback(checkFunc1) d2 = deferToThread(func2).addCallback(checkFunc2) d.addCallback(????) return d Clearly d, d1, and d2 in the above are disconnected. And I don't want to serialize d1 and d2 which would be a way to connect them all. The problem is I don't know how d should be defined. The idea is that the 'done' function in the client function is only called when both threads d1 and d2 are done. Can you give me any suggestions on how to implement myfunc()? Thank you, -- Pedro

1 0

URL length limits in twisted-web
by Lenny G Arbage 25 Aug '05

25 Aug '05

I'm noticing silent failures using HTTPClientFactory with len(url) >= 16400. The factory's deferred object immediately calls its associated callBack() method without sending any data (the server is likewise written with twisted python, but it doesn't appear to ever receive data). The errorBack() doesn't fire either. A url with len <= 16398 works just fine, firing the callback /after/ sending the data. According to RFC 2616, arbitrary length URLs are allowed by HTTP, but I understand that there are practical limitations in many implementations. Is the 16400 url-length ceiling a known limitation in twisted, or am I banging up against something else? thanks, Lenny ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs

2 1

registerAdapter and implemented interfaces
by en.karpachov＠ospaz.ru 25 Aug '05

25 Aug '05

In nevow, when I adapt a context to ISession, I get a twisted.web.server.Session object by default. If I want to adapt this session-like object to my custom IUser interface, I register an adapter: components.registerAdapter(UserAdapter, twisted.web.server.Session, IUser) so I could write: session = ISession(ctx) user = IUser(session) But, if I eventually move to the nevow guard, this code won't work because ISession adapter now returns an object of nevow.guard.GuardSession. What is the right way to register an adapter so the code won't break? components.registerAdapter(UserAdapter, <what is here?>, IUser) I can use the Componentized class, but it is not obvious and there is no guarantee either that ISession will always adapt to the Componentized. It would be easier if there was a precondition that an adapter itself implemented the interface it adapts to, so one could write: components.registerAdapter(UserAdapter, ISession, IUser) Is there any rationale not to have such a precondition? What I missed here? -- jk

1 0

Higher level application servers
by Phil Mayers 24 Aug '05

24 Aug '05

OR: Zope is great, but... I've got a large web-based application (forms-based SQL-backed, but a very very comprehensive app) written over a period of about 4 years in Zope. I have been overall very impressed by some features of Zope, but I feel it's increasingly moving away from where I want to be - there's a heavy CMF/doc repository/company intranet feel to the new X3 releases, and the benefits of async versus threadpool-based and multiprotocol support are just too much. But I don't want to lose a lot of what Zope does offer, in particular: * transaction manager - integrated ZODB, SQL (and other) transaction(s) wrapping rendering of the entire requests * TTW editing - very hand for RAD * rollback of changes / versioning of code/data objects I can turn around new (good, well thought out) features very quickly using a combinaton of Zopes PythonScripts and/or templated SQL, and if I make a coding error, the exception rolls back any transactions for any and *all* transactional-aware components used in that request (SQL, LDAP, filestore) without me doing any work, then I take a look at the exact diff of what I typed and maybe rollback the changes. A great crutch! Of course writing all these would be relatively trivial, but the time spent piles up. I'm aware the divmod suite has an app server (Mantissa) but to be frank I don't like Atop. I'm quite happy (actually I prefer) filesystem-backed code. So what's the state of play for layers between t.web -> (appserver) -> appcode. I would expect the obvious choice for components within the app server to be t.e.adbapi, nevow, t.cred and then onto the more narrow components like a caching framework, indexing, etc. Thoughts?

2 3

Generating cache headers for dymanic data
by Mary Gardiner 23 Aug '05

23 Aug '05

I think this is round 2 of this discussion, but I'd appreciate thoughts anyway. I have a Nevow application. It has the following properties: 1. it is the same for all users, there is no user specific output 2. part of the data source, and hence some of the pages (say 5-10 out 600 odd) changes about twice a week 3. the vast majority of pages remain the same for periods of weeks or months As far as I can tell, this makes it a pretty good candidate for the various cache headers (Last-Modified and ETag). In fact I'm sure it's a pretty good candidate, because I've seen how much bandwidth the Googlebot and the RSS aggregators suck if I don't use them. The problem is giving a sufficient guarentee of byte-equality. As I understand the purpose of those headers, if a page has a Last-Modified unchanged from the last time it was accessed, or has an ETag unchanged from the last time it was accessed, the body of the response should be guarenteed to be byte-for-byte identical in so far as that is possible. (I think foom told me that in practice, Apache regards mtime, file size and inode numbers as a sufficient guarentee of this.) As best I can tell, a realistic and reasonably bullet-proof way for me to tell if there's been a change in my output would be: - the same version of Nevow - the same version of my Nevow application - the same template (as checked by the above logic of Apache's) - the same data source (mine are files on disk, so again I can use the above logic) So that's all well and good for the last two, but does anyone have any ways to detect the first two in any other way than storing a version variable in my source code and in Nevow source code? -Mary

2 3

On Flatteners and Adapters
by Pedro Sanchez 20 Aug '05

20 Aug '05

Hello, I'm trying to implement a menu with the attached code, but it fails with the following error: exceptions.NotImplementedError: There is no flattener function registered for object <Menu object at 0xb76dd42c> of type <class 'Menu'>. Locals flattener None obj <Menu object at 0xb76dd42c> context WovenContext(tag=Tag('li', children=[<Menu object at 0xb76dd42c>])) I'm using http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286260 as a reference for this sample implementation. The idea is to define two objects, Menu and MenuItem, and then to define and to register adapters for each object type. I don't understand why it complains about flatteners not defined. Isn't that covered by the registered adapters? I'd appreciate any help. Thank you, -- Pedro PS. I test the program running: $ twistd -noy menu.py

2 3

Dynamic SSL context
by Marek Habersack 19 Aug '05

19 Aug '05

Hello, I am wondering, is it possible with Twisted to delay creating the SSL context till the connection time? I would need something like that to support virtual hosts on the same interface:port with different SSL certs. tia, marek

5 6

Garbage SSL application data
by Joshua Boverhof 18 Aug '05

18 Aug '05

I'm using the twisted.web.getPage for https, but for some reason garbage application data is preceeding the data I'm specifying (it completes the handshake). This is causing a java server to bomb because it is expecting a POST or GET command: It does a readLine, gets nothing and throws an IOException. A twisted.web server digests the message w/o problem, apparently it's ignoring the garbage bytes (which change w/each invocation). I'm also getting a few failures w/ "twisted.test.test_ssl". I'm interpretting the error message below "startTLS doesn't empty buffer before starting TLS. :(" as the cause of my problem... I'm seeing this behavior w/clients running on a Tiger laptop and a Linux box w/the following packages: Twisted-2.0 TwistedWeb-0.5 pyOpenSSL-0.6/openssl-0.9.7 thanks, -josh Code f = client.getPage(url, contextFactory=None, postdata=DATA, agent=agent, method=method, headers=headers) SSLDump after handshake --------------------------------------------------------------- 2 13 0.6668 (0.0629) C>SV3.0(24) application_data --------------------------------------------------------------- application_data(23)/major(3)/minor(0)/length(24) 17 03 00 00 18 a7 f3 45 ee 7b e1 19 d6 0c 0f fa 7a 9d a4 eb 98 ce 84 7f e3 1e a1 cc 70 GARBAGE --------------------------------------------------------------- 2 14 0.6668 (0.0000) C>SV3.0(1224) application_data --------------------------------------------------------------- application_data(23)/major(3)/minor(0)/length(1224) 17 03 00 04 c8 50 4f 53 54 20 2f 77 73 72 66 2f ..... P O S T / w s r f / ..... bundy:~/Desktop/Python/Twisted-2.0.0/bin boverhof$ ./trial twisted.test.test_ssl .... .... [TODO]: testTLS (twisted.test.test_ssl.SpammyTLSTestCase) startTLS doesn't empty buffer before starting TLS. :( File "/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/internet/defer.py", line 117, in maybeDeferred result = f(*args, **kw) File "/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/test/test_ssl.py", line 285, in testTLS TLSTestCase.testTLS(self) File "/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/test/test_ssl.py", line 225, in testTLS UnintelligentProtocol.pretext + UnintelligentProtocol.posttext File "/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/trial/unittest.py", line 415, in <lambda> failUnlessEqual = lambda self, a, b, msg=None: failUnlessEqual(a, b, msg) File "/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/trial/assertions.py", line 77, in failUnlessEqual raise FailTest, (msg or '%r != %r' % (first, second)) twisted.trial.assertions.FailTest: ['first line', 'last thing before tls starts', 'STARTTLS'] != ['first line', 'last thing before tls starts', 'STARTTLS', 'first thing after tls started', 'last thing ever'] ------------------------------------------------------------------------------- Ran 10 tests in 5.512s PASSED (expectedFailures=3, successes=7)

2 3

PATCH: Make nevow.URL.__str__ return a string
by Jonathan Lange 15 Aug '05

15 Aug '05

I said, wtf exarkun said, patches accepted unit tests said, sure, why not radix said, mail it to twisted-web

1 0

[PATCH] Unquote URI when converting to filesystem path
by Wilfredo Sánchez Vega 12 Aug '05

12 Aug '05

The following patch fixes web2 to unquote a path string derived from a request URI before sticking into a filesystem path. Otherwise, URIs like "/foo%20bar" don't get properly translated to "/ foo bar". -wsv Index: twisted/web2/static.py =================================================================== --- twisted/web2/static.py (revision 14235) +++ twisted/web2/static.py (working copy) @@ -6,7 +6,7 @@ """ # System Imports -import os, time, stat +import os, time, stat, urllib # Sibling Imports from twisted.web2 import http_headers, resource @@ -177,7 +177,7 @@ return None, () if path: - fpath = self.fp.child(path) + fpath = self.fp.child(urllib.unquote(path)) else: fpath = self.fp.childSearchPreauth(*self.indexNames) if fpath is None:

1 0