Mailman 3 October 2004 - Twisted-web

Test
by Philippe Lafoucrière 05 Aug '21

05 Aug '21

Hi all just a test

3 2

How to get rid of web logs?
by Yun Mao 05 Jan '05

05 Jan '05

Right now I'm using the log for debugging, and I don't wan to log those info like below. How should I achieve this? Thanks! 2004/09/30 13:31 Eastern Daylight Time [HTTPChannel,1,127.0.0.1] 127.0.0.1 - - [ 30/Sep/2004:17:31:56 +0000] "GET /?view=all HTTP/1.1" 200 2767 "http://localhost :1083/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3" Yun

3 3

Nevow: Authenticated site with "remember me" and vhosts
by Paul Moore 04 Nov '04

04 Nov '04

I'm trying to use Nevow to build a site which displays the results of a series of SQL queries. It looks good so far, I have the basic site structure all sorted out in my mind (it's *very* impressive how quick that was in Nevow). Before I flesh things out much further, I want to add some authentication - basically, "guest" users will be able to see a limited subset of the data. Adding authentication including a "login" form in the sidebar (a bit like Roundup does) was again very easy. I basically cribbed from the existing examples. But there are a few things I's now like to add: 1. Ultimately, I want to proxy the app through Apache, so I was using the Virtual Host Monster. Im ny original, unauthenticated, application, I added a "vhost" child to the main application resource, and that was it. I can no longer do that, as the resource is wrapped in a guard.SessionWrapper, which doesn't have a putChild method. I'm not clear that I want the vhost child to be authenticated anyway - what I *think* I want is for "/" to be authenticated, but "/vhost" not to be. Or do I? Any suggestions? 2. I'd like to add a "remember me" checkbox, which if checked sets a cookie to remember the user's ID, so that they don't have to log in again for (say) 2 weeks. Much like gmail, or Yahoo, do. But I'm not too sure how I'd go about this. I'm not even sure where I'd *put* auto-login code: the page is too late (the guard has done its thing by then) but anything else seems to be too early (ie, not per-session, if you see what I mean). I suspect I need to subclass the Portal, or something, but I'm not sure... I've scanned the various examples, both in the distribution and in the Nevow sandbox, but nothing seems to do what I want. Can anyone give me some pointers? <off on a tangent> This is the sort of area where the lack of documentation really hurts. Examples are great (and the wealth of Nevow examples is realy nice) but there's a lack of "general principle" documentation to make it easy to generalise the example code. My best example of this is that I can't work out what to return from a render_* method - should it be a function, or a string, or a stan tree, or what? I suspect that by the magic of interfaces, the answer is "anything that can be adapted to an ISomething" (which needs further documentation to say what standard adapters from string, sequence, whatever to ISomething exist) - but it's not stated anywhere, which makes writing render_* methods an exciting exercise in experimentation :-) At the other end of the scale, the source is really too low-level to follow for this sort of thing (I have tried). Is there any documentation effort for Nevow going on anywhere? I'd happily contribute, although it would more likely be in the form of questions than answers at the moment :-) </off on a tangent> FWIW, I attach my driver script below, in case it clarifies my authentication question above. from twisted.application import internet from twisted.application import service from twisted.web import static from nevow import appserver from nevow import vhost from nevow import guard from nevow import inevow from twisted.cred import portal from twisted.cred import checkers from twisted.cred import credentials from testapp.web import pages # Authorisation setup. # First we need a realm, which provides requestAvatar # TODO: need NotLoggedIn, LoggedIn and noLogout class MyRealm: """A simple implementor of cred's IRealm. For web, this gives us the LoggedIn page. """ __implements__ = portal.IRealm def requestAvatar(self, avatarId, mind, *interfaces): for iface in interfaces: if iface is inevow.IResource: # do web stuff if avatarId is checkers.ANONYMOUS: resc = pages.BasePage() else: resc = pages.BasePage(avatarId) resc.realm = self return (inevow.IResource, resc, resc.logout) raise NotImplementedError("Can't support that interface.") # Now create a portal to our realm realm = MyRealm() portal = portal.Portal(realm) # Create a checker for a static list of test users myChecker = checkers.InMemoryUsernamePasswordDatabaseDontUse() myChecker.addUser("user","password") myChecker.addUser("fred", "flintstone") # Register our checker, and an anonymous user checker, with the portal portal.registerChecker(checkers.AllowAnonymousAccess(), credentials.IAnonymous) portal.registerChecker(myChecker) application = service.Application('testapp') # Instead of the application resource being the base page, it becomes a # SessionWrapper round the portal we created. The base page is part of the # realm - see above... # appResource = pages.BasePage() appResource = guard.SessionWrapper(portal) #appResource.putChild('robots.txt', static.File('static/robots.txt')) # SessionWrapper doesn't have a putChild method... #vResource = vhost.VHostMonsterResource() #appResource.putChild('vhost', vResource) webServer = internet.TCPServer(8080, appserver.NevowSite(appResource)) webServer.setServiceParent(application) # vim:ft=python: Paul -- The scientific name for an animal that doesn't either run from or fight its enemies is lunch. -- Michael Friedman

4 4

Twisted as reverse proxy for access control?
by Bud P. Bruegger 29 Oct '04

29 Oct '04

Hello, I'm new to Twisted and would like to find out whether it is a good choice to solve my problem. For OpenPortalGuard's GateKeeper component (http://openportalguard.sourceforge.net/wiki/index.php/Specification/Archite…), I am looking for a platform to implement a reverse proxy that handles the following tasks before giving access to a private network that hosts multiple application servers: * ssl processing (offloaded from application servers) * ssl client cert authentication * a custom authentication module that writes cookies * a custom access control module (role based) * an URL-rewriting engine like Apache's mod-proxy or mod-rewrite My first idea was to use Apache with mod-ssl, mod-proxy, and mod-rewrite out of the box and write the two custom modules with mod-python. My concern with Apache is scalability. I believe that Twisted (implementing the Reactor Pattern) should be able to handle many more concurrent connections. Can anybody confirm this and possibly quantify this with examples? My concern with Twisted is (due to ignorance ;-) whether it has the out of the box functionality equivalent to Apache's mod-ssl and mod-rewrite (to proxy to remote hosts). A first look seems to indicate that twisted.internet.ssl, twisted.web.proxy, and twisted.web.rewrite are my friends. Is this correct? Are there any examples out there? Any experience on performance? Alltogether, is Twisted a good choice for my plan? Has anyone already used Twisted for such a purpose? Many thanks in advance for any input and suggestions. -b ------------------------------------------------------------------------------------------------- Ing. Bud P. Bruegger, Ph.D. 0564-488 577 (voice) Servizio Elaborazione Dati 0564- 21139 (fax) Comune di Grosseto e-mail: bud(a)comune.grosseto.it Via Ginori, 43 jabber: bud(a)amessage.info 58100 Grosseto icq: 249-858-685 Collaborazione Open Source per la CIE e CNS http://www.comune.grosseto.it/cie/ Software Libero/Open Source in P.A.: Non solo una buona idea, ma una necessita'

1 0

How to use Woven.form
by Young-gue Park 28 Oct '04

28 Oct '04

i want to use Woven.form but i don't know exact using If I use Woven.form, I will really easy make form and process form data please help me Any example code and any hint make me so happy ps... why don't you complete form in depth document? and When was document completed?

2 1

Nevow bug?
by Peter Hunt 28 Oct '04

28 Oct '04

nevow.stan.Comment.__call__ has the following line: return NotImplementedError('comments are not callable') Shouldn't it be: raise NotImplementedError('comments are not callable') This was a tough bug to track down.

2 1

Redirect with deferred
by Vicky Lupien 27 Oct '04

27 Oct '04

Hi, I have a little problem. I have a deferred and I want to redirect after the defered is done. But it seems that my page is rendered and my redirect is never done. Can someone help me. Here's my code: class HousesQuickSearch(pagelib.QuebecPage, sqlQueries): def render_content(self, context, data): request = context.locate(inevow.IRequest) def _(data): print data ses = request.getSession() self.houseSR = sessions.ISessionPrefs(ses).get('houseSR') sessions.ISessionPrefs(ses).set(self.houseSR[0], data) sessions.ISessionPrefs(ses).set(self.houseSR[4], 0) lengthPage = sessions.ISessionPrefs(ses).get(self.houseSR[2]) widthPage = sessions.ISessionPrefs(ses).get(self.houseSR[3]) currentPage, pageBatchItems = self.nextPage(data, lengthPage, widthPage, 0) sessions.ISessionPrefs(ses).set(self.houseSR[4], currentPage) sessions.ISessionPrefs(ses).set(self.houseSR[1], pageBatchItems) dir = '/'.join(['', '/'.join(request.prepath[0:-1]), "search", "results"]) #this is not done return request.redirect(dir) conditions = " where 1" if request.args.get('plan_number')[0]: conditions += " and plan_number='"+str(request.args.get('plan_number')[0])+"'" sql = self.getHouseSearchResultsInfo(conditions) dbpool = request.getSession().guard.resource.portal.realm.dbpool searchResults = dbpool.runQueryDict(sql) searchResults.addCallback(_) Vicky

2 2

Re: Case for an Async MySQL Driver?
by Peter Hunt 27 Oct '04

27 Oct '04

I'd be willing to help; but I think we should shell out an async DB-API, too. Maybe go over to DB-SIG and propose an async version of the new upcoming spec?

1 0

Diagnosing a socket leak in nevow code
by Joseph Bowers 26 Oct '04

26 Oct '04

Hey, all! I'm working with a release nevow (0.1) on Twisted 1.3, and somewhere I've introduced a socket leak. Unfortunately, I'm not sure how to go about debugging this: Does anybody have recommendations for tracking down this kind of error? What are common screw-ups with nevow code that might lead to zombie sockets? Thanks! -------------------- Joe Bowers Technical Director Seventy-two dpi 828.252.2408 [x308] joe(a)seventytwodpi.com http://www.seventytwodpi.com

2 1

HTTP/1.1 Persistent Connections and Pipelining
by Seun Osewa 26 Oct '04

26 Oct '04

Hi, Does Twisted.Web support persistent connections and pipelining? how can I take advantage of this, if it does? And, if not, do you feel the interface is such that support for persistent connections can be transparently added? Regards, Seun Osewa.

3 4

RE: [Twisted-web] Documentation for Twisted.Web
by Harald.Nyman＠astrazeneca.com 23 Oct '04

23 Oct '04

I agree. The info is there (see PS), but it took quite some time before it made any sense to me, and I still don't understand most of the "Using twisted.web" section, being probably unusually stupid. Here's a humble suggestion: Add something like these 2 points - 1) Refer to Jason Hunter's "Java Servlet Programming" # bcos that's pretty similar (I think) 2) Try this code snippet: class Servlet(resource.Resource): def render_GET(self, request): print request.__dict__ # .. and see what it contains Using that approach I've just managed to put together the coolest asynchronous, message-driven, "don't-hang-around-waiting-for-it" web GUI we've ever had. The more I fumble with Twisted, the more I appreciate it. Or did I get it all wrong again? ;-) Harald PS The Resource class, which is usually what one's Resource classes subclass, has a convenient default implementation of render. It will call a method named self.render_METHOD where "METHOD" is whatever HTTP method was used to request this resource. Examples: request_GET, request_POST, request_HEAD, and so on. It is recommended that you have your resource classes subclass Resource and implement render_METHOD methods as opposed to render itself. Note that for certain resources, request_POST = request_GET may be desirable in case one wants to process arguments passed to the resource regardless of whether they used GET (?foo=bar&baz=quux, and so forth) or POST. -----Original Message----- From: Seun Osewa [mailto:naijasms@gmail.com] Sent: lördag, 23 oktober 2004 10:50 To: twisted-web(a)twistedmatrix.com Subject: [Twisted-web] Documentation for Twisted.Web Hi, I sure wish documentation for Twisted and Twisted.Web was more complete! Right now I feel the information about using Twisted.Wed objects for managing http connections asynchronously _without_ the configuration system (Tap files etc.) is incomplete. How can one get the required info? It appears things are changing really fast ... Seun _______________________________________________ Twisted-web mailing list Twisted-web(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

1 0

Documentation for Twisted.Web
by Seun Osewa 23 Oct '04

23 Oct '04

Hi, I sure wish documentation for Twisted and Twisted.Web was more complete! Right now I feel the information about using Twisted.Wed objects for managing http connections asynchronously _without_ the configuration system (Tap files etc.) is incomplete. How can one get the required info? It appears things are changing really fast ... Seun

1 0

Re: [Nevow-commits] r740 - Refactor urlToChild API to avoid passing superfluous data (#1 from issue 121).
by James Y Knight 23 Oct '04

23 Oct '04

I believe this change is wrong: prePathURL() changes. setupSession has to use the prePathURL in place when guard's locateChild() is called, not the one in place at the time some other resource calls setupSession(). James On Oct 22, 2004, at 11:02 AM, <mg(a)divmod.net> wrote: > Refactor urlToChild API to avoid passing superfluous data (#1 from > issue 121).

2 1

Re: [Twisted-Python] twisted webdav server
by James Y Knight 23 Oct '04

23 Oct '04

Following up on twisted-web mailing list. On Oct 22, 2004, at 11:23 PM, Yasushi Iwata wrote: > On Fri, 22 Oct 2004 12:31:57 -0400 Bob Ippolito wrote: >> This particular package is not going to go into Twisted unless the >> license changes. LGPL is not compatible with MIT. > > Some code of akaDAV were adapted from Twisted 1.3.0 which is > distributed under LGPL, so I cannot change the lisence of akaDAV. > > I'm glad to change the lincense and contribute what I wrote to Twisted > project. Should I rewrite all the code adapted from Twisted 1.3.0 to > change the license? To change the license, you should copy the code from Twisted again, but this time from a version of Twisted under the MIT license (current SVN or 2.0 when it's released). Or you could just say you did that, and no one would know the difference, since the code you copied likely hasn't changed much since 1.3. ;) I'd *love* to have a WebDAV package as part of Twisted, with a few caveats (I have not looked at your package yet, so none of this is criticism, just generic comments): - I would require it to be completely RFC conforming. Bugs are one thing, but some of the other python WebDAV implementations I've looked at have left me very unimpressed. A quick glance through the code and the RFC found numerous MUSTs that they didn't even _attempt_ to implement. It seemed as if the authors wrote the absolute minimum to make it "work" with some popular clients and left it at that. An absolutely minimal implementation is okay for some purposes, I suppose, but not for inclusion as a protocol implementation in Twisted. - I'm not sure how much code you duplicated, but that would likely have to be resolved somehow, as it's clearly not acceptable to have two slightly different copies of a large portion of code in twisted. - It should be very well unit-tested. - I'd like it based on the web2 API (which is currently still unfinished, so don't write to it yet expecting it to be stable). Hopefully it will be easier to not copy code with the new API. If not, the new API should be fixed so that it is. ;) - It should integrate nicely into a standard twisted web resource tree. I have muttered about possibly doing WebDAV for Twisted at "some point in the future" a few times, and that still stands. But, for now, I don't have time to do it. I've got more than enough on my plate just trying to get the web2 rewrite done, without adding highly complicated new protocols. So, either you or someone else would have to volunteer to get it into merge-able shape *and* to be the primary Twisted maintainer to handle any bugs which will inevitably show up. Or wait probably at least a year for me to become able to do it. :) James

2 1

Twisted vs Webware
by Tom Fillmore 22 Oct '04

22 Oct '04

Can anyone shoot me their (fact-based) opinion as to why Twisted is more appropriate than Webware? I'm not trolling here, I've just never used Webware. Thanks Tom Fillmore in traffic-clogged southern California

5 7

Template engines
by Tom Fillmore 21 Oct '04

21 Oct '04

Hi, all - I'm a newbie so please be gentle... 8-) I am really impressed with Twisted and its capabilities and have looked at Nevow but the lack of docs (outside the source code that is) leaves it essentially unusable for me at this point. I see on the site docs that Quixote is listed as a template engine that can be used by Twisted. Can any others, such as Cheetah? Thanks in advance. Tom Fillmore Rainy Southern California

2 1

Alternative to <nevow:attr>
by David Reid 20 Oct '04

20 Oct '04

Ok, so currently if you want dynamic stuff to happen in an attribute you have to do something along the lines of this. <a render="mylink"> <nevow:attr name="href"> <nevow:slot name="link" /> </nevow:attr> <nevow:slot name="label" /> </a> def render_mylink(self, ctx, data): ctx.fillSlots('link', data[0]) ctx.fillSlots('label', data[1]) return ctx.tag Which is horrendously ugly and difficult to explain to pretty much anyone. Now, there are plenty of other solutions to this problem, but none of them provide the flexability for the template author as <nevow:attr> and <nevow:slot> do. For instance: <nevow:attr name="href"> /users/<nevow:slot name="username" />/images </ nevow:attr> Works perfectly well. Which according to fzZzy is the main reason for choosing this syntax. A comparison to this with the syntax of TAL was recently mentioned on the ML (Comparison with ZPT [Was: [Twisted-web] Sub-tag level templating in Nevow: possible?]), and the TAL syntax seems to be cleaner, though I'm not sure if it provides the above functionality. While discussing it on IRC I came to a conclusion that the following would be comfortable, and flexible syntax alternative (not replacement) for <nevow:attr> <a href="/images/archives/%(year)/%(month)/%(day)" nevow:attributes="href=year,month,day alt=label"> <nevow:slot name="label" /> </a> basically when the template was getting parsed, the Tag instance would end up something like this. Tag(name='a', attributes={'href': ['/images/archives/', slot(name='year'), slot(name='month'), slot(name='day')] }) It provides the flexibility, and in my opinion is just a little more intuitive than <nevow:attr>. --David

6 8

Passing context through/around/across webform
by Alex Levy 20 Oct '04

20 Oct '04

So I've started to move Payago (formerly Go) towards compatibility with Nevow 0.3's API changes; the biggest impact has been the ubiquitous context, and not being able to rely on a Page having self.context. Previously, I had _many_ TypedInterfaces which did something along the lines of: class ICanDoSomething(TypedInterface): def doSomething(self, arg = Choice(choicesAttribute='choices')): pass doSomething = autocallable(doSomething) class MyPage(Page): __implements__ = ICanDoSomething def _get_choices(self): ## do something involving self.context return some_list choices = property(_get_choices) Now, with the ubiquitous context changes, self.context doesn't exist. (Makes you wonder what "ubiquitous" really means.) So there are a couple solutions that I've thought of: 1. Make annotate.Choice support 'choicesMethod', which would take context as an argument. This seems very difficult because not only webform, but Typed.coerce(), would have to be able to retrieve the context. Then we'd have to support Deferreds, because some crazy person would obviously want that at some point. ;) I tried to write a patch, and it got ugly. 2. Set self.context whenever Page.locateChild or Page.renderHTTP is called. This is a lot simpler, but it seems like a hack to me, and I'm not sure whether this will _always_ work. What do other people think? Is option #1 easier to implement than I'm thinking? Is option #2 safer than it seems to me? Is there an option #3 that I don't see right now? -- Alex Levy WWW: http://mesozoic.geecs.org/ "Never let your sense of morals prevent you from doing what is right." -- Salvor Hardin, Isaac Asimov's _Foundation_

3 6

Re: hi? i have some problem in using simpleguard example
by Young-gue Park 20 Oct '04

20 Oct '04

I just add the last line at the whole source resource = buildGuardedResource() Don't you feel me stupid? %^.^% On Wed, 20 Oct 2004 17:32:07 +0900, Young-gue Park <jackup(a)gmail.com> wrote: > i execute your sample example code > > but the result is like next > > the page can not display > > i don't know the reason and why doesn't it work > > Is There anybody to answer my confusing? > > i hope this problem solve as soon as possible > > please help me >

1 0

mod_rewrite while persisting client ips
by Miika Keskinen 20 Oct '04

20 Oct '04

Hi there. I'm writing to the list in case someone finds this useful, I include short ProxyIPResource-'hack' and corresponding rewriterule to set request.client.host correctly :) here it goes: --snip-- from twisted.application import service, internet from nevow import rend, loaders, appserver, vhost, inevow from nevow import tags as T """ Example apache configuration NameVirtualHost * <VirtualHost *> RewriteEngine on RewriteRule (.*)/ http://localhost:8080/proxyip/%{REMOTE_HOST}/vhost/http/test.com/$1 [P] ServerName test.com </VirtualHost> """ class ProxyIPResource: __implements__ = inevow.IResource, def putChild(self, child, res): setattr(self, 'child_%s' % (child,), res) def locateChild(self, ctx, segments): if len(segments) < 1: return rend.NotFound elif "child_%s" % (segments[0],) in self.__dict__: return self.__dict__['child_%s' % (segments[0],)], segments[1:] else: request = inevow.IRequest(ctx) request.client.host = segments[0] prefixLen = len('/'+'/'.join(request.prepath)+'/'+'/'.join(segments[:1])) request.path = '/'+'/'.join(segments[1:]) request.uri = request.uri[prefixLen:] request.prepath = request.prepath[2:] return self, segments[1:] class Simple(rend.Page): addSlash = True docFactory = loaders.stan( T.html[ T.body[ T.h1["hello"] ] ]) application = service.Application("Simple") res = Simple() vResource = vhost.VHostMonsterResource() pResource = ProxyIPResource() res.putChild('proxyip', pResource) pResource.putChild('vhost', vResource) internet.TCPServer(8080, appserver.NevowSite(res)).setServiceParent(application) --snip--

2 1

hi? i have some problem in using simpleguard example
by Young-gue Park 20 Oct '04

20 Oct '04

i execute your sample example code but the result is like next the page can not display i don't know the reason and why doesn't it work Is There anybody to answer my confusing? i hope this problem solve as soon as possible please help me

1 0

Re: [Twisted-web] Re: [Web-SIG] A more Twisted approach to async apps in WSGI
by Phillip J. Eby 15 Oct '04

15 Oct '04

At 12:59 AM 10/7/04 -0400, James Y Knight wrote: >On Oct 5, 2004, at 2:37 AM, Phillip J. Eby wrote: >>Although you probably want something more like a pipe error if the input >>times out or the connection is broken. > >You normally only get pipe errors on writes, read just sees EOF. > >But that does bring up a good point: How does the server notify the >application that the client has gone away, and any further work is useless? >- For non-async apps that use the iterator model: I think the server is >allowed to just call iterable.close() and never iterate again. Yes. >- For async applications, with the proposed API, that may not be an >option, because the iterable returned is the special wrapper, not a >user-created class. Although, actually, I guess the app can return its own >iterable whose __iter__ calls through and returns the wrapper's __iter__. Not if the server wants to be able to handle that iterable specially. But anyway, it seems that the wrapper's constructor should take a close method, or have a way to set one. >- What about for non-async applications that use the write callable? >Should write be allowed to raise an exception? Or should it just become a >no-op when the client is disconnected? It's allowed to raise an exception, though this was never explicitly put in the spec; I'll have to fix that. The actual process for that scenario looks something like this: * app calls write() * write() raises error * app catches error (maybe) and calls start_response() with exc_info * start_response() reraises the error, because it has already sent headers to the client and can't restart the response * application error handler bombs out and returns to server/gateway * server/gateway logs the exception (maybe) and gets on with life in the big 'net >Hmm, yes. I totally missed the option of just yielding ''. Of course it's >a very bad idea to repeatedly yield '' to a server if you don't know the >server can properly handle it (by e.g. delaying longer and longer), but, >in this case, since the server itself is providing the special iterable, >that should be fine. Yes. Also, when we finally settle on an async API, I do want to cover the issue of backing off iteration when empty strings are yielded. I'm actually inclined to suggest that an async application should take responsibility for doing the delaying if it's called repeatedly, and the async API isn't available. >It seems like it should be possible to make a generic class that >implements this async API for use with sync servers that do not support it >natively. That would allow async apps to run on a sync server without >modification, which is potentially useful. To do that, though, I think the >it'd have to spawn an extra thread per request that is waiting to read >data, for the read() call to block on. Unless, of course, the app never >needs to yield outgoing data while waiting for incoming data. Well, with Twisted you could deferToThread the read() operations, though it's hard for me to think straight about that scenario because I keep finding it hard to imagine an async web app that isn't just written to the Twisted API to start with... ;) >The one remaining issue I have is the required thread-safeness of various >APIs. > >The spec doesn't mention much of anything about threadsafeness: is it ok >to call wsgi methods from a different thread than the one the server >originally called the request on? Especially interesting for implementing >the above sync->async adapter: environ['wsgi.input'].read(x) would be >called from a second thread. Excellent question; I should add the answer to the spec, as soon as I decide precisely what it is. :) One point: the spec should absolutely forbid servers from using thread identity to identify the application/caller. The "what can you call while what else is executing" part of the question is a bit trickier. >What thread (if there's a choice) does the on_get callback get called on. Etc. My inclination is to make threading issues symmetrical. That is, the application doesn't get any thread-identity guarantees either. > I haven't really thought about these thready questions much either, so > maybe the answers are obvious, but in my experience, that's usually not > the case when it comes to threads. Yep. :) However, the more I think about it, the more it seems to me that WSGI should emulate single-threadedness with respect to any function/method/iterator invocations associated with a given application invocation. However, it is *not* guaranteed that all such invocations will occur from the same thread. Basically, it means "no multitasking with the other guy's objects", and puts the locking burdens on whoever's trying to mix multitasking into the works. >That's why async apps are nice. ;) Not to mention fork(). :) By the way, after all this discussion... do you think it would be better to: 1) Push towards a full async API, nailing down all these loose ends 2) Use the simple-but-klugdy "pause iteration" API idea 3) Don't make an "official" async API, and just leave it open to server authors to create their own extensions, and maybe cherry pick the best ideas for WSGI 2.0, or 4) Do something else altogether?

2 2

nevow.compy
by Peter Hunt 14 Oct '04

14 Oct '04

I've been looking into adaption frameworks recently. I've found the following different frameworks: PEP 246 PyProtocols Zope3 Twisted Nevow.compy Can someone explain to me the differences between them, and why we have so many?

4 5

Fragment's IGettable
by Christopher Armstrong 13 Oct '04

13 Oct '04

I recently noticed a difference in Nevow from some old SVN checkout vs 0.3. 0.3's behavior got in my way while writing a nevow app, but I'm not sure if it's considered a bug. The minimal example is below. The difference is that in the old Nevow, it would display "Hi"; now it displays big-red-NONE. After some debugging with the help of Tv and dreid on #twisted-web, we discovered that it was the change that added an IGettable to rend.Fragment. It __implements__ IGettable and implements 'get' as 'return self.original', which is None. Removing IGettable from __implements__ and deleting 'def get' makes it display "hi". I'll probably work around this problem by not passing the Fragment through the data machinery, and instead look it up and return it directly in a render function. I'm just notifying you guys about this issue and you can decide whether or not it should be considered a bug. I'm interested in knowing why this IGettable implementation was added to Fragment, as well. Thanks, -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://radix.twistedmatrix.com snip: from nevow import appserver, rend, loaders, inevow, tags as T class Page(rend.Page): def data_foo(self, ctx, data): return rend.Fragment(docFactory=loaders.stan(T.div["Hi"])) def render_default(self, ctx, data): # data is None here in 0.3 (as is IData(ctx), not so in old-nevow return data docFactory = loaders.stan(T.html[T.body[ T.div(data=T.directive("foo"), render=T.directive("default")) ]]) from twisted.application import internet, service application = service.Application("nevow") internet.TCPServer(8080, appserver.NevowSite(Page()) ).setServiceParent(application)

2 2

request.getHost() warning message
by Tsai Li Ming 12 Oct '04

12 Oct '04

Hi I'm getting the following warning message in my twistd.log when I use request.getHost 2004/10/12 18:25 SGT [HTTPChannel,0,192.168.1.213] /root/server.tac:69: exceptions.DeprecationWarning: IPv4Address.__getitem__ is deprecated. Use attributes instead. Should I be concern at all? Liming

1 0

Trivial diff -- automatically adapt renderers
by Cory Dodt 11 Oct '04

11 Oct '04

When using a hand-made (doesn't inherit from rend.Page) Page that wraps real pages, I discovered that registering an adapter for IRendererFactory doesn't automatically allow .renderer to get called. Attached is a diff that automatically adapts rendererFactories to IRendererFactory in flatstan.py. The following is a minimal app that exhibits the buggy behavior: e.g. ----------- from nevow import inevow, rend, loaders, appserver, tags as T from twisted.application import service, internet from twisted.python import components class MyWrapper: __implements__ = inevow.IResource, def __init__(self, wrapped): self.wrapped = inevow.IResource(wrapped) def renderHTTP(self, ctx): return self.wrapped.renderHTTP(ctx) def locateChild(self, ctx, segments): wr, segments = self.wrapped.locateChild(ctx, segments) return MyWrapper(wr), () components.registerAdapter(lambda self: self.wrapped, MyWrapper, inevow.IRendererFactory) class DumbResource(rend.Page): def render_x(self, ctx, data): return T.body['hi'] docFactory = loaders.xmlstr("<html xmlns:n='http://nevow.com/ns/nevow/0.1'><a n:render='x' /></html>") application = service.Application("brokenIRendererFactory") svc = internet.TCPServer(8080, appserver.NevowSite(MyWrapper(DumbResource()))) svc.setServiceParent(application) ----------- # ===> MyWrapper instance has no attribute 'renderer' (Donovan, you might remember me bringing this up on #t.web a few months ago.) And, as an anticlimax, the patch to fix the above: Index: nevow/flat/flatstan.py =================================================================== --- nevow/flat/flatstan.py (revision 695) +++ nevow/flat/flatstan.py (working copy) @@ -204,7 +204,7 @@ if context.precompile: return original - rendererFactory = context.locate(IRendererFactory) + rendererFactory = IRendererFactory(context.locate(IRendererFactory)) renderer = rendererFactory.renderer(context, original.name) return serialize(renderer, context)

3 3

Sub-tag level templating in Nevow: possible?
by Mary Gardiner 10 Oct '04

10 Oct '04

I have a use-case for sub-tag level templating in Nevow. Imagine that for some reason I have a list of categories that I want to link to. I might want to link to them like this: <a href="catname1">catname1</a> <a href="catname2">catname2</a> <a href="catname3">catname3</a> Or I might want to link to them like this: <a href="catname1">Category catname1</a> <a href="catname2">Category catname2</a> <a href="catname3">Category catname3</a> As best I can tell, the only way to do this is by changing a render_ method to be either: render_atag(self, ctx, data): return ctx.tag(href=data)[data] or render_atag(self, ctx, data): return ctx.tag(href=data)["Category: " + data] I'd prefer to specify it in an HTML template rather than the Python code for various reasons (for starters, it isn't "business logic", it's a design choice). Is there any way to do that? -Mary

3 2

[PATH] Twisted.web2 little bug fixes.
by Valentino Volonghi 08 Oct '04

08 Oct '04

Index: twisted/web2/server.py =================================================================== --- twisted/web2/server.py (revision 11928) +++ twisted/web2/server.py (working copy) @@ -111,7 +111,7 @@ requestContext = context.RequestContext(tag=self) - self.deferredContext = self._getChild(requestContext, site.getRootResource(), self.postpath) + self.deferredContext = self._getChild(requestContext, self.site.getRootResource(), self.postpath) self.deferredContext.addErrback(self._processingFailed, requestContext) def handleContentChunk(self, data): @@ -140,7 +140,7 @@ ).addErrback( self._processingFailed, newctx ).addCallback( - self.handleSegment, path, newctx + self._handleSegment, path, newctx ) def _handleSegment(self, result, path, pageContext): @@ -239,7 +239,7 @@ raise TypeError("html is not a string") return - def notifyFinish(self): + def notifyFinish(self): """Notify when finishing the request @return: A deferred. The deferred will be triggered when the -- Valentino Volonghi aka Dialtone Now running FreeBSD 5.3-beta6 Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/

1 0

[BUG] static.File and nevow.static.File
by Valentino Volonghi 08 Oct '04

08 Oct '04

Every page that has a static.File() inside when is reloaded from the browser causes a timeout error. Removing the static.File from the page solves the situation so I guess that the problem is inside static.File. The same erroneous behaviour could be observed in customform.tac example if you refresh it. In some cases refreshing 2 or 3 times the same example renders it. I don't remember having problems with this stuff except in the last 2 days, so it could have something to do with the latest changes in tcp and abstract, anyway I can reproduce this problem all the times is needed. I also have ethereal packets file that I used to sniff the connection from the browser and the server (if needed). Here is a minimal example to demonstrate the bug: == from twisted.application import service, internet from nevow import appserver, rend, loaders, static class Page(rend.Page): docFactory = loaders.xmlfile('min.html') children = {'style.css':static.File('style.css')} site = appserver.NevowSite( Page() ) application = service.Application("minimal") internet.TCPServer(8080, site).setServiceParent(application) == <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>Error</title> <style type="text/css">@import url('style.css');</style> </head> <body> I'm not starting when you hit refresh on me </body> </html> == body { background-color: #CE9090; } == -- Valentino Volonghi aka Dialtone Now running FreeBSD 5.3-beta6 Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/

1 1

A more Twisted approach to async apps in WSGI
by Phillip J. Eby 05 Oct '04

05 Oct '04

Hi all. I've been away for a few days due to loss of e-mail service when my dedicated server lost a hard drive. Unfortunately my ISP didn't support the OS version any more, so I had to rebuild everything for the new OS version. Anyway, on to the topic of my post. Should 'wsgi.input' become an iterator? Or should we develop a different API for asynchronous applications? On the positive side of the iterator approach, it could make it easier for asynchronous applications to pause waiting for input, and it could in principle support "chunked" transfer encoding of the input stream. However, since we last discussed this, I did some Googling on CGI and chunked encoding. By far and away, the most popular links regarding chunked encoding and CGI, are all about bugs in IIS and Apache leading to various vulnerabilities when chunked encoding is used. :( Once you get past those items (e.g. by adding "-IIS -vulnerability" to your search), you then find *our* discussion here on the Web-SIG! Finally, digging further, I found some 1998 discussion from the IPP (Internet Printing Protocol!) mailing list about what HTTP/1.1 servers support chunked encoding for CGI and which don't. Anyway, the long and short of it is that CGI and chunked encoding are quite simply incompatible, which means that relying on its availability would be nonportable in a WSGI application anyway. That leaves the asynchronous use case, but the benefit is rather strained at that point. Many frameworks reuse the 'cgi' module's 'FieldStorage' class in order to parse browser input, and the 'cgi' module's implementation requires an object with a 'readline()' method. That means that if we switch from an input stream to an iterator, a lot of people are going to be trying to make sensible wrappers to convert the iterator back to an input stream, and that's just getting ridiculous, especially since in many cases the server or gateway has a file-like object to start with. So, I'm thinking we should shift the burden to an async-specific API. But, in this case, "burden" means that we get to give asynchronous apps an API much more suited to their use cases. Suppose that we did something similar to 'wsgi.file_wrapper'? That is, suppose we had an optional extension that a server could provide, to wrap specialized application object(s) in a fashion that then provides backward compatibility to the spec? That is, suppose we had a 'wsgi.async_wrapper', used like this: if 'wsgi.async_wrapper' in environ: controller=environ['wsgi.async_wrapper'](environ) # do stuff with controller, like register its # methods as callbacks return controller The idea is that this would create an iterator that the server/gateway could recognize as "special", similar to the file-wrapper trick. But, the object returned would provide an extra API for use by the asynchronous application, maybe something like: put(data) -- queue data for retrieval when the controller is iterated over finish() -- mark the iterator finished, so it raises StopIteration on_get(length,callback) -- call 'callback(data)' when 'length' bytes are available on 'wsgi.input' (but return immediately from the 'on_get()' call) While this API is an optional extension, it seems it would be closer to what some async fans wanted, and less of a kludge. It won't do away with the possibility that middleware might block waiting for input, of course, but when no middleware is present or the middleware isn't transforming the input stream, it should work out quite well. In any case, the implementation of the methods and the iterator interface are pretty straightforward, either for synchronous or asynchronous servers. What do y'all think? I'd especially like feedback from Twisted folk, as to whether this looks anything like the right kind of API for async apps. (I expect it will need some tweaking and tuning.) But if this is the overall right approach, I'd like to drop the current proposals to make 'wsgi.input' an iterator and add optional 'pause'/'resume' APIs, since they were rather kludgy compared to giving async apps their own mini-API for nonblocking I/O. Comments? Questions?

4 5

[PATCH] twisted.web2 support for gzip stream in/out, this time for real
by Valentino Volonghi 05 Oct '04

05 Oct '04

The patch is attached and should work. At least this is what I can see from the tests I wrote. Attached there are also the 2 files needed for this patch to work. Both files should go in twsited/web2, but since they can be useful to other protocols too maybe there is a better location. -- Valentino Volonghi aka Dialtone Now running FreeBSD 5.3-beta6 Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/

1 0

nevow/stackless bugs -- patch
by Peter Hunt 03 Oct '04

03 Oct '04

I've been trying to run stan on stackless when I ran into a problem with render methods. Stackless uses "stackless.instancemethod" instead of "__builtin__.instancemethod", and "stackless.function" instead of "__builtin__.function". Attached is a svn diff patch of the changes I made to __init__.py in order for stan to support stackless properly. HTH, Peter Hunt

3 3

[PATCH] twisted.web2 support for gzip stream in/out
by Valentino Volonghi 01 Oct '04

01 Oct '04

Following there's a patch for twisted.web2 to support gzip Content-Encoding of chunks. Thanks to Bob Ippolito for answering immediately to my mail about the permission to include his gzstream work in twisted.web2. Here is his mail: === From: Bob Ippolito <bob(a)redivi.com> To: Valentino Volonghi <dialtone(a)gmail.com> Date: Fri, 1 Oct 2004 20:50:35 -0400 Subject: Re: gzstream integration with twisted Twisted.web2 may use my gzstream. On Oct 1, 2004, at 8:33 PM, Valentino Volonghi wrote: > Hi, > I'm helping out Twisted developers to get twisted.web2 out before > October 31. > > We are going to have a very complete implementation of HTTP 1.1 RFC. > Doing so requires also having a gzip/gunzip stream to handle the > Transfer-Coding. > > I've already talked to foom (the main developer of t.web2) and he > asked for your permission to include gzstream in twisted.web2. > > That would be very helpful and your copyright will be kept entirely. > > Let me now your decision please. Thanks for attention. > > -- > Valentino Volonghi aka Dialtone > Now running FreeBSD 5.3-beta6 > Blog: http://vvolonghi.blogspot.com > Home Page: http://xoomer.virgilio.it/dialtone/ === files needed are attached. and patch following. I'll write also a bug for web2 and attach everything, but it's 3.30 AM now, and I'll do that tomorrow. Comments are welcome. Index: twisted/web2/http.py =================================================================== --- twisted/web2/http.py (revision 11886) +++ twisted/web2/http.py (working copy) @@ -33,6 +33,7 @@ from twisted.web2 import http_headers from twisted.web2 import iweb from twisted.web2 import error +from twisted.web2 import gzstream PERSIST_NO_PIPELINE = 2 @@ -301,6 +302,12 @@ else: return ifrange == self.out_headers.getHeader("last-modified") + def requestContentEncoding(self, name): + if name.lower() != 'gzip': + warnings.warn("WARNING! Currently only gzip content is supported") + return + self.out_headers.setHeader('Content-Encoding', name) + # FIXME: these last 3 methods don't belong here. @@ -347,6 +354,8 @@ partialHeader = '' queued = 0 finishedReading = False + inChunkHandler = False + outChunkHandler = False channel = None request = None @@ -449,13 +458,14 @@ def rawDataReceived(self, data): datalen = len(data) + if datalen < self.length: if not self.finished: - self.request.handleContentChunk(data) + self.inChunkHandler(data) self.length = self.length - datalen else: if not self.finished: - self.request.handleContentChunk(data[:self.length]) + self.inChunkHandler(data[:self.length]) extraneous = data[self.length:] channel = self.channel # could go away from allContentReceived. if not self.chunkedIn: @@ -493,6 +503,18 @@ self.channel.queueRequest(self) request = self.channel.requestFactory(self, self.command, self.path, self.version, self.reqHeaders) + + def decode_gzip(self, chunkHandler): + gunzipper = gzstream.GunzipStream() + gunzipper.write = chunkHandler + return gunzipper + + # Set the client chunkhandler if the content is gzipped. + for encoding in self.reqHeaders.get('Content-Encoding', ()): + if encoding == 'gzip': + self.inChunkHandler = decode_gzip(request.handleContentChunk) + else: + self.inChunkHandler = request.handleContentChunk # Reset header state variables del self.reqHeaders @@ -633,11 +655,22 @@ l.append("\r\n") self.transport.writeSequence(l) + + def encode_gzip(chunkHandler): + gzipper = gzstream.GzipStream() + gzipper.write = chunkHandler + return gzipper + + for encoding in headers.getHeader('Content-Encoding', ()): + if encoding == 'gzip': + self.outChunkHandler = encode_gzip(toChunk) + else: + self.outChunkHandler = toChunk def writeData(self, data): if self.chunkedOut: - self.transport.writeSequence(toChunk(data)) + self.transport.writeSequence(self.outChunkHandler(data)) else: self.transport.write(data) ======= HTH -- Valentino Volonghi aka Dialtone Now running FreeBSD 5.3-beta6 Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/

1 1