Mailman 3 November 2005 - Twisted-web

Test
by Philippe Lafoucrière 05 Aug '21

05 Aug '21

Hi all just a test

3 2

web2 response.setCookie and request.getCookie shortcuts?
by Clark C. Evans 21 Jan '06

21 Jan '06

I'm still having fun with web2, it is such a huge improvment over twisted.web and I'm absolutely thrilled with it. However, I have a few suggestions: 1. It would be supreme if render() and locateChild() just provided me the Request object; the incantation to ctx.locate() seems a tedious an unnecessary step. 2. I would love a request.getCookie('cookie-name') and response.setCookie('cookie-name','cookie-value', path = '/') helper; that adds the given cookie to the current list. 3. Make the 'path' in the Cookie constructor and setCookie default to '/'. Almost all of the time you want your cookies to be site-wide; adding path = '/' to each request adds a point of failure that makes the system less usable. Kind Regards, Clark P.S. Following is sample cookie code that works with web2 --- from twisted.internet import reactor, defer from twisted.web2 import server, http, resource, channel, iweb from twisted.web2.http_headers import Cookie class Toplevel(resource.Resource): addSlash = True def render(self, ctx): request = ctx.locate(iweb.IRequest) found = False for cookie in request.headers.getHeader('cookie',[]): if cookie.name == 'testing': print "found", cookie.value found = True response = http.Response(stream="Hello monkey!") if not found: print "not-found" cookie = Cookie("testing","womble",path="/") response.headers.setHeader('Set-Cookie',(cookie,)) return response # Standard twisted application Boilerplate from twisted.application import service, strports application = service.Application("demoserver") site = server.Site(Toplevel()) s = strports.service('tcp:8080', channel.HTTPFactory(site)) s.setServiceParent(application)

6 12

Error output for bad deferred method signatures?
by Lenny G Arbage 02 Dec '05

02 Dec '05

I'm curious why errors aren't reported when deferreds are fired if the deferred method has a bad signature. For example, if the following is run: import httplib, sys from twisted.web import client from twisted.internet import reactor, threads, defer from twisted.protocols import http from twisted.python import failure class getYahoo(object): def __init__(self): factory = client.HTTPClientFactory("http://www.yahoo.com") reactor.connectTCP("www.yahoo.com",80,factory) factory.deferred.addCallback(self._gotResponse, factory) factory.deferred.addErrback(self._errSendReq, factory) self.deferred = factory.deferred def _gotResponse(self, resp, factory): print "yeah!" print resp def _errSendReq(self, err, factory): print "shucks!" print err def sendRequest(): return getYahoo().deferred if __name__ == '__main__': def test(msg): print msg print respOrErr d = sendRequest() d.addCallback(test, "success") d.addErrback(test, "failure") reactor.run() The deferred _gotResponse() and _errSendReq() fire perfectly, but the test() function in __main__ is not called for "success" or "failure". This is expected as the signature is wrong -- it should be 'test(respOrErr, msg)'. What isn't clear [to me] is why this doesn't produce an error upon firing the deferred, but instead produces an error when the reactor shuts down [via ctrl-c], which could be hours later. I have an instance of this that is more severe, where the error output is *never* produced (maybe because of threading?). Is this the expected behavior, or am I doing something atrocious to produce it? Thanks, Lenny __________________________________ Yahoo! Music Unlimited Access over 1 million songs. Try it free. http://music.yahoo.com/unlimited/

2 3

Nevow Athena LiveFragment without allowedMethods
by Paul Reznicek 01 Dec '05

01 Dec '05

Hi JP & co., I've just started playing with current ATHENA - very nice !!! Is there any deeper security reason to use the "allowedMethods" mapping additionally to the simple fact, that a fragment implements a method only if it is necessary? If not, please consider to apply attached patch. It makes the code and use of it a little bit simpler. Thanks for your great work, best regards, Paul Reznicek Index: nevow/athena.py =================================================================== --- nevow/athena.py (Revision 3368) +++ nevow/athena.py (Arbeitskopie) @@ -407,20 +407,18 @@ <form onsubmit="Nevow.Athena.refByDOM(this).callRemote('foo', bar); return false;"> - By default, only methods named in the C{allowedMethods} mapping - may be invoked by the client. """ - allowedMethods = {} - def rend(self, context, data): myID = self.page.addLocalObject(self) context.fillSlots('nevow:athena_id', myID) return super(LiveFragment, self).rend(context, data) def locateMethod(self, ctx, methodName): - if methodName in self.allowedMethods: - return getattr(self, methodName) + if not methodName.startswith('_'): + method = getattr(self, methodName, None) + if method: + return method raise AttributeError(methodName) Index: examples/athenademo/athenatest.py =================================================================== --- examples/athenademo/athenatest.py (Revision 3368) +++ examples/athenademo/athenatest.py (Arbeitskopie) @@ -23,7 +23,6 @@ tags.form(action='#', onsubmit='return test(test_ClientToServerArgumentSerialization(this));')[ tags.input(type='submit', value='Test Client To Server Argument Serialization')]]) - allowedMethods = {'test': True} def test(self, i, f, s, l, d): self.assertEquals(i, 1) self.assertEquals(f, 1.5) @@ -65,7 +64,6 @@ tags.form(action='#', onsubmit='return test(test_ClientToServerResultSerialization(this));')[ tags.input(type='submit', value='Test Client To Server Result Serialization')]]) - allowedMethods = {'test': True} def test(self, i, f, s, l, d): return (i, f, s, l, d) @@ -104,7 +102,6 @@ tags.input(type='submit', value='Test Client To Server Asynchronous Exception Result')]]) - allowedMethods = {'testSync': True, 'testAsync': True} def testSync(self, s): raise Exception(s) @@ -135,7 +132,6 @@ tags.form(action='#', onsubmit='return test(test_ServerToClientArgumentSerialization(this));')[ tags.input(type='submit', value='Test Server To Client Argument Serialization')]]) - allowedMethods = {'test': True} def test(self): return self.page.callRemote('test_Reverse_ServerToClientArgumentSerialization', 1, 1.5, u'hello', {u'world': u'value'}); @@ -159,7 +155,6 @@ tags.form(action='#', onsubmit='return test(test_ServerToClientResultSerialization(this));')[ tags.input(type='submit', value='Test Server To Client Result Serialization')]]) - allowedMethods = {'test': True} def test(self): def cbResults(result): self.assertEquals(result[0], 1) Index: examples/athenademo/typeahead.py =================================================================== --- examples/athenademo/typeahead.py (Revision 3368) +++ examples/athenademo/typeahead.py (Arbeitskopie) @@ -20,7 +20,6 @@ class TypeAheadFieldFragment(athena.LiveFragment): docFactory = loaders.stan(T.input(type="text", id="typehere", **athena.liveFragmentID)) - allowedMethods = { 'loadDescription' : True } def loadDescription(self, typed): if typed == '': Index: examples/livefragments/livefrag.py =================================================================== --- examples/livefragments/livefrag.py (Revision 3368) +++ examples/livefragments/livefrag.py (Arbeitskopie) @@ -40,11 +40,10 @@ def __init__(self, label, *a, **kw): super(CooperativeFrag, self).__init__(*a, **kw) - self.allowedMethods = {label: True} setattr(self, label, lambda: unicode(label)) def render_submit(self, ctx, data): - return "submitIt(this, '%s'); return false;" % (self.allowedMethods.keys()[0],) + return "submitIt(this, 'label'); return false;" class Root(rend.Page): def child_(self, ctx):

2 3

Sessions and Authentication for Web2
by Clark C. Evans 30 Nov '05

30 Nov '05

Hello. I am in the middle of porting my application(s) to web2, and it seems that general authentication and session handling is not yet implemented. My requirement is rather simple: 1. When the user connects to my system (via HTTPS), I check for a cookie that indicates their server-side session. If they have a session active, I touch() the session so that the expiration date is pushed out. 2. If the cookie does not exist, or the session has expired, then I send them through one of 3 authentication mechanisms depending on either their IP, a query argument, or what resource they are trying to access. a) If they are coming from a YALE IP address and have not explicitly asked for HTTP Digest authentication, I use the CAS authentication mechanism: http://www.yale.edu/tp/auth/cas20.html b) Else, if they are viewing a limited "semi-restricted" set of resources, I use another custom challenge-response mechanism that is similar to CAS (but slightly different) c) Otherwise, I use HTTP Digest Authenitication. All of these methods require that the request be cut short, and either a 401 is returned, or a HTTP Redirect (for CAS). 3. Once I have an authorized user; I create a session for them, and set a 20 minute timeout. I might have multiple requests active for the same session. Once I have a session, I need a few things: 1. A way to getSession() from a given request; in particular, most resource authorization is based on the user's identity. 2. Request specific storage of temporary working data so that requests in-progress can accumulate results before the response is generated. 3. Session specific storage, so that information about the user's context can be maintained. This needs to be synronized with a database server since the Session may be shared across multiple servers. 4. A JobQueue for that particular Session. Many queries take a long time to run; and incremental results and/or failure notices need to be managed between each request. Of these requirements, #1 is most important, and #4 is a wish-list item for the next version of my application. We had a chat on IRC today, and a few random notes/ideas emerged: a) There was a strong preference to keep Session and Authentication completely separate. Currently Nevow's Guard mixes the two and thus is overly complicated and non-modular. Nevow also uses Componentized, which is depreciated (in favor of... ?) http://divmod.org/trac/browser/trunk/Nevow/nevow/guard.py http://divmod.org/trac/browser/import/Nevow/sandbox/mg/guarded/guard.py b) Besides Nevow's GuardSession, there are a few other session implementations besides the older twisted.web implementation. A requtil.py in web2 seems to be a straight-forward port of tw.web's sessions. Quixote also has a session manager, http://quixote.idyll.org/session2/ which is licensed to steal. c) Exarkun expressed a strong (ok, mandatory) preference for the use of tw.cred in any Authentication solution. However, it was noted that tw.cred does not allow for challenge-response authentication mechanisms (which all of mine are). Specific examples were noted: twisted.protocols.sip, SASL, OTP d) There was talk that the Session manager should actually be a filter on the Response object; so that it can rewite paths and/or inject cookies as needed before the response is sent. This seems a bit complicated, no? e) I can probalby help in some ways, but I'm not a Twisted expert. I could continue to collect requirements and post; but I have short term deadlines (as usual). If you wish I could put two files in twisted.web2, marked 'experimental': session.py This would contain session manager, how ever it is eventually implemented. guard.py This would contain the gule to link tw.cred to web2. I prefer an iterative; implement the simplest solution first, and then grow it as needed approach (with clear marks on instability). I wouldn't mind owning these two files and collecting feedback, etc. Anyway, thanks for listening. Any thoughts as to how to best proceed would be very welcome. Kind Regards, Clark P.S. For now, I'm hacking my own session and auth code. There is a requtil.py in web2 which is a stab at sessions, and both David Reid and I have HTTP Auth code in our sandbox.

7 15

Concurrency in a twisted.web application
by Ananda Tallur 29 Nov '05

29 Nov '05

Dear twisted.web developpers and users, I have written a twisted application for viewing and exporting movies. These movies are in a format specific to my company. I use : twisted 2.1.0, twisted.web 0.5.0, and nevow 0.5.0 (all html templates are written using nevow). In this web application, there is a feature for exporting a movie file into mpeg, and sending it to the client web browser (file download). This feature is handled by a resource object which is a subclass of static.File. Exporting a movie into mpeg implies : -> opening the original video file -> doing the conversion (which can take as long as 1 minute or more) -> sending the resulting mpeg file to the browser (file download) The problem is that while the twisted web application is working handling the export request, the twisted web server is not responsive to any other HTTP request, as long as the video conversion and file transfer is not finished. I would like to know if there is a standard or recommanded way of setting up concurrency into a twisted.web application. And also if there is a sample program available somewhere. I would like my web application to be still responsive to other HTTP requests while : -> file conversion to mpeg is being processing -> mpeg file is being transfered to the client browser (which is handled by a static.File subclass resource) Thank you very much for all your answers. I appreciate very much working with twisted.web and interested to get more insight into the way twisted core handles requests. Anand

3 3

About Authentication/Membership
by QnA 29 Nov '05

29 Nov '05

Hi, I am new to Twisted project and my job is to write a few web-pages under twisted framework. I want to ask if Twisted provide a user authentication and session library just like PHP or JSP?? Or may anyone can give me some idea or articles I should read? Thank you very much for all kindly help. Regards, QnA

1 0

Some questions regarding nevow.athena
by Johannes Beigel 28 Nov '05

28 Nov '05

Hi. We are currently using Twisted+Nevow to build web frontends. There's some AJAX stuff involved and we are using several JavaScript and Python packages and mostly custom code for this. With the new Nevow versions a restructuring of LiveEvil/Livepage code to the new athena package has begun. I looked at it and it looks very promising, especially the way it integrates into existing Nevow code and templates. Is there some API documentation/example code available (some more than in the SVN tree)? The code states that the API could be rather unstable. Would you rather recommend to wait for a little more mature athena before using it in production code? -jojo

1 0

Json breakage
by Stephen Thorne 28 Nov '05

28 Nov '05

Hi, Cory noted that json is broken, jp told me about it, I looked at it and hated regexps. This patch *seems* to work, and passes the tests. The tests are insubstantial and my grasp of python's regular expressions is crap, so I'd like someone to review this patch before I commit it. Stephen.

1 0

Serving different resource types from the same directory
by Olivier Laurent 27 Nov '05

27 Nov '05

On dim, 27 nov 2005, at 16:18, oli wrote: I'm trying to serve 2 different kind of resources from the same directory. I can, with no problem, serve one kind or the other but not the two resource types at the same time. Examples abounds on the web on how to serve one kind of resources from subdirectory /foo and another kind from subdirectory /bar. But no example on how to serve different kind of resources from the same directory. Here is how I tried: # [...] PATH = '/var/www/' pt_resource = static.File(PATH) pt_resource.processors = { '.pt' : PTResource } pt_resource.indexNames = [ 'index' + '.pt' ] rst_resource = static.File( PATH ) rst_resource.processors = { '.rst' : RSTResource } rst_resource.indexNames = [ 'index' + '.rst' ] #root = Resource() # I tried this too root = static.File(PATH) root.putChild(PATH, pt_resource) root.putChild(PATH, rst_resource) reactor.listenTCP( 80, server.Site( root ) ) reactor.run( ) # this is the end What's wrong ? What did I forget ? Is it possible ? Thank you. -- Olivier Laurent

3 3