On Monday, Apr 28, 2003, at 12:17 America/New_York, Clark C. Evans wrote:

On Mon, Apr 28, 2003 at 04:09:35PM +0000, Clark C. Evans wrote: | I was just giving a quick Twisted tutorial to someone using | twisted and as we were breaking page construction into more | than one chunk... an unexpected stumbling block occurred -- | returning NOT_DONE_YET form the resource's render() function. | | I was thinking about two other options: | | 1. Perhaps NOT_DONE_YET could just be None, this | way, it can be the default return value. As I'm | browsing through my code this is the most common | return... why not make it the default.

Err, this isn't exactly what I was thinking. What I was proposing... if during the scope of the render() function, req.write() is called, then a None value would be an allowable return. And if None is returned, req.finish() would be called automagically.

| 2. Alternatively, allow a Deferred to be a return | value. Then the underlying caller can add | result.finish() to the deferred chain. This | has the advantage of not requiring finish() to | really be managed. Either the return value is | a string, a Deferred, (or for backwards compatibiliy | NOT_DONE_YET). In either of the primary cases, | result.finish() always gets called... thus making | it easier on newbies. | | Ideally, both of the above options could be used...

Or, you could just deprecate NOT_DONE_YET and say "if you're doing something that uses req.write you need to return a deferred". For case 1, you return a deferred with an empty rval (None or ''). This signifies that you took care of req.write. For case 2, you return a deferred with the page as the rval. This signifies that you want the server to req.write. I think that's the simplest way, and takes the return values for render down to three, but really two for new code: 1) a string 2) a deferred 3) a deprecated NOT_DONE_YET

On lundi, avr 28, 2003, at 21:57 Europe/Amsterdam, Glyph Lefkowitz wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On Monday, April 28, 2003, at 11:17 AM, Clark C. Evans wrote:

...

On Mon, Apr 28, 2003 at 04:09:35PM +0000, Clark C. Evans wrote: | I was just giving a quick Twisted tutorial to someone using | twisted and as we were breaking page construction into more | than one chunk... an unexpected stumbling block occurred -- | returning NOT_DONE_YET form the resource's render() function. | | I was thinking about two other options: | | 1. Perhaps NOT_DONE_YET could just be None, this | way, it can be the default return value. As I'm | browsing through my code this is the most common | return... why not make it the default.

The reasoning behind not allowing None as a default value is that forgetting is too easy. If you're writing a simple request that has a render() method that looks like

def render(self, request): if self.authenticated: return self.goodies

the default behavior should not be "hang forever".

...

Err, this isn't exactly what I was thinking. What I was proposing... if during the scope of the render() function, req.write() is called, then a None value would be an allowable return. And if None is returned, req.finish() would be called automagically.

What if you wanted to start writing the page in the render() method but keep writing it later? Then we have None as a synonym for NOT_DONE_YET except in certain situations where you've done something to the request?

...

| 2. Alternatively, allow a Deferred to be a return | value. Then the underlying caller can add | result.finish() to the deferred chain. This | has the advantage of not requiring finish() to | really be managed. Either the return value is | a string, a Deferred, (or for backwards compatibiliy | NOT_DONE_YET). In either of the primary cases, | result.finish() always gets called... thus making | it easier on newbies.

I've discussed this with several different people at various times... the trouble is, there isn't really a use-case that Deferreds make easier. render() ends up being a relatively low-level interface, and the NOT_DONE_YET/write/finish API is quite convenient for the stuff that has been implemented with it.

I am definitely a True Believer in the Deferred, but in this case it just doesn't seem worth the inconvenience of deprecating things and shuffling stuff around for a vanishingly small benefit.

Completely agree. Related to this issue, i feel, is what should be best practices for a twisted web application? With all that's available, and little imagination, one can very easily get all tangled up in blue ;) In view of a site that I have the intention to build, I have been trying to select what best implementation architecture to adopt, looking always for the most stupidly simple. I also want to be able to handle errors as nicely as possible for the users -- meaning I would like to, _whenever_ possible, return a fully consistent page (and hopefully the one expected by the user) that provides also the error info, but that would not require the user to do anything more than correct form input data on the _same_ page (without even hitting Back on the browser). I hate the feeling of being surprised with drastic error pages, with all my input data apparently disappeared (also dislike pop-ups asking to repost data, ...). Of course these errors will be of the type 'Ah, that one exists already please try another'... This is all somewhat application specific, and about interfaces, but i feel implementation style can both help or hinder this. A further issue that i find very nagging (at least during development), is that if an error occurs within the presentation layer, the response hangs forever. (These should of course be only development bugs, but i would not like to deliver an app that *may* have presentation bugs yet unknown.) I therefore wrap the call to the real presentation handler in a try/catch. Errors in the business logic are raised to the error handler, so this is not a problem. Since I want my errors to be rendered with a fully functional response, the error callback simply sets the additional response data object, and passes to the presentation layer (this assumes that the business layer has behaved nicely and attached to request the necessary and sufficient data objects, so that the presentation can render it into a coherent page). I am going for this anatomy: class WebAppPage(twisted.web.resource.Resource): def render(self, request): request.setHeader('Content-Type', 'text/html; charset=utf-8') ... d = threads.deferToThread(self.businessLogic, request) # to be threadpooled ;) d.addCallback(self.presentationLogic, request) d.addErrback(self.errBack, request) return NOT_DONE_YET def businessLogic(self, request): # uses functionality that is neatly factored in external modules ;) # and attaches the resulting data objects to request in a designated # container, separate from args (e.g. respargs). # Knows no HTML, never calls request.write() def presentationLogic(self, result, request, error=None): try: self._presentationLogic(result, request, error) except: import sys request.write( '''Ooops, error in presentation layer:<p>%s: %s</p>''' % ( sys.exc_info()[0], ' -- '.join(sys.exc_info()[1].args) )) def _presentationLogic(self, result, request, error=None): # combines the data objects in args, and result (if any - thus may be None) # with rendering templates, html, js, css, ... def errBack(self, error, request): self.presentationLogic([],request,error=error) This suggests that my site's rpy resources should inherit from a subclass of resource, to at least define a common error callback and presentation try/catch wrap for the whole site. However, a small thing escapes me -- how do i guarantee specific and automatic treatment to the request object for all my resources? E.g. setting specific headers, reading cookies/session info, etc., without coding this in every rpy instance? mario

On Mon, Apr 28, 2003 at 01:11:33PM -0400, Bob Ippolito wrote: | Or, you could just deprecate NOT_DONE_YET and say "if you're doing | something that uses req.write you need to return a deferred". ... | I think that's the simplest way, and takes the return values for | render down to three, but really two for new code: | 1) a string | 2) a deferred | 3) a deprecated NOT_DONE_YET Advantages: - keeps the clear error condition when mis-used - NOT_DONE_YET uglyness goes away - request.finish() can be handled automatically, ie, the default errback/callback and the end of the chain can make sure that request.finish() is called - could provide for better default exception handling, the first thing I did as a newbie was dig to find out how to get those wonderful traceback pages working for deferred failures... why not let it be the default? I'd add one more... option though: 4) If request.finish() has already been called, then it is acceptable to return None On Mon, Apr 28, 2003 at 06:14:57PM -0400, Bob Ippolito wrote: | On Monday, Apr 28, 2003, at 15:57 America/New_York, Glyph Lefkowitz wrote: | >>| 2. Alternatively, allow a Deferred to be a return | >>| value. Then the underlying caller can add | >>| result.finish() to the deferred chain. This | >>| has the advantage of not requiring finish() to | >>| really be managed. Either the return value is | >>| a string, a Deferred, (or for backwards compatibiliy | >>| NOT_DONE_YET). In either of the primary cases, | >>| result.finish() always gets called... thus making | >>| it easier on newbies. | > | >I've discussed this with several different people at various times... | >the trouble is, there isn't really a use-case that Deferreds make | >easier. render() ends up being a relatively low-level interface, and | >the NOT_DONE_YET/write/finish API is quite convenient for the stuff | >that has been implemented with it. | > | >I am definitely a True Believer in the Deferred, but in this case it | >just doesn't seem worth the inconvenience of deprecating things and | >shuffling stuff around for a vanishingly small benefit. | | The error handling inherent with Deferred makes it worth it IMHO. I | see a lot of NOT_DONE_YET responses fail and it just hangs for the web | user unless the developer of the function did a lot more work.. a | deferred could just errback on a failure rather than having to summon | the web traceback mechanism manually. I've had two use cases for NOT_DONE_YET they are: 1. You've already use req.write() several times followed by req.finish()... and then return NOT_DONE_YET. This is confusing usage, primarly beacuse you *are* done. This usage is supported by option #4 above. 2. You've setup a async call to finish handling the request, and in this case, it makes sence to return Deferred. In this case returning NOT_DONE_YET, and having to call req.finish() is just extra, unnecessary baggage; further, default error handling isn't provided when it should be. NOT_DONE_YET is less than optimal in both use cases where it is used, both of these cases confused a newbie that I was trying to teach. Best, Clark

On Fri, May 09, 2003 at 05:02:30PM +0200, Matthias Urlichs wrote: | Clark C. Evans schrub am Mon, 28 Apr 2003 16:09:35 +0000: | > 2. Alternatively, allow a Deferred to be a return | > value. Then the underlying caller can add result.finish() to the | > deferred chain. This has the advantage of not requiring finish() | > to really be managed. | | I think that wouldn't work. Consider: | | def render(self,req): | d = somewhere.GetDeferredData() | d.addCallback(self.render_more,req) | return d | def render_more(self,data,req): | d = somewhere.GetMoreDeferredData(data) | d.addCallback(self.render_even_more,req) | return d | def render_even_more(self,data,req): | req.write("Done!") | | This ends up calling req.finish() befor req.write(). Just beacuse you can write code that will violate a given protocol doesn't mean it's a bad protocol. The suggestion would work just fine... you just picked a way to get around the intent of returning Deferred. IMHO, Clark

Hi, Glyph Lefkowitz:

While in theory he _could_, this example doesn't :-) Chained Deferred results will make this Do The Right Thing. To be sure, I asked the Python interpreter about it -

stare at code scratch head stare some more *lightbulb goes off* Nice. Thank you. -- Matthias