On 03:01 pm, mithrandi-twisted-web@mithrandi.za.net wrote:
* glyph@divmod.com <glyph@divmod.com> [2007-09-07 01:43:19 -0000]:
Use a recursive generator that yields Deferreds for loading each row.
Thanks for the example! Combined with a DeferredQueue to pass the data from the callback to the rendering "loop", this approach seems to work perfectly.
No problem! Your Fan Club Dollars At Work!22.
Here's an example - which I even looked at in a web browser :) - that should help to get you started. (The athena stuff is beside the point, it's just the quickest way I know to get a server serving a Nevow page...)
The athena-widget plugin is definitely very handy for quick examples like this.
We've also been discussing a slightly more general "twistd nevow" plugin. More on this as it develops.
Warning: I haven't tested this on really large datasets, so it might have disastrous performance consequences. I don't *think* that it will, but I know there are some booby-traps in the rendering pipeline to snare the unwary.
As far as I can tell, nothing particularly nasty is happening when I use this; if the overhead of producing each row is very low, all the indirection through generators / deferreds might be costly, but that's certainly not the case for my code (the database and formatting operations are relatively expensive).
Good to know.
http://twistedmatrix.com/trac/ticket/1875
I'm not really sure what I can do to drive the ticket along, but if there is something, please let me know.
I vaguely remembered this ticket when I wrote that comment :-). Some of the things that are blocking it are on my list of stuff to get back to when I get back to Twisted development at all.