[Twisted-Python] Woven: access to submodels
I'm trying to replicate the functionality of http://eyes.puzzling.org/ (currently powered by a bunch of Python CGI scripts running on an Apache webserver) in Woven. eyes.puzzling.org is a kind of a minimalist blog, with pages that list recent entries, the authors of the entries, and so on. At the moment, I'm trying to replicate the authors functionality (2 different sets of pages: one which lists all the authors and links to their individual page; and one for each author, which lists their individual articles. The /authors/ page (listing all the authors and linking to their individual page) is generated by objects of the AuthorsPage subclass. For each author, I would like to generate a /author/AUTHORNAME/ page which lists their individual entries. I am trying to create these pages from AuthorsPage's getDynamicChild method. The idea is to pass the already constructed submodel -- a dictionary representing the individual author -- to the IndividualPage object I will construct to represent the individual author's page, rather than have IndividualPage make its own database query for that information. At present, as you can see below, I'm using getSubmodel to retrieve the (dictionary) submodel associated with a particular author. However, this returns a twisted.web.woven.model.DeferredWrapper instance, rather than the dictionary itself. Should I make IndividualPage add callbacks to the DeferredWrapper, or am I missing something crucial about retrieving submodels or is this model more fundamentally incorrect? For example, should I be passing submodels around like that at all? If so, is there a higher level call I should make to access the submodel? The two classes following are the AuthorsPage and IndividualPage classes described above. Thanks, -Mary class IndividualPage(page.Page): # getTemplate munges two template files together template = getTemplate("/home/mary/cvs/Projects/Eye/author-page.html") def initialize(self, *args, **kwargs): self.amodel = kwargs['amodel'] def wmfactory_author(self, request): return self.amodel['author'] class AuthorsPage(page.Page): template = getTemplate("/home/mary/cvs/Projects/Eye/authors-page.html") def initialize(self, *args, **kwargs): self.dbpool = kwargs['pool'] self.authors = None def wmfactory_authors(self, request): query = self.dbpool.runQuery("SELECT DISTINCT posts.name, link, password FROM authors, posts WHERE authors.name = posts.name ORDER BY name;") query.addCallback(self.makeAuthors) return query def makeAuthors(self, rows): ''' Construct self.authors, return them as a list @return a list of author dictionaries ''' authors = {} for row in rows: (name, link, password) = row authors[name] = { 'name' : name, 'link' : link, 'password' : password } return authors def wvfactory_alink(self, request, node, model): return AuthorLink(model) def getDynamicChild(self, name, request): return IndividualPage(amodel=self.getSubmodel('authors')[name]))
On Sun, 13 Jul 2003, Mary <mary-twisted@puzzling.org> wrote:
def getDynamicChild(self, name, request): return IndividualPage(amodel=self.getSubmodel('authors')[name]))
Rar! You are very evil. Please do not reinvent SQL in Python: write *another* select query, that only selects posts from this author. -- Moshe Zadka -- http://moshez.org/ Buffy: I don't like you hanging out with someone that... short. Riley: Yeah, a lot of young people nowadays are experimenting with shortness. Agile Programming Language -- http://www.python.org/
On Sun, Jul 13, 2003, Moshe Zadka wrote:
On Sun, 13 Jul 2003, Mary <mary-twisted@puzzling.org> wrote:
def getDynamicChild(self, name, request): return IndividualPage(amodel=self.getSubmodel('authors')[name]))
Rar! You are very evil. Please do not reinvent SQL in Python: write *another* select query, that only selects posts from this author.
I have to anyway... the SELECT query you saw doesn't extract any posts at all, it extracts the names of authors (posts.names -- plus a little extraneous information) who have made posts. So I take it that in this design the arguments to __init__/initialize of child IndividualPage would be just the name of the author? This leaves a few questions still: 1. Pure design issue: There still needs to be a query along the lines of "does this author in fact exist?" so that /author/Mary/ generates a list of my articles and /author/Humphrey/ generates an error page. This was the query I was trying to avoid by passing the submodel around -- the "get me all posts by Mary" query was always going to be part of IndividualPage. At present, my model does not have the IndividualPage make a query to this effect, it relies on its parent AuthorsPage to have accessed a list of all authors in the system (actually, the SELECT query given ignores authors who have made no posts, a deliberate choice). Should I have IndividualPage access this information itself? If not, I still continue to rely on the information presently accesible to AuthorsPage -- how should I make this information available to IndividualPage? (related to point 3). If I don't use the authors model, then I have to make an essentially identical query anyway. There's a few options about where I make the query, how many times I make it, and where I should store the results. And of course, where (and how) I pass a Deferred around. 2. More generally, since I'm unfamiliar with the design pattern(s) associated with Woven, what level of information sharing in this sense is acceptable between parents and children? (Pointers to other sources of information would be welcome, since this question is pretty open ended)? 3. (Generalisation of point 1.) To whatever extent it is even acceptable to share query results of some kind between parents and children (perhaps in the form of Models), what functions do I call in Woven to share the information? Should an object ever access its own sub-models? -Mary
On Sun, 13 Jul 2003, Mary <mary-twisted@puzzling.org> wrote:
1. Pure design issue: There still needs to be a query along the lines of "does this author in fact exist?" so that /author/Mary/ generates a list of my articles and /author/Humphrey/ generates an error page. This was the query I was trying to avoid by passing the submodel around -- the "get me all posts by Mary" query was always going to be part of IndividualPage.
Ah, I see. Well, a somewhat thorny issue in current web code [that we may fix someday in the future, but you're not interested in that] is that child-getting is synchronous. This means that an easy answer is currently beyond us. A non-easy way to do that is to hack around: from twisted.web import resource, server class MaybeIndividualPage(resource.Resource): isLeaf = 1 def __init__(self, author): resource.Resource.__init__(self) self.author = author def render(self, request): def raiseException(): raise LookupError def simulateServer(value): if value == server.NOT_DONE_YET: return request.write(value) request.finish() query = deferredWhichIsTrueIfExists(self.author).addCallback( lambda x: (x or error.NoResource()) and IndividualPage(self.author).getChildForRequest(request ).render(request)).addCallback( simulateServer) return server.NOT_DONE_YET Then, in getDynamicChild, return MaybeIndividualPage(author) instead of IndividualPage(author). Basically, this is explicitly asyncing the child-getting mechanism by going through the async-ready render method. I realize it's not the best-looking code in the world, but it should get the job done.
2. More generally, since I'm unfamiliar with the design pattern(s) associated with Woven, what level of information sharing in this sense is acceptable between parents and children? (Pointers to other sources of information would be welcome, since this question is pretty open ended)?
Well, we don't have any real understanding of this issue either, since Woven is not old enough. My gut instinct would be towards "pretty high", though. -- Moshe Zadka -- http://moshez.org/ Buffy: I don't like you hanging out with someone that... short. Riley: Yeah, a lot of young people nowadays are experimenting with shortness. Agile Programming Language -- http://www.python.org/
I apologize for not responding immediately. On Saturday, July 12, 2003, at 7:21PM, Mary wrote:
I'm trying to replicate the functionality of http://eyes.puzzling.org/ (currently powered by a bunch of Python CGI scripts running on an Apache webserver) in Woven.
eyes.puzzling.org is a kind of a minimalist blog, with pages that list recent entries, the authors of the entries, and so on.
Cool! Have you seen Blog, in the twistedmatrix.com cvs? It is out of date and needs work, but feel free to steal the code/take over the project/whatever. <snip>
For each author, I would like to generate a /author/AUTHORNAME/ page which lists their individual entries. I am trying to create these pages from AuthorsPage's getDynamicChild method. The idea is to pass the already constructed submodel -- a dictionary representing the individual author -- to the IndividualPage object I will construct to represent the individual author's page, rather than have IndividualPage make its own database query for that information.
That's good -- that sounds like a good design. There are no rules when it comes to Woven architecture, but reusing your Model objects, and model production logic, across multiple pages sounds like a good idea to me.
At present, as you can see below, I'm using getSubmodel to retrieve the (dictionary) submodel associated with a particular author. However, this returns a twisted.web.woven.model.DeferredWrapper instance, rather than the dictionary itself. Should I make IndividualPage add callbacks to the DeferredWrapper, or am I missing something crucial about retrieving submodels or is this model more fundamentally incorrect?
As Moshe said in his reply, currently the URL traversal process is synchronous, so if you need to wait on a Deferred to construct the next URL segment Resource you're going to be doing a lot more work than you should have to. Woven attempts to hide Deferreds from the programmer by pausing rendering and resuming it when the Deferred fires. So, fortunately, things may be simpler than they seem. You can just pass the DeferredWrapper as the Main Model to the next Page instance you construct, and as long as you are going to be rendering that Page next, Woven will just wait on it as it would any other Deferred. from twisted.web.woven import page from twisted.web.woven import model from twisted.internet import defer from twisted.internet import reactor class DeferredProducer(model.MethodModel): def wmfactory_someDeferred(self, reqeuest): d = defer.Deferred() reactor.callLater(0.5, self.produce, d) return d def produce(self, d): d.callback( "Hello, here is some text" ) def wmfactory_someNotDeferred(self, request): return "Hi there" class MainPage(page.Page): template = """<html> <h3 model="someNotDeferred" view="Text">qawsf</h3> <span model="someDeferred" view="Text">qwensd</span> </html>""" def getDynamicChild(self, name, request): return ChildPage( self.model.getSubmodel(request, 'someDeferred')) class ChildPage(page.Page): template = """<html> Hello! Our main model is a deferred, so the next node will have to wait <span model="." view="Text">ncxnz</span> </html>""" resource = MainPage(DeferredProducer())
On Sat, Jul 19, 2003 at 12:08:35PM -0700, Donovan Preston wrote:
As Moshe said in his reply, currently the URL traversal process is synchronous, so if you need to wait on a Deferred to construct the next URL segment Resource you're going to be doing a lot more work than you should have to.
However, there is (now) a twisted.web.util.DeferredResource. Any time a Resource is expected of you, but all you have is a Deferred that will eventually result in a Resource, you can wrap the Deferred up in one of these babies, and it'll DTRT. It's not necessary most of the time, but it can be extremely useful. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://twistedmatrix.com/users/radix.twistd/
On Saturday, July 19, 2003, at 1:15PM, Christopher Armstrong wrote:
On Sat, Jul 19, 2003 at 12:08:35PM -0700, Donovan Preston wrote:
As Moshe said in his reply, currently the URL traversal process is synchronous, so if you need to wait on a Deferred to construct the next URL segment Resource you're going to be doing a lot more work than you should have to.
However, there is (now) a twisted.web.util.DeferredResource. Any time a Resource is expected of you, but all you have is a Deferred that will eventually result in a Resource, you can wrap the Deferred up in one of these babies, and it'll DTRT. It's not necessary most of the time, but it can be extremely useful.
Thank you for moving that into util. However, it doesn't support chaining -- the thing that is returned will be rendered. So it's no different than what I suggested. dp
On Sat, Jul 19, 2003 at 03:24:38PM -0700, Donovan Preston wrote:
On Saturday, July 19, 2003, at 1:15PM, Christopher Armstrong wrote:
On Sat, Jul 19, 2003 at 12:08:35PM -0700, Donovan Preston wrote:
As Moshe said in his reply, currently the URL traversal process is synchronous, so if you need to wait on a Deferred to construct the next URL segment Resource you're going to be doing a lot more work than you should have to.
However, there is (now) a twisted.web.util.DeferredResource. Any time a Resource is expected of you, but all you have is a Deferred that will eventually result in a Resource, you can wrap the Deferred up in one of these babies, and it'll DTRT. It's not necessary most of the time, but it can be extremely useful.
Thank you for moving that into util. However, it doesn't support chaining -- the thing that is returned will be rendered. So it's no different than what I suggested.
Chaining? I don't understand. It can be getChilded, if that's what you mean. Or do you mean a DeferredResource wrapping a Deferred wrapping a DeferredResource wrapping a Deferred resulting in something else? I don't see why that would be a problem, but I haven't tried it. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://twistedmatrix.com/users/radix.twistd/
Replying to an old thread here, I work though my problems slowly... On Sat, Jul 19, 2003, Donovan Preston wrote:
So, fortunately, things may be simpler than they seem. You can just pass the DeferredWrapper as the Main Model to the next Page instance you construct, and as long as you are going to be rendering that Page next, Woven will just wait on it as it would any other Deferred.
[some bits of code snipped]
class DeferredProducer(model.MethodModel): def wmfactory_someDeferred(self, reqeuest): d = defer.Deferred() reactor.callLater(0.5, self.produce, d) return d
def produce(self, d): d.callback( "Hello, here is some text" )
class MainPage(page.Page): template = """<html> <h3 model="someNotDeferred" view="Text"> qawsf</h3> <span model="someDeferred" view="Text"> qwensd</span> </html> """
def getDynamicChild(self, name, request): return ChildPage( self.model.getSubmodel(request, 'someDeferred'))
class ChildPage(page.Page): template = """<html> Hello! Our main model is a deferred, so the next node will have to wait <span model="." view="Text"> ncxnz</span> </html> """
Sorry, I'm not too familiar with the stages in which this gets sent to the browser, but this code assumes that the ChildPage is always going to exist. In my blog model, obviously there are authors that don't exist. So /author/Mary/ would be represented by a ChildPage Resource, but /author/Donovan/ should 404. Is that possible in a code structure like this, or is it too late to produce the appropriate error at the: <span model="." view="Text"> ncxnz</span> stage, which is the first time Twisted Web needs to wait on our Deferred main model? -Mary
On Sat, Aug 09, 2003, Mary wrote:
In my blog model, obviously there are authors that don't exist. So /author/Mary/ would be represented by a ChildPage Resource, but /author/Donovan/ should 404. Is that possible in a code structure like this, or is it too late to produce the appropriate error at the: <span model="." view="Text"> ncxnz</span> stage, which is the first time Twisted Web needs to wait on our Deferred main model?
OK, as per discussion on IRC, I used a DeferredResource instead, and return a ChildPage if the database query finds an author and an ErrorPage otherwise. -Mary
participants (4)
-
Christopher Armstrong -
Donovan Preston -
Mary -
Moshe Zadka