[Twisted-Python] Deferred getChild (#3621)
Hi, I was wondering if anyone had any concrete thoughts about what should happen with IResource.getChild being able to return Deferreds, or, in general, the ability to asynchronously get children of a resource. The ticket for this is #3621 <http://tm.tl/3621>. My use case for this is txYoga <https://github.com/lvh/txyoga>. Long story short, it lets you write REST-y webapps. So, you could have something like: http://www.twistedmatrix.com/labs/glyph In local jargon, labs is a collection, and glyph is an element in it. Right now, the only collection that actually exists is an in-memory one, and the next one I'm going to write is a SQLite one, so blocking on child access seems relatively reasonable. In general, however, this should be implemented using Deferreds, since the information about glyph might be stored in a database far, far away. The logical way to do that for me seems to be that in getChild, you'd I have found a mailing list post<http://www.twistedmatrix.com/pipermail/twisted-python/2002-October/001847.html>(from 2002) where Christopher Armstrong ostensibly solved the problem. I can't find any replies to that mailing list post. I remember Glyph saying something about how that could potentially change/break public API. I understand that reservation, but I don't see how it'd be that bad. Existing code that always immediately returns a resource would still work -- it would merely only use a part of the API it's allowed to use (in this case, it'd ignore the fact that it is allowed to return a deferred). Since a Deferred isn't an IResource, it wouldn't be a legal value to return now, anyway. Obviously, code that previously relied on getChild always returning a child *right now*, when now that getChild will occasionally return a deferred, would break. I'm not sure yet if that's a serious problem, or just a breach of the contract that code previously had. I'm thinking it's the latter, but maybe my subconscious is cheating to get to the easier answer. I don't actually call getChild myself in my application code. I call it in tests, and Twisted calls it for me in production. Maybe I'm missing major use cases for getChild there? I could definitely see how that works. Thoughts welcome :) cheers lvh
On Monday 30 May 2011, Laurens Van Houtven wrote:
My use case for this is txYoga <https://github.com/lvh/txyoga>. Long story short, it lets you write REST-y webapps. So, you could have something like:
http://www.twistedmatrix.com/labs/glyph
In local jargon, labs is a collection, and glyph is an element in it. Right now, the only collection that actually exists is an in-memory one, and the next one I'm going to write is a SQLite one, so blocking on child access seems relatively reasonable. In general, however, this should be implemented using Deferreds, since the information about glyph might be stored in a database far, far away.
A possible alternative would be to create a child resource that remembers the child name and do the database lookup in the render method. If there is a matching record, render a page with its data, otherwise render a 404 page. I cannot predict whether the resulting code would be clean or ugly though, so I don't know how feasible this approach is. Bye, Maarten
Right, it's definitely possible to work around this problem using proxy children, but going forward I don't think that's how Twisted *should* behave. (I actually tried this, and nope, it's not exactly pretty :-( ) The reason for that is that that would mean there's two ways to do things. Suppose a child is missing. In the case of a blocking/immediate lookup, you'd return an error resource from getChild. In the case of an asynchronous lookup, you'd return a resource immediately, regardless of whether the lookup fails or not (since you don't know yet) whose render method returns NOT_DONE_YET and eventually finishes a request. If getChild supported deferreds, you'd return an deferred error page from getChild in the asynchronous case, and you'd return the error page as you always have in the synchronous case. That makes a lot more sense to me. What I want is "so, there's a value (==IResource), but I don't know what it is yet", and it seems to me the natural way of expressing that in Twisted is a Deferred. cheers lvh
On May 30, 2011, at 6:51 AM, Laurens Van Houtven wrote:
I remember Glyph saying something about how that could potentially change/break public API. I understand that reservation, but I don't see how it'd be that bad. Existing code that always immediately returns a resource would still work -- it would merely only use a part of the API it's allowed to use (in this case, it'd ignore the fact that it is allowed to return a deferred). Since a Deferred isn't an IResource, it wouldn't be a legal value to return now, anyway.
The issue is with code that calls getChild, or getChildWithDefault, or any of the APIs that are listed on #3621's description, or anything derived from them. Any kind of resource wrapper will have to deal with these APIs, and there are a surprisingly high number of these kinds of things out in the wild. Every public interface has callers and implementations, and both may exist out in the wild for just about any interface. So, the problem is that we need a new interface. The main other issue which affects IRequest is #288, so it may be worthwhile to land both of these branches at the same time. I've discussed this previously, but the general idea would be: create an integration branch, get #288 reviewed, merge it to the integration branch, get #3621 reviewed, merge it to the integration branch, and then merge the integration branch to trunk (since it would have no unreviewed commits). This would allow us to have only one new interface instead of two. (I am probably forgetting some other tickets which affect IResource and IRequest, others should feel free to chime in.)
participants (3)
-
Glyph Lefkowitz -
Laurens Van Houtven -
Maarten ter Huurne