[Twisted-Python] A new & improved IResource
![](https://secure.gravatar.com/avatar/415203f2727ceaf56d8f7f5e6d5d508b.jpg?s=120&d=mm&r=g)
Hey! In relation to my recent getChild problem (specifically, I want to return deferred resources from it), it has become evident that IResource just won't cut it for a number of reasons and we should probably replace the API with something else. The issues we should take care of in this new API: 1. http://tm.tl/3711 (returning deferreds from render methods) 2. http://tm.tl/3621 (returning deferreds from getChild and friends) 3. http://tm.tl/288 (accessing uploaded data while in progress) If there are any missing tickets I'm forgetting about that are only fixable by breaking the interface, now would be a good time to speak up. I have created a new ticket (http://tm.tl/5152) for the new interface using an integration branch that addresses the issues in the individual tickets on a new interface as Glyph proposed. I've also commented on #3711 to say that it just won't work that way and that we need a new interface, referring them to the new ticket number. What should the migration plan look like? At least from the code point of view, exarkun has already partially answered this question: http://twistedmatrix.com/trac/ticket/288#comment:31 cheers lvh
![](https://secure.gravatar.com/avatar/607cfd4a5b41fe6c886c978128b9c03e.jpg?s=120&d=mm&r=g)
On 01:18 pm, _@lvh.cc wrote:
Hey!
In relation to my recent getChild problem (specifically, I want to return deferred resources from it), it has become evident that IResource just won't cut it for a number of reasons and we should probably replace the API with something else.
The issues we should take care of in this new API:
1. http://tm.tl/3711 (returning deferreds from render methods) 2. http://tm.tl/3621 (returning deferreds from getChild and friends) 3. http://tm.tl/288 (accessing uploaded data while in progress)
If there are any missing tickets I'm forgetting about that are only fixable by breaking the interface, now would be a good time to speak up.
Another thing we want is something closer to locateChild as Nevow and web2 had. The key feature is being able to consume multiple path segments (or none at all) at a time if desired. Jean-Paul
I have created a new ticket (http://tm.tl/5152) for the new interface using an integration branch that addresses the issues in the individual tickets on a new interface as Glyph proposed. I've also commented on #3711 to say that it just won't work that way and that we need a new interface, referring them to the new ticket number.
What should the migration plan look like? At least from the code point of view, exarkun has already partially answered this question: http://twistedmatrix.com/trac/ticket/288#comment:31
cheers lvh
![](https://secure.gravatar.com/avatar/415203f2727ceaf56d8f7f5e6d5d508b.jpg?s=120&d=mm&r=g)
On Mon, Jun 13, 2011 at 3:38 PM, <exarkun@twistedmatrix.com> wrote:
Another thing we want is something closer to locateChild as Nevow and web2 had. The key feature is being able to consume multiple path segments (or none at all) at a time if desired.
Jean-Paul
I added it to the ticket. Someone on IRC mentioned that locateChild isn't exactly perfect either -- how does it differ from the idealized API? For the people who hadn't seen locateChild before (much like myself) -- the main difference is that instead of just taking the *next* segment, it takes all of the remaining ones, and instead of returning a single child, it returns a child and the remaining segments. A possible problem with this is that people might get the impression they should mess with the segments... Comments welcome. I'm also not sure if it should be allowed to return an IResource (and basically use the IResource getChild behavior whenever you see an IResource...) or if it should only return the new things. Perhaps there could be an adapter: if entity.locateChild([a, b, c]) returns a (resource, [b, c]), the resource is adapted to IEntity (working title for I2NewResource2), and the adapter's .locateChild([b, c]) call gets translated into self.wrappedResource.getChild(b).getChild(c), or something. cheers lvh
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jun 13, 2011, at 6:30 PM, Laurens Van Houtven wrote:
On Mon, Jun 13, 2011 at 3:38 PM, <exarkun@twistedmatrix.com> wrote: Another thing we want is something closer to locateChild as Nevow and web2 had. The key feature is being able to consume multiple path segments (or none at all) at a time if desired.
Jean-Paul
I added it to the ticket.
Someone on IRC mentioned that locateChild isn't exactly perfect either -- how does it differ from the idealized API?
For the people who hadn't seen locateChild before (much like myself) -- the main difference is that instead of just taking the *next* segment, it takes all of the remaining ones, and instead of returning a single child, it returns a child and the remaining segments. A possible problem with this is that people might get the impression they should mess with the segments... Comments welcome.
Yeah, messing with segments is an issue. There's a thing that some Nevow applications would do which, while I'm not entirely sure it was a terrible idea, definitely exposed a weakness in this model. We called it an "internal redirect". The basic idea is that your resource gets called with "request, [a, b, c]" and you return "resource, [d, e, f]". The problem with allowing this is that it decouples the resource from any sane way to identify its URL based on the request. It's handy to have something like the 'prepath' attribute to assist in generating absolute URIs that can be passed to external systems, but you can't make that consistent in the face of code that arbitrarily manipulates the segments list. Also, you can internally redirect this way based on path-based dispatch but you can't redirect based on hostname or other features of the request without mutating it. So it seemed like a lopsided half-feature. One off-the-cuff idea to address this would be to have a 'path' object with an API of its own; something like this: def locateDescendant(self, request, path): year, month, day = path.consume(3) rsrc = self.resourceFromDate(year, month, day) return rsrc .consume(integer) is something of a strawman here; you'd probably need a way to say stuff like "consume all of the path segments" or "consume everything up to a segment with the value 'x'", so that we didn't lose the (legitimate) flexibility of having a list. We could of course just pass in a mutable list and not return it, but that provides a little more flexibility than I'd like, because it would allow you to do all the same stuff as returning your own arbitrary sequence. Really this 'path' is actually cursor over the URI inside the request. It would be nice if the state associated with traversal were encapsulated entirely in that object, so that the request itself didn't get mutated as it traveled down the path.
I'm also not sure if it should be allowed to return an IResource (and basically use the IResource getChild behavior whenever you see an IResource...) or if it should only return the new things.
One return-type signature is a good idea. Convenience APIs are better. That said, if we do go with 2-tuple as the return type, good error reporting is crucial; it's very easy to accidentally return a resource.
Perhaps there could be an adapter: if entity.locateChild([a, b, c]) returns a (resource, [b, c]), the resource is adapted to IEntity (working title for I2NewResource2), and the adapter's .locateChild([b, c]) call gets translated into self.wrappedResource.getChild(b).getChild(c), or something.
That seems sensible.
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jun 13, 2011, at 9:18 AM, Laurens Van Houtven wrote:
Hey!
In relation to my recent getChild problem (specifically, I want to return deferred resources from it), it has become evident that IResource just won't cut it for a number of reasons and we should probably replace the API with something else.
The issues we should take care of in this new API:
1. http://tm.tl/3711 (returning deferreds from render methods) 2. http://tm.tl/3621 (returning deferreds from getChild and friends) 3. http://tm.tl/288 (accessing uploaded data while in progress)
A new IResource means an opportunity for a new IRequest, since implementing that new interface is the easiest way for a resource to say "I require the new IRequest interface as a parameter to these methods". This long-standing enhancement would be a nice thing to get in there: http://tm.tl/1569 (IRequest should have a uri object) This ticket predates my understanding of many of the nuances of URIs, URLs, URNs, IRIs, IDNA, etc, so URLPath is probably not quite up to the job. nevow.url is closer, but probably needs some fixing (especially in the area of IRIs). And, for that matter, some compatibility between the request object used by Agent and the request object used by the server would be great. That's not necessary for a new interface, but in any design of a new interface, that should be taken into account so that it can satisfy requirements on both sides.
If there are any missing tickets I'm forgetting about that are only fixable by breaking the interface, now would be a good time to speak up.
I have created a new ticket (http://tm.tl/5152) for the new interface using an integration branch that addresses the issues in the individual tickets on a new interface as Glyph proposed. I've also commented on #3711 to say that it just won't work that way and that we need a new interface, referring them to the new ticket number.
What should the migration plan look like? At least from the code point of view, exarkun has already partially answered this question: http://twistedmatrix.com/trac/ticket/288#comment:31
cheers lvh
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
![](https://secure.gravatar.com/avatar/415203f2727ceaf56d8f7f5e6d5d508b.jpg?s=120&d=mm&r=g)
I've been looking at this again recently and honestly it's beginning to feel like I'm reimplementing large parts of web2. Does that mean I'm doing it wrong?
![](https://secure.gravatar.com/avatar/15fa47f2847592672210af8a25cd1f34.jpg?s=120&d=mm&r=g)
On Jul 24, 2011, at 11:36 AM, Laurens Van Houtven wrote:
I've been looking at this again recently and honestly it's beginning to feel like I'm reimplementing large parts of web2. Does that mean I'm doing it wrong?
Either that or that web2 was doing it right... Of course, you don't really need to reimplement it, just copy the existing implementation from before it was removed, and s/web2/web/. :) J
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jul 24, 2011, at 2:13 PM, James Y Knight wrote:
On Jul 24, 2011, at 11:36 AM, Laurens Van Houtven wrote:
I've been looking at this again recently and honestly it's beginning to feel like I'm reimplementing large parts of web2. Does that mean I'm doing it wrong?
Either that or that web2 was doing it right...
Of course, you don't really need to reimplement it, just copy the existing implementation from before it was removed, and s/web2/web/. :)
The main thing that web2 did wrong was introducing a new namespace and thereby confusion, not any particular detail of its implementation. The web2 resource model was in most ways a nice upgrade from twisted.web. It even had a halfway decent compatibility layer. You should feel free to copy its implementation liberally (with the exception of the IStream interface, which has been discussed to death elsewhere). However, since web2 was half-baked in some regards, you may need to add test coverage, improve docstrings, or handle some corner cases in order to pass review. But, in terms of implementation details, web2 was in fact "doing it right" for the most part. -glyph
![](https://secure.gravatar.com/avatar/415203f2727ceaf56d8f7f5e6d5d508b.jpg?s=120&d=mm&r=g)
On Mon, Jul 25, 2011 at 1:22 AM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:
The main thing that web2 did wrong was introducing a new namespace and thereby confusion, not any particular detail of its implementation.
Well, my thing would introduce some namespaces (like "entity")-- just not one in the toplevel twisted package.
The web2 resource model was in most ways a nice upgrade from twisted.web. It even had a halfway decent compatibility layer. You should feel free to copy its implementation liberally (with the exception of the IStream interface, which has been discussed to death elsewhere). However, since web2 was half-baked in some regards, you may need to add test coverage, improve docstrings, or handle some corner cases in order to pass review.
Of course -- I'd end up doing that anyway because I wouldn't feel confident I understand the code otherwise.
But, in terms of implementation details, web2 was in fact "doing it right" for the most part.
Glad to know it's not just my mangled mind.
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- cheers lvh
participants (4)
-
exarkun@twistedmatrix.com
-
Glyph Lefkowitz
-
James Y Knight
-
Laurens Van Houtven