Finer-Grained Security System for Twisted Web/Nevow?
From reading the sources, it appears that the portal/realm/cred system only checks the user identity at the -start- of an HTTP request, prior to URL
Hi, I'm working with Twisted Web/Nevow and have questions about my options for user authentication. I've read all the docs on portal/realm/cred and have a simple website working using those (and guard.SessionWrapper) that prompts for login. However I'm not sure of the correct design approach to using that technique for a site with mixed access controls. traversal or page delivery. Once the realm has returned an appropriate avatar representing a specific tree of pages/resources, there appear to be no further security controls for finer control. I see two problems with that approach: 1) It is rather monolithic; you can't grant access to this page or that one selectively, or perhaps add a security check into the URL traversal steps to control access to a hierarchy of sub-pages. Viewing the portal as the frontdoor of a site requiring authentication, it makes it tricky to have some non-authenticating pages for visitors to register or have their forgotten password mailed to them. To do those tasks, it seems necessary to create multiple portals for a single site, one for the sign-up/password-reminder set of pages, and another portal for the members-only pages. 2) Alternatively, one could dynamically generate a custom tree of pages/resources within the realm object, returning a different tree depending upon the identity/permissions of the user. This would seem to make it complicated to (a) guarantee that all visitors see the exact same URL structure and, (b) consume more time/memory with constructing duplicate page/resource trees when thousands of users may be visiting the site, with a mix of permissions. I'm thinking I'll have to write something like decorators for page resources that front-end the locateChild method (for access control over traversal), and perhaps the renderHTTP method (for access control over page delivery) with permissions checking logic. Maybe I've misread the Twisted/Nevow sources and there is already a mechanism for, it seems to me, this common use case for a membership-type of website. Considering the complexity of the cred system that gives us such great flexiblity in authentication, I don't really want to have to bypass it and write my own mechanism. Surely others have been here before me. Any advice is appreciated, -Jeff
On Apr 19, 2009, at 5:40 AM, Jeff Rush wrote:
1) It is rather monolithic; you can't grant access to this page or that one selectively, or perhaps add a security check into the URL traversal steps to control access to a hierarchy of sub-pages. Viewing the portal as the frontdoor of a site requiring authentication, it makes it tricky to have some non-authenticating pages for visitors to register or have their forgotten password mailed to them. [...]
2) Alternatively, one could dynamically generate a custom tree of pages/resources within the realm object, returning a different tree depending upon the identity/permissions of the user. [...]
I haven't done any development like this for a number of years, but when I was, I also found cred/portal/guard rather cumbersome. Unfortunately I don't have anything actually useful to add to the discussion, but I feel your pain. :) James
2009/4/19 Jeff Rush <jeff@taupro.com>: [...]
2) Alternatively, one could dynamically generate a custom tree of pages/resources within the realm object, returning a different tree depending upon the identity/permissions of the user.
I think this is the preferred approach ATM.
This would seem to make it complicated to (a) guarantee that all visitors see the exact same URL structure and,
You can share the same URL mapper between all users of you site.
(b) consume more time/memory with constructing duplicate page/resource trees when thousands of users may be visiting the site, with a mix of permissions.
You don't have to duplicate page resources; you can create a single resource and return it for as many users as you want.
I'm thinking I'll have to write something like decorators for page resources that front-end the locateChild method (for access control over traversal), and perhaps the renderHTTP method (for access control over page delivery) with permissions checking logic.
What is your permission model exactly? What kind of limitations of current guard implementation would you like to solve? I think I like the current approach... Once you learn how to use it and how to set it up properly (this is a key phrase here), it will take a few things off your head. Even if setting it up is cumbersome, the idea of web resource wrapping an avatar object is pretty cool, as you don't have to put access control logic into web resource objects -- you just enable these and these resources for that level of privleges, which in fact gives you fine-grained security model. If you look for a tool, that handles setting up such things for you and lets you just concentrate on the code, there's Divmod Mantissa; unfortunatley, it is not a very well documented piece of software. -- m
On 12:32 am, michal.dtz@gmail.com wrote:
2009/4/19 Jeff Rush <jeff@taupro.com>:
I think I like the current approach... Once you learn how to use it and how to set it up properly (this is a key phrase here), it will take a few things off your head. Even if setting it up is cumbersome, the idea of web resource wrapping an avatar object is pretty cool, as you don't have to put access control logic into web resource objects -- you just enable these and these resources for that level of privleges, which in fact gives you fine-grained security model.
I agree with you that the way guard/cred work is kind of neat, but it's kind of neat as a cooperative low-level primitive for building more full-featured systems on top of. I can see the OP's point here: for a lot of use-cases, it's incomplete and you need to do a lot of extra work. Work which isn't particularly well documented. It would be really nice if we had more examples of *how* to build things on top of it. If anyone reading this thread has some code to link to, I encourage you to share it. Mantissa's sharing system is a good example, but it does a lot more than just wrap resources in a few simple permissions. Also, as you say:
unfortunatley, it is not a very well documented piece of software.
I look forward to your doc patches ;-).
Michal Pasternak wrote:
2009/4/19 Jeff Rush <jeff@taupro.com>: [...]
2) Alternatively, one could dynamically generate a custom tree of pages/resources within the realm object, returning a different tree depending upon the identity/permissions of the user.
I think this is the preferred approach ATM.
This would seem to make it complicated to (a) guarantee that all visitors see the exact same URL structure and,
You can share the same URL mapper between all users of you site.
Hmmm, I'm not aware of a centralized URL mapper in the Twisted framework as there is in other Python frameworks that use regular expression mappers. AFAIK, in Twisted, the URL tree is constructed piece by piece using a series of .putChild('segment', resource) calls (or child_XXX class attributes) which if you construct a complex tree with lots of conditionals based on the access rights of the user, can be messy.
(b) consume more time/memory with constructing duplicate page/resource trees when thousands of users may be visiting the site, with a mix of permissions.
You don't have to duplicate page resources; you can create a single resource and return it for as many users as you want.
WHile users with identical roles can share resource trees, it seems you'd need a unique tree for each 'kind' of user.
I'm thinking I'll have to write something like decorators for page resources that front-end the locateChild method (for access control over traversal), and perhaps the renderHTTP method (for access control over page delivery) with permissions checking logic.
What is your permission model exactly? What kind of limitations of current guard implementation would you like to solve?
It is role-based, where a user can possess a mix of multiple roles, such as user A is (member, ), user B is (member, instructor), user C is (admin, ). For that scenario I'd need three resource trees that include/exclude different things. I could pre-build trees for all possible combinations of roles, or build on demand and cache the last N trees keyed by role combination. Not very elegant though. Instead I'm looking for something more like: @access('instructor') class StudentRosterPage(rend.Page): ... where after traversal ends, a page's renderHTTP() method does: if request.user.session.role in self.page_roles: ...return content
I think I like the current approach... Once you learn how to use it and how to set it up properly (this is a key phrase here), it will take a few things off your head. Even if setting it up is cumbersome, the idea of web resource wrapping an avatar object is pretty cool,
It is pretty cool - and that is why I'm asking because Twisted has always introduced me to new ideas and ways of doing things. Perhaps there is significant security leverage already in there I'm not understanding, or maybe most people using Twisted roll their own web security mechanisms. The avatar approach certainly is a perfect fit for guarding SSH, IMAP, POP3 and such services. It just seems a bit awkward for fine-grained websites with a mix of access.
as you don't have to put access control logic into web resource objects -- you just enable these and these resources for that level of privleges, which in fact gives you fine-grained security model. If you look for a tool, that handles setting up such things for you and lets you just concentrate on the code, there's Divmod Mantissa; unfortunatley, it is not a very well documented piece of software.
I can see both sides of the issue of whether to place your access control logic into resources or centralize it into the part that builds up your website (realm/avatar) ie. a .tac file. Centralization permits tighter control over customization of use, but makes it harder to write pluggable subsystems reusable across different applications. My experiments so far are something along the lines of: class SiteSession(guard.SessionWrapper, membership.SessionMixin): # mixin adds new access-testing methods for current user # construct resource tree root = FrontDoor() membership.add_ResourcePages(root) return appserver.NevowSite(resource=root) In my case, after many years with other web frameworks like Zope I'm playing with Twisted to see how hard it is to write such a subsystem for membership management. Basically a drop-in for user registration, forgotten-password emails, generic user profile editing, login/logout panel to place into any page, and have it as easy to use as adding an entry to a buildout.cfg and a couple of lines to a .tac to choose your cred checkers and user info persistence stores, and poof, a membership-based website. I can see how to write it but I'd rather learn from others in the Twisted community if it has already been done. Besides security, I'm looking for examples of smart ways to build up a complex website from parts developed and packaged by others, without a lot of messy wiring logic. -Jeff
Jeff Rush pisze:
Michal Pasternak wrote:
2009/4/19 Jeff Rush <jeff@taupro.com>: [...]
2) Alternatively, one could dynamically generate a custom tree of pages/resources within the realm object, returning a different tree depending upon the identity/permissions of the user.
I think this is the preferred approach ATM.
This would seem to make it complicated to (a) guarantee that all visitors see the exact same URL structure and,
You can share the same URL mapper between all users of you site.
Hmmm, I'm not aware of a centralized URL mapper in the Twisted framework as there is in other Python frameworks that use regular expression mappers.
I think you can write one easily. The question is, do you really need one? :-)
AFAIK, in Twisted, the URL tree is constructed piece by piece using a series of .putChild('segment', resource) calls (or child_XXX class attributes) which if you construct a complex tree with lots of conditionals based on the access rights of the user, can be messy.
Or, you can override locateChild(). Conditionals make the code messy. That's why I like the idea of returning trees of resources, that wrap the avatar object and know nothing about access control.
(b) consume more time/memory with constructing duplicate page/resource trees when thousands of users may be visiting the site, with a mix of permissions.
You don't have to duplicate page resources; you can create a single resource and return it for as many users as you want.
WHile users with identical roles can share resource trees, it seems you'd need a unique tree for each 'kind' of user.
If you like, you can also remember the avatar object on the request object, and then make your resources (which may be a singleton instances) get the avatar from the request. I think that you will spare a few kilobytes of RAM at the cost of making your code a little bit more complicated.
I'm thinking I'll have to write something like decorators for page resources that front-end the locateChild method (for access control over traversal), and perhaps the renderHTTP method (for access control over page delivery) with permissions checking logic.
What is your permission model exactly? What kind of limitations of current guard implementation would you like to solve?
It is role-based, where a user can possess a mix of multiple roles, such as user A is (member, ), user B is (member, instructor), user C is (admin, ). For that scenario I'd need three resource trees that include/exclude different things. I could pre-build trees for all possible combinations of roles, or build on demand and cache the last N trees keyed by role combination. Not very elegant though.
Instead I'm looking for something more like:
@access('instructor') class StudentRosterPage(rend.Page): ...
where after traversal ends, a page's renderHTTP() method does:
if request.user.session.role in self.page_roles: ...return content
How about this: * create a central mapping of { privlege : [resourceClass, orInstances] } * make your decorators put classess into that mapping * show your resources where they belong (self.url, for example) * in your IRealm implementation, return trees of resources for a given privlege; you can return trees of classess and create instances on-fly, you can create all the instances that you share between users as singletons and put them in those trees too... I think it will work with guard; it will also allow you to write API you described. Do you see any potential problems with this approach?
It is pretty cool - and that is why I'm asking because Twisted has always introduced me to new ideas and ways of doing things. Perhaps there is significant security leverage already in there I'm not understanding, or maybe most people using Twisted roll their own web security mechanisms. [...] Besides security, I'm looking for examples of smart ways to build up a complex website from parts developed and packaged by others, without a lot of messy wiring logic.
Looks like you need Divmod Mantissa. http://www.divmod.org/trac/wiki/DivmodMantissa <http://www.divmod.org/trac/wiki/DivmodMantissa>http://www.divmod.org/trac/wiki/DivmodMantissa/Sharing BTW, list: it would be ubercool to have a downloadable repository of packaged LiveFragments (or Mantissa plugins). All the hard parts (code & tests) is already done... -- m
On 10:47 am, michal.dtz@gmail.com wrote:
Jeff Rush pisze:
Michal Pasternak wrote:
I think you can write one easily. The question is, do you really need one? :-)
AFAIK, in Twisted, the URL tree is constructed piece by piece using a series of .putChild('segment', resource) calls (or child_XXX class attributes) which if you construct a complex tree with lots of conditionals based on the access rights of the user, can be messy.
Or, you can override locateChild().
I would definitely do this reactively, in locateChild, rather than try to build up the whole hierarchy beforehand.
Conditionals make the code messy. That's why I like the idea of returning trees of resources, that wrap the avatar object and know nothing about access control.
They're not just messy. If you need to put the knowledge of your security model into your application logic, there's a chance you screw up, and when you screw up, you have a security hole. If the security logic always lives somewhere else, then you can apply security to application logic without changing what the application code does, and you only have to look at your security code for security bugs, not every line of code you've ever written in any application.
Jeff Rush wrote:
I'm thinking I'll have to write something like decorators for page resources that front-end the locateChild method (for access control over traversal),
and perhaps the renderHTTP method (for access control over page delivery) with permissions checking logic.
I did openid logins as a mixin class that replaces locateChild: http://bigasterisk.com/darcs/?r=exchangeMeeting;a=headblob;f=/nevowopenid.py Usage is like this: class Main(WithOpenid, rend.Page): docFactory = loaders.xmlfile("main.html") ..replace verify() and use self.identity..
Drew Perttula wrote:
Jeff Rush wrote:
I did openid logins as a mixin class that replaces locateChild:
http://bigasterisk.com/darcs/?r=exchangeMeeting;a=headblob;f=/nevowopenid.py
Draw (and Glyph with his suggestion of Mantissa's approach), thanks for the mentions of source I can study to better understand how I might wrap the web delivery mechanism. -Jeff
On Apr 20, 2009, at 4:05 AM, Jeff Rush wrote:
Drew Perttula wrote:
Jeff Rush wrote:
I did openid logins as a mixin class that replaces locateChild:
http://bigasterisk.com/darcs/?r=exchangeMeeting;a=headblob;f=/nevowopenid.py
Draw (and Glyph with his suggestion of Mantissa's approach), thanks for the mentions of source I can study to better understand how I might wrap the web delivery mechanism.
http://files.twisted.it/stiq_at_europython07.pdf This is a presentation I gave at europython 07 about some of the practices, that I consider best, in developing with Nevow. Basically I define a function that touches the database for some reason, then I append that function to a role (or a set of roles) using a decorator: @forRoles('admin') def deleteUser(avatar, email): @transact def _(email): return users.delete(users.c.email==email).execute() return _(email) @forRoles('anonymous', 'user', 'admin') def findUser(avatar, email): @transact def _(email): return users.select(sq.and_(users.c.email==email, users.c.enabled==True)).execute().fetchone() return _(email) This is one example of such a thing. The avatar object is exactly what's returned by cred/guard and it's defined in the following way: import zope.interface as z class Avatar(object): z.implements(IAvatar) def __init__(self, behavior, *args, **kw): self.username = None self.email = None self.role = None self.password = None self.conf = None super(Avatar, self).__init__(*args, **kw) self._actions = dict(((f.__name__, f) for f in behavior)) def __getattr__(self, name): try: fun = self._actions[name] except KeyError: raise NoPermission(name) return lambda *args, **kwargs: fun(self, *args, **kwargs) So that it takes a set of methods that it knows and allows you to call them, if a method is not present you probably don't have enough permissions (or you made a mistake in writing it). This is paired with some clever use of patterns: <nevow:invisible nevow:render="optional"> <nevow:invisible nevow:pattern="None"> Login to upload images. </nevow:invisible> <nevow:invisible nevow:pattern="default"> <nevow:invisible nevow:render="form upload" /> </nevow:invisible> </nevow:invisible> The optional (bad name I know) checks for the role of the current avatar, if it's None it means that the user is not logged in, otherwise you can use a specific role or default for 'all the others': This is how it's implemented: def render_optional(self, ctx, data): iq = inevow.IQ(ctx) try: t = iq.onePattern(str(self.avatar.role)) except stan.NodeNotFound: try: t = iq.onePattern('default') except stan.NodeNotFound: t = '' return ctx.tag[t] I'm not sure it's still usable though with newer nevow versions, although it might still be. The other part (navigating in the tree) should be solvable by using another decorator that could look like this: def requiresRole(role, loginPage=LoginPage): def _f(fun): def __f(avatar, *args, **kwargs): if role not in avatar.roles: return loginPage(avatar) return fun(avatar, *args, **kwargs) return __f return _f Or something similar (this is not tested). It's been a while since I last used this code but it's concept is still good. -- Valentino Volonghi aka Dialtone Now running MacOS X 10.5 Home Page: http://www.twisted.it http://www.adroll.com
Valentino Volonghi pisze:
http://files.twisted.it/stiq_at_europython07.pdf
This is a presentation I gave at europython 07 about some of the practices, that I consider best, in developing with Nevow.
Thank you very much for sharing those ideas & code samples! -- m
Valentino Volonghi wrote:
On Apr 20, 2009, at 4:05 AM, Jeff Rush wrote:
http://files.twisted.it/stiq_at_europython07.pdf
This is a presentation I gave at europython 07 about some of the practices, that I consider best, in developing with Nevow.
<snip> Valentino, thanks very much for that long email outlining how you use Nevow. It definitely helps me with ideas I've not seen anywhere else. -Jeff
On 04:49 am, jeff@taupro.com wrote:
Valentino Volonghi wrote:
On Apr 20, 2009, at 4:05 AM, Jeff Rush wrote:
http://files.twisted.it/stiq_at_europython07.pdf
This is a presentation I gave at europython 07 about some of the practices, that I consider best, in developing with Nevow.
<snip>
Valentino, thanks very much for that long email outlining how you use Nevow. It definitely helps me with ideas I've not seen anywhere else.
Jeff, I'd appreciate it if you could collate some of these ideas and experiences into some write-up of your own - there isn't a lot of folk knowledge about there about how to get the most out of Nevow or Twisted.Web. It would be nice if google had more to find on this subject than bugs in our ticket trackers :).
On Apr 22, 2009, at 6:55 AM, glyph@divmod.com wrote:
On 04:49 am, jeff@taupro.com wrote:
Valentino Volonghi wrote:
On Apr 20, 2009, at 4:05 AM, Jeff Rush wrote:
http://files.twisted.it/stiq_at_europython07.pdf
This is a presentation I gave at europython 07 about some of the practices, that I consider best, in developing with Nevow.
<snip>
Valentino, thanks very much for that long email outlining how you use Nevow. It definitely helps me with ideas I've not seen anywhere else.
Jeff, I'd appreciate it if you could collate some of these ideas and experiences into some write-up of your own - there isn't a lot of folk knowledge about there about how to get the most out of Nevow or Twisted.Web. It would be nice if google had more to find on this subject than bugs in our ticket trackers :).
I'd be happy to review it since I gave up in frustration after, as you said, finding not much beyond bug reports filed against them by people who already knew how to use them. I really need Twisted for some of my app but ended up having to use django, running behind CherryPy's WSGI with Twisted mixed in to do the real work. The Twisted WSGI server gave me fits with missing CSS files and other oddities I don't know enough to track down myself or I would have used that. Obviously, that's a different discussion but I wasted so much time trying to get Nevow to do _anything_ that I can't pursue it right now. S
participants (8)
-
Drew Perttula -
glyph＠divmod.com -
James Y Knight -
Jeff Rush -
Michal Pasternak -
Michał Pasternak
-
s s -
Valentino Volonghi