[Twisted-Python] twisted cred: why does avatarId need to be a str?
Hi, As some of you probably know I'm trying to beat OAuth2.0 into submission. I'm using twisted.cred to do it. The restriction that avatarIds need to be strs is somewhat getting in my way, and I'm not sure if the correct way to do it is: 1) ignore the restriction (credentials types are so specific to OAuth that it's very unlikely you'll find anything already pluggable...) 2) use the mind object (but then it becomes somewhat wonky what goes on the mind object and what should be the avatarId) I'll give you a practical example: An OAuth client makes a request for an access token. This basically means he trades some credentials in for an access token (I use the expression trade-in because most of the credentials can only be used once). It presents some credentials, which can be grouped into two parts: 1) credentials that authenticate the client itself 2) credentials for the access grant (eg I've got this thing that says I should have access) Because (1) is pretty much always the same (and is common to other parts of txOAuth) and (2) depends on the specific kind of request, this results in two portals: 1) for authenticating the client, which returns an IClient 2) for authenticating the credentials (which includes the authenticated client, since credentials are client-specific), which returns the token Coming from Perspective Broker I think the Client would be an apt mind object for the second portal, since it represents the entity on the other side trying to do auth. The appropriate credentials checker in the second portal checks (and invalidates, since most of the credentials are single use) the credentials. What avatarId is sent to the Realm? The Realm is also responsible for requesting the new token, so it needs to know details about the credential that the credential checker just checked (and in doing so, invalidated). These are not available through the IClient mind object. The alternative design is that all of the credentials checkers know how to request tokens, and instead give the tokenId as avatarId. This is still insufficient, because tokens can include extra information, such as scope, expiration time... That can again be solved by sending an opaque string value which contains all of the data in the token response, but in that case the Realm is completely useless and the checkers do all of the work. Additionally, this seems stupid because cred checkers are for checking credentials, not checking credentials + a bunch of other stuff. Here's my theory: 13:43 <lvh> aa_: fwiw I *think* it's because cred's supposed to be pluggable 13:43 <lvh> aa_: so avatarId is supposed to be the equivalent of a username 13:44 <lvh> aa_: and it's str not unicode because there's a helluva lot more things that store bytes than unicode objects So um, any thoughts from people that understand cred? Is it ever okay to send richer things than strings to a Realm? thanks lvh
On Sep 7, 2010, at 12:16 PM, Laurens Van Houtven wrote:
Hi,
As some of you probably know I'm trying to beat OAuth2.0 into submission. I'm using twisted.cred to do it. The restriction that avatarIds need to be strs is somewhat getting in my way, and I'm not sure if the correct way to do it is:
Let me begin my answer with Epigram 34: "The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information." If I recall correctly, the Divmod realm/checker integration required a particular format for avatar IDs; they had to be 'user@domain' so that the server could do virtual hosting. In some cases (POP for example) that actually meant the user needed to type in user@domain as their username. So, while strings may not be ideal, it's almost certainly possible for them to let you do what you want. I wouldn't recommend that approach if you have a bunch of extra data to deal with... but you don't really give a clear picture of what extra data you do have to deal with :).
1) ignore the restriction (credentials types are so specific to OAuth that it's very unlikely you'll find anything already pluggable...)
but the avatarID isn't a credential. It's a mapping between the checker (which is about authentication) and the realm (which is about application data). There hopefully _should_ be such a thing as some "already pluggable" application data.
2) use the mind object (but then it becomes somewhat wonky what goes on the mind object and what should be the avatarId)
Use the mind object for what??? You're asking this question completely backwards. Even after reading the whole email I'm still not quite sure what you're on about. I'll do my best to answer a few specific quirks here, but please re-post starting with what you're actually trying to do, i.e., "I got up today, and I wanted to paint the shed." If you're already most of the way into an implementation, feel free to present bits of the implementation itself, but you haven't really described enough of it here for me to understand what you're doing and why.
Only one of these things are really "credentials" from cred's perspective. Cred's purpose is to tie together two things: a realm that implements some protocol/application logic, and a checker that implements some authentication logic. _everything that happens before you start talking to the application_ is just authentication stuff, and is part of the checker. Even if that stuff, itself, needs to be pluggable. I don't know the specifics of how OAuth works, but I know vaguely what it does, and ultimately what it needs to do is give you access to an IResource that allows you to do ... stuff. The stuff-doing part is the interesting part. That is the part that the Realm gives you. The Realm ought to be able to work with Basic authentication, and the objects returned from the realm should be able to work even with no authentication (if you wanted to just put your resource endpoint up for everyone to use, unauthenticated, which is granted not very sueful).
Because (1) is pretty much always the same (and is common to other parts of txOAuth) and (2) depends on the specific kind of request, this results in two portals:
1) for authenticating the client, which returns an IClient
I have no idea what an "IClient" is. It sounds like you're using cred internally so that you can make the actual OAuth-ing process plugabble, and that's great, as long as you separate it from the post-OAuth part of actually producing a resource.
2) for authenticating the credentials (which includes the authenticated client, since credentials are client-specific), which returns the token
That part sounds like what I would expect.
Coming from Perspective Broker I think the Client would be an apt mind object for the second portal, since it represents the entity on the other side trying to do auth.
The Mind is the entity on the other side trying to get something done, not trying to auth. The docs sorely need to be updated to reflect this, but the canonical example of the 'mind' is the IChatClient interface in twisted.words (used in a cred context by twisted.words.service). Of course, the IChatClient provider may also be involved in authentication, for example receiving a NickServ challenge from the server, but by the time you start calling that object a 'mind', authentication is done.
The appropriate credentials checker in the second portal checks (and invalidates, since most of the credentials are single use) the credentials. What avatarId is sent to the Realm? The Realm is also responsible for requesting the new token,
OK, and that's where things go wrong. The realm is responsible for one thing only: returning an object which implements an interface that allows a user of the library to define some application logic. It should not be participating in auth. If the authentication layer needs to perform some additional logic on each request, it should wrap up the IResource (or whatever) provider returned by the Realm, so that the Realm can do the work. (and furthermore, why is anybody responsible for "requesting the new token" in the first place? why do any new tokens ever need to be requested on the service provider's side of things?)
so it needs to know details about the credential that the credential checker just checked (and in doing so, invalidated). These are not available through the IClient mind object.
The alternative design is that all of the credentials checkers know how to request tokens, and instead give the tokenId as avatarId. This is still insufficient, because tokens can include extra information, such as scope, expiration time... That can again be solved by sending an opaque string value which contains all of the data in the token response, but in that case the Realm is completely useless and the checkers do all of the work. Additionally, this seems stupid because cred checkers are for checking credentials, not checking credentials + a bunch of other stuff.
This has historically been a weak point for cred. While I'm trying to give you a better idea of how to use it "right", you should be keenly aware that it isn't perfect and doesn't necessarily handle every case, so there are some things that you might try to do which won't be well supported. Since cred is so mysterious and subtle, sometimes people believe that if they could just grasp its essence all their problems will go away; but it only solves one problem, so if that problem isn't the one you're facing, you may still be out of luck :). The idea is that the realm is supposed to be a backend that has its own storage and logic and so on, and the checker is supposed to have enough information to produce an avatar ID that points at something the realm knows about. The fiction that cred promulgates is that checkers are somehow agnostic to realms, but can still perform this mapping reliably based on their credentials. Of course, this can't be universally true: if the checker needs to map things into the realm, then it stands to reason it *does* need to know something about the realm. This level of flexibility is needed very rarely though, so what ends up happening is that all the existing checkers just relay the 'username' field to be the avatar ID. In the event that you do need more flexibility - let's say your accounts database uses a peculiar convention to encode the authentication type into the stored user ID so that the same user might get different behavior based on how they authenticate - you could easily write some custom checkers that wrap simple underlying checkers and sprinkle on a little special-case logic to mutate the avatarID appropriately before returning it. I think you're actually a little lucky to be implementing OAuth rather than OpenID, because my understanding is that with OAuth the expectation is that, as a provider of a service (i.e. someone likely to be using the cred interfaces, not a client) you expect that users have already created an account, and so the realm already has some data. "expiration time" can be handled by the checker, or at least by the code that's driving the portal, to set session expiration. (I'm not really sure what "scope" is, perhaps you could elaborate on some of these additional fields.) The place where cred really falls down is multi-field stuff. For example, in OpenID, a new user might show up with some OpenID metadata, like their preferred nickname, email address, and location. Cred provides no way to tell the realm "create an account like this" or "are there any other users with this email address", so if you need to do *that* kind of thing, you're going to need to work on expanding the API.
It hadn't really occurred to me to abuse the 'mind' object to provide that sort of structured data, but being able to specify the mind as "something adaptable to (interface X which provides the info you need)" is a nice big loophole :). And the realm can return whatever it wants; it doesn't need to be based on the avatarID. If you wanted to cram some more structured fields in there to smuggle them to the realm, that's a better place to put them than the strictly-mandated it's-just-some-bytes avatarID. (This would allow other 'regular' HTTP realms to work with your OAuth code, but provide extra OAuth stuff for realms that want it). I'm still not convinced that your realm should actually know about these extra fields, but maybe you just haven't given the relevant example yet.
A nomenclature distinction that might help this conversation: *authentication* verifies *identity* (via, e.g., userid/passwd, public key negotiation, etc.) and may issue a token (e.g., a cookie, Kerberos TGT, etc.). *authorization* determines what an authenticated identity (and its associated roles, if any) have access to in an application context (realm?). These are standard terms, and the activities they refer to should be orthogonal. Neither the OP nor Glyph use the term "authorization" in either of their messages, but that concept is clearly involved and is almost always useful for clarification. Looking at the OAuth 2.0 protocol specification, it seems quite similar in concept to Kerberos (and other SSO protocols), in which the user acquires a Ticket-Granting Ticket (TGT) upon authentication to a Kerberos domain controller, and then the TGT is used when interacting with applications in the Kerberos domain. Each app is registered with the Kerberos domain controller and can interact with it securely to verify the validity (also expiration time, etc.) of a TGT that has been offered to it. Once that interaction is complete, the app knows the identity associated with the TGT has been authenticated, and it can proceed with authorization, which of course depends on each application's context, and is completely separate from authentication. It might be useful to describe cred's structure in these terms, since both authentication and authorization are involved. (Might be useful to me, at least ... :) Steve
On Sep 8, 2010, at 1:27 PM, Stephen Waterbury wrote:
The checker authenticates; the realm authorizes. Authorization proceeds from the realm's idea of what a particular avatar ID (and, apparently, mind, as laurens has discovered this particular loophole in the API) is authorized to do; authentication proceeds from what the checker thinks makes some credentials valid. As you put it:
replace "application" with "realm" here and that's basically how twisted.cred works. The reason I didn't use the term authorization in my original message is that we're talking about an authentication protocol, and hopefully authorization can stay out of it :).
Actually, I believe much of the confusion is cleared up. Apparently I misunderstood what a Realm does. Oops! The application in the original mail, which I will continue to talk about in this one, is a token endpoint. I've been told an explanation of what that is is in order. To start out: it is NOT the thing that actually serves protected resources, that's a resource server, which is a different beast entirely. The job of a token endpoint is to turn certain kinds of credentials into an access token; the access token, in turn, can be used to access protected resources. So basically, these credentials are about already provided access grants, that haven't been turned into access tokens yet. Examples of such credentials which can be turned in for access tokens: - authorization code: This represents an access grant from the end user. Instead of receiving an access token when the grant was issued, you got this code, which basically is a credential that says "This person is allowed to access the protected resource, but we haven't given him a proper token to do that yet". The token endpoint is for turning one of these into a token. - refresh token: in case your access token has been used up or expired, one refresh token can be exchanged for one access token - assertion: this is basically just a pair of opaque strings (a type and a value); it's used for tying OAuth in with existing systems. Hence, this is deliberately vague. - user credentials: this is the classic username/password. It's a bit silly to use this; the only advantage is that you can limit which servers need access to these credentials. The key thing to remember is: token endpoints let you trade in some credentials that prove you're supposed to have access for a token that actually *gives* you that access. Since people asked for more detail as to which data exactly needs to be passed to the Realm (yes, even now that I know what the Realm actually does). This is basically all of the data in the token endpoint's response. The response contains the following data, in JSON format: (explanations of what each thing is in the following paragraphs) - the access token (an ascii string, mandatory) - the expiration time (optional) - the refresh token (another ascii string, similar to the access token, optional) - scope (an ascii string consisting of a set of space-separated words, optional) Optional here does not really mean "at my option" but "occasionally required", so it is necessary that all four can be passed to the IRealm at the same time. Furthermore, it's impossible to use the Mind for some of them, because ICredentialsCheckers never see the Mind, and all of that data (access token, refresh token, expiration time, scope) are only ever visible to the ICredentialsChecker. Again, the access token is something you use to actually access a protected resource. The refresh token is something you use to get a new token when your old one expires or is used up. Defining the scope properly is hard, it's deliberately vague in the spec as well; it's a set of strings that determine what you can do with your shiny new access token. For example, for Facebook, the scope might be "name friends likes" for a token that allows someone to access your name, your list of friends and what you like, but not your pictures, gender, date of birth. These values are not determined by the spec, and you can just make some up for your service. Also, to clear up a misunderstanding:
Does that help? Laurens
On Sep 8, 2010, at 8:12 PM, Laurens Van Houtven wrote:
Actually, I believe much of the confusion is cleared up. Apparently I misunderstood what a Realm does. Oops!
The application in the original mail, which I will continue to talk about in this one, is a token endpoint. I've been told an explanation of what that is is in order. To start out: it is NOT the thing that actually serves protected resources, that's a resource server, which is a different beast entirely. The job of a token endpoint is to turn certain kinds of credentials into an access token; the access token, in turn, can be used to access protected resources. So basically, these credentials are about already provided access grants, that haven't been turned into access tokens yet. Examples of such credentials which can be turned in for access tokens:
(...)
Does that help? Laurens
Somewhat, but it really casts the main issue into sharper focus: for any given cred implementation, there are two key questions: What is the avatar interface? What is the associated mind interface? If we're not talking about a system where the avatar interface (the one passed to 'login', the one that the realm must return something that gets implemented) is IResource, then that's the problem. What is that interface that you're talking about? Be very specific: what methods does it have and why? It sounds like OAuth is very precisely implementing something that maps almost exactly onto the checker part of cred, but you're doing it by implementing a realm to allow applications to provide their own logic. This might not be the wrong idea, but it needs to be spelled out very clearly. As you describe it:
token endpoints let you trade in some credentials that prove you're supposed to have access for a token that actually *gives* you that access
In cred-ese, that would be "checkers let you trade in some credentials that prove you're supposed to have access for an <avatar ID, which you give to a Realm> that actually *gives* you that access". As far as the OAuth response is concerned, it's like a checker. Looking at the individual parts, assuming that the avatar interface for the purposes of this discussion is IResource, it breaks down like this in my mind:
- the access token (an ascii string, mandatory)
Avatar ID. (See, it's a str!)
- the expiration time (optional)
Implementation detail of the session. Avatars are actually sessions, which expire: sometimes (as with POP) at the end of a connection, sometimes (as with nevow.guard's HTTP support) with a session timeout.
- the refresh token (another ascii string, similar to the access token, optional)
Implementation detail of the authentication protocol. The client library and server library should be transparently refreshing this without telling either the client application code or server application code, right?
- scope (an ascii string consisting of a set of space-separated words, optional)
This part doesn't quite fit, but could be expressed in one of two ways: a modification to the avatar ID, or as some extra structure on the Mind that modifies what functionality the Realm bundles in to your avatar implementation (without changing the interface provided by that object, of course).
I'll give a very practical example of an example request + response, because I'm not entirely sure I communicated that properly. The code we're discussing is part of a TokenEndpoint, which is an IResource. Something (a client, in OAuth terminology) makes a request that looks like this: --- POST /token HTTP/1.1 Host: server.example.com Content-Type: application/x-www-form-urlencoded grant_type=password&client_id=s6BhdRkqt3&client_secret=47HDu8s&username=johndoe&password=A3ddj3w --- And hopefully the code we're discussing answers with a request that looks like this: --- HTTP/1.1 200 OK Content-Type: application/json Cache-Control: no-store { "access_token":"SlAV32hkKG", "expires_in":3600, "refresh_token":"8xLOxBtZp8" "scope": "tummies cookies parrots" } --- Where, as mentioned before, everything besides "access_token" is not always required (but it must be possible to set them, because occasionally they are required). On Fri, Sep 10, 2010 at 2:34 AM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:
Okay, to be specific: I believe the appropriate interface is IResource. This is in line with t.web's general way of interacting with cred: you give me credentials, I give you a protected IResource.
Doing everything in the checker probably might make more sense. The reason I originally thought to do this in the IRealm is that I figured credentials checkers should just check credentials, and creating a new credential (the access token) wasn't part of ICredentialsChecker's job (more like IRealm's job). Apparently I was mistaken.
Heh, okay; as long as there's a plausible way to get the other important information (see rest of email) into the HTTP response, I'll believe you.
Right, but: a) The avatar needs to know about this timeout, since the timeout information needs to be able to make it into the HTTP response. b) The expiration time is only known to the thing that creates the access token (obviously). From the previous discussion, this is apparently ICredentialsChecker. c) (From (a, b)) The ICredentialsChecker needs to be able to communicate the expiration to the avatar. d) The ICredentialsChecker and IRealm can only communicate through the avatarId. Conclusion (from c,d): The expiration time needs to be in the avatarId?
Sure, but we're still writing library code here. If the avatar interface isn't actually IResource, but a new interface IAccessToken, getting the refresh token later might be feasible. (It should be a different interface, because in order to create a refresh token outside of this entire cred cycle, I need to know about the thing it's refreshing -- so, the information in the response needs to be easily accessible and not just an opaque IResource). I agree entirely that clever client library code would abstract this mess away from application code. However, right now this code still needs to somehow be able to eventually produce HTTP responses that don't abstract anything yet and just contain all the appropriate data, because that's just what the OAuth spec says it needs to be able to do. There isn't any real application code in the token endpoint; they're pretty similar for all OAuth setups, and customization would typically happen through implementing the appropriate ICredentialsChecker. - scope (an ascii string consisting of a set of space-separated words,
Unfortunately I don't believe this can be done through the mind. Again working under the previous assumption that ICredentialsChecker and not the IRealm is responsible for creating the access token, and scope is known to the thing that makes the access token, the ICredentialsChecker knows about the scope. Unfortunately the only way to pass stuff between the ICredentialsChecker and the IRealm is the avatar ID, so you don't have a choice. thanks for your infinite patience, Laurens
On Fri, Sep 10, 2010 at 10:29 PM, <exarkun@twistedmatrix.com> wrote:
Is the timeout authenticated?
Jean-Paul
In the sense that it's not a disaster if you lie about it -- no. Timeouts are generally optional and really just a friendly hint to the client receiving the token to not expect it to work after X time. Laurens
On Sep 10, 2010, at 1:01 PM, Laurens Van Houtven wrote:
Okay, to be specific: I believe the appropriate interface is IResource. This is in line with t.web's general way of interacting with cred: you give me credentials, I give you a protected IResource.
This doesn't quite make sense to me. Mostly, I don't understand what the moving pieces are here. If this is all to implement a "token endpoint" which doesn't "actually serve protected resources", then what is the IResource being returned by the realm actually *doing*? If it is not, itself, a "protected resource", then isn't the amount of stuff that its IResource can do severely restricted (i.e. restricted to doing a few OAuth-specific HTTP things) and therefore actually an avatar of a completely different, OAuth-specific interface, which just happens to be wrapped in an IResource by the OAuth implementation for the benefit of formatting those OAuth-specific things over an HTTP channel? Based on what you've said, I would expect the avatar interface to be 'IToken', since the point of this authentication is to 'generate tokens'.
It sounds like OAuth is very precisely implementing something that maps almost exactly onto the checker part of cred, but you're doing it by implementing a realm to allow applications to provide their own logic. This might not be the wrong idea, but it needs to be spelled out very clearly.
Doing everything in the checker probably might make more sense. The reason I originally thought to do this in the IRealm is that I figured credentials checkers should just check credentials, and creating a new credential (the access token) wasn't part of ICredentialsChecker's job (more like IRealm's job). Apparently I was mistaken.
I'm going to handwave a bit here and pretend we're talking about a thing that actually does serve arbitrary resources, so my answers may be slightly off. But I think they may be applicable either way. Speaking the authentication protocol is the authentication implementation's job, not the realm or the checker. For example: if you need to generate a challenge as part of the authentication process, the checker probably doesn't need to get involved there. If you need to ask the checker something about the challenge (let's say you have a list of persistent salts), then there needs to be a method on the credential interface to allow the checker to relay that information back. It strikes me that creating a new access token is also an authentication feature. If it is, then it goes into the OAuth resource somewhere. If it needs to be pluggable then it needs to go into the ICredentialsChecker or possibly the realm, but it's possible that this is just an OAuth specific interface that is plugged in to the OAuth-doing IResource separately, as a constructor argument that has nothing to do with cred.
Yes. The way to get it into the HTTP response is: put it into the HTTP response :). You don't necessarily need to cram it into the realm.
I'll stop you right there: No. (We can skip the rest of points b, c, and d, since they all apparently follow from this.) The avatar's job is to give you an IResource, but the OAuth implementation doesn't necessarily need to pass through either the arguments to that resource or that resource's return values (from getChild or render) exactly. If the timeout needs to be put into the HTTP response as a header, the OAuth resource could wrap the IResource value that it gets from the realm and set the header before calling the ultimate resource's child. Note that this isn't just the credentials checker: this is the OAuth implementation. Relaying information there is easy though: depending on how the checker itself gets created and parameterized, either you can have the checker simply have a reference back to the OAuth resource so it can call some methods, or you can expand the OAuth credentials interface to have a method that the checker uses to pass that information back along with the resource.
If the avatar interface isn't actually IResource, but a new interface IAccessToken, getting the refresh token later might be feasible. (It should be a different interface, because in order to create a refresh token outside of this entire cred cycle, I need to know about the thing it's refreshing -- so, the information in the response needs to be easily accessible and not just an opaque IResource).
I agree entirely that clever client library code would abstract this mess away from application code. However, right now this code still needs to somehow be able to eventually produce HTTP responses that don't abstract anything yet and just contain all the appropriate data, because that's just what the OAuth spec says it needs to be able to do. There isn't any real application code in the token endpoint; they're pretty similar for all OAuth setups, and customization would typically happen through implementing the appropriate ICredentialsChecker.
Again: producing HTTP responses is not the same thing as producing IResource implementations. You can have an HTTP response produced by an IResource that wraps another IResource and adds some special sauce to the response. So this information still never needs to make it to the realm.
OK, I admit it, by the time I get here I'm lost. We should try to meet up and have an interactive conversation about this, because I have a ton of questions about this, and I don't understand who generates the scope ID and why, and why this bit of information would need to be relayed between the checker and the realm in the process of authentication. Basically this goes back to the first confusion I mentioned at the beginning of this email: I don't understand why the realm is giving back an IResource if the whole purpose is to just generate tokens and return them.
On Sep 7, 2010, at 12:16 PM, Laurens Van Houtven wrote:
Hi,
As some of you probably know I'm trying to beat OAuth2.0 into submission. I'm using twisted.cred to do it. The restriction that avatarIds need to be strs is somewhat getting in my way, and I'm not sure if the correct way to do it is:
Let me begin my answer with Epigram 34: "The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information." If I recall correctly, the Divmod realm/checker integration required a particular format for avatar IDs; they had to be 'user@domain' so that the server could do virtual hosting. In some cases (POP for example) that actually meant the user needed to type in user@domain as their username. So, while strings may not be ideal, it's almost certainly possible for them to let you do what you want. I wouldn't recommend that approach if you have a bunch of extra data to deal with... but you don't really give a clear picture of what extra data you do have to deal with :).
1) ignore the restriction (credentials types are so specific to OAuth that it's very unlikely you'll find anything already pluggable...)
but the avatarID isn't a credential. It's a mapping between the checker (which is about authentication) and the realm (which is about application data). There hopefully _should_ be such a thing as some "already pluggable" application data.
2) use the mind object (but then it becomes somewhat wonky what goes on the mind object and what should be the avatarId)
Use the mind object for what??? You're asking this question completely backwards. Even after reading the whole email I'm still not quite sure what you're on about. I'll do my best to answer a few specific quirks here, but please re-post starting with what you're actually trying to do, i.e., "I got up today, and I wanted to paint the shed." If you're already most of the way into an implementation, feel free to present bits of the implementation itself, but you haven't really described enough of it here for me to understand what you're doing and why.
Only one of these things are really "credentials" from cred's perspective. Cred's purpose is to tie together two things: a realm that implements some protocol/application logic, and a checker that implements some authentication logic. _everything that happens before you start talking to the application_ is just authentication stuff, and is part of the checker. Even if that stuff, itself, needs to be pluggable. I don't know the specifics of how OAuth works, but I know vaguely what it does, and ultimately what it needs to do is give you access to an IResource that allows you to do ... stuff. The stuff-doing part is the interesting part. That is the part that the Realm gives you. The Realm ought to be able to work with Basic authentication, and the objects returned from the realm should be able to work even with no authentication (if you wanted to just put your resource endpoint up for everyone to use, unauthenticated, which is granted not very sueful).
Because (1) is pretty much always the same (and is common to other parts of txOAuth) and (2) depends on the specific kind of request, this results in two portals:
1) for authenticating the client, which returns an IClient
I have no idea what an "IClient" is. It sounds like you're using cred internally so that you can make the actual OAuth-ing process plugabble, and that's great, as long as you separate it from the post-OAuth part of actually producing a resource.
2) for authenticating the credentials (which includes the authenticated client, since credentials are client-specific), which returns the token
That part sounds like what I would expect.
Coming from Perspective Broker I think the Client would be an apt mind object for the second portal, since it represents the entity on the other side trying to do auth.
The Mind is the entity on the other side trying to get something done, not trying to auth. The docs sorely need to be updated to reflect this, but the canonical example of the 'mind' is the IChatClient interface in twisted.words (used in a cred context by twisted.words.service). Of course, the IChatClient provider may also be involved in authentication, for example receiving a NickServ challenge from the server, but by the time you start calling that object a 'mind', authentication is done.
The appropriate credentials checker in the second portal checks (and invalidates, since most of the credentials are single use) the credentials. What avatarId is sent to the Realm? The Realm is also responsible for requesting the new token,
OK, and that's where things go wrong. The realm is responsible for one thing only: returning an object which implements an interface that allows a user of the library to define some application logic. It should not be participating in auth. If the authentication layer needs to perform some additional logic on each request, it should wrap up the IResource (or whatever) provider returned by the Realm, so that the Realm can do the work. (and furthermore, why is anybody responsible for "requesting the new token" in the first place? why do any new tokens ever need to be requested on the service provider's side of things?)
so it needs to know details about the credential that the credential checker just checked (and in doing so, invalidated). These are not available through the IClient mind object.
The alternative design is that all of the credentials checkers know how to request tokens, and instead give the tokenId as avatarId. This is still insufficient, because tokens can include extra information, such as scope, expiration time... That can again be solved by sending an opaque string value which contains all of the data in the token response, but in that case the Realm is completely useless and the checkers do all of the work. Additionally, this seems stupid because cred checkers are for checking credentials, not checking credentials + a bunch of other stuff.
This has historically been a weak point for cred. While I'm trying to give you a better idea of how to use it "right", you should be keenly aware that it isn't perfect and doesn't necessarily handle every case, so there are some things that you might try to do which won't be well supported. Since cred is so mysterious and subtle, sometimes people believe that if they could just grasp its essence all their problems will go away; but it only solves one problem, so if that problem isn't the one you're facing, you may still be out of luck :). The idea is that the realm is supposed to be a backend that has its own storage and logic and so on, and the checker is supposed to have enough information to produce an avatar ID that points at something the realm knows about. The fiction that cred promulgates is that checkers are somehow agnostic to realms, but can still perform this mapping reliably based on their credentials. Of course, this can't be universally true: if the checker needs to map things into the realm, then it stands to reason it *does* need to know something about the realm. This level of flexibility is needed very rarely though, so what ends up happening is that all the existing checkers just relay the 'username' field to be the avatar ID. In the event that you do need more flexibility - let's say your accounts database uses a peculiar convention to encode the authentication type into the stored user ID so that the same user might get different behavior based on how they authenticate - you could easily write some custom checkers that wrap simple underlying checkers and sprinkle on a little special-case logic to mutate the avatarID appropriately before returning it. I think you're actually a little lucky to be implementing OAuth rather than OpenID, because my understanding is that with OAuth the expectation is that, as a provider of a service (i.e. someone likely to be using the cred interfaces, not a client) you expect that users have already created an account, and so the realm already has some data. "expiration time" can be handled by the checker, or at least by the code that's driving the portal, to set session expiration. (I'm not really sure what "scope" is, perhaps you could elaborate on some of these additional fields.) The place where cred really falls down is multi-field stuff. For example, in OpenID, a new user might show up with some OpenID metadata, like their preferred nickname, email address, and location. Cred provides no way to tell the realm "create an account like this" or "are there any other users with this email address", so if you need to do *that* kind of thing, you're going to need to work on expanding the API.
It hadn't really occurred to me to abuse the 'mind' object to provide that sort of structured data, but being able to specify the mind as "something adaptable to (interface X which provides the info you need)" is a nice big loophole :). And the realm can return whatever it wants; it doesn't need to be based on the avatarID. If you wanted to cram some more structured fields in there to smuggle them to the realm, that's a better place to put them than the strictly-mandated it's-just-some-bytes avatarID. (This would allow other 'regular' HTTP realms to work with your OAuth code, but provide extra OAuth stuff for realms that want it). I'm still not convinced that your realm should actually know about these extra fields, but maybe you just haven't given the relevant example yet.
A nomenclature distinction that might help this conversation: *authentication* verifies *identity* (via, e.g., userid/passwd, public key negotiation, etc.) and may issue a token (e.g., a cookie, Kerberos TGT, etc.). *authorization* determines what an authenticated identity (and its associated roles, if any) have access to in an application context (realm?). These are standard terms, and the activities they refer to should be orthogonal. Neither the OP nor Glyph use the term "authorization" in either of their messages, but that concept is clearly involved and is almost always useful for clarification. Looking at the OAuth 2.0 protocol specification, it seems quite similar in concept to Kerberos (and other SSO protocols), in which the user acquires a Ticket-Granting Ticket (TGT) upon authentication to a Kerberos domain controller, and then the TGT is used when interacting with applications in the Kerberos domain. Each app is registered with the Kerberos domain controller and can interact with it securely to verify the validity (also expiration time, etc.) of a TGT that has been offered to it. Once that interaction is complete, the app knows the identity associated with the TGT has been authenticated, and it can proceed with authorization, which of course depends on each application's context, and is completely separate from authentication. It might be useful to describe cred's structure in these terms, since both authentication and authorization are involved. (Might be useful to me, at least ... :) Steve
On Sep 8, 2010, at 1:27 PM, Stephen Waterbury wrote:
The checker authenticates; the realm authorizes. Authorization proceeds from the realm's idea of what a particular avatar ID (and, apparently, mind, as laurens has discovered this particular loophole in the API) is authorized to do; authentication proceeds from what the checker thinks makes some credentials valid. As you put it:
replace "application" with "realm" here and that's basically how twisted.cred works. The reason I didn't use the term authorization in my original message is that we're talking about an authentication protocol, and hopefully authorization can stay out of it :).
Actually, I believe much of the confusion is cleared up. Apparently I misunderstood what a Realm does. Oops! The application in the original mail, which I will continue to talk about in this one, is a token endpoint. I've been told an explanation of what that is is in order. To start out: it is NOT the thing that actually serves protected resources, that's a resource server, which is a different beast entirely. The job of a token endpoint is to turn certain kinds of credentials into an access token; the access token, in turn, can be used to access protected resources. So basically, these credentials are about already provided access grants, that haven't been turned into access tokens yet. Examples of such credentials which can be turned in for access tokens: - authorization code: This represents an access grant from the end user. Instead of receiving an access token when the grant was issued, you got this code, which basically is a credential that says "This person is allowed to access the protected resource, but we haven't given him a proper token to do that yet". The token endpoint is for turning one of these into a token. - refresh token: in case your access token has been used up or expired, one refresh token can be exchanged for one access token - assertion: this is basically just a pair of opaque strings (a type and a value); it's used for tying OAuth in with existing systems. Hence, this is deliberately vague. - user credentials: this is the classic username/password. It's a bit silly to use this; the only advantage is that you can limit which servers need access to these credentials. The key thing to remember is: token endpoints let you trade in some credentials that prove you're supposed to have access for a token that actually *gives* you that access. Since people asked for more detail as to which data exactly needs to be passed to the Realm (yes, even now that I know what the Realm actually does). This is basically all of the data in the token endpoint's response. The response contains the following data, in JSON format: (explanations of what each thing is in the following paragraphs) - the access token (an ascii string, mandatory) - the expiration time (optional) - the refresh token (another ascii string, similar to the access token, optional) - scope (an ascii string consisting of a set of space-separated words, optional) Optional here does not really mean "at my option" but "occasionally required", so it is necessary that all four can be passed to the IRealm at the same time. Furthermore, it's impossible to use the Mind for some of them, because ICredentialsCheckers never see the Mind, and all of that data (access token, refresh token, expiration time, scope) are only ever visible to the ICredentialsChecker. Again, the access token is something you use to actually access a protected resource. The refresh token is something you use to get a new token when your old one expires or is used up. Defining the scope properly is hard, it's deliberately vague in the spec as well; it's a set of strings that determine what you can do with your shiny new access token. For example, for Facebook, the scope might be "name friends likes" for a token that allows someone to access your name, your list of friends and what you like, but not your pictures, gender, date of birth. These values are not determined by the spec, and you can just make some up for your service. Also, to clear up a misunderstanding:
Does that help? Laurens
On Sep 8, 2010, at 8:12 PM, Laurens Van Houtven wrote:
Actually, I believe much of the confusion is cleared up. Apparently I misunderstood what a Realm does. Oops!
The application in the original mail, which I will continue to talk about in this one, is a token endpoint. I've been told an explanation of what that is is in order. To start out: it is NOT the thing that actually serves protected resources, that's a resource server, which is a different beast entirely. The job of a token endpoint is to turn certain kinds of credentials into an access token; the access token, in turn, can be used to access protected resources. So basically, these credentials are about already provided access grants, that haven't been turned into access tokens yet. Examples of such credentials which can be turned in for access tokens:
(...)
Does that help? Laurens
Somewhat, but it really casts the main issue into sharper focus: for any given cred implementation, there are two key questions: What is the avatar interface? What is the associated mind interface? If we're not talking about a system where the avatar interface (the one passed to 'login', the one that the realm must return something that gets implemented) is IResource, then that's the problem. What is that interface that you're talking about? Be very specific: what methods does it have and why? It sounds like OAuth is very precisely implementing something that maps almost exactly onto the checker part of cred, but you're doing it by implementing a realm to allow applications to provide their own logic. This might not be the wrong idea, but it needs to be spelled out very clearly. As you describe it:
token endpoints let you trade in some credentials that prove you're supposed to have access for a token that actually *gives* you that access
In cred-ese, that would be "checkers let you trade in some credentials that prove you're supposed to have access for an <avatar ID, which you give to a Realm> that actually *gives* you that access". As far as the OAuth response is concerned, it's like a checker. Looking at the individual parts, assuming that the avatar interface for the purposes of this discussion is IResource, it breaks down like this in my mind:
- the access token (an ascii string, mandatory)
Avatar ID. (See, it's a str!)
- the expiration time (optional)
Implementation detail of the session. Avatars are actually sessions, which expire: sometimes (as with POP) at the end of a connection, sometimes (as with nevow.guard's HTTP support) with a session timeout.
- the refresh token (another ascii string, similar to the access token, optional)
Implementation detail of the authentication protocol. The client library and server library should be transparently refreshing this without telling either the client application code or server application code, right?
- scope (an ascii string consisting of a set of space-separated words, optional)
This part doesn't quite fit, but could be expressed in one of two ways: a modification to the avatar ID, or as some extra structure on the Mind that modifies what functionality the Realm bundles in to your avatar implementation (without changing the interface provided by that object, of course).
I'll give a very practical example of an example request + response, because I'm not entirely sure I communicated that properly. The code we're discussing is part of a TokenEndpoint, which is an IResource. Something (a client, in OAuth terminology) makes a request that looks like this: --- POST /token HTTP/1.1 Host: server.example.com Content-Type: application/x-www-form-urlencoded grant_type=password&client_id=s6BhdRkqt3&client_secret=47HDu8s&username=johndoe&password=A3ddj3w --- And hopefully the code we're discussing answers with a request that looks like this: --- HTTP/1.1 200 OK Content-Type: application/json Cache-Control: no-store { "access_token":"SlAV32hkKG", "expires_in":3600, "refresh_token":"8xLOxBtZp8" "scope": "tummies cookies parrots" } --- Where, as mentioned before, everything besides "access_token" is not always required (but it must be possible to set them, because occasionally they are required). On Fri, Sep 10, 2010 at 2:34 AM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:
Okay, to be specific: I believe the appropriate interface is IResource. This is in line with t.web's general way of interacting with cred: you give me credentials, I give you a protected IResource.
Doing everything in the checker probably might make more sense. The reason I originally thought to do this in the IRealm is that I figured credentials checkers should just check credentials, and creating a new credential (the access token) wasn't part of ICredentialsChecker's job (more like IRealm's job). Apparently I was mistaken.
Heh, okay; as long as there's a plausible way to get the other important information (see rest of email) into the HTTP response, I'll believe you.
Right, but: a) The avatar needs to know about this timeout, since the timeout information needs to be able to make it into the HTTP response. b) The expiration time is only known to the thing that creates the access token (obviously). From the previous discussion, this is apparently ICredentialsChecker. c) (From (a, b)) The ICredentialsChecker needs to be able to communicate the expiration to the avatar. d) The ICredentialsChecker and IRealm can only communicate through the avatarId. Conclusion (from c,d): The expiration time needs to be in the avatarId?
Sure, but we're still writing library code here. If the avatar interface isn't actually IResource, but a new interface IAccessToken, getting the refresh token later might be feasible. (It should be a different interface, because in order to create a refresh token outside of this entire cred cycle, I need to know about the thing it's refreshing -- so, the information in the response needs to be easily accessible and not just an opaque IResource). I agree entirely that clever client library code would abstract this mess away from application code. However, right now this code still needs to somehow be able to eventually produce HTTP responses that don't abstract anything yet and just contain all the appropriate data, because that's just what the OAuth spec says it needs to be able to do. There isn't any real application code in the token endpoint; they're pretty similar for all OAuth setups, and customization would typically happen through implementing the appropriate ICredentialsChecker. - scope (an ascii string consisting of a set of space-separated words,
Unfortunately I don't believe this can be done through the mind. Again working under the previous assumption that ICredentialsChecker and not the IRealm is responsible for creating the access token, and scope is known to the thing that makes the access token, the ICredentialsChecker knows about the scope. Unfortunately the only way to pass stuff between the ICredentialsChecker and the IRealm is the avatar ID, so you don't have a choice. thanks for your infinite patience, Laurens
On Fri, Sep 10, 2010 at 10:29 PM, <exarkun@twistedmatrix.com> wrote:
Is the timeout authenticated?
Jean-Paul
In the sense that it's not a disaster if you lie about it -- no. Timeouts are generally optional and really just a friendly hint to the client receiving the token to not expect it to work after X time. Laurens
On Sep 10, 2010, at 1:01 PM, Laurens Van Houtven wrote:
Okay, to be specific: I believe the appropriate interface is IResource. This is in line with t.web's general way of interacting with cred: you give me credentials, I give you a protected IResource.
This doesn't quite make sense to me. Mostly, I don't understand what the moving pieces are here. If this is all to implement a "token endpoint" which doesn't "actually serve protected resources", then what is the IResource being returned by the realm actually *doing*? If it is not, itself, a "protected resource", then isn't the amount of stuff that its IResource can do severely restricted (i.e. restricted to doing a few OAuth-specific HTTP things) and therefore actually an avatar of a completely different, OAuth-specific interface, which just happens to be wrapped in an IResource by the OAuth implementation for the benefit of formatting those OAuth-specific things over an HTTP channel? Based on what you've said, I would expect the avatar interface to be 'IToken', since the point of this authentication is to 'generate tokens'.
It sounds like OAuth is very precisely implementing something that maps almost exactly onto the checker part of cred, but you're doing it by implementing a realm to allow applications to provide their own logic. This might not be the wrong idea, but it needs to be spelled out very clearly.
Doing everything in the checker probably might make more sense. The reason I originally thought to do this in the IRealm is that I figured credentials checkers should just check credentials, and creating a new credential (the access token) wasn't part of ICredentialsChecker's job (more like IRealm's job). Apparently I was mistaken.
I'm going to handwave a bit here and pretend we're talking about a thing that actually does serve arbitrary resources, so my answers may be slightly off. But I think they may be applicable either way. Speaking the authentication protocol is the authentication implementation's job, not the realm or the checker. For example: if you need to generate a challenge as part of the authentication process, the checker probably doesn't need to get involved there. If you need to ask the checker something about the challenge (let's say you have a list of persistent salts), then there needs to be a method on the credential interface to allow the checker to relay that information back. It strikes me that creating a new access token is also an authentication feature. If it is, then it goes into the OAuth resource somewhere. If it needs to be pluggable then it needs to go into the ICredentialsChecker or possibly the realm, but it's possible that this is just an OAuth specific interface that is plugged in to the OAuth-doing IResource separately, as a constructor argument that has nothing to do with cred.
Yes. The way to get it into the HTTP response is: put it into the HTTP response :). You don't necessarily need to cram it into the realm.
I'll stop you right there: No. (We can skip the rest of points b, c, and d, since they all apparently follow from this.) The avatar's job is to give you an IResource, but the OAuth implementation doesn't necessarily need to pass through either the arguments to that resource or that resource's return values (from getChild or render) exactly. If the timeout needs to be put into the HTTP response as a header, the OAuth resource could wrap the IResource value that it gets from the realm and set the header before calling the ultimate resource's child. Note that this isn't just the credentials checker: this is the OAuth implementation. Relaying information there is easy though: depending on how the checker itself gets created and parameterized, either you can have the checker simply have a reference back to the OAuth resource so it can call some methods, or you can expand the OAuth credentials interface to have a method that the checker uses to pass that information back along with the resource.
If the avatar interface isn't actually IResource, but a new interface IAccessToken, getting the refresh token later might be feasible. (It should be a different interface, because in order to create a refresh token outside of this entire cred cycle, I need to know about the thing it's refreshing -- so, the information in the response needs to be easily accessible and not just an opaque IResource).
I agree entirely that clever client library code would abstract this mess away from application code. However, right now this code still needs to somehow be able to eventually produce HTTP responses that don't abstract anything yet and just contain all the appropriate data, because that's just what the OAuth spec says it needs to be able to do. There isn't any real application code in the token endpoint; they're pretty similar for all OAuth setups, and customization would typically happen through implementing the appropriate ICredentialsChecker.
Again: producing HTTP responses is not the same thing as producing IResource implementations. You can have an HTTP response produced by an IResource that wraps another IResource and adds some special sauce to the response. So this information still never needs to make it to the realm.
OK, I admit it, by the time I get here I'm lost. We should try to meet up and have an interactive conversation about this, because I have a ton of questions about this, and I don't understand who generates the scope ID and why, and why this bit of information would need to be relayed between the checker and the realm in the process of authentication. Basically this goes back to the first confusion I mentioned at the beginning of this email: I don't understand why the realm is giving back an IResource if the whole purpose is to just generate tokens and return them.
participants (4)
-
exarkun＠twistedmatrix.com -
Glyph Lefkowitz -
Laurens Van Houtven -
Stephen Waterbury