[Twisted-Python] Potential PB Security Problem (And Solution)
While auditing some code that I'd written to use PB, I noticed a class of bug that I had previously not considered. When writing a PB remote interface, it's easy to write some code that looks like this: class MyObject(Referenceable): def remote_IOnlyExpectCopies(self, someCopy): someCopy.someOperation(self.somePrivilegedData) This is, of course, a potential security hole. Assuming somePrivilegedData is a basic type (and therefore serializeable), a hostile client could send a Referenceable of their own to IOnlyExpectCopies. A sufficiently vigilant application author would, of course, be able to protect against this by distrusting any arguments sent to a remote method and performing type checks on them, but then, application authors (including myself) are rarely "sufficiently" vigilant :-). My proposed solution is to change the way remote methods are invoked: instead of emulating regular Python methods, they would be accessed through a 'callRemote' method: fairly simply, calling the remote method 'foo' on 'bar' would look like this: bar.callRemote("foo", baz, boz=qux) This way, the above example of a local method call would blow up a remote object were passed to it. Pros of this approach: * it fixes this potentially common security problem * it makes it possible to grep for all places where a method is invoked remotely * it removes some overhead (creation of the RemoteMethod instance) Cons of this approach: * it's no longer possible to treat remote objects and objects that have methods which return Deferreds identically * slightly more typing * massive refactoring required, lots of user code might break I think the pros clearly outweigh the cons, so I'm going to start changing things over, potentially with a backwards-compatibility release in the interim. If anyone has a different idea, let me know. -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Fri, 2002-02-15 at 15:44, Glyph Lefkowitz wrote:
It's not "slightly more typing", it's a massive blow to the elegance and transparency of spreadable object code. I guess you're saying that the transparancy has become a liability, as you can't necessarily tell if you're invoking a local or remote object (...but I thought that was the point?). I'm also a little suprised because this is the first time I've seen you vouch for any sort of "safety" mechanism to protect the programmer from doing wayward things with eir data. It also sounds like that this use case is inherently insecure. You're passing privledged information to some-object-passed-to-you-from-who-knows-what. I'm not sure there's really anything you can do about that. -- The moon is waxing crescent, 10.3% illuminated, 3.1 days old.
On Fri, 2002-02-15 at 20:11, Kevin Turner wrote:
It was never my intention that remote object communication be transparent, only convenient. (The party line is "translucent".) In fact, one of the use-cases that radix and I discussed on IRC was the ability to have locally-callable asynchronous objects that behave like PB references. Since a PB object's semantics will differ slightly if it is local or remote if you were to just wrap its results in a deferred (thanks to serialization, and such) it would be desirable to have a local interface as well, which could perform the necessary mutations on the data to make it look as though it'd been serialized. As it stands now, this wrapper would be pretty thick, creating at least 3 separate instances for a single method invocation. In the case that I'm proposing, though, all you'd need is to subclass a mixin that implements callRemote and dispatches to "async_" prefixed method instead. From the user code point of view it would be about the same, but it would simplify the implementation a lot.
Well, what I propose to do about it <0.5 wink> is make it explicit that the information you're sending *may* be going to a remote source. This causes the broken case -- where the client is trying to masquerade a reference somewhere bad -- blow up, rather than merrily send your information somewhere it shouldn't go. If we intend to use this API for a general basis for client<->server communication, it is desirable that we make it require as *little thinking as possible* to write secure code. Buffer overflows are, after all, something that can be easily prevented in C; it's just that the fact that it's hard makes it a common mistake. -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Fri, 2002-02-15 at 20:39, Glyph Lefkowitz wrote:
It was never my intention that remote object communication be transparent, only convenient. (The party line is "translucent".)
Be that as it may, this seems like a rather large loss of convenience in most cases. surely we can make this behaviour a runtime option on Broker? Broker.enableParanoia, Broker.disableParanoia methods perhaps? i leave the question of which is the default up for debate. :) Allen Short Programmer-Archaeologist washort@twistedmatrix.com `Software Engineering: How to program if you cannot.' --- Dijkstra
On Fri, 2002-02-15 at 20:29, Allen Short wrote:
Especially given my angle of approach to Twisted. "What is it?" "It's this great stuff that makes network applications super-easy to write." "Oh? How's it do that?" "Well there's this cool thing called PB [...] and then you can just call methods on objects [...] internet!"
On Fri, 2002-02-15 at 22:29, Allen Short wrote:
The semantics of remote objects will remain identical, the syntax will change slightly. It will still be possible to emulate remote objects by implementing the callRemote method to do local dispatch (which could be automated). I guess I'm just being thick here. What is the convenience being lost? The additional 10 characters per remote invocation doesn't seem that significant to me. (After all, it is currently still necessary to emulate the behavior of remote objects, since they're always asynchronous.) -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Sat, 2002-02-16 at 05:47, Glyph Lefkowitz wrote:
it does to me, if only because of the existing code using it. if you want to deprecate the current way of doing things, that'd work, i think: but let's not spawn a massive refactoring frenzy all at once, please? -- Allen Short Programmer-Archaeologist washort@twistedmatrix.com "TECO was *IT* man, TECO was more fun than a sendmail.cf file. TECO was a bar you had to clear if you wanted to be a Real Hacker (tm)." - Foobar in afc
On Sat, 2002-02-16 at 11:55, Allen Short wrote:
Heh. I have to deal with a volume of code about 4 times the size that is publicly visible, so that is not lost on me :). I do plan to slowly deprecate it, with warning messages printed each time a "bad" remote method is called. Otherwise I'd be sure to miss at least one remote notification on the first pass (after all, part of the problem is that they're difficult to locate right now since they're not tagged with anything distinctive...) -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Fri, 2002-02-15 at 21:11, Kevin Turner wrote:
For as long as I remember, that was *never* the point. Haven't you ever heard glyph shouting from on high, "Explicit is better than implicit!"? :)
It's a different matter when you're talking about networked-app security.
Well, the point was, the code wasn't *meant* to pass privelaged information to an object across the wire - notice that it was a Copied. This could also be done for more things than Copied - imagine a method that takes a list as one of its arguments and appends some secure data to it. You could spoof a list with a remote object that has an append() method. While I think 'callRemote' is slightly less aesthetic than previously, this is still a serious issue that needs to be dealt with. I guess I can live with callRemote if it's the best way to explicit-ize remote methods. -- Chris Armstrong << radix@twistedmatrix.com >> http://twistedmatrix.com/users/carmstro.twistd/
On Fri, 2002-02-15 at 19:12, Christopher Armstrong wrote:
For as long as I remember, that was *never* the point. Haven't you ever heard glyph shouting from on high, "Explicit is better than implicit!"?
Funny words for a sect which celebrates in masqurading objects and generally deplores strict type checking.
What do you mean, "wasn't meant to pass privledged information"? Your example has you appending the "secure data" to a list that's from who knows where! You have no idea who holds a reference to that list, or who will hold references to the list (or certain items on the list) in the near future. And, as a happy little object, I don't see why you should care. You perform your operation with the given parameters, and that's all your job is, isn't it? I guess there's the "Referenceable vs Copyable" facet which I haven't been paying a lot of attention to in this discussion. When I choose between those two spreadable flavours (the module name is spelled wrong, BTW), I am thinking about what goes over the wire, but only in terms of how much it will change, how much the states of the objects need to be synchronized, how much traffic it's generating, etc. But I've never seen the use of Copyable as a security option. If I had data I did not want getting outside, I do not think I should entrust it to *any* object which came from Outside, whether it be a RemoteCopy or RemoteCache or a list-like object with whiskers. I guess it's only fair to admit that I've written very little application code with twisted.spread and read even less, so it's entirely possible that I lack the experience to understand the issues here... but some things bein' said just don't ring true to me. "So-at-least-ONE-of-us-has-been-replaced-by-an-evil-robot"-ly yours, Kevin -- The moon is waxing crescent, 10.8% illuminated, 3.1 days old.
On Fri, 2002-02-15 at 22:04, Kevin Turner wrote:
Dynamic typing is very cool, for reasons I've described in detail elsewhere. Hence, I want to make it still be easy to write secure PB code without mandating the writing of IDLs. Assuming that whatever code you write *must* be secure, which is more of a hindrance: foo.callRemote("bar") or requiring a full interface definition before you can even publish your object? The distinction between masquerading locally (fooling yourself) and masquerading remotely (fooling others) is the difference between having a vivid imagination and being a con artist...
There are two kinds of "who" here. One "who" is to know the objects that will have knowledge of your object. That's encapsulation, which is all well and good, but we break it sometimes, for various reasons; python's nifty because it lets us do that when necessary. The other "who" is more important, though: it's the actual *people* that will have access to the information that you're sending through that method call.
RemoteCopy and RemoteCache are both local classes; e.g. any information you pass to methods that exist on them gets dealt with locally. At least in the code I'm writing for work, I'll often want to embed "privileged" code in a RemoteCopy class, since it is dealing with *information* from the "outside", but behavior locally. For mirroring the view of a simulation on a server, this is a common design pattern, I'd imagine. The proposal on the table makes the locality of the behavior much more explicit -- unless your method begins with 'callRemote', the behavior you're invoking will *definitely* be local. Attempts to invoke (potentially) remote behavior without first saying so will just blow up.
WELL OF COURSE YOU HAVE NO IDEA WHAT YOU'RE TALKING ABOUT THEN!!!! ;-) Seriously, thanks for your response; this is exactly the sort of discussion I wanted to generate. You haven't convinced me that I was wrong yet ;-), but I would like it if the motivations for doing this are clearly understood. If you haven't read much PB code, the problem I'm describing might seem a trivial concern, but I've found at least two instances of this problem "in the wild". One was a mistake made by me and one was someone else. Neither could be made to divulge critical information, but both could be exploited to give a user an unfair advantage or consume a large proportion of the server's CPU/bandwidth. Do you have a solution that's less aesthetically unpleasant? I think that this is a good compromise all around (mumble, frotz, I want macros and promises and continuations so I can say [wait-for [tell obj message arg arg arg]]). But then again, I actually find it *more* aesthetically pleasant to say THIS HERE IS LIKELY TO BE REMOTE. Despite the other advantages of moving to PB methods returning Deferreds, I was glad that I could previously grep for 'pbcallback' and get most of the methods that would be invoked remotely.
"So-at-least-ONE-of-us-has-been-replaced-by-an-evil-robot"-ly yours,
Who would want to replace me with another evil robot? -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Sat, Feb 16, 2002 at 05:42:48AM -0600, Glyph Lefkowitz wrote:
coming completely out of the blue with no experience or understanding to give me enough qualifications to even comment (but doing it anyway :-). Just a thought; why is a local object any more trustworthy than a remote one? To me the local vs remote trust boundary seem to be a bit arbitary. It is better to think about the trust relationship between the objects than where they are located. Why and how do the objects trust each other? Perhaps this _can_ be reduced to "Because it's local and hence I or some other local object I trust created it". However, I think once you start going down the path of secure transactions between objects, it's better to try and provide a generalised solution. There are various ways that authenticated and secured communications between objects can be implemented, and perhaps even made "translucent". -- ---------------------------------------------------------------------- ABO: finger abo@minkirri.apana.org.au for more info, including pgp key ----------------------------------------------------------------------
On Sat, 2002-02-16 at 06:39, Donovan Baarda wrote:
On Sat, Feb 16, 2002 at 05:42:48AM -0600, Glyph Lefkowitz wrote:
You're confusing "objects" and "people". I don't trust the remote objects because (A) the communication to them is potentially visible over the wire and (B) they don't actually exist on physical hardware I control, e.g. their behavior might have been tampered with. They are under the complete control of another person, who might as well be speaking the wire-protocol personally. Thinking from a security viewpoint, they *have* already been tampered with, and the question is "how do I reduce the impact of that on my service?".
What gave you the idea we were talking about secure distributed transactions? I'm just talking about making our current implementation of remote communications more secure (by default). -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Sat, 2002-02-16 at 03:42, Glyph Lefkowitz wrote:
Is there a way to fix this? My lack of education, that is. There's more code within the Twisted repository left more me to read, but IIRC there are actually very few PB interfaces defined there. (We have Words and Manhole and...?) More pertinant here would be other applications (from the "wilds") which employ Twisted technology. Are there any such 0775 codebases I should be aware of? The accusation has been made that no one actually *uses* Twisted for anything (not to name names, but he's Canadian and works in Japan), but assuming that's not the case, maybe we could beef up the "sites using Twisted" list a little.
Who would want to replace me with another evil robot?
"Do not trust the Shover-robot, he is malfunctioning," - Kevin -- The moon is waxing crescent, 14.8% illuminated, 3.7 days old.
On Sat, 2002-02-16 at 14:26, Kevin Turner wrote:
Ohhh, excellent idea!! Let's break a bunch of interfaces, and then when everybody who uses Twisted starts complaining to us, we can get them to tell us where they use it!
Who would want to replace me with another evil robot?
"Do not trust the Shover-robot, he is malfunctioning,"
I wonder if glyph has stairs in his house... -- Chris Armstrong << radix@twistedmatrix.com >> http://twistedmatrix.com/users/carmstro.twistd/
On Sat, 2002-02-16 at 13:26, Kevin Turner wrote:
Distributed twisted.web, but you're right, that's about it.
It is indeed unfortunate that much Twisted-using code is closed. Not my decision, I assure you ;). I believe Itamar is using PB, so you could ask to look at his code -- I know that Sean was planning on releasing a small demo at some point. However, this is a circular problem. Much of this past week for me has been spent working on a new website for tm.com and trying to come up with introductory documentation for PB. It needs to be documented, standardized, and released as 1.0 before we can really expect droves of people to come flocking to it. Does anyone else on this list have some code to reference? The other folks I know of using Twisted are mostly using the other components at this point.
Who would want to replace me with another evil robot?
"Do not trust the Shover-robot, he is malfunctioning,"
"Do not trust the Pusher-robot, he is doing a little too much of his own stuff." -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
Glyph Lefkowitz wrote:
Yep. However, for clarity, lets not forget twisted.spread the implementation is not the same as the PB the protocol. E.g. Perspectives are an implementation choice, not part of the protocol. Or how to call remote methods - this is an implementation choice, in Java the __getattr__ eay isn't even relevant.
Kevin Turner <acapnotic@twistedmatrix.com> writes:
http://mc-foo.sourceforge.net/ (note that parts of it are sick, but not twisted) -- tv@{{hq.yok.utu,havoc,gaeshido}.fi,{debian,wanderer}.org,stonesoft.com} double a,b=4,c;main(){for(;++a<2e6;c-=(b=-b)/a++);printf("%f\n",c);}
well, as someone who HAS written a lot of spread code I'd say that the current interface that assumes method calls on remote objects are remote can be confusing. In a few cases I have found my code doing strange things on the network when I was expecting local behavior. The "callRemote" syntax would make remote operation explicit - which I am in favor of. Another observation from writing lots of spread code is that the remote interfaces for objects tend to be very small. This reduces the impact of the "more typing required" argument against this change. In my current application that uses pb which is 4300 lines of python and about 70 classes, there are only 19 remote methods defined. Changing my code over to use the "callRemote" syntax would take a couple of hours at most. (i have unit tests :) ). BTW, I was the other person that Glyph mention who had written code exploitable by this issue. -----Original Message----- From: twisted-python-admin@twistedmatrix.com [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Kevin Turner Sent: Friday, February 15, 2002 10:05 PM To: Twisted List Subject: Re: [Twisted-Python] Potential PB Security Problem (And Solution) On Fri, 2002-02-15 at 19:12, Christopher Armstrong wrote:
For as long as I remember, that was *never* the point. Haven't you ever heard glyph shouting from on high, "Explicit is better than implicit!"?
Funny words for a sect which celebrates in masqurading objects and generally deplores strict type checking.
What do you mean, "wasn't meant to pass privledged information"? Your example has you appending the "secure data" to a list that's from who knows where! You have no idea who holds a reference to that list, or who will hold references to the list (or certain items on the list) in the near future. And, as a happy little object, I don't see why you should care. You perform your operation with the given parameters, and that's all your job is, isn't it? I guess there's the "Referenceable vs Copyable" facet which I haven't been paying a lot of attention to in this discussion. When I choose between those two spreadable flavours (the module name is spelled wrong, BTW), I am thinking about what goes over the wire, but only in terms of how much it will change, how much the states of the objects need to be synchronized, how much traffic it's generating, etc. But I've never seen the use of Copyable as a security option. If I had data I did not want getting outside, I do not think I should entrust it to *any* object which came from Outside, whether it be a RemoteCopy or RemoteCache or a list-like object with whiskers. I guess it's only fair to admit that I've written very little application code with twisted.spread and read even less, so it's entirely possible that I lack the experience to understand the issues here... but some things bein' said just don't ring true to me. "So-at-least-ONE-of-us-has-been-replaced-by-an-evil-robot"-ly yours, Kevin -- The moon is waxing crescent, 10.8% illuminated, 3.1 days old.
OK, My first thoughts - I think that this is a good idea. The problem with pretending to be a regular method call is that, well, it isn't. You may get disconnected, for example. So we want to encourage people to realize thjat they can't just use the same code. Of course, this'll make web.distrib rather more difficult to implement nicely... However, if we're unto security, some form of static typing on arguments passed from remote clients to Referenceables methods would be very useful, some would say necessary, for secure programming. This would aid documentation as well. Switching to callRemote still doesn't solve the "passing the wrong objects maliciously" issue, just one very specialized instance of it. And I ain't suggesting IDL - I'm sure we could come up with a solution embedded in python source code that is easy to type (say, using oscar-style docstrings or some ther form of annotation, and a enforceInterface(klass) function that parses them and uses t.p.hook).
On Sun, 2002-02-17 at 01:27, Itamar Shtull-Trauring wrote:
Nah. We already have to manually frob the interface -- peeling off methods of a remote reference and sticking them to a copy. It would probably look cleaner if we were more explicit about it, anyway :) I suppose each of those one-line assignments would turn into a two-line method declaration (and gasp! a newline to separate them) but it would also provide for more flexible emulation of the request interface. I think that the existing code there may be an example of the "poltergeists" antipattern.
Well, yes. We need a documentation standard for return types (the sound you hear there is another section of the coding standard being exploded in a concrete bunker, several miles away) and we might as well use that to enforce interfaces when we find them. However -- any sort of polymorphism is a potential security hole in that regard. I would like to see some exploit demonstrations of the sorts of holes that one could create by passing the wrong serialized data, provided that remote calls were more explicit. I don't think it's really serious, or could be resolved by stricter type-checking. (After all, any behavior besides "blow up" pretty much assumes that the client is going to be passing an object that adheres to a correct interface.) -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Fri, 2002-02-15 at 15:44, Glyph Lefkowitz wrote:
It's not "slightly more typing", it's a massive blow to the elegance and transparency of spreadable object code. I guess you're saying that the transparancy has become a liability, as you can't necessarily tell if you're invoking a local or remote object (...but I thought that was the point?). I'm also a little suprised because this is the first time I've seen you vouch for any sort of "safety" mechanism to protect the programmer from doing wayward things with eir data. It also sounds like that this use case is inherently insecure. You're passing privledged information to some-object-passed-to-you-from-who-knows-what. I'm not sure there's really anything you can do about that. -- The moon is waxing crescent, 10.3% illuminated, 3.1 days old.
On Fri, 2002-02-15 at 20:11, Kevin Turner wrote:
It was never my intention that remote object communication be transparent, only convenient. (The party line is "translucent".) In fact, one of the use-cases that radix and I discussed on IRC was the ability to have locally-callable asynchronous objects that behave like PB references. Since a PB object's semantics will differ slightly if it is local or remote if you were to just wrap its results in a deferred (thanks to serialization, and such) it would be desirable to have a local interface as well, which could perform the necessary mutations on the data to make it look as though it'd been serialized. As it stands now, this wrapper would be pretty thick, creating at least 3 separate instances for a single method invocation. In the case that I'm proposing, though, all you'd need is to subclass a mixin that implements callRemote and dispatches to "async_" prefixed method instead. From the user code point of view it would be about the same, but it would simplify the implementation a lot.
Well, what I propose to do about it <0.5 wink> is make it explicit that the information you're sending *may* be going to a remote source. This causes the broken case -- where the client is trying to masquerade a reference somewhere bad -- blow up, rather than merrily send your information somewhere it shouldn't go. If we intend to use this API for a general basis for client<->server communication, it is desirable that we make it require as *little thinking as possible* to write secure code. Buffer overflows are, after all, something that can be easily prevented in C; it's just that the fact that it's hard makes it a common mistake. -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Fri, 2002-02-15 at 20:39, Glyph Lefkowitz wrote:
It was never my intention that remote object communication be transparent, only convenient. (The party line is "translucent".)
Be that as it may, this seems like a rather large loss of convenience in most cases. surely we can make this behaviour a runtime option on Broker? Broker.enableParanoia, Broker.disableParanoia methods perhaps? i leave the question of which is the default up for debate. :) Allen Short Programmer-Archaeologist washort@twistedmatrix.com `Software Engineering: How to program if you cannot.' --- Dijkstra
On Fri, 2002-02-15 at 20:29, Allen Short wrote:
Especially given my angle of approach to Twisted. "What is it?" "It's this great stuff that makes network applications super-easy to write." "Oh? How's it do that?" "Well there's this cool thing called PB [...] and then you can just call methods on objects [...] internet!"
On Fri, 2002-02-15 at 22:29, Allen Short wrote:
The semantics of remote objects will remain identical, the syntax will change slightly. It will still be possible to emulate remote objects by implementing the callRemote method to do local dispatch (which could be automated). I guess I'm just being thick here. What is the convenience being lost? The additional 10 characters per remote invocation doesn't seem that significant to me. (After all, it is currently still necessary to emulate the behavior of remote objects, since they're always asynchronous.) -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Sat, 2002-02-16 at 05:47, Glyph Lefkowitz wrote:
it does to me, if only because of the existing code using it. if you want to deprecate the current way of doing things, that'd work, i think: but let's not spawn a massive refactoring frenzy all at once, please? -- Allen Short Programmer-Archaeologist washort@twistedmatrix.com "TECO was *IT* man, TECO was more fun than a sendmail.cf file. TECO was a bar you had to clear if you wanted to be a Real Hacker (tm)." - Foobar in afc
On Sat, 2002-02-16 at 11:55, Allen Short wrote:
Heh. I have to deal with a volume of code about 4 times the size that is publicly visible, so that is not lost on me :). I do plan to slowly deprecate it, with warning messages printed each time a "bad" remote method is called. Otherwise I'd be sure to miss at least one remote notification on the first pass (after all, part of the problem is that they're difficult to locate right now since they're not tagged with anything distinctive...) -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Fri, 2002-02-15 at 21:11, Kevin Turner wrote:
For as long as I remember, that was *never* the point. Haven't you ever heard glyph shouting from on high, "Explicit is better than implicit!"? :)
It's a different matter when you're talking about networked-app security.
Well, the point was, the code wasn't *meant* to pass privelaged information to an object across the wire - notice that it was a Copied. This could also be done for more things than Copied - imagine a method that takes a list as one of its arguments and appends some secure data to it. You could spoof a list with a remote object that has an append() method. While I think 'callRemote' is slightly less aesthetic than previously, this is still a serious issue that needs to be dealt with. I guess I can live with callRemote if it's the best way to explicit-ize remote methods. -- Chris Armstrong << radix@twistedmatrix.com >> http://twistedmatrix.com/users/carmstro.twistd/
On Fri, 2002-02-15 at 19:12, Christopher Armstrong wrote:
For as long as I remember, that was *never* the point. Haven't you ever heard glyph shouting from on high, "Explicit is better than implicit!"?
Funny words for a sect which celebrates in masqurading objects and generally deplores strict type checking.
What do you mean, "wasn't meant to pass privledged information"? Your example has you appending the "secure data" to a list that's from who knows where! You have no idea who holds a reference to that list, or who will hold references to the list (or certain items on the list) in the near future. And, as a happy little object, I don't see why you should care. You perform your operation with the given parameters, and that's all your job is, isn't it? I guess there's the "Referenceable vs Copyable" facet which I haven't been paying a lot of attention to in this discussion. When I choose between those two spreadable flavours (the module name is spelled wrong, BTW), I am thinking about what goes over the wire, but only in terms of how much it will change, how much the states of the objects need to be synchronized, how much traffic it's generating, etc. But I've never seen the use of Copyable as a security option. If I had data I did not want getting outside, I do not think I should entrust it to *any* object which came from Outside, whether it be a RemoteCopy or RemoteCache or a list-like object with whiskers. I guess it's only fair to admit that I've written very little application code with twisted.spread and read even less, so it's entirely possible that I lack the experience to understand the issues here... but some things bein' said just don't ring true to me. "So-at-least-ONE-of-us-has-been-replaced-by-an-evil-robot"-ly yours, Kevin -- The moon is waxing crescent, 10.8% illuminated, 3.1 days old.
On Fri, 2002-02-15 at 22:04, Kevin Turner wrote:
Dynamic typing is very cool, for reasons I've described in detail elsewhere. Hence, I want to make it still be easy to write secure PB code without mandating the writing of IDLs. Assuming that whatever code you write *must* be secure, which is more of a hindrance: foo.callRemote("bar") or requiring a full interface definition before you can even publish your object? The distinction between masquerading locally (fooling yourself) and masquerading remotely (fooling others) is the difference between having a vivid imagination and being a con artist...
There are two kinds of "who" here. One "who" is to know the objects that will have knowledge of your object. That's encapsulation, which is all well and good, but we break it sometimes, for various reasons; python's nifty because it lets us do that when necessary. The other "who" is more important, though: it's the actual *people* that will have access to the information that you're sending through that method call.
RemoteCopy and RemoteCache are both local classes; e.g. any information you pass to methods that exist on them gets dealt with locally. At least in the code I'm writing for work, I'll often want to embed "privileged" code in a RemoteCopy class, since it is dealing with *information* from the "outside", but behavior locally. For mirroring the view of a simulation on a server, this is a common design pattern, I'd imagine. The proposal on the table makes the locality of the behavior much more explicit -- unless your method begins with 'callRemote', the behavior you're invoking will *definitely* be local. Attempts to invoke (potentially) remote behavior without first saying so will just blow up.
WELL OF COURSE YOU HAVE NO IDEA WHAT YOU'RE TALKING ABOUT THEN!!!! ;-) Seriously, thanks for your response; this is exactly the sort of discussion I wanted to generate. You haven't convinced me that I was wrong yet ;-), but I would like it if the motivations for doing this are clearly understood. If you haven't read much PB code, the problem I'm describing might seem a trivial concern, but I've found at least two instances of this problem "in the wild". One was a mistake made by me and one was someone else. Neither could be made to divulge critical information, but both could be exploited to give a user an unfair advantage or consume a large proportion of the server's CPU/bandwidth. Do you have a solution that's less aesthetically unpleasant? I think that this is a good compromise all around (mumble, frotz, I want macros and promises and continuations so I can say [wait-for [tell obj message arg arg arg]]). But then again, I actually find it *more* aesthetically pleasant to say THIS HERE IS LIKELY TO BE REMOTE. Despite the other advantages of moving to PB methods returning Deferreds, I was glad that I could previously grep for 'pbcallback' and get most of the methods that would be invoked remotely.
"So-at-least-ONE-of-us-has-been-replaced-by-an-evil-robot"-ly yours,
Who would want to replace me with another evil robot? -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Sat, Feb 16, 2002 at 05:42:48AM -0600, Glyph Lefkowitz wrote:
coming completely out of the blue with no experience or understanding to give me enough qualifications to even comment (but doing it anyway :-). Just a thought; why is a local object any more trustworthy than a remote one? To me the local vs remote trust boundary seem to be a bit arbitary. It is better to think about the trust relationship between the objects than where they are located. Why and how do the objects trust each other? Perhaps this _can_ be reduced to "Because it's local and hence I or some other local object I trust created it". However, I think once you start going down the path of secure transactions between objects, it's better to try and provide a generalised solution. There are various ways that authenticated and secured communications between objects can be implemented, and perhaps even made "translucent". -- ---------------------------------------------------------------------- ABO: finger abo@minkirri.apana.org.au for more info, including pgp key ----------------------------------------------------------------------
On Sat, 2002-02-16 at 06:39, Donovan Baarda wrote:
On Sat, Feb 16, 2002 at 05:42:48AM -0600, Glyph Lefkowitz wrote:
You're confusing "objects" and "people". I don't trust the remote objects because (A) the communication to them is potentially visible over the wire and (B) they don't actually exist on physical hardware I control, e.g. their behavior might have been tampered with. They are under the complete control of another person, who might as well be speaking the wire-protocol personally. Thinking from a security viewpoint, they *have* already been tampered with, and the question is "how do I reduce the impact of that on my service?".
What gave you the idea we were talking about secure distributed transactions? I'm just talking about making our current implementation of remote communications more secure (by default). -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
On Sat, 2002-02-16 at 03:42, Glyph Lefkowitz wrote:
Is there a way to fix this? My lack of education, that is. There's more code within the Twisted repository left more me to read, but IIRC there are actually very few PB interfaces defined there. (We have Words and Manhole and...?) More pertinant here would be other applications (from the "wilds") which employ Twisted technology. Are there any such 0775 codebases I should be aware of? The accusation has been made that no one actually *uses* Twisted for anything (not to name names, but he's Canadian and works in Japan), but assuming that's not the case, maybe we could beef up the "sites using Twisted" list a little.
Who would want to replace me with another evil robot?
"Do not trust the Shover-robot, he is malfunctioning," - Kevin -- The moon is waxing crescent, 14.8% illuminated, 3.7 days old.
On Sat, 2002-02-16 at 14:26, Kevin Turner wrote:
Ohhh, excellent idea!! Let's break a bunch of interfaces, and then when everybody who uses Twisted starts complaining to us, we can get them to tell us where they use it!
Who would want to replace me with another evil robot?
"Do not trust the Shover-robot, he is malfunctioning,"
I wonder if glyph has stairs in his house... -- Chris Armstrong << radix@twistedmatrix.com >> http://twistedmatrix.com/users/carmstro.twistd/
On Sat, 2002-02-16 at 13:26, Kevin Turner wrote:
Distributed twisted.web, but you're right, that's about it.
It is indeed unfortunate that much Twisted-using code is closed. Not my decision, I assure you ;). I believe Itamar is using PB, so you could ask to look at his code -- I know that Sean was planning on releasing a small demo at some point. However, this is a circular problem. Much of this past week for me has been spent working on a new website for tm.com and trying to come up with introductory documentation for PB. It needs to be documented, standardized, and released as 1.0 before we can really expect droves of people to come flocking to it. Does anyone else on this list have some code to reference? The other folks I know of using Twisted are mostly using the other components at this point.
Who would want to replace me with another evil robot?
"Do not trust the Shover-robot, he is malfunctioning,"
"Do not trust the Pusher-robot, he is doing a little too much of his own stuff." -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
Glyph Lefkowitz wrote:
Yep. However, for clarity, lets not forget twisted.spread the implementation is not the same as the PB the protocol. E.g. Perspectives are an implementation choice, not part of the protocol. Or how to call remote methods - this is an implementation choice, in Java the __getattr__ eay isn't even relevant.
Kevin Turner <acapnotic@twistedmatrix.com> writes:
http://mc-foo.sourceforge.net/ (note that parts of it are sick, but not twisted) -- tv@{{hq.yok.utu,havoc,gaeshido}.fi,{debian,wanderer}.org,stonesoft.com} double a,b=4,c;main(){for(;++a<2e6;c-=(b=-b)/a++);printf("%f\n",c);}
well, as someone who HAS written a lot of spread code I'd say that the current interface that assumes method calls on remote objects are remote can be confusing. In a few cases I have found my code doing strange things on the network when I was expecting local behavior. The "callRemote" syntax would make remote operation explicit - which I am in favor of. Another observation from writing lots of spread code is that the remote interfaces for objects tend to be very small. This reduces the impact of the "more typing required" argument against this change. In my current application that uses pb which is 4300 lines of python and about 70 classes, there are only 19 remote methods defined. Changing my code over to use the "callRemote" syntax would take a couple of hours at most. (i have unit tests :) ). BTW, I was the other person that Glyph mention who had written code exploitable by this issue. -----Original Message----- From: twisted-python-admin@twistedmatrix.com [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Kevin Turner Sent: Friday, February 15, 2002 10:05 PM To: Twisted List Subject: Re: [Twisted-Python] Potential PB Security Problem (And Solution) On Fri, 2002-02-15 at 19:12, Christopher Armstrong wrote:
For as long as I remember, that was *never* the point. Haven't you ever heard glyph shouting from on high, "Explicit is better than implicit!"?
Funny words for a sect which celebrates in masqurading objects and generally deplores strict type checking.
What do you mean, "wasn't meant to pass privledged information"? Your example has you appending the "secure data" to a list that's from who knows where! You have no idea who holds a reference to that list, or who will hold references to the list (or certain items on the list) in the near future. And, as a happy little object, I don't see why you should care. You perform your operation with the given parameters, and that's all your job is, isn't it? I guess there's the "Referenceable vs Copyable" facet which I haven't been paying a lot of attention to in this discussion. When I choose between those two spreadable flavours (the module name is spelled wrong, BTW), I am thinking about what goes over the wire, but only in terms of how much it will change, how much the states of the objects need to be synchronized, how much traffic it's generating, etc. But I've never seen the use of Copyable as a security option. If I had data I did not want getting outside, I do not think I should entrust it to *any* object which came from Outside, whether it be a RemoteCopy or RemoteCache or a list-like object with whiskers. I guess it's only fair to admit that I've written very little application code with twisted.spread and read even less, so it's entirely possible that I lack the experience to understand the issues here... but some things bein' said just don't ring true to me. "So-at-least-ONE-of-us-has-been-replaced-by-an-evil-robot"-ly yours, Kevin -- The moon is waxing crescent, 10.8% illuminated, 3.1 days old.
OK, My first thoughts - I think that this is a good idea. The problem with pretending to be a regular method call is that, well, it isn't. You may get disconnected, for example. So we want to encourage people to realize thjat they can't just use the same code. Of course, this'll make web.distrib rather more difficult to implement nicely... However, if we're unto security, some form of static typing on arguments passed from remote clients to Referenceables methods would be very useful, some would say necessary, for secure programming. This would aid documentation as well. Switching to callRemote still doesn't solve the "passing the wrong objects maliciously" issue, just one very specialized instance of it. And I ain't suggesting IDL - I'm sure we could come up with a solution embedded in python source code that is easy to type (say, using oscar-style docstrings or some ther form of annotation, and a enforceInterface(klass) function that parses them and uses t.p.hook).
On Sun, 2002-02-17 at 01:27, Itamar Shtull-Trauring wrote:
Nah. We already have to manually frob the interface -- peeling off methods of a remote reference and sticking them to a copy. It would probably look cleaner if we were more explicit about it, anyway :) I suppose each of those one-line assignments would turn into a two-line method declaration (and gasp! a newline to separate them) but it would also provide for more flexible emulation of the request interface. I think that the existing code there may be an example of the "poltergeists" antipattern.
Well, yes. We need a documentation standard for return types (the sound you hear there is another section of the coding standard being exploded in a concrete bunker, several miles away) and we might as well use that to enforce interfaces when we find them. However -- any sort of polymorphism is a potential security hole in that regard. I would like to see some exploit demonstrations of the sorts of holes that one could create by passing the wrong serialized data, provided that remote calls were more explicit. I don't think it's really serious, or could be resolved by stricter type-checking. (After all, any behavior besides "blow up" pretty much assumes that the client is going to be passing an object that adheres to a correct interface.) -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com
participants (9)
-
abo＠minkirri.apana.org.au -
Allen Short -
Christopher Armstrong -
Glyph Lefkowitz -
Itamar Shtull-Trauring -
Itamar Shtull-Trauring -
Kevin Turner -
Sean Riley -
Tommi Virtanen