What I am trying to nail down is exactly what needs doing to get us from where we are now to where capabilities actually work. As I understand it, what is needed is:

a) Fix restricted execution, which is in a state of disrepair

Yes.

b) Override import, open (and other stuff? what?)

Don't worry about this; it's taken care of by the rexec module; each application will probably want to do this a little differently (certainly Zope has its own way).

c) Wrap or replace some of the existing libraries, certify that others are "safe"

This should only be necessary for (core and 3rd party) extension modules. The rexec module has a framework for this.

It looks to me like a and b are shared with proxies, and c would be different, by definition. Is there anything else? Am I on the wrong track?

I don't know why you think (c) is different.

I am going to write this all up into a document which can be used as a starting point for work to complete this.

Excellent. --Guido van Rossum (home page: http://www.python.org/~guido/)

...

...

b) Override import, open (and other stuff? what?)

Don't worry about this; it's taken care of by the rexec module; each application will probably want to do this a little differently (certainly Zope has its own way).

I believe I heard way back that there was a lack of confidence rexec overrode everything that needed overriding - or am I getting mixed up with restricted execution?

Indeed.

...

...

c) Wrap or replace some of the existing libraries, certify that others are "safe"

This should only be necessary for (core and 3rd party) extension modules. The rexec module has a framework for this.

...

It looks to me like a and b are shared with proxies, and c would be different, by definition. Is there anything else? Am I on the wrong track?

I don't know why you think (c) is different.

Because with proxies you'd wrap with proxies, and with capabilities you'd wrap with capabilities. Or do you think there's a way that would work for both (which would, of course, be great)?

OK, fair enough. --Guido van Rossum (home page: http://www.python.org/~guido/)

Ka-Ping Yee wrote:

On Sat, 8 Mar 2003, Ben Laurie wrote:

...

...

...

c) Wrap or replace some of the existing libraries, certify that others are "safe"

This should only be necessary for (core and 3rd party) extension modules. The rexec module has a framework for this.

...

It looks to me like a and b are shared with proxies, and c would be different, by definition. Is there anything else? Am I on the wrong track?

I don't know why you think (c) is different.

Because with proxies you'd wrap with proxies, and with capabilities you'd wrap with capabilities. Or do you think there's a way that would work for both (which would, of course, be great)?

This doesn't make any sense to me. The standard libraries would provide proxy wrappers in either caes. The rexec vs. proxy issue doesn't enter into it.

We've got too much overloading here! I meant "proxy" as in "Zope proxy". Yes, in either case they'll be wrapped in some kind of (non-Zope) proxy, but the actual wrapper would be different.

By the way -- to avoid confusion between "proxies used to wrap unrestricted objects in order to make them into secure objects" and "proxies used to reduce the interface of an existing secure object", let's call the first "proxy" (as has been used in the "rexec vs. proxy" discussion so far), and call the second a "facet" (which is the term commonly used when capabilities people talk about reducing an interface). We often talk about providing, say, a "read-only facet" on an object.

This would be more applicable to an object-based capability model, which Jim and Guido seem to favour. In fact, perhaps it would be nicest to be able to do both - i.e. bound methods _and_ opaque objects. Then we'd all be happy. Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff

...

...

Guido van Rossum wrote The only truly secure object is None. :-)

You sure?

...

...

None.__class__.__class__.mro(type(None))[1]

Not sure what else it's possible to get to from None... Anthony -- Anthony Baxter It's never too late to have a happy childhood.

On Sat, 2003-03-08 at 07:27, Ben Laurie wrote:

Bound methods are not capabilities unless they are secured. It seems the correct way to do this is to use restricted execution, and perhaps some other tricks. What I am trying to nail down is exactly what needs doing to get us from where we are now to where capabilities actually work. As I understand it, what is needed is:

a) Fix restricted execution, which is in a state of disrepair

b) Override import, open (and other stuff? what?)

c) Wrap or replace some of the existing libraries, certify that others are "safe"

It looks to me like a and b are shared with proxies, and c would be different, by definition. Is there anything else? Am I on the wrong track?

I have been trying to argue, though I feel a bit muddled at times, that the proxy approach eliminates the need for rexec and makes it possible to build a "restricted environment" without relying on the rexec code in the interpreter. Any security scheme needs some kind of information hiding to guarantee that untrusted code does not break into the representation of an object, so that, for example, an object can be used as a capability. I think we've discussed two different ways to implement information hiding. The rexec approach is to add code to the interpreter to disable certain introspection features when running untrusted code. The proxy approach is to wrap protected objects in proxies before passing them to untrusted code. I think both techniques achieve the same end, but with different limitations. I prefer the proxy approach because it is more self contained. The rexec approach requires that all developers working in the core on introspection features be aware of security issues. The security kernel ends up being most of the core interpreter -- anything that can introspection on objects. The proxy approach is to create an object that specifically disables introspection by not exposing internals to the core. We need to do some more careful analysis to be sure that proxies really achieve the goal of information hiding. I think another benefit of proxies vs. rexec is that untrusted code can still use all of the standard introspection features when dealing with objects it creates itself. Code running in rexec can't use any introspective feature, period, because all those features are disabled. With the proxy approach, introspection is only disabled on protected objects.

I am going to write this all up into a document which can be used as a starting point for work to complete this.

It sounds like a PEP would be the right thing. It would be nice if the PEP could explain the rationale for a secure Python environment and then develop (at least) the capability approach to building that environment. Perhaps I could chip in with some explanation of the proxy approach. Jeremy

Jeremy Hylton wrote:

On Sat, 2003-03-08 at 07:27, Ben Laurie wrote:

...

Bound methods are not capabilities unless they are secured. It seems the correct way to do this is to use restricted execution, and perhaps some other tricks. What I am trying to nail down is exactly what needs doing to get us from where we are now to where capabilities actually work. As I understand it, what is needed is:

a) Fix restricted execution, which is in a state of disrepair

b) Override import, open (and other stuff? what?)

c) Wrap or replace some of the existing libraries, certify that others are "safe"

It looks to me like a and b are shared with proxies, and c would be different, by definition. Is there anything else? Am I on the wrong track?

I have been trying to argue, though I feel a bit muddled at times, that the proxy approach eliminates the need for rexec and makes it possible to build a "restricted environment" without relying on the rexec code in the interpreter.

Any security scheme needs some kind of information hiding to guarantee that untrusted code does not break into the representation of an object, so that, for example, an object can be used as a capability. I think we've discussed two different ways to implement information hiding.

The rexec approach is to add code to the interpreter to disable certain introspection features when running untrusted code.

The proxy approach is to wrap protected objects in proxies before passing them to untrusted code.

I think both techniques achieve the same end, but with different limitations. I prefer the proxy approach because it is more self contained. The rexec approach requires that all developers working in the core on introspection features be aware of security issues. The security kernel ends up being most of the core interpreter -- anything that can introspection on objects. The proxy approach is to create an object that specifically disables introspection by not exposing internals to the core. We need to do some more careful analysis to be sure that proxies really achieve the goal of information hiding.

I think another benefit of proxies vs. rexec is that untrusted code can still use all of the standard introspection features when dealing with objects it creates itself. Code running in rexec can't use any introspective feature, period, because all those features are disabled. With the proxy approach, introspection is only disabled on protected objects.

These are all good points. Proxies have a dark side though. They sometimes trip up standard facilities in Python that either depend on specific types or on identity comparisons. With a bit of effort, proxies can be made highly transparent, but they change an object's type and id. For example, you can't proxy exceptions without breaking exception handling. In Zope, we rely on restricted execution to prevent certian kinds of introspection on exceptions and exception classes. In Zope, we also don't proxy None, because None is usually checked for identity. We also don't proxy strings, and numbers. I think I agree that you could build a restricted environment with proxies alone, but, to do so, you would need to make Python far more proxy aware. I think that the language would need to be aware of proxies at a far deeper level. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

From: "Samuele Pedroni"

...

For example, you can't proxy exceptions without breaking exception handling. In Zope, we rely on restricted execution to

From: "Jim Fulton" prevent

...

certian kinds of introspection on exceptions and exception classes. In Zope, we also don't proxy None, because None is usually checked for identity. We also don't proxy strings, and numbers.

That was a question I was asking myself about proxies: exception handling. But I never had the time to play with it to check.

Does that mean that restricted code can get unproxied instances of classic classes as caught exceptions?

maybe the question was unclear, but it was serious, what I was asking is whether some restricted code can do: try: deliberate code to force exception except Exception,e: ... so that e is caught unproxied. Looking at zope/security/_proxy.c it seems this can be the case... then to be (likely) on the safe side, all exception class definitions for possible e classes: like e.g. class MyExc(Exception): ... ought to be executed _in restricted mode_, or be "trivial/empty": something like class MyExc(Exception): def __init__(self, msg): self.message = msg Exception.__init__(self, msg) def __str__(self): return self.message is already too much rope. Although it seems not to have the "nice" two-level-of-calls behavior of Bastion instances, an unproxied instance of MyExc if MyExc was defined outside of restricted execution, can be used to break out of restricted execution. regards.

How can it be used to break out of restricted execution?

I see three risks:

1. The exception provides methods to do harmful things, such as create side effects or provide access to data outside the exception.

2. The exception creates data that needs to be protected. For example Zope uses a NotFoundError exception taht contains an object being searched.

3. The exception methods meta data provide access to module globals.

Risk 1 needs to be mitigated through proper exception design. Exceptions need to be limited in what their methods do. This is a bit brittle, but all standard exceptions have this property.

Risk 2 is mitigated by proxying exception instance data. Proxies can do

From: "Jim Fulton" this.

This is what we've decided to do, although we haven't implemented it yet.

Risk 3 is mitigated by restricted execution.

Have I missed anything?

OK, I have had the time to really try what I was thinking about. I have not found a way to really break out from restricted execution (does not mean I'm sure there isn't) BUT: I'm considering: - Python 2.2.2 - Zope 3 3.0a1 and zope.security.interpreter.RestrictedInterpreter with zope.security.simplepolicies.ParanoidSecurityPolicy (the default) so 1. a bug (rexec had it too). If I remember correctly the solution is re-injecting __builtins__ before each exec C:\transit\Zope3-3.0a1\src\zope\security>\usr\python22\python Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.

...

...

import sys sys.path.append('..\..') from zope.security.interpreter import RestrictedInterpreter ri=RestrictedInterpreter() ri.ri_exec("class A: pass") ri.ri_exec("print A.__dict__") Traceback (most recent call last): File "<stdin>", line 1, in ? File "..\..\zope\security\interpreter.py", line 32, in ri_exec exec code in self.globals File "<string>", line 1, in ? RuntimeError: class.__dict__ not accessible in restricted mode ri.ri_exec("del __builtins__") ri.ri_exec("print A.__dict__") {'__module__': '__builtin__', '__doc__': None}

or be sure to call ri_exec only once on each RestrictedInterpreter instance. Assuming that fixed: 2. If code executed under a RestrictedInterpreter could obtain a MyExc instance and had a working unproxied/non-proxying 'property' built-in, it could very likely break out from restricted execution. Fortunately the 'property' passed to such code is not working. Given that that's not the case I skip the illustration. 3. How much this scenario is likely really depend on how RestrictedInterpreter is used, how and where exceptions are defined, if really restricted code can manage to get an instance of one of them ... if further restrictions e.g. on subclassing are added or removed ... if the general situation of restricted execution and new-style classes improve. All of this I don't know. Here I consider: a "dangerous" function ('sys.exit') is imported in the same module where MyExc is defined, MyExc is not defined under restricted execution, a proxied function is passed to restricted code such that it can capture an instance of MyExc (as I said whether this set of things is likely/unlikely I don't know): import sys from sys import exit # !!! same module as MyExc sys.path.append('C:/transit/Zope3-3.0a1/src') from zope.security.interpreter import RestrictedInterpreter from zope.security.checker import ProxyFactory class MyExc(Exception): # !!! definition outside of resticted execution def __init__(self,msg): self.message = msg Exception.__init__(self,msg) def __str__(self): return self.message def myfunc(): raise MyExc('foo') ri = RestrictedInterpreter() ri.globals['myfunc'] = ProxyFactory(myfunc) f = open('c:/Documenti/x.txt','r') code = f.read() f.close() ri.ri_exec(code) print "OK" Anyway I have a _very baroque_ x.txt that manages to call sys.exit. regards

[me]

attached is a modified version of s.py that takes a filename for the code to run inside the RestrictedInterpreter. Also myfunc is now myexc_source . There is also a new function candy, next mail on that.

Consider from s.py: -- * -- from sys import exit ... def candy(s): if s == "yes": return 'candy' else: return 'none' ri = RestrictedInterpreter() ri.globals['candy'] = ProxyFactory(candy) ... ri.ri_exec(code) print "OK" -- * -- No unproxied exceptions, on the other hand both rexec and the prototype RestrictedIntrepreter supply code with globals() [!], and apply() ... I have some _even more baroque_ code (xpl2) that exploits candy and manages to call sys.exit: ...>\usr\python22\python -i s.py xpl2 candy Got sys.exit ...> In this case xpl2 could be rewritten as a single expression of the form: candy(...) although that would make for a totally masochistic exercise and a total obfuscated python entry. No, I haven't done/ tried that :) regards.

Samuele Pedroni wrote:

From: "Jim Fulton"

...

For example, you can't proxy exceptions without breaking exception handling. In Zope, we rely on restricted execution to

prevent

...

certian kinds of introspection on exceptions and exception classes. In Zope,

we

...

also don't proxy None, because None is usually checked for identity. We also

don't

...

proxy strings, and numbers.

That was a question I was asking myself about proxies: exception handling. But I never had the time to play with it to check.

Does that mean that restricted code can get unproxied instances of classic classes as caught exceptions?

Right. What we can (and will do) is intercept the exceptions and proxy the exception's instance data. So we'll be relying on restricted execution to protect the exception method meta data and on proxies to protect the exception data. Of course, we'd prefer to be able to proxy the the exception instances themselves. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Jeremy Hylton wrote: ...

I think both techniques achieve the same end, but with different limitations. I prefer the proxy approach because it is more self contained. The rexec approach requires that all developers working in the core on introspection features be aware of security issues. The security kernel ends up being most of the core interpreter -- anything that can introspection on objects.

I think that there is an important corrolary. Changes to the security policy are very hard to make. For example, if we change our mind about what should be safe or not: we have many places to make the change, we have lot's of tests to redo. people have to reinstall or rebuild Python to get the change. With proxies, the update is provides as fairly small and self-contained library update. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Guido van Rossum wrote:

...

Maybe every Python object should have a flag which can be set to prevent introspection -- like the current restricted execution mechanism, but on a per-object basis. Then any object could be used as a capability.

I think the capability folks would object to calling it a capability though. :-)

No, objects are another way to do it, though it seems to me with somewhat less ease - because the most common use of capabilities is to restrict the type of access to objects other objects have, so you'd need to have multiple objects proxying to the "real" one if you do it at the object level. If we were going to go this route, I'd like the alternative of _also_ being able to set the flag on a bound method.

Two questions:

- Where to store the flag? It probably would cost 4 bytes per object.

You can swap space for time by storing it as an attribute, of course.

- Which attributes are considered introspective?

All of them, except methods. Of course, this is what my first approximation to capabilities did (that's what a "capclass" was). Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff

On Sun, 9 Mar 2003, Guido van Rossum wrote:

- Which attributes are considered introspective?

Here's a preliminary description of the boundary between "introspective" and "restricted", off the top of my head: 1. The only thing you can do with a bound method is to call it (bound methods have no attributes except __doc__). 2. The following instance attributes are off limits: __class__, __dict__, __module__. That might be a reasonable start. However, there is still the problem that the established technique for storing instance-specific state in Python is to use globally- accessible data attributes instead of a limited scope. We would also need to add a safe (private) place for instances to put state. -- ?!ng

On Mon, 10 Mar 2003, Jim Fulton wrote:

Ka-Ping Yee wrote:

...

Here's a preliminary description of the boundary between "introspective" and "restricted", off the top of my head:

1. The only thing you can do with a bound method is to call it (bound methods have no attributes except __doc__).

Well, I see no harm and much usefulness in allowing __name__, __repr__, and __str__.

Depends. In a truly secure system, classes would only reveal information about themselves if they wanted to. The default __repr__ gives away to the id() of the instance, and __name__ gives away the name of the method, which would prevent you from creating proxies that are indistinguishable from the original. Sometimes it is useful to be able to do that.

...

2. The following instance attributes are off limits: __class__, __dict__, __module__.

I generally want to be able to get the __class__. This is harmless in my case, because I get a proxy back.

We definitely do not want to provide access to __class__. Access to an instance should not give you the power to create more instances of its class. If you passed somebody a file object, access to the class would convey the power to open any file on the filesystem!

...

However, there is still the problem that the established technique for storing instance-specific state in Python is to use globally- accessible data attributes instead of a limited scope. We would also need to add a safe (private) place for instances to put state.

I'm don't understand why this is necessary. In general, you want to restrict what attributes (data, properties, methods, etc.) are accessible in certain situations. I don't follow what makes data attributes special.

Instances currently don't have a private place to put their state, and unless there is a convenient way do that, implementers will tend to expose their instance state in public data attributes. Even if the instance had properties, the properties still (as yet) have no way to conveniently distinguish if access is being attempted from within an instance method, or from outside the instance. -- ?!ng

Samuele Pedroni wrote:

From: "Ka-Ping Yee"

...

However, there is still the problem that the established technique for storing instance-specific state in Python is to use globally- accessible data attributes instead of a limited scope. We would also need to add a safe (private) place for instances to put state.

Indeed, that's the fact that implementations of methods are normal functions that access the instance attributes like everything else do, that's why Zope-proxies become necessary (and a bit brittle):

class A: def geta(self): return self.a # 1

a=A()

a.a # 2

(1) and (2) are using the same operation/execution path.

This points out a nice feature of zope proxies. The proxied object's methods are called with an unproxied self, so you can easily allow access to the object's methods without providing access to other attributes. Or, equivalently, you can provide access to one set of methods and those methods can use other methods that you don't provide access to. Could you explain why you say that zope proxies are brittle? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Samuele Pedroni wrote:

From: "Jim Fulton"

...

Could you explain why you say that zope proxies are brittle?

from my small experience playing with RestrictedIntepreter:

Um, er, I've been meaning to mention that RestrictedIntepreter is not does and isn't used anywhere yet. It's a bit of a decoy at this time. :] At this point, RestrictedIntepreter is really an incomplete prototype. OTOH, RestrictedBuiltins is used for Python expressions in Zope page templates. Simple Python expressions in zpt don't tend to run into the sorts of problems you've found.

you wrap into proxies a lot of builtins:

*) 'object' for example, then class C(object): ... does not work

Right. We will fix this. It should be possible to subclass proxied classes. The resulting classes should then be proxies. object and type should probably be special cases.

but given that some basic types are left alone, one can use Type = ''.__class__.__class__

class C: __metaclass__ = Type

*) iter seems not to work (deliberate decision or bug?)

bug I imagine.

*) proxied 'property' is unusable

ditto

*) built-in functions return proxies even if the argument were unproxied:

_12 = map(None,[1,2])

Interesting case. It looks like map shouldn't be proxied.

class A: pass a = A() a.a = [1,2]

_12 = getattr(a,'a')

Ditto.

in both cases with the proxied version of map and getattr the result _12 would be a proxied list.

deliberate safer-side decisions?

no.

I can see it both ways: - see other mail

I don't know what other mail you are refering to. Maybe it doesn't matter.

- map(None,[obj])[0] becomes a way to get a a proxied version of obj that can be passed to code that would maybe unwrap it and believe that is some other legit object.

Any code that unwraps proxies should be viewed with great suspicion. I currently consider any use of that API without an extensive accompanying comment to be a virtual "XXX" comment. I'm sorry to have had you spend so much time on what is a bit od a decoy. OTOH, you've pointed out a number of points that we do need to address to move our RestrictedInterpreter beyond the prototype stage. You've found a number of problems and issues in deciding how to proxy builtins. I would argue that these are not problems in the proxies themselves but in the applications to builtins. But perhaps I'm wrong. Another area that we haven't dealt with yet is how proxies will work in untrusted *persistent* modules. But you probably don't want to know about that. ;) Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

From: "Jim Fulton"

Samuele Pedroni wrote:

...

From: "Jim Fulton"

...

Could you explain why you say that zope proxies are brittle?

from my small experience playing with RestrictedIntepreter:

Um, er, I've been meaning to mention that RestrictedIntepreter is not does and isn't used anywhere yet. It's a bit of a decoy at this time. :]

I knew it isn't used. I'm not that naive, it seemed nevertheless a show-case/playground of proxy+restricted execution approach. regards.

Greg Ewing wrote:

...

E.g. I might have a service configuration registry object. The object behaves roughly like a dictionary. A certain user may be given read-only access to the registry.

Maybe every Python object should have a flag which can be set to prevent introspection -- like the current restricted execution mechanism, but on a per-object basis. Then any object could be used as a capability.

Yes, but not a very useful one. For example, given a file, you often want to create a "file read" capability which is an object that allows reading the file but not writing the file. Just preventing introspection isn't enough. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

On Sat, 29 Mar 2003, Ka-Ping Yee wrote:

Okay, at last to the example, then.

The following is a better formulation in the capability style -- please ignore the previous one. The previously posted code allows names to carry authority, which is a big no-no. This code gets rid of names altogether in the API for file access; it's better to deal with just objects. import os, __builtin__ class Namespace: def __init__(self, *args, **kw): for value in args: self.__dict__[value.__name__] = value for name, value in kw.items(): self.__dict__[name] = value class ImmutableNamespace(Namespace): def __setattr__(self, name, value): raise TypeError('read-only namespace') def ReadStream(file, name): def __repr__(): return '' % name return ImmutableNamespace(__repr__, file.read, file.close, name=name) def FileReader(path, name): def __repr__(): return '' % name def open(): return ReadStream(__builtin__.open(path, 'r'), name) def getsize(): return os.path.getsize(path) def getmtime(): return os.path.getmtime(path) return ImmutableNamespace(__repr__, open, getsize, getmtime, name=name) def DirectoryReader(path, name): def __repr__(): return '' % name def getfiles(): files = [] for name in os.listdir(path): fullpath = os.path.join(path, name) if os.path.isfile(fullpath): files.append(FileReader(fullpath, name)) return files def getdirs(): dirs = [] for name in os.listdir(path): fullpath = os.path.join(path, name) if os.path.isdir(fullpath): dirs.append(DirectoryReader(fullpath, name)) return dirs return ImmutableNamespace(__repr__, getfiles, getdirs, name=name) -- ?!ng

On Sun, 30 Mar 2003, Paul Prescod wrote:

It wouldn't have hurt for you to describe how the code achieves security by using lexical closure namespaces instead of dictionary-backed namespaces. ;)

Sorry. :) I assumed it would be clear.

I don't understand one thing.

The immutability imposed by the "ImmutableNamespace" trick is easy to turn off. But once I turn it off, I couldn't figure out any way to violate the security because the closure's variables are invisible to any code that is not defined within its block. Why bother with the ImmutableNamespace bit at all?

That immutability isn't required in order to prevent filesystem access. That immutability is only there to prevent multiple clients of the same DirectoryReader to use the DirectoryReader as a communication channel.

del x.__class__.__setattr__

Sneaky. :) In restricted mode you wouldn't be able to do that.

I can't see in this model how to implement what C++ calls a "friend" class.

I haven't tried an example that requires that yet, but two classes could communicate through access to a shared object if they wanted to.

If this technique became widespread, Python's restrictions on assigning to lexically inherited variables would probably become annoying.

The Namespace offers a possible workaround. I didn't end up using it in my second code example because none of the objects have mutable state, but here's how you could do it: def Counter(): self = Namespace() self.i = 0 def next(): self.i += 1 return self.i return ImmutableNamespace(next) It would be cool if you could suggest little "security challenges" to work through. Given specific scenarios requiring things like mutability or friend classes, i think trying to implement them in this style could be very instructive. -- ?!ng

On Sun, 30 Mar 2003, Paul Prescod wrote:

I'm not clear (because I've been following the thread with half my brain, over quite a few days) whether you are making or have made some specific proposal. I guess you are proposing a restricted mode that would make this example actually secure as opposed to almost secure. Are you also proposing any changes to the syntax?

Not yet. Although it's certainly tempting to propose syntax changes, it makes more sense to really understand what we want first. We can't know that until we've actually tried programming in the capability style in Python. That's why i want to explore the possibilities and try these exercises -- it will help us discover the shortest path from here to there.

Also, is restricted mode an interpreter mode or is it scoped by module?

Whether restricted mode is activated depends on the __builtins__ of the current namespace. So the short answer is "by module".

Yes, but why workaround rather than fix? Is there a deep reason Python objects can't write to intermediate namespaces?

No. There's just no syntax for it yet. But let's figure out what we can get away with first.

...

It would be cool if you could suggest little "security challenges" to work through. Given specific scenarios requiring things like mutability or friend classes, i think trying to implement them in this style could be very instructive.

Unfortunately, most of the examples I can come up with seem to be hacks, workarounds and optimizations. It isn't surprising that sometimes you lose some efficiency or simplicity when working in a secure system.

Hmm, i'm not sure you understood what i meant. The code example i posted is a solution to the design challenge: "provide read-only access to a directory and its subdirectories, but no access to the rest of the filesystem". I'm looking for other security design challenges to tackle in Python. Once enough of them have been tried, we'll have a better understanding of what Python would need to do to make secure programming easier. -- ?!ng

Paul Prescod wrote:

Are DOS issues in scope? How do we prevent untrusted code from just bringing the interpreter to a halt? A smart enough attacker could even block all threads in the current process by finding a task that is usually not time-sliced and making it go on for a very long time. without looking at the Python implementation, I can't remember an example off of the top of my head, but perhaps a large multiplication or search-and-replace in a string.

It seems to me that this is an issue orthogonal to capabilities (though access to mechanisms that regulate it might well be capability-based). Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff

Ka-Ping Yee wrote:

Hmm, i'm not sure you understood what i meant. The code example i posted is a solution to the design challenge: "provide read-only access to a directory and its subdirectories, but no access to the rest of the filesystem". I'm looking for other security design challenges to tackle in Python. Once enough of them have been tried, we'll have a better understanding of what Python would need to do to make secure programming easier.

Well, one of the favourites is to create a file selection dialog that will only give access (optionally readonly) to the file designated by the user. This may be rather more than you want to bite off as a working system at this stage, though! It might be a useful thought experiment, though. Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff

Ka-Ping Yee wrote:

Ben Laurie wrote:

...

BTW, if you would like to explain why you don't think bound methods are the way to go on python-dev, I'd love to hear it.

Guido van Rossum wrote:

...

Using capabilities, I would have to hand her a bunch of capabilities for various methods: __getitem__, has_key, get, keys, items, values, and many more. Using proxies I can simply give her a read-only proxy for the object. So proxies are more powerful.

I'm pretty sure that Guido meant to say "bound method" rather than "capability" in the text above. I think that the debate is partly whether to express capabilities (or some other scheme) in terms of bound methods or proxies, which expose entire interfaces.

There seems to be a persistent confusion here that i would like to dispel: a capability is not a single lambda.

There are a bunch of confusions floating around. :) A major one is a concise definition os what a capability and why the capability approach is good or bad. In reading about capabilities in E, http://www.erights.org/. I really need to read all that stuff again. Of course, as others pointed out, I ended up creating something for Zope 3 that isn't capabilities. I think you touch on a reason below.

Guido's paragraph, above, seems to believe that it is. In fact, the pattern he described is a common and powerful way of using capabilities. A capability is just an unforgeable object reference. In a pure capability system, the only thing you can do with a capability is to call methods on it (or, if you prefer, all you can do is send messages to it). Interposing an object to expose only a subset of another object's API, such as a read-only subset, is exactly the power capabilities give you.

It seems to me that the "rexec vs. proxy" debate is really about a very different question: How do we get from Python's currently promiscuous objects to properly restricted objects?

(Once we have properly restricted objects in either fashion, yes, definitely, using proxies to restrict access is a great technique.)

If i understand correctly, the "proxy" answer is "we create a special wrapper object, then the programmer has to individually wrap any object they want to be secure". And the "rexec" answer is "we create an interpreter mode in which all objects are secure".

I think the latter is far better. To have any sort of real chance at establishing security, you have to start from a place where everything is secure, instead of starting from a place where everything is insecure and you have to individually secure every single object with its own wrapper.

The eventual ideal is to have a system where all objects are "pure" objects (i.e. non-introspectable capabilities) by default. Anyone wanting to do introspection would simply have to obtain the "introspect" capability from a privileged place (e.g. sys). For example,

class Foo: pass

print Foo.__dict__ # fails

from sys import introspect print introspect(Foo).__dict__ # succeeds

When running the interpreter in secure mode, "introspect" would just be missing from the sys module (again, ideally sys.introspect wouldn't exist by default, and a command-line option would turn it on, but i realize that's far away).

This would have the effect of the "introspectable flag" that Guido mentioned, but without expending any storage at all, until you actually needed to introspect something.

You seem to be arguing that programmers should not have to explictly create capabilities, but that everythink should be a capability by default. Please correct me if I'm wrong. I thought that the main point of capabilities was that programmers *should* explictly bother to pass capabilities. Programmers should think about arguments passed to (or returned or raised to) other code as capabilities to do things and pass *just* the capabilities needed. I find a lot of appeal in this idea. Zope employs proxies in a way that falls somewhere between the extremes of capabilities and implicitly protecting everything. (I'm going to be a little sloppy hear for brevity. A Zope proxy is made up of two objects, a simple proxy that *could* be used to implement capabilities and a checker that provides policy. The policy we currently use in Zope is not a capability policy.) Zope security proxies assure that "everything" is proxied. (We choose not to proxy simple valies like numbers, strings, and None.) Values returned from operations on proxied. This maked it pretty straightforward to set up execution environments where untrusted code only has access to proxies. In addition, if untrusted code calls trusted code, the untrusted code can only pass proxies. This means that trusted code can't be tricked into performing operations that the untrusted code could not perform. Zope proxies achiev this level of automation by providing registries, mostly based on classes, that allow programmers to say how different kinds of objects should be proxied. Programmers decide what capabilities to expose at "compile" time (really program startup) rather than run time. Programmers *can* create proxies explicitly that provides non-default access. In fact, there are apis that actually provide the the equivalent of capabilities. I mention all of this because I think it's worth thinking/debating this issue about how explicit security should be. On the one hand, explictly giving *just* the capabilities needed for a task seems very appealing. OTOH, making sure that everything is protected by default is safer. I suspect that there are ways to combine (trade off?) these in reasonable ways. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Ben Laurie wrote:

...

BTW, if you would like to explain why you don't think bound methods are the way to go on python-dev, I'd love to hear it.

Guido van Rossum wrote:

...

Using capabilities, I would have to hand her a bunch of capabilities for various methods: __getitem__, has_key, get, keys, items, values, and many more. Using proxies I can simply give her a read-only proxy for the object. So proxies are more powerful.

(Jim surmised that I meant to write "bound methods". Alas, I don't get off that easily: at the time I wrote that I really did think that a capability had to be a single function.) [Ping]

There seems to be a persistent confusion here that i would like to dispel: a capability is not a single lambda.

I guess, I misunderstood.. I was sure that Ben told me this was so. Apparently I misread, or you have a different definition of capability than he does (wouldn't be the first time.)

Guido's paragraph, above, seems to believe that it is. In fact, the pattern he described is a common and powerful way of using capabilities. A capability is just an unforgeable object reference. In a pure capability system, the only thing you can do with a capability is to call methods on it (or, if you prefer, all you can do is send messages to it). Interposing an object to expose only a subset of another object's API, such as a read-only subset, is exactly the power capabilities give you.

So a proxy with a fixed (not depending on the caller) policy about which methods you can should be considered as equivalent to a capability -- in fact this would be a way to implement capabilities.

It seems to me that the "rexec vs. proxy" debate is really about a very different question: How do we get from Python's currently promiscuous objects to properly restricted objects?

(Once we have properly restricted objects in either fashion, yes, definitely, using proxies to restrict access is a great technique.)

If i understand correctly, the "proxy" answer is "we create a special wrapper object, then the programmer has to individually wrap any object they want to be secure". And the "rexec" answer is "we create an interpreter mode in which all objects are secure".

Well, actually, restricted execution as currently implemented does *not* strive to make all objects secure: untrusted code can still inspect all attributes of an object unless that object is proxied by a Bastion, or unless that object is one of a few built-in types (e.g. bound methods) for which some attributes are privatized.

I think the latter is far better. To have any sort of real chance at establishing security, you have to start from a place where everything is secure, instead of starting from a place where everything is insecure and you have to individually secure every single object with its own wrapper.

But we don't have the latter.

The eventual ideal is to have a system where all objects are "pure" objects (i.e. non-introspectable capabilities) by default.

That wouldn't be Python.

Anyone wanting to do introspection would simply have to obtain the "introspect" capability from a privileged place (e.g. sys). For example,

class Foo: pass

print Foo.__dict__ # fails

from sys import introspect print introspect(Foo).__dict__ # succeeds

When running the interpreter in secure mode, "introspect" would just be missing from the sys module (again, ideally sys.introspect wouldn't exist by default, and a command-line option would turn it on, but i realize that's far away).

This would have the effect of the "introspectable flag" that Guido mentioned, but without expending any storage at all, until you actually needed to introspect something.

That flag wasn't my idea, it was some one else's (Greg Ewing?). --Guido van Rossum (home page: http://www.python.org/~guido/)

[Greg Ewing]

I think I agree that to really get on top of this security business we need to move towards having dangerous things forbidden by default rather than allowed by default.

This is more or less what the rexec module implements, except for convenience it has a list of unsafe built-ins rather than a list of safe built-in.

To that end, it would be useful if we could pin down exactly what's dangerous and what isn't. It seems to me that most uses of introspection by most programs are harmless. Can we sort out those (hopefully few) things that are dangerous, and separate them from the existing introspection mechanisms?

Maybe, maybe not. The original restricted execution code (not the rexec module) arbitrarily decided that setting class attributes was dangerous but getting them was not. Samuele found that new-style classes allow both, but always disallows write-access to the class __dict__ (you have to use the setattr protocol); this is good or bad depending on how it's used. The real problem is that harmful access may be granted via innocent-looking access. For example, allowing read-only access to a function's globals gives you access to the unrestricted 'open' function...

Access to sys.modules has been mentioned as a key thing that needs to be restricted. Maybe this shouldn't be an arbitrarily-accessible variable? Maybe the sys module shouldn't be a module at all, but some special object that won't let you do nasty things with its contents unless you've got special privileges (which most code would *not* have by default).

That's pretty much what the rexec module implements; it overrides __import__ and when you ask for sys, you get a fake sys that only contains stuff that should be safe.

One of the "nasty" things would be picking the real __builtins__ out of sys.modules. Are there any others?

Picking an unsafe extension module out of sys.modules. --Guido van Rossum (home page: http://www.python.org/~guido/)

From: "Samuele Pedroni"

For sure a good thing would be for func_* and im_* attributes of functions and methods to be substituted by special accessor functions/objects, indipendently of restricted mode.

to clarify: I mean something like func_globals(f) vs f.func_globals regards

On Mon, 2003-03-10 at 05:51, Ka-Ping Yee wrote:

It seems to me that the "rexec vs. proxy" debate is really about a very different question: How do we get from Python's currently promiscuous objects to properly restricted objects?

I think that's the right question.

(Once we have properly restricted objects in either fashion, yes, definitely, using proxies to restrict access is a great technique.)

If i understand correctly, the "proxy" answer is "we create a special wrapper object, then the programmer has to individually wrap any object they want to be secure". And the "rexec" answer is "we create an interpreter mode in which all objects are secure".

The proxy answer is a bit more complex. Any object returned from a proxy is itself wrapped in a proxy, except for immutable objects like None, ints, and strings. The initial proxy creates a barrier between the code that created the proxy and the client that uses the proxy.

I think the latter is far better. To have any sort of real chance at establishing security, you have to start from a place where everything is secure, instead of starting from a place where everything is insecure and you have to individually secure every single object with its own wrapper.

It would indeed be impractical to wrap every object manually. I think both approaches tend towards the design principle of fail-safe defaults and complete mediation. A proxy mediates all access to the object it wraps. By default, it allows no access. When it allows access, it creates new proxies that provide the same facilities as the original. The one exception is for immutable objects. (Immutability is good for so many reasons.)

The eventual ideal is to have a system where all objects are "pure" objects (i.e. non-introspectable capabilities) by default. Anyone wanting to do introspection would simply have to obtain the "introspect" capability from a privileged place (e.g. sys). For example,

class Foo: pass

print Foo.__dict__ # fails

from sys import introspect print introspect(Foo).__dict__ # succeeds

When running the interpreter in secure mode, "introspect" would just be missing from the sys module (again, ideally sys.introspect wouldn't exist by default, and a command-line option would turn it on, but i realize that's far away).

This would have the effect of the "introspectable flag" that Guido mentioned, but without expending any storage at all, until you actually needed to introspect something.

If Python's introspection were less ad hoc, I suppose this issue would be easier to tackle. (Has anyone done security design for a CLOS-style meta-object protocol?) Note that the biggest problem with the introspectable flag is that it would need to be checked all over the interpreter internals. For example, the interpreter optimisizes bound method calls by extracting the im_self and im_func and calling im_func directly passing im_self and the rest of the arguments. This is all done within the mainloop using a single type check and a bunch of macros to extract fields from the bound method. It is pretty common to use macros that depend on the representation of builtin types like functions, methods, dictionaries, etc. Jeremy

On Tue, 11 Mar 2003, Greg Ewing wrote:

Perhaps it would be useful to distinguish between what might be called "read-only" introspection, and more powerful forms of introspection.

Usually it doesn't do any harm to be able to find out things like what class an object belongs to and what methods it supports, so perhaps these kinds of introspections don't need to be restricted by default.

A serious flaw with this particular point is that Python does not separate the identity of a class from the power to create instances of that class. Having access to a particular instance should certainly not allow one to ask it for its class, and then instantiate the class with arbitrary constructor arguments. -- ?!ng

On Sun, 30 Mar 2003, Guido van Rossum wrote:

[Ping]

...

Having access to a particular instance should certainly not allow one to ask it for its class, and then instantiate the class with arbitrary constructor arguments.

Assuming the Python code in the class itself is not empowered in any special way, I don't see why not. So that suggests that you assume classes can be empowered. I can see this for classes implemented in C; but how can classes implemented in pure Python be empowered?

In many classes, __init__ exercises authority. An obvious C type with the same problem is the "file" type (being able to ask a file object for its type gets you the ability to open any file on the filesystem). But many Python classes are in the same position -- they acquire authority upon initialization. To pick one at random, consider zipfile.ZipFile. At first glance it appears that once you create a ZipFile object with mode "r" you can hand it off to provide read-only access to a zip archive. (Even if a security audit of the code reveals holes, my point is that the API isn't far from accommodating such a design intent.) It's useful to be able to separate the authority to read one particular instance of ZipFile from the authority to instantiate new ZipFiles, which currently allows you to open any zip file on the filesystem for reading or writing. -- ?!ng

...

[Ping]

...

Having access to a particular instance should certainly not allow one to ask it for its class, and then instantiate the class with arbitrary constructor arguments.

[Guido]

...

Assuming the Python code in the class itself is not empowered in any special way, I don't see why not. So that suggests that you assume classes can be empowered. I can see this for classes implemented in C; but how can classes implemented in pure Python be empowered?

[Ping]

In many classes, __init__ exercises authority. An obvious C type with the same problem is the "file" type (being able to ask a file object for its type gets you the ability to open any file on the filesystem). But many Python classes are in the same position -- they acquire authority upon initialization.

What do you mean exactly by "exercise authority"? Again, I understand this for C code, but it would seem that all authority ultimately comes from C code, so I don't understand what authority __init__() can exercise.

To pick one at random, consider zipfile.ZipFile. At first glance it appears that once you create a ZipFile object with mode "r" you can hand it off to provide read-only access to a zip archive. (Even if a security audit of the code reveals holes, my point is that the API isn't far from accommodating such a design intent.)

But is it really ZipFile.__init__ that exercises the authority? Isn't its authority derived from that of the open() function that it calls?

It's useful to be able to separate the authority to read one particular instance of ZipFile from the authority to instantiate new ZipFiles, which currently allows you to open any zip file on the filesystem for reading or writing.

In what sense is the ZipFile class an entity by itself, rather than just a pile of Python statements that derive any and all authority from its caller? I understand how class ZipFile could exercise authority in a rexec-based world, if the zipfile module was trusted code. But I thought that a capability view of the world doesn't distinguish between trusted and untrusted code. I guess I need to understand better what kind of "barriers" the capability way of life *does* use. --Guido van Rossum (home page: http://www.python.org/~guido/)

Ka-Ping Yee :

On Sun, 30 Mar 2003, Guido van Rossum wrote:

...

[Ping]

...

Having access to a particular instance should certainly not allow one to ask it for its class, and then instantiate the class with arbitrary constructor arguments.

Assuming the Python code in the class itself is not empowered in any special way, I don't see why not. So that suggests that you assume classes can be empowered. I can see this for classes implemented in C; but how can classes implemented in pure Python be empowered?

In many classes, __init__ exercises authority. An obvious C type with the same problem is the "file" type

Yes, I think the solution to this is not to forbid getting hold of the class of an object, but to design constructors so that they don't do anything that might be a security problem. In the case of files, that would mean removing the feature that file("foo") means the same as open("foo"), so that only the open() function can open arbitrary files. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+