I've received some enthusiastic emails from someone who wants to revive restricted mode. He started out with a bunch of patches to the CPython runtime using ctypes, which he attached to an App Engine bug: http://code.google.com/p/googleappengine/issues/detail?id=671 Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.) I don't recall the exploits that Samuele once posted that caused the death of rexec.py -- does anyone recall, or have a pointer to the threads? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Sun, Feb 22, 2009 at 10:45 AM, Guido van Rossum <guido@python.org> wrote:
I've received some enthusiastic emails from someone who wants to revive restricted mode. He started out with a bunch of patches to the CPython runtime using ctypes, which he attached to an App Engine bug:
http://code.google.com/p/googleappengine/issues/detail?id=671
Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.)
We have to remember that it's quite trivial to segfault the interpreter with pure Python. (See Lib/test/crashers) Even if this patch manages to plug all the holes in the current Python, do we really want to commit our selves to maintaining it through language evolution which will surely introduce new subtle ways to circumvent the guard? IMO, the only fairly close to fool proof method of running restricted python is through something like the PyPy sandbox where all os level calls have to be checked by the master process.
I don't recall the exploits that Samuele once posted that caused the death of rexec.py -- does anyone recall, or have a pointer to the threads?
It was broken by the introduction of new-style classes: http://mail.python.org/pipermail/python-dev/2002-December/031160.html -- Regards, Benjamin
On Sun, Feb 22, 2009 at 9:54 AM, Benjamin Peterson <benjamin@python.org> wrote:
On Sun, Feb 22, 2009 at 10:45 AM, Guido van Rossum <guido@python.org> wrote:
I've received some enthusiastic emails from someone who wants to revive restricted mode. He started out with a bunch of patches to the CPython runtime using ctypes, which he attached to an App Engine bug:
http://code.google.com/p/googleappengine/issues/detail?id=671
Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.)
We have to remember that it's quite trivial to segfault the interpreter with pure Python. (See Lib/test/crashers)
I know it well, but for this particular use case that doesn't matter. Tav is interested in using this on app engine, which doesn't care about segfaults -- the process is simply restarted, nobody gains access to information they shouldn't have. App engine does care about overwriting memory, but none of the crashers get that far in the configuration on app engine.
Even if this patch manages to plug all the holes in the current Python, do we really want to commit our selves to maintaining it through language evolution which will surely introduce new subtle ways to circumvent the guard?
You'd have to talk to Tav about this (I've CC'ed him on this message). He seems quite convinced that his current approach is secure, all he wants is a way to do the equivalent in app engine (where ctypes is not supported and never will be).
IMO, the only fairly close to fool proof method of running restricted python is through something like the PyPy sandbox where all os level calls have to be checked by the master process.
Trust me, app engine has something similar -- Tav's desire is just to provide an added barrier between his app and code that his app will let users run within a context it provides. He's relying (forced to rely :-) on app engine's sandbox for protection against illegal syscalls.
I don't recall the exploits that Samuele once posted that caused the death of rexec.py -- does anyone recall, or have a pointer to the threads?
It was broken by the introduction of new-style classes:
http://mail.python.org/pipermail/python-dev/2002-December/031160.html
I think there were other threads where Samule showed some quite devious ways to access the os module. I don't see any posts by Samuele in that thread (and he just replied that he's too busy). The attack demonstrated at the top of that thread isn't particularly interesting -- it just shows that class-based security doesn't work so well, but that doesn't mean some other approach based more on functions couldn't work. For Tav's benefit, I think it would be good to at least add "IsRestricted" checks to __subclasses__(), go_code and gi_frame -- that's a trivial patch and if he believes it's enough he can create a sandbox on app engine and invite people to try to break out of it... If someone succeeds, the damage is limited by app engine's own perimeter defenses (which haven't been broken through since it was released 9 months ago). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Tav is interested in using this on app engine, which doesn't care about segfaults -- the process is simply restarted, nobody gains access to information they shouldn't have. App engine does care about overwriting memory,
That doesn't make sense -- how can something not care about segfaults, but care about memory overwriting? If something is capable of causing a segfault, you can't be sure it won't just corrupt memory in some way that doesn't segfault but causes some other problem. -- Greg
On Sun, Feb 22, 2009 at 12:10 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Tav is interested in using this on app engine, which doesn't care about segfaults -- the process is simply restarted, nobody gains access to information they shouldn't have. App engine does care about overwriting memory,
That doesn't make sense -- how can something not care about segfaults, but care about memory overwriting? If something is capable of causing a segfault, you can't be sure it won't just corrupt memory in some way that doesn't segfault but causes some other problem.
To be more precise, we don't care about crashes caused by NULL pointer dereferencing. Most of the demonstrated crashers work by causing a NULL pointer dereference. Since that crashes immediately, there is no possibility for a further exploit. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Hey guys, benjamin> Even if this patch manages to plug all the holes in the benjamin> current Python, do we really want to commit our benjamin> selves to maintaining it through language evolution benjamin> which will surely introduce new subtle ways to benjamin> circumvent the guard? If it would be helpful, I am happy to maintain this as Python evolves. I've already been maintaining the PJE-inspired ctypes-based approach and monkeypatches for various Python versions for a while now. See secure.py, secure25.py, secure26.py and secure30.py in: http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/... Also, my plans for world domination depend on a secure Python, so I have the necessary incentives ;p sameule> I don't have much time these days, for sure not samuele> until pycon us, to look at the proposed code. Thanks in advance if/when you get the time for this Samuele! samuele> E provides and incorporate a lot of thinking samuele> around [snip] The functions based approach I am taking is very much taken from E and inspired by an insight that Ka-Ping Yee had on Python-Dev years ago. See http://www.erights.org/elib/capability/ode/index.html for a direct parallel to the approach I've taken... guido> For Tav's benefit, I think it would be good to at guido> least add "IsRestricted" checks to guido> __subclasses__(), gi_code and gi_frame -- guido> that's a trivial patch and if he believes it's guido> enough he can create a sandbox on app engine guido> and invite people to try to break out of it... If guido> someone succeeds.... If someone succeeds... ...My missus might end up leaving me on account of so much crying ;p Seriously though, it's a relatively risk-free approach. The only person who stands to lose out is me if I'm wrong =) In the worst case scenario, this approach would help identify other "leak" attributes/methods -- which I'm hoping won't be found. And, in an ideal scenario, we'd have the basis for secure Python interpreter/programming... which, together with PyPy's sandboxed interpreter, would seriously rock! -- enthusiastically, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
I've already been maintaining the PJE-inspired ctypes-based approach and monkeypatches for various Python versions for a while now. See secure.py, secure25.py, secure26.py and secure30.py in:
http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/...
What is the objective of this code? Is it a complete sandbox? If not, is a complete sandbox based on it available somehow for review? Regards, Martin
On Feb 22, 2009, at 5:15 PM, Martin v. Löwis wrote:
What is the objective of this code? Is it a complete sandbox? If not, is a complete sandbox based on it available somehow for review?
From a cursory look at Tav's CPython patch, I'd describe it as follows: Requires: an existing Python code execution environment which does not expose unsafe interpreter functionality to the (untrusted) code it runs, Provides: a way for the trusted code outside of that restricted environment to wrap functions (which may reference trusted objects) in closures which can be passed into the restricted environment as completely opaque objects whose internal state (including any referenced trusted objects) cannot be accessed from the untrusted code. When I say 'requires', I mean 'requires to be useful'; the patch can be applied to vanilla CPython, but isn't useful on its own. Building blocks for a restricted execution environment as required by the patch are commonly provided in templating libraries; Tav mentions Genshi in particular. By default, Genshi templates don't come with security built in -- you can do what you please in a template. However, since Genshi manually constructs a Python AST from Python code in the template, one can restrict the AST and modify the builtins exposed to the template environment, making things like filesystem access, import and other sensitive system facilities unavailable. Even with those unavailable, untrusted code can break out of containment by accessing object.__subclasses__ or gaining access to gi_frame and gi_code. This means you can't, for instance, pass into the untrusted code a closure which operates on trusted objects. Tav's patch addresses this particular problem. To give you a complete sandbox to play with, I just violently ripped out the relevant code from Genshi, added some glue, and slapped it all together in a single file along with Tav's pure-Python code which is functionally equivalent to the CPython patch he submitted. The result, tested on 2.5.1 and nothing else: <http://radian.org/~krstic/sandbox.py> As is, sandbox allows you to execute untrusted Python code which won't have access to import, __import__, eval, exec, execfile, open, and to which you can pass closures which reference trusted objects which the untrusted code can't get at. Example:
import sandbox import sys import md5 def trusted_pwd_closure(password): ... def get_pwd(): ... return md5.md5(password).hexdigest() ... return get_pwd ... context = dict(pwd=trusted_pwd_closure('secret')) sandbox.run_string("print pwd()", context) sandbox.run_string("print pwd.func_closure[0].cell_contents", context) [snip] AttributeError: 'function' object has no attribute 'func_closure'
Without Tav's patch, the untrusted code can extract our password ('secret') from the closure with that last statement. To run whole files in the sandbox:
sandbox.run_file('/some/path/file.py')
Finally, disclaimer: I put this together in *15 minutes*. I'm sure I missed something, this isn't secure, etc. It's just a proof of concept. Void where prohibited, etc. Cheers, -- Ivan Krstić <krstic@solarsail.hcs.harvard.edu> | http://radian.org
I'm not familiar with Genshi -- what is the purpose of the AST transformation here? Traditionally, sandboxing environments in Python usually just exec the untrusted code in an environment with a __builtins__ dict that limits the built-in functions and overrides __import__ so as to provide an import implementation that allows import of pure-Python modules but not extensions (or only allows certain extensions that have been previously audited, or provides wrappers or subsets of selected others). --Guido On Sun, Feb 22, 2009 at 6:09 PM, Ivan Krstić <krstic@solarsail.hcs.harvard.edu> wrote:
On Feb 22, 2009, at 5:15 PM, Martin v. Löwis wrote:
What is the objective of this code? Is it a complete sandbox? If not, is a complete sandbox based on it available somehow for review?
From a cursory look at Tav's CPython patch, I'd describe it as follows:
Requires: an existing Python code execution environment which does not expose unsafe interpreter functionality to the (untrusted) code it runs,
Provides: a way for the trusted code outside of that restricted environment to wrap functions (which may reference trusted objects) in closures which can be passed into the restricted environment as completely opaque objects whose internal state (including any referenced trusted objects) cannot be accessed from the untrusted code.
When I say 'requires', I mean 'requires to be useful'; the patch can be applied to vanilla CPython, but isn't useful on its own.
Building blocks for a restricted execution environment as required by the patch are commonly provided in templating libraries; Tav mentions Genshi in particular. By default, Genshi templates don't come with security built in -- you can do what you please in a template. However, since Genshi manually constructs a Python AST from Python code in the template, one can restrict the AST and modify the builtins exposed to the template environment, making things like filesystem access, import and other sensitive system facilities unavailable.
Even with those unavailable, untrusted code can break out of containment by accessing object.__subclasses__ or gaining access to gi_frame and gi_code. This means you can't, for instance, pass into the untrusted code a closure which operates on trusted objects. Tav's patch addresses this particular problem.
To give you a complete sandbox to play with, I just violently ripped out the relevant code from Genshi, added some glue, and slapped it all together in a single file along with Tav's pure-Python code which is functionally equivalent to the CPython patch he submitted. The result, tested on 2.5.1 and nothing else:
<http://radian.org/~krstic/sandbox.py>
As is, sandbox allows you to execute untrusted Python code which won't have access to import, __import__, eval, exec, execfile, open, and to which you can pass closures which reference trusted objects which the untrusted code can't get at.
Example:
import sandbox import sys import md5 def trusted_pwd_closure(password): ... def get_pwd(): ... return md5.md5(password).hexdigest() ... return get_pwd ... context = dict(pwd=trusted_pwd_closure('secret')) sandbox.run_string("print pwd()", context) sandbox.run_string("print pwd.func_closure[0].cell_contents", context) [snip] AttributeError: 'function' object has no attribute 'func_closure'
Without Tav's patch, the untrusted code can extract our password ('secret') from the closure with that last statement.
To run whole files in the sandbox:
sandbox.run_file('/some/path/file.py')
Finally, disclaimer: I put this together in *15 minutes*. I'm sure I missed something, this isn't secure, etc. It's just a proof of concept. Void where prohibited, etc.
Cheers,
-- Ivan Krstić <krstic@solarsail.hcs.harvard.edu> | http://radian.org
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Feb 22, 2009, at 9:43 PM, Guido van Rossum wrote:
I'm not familiar with Genshi -- what is the purpose of the AST transformation here?
Sorry, I should have been clearer. If the only goal is to provide a restricted bare interpreter, you can certainly just exec with a restricted set of builtins and no __import__. Since Tav mentioned wanting restricted execution of Genshi templates in particular (which have a rather complicated mechanism for executing inline Python code), I threw together a realistic, self-contained 'restricting Genshi' demo which doesn't rely on outside restrictions, such as those provided by GAE. You can ignore the AST stuff; Genshi does it for its own (non- security) purposes. -- Ivan Krstić <krstic@solarsail.hcs.harvard.edu> | http://radian.org
Dearest fellow Python lovers, Could one of you please review: http://codereview.appspot.com/20051 The patch is a mere 6 lines of code and provides the absolute minimum that is needed to secure the Python interpreter! [This patch is for Python 2.5.4 -- I can create one for the other branches too if wanted...] It turns out that the __builtins__ based restricted framework is pure genius and gives us nearly everything that's needed to secure our beloved interpreter. This patch simply closes the two holes in 2.5.x -- type.__subclasses__ and GeneratorType.gi_frame. GeneratorType.gi_code would need to be restricted in 2.6 and above. The problem with rexec and brethren have simply been that we tried to do class-based sandboxing. This approach is filled with dark exploitable corners. In contrast, a function-based approach gives us security through the simplicity of the object-capability model. Or at least I currently believe so =) Once this patch gets through onto App Engine, I'll create an app with a sandboxed framework built around it and you can all prove me wrong. But, for the sake of my relationship with the missus, I hope not ;p Please note that this patch doesn't do anything to prevent any of the various crashers in Python. It is not the intention of this patch to make the interpreter invincible against segfaults or exhaustion of resource attacks. For that, I heartily recommend taking a look at PyPy's sandboxed interpreter and/or the magic of App Engine. In the extremely remote chance that I am right -- pigs could fly, right? ;p -- and the sandboxed App Engine app turns out to be impenetrable, I would like to then get approval to simplify the current restricted execution support in the interpreter and create a modernised equivalent of the rexec module. Does that seem reasonable to you all? tav> http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/... martin> What is the objective of this code? Is it a complete martin> sandbox? If not, is a complete sandbox based on martin> it available somehow for review? Martin, sorry, not yet. I'll code one once the patch gets through and release it for use/review. And if people like it, it could form the basis for the modernised rexec I mentioned above... krstic> http://radian.org/~krstic/sandbox.py Thank you Ivan for that Genshi sample! Would you be interested in working with me on the Genshi aspect for the new sandbox framework? pje> Just a question, but, if you just need a pure-python pje> restricted environment for App Engine, why not just pje> use the RestrictedPython package? I'm aware of the various Zope offerings. Performance is the answer to your question. Also, when it comes to security, I am a deep believer in simplicity. Thanks again! -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian
And, here's a version for Python 2.6+ -- diffed against an svn checkout of the current python/trunk: http://codereview.appspot.com/21051/show Please review also. Cheers! -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian
And, here's a version for Python 2.6+ -- diffed against an svn checkout of the current python/trunk:
http://codereview.appspot.com/21051/show
Please review also. Cheers!
No need to provide two versions. Regular back-merging should be able to deal with that just fine. Regards, Martin
On Mon, Feb 23, 2009 at 09:23, tav <tav@espians.com> wrote:
Dearest fellow Python lovers,
Could one of you please review:
http://codereview.appspot.com/20051
The patch is a mere 6 lines of code and provides the absolute minimum that is needed to secure the Python interpreter! [This patch is for Python 2.5.4 -- I can create one for the other branches too if wanted...]
It turns out that the __builtins__ based restricted framework is pure genius and gives us nearly everything that's needed to secure our beloved interpreter. This patch simply closes the two holes in 2.5.x -- type.__subclasses__ and GeneratorType.gi_frame. GeneratorType.gi_code would need to be restricted in 2.6 and above.
The problem with rexec and brethren have simply been that we tried to do class-based sandboxing. This approach is filled with dark exploitable corners. In contrast, a function-based approach gives us security through the simplicity of the object-capability model.
If you block __closure__ and __globals__ on function objects you will get a semblance of a private namespace. That way you might (I have not thought this one through like securing the interpreter for embedding) be able to get what you need to safely pass in Python code through the globals of the code being executed. -Brett
Or at least I currently believe so =)
Once this patch gets through onto App Engine, I'll create an app with a sandboxed framework built around it and you can all prove me wrong. But, for the sake of my relationship with the missus, I hope not ;p
Please note that this patch doesn't do anything to prevent any of the various crashers in Python. It is not the intention of this patch to make the interpreter invincible against segfaults or exhaustion of resource attacks. For that, I heartily recommend taking a look at PyPy's sandboxed interpreter and/or the magic of App Engine.
In the extremely remote chance that I am right -- pigs could fly, right? ;p -- and the sandboxed App Engine app turns out to be impenetrable, I would like to then get approval to simplify the current restricted execution support in the interpreter and create a modernised equivalent of the rexec module.
Does that seem reasonable to you all?
tav> http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/...
martin> What is the objective of this code? Is it a complete martin> sandbox? If not, is a complete sandbox based on martin> it available somehow for review?
Martin, sorry, not yet.
I'll code one once the patch gets through and release it for use/review.
And if people like it, it could form the basis for the modernised rexec I mentioned above...
krstic> http://radian.org/~krstic/sandbox.py<http://radian.org/%7Ekrstic/sandbox.py>
Thank you Ivan for that Genshi sample!
Would you be interested in working with me on the Genshi aspect for the new sandbox framework?
pje> Just a question, but, if you just need a pure-python pje> restricted environment for App Engine, why not just pje> use the RestrictedPython package?
I'm aware of the various Zope offerings.
Performance is the answer to your question.
Also, when it comes to security, I am a deep believer in simplicity.
Thanks again!
-- love, tav
plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
Could one of you please review:
http://codereview.appspot.com/20051
The patch is a mere 6 lines of code and provides the absolute minimum that is needed to secure the Python interpreter!
Unlike Guido, I'm not quite willing to your word for it. OTOH, the patch looks harmless (with minor corrections). It could be considered a bug fix for the current set of restricted attributes (although I do wish somebody would present a design telling what reflective attributes must be restricted and why; the current set looks arbitrary) Regards, Martin
Hey Martin,
The patch is a mere 6 lines of code and provides the absolute minimum that is needed to secure the Python interpreter!
Unlike Guido, I'm not quite willing to your word for it.
You are right. Sorry, I was a bit too enthusiastic and overstated the case. How about: "it could possibly enable a secured Python interpreter" ?
OTOH, the patch looks harmless (with minor corrections). It could be considered a bug fix for the current set of restricted attributes
Yes, and it is in that light that I would like the patch to be accepted. -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
This looks very interesting. What I like about Tav's approach is that it should also be directly applicable to Jython. Much like Jython in general, there's a tight correspondence between typeobject.c/PyType.java and genobject.c/PyGenerator.java. So we plan on trying out a similar, presumably small patch too. What will be very helpful here is identifying a set of tests that verify these claims of restricted execution. - Jim On Mon, Feb 23, 2009 at 4:03 PM, tav <tav@espians.com> wrote:
Hey Martin,
The patch is a mere 6 lines of code and provides the absolute minimum that is needed to secure the Python interpreter!
Unlike Guido, I'm not quite willing to your word for it.
You are right. Sorry, I was a bit too enthusiastic and overstated the case.
How about: "it could possibly enable a secured Python interpreter" ?
OTOH, the patch looks harmless (with minor corrections). It could be considered a bug fix for the current set of restricted attributes
Yes, and it is in that light that I would like the patch to be accepted.
-- love, tav
plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/jbaker%40zyasoft.com
-- Jim Baker jbaker@zyasoft.com
Guido van Rossum wrote:
I've received some enthusiastic emails from someone who wants to revive restricted mode. He started out with a bunch of patches to the CPython runtime using ctypes, which he attached to an App Engine bug:
http://code.google.com/p/googleappengine/issues/detail?id=671
Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.)
I don't recall the exploits that Samuele once posted that caused the death of rexec.py -- does anyone recall, or have a pointer to the threads?
I don't have much time these days, for sure not until pycon us, to look at the proposed code. A remark though, in many cases people want to use restricted execution to allow trusted and untrusted code to live side by side. In such a scenario the issue is not only how to prevent for example the untrusted code to get hold of open() but also how to deal with the security issues at the boundary when untrusted code calls into trusted code. AFAIK E provides and incorporate a lot of thinking around the concept of guards, a sort of generalized runtime "type checking" with very strong guarantees, in a language like Java instead type checking and final classes can be used to provide the required safety. Python instead over the years has grown more hooks for object to cheat about their types. In general safely programming such trusted code in python will be delicate and slightly cumbersome. Think for example of thinhs that can be done by objects redefining equality wrt data structures that old sensitive information unless such arguments are blocked.
Le Sunday 22 February 2009 17:45:27 Guido van Rossum, vous avez écrit :
I've received some enthusiastic emails from someone who wants to revive restricted mode. (...) Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.)
Some ways to "crash" Python: - use ctypes: invalid memory read/write - use os.kill(): kill the current process - call buggy function: invalid memory read/write or denial of service - "while 1: pass": denial of service - allocate many huge objects: MemoryError (maybe invalid memory read/write) - load a buggy .pyc file: invalid memory read/write - recursive structures/function calls: stack overflow (in buggy functions, see the bug tracker) - etc. Protections against these attacks: - Module whitelist (or a least use a blacklist of all modules written in C) - use system quota: resource.setrlimit() on Linux => set max CPU time and max memory limits (or signal.alarm() for the timeout) - Run a fuzzer on Python and fix all bugs :-) I wrote a short document in Python's wiki on the different security projects: http://wiki.python.org/moin/Security -- Victor Stinner aka haypo http://www.haypocalc.com/blog/
None of those are useful attacks on app engine though. On Mon, Feb 23, 2009 at 7:57 AM, Victor Stinner <victor.stinner@haypocalc.com> wrote:
Le Sunday 22 February 2009 17:45:27 Guido van Rossum, vous avez écrit :
I've received some enthusiastic emails from someone who wants to revive restricted mode. (...) Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.)
Some ways to "crash" Python:
- use ctypes: invalid memory read/write - use os.kill(): kill the current process - call buggy function: invalid memory read/write or denial of service - "while 1: pass": denial of service - allocate many huge objects: MemoryError (maybe invalid memory read/write) - load a buggy .pyc file: invalid memory read/write - recursive structures/function calls: stack overflow (in buggy functions, see the bug tracker) - etc.
Protections against these attacks:
- Module whitelist (or a least use a blacklist of all modules written in C) - use system quota: resource.setrlimit() on Linux => set max CPU time and max memory limits (or signal.alarm() for the timeout) - Run a fuzzer on Python and fix all bugs :-)
I wrote a short document in Python's wiki on the different security projects:
http://wiki.python.org/moin/Security
-- Victor Stinner aka haypo http://www.haypocalc.com/blog/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Sorry about being a bit late on this front (just 5 years...), but I've extended tav's jail to module level, and added the niceties. It's goal is similar to that of rexec, stopping IO, but not crashes. It is currently at https://github.com/matsjoyce/sandypython, and it has instructions as to its use. I've bashed it with all the exploits I've found online, and its still holding, so I thought the public might like ago.
Hi, 2014-07-21 21:26 GMT+02:00 matsjoyce <matsjoyce@gmail.com>:
Sorry about being a bit late on this front (just 5 years...), but I've extended tav's jail to module level, and added the niceties. It's goal is similar to that of rexec, stopping IO, but not crashes. It is currently at https://github.com/matsjoyce/sandypython, and it has instructions as to its use. I've bashed it with all the exploits I've found online, and its still holding, so I thought the public might like ago.
I wrote this project, started from tav's jail: https://github.com/haypo/pysandbox/ I gave up because I know consider that pysandbox is broken by design. Please read the LWN article: https://lwn.net/Articles/574215/ Don't hesitate to ask more specific questions. Victor
Yup, I read that post. However, those specific issues do not exist in my module, as there is a module whitelist, and a method whitelist. Builtins are now proxied, and all types going in to functions are checked for modification. There maybe some holes in my approach, but I can't find them.
On 11/08/2014 18:42, matsjoyce wrote:
Yup, I read that post. However, those specific issues do not exist in my module, as there is a module whitelist, and a method whitelist. Builtins are now proxied, and all types going in to functions are checked for modification. There maybe some holes in my approach, but I can't find them.
Any chance of giving us some context, or do I have to retrieve my crystal ball from the menders? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence
On Mon, 11 Aug 2014, Skip Montanaro wrote:
On Mon, Aug 11, 2014 at 12:42 PM, matsjoyce <matsjoyce@gmail.com> wrote:
There maybe some holes in my approach, but I can't find them.
There's the rub. Given time, I suspect someone will discover a hole or two.
Schneier's Law: Any person can invent a security system so clever that she or he can't think of how to break it. While I would not claim a Python sandbox is utterly impossible, I'm suspicious that the whole "consenting adults" approach in Python is incompatible with a sandbox. The whole idea of a sandbox is to absolutely prevent people from doing things even if they really want to and know what they are doing. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist
Unless you remove all the things labelled "keep away from children". I wrote this sandbox to allow python to be used as a "mods"/"add-ons" language for a game I'm writing, hence the perhaps too strict nature. About the crashers: as this is for games, its "fine" for the game to crash, as long as the sandbox is not broken while crashing. time and math can probably be allowed, but random imports a lot of undesirable modules. My sandbox doesn't use proxies, due to the introspection and complexity that it involves. Instead it completely isolates the sandboxed globals, and checks all arguments and globals for irregularities before passing control to non-sandboxed functions. On 13 August 2014 14:11, Isaac Morland <ijmorlan@uwaterloo.ca> wrote:
On Mon, 11 Aug 2014, Skip Montanaro wrote:
On Mon, Aug 11, 2014 at 12:42 PM, matsjoyce <matsjoyce@gmail.com> wrote:
There maybe some holes in my approach, but I can't find them.
There's the rub. Given time, I suspect someone will discover a hole or two.
Schneier's Law:
Any person can invent a security system so clever that she or he can't think of how to break it.
While I would not claim a Python sandbox is utterly impossible, I'm suspicious that the whole "consenting adults" approach in Python is incompatible with a sandbox. The whole idea of a sandbox is to absolutely prevent people from doing things even if they really want to and know what they are doing.
Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist
On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland <ijmorlan@uwaterloo.ca> wrote:
While I would not claim a Python sandbox is utterly impossible, I'm suspicious that the whole "consenting adults" approach in Python is incompatible with a sandbox. The whole idea of a sandbox is to absolutely prevent people from doing things even if they really want to and know what they are doing.
It's certainly not *fundamentally* impossible to sandbox Python. However, the question becomes one of how much effort you're going to go to and how much you're going to restrict the code. I think I remember reading about something that's like ast.literal_eval, but allows name references; with that, plus some tiny features of assignment, you could make a fairly straight-forward evaluator that lets you work comfortably with numbers, strings, lists, dicts, etc. That could be pretty useful - but it wouldn't so much be "Python in a sandbox" as "an expression evaluator that uses a severely restricted set of Python syntax". If you start with all of Python and then start cutting out the dangerous bits, you're doomed to miss something, and your sandbox is broken. If you start with nothing and then start adding functionality, you're looking at a gigantic job before it becomes anything that you could call an applications language. So while it's theoretically possible (I think - certainly I can't say for sure that it's impossible), it's fairly impractical. I've had my own try at it, and failed quite badly (fortunately noisily and at a sufficiently early stage of development to shift). ChrisA
On Thu, Aug 14, 2014 at 02:26:29AM +1000, Chris Angelico wrote:
On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland <ijmorlan@uwaterloo.ca> wrote:
While I would not claim a Python sandbox is utterly impossible, I'm suspicious that the whole "consenting adults" approach in Python is incompatible with a sandbox. The whole idea of a sandbox is to absolutely prevent people from doing things even if they really want to and know what they are doing.
The point of a sandbox is that I, the consenting adult writing the application in the first place, may want to allow *untrusted others* to call Python code without giving them control of the entire application. The consenting adults rule applies to me, the application writer, not them, the end-users, even if they happen to be writing Python code. If they want unrestricted access to the Python interpreter, they can run their code on their own machine, not mine.
It's certainly not *fundamentally* impossible to sandbox Python. However, the question becomes one of how much effort you're going to go to and how much you're going to restrict the code.
I believe that PyPy has an effective sandbox, but to what degree of effectiveness I don't know. http://pypy.readthedocs.org/en/latest/sandbox.html I've had rogue Javascript crash my browser or make my entire computer effectively unusable often enough that I am skeptical about claims that Javascript in the browser is effectively sandboxed, so I'm doubly cautious about Python. -- Steven
On Thu, Aug 14, 2014 at 2:58 AM, Steven D'Aprano <steve@pearwood.info> wrote:
It's certainly not *fundamentally* impossible to sandbox Python. However, the question becomes one of how much effort you're going to go to and how much you're going to restrict the code.
I believe that PyPy has an effective sandbox, but to what degree of effectiveness I don't know.
""" A potential attacker can have arbitrary code run in the subprocess, but cannot actually do any input/output not controlled by the outer process. Additional barriers are put to limit the amount of RAM and CPU time used. Note that this is very different from sandboxing at the Python language level, i.e. placing restrictions on what kind of Python code the attacker is allowed to run (why? read about pysandbox). """ That's quite useful, but isn't the same thing as a Python-in-Python sandbox (or even what I was doing, Python-in-C++). ChrisA
On Thu, 14 Aug 2014, Steven D'Aprano wrote:
On Thu, Aug 14, 2014 at 02:26:29AM +1000, Chris Angelico wrote:
On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland <ijmorlan@uwaterloo.ca> wrote:
While I would not claim a Python sandbox is utterly impossible, I'm suspicious that the whole "consenting adults" approach in Python is incompatible with a sandbox. The whole idea of a sandbox is to absolutely prevent people from doing things even if they really want to and know what they are doing.
The point of a sandbox is that I, the consenting adult writing the application in the first place, may want to allow *untrusted others* to call Python code without giving them control of the entire application. The consenting adults rule applies to me, the application writer, not them, the end-users, even if they happen to be writing Python code. If they want unrestricted access to the Python interpreter, they can run their code on their own machine, not mine.
Yes, absolutely, and I didn't mean to contradict what you are saying. What I am suggesting is that the basic design of Python isn't a good starting point for imposing mandatory restrictions on what code can do. By contrast, take something like Safe Haskell. I'm not absolutely certain that it really is safe as promised, but it's starting from a very different language in which the compiler performs extremely sophisticated type checking and simply won't compile programs that don't work within the type system. This isn't a knock on Python (which I love using, by the way), just being realistic about what the existing language is likely to be able to support. Having said that, I'll be very interested if somebody does come up with a restricted mode Python that is widely accepted as being secure - that would be a real achievement. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist
Hi, I heard that PyPy sandbox cannot be used out of the box. You have to write a policy to allow syscalls. The complexity is moved to this policy which is very hard to write, especially if you only use whitelists. Correct me if I'm wrong. To be honest, I never take a look at this sandbox. Victor Le mercredi 13 août 2014, Steven D'Aprano <steve@pearwood.info> a écrit :
On Thu, Aug 14, 2014 at 02:26:29AM +1000, Chris Angelico wrote:
On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland <ijmorlan@uwaterloo.ca <javascript:;>> wrote:
While I would not claim a Python sandbox is utterly impossible, I'm suspicious that the whole "consenting adults" approach in Python is incompatible with a sandbox. The whole idea of a sandbox is to absolutely prevent people from doing things even if they really want to and know what they are doing.
The point of a sandbox is that I, the consenting adult writing the application in the first place, may want to allow *untrusted others* to call Python code without giving them control of the entire application. The consenting adults rule applies to me, the application writer, not them, the end-users, even if they happen to be writing Python code. If they want unrestricted access to the Python interpreter, they can run their code on their own machine, not mine.
It's certainly not *fundamentally* impossible to sandbox Python. However, the question becomes one of how much effort you're going to go to and how much you're going to restrict the code.
I believe that PyPy has an effective sandbox, but to what degree of effectiveness I don't know.
http://pypy.readthedocs.org/en/latest/sandbox.html
I've had rogue Javascript crash my browser or make my entire computer effectively unusable often enough that I am skeptical about claims that Javascript in the browser is effectively sandboxed, so I'm doubly cautious about Python.
-- Steven _______________________________________________ Python-Dev mailing list Python-Dev@python.org <javascript:;> https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.co...
On 14 August 2014 07:25, Victor Stinner <victor.stinner@gmail.com> wrote:
Hi,
I heard that PyPy sandbox cannot be used out of the box. You have to write a policy to allow syscalls. The complexity is moved to this policy which is very hard to write, especially if you only use whitelists.
Correct me if I'm wrong. To be honest, I never take a look at this sandbox.
By default, the PyPy sandbox requires all system access to be proxied through the host application (which is running in a separate process). Similarly, using "sandbox" on Fedora (et al) will get you a default deny OS level sandbox, where you have to provide selective access to things outside the box. The effective decision taken when rexec and Bastion were removed from the standard library was "sandboxing is hard enough for operating systems to get right, we're not going to try to tackle the even harder problem of an in-process sandbox". "Deny all" sandboxes are relatively easy, but also relatively useless. It's "allow these activities, but no others" that's difficult, since any kind of access can often be leveraged into greater access than was intended. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
2014-08-11 19:42 GMT+02:00 matsjoyce <matsjoyce@gmail.com>:
Yup, I read that post. However, those specific issues do not exist in my module, as there is a module whitelist, and a method whitelist. Builtins are now proxied, and all types going in to functions are checked for modification. There maybe some holes in my approach, but I can't find them.
I take a look at your code and it looks like almost everything is blocked. Right now, I'm not sure that your sandbox is useful. For example, for a simple IRC bot, it would help to have access to some modules like math, time or random. The problem is to provide a way to allow these modules and ensure that the policy doesn't introduce a new hole. Allowing more functions increase the risk of new holes. Even if your sandbox is strong, CPython contains a lot of code written in C (50% of CPython is written in C), and the C code usually takes shortcuts which ignore your sandbox. CPython source code is huge (+210k of C lines just for the core). Bugs are common, your sandbox is vulnerable to all these bugs. See for example the Lib/test/crashers/ directory of CPython. For my pysandbox project, I wrote some proxies and many vulnerabilities were found in these proxies. They can be explained by the nature of Python, you can introspect everything, modify everything, etc. It's very hard to design such proxy in Python. Implementing such proxy in C helps a little bit. The rule is always the same: your sandbox is as strong as its weakest function. A very minor bug is enough to break the whole sandbox. See the history of pysandbox for examples of such bugs (called "vulnerabilities" in the case of a sandbox). Victor
Unless you remove all the things labelled "keep away from children". I wrote this sandbox to allow python to be used as a "mods"/"add-ons" language for a game I'm writing, hence the perhaps too strict nature. About the crashers: as this is for games, its "fine" for the game to crash, as long as the sandbox is not broken while crashing. time and math can probably be allowed, but random imports a lot of undesirable modules. My sandbox doesn't use proxies, due to the introspection and complexity that it involves. Instead it completely isolates the sandboxed globals, and checks all arguments and globals for irregularities before passing control to non- sandboxed functions.
On 8/13/2014 12:19 PM, matsjoyce wrote:
Unless you remove all the things labelled "keep away from children". I wrote this sandbox to allow python to be used as a "mods"/"add-ons" language for a game I'm writing, hence the perhaps too strict nature.
About the crashers: as this is for games, its "fine" for the game to crash, as long as the sandbox is not broken while crashing.
time and math can probably be allowed, but random imports a lot of undesirable modules.
My sandbox doesn't use proxies, due to the introspection and complexity that it involves. Instead it completely isolates the sandboxed globals, and checks all arguments and globals for irregularities before passing control to non- sandboxed functions.
pydev is for mainly for discussion of maintaining current versions and development of the next, and for discussion of PEPs which might apply to the one after next. This discussion should be on python-list or perhaps python-ideas if there is a semi-concrete proposal for a future python. -- Terry Jan Reedy
participants (19)
-
"Martin v. Löwis" -
Benjamin Peterson -
Brett Cannon -
Chris Angelico -
Greg Ewing -
Guido van Rossum -
Isaac Morland -
Ivan Krstić -
Jim Baker -
Mark Lawrence -
matsjoyce -
Nick Coghlan -
Samuele Pedroni -
Skip Montanaro -
Steven D'Aprano -
tav -
Terry Reedy -
Victor Stinner -
Victor Stinner