Hello, All, I'm trying to translate sandboxed pypy but I get following error: [sandbox:WARNING] Not Implemented: sandboxing for external function 'inet_ntop' * [rtyper] -=- specialized 3 more blocks -=- [Timer] Timings: [Timer] annotate --- 1380.9 s [Timer] rtype_lltype --- 1414.1 s [Timer] backendopt_lltype --- 647.0 s [Timer] stackcheckinsertion_lltype --- 218.3 s [Timer] database_c --- 1395.4 s [Timer] =========================================== [Timer] Total: --- 5055.7 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "./translate.py", line 278, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 802, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 270, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 480, in task_database_c [translation:ERROR] database = cbuilder.build_database() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/genc.py", line 73, in build_database [translation:ERROR] db.complete() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 310, in complete [translation:ERROR] add_dependencies(node.enum_dependencies()) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 298, in add_dependencies [translation:ERROR] self.get(value) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 218, in get [translation:ERROR] node = self.getcontainernode(container) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 156, in getcontainernode [translation:ERROR] node = nodefactory(self, T, container, **buildkwds) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 699, in __init__ [translation:ERROR] self.make_funcgens() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 705, in make_funcgens [translation:ERROR] self.funcgens = select_function_code_generators(self.obj, self.db, self.name) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 825, in select_function_code_generators [translation:ERROR] return sandbox_stub(fnobj, db) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 787, in sandbox_stub [translation:ERROR] force_stub=True) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/sandbox/rsandbox.py", line 143, in get_external_function_sandbox_graph [translation:ERROR] args_s = [annmodel.lltype_to_annotation(ARG) for ARG in FUNCTYPE.ARGS] [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/annotation/model.py", line 639, in lltype_to_annotation [translation:ERROR] return SomePtr(T) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/annotation/model.py", line 537, in __init__ [translation:ERROR] assert isinstance(ll_ptrtype, lltype.Ptr) [translation:ERROR] AssertionError [translation] start debugger...
I was running command: /pypy-dist/pypy/translator/goal$ ./translate.py --sandbox --opt=1 --gc=hybrid targetpypystandalone.py I have just quited this time but it failed with same error as I run ./translate.py --sandbox targetpypystandalone.py (without options). ll_ptrtype is sock_addr if I remember correctly (when I will run it next time I will check). My system: $ uname -a Linux dalius 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686 GNU/Linux $ python -V Python 2.5.2 -- Dalius http://blog.sandbox.lt
Unfortunately pypy/dist is largely outdated nowadays (we plan to copy trunk to dist soon) please try with pypy/trunk.
Hello, On Wed, Feb 18, 2009 at 3:08 PM, Antonio Cuni <anto.cuni@gmail.com> wrote: trunk has worked just perfect. I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...). -- Dalius http://blog.sandbox.lt
Dalius Dobravolskas wrote:
trunk has worked just perfect.
nice :-)
I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...).
that's cool. Please keep us informed, and feel free to come to #pypy on irc.freenode.net if you have any question. ciao, Anto
Hello,
Here is result: http://py.sandbox.lt/ Basically it is the same code as Django example but only written in WSGI (I will share that later). Here some of my experiences that I think I should mention: 1. I'm running my code behind Apache mod_wsgi. As you might know mod_wsgi is very strict WSGI implementation. pypy_interact failed when was calling isatty function (mod_wsgi.Log does not have this function). I have fixed that by adding "'isatty in dir(log)". I will give later exact locations and names if I have confused them. 2. It looks like that PyPySandboxedProc expects --heapsize as first argument in argument list. 3. Is settimeout actually working? If I run following code (please don't run it on my server): x = 1 for y in xrange(1, 1024*1024): x *= y print x Process is not terminated after one second and uses significant part of my CPU resources. I set timeout in following way: sandproc = pypy_interact.PyPySandboxedProc(SANDBOX_BIN, ['--heapsize', str(1024*1024), '-c', code]) sandproc.settimeout(1, interrupt_main=True) I have tried 5 seconds as well. I will analyze the code myself a little bit but if you have ideas already please share them ;-) -- Dalius http://blog.sandbox.lt
Hello, On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
-- Dalius http://blog.sandbox.lt
On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote: pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. That's how I solved problem now (modifed pypy/tool/killsubprocess.py). I will dig deeper of course but I will do that later. HTH for other people who will run sandboxed pypy. -- Dalius http://blog.sandbox.lt
On Fri, 20 Feb 2009 11:13:11 +0200, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
Check out the sigprocmask Apache sets for itself. Unfortunately, this is inherited by all processes Apache runs, usually leading to surprising behavior, since it prevents the delivery of almost all signals. Jean-Paul
If you are interested how it will work in the future, please follow this thread in modwsgi group: http://groups.google.com/group/modwsgi/browse_thread/thread/aae8d2d675ebcf9 Thanks everyone for your help. -- Dalius http://blog.sandbox.lt
Hi Dalius, On Fri, Feb 20, 2009 at 08:30:32AM +0200, Dalius Dobravolskas wrote:
2. It looks like that PyPySandboxedProc expects --heapsize as first argument in argument list.
Bit of a hack, but yes, indeed. It also doesn't work well with a PyPy translated with the default GC, which is the hybrid GC. I recommend the generational GC (--gc=generation). (This should really warn you...) A bientot, Armin.
Hello, Dalius Dobravolskas wrote:
Traceback (most recent call last):, referer: http://py.sandbox.lt/ File "/home/dalius/wsgi/pypysandboxserver.py", line 25, in __call__, referer: http://py.sandbox.lt/ sandproc.interact(stdout=code_output, stderr=code_output), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", line 307, in interact, referer: http://py.sandbox.lt/ returncode = self.handle_until_return(), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", line 228, in handle_until_return, referer: http://py.sandbox.lt/ log.call('%s(%s)' % (fnname,, referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/py/log/producer.py", line 55, in __call__, referer: http://py.sandbox.lt/ func(self.Message(self.keywords, args)), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/tool/ansi_print.py", line 72, in __call__, referer: http://py.sandbox.lt/ file=self.file, newline=newline, flush=flush), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/py/io/terminalwriter.py", line 34, in ansi_print, referer: http://py.sandbox.lt/ if esc and sys.platform != "win32" and file.isatty():, referer: http://py.sandbox.lt/ AttributeError: 'mod_wsgi.Log' object has no attribute 'isatty', referer: http://py.sandbox.lt/ Here is diff how I have avoided this problem: --- io/terminalwriter.py (revision 62015) +++ io/terminalwriter.py (working copy) @@ -31,7 +31,7 @@ if file is None: file = sys.stderr text = text.rstrip() - if esc and sys.platform != "win32" and file.isatty(): + if esc and sys.platform != "win32" and 'isatty' in dir(file) and file.isatty(): if not isinstance(esc, tuple): esc = (esc,) text = (''.join(['\x1b[%sm' % cod for cod in esc]) + Could it be fixed in the code? Regards, Dalius
Hello, On Wed, Feb 18, 2009 at 10:39 PM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...).
So here is final result: http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox Thank you everyone for your help. I think I will play around more with PyPy. -- Dalius http://blog.sandbox.lt
Unfortunately pypy/dist is largely outdated nowadays (we plan to copy trunk to dist soon) please try with pypy/trunk.
Hello, On Wed, Feb 18, 2009 at 3:08 PM, Antonio Cuni <anto.cuni@gmail.com> wrote: trunk has worked just perfect. I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...). -- Dalius http://blog.sandbox.lt
Dalius Dobravolskas wrote:
trunk has worked just perfect.
nice :-)
I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...).
that's cool. Please keep us informed, and feel free to come to #pypy on irc.freenode.net if you have any question. ciao, Anto
Hello,
Here is result: http://py.sandbox.lt/ Basically it is the same code as Django example but only written in WSGI (I will share that later). Here some of my experiences that I think I should mention: 1. I'm running my code behind Apache mod_wsgi. As you might know mod_wsgi is very strict WSGI implementation. pypy_interact failed when was calling isatty function (mod_wsgi.Log does not have this function). I have fixed that by adding "'isatty in dir(log)". I will give later exact locations and names if I have confused them. 2. It looks like that PyPySandboxedProc expects --heapsize as first argument in argument list. 3. Is settimeout actually working? If I run following code (please don't run it on my server): x = 1 for y in xrange(1, 1024*1024): x *= y print x Process is not terminated after one second and uses significant part of my CPU resources. I set timeout in following way: sandproc = pypy_interact.PyPySandboxedProc(SANDBOX_BIN, ['--heapsize', str(1024*1024), '-c', code]) sandproc.settimeout(1, interrupt_main=True) I have tried 5 seconds as well. I will analyze the code myself a little bit but if you have ideas already please share them ;-) -- Dalius http://blog.sandbox.lt
Hello, On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
-- Dalius http://blog.sandbox.lt
On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote: pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. That's how I solved problem now (modifed pypy/tool/killsubprocess.py). I will dig deeper of course but I will do that later. HTH for other people who will run sandboxed pypy. -- Dalius http://blog.sandbox.lt
On Fri, 20 Feb 2009 11:13:11 +0200, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
Check out the sigprocmask Apache sets for itself. Unfortunately, this is inherited by all processes Apache runs, usually leading to surprising behavior, since it prevents the delivery of almost all signals. Jean-Paul
If you are interested how it will work in the future, please follow this thread in modwsgi group: http://groups.google.com/group/modwsgi/browse_thread/thread/aae8d2d675ebcf9 Thanks everyone for your help. -- Dalius http://blog.sandbox.lt
Hi Dalius, On Fri, Feb 20, 2009 at 08:30:32AM +0200, Dalius Dobravolskas wrote:
2. It looks like that PyPySandboxedProc expects --heapsize as first argument in argument list.
Bit of a hack, but yes, indeed. It also doesn't work well with a PyPy translated with the default GC, which is the hybrid GC. I recommend the generational GC (--gc=generation). (This should really warn you...) A bientot, Armin.
Hello, Dalius Dobravolskas wrote:
Traceback (most recent call last):, referer: http://py.sandbox.lt/ File "/home/dalius/wsgi/pypysandboxserver.py", line 25, in __call__, referer: http://py.sandbox.lt/ sandproc.interact(stdout=code_output, stderr=code_output), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", line 307, in interact, referer: http://py.sandbox.lt/ returncode = self.handle_until_return(), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", line 228, in handle_until_return, referer: http://py.sandbox.lt/ log.call('%s(%s)' % (fnname,, referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/py/log/producer.py", line 55, in __call__, referer: http://py.sandbox.lt/ func(self.Message(self.keywords, args)), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/tool/ansi_print.py", line 72, in __call__, referer: http://py.sandbox.lt/ file=self.file, newline=newline, flush=flush), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/py/io/terminalwriter.py", line 34, in ansi_print, referer: http://py.sandbox.lt/ if esc and sys.platform != "win32" and file.isatty():, referer: http://py.sandbox.lt/ AttributeError: 'mod_wsgi.Log' object has no attribute 'isatty', referer: http://py.sandbox.lt/ Here is diff how I have avoided this problem: --- io/terminalwriter.py (revision 62015) +++ io/terminalwriter.py (working copy) @@ -31,7 +31,7 @@ if file is None: file = sys.stderr text = text.rstrip() - if esc and sys.platform != "win32" and file.isatty(): + if esc and sys.platform != "win32" and 'isatty' in dir(file) and file.isatty(): if not isinstance(esc, tuple): esc = (esc,) text = (''.join(['\x1b[%sm' % cod for cod in esc]) + Could it be fixed in the code? Regards, Dalius
Hello, On Wed, Feb 18, 2009 at 10:39 PM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...).
So here is final result: http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox Thank you everyone for your help. I think I will play around more with PyPy. -- Dalius http://blog.sandbox.lt
participants (6)
-
Antonio Cuni -
Armin Rigo -
Dalius Dobravolskas -
Jean-Paul Calderone -
Maciej Fijalkowski -
Marius Gedminas