Hello, All, I'm trying to translate sandboxed pypy but I get following error: [sandbox:WARNING] Not Implemented: sandboxing for external function 'inet_ntop' * [rtyper] -=- specialized 3 more blocks -=- [Timer] Timings: [Timer] annotate --- 1380.9 s [Timer] rtype_lltype --- 1414.1 s [Timer] backendopt_lltype --- 647.0 s [Timer] stackcheckinsertion_lltype --- 218.3 s [Timer] database_c --- 1395.4 s [Timer] =========================================== [Timer] Total: --- 5055.7 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "./translate.py", line 278, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 802, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 270, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 480, in task_database_c [translation:ERROR] database = cbuilder.build_database() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/genc.py", line 73, in build_database [translation:ERROR] db.complete() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 310, in complete [translation:ERROR] add_dependencies(node.enum_dependencies()) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 298, in add_dependencies [translation:ERROR] self.get(value) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 218, in get [translation:ERROR] node = self.getcontainernode(container) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 156, in getcontainernode [translation:ERROR] node = nodefactory(self, T, container, **buildkwds) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 699, in __init__ [translation:ERROR] self.make_funcgens() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 705, in make_funcgens [translation:ERROR] self.funcgens = select_function_code_generators(self.obj, self.db, self.name) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 825, in select_function_code_generators [translation:ERROR] return sandbox_stub(fnobj, db) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 787, in sandbox_stub [translation:ERROR] force_stub=True) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/sandbox/rsandbox.py", line 143, in get_external_function_sandbox_graph [translation:ERROR] args_s = [annmodel.lltype_to_annotation(ARG) for ARG in FUNCTYPE.ARGS] [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/annotation/model.py", line 639, in lltype_to_annotation [translation:ERROR] return SomePtr(T) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/annotation/model.py", line 537, in __init__ [translation:ERROR] assert isinstance(ll_ptrtype, lltype.Ptr) [translation:ERROR] AssertionError [translation] start debugger...
/home/dalius/projects/pypy-dist/pypy/annotation/model.py(537)__init__() -> assert isinstance(ll_ptrtype, lltype.Ptr) (Pdb+) q
I was running command: /pypy-dist/pypy/translator/goal$ ./translate.py --sandbox --opt=1 --gc=hybrid targetpypystandalone.py I have just quited this time but it failed with same error as I run ./translate.py --sandbox targetpypystandalone.py (without options). ll_ptrtype is sock_addr if I remember correctly (when I will run it next time I will check). My system: $ uname -a Linux dalius 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686 GNU/Linux $ python -V Python 2.5.2 -- Dalius http://blog.sandbox.lt
Unfortunately pypy/dist is largely outdated nowadays (we plan to copy trunk to dist soon) please try with pypy/trunk.
Hello, On Wed, Feb 18, 2009 at 3:08 PM, Antonio Cuni <anto.cuni@gmail.com> wrote: trunk has worked just perfect. I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...). -- Dalius http://blog.sandbox.lt
Dalius Dobravolskas wrote:
trunk has worked just perfect.
nice :-)
I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...).
that's cool. Please keep us informed, and feel free to come to #pypy on irc.freenode.net if you have any question. ciao, Anto
Hello,
I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...).
that's cool. Please keep us informed, and feel free to come to #pypy on irc.freenode.net if you have any question.
Here is result: http://py.sandbox.lt/ Basically it is the same code as Django example but only written in WSGI (I will share that later). Here some of my experiences that I think I should mention: 1. I'm running my code behind Apache mod_wsgi. As you might know mod_wsgi is very strict WSGI implementation. pypy_interact failed when was calling isatty function (mod_wsgi.Log does not have this function). I have fixed that by adding "'isatty in dir(log)". I will give later exact locations and names if I have confused them. 2. It looks like that PyPySandboxedProc expects --heapsize as first argument in argument list. 3. Is settimeout actually working? If I run following code (please don't run it on my server): x = 1 for y in xrange(1, 1024*1024): x *= y print x Process is not terminated after one second and uses significant part of my CPU resources. I set timeout in following way: sandproc = pypy_interact.PyPySandboxedProc(SANDBOX_BIN, ['--heapsize', str(1024*1024), '-c', code]) sandproc.settimeout(1, interrupt_main=True) I have tried 5 seconds as well. I will analyze the code myself a little bit but if you have ideas already please share them ;-) -- Dalius http://blog.sandbox.lt
Hello, On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
3. Is settimeout actually working? If I run following code (please don't run it on my server): OK. Everything is OK in PyPy code. timeout works (at least log indicates that). There is something specific about mod_wsgi and Apache in this case.
-- Dalius http://blog.sandbox.lt
Hello,
On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
3. Is settimeout actually working? If I run following code (please don't run it on my server): OK. Everything is OK in PyPy code. timeout works (at least log indicates that). There is something specific about mod_wsgi and Apache in this case. That's what I have found out:
On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote: pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. That's how I solved problem now (modifed pypy/tool/killsubprocess.py). I will dig deeper of course but I will do that later. HTH for other people who will run sandboxed pypy. -- Dalius http://blog.sandbox.lt
That's all very good news. Keep us informed! Cheers, fijal On Fri, Feb 20, 2009 at 10:13 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
Hello,
On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
3. Is settimeout actually working? If I run following code (please don't run it on my server): OK. Everything is OK in PyPy code. timeout works (at least log indicates that). There is something specific about mod_wsgi and Apache in this case. That's what I have found out:
On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote: pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. That's how I solved problem now (modifed pypy/tool/killsubprocess.py). I will dig deeper of course but I will do that later.
HTH for other people who will run sandboxed pypy.
-- Dalius http://blog.sandbox.lt _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
On Fri, 20 Feb 2009 11:13:11 +0200, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
Hello,
On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
3. Is settimeout actually working? If I run following code (please don't run it on my server): OK. Everything is OK in PyPy code. timeout works (at least log indicates that). There is something specific about mod_wsgi and Apache in this case. That's what I have found out:
On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote: pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. That's how I solved problem now (modifed pypy/tool/killsubprocess.py). I will dig deeper of course but I will do that later.
Check out the sigprocmask Apache sets for itself. Unfortunately, this is inherited by all processes Apache runs, usually leading to surprising behavior, since it prevents the delivery of almost all signals. Jean-Paul
Jean-Paul Calderone wrote:
That's what I have found out: pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. That's how I solved problem now (modifed pypy/tool/killsubprocess.py). I will dig deeper of course but I will do that later.
Check out the sigprocmask Apache sets for itself. Unfortunately, this is inherited by all processes Apache runs, usually leading to surprising behavior, since it prevents the delivery of almost all signals.
I fear that everything is more complicated here. There are some information in mod_wsgi documentation about signals. I will ask Graham (mod_wsgi author) about that. He might have some ideas what to do. Regards, Dalius
Check out the sigprocmask Apache sets for itself. Unfortunately, this is inherited by all processes Apache runs, usually leading to surprising behavior, since it prevents the delivery of almost all signals. FYI, I have contacted mod_wsgi author. If you are using mod_wsgi 2.X with deamon mode SIGTERM works. The problem still persists for embedded mode. That means there is solution that works with SIGTERM and no changes are required in pypy code.
If you are interested how it will work in the future, please follow this thread in modwsgi group: http://groups.google.com/group/modwsgi/browse_thread/thread/aae8d2d675ebcf9 Thanks everyone for your help. -- Dalius http://blog.sandbox.lt
Hi Dalius, On Fri, Feb 20, 2009 at 08:30:32AM +0200, Dalius Dobravolskas wrote:
2. It looks like that PyPySandboxedProc expects --heapsize as first argument in argument list.
Bit of a hack, but yes, indeed. It also doesn't work well with a PyPy translated with the default GC, which is the hybrid GC. I recommend the generational GC (--gc=generation). (This should really warn you...) A bientot, Armin.
Hi, On Fri, Feb 20, 2009 at 04:59:06PM +0100, Armin Rigo wrote:
Bit of a hack, but yes, indeed. It also doesn't work well with a PyPy translated with the default GC, which is the hybrid GC. I recommend the generational GC (--gc=generation). (This should really warn you...)
Ah, I just found a hack in pypy/config/translationoption.py. The GC defaults to 'generation' instead of 'hybrid' in case of a sandbox translation. So everything is fine :-) A bientot, Armin.
Hello, Dalius Dobravolskas wrote:
1. I'm running my code behind Apache mod_wsgi. As you might know mod_wsgi is very strict WSGI implementation. pypy_interact failed when was calling isatty function (mod_wsgi.Log does not have this function). I have fixed that by adding "'isatty in dir(log)". I will give later exact locations and names if I have confused them Here is traceback from this situation:
Traceback (most recent call last):, referer: http://py.sandbox.lt/ File "/home/dalius/wsgi/pypysandboxserver.py", line 25, in __call__, referer: http://py.sandbox.lt/ sandproc.interact(stdout=code_output, stderr=code_output), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", line 307, in interact, referer: http://py.sandbox.lt/ returncode = self.handle_until_return(), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", line 228, in handle_until_return, referer: http://py.sandbox.lt/ log.call('%s(%s)' % (fnname,, referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/py/log/producer.py", line 55, in __call__, referer: http://py.sandbox.lt/ func(self.Message(self.keywords, args)), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/tool/ansi_print.py", line 72, in __call__, referer: http://py.sandbox.lt/ file=self.file, newline=newline, flush=flush), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/py/io/terminalwriter.py", line 34, in ansi_print, referer: http://py.sandbox.lt/ if esc and sys.platform != "win32" and file.isatty():, referer: http://py.sandbox.lt/ AttributeError: 'mod_wsgi.Log' object has no attribute 'isatty', referer: http://py.sandbox.lt/ Here is diff how I have avoided this problem: --- io/terminalwriter.py (revision 62015) +++ io/terminalwriter.py (working copy) @@ -31,7 +31,7 @@ if file is None: file = sys.stderr text = text.rstrip() - if esc and sys.platform != "win32" and file.isatty(): + if esc and sys.platform != "win32" and 'isatty' in dir(file) and file.isatty(): if not isinstance(esc, tuple): esc = (esc,) text = (''.join(['\x1b[%sm' % cod for cod in esc]) + Could it be fixed in the code? Regards, Dalius
On Sun, Feb 22, 2009 at 09:57:22PM +0200, Dalius Dobravolskas wrote:
Here is diff how I have avoided this problem:
--- io/terminalwriter.py (revision 62015) +++ io/terminalwriter.py (working copy) @@ -31,7 +31,7 @@ if file is None: file = sys.stderr text = text.rstrip() - if esc and sys.platform != "win32" and file.isatty(): + if esc and sys.platform != "win32" and 'isatty' in dir(file) and file.isatty():
Please use hasattr(log, 'isatty') instead.
if not isinstance(esc, tuple): esc = (esc,) text = (''.join(['\x1b[%sm' % cod for cod in esc]) +
Marius Gedminas -- I am a computer. I am dumber than any human and smarter than any administrator.
+ if esc and sys.platform != "win32" and 'isatty' in dir(file) and file.isatty():
Please use hasattr(log, 'isatty') instead. Ačiū, Mariau. I felt that there should be more efficient way to do that ;-)
-- Dalius http://blog.sandbox.lt
Hello, On Wed, Feb 18, 2009 at 10:39 PM, Dalius Dobravolskas <dalius.dobravolskas@gmail.com> wrote:
I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.htm...).
So here is final result: http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox Thank you everyone for your help. I think I will play around more with PyPy. -- Dalius http://blog.sandbox.lt
participants (6)
-
Antonio Cuni -
Armin Rigo -
Dalius Dobravolskas -
Jean-Paul Calderone -
Maciej Fijalkowski -
Marius Gedminas