Hi, everybody. Tried to play with pypy, but translating to c failes because of lack of memory. Used command: python translate.py. OS: gentoo linux, amd64 RAM: 4Gb Traceback attached to the letter. nekto0n [cbuild:execute] cc -O3 -fomit-frame-pointer -pthread -c -O3 -fomit-frame-pointer -pthread -c __thread_test.c -o __thread_test.o [Timer] Timings: [Timer] annotate --- 679.5 s [Timer] rtype_lltype --- 487.4 s [Timer] backendopt_lltype --- 458.8 s [Timer] stackcheckinsertion_lltype --- 70.2 s [Timer] database_c --- 971.2 s [Timer] source_c --- 0.4 s [Timer] =========================================== [Timer] Total: --- 2667.5 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 278, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/driver.py", line 797, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/driver.py", line 268, in _do [translation:ERROR] res = func() [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/driver.py", line 492, in task_source_c [translation:ERROR] c_source_filename = cbuilder.generate_source(database) [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/c/genc.py", line 127, in generate_source [translation:ERROR] CBuilder.have___thread = check_under_under_thread() [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/tool/cbuild.py", line 696, in check_under_under_thread [translation:ERROR] noerr=True) [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/tool/cbuild.py", line 654, in build_executable [translation:ERROR] compiler.build(noerr=noerr) [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/tool/cbuild.py", line 578, in build [translation:ERROR] self._build() [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/tool/cbuild.py", line 638, in _build [translation:ERROR] extra_preargs=compile_extra) [translation:ERROR] File "/usr/lib64/python2.5/distutils/ccompiler.py", line 701, in compile [translation:ERROR] self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts) [translation:ERROR] File "/usr/lib64/python2.5/distutils/unixccompiler.py", line 180, in _compile [translation:ERROR] extra_postargs) [translation:ERROR] File "/var/tmp/pypy-svn/dist/pypy/translator/tool/cbuild.py", line 482, in spawn_and_log [translation:ERROR] return spawn(cmd, *args, **kwds) [translation:ERROR] File "/usr/lib64/python2.5/distutils/ccompiler.py", line 1042, in spawn [translation:ERROR] spawn (cmd, dry_run=self.dry_run) [translation:ERROR] File "/usr/lib64/python2.5/distutils/spawn.py", line 37, in spawn [translation:ERROR] _spawn_posix(cmd, search_path, dry_run=dry_run) [translation:ERROR] File "/usr/lib64/python2.5/distutils/spawn.py", line 127, in _spawn_posix [translation:ERROR] pid = os.fork() [translation:ERROR] OSError: [Errno 12] Cannot allocate memory [translation] start debugger...
/usr/lib64/python2.5/distutils/spawn.py(127)_spawn_posix() -> pid = os.fork()
Hi, On Thu, Aug 28, 2008 at 09:13:58PM +0600, ???????????????? ???????????? wrote:
Tried to play with pypy, but translating to c failes because of lack of memory.
This is a known issue. We will try to implement a workaround. For now you need to have more swap available. Note that the swap will not actually be used if you have >= 2GB of RAM; it is really a limitation of some Linux configurations in which a process that consumes about half of the total amount of memory+swap cannot spawn new processes because os.fork() fails. A bientot, Armin.
Is there a direction, where I could digg to help? Something to begin with. I don't have enough experience but I'd like to take part (even very small) in this project. Armin Rigo wrote:
Hi,
On Thu, Aug 28, 2008 at 09:13:58PM +0600, ???????????????? ???????????? wrote:
Tried to play with pypy, but translating to c failes because of lack of memory.
This is a known issue. We will try to implement a workaround. For now you need to have more swap available. Note that the swap will not actually be used if you have>= 2GB of RAM; it is really a limitation of some Linux configurations in which a process that consumes about half of the total amount of memory+swap cannot spawn new processes because os.fork() fails.
A bientot,
Armin.
-- --- С уважением, Ветошкин Никита Инженер поддержки систем IP-телефонии Сервисный центр Компания Naumen
Hi, On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
Is there a direction, where I could digg to help? Something to begin with. I don't have enough experience but I'd like to take part (even very small) in this project.
Sure. Do you understand the issue? It's related to the way a Unix process starts a new subprocess: os.fork() followed by os.execv() in the child process. The issue is that os.fork() fails if the parent process' total memory is more than half of your RAM+swap, because it tries to create a copy of the process, and the copy would also need more than half of your RAM+swap. (In reality, os.fork() doesn't copy the memory at all, but creates "shared pages" of memory so that memory pages are duplicated only when one of the two processes really modifies it; but still os.fork() has to mark all the memory as *potentially* used, and raises MemoryError if there isn't enough.) A solution might be to split translate.py in two processes: translate.py would start a subprocess when it starts, and then do all the work (consuming a lot of RAM); but when it needs to start new processes, e.g. call gcc, it would instead ask the subprocess to start the new processes. More precisely, the code in pypy.translator.tool.cbuild would stop using distutils directly, and instead ask the subprocess to use distutils. It looks even easy to do with the help of a py.execnet.PopenGateway() created when translate.py starts (see the py lib documentation at http://codespeak.net/py/). A bientot, Armin
Thanks a lot, Armin. I know how os.fork() works and about copy-on-write too, so as soon as I got time, I'll try to fix it. My interest in translating and playing then with pypy was exactly py.execnet =) Armin Rigo wrote:
Hi,
On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
Is there a direction, where I could digg to help? Something to begin with. I don't have enough experience but I'd like to take part (even very small) in this project.
Sure. Do you understand the issue? It's related to the way a Unix process starts a new subprocess: os.fork() followed by os.execv() in the child process. The issue is that os.fork() fails if the parent process' total memory is more than half of your RAM+swap, because it tries to create a copy of the process, and the copy would also need more than half of your RAM+swap. (In reality, os.fork() doesn't copy the memory at all, but creates "shared pages" of memory so that memory pages are duplicated only when one of the two processes really modifies it; but still os.fork() has to mark all the memory as *potentially* used, and raises MemoryError if there isn't enough.)
A solution might be to split translate.py in two processes: translate.py would start a subprocess when it starts, and then do all the work (consuming a lot of RAM); but when it needs to start new processes, e.g. call gcc, it would instead ask the subprocess to start the new processes. More precisely, the code in pypy.translator.tool.cbuild would stop using distutils directly, and instead ask the subprocess to use distutils. It looks even easy to do with the help of a py.execnet.PopenGateway() created when translate.py starts (see the py lib documentation at http://codespeak.net/py/).
A bientot,
Armin
-- --- С уважением, Ветошкин Никита Инженер поддержки систем IP-телефонии Сервисный центр Компания Naumen
Playing with py.execnet failes too. Full traceback attached. P.S. Seens like a hard start with my trials in PyPy =) Armin Rigo wrote:
Hi,
On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
Is there a direction, where I could digg to help? Something to begin with. I don't have enough experience but I'd like to take part (even very small) in this project.
Sure. Do you understand the issue? It's related to the way a Unix process starts a new subprocess: os.fork() followed by os.execv() in the child process. The issue is that os.fork() fails if the parent process' total memory is more than half of your RAM+swap, because it tries to create a copy of the process, and the copy would also need more than half of your RAM+swap. (In reality, os.fork() doesn't copy the memory at all, but creates "shared pages" of memory so that memory pages are duplicated only when one of the two processes really modifies it; but still os.fork() has to mark all the memory as *potentially* used, and raises MemoryError if there isn't enough.)
A solution might be to split translate.py in two processes: translate.py would start a subprocess when it starts, and then do all the work (consuming a lot of RAM); but when it needs to start new processes, e.g. call gcc, it would instead ask the subprocess to start the new processes. More precisely, the code in pypy.translator.tool.cbuild would stop using distutils directly, and instead ask the subprocess to use distutils. It looks even easy to do with the help of a py.execnet.PopenGateway() created when translate.py starts (see the py lib documentation at http://codespeak.net/py/).
A bientot,
Armin
gw = py.execnet.PopenGateway() faking <type '_socket.socket'> faking <class 'socket.timeout'> Traceback (most recent call last): File "./pypy/bin/py.py", line 152, in <module> sys.exit(main_(sys.argv)) File "./pypy/bin/py.py", line 138, in main_ con.interact(banner) File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 133, in interact code.InteractiveConsole.interact(self, banner) File "/usr/lib64/python2.5/code.py", line 239, in interact more = self.push(line) File "/usr/lib64/python2.5/code.py", line 261, in push more = self.runsource(source, self.filename) File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 192, in runsource main.run_toplevel(self.space, doit, verbose=self.verbose) File "/var/tmp/pypy-svn/dist/pypy/interpreter/main.py", line 103, in run_toplevel f() File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 184, in doit code.exec_code(self.space, self.w_globals, self.w_globals) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 695, in LOAD_ATTR w_value = f.space.getattr(w_obj, w_attributename) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, in getattr return DescrOperation.getattr(self, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 134, in getattr return space._handle_getattribute(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 147, in _handle_getattribute return space.get_and_call_function(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 89, in get_and_call_function return descr.funccall(w_obj, *args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 56, in funccall return code.fastcall_2(self.space, self, args_w[0], args_w[1]) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 88, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, in fastcall_4 w_result = self.fastfunc_4(space, w1, w2, w3, w4) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in fastfunc_importhook_4 File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
-- Nikita line 180, in importhook w_mod = absolute_import(space, modulename, 0, w_fromlist, tentative=0) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 192, in absolute_import w_fromlist, tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 211, in _absolute_import tentative=tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 272, in load_part w(partname)) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 90, in try_import_mod magic, timestamp, stream.readall()) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 499, in load_compiled_module code_w.exec_code(space, w_dic, w_dic) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 760, in IMPORT_NAME w_locals, w_fromlist) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 712, in call_function return w_func.funccall(*args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 62, in funccall args_w[1], args_w[2], args_w[3]) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, in fastcall_4 w_result = self.fastfunc_4(space, w1, w2, w3, w4) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in fastfunc_importhook_4 File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 180, in importhook w_mod = absolute_import(space, modulename, 0, w_fromlist, tentative=0) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 192, in absolute_import w_fromlist, tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 211, in _absolute_import tentative=tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 272, in load_part w(partname)) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 90, in try_import_mod magic, timestamp, stream.readall()) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 499, in load_compiled_module code_w.exec_code(space, w_dic, w_dic) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 769, in IMPORT_STAR import_all_from(f.space, w_module, w_locals) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 839, in appcaller return space.call_args(w_func, args) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 98, in call_args return w_obj.call_args(args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, in call_args return self.code.funcrun(self, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 483, in funcrun return BuiltinCode.funcrun_obj(self, func, None, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 491, in funcrun_obj w_result = activation._run(space, scope_w) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in _run_UWS_ObjSpace_W_Root_W_Root File "/var/tmp/pypy-svn/dist/pypy/_cache/pyopcode_4922ecf55c7db5d81d0c4a63f92bdd44.py", line 59, in import_all_from w_2 = space.getattr(w_module, gs___dict__) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, in getattr return DescrOperation.getattr(self, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 134, in getattr return space._handle_getattribute(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 140, in _handle_getattribute return space.get_and_call_function(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 89, in get_and_call_function return descr.funccall(w_obj, *args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 69, in funccall return self.call_args(Arguments(self.space, list(args_w))) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, in call_args return self.code.funcrun(self, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 61, in funcrun return frame.run() File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 171, in run return self.space.wrap(result) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 475, in wrap items_w = [(self.wrap(k), self.wrap(v)) for (k, v) in x.iteritems()] File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 522, in wrap return fake_object(self, x) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 20, in fake_object ft = fake_type(x) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 58, in fake_type faked_type = really_build_fake_type(cpy_type) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 108, in really_build_fake_type assert cpy_type.__base__ is basestring AssertionError
I'm not completele sure what you're trying to do. Can you explain in a bit more detail? Also feel free to drop by on IRC for a live discussion. cheers, fijal On Wed, Sep 3, 2008 at 9:43 PM, Vetoshkin Nikita <nvetoshkin@naumen.ru> wrote:
Playing with py.execnet failes too. Full traceback attached.
P.S. Seens like a hard start with my trials in PyPy =)
Armin Rigo wrote:
Hi,
On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
Is there a direction, where I could digg to help? Something to begin with. I don't have enough experience but I'd like to take part (even very small) in this project.
Sure. Do you understand the issue? It's related to the way a Unix process starts a new subprocess: os.fork() followed by os.execv() in the child process. The issue is that os.fork() fails if the parent process' total memory is more than half of your RAM+swap, because it tries to create a copy of the process, and the copy would also need more than half of your RAM+swap. (In reality, os.fork() doesn't copy the memory at all, but creates "shared pages" of memory so that memory pages are duplicated only when one of the two processes really modifies it; but still os.fork() has to mark all the memory as *potentially* used, and raises MemoryError if there isn't enough.)
A solution might be to split translate.py in two processes: translate.py would start a subprocess when it starts, and then do all the work (consuming a lot of RAM); but when it needs to start new processes, e.g. call gcc, it would instead ask the subprocess to start the new processes. More precisely, the code in pypy.translator.tool.cbuild would stop using distutils directly, and instead ask the subprocess to use distutils. It looks even easy to do with the help of a py.execnet.PopenGateway() created when translate.py starts (see the py lib documentation at http://codespeak.net/py/).
A bientot,
Armin
-- Nikita
gw = py.execnet.PopenGateway() faking <type '_socket.socket'> faking <class 'socket.timeout'> Traceback (most recent call last): File "./pypy/bin/py.py", line 152, in <module> sys.exit(main_(sys.argv)) File "./pypy/bin/py.py", line 138, in main_ con.interact(banner) File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 133, in interact code.InteractiveConsole.interact(self, banner) File "/usr/lib64/python2.5/code.py", line 239, in interact more = self.push(line) File "/usr/lib64/python2.5/code.py", line 261, in push more = self.runsource(source, self.filename) File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 192, in runsource main.run_toplevel(self.space, doit, verbose=self.verbose) File "/var/tmp/pypy-svn/dist/pypy/interpreter/main.py", line 103, in run_toplevel f() File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 184, in doit code.exec_code(self.space, self.w_globals, self.w_globals) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 695, in LOAD_ATTR w_value = f.space.getattr(w_obj, w_attributename) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, in getattr return DescrOperation.getattr(self, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 134, in getattr return space._handle_getattribute(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 147, in _handle_getattribute return space.get_and_call_function(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 89, in get_and_call_function return descr.funccall(w_obj, *args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 56, in funccall return code.fastcall_2(self.space, self, args_w[0], args_w[1]) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 88, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, in fastcall_4 w_result = self.fastfunc_4(space, w1, w2, w3, w4) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in fastfunc_importhook_4 File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 180, in importhook w_mod = absolute_import(space, modulename, 0, w_fromlist, tentative=0) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 192, in absolute_import w_fromlist, tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 211, in _absolute_import tentative=tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 272, in load_part w(partname)) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 90, in try_import_mod magic, timestamp, stream.readall()) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 499, in load_compiled_module code_w.exec_code(space, w_dic, w_dic) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 760, in IMPORT_NAME w_locals, w_fromlist) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 712, in call_function return w_func.funccall(*args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 62, in funccall args_w[1], args_w[2], args_w[3]) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, in fastcall_4 w_result = self.fastfunc_4(space, w1, w2, w3, w4) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in fastfunc_importhook_4 File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 180, in importhook w_mod = absolute_import(space, modulename, 0, w_fromlist, tentative=0) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 192, in absolute_import w_fromlist, tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 211, in _absolute_import tentative=tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 272, in load_part w(partname)) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 90, in try_import_mod magic, timestamp, stream.readall()) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 499, in load_compiled_module code_w.exec_code(space, w_dic, w_dic) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 769, in IMPORT_STAR import_all_from(f.space, w_module, w_locals) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 839, in appcaller return space.call_args(w_func, args) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 98, in call_args return w_obj.call_args(args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, in call_args return self.code.funcrun(self, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 483, in funcrun return BuiltinCode.funcrun_obj(self, func, None, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 491, in funcrun_obj w_result = activation._run(space, scope_w) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in _run_UWS_ObjSpace_W_Root_W_Root File "/var/tmp/pypy-svn/dist/pypy/_cache/pyopcode_4922ecf55c7db5d81d0c4a63f92bdd44.py", line 59, in import_all_from w_2 = space.getattr(w_module, gs___dict__) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, in getattr return DescrOperation.getattr(self, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 134, in getattr return space._handle_getattribute(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 140, in _handle_getattribute return space.get_and_call_function(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 89, in get_and_call_function return descr.funccall(w_obj, *args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 69, in funccall return self.call_args(Arguments(self.space, list(args_w))) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, in call_args return self.code.funcrun(self, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 61, in funcrun return frame.run() File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 171, in run return self.space.wrap(result) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 475, in wrap items_w = [(self.wrap(k), self.wrap(v)) for (k, v) in x.iteritems()] File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 522, in wrap return fake_object(self, x) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 20, in fake_object ft = fake_type(x) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 58, in fake_type faked_type = really_build_fake_type(cpy_type) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 108, in really_build_fake_type assert cpy_type.__base__ is basestring AssertionError
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Oh, sorry. Here are the details: launching pypy: ./pypy/bin/py.py then doing this: import py gw = py.execnet.PopenGateway() and getting that long long traceback. Machine: Gentoo Linux x64 Python 2.5.2 Would be glad to provide any useful information. I'm not very familiar with IRC, what is the server and channel to communicate on pypy related issues? Maciej Fijalkowski wrote:
I'm not completele sure what you're trying to do. Can you explain in a bit more detail? Also feel free to drop by on IRC for a live discussion.
cheers, fijal
On Wed, Sep 3, 2008 at 9:43 PM, Vetoshkin Nikita<nvetoshkin@naumen.ru> wrote:
Playing with py.execnet failes too. Full traceback attached.
P.S. Seens like a hard start with my trials in PyPy =)
Armin Rigo wrote:
Hi,
Is there a direction, where I could digg to help? Something to begin with. I don't have enough experience but I'd like to take part (even very small) in this project. Sure. Do you understand the issue? It's related to the way a Unix
On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote: process starts a new subprocess: os.fork() followed by os.execv() in the child process. The issue is that os.fork() fails if the parent process' total memory is more than half of your RAM+swap, because it tries to create a copy of the process, and the copy would also need more than half of your RAM+swap. (In reality, os.fork() doesn't copy the memory at all, but creates "shared pages" of memory so that memory pages are duplicated only when one of the two processes really modifies it; but still os.fork() has to mark all the memory as *potentially* used, and raises MemoryError if there isn't enough.)
A solution might be to split translate.py in two processes: translate.py would start a subprocess when it starts, and then do all the work (consuming a lot of RAM); but when it needs to start new processes, e.g. call gcc, it would instead ask the subprocess to start the new processes. More precisely, the code in pypy.translator.tool.cbuild would stop using distutils directly, and instead ask the subprocess to use distutils. It looks even easy to do with the help of a py.execnet.PopenGateway() created when translate.py starts (see the py lib documentation at http://codespeak.net/py/).
A bientot,
Armin -- Nikita
gw = py.execnet.PopenGateway() faking<type '_socket.socket'> faking<class 'socket.timeout'> Traceback (most recent call last): File "./pypy/bin/py.py", line 152, in<module> sys.exit(main_(sys.argv)) File "./pypy/bin/py.py", line 138, in main_ con.interact(banner) File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 133, in interact code.InteractiveConsole.interact(self, banner) File "/usr/lib64/python2.5/code.py", line 239, in interact more = self.push(line) File "/usr/lib64/python2.5/code.py", line 261, in push more = self.runsource(source, self.filename) File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 192, in runsource main.run_toplevel(self.space, doit, verbose=self.verbose) File "/var/tmp/pypy-svn/dist/pypy/interpreter/main.py", line 103, in run_toplevel f() File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 184, in doit code.exec_code(self.space, self.w_globals, self.w_globals) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 695, in LOAD_ATTR w_value = f.space.getattr(w_obj, w_attributename) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, in getattr return DescrOperation.getattr(self, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 134, in getattr return space._handle_getattribute(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 147, in _handle_getattribute return space.get_and_call_function(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 89, in get_and_call_function return descr.funccall(w_obj, *args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 56, in funccall return code.fastcall_2(self.space, self, args_w[0], args_w[1]) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 88, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, in fastcall_4 w_result = self.fastfunc_4(space, w1, w2, w3, w4) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in fastfunc_importhook_4 File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 180, in importhook w_mod = absolute_import(space, modulename, 0, w_fromlist, tentative=0) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 192, in absolute_import w_fromlist, tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 211, in _absolute_import tentative=tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 272, in load_part w(partname)) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 90, in try_import_mod magic, timestamp, stream.readall()) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 499, in load_compiled_module code_w.exec_code(space, w_dic, w_dic) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 760, in IMPORT_NAME w_locals, w_fromlist) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 712, in call_function return w_func.funccall(*args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 62, in funccall args_w[1], args_w[2], args_w[3]) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, in fastcall_4 w_result = self.fastfunc_4(space, w1, w2, w3, w4) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in fastfunc_importhook_4 File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 180, in importhook w_mod = absolute_import(space, modulename, 0, w_fromlist, tentative=0) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 192, in absolute_import w_fromlist, tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 211, in _absolute_import tentative=tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 272, in load_part w(partname)) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 90, in try_import_mod magic, timestamp, stream.readall()) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 499, in load_compiled_module code_w.exec_code(space, w_dic, w_dic) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 769, in IMPORT_STAR import_all_from(f.space, w_module, w_locals) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 839, in appcaller return space.call_args(w_func, args) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 98, in call_args return w_obj.call_args(args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, in call_args return self.code.funcrun(self, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 483, in funcrun return BuiltinCode.funcrun_obj(self, func, None, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 491, in funcrun_obj w_result = activation._run(space, scope_w) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in _run_UWS_ObjSpace_W_Root_W_Root File "/var/tmp/pypy-svn/dist/pypy/_cache/pyopcode_4922ecf55c7db5d81d0c4a63f92bdd44.py", line 59, in import_all_from w_2 = space.getattr(w_module, gs___dict__) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, in getattr return DescrOperation.getattr(self, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 134, in getattr return space._handle_getattribute(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 140, in _handle_getattribute return space.get_and_call_function(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 89, in get_and_call_function return descr.funccall(w_obj, *args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 69, in funccall return self.call_args(Arguments(self.space, list(args_w))) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, in call_args return self.code.funcrun(self, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 61, in funcrun return frame.run() File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 171, in run return self.space.wrap(result) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 475, in wrap items_w = [(self.wrap(k), self.wrap(v)) for (k, v) in x.iteritems()] File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 522, in wrap return fake_object(self, x) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 20, in fake_object ft = fake_type(x) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 58, in fake_type faked_type = really_build_fake_type(cpy_type) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 108, in really_build_fake_type assert cpy_type.__base__ is basestring AssertionError
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
-- Nikita
Hi Vetoshkin, i think for translate.py you rather need to run py.execnet over CPython or through a translated pypy-c - both of which should work fine. I think you ran it through py.py which means starting a PyPy interpreter on top of CPython. Then running py.execnet.PopenGateway() would need to create another process that also runs through py.py - would be very slow - not sure it's worth trying to make it work. So i suggest you continue your experiments with plain CPython or the translated pypy-c. best & thanks, holger On Thu, Sep 04, 2008 at 01:43 +0600, Vetoshkin Nikita wrote:
Playing with py.execnet failes too. Full traceback attached.
P.S. Seens like a hard start with my trials in PyPy =)
Armin Rigo wrote:
Hi,
On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
Is there a direction, where I could digg to help? Something to begin with. I don't have enough experience but I'd like to take part (even very small) in this project.
Sure. Do you understand the issue? It's related to the way a Unix process starts a new subprocess: os.fork() followed by os.execv() in the child process. The issue is that os.fork() fails if the parent process' total memory is more than half of your RAM+swap, because it tries to create a copy of the process, and the copy would also need more than half of your RAM+swap. (In reality, os.fork() doesn't copy the memory at all, but creates "shared pages" of memory so that memory pages are duplicated only when one of the two processes really modifies it; but still os.fork() has to mark all the memory as *potentially* used, and raises MemoryError if there isn't enough.)
A solution might be to split translate.py in two processes: translate.py would start a subprocess when it starts, and then do all the work (consuming a lot of RAM); but when it needs to start new processes, e.g. call gcc, it would instead ask the subprocess to start the new processes. More precisely, the code in pypy.translator.tool.cbuild would stop using distutils directly, and instead ask the subprocess to use distutils. It looks even easy to do with the help of a py.execnet.PopenGateway() created when translate.py starts (see the py lib documentation at http://codespeak.net/py/).
A bientot,
Armin
-- Nikita
gw = py.execnet.PopenGateway() faking <type '_socket.socket'> faking <class 'socket.timeout'> Traceback (most recent call last): File "./pypy/bin/py.py", line 152, in <module> sys.exit(main_(sys.argv)) File "./pypy/bin/py.py", line 138, in main_ con.interact(banner) File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 133, in interact code.InteractiveConsole.interact(self, banner) File "/usr/lib64/python2.5/code.py", line 239, in interact more = self.push(line) File "/usr/lib64/python2.5/code.py", line 261, in push more = self.runsource(source, self.filename) File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 192, in runsource main.run_toplevel(self.space, doit, verbose=self.verbose) File "/var/tmp/pypy-svn/dist/pypy/interpreter/main.py", line 103, in run_toplevel f() File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 184, in doit code.exec_code(self.space, self.w_globals, self.w_globals) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 695, in LOAD_ATTR w_value = f.space.getattr(w_obj, w_attributename) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, in getattr return DescrOperation.getattr(self, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 134, in getattr return space._handle_getattribute(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 147, in _handle_getattribute return space.get_and_call_function(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 89, in get_and_call_function return descr.funccall(w_obj, *args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 56, in funccall return code.fastcall_2(self.space, self, args_w[0], args_w[1]) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in fastcall_2 return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, in CALL_FUNCTION w_result = f.space.call_valuestack(w_function, nargs, f) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 736, in call_valuestack return w_func.funccall_valuestack(nargs, frame) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 88, in funccall_valuestack frame.peekvalue(0)) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, in fastcall_4 w_result = self.fastfunc_4(space, w1, w2, w3, w4) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in fastfunc_importhook_4 File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 180, in importhook w_mod = absolute_import(space, modulename, 0, w_fromlist, tentative=0) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 192, in absolute_import w_fromlist, tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 211, in _absolute_import tentative=tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 272, in load_part w(partname)) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 90, in try_import_mod magic, timestamp, stream.readall()) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 499, in load_compiled_module code_w.exec_code(space, w_dic, w_dic) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 760, in IMPORT_NAME w_locals, w_fromlist) File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 712, in call_function return w_func.funccall(*args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 62, in funccall args_w[1], args_w[2], args_w[3]) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, in fastcall_4 w_result = self.fastfunc_4(space, w1, w2, w3, w4) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in fastfunc_importhook_4 File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 180, in importhook w_mod = absolute_import(space, modulename, 0, w_fromlist, tentative=0) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 192, in absolute_import w_fromlist, tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 211, in _absolute_import tentative=tentative) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 272, in load_part w(partname)) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 90, in try_import_mod magic, timestamp, stream.readall()) File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", line 499, in load_compiled_module code_w.exec_code(space, w_dic, w_dic) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in exec_code return frame.run() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in run return self.execute_frame() File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, in execute_frame executioncontext) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, in dispatch next_instr = self.handle_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, in handle_bytecode next_instr = self.dispatch_bytecode(co_code, next_instr, ec) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, in dispatch_bytecode res = getattr(self, methodname)(oparg, next_instr) File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 769, in IMPORT_STAR import_all_from(f.space, w_module, w_locals) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 839, in appcaller return space.call_args(w_func, args) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 98, in call_args return w_obj.call_args(args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, in call_args return self.code.funcrun(self, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 483, in funcrun return BuiltinCode.funcrun_obj(self, func, None, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 491, in funcrun_obj w_result = activation._run(space, scope_w) File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in _run_UWS_ObjSpace_W_Root_W_Root File "/var/tmp/pypy-svn/dist/pypy/_cache/pyopcode_4922ecf55c7db5d81d0c4a63f92bdd44.py", line 59, in import_all_from w_2 = space.getattr(w_module, gs___dict__) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, in getattr return DescrOperation.getattr(self, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 134, in getattr return space._handle_getattribute(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 140, in _handle_getattribute return space.get_and_call_function(w_descr, w_obj, w_name) File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 89, in get_and_call_function return descr.funccall(w_obj, *args_w) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 69, in funccall return self.call_args(Arguments(self.space, list(args_w))) File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, in call_args return self.code.funcrun(self, args) File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 61, in funcrun return frame.run() File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 171, in run return self.space.wrap(result) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 475, in wrap items_w = [(self.wrap(k), self.wrap(v)) for (k, v) in x.iteritems()] File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 522, in wrap return fake_object(self, x) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 20, in fake_object ft = fake_type(x) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 58, in fake_type faked_type = really_build_fake_type(cpy_type) File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 108, in really_build_fake_type assert cpy_type.__base__ is basestring AssertionError
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
-- collaborative expert contracting: http://merlinux.eu PyPy Python/Compiler tool chain: http://codespeak.net/pypy pylib py.test/greenlets/svn APIs: http://pylib.org
Hi Holger, 2008/9/4 holger krekel <holger@merlinux.eu>:
i think for translate.py you rather need to run py.execnet over CPython or through a translated pypy-c - both of which should work fine.
I think you ran it through py.py which means starting a PyPy interpreter on top of CPython. Then running py.execnet.PopenGateway() would need to create another process that also runs through py.py - would be very slow - not sure it's worth trying to make it work.
So i suggest you continue your experiments with plain CPython or the translated pypy-c.
Well, suggesting to use pypy-c isn't really helpful, since his original problem was that PyPy doesn't translate :-). Cheers, Carl Friedrich
On Thu, Sep 04, 2008 at 09:43 +0200, Carl Friedrich Bolz wrote:
Hi Holger,
2008/9/4 holger krekel <holger@merlinux.eu>:
i think for translate.py you rather need to run py.execnet over CPython or through a translated pypy-c - both of which should work fine.
I think you ran it through py.py which means starting a PyPy interpreter on top of CPython. Then running py.execnet.PopenGateway() would need to create another process that also runs through py.py - would be very slow - not sure it's worth trying to make it work.
So i suggest you continue your experiments with plain CPython or the translated pypy-c.
Well, suggesting to use pypy-c isn't really helpful, since his original problem was that PyPy doesn't translate :-).
good point - actually i just wanted to make the point that the translation tool chain (and py.execnet) should work fine on top of pypy-c, even if py.py fails. FWIW, I didn't see the other answers before i posted - using mutt over a high latency line i no fun, btw. holger
participants (6)
-
Armin Rigo -
Carl Friedrich Bolz -
holger krekel -
Maciej Fijalkowski -
Vetoshkin Nikita -
Ветошкин Никита