[pypy-dev] translate.py failes
nvetoshkin at naumen.ru
Tue Sep 2 20:40:58 CEST 2008
Thanks a lot, Armin.
I know how os.fork() works and about copy-on-write too, so as soon as I
got time, I'll try to fix it.
My interest in translating and playing then with pypy was exactly
Armin Rigo wrote:
> On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
>> Is there a direction, where I could digg to help? Something to begin with.
>> I don't have enough experience but I'd like to take part (even very
>> small) in this project.
> Sure. Do you understand the issue? It's related to the way a Unix
> process starts a new subprocess: os.fork() followed by os.execv() in the
> child process. The issue is that os.fork() fails if the parent process'
> total memory is more than half of your RAM+swap, because it tries to
> create a copy of the process, and the copy would also need more than
> half of your RAM+swap. (In reality, os.fork() doesn't copy the memory
> at all, but creates "shared pages" of memory so that memory pages are
> duplicated only when one of the two processes really modifies it; but
> still os.fork() has to mark all the memory as *potentially* used, and
> raises MemoryError if there isn't enough.)
> A solution might be to split translate.py in two processes: translate.py
> would start a subprocess when it starts, and then do all the work
> (consuming a lot of RAM); but when it needs to start new processes, e.g.
> call gcc, it would instead ask the subprocess to start the new
> processes. More precisely, the code in pypy.translator.tool.cbuild
> would stop using distutils directly, and instead ask the subprocess to
> use distutils. It looks even easy to do with the help of a
> py.execnet.PopenGateway() created when translate.py starts (see the py
> lib documentation at http://codespeak.net/py/).
> A bientot,
Инженер поддержки систем IP-телефонии
More information about the Pypy-dev