[pypy-dev] translate.py failes

Vetoshkin Nikita nvetoshkin at naumen.ru
Tue Sep 2 20:40:58 CEST 2008

Thanks a lot, Armin.
I know how os.fork() works and about copy-on-write too, so as soon as I 
got time, I'll try to fix it.
My interest in translating and playing then with pypy was exactly 
py.execnet =)

Armin Rigo wrote:
> Hi,
> On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
>> Is there a direction, where I could digg to help? Something to begin with.
>> I don't have enough experience but I'd like to take part (even very
>> small) in this project.
> Sure.  Do you understand the issue?  It's related to the way a Unix
> process starts a new subprocess: os.fork() followed by os.execv() in the
> child process.  The issue is that os.fork() fails if the parent process'
> total memory is more than half of your RAM+swap, because it tries to
> create a copy of the process, and the copy would also need more than
> half of your RAM+swap.  (In reality, os.fork() doesn't copy the memory
> at all, but creates "shared pages" of memory so that memory pages are
> duplicated only when one of the two processes really modifies it; but
> still os.fork() has to mark all the memory as *potentially* used, and
> raises MemoryError if there isn't enough.)
> A solution might be to split translate.py in two processes: translate.py
> would start a subprocess when it starts, and then do all the work
> (consuming a lot of RAM); but when it needs to start new processes, e.g.
> call gcc, it would instead ask the subprocess to start the new
> processes.  More precisely, the code in pypy.translator.tool.cbuild
> would stop using distutils directly, and instead ask the subprocess to
> use distutils.  It looks even easy to do with the help of a
> py.execnet.PopenGateway() created when translate.py starts (see the py
> lib documentation at http://codespeak.net/py/).
> A bientot,
> Armin

С уважением,
Ветошкин Никита

Инженер поддержки систем IP-телефонии
Сервисный центр
Компания Naumen

More information about the Pypy-dev mailing list