[pypy-dev] translate.py failes

Vetoshkin Nikita nvetoshkin at naumen.ru
Wed Sep 3 21:43:31 CEST 2008


Playing with py.execnet failes too. Full traceback attached.

P.S. Seens like a hard start with my trials in PyPy =)

Armin Rigo wrote:
> Hi,
>
> On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
>> Is there a direction, where I could digg to help? Something to begin with.
>> I don't have enough experience but I'd like to take part (even very
>> small) in this project.
>
> Sure.  Do you understand the issue?  It's related to the way a Unix
> process starts a new subprocess: os.fork() followed by os.execv() in the
> child process.  The issue is that os.fork() fails if the parent process'
> total memory is more than half of your RAM+swap, because it tries to
> create a copy of the process, and the copy would also need more than
> half of your RAM+swap.  (In reality, os.fork() doesn't copy the memory
> at all, but creates "shared pages" of memory so that memory pages are
> duplicated only when one of the two processes really modifies it; but
> still os.fork() has to mark all the memory as *potentially* used, and
> raises MemoryError if there isn't enough.)
>
> A solution might be to split translate.py in two processes: translate.py
> would start a subprocess when it starts, and then do all the work
> (consuming a lot of RAM); but when it needs to start new processes, e.g.
> call gcc, it would instead ask the subprocess to start the new
> processes.  More precisely, the code in pypy.translator.tool.cbuild
> would stop using distutils directly, and instead ask the subprocess to
> use distutils.  It looks even easy to do with the help of a
> py.execnet.PopenGateway() created when translate.py starts (see the py
> lib documentation at http://codespeak.net/py/).
>
>
> A bientot,
>
> Armin

-- 
Nikita
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: execnet_fail.txt
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20080904/54587572/attachment.txt>


More information about the Pypy-dev mailing list