[pypy-dev] Parallel translation?

[Sven-Hendrik Haase]

On 07.12.2009 10:48, Stefan Behnel wrote:
> Armin Rigo, 05.12.2009 16:44:
>   
>> On Fri, Dec 04, 2009 at 06:18:13PM +0100, Antonio Cuni wrote:
>>     
>>> I agree that at this point in time we cannot or don't want to make 
>>> annotation/rtyping/backend parallelizable, but it should definitely be 
>>> possible to just pass the -j flag to 'make' in an automatic way.
>>>       
>> Of course, that is full of open problems too.  The main one is that each
>> gcc process consumes potentially a lot of RAM, so just passing "-j" is
>> not a great idea, as all gccs are started in parallel.  It looks like
>> some obscure tweak is needed, like setting -j to a number that depends
>> not only on the number of CPUs (as is classically done) but also on the
>> total RAM of the system...
>>     
>
> I just did a quick check with lxml.etree, for which Cython generates a
> 6.5MB C file with 150K lines (~96K non-empty/non-'#' lines in gcc -E).
> Running that through "gcc -O3 -march=core2 -pipe" keeps the peek virtual
> memory allocation in 'top' well below 350MB on my 32bit Linux system.
> Developer machines tend to be rather well equipped these days, so not much
> to worry about here, IMHO.
>
> Stefan
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
>   
Indeed, my own tests support this. Especially with kernel 2.6.32, memory
should no longer be the issue.

-- Sven-Hendrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20091208/da61981e/attachment.html>