On 07.12.2009 10:48, Stefan Behnel wrote:
Armin Rigo, 05.12.2009 16:44:
On Fri, Dec 04, 2009 at 06:18:13PM +0100, Antonio Cuni wrote:
I agree that at this point in time we cannot or don't want to make annotation/rtyping/backend parallelizable, but it should definitely be possible to just pass the -j flag to 'make' in an automatic way.
Of course, that is full of open problems too. The main one is that each gcc process consumes potentially a lot of RAM, so just passing "-j" is not a great idea, as all gccs are started in parallel. It looks like some obscure tweak is needed, like setting -j to a number that depends not only on the number of CPUs (as is classically done) but also on the total RAM of the system...
I just did a quick check with lxml.etree, for which Cython generates a 6.5MB C file with 150K lines (~96K non-empty/non-'#' lines in gcc -E). Running that through "gcc -O3 -march=core2 -pipe" keeps the peek virtual memory allocation in 'top' well below 350MB on my 32bit Linux system. Developer machines tend to be rather well equipped these days, so not much to worry about here, IMHO.
Stefan
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Indeed, my own tests support this. Especially with kernel 2.6.32, memory should no longer be the issue. -- Sven-Hendrik