[Python-Dev] Multigigabyte memory usage in the OpenIndiana Buildbot

Jesus Cea jcea at jcea.es
Thu Sep 8 03:12:33 CEST 2011


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/09/11 14:32, Antoine Pitrou wrote:
> If "-j" is used, tests get run in a separate process each, so that
>  approach might be an answer.

Antoine, I think this would be the answer. Each test would be a bit
slower, because I would launch a new python process per test, but I
could run 16 tests in parallel (I have 16 CPUs and, actually, most
tests are not CPU intensive). I sorry to bother you with these details
and waste of time, but could you possibly change my buildbot
configurarion to launch, let's say, 4 test processes in parallel, just
for testing?.

Another option would be to have a single Python process and "fork" for
each test. That would launch each test in a separate process without
requiring a full python interpreter launching each time. Is this the
way "-j" is implemented, or is "-j" something external, like "make -j"?.

BTW, the (nice and helpful) OpenIndiana folks have told me a few hours
ago that they would increase my swap limit to 16GB. I am now waiting
for this change to be done.

I want my six builds in parallel (2.7, 3.2, 3.x, in 32 and 64 bits) back!.

Sorry for wasting your time with these mundane details...

- -- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBTmgWgZlgi5GaxT1NAQI/eAP/anenlTjt7NxIzMLK+ME+f84zLurb8MS/
XiLRpVSNDn6TzKnqXtDLfOc6sua81h+ZlpHvuFNHOkK9u/PkmeUKidgoDvASj5Ti
ITUmUxigX1j9ZbD1ITkn53msm1xfug3rw/8+Rh//4ONhhbmhSm8ChZ0iNwtntToG
5SwL3BL2iSI=
=fCJe
-----END PGP SIGNATURE-----


More information about the Python-Dev mailing list