Translating pypy-sandbox eats 13GB of RAM
![](https://secure.gravatar.com/avatar/149e1b2be0d1bb91ef7f85e90e5cc603.jpg?s=120&d=mm&r=g)
Hi While trying to translate pypy-1.7 with the sandbox option, using pypy-1.6 I saw the translating pypy consume 13.3GB of memory (column Res in top). This is under FreeBSD-9RC2/amd64. I've never noticed this memory consumtion when translating pypy-1.6 with the stackless option. Is this perhaps a bug with pypy-1.6 or the source code for pypy-1.7? I'll try again using pypy-1.7 to do the translating once it has been translated (I needed to restart since I was running the translations in parallel). Regards
![](https://secure.gravatar.com/avatar/149e1b2be0d1bb91ef7f85e90e5cc603.jpg?s=120&d=mm&r=g)
On Monday, 21 November 2011 18:30:31 David Naylor wrote:
Hi
While trying to translate pypy-1.7 with the sandbox option, using pypy-1.6 I saw the translating pypy consume 13.3GB of memory (column Res in top). This is under FreeBSD-9RC2/amd64.
This isn't specific to the sandbox option. The default options used also cause this memory usage. On a box with 16GB of RAM the translation was causing extensive swapping (I estimate a peak memory of ~15GB). This seems to happen during jitcodewriter.
Is this perhaps a bug with pypy-1.6 or the source code for pypy-1.7? I'll try again using pypy-1.7 to do the translating once it has been translated (I needed to restart since I was running the translations in parallel).
I will also run a comparitive build with python-2.7 and report back...
![](https://secure.gravatar.com/avatar/5b37e6b4ac97453e4ba9dba37954cf79.jpg?s=120&d=mm&r=g)
Hi, On Mon, Nov 21, 2011 at 17:55, David Naylor <naylor.b.david@gmail.com> wrote:
While trying to translate pypy-1.7 with the sandbox option, using pypy-1.6 I saw the translating pypy consume 13.3GB of memory (column Res in top). This is under FreeBSD-9RC2/amd64.
This isn't specific to the sandbox option. The default options used also cause this memory usage.
PyPy used to contain a bug that causes infinite memory usage in one particular circumstance (literally -- so in practice, infinite swapping with no progress). It so happens that since about one month, translating a PyPy may trigger it. This means that in order to translate a PyPy, you need to run (either CPython or) a version of PyPy that contains the fix, which was done November 11. A bientôt, Armin.
![](https://secure.gravatar.com/avatar/83c1cb58fae2faedb80305a0798383d1.jpg?s=120&d=mm&r=g)
On 11/21/11, David Naylor <naylor.b.david@gmail.com> wrote:
Hi
While trying to translate pypy-1.7 with the sandbox option, using pypy-1.6 I saw the translating pypy consume 13.3GB of memory (column Res in top). This is under FreeBSD-9RC2/amd64.
I also get a lot of memory use when building pypy 1.6 on a 64 bit Debian system with gcc 4.4.5. The machine has 3 Gigabytes of physmem. I let one build attempt run for 2 or 3 days before killing it. The same system built pypy OK with a 32 bit OS, though they were builds of nearby pypy trunk's, not 1.6.
I've never noticed this memory consumtion when translating pypy-1.6 with the stackless option.
Is this perhaps a bug with pypy-1.6 or the source code for pypy-1.7? I'll try again using pypy-1.7 to do the translating once it has been translated (I needed to restart since I was running the translations in parallel).
Regards
![](https://secure.gravatar.com/avatar/bfc96d2a02d9113edb992eb96c205c5a.jpg?s=120&d=mm&r=g)
On Tue, Nov 22, 2011 at 3:07 AM, Dan Stromberg <drsalists@gmail.com> wrote:
On 11/21/11, David Naylor <naylor.b.david@gmail.com> wrote:
Hi
While trying to translate pypy-1.7 with the sandbox option, using pypy-1.6 I saw the translating pypy consume 13.3GB of memory (column Res in top). This is under FreeBSD-9RC2/amd64.
I also get a lot of memory use when building pypy 1.6 on a 64 bit Debian system with gcc 4.4.5.
The machine has 3 Gigabytes of physmem. I let one build attempt run for 2 or 3 days before killing it.
The same system built pypy OK with a 32 bit OS, though they were builds of nearby pypy trunk's, not 1.6.
3G is borderline for 64bit. I build pypy on 3G, but it requires closing browsers etc.
I've never noticed this memory consumtion when translating pypy-1.6 with the stackless option.
Is this perhaps a bug with pypy-1.6 or the source code for pypy-1.7? I'll try again using pypy-1.7 to do the translating once it has been translated (I needed to restart since I was running the translations in parallel).
Regards
_______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
participants (4)
-
Armin Rigo
-
Dan Stromberg
-
David Naylor
-
Maciej Fijalkowski