[pypy-issue] Issue #2258: Unable to translate PyPy 5.0 with JIT on a 32bit machine (pypy/pypy)

Stefano Rivera issues-reply at bitbucket.org
Mon Mar 14 21:39:32 EDT 2016


New issue 2258: Unable to translate PyPy 5.0 with JIT on a 32bit machine
https://bitbucket.org/pypy/pypy/issues/2258/unable-to-translate-pypy-50-with-jit-on-a

Stefano Rivera:

I can't translate PyPy 5.0 on an `armhf` machine (Novena with 4GiB of RAM), as I run into a MemoryError. This wasn't a problem in the past. On 32bit machines, userspace processes only get 3GB of address space, and we're using it all.

This is with cpython. I've tried using jemalloc, it didn't help.

Translating with `-O3` instead of `-Ojit` works.

The other way would be to use a 32bit arm chroot on an arm64 machine, that gives you a 4GB address space. Or possibly a 32bit ARM machine with LPAE.

Sample failure:
```
[backendopt:malloc] removed 16 simple mallocs in total
[backendopt:mergeifblocks] starting to merge if blocks
[c:database] GC transformer: finished helpers
[c]   907000 nodes  [ array: 146621  framework rtti: 2870  func: 32576  group: 1  struct: 725281 ]
[c:database] GC transformer: finished tables
[c]   908000 nodes  [ array: 147054  framework rtti: 2870  func: 32581  group: 1  struct: 727165 ]
[c]   909000 nodes  [ array: 147650  framework rtti: 2870  func: 32581  group: 1  struct: 727183 ]
[c]   910000 nodes  [ array: 147794  framework rtti: 2870  func: 32581  group: 1  struct: 727205 ]
[gctransform:info] assigned 3366 typeids
[gctransform:info] added 152628 push/pop stack root instructions
[gctransform:info] inserted 6719 write barrier calls
[gctransform:info] inserted 6537 write_barrier_from_array calls
[gctransform:info] found 3 static roots
[-26bad] translation-task}
[Timer] Timings:
[Timer] annotate                       ---  2718.8 s
[Timer] rtype_lltype                   ---  4222.4 s
[Timer] pyjitpl_lltype                 ---  6207.4 s
[Timer] backendopt_lltype              ---  1579.3 s
[Timer] stackcheckinsertion_lltype     ---  1856.9 s
[Timer] database_c                     ---  3279.4 s
[Timer] ============================================
[Timer] Total:                         --- 19864.2 s
[translation:info] Error:
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/goal/translate.py", line 318, in main
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/driver.py", line 550, in proceed
[translation:info]     result = self._execute(goals, task_skip = self._maybe_skip())
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/tool/taskengine.py", line 114, in _execute
[translation:info]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/driver.py", line 278, in _do
[translation:info]     res = func()
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/driver.py", line 432, in task_database_c
[translation:info]     database = cbuilder.build_database()
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/c/genc.py", line 159, in build_database
[translation:info]     db.complete()
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/c/database.py", line 351, in complete
[translation:info]     self.gctransformer.inline_helpers(self.all_graphs())
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/memory/gctransform/transform.py", line 130, in inline_helpers
[translation:info]     cleanup=False)
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/backendopt/inline.py", line 80, in inline_function
[translation:info]     return inliner.inline_all()
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/backendopt/inline.py", line 183, in inline_all
[translation:info]     self.inline_once(block, index_operation)
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/backendopt/inline.py", line 210, in inline_once
[translation:info]     self.do_inline(block, index_operation)
[translation:info]    File "/«BUILDDIR»/pypy-5.0+dfsg/rpython/translator/backendopt/inline.py", line 398, in do_inline
[translation:info]     splitlink = split_block(block, index_operation)
[translation:ERROR] MemoryError

```




More information about the pypy-issue mailing list