[pypy-dev] GC error

[Maciej Fijalkowski]

On Mon, Jan 16, 2012 at 8:44 PM, Rich Drewes <drewes at interstice.com> wrote:
> Hello all,
>
> Great work on pypy!  I've had good luck with pypy generally but on a program
> that loads a very large data set I am getting a GC related exception:
>
> ----
> loading reads, on record 25000000
> RPython traceback:
>  File "translator_goal_targetpypystandalone.c", line 888, in entry_point
>  File "interpreter_function.c", line 876, in funccall__star_1
>  File "interpreter_function.c", line 905, in funccall__star_1
>  File "rpython_memory_gc_minimark.c", line 2490, in
> MiniMarkGC_collect_and_reserve
>  File "rpython_memory_gc_minimark.c", line 2193, in
> MiniMarkGC_minor_collection
>  File "rpython_memory_gc_minimark.c", line 4535, in
> MiniMarkGC_collect_oldrefs_to_nursery
>  File "rpython_memory_gc_base.c", line 1761, in trace___trace_drag_out
>  File "rpython_memory_gc_minimarkpage.c", line 214, in
> ArenaCollection_malloc
>  File "rpython_memory_gc_minimarkpage.c", line 536, in
> ArenaCollection_allocate_new_page
>  File "rpython_memory_gc_minimarkpage.c", line 735, in
> ArenaCollection_allocate_new_arena
> Fatal RPython error: MemoryError
> Aborted

Hi Richard.

I don't quite know how you got the MemoryError, however using swap
with python (and pypy) is a very bad idea. Each time you have a
garbage collection cycle, it has to walk over all pages that are
addressed by the process, which means reading and writing to all the
pages that are swapped. This makes the program essentially not do
anything any more except reading and writing to the HD and as such,
you're very unlikely to achieve anything.

Cheers,
fijal