[pypy-issue] [issue907] Stacklets running out of memory on higher optimisation levels

Laurence Tratt tracker at bugs.pypy.org
Thu Oct 13 18:27:44 CEST 2011


New submission from Laurence Tratt <laurie at tratt.net>:

I've finally gotten around to implementing generator support in my prototype
Converge VM. For simple examples, this was super-duper easy to get up and
running - I was very impressed! However, for more involved examples, the
stacklet approach seems to run out of memory on --opt=2, --opt=3, and
--opt=jit and then dumps out a 2G core file (which probably doesn't do my
SSD much good!). I must admit, I have yet to work out why this is happening,
as I'm fairly sure I'm destroying every stacklet that's created as soon as I
can reasonably do so, and there doesn't appear to be a problem on --opt=1 or
--opt=0.

Here's an example error with --opt=3:

  RPython traceback:
    File "implement.c", line 310, in entry_point
    File "implement.c", line 5758, in VM_apply_closure
    File "implement.c", line 10764, in VM_execute
    File "implement.c", line 13910, in VM_bc_loop
    File "implement.c", line 17090, in VM__instr_apply
    File "implement.c", line 22377, in VM__apply_pump
    File "rpython_memory_gctransform_shadowstack.c", line 416, in
ShadowStackPool_save_current_state_away
  Fatal RPython error: MemoryError

You can see the full VM code here:

  https://github.com/ltratt/converge/tree/pypyvm/pypyvm

Here's an example Converge program which causes things to blow up:

  func g():
      i := 10
      while i > 0:
          yield i
          i -= 1
      fail

  func main():
      i := 5000000
      while i > 0:
          for g()

I attach a bytecode version of that file to this bug report so that you can
execute things with this command:

  ./main-c t

For me, the blow-up (on --opt=3) happens a few iterations into the while
loop executing the "for g()" statement. The switching code is in VM.py -
there is only one stacklet creation site, and two switches, so it's
reasonably easy to see what's going on, I hope.

I will warn you that it's possible I'm doing something very stupid here.
However, because there are no errors in --opt=1, but there are in --opt=3,
it seems possible that there's something a bit squiffy going on in PyPy,
perhaps in the interaction with the garbage collector.

----------
assignedto: arigo
files: t
messages: 3299
nosy: arigo, ltratt, pypy-issue
priority: bug
status: unread
title: Stacklets running out of memory on higher optimisation levels

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue907>
________________________________________


More information about the pypy-issue mailing list