Catching MemoryError in RPython
I cannot seem to generate code that catches OOM errors in RPython. I wrote a test like this: def test_memoryerror_due_to_oom(self): def fn(): try: lst = [] for i in range(sys.maxint): lst.append(i) except Exception: #MemoryError: return "OK" assert self.interpret(fn, []) == "OK" but the try:/except: seems to have disappeared by the time the JVM backend is invoked (it's not in the flow graphs anywhere). I assume that it is getting stripped away by a deadcode pass at some point, though I'm not sure where that might be. I haven't managed to concoct code that can fool it into keeping the exception block yet, but I'm pretty sure that if I only could, then the JVM backend would handle it correctly... :) If you comment out the test that causes this out of memory condition, then test_builtin.py *almost* passes in pypy-jvm. There are still two errors, though. :) Niko
Hi Niko, On Fri, Feb 22, 2008 at 07:54:52PM +0100, Niko Matsakis wrote:
I cannot seem to generate code that catches OOM errors in RPython.
A known issue: the flow space doesn't consider that almost any operation could raise MemoryError, and thus the except clause is never seen by it. The same issue applies to try:finally:. The workaround is to use a function call: def fn(): try: do_stuff() except MemoryError: return True else: return False def do_stuff(): ... A bientot, Armin.
Hi Armin, On Sat, Feb 23, 2008 at 13:44 +0100, Armin Rigo wrote:
On Fri, Feb 22, 2008 at 07:54:52PM +0100, Niko Matsakis wrote:
I cannot seem to generate code that catches OOM errors in RPython.
A known issue: the flow space doesn't consider that almost any operation could raise MemoryError, and thus the except clause is never seen by it. The same issue applies to try:finally:.
The workaround is to use a function call:
def fn(): try: do_stuff() except MemoryError: return True else: return False
def do_stuff(): ...
the "range" and "lst.append" in Niko's example are not considered to be function calls? Is it (also) the explicit catching of MemoryError here that is important? holger
Hi Holger, On Sat, Feb 23, 2008 at 02:33:23PM +0100, holger krekel wrote:
the "range" and "lst.append" in Niko's example are not considered to be function calls?
Good point. The range() and lst.append() are function calls, so the flow space keeps the MemoryError catching path. (That's not the case for the creation of the empty list done in the loop.) However, what occurs is that the rtyper kills these paths because rlist.py says that append() cannot raise. This could be changed in the rtyper to systematically keep the MemoryError path (not just for append() but generally). However I think that going for half-fixes like this one is not a good idea. Either we do a 100% solution that would include the flow space issue, or we keep the special rule about how we are supposed to catch MemoryError in RPython. A bientot, Armin.
participants (3)
-
Armin Rigo -
holger krekel -
Niko Matsakis