[Python-ideas] More details in MemoryError

[Steven D'Aprano]

On 22/01/13 09:12, Benjamin Peterson wrote:
> Serhiy Storchaka<storchaka at ...>  writes:
>
>>
>> I propose to add new optional attributes to MemoryError, which show how
>> many memory was required in failed allocation and how many memory was
>> used at this moment.
>
> What is this useful for?


After locking up a production machine with a foolishly large list
multiplication (I left it thrashing overnight, and 16+ hours later gave
up and power-cycled the machine), I have come to appreciate ulimit on
Linux systems. That means I often see MemoryErrors while testing.


[steve at ando ~]$ ulimit -v 20000
[steve at ando ~]$ python3.3
Python 3.3.0rc3 (default, Sep 27 2012, 18:44:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux
Type "help", "copyright", "credits" or "license" for more information.
=== startup script executed ===
py> x = [0]*1000000
py> x = [0]*123456789012  # oops what was I thinking?
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
MemoryError


For interactive use, it would be really useful in such a situation to
see how much memory was requested and how much was available. That
would allow me to roughly estimate (say) how big a list I could make
in the available memory, instead of tediously trying larger and smaller
lists.

Something like this could be used to decide whether or not to flush
unimportant in-memory caches, compact data structures, etc., or just
give up and exit.



-- 
Steven