Program uses twice as much memory in Python 3.6 than in Python 3.5

Peter Otten __peter__ at web.de
Mon Mar 27 18:30:12 EDT 2017


Jan Gosmann wrote:

> Hi,
> 
> I have a program which uses twice as much memory when I run it in Python
> 3.6 than when I run it in Python 3.5 (about 60GB instead of 30GB). I
> tried to find the reason for that, but the cumulated size (measured with
> sys.getsizeof) of all objects returned by gc.get_objects accumulates
> only to about 17GB in both cases. The program also uses NumPy and I
> tried tracking allocations with the NumPy allocation tracker in the
> relevant part of the program, but again the number of allocations are
> almost identical and the reported maximum memory usage perfectly agrees
> (it is about 18GB).
> 
> Any ideas where this significant increase in memory consumption could
> come from? Or any ideas how to further debug this?
> 
> Looking at the changelog it seems that the only change in Python 3.6
> affecting memory usage is the new dict implementation which is supposed
> to be more memory efficient. In fact, this is what I find if I run the
> program with a smaller test cases. There the memory consumption is less
> with Python 3.6.

Are you perchance comparing 32-bit Python 3.5 with 64-bit Python 3.6?



More information about the Python-list mailing list