python memory use

[Bearophile]

Gary Robinson:

>(I could test the particular case I mention, but I'm wondering if someone has some fundamental knowledge that would lead to a basic understanding.)<

Java is one of the languages most avid of memory, often even more than
Python 2.x. Some bad people have said that Java developers were not
that interested in saving RAM because Sun sells hardware, and the more
RAM it uses the more they can sell ;-)

More seriously, Java uses a complex hybrid generational garbage
collectors, while CPython uses a much simpler reference count GC +
cycle detector.

A reference counter usually has a lower performance compared to good
generational garbage collectors, especially if they are hybridized
with several other algorithms, but it's simpler (and most things in
CPython design are designed for simplicity even when they are a little
less efficient, and among other things this simplicity helps this
OpenSource project recruit and keep developers), it's almost
deterministic (so for example in some situations you can forget to
close a file) so it often uses less memory because in any moment you
have only the objects you are using (reference cycles add a little
extra complexity in this). While a generational GC keeps a lot of
extra memory unused, free, etc. There are studies that show that if
you use such kind of good generational GCs and you pay about a 2-5X
memory penalty you can have a language about as fast as ones where you
manually manage memory. Indeed today good Java programs are often no
more than 2X slower than C++ and sometimes are about as fast or even
faster (thanks to other optimizations, like a strong inlining of
virtual methods done by HotSpot).

If you want a language that uses less RAM you can try FreePascal :-)

I think that among the languages designed to work with a GC, the D
language is among the ones that uses less memory (because so far its
GC is not that good, so it saves memory while being slower than the
advanced GC used by Sun Java).

On 64 bit systems Java Sun has added an optimization, certain pointers
are compressed in 32 bits, reducing memory usage. Similar things may
be done by the LLVM too in future:
http://llvm.org/pubs/2005-06-12-MSP-PointerCompSlides.pdf
Maybe someday 64-bit CPython will do the same, or maybe
UnlandenSwallow or PyPy (Jthon in a sense may do it already if you use
it on a 64 bit Java. I don't know).

Bye,
bearophile