memory usage

Duncan Booth duncan at NOSPAMrcp.co.uk
Wed May 7 07:53:30 EDT 2003


Nagy Gabor <linux42 at freemail.c3.hu> wrote in 
news:mailman.1052303970.11712.python-list at python.org:

> What I don't understand is this:
>>>> fields = []
>>>> class A:
> ...     pass
>>>> for i in range( 1, 600000):
> ...     fields.append( A() )
> 
> When ran, the python2.2 process takes 116M of virtual memory. When
> started, it took about 2M. When I appended i, the process took 11M.
> So it seems when the element is a number, it takes roughly 20bytes, if it
> is an object, it takes roughly 200bytes.
> 
> __slots__ seems to be a great step, the above example with new class and
> with slots took only 29M.

When you create an instance of a class, then unless you specify a __slots__ 
attribute in the class definition, a dictionary is automatically created to 
hold the instance attributes.

An empty dictionary contains:

    the standard heading for any Python object: refcount and type
    count of filled slots, count of used slots
    an integer mask
    a pointer to the data
    a pointer to the lookup function
    and finally the actual data when the dictionary is small

Assuming all the simple fields are the same size, that is 7 words plus an 8 
element array. Each element of the array contains 3 words. Unfortunately, 
you may also get 1 word padding between the lookup function pointer and the 
data array, and also 1 word padding for each element of the data. 

If the compiler decided to pad all the structures an empty dictionary 
occupies 160 bytes, even though only 124 of them are used. In your case the 
padding alone would lose you 21Mb. So far as I can see the windows version 
of Python has this padding, I don't know about other implementations. It 
might be interesting to see whether removing the padding gave a net benefit 
or hit on performance.

-- 
Duncan Booth                                             duncan at rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?




More information about the Python-list mailing list