memory usage
Duncan Booth
duncan at NOSPAMrcp.co.uk
Wed May 7 07:53:30 EDT 2003
Nagy Gabor <linux42 at freemail.c3.hu> wrote in
news:mailman.1052303970.11712.python-list at python.org:
> What I don't understand is this:
>>>> fields = []
>>>> class A:
> ... pass
>>>> for i in range( 1, 600000):
> ... fields.append( A() )
>
> When ran, the python2.2 process takes 116M of virtual memory. When
> started, it took about 2M. When I appended i, the process took 11M.
> So it seems when the element is a number, it takes roughly 20bytes, if it
> is an object, it takes roughly 200bytes.
>
> __slots__ seems to be a great step, the above example with new class and
> with slots took only 29M.
When you create an instance of a class, then unless you specify a __slots__
attribute in the class definition, a dictionary is automatically created to
hold the instance attributes.
An empty dictionary contains:
the standard heading for any Python object: refcount and type
count of filled slots, count of used slots
an integer mask
a pointer to the data
a pointer to the lookup function
and finally the actual data when the dictionary is small
Assuming all the simple fields are the same size, that is 7 words plus an 8
element array. Each element of the array contains 3 words. Unfortunately,
you may also get 1 word padding between the lookup function pointer and the
data array, and also 1 word padding for each element of the data.
If the compiler decided to pad all the structures an empty dictionary
occupies 160 bytes, even though only 124 of them are used. In your case the
padding alone would lose you 21Mb. So far as I can see the windows version
of Python has this padding, I don't know about other implementations. It
might be interesting to see whether removing the padding gave a net benefit
or hit on performance.
--
Duncan Booth duncan at rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
More information about the Python-list
mailing list