Skipping on memory in Python classes

[Raymond Hettinger]

"the_rev_dharma_roadkill" <doug.hendricks at tnzi.com> wrote in message
news:fb91dbec.0308061948.2e328980 at posting.google.com...
> Hi,
> I'm a Python newbie, but I have some experience in other languages.
>
> I need to create about 100,000 instances of one class.  Each instance
> has two lists, one usually empty, the other containing exactly 200
> elements which differ widely between the 100,000 instances, but about
> half of the elements in these lists will be empty strings:
>
> instance x (one of 100,000) contains:
> list A, 200 elements but half are emptyString
> list B, usually empty (None, not []), but can contain a few small
> elements
> variable X, a fairly short string.
> I also set __init__, __cmp__, and an attribute access function.
>
> Question:  How can a reduce the memory used to a minimum?
>
> I have already set __slots__ = A,B,X
> and this shaved about 10% off of the used memory, which is well worth
> it.
>
> Any other proven techniques out there?  Is there much point in
> creating a new metaclass for my class?  How about replacing
> emptyStrings with Nones?  Is there a fast (runtime) way of translating
> between '' and None?

If the list of 200 elements doesn't change, it may be better to use a
tuple instead of a list.

If the list contents are all of the same type, the array module provides
a space efficient storage solution.

Empty strings are like None in that they all refer to a single object,
so there are no savings there.


Raymond Hettinger