initializing mutable class attributes

[Alex Martelli]

Dan Perl <dperl at rogers.com> wrote:

> There is something with initializing mutable class attributes that I am
> struggling with.  I'll use an example to explain:
>     class Father:
>         attr1=None   # this is OK
>         attr2=[ ]        # this is wrong
>         def foo(self, data):
>             self.attr1=data
>             self.attr2.append(data)
> The initialization of attr1 is obviously OK, all instances of Father
> redefine it in the method foo.  But the initialization of attr2 is wrong
> because all the instances of Father end up sharing the same value.  Maybe
> that is desired sometimes, but usually it is just a bug.

I disagree: good Python programmers often use mutable class attributes,
and use them to good effect.  I've done my share of Python teaching,
consulting and mentoring, and I definitely do *not* think this usage "is
desired sometimes but usually it is just a bug".


> So the only solution I see to this is to initialize attr2 in __init__:
>     class Father:
>         attr1=None
>         def __init__(self):
>             self.attr2=[ ]

This is the canonical way, sure.


> This is already awkward because there is such a difference between attr1 and
> attr2.

One is a class attribute (which nobody forced you to have), the other is
an instance attribute.  If you want both to be instance attributes,
initialize both in __init__ -- that's all there is to it.  Don't use
class attributes unless there's a reason for them to be class
attributes, that seems like a good and sensible rule of thumb to me.

>  But moreover, I think this forces subclasses of Father to do
> something like this:
>     class Child (Father):
>         def __init__(self):
>             Father.__init__(self)
>             self.attr3=[ ]

Assuming an instance of Child needs to have both attributes attr2 and
attr3, yes.

> 
> I find this even more awkward because many people will forget to do it.

Forget to do what -- call the parent class's __init__?  Again, this is
rare -- not as rare as the other "usually just a bug" you opined about,
but not common.  A class normally needs __init__ for a large number of
purposes, of course, not just to assign mutable attributes to each
instance, therefore people who learn subclassing do learn to call the
parent class __init__ very early on, to avoid everything breaking.

> Clearly, this is then a more general issue with __init__, but I think it is
> accentuated by the fact that you HAVE TO HAVE __init__ in order to
> initialize attributes that are mutable.

Making __init__ more common means people are more likely to get used to
it, and thus makes the bug of not calling a parent class's __init__
rarer, not "accentuated".

> 
> Is there something I don't know here and there is a better way to do this in
> Python?  I would like to get a better solution or otherwise start a
> discussion.

There is no BETTER way, IMHO, than normal Python coding, which I believe
is quite good -- but you appear to disagree, therefore it's hard for me
to gauge what you may consider "better".  There is definitely a
DIFFERENT way -- coding a custom metaclass which will tweak the
instantiation of all classes belonging to it, in order to insert some
funky initialization of mutable attributes, and perhaps automate calls
to superclasses' __init__ methods, and so on.  Such automation is a
ticklish issue, in the general case, so let's assume initializing
mutable attributes is all we ever want to do.  Then, for example...:

import copy
class metaImu(type):
    def __call__(cls, *a, **k):
        instance = type.__call__(cls, *a, **k)
        for sup in cls.mro():
            muts = gerattr(sup, '__mutables__', ())
            for mutname in muts:
                if hasattr(instance, mutname): continue
                mutvalue = muts[mutname]
                setattr(instance, mutname, copy.copy(mutvalue))
        return instance

class Father:
    __metaclass__ = metaImu
    __mutables__ = dict(attr1=None, attr2={})

class Child(Father):
    __mutables__ = dict(attr3=[])

I haven't tested this, but I believe it should behave as you appear to
desire (assuming you want shallow rather than deep copying of the
values, of course: I can't read your mind!).  I specifically showed that
you can use __mutables__ also for attributes which are in fact not
mutable, since copy.copy should be innocuous (identity) on those, just
in case you insist on having the same form for all such attributes.  I
have also assumed that an instance's __init__ may be allowed to set its
own value for an attribute in __mutables__ and if it does so then that
should be left alone.  But of course you can play with all of these
aspects at will.  You could even have that __call__ method examine all
attributes defined in the class (or any superclass) and force copies of
them into instance attributes for all whose types are [[whatever
criteria you like...]].


Alex