customized instance dictionaries, anyone?

Bengt Richter bokr at
Thu Jan 26 10:37:23 EST 2006

On 25 Jan 2006 09:35:50 -0800, "wolfgang.lipp at" <wolfgang.lipp at> wrote:

>thx! indeed, it worked -- took me some time to figure out how to
>implement the setting of attributes, too. i finally managed to get that
>done using super:
Seems good.
>the ``~.__setattr__()`` method tests for the special name ``_dict`` and
>defers execution to the super of the ``X`` instance, ``object``. other
>stuff is handled by the instance itself.
seems a clean way to do it.

>testing that with ::
>    x = X()
>    print x.__dict__[ 'foo' ]
>    print x.__dict__[ 'bar' ]
>    print
>    print
>    print x.__dict__
>    x.oops = 42
>    print x.__dict__
>yields ::
>    bar
>    bar
>    {'foo': 'bar'}
>    {'foo': 'bar', 'oops': 42}
>as expected.
>i tried to reason *why* the usage of a property makes this particular
>piece of code work, but i seemingly can't find out. anyone around who
>has thoughts on that?
I had an inspiration and I think succeeded in doing what you were originally
trying to do (replace the actual instance dict with your custom dict), which
I tried to do but didn't realize at first that the __init__ setting
of x.__dict__ was really setting x.__dict__['__dict__'] not setting the initial
x.__dict__ itself. __dict__ is a peculiar animal, and looking for an instance's
attribute dict doesn't start at the instance. If you look for instance.__dict__,
it is just like looking for any other attribute, and it starts at type(instance).mro()[0]
looking for a descriptor (which a method also is). But the first thing "found" is
a dict proxy for looking up attributes, and when it looks up '__dict__' it returns
a descriptor, which then gets its __get__ method called with the instance whose '__dict__'
is being sought. You have to use the corresponding __set__ method to set the value of '__dict__'
(in this case the CustomDict instance). Otherwise instance.__dict__ will automatically
be set to {} and then used so you have {'__dict__':CustomDict()} instead of CustomDict() itself.

Once this is initialized correctly, then all the machinery works, except we still need to intercept
__getattr__ for some reason. I suspect this is another symptom of some optimization. One of these
days I will have to look in the source ;-) You would think it could give up on the mro method search
and get the __dict__ normally to get the attribute, but some mechanism is not finding the custom
dict, or else maybe it's bypassing the __getitem__ and going directly to the base dict method.

Someday I'll have to look in the source ;-)

customdict as before

 >>> class CustomDict( dict ):
 ...     defaultValue = 'THIS ITEM NOT AVAILABLE'
 ...     def __getitem__( self, name ):
 ...         try:
 ...             return super( CustomDict, self ).__getitem__( name )
 ...         except KeyError:
 ...             return self.defaultValue
 ...     def __contains__( self, name ):
 ...         return True
 ...     def has_key( self, name ):
 ...         return True
 >>> class X( object ):
 ...     def __getattr__(self, attr):
 ...         return self.__dict__[attr]
 ...         #return type(self).__dict__['__dict__'].__get__(self)[attr]
 ...     def __init__( self, *args, **kw ):
 ...         type(self).__dict__['__dict__'].__set__(self, CustomDict(*args, **kw)
 >>> x = X(foo='bar')
 >>> print x.__dict__['foo']
 >>> print x.__dict__['bar']
 >>> print
 >>> print
 >>> x.oops = 42
 >>> print x.__dict__
 {'foo': 'bar', 'oops': 42}
Looking at a few things of interest:

 >>> vars(x)
 {'foo': 'bar', 'oops': 42}
 >>> type(vars(x))
 <class '__main__.CustomDict'>
 >>> type(x.__dict__)
 <class '__main__.CustomDict'>
 >>> vars(x)['?']
 >>> type(x)
 <class '__main__.X'>
 >>> type(x).__dict__
 <dictproxy object at 0x02E81554>
 >>> type(x).__dict__['__dict__']
 <attribute '__dict__' of 'X' objects>
 >>> type(x).__dict__['__dict__'].__get__
 <method-wrapper object at 0x02EF3B6C>
 >>> type(x).__dict__['__dict__'].__get__(x)
 {'foo': 'bar', 'oops': 42}
 >>> type(type(x).__dict__['__dict__'].__get__(x))
 <class '__main__.CustomDict'>

The reason the property was needed before was really several reasons.
First was that x.__dict__ wasn't being set properly, so the internal
machinery wasn't finding the custom dict. I changed the name to _dict
and let it be an ordinary attribute, but then used property to fake
what normal stuff would do if x.__dict__ itself was set, not x.__dict__['__dict__']

Wasted a bunch of time trying to get rid of that __getattr__ ;-/

Bengt Richter

More information about the Python-list mailing list