__getattr__, __setattr__ and pickle

[Bruno Desthuilliers]

mwojc a écrit :
> Bruno Desthuilliers wrote:
(snip)
>> FWIW, you'd be better using a property instead of __getattr__ /
>> __setattr__ if possible. 
> 
> You're probably right again, in this case it's better to use property.

Since you seem to have concerns wrt/ execution time, properties might be 
a little bit faster than __getattr__/__setattr__ hooks - cf below...

>> And while we're at it, you dont need to 
>> manually take care of your index in the for loop - you can use
>> enumerate(iterable) instead:
>>
>>               for j, net in enumerate(self.nets):
>>                   w1 = self.wmarks[j]
>>                   w2 = self.wmarks[j+1]
>>                   self._weights[w1:w2] = net.weights
>>               return self._weights
>>
> Sometimes i use manual handling of index because i'm convinced that
> enumeration is a bit slower than this. But i'm not really sure about it...

It's easy to try out:

bruno at bibi ~ $ python
Python 2.5.1 (r251:54863, Apr  6 2008, 17:20:35)
[GCC 4.1.2 (Gentoo 4.1.2 p1.0.2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> def manual(cnt):
...     j = 0
...     for i in xrange(cnt):
...         x = i
...         j += 1
...
 >>> def auto(cnt):
...     for j, i in enumerate(xrange(cnt)):
...         x = i
...
 >>> from timeit import Timer

 >>> tm = Timer("manual(10000)", "from __main__ import manual")
 >>> ta = Timer("auto(10000)", "from __main__ import auto")
 >>> tm.timeit(1000)
3.3354489803314209
 >>> tm.timeit(1000)
3.3376359939575195
 >>> tm.timeit(1000)
3.3400180339813232
 >>> ta.timeit(1000)
2.8350770473480225
 >>> ta.timeit(1000)
2.8400650024414062
 >>> ta.timeit(1000)
2.8361449241638184
 >>>

Looks like enum is a bit faster by a mostly constant factor here.

And while we're at it:

 >>> class Prop(object):
...     @apply
...     def prop():
...         def fget(self): return self._prop
...         def fset(self, val): self._prop = val
...         return property(**locals())
...     def __init__(self, val): self.prop=val
...
 >>> class Hook(object):
...     def __getattr__(self, name):
...         if name == 'prop':
...             return self._prop
...         raise AttributeError("yadda")
...     def __setattr__(self, name, val):
...         if name ==  'prop':
...             self.__dict__['_prop'] = val
...         else:
...             # XXX : INCORRECT IMPLEMENTATION, DONT DO THIS !
...             self.__dict__[name] = val
...             # correct implementation:
...             # super(Hook, self).__setattr__(name, value)
...     def __init__(self, val): self.prop=val
...
 >>> def testprop(cnt):
...     p = Prop('test')
...     for i in xrange(cnt):
...         p.prop = i
...         x = p.prop
...
 >>> def testhook(cnt):
...     h = Hook('test')
...     for i in xrange(cnt):
...         h.prop = i
...         x = h.prop
...
 >>> tp = Timer("testprop(1000)", "from __main__ import testprop")
 >>> th = Timer("testhook(1000)", "from __main__ import testhook")
 >>> tp.timeit(1000)
3.0640909671783447
 >>> tp.timeit(1000)
3.0650019645690918
 >>> th.timeit(1000)
7.0889511108398438
 >>> th.timeit(1000)
7.0815410614013672

Looks like properties are significatively faster than the 
__getattr__/__setattr__ hook too... Which is probably explained by the 
following facts:

Looking for binding descriptors (like read/write properties) is the 
first very stage of the attribute resolution algorithm (since they must 
be looked up before the instance's __dict__ to avoid setting an instance 
attribute which would then shadow the property).

OTHO, __getattr__ is always looked for last - which means it takes 
longer to resolve __getattr__ than to resolve a read access to a 
read/write property.

wrt/ __setattr__  - which is always called for attribute assignement, 
whatever -, overloading it (instead of relying on the optimized builtin 
implementation) is not only tricky (BTW, your implementation is broken - 
try to add a read/write property to your class and enjoy...[1]), but 
also cause a penalty for *all* attributes 'write' access.

[1] oh, yes, should I mention it ? The correct implementation is even 
slower:
 >>> class Hook(object):
...     def __getattr__(self, name):
...         if name == 'prop':
...             return self._prop
...         raise AttributeError("yadda")
...     def __setattr__(self, name, val):
...         if name ==  'prop':
...             self.__dict__['_prop'] = val
...         else:
...             #self.__dict__[name] = val
...             super(Hook, self).__setattr__(name, value)
...     def __init__(self, val): self.prop=val
...
 >>> th.timeit(1000)
7.1943540573120117
 >>> th.timeit(1000)
7.1930480003356934
 >>>


HTH