[Python-Dev] __getattr__ and new style classes

[Kristján Valur Jónsson]

Thanks for trying this out.
When I saw your comments, I realized that the difference is due to a speed patch that we have her at CCP.
The test was run using current 2.5 python, with a patch applied, as attached this message.
The patch was written by me a few years back when I noticed that python spends enormous time during regular
run creating exceptions that then get discarded internally.  Most of the cost comes from FormatException
when it is generating AttributeErrors.
So, when I thought I was complaining about slow new objects, I was really showing off my cool optimization.

The problem is, it is difficult to generalize this to python in general.  I spent some time last year to
try to improve new-style classes, by adding ts fields and telling the exception system to "expect a certain
type of exception, and if it is raised, use PyErr_SetNone() instead of the PyErr_Format because I will
clear it anyway".  I had macros such as
PyErr_Expect(AttributeError);
attr = PyObject_GetAttr(o, a);
PyErr_Expect(0);
if (!attr)
        PyErr_Clear()

The problem was, I wasn't able to come up with a simple patch that showed consistent speed improvements
in the performance testsuite without showing some slowdowns in other places.

Running regular python code through a profiler, and especially code that relies much on the use of
__getattr__() to emulate attribute access, will show hideous amounts of time spent formatting
attribute exceptions that get thrown away.

Any thoughts on how to do this better?

And, for completeness, a new test run of this, using python25, first using the neutered patch version
(where I have set softfail=0 in the relevant places) and then with the patch active:

import timeit
s = """
class dude:
        def bar(self):pass
        def __getattr__(self, a): return a
class dude2(object):
        def bar(self):pass
        def __getattr__(self, a): return a
d = dude()
d2 = dude2()
d.a = d2.a = 1
"""
print timeit.Timer("d.foo", s).timeit()
print timeit.Timer("d.bar", s).timeit()
print timeit.Timer("d.a", s).timeit()

print timeit.Timer("d2.foo", s).timeit()
print timeit.Timer("d2.bar", s).timeit()
print timeit.Timer("d2.a", s).timeit()

patch neutered
1.35734336404
0.157773452422
0.0937950608722
1.48494915604
0.240154539405
0.186524222345

patch active:
0.352850669641
0.147599760073
0.0910020300097
1.4453737036
0.212842069748
0.203442097864


Cheers,

Kristján
> -----Original Message-----
> From: Nick Coghlan [mailto:ncoghlan at gmail.com]
> Sent: Wednesday, October 08, 2008 21:37
> To: Kristján Valur Jónsson
> Cc: Python-Dev
> Subject: Re: [Python-Dev] __getattr__ and new style classes
>
> Kristján Valur Jónsson wrote:
> > Hello there.
> >
> > I‘ve just noticed what I consider a performance problem:
> >
> > Using new style classes to provide attribute-like access using
> > __getattr__ is considerably slower than old style classes:  Observe:
>
> I can't reproduce those relative numbers using SVN trunk. Using your
> setup code (copied and pasted directly from your message to my
> interpreter session) I got the following numbers:
>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: speedpatch.rar
Type: application/octet-stream
Size: 1639 bytes
Desc: speedpatch.rar
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081009/0834679a/attachment.obj>