Accessing __slots__ from C

[Chris]

Hrvoje Niksic <hniksic <at> xemacs.org> writes:
...
> Chris <ceball <at> users.sourceforge.net> writes:
> 
> >> PyObject_GetAttrString is convenient, but it creates a Python string
> >> only so it can intern it (and in most cases throw away the freshly
> >> created version).  For maximum efficiency, pre-create the string
> >> object using PyString_InternFromString, and use that with
> >> PyObject_GetAttr.
> >
> > Yes, we'd thought of that too, but it doesn't seem to be an
> > important factor compared to the actual attribute lookup.
> 
> I'd like to see your test code.  

Thanks for your interest in this. My test code is a whole function
that's part of a big simulator, unfortunately! I need to use the data
structures created by the simulator as part of the testing.  While the
source is freely available*, that doesn't make it easy for others to
run the tests...


> In my experience, as long as you're
> accessing simple slots, you should notice a difference.

(I'm not sure what you mean by a 'simple slot'. The slot we're
accessing is a numpy array.)

Sorry I wasn't clear before - we do notice a difference, but not as
big a difference as when we access the attributes (arrays) from a
pre-built list. Below are timings from running the simulator
(i.e. calling the function in question many times) using the three
approaches (GetAttrString, GetAttr, and instead using a list and
GetItem; times outside parentheses are from a stopwatch; times in
parentheses are from Python's cProfile module):


- GetAttrString: 55 seconds (58 seconds)
inside the loop:
PyObject *weights_obj = PyObject_GetAttrString(obj,"attr_one");


- GetAttr: 46 seconds (46 seconds)
outside the loop:
PyObject *name = PyString_FromString("attr_one"); 

inside the loop: 
PyObject *obj = PyObject_GetAttr(obj,name);


- PyList_GetItem: 35 seconds (37 seconds)


So, of course, you are right to say that we should notice a
difference!  But speed is critical for us here. 

Incidentally, what we have is a simulator written entirely in Python,
but we also provide optimized C versions of some parts of it.  These C
parts must be entirely optional.


> Here is a test program that shows a 4.6 time speedup simply by
> switching from PyObject_GetAttrString to PyObject_GetAttr:

Thanks for the illustration. While I didn't run your code myself,
I did try to study it. Illustrations like that are very helpful.


Chris


* from http://topographica.org/