[capi-sig] What are the various mechanisms of attribute access?

Hello PyC-API community,

I would like to know what are the various mechanisms for providing attributes for a module or class.

I'm interested in the structure, how they relate to one another, why they exist and when they should be used.

I'm particularly interested in which techniques provide inheritable attributes.

I can't find any documentation that provides a clear classification, and consequently I feel like I'm flying blind.

I can see we have TYPE attributes (via tp_dict) and INSTANCE attributes (via tp_dictoffset)

There is also another technique namely overriding attribute access e.g. getattr and returning a Callable. But can I assume this technique is not inheritable? It would be nice if it were, then the derived class could call super().getattr which would attempt to manually provide an attribute falling back onto super's tp_dict of TYPE attributes, and rinse and repeat that for each item in the MRO. But experimentation suggests this doesn't happen.

What exactly happens when we create an instance of a class with several MRO items? Does it straight away go through the tp_dict-s each item adding in attributes (but not duplicates, so as to provide an override mechanism)?

Can all attributes access be explained by the above (tp_dict & tp_dictoffset + overriding getattr&friends)?

Is there any way of providing inheritable attributes that does NOT use tp_dict? (the reason for this is given below -- basically it requires messy C++). The only way I can think of is to have code execute every time an instance of Foo is created that will add the attributes to tp_dictoffset using PyObject_SetAttr. But I'm not sure how to get this to run everytime Python does 'class Bar(Foo)...; bar=Bar()'

That's my basic enquiry, however I will detail the problem I'm trying to solve below in case the context is of some interest.

Very confused!

π

I am restructuring PyCXX (http://sourceforge.net/projects/cxx/ http://sourceforge.net/projects/cxx/) — mainly as an exercise. I will be offering my work back to the open source community once I feel it is of decent grade.

In PyCXX you construct a C++ class and Python can see it as a type You expose one of its instance methods and Python sees this method as an attribute of the type.

There is old style and new style class, and PyCXX provides a different mechanism for each:

For old-style class, it overrides getattr to return a Callable in which is packaged up the desired method. For new style class, it maintains an array of PyMethodDef-s which gets slotted into tp_methods.

Additionally, there is a third technique used for module attributes: it retrieves a Dictionary through PyModule_GetDict, and populates it with a PyCallable for each attribute.

For each of these three techniques, a trampoline function is required (as you can't typecast a pointer to a C++ instance method back to an 'extern C' function pointer.

I'm attempting to find a common technique that will work for all three situations.

It's worth noting that the old-style class technique doesn't support inherited attributes. i.e. If you create an old-style class Foo, and do "class Bar(Foo)...", Bar will be unable to see the methods added by PyCXX. However, new style class DOES work.

There seems to be several different mechanisms for adding attributes;

Module

PyModule_Create -> PyModuleDef.m_methods = PyMethodDef[] PyModule_GetDict PyModule_Add*() (uses PyModule_GetDict -- modsupport.c)

Extension Type overriding getattr/o PyTypeObject->tp_dict

Doc says "All the following add to tp_dict, and are not inherited by subtypes:" tp_members tp_methods tp_getset computed attributes

Both Calling PyObject_SetAttr (tries setattro then setattr -- object.c)

There is a problem with implementing tp_methods; I can't have all items in the array point to the same trampoline function, as that function would not know which attribute triggered it (even though it WOULD know which object INSTANCE triggered it, thanks to the first parameter). This means that a C++ solution is going to be a bit ugly, as there is no easy way to generate an arbitrary number of "extern C" functions each of which has its own distinct identity.

So I'm looking at other ways of accomplishing the task.