help with writing extension

[Alex Martelli]

<posted & mailed>

Eric Hagemann wrote:

> I have been handcrafting an extension module creating a new type
> 
> I am making use of the numeric methods (+,-,* etc)
> 
> I solved the problem of the object interating with other types by working
> through the __coerce__() function and have no issue with operations like
> <obj>+2 but I am getting an error on 2-<obj>.

As I understand, __coerce__ is not recommended for new code.  You should,
I believe, add Py_TPFLAGS_CHECKTYPES to your tp_flags slot, and be ready
in your numeric functions to deal with types as they come.  Sometimes
coercion can be handier, but in general avoiding it is better.


> In Beazley's book there are descriptions of functions like __radd__() when
> defining a new type in python, but I cannot find reference to these
> functions in C ?

If you grep Python's C sources (2.2 or better), you'll find some such
references.  Slots nb_add of the PyNumberMethods structure to which
slot tp_as_number of your PyTypeObject structure point supplies both
__add__ and __radd__ (when you call PyType_Ready on your type, suitable
wrappers on that slot are exposed with those two names).


> I have checked the doc's (not to say that I did not miss something ;) )

I think you still need to complement existing docs with some little
study of the Python sources when you want to design rich and complete
extension types -- and I think that dealing with __radd__ etc in the
right way is part of a "rich and complete" (rather than just a
really minimal) extension type.  I think this will still be true
when my "Python in a Nutshell" is finally out, because the chapter
on extending and embedding with C is limited to 40-50 pages (can't
be more precise right now because the book's being edited and I'm
not sure how many of my examples and details the editor will want
to keep -- or if even more will be added), and doing complete justice
to the subject would require about 10 times as much space.

But I also think that asking here about such issues is just right!
Best place to get help -- and this way the help gets archived (thanks
to google groups and other archivers) and can be pointed to in the
future (which alas doesn't work for other mailing lists such as
python-help, nor, of course, for private mail:-).


An example can help -- sorry if it's not very concise, but the
PyTypeObject structure is so large that any examples of its use
are also large, even though most fields are 0.  Say I have a
directory ~/extype with a setup.py file such as:

from distutils.core import setup, Extension

setup(name = "samp1",
    version = "1.0",
    description = "Sample extension type",
    maintainer = "Alex Martelli",
    maintainer_email = "aleaxit at yahoo.com",
       
    ext_modules = [ Extension('samp1', sources=['samp1.c']) ]
)

and a samp1.c file such as:

#include "Python.h"

typedef struct {
    PyObject_HEAD
} samp1;

static void
samp1_dealloc(PyObject *op)
{
    op->ob_type->tp_free(op);
}

static PyObject *
samp1_amethod(PyObject *self)
{
    Py_INCREF(Py_None);
    return Py_None;
}

static PyObject *
samp1_add(PyObject *self, PyObject *other)
{
    PyObject* result = PyTuple_New(2);
    if(!result)
        return NULL;
    Py_INCREF(self);
    PyTuple_SET_ITEM(result, 0, self);
    Py_INCREF(other);
    PyTuple_SET_ITEM(result, 1, other);
    return result;
}

static PyMethodDef samp1_methods[] = {
    {"amethod", (PyCFunction)samp1_amethod, METH_NOARGS},
    {NULL,  NULL}           /* sentinel */
};

static PyNumberMethods samp1_as_number = {
    samp1_add,
};

static PyTypeObject samp1_t = {
    PyObject_HEAD_INIT(0)
    0,
    "samp1",
    sizeof(samp1),
    0,
    samp1_dealloc,
    0,
    0,
    0,
    0,                                  /* tp_compare */
    0,                                  /* tp_repr */
    &samp1_as_number,                   /* tp_as_number */
    0,                                  /* tp_as_sequence */
    0,                                  /* tp_as_mapping */
    0,                                  /* tp_hash */
    0,                                  /* tp_call */
    0,                                  /* tp_str */
    0,                                  /* tp_getattro */
    0,                                  /* tp_setattro */
    0,                                  /* tp_as_buffer */
    Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE|Py_TPFLAGS_CHECKTYPES,
    "sample extension type",
    0,                                  /* tp_traverse */
    0,                                  /* tp_clear */
    0,                                  /* tp_richcompare */
    0,                                  /* tp_weaklistoffset */
    0,                                  /* tp_iter */
    0,                                  /* tp_iternext */
    samp1_methods,                      /* tp_methods */
    0,                                  /* tp_members */
    0,                                  /* tp_getset */
    0,                                  /* tp_base */
    0,                                  /* tp_dict */
    0,                                  /* tp_descr_get */
    0,                                  /* tp_descr_set */
    0,                                  /* tp_dictoffset */
    0,                                  /* tp_init */
    PyType_GenericAlloc,                /* tp_alloc */
    PyType_GenericNew,                  /* tp_new */
    _PyObject_Del,                      /* tp_free */
};

static PyMethodDef no_methods[] = { {0} };

void
initsamp1(void)
{
    PyObject* self;
    PyType_Ready(&samp1_t);
    self = Py_InitModule("samp1", no_methods);
    PyObject_SetAttrString(self, "samp1", (PyObject*)&samp1_t);
}


Now of course running python setup.py in this dictionary builds
and installs extension module samp1.  Let's look at it in action:

[alex at lancelot extype]$ python
Python 2.2.1 (#2, Jul 15 2002, 17:32:26)
[GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import samp1
>>> examp = samp1.samp1()
>>> examp + 'ciao!'
(<samp1 object at 0x8160f28>, 'ciao!')
>>> 'salute...' + examp
('salute...', <samp1 object at 0x8160f28>)
>>>

See what's going on?  Despite the (traditional) argument names,
samp1_add is being called with the LEFT operand of + as the first
argument ('self'), the RIGHT operand as the second argument
('other') -- your C code can test the types directly to do the
right thing in both the __add__ and __radd__ cases...!

Sure, you CAN make do with coercion (so have all Python extension
types until about last year), but bypassing it makes things a bit
clearer, I believe.  If you do want coercion in some but nor all
of your numeric-methods you can deal with it yourself -- call your
own "coercion-like" function from your methods' code as and where
appropriate.  I do think it's simpler than having Python do it for
you and sometimes having to "bypass" it, so to speak... this way
you can still supply a function to be called when the user does an
explicit coerce call, and yet have whatever precise behavior you
want for all operators that look like numeric operators ("look like",
because, for example, sequence concatenation isn't really numeric --
it just LOOKS LIKE it is, and has to live in PyNumberMethods, but
that's not quite where one might like to place it...).


Feel free to keep posting such questions to this list if there is
anything unclear in the above, and/or if you need any more help!


Alex