using lists in C-extensions?

Alex Martelli aleax at aleax.it
Mon Feb 18 11:39:37 CET 2002


Marcus Stojek wrote:
        ...
> EXAMPLE:
> 
> list=[]
> for i in range(100000):
>     list.append(i)

Note in passing that this is equivalent to
    list = range(100000)
since range does return a list.  If it didn't (if it returned
some other kind of sequence or iterator), then you could
ALMOST use:
    list = list(funkyvariantofrange(100000))
EXCEPT that by calling this local variable 'list' you've
shadowed the builtin type [function in 2.1 and earlier] of
the same name [if this code is within a function] -- so,
don't: call the variable alist or thelist or mylist instead.

Peripheral issues, but sometimes they DO give headaches.


> listnew=map((lambda x:x+1),list)  # this should be coded in C

You won't get all that much gain by recoding this in C, since
map is already C-coded and well-optimized and fetching and
setting the list elements is substantially the same amount of
work.  However, if you can start with an array.array (or, of
course, a Numeric array, as Jason suggested), then potential
for performance is indeed there.

Simplest, assuming you do want to work with integers:

import array
anarray = array.array('l', alist)

import mycode
mycode.incr(anarray)

In a directory called e.g. ~/mycode put two text files, one
called setup.py with:

from distutils.core import setup, Extension

setup (name = "mycode",
       version = "1.0",
       maintainer = "Alex Martelli",
       maintainer_email = "aleax at aleax.it",
       description = "Sample use of array",

       ext_modules = [Extension('mycode',sources=['mycode.c'])]

and one called mycode.c with:

#include "Python.h"

static PyObject*
incr(PyObject *self, PyObject *args)
{
    PyObject *the_array;
    long* pbuf;
    int blen, i;

    if(!PyArg_ParseTuple(args, "O", &the_array))
        return 0;
    if(PyObject_AsWriteBuffer(the_array, (void**)&pbuf, &blen))
        return 0;
    blen /= sizeof(long);
    for(i=0; i<blen; ++i)
        pbuf[i] += 1;

    return Py_BuildValue("");
}

static PyMethodDef mycode_methods[] = {
    {"incr",   incr,   METH_VARARGS},
    {0, 0}
};

void
initmycode(void)
{
    Py_InitModule("mycode", mycode_methods);
}

Run in this directory:
    python setup.py install

and the above sample Python snippet should work.  Of course,
you may wish to take more precautions in function incr, which,
as it stands, would produce funny results should you call
it e.g. with an array.array of typecode other than 'l' or
'L'.  But don't go too far in "precautions" (typetests) that
may in the future interfere with the ability of using this
code with objects of other types.  For example, you might
want to check PyObject_GetAttrString(the_array, "typecode"),
that it works and returns "l" or "L": this makes it easy in
the future to emulate the two important aspects of an
array.array's behavior (the typecode attribute and the
fact that it has a single contiguous writable buffer) by
another type should the need arise.  You may also want to
support multiple typecodes, such as "l" or "L" for longs
(signed/unsigned), "f" for floats, "d" for doubles, to have
incr transparently work on numeric arrays of different
types of numbers (you'll need to dispatch e.g. with a
case statement in your C code between different loops of
operations in this case, of course).


Alex




More information about the Python-list mailing list