[Python-Dev] C API for appending to arrays

[Mike Klaas]

On 2-Feb-09, at 9:21 AM, Hrvoje Niksic wrote:
>
> It turns out that an even faster method of creating an array is by  
> using the fromstring() method.  fromstring() requires an actual  
> string, not a buffer, so in C++ I created an std::vector<double>  
> with a contiguous array of doubles, passed that array to  
> PyString_FromStringAndSize, and called array.fromstring with the  
> resulting string.  Despite all the unnecessary copying, the result  
> was much faster than either of the previous versions.
>
>
> Would it be possible for the array module to define a C interface  
> for the most frequent operations on array objects, such as appending  
> an item, and getting/setting an item?  Failing that, could we at  
> least make fromstring() accept an arbitrary read buffer, not just an  
> actual string?

Do you need to append, or are you just looking to create/manipulate an  
array with a bunch of c-float values?  I find As{Write/Read}Buffer  
sufficient for most of these tasks.  I've included some example pyrex  
code that populates a new array.array at c speed.  (Note that you can  
get the size of the resulting c array more easily than you are by  
using PyObject_Length).  Of course, this still leaves difficult  
appending to an already-created array.

def calcW0(W1, colTotal):
     """ Calculate a W0 array from a W1 array.

     @param W1: array.array of doubles
     @param colTotal: value to which each column should sum

     @return W0 = [colTotal] * NA - W1
     """
     cdef int NA
     NA = len(W1)
     W0 = array('d', [colTotal]) * NA

     cdef double *cW1, *cW0
     cdef int i
     cdef Py_ssize_t dummy

     PyObject_AsReadBuffer(W1, <void**>&cW1, &dummy)
     PyObject_AsWriteBuffer(W0, <void**>&cW0, &dummy)

     for i from 0 <= i < NA:
         cW0[i] = cW0[i] - cW1[i]

     return W0

regards,
-Mike