[Numpy-discussion] libnumarray.h and NO_IMPORT_ARRAY

[David Abrahams]

Philip Austin <paustin at eos.ubc.ca> writes:

>  > What I thought I would see, was some form of direct coupling
>  > between a module in boost and either the Numeric or numarray C-API.
>  > However, when I search the whole source tree (*.cpp *.hpp) I can
>  > find neither PyArrayObject, arrayobject.h, NDInfo, nor
>  > libnumarray.h.  So I'm not sure what part of any Numeric/numarray
>  > C-API you're using now (if any) 
> 
> The answer is -- no part of the C-API if he can get away with it.
> Your move towards Numeric compatibility for numarray means that he
> now can get away with it.

I've been getting away with it already. It's easy enough to access all
that functionality through the usual Python/C API.

> numeric wraps the module pointer, and calls functions
> through it -- e.g. (from boost/libs/python/src/numeric.cpp):
> 
>   void array_base::resize(object const& shape)
>   {
>       attr("resize")(shape);
>   }
> 
> which invokes PyObject_CallObject to call the array's resize method,
> with the tuple shape (similar to CXX). If the method isn't part of the
> library (say for Numeric and getflat) then a python exception is
> thrown.

Slow but effective. If you want to touch the raw PyObject*, of course,
you can do this:

        void f(numeric::array x)
        {
            PyObject* danger = x.ptr();
            // Cast to whatever you like.
        }

>  > and in particular, what isn't
>  > working for you.  Note: numarray and Numeric are not totally
>  > compatible at either the C or Python levels, so either area could
>  > have been a source of difficulty.
> 
> The problem was that, as long as a call to getNDInfo was required
> to access an array's data pointer, a Python-only approach
> like the above wouldn't work.  Since it is now possible to
> do this via a cast to PyArrayObject* for a numarray, the problem
> has disappeared.

Not sure what you're saying here. I /do/ remember when I was writing
the Numeric/NumArray support that there seemed no publicly-accessible
definition of the structure of the underlying array objects.

> (there's nothing stopping you from using the C-API if you want,
> though.  For example you can create a numarray with data
> copied from a C array with something like:
> 
> numeric::array makeNum(int * data, int n=0){
>   object obj(detail::new_reference(PyArray_FromDims(1, &n, PyArray_INT)));
>   char *arr_data = ((PyArrayObject*) obj.ptr())->data; 
>   memcpy(arr_data, data, 4 * n); 
>   return numeric::array(obj);
> }

Well, this uses undocumented implementation details. The supported
approach is:

   handle<> raw_array(PyArray_FromDims(1, &n, PyArray_INT));
   object obj(raw_array);

Note, though, that you can also write:

   handle<PyArrayObject> raw_array(PyArray_FromDims(1, &n, PyArray_INT));

Assuming, of course, that PyArray_FromDims returns a
PyArrayObject*. And now you have access to the PyArrayObject* through
raw_array.

Also note that it's probably smarter to use

   extract<numeric::array>(obj)

Than

   numeric::array(obj)

See the note about pitfalls at
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/boost/boost/libs/python/doc/tutorial/doc/derived_object_types.html
for the reasons why.

-- 
           David Abrahams * Boost Consulting
dave at boost-consulting.com * http://www.boost-consulting.com