[Numpy-discussion] Problems creating a numarray of type PyArray_OBJECT

[Todd Miller]

On Wed, 2003-11-19 at 05:27, Leo Breebaart wrote:
> Todd Miller wrote:
> 
> > Leo Breebaart wrote:
> >
> > > [...]
> > >        pdata = PyArray_FromDimsAndData(num_dims, dim, PyArray_OBJECT, 
> > >                                        (char *)strarray);
> > >    }
> > > 
> > > In numarray 0.7, the same code yields the runtime error:
> > > 
> > >    libnumarray.error: Type object lookup returned NULL for type 14
> > > 
> > > in the PyArray_FromDimsAndData() call. [...]
> > 
> > > Is there somebody who knows what is going or what I am doing
> > > wrong?
> > 
> > Yes and no.   You're in an area that numarray supports differently than
> > Numeric.  numarray's support for character arrays is in the Python
> > numarray.strings module.  There's no direct support for using it from
> > C.
> 
> Thanks for the response, Todd.
> 
> Do I understand correctly that you are saying it's no longer
> possible in numarray (as opposed to Numeric) to create *any* kind
> of PyObject array? 

No.  numarray-0.7 added numarray.objects which provides basic support
for arrays of arbitrary objects: indexing, slicing, ufuncs, etc.  Access
from C, however, is not directly supported.  Internally, 
numarray.objects manages objects using a Python list so it is important
to respect the Python object interface and not try to directly munge the
data buffer as you might with numerical arrays.

> (After all, the fact that the PyObjects in the
> above example are really strings is basically accidental -- what
> if I had an array of pointers to e.g. PyLists or PyDicts that I
> wanted to store in a numarray -- in Numeric that would also be
> possible using the PyArray_OBJECT type, right?)

If, in Python, you had a list of dicts, you could create a numarray
object array from it.  If, in C, you had an array of pointers to dicts,
you could first turn it into a list of dicts, and then call back into
Python to create an object array using numarray.objects.fromlist().

> Am I also correct in thinking that at this point the only
> solution to my problem is to manually convert the string arrays
> in my data to direct Python lists of strings -- or is there
> perhaps some other trick by which I could get them into a
> numarray after all? 

It's possible to call back into Python to create an empty CharArray, and
then to fill it in C.  This is more straight forward for CharArrays
because the data is stored in the same sort of buffer as for numerical
arrays, and unlike object arrays, no reference counting is required.

Regards,
Todd

> -- 
> Todd Miller <jmiller at stsci.edu>