Problems creating a numarray of type PyArray_OBJECT
Hi all, Using the numarray C API, I am trying to create an array containing Python strings from an existing C (char **) array. In Numeric 23.1, the following code fragment works fine: PyObject *pdata = NULL; // num_elements has been defined earlier PyStringObject **strarray = malloc(num_elements * sizeof(PyStringObject *)); if (strarray != NULL) { int e; for (e = 0; e < num_elements; e++) { strarray[e] = (PyStringObject *) PyString_FromString(((char **)data)[e]); } // num_dims and the dim array have been defined earlier pdata = PyArray_FromDimsAndData(num_dims, dim, PyArray_OBJECT, (char *)strarray); } In numarray 0.7, the same code yields the runtime error: libnumarray.error: Type object lookup returned NULL for type 14 in the PyArray_FromDimsAndData() call. Using the current numarray CVS head instead of the 0.7 release makes no difference. Is there somebody who knows what is going or what I am doing wrong? Many thanks in advance, -- Leo Breebaart <leo@lspace.org>
On Tue, 2003-11-18 at 13:34, Leo Breebaart wrote:
Hi all,
Using the numarray C API, I am trying to create an array containing Python strings from an existing C (char **) array.
In Numeric 23.1, the following code fragment works fine:
PyObject *pdata = NULL; // num_elements has been defined earlier PyStringObject **strarray = malloc(num_elements * sizeof(PyStringObject *)); if (strarray != NULL) { int e; for (e = 0; e < num_elements; e++) { strarray[e] = (PyStringObject *) PyString_FromString(((char **)data)[e]); }
// num_dims and the dim array have been defined earlier pdata = PyArray_FromDimsAndData(num_dims, dim, PyArray_OBJECT, (char *)strarray); }
In numarray 0.7, the same code yields the runtime error:
libnumarray.error: Type object lookup returned NULL for type 14
in the PyArray_FromDimsAndData() call.
Using the current numarray CVS head instead of the 0.7 release makes no difference.
Is there somebody who knows what is going or what I am doing wrong?
Yes and no. You're in an area that numarray supports differently than Numeric. numarray's support for character arrays is in the Python numarray.strings module. There's no direct support for using it from C.
Many thanks in advance,
You're welcome, Todd -- Todd Miller <jmiller@stsci.edu>
Todd Miller wrote:
Leo Breebaart wrote:
[...] pdata = PyArray_FromDimsAndData(num_dims, dim, PyArray_OBJECT, (char *)strarray); }
In numarray 0.7, the same code yields the runtime error:
libnumarray.error: Type object lookup returned NULL for type 14
in the PyArray_FromDimsAndData() call. [...]
Is there somebody who knows what is going or what I am doing wrong?
Yes and no. You're in an area that numarray supports differently than Numeric. numarray's support for character arrays is in the Python numarray.strings module. There's no direct support for using it from C.
Thanks for the response, Todd. Do I understand correctly that you are saying it's no longer possible in numarray (as opposed to Numeric) to create *any* kind of PyObject array? (After all, the fact that the PyObjects in the above example are really strings is basically accidental -- what if I had an array of pointers to e.g. PyLists or PyDicts that I wanted to store in a numarray -- in Numeric that would also be possible using the PyArray_OBJECT type, right?) Am I also correct in thinking that at this point the only solution to my problem is to manually convert the string arrays in my data to direct Python lists of strings -- or is there perhaps some other trick by which I could get them into a numarray after all? I do a certain amount of Python-level 'post-processing' in my code anyway, and the string arrays I encounter are expected to have only a few elements each, so it would be no problem to do some munging after the fact, if that could help matters. -- Leo Breebaart <leo@lspace.org>
On Wed, 2003-11-19 at 05:27, Leo Breebaart wrote:
Todd Miller wrote:
Leo Breebaart wrote:
[...] pdata = PyArray_FromDimsAndData(num_dims, dim, PyArray_OBJECT, (char *)strarray); }
In numarray 0.7, the same code yields the runtime error:
libnumarray.error: Type object lookup returned NULL for type 14
in the PyArray_FromDimsAndData() call. [...]
Is there somebody who knows what is going or what I am doing wrong?
Yes and no. You're in an area that numarray supports differently than Numeric. numarray's support for character arrays is in the Python numarray.strings module. There's no direct support for using it from C.
Thanks for the response, Todd.
Do I understand correctly that you are saying it's no longer possible in numarray (as opposed to Numeric) to create *any* kind of PyObject array?
No. numarray-0.7 added numarray.objects which provides basic support for arrays of arbitrary objects: indexing, slicing, ufuncs, etc. Access from C, however, is not directly supported. Internally, numarray.objects manages objects using a Python list so it is important to respect the Python object interface and not try to directly munge the data buffer as you might with numerical arrays.
(After all, the fact that the PyObjects in the above example are really strings is basically accidental -- what if I had an array of pointers to e.g. PyLists or PyDicts that I wanted to store in a numarray -- in Numeric that would also be possible using the PyArray_OBJECT type, right?)
If, in Python, you had a list of dicts, you could create a numarray object array from it. If, in C, you had an array of pointers to dicts, you could first turn it into a list of dicts, and then call back into Python to create an object array using numarray.objects.fromlist().
Am I also correct in thinking that at this point the only solution to my problem is to manually convert the string arrays in my data to direct Python lists of strings -- or is there perhaps some other trick by which I could get them into a numarray after all?
It's possible to call back into Python to create an empty CharArray, and then to fill it in C. This is more straight forward for CharArrays because the data is stored in the same sort of buffer as for numerical arrays, and unlike object arrays, no reference counting is required. Regards, Todd
-- Todd Miller <jmiller@stsci.edu>
participants (2)
-
Leo Breebaart -
Todd Miller