[Numpy-discussion] subclassing array in c
Christoph Gohle
christoph.gohle at mpq.mpg.de
Thu Mar 8 02:49:18 EST 2012
Dear Val,
I agree that more detail is needed. Sorry for that it was late yesterday.
I am running Python 2.6.1, numpy development branch (numpy-2.0.0.dev_20101104-py2.6-macosx-10.6-universal.egg). maybe I should switch to release?
I compile with your setup.py using 'python setup.py build_ext -i' and run the commands below in ipython. As you can see, I don't get a crash for a single call to the constructor, consistent with your observation. And it looks like, one has to use dict in the unit to make it crash.
Can you make sense out of that?
In [1]: import spampub
In [4]: spampub.UnitArray(1,{'s':1})
Out[4]: UnitArray(1)
In [6]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)]
Segmentation fault
backtrace is the following:
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000021
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Thread 0 Crashed: Dispatch queue: com.apple.main-thread
0 org.python.python 0x00000001000b0b8e PyObject_GC_Track + 1473
1 org.python.python 0x00000001000b1255 _PyObject_GC_Malloc + 191
2 org.python.python 0x00000001000b12d0 _PyObject_GC_New + 21
3 org.python.python 0x000000010003a856 PyDict_New + 116
4 org.python.python 0x000000010003a99c _PyDict_NewPresized + 24
5 org.python.python 0x00000001000880cb PyEval_EvalFrameEx + 11033
6 org.python.python 0x000000010008acce PyEval_EvalCodeEx + 1803
7 org.python.python 0x000000010008735d PyEval_EvalFrameEx + 7595
8 org.python.python 0x000000010008acce PyEval_EvalCodeEx + 1803
9 org.python.python 0x000000010008935e PyEval_EvalFrameEx + 15788
10 org.python.python 0x000000010008acce PyEval_EvalCodeEx + 1803
11 org.python.python 0x000000010008935e PyEval_EvalFrameEx + 15788
12 org.python.python 0x00000001000892e1 PyEval_EvalFrameEx + 15663
13 org.python.python 0x000000010008acce PyEval_EvalCodeEx + 1803
14 org.python.python 0x000000010008935e PyEval_EvalFrameEx + 15788
15 org.python.python 0x000000010008acce PyEval_EvalCodeEx + 1803
16 org.python.python 0x000000010008935e PyEval_EvalFrameEx + 15788
17 org.python.python 0x000000010008acce PyEval_EvalCodeEx + 1803
18 org.python.python 0x000000010008935e PyEval_EvalFrameEx + 15788
19 org.python.python 0x000000010008acce PyEval_EvalCodeEx + 1803
20 org.python.python 0x000000010008935e PyEval_EvalFrameEx + 15788
21 org.python.python 0x000000010008acce PyEval_EvalCodeEx + 1803
22 org.python.python 0x000000010008ad61 PyEval_EvalCode + 54
23 org.python.python 0x00000001000a265a Py_CompileString + 78
24 org.python.python 0x00000001000a2723 PyRun_FileExFlags + 150
25 org.python.python 0x00000001000a423d PyRun_SimpleFileExFlags + 704
26 org.python.python 0x00000001000b0286 Py_Main + 2718
27 org.python.python.app 0x0000000100000e6c start + 52
Am 08.03.2012 um 02:36 schrieb Val Kalatsky:
>
> Tried it on my Ubuntu 10.10 box, no problem:
>
> 1) Saved as spampub.c
> 2) Compiled with (setup.py attached): python setup.py build_ext -i
> 3) Tested from ipython:
> In [1]: import spampub
> In [2]: ua=spampub.UnitArray([0,1,2,3.0],'liter')
> In [3]: ua
> Out[3]: UnitArray([ 0., 1., 2., 3.])
> In [4]: ua.unit
> Out[4]: 'liter'
>
>
> On Wed, Mar 7, 2012 at 7:15 PM, Val Kalatsky <kalatsky at gmail.com> wrote:
>
> Seeing the backtrace would be helpful.
> Can you do whatever leads to the segfault
> from python run from gdb?
> Val
>
>
> On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle <christoph.gohle at mpq.mpg.de> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> I have been struggeling for quite some time now. Desperate as I am, now I need help.
>
> I was trying to subclass ndarrays in a c extension (see code below) and do constantly get segfaults. I have been checking my INCREF and DECREF stuff up and down but can't find the error. Probably I got something completely wrong... anybody able to help?
>
> Thanks,
> Christoph
> - -----------------
> #include <Python.h>
> #include <structmember.h>
>
> #include <numpy/arrayobject.h>
>
> static PyObject *SpamError;
>
> typedef struct {
> PyArrayObject base;
> PyDictObject* unit;
> } UnitArrayObject;
>
> PyTypeObject unitArrayObjectType;
>
> static int
> checkunit(PyObject *unit1, PyObject *unit2) {
> return PyObject_Compare(unit1, unit2);
> }
>
> static PyObject *
> unitArray_new(PyTypeObject *cls, PyObject *args, PyObject *kwargs) {
> PyObject *data = NULL,
> *unit = NULL;
> PyArray_Descr* dtype = NULL;
> PyObject *res = NULL, *tmp = NULL;
>
> if (!PyArg_ParseTuple(args, "OO|O&", &data, &unit, PyArray_DescrConverter, &dtype)) {
> Py_XDECREF(dtype);
> return NULL;
> }
>
> res = PyArray_FromAny(data, dtype, 0, 0, NPY_ENSURECOPY, NULL);
> if (res == NULL) {
> Py_XDECREF(dtype);
> //TODO: raise exception?
> return NULL;
> }
>
> if (PyObject_IsInstance(data, (PyObject*)cls)) {
> if (unit!=NULL && !checkunit((PyObject*)((UnitArrayObject*)data)->unit,unit)) {
> Py_XDECREF(res);
> //TODO: raise exception
> return NULL;
> }
> } else {
> if (PyObject_IsTrue(unit)) {
> tmp = res;
> res = PyArray_View((PyArrayObject*)res, NULL, &unitArrayObjectType);
> if (tmp!=res) {
> Py_XDECREF(tmp);
> }
> ((UnitArrayObject*)res)->unit = (PyDictObject*)unit;
> Py_INCREF(unit);
> if (unit!=NULL) {
> }
> }
> }
> return res;
> }
>
> static PyObject*
> unitArray__array_finalize__(PyObject* new, PyObject* args) {
> PyObject *attr = NULL, *tmp = NULL;
> PyObject *parent = NULL;
>
> if (!PyArg_ParseTuple(args, "O", &parent)) {
> return NULL;
> }
> if (parent!=NULL) {
> attr = PyObject_GetAttrString(parent, "unit");
> if (attr == NULL) {
> //parent has no 'unit' so we make a new empty one
> attr = PyDict_New();
> PyErr_Clear();
> }
> }
> tmp = (PyObject*)((UnitArrayObject*)new)->unit;
> ((UnitArrayObject*)new)->unit = (PyDictObject*)attr;
>
> Py_INCREF(Py_None);
> return Py_None;
> }
>
> static PyObject*
> unitArray__array_wrap__(PyObject *self, PyObject *args) {
> PyObject *array = NULL, *context = NULL;
>
> if (!PyArg_ParseTuple(args, "OO", array, context)) {
> //TODO: raise exception
> return NULL;
> }
>
> printf("%s",PyString_AsString(PyObject_Str(context)));
>
> Py_INCREF(array);
> return array;
> }
>
>
> static PyMethodDef unitArrayMethods[] = {
> {"__array_finalize__", unitArray__array_finalize__, METH_VARARGS, "array finalize method"},
> {"__array_wrap__", unitArray__array_wrap__, METH_VARARGS, "array wrap method"},
> {NULL, NULL, 0, NULL}
> };
>
> static PyMemberDef unitArrayMembers[] = {
> {"unit", T_OBJECT, offsetof(UnitArrayObject, unit), 0, "dictionary containing unit info."},
> {NULL, 0, 0, 0, NULL}
> };
>
> PyTypeObject unitArrayObjectType = {
> PyObject_HEAD_INIT(NULL)
> 0, /* ob_size */
> "spam.UnitArray", /* tp_name */
> sizeof(UnitArrayObject), /* tp_basicsize */
> 0, /* tp_itemsize */
> 0, /* tp_dealloc */
> 0, /* tp_print */
> 0, /* tp_getattr */
> 0, /* tp_setattr */
> 0, /* tp_compare */
> 0, /* tp_repr */
> 0, /* tp_as_number */
> 0, /* tp_as_sequence */
> 0, /* tp_as_mapping */
> 0, /* tp_hash */
> 0, /* tp_call */
> 0, /* tp_str */
> 0, /* tp_getattro */
> 0, /* tp_setattro */
> 0, /* tp_as_buffer */
> Py_TPFLAGS_DEFAULT, /* tp_flags */
> "A numpy array with units", /* tp_doc */
> 0, /* traverseproc */
> 0, /* tp_clear*/
> 0, /* tp_richcompare */
> 0, /* tp_weaklistoffset */
> 0, /* tp_iter */
> 0, /* tp_iternext */
> unitArrayMethods, /* tp_methods */
> unitArrayMembers, /* tp_members */
> 0, /* tp_getset */
> 0, /* tp_base*/
> 0, /* tp_dict */
> 0, /* tp_descr_get*/
> 0, /* tp_descr_set */
> 0, /* tp_dictoffset */
> 0, /* tp_init */
> 0, /* tp_alloc */
> unitArray_new /* tp_new */
> };
>
>
> static PyMethodDef SpamMethods[] = {
> {NULL, NULL, 0, NULL} /* Sentinel */
> };
>
> PyObject *unitArray = NULL;
> PyMODINIT_FUNC
> initspampub(void)
> {
> import_array();
>
> PyObject *m;
>
> Py_INCREF(&PyArray_Type);
> unitArrayObjectType.tp_base = &PyArray_Type;
>
> if (PyType_Ready(&unitArrayObjectType) < 0)
> return;
>
>
> m = Py_InitModule3("spampub", SpamMethods, "some tests and a array type with units.");
> if (m == NULL)
> return;
>
> SpamError = PyErr_NewException("spampub.error", NULL, NULL);
> Py_INCREF(SpamError);
> PyModule_AddObject(m, "error", SpamError);
> Py_INCREF(&unitArrayObjectType);
> PyModule_AddObject(m, "UnitArray", (PyObject *)&unitArrayObjectType);
> (void) Py_InitModule("spampub", SpamMethods);
>
> }
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
>
> iEYEARECAAYFAk9YBbkACgkQLYu25rCEIzsExQCggHg0e0ibYP3bEsAE0Ce8Rbm3
> qOcAoKwuTuE4BDZgrqNpwISgMS6uSMnO
> =Qtek
> -----END PGP SIGNATURE-----
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> <setup.py>_______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
Christoph Gohle
--
Max-Planck-Institut für Quantenoptik
Abteilung Quantenvielteilchensysteme
Hans-Kopfermann-Strasse 1
85748 Garching
christoph.gohle at mpq.mpg.de
tel: +49 89 32905 283
fax: +49 89 32905 313
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120308/4a95ce0e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: Signierter Teil der Nachricht
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120308/4a95ce0e/attachment.sig>
More information about the NumPy-Discussion
mailing list