[Numpy-discussion] object array alignment issues

[Charles R Harris]

On Fri, Oct 16, 2009 at 9:35 PM, Travis Oliphant <oliphant at enthought.com>wrote:

>
> On Oct 15, 2009, at 11:40 AM, Michael Droettboom wrote:
>
> I recently committed a regression test and bugfix for object pointers in
> record arrays of unaligned size (meaning where each record is not a
> multiple of sizeof(PyObject **)).
>
> For example:
>
>        a1 = np.zeros((10,), dtype=[('o', 'O'), ('c', 'c')])
>        a2 = np.zeros((10,), 'S10')
>        # This copying would segfault
>        a1['o'] = a2
>
> http://projects.scipy.org/numpy/ticket/1198
>
> Unfortunately, this unit test has opened up a whole hornet's nest of
> alignment issues on Solaris.  The various reference counting functions
> (PyArray_INCREF etc.) in refcnt.c all fail on unaligned object pointers,
> for instance.  Interestingly, there are comments in there saying
> "handles misaligned data" (eg. line 190), but in fact it doesn't, and
> doesn't look to me like it would.  But I won't rule out a mistake in
> building it on my part.
>
>
> Thanks for this bug report.      It would be very helpful if you could
> provide the line number where the code is giving a bus error and explain why
> you think the code in question does not handle misaligned data (it still
> seems like it should to me --- but perhaps I must be missing something --- I
> don't have a Solaris box to test on).   Perhaps, the real problem is
> elsewhere (such as other places where the mistake of forgetting about
> striding needing to be aligned also before pursuing the fast alignment path
> that you pointed out in another place of code).
>
> This was the thinking for why the code (that I think is in question) should
> handle mis-aligned data:
>
> 1) pointers that are not aligned to the correct size need to be copied to
> an aligned memory area before being de-referenced.
> 2) static variables defined in a function will be aligned by the C
> compiler.
>
> So, what the code in refcnt.c does is to copy the value in the NumPy
> data-area (i.e. pointed to by it->dataptr) to another memory location (the
> stack variable temp), dereference it and then increment it's reference
> count.
>
> 196:  temp = (PyObject **)it->dataptr;
> 197:  Py_XINCREF(*temp);
>

Doesn't it->dataptr need to be copied to temp, not just assigned?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20091016/30850578/attachment.html>