[Numpy-discussion] numpy arrays, data allocation and SIMD alignement
peridot.faceted at gmail.com
Tue Aug 7 00:49:09 EDT 2007
On 06/08/07, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
> Well, when I proposed the SIMD extension, I was willing to implement the
> proposal, and this was for a simple goal: enabling better integration
> with many numeric libraries which need SIMD alignment.
> As nice as a custom allocator might be, I will certainly not implement
> it myself. For SIMD, I think the weight adding complexity / benefit
> worth it (since there is not much change to the API and implementation),
> and I know more or less how to do it; for custom allocator, that's an
> entirely different story. That's really more complex; static pools may
> be useful in some cases (but that's not obvious, since only the data are
> allocated with this buffer, everything else being allocated through the
> python memory allocator, and numpy arrays have pretty simple memory
> allocation patterns).
I have to agree. I can hardly volunteer David for anything, and I
don't have time to implement this myself, but I think a custom
allocator is a rather special-purpose tool; if one were to implement
one, I think the way to go would be to implement a subclass of ndarray
(or just a constructor) that allocated the memory. This could be done
from python, since you can make an ndarray from scratch using a given
memory array. Of course, making temporaries be allocated with the
correct allocator will be very complicated, since it's unclear which
allocator should be used.
Adding SIMD alignment should be a very small modification; it can be
done as simply as using ctypes to wrap posix_memalign (or a portable
version, possibly written in python) and writing a simple python
function that checks the beginning data address. There's really no
need to make it complicated.
More information about the NumPy-Discussion