[Numpy-discussion] Deprecating PyDataMem_RENEW ?

[David Cournapeau]

Anne Archibald wrote:
>
> How much does this matter? I mean, what if you simply left all the
> reallocs as-is? The arrays that resulted from reallocs would not
> typically be aligned, but we cannot in any case expect all arrays to
> be aligned.

The problem would be the interaction between the aligned allocator and 
realloc. If we do it by "ourselves", then it is no problem, but we 
cannot do it if we rely on posix_memalign/memalign. I was not precise 
enough when I mentioned portability: I meant that it was not possible to 
implement realloc in terms of malloc/free portably, and as such, it is 
not possible to implement an aligned realloc with memalign/free. Also, I 
have not found any indication that the pointer given by posix_memalign 
can be fed to realloc: depending on the implementation, it cannot be 
freed...

Maybe the solution is to never use posix_memalign...

> As I understand it, a
> portable aligned malloc allocates extra memory, then adds something to
> the pointer to make the memory arena aligned. 

Yes, that's how it works in fftw, and the implementation I have for 
numpy is derived from fftw code.

> I don't think any numpy function can assume that its input arrays are
> aligned, since users may like to pass, say, mmap()ed hunks of a file,
> over which numpy has no control of the alignment. In this case, then,
> how much of a problem is it if a few stray realloc()s produce
> non-aligned arrays?

There are a few realloc, but they are used in core routines 
(PyArray_FromIter, PyArray_Resize). The point is to get aligned arrays 
most of the time, and avoiding losing alignement unless it is too 
complicated to do otherwise.

cheers,

David