Andrew Straw <strawman@astraw.com> writes:
Alexander Schmolck wrote:
2. Despite this overhead, copying around large arrays (e.g. >=1e5 elements) in above way causes notable additional overhead. Whilst I don't think there's a sane way to avoid copying by sharing data between numpy and matlab the copying could likely be done better.
Alex, what do you think about "hybrid arrays"?
http://www.mail-archive.com/numpy-discussion@lists.sourceforge.net/msg03748....
Oh, that's why I said no *sane* way :) I read about hybrid arrays, but as far as I can tell the only way to use them to avoid copying stuff around is to create your own hybrid array memory pool (as you suggested downthread), which seems a highly unattractive pain-for-gain trade-off, especially given that you *do* have to reorder the data as you go from numpy to matlab -- unless of course I'm missing something. I have the impression that just memcpy'ing a single chunk of data isn't that much of a performance sink, but the reordering copying that mlabwrap currently does seems rather expensive. In other words, I think going from matlab to numpy is fine (just memcpy into a newly created fortran-order numpy array, or more or less equivalently, memcpy into a C-order array, transpose and reshape), the question appears to be how to best go from numpy to matlab when the numpy array isn't fortran-contiguous. I assume that it makes more sense to rely on some numpy functionality than to use a custom reordering-copy routine, especially if I want to move to ctypes later. Is there anything better than 1. allocating a matlab array 2. transposing and reshaping the numpy array 3. allocating (or keeping around) a temporary numpy array with data pointing to the matlab array data 4. using some function (PyArray_CopyInto?) to copy from the transposed, reshaped numpy array into the temporary numpy array thereby filling the matlab array with an appropriately reordered copy of the original array ? cheers, alex