[Numpy-discussion] Initializing array from buffer

[Andrea Arteaga]

Hi.
Yesterday I tried to make use of the C API, but I did not manage to have
anything useful. The reference is very well done, but I feel the lack for
some tutorial that would guide with some examples. Do you know of any?

The array interface looks sounds like a very good solution. In a sense it
is a Numpy specific version of the buffer protocol, a bit simpler but less
generic. It looks very easy to implement and clean. I will try this way.

Thank you so much for the useful links.
Andrea Arteaga

2014-11-17 18:08 GMT+01:00 Edison Gustavo Muenz <edisongustavo at gmail.com>:

> Have you tried using the C-API to create the array? This link might be of
> help:
> http://docs.scipy.org/doc/numpy/reference/c-api.array.html#creating-arrays
>
> I know that Boost.Python can handle this.
>
> On Sun, Nov 16, 2014 at 3:42 PM, Andrea Arteaga <andyspiros at gmail.com>
> wrote:
>
>> Hello.
>> Using the numpy.frombuffer function [1] one can initialize a numpy array
>> using an existing python object that implements the buffer protocol [2].
>> This is great, but currently this function supports only 1D buffers, even
>> if the provided buffer is multidimensional and it exposes all information
>> about its structure (shape, strides, data type).
>>
>> Apparently, one can extract every kind of buffer information out of a
>> buffer of a numpy array (pointer, number of dimensions, shape, strides,
>> suboffsets,...), but the other way around is only partially implemented:
>> providing a multidimensional buffer does not mean being able of creating a
>> numpy array the uses that buffer with the desired structure.
>>
>> My use case is the following: we have a some 3D arrays in our C++
>> framework. The ordering of the elements in these arrays is neither C nor
>> Fortran style: it might be IJK (i.e. C style, 3rd dimension contiguous in
>> memory), KJI (i.e. Fortran style, first dimension contiguous) or, e.g. IKJ.
>> Moreover we put some padding to optimize aligned access. This kind of
>> memory structure cannot be just expressed as 'C' or 'Fortran', but it can
>> be perfectly expressed using the Python buffer protocol by providing the
>> shape and the strides. We would like to export this structure to a numpy
>> array that should be able of accessing the same memory locations in a
>> consistent way and make some operations like initializing the content or
>> plotting it.
>>
>> Is this currently possible?
>> If not, is it planned to implement such a feature?
>>
>> ==========
>>
>> Maybe just to clarify I could show an example entirely in python. Assume
>> a in a 2D numpy array:
>>
>> a = np.ones((10,20))
>>
>> It contains information about its structure which can be portably
>> accessed using its data member:
>>
>> a.data.format == 'd'
>> a.data.ndim == 2
>> a.data.shape == (10,20)
>> a.data.strides == (160,8)
>>
>> Unfortunately, when initializing an array b from this buffer, the
>> structure of the buffer is "downgraded" to unidimensional shape:
>>
>> b = np.frombuffer(a.data)
>>
>> b.ndim == 1
>> b.shape == (200,)
>> b.strides == (8,)
>>
>> I wished b had the same multi-dimensional structure of a.
>>
>> (This is of course a very simple example. In my use case I would
>> initialize b with my own buffer instead of that of another numpy array).
>>
>> Best regards
>>
>> [1]
>> http://docs.scipy.org/doc/numpy/reference/generated/numpy.frombuffer.html
>> [2] https://docs.python.org/3/c-api/buffer.html
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20141117/68e8ea2a/attachment.html>