converting a C bytes array to two dimensional numpy array
Hi, I have a question, regarding conversion of C (unsigned char *) buffer to a two dimensional numpy array this is what i am doing: 1) I get a C network buffer of unsigned char * let's call it the source buffer the size of the source buffer is: W * H * 2 bytes 2) I am using PyByteArray_FromStringAndSize() to convert the source buffer (a C unsigned char *) to python bytes array. a = PyByteArray_FromStringAndSize(source buffer, W * H * 2) 3) i am using numpy.frombuffer to convert the python bytes array to a 1 dimensional numpy array of size W *H *2 bytes b = numpy.frombuffer(a, dtype = np.uint8) 4) i am creating a 2 dimensional numpy array from (3) when each element in that array is made of 2 bytes from the python bytes array c = b.view(np.uint16).reshape((H, W)) Is there a way to optimize this some how ? Can you suggest a faster and better solution ? Thanks,
On 16 Jul 2019, at 9:30 am, Omry Levy
I have a question, regarding conversion of C (unsigned char *) buffer to a two dimensional numpy array
this is what i am doing: 1) I get a C network buffer of unsigned char * let's call it the source buffer the size of the source buffer is: W * H * 2 bytes
2) I am using PyByteArray_FromStringAndSize() to convert the source buffer (a C unsigned char *) to python bytes array. a = PyByteArray_FromStringAndSize(source buffer, W * H * 2)
3) i am using numpy.frombuffer to convert the python bytes array to a 1 dimensional numpy array of size W *H *2 bytes b = numpy.frombuffer(a, dtype = np.uint8)
4) i am creating a 2 dimensional numpy array from (3) when each element in that array is made of 2 bytes from the python bytes array c = b.view(np.uint16).reshape((H, W))
Is there a way to optimize this some how ? Can you suggest a faster and better solution ?
The PyByteArray conversion seems unnecessary - if you can access your input as a buffer, calling np.frombuffer on it directly with the correct dtype should work just as well, and you can reshape it on the fly: c = np.frombuffer(source_buffer, dtype=np.uint16, [count=W*H]).reshape((H, W)) The optional ‘count’ argument would only be required if you cannot simply read the buffer to its end. HTH, Derek
You can also directly build a numpy array from a pointer with the numpy
API.
And I recommend Cython as an interface to make these things easy.
This does mean you’d need to have the numpy lib at build time, .which may
be a downside.
-CHB
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 x-apple-data-detectors://6/1 (206) 526-6317
main reception
On Jul 16, 2019, at 5:48 AM, Derek Homeier <
derek@astro.physik.uni-goettingen.de> wrote:
On 16 Jul 2019, at 9:30 am, Omry Levy
participants (3)
-
Chris Barker - NOAA Federal
-
Derek Homeier
-
Omry Levy