Re: [Numpy-discussion] reading 10 bit raw data into an array

On 8/1/07, Danny Chan wrote:

Hi Travis! I guess I will still have to pad my data to full bytes before reading it, correct?

Travis Oliphant schrieb: Danny Chan wrote:

...

Hi all! I'm trying to read a data file that contains a raw image file. Every pixel is assigned a value from 0 to 1023, and all pixels are stored from top left to bottom right pixel in binary format in this file. I know the width and the height of the image, so all that would be required is to read 10 bits at a time and store it these as an integer. I played around with the fromstring and fromfile function, and I read the documentation for dtype objects, but I'm still confused. It seems simple enough to read data in a format with a standard bitwidth, but how can I read data in a non-standard format. Can anyone help?

This kind of bit-manipulation must be done using bit operations on standard size data types even in C. The file reading and writing libraries use bytes as their common denominator.

I would read in the entire image into a numpy array of unsigned bytes and then use slicing, masking, and bit-shifting to take 5 bytes at a time and convert them to 4 values of a 16-bit unsigned image.

Basically, you would do something like

# read in entire image into 1-d unsigned byte array # create 16-bit array of the correct 2-D size # use flat indexing to store into the new array # new.flat[::4] = old[::5] + bitwise_or(old[1::5], MASK1b) << SHIFT1b # new.flat[1::4] = bitwise_or(old[1::5], MASK2a) << SHIFT2a + bitwise_or(old[2::5], MASK2b) << SHIFT2b

# etc.

The exact MASKS and shifts to use is left as an exercise for the reader :-)

Quick comment : are you really sure your camera produces the 12 bit data in a "12 bit stream" --- all I have ever seen is that cameras would just use 16 bit for each pixel. (All you had to know if it uses the left or the right part of those. In other words, you might have to divide (or use bit shifting) the data by 16.) Wasteful yes, but much simpler to handel !? -Sebastian Haase