[Numpy-discussion] reading 10 bit raw data into an array

[Sebastian Haase]

On 8/1/07, Danny Chan <chan_dhf at yahoo.de> wrote:
> Hi Travis!
> I guess I will still have to pad my data to full bytes before reading it,
> correct?
>
> Travis Oliphant <oliphant.travis at ieee.org> schrieb:
> Danny Chan wrote:
> > Hi all!
> > I'm trying to read a data file that contains a raw image file. Every
> > pixel is assigned a value from 0 to 1023, and all pixels are stored from
> > top left to bottom right pixel in binary format in this file. I know the
> > width and the height of the image, so all that would be required is to
> > read 10 bits at a time and store it these as an integer. I played around
> > with the fromstring and fromfile function, and I read the documentation
> > for dtype objects, but I'm still confused. It seems simple enough to
> > read data in a format with a standard bitwidth, but how can I read data
> > in a non-standard format. Can anyone help?
> >
>
> This kind of bit-manipulation must be done using bit operations on
> standard size data types even in C. The file reading and writing
> libraries use bytes as their common denominator.
>
> I would read in the entire image into a numpy array of unsigned bytes
> and then use slicing, masking, and bit-shifting to take 5 bytes at a
> time and convert them to 4 values of a 16-bit unsigned image.
>
> Basically, you would do something like
>
> # read in entire image into 1-d unsigned byte array
> # create 16-bit array of the correct 2-D size
> # use flat indexing to store into the new array
> # new.flat[::4] = old[::5] + bitwise_or(old[1::5], MASK1b) << SHIFT1b
> # new.flat[1::4] = bitwise_or(old[1::5], MASK2a) << SHIFT2a
> + bitwise_or(old[2::5], MASK2b) << SHIFT2b
>
> # etc.
>
>
> The exact MASKS and shifts to use is left as an exercise for the reader :-)


Quick comment :  are you really sure your  camera produces the 12 bit
data in a "12 bit stream" --- all I have ever seen is that cameras
would just use 16 bit for each pixel.  (All you had to know if it uses
the left or the right part of those.  In other words, you might have
to divide (or use  bit shifting) the data by 16.)
Wasteful yes, but much simpler to handel !?


-Sebastian Haase