reading 10 bit raw data into an array
Hi all! I'm trying to read a data file that contains a raw image file. Every pixel is assigned a value from 0 to 1023, and all pixels are stored from top left to bottom right pixel in binary format in this file. I know the width and the height of the image, so all that would be required is to read 10 bits at a time and store it these as an integer. I played around with the fromstring and fromfile function, and I read the documentation for dtype objects, but I'm still confused. It seems simple enough to read data in a format with a standard bitwidth, but how can I read data in a non-standard format. Can anyone help? Greets, Danny --------------------------------- Alles was der Gesundheit und Entspannung dient.BE A BETTER MEDIZINMANN!
On Mon, Jul 30, 2007 at 04:01:46PM +0200, Danny Chan wrote:
I'm trying to read a data file that contains a raw image file. Every pixel is assigned a value from 0 to 1023, and all pixels are stored from top left to bottom right pixel in binary format in this file. I know the width and the height of the image, so all that would be required is to read 10 bits at a time and store it these as an integer. I played around with the fromstring and fromfile function, and I read the documentation for dtype objects, but I'm still confused. It seems simple enough to read data in a format with a standard bitwidth, but how can I read data in a non-standard format. Can anyone help?
AFAIK, numpy's dtypes all have widths >= 1 byte. The easiest solution I can think of is to use fromfile to read 5 bytes at a time, and then to use divmod to obtain your 4 values. Cheers Stéfan
Danny Chan wrote:
Hi all! I'm trying to read a data file that contains a raw image file. Every pixel is assigned a value from 0 to 1023, and all pixels are stored from top left to bottom right pixel in binary format in this file. I know the width and the height of the image, so all that would be required is to read 10 bits at a time and store it these as an integer. I played around with the fromstring and fromfile function, and I read the documentation for dtype objects, but I'm still confused. It seems simple enough to read data in a format with a standard bitwidth, but how can I read data in a non-standard format. Can anyone help?
This kind of bit-manipulation must be done using bit operations on standard size data types even in C. The file reading and writing libraries use bytes as their common denominator. I would read in the entire image into a numpy array of unsigned bytes and then use slicing, masking, and bit-shifting to take 5 bytes at a time and convert them to 4 values of a 16-bit unsigned image. Basically, you would do something like # read in entire image into 1-d unsigned byte array # create 16-bit array of the correct 2-D size # use flat indexing to store into the new array # new.flat[::4] = old[::5] + bitwise_or(old[1::5], MASK1b) << SHIFT1b # new.flat[1::4] = bitwise_or(old[1::5], MASK2a) << SHIFT2a + bitwise_or(old[2::5], MASK2b) << SHIFT2b # etc. The exact MASKS and shifts to use is left as an exercise for the reader :-) -Travis
Hi Travis! I guess I will still have to pad my data to full bytes before reading it, correct? Travis Oliphant <oliphant.travis@ieee.org> schrieb: Danny Chan wrote:
Hi all! I'm trying to read a data file that contains a raw image file. Every pixel is assigned a value from 0 to 1023, and all pixels are stored from top left to bottom right pixel in binary format in this file. I know the width and the height of the image, so all that would be required is to read 10 bits at a time and store it these as an integer. I played around with the fromstring and fromfile function, and I read the documentation for dtype objects, but I'm still confused. It seems simple enough to read data in a format with a standard bitwidth, but how can I read data in a non-standard format. Can anyone help?
This kind of bit-manipulation must be done using bit operations on standard size data types even in C. The file reading and writing libraries use bytes as their common denominator. I would read in the entire image into a numpy array of unsigned bytes and then use slicing, masking, and bit-shifting to take 5 bytes at a time and convert them to 4 values of a 16-bit unsigned image. Basically, you would do something like # read in entire image into 1-d unsigned byte array # create 16-bit array of the correct 2-D size # use flat indexing to store into the new array # new.flat[::4] = old[::5] + bitwise_or(old[1::5], MASK1b) << SHIFT1b # new.flat[1::4] = bitwise_or(old[1::5], MASK2a) << SHIFT2a + bitwise_or(old[2::5], MASK2b) << SHIFT2b # etc. The exact MASKS and shifts to use is left as an exercise for the reader :-) -Travis _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion --------------------------------- Jetzt Mails schnell in einem Vorschaufenster überfliegen. Dies und viel mehr bietet das neue Yahoo! Mail.
On 8/1/07, Danny Chan <chan_dhf@yahoo.de> wrote:
Hi Travis! I guess I will still have to pad my data to full bytes before reading it, correct?
Travis Oliphant <oliphant.travis@ieee.org> schrieb: Danny Chan wrote:
Hi all! I'm trying to read a data file that contains a raw image file. Every pixel is assigned a value from 0 to 1023, and all pixels are stored from top left to bottom right pixel in binary format in this file. I know the width and the height of the image, so all that would be required is to read 10 bits at a time and store it these as an integer. I played around with the fromstring and fromfile function, and I read the documentation for dtype objects, but I'm still confused. It seems simple enough to read data in a format with a standard bitwidth, but how can I read data in a non-standard format. Can anyone help?
This kind of bit-manipulation must be done using bit operations on standard size data types even in C. The file reading and writing libraries use bytes as their common denominator.
I would read in the entire image into a numpy array of unsigned bytes and then use slicing, masking, and bit-shifting to take 5 bytes at a time and convert them to 4 values of a 16-bit unsigned image.
Basically, you would do something like
# read in entire image into 1-d unsigned byte array # create 16-bit array of the correct 2-D size # use flat indexing to store into the new array # new.flat[::4] = old[::5] + bitwise_or(old[1::5], MASK1b) << SHIFT1b # new.flat[1::4] = bitwise_or(old[1::5], MASK2a) << SHIFT2a + bitwise_or(old[2::5], MASK2b) << SHIFT2b
# etc.
The exact MASKS and shifts to use is left as an exercise for the reader :-)
Quick comment : are you really sure your camera produces the 12 bit data in a "12 bit stream" --- all I have ever seen is that cameras would just use 16 bit for each pixel. (All you had to know if it uses the left or the right part of those. In other words, you might have to divide (or use bit shifting) the data by 16.) Wasteful yes, but much simpler to handel !? -Sebastian Haase
Quick comment : are you really sure your camera produces the 12 bit data in a "12 bit stream" --- all I have ever seen is that cameras would just use 16 bit for each pixel. (All you had to know if it uses the left or the right part of those. In other words, you might have to divide (or use bit shifting) the data by 16.) Wasteful yes, but much simpler to handel !?
10 bit, but yes, I am sure. It is an embedded camera system, in fact, I get the data stream even before it is handled to any ISP for further processing of the picture. In the end, the ISP will convert the data stream to another format, but I have to simulate some of the algorithms that will be implemented in hardware.
participants (4)
-
Danny Chan -
Sebastian Haase -
Stefan van der Walt -
Travis Oliphant