[Numpy-discussion] Is there anyway to read raw binary file via pytable?

[Christopher Barker]

Francesc Alted wrote:
> A Wednesday 28 July 2010 18:05:11 脑关(BrainGateway)生命科学仪器 escrigué:
>> I had some 500GB numerical arrays
>> to process. Pytable claims to have some advance feature to enhance
>> processing speed and largely reduce physical memory requirement. However, I
>> do not wanna touch the raw data I had. Simply because I do not have doubled
>> diskspace to covert all 10TB data into HDF5. Is there any way to let
>>  pytable read raw binary files or alternatively to package raw files into
>>  HDF5 format without change the files themselves.?
> 
> No, PyTables does not support raw data directly.  You have to convert to HDF5 
> first.

Which is probably worth it.

But if your data files are pretty simple, you may be able to use a 
memory mapped array, or baring that, write your own reader with fromfile().


-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov