Re: [Numpy-discussion] checksum on numpy float array

josef.pktd@gmail.com wrote:

On Thu, Dec 4, 2008 at 6:17 PM, Brennan Williams wrote:

...

My app reads in one or more float arrays from a binary file.

Sometimes due to network timeouts etc the array is not read correctly.

What would be the best way of checking the validity of the data?

Would some sort of checksum approach be a good idea? Would that work with an array of floating point values? Or are checksums more for int,byte,string type data?

If you want to verify the file itself, then python provides several more or less secure checksums, my experience was that zlib.crc32 was pretty fast on moderate file sizes. crc32 is common inside archive files and for binary newsgroups. If you have large files transported over the network, e.g. GB size, I would work with par2 repair files, which verifies and repairs at the same time.

The file has multiple arrays stored in it. So I want to have some sort of validity check on just the array that I'm reading. I will need to add a check on the file as well as of course network problems could affect writing to the file as well as reading from the file.

Josef _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion