[Numpy-discussion] numpy.load raising IOError but EOFError expected

[Ruben Salvador]

Hi there,

I have a .npy file built by succesively adding results from different test
runs of an algorithm. Each time it's run, I save a numpy.array using
numpy.save as follows:

fn = 'file.npy'
f = open(fn, 'a+b')
np.save(f, arr)
f.close()

When I try to read the file with the following code, for a file containing 3
array saves (a is there to show when the error arises):

f = open(fn, 'rb')
arr = np.load(f)
a = 0
try:
    while True:
        print a
        a += 1
        arr = np.vstack((arr, np.load(f)))
except EOFError:
    pass
f.close()

I get the following output:

0
1
2
Traceback (most recent call last):
  File "./proc_stat.py", line 32, in <module>
    arr = np.vstack((arr, np.load(f)))
  File "/usr/lib/python2.5/site-packages/numpy/lib/io.py", line 201, in load
    "Failed to interpret file %s as a pickle" % repr(file)
IOError: Failed to interpret file <open file
'/home/rsalvador/trabajo/research/phd/devel/evowv/retest/avgfit_random_sigma_normal_Flp_20G.npy',
mode 'rb' at 0x174ca08> as a pickle

Using IOError in the except makes the code work, but this way I am masking
other possible sources of error.

I have tried with a file containing 3 dumps from numpy.save (this is, 3
arrays saved). As shown, the error is raised when trying to read a fourth
time (since EOFError is not raised).

Therefore, is this a bug? Shouldn't EOFError be raised instead of IOError?
Or am I missunderstanding something? If this is not a bug, how can I detect
the EOF to stop reading (I expect a way for this to work without tweaking
the code with saving first in the file the number of dumps done)?

Thanks in advance!

-- 
Rubén Salvador
PhD student @ Centro de Electrónica Industrial (CEI)
http://www.cei.upm.es
Blog: http://aesatcei.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100623/afcc8b59/attachment.html>