[Numpy-discussion] numpy.load raising IOError but EOFError expected

[Ruben Salvador]

Sorry, I forgot to include versions info:
Python 2.5.5
Numpy Version: 1:1.3.0-3+b1 (actual debian testing)


On Wed, Jun 23, 2010 at 12:46 PM, Ruben Salvador <rsalvador.wk at gmail.com>wrote:

> Hi there,
>
> I have a .npy file built by succesively adding results from different test
> runs of an algorithm. Each time it's run, I save a numpy.array using
> numpy.save as follows:
>
> fn = 'file.npy'
> f = open(fn, 'a+b')
> np.save(f, arr)
> f.close()
>
> When I try to read the file with the following code, for a file containing
> 3 array saves (a is there to show when the error arises):
>
> f = open(fn, 'rb')
> arr = np.load(f)
> a = 0
> try:
>     while True:
>         print a
>         a += 1
>         arr = np.vstack((arr, np.load(f)))
> except EOFError:
>     pass
> f.close()
>
> I get the following output:
>
> 0
> 1
> 2
> Traceback (most recent call last):
>   File "./proc_stat.py", line 32, in <module>
>     arr = np.vstack((arr, np.load(f)))
>   File "/usr/lib/python2.5/site-packages/numpy/lib/io.py", line 201, in
> load
>     "Failed to interpret file %s as a pickle" % repr(file)
> IOError: Failed to interpret file <open file
> '/home/rsalvador/trabajo/research/phd/devel/evowv/retest/avgfit_random_sigma_normal_Flp_20G.npy',
> mode 'rb' at 0x174ca08> as a pickle
>
> Using IOError in the except makes the code work, but this way I am masking
> other possible sources of error.
>
> I have tried with a file containing 3 dumps from numpy.save (this is, 3
> arrays saved). As shown, the error is raised when trying to read a fourth
> time (since EOFError is not raised).
>
> Therefore, is this a bug? Shouldn't EOFError be raised instead of IOError?
> Or am I missunderstanding something? If this is not a bug, how can I detect
> the EOF to stop reading (I expect a way for this to work without tweaking
> the code with saving first in the file the number of dumps done)?
>
> Thanks in advance!
>
> --
> Rubén Salvador
> PhD student @ Centro de Electrónica Industrial (CEI)
> http://www.cei.upm.es
> Blog: http://aesatcei.wordpress.com
>



-- 
Rubén Salvador
PhD student @ Centro de Electrónica Industrial (CEI)
http://www.cei.upm.es
Blog: http://aesatcei.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100623/a7484293/attachment.html>