genfromtxt and comment identifier

Peter Otten __peter__ at web.de
Fri Apr 15 19:20:42 CEST 2011


simona bellavista wrote:

> Hi All,
> 
> I have a problem with reading data from a file using genfromtxt of
> numpy module.
> 
> I have prepared a minimal example similar to the ones presented in
> 
> http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html#splitting-
the-lines-into-columns
> 
> The script is
> 
> import numpy as np
> from StringIO import StringIO
> file = open('esempio.dat')
> for line in file.xreadlines() :
>     if not line : break
>     print line
>     np.genfromtxt(StringIO(line), comments="#", delimiter=",")
> 
> I have a data file - esempio.dat - like the following:
> 
> #
> # Skip me !
> # Skip me too !
> 1, 2
> 3, 4
> 5, 6 #This is the third line of the data
> 7, 8
> # And here comes the last line
> 9, 0
> """
> 
> The code is breaking at the first line, it looks like it doesn't
> recognize # like comment identifier.
> 
> The error in the python interpreter is
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "esempio.py", line 7, in <module>
>     np.genfromtxt(StringIO(line), comments="#", delimiter=",")
>   File "/opt/numpy/1.5.1/lib/python2.6/site-packages/numpy/lib/
> npyio.py", line 1174, in genfromtxt
>     raise IOError('End-of-file reached before encountering data.')
> IOError: End-of-file reached before encountering data.
> 
> 
> It is clear that I haven't understood something, what am I doing
> wrong?

The examples use StringIO to simulate a file, but you are wrapping every 
line of your actual file. The first "simulated file" is then

StringIO("#\n")

i. e. it contains only a comment, no data -- and that's what genfromtxt() 
complains about. Read the file in one fell swoop and you should be OK:

import numpy as np

with open('esempio.dat') as instream:
    print np.genfromtxt(instream, comments="#", delimiter=",")





More information about the Python-list mailing list