[Numpy-discussion] List to Array question?

[Eric Firing]

Jason.Woolard at noaa.gov wrote:
> hi all,
> 
> I'm sort of new to Numpy and I haven't had any luck with the docs or examples on this so I thought I would ask here. I have this small piece of code that's working but I'm wondering if the list really needs to be created or if this is an extra step that could be eliminated and speed things up a bit. It seems like the data could be dumped directly into the numpy array. 
> 

If the file consists of a header followed by a sequence of binary 
numbers, then you can use the numpy.fromfile function to read the whole 
sequence into an array of shape (n,3).  You don't even have to know what 
n is beforehand if you are reading to the end of the file.  You do have 
to know how to construct a suitable dtype to match what is in the file. 
  And you do have to know how to read the header, at least to the extent 
of positioning the file at the start of the binary chunk before calling 
numpy.fromfile.

Eric


>>>> infile = file.File('test.dat',mode='r') #A binary file containing x,y,z data
>>>> xdata = []                                    #Create some empty lists 
>>>> ydata = []
>>>> zdata = []
>>>> for p in infile:
> 	     xdata.append(p.x)                    #Append data to list
> 	     ydata.append(p.y)
> 	     zdata.append(p.z)
> 
>>>> easting = numpy.array(xdata,dtype=float32) #Convert to array
>>>> northing = numpy.array(ydata,dtype=float32)
>>>> height = numpy.array(zdata,dtype=float32)
>>>> print height
> [-39.54999924 -39.54999924 -39.61000061 ..., -39.54000092 -39.52999878
>  -39.52999878]
> 
> I also tried this and it worked but I'd have to loop through the file each time (x,y,z) and that was slower than converting from the list.
> 
>>>> xiterator = (p.x for p in infile)
>>>> x = numpy.fromiter(xiterator, dtype=float32, count=-1)
>>>>
>>>> ziterator = (p.z for p in infile)
>>>> z = numpy.fromiter(ziterator, dtype=float32, count=-1)
> 
>>From what I've read the numpy arrays need to pre-allocated and I have a header that will give me this info but I can't seem to get the data into the array.
> 
> Sorry if this is something obvious. Thanks in advance.
> 
> JW
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion