file data => array(s)

[Dave Angel]

On 12/14/2011 05:20 PM, Eric wrote:
> I'm trying to read some file data into a set of arrays.  The file data
> is just four columns of numbers, like so:
>
>     1.2    2.2   3.3  0.5
>     0.1   0.2    1.0  10.1
>     ... and so on
>
> I'd like to read this into four arrays, one array for each column.
> Alternatively, I guess something like this is okay too:
>
>     [[1.2, 2.2, 3.3, 0.5], [0.1, 0.2, 1.0, 10.1], ... and so on]
>
> I came up with the following for the four array option:
>
>     file = open(fileName, 'r')
>     for line in file.readlines():
The readlines() call is a waste of time/space.  file is already an 
iterator that'll return lines for you.
>        d1, e1, d2, e2 = map(float, line.split())
>        data1.append(d1)  # where data1, err1, data2, err2 are init-ed
> as empty lists
>        err1.append(e1)
>        data2.append(d2)
>        err2.append(e2)
>     file.close()
>
> But somehow it doesn't seem very python-esque (I'm thinking there's a
> more elegant and succinct way to do it in python).  I've also tried
> replacing the above "map" line with:
>
>        d = d + map(float, line.split())  # where d is initialized as d
> = []
>
> But all I get is one long flat list, not what I want.
>
> So is the map and append method the best I can do or is there a
> slicker way?
>
> One more thing, no numpy.  Nothing against numpy but I'm curious to
> see what can be done with just the box stock python install.
>
> TIA,
> eric
When I see a problem like this, I turn to zip().  It's got some powerful 
uses when rows and columns need inverting.

I didn't try it on an actual file, but the following works:
linedata =    [[1.2, 2.2, 3.3, 0.5], [0.1, 0.2, 1.0, 10.1] ]

data, err1, data2, err2 = zip(*linedata)

print data
print err1
print data2
print err2

So you could try (untested)

file = open(filename, "r")
linedata = [ map(float, line) for line in file]
data, err1, data2, err2 = zip(*linedata)
file.close()

Note that your code won't work (and mine probably won't either) if one 
of the lines has 3 or 5 items.  Or if one of the numbers isn't legal 
format for a float.  So you need to think about error checking, or 
decide whether a partial result is important.

-- 

DaveA