On Oct 7, 2010, at 4:48 AM, Chris Fonnesbeck wrote:
The documentation for loadtxt and genfromtxt state that the unpack argument functions as follows:
If True, the returned array is transposed, so that arguments may be unpacked using x, y, z = loadtxt(...).
Provided that all the columns have the same dtype
In practice, this does not always occur. I have a csv file of mixed data types, and try importing it via:
genfromtxt("progestogens.csv", delimiter=",", names=True, dtype=dtype([('id', int),('study', '|S25'),('year', int),('treat', int),('drug', '|S25'),('form', '|S10'),('ptb', int),('mgest', int),('lab', int),('n', int),('y', int),('var', '|S5'),('wt', int),('sdwt', int)]), unpack=True)
With unpack=True, I would expect the data to be presented by columns, however the resulting array is by rows:
Well, you have a complex dtype, so your result array is 1D, each row corresponding to a tuple of elements with different dtypes.
array([(1, 'Meis', 2003, 1, '17P', 'IM', 1, 0, 0, 306, 111, 'ptb'), (1, 'Meis', 2003, 0, '17P', 'IM', 1, 0, 0, 153, 84, 'ptb'), (2, 'Rai', 2009, 1, 'Progesterone', 'Oral', 1, 0, 0, 74, 29, 'ptb'), (2, 'Rai', 2009, 0, 'Progesterone', 'Oral', 1, 0, 0, 74, 44, 'ptb'), ...
The same behaviour occurs using loadtxt. Moreover, this array is untransposeable, so I am stuck with having to iterate over all the rows, making genfromtxt no better than csv.reader.
Once again, your array is 1D, so you can't tranpose it. Now, you should be able to get each column through
[a[_] for _ in a.dtype.names]