[Numpy-discussion] unpack argument in loadtxt/genfromtxt does not work as documented

[Pierre GM]

On Oct 7, 2010, at 4:48 AM, Chris Fonnesbeck wrote:

> The documentation for loadtxt and genfromtxt state that the unpack
> argument functions as follows:
> 
> If True, the returned array is transposed, so that arguments may be
> unpacked using x, y, z = loadtxt(...).

Provided that all the columns have the same dtype

> 
> In practice, this does not always occur. I have a csv file of mixed
> data types, and try importing it via:
> 
> genfromtxt("progestogens.csv", delimiter=",", names=True,
> dtype=dtype([('id', int),('study', '|S25'),('year', int),('treat',
> int),('drug', '|S25'),('form', '|S10'),('ptb', int),('mgest',
> int),('lab', int),('n', int),('y', int),('var', '|S5'),('wt',
> int),('sdwt', int)]), unpack=True)
> 
> With unpack=True, I would expect the data to be presented by columns,
> however the resulting array is by rows:

Well, you have a complex dtype, so your result array is 1D, each row corresponding to a tuple of elements with different dtypes.


> 
> 
> array([(1, 'Meis', 2003, 1, '17P', 'IM', 1, 0, 0, 306, 111, 'ptb'),
>       (1, 'Meis', 2003, 0, '17P', 'IM', 1, 0, 0, 153, 84, 'ptb'),
>       (2, 'Rai', 2009, 1, 'Progesterone', 'Oral', 1, 0, 0, 74, 29, 'ptb'),
>       (2, 'Rai', 2009, 0, 'Progesterone', 'Oral', 1, 0, 0, 74, 44, 'ptb'),
> ...
> 
> The same behaviour occurs using loadtxt. Moreover, this array is
> untransposeable, so I am stuck with having to iterate over all the
> rows, making genfromtxt no better than csv.reader.

Once again, your array is 1D, so you can't tranpose it.
Now, you should be able to get each column through
>>> [a[_] for _ in a.dtype.names]