[SciPy-User] loadtxt and complicated dtype

[Giovanni Luca Ciampaglia]

Il 16. 08. 11 08:50, Gerrit Holl ha scritto:
> In my real case I
> have a 2-D array M with shape (5000, 839). I have my complicated
> dtype:
>
> [('temp',<type 'numpy.float64'>, 91),
>   ('hum',<type 'numpy.float64'>, 91),
>   ...,
>   ('gpoint',<type 'numpy.uint32'>, 1),
>   ('ind',<type 'numpy.uint16'>, 1)]
> ]
>
> whose numbers add up to 839. How do I turn this into an array of size
> (5000,) with my requested dtype?
> - .view(dtype) does not do what I mean, because this interprets the
> actual bytes, and my new array will have a different number of bytes
> compared to the old one
> - array(M, dtype) does not do what I mean, because this will try to
> expand every element of M according to the requested dtype, does
> making the array much larger (and throwing a MemoryError).
>
> I want this, because it's a very convenient way to access fields of my
> data. It's more convenient to say M["ciw"] than to say M[:, 455:546].
> If someone can suggest another way to achieve this convenience, I'm
> open for suggestions.

Hi Gerrit,
you could use numpy.empty:

# x has shape (5000, 184)

ty = dtype([('temp', float64, 91),
('hum', float64, 91),
('gpoint', int32, 1),
('ind', int16, 1)])

data = empty((5000,), ty)

# copy the individual columns

data['temp'] = x[:,:91]
data['hum'] = x[:,91:182]
data['gpoint'] = x[:,182]
data['ind'] = x[:,183]

you can probably do the assignments in a for loop using the shape 
information from the individual fields

cheers

-- 
Giovanni Luca Ciampaglia

Ph.D. Candidate
Faculty of Informatics
University of Lugano
Web: http://www.inf.usi.ch/phd/ciampaglia/

Bertastraße 36 ∙ 8003 Zürich ∙ Switzerland