[Numpy-discussion] genfromtxt converter question

[Derek Homeier]

Hi Gary,

On 17.06.2011, at 5:39PM, gary ruben wrote:
> Thanks for the hints Olivier and Bruce. Based on them, the following
> is a working solution, although I still have that itchy sense that genfromtxt
> should be able to do it directly.
> 
> import numpy as np
> from StringIO import StringIO
> 
> a = StringIO('''\
> (-3.9700,-5.0400) (-1.1318,-2.5693) (-4.6027,-0.1426) (-1.4249, 1.7330)
> (-5.4797, 0.0000) ( 1.8585,-1.5502) ( 4.4145,-0.7638) (-0.4805,-1.1976)
> ( 0.0000, 0.0000) ( 6.2673, 0.0000) (-0.4504,-0.0290) (-1.3467, 1.6579)
> ( 0.0000, 0.0000) ( 0.0000, 0.0000) (-3.5000, 0.0000) ( 2.5619,-3.3708)
> ''')
> 
> b = np.genfromtxt(a, dtype=str, delimiter=18)[:,:-1]
> b = np.vectorize(lambda x: complex(*eval(x)))(b)
> 
> print b

It should, I think you were very close in your earlier attempt:

> On Sat, Jun 18, 2011 at 12:31 AM, Bruce Southey <bsouthey at gmail.com> wrote:
>> On 06/17/2011 08:51 AM, Olivier Delalleau wrote:
>> 
>> 2011/6/17 Bruce Southey <bsouthey at gmail.com>
>>> 
>>> On 06/17/2011 08:22 AM, gary ruben wrote:
>>>> Thanks Olivier,
>>>> Your suggestion gets me a little closer to what I want, but doesn't
>>>> quite work. Replacing the conversion with
>>>> 
>>>> c = lambda x:np.cast[np.complex64](complex(*eval(x)))
>>>> b = np.genfromtxt(a,converters={0:c, 1:c, 2:c,
>>>> 3:c},dtype=None,delimiter=18,usecols=range(4))
>>>> 
>>>> produces
>>>> 
>>>> [[(-3.97000002861-5.03999996185j) (-1.1318000555-2.56929993629j)
>>>>   (-4.60270023346-0.142599999905j) (-1.42490005493+1.73300004005j)]
>>>>   [(-5.4797000885+0j) (1.85850000381-1.5501999855j)
>>>>   (4.41450023651-0.763800024986j) (-0.480500012636-1.19760000706j)]
>>>>   [0j (6.26730012894+0j) (-0.45039999485-0.0289999991655j)
>>>>   (-1.34669995308+1.65789997578j)]
>>>>   [0j 0j (-3.5+0j) (2.56189990044-3.37080001831j)]]
>>>> 
>>>> which is not yet an array of complex numbers. It seems close to the
>>>> solution though.

You were just overdoing it by already creating an array with the converter, this apparently caused genfromtxt to create a structured array from the input (which could be converted back to an ndarray, but that can prove tricky as well) - similar, if you omit the dtype=None. The following

cnv = dict.fromkeys(range(4), lambda x: complex(*eval(x)))
b = np.genfromtxt(a,converters=cnv, dtype=None, delimiter=18, usecols=range(4))

directly produces a shape(4,4) complex array for me (you may have to apply an .astype(np.complex64) afterwards if so desired).

BTW I think this is an interesting enough case of reading non-trivially structured data that it deserves to appear on some examples or cookbook page.

HTH,
								Derek