[Numpy-discussion] genfromtxt

[Derek Homeier]

Hi Nils,

On 11 Oct 2011, at 16:34, Nils Wagner wrote:

> How do I use genfromtxt to read a file with the following 
> lines
> 
>              1            1  2.2592365264892578D+01
>              2            2  2.2592365264892578D+01
>              1            3  2.6666669845581055D+00
>              3            3  2.2592365264892578D+01
>              2            4  2.6666669845581055D+00
>              4            4  2.2592365264892578D+01
>              3            5  2.6666669845581055D+00
>              5            5  2.2592365264892578D+01
>              4            6  2.6666669845581055D+00
>              6            6  2.2592365264892578D+01
>              1            7  2.9814243316650391D+00
>              7            7  1.7259031295776367D+01
>              2            8  2.9814243316650391D+00
>              8            8  1.7259031295776367D+01
> ...
> 
> 
> names =("i","j","v")
> A = 
> np.genfromtxt('bmll.mtl',dtype=[('i','int'),('j','int'),('v','d')],names=names)
> V = A[:]['v']
> 
>>>> V
> array([ NaN,  NaN,  NaN,  NaN,  NaN,  NaN,  NaN,  NaN, 
> NaN,  NaN,  NaN,
>         NaN,  NaN,  NaN,  NaN,  NaN,  NaN,  NaN,  NaN, 
> NaN,  NaN,  NaN,
>         NaN,  NaN,  NaN,  NaN,  NaN,  NaN,  NaN,  NaN, 
> NaN,  NaN,  NaN,
>         NaN,  NaN,  NaN])
> 
> yields NaN, while
> 
> convertfunc = lambda x: x.replace('D','E')
> names =("i","j","v")
> A = 
> np.genfromtxt('bmll.mtl',dtype=[('i','int'),('j','int'),('v','|S24')],names=names,converters={"v":convertfunc})
> V = A[:]['v'].astype(float)
>>>> V
> array([ 22.59236526,  22.59236526,   2.66666698, 
> 22.59236526,
>          2.66666698,  22.59236526,   2.66666698, 
> 22.59236526,
>          2.66666698,  22.59236526,   2.98142433, 
> 17.2590313 ,
>          2.98142433,  17.2590313 ,   2.98142433, 
>  2.98142433,
>          2.66666698,  22.59236526,   2.98142433, 
>  2.98142433,
>          2.66666698,  22.59236526,   2.98142433, 
>  2.98142433,
>          2.66666698,  22.59236526,   2.98142433, 
>  2.98142433,
>          2.66666698,  22.59236526,   2.98142433, 
>  2.66666698,
>         17.2590313 ,   2.98142433,   2.66666698, 
> 17.2590313 ])
> 
> 
> works fine.

took me a moment to figure out what the actual problem remaining was, 
but expect you'd prefer it to load directly into a float record?
The problem is simply that the converter _replaces_ the default converter 
function (which would be float(x) in this case), rather than operating on top of it.

Try instead
convertfunc = lambda x: float(x.replace('D','E')) 
and you should be ready to use ('v', 'd') as dtype (BTW, specifying 'names' 
is redundant in the above example). 

This behaviour is only hinted at in the docstring example, so maybe the 
documentation should be clearer here.

Cheers,
						Derek