[Numpy-discussion] bug in genfromtxt with missing values?

[Bruce Southey]

On 01/25/2011 10:56 AM, Skipper Seabold wrote:
> On Tue, Jan 25, 2011 at 11:17 AM, Pierre GM<pgmdevlist at gmail.com>  wrote:
>> On Jan 24, 2011, at 11:47 PM, Skipper Seabold wrote:
>>
>>> Am I misreading the docs or missing something?  Consider the following
>>> adapted from here:
>>> http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html
>>>
>>> from StringIO import StringIO
>>> import numpy as np
>>>
>>> data = "1, 2, 3\n4, ,5"
>>>
>>> np.genfromtxt(StringIO(data), delimiter=",", names="a,b,c",
>>> missing_values=" ", filling_values=0)
>>> array([(1.0, 2.0, 3.0), (4.0, nan, 5.0)],
>>>       dtype=[('a', '<f8'), ('b', '<f8'), ('c', '<f8')])
>>>
>>> np.genfromtxt(StringIO(data), delimiter=",", names="a,b,c",
>>> missing_values={'b':" "}, filling_values={'b' : 0})
>>> array([(1.0, 2.0, 3.0), (4.0, 0.0, 5.0)],
>>>       dtype=[('a', '<f8'), ('b', '<f8'), ('c', '<f8')])
>>>
>>> Unless I use the dict for missing_values, it doesn't fill them in.
>>>
>> It's probably a bug . Mind opening a ticket ? I'll try to care of it when I can.
>> Thx in advance
>> P.
> http://projects.scipy.org/numpy/ticket/1722
>
> Forgot to use the code formatting, and it doesn't look like I can edit.
>
> Thanks,
>
> Skipper
Hi,
Your filling_values is zero so there is this line (1295?) in the code:
user_filling_values = filling_values or []

Which of cause presumes your filling_values is not something like 0 or [0].

Now it can be a code bug or just undocumented feature that 
filling_values can not a single zero. Thus something like these work:

np.genfromtxt(StringIO(data), delimiter=",", names="a,b,c", 
filling_values=-90)
np.genfromtxt(StringIO(data), delimiter=",", names="a,b,c", 
filling_values=[0,0])
np.genfromtxt(StringIO(data), delimiter=",", names="a,b,c", 
filling_values=[0,0,0])

Bruce