[Numpy-discussion] `missing` argument in genfromtxt only a string?

[Skipper Seabold]

On Tue, Sep 15, 2009 at 1:56 PM, Pierre GM <pgmdevlist at gmail.com> wrote:
>
> On Sep 15, 2009, at 10:44 AM, Skipper Seabold wrote:
>>>>> How do you specify different dtypes in genfromtxt?
>>> I could not see the information in the docstring and the dtype
>>> argument
>>> does not appear to allow multiple dtypes.
>
> Just give a regular dtype, or something that could be interpreted as
> such. Have a look at
> http://docs.scipy.org/doc/numpy/reference/generated/numpy.dtype.html
>
>> # a few incorrect ones
>>
>> s = StringIO("11.3abcde")
>> data = np.genfromtxt(s, dtype=np.dtype(int, float, str), delimiter=
>> [1,3,5])
>
> Non-legit at all, but a good idea in that case.
>
>>
>> # one I expected to work but have probably made an obvious mistake
>>
>> s.seek(0)
>> data = np.genfromtxt(s, dtype=np.dtype('i8','f8','a5'),
>> names=['myint','myfloat','mystring'], delimiter=[1,3,5])
>
> But this one works:
> data=np.genfromtxt(s, dtype=np.dtype("i8,f8,a5"), names=
> ['myint','myfloat','mystring'], delimiter=[1,3,5])
>
>>
>> Btw, you don't have to pass it as a dtype.  It just needs to be able
>> to pass
>>
>> if dtype is not None:
>>    dtype = np.dtype(dtype)
>>
>> I would like to see something like this, as it does when dtype is
>> None, but then we would have to have a type argument, maybe rather
>> than a dtype argument.
>
> 'k. Gonna see what I can do.
>

Oh, given that this works though, I don't think my gripe is that
legitimate.  This is essentially the same thing, I just need to read
up on declaring a dtype and stick some examples in the docstrings, so
I don't forget...

data = np.genfromtxt(s, dtype=np.dtype("i8,f8,a5"),
names=['myint','myfloat','mystring'], delimiter=[1,3,5])

>> Again, while I'm on it...I noticed the argument to specify the
>> autostrip argument that can be provided to _iotools.LineSplitter is
>> always False.  If this does, what I think (no time to test yet), it
>> might be nice to be able to specify this in genfromtxt.
>
> Would you mind giving me an example of usage with the corresponding
> expected output, so that I can work on it ?
>

Sure,  I gave a longer example of this in the 2nd email in this
thread, where my "missing" fields were " ,     ,         ,     ,   ,",
ie., fixed width white space that I wanted to just strip down to "".
Also if you notice that when it reads the date I still have
"mm/dd/yyyy  " with the trailing whitespace.  I don't know how big of
a deal this though.  I think you can just define an autostrip argument
in genfromtxt and then split_line=LineSplitter(...,
autostrip=autostrip).  I haven't tested this yet though.

http://article.gmane.org/gmane.comp.python.numeric.general/32821

Skipper