[Numpy-discussion] genfromtxt - the return

[Skipper Seabold]

On Tue, Oct 6, 2009 at 10:08 PM, Bruce Southey <bsouthey at gmail.com> wrote:
> On Tue, Oct 6, 2009 at 4:04 PM, Pierre GM <pgmdevlist at gmail.com> wrote:
>>
>> On Oct 6, 2009, at 4:43 PM, Christopher Barker wrote:
>>
>>> Pierre GM wrote:
>>>>> I think that the default invalid_raise should be True.
>>>>
>>>> Mmh, OK, that's a +1/) for invalid_raise=true. Anybody else ?
>>>
>>> yup -- make it +2 -- ignoring erreos and losing data by default is a
>>> "bad idea"!
>>
>> OK then, that's enough for me: I'll put invalid_raise as True by
>> default. Note that a warning was emitted no matter what.
>>
>>
>>>
>>>>> One 'feature' is that there is no way to indicate multiple
>>>>> delimiters
>>>>> when the delimiter is whitespace.
>>>>> A B C D
>>>>> 1 2 3 4
>>>>> 1     4 5
>>>
>>> I'd say someone has made a very poor choice of file formats!
>
> No, just seeing what sort of problems I can create. This case is
> partly based on if someone is using tab-delimited then they need to
> set the delimiter='\t' otherwise it gives an error. Also I often parse
> text files so, yes, you have to be careful of the delimiters. It is
> also arises because certain programs like spreadsheets there is the
> option to merge delimiters - actually in SAS it is default (you need
> to specify the DSD option).
>
>>>
>>> Unless this s a fixed width file, in which case it should be processes
>>> as such, rather than as a delimited one. I suppose it wouldn't hurt to
>>> add that feature to genfromtxt.. or is it there already. Perhaps
>>> that's
>>> what this means:
>>>
>>>> Have you tried using a sequence of integers for the delimiter ?
>>
>> Yes, if you give a sequence of integers as delimiter, it is
>> interpreted as the length of each field. At least, should be.
>
> More to learn and test.
>

There's an example on using the fixed-width delimiter here:
http://docs.scipy.org/numpy/docs/numpy.lib.io.genfromtxt/

As far as I know, it works fine.

> Anyhow, I am really impressed on how this function works.
>

Agreed.  Genfromtxt and the derived are very useful.

Skipper