[Numpy-discussion] ticket 1562 on loadtxt (was Re: Numpy 2.0 schedule)

28 Feb 2011


      On 02/28/2011 09:47 AM, Benjamin Root wrote:
...
On Mon, Feb 28, 2011 at 9:25 AM, Bruce Southey mailto:bsouthey@gmail.com> wrote:
On 02/28/2011 09:02 AM, Benjamin Root wrote:
    [snip]
    >
    >
    > So, is there still no hope in addressing this old bug report of
    mine?
    >
    > http://projects.scipy.org/numpy/ticket/1562
    >
    > Ben Root
    >
    I think you need to add more details to this. So do you have an
    example
    of the problem that includes code and expected output?
Perhaps genfromtxt is probably more appropriate than loadtxt for what
    you want:
from StringIO import StringIO
    import numpy as np
    t = StringIO("1,1.3,abcde\n2,2.3,wxyz\n1\n3,3.3,mnop")
    data = np.genfromtxt(t,
    [('myint','i8'),('myfloat','f8'),('mystring','S5')], names =
    ['myint','myfloat','mystring'], delimiter=",", invalid_raise=False)
    print 'Bad data raise\n',data
This gives the output that skips the incomplete 3rd line:
/usr/lib64/python2.7/site-packages/numpy/lib/npyio.py:1507:
    ConversionWarning: Some errors were detected !
        Line #3 (got 1 columns instead of 3)
      warnings.warn(errmsg, ConversionWarning)
    Bad data raise
    [(1, 1.3, 'abcde') (2, 2.3, 'wxyz') (3, 3.3, 'mnop')]
Bruce
Bruce,
I think you mis-understood the problem I was reporting.
Probably - which is why I asked for more details.
You can find the discussion thread here:
http://www.mail-archive.com/numpy-discussion@scipy.org/msg26235.html
I have proposed that at the very least, an example of this problem is 
added to the documentation of loadtxt so that users know to be aware 
of this possibility.
I did not connect the ticket to that email thread. Removing the 
structured array part of your email, I think essentially the argument is 
which should be the output of:
np.loadtxt(StringIO("89.23"))
np.arange(5)[1]
These return an 0-d array and an rather old argument about that (which 
may address the other part of the ticket). Really I see this behavior as 
standard so you add an example to the documentation to reflect that.
...
In addition, loadtxt fails on empty files even when provided with a 
dtype.  I believe genfromtxt also fails as well in this case.
Ben Root
Errors on empty files probably should be a new bug report as that was 
not in the ticket.

Bruce