[Numpy-discussion] fromfile() -- help!

[Christopher Barker]

Pauli Virtanen wrote:
>> if I have NumPyOS_ascii_ftolf right, it should return 0 if it doesn't 
>> succesfully read a number. However, this looks like it sets the data in 
>> *ip, even if the return value is zero.
> 
> It may also return EOF (== -1) when encountering end-of-stream. Of
> course, I don't think any code should not rely on EOF being -1, and I
> doubt that relying on it is intended here.

OK, so it should explicitly check for EOF?


>> It does pass on that return value, but, from ctors.c:
>>
>> fromfile_next_element(FILE **fp, void *dptr, PyArray_Descr *dtype,
>>                        void *NPY_UNUSED(stream_data))
>> {
>>      /* the NULL argument is for backwards-compatibility */
>>      return dtype->f->scanfunc(*fp, dptr, NULL, dtype);
>> }
>>
>> just moves it on through. This is called from here:
>>
>>          if (next(&stream, dptr, dtype, stream_data) < 0) {
>>              break;
>>          }
>>
>> which is checking for < 0 , so if a zero is returned, it will just go in 
>> its merry way...
> 
> Yeah, this is of course wrong; for example a file containing "1,2,"
> results to np.fromfile("filename.txt", sep=",") == [1, 2, -1] where the
> last value is effectively undefined.

I get a zero, but yes, that's what I'm trying to fix

> Another point to note is that `next` may also be the
> fromstr_next_element function; when fixing things also its semantics
> should be corrected.

yup -- I know -- great fun! But I;'m writing unit test that ensure that 
fromstring and fromfile do the same thing, so I should catch it if I 
miss anything.

>> It does pass on that return value, but, from ctors.c:
>>
>> fromfile_next_element(FILE **fp, void *dptr, PyArray_Descr *dtype,
>>                        void *NPY_UNUSED(stream_data))
>> {
>>      /* the NULL argument is for backwards-compatibility */ return
>>      dtype->f->scanfunc(*fp, dptr, NULL, dtype);
>> }
> 
> This functions is IMHO where the fix should go; I believe it should do 
> something like
> 
> 	return (ret == 0 || ret == EOF) ? -1 : ret;
> 

Thanks -- I think that makes sense -- if nothing else, a change here 
will only effect fromfile(), so I won't accidentally break anything else.

-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov