[Numpy-discussion] Question about improving genfromtxt errors

[Christopher Barker]

One more thought:

Pierre GM wrote:
>>> That way, we don't slow
>>> things down when everything works,

how long can it really take to increment an integer as each line is 
parsed? I'd suspect no one would even notice!

 >>if you don't keep track of where you are, wouldn't you
>> need to re-parse the whole file to figure it out again?
> 
> Indeed. But does it really matter ? We're in a case where there's a  
> problem already...

no, it doesn't.

>> Maybe a "debug" mode that the user could turn on and off would fit  
>> the need.

> Not a bad idea. Another option would be to give the user the  
> possibility to skip the offending lines:

In either case, I think I'd tend to use it something like this:

try:
    LoadTheFile()
except GenFromTxtException:
    LoadTheFile(debug=True)


But I suppose that block could be built in if debug was on -- without 
debug, it would simply raise an error when one was hit, with debug, it 
would go back and figure out more about the error and report it.


or you could have multiple error modes:

error_mode is one of:
  "fast": does it as fast as possible, and craps out with not much info 
on error

"first_error": stops on first error, and gives you some info about it.

"all_errors": keeps going after an error, and logs them all and reports 
back at the end.

"ignore_errors": skips any line with errors, loading the rest of the 
data -- I think I'd still want the error report, though.


but who's going to write that code?

-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov