[Numpy-discussion] 1.5 release schedule proposal

Christopher Barker Chris.Barker at noaa.gov
Tue Jul 27 14:34:22 EDT 2010


Pauli Virtanen wrote:
>> These are a couple relevant tickets;
>>
>> http://projects.scipy.org/numpy/ticket/909
>> http://projects.scipy.org/numpy/ticket/883
> 
> The relevant thing for 1.5.x would probably be to check that fromfile/
> fromstring handle malformed data in a well-specified way, and being 
> compatible with previous behavior.

yup.

> I don't think we want to start changing the behavior (error handling etc) 

The error handling I refer to is internal -- what happens when the 
routine that a specific dataype uses to try to parse the file fails -- 
that information is currently not passed up the chain -- so fixing that 
is not a change to the API, unless you consider unspecified behavior in 
the face of malformed data part of the API.

> or feature set of fromfile at this point. It's a low-level function, and 
> meant to be one.

yes, it sure is -- I'm still convinced that adding newline handling (and 
maybe comments) will massively improve its usefulness -- on the one 
hand, it's low level, on the other hand, its only point is performance; 
you can use any number of python routines to do the same job otherwise. 
I think with a few changes, it could be a high performance routine that 
could be used by higher level routines (such as loadtxt) to do more 
complex jobs.

> The correct solution is probably to improve the speed (and possibly 
> extend the feature set) of `loadtxt`, especially as the previous thread 
> started from the observation that although `loadtxt` did what was 
> desired, it was too slow for big files.

That would be fine, too. Actually, as I dug into the fromfile code, I 
almost decided that it was just too complicated, and it would be better 
to simply write something else from scratch, maybe in Cython -- I'm 
still on the fence about that. If I did write something from scratch it 
would probably look something like loadtxt().

There was a thread on the Cython list recently about this.

-Chris






-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



More information about the NumPy-Discussion mailing list