[Numpy-discussion] np.loadtxt : yet a new implementation...

Joris De Ridder Joris.DeRidder at ster.kuleuven.be
Tue Dec 2 07:21:49 EST 2008


On 1 Dec 2008, at 21:47 , Stéfan van der Walt wrote:

> Hi Pierre
>
> 2008/12/1 Pierre GM <pgmdevlist at gmail.com>:
>> * `genloadtxt` is the base function that makes all the work. It
>> outputs 2 arrays, one for the data (missing values being substituted
>> by the appropriate default) and one for the mask. It would go in
>> np.lib.io
>
> I see the code length increased from 200 lines to 800.  This made me
> wonder about the execution time: initial benchmarks suggest a 3x
> slow-down.  Could this be a problem for loading large text files?  If
> so, should we consider keeping both versions around, or by default
> bypassing all the extra hooks?
>
> Regards
> Stéfan


As a historical note, we used to have scipy.io.read_array which at the  
time was considered by Travis too slow and too "grandiose" to be put  
in Numpy. As a consequence, numpy.loadtxt() was created which was  
simple and fast.  Now it looks like we're going back to something  
grandiose.   But perhaps it can be made grandiose *and* reasonably  
fast ;-).

Cheers,
Joris

P.S. As a reference: http://article.gmane.org/gmane.comp.python.numeric.general/5556/


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm




More information about the NumPy-Discussion mailing list