[Numpy-discussion] np.loadtxt : yet a new implementation...
Joris De Ridder
Joris.DeRidder at ster.kuleuven.be
Tue Dec 2 07:21:49 EST 2008
On 1 Dec 2008, at 21:47 , Stéfan van der Walt wrote:
> Hi Pierre
>
> 2008/12/1 Pierre GM <pgmdevlist at gmail.com>:
>> * `genloadtxt` is the base function that makes all the work. It
>> outputs 2 arrays, one for the data (missing values being substituted
>> by the appropriate default) and one for the mask. It would go in
>> np.lib.io
>
> I see the code length increased from 200 lines to 800. This made me
> wonder about the execution time: initial benchmarks suggest a 3x
> slow-down. Could this be a problem for loading large text files? If
> so, should we consider keeping both versions around, or by default
> bypassing all the extra hooks?
>
> Regards
> Stéfan
As a historical note, we used to have scipy.io.read_array which at the
time was considered by Travis too slow and too "grandiose" to be put
in Numpy. As a consequence, numpy.loadtxt() was created which was
simple and fast. Now it looks like we're going back to something
grandiose. But perhaps it can be made grandiose *and* reasonably
fast ;-).
Cheers,
Joris
P.S. As a reference: http://article.gmane.org/gmane.comp.python.numeric.general/5556/
Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
More information about the NumPy-Discussion
mailing list