[SciPy-Dev] possible speed-up for arffread

[Ralf Gommers]

On Wed, Jun 16, 2010 at 10:46 AM, Benjamin Root <ben.root at ou.edu> wrote:

> Hello,
>
> I was looking at the scipy.io.arff module to see if I could easily shave
> some processing time for loading an ARFF file.  Doing some profiling on a
> file with 40,000 floating point numbers pointed me to the safe_float()
> function in the arffread.py file.  In it, it was stripping the string token
> of any whitespace and then comparing it to '?' (which is ARFF's missing data
> indicator).  I found that if one just does a check for the '?' character,
> you can shave almost 30% of the processing time off of the safe_float()
> function.
>
> In addition, I found a very slight improvement by calculating the range(ni)
> once and reusing that variable in the generator function.  Attached is my
> patch file.
>
> It isn't much, but it is noticeable.
>
> That looks good, thanks. I included it in
http://github.com/rgommers/scipy/tree/io, will commit that soon.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20100617/b78e4432/attachment.html>