looking for speed-up ideas

[Andrew Dalke]

Ram Bhamidipaty wrote:
> With xreadlines the new speed is:

Thank you for the final timings.  Glad to know both what
sped things up (add to list only as needed) and what didn't
(so "for line in infile" is indeed as fast as readlines, and
my "make a long list and only occasionally sort" vs. using
a heapq).

> The sad part is this:
> 
> espring> cat du_sh_scipt
> grep \^F /tmp/foo | sort -t '/' -n -k 2,2 | tail -200 > /tmp/foo_sh
> 
> espring> time ~/du_sh_script
> 8.40u 0.58s 0:09.09 98.7%
> 
> To be fair the shell script does not output the full file name, but
> that could be handled with some post-processing.
  ...
> Ok - so now I have a script that is about half the speed of grep + sort.
> I guess thats not bad for an interpreted language.

Ahh, but if post-processing to get the filename is okay, then
you can speed up the Python even more.

					Andrew
					dalke at dalkescientific.com