Why is text file processing SO slow?

Rick Pasotto rickp at telocity.com
Wed Sep 20 22:52:41 EDT 2000


On 20 Sep 2000 17:46:46 -0700 in comp.lang.python, Aahz Maruch wrote:
> In article <8qb5j8$g17$1 at nnrp1.deja.com>,  <jaimewyant at my-deja.com> wrote:
> >
> >I have a program that processes large files ~25 megs.  The algorithm
> >that I use is as follows:
> 
> Use readlines() with the hint argument so you don't consume too much
> RAM.

What is this "hint argument?" It's not mentioned in the docs that I
have. Furthermore, the tutorial says that readlines() calls readline()
repeatedly so I don't see why there should be any speed difference.

-- 
"'Value' has no meaning other than in relation to living beings.  The
value of a thing is always relative to a particular person, is
completely personal & different in quantity for each living human --
'market value' is a fiction, merely a rough guess at the average of
personal values, all of which must be quantitatively different or
trade would be impossible...  The very personal relationship, 'value',
has 2 factors for a human being: 1st, what he can do with a thing, its
use for him... & 2nd, what he must do to get it, its cost to him.
There is an old song which asserts 'the best things in life are free'.
Not true!  Utterly false!   This was the tragic fallacy which brought
on the decadence & collapse of the democracies of the 20th century;
those noble experiments failed because the people had been led to
believe that they could simply vote for whatever they wanted... & get
it, without toil, without sweat, without tears.  Nothing of value is
free.  Even the breath of life is purchased at birth only through
gasping effort & pain...  If you boys & girls had to sweat for your
toys the way a newly born baby has to struggle to live you would be
happier... & much richer."
		-- Robert Anson Heinlein 1959 _StarShip Troopers_
		   Rick Pasotto email: rickp at telocity.com



More information about the Python-list mailing list