Lazy file.readlines()?
Marc Christiansen
tolot at utolot.toppoint.de
Fri Sep 17 02:02:42 EDT 1999
Neil Schemenauer <nascheme at ucalgary.ca> wrote:
> What would be cool is if readlines() returned a lazy sequence
> object (ie. only read as much is needed using a certain block
> size). This should give the advantages of readlines() without
> the concern about sucking up a huge file all at once.
>
> I implemented this idea (probably badly) in pure Python
> and got about a 2x speedup verses readline(). It is a small
> module so I will post it here.
Here it's the other way round:
14.SRam:> python BlockFile.py daten:programme/python/e4xtc.txt
block time 9.74225389957
normal time 5.96557199955
This is on an Amiga, using Python 1.5.2, the file is 276089 Bytes long.
I tried also with blocksizes of 100k and 8k for BlockFile, but the
difference was about nought, so the speed difference is not blocksize
related.
And there is a error in _get_block(self):
> self.lines[0] = self.lines[0] + self.end
This should read: self.lines[0] = self.end + self.lines[0]
Ciao
Marc
--
"Claims about the evils of cannabis have been exaggerated: The question now
is not whether it should be legalised, but how and when."
http://www.newscientist.com/ns/980221/contents.html
PGP-Key fingerprint EE 2E C1 06 D1 28 89 D2 F9 CE 35 7B F8 F4 42 52
More information about the Python-list
mailing list