How to speed up reading bytes from file?
Stephan Diehl
stephan.diehl at gmx.net
Fri Dec 6 11:39:34 EST 2002
Brad Clements wrote:
> Hi,
>
> I have a binary file that consists of variable length "elements". I need to
> scan over the file, reading in elements, seeking over some elements, etc.
>
> I do not want to load the entire file into memory.
>
> My "unpack" algorithm consists of reading a "byte", then optionally 1 to 15
> "chars", then some number of bytes of words (2-byte network order)
>
I haven't read your posting in great detail, so please excuse me, if I'm
too far of the mark...
Whatever you do, you should NEVER read single bytes from disk. The
reason is quite simple: the disk can only access chunks of data anyway
and not single bytes. So, for every byte you are reading, the disk (or
diskdriver) will read a chunk and extract the byte you extracted.
Just read in a whole block of data and work on that local in memory copy.
Cheers
Stephan
More information about the Python-list
mailing list