How to speed up reading bytes from file?

Stephan Diehl stephan.diehl at gmx.net
Fri Dec 6 11:39:34 EST 2002


Brad Clements wrote:
> Hi,
> 
> I have a binary file that consists of variable length "elements". I need to
> scan over the file, reading in elements, seeking over some elements, etc.
> 
> I do not want to load the entire file into memory.
> 
> My "unpack" algorithm consists of reading a "byte", then optionally 1 to 15
> "chars", then some number of bytes of words (2-byte network order)
> 

I haven't read your posting in great detail, so please excuse me, if I'm 
too far of the mark...

Whatever you do, you should NEVER read single bytes from disk. The 
reason is quite simple: the disk can only access chunks of data anyway 
and not single bytes. So, for every byte you are reading, the disk (or 
diskdriver) will read a chunk and extract the byte you extracted.
Just read in a whole block of data and work on that local in memory copy.

Cheers

Stephan




More information about the Python-list mailing list