Efficient MD5 (or similar) hashes
bokr at oz.net
Mon Dec 8 04:36:57 CET 2003
On Sun, 07 Dec 2003 17:21:04 -0800, Erik Max Francis <max at alcyone.com> wrote:
>Kamus of Kadizhar wrote:
>> I want to check the integrity of the files after transfer. I can
>> the obvious - date, file size - quickly, but what if I want an MD5
>> From reading the python docs, md5 reads the entire file as a string.
>> That's not practical on a 1 GB file that's network mounted.
>Python's md5 module just accepts updating strings; the driving code
>certainly doesn't have to read the file all in at once. Just read it in
>a chunk at a time:
PMJI, but don't forget to open the file in binary,
e.g., theFile = file(thePath, 'rb'), if you're on windows.
> hasher = md5.new()
> while True:
> chunk = theFile.read(CHUNK_SIZE)
> if not chunk:
> theHash = hasher.hexdigest()
More information about the Python-list