md5 for large files

[Andrew MacIntyre]

On Tue, 4 Mar 2003, Bob Ballard wrote:

> Large is any file that I want to transfer around.  For example the gpg
> library for windows that I just downloaded is a little over 1 mb and has a
> defined md5 value that I'd like to compare.  This for a starter but for a
> specific purpose, i'd like to include the hash val in a separate file and
> upload or download with the target file, no matter the size.  Any clues?

So how much memory does your machine have?  I routinely start the
interpreter and do:

>>> import md5
>>> md5.new(open('/some/file','rb').read()).hexdigest()

on files of megabytes.  On an 83MB ZIP file I just tried, this completed
in less than 10 seconds (Athlon 1.4, 512MB RAM, OS/2).

This approach won't work where the file is larger than available VM, but
others have given you recipes to deal with that.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  | Snail: PO Box 370
        andymac at pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia