large arbitrary binary data in python

[Steve Purcell]

Wayne Cuddy wrote:
>     1) simply read the data into a python binary string

Yes, see

    http://www.python.org/doc/lib/module-StringIO.html
    http://www.python.org/doc/lib/module-cStringIO.html


>     2) pipe the string to md5sum to verify integrity

Yes, but don't pipe it to md5sum if you can avoid it. Instead, use the
'md5' standard module:

    http://www.python.org/doc/lib/module-md5.html


>     3) pipe the data to a shell command like this "gunzip -dc | dd of=device"


Yep, see os.popen() and the popen2 module:

    http://www.python.org/doc/lib/os-newstreams.html
    http://www.python.org/doc/lib/module-popen2.html


> Is there a limit to the size of a binary string in python?

Yes. The limit is the amount of available virtual memory.

-Steve


-- 
Steve Purcell, Pythangelist
Get testing at http://pyunit.sourceforge.net/
Any opinions expressed herein are my own and not necessarily those of Yahoo