Using Gzip with as little as possible memory usage

Tw thomas at cintra.no
Mon Aug 21 18:00:36 EDT 2000


Hi,

I need to compress a file using the gzip module. Could I do this
completly without reading each chunk of the file and writing, etc. Is
there a compress(in, out)-kinda thing?

I need to fastest, least memory-hungry method available. The data to
be compressed is plain text, but huge.

PS! My reason for asking this is that when I try to use it the old
fashion way, like :

while 1:
	buff = in.read(1024)
	if not buff:break
	out.write(buff)
	# etc

it crashes. the code is called inside a larger object, and it the
error comes from inside the gzip module. Testing the same thing
outside my object works fine. No errors. Inside, no way. I hoped there
was some way of bypassing the whole thing, calling the library
directly with an input and an output file. 

Thanks



More information about the Python-list mailing list