read zipfile sequentially?
python.list at tim.thechases.com
Wed Oct 15 21:13:13 CEST 2008
> I have a large ASCII data set that is zipped to a reasonable size.
> Can I access the data without decompressing the whole file first?
> I would like to run through the data to produce a much smaller
> extract and some summary statistics, but without unzipping
> it (if that is even possible).
Yes, if you're willing to slightly hack your install if you're
running pre-2.6 Python.
I had the same question, and Gabriel suggested I try dropping
the 2.6 version of zipfile.py in my $PYTHONPATH so it's found
before the existing version.
Once available, you can use the ZipFile.open() method which has
an iterator you can use rather than reading the entire content
into memory. You can read through the thread for further details.
Works on My Machine(tm)
More information about the Python-list