Bob Ippolito wrote:
One of the most useful things that could happen to the zipfile module would be a stream interface for both reading and writing. Right now it's slow and memory hungry when dealing with large chunks. The use case that lead me to fix this bug is a tool that archives video to zip files of targa sequences with a reference QuickTime movie.. so I end up with thousands of bite sized chunks.
While it's probably not an improvement on the order of magnitude you're looking for, there's a patch (1121142) that lets you read large items out of a zip archive via a file-like object. I'm occasionally running into the 2GB problem myself, so if any changes are made to get around that I can at least help out by testing it against some "real-life" data sets. Alan