[Mailman-Developers]
[ mailman-Patches-647083 ] compress dies w/ MemoryError
noreply at sourceforge.net
noreply at sourceforge.net
Mon Dec 2 18:53:45 2002
Patches item #647083, was opened at 2002-12-02 13:38
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=300103&aid=647083&group_id=103
Category: command line scripts
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: John-Mark Gurney (jmg)
Assigned to: Nobody/Anonymous (nobody)
Summary: compress dies w/ MemoryError
Initial Comment:
If you have a large mailing list, it is very easy to
quickly exhast memory in many conditions. nightly_gzip
script tries to load the entire file into memory before
compressing it, not only is this terribly ineffecient,
it also means you are limited to file sizes that can
fit in memory. Some machines also further restrict how
much memory each process can access at a time. If that
limit is 512megs, and you have a 400+meg file, compress
will die.
Solution: read/write the file in blocks.
----------------------------------------------------------------------
Comment By: Bryan Fullerton (fehwalker)
Date: 2002-12-02 13:53
Message:
Logged In: YES
user_id=660772
would it make more sense to do this with zlib routines on a
stream than with gzip on blocks?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=300103&aid=647083&group_id=103
More information about the Mailman-Developers
mailing list