Brad Knowles wrote:
Mark Sapiro wrote:
Note that in addition to the logging change for unparseable messages, they will now be saved in the shunt queue, so if you have a lot, you'll have to deal with that too.
These are the *.psv files? Yeah, we've got almost 8000 of them on python.org. Do you have any documentation on what should be done with them?
Wow! I never thought there would be anything like this.
Here's the situation. Pre 2.1.9, when Mailman dequeued an incoming message and then the Python email library couldn't parse the MIME structure the message was just lost. There wasn't anything that could be done other than log the fact. This is not a big deal as the message's MIME structure was defective and the message was almost certainly spam.
Beginning in 2.1.9, we implemented the .bak files to back up an in process message so the message could be recovered if something died horribly (e.g. power failure) while it was in process. Nothing was done at that time about the unparseable messages.
I then realized that the unparseable message had an intact .bak file in the queue, so for 2.1.10, I decided to preserve this file (thus the .psv in the shunt queue) in case some human wanted to look at it with bin/dumpdb or whatever. Obviously, if you get 8000+ in a few hours, no one is going to look at them all or even much of a sample.
You could set up a cron to run every hour or some other interval to efectively do
rm $var_prefix/qfiles/shunt/*.psv
The problem with that is there can occasionally be queue entries preserved for other conditions which are hopefully much rarer, but you might actually want to look at those.
I think the best solution is to turn off the preservation of unparseable messages, and add an mm_cfg.py setting to turn it on. I can work up a patch.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan