[Mailman-Users] Mailman 2.1.10 has been released

[Mark Sapiro]

Brad Knowles wrote:

>Mark Sapiro wrote:
>
>> Note that in addition to the logging change for unparseable messages,
>> they will now be saved in the shunt queue, so if you have a lot,
>> you'll have to deal with that too.
>
>These are the *.psv files?  Yeah, we've got almost 8000 of them on 
>python.org.  Do you have any documentation on what should be done with them?


Wow! I never thought there would be anything like this.

Here's the situation. Pre 2.1.9, when Mailman dequeued an incoming
message and then the Python email library couldn't parse the MIME
structure  the message was just lost. There wasn't anything that could
be done other than log the fact. This is not a big deal as the
message's MIME structure was defective and the message was almost
certainly spam.

Beginning in 2.1.9, we implemented the .bak files to back up an in
process message so the message could be recovered if something died
horribly (e.g. power failure) while it was in process. Nothing was
done at that time about the unparseable messages.

I then realized that the unparseable message had an intact .bak file in
the queue, so for 2.1.10, I decided to preserve this file (thus the
.psv in the shunt queue) in case some human wanted to look at it with
bin/dumpdb or whatever. Obviously, if you get 8000+ in a few hours, no
one is going to look at them all or even much of a sample.

You could set up a cron to run every hour or some other interval to
efectively do

  rm $var_prefix/qfiles/shunt/*.psv

The problem with that is there can occasionally be queue entries
preserved for other conditions which are hopefully much rarer, but you
might actually want to look at those.

I think the best solution is to turn off the preservation of
unparseable messages, and add an mm_cfg.py setting to turn it on. I
can work up a patch.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan