[Mailman-Users] Is it OK to unshunt "bad" messages?

[Mark Sapiro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sebastian Hagedorn wrote:
| --On 25. April 2008 10:16:27 -0700 Mark Sapiro <mark at msapiro.net> wrote:
|
|> Sebastian Hagedorn wrote:
|> |
|> | No, it was a .pck file. I found this entry in the vette log:
|> |
|> | (3438) Message discarded, msgid: <c39b437d2808.47ee2e26 at wmich.edu>
|>
|>
|> This should have nothing to do with anything in the 'bad' queue.
|
| But it was definitely that message!


I have been overlooking something. I think it was the same message in
the 'bad' queue and in the vette log entry. This happens when the list's
Content filtering -> filter_action is Preserve and the message is
discarded because content filtering removed all its content.

If it was accepted when unshunted, the list's content filtering settings
must have been changed in the interim.


|> This
|> message is logged when a handler decides to discard an incoming post,
|> e.g. a post from a non-member in discard_these_nonmembers or any other
|> condition with a discard action.
|
| That's what I thought initially, but how did it end up in "bad" then?
| Could it be caused by this patch to 2.1.9 that I applied on March 8:
|
|                syslog('error', 'Ignoring unparseable message:
%s',filebase)
| +             self._switchboard.finish(filebase)


This patch doesn't put anything in 'bad'. In fact, without this patch,
the original unparseable queue entry is left in the original queue with
a .bak extension. The patch just removes the .bak file unless you also
have patches to the finish() method in Mailman/Queue/Switchboard.py.


| I see this line in "error" shortly after the message that I found in
| "bad" arrived:
|
| Mar 29 17:01:05 2008 (3438) Ignoring unparseable message:
| 1206806465.417665+d798425535212058c480aebf7900add235ac4cb9
| Mar 29 17:01:05 2008 (3438) Ignoring unparseable message:
| 1206806465.4255731+c874adc58f17010096988e4e388e31260a56677a
|
| But how can I tell what messages those really are?


Those are two separate messages/queue entries. In 2.1.9 with the patch
above, you can't tell anything, because the queue entries are gone. In
2.1.10 as released last Monday, the error log messages would have been
for example,

'Skipping and preserving unparseable message:
1206806465.4255731+c874adc58f17010096988e4e388e31260a56677a'

and the queue entry would have been saved as
1206806465.4255731+c874adc58f17010096988e4e388e31260a56677a.psv in
qfiles/shunt.

The patch I posted at
<http://mail.python.org/pipermail/mailman-users/2008-April/061360.html>
makes this conditional on QRUNNER_SAVE_BAD_MESSAGES and saves the entry
in qfiles/bad instead.



|> Changing long standing log messages is
|> not something to be done lightly as it potentially breaks peoples log
|> analysis tools.
|
| I can see that, but still I'd put it on my wishlist for 2.2 or 3.0,
| because it's happened a few times that list admins complained about
| "lost messages". It would've been much easier to see that mail to those
| lists had been discarded if the list name had been logged. The way it is
| now we first had to look at the MTA's log and then search the error log
| for the message-id. And when a single message is sent to more than one
| list, that still doesn't necessarily tell you the whole story ...


I understand. There are a number of log messages that could benefit from
additional information. I've put it on my 2.2 list to review these.

- --
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFIEiCcVVuXXpU7hpMRAlIVAKCiG+Rp647VLsAQYq13n23dDw4VfACg9t2V
dOJDWON4B/RdHQ4ZamXqXT8=
=9Qci
-----END PGP SIGNATURE-----