hi
mailman just started to behave strangely. i have a couple (20-30) lists, and one or two started to 'now work' all of a sudden. mail sent to that list disappears. i looked in /var/log/mailman/error, and there i found these errors:
May 27 12:02:42 2006 (10050) Uncaught runner exception: unpack list of wrong size May 27 12:02:42 2006 (10050) Traceback (most recent call last): File "/var/mailman/Mailman/Queue/Runner.py", line 111, in _oneloop self._onefile(msg, msgdata) File "/var/mailman/Mailman/Queue/Runner.py", line 167, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/var/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose more = self._dopipeline(mlist, msg, msgdata, pipeline) File "/var/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 91, in process send_digests(mlist, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 132, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 306, in send_i18n_digests msg = scrubber(mlist, msg) File "/var/mailman/Mailman/Handlers/Scrubber.py", line 240, in process url = save_attachment(mlist, part, dir) File "/var/mailman/Mailman/Handlers/Scrubber.py", line 359, in save_attachment fnext = os.path.splitext(msg.get_filename(''))[1] File "/var/tmp/mailman-root/var/mailman/pythonlib/email/Message.py", line 725, in get_filename File "/var/tmp/mailman-root/var/mailman/pythonlib/email/Message.py", line 608, in get_param File "/var/tmp/mailman-root/var/mailman/pythonlib/email/Message.py", line 555, in _get_params_preserve File "/var/tmp/mailman-root/var/mailman/pythonlib/email/Utils.py", line 337, in decode_params File "/var/tmp/mailman-root/var/mailman/pythonlib/email/Utils.py", line 284, in decode_rfc2231 ValueError: unpack list of wrong size
May 27 12:02:42 2006 (10050) SHUNTING: 1148720560.9632571+35d167125b7f3e005510df99b4728c1c6452c257
i have no idea what this means. i sent a couple of emails to this list, and to others lists on the same mailman, from the same email client (firefox), through the same server (postfix), and this particular list always gives this error.
somebody got a hint to what to do? until yesterday, everything was ok. i don't see any strange updates, or modifications in the packages.
[root@mail mailman]# rpm -qa | grep mailman mailman-2.1.5-10.fc2
[root@mail mailman]# rpm -qa |grep ^python python-devel-2.3.3-6 python-2.3.3-6
Imre Gergely wrote:
There is a malformed message - something wrong with an attachment filename - in lists/<listname>/digest.mbox for the offending list.
See <http://mail.python.org/pipermail/mailman-users/2006-May/051147.html> where the actual message problem is different, but the solution is the same.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
BTW, I don't think it is mentioned in the above thread, but after fixing the problem in digest.mbox, you can run bin/unshunt to reprocess the shunted messages, but before doing that, look at all the messages in the shunt queue (qfiles/shunt/*) with bin/show_qfiles or bin/dumpdb, to make sure there aren't old, no longer wanted messages there. If there are unwanted messages in the shunt queue, just remove the unwanted files before running bin/unshunt.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
it was kinda urgent, that's why i tried on the list first. after that i did some search and found the solution, exactly what you said. thx a lot for the help tho.
after solving this, i figured, on my lists nobody ever uses digest mode, so i just disabled it on all lists, then deleted digest.mbox. i guess with this setup, if a list gets a broken message which mailman can't handle, it will be shunted and that's it, the rest of the list and messages won't be affected, am i right?
and what happens with the archiving? the message gets archived before, or after a successful delivery? in another words, is there a chance that a broken message will be archived?
Imre Gergely wrote:
Yes.
Most of these problems that stop a list involve the digest.mbox. What happens is digest.mbox contains a message that throws an exception in Scrubber. When digest.mbox reaches digest_size_threshold, we try to create both a MIME and plain text digest and finally generate the recipient list for each digest and queue it to be sent.
If Scrubber (which is invoked to flatten the message for the plain digest) throws an exception in this process, pre Mailman 2.1.7 this caused the message to be shunted and the list would effectively stop as this all occurs before archiving and regular delivery, and the whole thing repeats with each new message.
By setting the list digestable to No, digest processing is skipped so the problem doesn't occur.
In Mailman 2.1.7 and later, all exceptions, even unanticipated ones, in the send_digests process are caught and logged as an error, so list processing other than sending of digests doesn't stop.
As far as archiving is concerned, Scrubber will also be invoked on the broken message in order to flatten it for the archive. It will throw some exception which will be caught in ArchRunner and the message will be shunted, but in this case, it only affects the one message and only the archive, not delivery.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
Just a clarification. The shunted message in the above case is not the 'problem' message. It is the current message which triggers the digest, and since the digest isn't produced and digest.mbox isn't removed, each subsequent message does the same thing and is also shunted.
The original 'problem' message was possibly shunted in processing it for the archive, but it was most likely delivered to the list before the list 'shut down'.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Imre Gergely wrote:
There is a malformed message - something wrong with an attachment filename - in lists/<listname>/digest.mbox for the offending list.
See <http://mail.python.org/pipermail/mailman-users/2006-May/051147.html> where the actual message problem is different, but the solution is the same.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
BTW, I don't think it is mentioned in the above thread, but after fixing the problem in digest.mbox, you can run bin/unshunt to reprocess the shunted messages, but before doing that, look at all the messages in the shunt queue (qfiles/shunt/*) with bin/show_qfiles or bin/dumpdb, to make sure there aren't old, no longer wanted messages there. If there are unwanted messages in the shunt queue, just remove the unwanted files before running bin/unshunt.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
it was kinda urgent, that's why i tried on the list first. after that i did some search and found the solution, exactly what you said. thx a lot for the help tho.
after solving this, i figured, on my lists nobody ever uses digest mode, so i just disabled it on all lists, then deleted digest.mbox. i guess with this setup, if a list gets a broken message which mailman can't handle, it will be shunted and that's it, the rest of the list and messages won't be affected, am i right?
and what happens with the archiving? the message gets archived before, or after a successful delivery? in another words, is there a chance that a broken message will be archived?
Imre Gergely wrote:
Yes.
Most of these problems that stop a list involve the digest.mbox. What happens is digest.mbox contains a message that throws an exception in Scrubber. When digest.mbox reaches digest_size_threshold, we try to create both a MIME and plain text digest and finally generate the recipient list for each digest and queue it to be sent.
If Scrubber (which is invoked to flatten the message for the plain digest) throws an exception in this process, pre Mailman 2.1.7 this caused the message to be shunted and the list would effectively stop as this all occurs before archiving and regular delivery, and the whole thing repeats with each new message.
By setting the list digestable to No, digest processing is skipped so the problem doesn't occur.
In Mailman 2.1.7 and later, all exceptions, even unanticipated ones, in the send_digests process are caught and logged as an error, so list processing other than sending of digests doesn't stop.
As far as archiving is concerned, Scrubber will also be invoked on the broken message in order to flatten it for the archive. It will throw some exception which will be caught in ArchRunner and the message will be shunted, but in this case, it only affects the one message and only the archive, not delivery.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
Just a clarification. The shunted message in the above case is not the 'problem' message. It is the current message which triggers the digest, and since the digest isn't produced and digest.mbox isn't removed, each subsequent message does the same thing and is also shunted.
The original 'problem' message was possibly shunted in processing it for the archive, but it was most likely delivered to the list before the list 'shut down'.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Imre Gergely -
Mark Sapiro