shunting problem (unpack list of wrong size)
Hello all,
Of our 770+ lists, a couple have some shunting problems (the unshunt command fails to release queued up messages in the shunt directory). We are still running v2.1.5 with plans to upgrade in May, but I was hoping to get a fix for this before then.
The error message we get is:
==========================================================
Mar 23 12:37:33 2006 (19290) Uncaught runner exception: unpack list of wrong size . . . File "/mail/mailman/pythonlib/email/Utils.py", line 337, in decode_params charset, language, value = decode_rfc2231(EMPTYSTRING.join(value)) File "/mail/mailman/pythonlib/email/Utils.py", line 284, in decode_rfc2231 charset, language, s = parts ValueError: unpack list of wrong size
In pouring over the archives, Mark Shapiro wrote (in response to someone with the same problem):
Other lists aren't having a problem because it's probably caused by one
particular list member with a pathological MUA.And that this is related to the "scrubbing of attachments" for the digest or archives.
But no one on this list is using the digest feature and archiving is turned off. Plus the user in question (the one sending the message) is using a generic webmail client, the same one many of our users employ, with no special character set or language options (that I can see anyway).
Any ideas? Would an upgrade to 2.1.7 resolve this problem (I only say that because I perused the Mailman mailing list in reverse order from present date, and this issue was never raised again by anyone after 2.1.6 came out).
Thanks!
- jim -
Jim Savoy wrote:
The error message we get is:
==========================================================
Mar 23 12:37:33 2006 (19290) Uncaught runner exception: unpack list of wrong size . . . File "/mail/mailman/pythonlib/email/Utils.py", line 337, in decode_params charset, language, value = decode_rfc2231(EMPTYSTRING.join(value)) File "/mail/mailman/pythonlib/email/Utils.py", line 284, in decode_rfc2231 charset, language, s = parts ValueError: unpack list of wrong size
It appears that the message has a malformed rfc-2231 <http://www.faqs.org/rfcs/rfc2231.html> encoded header or some header with exactly one single quote "'" character that's being misidentified as an RFC-2231 encoded header.
Beyond that, it would help to see the rest of the traceback to see how we got here.
Also, can you do either bin/dumpdb or bin/show_qfiles on the shunt queue entry and perhaps see what the problem with the message is?
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Also, can you do either bin/dumpdb or bin/show_qfiles on the shunt queue entry and perhaps see what the problem with the message is?
OK Mike - here is the dumbdb (the names were changed to protect the innocent). I don't see anything unusual, and I should point out that this list has been up and running for years without incident, and all of its users (only a dozen or so) have sent to it successfully until a couple of weeks ago, using the same mail clients they've always used. I even blew away and re-created the list from scratch last week, and it was working fine until yesterday. Now, no users can send to it (myself included). Everything is put right into the shunt dira dn it stays there and cannot be unshunted. Thanks. This user is sending with the squirrelmail web program, but the list members use other clients as well (most of the users on this particular list are Mac users).
============================================================== [----- start pickle file -----] <----- start object 1 -----> From usera@uleth.ca Wed Mar 22 21:27:56 2006 Received: from venus.netsrv.uleth.ca ([142.66.6.26]:47530 helo=webmail.uleth.ca) by juliet.netsrv.uleth.ca with esmtpa (Exim 4.52) id 1FMHQS-00055L-4S for ccc-l@uleth.ca; Wed, 22 Mar 2006 21:27:56 -0700 Received: from 142.66.34.6 (SquirrelMail authenticated user usera) by webmail.uleth.ca with HTTP; Wed, 22 Mar 2006 21:27:56 -0700 (MST) Message-ID: <49183.142.66.34.6.1143088076.squirrel@webmail.uleth.ca> Date: Wed, 22 Mar 2006 21:27:56 -0700 (MST) From: User A <usera@uleth.ca> To: ccc-l@uleth.ca User-Agent: SquirrelMail/1.4.5 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Subject: [ccc-l] Update X-BeenThere: ccc-l@uleth.ca X-Mailman-Version: 2.1.5 Precedence: list List-Id: CCC Staff Mailing List <ccc-l.uleth.ca> List-Unsubscribe: <http://listserv.uleth.ca/mailman/listinfo/ccc-l>, <mailto:ccc-l-request@uleth.ca?subject=unsubscribe> List-Post: <mailto:ccc-l@uleth.ca> List-Help: <mailto:ccc-l-request@uleth.ca?subject=help> List-Subscribe: <http://listserv.uleth.ca/mailman/listinfo/ccc-l>, <mailto:ccc-l-request@uleth.ca?subject=subscribe>
Hello all. Here is an update:
<----- start object 2 -----> { '_parsemsg': False, 'lang': 'en', 'listname': 'ccc-l', 'original_sender': 'usera@uleth.ca', 'origsubj': 'Update', 'pipeline': [ 'ToDigest', 'ToArchive', 'ToUsenet', 'AfterDelivery', 'Acknowledge', 'ToOutgoing'], 'received_time': 1143088076.3202429, 'recips': [ 'userb@uleth.ca', 'usera@uleth.ca', 'userc@uleth.ca', 'userd@uleth.ca', 'usere@uleth.ca'], 'tolist': 1, 'version': 3, 'whichq': '/mail/mailman/qfiles/in'} [----- end pickle file -----]
Jim Savoy wrote:
here is the dumbdb (the names were changed to protect the innocent). I don't see anything unusual, and I should point out that this list has been up and running for years without incident, and all of its users (only a dozen or so) have sent to it successfully until a couple of weeks ago, using the same mail clients they've always used. I even blew away and re-created the list from scratch last week, and it was working fine until yesterday.
The particular message you dumped looks OK. We've possibly got more than one issue. I'd still like to see the rest of that traceback.
Now, no users can send to it (myself included). Everything is put right into the shunt dira dn it stays there and cannot be unshunted. Thanks. This user is sending with the squirrelmail web program, but the list members use other clients as well (most of the users on this particular list are Mac users).
The most likely cause of this immediate problem in Mailman 2.1.5 is a bad message in the list's digest.mbox file. If you look at lists/list_name/digest.mbox with an editor, and you can't immediately see a 'problematic' header that you can fix, rename the digest.mbox to something else. This will probably allow normal processing to resume. You may be able to unshunt the shunted messages, but there is likely still a bad one there that will cause the problem all over again.
It may also be difficult to properly recover the digest.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mike wrote:
It may also be difficult to properly recover the digest.
Hi Mike - problem solved and it had nothing to do with python, Mailman or any of the mail clients. When I went to edit the digest.mbox file I noticed it was 1.4 gigabytes in size (!). And this from a list that was just re-created a week ago and only had 5 messages sent to it. Turns out the list owner set it up so that there was no limit on the size of the files users were allowed to send and one of the users sent a 1.4 gig .sit (Stuffit) file to the list (probably inadvertantly). Our MTA (exim) wouldn't have delivered it anyway, but Mailman and python were going wild trying to accomodate this large attachment (memory exhausted, plus a huge swap file being created each time), I also run a cron job every hour to unshunt things and that certainly added to the chaos.
This also explains why other lists were either failing to deliver, or getting their messages late. Mailman was busy doing other things.
It was not apparent from the messages in the shunt dir that this was a large file. Only digest.mbox revealed that. I am going to go through all of our lists now, searching for large digest files.
Thanks for all your help.
- jim -
participants (2)
-
Jim Savoy -
Mark Sapiro