[Mailman-Users] How to unshunt?

Mark Sapiro mark at msapiro.net
Mon May 25 18:46:57 CEST 2009

Ulf Dunkel wrote:
>/qfiles/retry/ contains *.pck files.

Which are messages that have "retryable" delivery problems which should
be retried at intervals (default 15 minutes) for some period (default
5 days) or until they are delivered. There should be smtp-failure log
entries for the failed deliveries.

>/qfiles/virgin/ contains one *.pck.tmp file only.

This is a Mailman generated message that never got fully queued because
the process queueing it died.

>> Does /var/lib/mailman/bin/unshunt exist and is it executable (mode
>> -rwxr-xr-x)?
>Yes, it exists there physically (no link or stuff) and is executable 
>with mode -rwxr-xr-x.

Then the #! first line points at a non-existent python, or if it says
"#! @PYTHON@", this is a file that has not been configured by
Mailman's configure - i.e it is a file unpacked from the tarball and
not processed through configure and make.

>>> What am I doing wrong? When I try
>>> $ /var/lib/mailman/bin/python unshunt
>> Do you really have a python in /var/lib/mailman/bin or did you perhaps
>> mean
>> $ python /var/lib/mailman/bin/unshunt
>Yepp, of course I meant the latter. Sorry.
>> Unshunt will process *.bak and *.pck files in qfiles/shunt/ first
>> renaming the .bak files to .pck and then attempting to restore the
>> .pck files to their original queue. For any .pck files it can't
>> requeue, it should print a "Cannot unshunt message xxxxx, skipping"
>> message and leave that file as a .bak in the shunt/ queue.
>Sorry, but it doesn't print anything and simply quits. Afterwards, all 
>existing .pck files in the shunt/ folder have been updated to the 
>current date+time. That seems to be all. :-o

Which indicates they were unshunted, processed and encountered the same
exception as before and were shunted again.

>> You can look at these files with bin/dumpdb -p to see the message and
>> the metadata. Each file should contain two objects, the message and
>> the metadata. The metadata should have a 'whichq' attribute indicating
>> which queue to put it in.
>I can see these data for all files, seem to be fine.
>> Is it possible that the messages are being unshunted and then throwing
>> exceptions in processing and being shunted again?
>> What do mailman's error and smtp-failure (for retry) logs say?
>error says:
>May 25 17:57:06 2009 (5320) Uncaught runner exception: 'utf8' codec 
>can't decode bytes in position 30-35: unsupported Unicode code range
>May 25 17:57:06 2009 (5320) Traceback (most recent call last):
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 112, in _oneloop
>     self._onefile(msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 170, in _onefile
>     keepqueued = self._dispose(mlist, msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/VirginRunner.py", line 38, in 
>     return IncomingRunner._dispose(self, mlist, msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in 
>     more = self._dopipeline(mlist, msg, msgdata, pipeline)
>   File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in 
>     sys.modules[modname].process(mlist, msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Handlers/CookHeaders.py", line 188, in 
>     i18ndesc = uheader(mlist, mlist.description, 'List-Id', maxlinelen=998)
>   File "/usr/lib/mailman/Mailman/Handlers/CookHeaders.py", line 65, in 
>     return Header(s, charset, maxlinelen, header_name, continuation_ws)
>   File "/usr/lib/mailman/pythonlib/email/Header.py", line 188, in __init__
>     self.append(s, charset, errors)
>   File "/usr/lib/mailman/pythonlib/email/Header.py", line 272, in append
>     ustr = unicode(s, incodec, errors)
>UnicodeDecodeError: 'utf8' codec can't decode bytes in position 30-35: 
>unsupported Unicode code range
>May 25 17:57:06 2009 (5320) SHUNTING: 
>smtp-failure says:
>----- snip -----
>May 25 17:56:38 2009 (5319) All recipients refused: 
>{'udo_discussion-owner at lists.udo-open-source.org': (451, 'Temporary 
>local problem - please try later')}, msg
>May 25 17:56:38 2009 (5319) delivery to 
>udo_discussion-owner at lists.udo-open-source.org failed with code 451: 
>Temporary local problem - please try later
>----- snap -----
>What the heck should I do? I have no idea where else I should search for 
>"wrong" utf8 stuff and what "Temporary local problem" means.

The "Temporary local problem" message is reported by your MTA to
Mailman. Check the MTA's logs which may have more information.

Regarding the utf8 issue, your OP says "I recently changed my Mailman
2.1.9 installation to use and support Unicode." The error occurs when
CookHeaders is trying to encode the list's 'description' for the
List-ID header. I suspect that the charset for the list's preferred
language is now utf-8, but the list's description is still iso-8859-1
or whatever and thus contains non utf-8 codes.

Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

More information about the Mailman-Users mailing list