[Mailman-Users] sendgiest cron job failure

[Mark Sapiro]

Xueshan Feng wrote:
>
>Since this happens only to two lists out of 18K lists, I suspect this
>might not be a general system configuration error, but I wanted to
>confirm that.


The problem is some mal-formed message in lists/<listname>/digest.mbox


>The work-around I have now is to have exception handling in "senddigest"
>code which skips the bad digest and log the error in mailman's error
>log. 
>
>Both the patch and the digest error messages are included below.
>
>Anyone knows if the problem is fixable? 
>
>Thanks,
>
>Xueshan
>
>--- senddigests 2007-06-13 10:48:54.000000000 -0700
>+++ senddigests.orig    2007-06-12 10:53:08.000000000 -0700
>@@ -38,7 +38,6 @@
> from Mailman import Utils
> from Mailman import MailList
> from Mailman.i18n import _
>-from Mailman.Logging.Syslog import syslog
> 
> # Work around known problems with some RedHat cron daemons
> import signal
>@@ -83,12 +82,9 @@
>         mlist = MailList.MailList(listname, lock=0)
>         if mlist.digest_send_periodic:
>             mlist.Lock()
>-           try:
>-                try:
>-                    mlist.send_digest_now()
>-                    mlist.Save()
>-               except (ValueError, TypeError), e:


I don't know if this is your patch or an emailing artifact, but the
above except and the preceding try should be indented the same amount.


>-                    syslog ('error', 'Broken digest: %s: %s', listname,
>e)
>+            try:
>+                mlist.send_digest_now()
>+                mlist.Save()
>             finally:
>                 mlist.Unlock()



The patch looks good. I may put something similar in cron/senddigests
for the next release.


>
>The digest error messages are two types:
>
>
>1. Traceback (most recent call last):
>  File "./senddigests", line 95, in ?
>    main()
>  File "./senddigests", line 87, in main
>    mlist.send_digest_now()
>  File "/var/lib/mailman/Mailman/Digester.py", line 60, in
>send_digest_now
>    ToDigest.send_digests(self, mboxfp)
>  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in
>send_digest
>s
>    send_i18n_digests(mlist, mboxfp)
>  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 324, in
>send_i18n_d
>igests
>    msg = scrubber(mlist, msg)
>  File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 372, in
>process
>    t = unicode(t, partcharset, 'replace')
>TypeError: coercing to Unicode: need string or buffer, NoneType found


Possibly a message with an empty sub-part. This is something we should
check in Scrubber. Try the following patch to
Mailman/Handlers/Scrubber.py

--- Scrubber.py	(revision 8236)
+++ Scrubber.py	(working copy)
@@ -373,7 +373,7 @@
                 partcharset = str(partcharset)
             else:
                 partcharset = part.get_content_charset()
-            if partcharset and partcharset <> charset:
+            if t and partcharset and partcharset <> charset:
                 try:
                     t = unicode(t, partcharset, 'replace')
                 except (UnicodeError, LookupError, ValueError,
AssertionError):

(The last line above is wrapped - the 'except' should be all one line).



>2.Traceback (most recent call last):
>  File "./senddigests", line 95, in ?
>    main()
>  File "./senddigests", line 87, in main
>    mlist.send_digest_now()
>  File "/var/lib/mailman/Mailman/Digester.py", line 60, in
>send_digest_now
>    ToDigest.send_digests(self, mboxfp)
>  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in
>send_digests
>    send_i18n_digests(mlist, mboxfp)
>  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 324, in
>send_i18n_digests
>    msg = scrubber(mlist, msg)
>  File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 306, in
>process
>    url = save_attachment(mlist, part, dir)
>  File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 483, in
>save_attachment
>    if os.path.exists(path):
>  File "/usr/lib/python2.3/posixpath.py", line 174, in exists
>    st = os.stat(path)
>TypeError: stat() argument 1 must be (encoded string without NULL
>bytes), not str


This too looks like a possible Scrubber bug. There is something wierd
abour the filename= parameter of an attachment, but it looks to me
like the code is pretty well protected. I'd like to see the message
(or .mbox) that causes this one and maybe the first one too.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan