Problems with virtual domains on postfix...
Help! I can't get my list to work. For some reason postfix keeps redirecting email sent to the list to the non-list domain.
Thanks for you help...
Roberto
Details:
The List: lug-nuts@saclug.org
The "Real" Domain: leibmanland.com
The message from a bounced back message sent to lug-nuts@saclug.org: <lug-nuts@leibmanland.com>: data format error. Command output: lug-nuts: Mailbox does not exist
The postfix message (from mail.log) 6ADC07040E6: to=<lug-nuts@leibmanland.com>, orig_to=<lug-nuts@saclug.org>, relay=spamassassin, delay=2.4, delays=0.12/0.01/0/2.3, dsn=2.0.0, status=sent (delivered via spamassassin service)
The configuration: Environment: Ubuntu feisty running on an AMD64. Postfix version: 2.3.8. Mailman (2.1.9... I think)
Excerpts from /etc/postfix/main.cf alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases alias_database = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases mydestination = localhost, localhost.localdomain, localhost, mail, mail.leibmanland.com, [69.62.235.30] virtual_alias_maps = hash:/etc/postfix/virtual, hash:/var/lib/mailman/data/virtual-mailman virtual_mailbox_domains=leibmanland.com list.saclug.org placerliberty.org freezlets.com leibman.net mail.leibman.net byrongalvez.com mail.freezlets.com mail.byrongalvez.com lpsacramento.com mail.lpsacramento.com lpsacramento.org mail.lpsacramento.org saclug.org owner_request_special=no allow_mail_to_commands = alias mailman_destination_recipient_limit = 1
Excerpts from /etc/postfix/master.cf mailman unix - n n - - pipe flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
Excerpts from /etc/mailman/mm_cfg.py DEFAULT_EMAIL_HOST = 'leibmanland.com' DEFAULT_URL_HOST = 'www.leibmanland.com' add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST) add_virtualhost(DEFAULT_URL_HOST, 'saclug.org') MTA='Postfix' POSTFIX_STYLE_VIRTUAL_DOMAINS=['placerliberty.org', 'saclug.org']
Excerpts from /var/lib/mailman/data/virtual-mailman # STANZA START: lug-nuts # CREATED: Tue Jun 12 17:04:53 2007 lug-nuts@saclug.org lug-nuts lug-nuts-admin@saclug.org lug-nuts-admin lug-nuts-bounces@saclug.org lug-nuts-bounces lug-nuts-confirm@saclug.org lug-nuts-confirm lug-nuts-join@saclug.org lug-nuts-join lug-nuts-leave@saclug.org lug-nuts-leave lug-nuts-owner@saclug.org lug-nuts-owner lug-nuts-request@saclug.org lug-nuts-request lug-nuts-subscribe@saclug.org lug-nuts-subscribe lug-nuts-unsubscribe@saclug.org lug-nuts-unsubscribe # STANZA END: lug-nuts
Excerpts from /var/lib/mailman/data/aliases # STANZA START: lug-nuts # CREATED: Tue Jun 12 17:04:53 2007 lug-nuts: "|/var/lib/mailman/mail/mailman post lug-nuts" lug-nuts-admin: "|/var/lib/mailman/mail/mailman admin lug-nuts" lug-nuts-bounces: "|/var/lib/mailman/mail/mailman bounces lug-nuts" lug-nuts-confirm: "|/var/lib/mailman/mail/mailman confirm lug-nuts" lug-nuts-join: "|/var/lib/mailman/mail/mailman join lug-nuts" lug-nuts-leave: "|/var/lib/mailman/mail/mailman leave lug-nuts" lug-nuts-owner: "|/var/lib/mailman/mail/mailman owner lug-nuts" lug-nuts-request: "|/var/lib/mailman/mail/mailman request lug-nuts" lug-nuts-subscribe: "|/var/lib/mailman/mail/mailman subscribe lug-nuts" lug-nuts-unsubscribe: "|/var/lib/mailman/mail/mailman unsubscribe lug-nuts" # STANZA END: lug-nuts
We have 18K mailing lists. The "senddigest" cron job runs at noon daily. It came to our attention that many lists don't get digests. I put some debugging code in "senddigest", and I found out that there were two lists's digests had TypeError while being processed and 'senddigest" job just bailed out on the exception. No further lists will be processed for digest. Since this happens only to two lists out of 18K lists, I suspect this might not be a general system configuration error, but I wanted to confirm that. The work-around I have now is to have exception handling in "senddigest" code which skips the bad digest and log the error in mailman's error log. Both the patch and the digest error messages are included below. Anyone knows if the problem is fixable? Thanks, Xueshan --- senddigests 2007-06-13 10:48:54.000000000 -0700 +++ senddigests.orig 2007-06-12 10:53:08.000000000 -0700 @@ -38,7 +38,6 @@ from Mailman import Utils from Mailman import MailList from Mailman.i18n import _ -from Mailman.Logging.Syslog import syslog # Work around known problems with some RedHat cron daemons import signal @@ -83,12 +82,9 @@ mlist = MailList.MailList(listname, lock=0) if mlist.digest_send_periodic: mlist.Lock() - try: - try: - mlist.send_digest_now() - mlist.Save() - except (ValueError, TypeError), e: - syslog ('error', 'Broken digest: %s: %s', listname, e) + try: + mlist.send_digest_now() + mlist.Save() finally: mlist.Unlock() The digest error messages are two types: 1. Traceback (most recent call last): File "./senddigests", line 95, in ? main() File "./senddigests", line 87, in main mlist.send_digest_now() File "/var/lib/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digest s send_i18n_digests(mlist, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 324, in send_i18n_d igests msg = scrubber(mlist, msg) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 372, in process t = unicode(t, partcharset, 'replace') TypeError: coercing to Unicode: need string or buffer, NoneType found 2.Traceback (most recent call last): File "./senddigests", line 95, in ? main() File "./senddigests", line 87, in main mlist.send_digest_now() File "/var/lib/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 324, in send_i18n_digests msg = scrubber(mlist, msg) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 306, in process url = save_attachment(mlist, part, dir) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 483, in save_attachment if os.path.exists(path): File "/usr/lib/python2.3/posixpath.py", line 174, in exists st = os.stat(path) TypeError: stat() argument 1 must be (encoded string without NULL bytes), not str
Xueshan Feng wrote:
Since this happens only to two lists out of 18K lists, I suspect this might not be a general system configuration error, but I wanted to confirm that.
The problem is some mal-formed message in lists/<listname>/digest.mbox
The work-around I have now is to have exception handling in "senddigest" code which skips the bad digest and log the error in mailman's error log.
Both the patch and the digest error messages are included below.
Anyone knows if the problem is fixable?
Thanks,
Xueshan
--- senddigests 2007-06-13 10:48:54.000000000 -0700 +++ senddigests.orig 2007-06-12 10:53:08.000000000 -0700 @@ -38,7 +38,6 @@ from Mailman import Utils from Mailman import MailList from Mailman.i18n import _ -from Mailman.Logging.Syslog import syslog
# Work around known problems with some RedHat cron daemons import signal @@ -83,12 +82,9 @@ mlist = MailList.MailList(listname, lock=0) if mlist.digest_send_periodic: mlist.Lock() - try: - try: - mlist.send_digest_now() - mlist.Save() - except (ValueError, TypeError), e:
I don't know if this is your patch or an emailing artifact, but the above except and the preceding try should be indented the same amount.
- syslog ('error', 'Broken digest: %s: %s', listname, e) + try: + mlist.send_digest_now() + mlist.Save() finally: mlist.Unlock()
The patch looks good. I may put something similar in cron/senddigests for the next release.
The digest error messages are two types:
1. Traceback (most recent call last): File "./senddigests", line 95, in ? main() File "./senddigests", line 87, in main mlist.send_digest_now() File "/var/lib/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digest s send_i18n_digests(mlist, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 324, in send_i18n_d igests msg = scrubber(mlist, msg) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 372, in process t = unicode(t, partcharset, 'replace') TypeError: coercing to Unicode: need string or buffer, NoneType found
Possibly a message with an empty sub-part. This is something we should check in Scrubber. Try the following patch to Mailman/Handlers/Scrubber.py --- Scrubber.py (revision 8236) +++ Scrubber.py (working copy) @@ -373,7 +373,7 @@ partcharset = str(partcharset) else: partcharset = part.get_content_charset() - if partcharset and partcharset <> charset: + if t and partcharset and partcharset <> charset: try: t = unicode(t, partcharset, 'replace') except (UnicodeError, LookupError, ValueError, AssertionError): (The last line above is wrapped - the 'except' should be all one line).
2.Traceback (most recent call last): File "./senddigests", line 95, in ? main() File "./senddigests", line 87, in main mlist.send_digest_now() File "/var/lib/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 324, in send_i18n_digests msg = scrubber(mlist, msg) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 306, in process url = save_attachment(mlist, part, dir) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 483, in save_attachment if os.path.exists(path): File "/usr/lib/python2.3/posixpath.py", line 174, in exists st = os.stat(path) TypeError: stat() argument 1 must be (encoded string without NULL bytes), not str
This too looks like a possible Scrubber bug. There is something wierd abour the filename= parameter of an attachment, but it looks to me like the code is pretty well protected. I'd like to see the message (or .mbox) that causes this one and maybe the first one too. -- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
Xueshan Feng wrote:
The digest error messages are two types:
- Traceback (most recent call last): File "./senddigests", line 95, in ? main() File "./senddigests", line 87, in main mlist.send_digest_now() File "/var/lib/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digest s send_i18n_digests(mlist, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 324, in send_i18n_d igests msg = scrubber(mlist, msg) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 372, in process t = unicode(t, partcharset, 'replace') TypeError: coercing to Unicode: need string or buffer, NoneType found
Possibly a message with an empty sub-part. This is something we should check in Scrubber.
Actually, it turns out that this was a delivery status notification returned to the list. The Python email package handles message/delivery-status parts in a speial way which results in part payloads of None.
2.Traceback (most recent call last): File "./senddigests", line 95, in ? main() File "./senddigests", line 87, in main mlist.send_digest_now() File "/var/lib/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 324, in send_i18n_digests msg = scrubber(mlist, msg) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 306, in process url = save_attachment(mlist, part, dir) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 483, in save_attachment if os.path.exists(path): File "/usr/lib/python2.3/posixpath.py", line 174, in exists st = os.stat(path) TypeError: stat() argument 1 must be (encoded string without NULL bytes), not str
And this was a message with an improprly encoded RFC 2047 filename parameter which had a trailing null byte.
Both of these issues are fixed by the attached Scrubber.patch.txt patch to Mailman/Handlers/Scrubber.py.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Roberto Leibman wrote:
Help! I can't get my list to work. For some reason postfix keeps redirecting email sent to the list to the non-list domain.
I've looked over the information you includes, and I don't see anything obviously wrong, but I know almost nothing about Postfix.
Perhaps someone else on this list can help, but the Mailman stuff that relates to Postfix looks OK. This is really a Postfix question, and I suggest you may get better help from Postfix support resources.
There is one problem in you Mailman config that I tont't think affects the problem you're having, but will affect other things.
In mm_cfg.py, you have
add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST) add_virtualhost(DEFAULT_URL_HOST, 'saclug.org')
The second of these effectively nullifies the first. All 'url hosts' in add_virtualhost() directives must be unique because the 'url host' is a key in a python dictionary mapping and the 'email host' is the value of that key. So, in the above, the first add_virtualhost() just sets the value of the 'www.leibmanland.com' key to 'leibmanland.com' and the second replaces that value with 'saclug.org'.
See <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.062.htp> for some consequences of this.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Mark Sapiro -
Roberto Leibman -
Xueshan Feng