Mailman 3 April 2018 - Mailman-Users

Mailman not sending mail???
by Jon Clements 19 Apr '18

19 Apr '18

Hi, my Mailman seems to have stopped sending mail. Posts to a list(s) are not going anywhere, nor for example, subscribe notifications, etc. are not going out. It seems to be receiving mail as pending moderator requests are there. Webadmin seems to be working normally (except for not sending mail when requested). Where do I start looking? Sorry, not very technically proficient, it's a wonder I got it (Mailman v. 2.1.20) up and running a couple years ago on Mac OS 10.10.5 (using Server v. 5.0.15) but it has been running great since then (until now). I will say there was quite a bit of spam coming in before the problem started, and I stopped Mail service (on Server) for about a day to hopefully make it go away, and then the Mailman sending problem cropped up after I re-started Mail service (I think). Local/normal Mail accounts seem to be working fine. Appreciate any help you can give me... Jon -- Jon Clements aka 'Mr Honeycrisp' University of Massachusetts Amherst Extension UMass Cold Spring Orchard 393 Sabin St. Belchertown, MA 01007 413-478-7219 Verizon 413-378-3068 Project Fi umassfruit.com

3 2

GSOC idea: mail server/DNS server/mailing list healthcheck
by Rich Kulawiec 17 Apr '18

17 Apr '18

The idea for this comes from some of the web sites that perform this; unfortunately most of them are "upgrading" from simple, fast, easy checks to bloated ones that use a ton of Javascript, can't be scripted, and are increasingly behind signups/paywalls/etc. The concept is simple: given a domain, check its DNS, mail, etc. setup for correctness and consistency. For example: - does it have an A record? - is that valid hostname? - does it have an AAAA record? - is that valid hostname? - does it have MX records? - are the MX records *not* CNAMEs? - are they valid hostnames? - do those hostnames resolve to public IP addresses? - are any of those MX records duplicates? - are all the MX responding? - are the MX weights valid? - do all MX's pass FCrDNS check? - does it have NS records? - are they valid hostnames? - do they have A, AAAA records? - are they in public IP space? - are the NS records *not* CNAMEs? - do all NS pass FCrDNS check? - are any of those NS records duplicates? - does the list of NS match the list of authoritative NS? - do all the NS return the same list of NS? - do all the NS return the same list of MX? - do the NS *not* allow recursion? - are any of the NS lame? - are any of the NS missing? - are all the NS responding? - is there a working postmaster address? - is there a working abuse address? - is there a working hostmaster address? - if not is there a working address that matches the one in the SOA? - is the domain's SOA sane? (e.g. plausible serial number, refresh, retry, etc.) - do all the NS return the same SOA with the same serial number? - is there ASN diversity among the NS? - and so on Those are the sort of checks that pertain to the operation of any domain and its nameservers and mail exchangers. But in addition, if run on a Mailman 2 or 3 host: - what mailing lists are public? - what mailing lists are private? - does every list have a working -request address? - does every list have a working -owner address? - does every list have a working -bounces address? - if any list supports the optional -subscribe address, does it have a -unsubscribe address? - if any list supports the optional -join address, does it have a -leave address? - and so on Note: command-line tool. It has to be, because then it can be scripted and run via cron and so on. Besides, anyone running name servers, mail servers, etc., had better be able to work at the command line. Note: should work on systems that aren't running Mailman: just can't analyze Mailman then, of course. This leaves open the door for people using other MLMs to write checks that work with those. And maybe that'd be a nice thing to do. Note: should have varying levels of verbosity, including one that explains why something is wrong by referencing RFCs/BCPs/manual by chapter and verse. Note: the second set of checks (above) are outside the scope of what Mailman checks inside itself. That is, they require correlating what Mailman thinks should be in place versus what's actually in place in the MTA. ---rsk

2 1

GSOC idea: The central scrutinizer ;)
by Rich Kulawiec 17 Apr '18

17 Apr '18

I have a partially-completed spec for a module that will examine messages for various issues but my Python-fu is likely not sufficient to realize it and I'm busy writing anyway. This is probably a GSOC-size and GSOC-scope project, so if anybody is game, below is a poorly-written and large incomplete description of what I have in mind. 1. As I've said for years, anti-abuse controls should be layered and should start at the network perimeter (with things like the Spamhaus DROP list in border routers). Additional layers may be in firewalls, in the MTA, in the MLM (such as Mailman). No one layer can catch everything because it doesn't have contextual knowledge, e.g., the firewall doesn't know who the members of a mailing list are or even that a particular SMTP connection it's allowing contains traffic for a mailing list. Unfortunately, email accounts have been hijacked by the billions (and that's just counting Yahoo). The new owners of those accounts likely enjoy the same SMTP reachability that the old owners did and can thus drop messages onto mailing lists. (I'm going to omit the long explanation about why context-sensitive measures in the MTA are not only inadequate to stop this, but are also highly undesirable.) To put it another way: we don't have many defenses against our friends. But we need them. 2. We all have our own views on proper netiquette for mail/mailing lists, and of course, mine are the only correct ones. ;) But regardless of those, it'd be useful to have a mechanism to scrutinize messages for things like "800 lines of quoted digest with one top-posted line above it". Of course some of this is easy to see and hard to code, but I think a modest attempt in this direction would be helpful. (Besides, if the action taken on detection of these is to merely hold the message, then little harm is done by false positives.) 3. 1 and 2 are related. The same kinds of criteria that are useful in detecting and putting a hold on spam are useful in detecting undesirable content in messages, like URL shorteners and third-party tracking links. So it makes sense to have a highly configurable module that comes with a minimal/loose default configuration that can be salted to taste. So what I have in mind is a module that scrutinizes messages based on a set of (enabled/disabled) criteria, each one of which is configurable. I know that's not very clear. Let me make up an example and see if that helps. Let's suppose we call this module the Central Scrutinizer (CS) because oblique tributes to Frank Zappa are always in order. The CS might have a list of dozens of checks like this: - URL shorteners (1) - tracking links (2) - full digest quoting (3) Check (1) would consult a list of known URL shortening domains and do pattern matching of any URLs in the message against them. Check (2) would attempt to detect tracking links/"web bugs". Check (3) would check messages to see if it includes an entire quoted digest. Each one of these would be associated with an action -- and I strongly think that "hold" would be best, because these tests are going to make lots of mistakes for a while. The results would be presented to list owners (in email notifications and in the browser interface) with something like: Central scrutinizer report: - URL shorteners - fail, example.net detected - tracking links - pass, none detected - full digest quoting - pass, none detected with appropriate presentation so that it's easy to read and so that when a test fails, it reports *why* it failed. Let me note that the MTA is wrong place to do this stuff for a whole bunch of reasons. Among other things, lots and lots of people want different policies applied to mailing list traffic (which will result in outbound mail traffic) than they due to local traffic (which won't). This is a serious issue for anybody concerned with their mail system's reputation, which is based on what it emits, not what it accepts. Of course not everyone will agree with every check, and not every check is appropriate on every mailing list. (For example, a one-way announce-only list is unlikely to need most of these.) But if this is modular, and if individual checks can be switched on/off readily, then it can ship with everything off and folks can enable whatever subset they find palatable. Let me also note that the motivation for this comes not just from things I've bumped into while running lists, but things I've seen on other lists. (And I'm on rather a lot of them.) I've been thinking about this for years, but have become convinced in the last six months or so that the problem has now reached a point where a solution is worth constructing. ---rsk

1 0

Bug in Mailman version 2.1.25
by MPN Kazishe 14 Apr '18

14 Apr '18

Dear All, I am running mailman version 2.1.25 on FreeBSD 11.1-RELEASE. I am getting this error after a power outage on one of the lists. The web page give this error here: Bug in Mailman version 2.1.25 We're sorry, we hit a bug! Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs. >From the shell whilst doing a db check as follows: root@snappers:~ # /usr/local/mailman/bin/check_db -v myproblemlist Traceback (most recent call last): File "/usr/local/mailman/bin/check_db", line 153, in <module> main() File "/usr/local/mailman/bin/check_db", line 121, in main mlist = MailList(listname, lock=0) File "/usr/local/mailman/Mailman/MailList.py", line 131, in __init__ self.Load() File "/usr/local/mailman/Mailman/MailList.py", line 684, in Load raise Errors.MMCorruptListDatabaseError, e Mailman.Errors.MMCorruptListDatabaseError: [Errno 2] No such file or directory: '/usr/local/mailman/lists/myproblemlist/config.db.last' In the error logs I am getting the following: Apr 13 13:04:49 2018 (58769) couldn't load config file /usr/local/mailman/lists/myproblemlist/config.db.last [Errno 2] No such file or directory: '/usr/local/mailman/lists/myproblemlist/config.db.last' Apr 13 13:04:49 2018 (58769) All myproblemlist fallbacks were corrupt, giving up Can someone please let me know, where I am getting it wrong, and what to look at. Many thanks MPN/

3 5

Blank emails due to convert_html_to_plaintext
by Michael Bakonyi 10 Apr '18

10 Apr '18

Dear Mailman-enthusiasts, I administer a list, where emails by some senders (e.g. the german mass-mailprovider GMX) arrive with a stripped message (but existing Mailman-footer) due to convert_html_to_plaintext enabled. Following you find other settings: filter_content: yes pass_mime_types: multipart/mixed multipart/alternative text/rtf text/plain text/comma-separated-values text/tab-separated-values image text/html message/rfc822 I don't see the option "collapse multipart/alternative" anywhere (which is mentioned in other threads regarding empty mails) so I guess it's an older version of Mailman. I don't have access to any configuration-files as it's a Mailman-instance of a shared hoster. Here's a test-email where the message would be stripped (X-Provags-ID and X-UI-Out-Filterresults shortened for better readability). Delivery-date: Sun, 08 Apr 2018 13:56:12 +0200 Received: from [80.67.18.6] (helo=mx06.ispgateway.de) by mailcluster2-1.ispgateway.de with esmtp (Exim 4.90_1) (envelope-from <m.bakonyi(a)gmx.net>) id 1f58vc-0007ME-Ov; Sun, 08 Apr 2018 13:56:12 +0200 Return-path: <m.bakonyi(a)gmx.net> X-Envelope-to: bla(a)blub.de Received: from [212.227.17.20] (helo=mout.gmx.net) by mx06.ispgateway.de with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.90_1) (envelope-from <m.bakonyi(a)gmx.net>) id 1f58vi-00032J-W2 for bla(a)blub.de; Sun, 08 Apr 2018 13:56:19 +0200 Received: from [91.63.233.19] ([91.63.233.19]) by 3c-app-gmx-bs65.server.lan (via HTTP); Sun, 8 Apr 2018 13:56:12 +0200 MIME-Version: 1.0 Message-ID: <trinity-cce054f8-f6f7-442a-a026-7132987a4a59-1523188571988@3c-app-gmx-bs65> From: "Michael Bakonyi" <m.bakonyi(a)gmx.net> To: bla(a)blub.de Subject: test Content-Type: text/html; charset=UTF-8 Date: Sun, 8 Apr 2018 13:56:12 +0200 Importance: normal Sensitivity: Normal X-Priority: 3 X-Provags-ID: shortened X-UI-Out-Filterresults: notjunk:1;shortened X-Received-SPF: pass ( mx06.ispgateway.de: domain of gmx.net designates 212.227.17.20 as permitted sender ) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on spamfilter16.ispgateway.de X-Spam-Level: X-Spam-Status: No, hits=0.0 required=9999.0 tests=BAYES_50 autolearn=disabled version=3.4.0 X-Spam-CMAETAG: v=2.2 cv=TruWvHfh c=1 sm=1 tr=0 a=EDFhQv9wJ3PvMGwfvWgztQ==:17 a=IkcTkHD0fZMA:10 a=x7bEGLp0ZPQA:10 a=z6JOVgwrxLwA:10 a=Kd1tUaAdevIA:10 a=7lxF1IeVJ3SWydATLDQA:9 a=_W_S_7VecoQA:10 a=QEXdDO2ut3YA:10 X-Spam-CMAECATEGORY: X-Spam-CMAESUBCATEGORY: X-Spam-CMAESCORE: <html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.</div></div></body></html> If i turn off convert_html_to_plaintext the mails of the user passes. Is there anything I can do other to turning conversion of HTML off? Actually I don't want to allow HTML-mails ... Maybe a charset-related problem? Cheers, Michael

2 6

(relatively) new DMARC issues - and Gmail
by Lindsay Haisley 02 Apr '18

02 Apr '18

I've been working with a From-rewriting tool with code taken from Mailman (thanks, Mark!) and discovered a couple of things which I thought deserved posting about. I expect that they peripherally affect Mailman, too. At some point Amazon (amazon.com) started publishing a DMARC "p=quarantine" policy, which means that any email which gets redirected and hits my dmarc_shield piece is going to have its From address re- written to "postmaster(a)fmp.com" (fmp.com has a proper SPF record). I don't know what Gmail's policy is with regard to "p=quarantine" - whether it rejects such email outright or relegates it to the recipient's spam folder. I know that if the sending site publishes "p=reject", redirected email is refused by Gmail at the front door. I'll have to test the "p=quarantine" behavior. Here's the really annoying thing. My dmarc_shield processor rewrites the From header as per SOP for Mailman with the proper switch turned on. The From header address becomes "postmaster(a)fmp.com" with the original From address in the address comment (from xxx at yyz.com). If the email didn't already have a Reply-To address, the original From address is inserted as the Reply-To address. If a Gmail user replies to such an email, the reply goes to the Reply-To address, but Gmail **whitelists** the From address! Thereafter, any email which comes in with a munged From address is accepted, bypassing Gmail's otherwise pretty good spam filtering. I'm noticing a lot of spam email going out with From addresses for which a DMARC "p=reject" policy is published, which means that any such spam redirected to the Gmail user via FMP is also whitelisted. Bah! It's a fucking war zone out there! The only possible solution here would be to randomize the username portion of the rewritten From address, which makes the email look more like spam, and the Gmail user would end up with a whole lot of useless whitelisted address which would need to be deleted. Not to mention the fact that FMP's mail server might be blocked from sending ANY email to Gmail. -- Lindsay Haisley | "The first casualty when FMP Computer Services | war comes is truth." 512-259-1190 | http://www.fmp.com | -- Hiram W Johnson

4 10