Mailman 3 February 2005 - Mailman-Developers

mmdsr -- Mailman Daily Status Reports script...
by Brad Knowles 16 Feb '05

16 Feb '05

Folks, I just posted a patch for this at <https://sourceforge.net/tracker/index.php?func=detail&aid=1123383&group_id=…>. I'd appreciate any feedback that you may have. Thanks! -- Brad Knowles, <brad(a)stop.mail-abuse.org> "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 SAGE member since 1995. See <http://www.sage.org/> for more info.

1 0

Html headers & footers, unsubscribe
by Adrian Bye 15 Feb '05

15 Feb '05

Hi guys, I have had patches made for two issues: 1. HTML headers & footers handling 2. 1 click list unsubscribe You can get our patches here: https://sourceforge.net/tracker/index.php?func=detail&aid=1121257&group_id=… tid=300103 Or also here: http://tasdevil.com/mailman_developers.zip Installation instructions are found in readme.txt What we did: 1. Reasonably good handling of HTML headers & footers The issue we wanted to address is this one: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.039.htp I wanted to be able to put html headers and footers into messages. I did not like the current mailman style of adding an attachment. I knew this must be possible to do because yahoo groups is doing it. So we spent a LOT of time testing how yahoo groups adds footers to mails. We tested multipart messages with varying numbers of attachments, badly formed html and a bunch of other stuff. We have documented our testing process in mime.txt. Our implementation is about 95% perfect, and works identical to yahoo groups, which is sending millions of emails/day. It will correctly handle badly formed html, and various mimetypes. Sometimes, depending on the colors of a message, it will not be possible to view the headers & footers. Otherwise it works really well. 2. 1-click unsubscribe with the URL under 60 characters. I've been involved in some discussions on this topic on the developers list. The current unsubscribe process was not clear to me, and I wanted it to be very easy for my users. I also wanted the URLs to be short, so they would never get broken by various mail clients. We made some small changes to encode the users email address and listnames to the footer, along with the user password. When a user wishes to unsubscribe, they click on the link, it passes this information to mailman, and a confirmation page is then shown. The user can choose to unsubscribe by clicking on the "yes, I want to unsubscribe" button. I work with users who are less advanced than your average AOL customer, so both of these points were extremely important to solve. While many do feel its important to continue requiring passwords for mailman accounts, I would strongly advocate that those passwords become hidden just for access to archives. I'd like to see mailman become capable of handling millions of messages/day, and it is only when simple unsubscribe functionality is incorporated that the resources will start to appear to make that possible. Yes, this may represent some change, but I believe it can be done in a way which leaves everyone happy. Anyways, those are our modifications. If you have code questions, all coding was done by Bushmanov Efim, who can be reached at binworkers(a)dcemail.com. He is available for more work of this kind if you need it, and I would highly recommend him as a great programmer. For other questions drop me a line. Cheers, Adrian

2 1

Mail-to-news gatewaying?
by Brad Knowles 15 Feb '05

15 Feb '05

Folks, Okay, so we know about the issue of munging Message-ID: (if you don't know what I'm talking about, see <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.059.htp>). The latest crisis that has me wondering is the subject prefix stuff. Looking in NewsRunner.py, starting at line 104, we have: # Should we restore the original, non-prefixed subject for gatewayed # messages? origsubj = msgdata.get('origsubj') if not mlist.news_prefix_subject_too and origsubj is not None: del msg['subject'] msg['subject'] = origsubj However, we still have a problem. If mail clients reply to the message to the list (which will include the subject prefix), then this will result in a new message with a subject prefix for the list already present. Now, Mailman is smart and avoids adding a second subject prefix, but it doesn't strip out the subject prefix if one already exists. For the mailing list, this subject prefix stripping isn't necessary. But for gatewaying to the newsgroup, it might be desirable. This would also resolve the issue of restoring the original subject when gatewaying -- don't bother, and just let the subject prefix stripper deal with it. If this were sed or Perl, I could quickly put together a one-line hack to implement this feature. For example, just doing something off the top of my head, you might be able to use something like: subject=y/\w*$prefix\w*/ /; But this is Python, and I don't really grok Python. Can someone help me out here? Looking at <http://www.python.org/doc/2.3.4/lib/module-string.html>, I'm imagining some sort of permutation of find(), split(), either lstrip() or rstrip(), and then join(), should be sufficient to do the job. But I'm not quite able to figure out how to put them all together. Of course, the reason why I ask is that I'm getting complaints about the mail-to-news gateway I'm running at another site, and I don't want this to blow up into an ugly situation. Sigh.... -- Brad Knowles, <brad(a)stop.mail-abuse.org> "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 SAGE member since 1995. See <http://www.sage.org/> for more info.

5 9

Userdefined HELO with SMTPDirect
by Palomäki Kai 15 Feb '05

15 Feb '05

Hello, It would be useful in some situations (for example Mailman and Postfix running on same host) to define custom HELO when using Mailman SMTPDirect delivery module. I made simple patch to achieve this. But since my experience with Python is minimal I don't know if this is "the right way(tm)". diff -u -r mailman-2.1.6b3/Mailman/Defaults.py.in mailman-2.1.6b3-patched/Mailman/Defaults.py.in --- mailman-2.1.6b3/Mailman/Defaults.py.in 2005-01-22 08:55:28.000000000 +0200 +++ mailman-2.1.6b3-patched/Mailman/Defaults.py.in 2005-02-13 12:59:28.000000000 +0200 @@ -422,6 +422,11 @@ SMTPHOST = 'localhost' SMTPPORT = 0 # default from smtplib +# SMTP HELO sent by Mailman, when DELIVERY_MODULE is 'SMTPDirect'. Default +# is FQDN of localhost (as returned by socket.getfqdn()) +# SMTPHELO = 'my.own.fqdn.hostname' +SMTPHELO = None + # Command for direct command pipe delivery to sendmail compatible program, # when DELIVERY_MODULE is 'Sendmail'. SENDMAIL_CMD = '/usr/lib/sendmail' diff -u -r mailman-2.1.6b3/Mailman/Handlers/SMTPDirect.py mailman-2.1.6b3-patched/Mailman/Handlers/SMTPDirect.py --- mailman-2.1.6b3/Mailman/Handlers/SMTPDirect.py 2004-01-23 01:02:07.000000000 +0200 +++ mailman-2.1.6b3-patched/Mailman/Handlers/SMTPDirect.py 2005-02-13 12:29:59.000000000 +0200 @@ -59,7 +59,7 @@ self.__conn = None def __connect(self): - self.__conn = smtplib.SMTP() + self.__conn = smtplib.SMTP('', 0, mm_cfg.SMTPHELO) self.__conn.connect(mm_cfg.SMTPHOST, mm_cfg.SMTPPORT) self.__numsessions = mm_cfg.SMTP_MAX_SESSIONS_PER_CONNECTION

1 0

Re: [Mailman-Users] security heads up - path traversal with 2.1.5
by Tokio Kikuchi 15 Feb '05

15 Feb '05

Hi, Ron Brogden wrote: > Hey folks. I haven't see an official post here yet but as this has already > gone out on at least one full-disclosure list I thought it worth mentioning > since this will be an actively exploited 0 day: > > http://lists.netsys.com/pipermail/full-disclosure/2005-February/031562.html Barry and I are notified on this subject but both are busy on their job so he requested for delay in the disclosure. > > Basically, there is a path traversal issue with mailman 2.1.5 which will let > you access any file that the Mailman user has read access to (at least under > Apache 1.3, can't speak for other web servers). I have tested this on a > personal box and it does indeed work as advertised. I've tested with my 1.3.29 installation and verified apache PATH_INFO does convert '//' to '/'. Barry also wanted to clarify which apache version/installation (combination with mailman) is valnerable. Return code of 200 doesn't mean sucessful exploit. You should check mailman logs/error also. (If there is none chances are succesful exploit.) > > One temporary workaround is to stop access to "/mailman/private" via your web > server configuration. I would wait for a formal patch notice from the > developers before patching the actual Mailman code. Also newly introduced script bin/reset_pw.py may be useful if your list has been really exploited. (It should be veiwable from SourceForge CVS but it looks like currently in trouble.) -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/

2 2

Re: [Mailman-Users] security heads up - path traversal with 2.1.5
by Brad Knowles 15 Feb '05

15 Feb '05

At 2:09 PM +0100 2005-02-14, Florian Weimer wrote: > The underlying assumption seems to be that Mailman security bugs can > only be disclosed by posting them on the Mailman lists. In response to this issue, FAQ 1.27 has been updated, and the mailman-users and mailman-developers mailing lists have likewise been modified to include suitable text at the bottom of every message, as well as on the "listinfo" page, and in the "welcome" message that is sent to all new subscribers. This matter is now closed. -- Brad Knowles <brad(a)python.org> Python.org Postmaster Team

3 2

Re: [Mailman-Announce] Critical security update for Mailman 2.1.5 and earlier
by Michael Janssen 14 Feb '05

14 Feb '05

On Thu, 10 Feb 2005, Axel Beckert wrote: > Am Thu, Feb 10, 2005 at 09:41:05AM -0500, Barry Warsaw schrieb: > > Until Mailman 2.1.6 is released, the longer term fix is to apply > > this patch: > > > > http://www.list.org/CAN-2005-0202.txt > > Which unfortunately only works with Python 2. Python 1 (respective at > least 1.5.2) complains about syntax errors. (Which, in fact, also > helps against the vulnerability by displaying the "You've found a > Mailman bug" page. ;-) > > Is there any patch which complies with Python 1 syntax? not as a patch, cause I haven't got a CVS repository available but you can change the line safe = [x for x in parts if x not in ('.', '..')] which is list comprehension syntax into: safe = [] for x in parts: if x not in ('.', '..'): safe.append(x) which does obviously the same: Append to safe every x in parts which are not "." or ".." regards Michael

1 0

mailman email harvester
by Bernhard Kuemel 12 Feb '05

12 Feb '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks(a)vt.edu wrote: | On Sat, 12 Feb 2005 02:48:56 +0100, Bernhard Kuemel said: | | |>If hashcash (http://www.hashcash.org/) gets integrated in our mail |>systems we no longer need to hide or obfuscate our email addresses. | | | On the other hand, widespread distribution of hashcash will probably mean | the end of many mailing lists, because you can't trust users to actually | whitelist everything they subscribe to. If a user choses to use hashcash he must understand it. If he doesn't and subscribes to a mailing list all the list mail will go to his spam folder. He will learn from that and whitelist list mail. | And remember that the whole *idea* | of hashcash is that you make it impractical for somebody to send 3,000 pieces | of mail. I'm sure netsys.com wouldn't want to keep full-disclosure if they had | to do hashcash for even 10% of their users. They would not hashcash every mail, but sign each incoming mail so spammers can't spam suscribers whose addresses then can be published again. | I'll overlook the issues caused when you *dont know* what to whitelist. | For instance - many mailing lists (including this one) have a "confirmation | of subscription" check. For bonus points - should you have whitelisted: | | a) full-disclosure(a)lists.netsys.com (the actual list name) | b) full-disclosure-request(a)lists.netsys.com (the rfc822 header on my confirm) | c) full-disclosure-admin(a)lists.netsys.com (the rfc821 MAIL FROM:) | d) mailman@ | e) majordomo@ | f) listserv@ Subscribing to mailing lists has always been a process of following instructions. If you subscribe via a web page, this web page will tell you which addresses to whitelist. If you subscribe via email firstly there will also be some source of instructions how to subscribe, and secondly you can whitelist replies that reference (private) emails you sent recently. | There's also all the stuff that things like amazon, ebay, your bank, | your insurance company, your utility companies, etc... all send out, | that users will forget to whitelist. They can send hashcashed requests for being whitelisted which will pop up a window similar to message receipt requests. | Hashcash really sucks if you're a mail server admin who has to crank 50,000 | hash cashes a day at 5 CPU seconds a pop because people forgot to whitelist | your server. I don't understand the situation. Human edited mail is usually created on a workstation that is capable of making hashcash while the mail is edited. Mass mail generated on a server falls into several categories: 1) spam: let them make hashcash 2) solicited recurring mail: send hashcashed whitelist request and follow up with unpaid mail. If unpaid mail gets rejected stop sending mail. Actually, there is little reason not to make the whitelisting part of the service subscription process. 3) Replies should be whitelisted automatically. 4) legitimate systems that initiate mail conversation must make hashcash. Can you think of any examples? | Hashcash isn't even a tiny speed bump if you're a spammer and have 50,000 | zombies - each one only takes a 5 second hiccup and continues spamming.... Configure your system to require more. 1 minute. Or 10. Or 20. The amount of hashcash can be put in an email address comment or if insufficient cash is sent, the receiving system can automatically request more. | But yeah, other than all those minor details, hashcash is a fine solution. ;) ecash may be even better. You don't have to accept the postage. Only take it from unwanted mail. Bernhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Debian - http://enigmail.mozdev.org iD8DBQFCDfJ89zL78+QhnUgRAu+pAJ95pzHYaMatinzyQ3wtIIeQqGb/uwCgi+4o 4I44MDzL2TeHQ1KLQGW7kts= =HCYs -----END PGP SIGNATURE-----

1 0

mailman email harvester
by Bernhard Kuemel 12 Feb '05

12 Feb '05

This mail got blocked at first because I was not subscribed with my current email address. Some of you may remember that I announced the release of the harvester script. We discussed the issue and as the result was, not to change mailman, I now released the script to raise public awareness of the problem. Bernhard -------- Original Message -------- Subject: mailman email harvester Date: Mon, 07 Feb 2005 23:48:44 +0100 From: Bernhard Kuemel <bernhard(a)bksys.at> To: full-disclosure(a)lists.netsys.com, bugtraq(a)securityfocus.com, mailman-developers(a)python.org Hi! Tons of email addresses from mailman mailing lists are vulnerable to be collected by spammers. They are "protected" by obfuscation (user(a)example.com -> user at example.com) and access to the subscriber list can be restricted to subscribers. The obfuscation is trivially reversed and harvester scripts can subscribe to gain access to restricted lists. I suggested a graphical turing test that would bar scripts but the mailman developers argued spammers might hire a couple of temps that would solve the test as it already happened for the creation of email accounts. The only solution would be not to have the desired information available. This is already an option by restricting access to the member list to the list administrator. However, still many lists either have the member list openly published, or available to the list members. To raise awareness to this issue I wrote a script that collects addresses from openly accessible lists. It stops after processing 1000 (the maximum allowed) search results from google and collects 76772 email addresses (61124 unique). It is attached as mmxp1. An improved version that collects addresses that are restricted to subscribers, processes more lists and works more parallelized is planned. Bye, Bernhard #!/usr/bin/perl -w #http://www.google.com/search?q=%22list+is+only+available+to+the+list+members%22+mailman/listinfo&start=600&num=100 #2.1.4 "current archive" "private list which" mailman/listinfo site:org $n=0; $u=0; for ($i=0;1;$i+=10) { $#urls=-1; $google=`wget -qO - -U 'any browser' 'http://www.google.com/search?q=%22Click+here+for+the+list%22+mailman%2Flist…'`; # print $google; @urls=($google=~m*<p class=g><a href=(http://\S+?)>*g); # print join("\n",@urls); if ($#urls==-1) {last;} # print "\naoeu $#urls\n"; foreach $url (@urls) { $u++; $url=~s*/listinfo/*/roster/*; print STDERR "$url...\n"; $roster=`lynx -connect_timeout=10 -dump $url`; # print $roster; @mails=$roster=~/^ +\* $?\[\d+\](.* at .*?)$?$/mgo; foreach $mail (@mails) { $mail=~s/ at /@/; print "$mail\n"; $n++; } print STDERR "mails=".($#mails+1).", total=$n, url=$u, google=$i\n"; # exit; } #foreach url } #while google

5 6

Mailman 2.1.6 beta 3 released
by Tokio Kikuchi 11 Feb '05

11 Feb '05

Hi, I put together a tarball for Mailman 2.1.6 beta 3 and placed on my Japanese Mailman site at: http://mm.tkikuchi.net/mailman-2.1.6b3.tgz This is the third beta release of 2.1.6 which are roughly scheduled to be released by the end of February. Please grab it from above site and upgrade your mailman. Change from 2.1.6b2 are mainly to fix the directory traversal vulnerability CVE number CAN-2005-0202 http://www.list.org/security.html Here is excerpt from NEWS file: 2.1.6 (XX-XXX-200X) - Critical security patch for path traversal vulnerability in private archive script (CAN-2005-0202). - Date and Message-Id headers are added for digests. (1116952) - List owners can now cusomize the non-member rejection notice from admin/<listname>/privacy/sender page. (1107169) - Most of the installation instructions have been moved to a latex document. See admin/www/mailman-install/index.html for details. - VERP_PROBES is disabled by default. - bin/withlist can be run without a list name, but only if -i is given. Also, withlist puts the directory it's found in at the end of sys.path, making it easier to run withlist scripts that live in $prefix/bin. - bin/newlist grew two new options: -u/--urlhost and -e/--emailhost which lets the user provide the web and email hostnames for the new mailing list. This is a better way to specify the domain for the list, rather than the old 'mylist@hostname' syntax (which is still supported for backward compatibility, but deprecated). - Added the ability for Mailman generated passwords (both member and list admin) to be more cryptographically secure. See new configuration variables USER_FRIENDLY_PASSWORDS, MEMBER_PASSWORD_LENGTH, and ADMIN_PASSWORD_LENGTH. Also added a new bin/withlist script called reset_pw.py which can be used to reset all member passwords. Passwords generated by Mailman are now 8 characters by default for members, and 10 characters for list administrators. - Allow editing of the welcome message from the admin page (1085501). - A potential cross-site scripting hole in the driver script has been closed. Thanks to Florian Weimer for its discovery. Also, turn STEALTH_MODE on by default. - Chinese languages moved from 'big5' and 'gb' to 'zh_TW' and 'zh_CN' respectively for compliance to the IANA spec. Note that neither language is supported yet. - Python 2.4 compatibility issue: time.strftime() became strict about the 'day of year' range. (1078482) - New feature: automatic discards of held messages. List owners can now set how many days to hold the messages in the moderator request queue. cron/checkdb will automatically discard old messages. (790494) - Improved mail address sanity check. (1030228) - SpamDetect.py now checks attachment header. (1026977) - New feature: subject_prefix can be configured to include a sequence number which is taken from the post_id variable. Also, the prefix is always put at the start of the subject, i.e. "[list-name] Re: original subject", if mm_cfg.OLD_STYLE_PREFIXING is set No. The default style is "Re: [list-name]" if numbering is not set, for backward compatibility. If the list owner is using numbering feature by "%d" directive, the new style, "[list-name 123] Re:", is always used. - List owners can now use Scrubber to get the attachments scrubbed (held in the web archive), if the site admin permits it in mm_cfg.py. New variables introduced are SCRUBBER_DONT_USE_ATTACHMENT_FILENAME and SCRUBBER_USE_ATTACHMENT_FILENAME_EXTENSION in Defaults.py for scrubber behavior. (904850) - Filter attachments by filename extensions. (1027882) - Bugs and patches: 955381 (older Python compatibility), 1020102/1013079/ 1020013 (fix spam filter removed), 665569 (newer Postfix bounce detection), 970383 (moderator -1 admin requests pending), 873035 (subject handling in -request mail), 799166/946554 (makefile compatibility), 872068 (add header/footer via unicode), 1032434 (KNOWN_SPAMMERS check for multi-header), 1025372 (empty Cc:), 789015 (fix pipermail URL), 948152 (Out of date link on Docs), 1099138 (Scrubber.py breaks on None part), 1099840/1099840 (deprecated % insertion), 880073/933762 (List-ID RFC compliance), 1090439 (passwd reminder shunted), 1112349 (case insensitivity in acceptable_aliases) -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/

2 1