Mailman 3 June 2012 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett 07 Nov '22

07 Nov '22

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Re: [Mailman-Developers] [GSoC 2012] Metrics
by Stephen J. Turnbull 15 Jul '12

15 Jul '12

George Chatzisofroniou writes: > Author > > This model represents an author of the mailing list. It mostly keeps > track of the number of postings and number of threads started. It has > the following fields: > > - authorid – IntegerField AFAIK every Django object has an internal ID. Why do authors need a separate, human-unfriendly "authorid"? > - authormail – CharField Authors are people. They typically have names<wink/> and often multiple email addresses. There may also be other information (organization, etc) that is available from the headers. > - totalmails – IntegerField > - totalthreads – IntegerField > - firstmsgdate – DateTimeField > - lastmsgdate – DateTimeField > > MailingList > > This model counts the total number of postings and threads started. > > - totalmails – IntegerField > - totalthreads – IntegerField longestthread? > > Month > > Year > > Views > > To display the metrics the Django template system will be used. To > output the charts i will create some custom tags. The three following > views will be used: > > - General page – On top, there will be general metrics about total > authors, total mails and total threads and below three charts (AJAX > based) "AJAX based" doesn't belong in the spec; it's an implementation detail. > that represent number of posts per author, number of threads > per author and mailing list’s yearly usage. Even below there will be a > number of charts (equal to the number of years of list’s existence) > that output monthly usage. Why multiple charts? If you can afford a 640x480 chart area, with 4 pixel wide bars you can have 160 months > 13 years in one chart. I personally wouldn't hesitate to go to pixel width bars, which gives you > 53 years. I don't think people will be looking at charts for precision, but rather to get an overview. > At the end, there will be tabular data representing the authors > > - Author page – Each user will have his own page with his own metrics. > > Django Admin page – A ‘Generate’ button will be added to the Django admin page. > Settings > > The Django app should handle the following configuration parameters: > > - Host – Message store data host > - Port – Message store data port > - Masking – A multi-state variable (None, abbreviated, full) for > masking email addresses at the results (we don’t want the emails to be > spammed) (1) If at all possible, this should be inherited from the list configuration (DRY). It's not useful if the addresses are available from the archives or by subscribing to the list. (Actually, a really sophisticated spammer might want to attack by spoofing frequent posters on the assumption they're more trusted and more read, but that seems second-order to me.) (2) It would be preferable if authors could supply nicknames, full names, or avatars for this purpose. > Interface to the Mailman core > > - Metrics class > > - Generate class

4 4

Questions in regard to the database operations
by Danci Emanuel 02 Jul '12

02 Jul '12

Hi! My name is Emanuel Danci and I am the GSoC student working on integrating the Systers` feature, dynamic sublists - http://systers.org/systers-dev/doku.php/good_to_know?s%5B%5D=postgresql , with Mm 3.0 and I have 2 questions in regard to how the database works. First of all, how is the aliases set populated in time? Second of all, when a user is unsubscribed, is all the data related to him/her deleted (his/hers preferences, the info related to the address etc) ? Thank you, Emanuel

4 13

smtp error - [Errno 111] Connection refused - when trying to install MM 3.0
by Danci Emanuel 01 Jul '12

01 Jul '12

Hello everyone! This is the continuation of the topic started on the mailman-users mailing list. Here is the original post: http://mail.python.org/pipermail/mailman-users/2012-June/073632.html And this is the post at which the conversation stopped: http://mail.python.org/pipermail/mailman-users/2012-June/073636.html Basically, what I am trying to do is to set up a running version of Mailman on my local machine, by using the 5 minute guide (http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/view/head:/src/mail…). After doing what Mark Sapiro suggested in the last post, I got this: Jun 27 20:01:09 2012 (5011) [SMTPServer] listening: localhost:9025 Jun 27 20:01:09 2012 (5011) starting the SMTP server thread Jun 27 20:01:09 2012 (5011) connecting to localhost:9025 Jun 27 20:01:09 2012 (5011) [SMTPServer] starting asyncore loop Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/local/mm3/mailman/src/mailman/testing/layers.py", line 254, in setUp cls.smtpd.start() File "/usr/local/mm3/mailman/src/mailman/testing/mta.py", line 197, in start QueueController.start(self) File "/usr/local/mm3/mailman/eggs/lazr.smtptest-1.3-py2.6.egg/lazr/smtptest/controller.py", line 69, in start smtpd = self._connect() File "/usr/local/mm3/mailman/eggs/lazr.smtptest-1.3-py2.6.egg/lazr/smtptest/controller.py", line 60, in _connect smtpd.connect(self._server.host, self._server.port) File "/usr/lib/python2.6/smtplib.py", line 295, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib/python2.6/smtplib.py", line 273, in _get_socket return socket.create_connection((port, host), timeout) File "/usr/lib/python2.6/socket.py", line 514, in create_connection raise error, msg error: [Errno 111] Connection refused Also, there is one more thing that could be related to this problem. I also have installed Mailman 2.1.10 and after setting it up for the proper domain, when a user tries to subscribe to one of the lists, although the subscription is recorded, the confirmation email is never sent back. The message that appears in the smtp-failure log is this: Jun 04 20:55:57 2012 (10427) Low level smtp error: (111, 'Connection refused'), msgid: <mailman.0.1338832556.10889.systers-admin(a)manu.pensiunealdyna.ro> Jun 04 20:55:57 2012 (10427) delivery to esdanci(a)acm.ro failed with code -1: (111, 'Connection refused') Thanks, Emanuel

3 7

Re: [Mailman-Developers] [GSoC 2012] Metrics
by Patrick Ben Koetter 26 Jun '12

26 Jun '12

George, let me throw in some thoughts just to annoy you ;) Like with most statistical data I mostly see the figures being used to give statements on quantity - top poster, number of threads etc. Do you think it would be possible to also make some statements on quality? Let me give an example: Mailing lists are often places where people go to ask for advice. Someone asking usually starts a thread and continually keeps replying. That easily makes a person top poster and might make the same person a thread starter, but number of posts and threads started gives no indication of that persons knowledge (concerning the mailing lists topic). OTOH someone who has been on the list for ages, who replies more often than starting threads and who ends threads often after she has replied might very well be a very knowledgeable person, because she gives the one answer that solves the problem. Do you think it would be possible to deduct such quality oriented statements? p@rick P.S. Do you also plan to deliver a tool that analyzes a mailing list archive in order to gather your statistical data? Having the statistical data might be a good reason for people to upgrade their MMx installation to MM3. * George Chatzisofroniou <sophron(a)latthi.com>: > The following document is the lowest level of my design concept. You > may also read it in my blog [1]. Of course, comments are very welcome. > > -- > Models > > In order to store statistical data, the app will use some Django models: > > Author > > This model represents an author of the mailing list. It mostly keeps > track of the number of postings and number of threads started. It has > the following fields: > > - authorid – IntegerField > - authormail – CharField > - totalmails – IntegerField > - totalthreads – IntegerField > - firstmsgdate – DateTimeField > - lastmsgdate – DateTimeField > > MailingList > > This model counts the total number of postings and threads started. > > - totalmails – IntegerField > - totalthreads – IntegerField > > Month > > This model associates the author and the mailing list with each month. > > - author – ForeignKey > - month – CharField > - postscount – IntegerField > - threadscount – IntegerField > - mailinglist – Boolean (if this is true it corresponds to the whole > mailing list) > > Year > > This model is similar to month. It has a year field instead of a month field. > > Views > > To display the metrics the Django template system will be used. To > output the charts i will create some custom tags. The three following > views will be used: > > - General page – On top, there will be general metrics about total > authors, total mails and total threads and below three charts (AJAX > based) that represent number of posts per author, number of threads > per author and mailing list’s yearly usage. Even below there will be a > number of charts (equal to the number of years of list’s existence) > that output monthly usage. At the end, there will be tabular data > representing the authors of the mailing list along with their number > of posts, number of threads started and the date of their last post. > The user will be able to order the tabular data (alphabetically, > ascending/descending on number of posts, number of threads, date of > last message) by clicking on the table’s headings (Mail, Mails Sent, > Threads Started, Last Message). > > - Author page – Each user will have his own page with his own metrics. > On top, there will be the email of the author, number of posts, number > of threads started and the dates of first and last message. Below > there will be monthly usage charts for each year the user is > subscribed to the mailing list. > Django Admin page – A ‘Generate’ button will be added to the Django admin page. > Settings > > The Django app should handle the following configuration parameters: > > - Host – Message store data host > - Port – Message store data port > - Masking – A multi-state variable (None, abbreviated, full) for > masking email addresses at the results (we don’t want the emails to be > spammed) > > Interface to the Mailman core > > - Metrics class – When a new post is sent, the Metrics class will > receive it through the IArchiver interface. The Posts field of the > Mailing List model (as well as the the related rows on the Month and > Year models) will increase by one. If the author’s email is not in the > database, it will query the mailman core database with the email, grab > the author’s id and a new Author row will be created. Otherwise if the > author is already in the database, the Posts field and the two foreign > fields (Month and Year) will increase by one > > - Generate class – When the ‘Generate’ button on the Admin page is pressed: > * The Django models will be initialized (the metrics will go back to > zero). A progress bar will inform the administrator that the operation > is being processed. > * All the messages of the archive will be parsed by performing a > direct Python call to the IArchiver. Another instance to the IArchiver > will grab any mails sent while the parsing is going on. > * The metrics will be generated from scratch. > * The administrator will be informed with a success message when the > process is over. > > [1]: http://sophron.latthi.com/gsoc-mailman/ > > -- > George Chatzisofroniou > sophron.latthi.com > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/p%40state-of-mind… > > Security Policy: http://wiki.list.org/x/QIA9 -- state of mind () http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht München Partnerschaftsregister PR 563

2 2

Trying to install mm 3.0.0b1
by Lindsay Haisley 21 Jun '12

21 Jun '12

I'm trying to install mm 3.0.0b1 as per the directions in START.rst. I'm running into a bit of a problem with zc.buildout. in the bootstrap.py script. The script reports: $ python bootstrap.py Downloading http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg Traceback (most recent call last): File "bootstrap.py", line 254, in <module> ws.require(requirement) File "/tmp/tmp6ebmh9/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 666, in require File "/tmp/tmp6ebmh9/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 565, in resolve pkg_resources.DistributionNotFound: zc.buildout==1.5.2 However; $ dpkg -l python-zc.buildout Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-==================================-==================================-== ii python-zc.buildout 1.5.2-1 system for managing development buildouts I explicitly installed it! Do I need to update the mm3 package? Where do I go from here? -- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |

5 11

Personal patch
by Lindsay Haisley 18 Jun '12

18 Jun '12

Can someone give me some feedback on the following patch to SMTPDirect.py - whatever I've overlooked, or done that might be dangerous? The purpose of this patch is to insert a header, "X-subdata" into VERPed emails which won't be flagged and redacted by AOL's brain-dead "Email Feedback Report" system, and will continue to allow my local scripts to unsubscribe subscribers who hit the "Report Spam" button on their AOL mail UI. The content of the header is an MD5 hash of the receiving subscriber's email address - the same information contained in the Sender and Return-path headers, normally munged ("redacted") by AOL. The hope is that this hash will address AOL's privacy concerns, and/or else fall beneath the intelligence level of their scrutiny. The address hash can be compared against the list of subscribers to the list, identified in several (improperly redacted or un-redacted) headers. I'm not submitting this as a suggested patch for Mailman, but just asking for some feedback from people who know the code better than I do. --- SMTPDirect.py.orig 2012-06-17 17:16:25.000000000 -0500 +++ SMTPDirect.py 2012-06-17 21:17:25.000000000 -0500 @@ -43,6 +43,8 @@ from email.Utils import formataddr from email.Header import Header from email.Charset import Charset +from md5crypt import md5crypt +from random import choice DOT = '.' @@ -307,6 +309,9 @@ 'host' : DOT.join(rdomain), } envsender = '%s@%s' % ((mm_cfg.VERP_FORMAT % d), DOT.join(bdomain)) + saltmarsh = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrsyuvwxyz1234567890./" + if not msgdata.has_key("X-subdata"): + msgcopy["X-Subdata"] = md5crypt(rmailbox + "@" + DOT.join(rdomain), choice(saltmarsh) + choice(saltmarsh)) if mlist.personalize == 2: # When fully personalizing, we want the To address to point to the # recipient, not to the mailing list

2 2

Mailman 2.1.15 final released.
by Mark Sapiro 15 Jun '12

15 Jun '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am happy to announce the final release of Mailman 2.1.15. This release is identical to the 2.1.15rc1 release except for the version number and the inclusion of a missing part of the HTML installation manual. Python 2.4 is the minimum supported, but Python 2.6 is recommended. This release should work with Python 2.7, but has not been tested with that version. This release includes minor security enhancements, new features and bug fixes. See the Changelog at <https://launchpad.net/mailman/2.1/2.1.15> for more details. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. For more information, please see: http://www.list.org http://www.gnu.org/software/mailman http://mailman.sourceforge.net/ Mailman 2.1.15 can be downloaded from https://launchpad.net/mailman/2.1/ http://ftp.gnu.org/gnu/mailman/ https://sourceforge.net/projects/mailman/ - -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFP22s7VVuXXpU7hpMRAmqOAJ9rDBhSkYDW2yiaqljJzjQtPX8xgwCfXpS7 4I6YOtQjk4tn/UGrtyCGgIQ= =rsB+ -----END PGP SIGNATURE-----

3 4

MM3 - Using fqdn in the exposed URLs
by Richard Wackerbarth 12 Jun '12

12 Jun '12

Are we making a design mistake? The current design of the Postorius and Hyperkitty web interfaces to the mailing list and its archives uses the fully qualified list submission email address as a component of the URLs presented to the public. Is this really a good idea? Just think of the exposure that search engines, etc. will give to these email addresses. I fear that doing this will create an even greater invitation to those who harvest email addresses for the purpose of spamming and other nefarious reasons. Additionally, in the most common usage case, it makes the URL significantly longer than it needs to be. In most cases, the website address determines the email domain of the associated lists. Only a few websites are serving mailing lists from multiple email domains. Those sites would need to have some mechanism to unambiguiously identify the list being referenced. But for most sites, the common name of the list is sufficient. One of the design principles of Django is that the website designer can present his content by way of URLs of his choosing. Presenting the actual email address of a list may "leak" information that the user wishes to obscure. I think that we should rethink this decision and follow a "slug" approach to the identification of the mailing lists in URLs. Those who choose to do so can use the fqdn as their slug. But others would be able to readily change the mapping without having to rewrite significant parts of the interface code. Comments?

6 10

Re: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter
by Richard Wackerbarth 11 Jun '12

11 Jun '12

Jessy, Yes, there are plans (and activity) related to the presentation of list archives within the django framework. In fact, Aamir, one of our GSoC interns, is working on that very subject as his project for the summer. In keeping with the django philosophy of combining many special purpose "apps" to create the overall website, the archive access will be packaged as a component separate from the postorius administrative interface. Richard On Jun 10, 2012, at 11:57 PM, Jessy Kate Schingler wrote: > one question i had about the UI - are there plans for including views/templates for list archives?

2 1