Mailman 3 August 2007 - Mailman-Developers

Improving the archives
by Terri Oda Oct. 3, 2007

Oct. 3, 2007

Since I've largely finished up the coding contract that was eating up a lot of my time, I'm thinking that I'd like to do some coding for fun. And nothing says fun like trying to fix the Mailman archives! ;) I'm trying to remember all the things people have suggested for the archives in the past so I can figure out what needs to be done and what might be nice to have, and see if this is doable in the time I have in the foreseeable future. The big things people wanted most, if I recall correctly, included: - modernized HTML/CSS/Themes (preferably to match a modernized web interface... is that all set up now?) - archive links that won't break if the archive is rebuilt - better address obfuscation (maybe by generating pages through cgi) - search - not adding a billion dependencies to Mailman Here's the list from the wiki's Mailman 2.2 page: http:// wiki.list.org/display/DEV/Mailman+2.2 * Reconsider using a 3rd-party archiver * Perhaps URLs to messages should be based on message-ids instead of message numbers so that regenerating archives can't break links. This must include backward compatible links * Ditch direct access and vend all archive messages through CGI so that we can do address obfuscation, and message deletion, etc. on the fly (with caching of course, but have to worry about web crawlers). * Add RSS feed * Allow for admins to remove or edit messages through the web. * Move archive threads into another list? * Put archives in the list/mylist directory. * Add a search option * Make archives default template look and feel similar to Web UI (whatever it looks like after the Summer of Code project is done) * Make archive templatable (at least by changing CSS) so they can match people's existing site look-and-feel * MUAs usually make URLs clickable. An new Archive could be used when posts are distributed, in the footer, so that each message has a link to the whole thread in the Archive. * Present all messages in a thread at once, and offer plaintext download of the whole thread * Put messages into a database and/or move away from mbox as the canonical storage format. So the questions are: (1) Is anyone working on this already? (2) What else is on people's wish lists for a pipermail replacement? Terri

16 63

Required template files
by Aaron Crosman Aug. 19, 2007

Aug. 19, 2007

I've been working that last day or two to help update the templates for the interim step of allowing CSS files on the templates. I've noticed several translations are missing some template files. Is this a known condition that behaves just fine, or is this a problem that needs to be addressed? Also, is there a list in the wiki of all the template files? I'm assuming the English translation is complete and current and I have based my work off that template, but other translations include files missing in the en folder, while more than 1/2 of the other languages are missing files present in en. By my count, of the 41 translations, at least 25 are missing files. Aaron

2 1

[patch] Spam report functionality
by Rick van der Zwet Aug. 10, 2007

Aug. 10, 2007

Hi All, The patch attached (created against 2.1.9), will add functionality where the moderator is able to report a moderated message as spam. This message will than either be stored on the disk or will be send -to a in the config- definable email address. Together will a small shell script it will be pretty easy to load this message into a spam learning program like sa-learn. This will make the moderators more happy, cause they are able to contribute to the improvement of the spam filter. I do believe this will gradually lower the amount of false negatives, cause having people to teach the spam filter is great :-) Best regards, /Rick -- http://rickvanderzwet.nl diff -u -x *.pyc -r ./Cgi/admindb.py /usr/local/mailman/Mailman/Cgi/admindb.py --- ./Cgi/admindb.py Wed Aug 30 16:54:22 2006 +++ /usr/local/mailman/Mailman/Cgi/admindb.py Fri Aug 10 01:53:24 2007 @@ -196,6 +196,11 @@ ' ' + _('Discard all messages marked Defer') )) + form.AddItem(Center( + CheckBox('reportspamalldefersp', 0).Format() + + ' ' + + _('Report all messages marked Defer as SPAM') + )) # Add a link back to the overview, if we're not viewing the overview! adminurl = mlist.GetScriptURL('admin', absolute=1) d = {'listname' : mlist.real_name, @@ -244,6 +249,11 @@ ' ' + _('Discard all messages marked Defer') )) + form.AddItem(Center( + CheckBox('reportspamalldefersp', 0).Format() + + ' ' + + _('Report all messages marked Defer as SPAM') + )) form.AddItem(Center(SubmitButton('submit', _('Submit All Data')))) doc.AddItem(mlist.GetMailmanFooter()) print doc.Format() @@ -302,11 +312,13 @@ radio = RadioButtonArray(id, (_('Defer'), _('Approve'), _('Reject'), - _('Discard')), + _('Discard'), + _('Spam')), values=(mm_cfg.DEFER, mm_cfg.SUBSCRIBE, mm_cfg.REJECT, - mm_cfg.DISCARD), + mm_cfg.DISCARD, + mm_cfg.SPAM), checked=0).Format() if addr not in mlist.ban_list: radio += ' ' + CheckBox('ban-%d' % id, 1).Format() + \ @@ -361,11 +373,13 @@ RadioButtonArray(id, (_('Defer'), _('Approve'), _('Reject'), - _('Discard')), + _('Discard'), + _('Spam')), values=(mm_cfg.DEFER, mm_cfg.UNSUBSCRIBE, mm_cfg.REJECT, - mm_cfg.DISCARD), + mm_cfg.DISCARD, + mm_cfg.SPAM), checked=0), TextBox('comment-%d' % id, size=45) ]) @@ -401,9 +415,11 @@ left.AddCellInfo(left.GetCurrentRowIndex(), 0, colspan=2) btns = hacky_radio_buttons( 'senderaction-' + qsender, - (_('Defer'), _('Accept'), _('Reject'), _('Discard')), - (mm_cfg.DEFER, mm_cfg.APPROVE, mm_cfg.REJECT, mm_cfg.DISCARD), - (1, 0, 0, 0)) + (_('Defer'), _('Accept'), _('Reject'), + _('Discard'), _('Spam')), + (mm_cfg.DEFER, mm_cfg.APPROVE, mm_cfg.REJECT, + mm_cfg.DISCARD,mm_cfg.SPAM), + (1, 0, 0, 0, 0)) left.AddRow([btns]) left.AddCellInfo(left.GetCurrentRowIndex(), 0, colspan=2) left.AddRow([ @@ -633,13 +649,15 @@ # We can't use a RadioButtonArray here because horizontal placement can be # confusing to the user and vertical placement takes up too much # real-estate. This is a hack! - buttons = Table(cellspacing="5", cellpadding="0") - buttons.AddRow(map(lambda x, s=' '*5: s+x+s, - (_('Defer'), _('Approve'), _('Reject'), _('Discard')))) + buttons = Table(cellspacing="6", cellpadding="0") + buttons.AddRow(map(lambda x, s=' '*6: s+x+s, + (_('Defer'), _('Approve'), _('Reject'), + _('Discard'), _('Spam')))) buttons.AddRow([Center(RadioButton(id, mm_cfg.DEFER, 1)), Center(RadioButton(id, mm_cfg.APPROVE, 0)), Center(RadioButton(id, mm_cfg.REJECT, 0)), Center(RadioButton(id, mm_cfg.DISCARD, 0)), + Center(RadioButton(id, mm_cfg.SPAM, 0)), ]) t.AddRow([Bold(_('Action:')), buttons]) t.AddCellInfo(row+3, col-1, align='right') @@ -692,6 +710,10 @@ discardalldefersp = cgidata.getvalue('discardalldefersp', 0) except ValueError: discardalldefersp = 0 + try: + reportspamalldefersp = cgidata.getvalue('reportspamalldefersp', 0) + except ValueError: + reportspamalldefersp = 0 for sender in senderactions.keys(): actions = senderactions[sender] # Handle what to do about all this sender's held messages @@ -701,11 +723,19 @@ action = mm_cfg.DEFER if action == mm_cfg.DEFER and discardalldefersp: action = mm_cfg.DISCARD + elif action == mm_cfg.DEFER and reportspamalldefersp: + action = mm_cfg.SPAM if action in (mm_cfg.DEFER, mm_cfg.APPROVE, - mm_cfg.REJECT, mm_cfg.DISCARD): - preserve = actions.get('senderpreserve', 0) - forward = actions.get('senderforward', 0) - forwardaddr = actions.get('senderforwardto', '') + mm_cfg.REJECT, mm_cfg.DISCARD, + mm_cfg.SPAM): + if action == mm_cfg.SPAM: + preserve = mm_cfg.SPAM_STORE + forward = mm_cfg.SPAM_FORWARD + forwardaddr = mm_cfg.SPAM_FORWARD_ADDRESS + else: + preserve = actions.get('senderpreserve', 0) + forward = actions.get('senderforward', 0) + forwardaddr = actions.get('senderforwardto', '') comment = _('No reason given') bysender = helds_by_sender(mlist) for id in bysender.get(sender, []): @@ -759,7 +789,7 @@ continue if v not in (mm_cfg.DEFER, mm_cfg.APPROVE, mm_cfg.REJECT, mm_cfg.DISCARD, mm_cfg.SUBSCRIBE, mm_cfg.UNSUBSCRIBE, - mm_cfg.ACCEPT, mm_cfg.HOLD): + mm_cfg.ACCEPT, mm_cfg.HOLD, mm_cfg.SPAM): continue # Get the action comment and reasons if present. commentkey = 'comment-%d' % request_id diff -u -x *.pyc -r ./Defaults.py /usr/local/mailman/Mailman/Defaults.py --- ./Defaults.py Fri Aug 10 00:30:06 2007 +++ /usr/local/mailman/Mailman/Defaults.py Fri Aug 10 01:48:45 2007 @@ -167,6 +167,12 @@ # -owners address, unless the message is explicitly approved. KNOWN_SPAMMERS = [] +# Spam actions +# Default to store spam messages +SPAM_STORE = 1 +SPAM_FORWARD = 0 +SPAM_FORWARD_ADDRESS = 'spam2trainer(a)example.com' + ##### @@ -1220,6 +1226,7 @@ UNSUBSCRIBE = 5 ACCEPT = 6 HOLD = 7 +SPAM = 8 # Standard text field width TEXTFIELDWIDTH = 40 diff -u -x *.pyc -r ./ListAdmin.py /usr/local/mailman/Mailman/ListAdmin.py --- ./ListAdmin.py Sat Aug 27 03:40:17 2005 +++ /usr/local/mailman/Mailman/ListAdmin.py Fri Aug 10 00:50:36 2007 @@ -295,10 +295,13 @@ self.__refuse(_('Posting of your message titled "%(subject)s"'), sender, comment or _('[No reason given]'), lang=self.getMemberLanguage(sender)) - else: - assert value == mm_cfg.DISCARD + elif value == mm_cfg.DISCARD: # Discarded rejection = 'Discarded' + else: + assert value == mm_cfg.SPAM + # Spam reported + rejection = 'SpamReported' # Forward the message if forward and addr: # If we've approved the message, we need to be sure to craft a

2 2

Major updates to Mailman 3.0 branch
by Barry Warsaw Aug. 4, 2007

Aug. 4, 2007

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've just merged in my 'setuptools' branch to the official Mailman 3 branch. This means that all the autoconf-based building cruft is gone, dead, and buried. Can you say "Yay"? :) Instead, Mailman is now a setuptools based project, which is rapidly becoming the standard for Python applications and libraries. There are lots and lots of benefits to this change; for developers, the most immediate benefit is that there's no more configure/make/ makeinstall dance for every little change. It means you can build a 'development' installation of Mailman and edit and test right in place. This will hugely reduce the overhead for developing the code. Also gone are the C wrapper programs, so Mailman is now a pure Python application. With the change to wsgi-based web integration, and maildir delivery from the MTA, we really don't need them any more. You'll notice a bunch of other things have disappeared too, like the 3rd party packages we were distributing in the misc/ directory. Instead, these are downloaded on demand from the Python Cheeseshop <http://cheeseshop.python.org>. All the packages Mailman depends on live in the Cheeseshop so we don't need to distribute them any more. And now that we're a setuptools-based project, when you build Mailman (see below), these dependent packages will be automatically downloaded and installed as necessary. You'll need a net connection for the initial build, but after that, once the packages are installed, you're good to go. To prepare your existing branch for the update, start by doing a 'make distclean' followed by a 'bzr revert'. IMPORTANT - don't do this if you have uncommitted changes! 'bzr stat' should now report no changes. Next, cd into your 'misc' directory and remove the following directories: - - Elixir-0.3.0 - - SQLAlchemy-0.3.3 - - setuptools-0.6c3 - - zope.interface-3.3.0.1 These are the unpacked dependent packages and you don't need them any more (in fact, they'll get in your way now). Next, remove any residual .pyc files laying around, via: % find . -name \*.pyc -print | xargs rm Now do your 'bzr pull' to get the latest 3.0 branch changes. If you have local modifications, you'll need to do a 'bzr merge' and resolve any conflicts. To see if everything's cool, pick a 'development' directory. I usually use a subdirectory called 'staging' in my 3.0 working tree. This development directory can be anything, but then do this: % export PYTHONPATH=<my staging dir> for me "<my staging dir>" would be `pwd`/staging. Make sure this directory exists. Next do this: % python2.5 setup.py develop --install-dir <my staging dir> After churning for a bit, downloaded some stuff, etc., look in your staging directory. You'll have a bin directory there with all the Mailman command line scripts, but all the code will remain in your working tree. You can now do this to run the test suite: % <my staging dir>/bin/testall You should see 72 tests passing and no failures. Though you probably won't get very far, the way this is going to work in deployed installations is that you'll have to run bin/ make_instance to create some directories and do a few other things that the configure and Makefile.in's used to do. After running that, you'll have an etc/mailman.cfg file that you can tweak for your installation. Note that you don't need to do this for bin/testall because the testing infrastructure creates a temporary instance that it uses. Let me now if you have any problems. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRqJro3EjvBPtnXfVAQL0OgQAktryl9zYZlZrm9dC644EL2hF+FMHs50v 2JTh2w6UhULAdpj+xneKR04pEfCw6HHNult03jo6NoNjYjMmzpUycDHXj7e0RaKE mbhMkX2v2b6d01OsMrAbAyWBTZH+2rtmjEKANd/1/+LlIdy3KlJe09m+xgNY9VsV keEIEdVQcYc= =J3Mq -----END PGP SIGNATURE-----

2 2

add_members: Support for invite & pre-generated passwords [patch]
by Julian Mehnle Aug. 1, 2007

Aug. 1, 2007

Hi all, Here is a patch for the add_members command-line utility that adds support for inviting users and specifying pre-generated passwords: http://sourceforge.net/tracker/index.php?func=detail&aid=1765041&group_id=1… http://files.mehnle.net/software/mailman/mailman-add_members:passwords-inpu… If you specify a "--invite-only" option, the users are invited instead of being subscribed right away. You can also specify pre-generated passwords as the last "word" (of non- whitespace characters) in address list files. For example: John Doe <john.doe(a)example.com> PaSsWoRd jane.doe(a)example.org pAsSwOrD user.with.autogen.password(a)example.net Consider the patch as being in the public domain. I'd be happy if these features could make it into one of the next Mailman releases. I was missing them so hard, I actually sat down and started learning Python (I mostly just write Perl otherwise). Julian.

1 0