Mailman 3 September 2013 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett 07 Nov '22

07 Nov '22

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Re: [Mailman-Developers] Migration script from MM2 to MM3
by Barry Warsaw 15 Jan '14

15 Jan '14

On Sep 20, 2013, at 12:46 PM, Aurelien Bompard wrote: >What do you think? Bad timing? Any existing work I'm missing? I think it's a *great* idea. No, perfect timing. Not that I know of! :) What we've said in the past is that we weren't going to promise conversion scripts from 2.1 to 3.0, partly because that work was incomplete, but also because we wanted to caution users not to just blindly upgrade as soon as 3.0 is released. But I think having a reliable, well-tested, non-destructive option to upgrade would be wonderful, especially because I fully hope that a site could run 2.1 and 3.0 together for a while. Any help you need, let me know. (I've been ridiculously swamped at work these past few weeks. Apologies for being rather quiet lately. I hope that'll improve in the next couple of weeks.) Cheers, -Barry

3 3

Mailman 2.1.16 release.
by Mark Sapiro 23 Nov '13

23 Nov '13

FYI, I will be releasing 2.1.16rc1 in a few days with a target of a final release in early September. I believe the release will be very solid and stable. The main purpose of the candidate release is to expose any i18n changes so that translators can submit any updates and get them in the final release. There are a few new features, contributed programs, i18n changes and bug fixes, all of which will be announced when the candidate is released. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 10

GSoC progress report
by Manish Gill 26 Sep '13

26 Sep '13

This is the final progress report for my project, Authenticated REST API for Postorius/Django. The code is available at https://bitbucket.org/mgill25/mm-rest I'll go over the major points of the work done so far: * The public-facing REST Interface is mostly done. Most business use cases, things like creating/deleting lists, subscriptions etc are working via REST. * The REST Interface currently works with Basic Auth, but other authentication schemes are easily pluggable via the Django REST framework. * Postorius has been modified in order to use the MM-REST models instead of directly interacting with Mailman Core. * Hence, in the case of no connection available to Core, MM-REST, and consequently Postorius, continue to work on their own, saving data in their own level. Which brings me to... * The Caching Interface b/w Mailman Core and MM-REST, which is yet to be completely finished. I have a (somewhat) generic interface, but haven't managed to make it work for all the models, primarily Memberships. * Workaround in mm-rest or modifications in Core for allowing backing up of certain Models, which is currently limited by the Core REST API. For example: Registering new Email addresses for a User. * And there's a small todo file in the project containing some things to be done yet. The project is not 100% finished yet, and after the GSoC is formally over, I shall continue working on it.

1 0

Re: [Mailman-Developers] Author_is_list option in upcoming mailman 2.1.16
by Stephen J. Turnbull 25 Sep '13

25 Sep '13

Murray S. Kucherawy writes: > I wonder how long we can hold out before we start trying to drag > [the MUA developers] into our conversations, which might be the > only way to solve these pain points long term. It seems to me that > Gmail, Yahoo Mail, Thunderbird, etc., must have either a team or an > individual that spends time thinking about and testing user-visible > solutions to these problems, so perhaps it's time to start asking > them for help. To be honest, I don't think they really care. Even larsi, who never saw an internet-draft he wouldn't implement, is not terribly systematic about it -- as long as things are smooth between Gnus users, well, tough luck for the rest of the world. When I was talking to MUA developers about best practices for dealing with reply-to-list the basic response was "we already have a function for that" or "sounds great, patches welcome". The Mozilla people were clearly much more interested in features like calendars and vcards. I think it's important we get started on it (and maybe I can contribute something now that GSoC is almost over), but I don't have much hope that the MUA developers will put a high priority on it. It's not really in their purview (see Franck Martin's comments about MUA irrelevance in this thread, for example, and he's not even an MUA developer). And of course the worst offender is Microsoft, which AFAICS is somewhat actively undermining the RFC 822 standard for reasons I don't understand. Regards,

1 0

Re: [Mailman-Developers] Author_is_list option in upcoming mailman 2.1.16
by Franck Martin 25 Sep '13

25 Sep '13

On Sep 17, 2013, at 6:21 PM, Mark Sapiro <mark(a)msapiro.net> wrote: > On 09/17/2013 05:28 PM, Barry Warsaw wrote: >> On Sep 15, 2013, at 08:24 AM, Mark Sapiro wrote: >> >>> Because the issue remains controversial, I will soon release 2.1.16 >>> final with the feature disabled by default, and will consider the >>> message encapsulation approach or other possibilities based on >>> experience with 2.1.16 for a 2.1.17 release perhaps early next year. >> >> This makes sense to me, although I would label the feature "provisional" or >> "experimental". There just isn't any good experience here we can draw on, so >> it seems reasonable to provide the knobs that will allow motivated folks to >> gather such experience, but generally keep it out of the way for the majority >> of users. > > > I intend to so indicate in the NEWS about the feature. > > I have given some thought to the encapsulation approach and have some > questions about it. > > My thoughts on how to do it include the following if the feature is enabled. > > CookHeaders saves the original Subject: before prefixing in the metadata. > > A new handler before ToOutgoing but after ToDigest, ToArchive and > ToUsenet creates a new message From: the list with Content-Type: > message/rfc822, a unique Message-ID: and Subject:, References: and > In-Reply-To: copied from the current message. It then replaces the > Subject: in the current message with that saved in the metadata and sets > the payload of the new message to the current message. > > This will result in a single-part message with a payload of the content > filtered original message. If content filtering hasn't removed anything, > the original message's DKIM signatures if any should still be valid. > > The message then goes to ToOutgoing and eventually OutgoingRunner and > SMTPDirect which will call Decorate and if any msg_header or msg_footer > is added, these will be added as is currently done which will result in > the message becoming multipart/mixed with the addition of one or two > text/plain, Content-Disposition: inline parts containing the header > and/or footer. > > The idea is the message/rfc822 part preserves as much of the original as > possible so its DKIM sigs if any may still validate and to put enough > into the headers of the new message so MUAs can still thread it properly > and render it nicely. Also, the message is unchanged over current > behavior for digests, archives and usenet. > > The sticky questions are what to do with the original From: and > Reply-To: in the face of possible Reply-To: munging options and so on. > Presumably, we want to still be able to reply to the original author, > and preserve the behavior of a simple MUA 'reply' going to the original > author and not the list in the absence of Reply-To: munging, and that > could be accomplished by putting the original author's Reply-To: (or the > original From: if no original Reply-To:) in the new message's Reply-To:, > but it's not yet clear to me how to handle the munging options (maybe > just ignore them ;). > > I could actually implement this approach for the 2.1.16 release, but > that would negate the i18n work that's already been done as the > descriptive string on General Options would change, so I'm more inclined > to label this feature as experimental and subject to change > significantly in a future release. > 1) If you keep the From: header as it is then, we will still have the same problems 2) the purpose of the encapsulation message/rfc822 is not meant to preserve the DKIM signature, DKIM is not meant to be verified by MUAs 3) encapsulation is not there to provide a transitive trust, there is another method explored for that which is Original-Authentication-Results (OAR). There is an expired Internet Draft on it. I think, it is risky to code this encapsulation method directly and now, it requires a branch some testing and then merging back into the main branch. The author_is_list has had deployment and testing for over a year in a DMARC environment. Limited testing I agree but nevertheless proved not to break things ( besides people ideal view of email ;) ) and be useable with many MTAs and especially MUAs Here is a recent test, deployment and analysis: http://sys4.de/en/blog/2013/08/11/mailman-dmarc-konform-betreiben/ I'd like to see somebody operating a mailing list with this encapsulation method first, before merging.

5 5

Exim4 support for Mailman 3
by Stephen J. Turnbull 24 Sep '13

24 Sep '13

Hi all, I have successfully added Exim 4 support to Mailman 3. The branch is here: lp:~stephen-xemacs/mailman/exim4 There's an instance at maxking-test(a)turnbull.sk.tsukuba.ac.jp which also contains Abhilash's signed-list feature. Since it's very strict about checking signatures, it probably won't be much fun to use. :-P Postorius is running at http://turnbull.sk.tsukuba.ac.jp/postorius It's a separate process group from my main site, so the URLs you'll see in the address bar aren't very pretty, it needs the extra "postorius" in front to distinguish the top page, and I haven't bothered to fix the rest of the name space. No HyperKitty yet, sorry.

1 0

GSoC Final report
by Abhilash Raj 24 Sep '13

24 Sep '13

Hi, So I have pushed my final commit for now for my GSoC project. I will try to be brief on all the accomplishments and mention concisely what-is-working. * Signature rule following strict rfc 3156 can now be verified, only pgp-mime messages are accepted and messages without a (valid) signature are held. * SignMessage handler creates multipart/signed message, first of whose part is the original message formed after all the processing(in default-posting-pipeline this handler is at last before the message is send out or copied to nntp or any other queue). It expects a `multipart/signed` or `multipart/mixeed` message. The signature is calculated on whole part using the list's secret key. * A List's key can be created from postorius and other signature parameters can also be changed from there( only 'signature_max_age for now). * A user can import his public key from public keyservers( defaults to ha.pool.sks-keyservers.net) by entering their public key id or they can copy paste their public key data to upload their public key.( Exceptions handling is not implemented yet in postorius, if works fine in perfect situations). I hope to improve all these things even more but seems like I have a short of time now, I will update you( and the mm-dev list) with what other plans I have to furthur improve these parts. P.S.: There is a working version of my copy of mailman here[1] if you want to have a look. -- Abhilash Raj [1]: http://maxking.emoir.co.in

2 1

Migration script from MM2 to MM3
by Aurelien Bompard 20 Sep '13

20 Sep '13

Hi all, I'm in the process of testing the conversion from Fedora's current mailman 2.1 instance to a test 3.0 instance. I realize the import21 command is currently pretty naive and does not migrate most of the settings, nor the member lists. I've looked at bug 965532 [1] and more recently the email thread about creating such a script during GSOC [2], and unless I'm mistaken nothing moved in that general area during the summer. Am I right or did I miss something? [1] https://bugs.launchpad.net/mailman/+bug/965532 [2] http://www.mail-archive.com/mailman-developers@python.org/msg13464.html I'd like to work on that, if that's the right time and no-one else is working on it. I'll probably extend the import21 command to convert the other settings. I understand the tricky part is to convert email addresses in 2.1 to users in 3.0, so that's probably going to be a separate module with its own tests. At least it'll serve as a point of reference and we'll be able to refine the use cases and the algorithm later on. What do you think? Bad timing? Any existing work I'm missing? Aurélien

1 0

Author_is_list option in upcoming mailman 2.1.16
by Franck Martin 19 Sep '13

19 Sep '13

In the upcoming mailman 2.1.16 there has been the introduction of the optional feature author_is_list "Replace the sender with the list address to conform with policies like ADSP and DMARC. It replaces the poster's address in the From: header with the list address and adds the poster to the Reply-To: header, but the anonymous_list and Reply-To: header munging settings below take priority. If setting this to Yes, it is advised to set the MTA to DKIM sign all emails. " Usage of this feature on lists have indicated no adverse issue for the members, provided proper communication is done when the feature is enabled (as to allow inbox rules to be changed if needed). In the 2.1.16 Release Candidate the feature author_is_list is controlled by the following switches in the mm_cfg.py ALLOW_AUTHOR_IS_LIST = No DEFAULT_AUTHOR_IS_LIST = No The first one will enable the GUI to present an option to the list admin to enable the author_is_list feature, the second controls if new lists or upgraded lists should have the author_is_list feature set to yes While it is better for the MTA to DKIM sign emails if author_is_list is enabled, this is not a requirement and the list will work fine without DKIM. While DEFAULT_AUTHOR_IS_LIST is important to avoid new lists and upgraded lists change behavior, setting ALLOW_AUTHOR_IS_LIST to Yes does not change how any list is handled (author_is_list in the GUI is No by default). it only provides an option to the list admin to change the list behavior. Unfortunately some list admins cannot edit mm_cfg.py (like CPANEL type install), therefore it would be nice to remove ALLOW_AUTHOR_IS_LIST or set it to Yes by default to let the list admin decide how he/she wants the list to behave. Otherwise the list admin needs to contact the mailman admin to enable this config. Please indicates if you are ok to set ALLOW_AUTHOR_IS_LIST to Yes or remove this option from mm_cfg.py Note: a few organizations have indicated they will provide the necessary translation of this feature in the GUI for the most common languages.

6 8