Mailman 3 July 2015 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett 07 Nov '22

07 Nov '22

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Mailman 3 Error & some questions
by Krinetzki, Stephan 05 Aug '15

05 Aug '15

Hello, I am planning to migrate from mailman 2.1.14 to mailman 3. At the moment mailman 3 is running on a new CentOS 7 server with postfix as MTA. But I have some questions and problems: 1. I have an error with hyperkitty ERROR 2015-07-29 08:11:02,287 update_index 7216 140406344681280 Failed to update the fulltext index: 'Command' object has no attribute 'commit' Traceback (most recent call last): File "/opt/mailman/mailman-bundler/eggs/HyperKitty-1.0.1-py2.7.egg/hyperkitty/job s/update_index.py", line 58, in execute update_index() File "/opt/mailman/mailman-bundler/eggs/HyperKitty-1.0.1-py2.7.egg/hyperkitty/sea rch_indexes.py", line 73, in update_index update_cmd.update_backend("hyperkitty", "default") File "/opt/mailman/mailman-bundler/eggs/django_haystack-2.4.0-py2.7.egg/haystack/ management/commands/update_index.py", line 233, in update_backend do_update(backend, index, qs, start, end, total, verbosity=self.verbosity, commit=self.commit) AttributeError: 'Command' object has no attribute 'commit' As far as I can tell, there is an error with the haystack indexing of archived mails. But how to fix this? I'm clueless. 2. In mailman 2, I've created a list and added a new owner. Then the owner get an email, with his password and a link to the admin page of the list. How does this work in mailman3? If I create an list in mailman3 and set the owner, then the email address is the owner, but he isn't noticed. So what steps are needed to get an password for the new owner and admin of the list? 3. Is it possible to get Shibboleth ( <https://www.shibboleth.net> https://www.shibboleth.net) working with mailman3? 4. The link in the welcome mail to the info page of the list is wrong. It says <http://lists.example.com/listinfo/list@lists.example.com> http://lists.example.com/listinfo/list@lists.example.com and if I visit this link, I get an 404 error. (example.com is here a real domain). The Second link says <http://example.com/me@host.com> http://example.com/me@host.com where the example.com part seems to be hardcoded. Thank you for your help and I hope my English is not too bad (I'm just a German IT Administrator...) Greetings from Germany Stephan Krinetzki

4 7

Re: [Mailman-Developers] Merging users
by Barry Warsaw 04 Aug '15

04 Aug '15

On Jul 21, 2015, at 06:39 PM, Aurelien Bompard wrote: >If I try to associate an existing address to an existing user (without >this address obviously), the Mailman API will reply with a 400 error >"User already exists". Postorius currently does not detect that and >just errors out with a 500 page (but that's not the issue here). > >I think I need to merge the two users, and that's currently not >supported. I believe it would be a useful feature, so I'd like to >discuss it here. IMHO, the only tricky part would be to merge the >preferences associated with each user, but if I know which of my two >users has priority, I can just resolve the conflicts that way. >Am I missing something? Do you agree it would be a useful feature to have? So, Abhilash submitted a merge request to allow for unlinked addresses to be linked to existing users via the REST API. https://gitlab.com/mailman/mailman/merge_requests/30 That's essentially the correct fix for the first order problem (though there's a subtle bug in the mr that I will fix manually). If that answers your question, then great! Actually merging two existing users is indeed a tricky thing because users link to so many other interesting bits and pieces. Two separate users have two separate memberships, linked addresses, preferred addresses, and yes, preferences. There are also the created_on, display_name, and (the mostly unused) password attributes. How deep does merging users go? Cheers, -Barry

2 1

Re: [Mailman-Developers] [Mailman-Users] Mailman 3: Deployment without LMTP
by Barry Warsaw 31 Jul '15

31 Jul '15

[Redirecting to mailman-developers] On Jul 25, 2015, at 01:34 PM, Michael Hierweck wrote: >We are currently running Mailman 2 in an very low-traffic environment >without access to the mail server configuration. We can either use >.forward or aliases to forward mails directly to Mailman via pipe. >Fetching mails from POP3/IMAP mailboxes would be possible, too. > >We would like to upgrade to Mailman 3 now. Thanks for lots of good work >over many years btw. However Mailman 3 only supports feeding mails via >LTMP. Is is possible to build a bridge between .forward/aliases and >Mailmans LMTP interface in a convenient and secure manner? > >An option might be letting fetchmail fetch the mails from a mailbox and >forward them via LTMP. As the mails have already been successfully >delivered to the mailbox before Mailman might reject them I can't image >a chance of handling bounces due to rejects correctly in this setup. > >Therefore I'm thinking of a small Python script that receives mail from >stdin and forwards them via LMTP but I'm unsure whether this would be a >correct way to achieve the goal because there might by problems with >bounces when Mailman rejects those mails. This script oughts to >terminate unsuccessfully when the LTMP delivery fails and should >therefore make Postfix bounce the mail itself. > >Postfix -> .forward -> pipe --(stdin)--> script --(LMTP)--> Mailman 3 > >Any opinions or hints are appreciated. Thanks in advance. It wouldn't be out of the question to do it this way, although you're right that you have to have some special handling for rejections. Technically speaking, Mailman would still support "bounces", which are messages created out of whole-cloth by Mailman sent to the original sender out of band to the LMTP conversation, encapsulating the original message. In fact, your dot-forward-to-LMTP bridge would probably have to capture the LMTP 4xx and 5xx rejection codes and craft a bounce, although it could potentially allow Mailman itself to deliver this. Remember though that the LMTP runner isn't all *that* special, in that upon successful acceptance of the message, it will just drop it in the appropriate queue for further processing. It's certainly conceivable that one could implement a new type of runner that interfaced with dot-forward files in much the same way that MM2 did, but with logic a bit more compatible with MM3's queue structure and database. The trickiest part of all this is handling the security implications of using a dot-forward program. The core only supports LMTP delivery by default in MM3 so we get to ignore all the C wrappers and gid-mismatch fun of MM2. FWIW, this could all be built as a separate plugin for the core. Cheers, -Barry

1 0

Migration from mailman2 to mailman3
by Shivam Shukla 31 Jul '15

31 Jul '15

Hi, I want to install mailman3 and use it for all my mailing facilities. Previously i was using mailman2. So i need to migrate all my mailman2 data to mailman3 for which i am not able to find any resource. Although i read at few places that migration will be supported in mailman3.1. Can anyone please provide more information if migration will be supported in mailman3.1 and if it will be then at what time i can expect that release? Thanks

2 1

Re: [Mailman-Developers] REST API speed
by Eric Searcy 30 Jul '15

30 Jul '15

On 7/29/15 10:38 PM, Abhilash Raj wrote: > Hi Eric, > > It's curious to me that Mailman Bundler makes references to Gunicorn > support, but it seems there code would run into the same problems I did > (multiple initiations and argument count mismatch). Is anybody using > this from Bundler currently and can help me figure out why the > difference? > > > The references to Gunicorn in Mailman-Bundler is just for deplying the > django > project in which the Postorius and Hyperkitty run. Oh, of course. Thanks, Eric

1 0

REST API speed
by Eric Searcy 30 Jul '15

30 Jul '15

I'm curious about how the REST interface scales out. I've posted a bug to Gitlab about Postorius making a REST request for every list member on the list info page [1], but aside from waiting for a fix for that, I've also been looking more into the performance of the Mailman3 REST interface itself. Testing with siege (15 concurrent users) on a production VM, I get 29 requests/sec and a concurrency of just 1.3x requesting the "/3.0/domains" URL. For "/3.0/system/versions" I get only about 0.2x concurrency. This may not scale too well to a very active Postorius, especially if 1 Postorious request results in many Mailman API requests. Is anyone using anything besides wsgiref for the REST interface? (uwsgi, gunicorn, ?) I thought I could get some extra performance that way... Eric [1] https://gitlab.com/mailman/postorius/issues/25

5 6

Re: [Mailman-Developers] Merging users
by Abhilash Raj 29 Jul '15

29 Jul '15

On 29 July 2015 at 20:03, Abhilash Raj <raj.abhilash1(a)gmail.com> wrote: > Hi Eric, > > On 21 July 2015 at 23:31, Eric Searcy <eric(a)linuxfoundation.org> wrote: > >> On 7/21/15 9:39 AM, Aurelien Bompard wrote: >> > Hey devs! >> > >> > I'm hitting a problem when migrating from 2.1 to 3, but I think it's a >> > generic issue (or rather a missing feature) and I'd like to fix it. >> > >> > If I try to associate an existing address to an existing user (without >> > this address obviously), the Mailman API will reply with a 400 error >> > "User already exists". Postorius currently does not detect that and >> > just errors out with a 500 page (but that's not the issue here). >> > >> > I think I need to merge the two users, and that's currently not >> > supported. I believe it would be a useful feature, so I'd like to >> > discuss it here. IMHO, the only tricky part would be to merge the >> > preferences associated with each user, but if I know which of my two >> > users has priority, I can just resolve the conflicts that way. >> > Am I missing something? Do you agree it would be a useful feature to >> have? >> >> This is a generic issue for us too; not just migration-related. I also >> would support consolidation/merging of Postorius users to solve this. >> >> In fact, there are a few inter-related blockers for us: >> >> a) authenticating as the wrong email address + attempt to add subscribed >> email as alternate (especially happens with social-type logins where >> correcting the authenticated email is a pain) >> > > I don't get any error while adding an already existing user. However, > since > I don't have postfix setup, I can't verify the address from email. I can > however. > do it from the backend. > Sorry for this one. Yes it indeed raise a 400 error when adding an already subscribed address even if there is no User objects associated with it in core. This probably is a bug. Also, probably postorius should check this before sending the confirmation email. > > >> b) cannot social-authenticate as an email address which is an alternate, >> not the primary >> > > So, it is correct that you cannot social-authenticate to the same account > using an alternate address. I am thinking about how we can fix this. > However, this is not a very critical bug. Even if you login into two > seperate > Postorius accounts, the underlying MailmanUser is the same. You can > view your subscriptions from both the addresses in both the accounts. > Also, you can change subscriptions to other alternate addresses in both > the accounts. > > >> c) attempting to social-authenticate as an email address which exists >> but has no account ... at least I think this is what is described by >> https://bugs.launchpad.net/postorius/+bug/1227461 (no matching gitlab >> bug) >> > > I checked out this one it is no more a bug. I can login with person to an > address > that already exists and is not associated with any account. > > However, note that I am using the master branch from source code. > > -- > thanks, > Abhilash Raj > -- thanks, Abhilash Raj

1 0

Running Mailman 3.0 on Shared Hosting
by Jason Glazer 29 Jul '15

29 Jul '15

I have a shared hosting account (Dreamhost) that I have been running Mailman 2.1.17 for many years with 15 low volume lists (5 to 100 messages per month). I would like to switch to using Mailman 3.0 and stay using a shared hosting environment. Has anyone successfully used Mailman 3.0 in a shared hosting environment? Were there any performance issues or other problems? Thanks Jason

1 0

Re: [Mailman-Developers] Merging users
by Eric Searcy 29 Jul '15

29 Jul '15

On 7/21/15 9:39 AM, Aurelien Bompard wrote: > Hey devs! > > I'm hitting a problem when migrating from 2.1 to 3, but I think it's a > generic issue (or rather a missing feature) and I'd like to fix it. > > If I try to associate an existing address to an existing user (without > this address obviously), the Mailman API will reply with a 400 error > "User already exists". Postorius currently does not detect that and > just errors out with a 500 page (but that's not the issue here). > > I think I need to merge the two users, and that's currently not > supported. I believe it would be a useful feature, so I'd like to > discuss it here. IMHO, the only tricky part would be to merge the > preferences associated with each user, but if I know which of my two > users has priority, I can just resolve the conflicts that way. > Am I missing something? Do you agree it would be a useful feature to have? This is a generic issue for us too; not just migration-related. I also would support consolidation/merging of Postorius users to solve this. In fact, there are a few inter-related blockers for us: a) authenticating as the wrong email address + attempt to add subscribed email as alternate (especially happens with social-type logins where correcting the authenticated email is a pain) b) cannot social-authenticate as an email address which is an alternate, not the primary c) attempting to social-authenticate as an email address which exists but has no account ... at least I think this is what is described by https://bugs.launchpad.net/postorius/+bug/1227461 (no matching gitlab bug) Eric

2 1