I was just hit with a subscription flood, along the lines of https://mail.python.org/pipermail/mailman-users/2014-May/076880.html https://bugs.launchpad.net/mailman/+bug/1082746
I've mitigated the current attack, but it's happened before and will happen again. I'm already using SUBSCRIBE_FORM_SECRET. I also saw Mark's patch in the thread above to disable subscriptions for a particular list, which is helpful.
Still, trying to look ahead, I have two further questions:
- The above mailman-users thread refers to using fail2ban. This sounds
sensible. Does anyone have a a working fail2ban filter they can share for this? I'd rather not write one from scratch; my previous attempts at doing so have been rather uneven. (Not the easiest thing to search for, since fail2ban itself uses mailman for its mailing lists, it seems. I couldn't find anything, at any rate.)
- At least in my cases, the floods try to subscribe the same address
over and over (and over and ...). It occurs to me that mailman could silently discard a request to subscribe an address foo@bar.com if foo@bar.com already has a pending subscription -- that is, not sending out the confirmation request. Would this be doable? Mark, anyone?
Although I realize that has downsides, for myself at least I'd prefer to minimize the backscatter pain for the random targeted addresses. Real people who are failing to subscribe can write the owner.
Thanks in advance, Karl
On 07/28/21 15:24, Karl Berry wrote:
- At least in my cases, the floods try to subscribe the same address
over and over (and over and ...). It occurs to me that mailman could silently discard a request to subscribe an address foo@bar.com if foo@bar.com already has a pending subscription -- that is, not sending out the confirmation request. Would this be doable? Mark, anyone?
You can probably do this with a procmail filter before anything hits mailman itself. (I filter spam this way.) I have not worked out the details. But the procmail recipe would run a script that would extract the email address from the message, call it "Fromaddress", and then
grep Fromaddress /var/log/mailman/subscribe | grep pending
and then if that is empty, pass the subscribe message on to the usual place (which, for me is
|/etc/smrsh/mailman join [name of list]
(This is the line in procmailrc that does it.)
Otherwise send the message to spam or /dev/null
Sorry about not working out the details, but I thought it might be better to say something rather than nothing.
Jon
Jonathan Baron, Professor of Psychology, University of Pennsylvania Home page: https://www.sas.upenn.edu/~baron Associate webmaster: sjdm.org
On 7/28/21 4:24 PM, Karl Berry wrote:
- The above mailman-users thread refers to using fail2ban. This sounds
sensible. Does anyone have a a working fail2ban filter they can share for this?
I have it setup, but it's not very sophisticated ...
failregex = .*/<HOST>\s+-\s+-\s+[.*]\s+"POST\s+/mailman/subscribe
It's just looking for repeated subscribe attempts.
david
Karl Berry writes:
I'm surprised Mark hasn't chimed in, maybe he's out on a boat catching salmon. Don't know when he'll be back, so here's what little I can say.
- The above mailman-users thread refers to using fail2ban.
The set in https://github.com/fail2ban/fail2ban/tree/master/config/filters.d looks quite comprehensive but there's nothing there that looks like it's to protect mailing lists. I hope somebody already has one, but I don't, and can't really suggest what might be similar that's already in the fail2ban distribution.
- At least in my cases, the floods try to subscribe the same address
over and over (and over and ...). It occurs to me that mailman could silently discard a request to subscribe an address foo@bar.com if foo@bar.com already has a pending subscription -- that is, not sending out the confirmation request. Would this be doable? Mark, anyone?
I'm pretty sure Mailman 3 has this although it may be recent. I doubt Mark would be willing to add it if it's not already in Mailman 2, although it's the kind of think I would think he would have back ported. Do you have the most recent version of Mailman?
If it's not in the most recent version of Mailman 2, or you don't want to upgrade for some reason, it's certainly doable as a patch, but how to implement will depend on how fast the requests are coming. I'm pretty sure that Mailman does not have a database of pending subscriptions that's efficient for this purpose. Your sources of information would be the log files (which presumably get rotated, may not have successful subscriptions, and so can't be 100% depended on) and the pickled subscription objects (.pck files) in the queue. The .pcks have a name that matches the one-time key so can easily be identified from a confirmation request, but the information needed to identify that this is a duplicated subscription is inside the object, which would need to be unpickled. This is not a terribly time-intensive operation, but could take a while if the evil requests come in frequently enough to build up a long queue, and in this design you'd be repeating this for every request.
If you don't mind missing one occasionally (ie, across log rotations) and the log files aren't humongous, you could just read the whole log every time, which would be much faster than working with the pickles, and look for matches on the subscription address.
The main issue I could see with either method is that this will slow down subscription processing substantially if the queue is long, which could happen with a "shotgun" of subscription attempts with different addresses. Keeping the database of pending subscriptions in memory would help with that, but you'd need to reread the .pcks on restarts.
I'm not really sure where this code would go. Probably in Mailman/ListAdmin.py, although it might be called from MailingList.py.
Steve
On 7/28/21 2:24 PM, Karl Berry wrote:
I've mitigated the current attack, but it's happened before and will happen again. I'm already using SUBSCRIBE_FORM_SECRET. I also saw Mark's patch in the thread above to disable subscriptions for a particular list, which is helpful.
Beginning with Mailman 2.1.26, there is the ability to add Google reCAPTCHA to the subscribe form, and in 2.1.30, there is the ability to add text based captchas (aka textchas). You can use either or both in combination.
Note however that experience on mail.python.org where we have both SUBSCRIBE_FORM_SECRET and Google reCAPTCHA (but not textcha) enabled is that we have still seen successful apparently robotic subscribe attacks across multiple lists (but not recently).
- At least in my cases, the floods try to subscribe the same address
over and over (and over and ...). It occurs to me that mailman could silently discard a request to subscribe an address foo@bar.com if foo@bar.com already has a pending subscription -- that is, not sending out the confirmation request. Would this be doable? Mark, anyone?
As Steve notes, this is done in Mailman 3, but not in Mailman 2.1. I will consider adding it to 2.1.
Also note that while it won't stop an initial attack, adding a pattern to a list's ban_list (or starting with 2.1.21, the GLOBAL_BAN_LIST) can help stem an ongoing attack.
On 7/29/21 11:29 AM, Mark Sapiro wrote:
On 7/28/21 2:24 PM, Karl Berry wrote:
- At least in my cases, the floods try to subscribe the same address
over and over (and over and ...). It occurs to me that mailman could silently discard a request to subscribe an address foo@bar.com if foo@bar.com already has a pending subscription -- that is, not sending out the confirmation request. Would this be doable? Mark, anyone?
As Steve notes, this is done in Mailman 3, but not in Mailman 2.1. I will consider adding it to 2.1.
Actually, it is in Mailman 2.1.30. Set
REFUSE_SECOND_PENDING = Yes
in mm_cfg.py to enable it.
Thanks to everyone for the great replies.
davidg> I have it setup, but it's not very sophisticated ...
failregex = .*\/<HOST>\s+-\s+-\s+\[.*\]\s+"POST\s+\/mailman\/subscribe
It's just looking for repeated subscribe attempts.
Thanks David! What are you using for maxretry, findtime, bantime, etc., in jail.local (or whatever)? I find it's often as hard to figure out good values for those as to write the regexps ...
marks> Actually, it is in Mailman 2.1.30. Set
REFUSE_SECOND_PENDING = Yes
in mm_cfg.py to enable it.
Thanks Mark! I've been using the mailman from my distro, which is (sigh) older. I'll look into going back to installing mailman from scratch, as I've done before.
jonb> You can probably do this with a procmail filter before anything hits
I'm not sure. My impression is the bad guys are hitting the subscribe cgi directly, not sending mail requests. But procmail could work for mail floods, for sure.
Sorry about not working out the details, but I thought it might be
better to say something rather than nothing.
Definitely :).
Thanks again, Karl
On 7/29/21 3:05 PM, Karl Berry wrote:
Thanks Mark! I've been using the mailman from my distro, which is (sigh) older. I'll look into going back to installing mailman from scratch, as I've done before.
We have information about upgrading a Debian/Ubuntu package from source at https://wiki.list.org/x/17891606 and upgrading a RHEL/CentOS package from source at https://wiki.list.org/x/17892071
Also, see https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1829 for the implementation of REFUSE_SECOND_PENDING and https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1851 for its extension to unsubscribes.
Karl Berry wrote:
The above mailman-users thread refers to using fail2ban. This sounds sensible. Does anyone have a a working fail2ban filter they can share for this?
Just on more thing on this. In the case of the attacks I've seen on mail.python.org, fail2ban is unlikely to help much as the attacks come from botnets with many different originating IPs and it's not clear that blocking individual IPs will have much effect. fail2ban could be effective against attacks that come from a single or only a few IPs, but that is not the pattern I've seen. YMMV
participants (5)
-
David Gibbs -
Jon Baron -
Karl Berry -
Mark Sapiro -
Stephen J. Turnbull