Re: [Mailman-Users] Automated Subscription Bots Inundating List Owners With Subscription Requests
Gary, et al:
The Mailman lists at Oregon State University have been receiving excessive request for subscriptions since mid-October as well. Our list administrators were suspicious because often the names on the requests did not match the email addresses. Also, many lists that had been defunct for years were receiving requests, too.
I spent some time trying to figure out what the lists that were being hit had in common. Not all of the lists receiving requests were advertised on the listinfo page. Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve". So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.
My folks are beating down my door for a solution, too, and I can't think of a good one. We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use. I am going to advise my list admins to enable confirmation, which should discourage these attempts. It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
If anyone else has any bright ideas about this problem, I would love to hear it.
-Kirsten Petersen Network Services, Oregon State University http://oregonstate.edu/is/services/network-services net@oregonstate.edumailto:net@oregonstate.edu (7-HELP, option 2) itconsult@oregonstate.edumailto:itconsult@oregonstate.edu (7-4710)
Petersen, Kirsten J - NET wrote:
Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve". So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.
Most likely, they are hitting all your lists but not answering confirmation requests because the bots don't know how or the confirmation requests are going to invalid or spoofed addresses.
My folks are beating down my door for a solution, too, and I can't think of a good one. We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use. I am going to advise my list admins to enable confirmation, which should discourage these attempts.
It seems this is a solution.
It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
If anyone else has any bright ideas about this problem, I would love to hear it.
For some time, there has been a withlist script, discard_address.py, at http://www.msapiro.net/scripts/ (mirrored at http://fog.ccsf.cc.ca.us/~msapiro/scripts/) which would discard all subscription requests and help posts from a specific address. While this it probably not too useful here, I have just created a new discard_subs.py script available at the same place which will discard all held subscription requests older than N days (can be 0) for a list or all lists.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
[Post by list member from an unsubscribed address]
On Nov 18, 2012 4:07 AM, "Petersen, Kirsten J - NET" < Kirsten.Petersen@oregonstate.edu> wrote:
Gary, et al:
The Mailman lists at Oregon State University have been receiving
excessive request for subscriptions since mid-October as well. Our list administrators were suspicious because often the names on the requests did not match the email addresses. Also, many lists that had been defunct for years were receiving requests, too.
I spent some time trying to figure out what the lists that were being hit
had in common. Not all of the lists receiving requests were advertised on the listinfo page. Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve". So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.
My folks are beating down my door for a solution, too, and I can't think
of a good one. We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use. I am going to advise my list admins to enable confirmation, which should discourage these attempts. It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
At KDE we took the semi drastic measure of allowing the commencement of mailing list subscription by email only as the attackers use HTTP POST to perform their attacks.
If Mailman were to implement basic CSRF protection for all POST requests that would also slow the attackers down I suspect (as they would have to make a GET request first and parse it).
One thing I do know is that at least for us the attacks all appeared to be coming from Tor endpoints or open web proxies.
Regards, Ben
[Quoted footers removed by moderator]
Ben Cooksley wrote:
If Mailman were to implement basic CSRF protection for all POST requests that would also slow the attackers down I suspect (as they would have to make a GET request first and parse it).
I have implemented a simple version of what I think you requested in your post at http://mail.python.org/pipermail/mailman-users/2012-October/074287.html.
It is implemented by the attached patch against Mailman 2.1.15. The patch will apply to versions 2.1.12 and later with at most line number changes, For older versions, the hashing function Mailman.Utils.sha_new doesn't exist and will need to be changed in the patch to something else. Note that the patch only enables configuring the token for the listinfo subscribe form. To actually enable placement and checking of the token, one must assign a non-empty string value to SUBSCRIBE_FORM_SECRET in mm_cfg.py. I.e.,
SUBSCRIBE_FORM_SECRET = 'Some site specific string'
The actual token is a hex digest of a sha hash of this string plus the list's internal name plus the IP address of the caller.
A more secure token would include something more random such as the time of day, but would be a bit more cumbersome to implement - volunteers are welcome.
Let us know if this helps.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Ben Cooksley writes:
If Mailman were to implement basic CSRF protection for all POST requests that would also slow the attackers down I suspect (as they would have to make a GET request first and parse it).
It might slow a human down, but as soon as it becomes a feature of Mailman, the attackers will implement the necessary countermeasure (if it isn't already implemented because they use libcurl or so in their program!) Bandwidth? CPU? These guys have no such constraints.
One thing I do know is that at least for us the attacks all appeared to be coming from Tor endpoints or open web proxies.
Big surprise. Not to mention demonstrating that CSRF protection won't help, because you're dealing with real players, not junior high school students from a fishing village in western Japan.
The problem here is that you cannot authenticate users you don't already know. So CSRF just adds a "get a free token" step to the automated process. I'm sure all the major libraries already implement this, so unless the attackers are remarkably stupid, undoubtedly the needed code is immediately to hand.
One partial solution would be to allow OpenID logins to the website to use it to register subscriptions. Of course you probably can't trust Google or Yahoo accounts. ;-)
Mark Sapiro wrote at http://mail.python.org/pipermail/mailman-users/2012-November/074415.html:
I have implemented a simple version of what I think you requested in your post at http://mail.python.org/pipermail/mailman-users/2012-October/074287.html.
It is implemented by the attached patch against Mailman 2.1.15.
I have augmented that patch with a timestamp and it now also checks that the hash is no older than mm_cfg.FORM_LIFETIME. See https://launchpad.net/bugs/1082746 and http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1371 for a bug report and the patch which will be released with Mailman 2.1.16.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
- Mark Sapiro mark@msapiro.net:
I have augmented that patch with a timestamp and it now also checks that the hash is no older than mm_cfg.FORM_LIFETIME. See https://launchpad.net/bugs/1082746 and http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1371 for a bug report and the patch which will be released with Mailman 2.1.16.
Excellent!!!
Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebrandt@charite.de Campus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155
On Nov 24, 2012, at 4:11 PM, Mark Sapiro mark@msapiro.net wrote:
Mark Sapiro wrote at http://mail.python.org/pipermail/mailman-users/2012-November/074415.html:
I have implemented a simple version of what I think you requested in your post at http://mail.python.org/pipermail/mailman-users/2012-October/074287.html.
It is implemented by the attached patch against Mailman 2.1.15.
I have augmented that patch with a timestamp and it now also checks that the hash is no older than mm_cfg.FORM_LIFETIME. See https://launchpad.net/bugs/1082746 and http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1371 for a bug report and the patch which will be released with Mailman 2.1.16.
I would like to apply this same patch to a 2.1.14 installation.
I downloaded the three modified files from this patch, and diffed them against 2.1.14 files. It looks like this patch will mostly apply to 2.1.14, but I'm not sure about the differences relating to comparing passwords, and the use of "strip." See 128,129c146,147 in the patch below for the what I am asking about.
Here is my diff:
1c1 < # Copyright (C) 1998-2009 by the Free Software Foundation, Inc.
# Copyright (C) 1998-2012 by the Free Software Foundation, Inc. 22a23 import time 112c113 < email = cgidata.getvalue('email', '')
email = cgidata.getvalue('email', '').strip()
122a124,140
# Are we checking the hidden data? if mm_cfg.SUBSCRIBE_FORM_SECRET: now = int(time.time()) try: ftime, fhash = cgidata.getvalue('sub_form_token', '').split(':') then = int(ftime) except ValueError: ftime = fhash = '' then = now token = Utils.sha_new(mm_cfg.SUBSCRIBE_FORM_SECRET + ftime + mlist.internal_name() + remote).hexdigest() if now - then > mm_cfg.FORM_LIFETIME: results.append(_('The form is too old. Please GET it again.')) if token != fhash: results.append(_('You must GET the form before submitting it.'))
128,129c146,147 < password = cgidata.getvalue('pw') < confirmed = cgidata.getvalue('pw-conf')
password = cgidata.getvalue('pw', '').strip() confirmed = cgidata.getvalue('pw-conf', '').strip()
131c149 < if password is None and confirmed is None:
if not password and not confirmed:
133c151 < elif password is None or confirmed is None:
elif not password or not confirmed:
On 12/9/2012 8:14 AM, Ivan Fetch wrote:
I downloaded the three modified files from this patch, and diffed them against 2.1.14 files. It looks like this patch will mostly apply to 2.1.14, but I'm not sure about the differences relating to comparing passwords, and the use of "strip." See 128,129c146,147 in the patch below for the what I am asking about.
Here is my diff:
[...]
112c113 < email = cgidata.getvalue('email', '')
email = cgidata.getvalue('email', '').strip()
The above is the fix for https://bugs.launchpad.net/mailman/+bug/745432. It would be better to include the .strip() as it fixes the bug.
[...]
128,129c146,147 < password = cgidata.getvalue('pw') < confirmed = cgidata.getvalue('pw-conf')
password = cgidata.getvalue('pw', '').strip() confirmed = cgidata.getvalue('pw-conf', '').strip()131c149 < if password is None and confirmed is None:
if not password and not confirmed:133c151 < elif password is None or confirmed is None:
elif not password or not confirmed:
The above is all part of the fix for https://bugs.launchpad.net/mailman/+bug/778088. The full fix is at http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1303.
It would be better to include the .strip() as that fixes the bug for the subscribe form. The other changes at 131 and 133 are important if you make the changes at 128,129c146,147 because the change from cgidata.getvalue('..') to cgidata.getvalue('..', '') returns the null string rather than None if the value is missing.
In other words, it is safe to install the patched 2.1.15 module in 2.1.14. The changes will fix bugs and not break anything.
Or you can apply just the patch to 2.1.14 and continue to live with the bugs.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (6)
-
Ben Cooksley -
Ivan Fetch -
Mark Sapiro -
Petersen, Kirsten J - NET -
Ralf Hildebrandt -
Stephen J. Turnbull