Re: [Mailman-Users] Automated Subscription Bots Inundating List Owners With Subscription Requests
![](https://secure.gravatar.com/avatar/80db113027cc1a0825b93fb879832080.jpg?s=120&d=mm&r=g)
Gary, et al:
The Mailman lists at Oregon State University have been receiving excessive request for subscriptions since mid-October as well. Our list administrators were suspicious because often the names on the requests did not match the email addresses. Also, many lists that had been defunct for years were receiving requests, too.
I spent some time trying to figure out what the lists that were being hit had in common. Not all of the lists receiving requests were advertised on the listinfo page. Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve". So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.
My folks are beating down my door for a solution, too, and I can't think of a good one. We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use. I am going to advise my list admins to enable confirmation, which should discourage these attempts. It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
If anyone else has any bright ideas about this problem, I would love to hear it.
-Kirsten Petersen Network Services, Oregon State University http://oregonstate.edu/is/services/network-services net@oregonstate.edu<mailto:net@oregonstate.edu> (7-HELP, option 2) itconsult@oregonstate.edu<mailto:itconsult@oregonstate.edu> (7-4710)
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Petersen, Kirsten J - NET wrote:
Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve". So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.
Most likely, they are hitting all your lists but not answering confirmation requests because the bots don't know how or the confirmation requests are going to invalid or spoofed addresses.
My folks are beating down my door for a solution, too, and I can't think of a good one. We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use. I am going to advise my list admins to enable confirmation, which should discourage these attempts.
It seems this is a solution.
It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
If anyone else has any bright ideas about this problem, I would love to hear it.
For some time, there has been a withlist script, discard_address.py, at <http://www.msapiro.net/scripts/> (mirrored at <http://fog.ccsf.cc.ca.us/~msapiro/scripts/>) which would discard all subscription requests and help posts from a specific address. While this it probably not too useful here, I have just created a new discard_subs.py script available at the same place which will discard all held subscription requests older than N days (can be 0) for a list or all lists.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/0d1f983fb6dde8d1d7f25af5ad738799.jpg?s=120&d=mm&r=g)
[Post by list member from an unsubscribed address]
On Nov 18, 2012 4:07 AM, "Petersen, Kirsten J - NET" < Kirsten.Petersen@oregonstate.edu> wrote:
Gary, et al:
The Mailman lists at Oregon State University have been receiving
excessive request for subscriptions since mid-October as well. Our list administrators were suspicious because often the names on the requests did not match the email addresses. Also, many lists that had been defunct for years were receiving requests, too.
I spent some time trying to figure out what the lists that were being hit
had in common. Not all of the lists receiving requests were advertised on the listinfo page. Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve". So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.
My folks are beating down my door for a solution, too, and I can't think
of a good one. We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use. I am going to advise my list admins to enable confirmation, which should discourage these attempts. It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
At KDE we took the semi drastic measure of allowing the commencement of mailing list subscription by email only as the attackers use HTTP POST to perform their attacks.
If Mailman were to implement basic CSRF protection for all POST requests that would also slow the attackers down I suspect (as they would have to make a GET request first and parse it).
One thing I do know is that at least for us the attacks all appeared to be coming from Tor endpoints or open web proxies.
Regards, Ben
[Quoted footers removed by moderator]
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Ben Cooksley wrote:
I have implemented a simple version of what I think you requested in your post at <http://mail.python.org/pipermail/mailman-users/2012-October/074287.html>.
It is implemented by the attached patch against Mailman 2.1.15. The patch will apply to versions 2.1.12 and later with at most line number changes, For older versions, the hashing function Mailman.Utils.sha_new doesn't exist and will need to be changed in the patch to something else. Note that the patch only enables configuring the token for the listinfo subscribe form. To actually enable placement and checking of the token, one must assign a non-empty string value to SUBSCRIBE_FORM_SECRET in mm_cfg.py. I.e.,
SUBSCRIBE_FORM_SECRET = 'Some site specific string'
The actual token is a hex digest of a sha hash of this string plus the list's internal name plus the IP address of the caller.
A more secure token would include something more random such as the time of day, but would be a bit more cumbersome to implement - volunteers are welcome.
Let us know if this helps.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Mark Sapiro wrote at <http://mail.python.org/pipermail/mailman-users/2012-November/074415.html>:
I have augmented that patch with a timestamp and it now also checks that the hash is no older than mm_cfg.FORM_LIFETIME. See <https://launchpad.net/bugs/1082746> and <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1371> for a bug report and the patch which will be released with Mailman 2.1.16.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/267565c6ab7816fe29beedf9a9cbcd44.jpg?s=120&d=mm&r=g)
- Mark Sapiro <mark@msapiro.net>:
Excellent!!!
Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebrandt@charite.de Campus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155
![](https://secure.gravatar.com/avatar/67d638414621308cb1b9706400582785.jpg?s=120&d=mm&r=g)
On Nov 24, 2012, at 4:11 PM, Mark Sapiro <mark@msapiro.net> wrote:
I would like to apply this same patch to a 2.1.14 installation.
I downloaded the three modified files from this patch, and diffed them against 2.1.14 files. It looks like this patch will mostly apply to 2.1.14, but I'm not sure about the differences relating to comparing passwords, and the use of "strip." See 128,129c146,147 in the patch below for the what I am asking about.
Here is my diff:
1c1 < # Copyright (C) 1998-2009 by the Free Software Foundation, Inc.
email = cgidata.getvalue('email', '').strip()
122a124,140
128,129c146,147 < password = cgidata.getvalue('pw') < confirmed = cgidata.getvalue('pw-conf')
password = cgidata.getvalue('pw', '').strip() confirmed = cgidata.getvalue('pw-conf', '').strip()
131c149 < if password is None and confirmed is None:
if not password and not confirmed:
133c151 < elif password is None or confirmed is None:
elif not password or not confirmed:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 12/9/2012 8:14 AM, Ivan Fetch wrote:
I downloaded the three modified files from this patch, and diffed them against 2.1.14 files. It looks like this patch will mostly apply to 2.1.14, but I'm not sure about the differences relating to comparing passwords, and the use of "strip." See 128,129c146,147 in the patch below for the what I am asking about.
Here is my diff:
[...]
The above is the fix for <https://bugs.launchpad.net/mailman/+bug/745432>. It would be better to include the .strip() as it fixes the bug.
[...]
The above is all part of the fix for <https://bugs.launchpad.net/mailman/+bug/778088>. The full fix is at <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1303>.
It would be better to include the .strip() as that fixes the bug for the subscribe form. The other changes at 131 and 133 are important if you make the changes at 128,129c146,147 because the change from cgidata.getvalue('..') to cgidata.getvalue('..', '') returns the null string rather than None if the value is missing.
In other words, it is safe to install the patched 2.1.15 module in 2.1.14. The changes will fix bugs and not break anything.
Or you can apply just the patch to 2.1.14 and continue to live with the bugs.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/334b870d5b26878a79b2dc4cfcc500bc.jpg?s=120&d=mm&r=g)
Ben Cooksley writes:
It might slow a human down, but as soon as it becomes a feature of Mailman, the attackers will implement the necessary countermeasure (if it isn't already implemented because they use libcurl or so in their program!) Bandwidth? CPU? These guys have no such constraints.
One thing I do know is that at least for us the attacks all appeared to be coming from Tor endpoints or open web proxies.
Big surprise. Not to mention demonstrating that CSRF protection won't help, because you're dealing with real players, not junior high school students from a fishing village in western Japan.
The problem here is that you cannot authenticate users you don't already know. So CSRF just adds a "get a free token" step to the automated process. I'm sure all the major libraries already implement this, so unless the attackers are remarkably stupid, undoubtedly the needed code is immediately to hand.
One partial solution would be to allow OpenID logins to the website to use it to register subscriptions. Of course you probably can't trust Google or Yahoo accounts. ;-)
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Petersen, Kirsten J - NET wrote:
Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve". So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.
Most likely, they are hitting all your lists but not answering confirmation requests because the bots don't know how or the confirmation requests are going to invalid or spoofed addresses.
My folks are beating down my door for a solution, too, and I can't think of a good one. We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use. I am going to advise my list admins to enable confirmation, which should discourage these attempts.
It seems this is a solution.
It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
If anyone else has any bright ideas about this problem, I would love to hear it.
For some time, there has been a withlist script, discard_address.py, at <http://www.msapiro.net/scripts/> (mirrored at <http://fog.ccsf.cc.ca.us/~msapiro/scripts/>) which would discard all subscription requests and help posts from a specific address. While this it probably not too useful here, I have just created a new discard_subs.py script available at the same place which will discard all held subscription requests older than N days (can be 0) for a list or all lists.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/0d1f983fb6dde8d1d7f25af5ad738799.jpg?s=120&d=mm&r=g)
[Post by list member from an unsubscribed address]
On Nov 18, 2012 4:07 AM, "Petersen, Kirsten J - NET" < Kirsten.Petersen@oregonstate.edu> wrote:
Gary, et al:
The Mailman lists at Oregon State University have been receiving
excessive request for subscriptions since mid-October as well. Our list administrators were suspicious because often the names on the requests did not match the email addresses. Also, many lists that had been defunct for years were receiving requests, too.
I spent some time trying to figure out what the lists that were being hit
had in common. Not all of the lists receiving requests were advertised on the listinfo page. Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve". So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.
My folks are beating down my door for a solution, too, and I can't think
of a good one. We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use. I am going to advise my list admins to enable confirmation, which should discourage these attempts. It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
At KDE we took the semi drastic measure of allowing the commencement of mailing list subscription by email only as the attackers use HTTP POST to perform their attacks.
If Mailman were to implement basic CSRF protection for all POST requests that would also slow the attackers down I suspect (as they would have to make a GET request first and parse it).
One thing I do know is that at least for us the attacks all appeared to be coming from Tor endpoints or open web proxies.
Regards, Ben
[Quoted footers removed by moderator]
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Ben Cooksley wrote:
I have implemented a simple version of what I think you requested in your post at <http://mail.python.org/pipermail/mailman-users/2012-October/074287.html>.
It is implemented by the attached patch against Mailman 2.1.15. The patch will apply to versions 2.1.12 and later with at most line number changes, For older versions, the hashing function Mailman.Utils.sha_new doesn't exist and will need to be changed in the patch to something else. Note that the patch only enables configuring the token for the listinfo subscribe form. To actually enable placement and checking of the token, one must assign a non-empty string value to SUBSCRIBE_FORM_SECRET in mm_cfg.py. I.e.,
SUBSCRIBE_FORM_SECRET = 'Some site specific string'
The actual token is a hex digest of a sha hash of this string plus the list's internal name plus the IP address of the caller.
A more secure token would include something more random such as the time of day, but would be a bit more cumbersome to implement - volunteers are welcome.
Let us know if this helps.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Mark Sapiro wrote at <http://mail.python.org/pipermail/mailman-users/2012-November/074415.html>:
I have augmented that patch with a timestamp and it now also checks that the hash is no older than mm_cfg.FORM_LIFETIME. See <https://launchpad.net/bugs/1082746> and <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1371> for a bug report and the patch which will be released with Mailman 2.1.16.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/267565c6ab7816fe29beedf9a9cbcd44.jpg?s=120&d=mm&r=g)
- Mark Sapiro <mark@msapiro.net>:
Excellent!!!
Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebrandt@charite.de Campus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155
![](https://secure.gravatar.com/avatar/67d638414621308cb1b9706400582785.jpg?s=120&d=mm&r=g)
On Nov 24, 2012, at 4:11 PM, Mark Sapiro <mark@msapiro.net> wrote:
I would like to apply this same patch to a 2.1.14 installation.
I downloaded the three modified files from this patch, and diffed them against 2.1.14 files. It looks like this patch will mostly apply to 2.1.14, but I'm not sure about the differences relating to comparing passwords, and the use of "strip." See 128,129c146,147 in the patch below for the what I am asking about.
Here is my diff:
1c1 < # Copyright (C) 1998-2009 by the Free Software Foundation, Inc.
email = cgidata.getvalue('email', '').strip()
122a124,140
128,129c146,147 < password = cgidata.getvalue('pw') < confirmed = cgidata.getvalue('pw-conf')
password = cgidata.getvalue('pw', '').strip() confirmed = cgidata.getvalue('pw-conf', '').strip()
131c149 < if password is None and confirmed is None:
if not password and not confirmed:
133c151 < elif password is None or confirmed is None:
elif not password or not confirmed:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 12/9/2012 8:14 AM, Ivan Fetch wrote:
I downloaded the three modified files from this patch, and diffed them against 2.1.14 files. It looks like this patch will mostly apply to 2.1.14, but I'm not sure about the differences relating to comparing passwords, and the use of "strip." See 128,129c146,147 in the patch below for the what I am asking about.
Here is my diff:
[...]
The above is the fix for <https://bugs.launchpad.net/mailman/+bug/745432>. It would be better to include the .strip() as it fixes the bug.
[...]
The above is all part of the fix for <https://bugs.launchpad.net/mailman/+bug/778088>. The full fix is at <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1303>.
It would be better to include the .strip() as that fixes the bug for the subscribe form. The other changes at 131 and 133 are important if you make the changes at 128,129c146,147 because the change from cgidata.getvalue('..') to cgidata.getvalue('..', '') returns the null string rather than None if the value is missing.
In other words, it is safe to install the patched 2.1.15 module in 2.1.14. The changes will fix bugs and not break anything.
Or you can apply just the patch to 2.1.14 and continue to live with the bugs.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/334b870d5b26878a79b2dc4cfcc500bc.jpg?s=120&d=mm&r=g)
Ben Cooksley writes:
It might slow a human down, but as soon as it becomes a feature of Mailman, the attackers will implement the necessary countermeasure (if it isn't already implemented because they use libcurl or so in their program!) Bandwidth? CPU? These guys have no such constraints.
One thing I do know is that at least for us the attacks all appeared to be coming from Tor endpoints or open web proxies.
Big surprise. Not to mention demonstrating that CSRF protection won't help, because you're dealing with real players, not junior high school students from a fishing village in western Japan.
The problem here is that you cannot authenticate users you don't already know. So CSRF just adds a "get a free token" step to the automated process. I'm sure all the major libraries already implement this, so unless the attackers are remarkably stupid, undoubtedly the needed code is immediately to hand.
One partial solution would be to allow OpenID logins to the website to use it to register subscriptions. Of course you probably can't trust Google or Yahoo accounts. ;-)
participants (6)
-
Ben Cooksley
-
Ivan Fetch
-
Mark Sapiro
-
Petersen, Kirsten J - NET
-
Ralf Hildebrandt
-
Stephen J. Turnbull