
In the last few days we've seen several thousand bogus subscription requests for various lists we host, send through the web interface. They seem to mostly originate in China.
We see log entries such as /var/log/mailman/subscribe Jan 11 20:50:30 2016 (27666) grsi-users: pending hellocatboots+80339132@gmail.com 221.178.182.31 and in the webserver logs 221.178.182.31 - - [10/Jan/2016:03:27:18 -0800] "POST /mailman/subscribe/grsi-users HTTP/1.1" 200
I'm not sure what the point is - a DoS attack on a few users, perhaps. I see that gmail gives you infinite aliases, so that hellocatboots+80339132 is the same as hellocatboots+96529823@gmail.com
Since most of these seem to originate with one netblock where we have, I believe, no legitimate users, I've added a Deny rule in httpd.conf.
I was wondering if other admins had seen this, and if there was a better way to control it than blocking an ip range.
Apart from all the variants of hellocatboots, we've seen a lot of posts for one unique user at kezukaya.com. The subscribe log shows hundreds of pending requests, from which I infer that mailman has no mechanism to track the fact that it already sent a "please confirm" message (we have mailman-2.1.18 on Centos 5).
-- Andrew Daviel, TRIUMF, Canada

From the "NEWS" file:
- There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET which will put
a dynamically generated, hidden hash in the listinfo subscribe form and
check it upon submission. Setting this will prevent automated processes
(bots) from successfully POSTing web subscribes without first retrieving
and parsing the form from the listinfo page. The form must also be
submitted no later than FORM_LIFETIME nor no earlier than
SUBSCRIBE_FORM_MIN_TIME after retrieval. Note that enabling this will
break any static subscribe forms on your site. See the description in
Defaults.py for more info. (LP: #1082746)
-----Original Message----- From: Mailman-Users [mailto:mailman-users- bounces+rosenbaumlm=ornl.gov@python.org] On Behalf Of Andrew Daviel Sent: Tuesday, January 12, 2016 4:18 AM To: mailman-users@python.org Subject: [Mailman-Users] Handling bogus subscribe requests
In the last few days we've seen several thousand bogus subscription requests for various lists we host, send through the web interface. They seem to mostly originate in China.
We see log entries such as /var/log/mailman/subscribe Jan 11 20:50:30 2016 (27666) grsi-users: pending hellocatboots+80339132@gmail.com 221.178.182.31 and in the webserver logs 221.178.182.31 - - [10/Jan/2016:03:27:18 -0800] "POST /mailman/subscribe/grsi-users HTTP/1.1" 200
I'm not sure what the point is - a DoS attack on a few users, perhaps. I see that gmail gives you infinite aliases, so that hellocatboots+80339132 is the same as hellocatboots+96529823@gmail.com
Since most of these seem to originate with one netblock where we have, I believe, no legitimate users, I've added a Deny rule in httpd.conf.
I was wondering if other admins had seen this, and if there was a better way to control it than blocking an ip range.
Apart from all the variants of hellocatboots, we've seen a lot of posts for one unique user at kezukaya.com. The subscribe log shows hundreds of pending requests, from which I infer that mailman has no mechanism to track the fact that it already sent a "please confirm" message (we have mailman-2.1.18 on Centos 5).
-- Andrew Daviel, TRIUMF, Canada
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman- users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman- users/rosenbaumlm%40ornl.gov

On 01/12/2016 08:18 AM, Rosenbaum, Larry M. wrote:
From the "NEWS" file:
- There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET ...
This is only partially effective against this attack.
On the @python.org lists we see this attack come and go and even with SUBSCRIBE_FORM_MIN_TIME = 10 seconds, enough of the bot subscribes delay long enough to get through.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On Tue, 12 Jan 2016, Mark Sapiro wrote:
On 01/12/2016 08:18 AM, Rosenbaum, Larry M. wrote:
From the "NEWS" file:
- There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET ...
This is only partially effective against this attack.
Thanks for the info.
Typical of me, I kept looking for a workaround after posting and didn't see this straight away. I will look into SUBSCRIBE_FORM_SECRET.
Meanwhile, I found the bot was evading the address block ban by using other hosts, and have tried adding a simple CAPTCHA based on Apache anonymous authentication. If a user tries to access a mailman script from offsite, they get a 401 rejection and a prompt to login with a simple username (which changes every hour)
-- Andrew Daviel, TRIUMF, Canada Tel. +1 (604) 222-7376 (Pacific Time) Network Security Manager

On 01/12/2016 01:18 AM, Andrew Daviel wrote:
In the last few days we've seen several thousand bogus subscription requests for various lists we host, send through the web interface. They seem to mostly originate in China.
We see log entries such as /var/log/mailman/subscribe Jan 11 20:50:30 2016 (27666) grsi-users: pending hellocatboots+80339132@gmail.com 221.178.182.31 and in the webserver logs 221.178.182.31 - - [10/Jan/2016:03:27:18 -0800] "POST /mailman/subscribe/grsi-users HTTP/1.1" 200
I'm not sure what the point is - a DoS attack on a few users, perhaps. I see that gmail gives you infinite aliases, so that hellocatboots+80339132 is the same as hellocatboots+96529823@gmail.com
There are threads on this in the archives of this list. See threads containing the posts <https://mail.python.org/pipermail/mailman-users/2015-September/079829.html> and <https://mail.python.org/pipermail/mailman-users/2015-September/079844.html> and perhaps the thread starting at https://mail.python.org/pipermail/mailman-users/2015-September/079855.html.
For the @python.org lists, we use the regexp '^.*\+.*\d{3,}@' in the newly implemented, not yet released GLOBAL_BAN_LIST to ban all addresses with a '+' followed by anything followed by at least 3 digits up to the '@'.
Read <https://mail.python.org/pipermail/mailman-users/2015-September/079844.html> for more on that.
It's been effective so far.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 1/12/2016 11:54 AM, Mark Sapiro wrote:
There are threads on this in the archives of this list. See threads containing the posts <https://mail.python.org/pipermail/mailman-users/2015-September/079829.html>
I can confirm that the technique used in this post work great.
david
-- IBM i on Power Systems: For when you can't afford to be out of business!
I'm riding 100 miles (a full century) in the American Diabetes Association's Tour de Cure to raise money for diabetes research, education, advocacy, and awareness. You can make a tax deductible donation to my ride by visiting http://gmanesig.diabetessucks.net. My goal is $6000 but any amount is appreciated.
See where I get my donations from ... visit http://gmanesig.diabetessucks.net/map for an interactive map (it's a geeky thing).

On Thu, 14 Jan 2016 08:55:21 -0600 "Gibbs, David" <david@midrange.com> wrote:
On 1/12/2016 11:54 AM, Mark Sapiro wrote:
There are threads on this in the archives of this list. See threads containing the posts <https://mail.python.org/pipermail/mailman-users/2015-September/079829.html>
I can confirm that the technique used in this post work great.
I have direct evidence that the asshats are now using "+" strings after the main address that are not strictly numeric. They seem to have responded to the simple ways of stopping them.
What's a good technique at this point to slow them down besides regexps? I'm stuck on 2.1.18 at the moment as a debian user.
Perry
Perry E. Metzger perry@piermont.com

On Sat, 16 Jan 2016 19:02:58 -0500 "Perry E. Metzger" <perry@piermont.com> wrote:
On Thu, 14 Jan 2016 08:55:21 -0600 "Gibbs, David" <david@midrange.com> wrote:
On 1/12/2016 11:54 AM, Mark Sapiro wrote:
There are threads on this in the archives of this list. See threads containing the posts <https://mail.python.org/pipermail/mailman-users/2015-September/079829.html>
I can confirm that the technique used in this post work great.
I have direct evidence that the asshats are now using "+" strings after the main address that are not strictly numeric. They seem to have responded to the simple ways of stopping them.
What's a good technique at this point to slow them down besides regexps? I'm stuck on 2.1.18 at the moment as a debian user.
Oh, and by the way, the documentation for SUBSCRIBE_FORM_SECRET (such as it is) does not mention that it needs to be set to a string. I only figured that out when setting it to True and 1 both failed spectacularly and reading the traceback.
Perry
Perry E. Metzger perry@piermont.com

On 01/16/2016 04:51 PM, Perry E. Metzger wrote:
Oh, and by the way, the documentation for SUBSCRIBE_FORM_SECRET (such as it is) does not mention that it needs to be set to a string. I only figured that out when setting it to True and 1 both failed spectacularly and reading the traceback.
It seems clear to me. The paragraph in Defaults.py begins
# If the following is set to a non-empty string, this string in combination # with the time, list name and the IP address of the requestor is used to # create a hidden hash as part of the subscribe form on the listinfo page.
Perhaps you were only looking at the NEWS file?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 01/16/2016 04:02 PM, Perry E. Metzger wrote:
I have direct evidence that the asshats are now using "+" strings after the main address that are not strictly numeric. They seem to have responded to the simple ways of stopping them.
I haven't seen any like that yet. The regexp I use is ^.*\+.*\d{3,}@ which will block anything between '+' and '@' as long as it ends with 3 digits.
Please provide some examples. If there is any discernable pattern, it might be blockable without impacting real subscribers.
I haven't actually seen kemo.mart or kezukaya + digits @gmail for 5 days. The last five days have been all hellocatboots+<eight digits>@gmail.com, although a few days ago a bunch of 'ichiba at kezukaya.com' actually got through. (I'm now banning that one too)
What's a good technique at this point to slow them down besides regexps? I'm stuck on 2.1.18 at the moment as a debian user.
Based on the one above getting through, I wrote the script at <https://www.msapiro.net/scripts/erase> (mirrored at <http://fog.ccsf.edu/~msapiro/scripts/erase>) to 'erase' an address from all lists. I.e. for every list, if the address is a member, it is removed. If there are any posts or (un)subscription requests from the address awaiting moderator action, they are removed and optionally any subscripions for the address awaiting user confirmation are discarded too.
As far as I can tell, even if the address isn't banned, unless you allow open subscribe which is a bad idea for many reasons, the subscription will require user confirmation, moderator approval, or both.
Those that require confirmation (or both) will never be confirmed because the bots don't get the confirmation request email, but they do create back-scatter (the confirmation request email) to the address. Those that require moderator approval will presumably be discarded, but are a moderator burden. The script can deal with all that at once.
Other possibilities are disabling web subscribe all together or installing some kind of captcha (ugh) in the page. Experience shows that SUBSCRIBE_FORM_SECRET doesn't stop them unless perhaps SUBSCRIBE_FORM_MIN_TIME is set long enough that it becomes a problem for real subscribers.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 01/16/2016 04:52 PM, Mark Sapiro wrote:
Based on the one above getting through, I wrote the script at <https://www.msapiro.net/scripts/erase> (mirrored at <http://fog.ccsf.edu/~msapiro/scripts/erase>) to 'erase' an address from all lists.
Note, this script uses the Python argparse module to parse the command line. As such it requires Python 2.7. Converting it to optparse is left as an exercise for the reader.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On Sat, 16 Jan 2016 16:52:29 -0800 Mark Sapiro <mark@msapiro.net> wrote:
On 01/16/2016 04:02 PM, Perry E. Metzger wrote:
I have direct evidence that the asshats are now using "+" strings after the main address that are not strictly numeric. They seem to have responded to the simple ways of stopping them.
I haven't seen any like that yet. The regexp I use is ^.*\+.*\d{3,}@ which will block anything between '+' and '@' as long as it ends with 3 digits.
Please provide some examples. If there is any discernable pattern, it might be blockable without impacting real subscribers.
I don't have a lot of examples (haven't been saving them as I nuke stuff out of the postfix queue) but I just nuked one aimed at what I could characterize as user\+[a-z]+[0-9]@gmail.com
In each case, the instances in the queue had this or equivalent back from google:
(host gmail-smtp-in.l.google.com[74.125.29.26] said: 450-4.2.1 The user you are trying to contact is receiving mail at a rate that 450-4.2.1 prevents additional messages from being delivered. Please resend your 450-4.2.1 message at a later time. If the user is able to receive mail at that 450-4.2.1 time, your message will be delivered. For more information, please 450-4.2.1 visit 450 4.2.1 https://support.google.com/mail/answer/6592 75si22222236qgm.43 - gsmtp (in reply to RCPT TO command))
I already had a regexp in to nuke everything aimed at a post-+ section with just digits. I'm reluctant to go further than that immediately, although I suspect trailing digits after alphabeticals are also unlikely to be real submailboxes.
Other possibilities are disabling web subscribe all together or installing some kind of captcha (ugh) in the page. Experience shows that SUBSCRIBE_FORM_SECRET doesn't stop them unless perhaps SUBSCRIBE_FORM_MIN_TIME is set long enough that it becomes a problem for real subscribers.
Would it be hard to add optional recaptcha support for the pages with forms in a future release? That would probably prevent most such games and it doesn't seem so bad.
Perry
Perry E. Metzger perry@piermont.com

On 01/16/2016 05:13 PM, Perry E. Metzger wrote:
On Sat, 16 Jan 2016 16:52:29 -0800 Mark Sapiro <mark@msapiro.net> wrote:
Please provide some examples. If there is any discernable pattern, it might be blockable without impacting real subscribers.
I don't have a lot of examples (haven't been saving them as I nuke stuff out of the postfix queue) but I just nuked one aimed at what I could characterize as user\+[a-z]+[0-9]@gmail.com
Is that just one digit? If so, that's a tough one to separate from legit ones, but in what I've seen the 'user' part doesn't vary all that much so you could focus on that.
I already had a regexp in to nuke everything aimed at a post-+ section with just digits. I'm reluctant to go further than that immediately, although I suspect trailing digits after alphabeticals are also unlikely to be real submailboxes.
I certainly agree that 3 or more and perhaps even 2 trailing digits is unlikely, but a single trailing digit is likely too agressive.
Just for info, there are a total of 60580 subscriptions with 41715 unique email addresses to lists @python.org. Of the 41745, 620 have a '+' in the address and of those, 24 have at least one digit immediately before the @. of the 25, 15 are subscriptions for the nabble.com archiving service and 2 are subscriptions for a googlegroups archiving service. Of the remaining 7, there are 2 +python3000, 1 +python-3000 and 1 +py3000, and the other 3 have only one digit including one +mm3.
So, in this installation at least, \+.*[0-9]@ is rare and most of even those are archivers with multiple digits or are probably motivated by the nature of the installation.
Would it be hard to add optional recaptcha support for the pages with forms in a future release? That would probably prevent most such games and it doesn't seem so bad.
I hate them. I'd really have to be convinced of the need.
In any case, you already have that ability. The only place I think you need it is the subscribe form on the listinfo page. It is a simple matter to install a sitewide custom listinfo template in (for example for english) $prefix/templates/site/en/listinfo.html or to make a list specific one through the web admin UI.
Since reCAPTCHA in particular requires registration and perhaps other steps outside of Mailman, I don't think it's too burdensome that the Mailman part requires making a custom template.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On Sat, 16 Jan 2016 18:27:00 -0800 Mark Sapiro <mark@msapiro.net> wrote:
Would it be hard to add optional recaptcha support for the pages with forms in a future release? That would probably prevent most such games and it doesn't seem so bad.
I hate them. I'd really have to be convinced of the need.
In any case, you already have that ability. The only place I think you need it is the subscribe form on the listinfo page. It is a simple matter to install a sitewide custom listinfo template in (for example for english) $prefix/templates/site/en/listinfo.html or to make a list specific one through the web admin UI.
Since reCAPTCHA in particular requires registration and perhaps other steps outside of Mailman, I don't think it's too burdensome that the Mailman part requires making a custom template.
Mostly it just requires registration. Doing a custom template is probably fine for someone like me who is able to deal with the technical steps involved but it might be too much of a burden for many users. If such code was contributed would you be willing to include it? You don't have to use it yourself of course if you find it overly vile.
Perry
Perry E. Metzger perry@piermont.com

Hi,
Just my $0.02 worth, but as a blind person, were I going to implement a CAPTCHA, I would probably write some sort of simple math problem system or something. Besides the benefit of not needing registration, this CAPTCHA can be solved by anybody who can solve simple problems. ReCaptcha has a visual challenge and an audio challenge. That's great for those of us who are blind. However, as far as I know, they offer no alternative for people who are deaf blind.
Jayson
On 1/17/2016 9:34 AM, Perry E. Metzger wrote:
Would it be hard to add optional recaptcha support for the pages with forms in a future release? That would probably prevent most such games and it doesn't seem so bad. I hate them. I'd really have to be convinced of the need.
In any case, you already have that ability. The only place I think you need it is the subscribe form on the listinfo page. It is a simple matter to install a sitewide custom listinfo template in (for example for english) $prefix/templates/site/en/listinfo.html or to make a list specific one through the web admin UI.
Since reCAPTCHA in particular requires registration and perhaps other steps outside of Mailman, I don't think it's too burdensome that the Mailman part requires making a custom template. Mostly it just requires registration. Doing a custom template is
On Sat, 16 Jan 2016 18:27:00 -0800 Mark Sapiro <mark@msapiro.net> wrote: probably fine for someone like me who is able to deal with the technical steps involved but it might be too much of a burden for many users. If such code was contributed would you be willing to include it? You don't have to use it yourself of course if you find it overly vile.
Perry

On 01/17/2016 06:34 AM, Perry E. Metzger wrote:
Mostly it just requires registration. Doing a custom template is probably fine for someone like me who is able to deal with the technical steps involved but it might be too much of a burden for many users. If such code was contributed would you be willing to include it? You don't have to use it yourself of course if you find it overly vile.
I would look at it, but it's not likely I will include it. Keep in mind that MM 2.1 is end of life. By the time such code would be released, MM 3.1 with a migration path for MM 2.1 users would probably be available
I think a more reasonable approach would be to develop a listinfo.html template and documentation on how to use it, and I would put that in the contrib directory of the release, or you could put it on the wiki <http://wiki.list.org/>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On Sun, 17 Jan 2016 09:34:35 -0800 Mark Sapiro <mark@msapiro.net> wrote:
On 01/17/2016 06:34 AM, Perry E. Metzger wrote:
Mostly it just requires registration. Doing a custom template is probably fine for someone like me who is able to deal with the technical steps involved but it might be too much of a burden for many users. If such code was contributed would you be willing to include it? You don't have to use it yourself of course if you find it overly vile.
I would look at it, but it's not likely I will include it. Keep in mind that MM 2.1 is end of life. By the time such code would be released, MM 3.1 with a migration path for MM 2.1 users would probably be available
What if the patches targeted MM 3?
Perry
Perry E. Metzger perry@piermont.com

On 01/17/2016 05:50 PM, Perry E. Metzger wrote:
On Sun, 17 Jan 2016 09:34:35 -0800 Mark Sapiro <mark@msapiro.net> wrote:
I would look at it, but it's not likely I will include it. Keep in mind that MM 2.1 is end of life. By the time such code would be released, MM 3.1 with a migration path for MM 2.1 users would probably be available
What if the patches targeted MM 3?
In that case, they would be against Postorius. See <http://postorius.readthedocs.org/en/latest/development.html> although that is out of date as it refers to Postorius being hosted on Launchpad, and it is now on gitlab <https://gitlab.com/mailman/postorius>.
The best way to contact Postorius developers is either the mailman-developers@python.org list or the freenode irc channel #mailman.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (6)
-
Andrew Daviel
-
Gibbs, David
-
Jayson Smith
-
Mark Sapiro
-
Perry E. Metzger
-
Rosenbaum, Larry M.