Hi all,
I have a long running list with thousands of subscribers that needs to move to a new set of email addresses.
The existing list is announce-only, so doesn't get a whole lot of traffic - typically at most a few outgoing emails a year. But some time back a bot started sending spam registrations via hits to the web server in the form
<domain name> /mailman/subscribe.cgi/bwbnewsletter?email=<bogus_registrant>@<somedomain.tld><mailto:ruanwei12@hotmail.com>&fullname=&pw=123456789&pw-conf=123456789&language=en&digest=0&email-button=Subscribe.
I long ago routed real users to an alternative signup, but the spam keeps coming, unrelenting, and are now anywhere between 1k and 10k per day.
I'm thinking that the best way to deal with this is to move everything, including the archives, to a newly created list on the same domain name. The alternative signup method i've set up works, so if i set the new list to "Advertise this list..." = NO in Privacy, I'm hoping the spammers won't figure out that the new mailman list exists. I could also set it up as a closed list if that's possible since the new registrations will be entered manually in bulk, though I don't see any settings which disallow all subscriptions.
So I'd like confirmation on how to move a list to a new address under the same domain. I figure I need to:
take a backup copy of the <oldlistname>.mbox file in /archives/private
take a copy of the current subscriber list (already done using the list_members command)
create the new list, aliases, etc and associated email addresses
remove /archives/private/<newlistname>.mbox
rename <oldlistname>.mbox to <newlistname>.mbox
5b) do I need to replace all instances of <oldlistname> with <newlistname> everywhere they occur in the .mbox file? Would doing so cause any problems?
rebuild the archives
test the new list
bulk subscribe the subscriber list to <newlistname>
delete the old list
Did I miss anything?
Thanks in advance.
On 05/14/2015 09:32 PM, Bill Christensen wrote:
I'm thinking that the best way to deal with this is to move everything, including the archives, to a newly created list on the same domain name. The alternative signup method i've set up works, so if i set the new list to "Advertise this list..." = NO in Privacy, I'm hoping the spammers won't figure out that the new mailman list exists. I could also set it up as a closed list if that's possible since the new registrations will be entered manually in bulk, though I don't see any settings which disallow all subscriptions.
Privacy options... -> Subscription rules -> subscribe_policy st to Require approval and then discard all requests.
So I'd like confirmation on how to move a list to a new address under the same domain. I figure I need to:
take a backup copy of the <oldlistname>.mbox file in /archives/private
take a copy of the current subscriber list (already done using the list_members command)
create the new list, aliases, etc and associated email addresses
remove /archives/private/<newlistname>.mbox
rename <oldlistname>.mbox to <newlistname>.mbox
5b) do I need to replace all instances of <oldlistname> with <newlistname> everywhere they occur in the .mbox file? Would doing so cause any problems?
5b) No. You don't need to edit the old .mbox file in any way. Of course, if there are say URLs in the bodies of messages (other than scrubbed attachment links) that point to the old list, you may want to fix those.
You can do all the above (1-5) in one step with the script at <http://www.msapiro.net/scripts/clone_list>.
See the FAQ at <http://wiki.list.org/x/4030617> which mentions this and a few other methods.
- rebuild the archives
See the note in the second half of that FAQ about scrub_nondigest. Similar info is in the script's -h/--help output.
The archives will be moved with the -a/--archives option to the script.
The reason you may want to rebuild the archives are the "more information about this list" links still point to the old list name and likewise, links to scrubbed attachments if any.
test the new list
bulk subscribe the subscriber list to <newlistname>
Already done with the -m/--members option to the list.
- delete the old list
Did I miss anything?
If you have and pending requests, a digest mbox or list specific templates, you may want to move those, but the -e/--extra_files option to the script will do that. It also takes care of alias generation for the new list if you use Mailman/Postfix integration. Otherwise, you may need to update aliases manually.
In short, the whole process is
Install the clone_list script in mailman's bin/ directory.
bin/clone_list -m -a -e old_list_name new_list_name (there are also options to change subject_prefix, list owner and list password, but they default to keeping current values.)
2a) maybe bin/arch --wipe new_list_name (or maybe 3a)
Test
bin/rmlist old_list_name
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Excellent. I didn't know about the clone_list script.
On 5/15/15 12:21 AM, Mark Sapiro wrote:
Privacy options... -> Subscription rules -> subscribe_policy st to Require approval and then discard all requests. when you say "discard all requests", is there a way to do that automatically? Without email notices to the owner address?
On 05/15/2015 06:51 PM, Bill Christensen wrote:
On 5/15/15 12:21 AM, Mark Sapiro wrote:
Privacy options... -> Subscription rules -> subscribe_policy st to Require approval and then discard all requests. when you say "discard all requests", is there a way to do that automatically? Without email notices to the owner address?
Not really. If you set General Options -> admin_immed_notify to No, there will be only one summary notice per day, and not the one per request and you could modify Mailman's crontab to not run checkdbs or run it less frequently than daily, but the requests will still be there.
There is no automatic discard, but it wouldn't be hard to create a script that discards all outstanding subscription requests for a list (or all lists) and run it daily just before checkdbs runs so checkdbs would have nothing to report.
In fact, if you wanted to be really crude and ignore all requests for a list you could just remove the lists/listname/request.pck file. This would effectively discard all requests. You have to be carful because you don't want to do this if there are held messages because then the message files would be orphaned, but something like (adjust list, prefix and var_prefix to suit
#! /bin/sh list=LISTNAME prefix=/usr/lib/mailman var_prefix=/var/lib/mailman ls $var_prefix/data/heldmsg-$list-* | xargs $prefix/bin/discard rm $var_prefix/lists/$list/request.pck
would discard all held messages and then discard the remaining subscribe (and unsubscribe if unsubscribe_policy is Yes) requests. You could add this to the crontab to run just before checkdbs, or add
$prefix/cron/checkdbs
to the end of it and run it instead of checkdbs. That all assumes you might have other lists you want to treat normally.
There are other things you can do as well, but, looking at your original post, I see I didn't fully register the situation. If your Mailman is at least 2.1.16, all you need to do is set
SUBSCRIBE_FORM_SECRET = 'Some string unique to your site"
in mm_cfg.py, and that attack will no longer work. Doing that requires that for the subscribe to be successful, the URL must contain a token which was generated by the listinfo subscribe form at least 5 seconds and no more than an hour before submission. The token contains a hash of the secret, the requesting IP, the list name and the time which means if a bot is going to subscribe, it has to first retrieve the listinfo page to get the token and then wait 5 seconds (which can be configured to be longer by setting SUBSCRIBE_FORM_MIN_TIME in mm_cfg.py) and then submit the request with the token. I don't think the bots have figured it out yet.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 15/05/15 05:32, Bill Christensen wrote:
I have a long running list with thousands of subscribers that needs to move to a new set of email addresses.
The existing list is announce-only, so doesn't get a whole lot of traffic - typically at most a few outgoing emails a year. But some time back a bot started sending spam registrations via hits to the web server in the form
<domain name>/mailman/subscribe.cgi/bwbnewsletter?email=<bogus_registrant>@<somedomain.tld><mailto:ruanwei12@hotmail.com>&fullname=&pw=123456789&pw-conf=123456789&language=en&digest=0&email-button=Subscribe.
I long ago routed real users to an alternative signup, but the spam keeps coming, unrelenting, and are now anywhere between 1k and 10k per day.
One of our lists was spammed in a similar way. The approach I took was to configure Apache to allow requests to /mailman/subscribe only when the referring page was on our server:
SetEnvIf Referer lists\.my\.domain localreferer
<Location /mailman/subscribe> Order deny,allow Deny from all Allow from env=localreferer </Location>
-- David Osborne University of Nottingham, UK
This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it.
Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham.
This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system, you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation.
On Wed, May 20, 2015 at 02:38:01PM +0100, David Osborne wrote:
On 15/05/15 05:32, Bill Christensen wrote:
I long ago routed real users to an alternative signup, but the spam keeps coming, unrelenting, and are now anywhere between 1k and 10k per day.
One of our lists was spammed in a similar way. The approach I took was to configure Apache to allow requests to /mailman/subscribe only when the referring page was on our server:
I've used mod_security/fail2ban in the past, both work as well as might be expected.
[…]
This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it.
Hum.
-- "Celebrity can be malign in that it becomes a form of idolatry, and people live their lives vicariously through the rich and famous rather than attending to their own lives." -- John Sentamu
Mark's fix:
If your Mailman is at least 2.1.16, all you need to do is set
SUBSCRIBE_FORM_SECRET = 'Some string unique to your site"
in mm_cfg.py, and that attack will no longer work.
Is working fine. But thanks for the alternative suggestions. I've got a copy of fail2ban ready to install but just haven't had time to configure it yet.
On Wed, May 20, 2015 at 9:07 AM, Adam McGreggor <adam-mailman@amyl.org.uk> wrote:
On Wed, May 20, 2015 at 02:38:01PM +0100, David Osborne wrote:
On 15/05/15 05:32, Bill Christensen wrote:
I long ago routed real users to an alternative signup, but the spam keeps coming, unrelenting, and are now anywhere between 1k and 10k per day.
One of our lists was spammed in a similar way. The approach I took was to configure Apache to allow requests to /mailman/subscribe only when the referring page was on our server:
I've used mod_security/fail2ban in the past, both work as well as might be expected.
[…]
This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it.
Hum.
-- "Celebrity can be malign in that it becomes a form of idolatry, and people live their lives vicariously through the rich and famous rather than attending to their own lives." -- John Sentamu
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/billc_lists%40greenbui...
participants (4)
-
Adam McGreggor -
Bill Christensen -
David Osborne -
Mark Sapiro