I have a mailman installation with over 300 lists. It is cPanel, but I am the administrator so have access to command line etc.
I have just two lists that receive a bunch of spam subscribes each day -- hundreds of them, in fact. For some reason -- which is good, they are held, so don't go through, not quite sure why. Two questions -- first is there a file I can erase for each list that will get rid of all the held subscriptions, without breaking anything else. I tried once, and my installation broke -- don't know if it is related, but don't want to try again unless I do it right.
Secondly, there is some commonality in the subscribe addresses, are there strings I can use to discard the subscribes so I never have to see them.
Below are examples, there is a common word, or a common word, a period ., and another common word, then a plus sign + then a 4 5 or 6 character word, all alpha, and @gmail.com Here are examples:
dragonommz+ jwmidnight+ nommz.naidoo+
If I could knock these out, it would be helpful. This has happened several times previously, but has always stopped after a few weeks. This time it has been a couple months.
Finally, I know it is probably too late in the Mailman2 cycle to get a new feature, but in the web UI, it would be nice if you could delete all deferred subscriptions. You can do so with deferred messages, that are held, but not subscriptions.
Thanks!
Dave
On 02/23/18 10:07, David Andrews wrote:
Secondly, there is some commonality in the subscribe addresses, are there strings I can use to discard the subscribes so I never have to see them.
Below are examples, there is a common word, or a common word, a period ., and another common word, then a plus sign + then a 4 5 or 6 character word, all alpha, and @gmail.com Here are examples:
dragonommz+ jwmidnight+ nommz.naidoo+
If I could knock these out, it would be helpful. This has happened several times previously, but has always stopped after a few weeks. This time it has been a couple months.
You can't filter based on that address format. (At least, not and be correct.)
This format, plus-extension, is a legitimate address structure specifically for the purpose of generating traceable throwaway addresses. If I give you reddog+thislist@example.com as my email address, which I receive at my address reddog@example.com, and I've given that to no-one else, and a few weeks later I start getting random spam sent to reddog+thislist@example.com, I know you have (intentionally or otherwise) leaked my email address.
Just because an address is plus-extended does not mean it is spam. If you choose to refuse extended addresses, you risk refusing legitimate subscribers.
Have you considered requiring CAPTCHAs for subscription?
-- Phil Stracchino Babylon Communications phils@caerllewys.net phil@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958
On 02/23/18 10:07, David Andrews wrote:
I have just two lists that receive a bunch of spam subscribes each day -- hundreds of them, in fact. For some reason -- which is good, they are held, so don't go through, not quite sure why. Two questions -- first is there a file I can erase for each list that will get rid of all the held subscriptions, without breaking anything else. I tried once, and my installation broke -- don't know if it is related, but don't want to try again unless I do it right.
As for the held subscriptions, you should be able to go to the list's admin interface and drop all of the pending subscription requests as a single operation.
-- Phil Stracchino Babylon Communications phils@caerllewys.net phil@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958
I have a mailman installation with over 300 lists. It is cPanel, but I am the administrator so have access to command line etc.
I have just two lists that receive a bunch of spam subscribes each day -- hundreds of them, in fact. For some reason -- which is good, they are held, so don't go through, not quite sure why. Two questions -- first is there a file I can erase for each list that will get rid of all the held subscriptions, without breaking anything else. I tried once, and my installation broke -- don't know if it is related, but don't want to try again unless I do it right.
Secondly, there is some commonality in the subscribe addresses, are there strings I can use to discard the subscribes so I never have to see them.
Below are examples, there is a common word, or a common word, a period ., and another common word, then a plus sign + then a 4 5 or 6 character word, all alpha, and @gmail.com Here are examples:
dragonommz+ jwmidnight+ nommz.naidoo+
If I could knock these out, it would be helpful. This has happened several times previously, but has always stopped after a few weeks. This time it has been a couple months.
Finally, I know it is probably too late in the Mailman2 cycle to get a new feature, but in the web UI, it would be nice if you could delete all deferred subscriptions. You can do so with deferred messages, that are held, but not subscriptions.
Thanks!
Dave
Hey Dave,
If you are using cPanel then running some Exim filters may be a better approach to handling this subscription problem. We get tons of spam from China via two domains and Exim is great at filtering them out.
Brian Carpenter Owner
Providing Cloud Services and more for over 15 years.
T: 336.755.0685 E: brian@emwd.com www.emwd.com
On 02/23/2018 07:07 AM, David Andrews wrote:
I have just two lists that receive a bunch of spam subscribes each day -- hundreds of them, in fact. For some reason -- which is good, they are held, so don't go through, not quite sure why. Two questions -- first is there a file I can erase for each list that will get rid of all the held subscriptions, without breaking anything else. I tried once, and my installation broke -- don't know if it is related, but don't want to try again unless I do it right.
See the script at <https://www.msapiro.net/scripts/erase> (mirrored at <https://fog.ccsf.edu/~msapiro/scripts/erase>). This will remove everything for an address or addresses that match a regexp.
Also for any list you can remove the lists/LISTNAME/request.pck file, but if there are any held messages for the list, they too will disappear from the pending requests although the data/heldmsg-LISTNAME-nnn.pck file will still be there. The best thing is to handle all held messages before removing the requests.pck file, but there is a script at <https://www.msapiro.net/scripts/hold_again> (mirrored as above) that can reprocess the data/heldmsg-LISTNAME-nnn.pck files or they can be removed if not wanted.
Secondly, there is some commonality in the subscribe addresses, are there strings I can use to discard the subscribes so I never have to see them.
Below are examples, there is a common word, or a common word, a period ., and another common word, then a plus sign + then a 4 5 or 6 character word, all alpha, and @gmail.com Here are examples:
dragonommz+ jwmidnight+ nommz.naidoo+
Since Mailman 2.1.21 there is a GLOBAL_BAN_LIST. See <https://mail.python.org/pipermail/mailman-users/2018-January/082905.html> for a bit on how to use this. You will find more in the archives from this Google search <https://www.google.com/search?q=site%3Amail.python.org+inurl%3Amailman-users...>
Also, if you haven't done so, set SUBSCRIBE_FORM_SECRET to some string unique to your site.
Both the above are mm_cfg.py settings.
Also, I don't know when cPanel will upgrade to Mailman 2.1.26 but it contains an ability to enable reCAPTCHA on the listinfo page subscribe form.
Finally, I know it is probably too late in the Mailman2 cycle to get a new feature, but in the web UI, it would be nice if you could delete all deferred subscriptions. You can do so with deferred messages, that are held, but not subscriptions.
If someone wants to do it, I'd accept a merge request, but I'm not likely to do it myself
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
A couple months ago I asked a question and got a response from Mark Sapiro, see below. We are having trouble implementing anything. We are trying recaptcha, but it isn't popular with our users, thousands of whom are blind. Here is what my Linux guy asks:
Does anyone have any solution for dealing with spam subscriptions from gmail addresses? The requests are coming from random addresses that contain a few words, a plus sign, then another random string of characters. I can't figure out how we block this without blocking all addresses with plus characters in them, which is not a good option. We are getting hundreds of held subscription messages per day. Is blocking this kind of thing through Exim an option? We are using cpanel.
p.s. The number of messages is causing my ISP to throttle my e-mail!
Dave
At 01:50 PM 2/23/2018, Mark Sapiro wrote:
On 02/23/2018 07:07 AM, David Andrews wrote: > > I have just two lists that receive a bunch of spam subscribes each day > -- hundreds of them, in fact. For some reason -- which is good, they are > held, so don't go through, not quite sure why. Two questions -- first > is there a file I can erase for each list that will get rid of all the > held subscriptions, without breaking anything else. I tried once, and > my installation broke -- don't know if it is related, but don't want to > try again unless I do it right. See the script at <https://www.msapiro.net/scripts/erase> (mirrored at <https://fog.ccsf.edu/~msapiro/scripts/erase>). This will remove everything for an address or addresses that match a regexp. Also for any list you can remove the lists/LISTNAME/request.pck file, but if there are any held messages for the list, they too will disappear from the pending requests although the data/heldmsg-LISTNAME-nnn.pck file will still be there. The best thing is to handle all held messages before removing the requests.pck file, but there is a script at <https://www.msapiro.net/scripts/hold_again> (mirrored as above) that can reprocess the data/heldmsg-LISTNAME-nnn.pck files or they can be removed if not wanted. > Secondly, there is some commonality in the subscribe addresses, are > there strings I can use to discard the subscribes so I never have to see > them. > > Below are examples, there is a common word, or a common word, a period > ., and another common word, then a plus sign + then a 4 5 or 6 character > word, all alpha, and @gmail.com > Here are examples: > > > dragonommz+ > jwmidnight+ > nommz.naidoo+ Since Mailman 2.1.21 there is a GLOBAL_BAN_LIST. See <https://mail.python.org/pipermail/mailman-users/2018-January/082905.html> for a bit on how to use this. You will find more in the archives from this Google search <https://www.google.com/search?q=site%3Amail.python.org+inurl%3Amailman-users...> Also, if you haven't done so, set SUBSCRIBE_FORM_SECRET to some string unique to your site. Both the above are mm_cfg.py settings. Also, I don't know when cPanel will upgrade to Mailman 2.1.26 but it contains an ability to enable reCAPTCHA on the listinfo page subscribe form. > Finally, I know it is probably too late in the Mailman2 cycle to get a > new feature, but in the web UI, it would be nice if you could delete all > deferred subscriptions. You can do so with deferred messages, that are > held, but not subscriptions. If someone wants to do it, I'd accept a merge request, but I'm not likely to do it myself -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/dandrews%40visi.com
This email has been checked for viruses by AVG. https://www.avg.com
On 06/02/2018 06:55 PM, David Andrews wrote:
Does anyone have any solution for dealing with spam subscriptions from gmail addresses? The requests are coming from random addresses that contain a few words, a plus sign, then another random string of characters.
I use this regexp in the GLOBAL_BAN_LIST
^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$
That blocks subscribe attempts from any address which is 8 or more letters, digits and periods followed by a plus followed by 4 or more letters and digits @gmail.com.
Recently, I've seen some with only 6 letters before the + so you might reduce {8,} to {6,}. I think I'll try that too.
I also have
^.*\+.*\d{3,}@
which blocks anything with a + followed by anything ending in 3 or more digits. Scanning the membership of all the Mailman 2.1 lists @python.org (over 132K addresses) shows only 10 matches 4 of which were members of the python-3000@python.org with addresses .*+python-3000@.* and the other 6 were nabble.com or googlegroups.com, so it's very unlikely that legitimate regular subscribers will match that.
The advantage of the global ban list for this is all the ones I've seen are web subscribes. This blocks them with a web response and doesn't send any confirmation email.
We are getting hundreds of held subscription messages per day. Is blocking this kind of thing through Exim an option? We are using cpanel.
If these as I've seen are all web subscribes, the only thing you could do in Exim is drop the outgoing confirmation email, but banning them stops the subscribe attempt before any mail is sent.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I have a different question.
For a few weeks now the Mailman 2.1 lists @python.org have seen a massive number of web subscribes from addresses @yahoo.com and @aol.com addresses. The aol.com ones seem to have abated but yahoo.com continues. They mostly have local parts that look like first and last names and display names that don't match the local part name. I implemented reCAPTCHA on the listinfo subscribe forms and that didn't seem to slow them down. Also, at first at least some of the subscriptions waiting user confirmation were being confirmed, some by email and some by web.
I have resorted to scraping Mailman's logs with an hourly cron looking for subscribes and attempts and when it find 4 or more for a single address, it uses my erase script to remove them.
This seems to slow down on weekends and pick up during the week.
My question is does anyone have a clue as to who might be doing this and what they are trying to accomplish. As far as I know, even when they've succeeded in subscribing, they don't try to post.
Are they just script kiddies trying to be noticed or are they actually trying to accomplish something.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
On Sat, 2018-06-02 at 20:50 -0700, Mark Sapiro wrote:
Are they just script kiddies trying to be noticed or are they actually trying to accomplish something.
I don't think they know what potential they have, but they know there has to be something worth building a collection for. I'm fairly confident that they've achieved some level of non-moderated subscriptions on some lists, the question is what do they intend to do with that megaphone.
- -Jim P. -----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEEPxwe8uYBnqxkbORSJxVetMRaJwUFAlsTaagACgkQJxVetMRa JwUguA/8DdKmlfjve8q8I7RyAUq3c2FwHfmsrn4iph7cdjs+kek/6dI5Vi59A11n ctvkbZkzCBrLwJH1T2nSZkeQ0jNbVcScCX4Iy8nscjUhU+k7Rq4E46kZbkzIzCVl nCzhi4HEgiDv3wFCxNThBFYBxJPH8yiHIaIne4B324Q62ZordzR2KRKkoOGefU/w wtCmUlLujvyeW1QGbBxQO5b5B9hwCwsu9izqtsS0evu+fV9uNF74Eu9J/XT82dQR qRc+g3pDTdwMo71HLMxsY5mrZ1ZRBe0z4sSN2baDe3mr4yy2X+ebNZbPms4g7GaA TV9vmxWDV4+SPlHLXfauD21ByTGC98GeNMaOOHsaBLiZ9CGQ41GtAsSzuR7xqmqb PzPx/tui8bXvL+Yt6GXZq9qLBcQVeQxO3OSoYrsJt8I0+XNqtU8z/UzEoaEoqxrF ZFXypa/+bPavINNW5h4jSCAvtT+QJso501N4HzGG/pGbWvI8+6Q80nXFB5PqFrCp NNEAAfZmNZtsqm4nMYG65lcsmB2d3t/KpLWz9yB3y2uZWZUD0Hh/2Tet8zIrfMCT p7sigLH0gbjaCR5dcsXoyiQoXQAraubG/D/LJ5yEhrBIuM4QLllAbqUwnqLkFu+A S/TLW0uext7nL1HnhzRa4w/MXwme8LRF31UoDsh1hdIxiRjAYA0= =YSGk -----END PGP SIGNATURE-----
At 10:29 PM 6/2/2018, Mark Sapiro wrote:
On 06/02/2018 06:55 PM, David Andrews wrote:
Does anyone have any solution for dealing with spam subscriptions from gmail addresses? The requests are coming from random addresses that contain a few words, a plus sign, then another random string of characters.
I use this regexp in the GLOBAL_BAN_LIST
^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$
That blocks subscribe attempts from any address which is 8 or more letters, digits and periods followed by a plus followed by 4 or more letters and digits @gmail.com.
Recently, I've seen some with only 6 letters before the + so you might reduce {8,} to {6,}. I think I'll try that too.
I also have
^.*\+.*\d{3,}@
which blocks anything with a + followed by anything ending in 3 or more digits. Scanning the membership of all the Mailman 2.1 lists @python.org (over 132K addresses) shows only 10 matches 4 of which were members of the python-3000@python.org with addresses .*+python-3000@.* and the other 6 were nabble.com or googlegroups.com, so it's very unlikely that legitimate regular subscribers will match that.
The advantage of the global ban list for this is all the ones I've seen are web subscribes. This blocks them with a web response and doesn't send any confirmation email.
Thanks very much -- we are trying it!
You always have the answer -- thanks!
Dave
We are getting hundreds of held subscription messages per day. Is blocking this kind of thing through Exim an option? We are using cpanel.
If these as I've seen are all web subscribes, the only thing you could do in Exim is drop the outgoing confirmation email, but banning them stops the subscribe attempt before any mail is sent.
This email has been checked for viruses by AVG. https://www.avg.com
On 06/02/2018 09:29 PM, Mark Sapiro wrote:
I use this regexp in the GLOBAL_BAN_LIST
^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$
Are you not looking for capital letters?
I can see how the period in the first class would work, but I don't see that in the second class.
What am I missing?
-- Grant. . . . unix || die
On 06/03/2018 09:53 AM, Grant Taylor via Mailman-Users wrote:
On 06/02/2018 09:29 PM, Mark Sapiro wrote:
I use this regexp in the GLOBAL_BAN_LIST
^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$
Are you not looking for capital letters?
Ban list regexps are case insensitive.
I can see how the period in the first class would work, but I don't see that in the second class.
What am I missing?
The fact that the ones I saw never had periods following the plus sign.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I use this regexp in the GLOBAL_BAN_LIST
^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$
I'm getting errors with the above however it seems to do the job if I enclose it in quotes and remove the trailing $ - like so:
GLOBAL_BAN_LIST = ['^[0-9a-z.]{6,}\+[0-9a-z]{4,}@gmail\.com']
Am I missing something?
Cheers, Mark
On 06/03/2018 04:28 PM, Mark Dale wrote:
I use this regexp in the GLOBAL_BAN_LIST
^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$
I'm getting errors with the above however it seems to do the job if I enclose it in quotes and remove the trailing $ - like so:
GLOBAL_BAN_LIST = ['^[0-9a-z.]{6,}\+[0-9a-z]{4,}@gmail\.com']
What you have done is correct. I don't know why you would have needed to remove the '$'. Did you get an error and if so, what?.
The regexp I gave was just intended to be an example regexp. The BAN_LIST is actually a list of strings so regexps in the BAN_LIST have to be quoted and enclosed in [] and comma separated if more than one.
Also, it doesn't matter in this case because \+ and \. are not meaningful string metacharacters, but it never hurts to define them as raw strings like, e.g.,
GLOBAL_BAN_LIST = [r'^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$']
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$
I'm getting errors with the above however it seems to do the job if I enclose it in quotes and remove the trailing $ - like so:
GLOBAL_BAN_LIST = ['^[0-9a-z.]{6,}\+[0-9a-z]{4,}@gmail\.com']
What you have done is correct. I don't know why you would have needed to remove the '$'. Did you get an error and if so, what?.
I can't see why either, but with the '$' left in place, the Mailman Web UI displayed the error "Sorry, we hit a bug..."
Also, it doesn't matter in this case because \+ and \. are not meaningful string metacharacters, but it never hurts to define them as raw strings like, e.g.,
Understood. Thanks.
GLOBAL_BAN_LIST = [r'^[0-9a-z.]{8,}\+[0-9a-z]{4,}@gmail\.com$']
The 'r' that precedes the first quote - is that correct?
Cheers.
On 06/03/2018 05:58 PM, Mark Dale wrote:
I can't see why either, but with the '$' left in place, the Mailman Web UI displayed the error "Sorry, we hit a bug..."
And what is the error in Mailman's error log.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
And what is the error in Mailman's error log.
GLOBAL_BAN_LIST = ['^[0-9a-z.]{6,}\+[0-9a-z]{4,}@gmail\.com$']
D'oh. My apologies. The error was not from the trailing '$' but from not having the quotes in place originally. All is now well (with the above).
Thanks, Mark
2018/06/04 10:33:14 [error] 17956#17956: *4956 FastCGI sent in stderr: "[Mailman: low level unrecoverable exception]" while reading response header from upstream, client: 68.235.48.108, server: mailmanlists.sg, request: "GET /mailman/listinfo HTTP/1.1", upstream: "fastcgi://unix:///var/run/fcgiwrap.socket:", host: "www.mailmanlists.sg"
Mark Dale writes:
D'oh. My apologies. The error was not from the trailing '$' but from not having the quotes in place originally. All is now well (with the above).
No big deal; on the contrary, we really appreciate your report confirming that the regex works as expected for you, after all.
Thank *you*!
Steve
participants (8)
-
Brian Carpenter -
David Andrews -
Grant Taylor -
Jim Popovitch -
Mark Dale -
Mark Sapiro -
Phil Stracchino -
Stephen J. Turnbull