I've got a new problem with my mailing list. I run a local announcements/trade list that should be of no interest to non-locals.
I started seeing warnings from Yahoo about users marking messages as spam.. (I'm subscribed to Yahoo's Antispam Feedback. I never got other feedback loops from others - like Microsoft - to work). The messages that were being marked as spam by users were the confirmation emails sent by mailman to confirm a signup. To avoid having Yahoo shut down the list as spam (for its subscribers), I set the subscription to be approved by admin, so I could review who was trying to sign up.
More and more now I'm seeing what appears to be spammers trying to subscribe, but I can't be sure. I'm seeing emails like:
hirofeet0916@yahoo.co.jp (doesn't seem local) blvckpvlm@gmail.com (not many use user names such as that) fsafwcasgsadf2@gwqc.com (obviously not real - couldn't find domain with minimal searching)
Plus some that could be local but how would I know.
I could take off the "approval by admin" for subscription and just deal with anyone that is a problem afterward, but I do worry that they may be harvesting emails from subscribers, which are available in the reply-to headers.
Don't know if there's anything I can do. Anyone else dealing with this?
thanks. Jim
I started seeing these as well, in pretty large quantities just recently. I wasn't seeing spam notices, but an increase in my deferred mail queue (Postfix) from emails that couldn't be delivered. Looking in my Mailman logs, I had hundreds of subscription/signup requests without any subsequent confirmations, and they were coming in once a minute or so from bots.
I had added 'SUBSCRIBE_FORM_SECRET' with a key some time ago, but that seemed to have stopped working as effectively. Implementing the google captcha solution to Mailman a week or so ago stopped it dead. For now...
Quoting Jim Dory (james@dorydesign.com):
I've got a new problem with my mailing list. I run a local announcements/trade list that should be of no interest to non-locals.
I started seeing warnings from Yahoo about users marking messages as spam.. (I'm subscribed to Yahoo's Antispam Feedback. I never got other feedback loops from others - like Microsoft - to work). The messages that were being marked as spam by users were the confirmation emails sent by mailman to confirm a signup. To avoid having Yahoo shut down the list as spam (for its subscribers), I set the subscription to be approved by admin, so I could review who was trying to sign up.
More and more now I'm seeing what appears to be spammers trying to subscribe, but I can't be sure. I'm seeing emails like:
hirofeet0916@yahoo.co.jp (doesn't seem local) blvckpvlm@gmail.com (not many use user names such as that) fsafwcasgsadf2@gwqc.com (obviously not real - couldn't find domain with minimal searching)
Plus some that could be local but how would I know.
I could take off the "approval by admin" for subscription and just deal with anyone that is a problem afterward, but I do worry that they may be harvesting emails from subscribers, which are available in the reply-to headers.
Don't know if there's anything I can do. Anyone else dealing with this?
thanks. Jim
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/ Member address: ddewey@cyberthugs.com
On Mon, 23 Oct 2023, ddewey@cyberthugs.com wrote:
...<snip!>...
Implementing the google captcha solution to Mailman a week or so ago stopped it dead. For now...
Hi Jim,
Interesting, and thanks for posting.
Can you please describe, briefly, as an overview only, what that interface is like?
I'm sure I can look up details, but, well, "details matter!" (And I don't mean the nitty gritty of installation or whatever.) I'm interested in implementation overview in how that relates to the user's experience - I already know what 'captcha' is like! We're talking web interface details, right?
Thanks, Richard
On 10/23/23 17:38, richard@karmannghia.org wrote:
On Mon, 23 Oct 2023, ddewey@cyberthugs.com wrote:
...<snip!>...
Implementing the google captcha solution to Mailman a week or so ago stopped it dead. For now...
Hi Jim,
Interesting, and thanks for posting.
Can you please describe, briefly, as an overview only, what that interface is like?
I'm sure I can look up details, but, well, "details matter!" (And I don't mean the nitty gritty of installation or whatever.) I'm interested in implementation overview in how that relates to the user's experience - I already know what 'captcha' is like! We're talking web interface details, right?
Thanks, Richard
Hello Richard,
It was ddewey that mentioned the captcha. I am interested in implementing it and googled it - found things from about 10 years ago, and mailman post from 2017. I have mailman version 2.1.39 on a VPS hosted server (with WHM and CPanel) with root privileges, though not sure I have the chutzpa to install it. Could give it a try I suppose.
The mailman post was https://mail.python.org/pipermail/mailman-users/2017-December/082820.html
Jim
On 10/23/23 6:46 PM, Jim Dory wrote:
On 10/23/23 17:38, richard@karmannghia.org wrote:
On Mon, 23 Oct 2023, ddewey@cyberthugs.com wrote:
...<snip!>...
Implementing the google captcha solution to Mailman a week or so ago stopped it dead. For now...
Hi Jim,
Interesting, and thanks for posting.
Can you please describe, briefly, as an overview only, what that interface is like?
I'm sure I can look up details, but, well, "details matter!" (And I don't mean the nitty gritty of installation or whatever.) I'm interested in implementation overview in how that relates to the user's experience - I already know what 'captcha' is like! We're talking web interface details, right?
Thanks, Richard
Hello Richard,
It was ddewey that mentioned the captcha. I am interested in implementing it and googled it - found things from about 10 years ago, and mailman post from 2017. I have mailman version 2.1.39 on a VPS hosted server (with WHM and CPanel) with root privileges, though not sure I have the chutzpa to install it. Could give it a try I suppose.
The mailman post was https://mail.python.org/pipermail/mailman-users/2017-December/082820.ht
I don't think that's required. We have reCAPTCHA implemented for MM 2.1 at https://mail.python.org/mailman/listinfo/ by just following the doc at https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/...
- there is also a custom CAPTCHA test that can be implemented as documented at https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/....
You will find Defaults.py and mm_cfg.py in /usr/local/cpanel/3rdparty/mailman/Mailman/ on cPanel. Any changes should be made by settings in mm_cfg.py which will override the defaults from Defaults.py.
Also see https://wiki.list.org/DOC/Mailman%20and%20CPanel for info about cPanel.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Quoting Jim Dory (james@dorydesign.com):
On 10/23/23 17:38, richard@karmannghia.org wrote:
On Mon, 23 Oct 2023, ddewey@cyberthugs.com wrote:
...<snip!>...
Implementing the google captcha solution to Mailman a week or so ago stopped it dead. For now...
Hi Jim,
Interesting, and thanks for posting.
Can you please describe, briefly, as an overview only, what that interface is like?
I'm sure I can look up details, but, well, "details matter!" (And I don't mean the nitty gritty of installation or whatever.) I'm interested in implementation overview in how that relates to the user's experience - I already know what 'captcha' is like! We're talking web interface details, right?
Thanks, Richard
Hello Richard,
It was ddewey that mentioned the captcha. I am interested in implementing it and googled it - found things from about 10 years ago, and mailman post from 2017. I have mailman version 2.1.39 on a VPS hosted server (with WHM and CPanel) with root privileges, though not sure I have the chutzpa to install it. Could give it a try I suppose.
The mailman post was https://mail.python.org/pipermail/mailman-users/2017-December/082820.html
for 2.1.39 (same as I'm running) it's built-in. No need to install any python modules (I didn't, on a RHEL 7 instance).
Apply for the (very important) v2 captcha keys from Google. Add the following 2 keys to the end of your mm_cfg.py: RECAPTCHA_SITE_KEY = xxxx RECAPTCHA_SECRET_KEY = xxxxxxx
If you have not modified your list_info files, you're done. If you have, then you'll need to add the tags from that above post to your list_info files.
On 10/24/23 06:04, David L. Dewey wrote:
Quoting Jim Dory (james@dorydesign.com):
On 10/23/23 17:38, richard@karmannghia.org wrote:
On Mon, 23 Oct 2023, ddewey@cyberthugs.com wrote:
...<snip!>...
Implementing the google captcha solution to Mailman a week or so ago stopped it dead. For now... Hi Jim,
Interesting, and thanks for posting.
Can you please describe, briefly, as an overview only, what that interface is like?
I'm sure I can look up details, but, well, "details matter!" (And I don't mean the nitty gritty of installation or whatever.) I'm interested in implementation overview in how that relates to the user's experience - I already know what 'captcha' is like! We're talking web interface details, right?
Thanks, Richard Hello Richard,
It was ddewey that mentioned the captcha. I am interested in implementing it and googled it - found things from about 10 years ago, and mailman post from 2017. I have mailman version 2.1.39 on a VPS hosted server (with WHM and CPanel) with root privileges, though not sure I have the chutzpa to install it. Could give it a try I suppose.
The mailman post was https://mail.python.org/pipermail/mailman-users/2017-December/082820.html for 2.1.39 (same as I'm running) it's built-in. No need to install any python modules (I didn't, on a RHEL 7 instance).
Apply for the (very important) v2 captcha keys from Google. Add the following 2 keys to the end of your mm_cfg.py: RECAPTCHA_SITE_KEY = xxxx RECAPTCHA_SECRET_KEY = xxxxxxx
If you have not modified your list_info files, you're done. If you have, then you'll need to add the tags from that above post to your list_info files.
I've started getting these spamming attacks again so thought I would dive into trying this recaptcha. I got the keys for V2 recaptcha from google and put the 2 lines at the bottom of the mm_cfg.py with proper keys from google. Spelling double checked. After saving the file, I can't log into the web interface of mailman - I get a Bad Request error page. I commented out the RECAPTCHA_*_* lines and could then access the admin web pages again.
David wrote that if I modified the list_info files then I needed to add something to them. As far as modifying them, I did add a paragraph description to the "Details for info" and the description under the General Options section of the web admin pages. What would I add and to which files? I don't see list_info under /usr/local/cpanel/3rdparty/mailman/Mailman/ .
Any advice? thanks, Jim
Jim Dory writes:
I've started getting these spamming attacks again so thought I would dive into trying this recaptcha. I got the keys for V2 recaptcha from google and put the 2 lines at the bottom of the mm_cfg.py with proper keys from google. Spelling double checked. After saving the file, I can't log into the web interface of mailman - I get a Bad Request error page. I commented out the RECAPTCHA_*_* lines and could then access the admin web pages again.
There's a lot missing here.
- What version of what operating system are you using? Ubuntu and Debian are likely to require some hoop-jumping to get the needed software installed.
- What version of Python are you using?
- What version of Mailman are you using? If it's recent enough, the listinfo.* pages will include a tag "<mm-recaptcha-ui>" which does all the heavy lifting for you.
- How did you install Mailman? Preinstalled on a cPanel host, from the OS, from source in a virtual environment, other from source?
web admin pages. What would I add and to which files? I don't see list_info under� /usr/local/cpanel/3rdparty/mailman/Mailman/ .
Try find /usr/local/cpanel/3rdparty/mailman -name 'listinfo.*'
and you should see a bunch of them. Most likely you are only
interested in
/usr/local/cpanel/3rdparty/mailman/templates/en/listinfo.html
and maybe the .txt version of that file if it exists, but if you offer
other languages to your users, you may need to deal with the
$TWO_LETTER_LANGUAGE_CODE/listinfo.* versions for those languages.
On 12/13/23 01:04, Stephen J. Turnbull wrote:
Jim Dory writes:
I've started getting these spamming attacks again so thought I would dive into trying this recaptcha. I got the keys for V2 recaptcha from google and put the 2 lines at the bottom of the mm_cfg.py with proper keys from google. Spelling double checked. After saving the file, I can't log into the web interface of mailman - I get a Bad Request error page. I commented out the RECAPTCHA_*_* lines and could then access the admin web pages again.
There's a lot missing here.
- What version of what operating system are you using? Ubuntu and Debian are likely to require some hoop-jumping to get the needed software installed.
- What version of Python are you using?
- What version of Mailman are you using? If it's recent enough, the listinfo.* pages will include a tag "<mm-recaptcha-ui>" which does all the heavy lifting for you.
- How did you install Mailman? Preinstalled on a cPanel host, from the OS, from source in a virtual environment, other from source?
web admin pages. What would I add and to which files? I don't see list_info under /usr/local/cpanel/3rdparty/mailman/Mailman/ .
Try
find /usr/local/cpanel/3rdparty/mailman -name 'listinfo.*'and you should see a bunch of them. Most likely you are only interested in /usr/local/cpanel/3rdparty/mailman/templates/en/listinfo.html and maybe the .txt version of that file if it exists, but if you offer other languages to your users, you may need to deal with the $TWO_LETTER_LANGUAGE_CODE/listinfo.* versions for those languages.
Thank you Stephen. Apologies for being vague.
I did find the listinfo.html file yesterday - I hadn't ever altered that particular file directly.
- CentOS v7.9.2009 STANDARD kvm, cPanel Version 110.0.17. I need to upgrade the OS AlmaLinux 8 by this summer. as CentOS (and the cPanel version) is deprecated. Would do it now but afraid of mucking things up. I'm just a volunteer and do this for the community - not an expert by any means.
2, Python 2.7.5
Mailman 2.1.39
Mailman installed by host. I'm on a vps with root access.
As for listinfo.html, I see 2 pertinent files. on under ../en/templates (this list is just english) and under ../lists/[name of our list]/ . The templates version includes a few lines of captcha which the lists version doesn't. Here's a snippet of the templates version:
[snip] </td> <td><MM-Undigest-Radio-Button> No <MM-Digest-Radio-Button> Yes </TD> </tr> <mm-digest-question-end> <mm-recaptcha-ui> <mm-captcha-ui> <tr> <td colspan="3"> <center><MM-Subscribe-Button></center> </td> </tr> </TABLE> <MM-Form-End> </ul>
On 12/13/23 10:00, Jim Dory wrote:
As for listinfo.html, I see 2 pertinent files. on under ../en/templates (this list is just english) and under ../lists/[name of our list]/ . The templates version includes a few lines of captcha which the lists version doesn't. Here's a snippet of the templates version:
This is your problem. You have a list specific version of the listinfo.html template in lists/listname/en/listinfo.html which was probably created on an older version before the captchas were implemented.
You need to diff lists/listname/en/listinfo.html with templates/en/listinfo.html. Part of the diff will be the absence of the
<mm-recaptcha-ui>
<mm-captcha-ui>tags in lists/listname/en/listinfo.html which need to be added. If that's the only diff, you can simply remove lists/listname/en/listinfo.html and fall back to the default, but if you had local changes in lists/listname/en/listinfo.html, you probably want to keep those and just add the missing captcha tags.
However, the only issue from those missing tags should be an inability to subscribe via the listinfo page. It shouldn't affect login to the admin or admindb pages and it shouldn't cause a Bad Request error. If after fixing the template, you still can't log in, We would like to see the full traceback from Mailman's logs/error log.
It's possible this is a cPanel issue.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12/13/23 12:08, Mark Sapiro wrote:
On 12/13/23 10:00, Jim Dory wrote:
As for listinfo.html, I see 2 pertinent files. on under ../en/templates (this list is just english) and under ../lists/[name of our list]/ . The templates version includes a few lines of captcha which the lists version doesn't. Here's a snippet of the templates version:
This is your problem. You have a list specific version of the listinfo.html template in lists/listname/en/listinfo.html which was probably created on an older version before the captchas were implemented.
You need to diff lists/listname/en/listinfo.html with templates/en/listinfo.html. Part of the diff will be the absence of the
<mm-recaptcha-ui> <mm-captcha-ui>
tags in lists/listname/en/listinfo.html which need to be added. If that's the only diff, you can simply remove lists/listname/en/listinfo.html and fall back to the default, but if you had local changes in lists/listname/en/listinfo.html, you probably want to keep those and just add the missing captcha tags.
However, the only issue from those missing tags should be an inability to subscribe via the listinfo page. It shouldn't affect login to the admin or admindb pages and it shouldn't cause a Bad Request error. If after fixing the template, you still can't log in, We would like to see the full traceback from Mailman's logs/error log.
It's possible this is a cPanel issue.
Thank you Mark,
My bad.. rather than Bad Request error (oiy, my memory), the actual error when enabling the RECAPTCHA strings is this:
Bug in Mailman version <undetermined>
We're sorry, we hit a bug!Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs or the web server logs.
So I looked in the mailman logs - I see several logs but the error one did not have any entry from today. Not sure where I would find a pertinent log such as the "web server logs" in the page error above.
What I tried was to hide the lists/[list-name]/en/listinfo.html and replace with the one in templates. I tried without replacing the hidden one first (without yet enacting the RECAPTCHA strings) just to see if the webpage listinfo would load. It did - then tried by replacing it with the templates file, and it also loaded fine. Then I added the RECAPTCHA in the cfg file and that broke the web pages.
I did diff the two listinfo files and it seemed there were just too many differences - practically the whole files.. for me to grok. The listinfo file in the lists directory (rather than templates directory) is probably very old. I started the list I think around 2008 (at least that's how far back for the archives), though I did change server once or twice so not sure if it would have carried along.
So I'm stuck.. I did look in the mailman/logs/subscribe file and boy - lots of nasty spamming going on there. Sure would like to resolve this - might be time to go for the OS upgrade and maybe a try at Mailman v3.
On 12/13/23 17:48, Jim Dory wrote:
My bad.. rather than Bad Request error (oiy, my memory), the actual error when enabling the RECAPTCHA strings is this:
Bug in Mailman version <undetermined>
We're sorry, we hit a bug!
Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs or the web server logs.
So I looked in the mailman logs - I see several logs but the error one did not have any entry from today. Not sure where I would find a pertinent log such as the "web server logs" in the page error above.
The error and traceback should be in /usr/local/cpanel/3rdparty/mailman/logs/error. Is that where you're looking? If you can't find it there, there might be a permissions error.
All of the cgi wrappers in /usr/local/cpanel/3rdparty/mailman/cgi-bin should be SETGID and group mailman and all the files in /usr/local/cpanel/3rdparty/mailman/logs/ should be group writable and group mailman.
There may be cPanel specifics affecting this that I am unaware of, but see https://wiki.list.org/DOC/Mailman%20and%20CPanel
If all else fails you can edit
/usr/local/cpanel/3rdparty/mailman/scripts/driver and set STEALTH_MODE = 0
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12/13/23 17:07, Mark Sapiro wrote:
On 12/13/23 17:48, Jim Dory wrote:
My bad.. rather than Bad Request error (oiy, my memory), the actual error when enabling the RECAPTCHA strings is this:
Bug in Mailman version <undetermined>
We're sorry, we hit a bug!
Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs or the web server logs.
So I looked in the mailman logs - I see several logs but the error one did not have any entry from today. Not sure where I would find a pertinent log such as the "web server logs" in the page error above.
The error and traceback should be in /usr/local/cpanel/3rdparty/mailman/logs/error. Is that where you're looking? If you can't find it there, there might be a permissions error.
All of the cgi wrappers in /usr/local/cpanel/3rdparty/mailman/cgi-bin should be SETGID and group mailman and all the files in /usr/local/cpanel/3rdparty/mailman/logs/ should be group writable and group mailman.
There may be cPanel specifics affecting this that I am unaware of, but see https://wiki.list.org/DOC/Mailman%20and%20CPanel
If all else fails you can edit /usr/local/cpanel/3rdparty/mailman/scripts/driver and set
STEALTH_MODE = 0
So I believe the cgi-bin files are correct - though it is a bit disturbing since they are highlighted in Red in my terminal.
ls -la cgi-bin/ total 316 drwxrwsr-x 2 mailman mailman 4096 Feb 6 2023 . drwxrwsr-x 19 mailman mailman 4096 Dec 13 16:17 .. -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 admin -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 admindb -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 confirm ---------- 1 mailman mailman 25705 Aug 30 2022 create -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 edithtml -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 listinfo -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 options -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 private ---------- 1 mailman mailman 25705 Aug 30 2022 rmlist -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 roster -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 subscribe
The logs are as you say. The /usr/local/cpanel/3rdparty/mailman/logs/error log still nothing for today. Things like:
Dec 03 13:14:26 2023 (68081) private: No such list "xmlrpc.php":
Dec 03 13:14:34 2023 (68083) private: No such list "xmlrpc.php":
Dec 12 07:44:10 2023 (51139) private: No such list "xmlrpc.php":
Dec 12 07:44:10 2023 (51140) listinfo: No such list "xmlrpc.php": Dec 12 07:44:11 2023 (51141) private: No such list "xmlrpc.php":
Dec 12 07:44:11 2023 (51144) listinfo: No such list "xmlrpc.php":
I checked the link you provided and got my hopes up when I saw one guy with a fix for my particular webpage error - but it didn't affect anything when I tried it.
So I'll keep poking around.
Jim
On 12/13/23 18:34, Jim Dory wrote:
On 12/13/23 17:07, Mark Sapiro wrote:
On 12/13/23 17:48, Jim Dory wrote:
My bad.. rather than Bad Request error (oiy, my memory), the actual error when enabling the RECAPTCHA strings is this:
Bug in Mailman version <undetermined>
We're sorry, we hit a bug!
Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs or the web server logs.
So I looked in the mailman logs - I see several logs but the error one did not have any entry from today. Not sure where I would find a pertinent log such as the "web server logs" in the page error above.
The error and traceback should be in /usr/local/cpanel/3rdparty/mailman/logs/error. Is that where you're looking? If you can't find it there, there might be a permissions error.
All of the cgi wrappers in /usr/local/cpanel/3rdparty/mailman/cgi-bin should be SETGID and group mailman and all the files in /usr/local/cpanel/3rdparty/mailman/logs/ should be group writable and group mailman.
There may be cPanel specifics affecting this that I am unaware of, but see https://wiki.list.org/DOC/Mailman%20and%20CPanel
If all else fails you can edit /usr/local/cpanel/3rdparty/mailman/scripts/driver and set
STEALTH_MODE = 0So I believe the cgi-bin files are correct - though it is a bit disturbing since they are highlighted in Red in my terminal.
ls -la cgi-bin/ total 316 drwxrwsr-x 2 mailman mailman 4096 Feb 6 2023 . drwxrwsr-x 19 mailman mailman 4096 Dec 13 16:17 .. -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 admin -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 admindb -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 confirm ---------- 1 mailman mailman 25705 Aug 30 2022 create -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 edithtml -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 listinfo -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 options -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 private ---------- 1 mailman mailman 25705 Aug 30 2022 rmlist -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 roster -rwsr-sr-x 1 mailman mailman 25705 Aug 30 2022 subscribe
This looks good except for no permissions on create and rmlist, but I
suppose that's a cPanel thing because cPanel has it's own way of
creating and removing lists via the control panel. The fact that they
show red is just ls -l emphasizing to you that they are SETGID.
The logs are as you say. The /usr/local/cpanel/3rdparty/mailman/logs/error log still nothing for today. Things like:
Dec 03 13:14:26 2023 (68081) private: No such list "xmlrpc.php":
Dec 03 13:14:34 2023 (68083) private: No such list "xmlrpc.php":
Dec 12 07:44:10 2023 (51139) private: No such list "xmlrpc.php":
Dec 12 07:44:10 2023 (51140) listinfo: No such list "xmlrpc.php": Dec 12 07:44:11 2023 (51141) private: No such list "xmlrpc.php":
Dec 12 07:44:11 2023 (51144) listinfo: No such list "xmlrpc.php":
There should be an entry for every "We hit a bug" instance. The fact that there isn't may also be a cPanel thing.
Under some circumstances, this info can be written the the stderr of the process which should result in it being written to the web server's error log, e.g. for apache this might be /var/log/apache2/error.log. The web server's config at /etc/<name>/* should point to where these logs are.
As I said, you can always, at least temporarily, edit
/usr/local/cpanel/3rdparty/mailman/scripts/driver and set STEALTH_MODE = 0 which should display the error info in your browser.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I've been getting some great advice on trying to block these bots that are hitting our subscription page with random email subscribe requests. It won't be long and I'll have this mailing list black-listed, since I've failed to get any suggestions to work. I think I need to move off this older Centos server.
I will still play around with the captcha solution and see if I was missing anything.. but can't get past a "We hit a bug" when enabled.
I tried firewalld, but locked myself out of the web interface and not real sure how to configure that to work. Could do some more research on it.
Another thought I had (from googling) is to use an .htaccess file. Curious if that would work?
It looks like the webpage I would want an .htaccess file (with offending IP addresses denied) is the [list name]/listinfo.html and curious if I should drop an .htaccess file there and would it work? Or other recommendation(s)? I would do something like: "|deny from 123.456.789.123" for the file without quotes. I'll have to figure out how to add multiple different ip addresses there, if this method would work.|
|thanks, JD |
On 12/15/23 15:51, Jim Dory wrote:
I've been getting some great advice on trying to block these bots that are hitting our subscription page with random email subscribe requests. It won't be long and I'll have this mailing list black-listed, since I've failed to get any suggestions to work. I think I need to move off this older Centos server.
I will still play around with the captcha solution and see if I was missing anything.. but can't get past a "We hit a bug" when enabled.
I tried firewalld, but locked myself out of the web interface and not real sure how to configure that to work. Could do some more research on it.
Another thought I had (from googling) is to use an .htaccess file. Curious if that would work?
It looks like the webpage I would want an .htaccess file (with offending IP addresses denied) is the [list name]/listinfo.html and curious if I should drop an .htaccess file there and would it work? Or other recommendation(s)? I would do something like: "|deny from 123.456.789.123" for the file without quotes. I'll have to figure out how to add multiple different ip addresses there, if this method would work.|
Well, I tried the htaccess by putting them in with the listinfo.html files both under the list name directory and the templates directory. Doesn't work, so far.
On 12/15/23 6:24 PM, Jim Dory wrote:
Well, I tried the htaccess by putting them in with the listinfo.html files both under the list name directory and the templates directory. Doesn't work, so far.
.htaccess would need to go in /usr/local/cpanel/3rdparty/mailman/cgi-bin/
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12/15/23 18:51, Jim Dory wrote:
I tried firewalld, but locked myself out of the web interface and not real sure how to configure that to work. Could do some more research on it.
(IMO firewalld has no place on a server, I'd replace it with plain iptables, but aside from that --) ITYM fail2ban. The problem with bots is if they're DDoS'ing you and not coming from the same source IP, fail2ban won't be much help either.
Fail2ban can parse logs and react to things at higher-than-packet level, but if it's a new IP every time... :(
Dima
On 12/15/23 18:40, Dmitri Maziuk wrote:
On 12/15/23 18:51, Jim Dory wrote:
I tried firewalld, but locked myself out of the web interface and not real sure how to configure that to work. Could do some more research on it.
(IMO firewalld has no place on a server, I'd replace it with plain iptables, but aside from that --) ITYM fail2ban. The problem with bots is if they're DDoS'ing you and not coming from the same source IP, fail2ban won't be much help either.
Fail2ban can parse logs and react to things at higher-than-packet level, but if it's a new IP every time... :(
Dima
Right now it is the same dozen or so IP addresses. I can look at fail2ban.. have it on my personal desktop but not real familiar with its workings.
Jim
Jim Dory writes:
[S]ince I've failed to get any suggestions to work[,] I think I need to move off this older Centos server.
Keeping the server upgraded is always a good idea, but I don't think that is going to help, and it's likely to bring a host of new issues. To slightly misquote from Zork, "you will find yourself in a twisty maze of little passages, all alike." And unlike the situation now, they won't be relevant to solving your spamming problem, just annoying glitches because of the upgrade. Postpone that in favor of locking out the spammers first, IMHO, YMMV.
I will still play around with the captcha solution and see if I was missing anything.. but can't get past a "We hit a bug" when enabled.
I would like to figure why this isn't working for you.
I tried firewalld, but locked myself out of the web interface and not real sure how to configure that to work. Could do some more research on it.
You say elsewhere that you know which IP addresses are currently causing the problem. I'm with Dmitri, just use iptables (or whatever native packet filter is available if it's so old it doesn't have iptables). I assume you just want them to go away period, right? For iptables, just su or sudo and
/usr/sbin/iptables -A INPUT -p tcp -j DROP -s 999.888.777.666/32does the trick for the various bots that were hammering on my Django site.[1][2] You just replace the fake IP address with a malicious address, and if you identify a subnet full of them you can replace the /32 with /24 (my rule is "> 2 from a /24") or even as wide as /16 ("> 5", but I look up the net in whois first to figure out how many).[3] Dmitri is also spot on about using fail2ban and its weakness against somebody with access to even a small botnet.
Another thought I had (from googling) is to use an .htaccess file. Curious if that would work?
It could help, but it suffers from the problem that it tells the miscreants your site is still alive (see also footnote [1]). You could also password all the cgi-bins that are dangerous (eg, subscribe). If they aren't targeting you personally, you can put the password in the listinfo page. If they *are* targeting you personally, they'll social engineer the password out of you, so it won't work.
Footnotes: [1] Use "-j DROP" not "-j REJECT" because the former just ignores those IPs so they won't necessarily realize they've been filtered, while the latter tells the source that you are still alive, you just hate them, which is not what you want if they have it in for you personally.
[2] If you want to check if they go away or something like that you can also log these attempts by duplicating the iptables command but using "-j LOG" instead of "-j DROP" and placing it before the corresponding DROP command. There's probably a better way to do this but it worked for me. (I stopped logging after a while because I never checked the logs.)
[3] You probably want to do this with a file-based approach: iptables-save to get any current rules, add the new rules, then iptables-apply to implement everything at once. Then your site appears to have just dropped off the net to those IPs.
On 12/15/23 23:47, Stephen J. Turnbull wrote:
Jim Dory writes:
[S]ince I've failed to get any suggestions to work[,] I think I need to move off this older Centos server.
Keeping the server upgraded is always a good idea, but I don't think that is going to help, and it's likely to bring a host of new issues. To slightly misquote from Zork, "you will find yourself in a twisty maze of little passages, all alike." And unlike the situation now, they won't be relevant to solving your spamming problem, just annoying glitches because of the upgrade. Postpone that in favor of locking out the spammers first, IMHO, YMMV.
I will still play around with the captcha solution and see if I was missing anything.. but can't get past a "We hit a bug" when enabled.
I would like to figure why this isn't working for you.
I tried firewalld, but locked myself out of the web interface and not real sure how to configure that to work. Could do some more research on it.
You say elsewhere that you know which IP addresses are currently causing the problem. I'm with Dmitri, just use iptables (or whatever native packet filter is available if it's so old it doesn't have iptables). I assume you just want them to go away period, right? For iptables, just su or sudo and
/usr/sbin/iptables -A INPUT -p tcp -j DROP -s 999.888.777.666/32does the trick for the various bots that were hammering on my Django site.[1][2] You just replace the fake IP address with a malicious address, and if you identify a subnet full of them you can replace the /32 with /24 (my rule is "> 2 from a /24") or even as wide as /16 ("> 5", but I look up the net in whois first to figure out how many).[3] Dmitri is also spot on about using fail2ban and its weakness against somebody with access to even a small botnet.
Another thought I had (from googling) is to use an .htaccess file. Curious if that would work?
It could help, but it suffers from the problem that it tells the miscreants your site is still alive (see also footnote [1]). You could also password all the cgi-bins that are dangerous (eg, subscribe). If they aren't targeting you personally, you can put the password in the listinfo page. If they *are* targeting you personally, they'll social engineer the password out of you, so it won't work.
Footnotes: [1] Use "-j DROP" not "-j REJECT" because the former just ignores those IPs so they won't necessarily realize they've been filtered, while the latter tells the source that you are still alive, you just hate them, which is not what you want if they have it in for you personally.
[2] If you want to check if they go away or something like that you can also log these attempts by duplicating the iptables command but using "-j LOG" instead of "-j DROP" and placing it before the corresponding DROP command. There's probably a better way to do this but it worked for me. (I stopped logging after a while because I never checked the logs.)
[3] You probably want to do this with a file-based approach: iptables-save to get any current rules, add the new rules, then iptables-apply to implement everything at once. Then your site appears to have just dropped off the net to those IPs.
Excellent write-up. !!
The .htaccess file could be working.. though I understand about what you say about that. Not sure I'm worried about these bots knowing the site exists - just want to have them stop submitting for subscribing the poor blokes they harvested emails from. But maybe Drop is a better solution.
- So I tried your one line command for dropping on one of the offending ip addresses:
/usr/sbin/iptables -A INPUT -p tcp -j DROP -s 23.94.6.51/32
For logging I am looking at the mailman log "subscribe". Let's me know if successful or not at blocking these guys.
Not sure how to deal with this. I ran the command:
iptables-save
# Generated by iptables-save v1.4.21 on Fri Dec 15 23:58:49 2023 *filter :INPUT ACCEPT [###:###] (I don't know if those numbers are ports to hide) :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [###:####] -A INPUT -s 23.94.6.51/32 -p tcp -j DROP COMMIT # Completed on Fri Dec 15 23:58:49 2023
But don't know how to proceed from that to add the around 16 ip addresses for the -apply all at once. Would you use a text editor or nano or something to create a file, or how would it work? I guess I could head back to duckduckgo for that answer.
Jim Dory writes:
The .htaccess file could be working.. though I understand about what you say about that. Not sure I'm worried about these bots knowing the site exists - just want to have them stop submitting for subscribing the poor blokes they harvested emails from.
If you're not being targeted specifically, that's OK. But if they are, then they'll move to different IP addresses, and carry on using your site to harrass people.
- So I tried your one line command for dropping on one of the offending ip addresses:
/usr/sbin/iptables -A INPUT -p tcp -j DROP -s 23.94.6.51/32
- For logging I am looking at the mailman log "subscribe". Let's me know if successful or not at blocking these guys.
The reason for using the iptables -j LOG target is that this way you know whether they disappeared because they're trapped by iptables, and which rules.
- Not sure how to deal with this. I ran the command:
iptables-save
# Generated by iptables-save v1.4.21 on Fri Dec 15 23:58:49 2023 *filter :INPUT ACCEPT [###:###] (I don't know if those numbers are ports to hide)
Those numbers are counts of packets and bytes that hit each rule.
:FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [###:####] -A INPUT -s 23.94.6.51/32 -p tcp -j DROP
Add the additional addresses here, one line at a time, with the same format (ie, same as the command you would use on the terminal but omit the "iptables" command at the beginning).
Order of options on each line doesn't matter as long as the option "-A" comes immediately before the chain "INPUT", etc. I put -s IP/NETMASK at the end to make it easy to edit the IP/NETMASK argument in any edit that has a "go to end of line" command.
COMMIT # Completed on Fri Dec 15 23:58:49 2023
Use the "-f FILE" option to iptables-save to save to a file, edit as described above, and run "iptables-apply FILE". That's it!
Steve
On 12/16/23 02:47, Stephen J. Turnbull wrote:
You say elsewhere that you know which IP addresses are currently causing the problem. I'm with Dmitri, just use iptables (or whatever native packet filter is available if it's so old it doesn't have iptables). I assume you just want them to go away period, right? For iptables, just su or sudo and
/usr/sbin/iptables -A INPUT -p tcp -j DROP -s 999.888.777.666/32does the trick for the various bots that were hammering on my Django site.
I don't think centos existed back iptables didn't. ;) Firewalld is an attempt to add a "windows defender"-type UI to iptables, it's a wrong tool for this job.
Iptables has a "recent" module (usually is an "extensions" package) that can be used to auto-block IPs that create too many connections too fast (DoS pattern), see e.g. https://unix.stackexchange.com/questions/110453/how-to-block-ssh-brute-force...
Fail2ban monitors log files and can potentially detect more/smarter patterns than just too many connections per time quantum, as the above. It comes with a couple of pre-defined pattern matches for apache (never used those myself): https://www.digitalocean.com/community/tutorials/how-to-protect-an-apache-se...
The main problem with either of the above in this case, is that several subscription requests will get through before the pattern is detected and the bot is blocked.
Captcha would be my first line of defense. I keep hearing about bots getting smarter than humans at solving captchas, and I am getting increasingly more skeptical about captcha "services" by the likes of Google and Cloudflare, so I'd go "belt and suspenders" and add fail2ban for the smart bots that get past captcha.
Dima
On 12/16/23 12:47 AM, Stephen J. Turnbull wrote:
[2] If you want to check if they go away or something like that you can also log these attempts by duplicating the iptables command but using "-j LOG" instead of "-j DROP" and placing it before the corresponding DROP command. There's probably a better way to do this but it worked for me. (I stopped logging after a while because I never checked the logs.)
You don't need anything special. iptables -L -v will list all the
rules together with the packet and byte counters for that rule.
iptables -L -v -n will avoid time consuming rDNS lookups on the IPs.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12/13/23 12:08, Mark Sapiro wrote:
On 12/13/23 10:00, Jim Dory wrote:
As for listinfo.html, I see 2 pertinent files. on under ../en/templates (this list is just english) and under ../lists/[name of our list]/ . The templates version includes a few lines of captcha which the lists version doesn't. Here's a snippet of the templates version:
This is your problem. You have a list specific version of the listinfo.html template in lists/listname/en/listinfo.html which was probably created on an older version before the captchas were implemented.
You need to diff lists/listname/en/listinfo.html with templates/en/listinfo.html. Part of the diff will be the absence of the
<mm-recaptcha-ui> <mm-captcha-ui>
tags in lists/listname/en/listinfo.html which need to be added. If that's the only diff, you can simply remove lists/listname/en/listinfo.html and fall back to the default, but if you had local changes in lists/listname/en/listinfo.html, you probably want to keep those and just add the missing captcha tags.
Hello - started getting someone reporting to yahoo the subscription confirmation email as spam which can cause yahoo to block our list if enough reports come to it. So I've revisited this captcha thing again. I've input the keys into mm_cfg.py and added
<mm-recaptcha-ui> <mm-captcha-ui>
to the lists/listname/listinfo.html. When trying to subscribe, the very obnoxious squares appear asking to check whatever box contains whatever.. many are hard to see. But if I go ahead and check the boxes I get a green checkmark in the I'm not a robot square. I hit subscribe, and I get "reCAPTCHA validation failed: invalid-input-response "
I've tried several times, so not sure if it is because I'm missing the squares in the test or something else is wrong. It would be best if I could just check the box "I'm not a robot" rather than go thru the square puzzle. Possible?
thanks, Jim
On 8/7/24 14:45, Jim Dory wrote:
On 12/13/23 12:08, Mark Sapiro wrote:
On 12/13/23 10:00, Jim Dory wrote:
As for listinfo.html, I see 2 pertinent files. on under ../en/templates (this list is just english) and under ../lists/[name of our list]/ . The templates version includes a few lines of captcha which the lists version doesn't. Here's a snippet of the templates version:
This is your problem. You have a list specific version of the listinfo.html template in lists/listname/en/listinfo.html which was probably created on an older version before the captchas were implemented.
You need to diff lists/listname/en/listinfo.html with templates/en/listinfo.html. Part of the diff will be the absence of the
<mm-recaptcha-ui> <mm-captcha-ui>
tags in lists/listname/en/listinfo.html which need to be added. If that's the only diff, you can simply remove lists/listname/en/listinfo.html and fall back to the default, but if you had local changes in lists/listname/en/listinfo.html, you probably want to keep those and just add the missing captcha tags.
Hello - started getting someone reporting to yahoo the subscription confirmation email as spam which can cause yahoo to block our list if enough reports come to it. So I've revisited this captcha thing again. I've input the keys into mm_cfg.py and added
<mm-recaptcha-ui> <mm-captcha-ui>
to the lists/listname/listinfo.html. When trying to subscribe, the very obnoxious squares appear asking to check whatever box contains whatever.. many are hard to see. But if I go ahead and check the boxes I get a green checkmark in the I'm not a robot square. I hit subscribe, and I get "reCAPTCHA validation failed: invalid-input-response "
I've tried several times, so not sure if it is because I'm missing the squares in the test or something else is wrong. It would be best if I could just check the box "I'm not a robot" rather than go thru the square puzzle. Possible?
thanks, Jim
Perhaps my problem is in needing to follow these instructions - which I haven't done and don't understand:
"Verify the reCAPTCHA token Submit the generated response token to reCAPTCHA for verification. reCAPTCHA returns a risk score indicating the likelihood of a legitimate interaction. Before proceeding, you must authenticate with reCAPTCHA. API requests to reCAPTCHA will fail until authentication steps are complete.
"
I've added an attachment showing part of the message. The rest includes my sitekey, which I'm not sure is safe to post.
On 8/7/24 15:45, Jim Dory wrote:
Hello - started getting someone reporting to yahoo the subscription confirmation email as spam which can cause yahoo to block our list if enough reports come to it. So I've revisited this captcha thing again. I've input the keys into mm_cfg.py and added
<mm-recaptcha-ui> <mm-captcha-ui>
to the lists/listname/listinfo.html. When trying to subscribe, the very obnoxious squares appear asking to check whatever box contains whatever.. many are hard to see. But if I go ahead and check the boxes I get a green checkmark in the I'm not a robot square. I hit subscribe, and I get "reCAPTCHA validation failed: invalid-input-response "
The obnoxious squares are a Google reCaptcha thing. You won't always get them, but Mailman has no control over when you do.
If you get the green check in the I'm not a robot square, your selections were accepted, however Google didn't return success.
Are you using reCaptcha v3. If so, you need v2. Mailman's reCaptcha support requires v2.
Also, V2 may not do the squares thing.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 8/8/24 21:41, Mark Sapiro wrote:
On 8/7/24 15:45, Jim Dory wrote:
Hello - started getting someone reporting to yahoo the subscription confirmation email as spam which can cause yahoo to block our list if enough reports come to it. So I've revisited this captcha thing again. I've input the keys into mm_cfg.py and added
<mm-recaptcha-ui> <mm-captcha-ui>
to the lists/listname/listinfo.html. When trying to subscribe, the very obnoxious squares appear asking to check whatever box contains whatever.. many are hard to see. But if I go ahead and check the boxes I get a green checkmark in the I'm not a robot square. I hit subscribe, and I get "reCAPTCHA validation failed: invalid-input-response "
The obnoxious squares are a Google reCaptcha thing. You won't always get them, but Mailman has no control over when you do.
If you get the green check in the I'm not a robot square, your selections were accepted, however Google didn't return success.
Are you using reCaptcha v3. If so, you need v2. Mailman's reCaptcha support requires v2.
Also, V2 may not do the squares thing.
Thanks Mark. I was aware that we needed V2 so thought I used that. I'll have to double check.. could have easily been led astray.
On 8/9/24 09:44, Jim Dory wrote:
On 8/8/24 21:41, Mark Sapiro wrote:
On 8/7/24 15:45, Jim Dory wrote:
Hello - started getting someone reporting to yahoo the subscription confirmation email as spam which can cause yahoo to block our list if enough reports come to it. So I've revisited this captcha thing again. I've input the keys into mm_cfg.py and added
<mm-recaptcha-ui> <mm-captcha-ui>
to the lists/listname/listinfo.html. When trying to subscribe, the very obnoxious squares appear asking to check whatever box contains whatever.. many are hard to see. But if I go ahead and check the boxes I get a green checkmark in the I'm not a robot square. I hit subscribe, and I get "reCAPTCHA validation failed: invalid-input-response "
The obnoxious squares are a Google reCaptcha thing. You won't always get them, but Mailman has no control over when you do.
If you get the green check in the I'm not a robot square, your selections were accepted, however Google didn't return success.
Are you using reCaptcha v3. If so, you need v2. Mailman's reCaptcha support requires v2.
Also, V2 may not do the squares thing.
Thanks Mark. I was aware that we needed V2 so thought I used that. I'll have to double check.. could have easily been led astray.
Definitely v2. Got another thing to check thru the google site.. I see I have two systems.. the one I just created and an earlier one I suppose. Maybe I have the wrong set of keys? I'll look into that.
Thanks for following up. I have nothing to add to what Mark says in the other branch of the thread, so I will avoid confusion and remain silent for now.
Steve 1
any python modules (I didn't, on a RHEL 7 instance).
Apply for the (very important) v2 captcha keys from Google. Add the following 2 keys to the end of your mm_cfg.py: RECAPTCHA_SITE_KEY = xxxx RECAPTCHA_SECRET_KEY = xxxxxxx
If you have not modified your list_info files, you're done. If you have, then you'll need to add the tags from that above post to your list_info files.
I've started getting these spamming attacks again so thought I would dive into trying this recaptcha. I got the keys for V2 recaptcha from google and put the 2 lines at the bottom of the mm_cfg.py with proper keys from google. Spelling double checked. After saving the file, I can't log into the web interface of mailman - I get a Bad Request error page. I commented out the RECAPTCHA_*_* lines and could then access the admin web pages again.
David wrote that if I modified the list_info files then I needed to add something to them. As far as modifying them, I did add a paragraph description to the "Details for info" and the description under the General Options section of the web admin pages. What would I add and to which files? I don't see list_info under /usr/local/cpanel/3rdparty/mailman/Mailman/ .
I just received a message from the support folks at my hosting that the "We've hit a bug" error I was having resulted in not putting the keys in quotes. So rather than:
RECAPTCHA_SITE_KEY = xxxx RECAPTCHA_SECRET_KEY = xxxxxxx
I should be:
RECAPTCHA_SITE_KEY = "xxxx" RECAPTCHA_SECRET_KEY = "xxxxxxx"
DOH! I haven't tried it again yet.. since the iptables might work for my situation. But I have it in my back pocket now.
JD
- Jim Dory <james@dorydesign.com>:
I've got a new problem with my mailing list. I run a local announcements/trade list that should be of no interest to non-locals.
I started seeing warnings from Yahoo about users marking messages as spam.. (I'm subscribed to Yahoo's Antispam Feedback. I never got other feedback loops from others - like Microsoft - to work). The messages that were being marked as spam by users were the confirmation emails sent by mailman to confirm a signup. To avoid having Yahoo shut down the list as spam (for its subscribers), I set the subscription to be approved by admin, so I could review who was trying to sign up.
Yes, I've ssen that as well. All were coming from the same ip.
-- Ralf Hildebrandt Charité - Universitätsmedizin Berlin Geschäftsbereich IT | Abteilung Netz | Netzwerk-Administration Invalidenstraße 120/121 | D-10115 Berlin
Tel. +49 30 450 570 155 ralf.hildebrandt@charite.de https://www.charite.de
participants (8)
-
David L. Dewey -
ddewey＠cyberthugs.com
-
Dmitri Maziuk -
Jim Dory -
Mark Sapiro -
Ralf Hildebrandt -
richard＠karmannghia.org -
Stephen J. Turnbull