List not archiving, not sending mail, mischief log shows 'Hostile listname: <list>"
Mark,
Adding the line
SUBSCRIBE_FORM_SECRET = 'Some string unique to your site"in mm_cfg.py definitely made a huge difference to the spam registration problem I was having. Looks like I may not need to migrate the list to a new address after all.
One more problem to solve.
I'm using Gmail for all our email addresses, and pulling the mail into Mailman using Fetchmail. It's been working fine for years.
One of my more important lists recently started getting this in the Fetchmail console:
post script, list not found: <listname>
fetchmail: Error writing to MDA: Broken pipeand the Mailman error log says:
<date> gate_news(12061): IOError : [Errno 13] Permission denied:'/opt/local/var/mailman/locks/gate_news.lock.<site FQDN>.12061.0'
<date> post(12068): post script, list not found: <listname>Mailman mischief log says:
Hostile listname: <listname>In the smtp-failure log I'm seeing
Message size exceeds fixed limitIn General Options => Maximum length in kilobytes (KB) of a message body. Use 0 for no limit. I have '0'.
I've run the check permissions script ( with -f), it changes nothing. It appears that there are 5 messages waiting for delivery, though I don't see them in any of the queues.
This appears to have started around the time that I upgraded from Mailman 2.1.17 (or so) to 2.1.20
That list is also not archiving properly, so it clearly has problems
All the other lists on the server are working fine as far as I can tell.
Any ideas as to what's wrong?
On 05/17/2015 01:58 PM, Bill Christensen wrote:
Mark,
Adding the line
SUBSCRIBE_FORM_SECRET = 'Some string unique to your site"in mm_cfg.py definitely made a huge difference to the spam registration problem I was having. Looks like I may not need to migrate the list to a new address after all.
One more problem to solve.
I'm using Gmail for all our email addresses, and pulling the mail into Mailman using Fetchmail. It's been working fine for years.
One of my more important lists recently started getting this in the Fetchmail console:
post script, list not found: <listname> fetchmail: Error writing to MDA: Broken pipeand the Mailman error log says:
<date> gate_news(12061): IOError : [Errno 13] Permission denied:'/opt/local/var/mailman/locks/gate_news.lock.<site FQDN>.12061.0'
This is probably not related. Do you even have any lists with Mail<->News gateways -> gateway_to_mail = Yes?. Mailman's cron/gate_news runs by default every five minutes and does use this lock to prevent concurrent updates. I would also expect to see entries like 'Could not acquire gate_news lock' in the fromusenet log.
If this is a one-time occurrence, It was just a glitch. Maybe someone manually ran cron/gate_news as a user not in Mailman's group. If you get these every 5 minutes, there is a permissions issue, probably with the user running this cron not being mailman, because if the locks directory were not group mailman and group writable, lots more would be broken.
<date> post(12068): post script, list not found: <listname>
The post script invoked by
/path/to/mailman/mail/mailman post <listname>
in your fetchmail script says <listname> doesn't exist.
Mailman mischief log says:
Hostile listname: <listname>
This is consistent with the 'post script' report and says that <listname> contains one or more characters not in the Defaults.py/mm_cfg.py setting ACCEPTABLE_LISTNAME_CHARACTERS. The default setting only allows letters, digits, '-', '+', '_', '.' and '='.
Does <listname> contain characters outside this set or whatever set you've specified?
In the smtp-failure log I'm seeing
Message size exceeds fixed limitIn General Options => Maximum length in kilobytes (KB) of a message body. Use 0 for no limit. I have '0'.
These two things are unrelated. The message in smtp-failure says Mailman attempted to send a message which was larger than the MTA would accept. In Postfix for example, this is the main.cf parameter message_size_limit.
I've run the check permissions script ( with -f), it changes nothing. It appears that there are 5 messages waiting for delivery, though I don't see them in any of the queues.
If Mailman is retrying the smtp-failure, the message would be in Mailman's 'retry' queue until Mailman eventually gives up. Or, they could be still in the gmail inbox since fetchmail probably wouldn't delete them as it couldn't deliver them.
This appears to have started around the time that I upgraded from Mailman 2.1.17 (or so) to 2.1.20
That list is also not archiving properly, so it clearly has problems
The listname problem would mess up a lot, but it looks like there would be nothing to archive since nothing for that list is getting posted.
All the other lists on the server are working fine as far as I can tell.
Any ideas as to what's wrong?
First, what's the list name? Once you fix that issue, that may be all, or there may be shunted messages in Mailman's shunt queue to be unshunted.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, May 17, 2015 at 6:26 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 05/17/2015 01:58 PM, Bill Christensen wrote:
I'm using Gmail for all our email addresses, and pulling the mail into Mailman using Fetchmail. It's been working fine for years.
One of my more important lists recently started getting this in the Fetchmail console:
post script, list not found: <listname> fetchmail: Error writing to MDA: Broken pipeand the Mailman error log says:
<date> gate_news(12061): IOError : [Errno 13] Permission denied:'/opt/local/var/mailman/locks/gate_news.lock.<site FQDN>.12061.0'
This is probably not related. Do you even have any lists with Mail<->News gateways -> gateway_to_mail = Yes?. Mailman's cron/gate_news runs by default every five minutes and does use this lock to prevent concurrent updates. I would also expect to see entries like 'Could not acquire gate_news lock' in the fromusenet log.
No news gateways.
If this is a one-time occurrence, It was just a glitch. Maybe someone manually ran cron/gate_news as a user not in Mailman's group. If you get these every 5 minutes, there is a permissions issue, probably with the user running this cron not being mailman, because if the locks directory were not group mailman and group writable, lots more would be broken.
Yes, definitely every 5 minutes.
<date> post(12068): post script, list not found: <listname>The post script invoked by
/path/to/mailman/mail/mailman post <listname>
in your fetchmail script says <listname> doesn't exist.
Mailman mischief log says:
Hostile listname: <listname>This is consistent with the 'post script' report and says that <listname> contains one or more characters not in the Defaults.py/mm_cfg.py setting ACCEPTABLE_LISTNAME_CHARACTERS. The default setting only allows letters, digits, '-', '+', '_', '.' and '='.
Does <listname> contain characters outside this set or whatever set you've specified?
Nope, it's four uppercase letters. No spaces, no punctuation.
In the smtp-failure log I'm seeing
Message size exceeds fixed limitIn General Options => Maximum length in kilobytes (KB) of a message body. Use 0 for no limit. I have '0'.
These two things are unrelated. The message in smtp-failure says Mailman attempted to send a message which was larger than the MTA would accept. In Postfix for example, this is the main.cf parameter message_size_limit.
Ok.
I checked the messages waiting in Gmail. They're all small enough, no worrisome attachments or anything. I tried marking the oldest as read just in case it was jamming up the works, but the others are still in a logjam.
I've run the check permissions script ( with -f), it changes nothing. It appears that there are 5 messages waiting for delivery, though I don't see them in any of the queues.
If Mailman is retrying the smtp-failure, the message would be in Mailman's 'retry' queue until Mailman eventually gives up. Or, they could be still in the gmail inbox since fetchmail probably wouldn't delete them as it couldn't deliver them.
Yes, there are still 5 messages waiting in Gmail. Fetchmail apparently hasn't marked them as read as it can't put them anywhere.
This appears to have started around the time that I upgraded from Mailman 2.1.17 (or so) to 2.1.20
That list is also not archiving properly, so it clearly has problems
The listname problem would mess up a lot, but it looks like there would be nothing to archive since nothing for that list is getting posted.
Makes sense.
All the other lists on the server are working fine as far as I can tell.
Any ideas as to what's wrong?
First, what's the list name? Once you fix that issue, that may be all, or there may be shunted messages in Mailman's shunt queue to be unshunted.
As far as I can tell, the list name is correct. I checked
bin/list_lists
/etc/aliases /etc/postfix/virtual /etc/fetchmailrc and ran sudo newaliases
it was lower case in a few places, so I changed those to upper case to match, just in case.
then stopped/started Mailman. No change.
I decided to take a chance and used your clone_list script to make a copy, moved the original archives to a backup location, then removed the original list and cloned back from the copy to the original name.
Ran check_perms -f and checked them manually as well. All looks correct.
And yet, it's still throwing the same errors.
One improvement - after correcting permissions manually on <listname>.mbox/<listname>.mbox I can get to the archives again. They were -rw-rw-r-- 1 root staff and I set them to -rwxrwxr-x 1 root staff to match other working lists with public archives.
Then I looked at mailman/lists/. The list in question was owned by my user, not root like the rest of them (possibly a result of clone list? the only two with that user were this list and the clone I had made earlier). Changed that to match the rest.
Ownership of the enclosed files has me confused. All have the following files
config.pck
config.pck.last
pending.pck
request.pck
some lists have some files with owner = _www, other files with owner = _mailman and still others lists have files with owner = my user . They're not consistent and I'm not really sure what they're all supposed to be, so I'd appreciate it if you'd enlighten me.
Some also have 'en' directories, and I'm similarly unsure of what the correct owners are for those and their enclosed files - but I'm pretty sure they're probably not supposed to be my user.
Next steps?
On 05/17/2015 10:14 PM, Bill Christensen wrote:
On Sun, May 17, 2015 at 6:26 PM, Mark Sapiro <mark@msapiro.net> wrote:
If this is a one-time occurrence, It was just a glitch. Maybe someone manually ran cron/gate_news as a user not in Mailman's group. If you get these every 5 minutes, there is a permissions issue, probably with the user running this cron not being mailman, because if the locks directory were not group mailman and group writable, lots more would be broken.
Yes, definitely every 5 minutes.
There is a permissions issue on the locks directory. If you do
ls -la /opt/local/var/mailman/locks/
you should see something similar to
drwxrwsr-x 2 root mailman 4096 May 18 20:11 . drwxrwsr-x 9 root mailman 4096 May 10 2008 .. -rw-rw-r-- 2 mailman mailman 62 May 19 2015 master-qrunner -rw-rw-r-- 2 mailman mailman 62 May 19 2015 master-qrunner.host.pid
There may or may not be other files, but the two master-qrunner files should be there and everything should be mailman's group (owner is not important) and both . and .. should have permissions as shown.
If all this is correct, then either cron/gate_news is not running as the Mailman user (where is its crontab and what's in it?) or there may be some SELinux, apparmor or other security policy issue.
<date> post(12068): post script, list not found: <listname>The post script invoked by
/path/to/mailman/mail/mailman post <listname>
in your fetchmail script says <listname> doesn't exist.
Mailman mischief log says:
Hostile listname: <listname>This is consistent with the 'post script' report and says that <listname> contains one or more characters not in the Defaults.py/mm_cfg.py setting ACCEPTABLE_LISTNAME_CHARACTERS. The default setting only allows letters, digits, '-', '+', '_', '.' and '='.
Does <listname> contain characters outside this set or whatever set you've specified?
Nope, it's four uppercase letters. No spaces, no punctuation.
List names are always all lower-case. The list may have a real_name attribute which is mixed case but it must differ only in case from the list's internal name.
The name in the pipe to 'mailman/mail/mailman post <listname>' command MUST be all lower case.
I checked the messages waiting in Gmail. They're all small enough, no worrisome attachments or anything. I tried marking the oldest as read just in case it was jamming up the works, but the others are still in a logjam.
Because fetchmail can't deliver to the list until you fix the listname you pass to the 'mailman' wrapper if that's the issue.
Yes, there are still 5 messages waiting in Gmail. Fetchmail apparently hasn't marked them as read as it can't put them anywhere.
Yes.
As far as I can tell, the list name is correct. I checked
bin/list_lists
/etc/aliases /etc/postfix/virtual /etc/fetchmailrc and ran sudo newaliases
it was lower case in a few places, so I changed those to upper case to match, just in case.
That was the reverse of what you should have done. Hopefilly, the name of the list's directory in the /opt/local/var/mailman/lists/ directory is all lower case.
It must also be all lower case in fetchmailrc and in the commands in /etc/aliases.
I.e. an alias could be
LISTNAME: '|/path/to/mailman/mail/mailman post listname' LISTNAME-admin: '|/path/to/mailman/mail/mailman admin listname'
But it should match /etc/postfix/virtual, but it's best if it's all lower case.
But then if fetchmail is delivering directly to /path/to/mailman/mail/mailman, none of the postfix stuff matters as it isn't used.
then stopped/started Mailman. No change.
I decided to take a chance and used your clone_list script to make a copy, moved the original archives to a backup location, then removed the original list and cloned back from the copy to the original name.
Ran check_perms -f and checked them manually as well. All looks correct.
And yet, it's still throwing the same errors.
One improvement - after correcting permissions manually on <listname>.mbox/<listname>.mbox I can get to the archives again. They were -rw-rw-r-- 1 root staff and I set them to -rwxrwxr-x 1 root staff to match other working lists with public archives.
It's a file. +x is not necessary on files. archives/private/<listname>.mbox should be drwxrwsr-x (note the SETGID), but no x of the file is needed (it's not executable). And I hope <listname> is all lower case.
Also, it appears that Mailman's group is 'staff', not 'mailman' so everywhere I refer to it as 'mailman', read 'staff'
Then I looked at mailman/lists/. The list in question was owned by my user, not root like the rest of them (possibly a result of clone list? the only two with that user were this list and the clone I had made earlier). Changed that to match the rest.
Ownership of the enclosed files has me confused. All have the following files
config.pck
config.pck.last
pending.pck
request.pck
some lists have some files with owner = _www, other files with owner = _mailman and still others lists have files with owner = my user . They're not consistent and I'm not really sure what they're all supposed to be, so I'd appreciate it if you'd enlighten me.
It doesn't matter. It is only the group that's important. The owner will reflect the user/process that created it, but in Mailman all the directories are SETGID and thus, things are created with the group of the containing directory, and permissions are by group. There are a couple of places where owner might matter. These are the owner of the aliases.db file with mailman's aliases and the owner of the archives/private/ directory itself if that directory is not o+x, but everywhere else, owner doesn't matter.
But if _mailman is an owner, that says to me FreeBSD or maybe Darwin, and I would have thought Mailman's group would also be _mailman, not staff.
Some also have 'en' directories, and I'm similarly unsure of what the correct owners are for those and their enclosed files - but I'm pretty sure they're probably not supposed to be my user.
The en/ directories contain list specific templates, and again, the owner doesn't matter as long as everything is in Mailman's group.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Great! I fixed the list name uppercase problem and enough of the permissions and group assignments to get things running again last night.
I still need to comb through and make sure that everything that's supposed to be is in the Mailman group, but that'll happen soon and the other major lists we run continue to work as intended.
Thanks for your help!
On Mon, May 18, 2015 at 11:07 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 05/17/2015 10:14 PM, Bill Christensen wrote:
On Sun, May 17, 2015 at 6:26 PM, Mark Sapiro <mark@msapiro.net> wrote:
If this is a one-time occurrence, It was just a glitch. Maybe someone manually ran cron/gate_news as a user not in Mailman's group. If you get these every 5 minutes, there is a permissions issue, probably with the user running this cron not being mailman, because if the locks directory were not group mailman and group writable, lots more would be broken.
Yes, definitely every 5 minutes.
There is a permissions issue on the locks directory. If you do
ls -la /opt/local/var/mailman/locks/
you should see something similar to
drwxrwsr-x 2 root mailman 4096 May 18 20:11 . drwxrwsr-x 9 root mailman 4096 May 10 2008 .. -rw-rw-r-- 2 mailman mailman 62 May 19 2015 master-qrunner -rw-rw-r-- 2 mailman mailman 62 May 19 2015 master-qrunner.host.pid
There may or may not be other files, but the two master-qrunner files should be there and everything should be mailman's group (owner is not important) and both . and .. should have permissions as shown.
If all this is correct, then either cron/gate_news is not running as the Mailman user (where is its crontab and what's in it?) or there may be some SELinux, apparmor or other security policy issue.
<date> post(12068): post script, list not found: <listname>The post script invoked by
/path/to/mailman/mail/mailman post <listname>
in your fetchmail script says <listname> doesn't exist.
Mailman mischief log says:
Hostile listname: <listname>This is consistent with the 'post script' report and says that <listname> contains one or more characters not in the Defaults.py/mm_cfg.py setting ACCEPTABLE_LISTNAME_CHARACTERS. The default setting only allows letters, digits, '-', '+', '_', '.' and '='.
Does <listname> contain characters outside this set or whatever set you've specified?
Nope, it's four uppercase letters. No spaces, no punctuation.
List names are always all lower-case. The list may have a real_name attribute which is mixed case but it must differ only in case from the list's internal name.
The name in the pipe to 'mailman/mail/mailman post <listname>' command MUST be all lower case.
I checked the messages waiting in Gmail. They're all small enough, no worrisome attachments or anything. I tried marking the oldest as read just in case it was jamming up the works, but the others are still in a logjam.
Because fetchmail can't deliver to the list until you fix the listname you pass to the 'mailman' wrapper if that's the issue.
Yes, there are still 5 messages waiting in Gmail. Fetchmail apparently hasn't marked them as read as it can't put them anywhere.
Yes.
As far as I can tell, the list name is correct. I checked
bin/list_lists
/etc/aliases /etc/postfix/virtual /etc/fetchmailrc and ran sudo newaliases
it was lower case in a few places, so I changed those to upper case to match, just in case.
That was the reverse of what you should have done. Hopefilly, the name of the list's directory in the /opt/local/var/mailman/lists/ directory is all lower case.
It must also be all lower case in fetchmailrc and in the commands in /etc/aliases.
I.e. an alias could be
LISTNAME: '|/path/to/mailman/mail/mailman post listname' LISTNAME-admin: '|/path/to/mailman/mail/mailman admin listname'
But it should match /etc/postfix/virtual, but it's best if it's all lower case.
But then if fetchmail is delivering directly to /path/to/mailman/mail/mailman, none of the postfix stuff matters as it isn't used.
then stopped/started Mailman. No change.
I decided to take a chance and used your clone_list script to make a copy, moved the original archives to a backup location, then removed the original list and cloned back from the copy to the original name.
Ran check_perms -f and checked them manually as well. All looks correct.
And yet, it's still throwing the same errors.
One improvement - after correcting permissions manually on <listname>.mbox/<listname>.mbox I can get to the archives again. They were -rw-rw-r-- 1 root staff and I set them to -rwxrwxr-x 1 root staff to match other working lists with public archives.
It's a file. +x is not necessary on files. archives/private/<listname>.mbox should be drwxrwsr-x (note the SETGID), but no x of the file is needed (it's not executable). And I hope <listname> is all lower case.
Also, it appears that Mailman's group is 'staff', not 'mailman' so everywhere I refer to it as 'mailman', read 'staff'
Then I looked at mailman/lists/. The list in question was owned by my user, not root like the rest of them (possibly a result of clone list?
the
only two with that user were this list and the clone I had made earlier). Changed that to match the rest.
Ownership of the enclosed files has me confused. All have the following files
config.pck
config.pck.last
pending.pck
request.pck
some lists have some files with owner = _www, other files with owner = _mailman and still others lists have files with owner = my user . They're not consistent and I'm not really sure what they're all supposed to be, so I'd appreciate it if you'd enlighten me.
It doesn't matter. It is only the group that's important. The owner will reflect the user/process that created it, but in Mailman all the directories are SETGID and thus, things are created with the group of the containing directory, and permissions are by group. There are a couple of places where owner might matter. These are the owner of the aliases.db file with mailman's aliases and the owner of the archives/private/ directory itself if that directory is not o+x, but everywhere else, owner doesn't matter.
But if _mailman is an owner, that says to me FreeBSD or maybe Darwin, and I would have thought Mailman's group would also be _mailman, not staff.
Some also have 'en' directories, and I'm similarly unsure of what the correct owners are for those and their enclosed files - but I'm pretty sure they're probably not supposed to be my user.
The en/ directories contain list specific templates, and again, the owner doesn't matter as long as everything is in Mailman's group.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/billc_lists%40greenbui...
On 05/19/2015 07:19 PM, Bill Christensen wrote:
Great! I fixed the list name uppercase problem and enough of the permissions and group assignments to get things running again last night.
I still need to comb through and make sure that everything that's supposed to be is in the Mailman group, but that'll happen soon and the other major lists we run continue to work as intended.
I know you said in your original post that you'd "run the check permissions script ( with -f), it changes nothing." Did you run it as root? It should fix most things with ownership and permission.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 05/17/2015 10:14 PM, Bill Christensen wrote:
On Sun, May 17, 2015 at 6:26 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 05/17/2015 01:58 PM, Bill Christensen wrote:
In the smtp-failure log I'm seeing
Message size exceeds fixed limitIn General Options => Maximum length in kilobytes (KB) of a message body. Use 0 for no limit. I have '0'.
These two things are unrelated. The message in smtp-failure says Mailman attempted to send a message which was larger than the MTA would accept. In Postfix for example, this is the main.cf parameter message_size_limit.
Ok.
I checked the messages waiting in Gmail. They're all small enough, no worrisome attachments or anything. I tried marking the oldest as read just in case it was jamming up the works, but the others are still in a logjam.
The Message size exceeds fixed limit errors have nothing to do with the list messages that are not getting posted. They are messages being sent from Mailman to the MTA (Postfix ?) that are bigger than Postfix's message_size_limit.
You'll have to look at the Postfix log for more info.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Bill Christensen -
Mark Sapiro