Virtual Domains for Mailing Lists in MailMan
Hello
I found this E-Mail address on the MailMan homepage and decided to
contact you as I don't seem to be able to locate the information I
need elsewhere.
To explain my circumstance. I am a visually impaired person, trying
to get to grips with administrating a mail server under Apple Mac OS X
10.5.7 (Leopard) server.
I need to host mailing lists under virtual domains hosted on our
server but can't see quite how this is accomplished.
Is there anywhere I can obtain this information as Apple's
documentation on this subject seems poor from what I've seen.
I really do apologise for any inconvenience I may cause you by
contacting you - but if you could point me to the relevant information
pages as to how to accomplish virtual domain list configuration under
Apple's server platform I'd be very grateful.
As a blind person this information is not always easy for me to
locate. Thank you in advance for any help.
Gordon
Gordon Smith wrote:
To explain my circumstance. I am a visually impaired person, trying to get to grips with administrating a mail server under Apple Mac OS X 10.5.7 (Leopard) server.
I need to host mailing lists under virtual domains hosted on our server but can't see quite how this is accomplished.
Is there anywhere I can obtain this information as Apple's documentation on this subject seems poor from what I've seen.
Gordon,
Unfortunately, our documentation of virtual hosts with Mailman is probably no better than Apple's. Also, I'm not sure how accessible it is to a visually impaired person. Finally, most of us know nothing about Apple's GUI for Mailman. We are familiar only with our own web interface and configuration methods.
That said, virtual hosting within Mailman is pretty simple. Mailman keeps a web host and an email host for each list, so list's can have email addresses and or web addresses in different domains. There is one possibly major restriction. List names must be globally unique within an installation.
Within Mailman, there is very little configuration for virtual hosts. There is a DEFAULT_URL_HOST and a corresponding DEFAULT_EMAIL_HOST. Beyond that, you add additional hosts with entries like
add_virtualhost('www.example.com', 'mail.example.com')
in mm_cfg.py for each host. The purpose of this is primarily so when you go to a URL like http://www.example.com/mailman/create to create a list, it gets created with the corresponding email host.
There is also a bit more configuration if Postfix is your MTA to make Postfix integration work with virtual domains. This is described in section 6.1 of the installation manual at http://www.list.org/mailman-install/index.html.
The rest is in the MTA, and possibly the web server, although (assuming Apache) if the Mailman specific Alias, ScriptAlias and Directory settings are in a global section of the configuration, the web server will probably work OK without change.
Most of the documentation we have is in the above referenced installation manual and the FAQ at http://www.list.org/mailman-install/index.html
Please don't hesitate to post any followup questions to mailman-users@python.org.
Gordon,
I happened to come across your post while trying to accomplish exactly the same thing: mailman virtual domains under OS X Server 10.5.7. Fun, isn't it?
I've taken the step of downloading and installing the standard, current mailman distribution today, and have successfully compiled a configuration that mimics Apple's various installation locations.
Now let's see if I can get it to run, or accomplish anything useful. Right now it's startup endless respawns, with no useful error messages in any log I can find.
But perhaps we can assist each other. I'm expert on neither Postfix nor Mailman, but may have to become one by the end of the week.
-Bryan
rbryanh wrote:
I've taken the step of downloading and installing the standard, current mailman distribution today, and have successfully compiled a configuration that mimics Apple's various installation locations.
See the FAQ at http://wiki.list.org/x/O4A9 for information on installing a standard Mailman distribution on Mac OS X.
Now let's see if I can get it to run, or accomplish anything useful. Right now it's startup endless respawns, with no useful error messages in any log I can find.
Mailman's logs are in the logs/ directory in the 'var-prefix' directory. There should be useful messages in the 'error' and 'qrunner' logs.
Mark,
Thank you.
I've found the resources you recommended helpful, but am stymied
nonetheless. I'm bright but not expert with Postfix or Mailman, and
have been butting my head against this for days.
As a test, I'm trying to install mailman on the server gilded-
bat.laughingboot.net, and create functioning lists mailman@laughingboot.net
and galaxy@gurgitate.org. Virtual domains are humming along nicely
in both Apache and Postfix.
Mailman's processes run and I can browse the interface site, but all
my tests now bounce with:
<"|/usr/share/mailman/mail/mailman post galaxy"@gilded- bat.laughingboot.net>: data format error. Command output: |/usr/share/mailman/mail/ mailman post galaxy: Bad protocol
If you're feeling particularly saintly, I've P.S.'d install_mailman.sh
below the very rough script I'm using to install and configure
mailman.
Relevant lines from main.cf are
virtual_alias_maps = hash:/etc/postfix/virtual,hash:/var/mailman/data/ aliases,hash:/var/mailman/data/virtual-mailman virtual_mailbox_domains = hash:/etc/postfix/virtual_domains
mm_cfg.pys looks like this:
# Include Defaults:
from Defaults import *
# Site-specific overrides:
MTA = 'Postfix' DEFAULT_EMAIL_HOST = 'laughingboot.net' DEFAULT_URL_HOST = 'gilded-bat.laughingboot.net' add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST) add_virtualhost('gurgitate.org', 'gurgitate.org') POSTFIX_STYLE_VIRTUAL_DOMAINS = 'gurgitate.org' DEFAULT_MAX_MESSAGE_SIZE = 10000 # KB
I'm running OS X Server 10.5.7.
I'll burn an SUV to the Gods on your behalf if you care to help me out.
Best regards, Bryan
# !/bin/tcsh # INSTALL MAILMAN TEST # Last Updated: 2009-06-15 # Assumes prior installation of Xcode.
serveradmin stop mail /usr/share/mailman/bin/mailmanctl stop
rm -Rf /var/mailman/* rm -Rf /usr/share/mailman/* mkdir -p -m ug=rwx,o=rx /var/mailman mkdir -p -m a+rx,g+ws /usr/share/mailman chown _mailman:_mailman /var/mailman chown _mailman:_mailman /usr/share/mailman chmod -R 2777 /var/mailman chmod -R 2777 /usr/share/mailman
mkdir -p /Library/TemporaryItems cd /Library/TemporaryItems
curl -O http://superb-west.dl.sourceforge.net/sourceforge/mailman/mailman-2.1.12.tgz tar -xzf mailman-2.1.12.tgz cd mailman-2.1.12
# Pick One, anyone, none of them work:
# ./configure --prefix=/usr/share/mailman --with-var-prefix=/var/
mailman --with-username=_mailman --with-groupname=_mailman --with-cgi-
gid=_www
# ./configure --prefix=/usr/share/mailman --with-var-prefix=/var/
mailman --with-username=_mailman --with-groupname=_mailman --with-mail-
gid=_mailman --with-cgi-gid=_www --without-permcheck
./configure --prefix=/usr/share/mailman --with-var-prefix=/var/mailman
--with-username=_mailman --with-groupname=_mailman --with-mail-
gid=_postfix --with-cgi-gid=_www --without-permcheck
make make install
chown _www:_mailman /var/mailman/archives/private chmod o-r-x /var/mailman/archives/private touch /var/mailman/data/aliases touch /var/mailman/data/virtual-mailman scp #copy mm_cfg.py from where I'm working on it to its proper home.
/usr/share/mailman/bin/newlist -q mailman bryan@laughingboot.net
mypassword
/usr/share/mailman/bin/config_list -i /var/mailman/data/sitelist.cfg
mailman
/usr/share/mailman/bin/newlist -q galaxy bryan@gurgitate.org mypassword
/usr/share/mailman/bin/config_list -i /var/mailman/data/sitelist.cfg
galaxy
/usr/share/mailman/bin/withlist -l -r fix_url galaxy --
urlhost=gurgitate.org
cd /usr/share/mailman/bin ./genaliases serveradmin start mail ./mailmanctl -s start ./check_perms -f
rm -R /Library/TemporaryItems/
P.S. "error" is empty, and "qrunner" says
Jun 15 16:01:12 2009 (7287) BounceRunner qrunner started. Jun 15 16:01:13 2009 (7293) RetryRunner qrunner started. Jun 15 16:01:13 2009 (7288) CommandRunner qrunner started. Jun 15 16:01:13 2009 (7292) VirginRunner qrunner started. Jun 15 16:01:13 2009 (7286) ArchRunner qrunner started. Jun 15 16:01:13 2009 (7290) NewsRunner qrunner started. Jun 15 16:01:13 2009 (7289) IncomingRunner qrunner started. Jun 15 16:01:13 2009 (7291) OutgoingRunner qrunner started.
Bryan Harrison wrote:
P.S. "error" is empty, and "qrunner" says
Jun 15 16:01:12 2009 (7287) BounceRunner qrunner started. Jun 15 16:01:13 2009 (7293) RetryRunner qrunner started. Jun 15 16:01:13 2009 (7288) CommandRunner qrunner started. Jun 15 16:01:13 2009 (7292) VirginRunner qrunner started. Jun 15 16:01:13 2009 (7286) ArchRunner qrunner started. Jun 15 16:01:13 2009 (7290) NewsRunner qrunner started. Jun 15 16:01:13 2009 (7289) IncomingRunner qrunner started. Jun 15 16:01:13 2009 (7291) OutgoingRunner qrunner started.
This doesn't look like "Right now it's startup endless respawns" to me. What did you mean by that?
Bryan Harrison wrote:
I've found the resources you recommended helpful, but am stymied
nonetheless. I'm bright but not expert with Postfix or Mailman, and
have been butting my head against this for days.As a test, I'm trying to install mailman on the server gilded- bat.laughingboot.net, and create functioning lists mailman@laughingboot.net and galaxy@gurgitate.org. Virtual domains are humming along nicely
in both Apache and Postfix.Mailman's processes run and I can browse the interface site, but all
my tests now bounce with:<"|/usr/share/mailman/mail/mailman post galaxy"@gilded- bat.laughingboot.net>: data format error. Command output: |/usr/share/mailman/mail/ mailman post galaxy: Bad protocol
See below
If you're feeling particularly saintly, I've P.S.'d install_mailman.sh
below the very rough script I'm using to install and configure
mailman.Relevant lines from main.cf are
virtual_alias_maps = hash:/etc/postfix/virtual,hash:/var/mailman/data/ aliases,hash:/var/mailman/data/virtual-mailman
The entry "hash:/var/mailman/data/aliases" should be in alias_maps, not virtual_alias_maps.
virtual_mailbox_domains = hash:/etc/postfix/virtual_domains
Your Mailman domains should be virtual_alias_domains. virtual_mailbox_domains can't deliver properly to Mailman aliases.
mm_cfg.pys looks like this:
# Include Defaults:
from Defaults import *
# Site-specific overrides:
MTA = 'Postfix' DEFAULT_EMAIL_HOST = 'laughingboot.net' DEFAULT_URL_HOST = 'gilded-bat.laughingboot.net'
Probably not absolutely necessary, but put
VIRTUAL_HOSTS.clear()
here to remove the add_virtualhosts entry from Defaults.py.
add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST) add_virtualhost('gurgitate.org', 'gurgitate.org') POSTFIX_STYLE_VIRTUAL_DOMAINS = 'gurgitate.org' DEFAULT_MAX_MESSAGE_SIZE = 10000 # KB
I'm running OS X Server 10.5.7.
I'll burn an SUV to the Gods on your behalf if you care to help me out.
Best regards, Bryan
# !/bin/tcsh # INSTALL MAILMAN TEST # Last Updated: 2009-06-15 # Assumes prior installation of Xcode.
serveradmin stop mail /usr/share/mailman/bin/mailmanctl stop
rm -Rf /var/mailman/* rm -Rf /usr/share/mailman/* mkdir -p -m ug=rwx,o=rx /var/mailman mkdir -p -m a+rx,g+ws /usr/share/mailman chown _mailman:_mailman /var/mailman chown _mailman:_mailman /usr/share/mailman chmod -R 2777 /var/mailman chmod -R 2777 /usr/share/mailman
These are a bit permissive - should be 2775.
mkdir -p /Library/TemporaryItems cd /Library/TemporaryItems
curl -O http://superb-west.dl.sourceforge.net/sourceforge/mailman/mailman-2.1.12.tgz tar -xzf mailman-2.1.12.tgz cd mailman-2.1.12
# Pick One, anyone, none of them work:
# ./configure --prefix=/usr/share/mailman --with-var-prefix=/var/ mailman --with-username=_mailman --with-groupname=_mailman --with-cgi- gid=_www # ./configure --prefix=/usr/share/mailman --with-var-prefix=/var/ mailman --with-username=_mailman --with-groupname=_mailman --with-mail- gid=_mailman --with-cgi-gid=_www --without-permcheck
The one above should be correct. With Postfix integration, the "mail-gid" is the primary group of the owner of /var/mailman/data/aliases.db which should be _mailman. It shouldn't need --without-permcheck
./configure --prefix=/usr/share/mailman --with-var-prefix=/var/mailman
--with-username=_mailman --with-groupname=_mailman --with-mail- gid=_postfix --with-cgi-gid=_www --without-permcheckmake make install
chown _www:_mailman /var/mailman/archives/private chmod o-r-x /var/mailman/archives/private touch /var/mailman/data/aliases touch /var/mailman/data/virtual-mailman
The above two touch shouldn't be needed.
scp #copy mm_cfg.py from where I'm working on it to its proper home.
/usr/share/mailman/bin/newlist -q mailman bryan@laughingboot.net
mypassword /usr/share/mailman/bin/config_list -i /var/mailman/data/sitelist.cfg
mailman/usr/share/mailman/bin/newlist -q galaxy bryan@gurgitate.org mypassword /usr/share/mailman/bin/config_list -i /var/mailman/data/sitelist.cfg
galaxy
/var/mailman/data/sitelist.cfg is only intended for the site list named 'mailman' (unless you set something different for MAILMAN_SITE_LISTin mm_cfg.py). You need something like
/usr/share/mailman/bin/newlist -q mailman bryan@gurgitate.org mypassword /usr/share/mailman/bin/config_list -i /var/mailman/data/sitelist.cfg mailman
Yo don't want the config_list for other lists.
/usr/share/mailman/bin/withlist -l -r fix_url galaxy -- urlhost=gurgitate.org
Instead of the above, just create it with
/usr/share/mailman/bin/newlist -q -u gurgitate.org galaxy bryan@gurgitate.org mypassword
cd /usr/share/mailman/bin ./genaliases
If you create the lists in the proper domain with newlist, you don't need genaliases at this point.
serveradmin start mail ./mailmanctl -s start ./check_perms -f
Do the check_perms before the mailmanctl start
rm -R /Library/TemporaryItems/
I've built and installed Mailman 2.12 from current source code and
have it running just fine under OS X Server 10.5.7, with on exception,
which is not properly speaking a Mailman problem at all.
If I start Mailman from the CLI, using /usr/share/mailman/bin/ mailmanctl -s start, all is well.
If I allow launchd to start it using the following Launch Daemon…
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd ">
<plist version="1.0"> <dict> <key>Label</key> <string>org.list.mailmanctl</string> <key>OnDemand</key> <false/> <key>Program</key> <string>/usr/share/mailman/bin/mailmanctl</string> <key>ProgramArguments</key> <array> <string>mailmanctl</string> <string>-s</string> <string>start</string> </array> <key>ServiceIPC</key> <false/> </dict> </plist>
…I get an endless succession of…
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master
qrunner.
Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]):
Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python
Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling
respawn: Will start in 9 seconds
…complete with dawn-of-the-dead respawning processes.
I configured Mailman to replicate OSXS's default locations, and it
runs fine. It doesn't even break Server Admin, aside from my use of
virtual domains which ServerAdmin can't handle.
The only difference between my org.list.mailmanctl.plist and Apple's
is my verb *start*, as opposed to Apple's *startf*. I'm baffled by
*startf*, which is not supported by mailman and does nothing, either
in the CLI or in org.list.mailmanctl.plist.
Obviously this isn't fatal and there are even other ways to insure an
automatic startup, but I use launchd for such things, Apple more or
less deprecates other approaches, and it drives me nuts not to
understand what's going on.
Speak, oh voices of wisdom!
-Bryan
On 6/18/09 7:11 PM, Bryan Harrison at bryan@bryanharrison.com wrote:
I've built and installed Mailman 2.12 from current source code and have it running just fine under OS X Server 10.5.7, with on exception, which is not properly speaking a Mailman problem at all.
If I start Mailman from the CLI, using /usr/share/mailman/bin/ mailmanctl -s start, all is well.
If I allow launchd to start it I get an endless succession of
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master qrunner. Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]): Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds
Known issue, no idea why. I've been running Mailman on Client OS X since the Tiger days. As soon as I upgraded to Leopard, I started seeing this happen intermittently. I have a launchd plist almost identical to yours.
It is intermittent for me. Probably happens about 1 in 3 boots. I try to remember to check that it's running after reboots. If not, I have a cron job that runs hourly to check for the proper number of qrunners and send me e-mail if any or all are missing.
Am/On Thu, 18 Jun 2009 19:22:02 -0500 schrieb/wrote Larry Stone:
On 6/18/09 7:11 PM, Bryan Harrison at bryan@bryanharrison.com wrote:
I've built and installed Mailman 2.12 from current source code and have it running just fine under OS X Server 10.5.7, with on exception, which is not properly speaking a Mailman problem at all.
If I start Mailman from the CLI, using /usr/share/mailman/bin/ mailmanctl -s start, all is well.
If I allow launchd to start it S<caron>I get an endless succession ofS<caron>
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master qrunner. Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]): Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds
Known issue, no idea why. I've been running Mailman on Client OS X since the Tiger days. As soon as I upgraded to Leopard, I started seeing this happen intermittently. I have a launchd plist almost identical to yours.
It is intermittent for me. Probably happens about 1 in 3 boots. I try to remember to check that it's running after reboots. If not, I have a cron job that runs hourly to check for the proper number of qrunners and send me e-mail if any or all are missing.
I have the same issue as Bryan (and my plist looks simular) and find this in my logs, when I start mailman via launchd, if I start it by CLI, I don't see this issue:
Jun 26 20:16:29 myMac org.list.mailmanctl[40673]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:16:29 myMac org.list.mailmanctl[40673]: master qrunner is already running. Jun 26 20:16:29 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds Jun 26 20:16:41 myMac org.list.mailmanctl[40675]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:16:41 myMac org.list.mailmanctl[40675]: master qrunner is already running. Jun 26 20:16:41 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 8 seconds Jun 26 20:16:52 myMac org.list.mailmanctl[40680]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:16:52 myMac org.list.mailmanctl[40680]: master qrunner is already running. Jun 26 20:16:52 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 8 seconds Jun 26 20:17:02 myMac org.list.mailmanctl[40757]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:17:02 myMac org.list.mailmanctl[40757]: master qrunner is already running. Jun 26 20:17:03 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 8 seconds Jun 26 20:17:13 myMac org.list.mailmanctl[40760]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:17:13 myMac org.list.mailmanctl[40760]: master qrunner is already running. Jun 26 20:17:13 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 8 seconds
But I have this issue always on a 10.5.7 PPC-Server ..... So what would be the alternative?
cheers, Matthias
--On 26. Juni 2009 20:22:13 +0900 Matthias Schmidt beta@admilon.net wrote:
Am/On Thu, 18 Jun 2009 19:22:02 -0500 schrieb/wrote Larry Stone:
On 6/18/09 7:11 PM, Bryan Harrison at bryan@bryanharrison.com wrote:
I've built and installed Mailman 2.12 from current source code and have it running just fine under OS X Server 10.5.7, with on exception, which is not properly speaking a Mailman problem at all.
If I start Mailman from the CLI, using /usr/share/mailman/bin/ mailmanctl -s start, all is well.
If I allow launchd to start it S<caron>I get an endless succession ofS<caron>
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master qrunner. Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]): Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds
The reason this happens is that launchd expects the processes it launches to stick around. They must not daemonize. But mailmanctl terminates after it has started the runners. I assume that Apple modified their own distribution of Mailman so that mailmanctl behaves differently. I'd suggest comparing the two.
On 6/26/09 6:22 AM, Matthias Schmidt at beta@admilon.net wrote:
Am/On Thu, 18 Jun 2009 19:22:02 -0500 schrieb/wrote Larry Stone:
On 6/18/09 7:11 PM, Bryan Harrison at bryan@bryanharrison.com wrote:
I've built and installed Mailman 2.12 from current source code and have it running just fine under OS X Server 10.5.7, with on exception, which is not properly speaking a Mailman problem at all.
If I start Mailman from the CLI, using /usr/share/mailman/bin/ mailmanctl -s start, all is well.
If I allow launchd to start it S<caron>I get an endless succession ofS<caron>
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master qrunner. Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]): Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds
Known issue, no idea why. I've been running Mailman on Client OS X since the Tiger days. As soon as I upgraded to Leopard, I started seeing this happen intermittently. I have a launchd plist almost identical to yours.
It is intermittent for me. Probably happens about 1 in 3 boots. I try to remember to check that it's running after reboots. If not, I have a cron job that runs hourly to check for the proper number of qrunners and send me e-mail if any or all are missing.
I have the same issue as Bryan (and my plist looks simular) and find this in my logs, when I start mailman via launchd, if I start it by CLI, I don't see this issue:
Jun 26 20:16:29 myMac org.list.mailmanctl[40673]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:16:29 myMac org.list.mailmanctl[40673]: master qrunner is already running. Jun 26 20:16:29 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds Jun 26 20:16:41 myMac org.list.mailmanctl[40675]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:16:41 myMac org.list.mailmanctl[40675]: master qrunner is already running. Jun 26 20:16:41 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 8 seconds Jun 26 20:16:52 myMac org.list.mailmanctl[40680]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:16:52 myMac org.list.mailmanctl[40680]: master qrunner is already running. Jun 26 20:16:52 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 8 seconds Jun 26 20:17:02 myMac org.list.mailmanctl[40757]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:17:02 myMac org.list.mailmanctl[40757]: master qrunner is already running. Jun 26 20:17:03 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 8 seconds Jun 26 20:17:13 myMac org.list.mailmanctl[40760]: The master qrunner lock could not be acquired because it appears as if another Jun 26 20:17:13 myMac org.list.mailmanctl[40760]: master qrunner is already running. Jun 26 20:17:13 myMac com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 8 seconds
This is not the same issue as Bryan and I have. The log entries you have posted indicate that you are trying to start Mailman twice (the issue Bryan and I have is Mailman dying with the "stray process" message). Perhaps you have two different launchd scripts that try to to start Mailman. Launchd looks in both /System/Library/LaunchDaemons and /Library/LaunchDaemons. The former is supposed to be for Apple provided scripts with the latter for user/3rd party application scripts. My "mailman.plist" is in /Library/LaunchDaemons but you should check both and grep for mailmanctl.
On Jun 26, 2009, at 7:49 AM, Sebastian Hagedorn wrote:
The reason this happens is that launchd expects the processes it
launches to stick around. They must not daemonize. But mailmanctl
terminates after it has started the runners. I assume that Apple
modified their own distribution of Mailman so that mailmanctl
behaves differently. I'd suggest comparing the two.
FWIW in MM3, the two functions of mailmanctl have been split into two
separate scripts. bin/master is the master queue runner watcher script
and it does not daemonize. bin/mailmanctl is simply the control
interface to master and it does daemonize. There may not be too much
Mailman3-ish about the new scripts so it could be backported (though
it probably requires Python 2.6 or possibly 2.5).
-Barry
Am/On Fri, 26 Jun 2009 13:49:57 +0200 schrieb/wrote Sebastian Hagedorn:
--On 26. Juni 2009 20:22:13 +0900 Matthias Schmidt beta@admilon.net wrote:
Am/On Thu, 18 Jun 2009 19:22:02 -0500 schrieb/wrote Larry Stone:
On 6/18/09 7:11 PM, Bryan Harrison at bryan@bryanharrison.com wrote:
I've built and installed Mailman 2.12 from current source code and have it running just fine under OS X Server 10.5.7, with on exception, which is not properly speaking a Mailman problem at all.
If I start Mailman from the CLI, using /usr/share/mailman/bin/ mailmanctl -s start, all is well.
If I allow launchd to start it S<caron>I get an endless succession ofS<caron>
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master qrunner. Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]): Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds
The reason this happens is that launchd expects the processes it launches to stick around. They must not daemonize. But mailmanctl terminates after it has started the runners. I assume that Apple modified their own distribution of Mailman so that mailmanctl behaves differently. I'd suggest comparing the two.
thanks, that was the information I missed and it also explains why Apple uses -startf in the plist, which is not documented for mailman. I meanwhile found out, that the voodoo here is to use mailmanctl from the original Apple distribution.
cheers, Matthias
Larry & Bryan --
If it helps you at all, I saw the same problems with launchd on my Mailman 2.1.11 install on Mac OS X Server 10.5.5 when we migrated that machine to 10.5 from 10.4. We abandoned launchd in favor of the older /Library/StartupItems/ method and have had no problems since then.
I know that we'll have to replace this with launchd at some point, so please keep us informed if you find true solution to the problem.
Drew Tenenholz
On Fri, 26 Jun 2009, Sebastian Hagedorn wrote:
On 6/18/09 7:11 PM, Bryan Harrison at bryan@bryanharrison.com wrote:
If I allow launchd to start it S<caron>I get an endless succession ofS<caron>
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master qrunner. Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]): Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds
I just caught that Bryan notes that he receives an "endless succession" of those messages where as I see one and that's it.
Comparing our launchd .plist files, I see a few differences: Bryan: <key>OnDemand</key> <false/> Me: <key>OnDemand</key> <true/> <key>RunAtLoad</key> <true/>
Bryan does not have RunAtLoad but the default is false. I think this is why he sees the "endless succession" and I don't. Wiht RunAtLoad but OnDemand set true, my system tries once at startup and that's it. Bryan, due to OnDemand being false (meaning keep it running continuously), keeps trying.
Another difference (but I think making no difference is: Bryan: <key>Program</key> <string>/usr/share/mailman/bin/mailmanctl</string> <key>ProgramArguments</key> <array> <string>mailmanctl</string> <string>-s</string> <string>start</string> </array>
Me: <key>ProgramArguments</key> <array> <string>/Applications/Mailman/bin/mailmanctl</string> <string>-s</string> <string>start</string> </array>
I don't have a Program key. I don't think this is relevant as I specify the full path in ProgramArguments which Bryan does not need to do since he specified the path in Program.
The reason this happens is that launchd expects the processes it launches to stick around. They must not daemonize. But mailmanctl terminates after it has started the runners. I assume that Apple modified their own distribution of Mailman so that mailmanctl behaves differently. I'd suggest comparing the two.
I'm not so sure. It worked fine for me under Tiger (client). Drew reports the same thing with OS X Server - worked fine under Tiger (10.4.x) and then the same problems as the rest of us with Leopard.
Also, I can't say I agree that launchd expects the program to stick around. With it the way I have it set (RunAtLoad true and OnDemand true), it makes one attempt at startup and does not care after that whether it started or not. I'll note that I also have Clam-AV's daemon (clamd) starting the same way and it seems to work OK but some of the technical issues of launching daemon with launchd are beyond me. However, the launchd.plist(5) man page says:
EXPECTATIONS Daemons or agents managed by launchd are expected to behave certain ways.
A daemon or agent launched by launchd MUST NOT do the following in the process directly launched by launchd:
o fork(2) and have the parent process exit(3) or _exit(2).
o Call daemon(3)
A daemon or agent launched by launchd SHOULD NOT do the following as a part of their startup initialization:
Setup the user ID or group ID.
o Setup the working directory.
o chroot(2)
o setsid(2)
o Close "stray" file descriptors.
o Change stdio(3) to /dev/null.
o Setup resource limits with setrusage(2).
o Setup priority with setpriority(2).
o Ignore the SIGTERM signal.
A daemon or agent launched by launchd SHOULD:
o Launch on demand given criteria specified in the XML property
list. More information can be found in launch(3).
o Catch the SIGTERM signal.
I'll leave it to those better into the technical end of things to interpret that relative to what mailmanctl does.
-- Larry Stone lstone19@stonejongleux.com
Am/On Fri, 26 Jun 2009 11:06:40 -0500 schrieb/wrote Larry Stone:
On Fri, 26 Jun 2009, Sebastian Hagedorn wrote:
On 6/18/09 7:11 PM, Bryan Harrison at bryan@bryanharrison.com wrote:
If I allow launchd to start it S<caron>I get an endless succession ofS<caron>
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master qrunner. Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]): Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds
I just caught that Bryan notes that he receives an "endless succession" of those messages where as I see one and that's it.
Comparing our launchd .plist files, I see a few differences: Bryan: <key>OnDemand</key>
<false/> Me: <key>OnDemand</key> <true/> <key>RunAtLoad</key> <true/>
Larry,
you are my hero, thanks! That looks like working :-)
cheers, Matthias
I know, I'm reopening a ten month old thread. But I think I have a good workaround.
On Fri, 26 Jun 2009, Sebastian Hagedorn wrote:
--On 26. Juni 2009 20:22:13 +0900 Matthias Schmidt beta@admilon.net wrote:
Am/On Thu, 18 Jun 2009 19:22:02 -0500 schrieb/wrote Larry Stone:
On 6/18/09 7:11 PM, Bryan Harrison at bryan@bryanharrison.com wrote:
I've built and installed Mailman 2.12 from current source code and have it running just fine under OS X Server 10.5.7, with on exception, which is not properly speaking a Mailman problem at all.
If I start Mailman from the CLI, using /usr/share/mailman/bin/ mailmanctl -s start, all is well.
If I allow launchd to start it S<caron>I get an endless succession ofS<caron>
Jun 18 16:50:26 org.list.mailmanctl[1697]: Starting Mailman's master qrunner. Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl[1697]): Stray process with PGID equal to this dead job: PID 1698 PPID 1 Python Jun 18 16:50:26 com.apple.launchd[1] (org.list.mailmanctl): Throttling respawn: Will start in 9 seconds
The reason this happens is that launchd expects the processes it launches to stick around. They must not daemonize. But mailmanctl terminates after it has started the runners. I assume that Apple modified their own distribution of Mailman so that mailmanctl behaves differently. I'd suggest comparing the two.
I've been playing around with Mailman on a 10.6 (Snow Leopard) machine. It's for testing right now (at the next refresh of Mac Minis or iMacs, I plan to buy one to replace my PPC iMac that is currently my everything server - for now I'm testing on my laptop, the only Intel Mac I currently have).
So I found that unlike on the PPC iMac, where Mailman starts OK after about 75% of the boots, on the laptop, it consistently died. No "stray process" system.log message but perhaps that's a 10.5/10.6 difference. But I did see that the lock files were being created in $mailman_prefix/locks. So what's different? Well, the laptop is a lot faster. Factor? I think so.
I know almost nothing about the internals of forking processes and daemonizing but if I have it right, when 'mailmanctl start' is run, it forks a second copy which daemonizes. Meanwhile, the first copy of mailmanctl exits. My guess is that the first copy was exiting and the process terminating before the second copy had performed whatever magic it does to deamonize and run on its own. My workaround, which appears to work so far, is to have launchd, rather than run mailmanctl directly (via a .plist in /Library/LaunchDaemons), instead run a script in root's home directory that runs mailmanctl, then sleeps for one second before exiting: #!/bin/sh /usr/local/mailman/bin/mailmanctl -s start sleep 1
-- Larry Stone lstone19@stonejongleux.com
Larry Stone wrote:
I know almost nothing about the internals of forking processes and daemonizing but if I have it right, when 'mailmanctl start' is run, it forks a second copy which daemonizes. Meanwhile, the first copy of mailmanctl exits. My guess is that the first copy was exiting and the process terminating before the second copy had performed whatever magic it does to deamonize and run on its own.
It probably has nothing to do with this, but through Mailman 2.1.13, mailmanctl forks the eventual daemon with file descriptors for stdin, stdout and stderr, possibly attached to the terminal. Normally, the only symptom is if you start Mailman in a terminal session and then log out, the session doesn't completely close until you physically disconnect, but this may have an effect on your situation. I do have an Intel iMac amongst several other machines, but I know essentially nothing about launchd.
Anyway, there is a patch to mailmanctl at http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1221 which probably won't make a difference to your case, but might.
participants (9)
-
Barry Warsaw -
Bryan Harrison -
Drew Tenenholz -
Gordon Smith -
Larry Stone -
Mark Sapiro -
Matthias Schmidt -
rbryanh -
Sebastian Hagedorn