![](https://secure.gravatar.com/avatar/7ef6790ad69919683f6fef96d0bd3168.jpg?s=120&d=mm&r=g)
hi,
I have several vps servers with very limited resources running several large mailman lists, n these servers keep stopping from responding n sometimes mailq shows several errors like: too many open files in the system and not enough resourses ... etc.
so every hour I do the following:
mailmanctl stop postfix stop reboot
to free up the system resources, and eventually the mailq shows empty n the mail gets delivered..
is that a good thing I’m doing? for I can’t afford to upgrade the hardware, is there a better solution? is there any side effects??
Thanks..
![](https://secure.gravatar.com/avatar/267565c6ab7816fe29beedf9a9cbcd44.jpg?s=120&d=mm&r=g)
- Khalil Abbas <khillo100@hotmail.com>:
It would be better to lower the default_process_limit in postfix
postconf -e "default_process_limit = 5" postfix reload
-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de
![](https://secure.gravatar.com/avatar/7ef6790ad69919683f6fef96d0bd3168.jpg?s=120&d=mm&r=g)
The servers are all vps'es with 256 ram.. and the lists are as big as 33,000 subscribers per list, 3 lists per vps.. it's one big mailing list for a TV station and newspaper.. only one message (newsletter) is sent out every day to the subscribers.. but setting:
SMTP_MAX_RCPTS to only 5 in mailman resulted in a huge number of messages generated by mailman that halts the servers, that's why I every hour stop mailman then stop postfix then reboot the servers to free up the memory and continue distributing the mail..
my question is, is this a good solution? or should I split them into very small lists then send the same message every hour after making sure that the servers finished distributing the mail for the previous lists?
Thanks..
-----Original Message----- From: Geoff Shang Sent: Wednesday, June 01, 2011 2:04 PM To: Khalil Abbas Cc: mailman-users@python.org Subject: Re: [Mailman-Users] reboot servers
On Tue, 31 May 2011, Khalil Abbas wrote:
How limited are these servers? How big are these lists?
Geoff.
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 6/3/11 4:56 AM, Khalil Abbas wrote:
What problem were you trying to solve by setting SMTP_MAX_RCPTS = 5? Maybe you should consider raising it or letting it go back to its default of 500 if things worked better then.
See the last paragraph of the FAQ at <http://wiki.list.org/x/j4A9>. The throttling patch may help.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California Better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/7ef6790ad69919683f6fef96d0bd3168.jpg?s=120&d=mm&r=g)
well it was your advise on a previous post to set the SMTP_MAX_RCPTS to 5 in order to let hotmail and yahoo and others accept mail coming from my lists.. if you remember the smtp_max_rcpts was 500 and thousands of subscribers used to bounce back because Hotmail rejected them.. after setting it to 5 it worked like a charm .. but I fell into this new problem of large number of messages flowing and halting the servers..
I don't want to use the throttling patch as it's not stable and not tested before .. so I have 2 ways, either rebooting the servers like I'm doing now, or splitting the lists into smaller ones and send to bunch of small lists every our ..
what do u think?
-----Original Message----- From: Mark Sapiro Sent: Friday, June 03, 2011 5:44 PM To: Khalil Abbas Cc: Geoff Shang ; mailman-users@python.org Subject: Re: [Mailman-Users] reboot servers
On 6/3/11 4:56 AM, Khalil Abbas wrote:
What problem were you trying to solve by setting SMTP_MAX_RCPTS = 5? Maybe you should consider raising it or letting it go back to its default of 500 if things worked better then.
See the last paragraph of the FAQ at <http://wiki.list.org/x/j4A9>. The throttling patch may help.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California Better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/b273ab068bc220d17a3e4c710c401c4b.jpg?s=120&d=mm&r=g)
On 6/3/2011 9:10 AM, Khalil Abbas wrote:
Not to be harsh, but that you're not looking in the right direction. Yes, splitting the lists -might- help, but then might not. You'll also loose some advantages of one list, such as grouping recipients on the same domain into fewer SMTP transactions, and list maintenance will become more of a hassle. Restarting the processes, or, gasp, rebooting the server does not solve the problem, it makes it go away for a while. It's not a valid -solution-.
And, you -do- have other ways to analyze and solve the problem but seem reluctant to do that. Other people have provided good suggestions (more RAM or CPU, remove other processes that may be hogging the system, get off a VM if that's what you're using, etc). You can use thing like 'top' to see where the resources are going. If mailman needs 500mb of virtual memory and takes a long time to deliver that list, everything will fall over when the next list starts trying. Eventually, the system will spend more time paging than running code.
FWIW, my mail server has been up for 436 days and has only 64mb of ram :). Granted it doesn't try to run a 30k member list. I'd use a larger server for that.
Have you read: http://wiki.list.org/display/DOC/Mailman+Performance+Tuning+for+Mail+Deliver... ? http://wiki.list.org/pages/viewpage.action?pageId=4030582 http://wiki.list.org/pages/viewpage.action?pageId=4030711
z!
![](https://secure.gravatar.com/avatar/b273ab068bc220d17a3e4c710c401c4b.jpg?s=120&d=mm&r=g)
On 6/3/2011 4:56 AM, Khalil Abbas wrote:
The servers are all vps'es with 256 ram..
vps? If you mean they're vertual servers, then at least give them more RAM! (Have you watched the processes with top while running? Where are the -actual- bottlenecks?)
I think the consensus is that it's not. Might work, but not good. But... RAM is cheap, servers are cheap, why not just replace the hardware? A few months ago I bought a used dual Opteron 2.6gH server with 2g ram for about $125 at a local reseller. Just needed a disk. You may spend more time hacking around with parameters and segmenting lists than building and replacing the box, and then you'll have have a more generally powerful server.
z!
![](https://secure.gravatar.com/avatar/29d2b1aeb89386b1fa98d58be3755b7f.jpg?s=120&d=mm&r=g)
The servers are all vps'es with 256 ram..
I'm assuming you meant 256 Megs RAM.. to which I say "Luxury!!!!" :)
..and a well-behaved SMTP daemon will just queue up the emails and file them out onto the internets as and when it sees a gap in the traffic.
Are you just using these servers for mailing purposes, or have you got all the extra gubbins like samba daemons, webservers, named/bind/dns servers, enterprise management suites, X11 servers and the like running on there as well?
Of course the other problem might be that your system is not interfacing very well with the upstream mailer. How reliable/how quickly does a standard non-list email take to send from that system to... somewhere else?
I'd really look at pruning down the excess cruft in the runspace of those servers, getting rid of all the unnecessary processes that seem to get foisted onto the box when the install CD does its job. If you think that memory may be a problem then up the size of your swap/virtual memory. Disk-space is cheap right now and if you're just stacking a load of processes then having a useful lump of swap can come in very handy.
"mailq shows several errors like: too many open files"
Ahhh! The old 'not enough open file descriptors to do a decent job' problem. You haven't said what O/S you're using so I couldn't really give you a definite on this one but have a look on google (other search engines are available!) for how to increase the number of open file descriptors (or handles) in your O/S. It'll usually involve sticking a line or two into the /etc/system file and rebooting.
It sounds a bit like your O/S hasn't been tuned to handle the number of processes it has running so either kill off some of the excess/unwanted/unneeded daemon processes, twiddle arund with the etc/system file.. And good luck! :)
Regards, Steff
========
![](https://secure.gravatar.com/avatar/267565c6ab7816fe29beedf9a9cbcd44.jpg?s=120&d=mm&r=g)
- Khalil Abbas <khillo100@hotmail.com>:
It would be better to lower the default_process_limit in postfix
postconf -e "default_process_limit = 5" postfix reload
-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de
![](https://secure.gravatar.com/avatar/7ef6790ad69919683f6fef96d0bd3168.jpg?s=120&d=mm&r=g)
The servers are all vps'es with 256 ram.. and the lists are as big as 33,000 subscribers per list, 3 lists per vps.. it's one big mailing list for a TV station and newspaper.. only one message (newsletter) is sent out every day to the subscribers.. but setting:
SMTP_MAX_RCPTS to only 5 in mailman resulted in a huge number of messages generated by mailman that halts the servers, that's why I every hour stop mailman then stop postfix then reboot the servers to free up the memory and continue distributing the mail..
my question is, is this a good solution? or should I split them into very small lists then send the same message every hour after making sure that the servers finished distributing the mail for the previous lists?
Thanks..
-----Original Message----- From: Geoff Shang Sent: Wednesday, June 01, 2011 2:04 PM To: Khalil Abbas Cc: mailman-users@python.org Subject: Re: [Mailman-Users] reboot servers
On Tue, 31 May 2011, Khalil Abbas wrote:
How limited are these servers? How big are these lists?
Geoff.
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 6/3/11 4:56 AM, Khalil Abbas wrote:
What problem were you trying to solve by setting SMTP_MAX_RCPTS = 5? Maybe you should consider raising it or letting it go back to its default of 500 if things worked better then.
See the last paragraph of the FAQ at <http://wiki.list.org/x/j4A9>. The throttling patch may help.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California Better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/7ef6790ad69919683f6fef96d0bd3168.jpg?s=120&d=mm&r=g)
well it was your advise on a previous post to set the SMTP_MAX_RCPTS to 5 in order to let hotmail and yahoo and others accept mail coming from my lists.. if you remember the smtp_max_rcpts was 500 and thousands of subscribers used to bounce back because Hotmail rejected them.. after setting it to 5 it worked like a charm .. but I fell into this new problem of large number of messages flowing and halting the servers..
I don't want to use the throttling patch as it's not stable and not tested before .. so I have 2 ways, either rebooting the servers like I'm doing now, or splitting the lists into smaller ones and send to bunch of small lists every our ..
what do u think?
-----Original Message----- From: Mark Sapiro Sent: Friday, June 03, 2011 5:44 PM To: Khalil Abbas Cc: Geoff Shang ; mailman-users@python.org Subject: Re: [Mailman-Users] reboot servers
On 6/3/11 4:56 AM, Khalil Abbas wrote:
What problem were you trying to solve by setting SMTP_MAX_RCPTS = 5? Maybe you should consider raising it or letting it go back to its default of 500 if things worked better then.
See the last paragraph of the FAQ at <http://wiki.list.org/x/j4A9>. The throttling patch may help.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California Better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/b273ab068bc220d17a3e4c710c401c4b.jpg?s=120&d=mm&r=g)
On 6/3/2011 9:10 AM, Khalil Abbas wrote:
Not to be harsh, but that you're not looking in the right direction. Yes, splitting the lists -might- help, but then might not. You'll also loose some advantages of one list, such as grouping recipients on the same domain into fewer SMTP transactions, and list maintenance will become more of a hassle. Restarting the processes, or, gasp, rebooting the server does not solve the problem, it makes it go away for a while. It's not a valid -solution-.
And, you -do- have other ways to analyze and solve the problem but seem reluctant to do that. Other people have provided good suggestions (more RAM or CPU, remove other processes that may be hogging the system, get off a VM if that's what you're using, etc). You can use thing like 'top' to see where the resources are going. If mailman needs 500mb of virtual memory and takes a long time to deliver that list, everything will fall over when the next list starts trying. Eventually, the system will spend more time paging than running code.
FWIW, my mail server has been up for 436 days and has only 64mb of ram :). Granted it doesn't try to run a 30k member list. I'd use a larger server for that.
Have you read: http://wiki.list.org/display/DOC/Mailman+Performance+Tuning+for+Mail+Deliver... ? http://wiki.list.org/pages/viewpage.action?pageId=4030582 http://wiki.list.org/pages/viewpage.action?pageId=4030711
z!
![](https://secure.gravatar.com/avatar/b273ab068bc220d17a3e4c710c401c4b.jpg?s=120&d=mm&r=g)
On 6/3/2011 4:56 AM, Khalil Abbas wrote:
The servers are all vps'es with 256 ram..
vps? If you mean they're vertual servers, then at least give them more RAM! (Have you watched the processes with top while running? Where are the -actual- bottlenecks?)
I think the consensus is that it's not. Might work, but not good. But... RAM is cheap, servers are cheap, why not just replace the hardware? A few months ago I bought a used dual Opteron 2.6gH server with 2g ram for about $125 at a local reseller. Just needed a disk. You may spend more time hacking around with parameters and segmenting lists than building and replacing the box, and then you'll have have a more generally powerful server.
z!
![](https://secure.gravatar.com/avatar/29d2b1aeb89386b1fa98d58be3755b7f.jpg?s=120&d=mm&r=g)
The servers are all vps'es with 256 ram..
I'm assuming you meant 256 Megs RAM.. to which I say "Luxury!!!!" :)
..and a well-behaved SMTP daemon will just queue up the emails and file them out onto the internets as and when it sees a gap in the traffic.
Are you just using these servers for mailing purposes, or have you got all the extra gubbins like samba daemons, webservers, named/bind/dns servers, enterprise management suites, X11 servers and the like running on there as well?
Of course the other problem might be that your system is not interfacing very well with the upstream mailer. How reliable/how quickly does a standard non-list email take to send from that system to... somewhere else?
I'd really look at pruning down the excess cruft in the runspace of those servers, getting rid of all the unnecessary processes that seem to get foisted onto the box when the install CD does its job. If you think that memory may be a problem then up the size of your swap/virtual memory. Disk-space is cheap right now and if you're just stacking a load of processes then having a useful lump of swap can come in very handy.
"mailq shows several errors like: too many open files"
Ahhh! The old 'not enough open file descriptors to do a decent job' problem. You haven't said what O/S you're using so I couldn't really give you a definite on this one but have a look on google (other search engines are available!) for how to increase the number of open file descriptors (or handles) in your O/S. It'll usually involve sticking a line or two into the /etc/system file and rebooting.
It sounds a bit like your O/S hasn't been tuned to handle the number of processes it has running so either kill off some of the excess/unwanted/unneeded daemon processes, twiddle arund with the etc/system file.. And good luck! :)
Regards, Steff
========
participants (6)
-
Carl Zwanzig
-
Geoff Shang
-
Khalil Abbas
-
Mark Sapiro
-
Ralf Hildebrandt
-
Steff Watkins