[Mailman-Users] Mailman-Users Digest, Vol 47, Issue 55

[Brad Knowles]

Khalil Abbas wrote:

> my question is, I have a 100,000+ subscribers mailing list, and hosting
> it with godaddy.com (Celeron 2000, 2 GBs RAM) .. but they only allow
> 100,000 emails per day for each dedicated server, so I have to buy a new
> server now, and it's a bit expensive.. I found another cheap service,
> $30/month dedicated servers and they offer unlimited emails per day.. but
> its characteristics are a bit low: AMD, 512 MB RAM ..
> 
> my question is, will mailman with such a huge list which is also rapidly
> growing work on such server? knowing that I only send 1 email newsletter
> a day to the list and none of the users can send to the list but me..

We try to answer all these kinds of questions in the Mailman FAQ Wizard 
entry 1.15 at 
<http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.015.htp>.

However, your specific case may need a bit more explanation.


If you read through the various FAQ entries regarding performance, you'll 
see that there are some critical factors.  Pay special attention to FAQ 4.11 
at <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.011.htp>, 
and the stuff down in section 6.

One critical factor is MTA performance.  You need to be running the right 
MTA in the right configuration, if you're going to be doing that kind of 
traffic.  This has knock-on requirements for memory, disk I/O capacity, DNS 
cache performance and location, etc....

Another critical factor is Mailman lock, pickle, and queue contention.  With 
a large list, all incoming and outgoing messages are processed in 
single-threaded mode, because there's just one pickle for the list which has 
to be locked during these kinds of operations, preventing any other list 
operations from occurring at the same time.  There's also on-disk queue 
contention, especially as directory sizes get large.  Here, you're going to 
want a filesystem that incorporates an internal directory hashing scheme 
(e.g., SGI XFS).  You can get around the pickle contention by creating a 
number of smaller sub-lists, all of which are subscribed to a higher-level 
"umbrella" list.  All traffic to an individual sub-list will be processed 
sequentially, but traffic between sublists can be processed in parallel.  An 
ideal breakdown for sub-list size would probably be something between the 
natural logarithm and the square-root of the total number of subscribers you 
have.  For 100,000 subscribers, that would be somewhere between 11 and 316 
sub-lists.

For really advanced stuff, you might want to look at splitting the Mailman 
queues and running multiple queue runners for each.  The documentation for 
this stuff is *not* found in the FAQ or the official written documentation, 
and FAQ 4.75 won't help you since you're not load-balancing across multiple 
servers.  The only documentation for setting up queue slices is found in the 
Mailman mm_cfg.py file, so you had better be pretty comfortable with this 
kind of stuff.


All that said, I would be surprised if a low-end system like this could 
reasonably keep up with such a large list (see that FAQ Wizard entry 
mentioned above and pay particular attention to the articles that it links 
to for the larger sites).  On the other hand, if you can get it to work, I'd 
love to hear about it.

If you want to get a sense of what kind of hardware we use for handling the 
mail for python.org (including all our mailing lists), see FAQ 1.15 
(mentioned above), FAQ 1.24 at 
<http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.024.htp>, and 
FAQ 4.56 at 
<http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.056.htp>.

> another question, is it easy to install mailman manually? cause in
> godaddy's control panel it's ready to install with 1 click.. but with the
> new cheap service I have to install it myself..

It's about as easy as any software I've ever installed, but then I've been 
doing this sort of thing for nearly twenty years.  So, I'm not sure I'm the 
right person to be answering your "ease-of-installation" questions.

-- 
Brad Knowles <brad at python.org>
Member of the Python.org Postmaster Team, & Co-moderator of the
mailman-users and mailman-developers mailing lists