[Mailman-Developers] Serious I/O contention issue

[Mike Bradley]

I have a rather large mailing list (35000 members) which is used for
announcements only, but I have been experiencing terrible performance
problems when the qrunner is active.  The mail server is Postfix,
Mailman is version 2.1.3 (previous versions showed the same problems).

When I send an outgoing mail, the qrunner processes take up all my CPU
and the hard disk becomes inaccessible due to constant IO.  Even when
there is very little activity in the list and the queues are cleared,
the server locks up for several seconds every time an operation is
performed.

I have not had much experience with Python, but have tried to track down
what might be causing this.  It seems that the OutgoingRunner and
VirginRunner in particular are causing the worst of the problems,
specifically during load and save of the mlist.  It appears that the
list is being reloaded or resaved from disk after EVERY operation (so
rendering useless the handy looking caching of the list in the main
Runner class).

I am not familiar with the exact way the application operates, but are
the mlist arrays shared/marshalled across the qrunner processes via the
_listcache?  If so, then would it be possible to eliminate the
mlist.Load() in the OutgoingRunner and implement delayed saving in the
mailing list class so that Save() didn't cause a write to disk each
time?

For the moment, I have had to shut down my list completely as it is
chewing my server!

Mike

-- 
Make more of your Nokia Phone with LogoManager 
http://www.logomanager.co.uk