[Mailman-Developers] Serious I/O contention issue

[Brad Knowles]

At 1:15 PM +0100 2003/10/15, Mike Bradley wrote:

>  One thing I did notice was that disable_dns_lookups=yes is recommended
>  for performance reasons - surely this would stop Postfix from working
>  altogether, as DNS lookups are needed to send mail (I tried switching
>  this option and mail delivery did indeed stop working!)

	I didn't notice that particular configuration option.


	I can say that you can use different copies of postfix on the 
server, listening to different IP address/port combinations, and 
configure one of them to minimize the various checks that it performs 
for incoming connections, and then configure mailman to use it 
instead of the other.  Best way to do this would be to have the 
version that minimizes the lookups listen to port 25 on 127.0.0.1, 
and have the other copy listen to port 25 on the other IP address(es) 
on the system.

	This way, incoming connections are still handled correctly 
through the external copy of postfix, while outgoing connections are 
sped up by eliminating unnecessary checks.

>  This is an area that I have shied away from in the past, as our server
>  is managed, so my knowledge of how to tune the filesystem is very
>  sketchy!  I would say that the server is quite busy with lots of
>  database accesses, and there have been no noticable filesystem
>  performance problems in the past, no matter how much load I have put on.
>  It also had no problem dealing with virus scanning and bouncing 10
>  incoming Slapper viruses per second last month while running the rest of
>  the stuff I have.

	Problem is, filesystem problems with synchronous meta-data issues 
(as typically plague most mail servers) usually don't *appear* to be 
filesystem problems.

	Instead, they appear to be things like not having enough RAM -- 
you get too many programs stacked up in memory, and you page/swap 
yourself to death.  In reality, the problem is that too many 
processes are bottlenecking on trying to create/delete too many 
temporary files in a particular directory structure, thus causing 
them all to slow down and stack up.

	There are a variety of other ways that filesystem synchronous 
meta-data issues will manifest themselves, but it's almost always in 
a manner that would lead you to think of the filesystem last.  Of 
course, the filesystem is one of the first things you should look at 
in cases like this.


	This is the reason why I asked the questions regarding the OS, 
how many disks, how the filesystem is configured, etc....  There are 
a lot of things you can do to speed these processes up, if you have 
enough information and know what you're doing.

	I'm trying to get that additional information.

>  If this was indeed a valid thing to do, then it might also be OK to
>  delay Save() operations so that they didn't occur so often (i.e. every N
>  operations, when idle, or on the final release of the mlist).  As I
>  said, it APPEARS that the caching of the lists in Runner.py is intended
>  to work this way, but I don't know enough about it to say for sure!

	I can't speak to the way the Python code runs.  I can say that 
I've got lots of experience doing filesystem tuning for mail systems, 
and I know that there are a myriads of ways that this kind of problem 
can masquerade as something else.

-- 
Brad Knowles, <brad.knowles at skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)