Performance under load (bursty message flow)
In running a number of lists at VA under mailman, I've noticed an apparent performance problem: multiple simultaneous deliveries to the same list performs badly. Locking?
One particular list tends to receive large numbers of messages over short time periods (eg 50 messages with less than a minute) followed by long periods of inactivity (eg an hour). The result is that multiple (typically a dozen) simultaneous instances of mailman are invoked for that list.
Watching a mail bomb come thru the apparent behavious is curious: The first batch gets thru fine (ie # of simultansous deliveries as set in the MTS cfg), each individual message seperated from the others by fractions of a second. But the second run (which all start within milliseconds of each other) all seem to block on <listname>.lock for the same time period, and to then spin, all unlocking and relocking in synch and getting nowhere. Certainly the wait time on the lock is rather long (I haven't checked the source).
Perhaps a shorter default lock time plus a random factor?
Eeek. System is a 400MHz PII, 128Meg RAM, Python 1.5, RH 5.2, all updates, latest CVS version of MailMan.
-- J C Lawrence Home: claw@kanga.nu ---------(*) Linux/IA64 - Work: claw@varesearch.com ... Beware of cromagnons wearing chewing gum and palm pilots ...
"JCL" == J C Lawrence <claw@varesearch.com> writes:
JCL> In running a number of lists at VA under mailman, I've
JCL> noticed an apparent performance problem: multiple
JCL> simultaneous deliveries to the same list performs badly.
JCL> Locking?
JC, I am now starting to see the same behavior on python.org. My machine is slammed at 100% cpu almost continuously doing about the same thing you're seeing. I'd classify this as a problem that requires a solution before 1.0 can go out.
I have no idea what's going on, or how to fix it, and as I mentioned before I don't have the time right now to look into it. But I just wanted to say that it's a problem I have a vested interest in fixing (although I'd be pleased as punch if someone else works on it in the meantime :).
-Barry
participants (2)
-
Barry A. Warsaw
-
J C Lawrence