[Mailman-Users] Mailman + giant lists + the infinite weight of the cosmos

[Cabel Sasser]

We're a Macintosh shareware company that, like some others on this 
list, use Mailman for announce-only mailings that our software users 
can voluntarily subscribe to to be alerted of new software versions.

I'll get right to the point: one of our Mailman lists currently has 
121,047 members. It uses full personalization and auto bounce 
processing. I'm sure you know where this is headed... :)

My problem is not with the mailing, but seemingly rather with the user 
management. Adding and removing members can take up to 60 seconds to 
complete -- many people bail before it's done. The web based management 
interface is also very, very slow as well. And, the last time I tried 
to send an e-mail to the list, I'm not even sure it made it through all 
the way -- the Xserve eventually crashed (!) because I think we were 
hitting a massive amount of swap (particularly once the bounces started 
rolling in and it tried to remove the members, taking 60 seconds of 
processing for each one.)

I could really use this lists' advice!

	1. Should I split the list into a series of smaller lists? What's the 
best way to do that easily?

	2. Is there any way to "optimize" this database, other than throwing 
more memory at the machine, etc.?

	3. Any general advice for handling ridiculously large lists?

I've seen in the FAQ (1.15) that The Guardian ran a list with 147,000 
subscribers, and I'm wondering how it was done! I've read performance 
tuning (4.11) but that seems to focus on the mail _delivery_, which is 
generally working just fine, not the user management -- think bursting 
Python pickles.

Thanks so much in advance for any -- any! -- help you can provide.

Best,
Cabel
Panic