On Fri, 2004-01-30 at 13:49, Somuchfun wrote:
Here is what I do not understand from the discussion: Mailman in its current form is slow and if personalization is turned on users cannot even get into the mailman site anymore because it takes up all available resources.
I don't totally believe that.
Understand that IMO, MM2.1's biggest architectural flaw right now is the list data storage arrangement. Pickles and list locks simply do not scale. Fixing this is a priority for MM3, but of course there will be costs. Backing Mailman with a real database (be it BerkeleyDB, MySQL, or whatever) increases the administrative costs. No way around it.
That said, MM2.1 does not retain a list lock while it delivers messages to its MTA, so it should not lock out other access to the site.
We are running a list with about 50,000 subscribers. As an admin I do not really care if some people think AOL does not have their act together or not - if I want to have my emails reach them then I have to play by their rules. Like I said, I have tried other softwares on the market and used their personalization feature. I even tried the same list on the same machine. Mailman needed with personalization about 8.5 hrs. to send out one message to all 50,000 people and Lyris Listmanager needed about 4.5 hours. Is disk I/O a problem? Of course it is, but it is a problem for all list managing software packages. My experience is that mailman is just very slow when it comes to db access. Just try to add 10,000 users at once and most likely you get a time out.
Again, the issue is likely deeper than it first appears. I will bet you that Mailman's "db access" is about as fast as you can possibly get, because the list data resides completely in memory. Lookups are a simple dictionary access, which is very fast.
Where I believe you're getting clobbered is in the specific code that generates the unique recipient copies. The technique I'm using is about as good as you can do in Python 2.1, which is MM2.1's minimum requirement. I can do a lot better if we set Python 2.3 as a baseline and make other incompatible changes. That's why it's all pushed off to Mailman 3.
And being twice as slow as Lyris is actually not bad, IMO. Lyris is probably written in C or C++. For a pure Python application like Mailman to only take twice as long is not bad.
So perhaps mailman is better for smaller discussion list than for larger email lists.
As Mailman gains in popularity, people will try to make it do things it wasn't necessarily designed for, or that weren't conceivable 6 years ago when many of the basic architectural decisions were made.
Some people here have suggested that anything besides email discussion lists are spam, I find statements like this alarming.
Spam is anything the user doesn't want to get.
We run a newsletter where people actively want to get the newsletter and we do not consider ourselves spamming these people. In fact we try very hard to comply with all rules, regulations and expectations - more so than some ISPs.
I have no problem with that.
All I want is a fast and cheap engine that can help me reach my goal - to get the email to my customers quickly and to offer easy management capabilities. So far I like mailman's management capabilities. The performance has left me being disappointed.
So how much would you pay to improve Mailman's performance? If we could raise a quarter million dollars in development funds, I doubt you'd be disappointed for long <wink>.
-Barry