[Mailman-Developers] AOL's requirements for spam complaints

[Barry Warsaw]

On Fri, 2004-01-30 at 13:49, Somuchfun wrote:
> Here is what I do not understand from the discussion:
> Mailman in its current form is slow and if personalization is turned on
> users cannot even get into the mailman site anymore because it takes up all
> available resources.

I don't totally believe that.

Understand that IMO, MM2.1's biggest architectural flaw right now is the
list data storage arrangement.  Pickles and list locks simply do not
scale.  Fixing this is a priority for MM3, but of course there will be
costs.  Backing Mailman with a real database (be it BerkeleyDB, MySQL,
or whatever) increases the administrative costs.  No way around it.

That said, MM2.1 does not retain a list lock while it delivers messages
to its MTA, so it should not lock out other access to the site.

> We are running a list with about 50,000 subscribers.
> As an admin I do not really care if some people think AOL does not have
> their act together or not - if I want to have my emails reach them then I
> have to play by their rules.
> Like I said, I have tried other softwares on the market and used their
> personalization feature. I even tried the same list on the same machine.
> Mailman needed with personalization about 8.5 hrs. to send out one message
> to all 50,000 people and Lyris Listmanager needed about 4.5 hours.
> Is disk I/O a problem? Of course it is, but it is a problem for all list
> managing software packages. My experience is that mailman is just very slow
> when it comes to db access. Just try to add 10,000 users at once and most
> likely you get a time out.

Again, the issue is likely deeper than it first appears.  I will bet you
that Mailman's "db access" is about as fast as you can possibly get,
because the list data resides completely in memory.  Lookups are a
simple dictionary access, which is very fast.

Where I believe you're getting clobbered is in the specific code that
generates the unique recipient copies.  The technique I'm using is about
as good as you can do in Python 2.1, which is MM2.1's minimum
requirement.  I can do a lot better if we set Python 2.3 as a baseline
and make other incompatible changes.  That's why it's all pushed off to
Mailman 3.

And being twice as slow as Lyris is actually not bad, IMO.  Lyris is
probably written in C or C++.  For a pure Python application like
Mailman to only take twice as long is not bad.  

> So perhaps mailman is better for smaller discussion list than for larger
> email lists.

As Mailman gains in popularity, people will try to make it do things it
wasn't necessarily designed for, or that weren't conceivable 6 years ago
when many of the basic architectural decisions were made.

> Some people here have suggested that anything besides email discussion lists
> are spam, I find statements like this alarming.

Spam is anything the user doesn't want to get.

> We run a newsletter where people actively want to get the newsletter and we
> do not consider ourselves spamming these people. In fact we try very hard to
> comply with all rules, regulations and expectations - more so than some
> ISPs.

I have no problem with that.

> All I want is a fast and cheap engine that can help me reach my goal - to
> get the email to my customers quickly and to offer easy management
> capabilities.
> So far I like mailman's management capabilities. The performance has left me
> being disappointed.

So how much would you pay to improve Mailman's performance?  If we could
raise a quarter million dollars in development funds, I doubt you'd be
disappointed for long <wink>.

-Barry