-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On May 12, 2008, at 6:57 PM, Mark Sapiro wrote:
This predates my experience with Mailman. It is based on the
statistics provided by Chuq and outlined in the FAQ. It's true that these statistics may only be applicable to lists with primarily US members, and may be outdated in any case, but I can't provide any more information on why it's done that way. Perhaps it's an idea that's outlived its usefulness.
It may have. I see from the NEWS file that SMTP_MAX_RCPTS was
introduced on the last day of 1998, and these chunking tricks were
essential back then given the state of the art in MTAs. A decade is a
long time in this field, so it makes sense to re-evaluate Mailman's
MTA hand-off machinery.
What makes the most sense for us to do today? In principle, I would
like to simplify this code rather than complicate it with yet another
configuration option. So what use cases do we need to support? Can
we get closer interaction with the upstream MTA to hand it the
outgoing message and recipients more efficiently? The more we can
push these decisions into the MTA, the better off Mailman will be.
As an example of an efficiency I'm likely to add to MM3: I believe I
can avoid calculating the full recipient list, requiring storing it in
memory or in a queue file. I think I can build it so that we only
iterate through the recipients one at a time, as we're talking to the
upstream MTA. That would be a big I/O and memory footprint savings on
the Mailman side. Are there other thing we can do to push most of the
processing, chunking, and efficiency calculations into the MTA, where
it belongs (IMO)? To gain this benefit, we may have to ditch
chunking, or only support what grouping is available in SQL.
We clearly need to support personalization[1] in Mailman, although it
would be nice if there were extensions we could rely on in the MTA to
push even that out farther. I'd like to hear from any Sendmail,
Postfix, or Exim experts on this list to see if the chunking even
makes sense any more.
I will note that we still get requests to batch deliveries to the
outgoing MTA. People want to be able to throttle Mailman's outgoing
bandwidth to X number of messages per hour or day. Ideally, this
shouldn't be Mailman's task, but it's a valid request and one we
should keep in mind.
I would be happy if we could determine the two or three strategies
that address 80% of Mailman's users, package them up as simple-to-
configure defaults, and then provide a robust plugin architecture that
people can use to build all kinds of wacky strategies for the other 20%.
- -Barry
[1] Technically, not VERP, but close enough for our purposes.