Efficient final message disposition (was Re: [Mailman-Developers] Requirements for a new archiver)

[Brad Knowles]

At 9:53 AM -0500 2003/10/30, Barry Warsaw wrote:

>  I'm sure you guys can identify more issues <wink>.  Look at the
>  complexity in SMTPDirect.py, and even there, we still have problems.

	I'm not a programmer, so I can't really help you there.  ;-(

>  So how do we design a system where we can push the complexity and
>  efficiency concerns out past our boundary?

	I can say that I think we need to look at all of the 
recommendations in the following papers:

		"Tuning Sendmail for Large Mailing Lists"
		Rob Kolstad
		Proceedings of LISA '97
		http://tinyurl.com/t09c


		"Drinking from the Fire(walls) Hose:
			Another Approach to Very Large Mailing Lists"
		Strata Rose Chalup, Christine Hogan, Greg Kulosa, Bryan McDonald,
			and Bryan Stansell
		Proceedings of LISA '98
		http://tinyurl.com/t09k

	There may be others that we need to look at, but of which I am 
not (yet) aware.  If anyone knows of any, please let me know.


	We're already doing some of the things recommended in these 
papers, but not everything.  And I think there may be a couple more 
things we can do that are not mentioned, but which would be a further 
help.

	However, if you want to hand all this work to an external "final 
mail-merge delivery agent", this is moot.  We just need to make sure 
that the selected FMMDA addresses all these issues.  We could use an 
existing tool (e.g., bulk_mailer from 
<ftp://cs.utk.edu/pub/moore/bulk_mailer/>), or we could create a 
separate package to address this issue (of course, that brings the 
ball back into our court).

	Or, you could just have Chuq solve this problem for you, as he 
mentioned in 
<http://mail.python.org/pipermail/mailman-developers/2000-May/006820.html>. 
;-)

>  So what can we do here to improve matters?

	Sounds to me like you want to externalize this whole process. 
Problem is, bulk_mailer is the only tool I know of that currently 
exists as a partial attempt to address this problem, although perhaps 
some additional work on it could fill in the rest.  Alternatively, 
you develop, or work with someone else to develop, an alternative to 
bulk_mailer that does all the things you want and which can be used 
as an external tool.

-- 
Brad Knowles, <brad.knowles at skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)