I'd really love to add somme sleep() function just after the deliveryfunc() call, something that would set Mailman to sleep about 1/10th of a second per recipient.
I just don't see the point of this. You may have stopped a large list inject being so aggressive on CPU usage, but you have increased the lifetime of processes by a factor of 30 or so (on your figures), and so increased the memory pressure and likelihood of swapping etc due to processes being just as fat but lasting longer.
The process we're talking about is OutgoingRunner, which is always in memory. I don't mind keeping the whole list in RAM -- I have 3 Gb, and the list (and chunks) probably eat up something like twice the size of the config.pck file, so about 44Mb. The only problem I see is if the list were locked during all that time, but I don't think (from what I understand from the code) that it is.
Your users may notice the big lists getting much slower - and having deliveries smeared over a much longer period.
Well that's already the case as postfix (my MTA) has 150 K messages to deliver, isn't it? But right now Mailman injects everything into postfix w/o considering that it might explode the load and system hit. It's brutal, with the load going to 25, and CPU to 70% (and I have a very nice machine, a brand new quadri-processor thing :-] ).
You may find you have made things less efficient by having 2 deliveries to a single list slowed down so that different messages to the same recipient can no longer be put in the same SMTP session (if your MTA does that).
I'm not sure postfix does that, no. And it's not really relevant in this case, as I really only have ONE big list with ONE message to send (and alot of small lists and normal emails to try to sneak through if possible).
For that matter can multiple deliveries be made against the same list at the same time?
Why not just run the cron jobs under nice instead?
What cron jobs? Sure I can renice the OutgoingRunner, but it won't change much (except that it will get swapped first if something needs to get swapped).