[Mailman-Users] Untangling an unruly jungle of lists

[mailmanu-20100705 at billmail.scconsult.com]

I have been tasked with cleaning up a collection of 23 lists running on 
Mailman that have been administered and moderated by a customer's 
non-technical staff for some time (> 2 years) without much training or 
assistance. At the top of my task list is the elimination of duplicate 
messages, which appear to be happening through two mechanisms:

1. Manually sending one message to multiple lists which have overlapping 
memberships.
2. A misnamed "all-lists" announcement list whose membership included a 
dozen of the other lists (with overlapping memberships) plus a handful of 
individual users who also were subscribed to those lists.

It has been many years and versions since I've worked with Mailman in any 
significant way, so I feel like I need a sanity check on my repair plans.

I have started with what seems to me to be the obvious first step to address 
(2). I have removed all of the other lists as members of "all-lists" and 
added their addresses to regular_include_lists instead, and since all of the 
direct subscribers wee in one or more of the sub-lists, I've removed them as 
well. This has left me with 4 questions:

1. Am I correct in my understanding that regular_include_lists does a 
de-duplication of the aggregate of all included lists?

2. Are the direct subscribers to the parent list included in the 
de-duplication? In other words: could I have left the direct subscribers in 
place without causing duplicates?

3. Since regular_include_lists is part of the non-digest settings, what 
exactly does a member of one or more of the included lists who is set to 
digest mode get when the parent list is mailed?

4. Are there non-obvious aspects of the regular_include_lists feature that I 
should be aware of, i.e. anything that argues in favor of using a script to 
assemble and de-dupe a membership list for it instead?

To handle the mailings to arbitrary sets of overlapping lists, I gather that 
regular_exclude_lists is the tool to use, but I am not sure exactly how to 
build the configuration. I have a set of 21 lists that have some users in 
common with 2-17 others members of that set. To evade the complexity of that 
setup, I'm starting with a set of 4 that the customer has cited as 
particularly problematic, and I think what I need to do to get 
cross-mailings to arbitrary combinations of them de-duped I need something 
like these regular_exclude_lists:

list1: list2, list3, list4
list2: list3, list4
list3: list4
list4: (empty)

So, if I really want to de-dupe arbitrary cross-mailings between all 21 
lists that have common members, I'd have to put lists 2-21 in for list1, 
3-21 for list2, etc., and if the customer wants a new list that might 
overlap, it would need to get all of the existing lists in its 
regular_exclude_lists.  Ugh. The questions I have about this:

1. Is that config correct?

2. Is there any useful basis for ordering the 4 lists I am making siblings? 
Should I make the biggest one list1 and the shortest list4 or should I order 
them based on the degree of overlap  or something else?

3. Does setting regular_exclude_lists have any effect on how lists are 
handled as members of another list's regular_include_lists?

4. Has anyone devised an easier and less fragile approach? Maybe for Mailman 
3? What I'd like most is a simple setting to have Mailman look at each 
message for other lists in the same domain or handled by the same server and 
work out the de-dupe cascade itself as needed.

More broadly and philosophically, I am curious about how others handle 
similar list jungles. My first instinct is to decree that there shall be a 
sane hierarchy of parent lists with regular_include_lists such that there 
shall be no need to cross-post, but that seems a bit unrealistic at this 
point. Any helpful clues?