[Mailman-Developers] Requirements for a new archiver

[J C Lawrence]

On Tue, 28 Oct 2003 12:49:51 -0500 
Barry Warsaw <barry at python.org> wrote:
> On Tue, 2003-10-28 at 00:41, J C Lawrence wrote:

>> If the URL is predictably based on the Message-ID this is not a
>> problem.

>> Quite, this is how/why NNTP uses Message-IDs are unique indexing
>> qualifiers.

> Yep, and we'd have to do the same thing (ensure that Message-IDs are
> unique).  

Yup.  Of course this heads directly into that beautiful debate of
whether MLMs should rewrite Message IDs.  Summarising briefly:

  If we rewrite all IDs we'll piss off the people who use ID to do dupe
  detection/deletion for courtesy copies.

  If we don't do some rewriting some messages won't make it through NNTP
  and some other people will be pissed off.

Two contrasting approaches:

  1) We guarantee uniqueness of all Message IDs.  The only way to do
  this is to rewrite all IDs.  This will piss off some people.

  2) We best-effort guarantee uniqueness by only guaranteeing uniqueness
  within the last N messages to the list.  This could be one by
  rewriting all IDs, in which case we might as well guarantee total
  uniqueness, or it could be done by keeping a DB of the last N (cf
  CDBD) and either discarding or rewriting detected collisions.  This of
  course means that some messages will be discarded by NNTP and we won't
  know about it.  Some may be willing to accept those risks.

Premise:

  Mailman just doesn't have enough configuration options.

My temptation would be to add the following configuration options to
Mailman:

  Keep track of the Messages IDs we've seen for the last N messages? (0
  disables, value sets N)

  How to handle messages with IDs we've seen before?  (discard/rewrite
  radio button)
  
  Replace the Message IDs for all the messages we rebroadcast with our
  own unique values? (checkbox)

> Note that we sometimes get shit from people who complain about
> Mailman's NNTP posting code modifying Message-IDs to adhere to the
> stricter NNTP requirements.  

Aye.  It is very easy to criticise.  It is a little less easy to define,
implement and maintain solutions which can't be criticised.

> But Mailman can't rely on the good graces of remote mail tools to
> ensure globally unique Message-IDs, so it has to check and munge if it
> gets a dup (or, it's within it's right to treat a dup /as/ a dup,
> e.g. discarding it).

Yeah, thus the above logic.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
claw at kanga.nu               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.