[John A. Martin]
"Harald" == Harald Meland
Harald> It is not clear to me that Mailman *is* an MTA. It is not Harald> an SMTP server, and is not (necessarily) an SMTP client.To have been precise perhaps I should have said something like "a mail agent must not muck with an existing Message-Id except as specified by the applicable standards". The Applicable Standards, to quote for example rfc2822,, apply as follows:
This standard specifies a syntax for text messages that are sent between computer users, within the framework of "electronic mail" messages.
I agree that it is obvious that Mailman should strive to avoid sending non-RFC2822-compliant messages.
However, I would think that the issue at hand is not about message *syntax*, but rather about the *semantic* value of a message's Message-Id.
Now that that nit is off my chest :-), I'll be quick to agree that RFC 2822 surely do contain a fair bit of semantic specifications as well; more on that below.
The applicable standards govern what goes on the wire and therefore what Mailman causes to be put on the wire through a MTA should be compliant.
Mailman is sort of between a rock and a hard place here, as it occupies a double role:
Mailman should be liberal in what it accepts -- which seems to imply that it should accept incoming messages even if they do not not conform strictly to all aspects of RFC 2822.
As one example, Mailman shouldn't offhandedly reject an incoming message just because there is a slight address syntax error in the message's From: header.
At the same time, Mailman should be conservative in what it sends. Naively, this would mean that Mailman ought to ensure that any message it puts on the wire conforms with RFC 2822; however, that would then have to either clash with the "liberal in what you expect" idea, or with the "don't change the message" maxim.
Harald> However, even if Mailman isn't an MTA, it would be nice if Harald> it *mostly* tries to follow the MTA rules. Harald> (As a side note, I am unable to find *clear* references to Harald> the effect of your statement in RFCs 2821 or 2822.)Rfc2822 Section 3.6.4 (the first paragraph below is the same paragraph you quoted elsewhere)
[[ ... ]]
The "Message-ID:" field provides a unique message identifier that refers to a particular version of a particular message. The uniqueness of the message identifier is guaranteed by the host that generates it (see below). This message identifier is intended to be machine readable and not necessarily meaningful to humans. A message identifier pertains to exactly one instantiation of a particular message; subsequent revisions to the message each receive new message identifiers.
Note: There are many instances when messages are "changed", but those changes do not constitute a new instantiation of that message, and therefore the message would not get a new message identifier. For example, when messages are introduced into the transport system, they are often prepended with additional header fields such as trace fields (described in section 3.6.7) and resent fields (described in section 3.6.6). The addition of such header fields does not change the identity of the message and therefore the original "Message-ID:" field is retained. In all cases, it is the meaning that the sender of the message wishes to convey (i.e., whether this is the same message or a different message) that determines whether or not the "Message-ID:" field changes, not any particular syntactic difference that appears (or does not appear) in the message.
Rfc822 Section 4.6.1 (in its entirety):
This field contains a unique identifier (the local-part address unit) which refers to THIS version of THIS message. The uniqueness of the message identifier is guaranteed by the host which generates it. This identifier is intended to be machine readable and not necessarily meaningful to humans. A message identifier pertains to exactly one instantiation of a particular message; subsequent revisions to the message should each receive new message identifiers.Rfc2822 in this case merely codifies long established practice interpreting rfc822. Rfc2822 Appendix A.3 may be helpful for the present discussion.
The part that (still) isn't clear to me, is whether Mailman's action of putting the message back on the wire can be said to be either 1) generation of a new message (personally, I wouldn't think so) or 2) a new instantiation of the message.
To test for compliance with the rfc2822 determination "whether this is the same message or a different message" one might stipulate that if the PGP signature verifies it is the same message, if the PGP signature does not verify it is a different message.
Now we're deeply into message semantics. :-)
I'd like to point out to things about your argument:
Firstly, the RFC does not merely distinguish between "the same message or a different message"; it also allows Message-ID: to be changed whenever there is a new instantiation of a (single) message.
Secondly, having to resort to (your) *interpretation* of the RFC, by using verification of PGP signatures for the test, is in my book a clear indication that the RFC is *not* crystal clear on this issue.
(One certainly can see by inspection what would break a signature without actually verifying the signature, right?)
That is my (rather shallow, I'm afraid) understanding of PGP email signatures, yes.
Harald> Um. Mailman lists have numerous configuration options for Harald> changing messages (e.g. adding footers) before they are Harald> sent to the list members, and it has had such options Harald> since time immemorial.Who reads the RFCs to say that footers cannot be added without changing the message?
The more interesting issue, I think, is where should the line be drawn; how much is Mailman allowed to change (various parts of) a message before it should be considered a new message?
And, how does the Mailman modus operandi fit in with the RFCs "new instantiation" use of words?
Harald> * To my mind it would not be obviously wrong to view Harald> Mailman as the *generator* of messages, at the very Harald> least in the cases where it is obvious that the Harald> previous generator didn't do its job of guaranteeing Harald> message-id uniqueness properly.Why?
Given that there exists two (or more) distinct messages that share the same message-id, the uniqueness of this identifier (as proscribed by RFC 2822) is clearly not satisfied. Hence, if Mailman really wants to have the messages it puts on the wire conform with RFC 2822, it should take on the role of message generator, and issue distinct message-ids for such distinct messages.
The hard problem, of course, is to properly discover whether or not two messages are indeed distinct; they might differ slightly by e.g. an automatically added footer, or in some other minor, but programmatically hard to discover, fashion.
ISTM the problem you are trying to solve is how to identify the archive image of the message.
Why not construct a URL containing a scrubbed Message-Id (as Brad Knowles has indicated) and a serial number (as I have indicated)?
Because, as Barry said, that would mean the "archive image identity" of the messages could change whenever the archive needs to be rebuilt (e.g. after a disk crash, the archives are gone, and there are no backups; then some kind list member comes forward with a partial archive constructed from the messages they've received from the list).
Such a URL should go into the "List-Archive" header field pointing to the specific message without doing violence to rfc2369 Section 3.6, right?
I don't think that's too far from the intention of that header, no. That section seems rather loosely worded, something I hope was done intentionally:
3.6. List-Archive
The List-Archive field describes how to access archives for the list.
Examples:
List-Archive: <mailto:archive@host.com?subject=index%20list>
List-Archive: <ftp://ftp.host.com/pub/list/archive/>
List-Archive: <http://www.host.com/list/archive/> (Web Archive)-- Harald