Re: [Mailman-Developers] New RFC on using DKIM with MLMs
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA.
I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
X-Message-ID-Hash X-Mailman-Rule-Hits X-Mailman-Rule-Misses X-BeenThere X-Mailman-Version X-Mailman-Approved-At X-Archive X-No-Archive X-Ack X-No-Ack X-Peer X-MailFrom X-RcptTo X-Originally-To X-Original-CC X-Original-Content-Transfer-Encoding X-MIME-Version X-Mailman-Copy X-List-Administrivia X-Content-Filtered-By X-Topics X-Mailer X-List-Received-Date X-Approve X-Approved
-Barry
On 10/24/2011 8:04 PM, Barry Warsaw wrote:
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA. I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
I believe the rule of thumb is you're supposed to use the X- prefix if it's not registered, so until the header is registered at IANA, I would vote that the X- prefix stays retained.
-- Joshua Cranmer News submodule owner DXR coauthor
Joshua Cranmer writes:
On 10/24/2011 8:04 PM, Barry Warsaw wrote:
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA.
I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
I would say that anything that is used only in the test suite should still get an X-, although I suppose you could use Mailman-Test- too.
I believe the rule of thumb is you're supposed to use the X- prefix if it's not registered, so until the header is registered at IANA, I would vote that the X- prefix stays retained.
What Murray is saying is that the rule of thumb is changing in response to experience. What has happened is that the experience with promoting an X-Foo header to just Foo has been poor, and the attendant confusion often hinders adoption. So many people have been in the habit of ignoring the X- namespace anyway (the most widespread example I know of is the adoption of Mail-Followup-To in mail, which has no[1] sanction in the RFCs, although it's a long-standard header in news).
Footnotes: [1] Last I checked, anyway, a couple years ago, but widespread usage dates back to at least 2003.
- Stephen J. Turnbull stephen@xemacs.org:
Joshua Cranmer writes:
On 10/24/2011 8:04 PM, Barry Warsaw wrote:
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA.
I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
I would say that anything that is used only in the test suite should still get an X-, although I suppose you could use Mailman-Test- too.
Is it possible to register a prefix (namespace) such as mailman-. Anything below would be mailman related. Stupid idea?
p@rick
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
Patrick Ben Koetter writes:
Is it possible to register a prefix (namespace) such as mailman-. Anything below would be mailman related. Stupid idea?
Very plausible, but I suspect there are good reasons not to allow it in the RFC 822/1036 messaging series. In any case, no, it's not possible.
There is a way to do it in other namespaces though, eg, in MIME media types you have the vnd.* space.
-----Original Message----- From: mailman-developers-bounces+msk=cloudmark.com@python.org [mailto:mailman-developers-bounces+msk=cloudmark.com@python.org] On Behalf Of Stephen J. Turnbull Sent: Tuesday, October 25, 2011 6:04 AM To: Patrick Ben Koetter Cc: mailman-developers@python.org Subject: Re: [Mailman-Developers] New RFC on using DKIM with MLMs
Patrick Ben Koetter writes:
Is it possible to register a prefix (namespace) such as mailman-. Anything below would be mailman related. Stupid idea?
Very plausible, but I suspect there are good reasons not to allow it in the RFC 822/1036 messaging series. In any case, no, it's not possible.
There is a way to do it in other namespaces though, eg, in MIME media types you have the vnd.* space.
I agree. The best you could do is just claim the "Mailman-*" space de facto by starting to use it, and then start registering the ones that appear to be useful and popular.
On 10/25/2011 2:47 AM, Stephen J. Turnbull wrote:
Joshua Cranmer writes:
On 10/24/2011 8:04 PM, Barry Warsaw wrote:
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA.
I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
I would say that anything that is used only in the test suite should still get an X-, although I suppose you could use Mailman-Test- too.
I believe the rule of thumb is you're supposed to use the X- prefix if it's not registered, so until the header is registered at IANA, I would vote that the X- prefix stays retained.
What Murray is saying is that the rule of thumb is changing in response to experience. What has happened is that the experience with promoting an X-Foo header to just Foo has been poor, and the attendant confusion often hinders adoption. So many people have been in the habit of ignoring the X- namespace anyway (the most widespread example I know of is the adoption of Mail-Followup-To in mail, which has no[1] sanction in the RFCs, although it's a long-standard header in news).
There is a formal procedure in place for the creation of new headers (RFC 3864); if you're going to drop the X-, you might as well at least attempt to get them provisionally accepted...
-- Joshua Cranmer News submodule owner DXR coauthor
On 25 Oct 2011, at 02:04, Barry Warsaw wrote:
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA.
I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
X-Message-ID-Hash
This could be replaced with DKIM sigs, I guess.
X-Mailman-Rule-Hits X-Mailman-Rule-Misses
This might be useful for diagnostics, but probably wants to be off in general. My view is that Mailman should not be doing message filtering.
X-BeenThere
I guess that's useful for avoiding list loops, perhaps?
X-Mailman-Version
I think this should be replaced with X-Mailer, or even User-Agent. That's not currently an SMTP header, but I think it should be. And it is in quite widespread use.
X-Mailman-Approved-At X-Archive
Does this do the same as List-Archive?
X-No-Archive
What does this mean?
X-Ack X-No-Ack X-Peer X-MailFrom X-RcptTo Isn't that usually in the Received header?
X-Originally-To
Doesn't that do the same thing as List-post?
X-Original-CC What's the purpose of including this?
X-Original-Content-Transfer-Encoding X-MIME-Version X-Mailman-Copy X-List-Administrivia
List-help?
X-Content-Filtered-By X-Topics X-Mailer
I think we should use User-Agent here. Thunderbird does, as do some other mail clients. Or, we should push for introduction of a List-Agent header.
X-List-Received-Date
Don't the Received headers carry this information?
X-Approve X-Approved
-Barry
Generally, I think we should avoid the use of headers that duplicate other existing headers. Where we want to add more information, then extend the List-* header set if the information might be generally useful for mailing list software. Otherwise, use X-Mailman-* (or even Mailman-*) so that people know where the header came from.
-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148
On 10/26/2011 5:43 AM, Ian Eiloart wrote:
On 25 Oct 2011, at 02:04, Barry Warsaw wrote:
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA.
If registering headers is to be pursued, perhaps a standardized list of headers should be discussed with other MLM's and then put forth to IANA. The standardized headers should be able to handle generic list information along with MLM specific. My thought would be for
List-* are generic headers all MLM's and MUA's can refer to for information. List-Mailman-* would be Mailman specific headers List-ListServ-* would be ListServ specific headers
I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
X-Message-ID-Hash This could be replaced with DKIM sigs, I guess.
X-Mailman-Rule-Hits X-Mailman-Rule-Misses This might be useful for diagnostics, but probably wants to be off in general. My view is that Mailman should not be doing message filtering.
X-BeenThere I guess that's useful for avoiding list loops, perhaps?
X-Mailman-Version I think this should be replaced with X-Mailer, or even User-Agent. That's not currently an SMTP header, but I think it should be. And it is in quite widespread use.
I agree and disagree. I may be wrong but I see the X-Mailer specifying the MTA and User-Agent specifying the MUA. Perhaps a different header added by the MLM called List-Agent or List-ListAgent.
X-Mailman-Approved-At X-Archive Does this do the same as List-Archive?
X-No-Archive What does this mean?
X-Ack X-No-Ack X-Peer X-MailFrom X-RcptTo Isn't that usually in the Received header?
X-Originally-To Doesn't that do the same thing as List-post?
X-Original-CC What's the purpose of including this?
X-Original-Content-Transfer-Encoding X-MIME-Version X-Mailman-Copy X-List-Administrivia List-help?
X-Content-Filtered-By X-Topics X-Mailer I think we should use User-Agent here. Thunderbird does, as do some other mail clients. Or, we should push for introduction of a List-Agent header.
List-Agent header is my vote (see above :))
X-List-Received-Date Don't the Received headers carry this information?
Maybe. They certainly contain the date the MTA received the message but not necessarily when the MLM received the message. Consider a message to be posted to a list. The MTA receives it and it is passed on to a Spam/AntiVirus device. The message is held at the device because is fails whatever checks are in place. Five days later, the device administrator goes through the held queue on the device, sees the message as okay and releases it. The MLM then receives the message to be posted. Now, when did the MLM really receive the message? When the MTA received it or five days later. I think there is enough of a possibility that keeping Received headers different from say a List-Received-Date header.
X-Approve X-Approved
-Barry Generally, I think we should avoid the use of headers that duplicate other existing headers. Where we want to add more information, then extend the List-* header set if the information might be generally useful for mailing list software. Otherwise, use X-Mailman-* (or even Mailman-*) so that people know where the header came from.
I agree. Reduce and avoid duplication. Discussions should also occur with other MLM's to come up with a generic List-* header set.
-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148
Hope I made sense, Thanks, Chris
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/cnulk%40scu.edu
Security Policy: http://wiki.list.org/x/QIA9
I searched mailman 2.1.14 sources and changelog to find out what they stand for. Read below. Which could/should we replace with already existing standardized headers?
- Ian Eiloart iane@sussex.ac.uk:
On 25 Oct 2011, at 02:04, Barry Warsaw wrote:
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA.
I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
X-Message-ID-Hash
This could be replaced with DKIM sigs, I guess.
X-Mailman-Rule-Hits X-Mailman-Rule-Misses
This might be useful for diagnostics, but probably wants to be off in general. My view is that Mailman should not be doing message filtering.
Are they related to SpamAssassin filtering that can take place in MM?
X-BeenThere
I guess that's useful for avoiding list loops, perhaps?
From Mailman/Handlers/CookHeaders.py:
# Mark message so we know we've been here, but leave any existing
# X-BeenThere's intact.X-Mailman-Version
I think this should be replaced with X-Mailer, or even User-Agent. That's not currently an SMTP header, but I think it should be. And it is in quite widespread use.
From Mailman/Handlers/CookHeaders.py:
msg['X-Mailman-Version'] = mm_cfg.VERSIONSeems to add the product version and not the User-Agent.
X-Mailman-Approved-At
From Mailman/ListAdmin.py:
# Queue the file for delivery by qrunner. Trying to deliver the
# message directly here can lead to a huge delay in web
# turnaround. Log the moderation and add a header.
msg['X-Mailman-Approved-At'] = email.Utils.formatdate(localtime=1)X-Archive
Does this do the same as List-Archive?
Looks like a control feature for senders:
From doc/mailman-admin.txt:
Note that senders can control whether their own posts are
archived, on an individual per-message basis. If the posted
message has a X-No-Archive: header (regardless of value), or a
X-Archive: header with a value of No (case insensitive), then
the message will not be archived, although it will be treated
as normal in all other ways.From the changelog:
o When a moderated message is approved for the list, add an
X-Mailman-Approved-At: header which contains the timestamp
of the approval action (changed from X-Moderated: with a
different format).
- Honor "X-Archive: No" header by not putting this message in the
archive.X-No-Archive
What does this mean?
Seems like a dupe from X-Archive:
From the changelog:
o Just the mere presence of an X-No-Archive: is enough to
inhibit archiving for this message; the value of the header
is now ignored.X-Ack
From the changelog: - Autoresponses, -request command responses, and posting hold notifications are inhibited for any message that has a Precedence: {bulk|list|junk} header. This is to avoid mail loops between email 'bots. If the original message has an X-Ack: yes header, the response is sent.
X-No-Ack
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
X-Peer
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
X-MailFrom
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
X-RcptTo
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
Isn't that usually in the Received header?
X-Originally-To
Doesn't that do the same thing as List-post?
Seems like a copy of Postfix' X-Original-To-header. Is it?
From cron/gate_news:
del msg['X-Originally-To']
msg['X-Originally-To'] = msg['To']X-Original-CC What's the purpose of including this?
From Mailman/Defaults.py.in:
# Next, these headers are left alone, unless there are duplicates in the
# original message. Any second and subsequent headers are rewritten to the
# second named header (case preserved).
NNTP_REWRITE_DUPLICATE_HEADERS = [
('to', 'X-Original-To'),
('cc', 'X-Original-Cc'),
('content-transfer-encoding', 'X-Original-Content-Transfer-Encoding'),
('mime-version', 'X-MIME-Version'),
]X-Original-Content-Transfer-Encoding
From Mailman/Defaults.py.in:
# Next, these headers are left alone, unless there are duplicates in the
# original message. Any second and subsequent headers are rewritten to the
# second named header (case preserved).
NNTP_REWRITE_DUPLICATE_HEADERS = [
('to', 'X-Original-To'),
('cc', 'X-Original-Cc'),
('content-transfer-encoding', 'X-Original-Content-Transfer-Encoding'),
('mime-version', 'X-MIME-Version'),
]X-MIME-Version
From Mailman/Defaults.py.in:
# Next, these headers are left alone, unless there are duplicates in the
# original message. Any second and subsequent headers are rewritten to the
# second named header (case preserved).
NNTP_REWRITE_DUPLICATE_HEADERS = [
('to', 'X-Original-To'),
('cc', 'X-Original-Cc'),
('content-transfer-encoding', 'X-Original-Content-Transfer-Encoding'),
('mime-version', 'X-MIME-Version'),
]X-Mailman-Copy
Nothing in the changelog
From templates/en/options.html:
Avoid duplicate copies of messages?
When you are listed explicitly in the To: or Cc: headers of a list
message, you can opt to not receive another copy from the mailing
list. Select Yes to avoid receiving copies from the mailing list;
select No to receive copies.
If the list has member personalized messages enabled, and you elect to
receive copies, every copy will have a X-Mailman-Copy: yes header
added to it. From Mailman/Handlers/SMTPDirect.py:
# We can flag the mail as a duplicate for each member, if they've
# already received this message, as calculated by Message-ID. See
# AvoidDuplicates.py for details.
del msgcopy['x-mailman-copy']
if msgdata.get('add-dup-header', {}).has_key(recip):
msgcopy['X-Mailman-Copy'] = 'yes'X-List-Administrivia
From the changelog: o X-List-Administrivia: is only added to messages Mailman creates and sends out of its own accord.
From Mailman/Handlers/CookHeaders.py:
# For internally crafted messages, we also add a (nonstandard),
# "X-List-Administrivia: yes" header. For all others (i.e. those coming
# from list posts), we add a bunch of other RFC 2369 headers.List-help?
X-Content-Filtered-By
Nothing in the changelog
From Mailman/Handlers/MimeDel.py:
# If we're left with only two parts, an empty body and one attachment,
# recast the message to one of just that partX-Topics
Nothing in the changelog
From Mailman/Handlers/Tagger.py:
# For each regular expression in the topics list, see if any of the lines
# of interest from the message match the regexp. If so, the message gets
# added to the specific topics bucket.X-Mailer
I think we should use User-Agent here. Thunderbird does, as do some other mail clients. Or, we should push for introduction of a List-Agent header.
Nothing in the changelog
Some references to code in Mailman/Bouncers/* Dunno what it does.
X-List-Received-Date
Don't the Received headers carry this information?
From the changelog: The archived copy of messages grows an X-List-Received-Date: header indicating the time the message was received by Mailman.
# Always put an indication of when we received the message.
Seems to decide where messages should be archivedX-Approve
Nothing in the changelog
From Mailman/Handlers/Approve.py:
# See if the message has an Approved or Approve header with a valid
# list-moderator, list-admin. Also look at the first non-whitespace line
# in the file to see if it looks like an Approved header. We are
# specifically /not/ allowing the site admins password to work here
# because we want to discourage the practice of sending the site admin
# password through email in the clear.X-Approved
Nothing in the changelog
X-Approve dupe?
Generally, I think we should avoid the use of headers that duplicate other existing headers. Where we want to add more information, then extend the List-* header set if the information might be generally useful for mailing list software. Otherwise, use X-Mailman-* (or even Mailman-*) so that people know where the header came from.
+1
p@rick
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
On 26 Oct 2011, at 16:33, Patrick Ben Koetter wrote:
I searched mailman 2.1.14 sources and changelog to find out what they stand for. Read below. Which could/should we replace with already existing standardized headers?
- Ian Eiloart iane@sussex.ac.uk:
On 25 Oct 2011, at 02:04, Barry Warsaw wrote:
On Oct 13, 2011, at 11:41 PM, Murray S. Kucherawy wrote:
There's movement afoot to deprecate use of "X-" in header field names. Just call it "Mailman-Topic". And if it's worthwhile, consider registering it with IANA.
I wonder if we should remove the X- prefixes for Mailman 3. Here's a list of ones we still add or recognize (some might be used only in the test suite):
X-Message-ID-Hash
This could be replaced with DKIM sigs, I guess.
X-Mailman-Rule-Hits X-Mailman-Rule-Misses
This might be useful for diagnostics, but probably wants to be off in general. My view is that Mailman should not be doing message filtering.
Are they related to SpamAssassin filtering that can take place in MM?
X-BeenThere
I guess that's useful for avoiding list loops, perhaps?
From Mailman/Handlers/CookHeaders.py:
# Mark message so we know we've been here, but leave any existing # X-BeenThere's intact.
The message that I'm replying to carried this header: X-Beenthere: mailman-developers@python.org
I guess that should stay, then. Unless we find a different place to put it. Moving it to a different place probably would not cause problems, depending on how it's used. If the header means "don't relay the message to this address", then there would be problems. If it means "drop this message, if you are mailman-developers@python.org" then it can be moved.
X-Mailman-Version
I think this should be replaced with X-Mailer, or even User-Agent. That's not currently an SMTP header, but I think it should be. And it is in quite widespread use.
From Mailman/Handlers/CookHeaders.py:
msg['X-Mailman-Version'] = mm_cfg.VERSIONSeems to add the product version and not the User-Agent.
Yes, but a User-Agent, header would have the product and the product version. A List-Agent header should do the same.
X-Mailman-Approved-At
From Mailman/ListAdmin.py:
# Queue the file for delivery by qrunner. Trying to deliver the # message directly here can lead to a huge delay in web # turnaround. Log the moderation and add a header. msg['X-Mailman-Approved-At'] = email.Utils.formatdate(localtime=1
So, I guess that the web moderation works by adding this header, so that the message can be delivered when the queue runner sees it. It looks like useful trace information, so it should stay.
This also looks like a candidate for, say, a List-Approved-Date header.
X-Archive
Does this do the same as List-Archive?
Looks like a control feature for senders:
From doc/mailman-admin.txt:
Note that senders can control whether their own posts are archived, on an individual per-message basis. If the posted message has a X-No-Archive: header (regardless of value), or a X-Archive: header with a value of No (case insensitive), then the message will not be archived, although it will be treated as normal in all other ways.From the changelog:
o When a moderated message is approved for the list, add an X-Mailman-Approved-At: header which contains the timestamp of the approval action (changed from X-Moderated: with a different format).- Honor "X-Archive: No" header by not putting this message in the archive.
Ah, well that's also a feature that various lists might want to implement. However, List-Archive says where archives are kept. Also, I guess an MUA is never going to have a guarantee that an MLM will honour a request. To use a List-* header might confuse, since all other List-* headers are added by the MLM.
Actually, I think that perhaps it and X-No-Archive could be deprecated. Does anyone actually use them?
X-No-Archive
What does this mean?
Seems like a dupe from X-Archive: From the changelog:
o Just the mere presence of an X-No-Archive: is enough to inhibit archiving for this message; the value of the header is now ignored.
See above.
X-Ack
From the changelog:
- Autoresponses, -request command responses, and posting hold notifications are inhibited for any message that has a Precedence: {bulk|list|junk} header. This is to avoid mail loops between email 'bots. If the original message has an X-Ack: yes header, the response is sent.
X-No-Ack
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
I think X-Ack and X-No-Ack could be deprecated. Just advise people to use the correct precedence.
X-Peer
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
I guess that should be deprecated, then!
X-MailFrom
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
I guess that should be deprecated, then!
X-RcptTo
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
I guess that should be deprecated, then!
Isn't that usually in the Received header?
X-Originally-To
Doesn't that do the same thing as List-post?
Seems like a copy of Postfix' X-Original-To-header. Is it?
From cron/gate_news:
del msg['X-Originally-To'] msg['X-Originally-To'] = msg['To']
Does Mailman 3 gate news?
X-Original-CC What's the purpose of including this?
From Mailman/Defaults.py.in:
# Next, these headers are left alone, unless there are duplicates in the # original message. Any second and subsequent headers are rewritten to the # second named header (case preserved). NNTP_REWRITE_DUPLICATE_HEADERS = [ ('to', 'X-Original-To'), ('cc', 'X-Original-Cc'), ('content-transfer-encoding', 'X-Original-Content-Transfer-Encoding'), ('mime-version', 'X-MIME-Version'), ]
So, this is about tracking original message headers. Why do we want to do that? Why not leave them intact?
X-Original-Content-Transfer-Encoding
From Mailman/Defaults.py.in:
# Next, these headers are left alone, unless there are duplicates in the # original message. Any second and subsequent headers are rewritten to the # second named header (case preserved). NNTP_REWRITE_DUPLICATE_HEADERS = [ ('to', 'X-Original-To'), ('cc', 'X-Original-Cc'), ('content-transfer-encoding', 'X-Original-Content-Transfer-Encoding'), ('mime-version', 'X-MIME-Version'), ]X-MIME-Version
From Mailman/Defaults.py.in:
# Next, these headers are left alone, unless there are duplicates in the # original message. Any second and subsequent headers are rewritten to the # second named header (case preserved). NNTP_REWRITE_DUPLICATE_HEADERS = [ ('to', 'X-Original-To'), ('cc', 'X-Original-Cc'), ('content-transfer-encoding', 'X-Original-Content-Transfer-Encoding'), ('mime-version', 'X-MIME-Version'), ]X-Mailman-Copy
Nothing in the changelog
From templates/en/options.html:
Avoid duplicate copies of messages? When you are listed explicitly in the To: or Cc: headers of a list message, you can opt to not receive another copy from the mailing list. Select Yes to avoid receiving copies from the mailing list; select No to receive copies. If the list has member personalized messages enabled, and you elect to receive copies, every copy will have a X-Mailman-Copy: yes header added to it.From Mailman/Handlers/SMTPDirect.py:
# We can flag the mail as a duplicate for each member, if they've # already received this message, as calculated by Message-ID. See # AvoidDuplicates.py for details. del msgcopy['x-mailman-copy'] if msgdata.get('add-dup-header', {}).has_key(recip): msgcopy['X-Mailman-Copy'] = 'yes'
This seems to be redundant. The Received headers will tell me which copy was processed by Mailman.
X-List-Administrivia
From the changelog: o X-List-Administrivia: is only added to messages Mailman creates and sends out of its own accord.
From Mailman/Handlers/CookHeaders.py:
# For internally crafted messages, we also add a (nonstandard), # "X-List-Administrivia: yes" header. For all others (i.e. those coming # from list posts), we add a bunch of other RFC 2369 headers.
This seems like a candidate for a (collection?) of new List-* headers. It could assist MUAs, for example, in identifying the nature of messages generated by lists.
List-help?
X-Content-Filtered-By
Nothing in the changelog
From Mailman/Handlers/MimeDel.py:
# If we're left with only two parts, an empty body and one attachment, # recast the message to one of just that partX-Topics
Nothing in the changelog
From Mailman/Handlers/Tagger.py:
# For each regular expression in the topics list, see if any of the lines # of interest from the message match the regexp. If so, the message gets # added to the specific topics bucket.
So, is this used by Mailman to decide who to send the message to? Or is it supposed to help recipients to build filters. Either way, it might be useful for the latter purpose. Perhaps it's a candidate for List-Topics?
X-Mailer
I think we should use User-Agent here. Thunderbird does, as do some other mail clients. Or, we should push for introduction of a List-Agent header.
Nothing in the changelog
Some references to code in Mailman/Bouncers/* Dunno what it does.
The message that I'm replying to doesn't seem to contain an X-mailer header.
X-List-Received-Date
Don't the Received headers carry this information?
From the changelog: The archived copy of messages grows an X-List-Received-Date: header indicating the time the message was received by Mailman.
# Always put an indication of when we received the message. Seems to decide where messages should be archived
I think that's a candidate for a List-Archived-Date header. Because that's potentially helpful for people looking to find the message in an archive.
X-Approve
Nothing in the changelog
From Mailman/Handlers/Approve.py:
# See if the message has an Approved or Approve header with a valid # list-moderator, list-admin. Also look at the first non-whitespace line # in the file to see if it looks like an Approved header. We are # specifically /not/ allowing the site admins password to work here # because we want to discourage the practice of sending the site admin # password through email in the clear.
X-Approved
Nothing in the changelog
X-Approve dupe?
Generally, I think we should avoid the use of headers that duplicate other existing headers. Where we want to add more information, then extend the List-* header set if the information might be generally useful for mailing list software. Otherwise, use X-Mailman-* (or even Mailman-*) so that people know where the header came from.
+1
p@rick
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/iane%40sussex.ac.u...
Security Policy: http://wiki.list.org/x/QIA9
-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148
On 10/27/2011 7:29 AM, Ian Eiloart wrote:
On 26 Oct 2011, at 16:33, Patrick Ben Koetter wrote:
[...]
The message that I'm replying to carried this header: X-Beenthere: mailman-developers@python.org
I guess that should stay, then. Unless we find a different place to put it. Moving it to a different place probably would not cause problems, depending on how it's used. If the header means "don't relay the message to this address", then there would be problems. If it means "drop this message, if you are mailman-developers@python.org" then it can be moved.
It means the latter - "drop this message, if you are mailman-developers@python.org".
[...]
X-Mailman-Approved-At
From Mailman/ListAdmin.py:
# Queue the file for delivery by qrunner. Trying to deliver the # message directly here can lead to a huge delay in web # turnaround. Log the moderation and add a header. msg['X-Mailman-Approved-At'] = email.Utils.formatdate(localtime=1So, I guess that the web moderation works by adding this header, so that the message can be delivered when the queue runner sees it. It looks like useful trace information, so it should stay.
It has nothing to do with delivery. Approval for delivery is noted by flags in the message's metadata. The X-Mailman-Approved-At: header is added solely for documentation to explain delays to message recipients.
This also looks like a candidate for, say, a List-Approved-Date header.
Yes.
[...]
X-Peer
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
I guess that should be deprecated, then!
This header is used in MM 3.
X-MailFrom
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
I guess that should be deprecated, then!
This header is used in MM 3.
X-RcptTo
Is this still being used?
Nothing in the changelog Nothing in the mailman-2.1.14 sources.
I guess that should be deprecated, then!
It is only in the headers of of some test bounce messages. It is not in the code. It is a header added to the headers of a bounce notification (DSN) from (I'll refrain from giving my opinion) the Windows MTA SMTPD32. You'd have to ask ipswitch if you want to know what it means, but it appears to duplicate the To: header.
[...]
X-Topics
Nothing in the changelog
From Mailman/Handlers/Tagger.py:
# For each regular expression in the topics list, see if any of the lines # of interest from the message match the regexp. If so, the message gets # added to the specific topics bucket.
So, is this used by Mailman to decide who to send the message to? Or is it supposed to help recipients to build filters. Either way, it might be useful for the latter purpose. Perhaps it's a candidate for List-Topics?
If Topics are enabled for a list, Mailman decides the topics of a post by matching the defining topic regexps against the Subject: and any Keywords: header or pseudo-header.
The X-Topics: header is added to the message and contains the names of all matching topics, but this is informational only. Delivery to those users subscribed to matching topics is controlled by an equivalent list of matching topics in the message's metadata.
X-Mailer
I think we should use User-Agent here. Thunderbird does, as do some other mail clients. Or, we should push for introduction of a List-Agent header.
Nothing in the changelog
Some references to code in Mailman/Bouncers/* Dunno what it does.
The message that I'm replying to doesn't seem to contain an X-mailer header.
The header is added by some mailers to identify themselves. It is used by some of Mailman's heuristic bounce recognizers as part of their recognition of non-standard DSNs.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Oct 27, 2011, at 02:29 PM, Ian Eiloart wrote:
The message that I'm replying to carried this header: X-Beenthere: mailman-developers@python.org
Note that I responded in the context of Mailman 3. We have an opportunity there to clean that all up, but it probably isn't a good idea to radically change these for MM2.
From Mailman/Handlers/CookHeaders.py:
msg['X-Mailman-Version'] = mm_cfg.VERSIONSeems to add the product version and not the User-Agent.
Yes, but a User-Agent, header would have the product and the product version. A List-Agent header should do the same.
List-Agent is not a bad idea.
X-Mailman-Approved-At
From Mailman/ListAdmin.py:
# Queue the file for delivery by qrunner. Trying to deliver the # message directly here can lead to a huge delay in web # turnaround. Log the moderation and add a header. msg['X-Mailman-Approved-At'] = email.Utils.formatdate(localtime=1So, I guess that the web moderation works by adding this header, so that the message can be delivered when the queue runner sees it. It looks like useful trace information, so it should stay.
This also looks like a candidate for, say, a List-Approved-Date header.
That would make sense too.
From cron/gate_news:
del msg['X-Originally-To'] msg['X-Originally-To'] = msg['To']Does Mailman 3 gate news?
It's supposed to. :)
From Mailman/Defaults.py.in:
# Next, these headers are left alone, unless there are duplicates in the # original message. Any second and subsequent headers are rewritten to the # second named header (case preserved). NNTP_REWRITE_DUPLICATE_HEADERS = [ ('to', 'X-Original-To'), ('cc', 'X-Original-Cc'), ('content-transfer-encoding', 'X-Original-Content-Transfer-Encoding'), ('mime-version', 'X-MIME-Version'), ]So, this is about tracking original message headers. Why do we want to do that? Why not leave them intact?
Since this is a configuration option, they're under the control of the site administrator. We can certainly discuss whether it makes sense to change or extend the defaults. The values are the same in MM3, even if they're expressed differently in the configuration file.
From Mailman/Handlers/Tagger.py:
# For each regular expression in the topics list, see if any of the lines # of interest from the message match the regexp. If so, the message gets # added to the specific topics bucket.
So, is this used by Mailman to decide who to send the message to? Or is it supposed to help recipients to build filters. Either way, it might be useful for the latter purpose. Perhaps it's a candidate for List-Topics?
Are there any other MLMs that support topics in a way that would make that header generally useful?
From the changelog: The archived copy of messages grows an X-List-Received-Date: header indicating the time the message was received by Mailman.
# Always put an indication of when we received the message. Seems to decide where messages should be archivedI think that's a candidate for a List-Archived-Date header. Because that's potentially helpful for people looking to find the message in an archive.
Possibly so, if it's a header added by the MLM. E.g. an archiver might have its own "archive date" value so it would either have to chose a different header, or use multiple instances (with the loss of fidelity).
-Barry
I've begun to sort the various comments into a list that sorts all changes in categories like 'decide', 'delete', 'modify', 'keep'. I will send it to mailman-developers@python.org once I've clarified a few header fields.
As a reference I looked for existing list-relevant header fields in RFC 2369 http://www.rfc-editor.org/rfc/rfc2369.txt, which doesn't seem to have seen an update in a while.
All my comments to List-* headers refer to RFC 2369. Please find them below including some minor nitpicking at the end.
- Barry Warsaw barry@list.org:
On Oct 27, 2011, at 02:29 PM, Ian Eiloart wrote: Note that I responded in the context of Mailman 3. We have an opportunity there to clean that all up, but it probably isn't a good idea to radically change these for MM2.
ACK. I'd keep MM2 as it is and introduce changes for MM3 only.
From Mailman/Handlers/CookHeaders.py:
msg['X-Mailman-Version'] = mm_cfg.VERSIONSeems to add the product version and not the User-Agent.
Yes, but a User-Agent, header would have the product and the product version. A List-Agent header should do the same.
List-Agent is not a bad idea.
It's not available in RFC 2369. We will have to propose 'List-Agent' as a new standard. Should we?
X-Mailman-Approved-At
From Mailman/ListAdmin.py:
# Queue the file for delivery by qrunner. Trying to deliver the # message directly here can lead to a huge delay in web # turnaround. Log the moderation and add a header. msg['X-Mailman-Approved-At'] = email.Utils.formatdate(localtime=1So, I guess that the web moderation works by adding this header, so that the message can be delivered when the queue runner sees it. It looks like useful trace information, so it should stay.
This also looks like a candidate for, say, a List-Approved-Date header.
It's not available in RFC 2369. We will have to propose 'List-Approved-Date' as a new standard. Should we?
From Mailman/Handlers/Tagger.py:
# For each regular expression in the topics list, see if any of the lines # of interest from the message match the regexp. If so, the message gets # added to the specific topics bucket.
So, is this used by Mailman to decide who to send the message to? Or is it supposed to help recipients to build filters. Either way, it might be useful for the latter purpose. Perhaps it's a candidate for List-Topics?
Are there any other MLMs that support topics in a way that would make that header generally useful?
Not that I am aware of. But maybe we could give it a more generic view. I could imagine calling them 'Tags', which would make them usable in any protocol that sends headers.
From the changelog: The archived copy of messages grows an X-List-Received-Date: header indicating the time the message was received by Mailman.
# Always put an indication of when we received the message. Seems to decide where messages should be archivedI think that's a candidate for a List-Archived-Date header. Because that's potentially helpful for people looking to find the message in an archive.
Possibly so, if it's a header added by the MLM. E.g. an archiver might have its own "archive date" value so it would either have to chose a different header, or use multiple instances (with the loss of fidelity).
When I read 'List-Archived-Date' I understand it specifies the date the message had been archived by some archival software and not the date is was sent to the archive by an MLM.
That makes a difference to me because copying the message to an archive might not be the only way to transport it there. The MLM might as well use an LMTP transport and queue it up for delivery. The queued message might get stuck in a delivery queue for hours. Being able to tell the difference between 'sent off to' and 'stored at' might be very useful to a postmaster when debugging archival problems.
Choosing a more specific header field would probably also avoid getting into an archivers way when it creats an "archive date" value as Barry mentioned above.
Not sure. Am I overly pedantic on this?
p@rick
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
On Oct 28, 2011, at 10:36 PM, Patrick Ben Koetter wrote:
From Mailman/Handlers/CookHeaders.py:
msg['X-Mailman-Version'] = mm_cfg.VERSIONSeems to add the product version and not the User-Agent.
Yes, but a User-Agent, header would have the product and the product version. A List-Agent header should do the same.
List-Agent is not a bad idea.
It's not available in RFC 2369. We will have to propose 'List-Agent' as a new standard. Should we?
I think it makes sense to have a header identifying the MLM that the message flowed through, and List-Agent seems like a good choice.
X-Mailman-Approved-At
From Mailman/ListAdmin.py:
# Queue the file for delivery by qrunner. Trying to deliver the # message directly here can lead to a huge delay in web # turnaround. Log the moderation and add a header. msg['X-Mailman-Approved-At'] = email.Utils.formatdate(localtime=1So, I guess that the web moderation works by adding this header, so that the message can be delivered when the queue runner sees it. It looks like useful trace information, so it should stay.
This also looks like a candidate for, say, a List-Approved-Date header.
It's not available in RFC 2369. We will have to propose 'List-Approved-Date' as a new standard. Should we?
I also think it makes sense to have a date header marking the time when the message was approved. This implies of course that the message was held for moderation. Would there be a L-A-D header if the posting was automatically accepted? Maybe the header should be List-Posted-Date, although there some subtly different semantics implied by that. E.g. if the date the message was approved isn't quite the date the message was posted (idk, some delay from moderator acceptance to actually posting through the MLM).
Another header that might be useful here would be List-Approved-By which could be the name or email address of the moderator who approved it. Right now, MM3 doesn't fill that in, and it could of course be filled in by say list-owner@example.com, but in MM3 it could be potentially filled in with the preferred address for the moderator that approved it.
From Mailman/Handlers/Tagger.py:
# For each regular expression in the topics list, see if any of the lines # of interest from the message match the regexp. If so, the message gets # added to the specific topics bucket.
So, is this used by Mailman to decide who to send the message to? Or is it supposed to help recipients to build filters. Either way, it might be useful for the latter purpose. Perhaps it's a candidate for List-Topics?
Are there any other MLMs that support topics in a way that would make that header generally useful?
Not that I am aware of. But maybe we could give it a more generic view. I could imagine calling them 'Tags', which would make them usable in any protocol that sends headers.
Hmm, adopting a hash-tags format here would be kind of interesting for interop with social networking sites. It wouldn't have to be reflected in the name of the header, but it could be SHOULD in the spec. I ought to be pretty trivial for MM3 generate hash-tags here.
What's interesting of course is that the Mailman module that fills this information in is the tagger handler.
From the changelog: The archived copy of messages grows an X-List-Received-Date: header indicating the time the message was received by Mailman.
# Always put an indication of when we received the message. Seems to decide where messages should be archivedI think that's a candidate for a List-Archived-Date header. Because that's potentially helpful for people looking to find the message in an archive.
Possibly so, if it's a header added by the MLM. E.g. an archiver might have its own "archive date" value so it would either have to chose a different header, or use multiple instances (with the loss of fidelity).
When I read 'List-Archived-Date' I understand it specifies the date the message had been archived by some archival software and not the date is was sent to the archive by an MLM.
Yep, to me too.
That makes a difference to me because copying the message to an archive might not be the only way to transport it there. The MLM might as well use an LMTP transport and queue it up for delivery. The queued message might get stuck in a delivery queue for hours. Being able to tell the difference between 'sent off to' and 'stored at' might be very useful to a postmaster when debugging archival problems.
Agreed.
Choosing a more specific header field would probably also avoid getting into an archivers way when it creats an "archive date" value as Barry mentioned above.
Not sure. Am I overly pedantic on this?
No, I think it's an important distinction. Also note that in MM3 you can actually have any number of archivers, so I think you want a field that clearly describes the actions of the MLM. I'm not sure what that would be other than List-Archived-Date though. :/
Suggestions?
-Barry
Barry Warsaw writes:
I think it makes sense to have a header identifying the MLM that the message flowed through, and List-Agent seems like a good choice.
Are lists the only agents that forward-with-changes? (Obviously MTAs forward, and they do in fact make changes to the headers, but those changes are restricted to those mandated by RFCs. And they don't touch bodies, ever, AFAIK, with the exception of From-stuffing.)
What about the case of iterated lists (eg, internal aggregators used by an organization)?
On 10/28/11 8:42 PM, Stephen J. Turnbull wrote:
Barry Warsaw writes:
I think it makes sense to have a header identifying the MLM that the message flowed through, and List-Agent seems like a good choice.
Are lists the only agents that forward-with-changes? (Obviously MTAs forward, and they do in fact make changes to the headers, but those changes are restricted to those mandated by RFCs. And they don't touch bodies, ever, AFAIK, with the exception of From-stuffing.)
Actually, MTAs might adjust the body, in specific ways if needed to change 8 bit encoded messages into a 7 bit safe encoding if needed for transport.
-- Richard Damon
- Barry Warsaw barry@list.org:
On Oct 28, 2011, at 10:36 PM, Patrick Ben Koetter wrote:
From Mailman/Handlers/CookHeaders.py:
msg['X-Mailman-Version'] = mm_cfg.VERSIONSeems to add the product version and not the User-Agent.
Yes, but a User-Agent, header would have the product and the product version. A List-Agent header should do the same.
List-Agent is not a bad idea.
It's not available in RFC 2369. We will have to propose 'List-Agent' as a new standard. Should we?
I think it makes sense to have a header identifying the MLM that the message flowed through, and List-Agent seems like a good choice.
Let me merge Stephen's comment from 87obx04ocb.fsf@uwakimon.sk.tsukuba.ac.jp to stick with a more generic term such as 'User-Agent'.
While I disagree with having Mailman be a User-Agent, I can't actually
say there are useful semantics to having a separate List-Agent. Nor
do I see a real problem with having multiple User-Agent headers, if
multiple agents have handled the message.I agree with Stephen that List-Agent is too specific leaving no semantic room for other types of message processing software. At the same time I think User-Agent is not apt, because RFC 1945 defines a User-Agent as originator, which is not the case with any MLM - it distributes a message which originated somewhere else:
The User-Agent request-header field contains information about the
user agent originating the request.
-- http://www.ietf.org/rfc/rfc1945.txtI suggest we use the term 'Mediator' as introduced by D. Crocker in RFC 5598 http://www.rfc-editor.org/in-notes/rfc5598.txt instead:
A Mediator attempts to preserve the original Author's information in the message it reformulates but is permitted to make meaningful changes to the message content or envelope.
A Mediator's role is complex and contingent, for example, modifying and adding content or regulating which Users are allowed to participate and when. The common example of this role is a group Mailing List.
(see section "2.1.4. Mediator" and also section "5. Mediators")
A 'Mediator' would leave the original User-Agent intact AND tell the message had been processed by another mail handling service AND it would allow for other Mediators to be added in case more mail processing was done after an MLM's (here: Mailman) job.
> X-Mailman-Approved-At
From Mailman/ListAdmin.py:
# Queue the file for delivery by qrunner. Trying to deliver the # message directly here can lead to a huge delay in web # turnaround. Log the moderation and add a header. msg['X-Mailman-Approved-At'] = email.Utils.formatdate(localtime=1So, I guess that the web moderation works by adding this header, so that the message can be delivered when the queue runner sees it. It looks like useful trace information, so it should stay.
This also looks like a candidate for, say, a List-Approved-Date header.
It's not available in RFC 2369. We will have to propose 'List-Approved-Date' as a new standard. Should we?
I also think it makes sense to have a date header marking the time when the message was approved. This implies of course that the message was held for moderation. Would there be a L-A-D header if the posting was automatically accepted? Maybe the header should be List-Posted-Date, although there some subtly different semantics implied by that. E.g. if the date the message was approved isn't quite the date the message was posted (idk, some delay from moderator acceptance to actually posting through the MLM).
Hmm, if there are no intermediate processes between receiving a message and approving it a List-Approved-Date seems fine. But if there are we run into the same problem as described below with List-Archived-Date - you can't tell when it was queued and when processing took place.
Adding a second header might make the useful distinction:
List-Received-Date RFC 2822 date timestamp when message was received by MLM
List-Approved-Date RFC 2822 date timestamp when message was approved by moderator
Another header that might be useful here would be List-Approved-By which could be the name or email address of the moderator who approved it. Right now, MM3 doesn't fill that in, and it could of course be filled in by say list-owner@example.com, but in MM3 it could be potentially filled in with the preferred address for the moderator that approved it.
I see the benefit because it helps if you moderate in a team. But I fear the anger of people whose postings we decline. They search for moderator identities and then start molesting them e.g. by subscribing them to mailing lists that don't require opt-in. (Happend to me python.org postmaster. The angry person subscribed my address to various pr0n mailing lists and it took me weeks to get unsubscribed.)
All this said, I'd say the policy should be to add it as an optional header field.
From Mailman/Handlers/Tagger.py:
# For each regular expression in the topics list, see if any of the lines # of interest from the message match the regexp. If so, the message gets # added to the specific topics bucket.
So, is this used by Mailman to decide who to send the message to? Or is it supposed to help recipients to build filters. Either way, it might be useful for the latter purpose. Perhaps it's a candidate for List-Topics?
Are there any other MLMs that support topics in a way that would make that header generally useful?
Not that I am aware of. But maybe we could give it a more generic view. I could imagine calling them 'Tags', which would make them usable in any protocol that sends headers.
Hmm, adopting a hash-tags format here would be kind of interesting for interop with social networking sites. It wouldn't have to be reflected in the name of
ACK with the notion that hashtag seems to closely realted with twitter and a more general 'tag' would stay away from that.
Is there or should there be a distinction between 'tag' and http 'keywords' http://en.wikipedia.org/wiki/Meta_element#The_keywords_attribute? Should we use 'keywords' instead?
the header, but it could be SHOULD in the spec. I ought to be pretty trivial for MM3 generate hash-tags here.
What's interesting of course is that the Mailman module that fills this information in is the tagger handler.
There you go ... ;)
From the changelog: The archived copy of messages grows an X-List-Received-Date: header indicating the time the message was received by Mailman.
# Always put an indication of when we received the message. Seems to decide where messages should be archivedI think that's a candidate for a List-Archived-Date header. Because that's potentially helpful for people looking to find the message in an archive.
Possibly so, if it's a header added by the MLM. E.g. an archiver might have its own "archive date" value so it would either have to chose a different header, or use multiple instances (with the loss of fidelity).
When I read 'List-Archived-Date' I understand it specifies the date the message had been archived by some archival software and not the date is was sent to the archive by an MLM.
Yep, to me too.
That makes a difference to me because copying the message to an archive might not be the only way to transport it there. The MLM might as well use an LMTP transport and queue it up for delivery. The queued message might get stuck in a delivery queue for hours. Being able to tell the difference between 'sent off to' and 'stored at' might be very useful to a postmaster when debugging archival problems.
Agreed.
Choosing a more specific header field would probably also avoid getting into an archivers way when it creats an "archive date" value as Barry mentioned above.
Not sure. Am I overly pedantic on this?
No, I think it's an important distinction. Also note that in MM3 you can actually have any number of archivers, so I think you want a field that clearly describes the actions of the MLM. I'm not sure what that would be other than List-Archived-Date though. :/
Suggestions?
List-Archive-Send-Date 'List-Archive-Send-Date' sounds pretty clumsy and overly long. OTOH we needn't care, as it will only be added to messages that go to the archive, right?
Archive-Transmit-Date, Archive-Transfer-Date, Archiv-Transfer Marks the beginning in opposition to Archive-Received-Date or Archive-Received. But then again an archiver could simply add a Received:-header!
Not an easy one.
p@rick
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
On 29 Oct 2011, at 05:39, Patrick Ben Koetter wrote:
Hmm, adopting a hash-tags format here would be kind of interesting for interop with social networking sites. It wouldn't have to be reflected in the name of
ACK with the notion that hashtag seems to closely realted with twitter and a more general 'tag' would stay away from that.
Is there or should there be a distinction between 'tag' and http 'keywords' http://en.wikipedia.org/wiki/Meta_element#The_keywords_attribute? Should we use 'keywords' instead?
Well, both are just ways of sorting messages into sets, rather than categories. By definition (using the usual mathematical definition), categories don't overlap but sets may, so objects may be in only one category, but may be in several sets. So, yes, keywords, tags and hashtags are pretty much the same thing.
And, it turns out that the Keywords: header is already defined in RFC5322:
3.6.5. Informational Fields
keywords = "Keywords:" phrase *("," phrase) CRLF
The "Keywords:" field contains a comma- separated list of important words and phrases that might be useful for the recipient.
There's also a "Comments:" field defined, apparently.
-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148
On 29 Oct 2011, at 05:39, Patrick Ben Koetter wrote:
List-Archive-Send-Date 'List-Archive-Send-Date' sounds pretty clumsy and overly long. OTOH we needn't care, as it will only be added to messages that go to the archive, right?
Archive-Transmit-Date, Archive-Transfer-Date, Archiv-Transfer Marks the beginning in opposition to Archive-Received-Date or Archive-Received. But then again an archiver could simply add a Received:-header!
Good point. I'd argue that the list manager, and the archiver should simply do that: put a received header into the message at the appropriate point. That would make it much easier to track the message progress.
-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148
Ian Eiloart writes:
Good point. I'd argue that the list manager, and the archiver should simply do that: put a received header into the message at the appropriate point. That would make it much easier to track the message progress.
I don't have a strong opinion on whether or not that's the best way to handle this, but I'd like to point out that RFC 5321 has quite a bit to say about the format of such lines. It also seems to imply that these are added by "gateway, relay, and forwarding systems". I would guess that Mailman qualifies as a forwarding system in which case it whould probably try to conform to the RFC 5321 syntax.
On 29 Oct 2011, at 05:39, Patrick Ben Koetter wrote:
I suggest we use the term 'Mediator' as introduced by D. Crocker in RFC 5598 http://www.rfc-editor.org/in-notes/rfc5598.txt instead:
A Mediator attempts to preserve the original Author's information in the message it reformulates but is permitted to make meaningful changes to the message content or envelope.
A Mediator's role is complex and contingent, for example, modifying and adding content or regulating which Users are allowed to participate and when. The common example of this role is a group Mailing List.
(see section "2.1.4. Mediator" and also section "5. Mediators")
A 'Mediator' would leave the original User-Agent intact AND tell the message had been processed by another mail handling service AND it would allow for other Mediators to be added in case more mail processing was done after an MLM's (here: Mailman) job.
That seems sensible to me.
-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148
On Oct 29, 2011, at 06:39 AM, Patrick Ben Koetter wrote:
I suggest we use the term 'Mediator' as introduced by D. Crocker in RFC 5598 http://www.rfc-editor.org/in-notes/rfc5598.txt instead:
A Mediator attempts to preserve the original Author's information in the message it reformulates but is permitted to make meaningful changes to the message content or envelope.
A Mediator's role is complex and contingent, for example, modifying and adding content or regulating which Users are allowed to participate and when. The common example of this role is a group Mailing List.
(see section "2.1.4. Mediator" and also section "5. Mediators")
That makes a good case for Mediator.
Hmm, if there are no intermediate processes between receiving a message and approving it a List-Approved-Date seems fine. But if there are we run into the same problem as described below with List-Archived-Date - you can't tell when it was queued and when processing took place.
Adding a second header might make the useful distinction:
List-Received-Date RFC 2822 date timestamp when message was received by MLM
List-Approved-Date RFC 2822 date timestamp when message was approved by moderator
What if the message is automatically approved? Does it get a List-Approved-Date header? Merging with Murray's concept of Received states, it might just make more sense to put all that information into Received headers.
Another header that might be useful here would be List-Approved-By which could be the name or email address of the moderator who approved it. Right now, MM3 doesn't fill that in, and it could of course be filled in by say list-owner@example.com, but in MM3 it could be potentially filled in with the preferred address for the moderator that approved it.
I see the benefit because it helps if you moderate in a team. But I fear the anger of people whose postings we decline. They search for moderator identities and then start molesting them e.g. by subscribing them to mailing lists that don't require opt-in. (Happend to me python.org postmaster. The angry person subscribed my address to various pr0n mailing lists and it took me weeks to get unsubscribed.)
Good point. I do want to provide the opportunity to "anonymize" ownership roles via generic owner email addresses. E.g. on the listinfo page.
ACK with the notion that hashtag seems to closely realted with twitter and a more general 'tag' would stay away from that.
Is there or should there be a distinction between 'tag' and http 'keywords' http://en.wikipedia.org/wiki/Meta_element#The_keywords_attribute? Should we use 'keywords' instead?
We can't use Keywords, because that header is already used as input to various functions such as the topic tagger. We have to use a different header for "output". I can't think of anything better than List-Tags though.
List-Archive-Send-Date 'List-Archive-Send-Date' sounds pretty clumsy and overly long. OTOH we needn't care, as it will only be added to messages that go to the archive, right?
Archive-Transmit-Date, Archive-Transfer-Date, Archiv-Transfer Marks the beginning in opposition to Archive-Received-Date or Archive-Received. But then again an archiver could simply add a Received:-header!
Not an easy one.
Agreed. :/
-Barry
Barry Warsaw writes:
On Oct 29, 2011, at 06:39 AM, Patrick Ben Koetter wrote:
I suggest we use the term 'Mediator' as introduced by D. Crocker in RFC 5598 http://www.rfc-editor.org/in-notes/rfc5598.txt instead: [...]
That makes a good case for Mediator.
+1, but only for the case of modification IMO.
Adding a second header might make the useful distinction:
List-Received-Date RFC 2822 date timestamp when message was received by MLM
I think this is best done with an RFC 5321 Received header as Murray suggests, even though I don't really think of MLMs as MTAs (or MUAs for that matter). OTOH, RFC 5321 requires that MTAs be pretty lenient about the presence of non-standardized trace headers, and specifically forbids altering them IIRC.
List-Approved-Date RFC 2822 date timestamp when message was approved by moderator
What if the message is automatically approved? Does it get a List-Approved-Date header? Merging with Murray's concept of Received states, it might just make more sense to put all that information into Received headers.
-1 on using "Received" for approvals. The approval
Also (per site or list policy) I would suggest that instead of a family of List-Approved-* headers, there should be a single List-Approved header with variable format. If present, it MUST contain an action ("Approved", "Denied", "Rejected", "Held"), MUST contain "at" and a timestamp, MAY contain "because" and a rule ("list member", "moderator", "spam score exceeded threshold"), and MAY contain "by" and a moderator ID (arbitrary token, could be a name, a numerial ID, or generic tokens like "mailman" (the name of the MLM in general), "list-owner", "moderator"). (I'm not wedded to the field tags, of course.)
The actions other than "Approved" would be used in cases where non-approvals are sent to the list or site owner for debugging purposes etc. Maybe "List-Action" or "List-Approval" or some such would be a better field tag.
I would suggest that it be inserted into the trace stream "as if" it were a "Received" header (or maybe before the Resent-* cluster?)
I see the benefit because it helps if you moderate in a team. But I fear the anger of people whose postings we decline. They search for moderator identities
Secret moderator IDs would serve to anonymize.
We can't use Keywords, because that header is already used as input to various functions such as the topic tagger.
Worse, it's already standardized by RFC 5322. In general, I take a dim view of a MLM hijacking any headers standardized by RFC 5322; those headers are basically for use of *authors*, though the common practice of adding various tags to Subject doesn't bother me *too* much, and exceptions might need to be made for Date and (especially) Message-ID, although maybe the MLM can just add those as Resent-* fields and abdicate responsibility if the user did.
We have to use a different header for "output". I can't think of anything better than List-Tags though.
What's wrong with List-Topics or List-Keywords? List-Tags is OK, I guess, but I think of tags as user-provided information for other users (not necessarily provided by the author, and not necessarily usable for automatic association of different posts).
List-Archive-Send-Date
How about just List-Archived, make it a formatted field "in[to] $ARCHIVE at $TIMESTAMP", and let the sent/received distinction be made by using the standard trace header of "Received" (as already suggested)? The "in" version is used when ARCHIVE contains a user-accessible URL (either generic for the list or post-specific), and "to" is for cases where the user URL is not necessarily known but the message is sent "to" an archiver and information about user access is given out-of-band.
-----Original Message----- From: mailman-developers-bounces+msk=cloudmark.com@python.org [mailto:mailman-developers-bounces+msk=cloudmark.com@python.org] On Behalf Of Stephen J. Turnbull Sent: Tuesday, November 15, 2011 8:02 PM To: Barry Warsaw Cc: mailman-developers@python.org Subject: Re: [Mailman-Developers] New RFC on using DKIM with MLMs
Barry Warsaw writes:
On Oct 29, 2011, at 06:39 AM, Patrick Ben Koetter wrote:
I suggest we use the term 'Mediator' as introduced by D. Crocker in RFC 5598 > >http://www.rfc-editor.org/in-notes/rfc5598.txt instead: [...]
That makes a good case for Mediator.
+1, but only for the case of modification IMO.
Works for me. If it includes the hostname where the mediator is running, it can go a long way toward debugging things like DKIM validation problems or other "What the hell happened to this message?" things.
But then again, if we go with the received-state idea, mailman could do what other MTAs do and write its name and version information in a comment in a Received field.
List-Approved-Date RFC 2822 date timestamp when message was approved by moderator
What if the message is automatically approved? Does it get a > List-Approved-Date header? Merging with Murray's concept of Received states, > it might just make more sense to put all that information into Received > headers.
-1 on using "Received" for approvals. The approval
Incomplete thought here?
If the received-state idea is adopted, a second Received: is the perfect way to show when something entered an approval hold and when it came out, without registering a new header field dedicated for this purpose.
Also (per site or list policy) I would suggest that instead of a family of List-Approved-* headers, there should be a single List-Approved header with variable format. If present, it MUST contain an action ("Approved", "Denied", "Rejected", "Held"), MUST contain "at" and a timestamp, MAY contain "because" and a rule ("list member", "moderator", "spam score exceeded threshold"), and MAY contain "by" and a moderator ID (arbitrary token, could be a name, a numerial ID, or generic tokens like "mailman" (the name of the MLM in general), "list- owner", "moderator"). (I'm not wedded to the field tags, of course.) [...]
If this is meant to be machine-parsable versus user-parsable, it'll need to be more rigid than that. But I see where you're going. Phrases in particular are a pain unless they're meant to be quoted strings that machines don't care about using.
So I guess we need to be clear on how these various new fields are expected to be used.
Secret moderator IDs would serve to anonymize.
As long as the MLM keeps track of who they are so administrators can track actions, sure.
We can't use Keywords, because that header is already used as input to various functions such as the topic tagger.
Worse, it's already standardized by RFC 5322. In general, I take a dim view of a MLM hijacking any headers standardized by RFC 5322; those headers are basically for use of *authors*, though the common practice of adding various tags to Subject doesn't bother me *too* much, and exceptions might need to be made for Date and (especially) Message-ID, although maybe the MLM can just add those as Resent-* fields and abdicate responsibility if the user did.
I think RFC5598 suggests that the MLM re-mailing should generate a new Message-ID and a new Date, and observes the common Subject: tagging practice, but I don't think anyone expects the other stuff to be renamed to Resent-* first.
What's wrong with List-Topics or List-Keywords? List-Tags is OK, I guess, but I think of tags as user-provided information for other users (not necessarily provided by the author, and not necessarily usable for automatic association of different posts).
As above, how are they typically consumed? That might help to answer these questions.
-MSK
On 11/15/2011 6:52 PM, Barry Warsaw wrote:
On Oct 29, 2011, at 06:39 AM, Patrick Ben Koetter wrote:
I suggest we use the term 'Mediator' as introduced by D. Crocker in RFC 5598 http://www.rfc-editor.org/in-notes/rfc5598.txt instead:
A Mediator attempts to preserve the original Author's information in the message it reformulates but is permitted to make meaningful changes to the message content or envelope.
A Mediator's role is complex and contingent, for example, modifying and adding content or regulating which Users are allowed to participate and when. The common example of this role is a group Mailing List.
(see section "2.1.4. Mediator" and also section "5. Mediators") That makes a good case for Mediator.
Hmmm. To me, the above definition for Mediator is reason enough to abandon it (for the time at least) because it is not well thought out regarding everything that may touch, modify, add content, regulate user, etc. Together with the above and a line from a previous message Patrick sent to the list:
"The term 'Mediator' describes the same, but it is general inmeaning. Any instance modifying and forwarding a message can use it. "
I can't help but consider the rash of useless Mediator headers. Consider the following example:
Person 1 sends a message to a list which is then sent to Person 2. Person 1's site has separate appliances for the MTA, Spam checking, Anti-Virus, and Spy-ware, likewise for the receiver's site and the site the MLM is for the list. My reading of the above definitions tells me:
- Person 1's MUA adds a Mediator header (creating a message is a simple special case of modifying/adding a message,
- Person 1's site adds 4 additional Mediator headers (one each for the MTA, Spam, Anti-Virus, Spy-ware since each modify/forward/add to the message,
- The MLM's site adds 9 additional Mediator headers (4 inbound [item 2], 1 for the MLM [maybe more], and 4 outbound),
- Person 2's site adds 4 additional Mediator headers (4 inbound [item 2]),
- Person 2's MUA may or may not add a Mediator header depending on any rules/filters Person 2 has in place.
So, at the end of this simple posting of a message to a list, we have 18+ Mediator headers, one for each time an instance adds content, modifies a message, or regulates a user.
Granted, I may be making this a worst case scenario, but also consider any sites the message is relayed through - adding even more Mediator headers. Looking at a worst case scenario helps see where the problems occur and cracks in specifications happen.
I am not opposed to a Mediator header. I just think since our discussion is focused on MLMs that List-Agent is the best choice that all MLM developers can come to an agreement on.
As for other applications (news, web forum, etc), a discussion with those developers should (and needs to) take place to come up with a more generic header - be it Mediator or not.
In any case, let's focus on MLM related headers, like List-Agent :), and then work on a more generic, non-conflicting header for others at a later time. I just see including non-MLM information into a MLM header discussion as feature-creep.
Hmm, if there are no intermediate processes between receiving a message and approving it a List-Approved-Date seems fine. But if there are we run into the same problem as described below with List-Archived-Date - you can't tell when it was queued and when processing took place.
Adding a second header might make the useful distinction:
List-Received-Date RFC 2822 date timestamp when message was received by MLM
List-Approved-Date RFC 2822 date timestamp when message was approved by moderator What if the message is automatically approved? Does it get a List-Approved-Date header? Merging with Murray's concept of Received states, it might just make more sense to put all that information into Received headers.
The only issue I can see is if the Received headers are removed as part of processing, whether in an appliance, an MTA, or an MUA. I know it shouldn't happen but I received messages with missing Received headers. Having the information in a List-* header wouldn't hurt.
Another header that might be useful here would be List-Approved-By which could be the name or email address of the moderator who approved it. Right now, MM3 doesn't fill that in, and it could of course be filled in by say list-owner@example.com, but in MM3 it could be potentially filled in with the preferred address for the moderator that approved it. I see the benefit because it helps if you moderate in a team. But I fear the anger of people whose postings we decline. They search for moderator identities and then start molesting them e.g. by subscribing them to mailing lists that don't require opt-in. (Happend to me python.org postmaster. The angry person subscribed my address to various pr0n mailing lists and it took me weeks to get unsubscribed.) Good point. I do want to provide the opportunity to "anonymize" ownership roles via generic owner email addresses. E.g. on the listinfo page.
:) So far, I haven't had anyone that grumpy at me to add me to various "other" lists. But they do occasionally go over my head to get authorization so they can post without being moderated (at least until they send a message that exceeds the size limit).
ACK with the notion that hashtag seems to closely realted with twitter and a more general 'tag' would stay away from that.
Is there or should there be a distinction between 'tag' and http 'keywords' http://en.wikipedia.org/wiki/Meta_element#The_keywords_attribute? Should we use 'keywords' instead? We can't use Keywords, because that header is already used as input to various functions such as the topic tagger. We have to use a different header for "output". I can't think of anything better than List-Tags though.
List-Archive-Send-Date 'List-Archive-Send-Date' sounds pretty clumsy and overly long. OTOH we needn't care, as it will only be added to messages that go to the archive, right?
Archive-Transmit-Date, Archive-Transfer-Date, Archiv-Transfer Marks the beginning in opposition to Archive-Received-Date or Archive-Received. But then again an archiver could simply add a Received:-header!
Not an easy one. Agreed. :/
-Barry
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/cnulk%40scu.edu
Security Policy: http://wiki.list.org/x/QIA9
Thanks for bearing with me, Chris
On 16 Nov 2011, at 16:10, C Nulk wrote:
I can't help but consider the rash of useless Mediator headers. Consider the following example:
Person 1 sends a message to a list which is then sent to Person 2. Person 1's site has separate appliances for the MTA, Spam checking, Anti-Virus, and Spy-ware, likewise for the receiver's site and the site the MLM is for the list. My reading of the above definitions tells me:
- Person 1's MUA adds a Mediator header (creating a message is a simple special case of modifying/adding a message,
But, there are already headers to identify the creator. There's no need to use a mediator header to duplicate other information.
- Person 1's site adds 4 additional Mediator headers (one each for the MTA, Spam, Anti-Virus, Spy-ware since each modify/forward/add to the message,
The MTA would use a "received" header, not a mediator header. If the message is routed through the other systems, then they should already add received headers. If the message isn't routed through those systems (e.g., the systems are plugged in to the MTA, then there's no need to add mediator headers).
- The MLM's site adds 9 additional Mediator headers (4 inbound [item 2], 1 for the MLM [maybe more], and 4 outbound),
- Person 2's site adds 4 additional Mediator headers (4 inbound [item 2]),
- Person 2's MUA may or may not add a Mediator header depending on any rules/filters Person 2 has in place.
-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148
Hello Ian,
Sorry I am a bit late in responding but with our Thanksgiving holiday and me taking a few days of vacation I was away from email.
I understand what you are saying in your message but I think you possibly missed my point. Given the definitions previously proposed for the "Mediator" header, I believe it is far too generic to be useful.
To paraphrase the previous definitions for the "Mediator" header,
"any program/instance/etc. that modifies, adds content, regulates Users, etc. can use it [the 'Mediator' header]".
To me, a reasonable reading and understanding of the definition[s] tells me the MUA, the MTA, any appliances, any other program/instance [including devices]/etc. can add the "Mediator" header REGARDLESS of whether or not that appliance/program/instance/device has added a "Received" header, a "User-Agent" header, or any other pertinent header(s).
Since I think the "Mediator" header is, at this time, poorly defined and the initial point of the discussion was about having a "List-Agent" header for MLMs, I say "List-Agent".
As for my example you pulled apart :), I deliberately made it a worst case scenario just to highlight how the "Mediator" header was poorly defined. In fact, given the "Mediator" header definitions and your comments, you also highlight how poor the definition is since not only will the "Mediator" headers be added but also all the additional "Received" headers. I may be wrong in that you assume that if "Received" headers are used then the "Mediator" headers won't be used. However, the definition does not precluded having both, thus doubling (or more) on the number of headers.
Again, I say stick with the "List-Agent" header for MLMs. It is narrowly defined and limited in scope, precisely what a header should be to be effective.
Thanks for listening/reading :), Chris
On 11/23/2011 5:16 AM, Ian Eiloart wrote:
On 16 Nov 2011, at 16:10, C Nulk wrote:
I can't help but consider the rash of useless Mediator headers. Consider the following example:
Person 1 sends a message to a list which is then sent to Person 2. Person 1's site has separate appliances for the MTA, Spam checking, Anti-Virus, and Spy-ware, likewise for the receiver's site and the site the MLM is for the list. My reading of the above definitions tells me:
- Person 1's MUA adds a Mediator header (creating a message is a simple special case of modifying/adding a message, But, there are already headers to identify the creator. There's no need to use a mediator header to duplicate other information.
- Person 1's site adds 4 additional Mediator headers (one each for the MTA, Spam, Anti-Virus, Spy-ware since each modify/forward/add to the message, The MTA would use a "received" header, not a mediator header. If the message is routed through the other systems, then they should already add received headers. If the message isn't routed through those systems (e.g., the systems are plugged in to the MTA, then there's no need to add mediator headers).
- The MLM's site adds 9 additional Mediator headers (4 inbound [item 2], 1 for the MLM [maybe more], and 4 outbound),
- Person 2's site adds 4 additional Mediator headers (4 inbound [item 2]),
- Person 2's MUA may or may not add a Mediator header depending on any rules/filters Person 2 has in place.
-----Original Message----- From: mailman-developers-bounces+msk=cloudmark.com@python.org [mailto:mailman-developers-bounces+msk=cloudmark.com@python.org] On Behalf Of Barry Warsaw Sent: Friday, October 28, 2011 3:43 PM To: mailman-developers@python.org Subject: Re: [Mailman-Developers] New RFC on using DKIM with MLMs
This also looks like a candidate for, say, a List-Approved-Date header.
It's not available in RFC 2369. We will have to propose 'List-Approved-Date' as a new standard. Should we?
I've got a separate draft that adds to Received: fields a tag that indicates transitions of messages into administrative hold states (quarantining, timed delivery and list moderation are included in the initial list of reasons) ready to go. I just missed the -00 submission deadline prior to the next IETF meeting in November, so it's not in the IETF datatracker but it is here: http://www.blackops.org/~msk/draft-kucherawy-received-state.txt. Comments welcome.
The idea: When the MLM selects a message for moderation it would a Received: field with such a tag, and then on release the next MTA in the chain adds another Received: as per normal which, presumably, completes the handling chain in a way that the end user can see what happened (i.e., when it entered the hold and when it came out).
This doesn't include a mechanism for tracking who did the approval, but you've got that separately on your list anyway.
-MSK
Right, sorry. I was in a hurry when I wrote my follow up, so let me provide more detail about the headers, and my thoughts about the X-iness.
In general, Mailman should "play nice" but I also think it's not unreasonable to claim the Mailman-* prefix and just start using it for Mailman-specific headers. If the header has some more general utility, then keeping the X- for now and registering them may make more sense.
On Oct 26, 2011, at 12:43 PM, Ian Eiloart wrote:
X-Message-ID-Hash
This could be replaced with DKIM sigs, I guess.
Actually, no, this one comes from here:
http://wiki.list.org/display/DEV/Stable+URLs
I think there was a discussion about this ages ago, although it might have been (partly) a private discussion with the Mail Archive folks. The idea is that given only the List-Archive or RFC 5064 Archived-At header and Message-ID header, you should be able to calculate a stable url for an archived message. If you only know the Message-ID then using the hash should allow you to search an archive for the message.
X-Message-ID-Hash is essentially just a convenience, since it can be calculated from the Message-ID. I think it's still appropriate to leave this one an X- header, although it might be interesting to register it, and even more interesting to propose an RFC as an extension of RFC 5064.
X-Mailman-Rule-Hits X-Mailman-Rule-Misses
This might be useful for diagnostics, but probably wants to be off in general. My view is that Mailman should not be doing message filtering.
Mailman's always done some form of message filtering, so this is just MM3's way of recording the results of the various tests for acceptance. In MM2, the processing pipeline contained both moderation handlers (e.g. "is this administrivia") and modification handlers (e.g. "futz with the headers"), but this has been refactored in MM3 so that there are different processing subsystems for moderation and modification. The headers above will contain a list of the moderations rules that matched or missed, so their inclusion in the outgoing message is *mostly* diagnostics.
Since they'll probably be hidden anyway, I think it's useful to keep these on by default. They could certainly lose the X- prefix.
X-BeenThere
I guess that's useful for avoiding list loops, perhaps?
Right, but MM3 uses List-Post for that now, so this is legacy. I think this can be eradicated from the MM3 source, since I think it makes no sense to recognize the MM2 loop-detection header in MM3.
https://bugs.launchpad.net/mailman/+bug/883158
X-Mailman-Version
I think this should be replaced with X-Mailer, or even User-Agent. That's not currently an SMTP header, but I think it should be. And it is in quite widespread use.
This is just the version of Mailman that sent the message. It can lose the X- prefix.
X-Mailman-Approved-At
This is a time stamp, so it'll help you understand why a message you sent to a list a year ago is only showing up now <wink>. It can lose the X- prefix.
X-Archive X-No-Archive
These are Postel's law headers which control whether a message is archived or not. Yeah, we've seen them both. MM3 doesn't set them, so ignore these for the purposes of this discussion.
What does this mean?
X-Ack X-No-Ack
Similarly, these control whether an acknowledgment of a post is sent to the original author. MM3 does set this to "no" when a user's noack flag is set. These should keep the X- prefix.
X-Peer
Ignore this, it's used in the test suite only. lazr.smtptest sets it based on the host:port of the connecting socket.
X-MailFrom X-RcptTo
These are also primarily used in the test suite. Again, lazr.smtptest sets them, however MM3's LMTP server does add this header based on the RFC 5321 MAIL FROM value. It's mostly used as a diagnostic though.
I think we could probably rename this to Mailman-LMTP-MailFrom if we still wanted to keep it.
X-Originally-To
Mailman only sets this in gate_news.py when it needs to rewrite the To: header. Probably not worth changing.
X-Original-CC X-Original-Content-Transfer-Encoding X-MIME-Version
Ignore these. They are used in the news gating code when it has to rewrite duplicate headers. E.g. if a message destined for NNTP has more than one CC header, MM3 collapses this into a single X-Original-CC and deletes all of the original CC headers. At least at one point, NNTP software did not accept messages with multiple CC headers.
X-Mailman-Copy
This one is used when VERP is enabled. When a message is posted to the list, personalization is enabled, and a user wants to receive duplicates (i.e. they have not enabled list-copy suppression), the copy of the message coming from Mailman will have X-Mailman-Copy: yes set.
This is probably a fairly useless redundancy, since I always tend to use one of the List-* headers to determine whether a copy comes from the list or not. It also probably does no harm. It can either be removed or lose the X- prefix.
X-List-Administrivia
Honestly, I don't remember what the purpose of this header is. It used to be used when reduced RFC 2369 header were requested by the list administrator, e.g. because they had users with, um, unhelpful mail clients and got complaints about all the List-* headers. Now we don't support reduced headers so X-List-Administrivia is just useless and should be removed.
X-Content-Filtered-By
This is an identification header added when MIME contents of the original message are filtered. I'm not sure how useful it is, except that it does signal to the recipient that the message they received via the list has been modified. I think this should be renamed to (X-)Mailman-Content-Filter.
X-Topics
This contains a list of all the topic names that matched the message.
X-Mailer
I think we should use User-Agent here. Thunderbird does, as do some other mail clients. Or, we should push for introduction of a List-Agent header.
Mailman's autoresponder is the only thing that adds this, and it only contains an identity string. It may or may not be useful to keep this or change it. I'm not sure User-Agent would be right though.
X-List-Received-Date
This only gets added when the message is sent to the archive. It's consumed by the scrubber to help calculate the directory that attachments are saved in.
X-Approve X-Approved
These are used for admin bypass of the posting rules. We could change these to Mailman-* headers, but we'd still need to keep the above two for backward compatibility, so I'm not sure it's worth it.
Generally, I think we should avoid the use of headers that duplicate other existing headers. Where we want to add more information, then extend the List-* header set if the information might be generally useful for mailing list software. Otherwise, use X-Mailman-* (or even Mailman-*) so that people know where the header came from.
I generally agree.
https://bugs.launchpad.net/mailman/+bug/883193
-Barry
Barry Warsaw writes:
X-Mailman-Version
I think this should be replaced with X-Mailer, or even User-Agent. That's not currently an SMTP header, but I think it should be. And it is in quite widespread use.
This is just the version of Mailman that sent the message. It can lose the X- prefix.
I think that's a bad idea. The version string should go in a *-Agent header, along with the agent's identity.
While I disagree with having Mailman be a User-Agent, I can't actually say there are useful semantics to having a separate List-Agent. Nor do I see a real problem with having multiple User-Agent headers, if multiple agents have handled the message.
-----Original Message----- From: mailman-developers-bounces+msk=cloudmark.com@python.org [mailto:mailman-developers-bounces+msk=cloudmark.com@python.org] On Behalf Of Stephen J. Turnbull Sent: Friday, October 28, 2011 5:46 PM To: Barry Warsaw Cc: mailman-developers@python.org Subject: [Mailman-Developers] Mailman headers (was Re: New RFC on using DKIM with MLMs)
I think that's a bad idea. The version string should go in a *-Agent header, along with the agent's identity.
Agreed.
While I disagree with having Mailman be a User-Agent, I can't actually say there are useful semantics to having a separate List-Agent. Nor do I see a real problem with having multiple User-Agent headers, if multiple agents have handled the message.
I think having a message with User-Agent and List-Agent is less confusing than one with two User-Agents.
participants (10)
-
Barry Warsaw -
Barry Warsaw -
C Nulk -
Ian Eiloart -
Joshua Cranmer -
Mark Sapiro -
Murray S. Kucherawy -
Patrick Ben Koetter -
Richard Damon -
Stephen J. Turnbull