Adding CSS to mailman archives
A little while ago (November 2012 probably) I changed a mailing list from Hypermail to Mailman and wanted to keep some of the look-and-feel of the archives, e.g. background colour and fonts. So I patched mailman 2.1.9 in CentOS 5 to include a CSS stylesheet link in each generated HTML page, replacing e.g. <BODY BGCOLOR="#ffffff"> with just <body> and adding e.g. body {background-color: #ffffff ; color: black; } in /pipermail/mailman.css, so that HTML style such as FONT and SIZE is removed from the Python code. (I only have one list on that server and didn't implement per-list style) I also changed a few things e.g. content-type to get zero warnings and errors from the W3C online validator.
Now if I want to get DMARC functionality I'll have to jump past the RedHat/CentOS release and build 2.1.18, and re-patch for CSS.
I was wondering if there was any interest in incorporating CSS into the official releases.
-- Andrew Daviel, TRIUMF, Canada Tel. +1 (604) 222-7376 (Pacific Time)
On 07/02/2014 06:27 PM, Andrew Daviel wrote:
Now if I want to get DMARC functionality I'll have to jump past the RedHat/CentOS release and build 2.1.18, and re-patch for CSS.
See the FAQ at <http://wiki.list.org/x/jYA9>. It is fairly straightforward to make list, domain or site specific HTML templates that will survive upgrades.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, 2 Jul 2014, Mark Sapiro wrote:
On 07/02/2014 06:27 PM, Andrew Daviel wrote:
Now if I want to get DMARC functionality I'll have to jump past the RedHat/CentOS release and build 2.1.18, and re-patch for CSS.
See the FAQ at <http://wiki.list.org/x/jYA9>. It is fairly straightforward to make list, domain or site specific HTML templates that will survive upgrades.
As I recall, although most things can be tuned in the templates, there is some non-verifier-compliant HTML generated directly by htmlformat.py which would be missing CSS defined in the templates.
-- Andrew Daviel, TRIUMF, Canada
This is probably a case of beating a dead horse after everyone switched to automobiles, but nevertheless:
We had an occasion where some private information got forwarded to a public mailing list and then got found by a search engine. Probably happens all the time. At least one of these messages had some boilerplate on the bottom saying "may be confidential, do not forward" etc.
Computers are better than people at reading boilerplate. It would be nice if there was some way to flag a message "do not archive", or even "do not forward" so that a user would have to do something to override that. I found "X-No-Archive" in wikipedia that was apparently honored by google when archiving usenet.
In an ideal world hundreds of people would immediately agree that this was an excellent idea, get an RFC written, and everyone would implement it by next year. Not going to happen, I know.
I wondered if this had been done already somehow, or considered and shot down, and how others handle the problem.
-- Andrew Daviel, TRIUMF, Canada Tel. +1 (604) 222-7376 (Pacific Time)
On 02/11/2015 03:31 PM, Andrew Daviel wrote:
In an ideal world hundreds of people would immediately agree that this was an excellent idea, get an RFC written, and everyone would implement it by next year. Not going to happen, I know.
I wondered if this had been done already somehow, or considered and shot down, and how others handle the problem.
Mailman honors this in that messages with either an
X-No-Archive:
or an
X-Archive: No
header (case insensitive) will not be archived. Also, Monthly and on demand password reminders and list welcome messages are sent with an
X-No-Archive: Yes
header.
I think the X-No-Archive header usage is fairly standard.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Andrew Daviel writes:
At least one of these messages had some boilerplate on the bottom saying "may be confidential, do not forward" etc.
Computers are better than people at reading boilerplate.
Unfortunately, the footer boilerplate is non-standard.
In an ideal world hundreds of people would immediately agree that this was an excellent idea, get an RFC written, and everyone would implement it by next year. Not going to happen, I know.
Only because it already has, except for the RFC, and
http://people.dsv.su.se/~jpalme/ietf/ietf-mail-attributes.htmlmentions the definition of "X-No-Archive: Yes".
In practice, I believe most of the standard (Mailman's Pipermail[1], mail-archive.com, GMane) and large-scale proprietary archiving engines (Google and Yahoo! Groups) already do.
I wondered if this had been done already somehow, or considered and shot down, and how others handle the problem.
I'm not sure why Palme's I-D wasn't taken to RFC (Informational) status; probably it just was considered insufficiently authoritative since new headers are being invented all the time, eg, X-Spam and relatives aren't in it).
However, the real practical problem is that to be reliable, it needs to be a header field, and insertion of header fields is the province of mail clients, over which the IETF has no authority. (That is, the IETF can define a header field's semantics, but it is up to the user or the mail client to add it, and the IETF can't mandate addition of a user choice field the way it can mandate addition of From, Date, and Message-ID.) Although archivers recognize X-No-Archive, few users realize that there is a header in mail to which they can (in principle) add arbitrary fields, and even fewer know how to add the field with their mail-client-of-choice.
I think the best strategy is for mail clients and outgoing mail gateways that add confidentiality notices to also automatically add "X-No-Archive: Yes" to the header. And that's what I suggest you recommend to your posters. Not all of them will have the skills and access to do so, of course, but it will save the competent ones the occasional headache.
Footnotes: [1] I need to check Mailman 3's HyperKitty.
On Thu, 12 Feb 2015, Stephen J. Turnbull wrote:
Only because it already has, except for the RFC, and
http://people.dsv.su.se/~jpalme/ietf/ietf-mail-attributes.html
mentions the definition of "X-No-Archive: Yes".
Thanks. Amazing. I assumed that it had expired with Usenet.
I found it implemented in /usr/lib/mailman/Mailman/Handlers/ToArchive.py, viz. # Common practice seems to favor "X-No-Archive: yes". No other value for # this header seems to make sense, so we'll just test for it's presence. # I'm keeping "X-Archive: no" for backwards compatibility. if msg.has_key('x-no-archive') or msg.get('x-archive', '').lower() == 'no': return
http://en.wikipedia.org/wiki/X-No-Archive says "If the X-No-Archive field is set to "No", or the field is absent, a Usenet archive will not recognize a prohibition on archiving the message."
Experimentally, if I add "X-No-Archive: no" in Alpine or Thunderbird, pipermail will not archive the message, while if I add just "X-No-Archive:" to my custom headers, both programs will suppress the empty header so that mailman will archive the message.
(double negatives are confusing, I know)
I found http://kb.mozillazine.org/Custom_headers in wikipedia which explains how to add a custom header in Thunderbird, though the "easy" way seemed to not exist anymore, at least I could not find it.
GMane has a different X-Archive syntax, per http://gmane.org/expiry.php viz. X-Archive: expiry=7 http://www.mail-archive.com/faq.html specifies "X-No-Archive: yes"
I've updated the wikipedia page for fun.
-- Andrew Daviel, TRIUMF, Canada Tel. +1 (604) 222-7376 (Pacific Time)
http://www.mail-archive.com/faq.html specifies "X-No-Archive: yes"
It's been a long time, but I vaguely remember lowercase 'yes' being canonical which is why we specify it that way in the FAQ. In practice mail-archive.com treats the header as case insensitive.
Here's some data on common practices, taken from a recent week of mail-archive.com's inbound mail.
$ zgrep -i "^x\-no\-archive:" *.mbox.gz | cut -f3 -d":" | sort | uniq -c
72 no
183 No6407 yes 108 Yes 47 YES
Andrew Daviel writes:
http://en.wikipedia.org/wiki/X-No-Archive says "If the X-No-Archive field is set to "No", or the field is absent, a Usenet archive will not recognize a prohibition on archiving the message."
Experimentally, if I add "X-No-Archive: no" in Alpine or Thunderbird, pipermail will not archive the message,
Do you consider that a bug?
I'm not sure why anybody would use "X-No-Archive: no" in reality, I wonder if it's a mistake in interpreting the double negation. But seeing it in Jeff's list, I guess people actually do use it. I wonder if Jeff would be willing to look at (some of) the 255 nos and see what fraction seem to have appropriate insemantics (ie, there is reason to believe the user was trying to prevent archiving, such as a "confidentiality" footer).
On 02/12/2015 12:28 AM, Stephen J. Turnbull wrote:
Andrew Daviel writes:
Experimentally, if I add "X-No-Archive: no" in Alpine or Thunderbird, pipermail will not archive the message,
Do you consider that a bug?
I'm not sure why anybody would use "X-No-Archive: no" in reality, I wonder if it's a mistake in interpreting the double negation.
As I posted earlier Mailman does not archive a message with an X-No-Archive: header. It doesn't look at the content which could be yes, no, empty or anything else. This is deliberate.
It does look at the content of an X-Archive: header which must be (case insensitive) no to suppress archiving.
This is explained in the code and comments which Andrew quoted in the same post partially quoted above
I found it implemented in /usr/lib/mailman/Mailman/Handlers/ToArchive.py, viz. # Common practice seems to favor "X-No-Archive: yes". No other value for # this header seems to make sense, so we'll just test for it's presence. # I'm keeping "X-Archive: no" for backwards compatibility. if msg.has_key('x-no-archive') or msg.get('x-archive', '').lower() == 'no': return
So the experiment just confirms that the code does what it says. Given that
X-No-Archive: no
apparently does occur in the wild, if we can confirm that the semantics of this should be 'do archive', file a bug and I'll fix it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
As I posted earlier Mailman does not archive a message with an X-No-Archive: header. It doesn't look at the content which could be yes, no, empty or anything else. This is deliberate.
Yes, I understand that, and the fact that the code uses .has_key makes it clear that it's deliberate. I'm asking whether non-Mailman- developers have an opinion, and more important (since I expect non- developers to say "Gee, I dunno ... whatever you think" :-), whether the mail-archive.com folks can tell something from their corpus. (I've checked my own folders and xemacs's: found *one*. From Dec 1999. :-)
X-No-Archive: no
apparently does occur in the wild, if we can confirm that the semantics of this should be 'do archive', file a bug and I'll fix it.
Yup.
On Feb 12, 2015, at 01:42 AM, Mark Sapiro wrote:
apparently does occur in the wild, if we can confirm that the semantics of this should be 'do archive', file a bug and I'll fix it.
It's been ages, but IIRC the reason for accepting any value is exactly because of the double negative. People had a difficult time parsing it out, and as you observe, it doesn't make much sense to deliberately say "X-No-Archive: no".
It's crappy (inherited, historical) ui but it's not a bug.
Cheers, -Barry
I checked and these two lists are responsible. One uses 'No' and the other uses 'no' for every message. I can't speak to the intention. There are an impressive number of headers on each message including DKIM but I don't see a clue as to the list server software.
http://www.mail-archive.com/iagi-net@iagi.or.id http://www.mail-archive.com/fgmi@iagi.or.id
On Thu, 12 Feb 2015, Stephen J. Turnbull wrote:
Andrew Daviel writes:
http://en.wikipedia.org/wiki/X-No-Archive says "If the X-No-Archive field is set to "No", or the field is absent, a Usenet archive will not recognize a prohibition on archiving the message."
Experimentally, if I add "X-No-Archive: no" in Alpine or Thunderbird, pipermail will not archive the message,
Do you consider that a bug?
I'm not sure. When I look at the Mailman documentation, it's clear enough, so it's a feature. But it seems at variance with the other applications mentioned.
I was trying to add the header without reading the documentation - the other custom header already in my configuration file had a value, so I assumed that a value was required. Turned out it was not.
- I realized later that XNA doesn't actually solve my problem; mail headers don't get promoted when forwarding mail. I need X-No-Forward...
Andrew
On 2/11/2015 7:37 PM, Stephen J. Turnbull wrote:
Only because it already has, except for the RFC, and
http://people.dsv.su.se/~jpalme/ietf/ietf-mail-attributes.htmlmentions the definition of "X-No-Archive: Yes".
In practice, I believe most of the standard (Mailman's Pipermail[1], mail-archive.com, GMane) and large-scale proprietary archiving engines (Google and Yahoo! Groups) already do.
I wondered if this had been done already somehow, or considered and shot down, and how others handle the problem.
I'm not sure why Palme's I-D wasn't taken to RFC (Informational) status; probably it just was considered insufficiently authoritative since new headers are being invented all the time, eg, X-Spam and relatives aren't in it).
There is an IANA registry for message header fields (see RFC 3864 for registration procedures). Of course, I don't know how many people who write email libraries on any position in the spectrum are aware of this in the first place, so adding X-No-Archive to that list is not likely to matter one whit.
-- Joshua Cranmer Thunderbird and DXR developer Source code archæologist
Andrew Daviel wrote:
I found "X-No-Archive" in wikipedia that was apparently honored by google when archiving usenet.
Keep in mind that all archivers probably archive the message anyway but might treat "X-No-Archive: yes" as a cue to not release that message to public queries. In other words, "X-No-Archive: yes" netnews posts are archived right alongside all the other posts and Google employees can find them in Google's repository of netnews posts.
I believe this is the case for two reasons: posts Google's search engine Google released copies of posts labeled
Structurally: Few search engine companies will throw away an opportunity to collect data when there's so much for that organization to gain by indexing that data.
Specific to Google: When Google took on DejaNews's archive of Usenet
"X-No-Archive: yes" for a while.
Overall, I think people are better off understanding that their boilerplate is not binding on most people (if anyone) and they need to be more careful in what they send.
On Feb 12, 2015, at 12:50 PM, J.B. Nicholson-Owens wrote:
I believe this is the case for two reasons:
Structurally: Few search engine companies will throw away an opportunity to collect data when there's so much for that organization to gain by indexing that data.
Specific to Google: When Google took on DejaNews's archive of Usenet posts Google's search engine Google released copies of posts labeled "X-No-Archive: yes" for a while.
Overall, I think people are better off understanding that their boilerplate is not binding on most people (if anyone) and they need to be more careful in what they send.
Aside from the fact that you should just assume that nothing on the intarwebs is secret. ;)
Cheers, -Barry
J.B. Nicholson-Owens writes:
Overall, I think people are better off understanding that their boilerplate is not binding on most people (if anyone) and they need to be more careful in what they send.
I don't think anyone here disagrees. On the other hand, people *do* have an expectation of privacy (or any other attribute of service), and we're doing our users[1] a big favor if we just conform to those expectations where we can.
Footnotes: [1] Here I mean the site and list admins who will handle complaints.
participants (7)
-
Andrew Daviel -
Barry Warsaw -
J.B. Nicholson-Owens -
Jeff Breidenbach -
Joshua Cranmer 🐧 -
Mark Sapiro -
Stephen J. Turnbull