I see this topic has been discussed years ago, but I recently started running some MailMan lists and this problem came up almost as soon as I started running it. (Version 2.1.5)
Users WILL NOT set their clocks to some nicely synchronized time. This
is especially true since many of my subscribers are computer neophytes.
When sorted by date, some replies come before the original posting. The
worst offender is actually a highly competent computer user who claims
to have a defective clock in his laptop that produces messages from the
year 2080. (The truth of his claim is immaterial.) In an ideal world,
these things would never happen. This is not an ideal world.
There was a discussion about "not munging" the headers as a validation for objecting to altering the "Date" header. Considering the actions of "CookHeaders.py" and other files that variously alter the "Reply To" header, delete unwanted headers, and obviously alter the "Subject" line, these objections seem weak, if not ironic. The date set by the senders computer is not sacrosanct. While it may have some "meaning," it is of no "use." Also, if the date isn't set properly, it probably isn't even important to the sender.
Since all of these messages are, in fact, being sent by my server I think it quite reasonable to change the "Date" to reflect the time that it was processed and changed by the server. I'm the one who has to answer questions about why the archive dates seem wrong and who is ultimately responsible for the smooth operation of the service. I would LIKE this to be an option that I could set. I do not have the time or inclination to write the code to make this a user option. However, since it something I wanted applied to all of my lists, I added the code to copy the "Date" field to "X-Original-Date" and put in the server's time as the "Date."
Suddenly, my new archives are coherent. I don't have to worry about aberrant or obnoxious users. I only added six lines of code (and two imports) to "CookHeaders.py" to make this work. PLEASE tell me I wasted my time and there is a simple option I can set! Tell me that I don't have to port this code when I want to upgrade!
I don't speak Python, and I know I imported more than necessary, but it seems to work. I'm including the code changes both to share what I found to be Very Useful and to have others sanity-check my code. As I said, I never wrote a line of Python before this.
Steven Kuck PS I considered sending this message from the year 2080 as I've had to deal with, but I thought I'd give you a chance to implement this patch first. PPS No, I don't WANT there to be a "fuzzfactor" allowing "close enough" dates. Server Time Only.
# SKuck Header additions to CookHeaders.py of Mailman 2.1.5 - Nov 18, 2004 import time from email.Utils import parsedate_tz, mktime_tz, formatdate # End SKuck Header additions
...
msg['List-Id'] = listid_h
# The above included as an insertion location reference. I think it'll work from anywhere in "def process" # SKuck added to pounce on bad email time stamps. originaldate = msg.get('date') receivedtime = formatdate(time.time()) del msg['date'] del msg['x-original-date'] msg['Date'] = receivedtime msg['X-Original-Date'] = originaldate # End SKuck added ...
On Sat, Nov 20, 2004 at 03:31:38AM -0500, Steven Kuck wrote:
Suddenly, my new archives are coherent. I don't have to worry about
To keep your archives coherent, you can just set: ARCHIVER_CLOBBER_DATE_POLICY = 1 in your mm_cfg.py.
my time and there is a simple option I can set! Tell me that I don't have to port this code when I want to upgrade!
Rather than patching CookHeaders, you might consider making your own handler that only mungs the date, and then add that handler to the global pipeline in your mm_cfg. Then a new version of CookHeaders.py is not a problem.
-- Jim Tittsler http://www.OnJapan.net/ GPG: 0x01159DB6 Python Starship http://Starship.Python.net/ Ringo MUG Tokyo http://www.ringo.net/rss.html
At 3:31 AM -0500 2004-11-20, Steven Kuck wrote:
Since all of these messages are, in fact, being sent by my server I think it quite reasonable to change the "Date" to reflect the time that it was processed and changed by the server.
The problem is that you can't tell which "Date:" headers are good and which ones aren't. The message may have been held up in a queue for several days, or the date it was sent could be off by several days. By the time the message gets to you, it is impossible to distinguish between these two events.
IMO, you should adhere to the principle of minimal munging, and not replace a "Date:" header unless you can pretty conclusively prove that it was set wrong to begin with.
I'm the one who has to answerquestions about why the archive dates seem wrong and who is ultimately responsible for the smooth operation of the service. I would LIKE this to be an option that I could set. I do not have the time or inclination to write the code to make this a user option. However, since it something I wanted applied to all of my lists, I added the code to copy the "Date" field to "X-Original-Date" and put in the server's time as the "Date."
I would be violently opposed to any system-wide modification that would arbitrarily replace all "Date:" headers with ones based on the time of reception.
If there were a way to effectively detect when a "Date:" header was wrong and when it was okay, then I might be willing to allow the system to correct the "Date:" header in those particular cases. If you've got a patch or additional code that can do that, I'd like to see it, although I can't promise it would be accepted by the Mailman developers for inclusion in an upcoming version.
PS I considered sending this message from the year 2080 as I've had to deal with, but I thought I'd give you a chance to implement this patch first.
We've all had the problem.PPS No, I don't WANT there to be a "fuzzfactor" allowing "close enough" dates. Server Time Only.
I can't speak for Barry, Tokio, or the other Mailman developers, but I would be violently opposed to a change of this sort.
-- Brad Knowles, brad@stop.mail-abuse.org
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See http://www.sage.org/ for more info.
Brad Knowles wrote:
At 3:31 AM -0500 2004-11-20, Steven Kuck wrote:
Since all of these messages are, in fact, being sent by my server I think it quite reasonable to change the "Date" to reflect the time that it was processed and changed by the server.
The problem is that you can't tell which "Date:" headers are goodand which ones aren't. The message may have been held up in a queue for several days, or the date it was sent could be off by several days. By the time the message gets to you, it is impossible to distinguish between these two events.
I feel as though I've suggested making burgers from a sacred cow... I would like to figure out why it is sacred, so pardon my sacrilegious question: why does it matter? Please, no flames... Can you point me to the definitive text for mail header definitions? RFC 733 only defines the format - it doesn't say they are any more inviolate than the "Subject" line.
True, you can't tell if a message was sent from a misbehaving computer or if it got stuck in the queue of a misbehaving server. Surely, any dates in the future would be fair game for alteration - they can't have been "stuck" there since NEXT week. I'd also be suspect of any dates before 1970. Frankly, I'd be suspect of any date older than a three days (over a weekend).
I still think that the time on the originating computer (assuming the sender isn't inserting false dates) is a matter for trivia, and archival in a "X-Sender-Universal-Disjointed-Time" field. Why do I care if the message was stuck (unless it's my server) or if the user's clock is wrong?
IMO, you should adhere to the principle of minimal munging, andnot replace a "Date:" header unless you can pretty conclusively prove that it was set wrong to begin with.
Come now, it's not like I'm adding content to the body of the message!
Oh wait, mailman does that too.
I would be violently opposed to any system-wide modification thatwould arbitrarily replace all "Date:" headers with ones based on the time of reception.
As I said, I can guarantee messages from the future are wrong. Disagree?
Perhaps messages from more than a day (or N days) in the past could be bounced saying: "Either your system clock is wrong or your message was unreasonably delayed. Either fix your clock, or make sure your message is still current and send it again." Alternately, the message could be held for approval or date fixing, and you could set that user as "Date Impaired" so that all messages from that individual get fixed - if they're off.
PS I considered sending this message from the year 2080 as I've had to deal with, but I thought I'd give you a chance to implement this patch first.
We've all had the problem.
At least you recognize it as a "problem." I would be more than happy to
work on a reasonable, if not totally infallible "solution." I know as a
current solution we tolerate a message archive on this list from 2006.
If date > now, replace. Can we at least get that in there? I'll worry
about messages from 1989 or earlier on my own.
Steven Kuck - no need for "violent opposition" - I'll behave.
On Sat, 2004-11-20 at 04:04, Jim Tittsler wrote:
On Sat, Nov 20, 2004 at 03:31:38AM -0500, Steven Kuck wrote:
Suddenly, my new archives are coherent. I don't have to worry about
To keep your archives coherent, you can just set: ARCHIVER_CLOBBER_DATE_POLICY = 1 in your mm_cfg.py.
Right, and if Steven really wants server time to always override, you can set ARCHIVE_CLOBBER_DATE_POLICY = 2 and ARCHIVER_ALLOWABLE_SANE_DATE_SKEW to some really small value (minutes(5) or even 0).
-Barry
--On Saturday, November 20, 2004 6:22 AM -0500 Steven Kuck scrib@afn.org wrote:
As I said, I can guarantee messages from the future are wrong. Disagree?
Perhaps messages from more than a day (or N days) in the past could be bounced saying: "Either your system clock is wrong or your message was unreasonably delayed. Either fix your clock, or make sure your message is still current and send it again." Alternately, the message could be held for approval or date fixing, and you could set that user as "Date Impaired" so that all messages from that individual get fixed - if they're off.
This seems a reasonable approach, given a configurable delivery delay tolerance. One could also cross-check References headers against messages already received, to set a lower limit on the time stamp. If a message claims to have been sent prior to one it references (modulo some tolerance), then it can be bounced/modified/moderated.
On 11/20/2004 13:26, "Kenneth Porter" shiva@sewingwitch.com wrote:
--On Saturday, November 20, 2004 6:22 AM -0500 Steven Kuck scrib@afn.org wrote:
As I said, I can guarantee messages from the future are wrong. Disagree?
Perhaps messages from more than a day (or N days) in the past could be bounced saying: "Either your system clock is wrong or your message was unreasonably delayed. Either fix your clock, or make sure your message is still current and send it again." Alternately, the message could be held for approval or date fixing, and you could set that user as "Date Impaired" so that all messages from that individual get fixed - if they're off.
This seems a reasonable approach, given a configurable delivery delay tolerance. One could also cross-check References headers against messages already received, to set a lower limit on the time stamp. If a message claims to have been sent prior to one it references (modulo some tolerance), then it can be bounced/modified/moderated.
Keep in mind that there are MUAs which set the Date: as the message is begun, not when it is sent. If the sender leaves it as a draft for a couple of days before sending, the Date: will be "two days old". (What one wants to do with that date is another matter.)
--John
At 6:22 AM -0500 2004-11-20, Steven Kuck wrote:
I feel as though I've suggested making burgers from a sacred cow... I would like to figure out why it is sacred, so pardon my sacrilegious question: why does it matter? Please, no flames...
Riiiiiiiiiiiiiiiiiiiiiiiight.
You feel free to include all your own inflamatory comments before the question, and then insist that this can't lead to a flame war. This is not a good start.
Can you point me tothe definitive text for mail header definitions? RFC 733 only defines the format - it doesn't say they are any more inviolate than the "Subject" line.
I believe that the current RFC on this matter is 2822, see http://www.faqs.org/rfcs/rfc2822.html. The last paragraph of Section 3.6 says:
| The only required header fields are the origination date field and | the originator address field(s). All other header fields are | syntactically optional. More information is contained in the table | following this definition.
This implies to me that the "Date:" header is one of the two most important headers on the entire message. Keep that in mind. Section 3.6.1 says:
|3.6.1. The origination date field | | The origination date field consists of the field name "Date" followed | by a date-time specification. | |orig-date = "Date:" date-time CRLF | | The origination date specifies the date and time at which the creator | of the message indicated that the message was complete and ready to | enter the mail delivery system. For instance, this might be the time | that a user pushes the "send" or "submit" button in an application | program. In any case, it is specifically not intended to convey the | time that the message is actually transported, but rather the time at | which the human or other creator of the message has put the message | into its final form, ready for transport. (For example, a portable | computer user who is not connected to a network might queue a message | for delivery. The origination date is intended to contain the date | and time that the user queued the message, not the time when the user | connected to the network to send the message.)
There's nothing here that says that an MTA should, or can, correct a "Date:" header that it believes to be incorrect. There's nothing here that says an application can or should correct a "Date:" header that it believes to be incorrect.
You should also read section 3.6.6, and note that it is not applicable since "Resent-" fields are for when a user has re-introduced a message into the transport system, and this is not what we are doing. Moreover, the "Resent-" fields are purely informational, and do not over-ride the original fields in the message in any way.
For my part, all this means that the "Date:" header is even more inviolable than the "Reply-to:" header, which is a very old flamewar. See http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.048.htp.
As for the rest of your message, I consider it to be sufficiently inflamatory that I will not attempt to respond. You seem to have a clear agenda here, and there is nothing that will deter you from this path. If you wish to modify your system to function the way you want, there's not really much that any of us can do to try to change your mind.
However, if you do actually want to try to benefit the rest of the Mailman community, or at least to avoid having to keep re-porting your change to every new version of Mailman, you're going to have to make some concessions to the patch you are proposing.
-- Brad Knowles, brad@stop.mail-abuse.org
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See http://www.sage.org/ for more info.
"Steven" == Steven Kuck scrib@afn.org writes:
>> At 3:31 AM -0500 2004-11-20, Steven Kuck wrote:
>>> Since all of these messages are, in fact, being sent by my
>>> server I think it quite reasonable to change the "Date" to
>>> reflect the time that it was processed and changed by the
>>> server.It's equally reasonable to disobey "no trespassing" signs if you plan to avoid making a mess. In both cases, however, you're using a facility that is somebody else's property, and in both cases you may inadvertantly cause problems for the person who has the right to use it.
Steven> Can you point me to the definitive text for mail header
Steven> definitions? RFC 733 only defines the format - it doesn't
Steven> say they are any more inviolate than the "Subject" line.That's true. In fact, they're equally property of the author(s) (in the wording of RFC 822 and reaffirmed by 2822), and I detest mailing lists that insist on munging the Subject header, too. Even this one.<wink> If an MUA can't do the right thing based on the List-* headers, I want no part of it, not even to share a mailing list with it.
Of course, unlike "no trespassing", there are no courts to enforce the RFCs. But don't let that fool you. They are "legislation", the product of the skull sweat of dozens of "representatives" far more competent and hardworking than my Congresszombie, tested in practice, introduced and maintained through the cooperation and hard work of hundreds or thousands of programmers, distributors, and sys admins. The social contract says "These headers are for the use of authors", and you (as a list admin) want to hijack the "Date" header.
You say "well, it would be very convenient if I could use these facilities for my own purposes". I can't blame you for _wishing_; still, an honest man would create an appropriate header _for the use of the mailing list manager_, and then lobby and write the software to get it accepted and diffused through the community.
Specifically, you could use either an X-List-Sequence-Number header or an X-List-Receipt-Date header for your purpose. Teach the archivers about them (and the versions without "X-", since of course you'll submit an RFC, right?) for the benefit of web readers.
Re your examples:
Steven> Why do I care if the message was stuck (unless it's my
Steven> server) or if the user's clock is wrong?You don't. You've already proved that. But authors will care, if they are not responsible for the stuckage, and they look like idiots because their post was written without the benefit of a week's worth of further information, but nonetheless appears with a current date.
Steven> As I said, I can guarantee messages from the future are
Steven> wrong. Disagree?Depends on what you mean by "wrong". Your server may very well change the interval between, or even the order of, two posts from the same (ie, consistently translated) future. Whether this information is interesting or not is another question; you are, however, destroying it, and there's nothing in the RFCs that permits you to do that.
-- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.
On Mon, 2004-11-22 at 01:28, Stephen J. Turnbull wrote:
Specifically, you could use either an X-List-Sequence-Number header or an X-List-Receipt-Date header for your purpose. Teach the archivers about them (and the versions without "X-", since of course you'll submit an RFC, right?) for the benefit of web readers. Cool three Steves.
<humor style="sarcasm"> Well I think that a more generic approach might be great. Something that could show who received the message from who, at what time, for what address, and maybe other information as well. You could then teach the archivers what is the name of your mailing list. I think we can call it simply "received". Quick someone write a RFC for this one; I think it's a winner. Below are a few examples of just what might be possible. </humor>
Anyway I think that maybe the existing headers might have enough information to do the job. I also think that if the date shows too much skew into either the past or the future you shouldn't be forwarded seems someone already gave a config option that you can tweak for that.
Actually I think it could be cool to integrate in something like http://hashcash.org/ which would also force clocks to be in sync by at least a month.
I guess that I'm just trying to think of how things like spf/ses/srs, http://www.doaml.net/ , http://sourceforge.net/projects/gossip-project/ , hashcash, and a few other things might fit into the mailman world.
Received: from smtp-vbr8.xs4all.nl ([194.109.24.28]) by sccrmxc19.comcast.net (sccrmxc19) with ESMTP id <20041122092847s1900g6dkoe>; Mon, 22 Nov 2004 09:28:47 +0000 Received: from bag.python.org (bag.python.org [194.109.207.14]) by smtp-vbr8.xs4all.nl (8.12.11/8.12.11) with ESMTP id iAM9SliV035191 for stephen_pollei@comcast.net; Mon, 22 Nov 2004 10:28:47 +0100 (CET) (envelope-from mailman-developers-bounces+stephen_pollei=comcast.net@python.org) Received: from bag.python.org (bag [127.0.0.1]) by bag.python.org (Postfix) with ESMTP id 3DE791E400B for stephen_pollei@comcast.net; Mon, 22 Nov 2004 10:28:46 +0100 (CET) Received: from bag.python.org (bag [127.0.0.1]) by bag.python.org (Postfix) with ESMTP id EEFE01E4002 for mailman-developers@python.org; Mon, 22 Nov 2004 10:28:32 +0100 (CET) Received: from bag (HELO bag.python.org) (127.0.0.1) by bag.python.org with SMTP; 22 Nov 2004 10:28:32 +0100 Received: from tleepslib.sk.tsukuba.ac.jp (tleepslib.sk.tsukuba.ac.jp [130.158.98.109]) by bag.python.org (Postfix) with ESMTP for mailman-developers@python.org; Mon, 22 Nov 2004 10:28:32 +0100 (CET) Received: from steve by tleepslib.sk.tsukuba.ac.jp with local (Exim 4.34) id 1CWAUh-0005yb-Ai; Mon, 22 Nov 2004 18:28:23 +0900
-- http://dmoz.org/profiles/pollei.html http://sourceforge.net/users/stephen_pollei/ http://www.orkut.com/Profile.aspx?uid=2455954990164098214 http://stephen_pollei.home.comcast.net/ GPG Key fingerprint = EF6F 1486 EC27 B5E7 E6E1 3C01 910F 6BB5 4A7D 9677
On Sat, Nov 20, 2004 at 10:42:24AM +0100, Brad Knowles wrote:
If there were a way to effectively detect when a "Date:" header was wrong and when it was okay, then I might be willing to allow the system to correct the "Date:" header in those particular cases. If you've got a patch or additional code that can do that, I'd like to see it, although I can't promise it would be accepted by the Mailman developers for inclusion in an upcoming version.
I can't speak for Barry, Tokio, or the other Mailman developers, but I would be violently opposed to a change of this sort.
Out of curiosity, am I overlooking something or are we over-generalizing the solution?
My straw man (which may be wildly under-educated) is that we have no need at all to modify any headers, but instead to ignore the Date: header entirely for archive and digest purposes. Leave the Date header there, but don't use it. Instead use time of receipt.
Now, go grab a 2x4 and clue me in...
-- Joe Rhett Senior Geek Meer.net
At 7:13 PM -0800 2004-11-23, Joe Rhett wrote:
My straw man (which may be wildly under-educated) is that we have no need at all to modify any headers, but instead to ignore the Date: header entirely for archive and digest purposes. Leave the Date header there, but don't use it. Instead use time of receipt.
Date is one of the ways that the archive is sorted. You'd have to display a different header which would be consistent with the sort order, and change some other links to read "time of receipt" instead of "date". Overall, more work than just correcting the "Date:" header, although it would be more correct.
Do-able, yes. But not currently being done.
Moreover, there are already other methods currently built into the code for handling this issue with respect to the archives -- see previous messages from Barry and Jim.
For my part, this issue has as much to do with "correcting" messages as they arrive into the system (and before they are archived or sent back out again), as it is keeping the archives clean. It's one thing to just keep the archives clean, it's another to "correct" the message before it gets recorded in the archives or sent back out again.
-- Brad Knowles, brad@stop.mail-abuse.org
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See http://www.sage.org/ for more info.
--On Tuesday, November 23, 2004 19:13:58 GMT -0800 Joe Rhett jrhett@meer.net wrote:
On Sat, Nov 20, 2004 at 10:42:24AM +0100, Brad Knowles wrote:
If there were a way to effectively detect when a "Date:" header was wrong and when it was okay, then I might be willing to allow the system to correct the "Date:" header in those particular cases. If you've got a patch or additional code that can do that, I'd like to see it, although I can't promise it would be accepted by the Mailman developers for inclusion in an upcoming version.
I can't speak for Barry, Tokio, or the other Mailman developers, but I would be violently opposed to a change of this sort.
Out of curiosity, am I overlooking something or are we over-generalizing the solution?
My straw man (which may be wildly under-educated) is that we have no need at all to modify any headers, but instead to ignore the Date: header entirely for archive and digest purposes. Leave the Date header there, but don't use it. Instead use time of receipt.
Now, go grab a 2x4 and clue me in...
Hmm, I think that this whole conversation is not seeing the wood for the trees. Why would anyone want to sort a list by date? The date of a posting isn't really relevant except:
So that we can understand the currency of the information. For example, I don't want to be reading 1998 postings if I'm looking for information on the latest version of - say - Apache. For that it doesn't matter much that a date might be a few hours out.
Location in a conversational thread. It is important to read threads in order. For that, the important thing is the time that the email was sent, not the time that it arrived - which may differ by minutes, hours or days. However, the better solution is to sort by threads using in-reply-to headers. Google's gmail, and Apple's Mail application do this nicely.
I think that development focus should be on displaying threads properly, not on "fixing" dates that the system can't possibly know are right or wrong.
-- Ian Eiloart Servers Team Sussex University ITS
On Nov 24, 2004, at 2:09 PM, Ian Eiloart wrote:
Hmm, I think that this whole conversation is not seeing the wood for the trees. Why would anyone want to sort a list by date? The date of a posting isn't really relevant except: [snip several good reasons]
- Helping users sort the email in their inboxes. This can be important to some people, since I know when we had one user posting from 1980, many other list members weren't noticing her posts because their mail clients put those messages where the date: header said they should be.
That said, I think this is something that is better handled by mail clients and the original senders, not Mailman. But in the past, I've been asked for this feature for this reason, so I imagine other people want it for similar reasons.
Terri
At 7:09 PM +0000 2004-11-24, Ian Eiloart wrote:
I think that development focus should be on displaying threads properly, not on "fixing" dates that the system can't possibly know are right or wrong.
There is already a "thread sort" mode within the archiving system. There is also a "date sort" mode. If we're going to have a given mode, it makes sense that we'd want to be able to display that mode "correctly".
My issue has more to do with doing that as a display-only matter as opposed to "correcting" the messages on input and before they are sent back out to the list recipients, as well as being recorded in the archives. Where we do the "correction" is as important as the correction itself.
-- Brad Knowles, brad@stop.mail-abuse.org
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See http://www.sage.org/ for more info.
--On Wednesday, November 24, 2004 3:25 PM -0500 Terri Oda terri@zone12.com wrote:
- Helping users sort the email in their inboxes. This can be important to some people, since I know when we had one user posting from 1980, many other list members weren't noticing her posts because their mail clients put those messages where the date: header said they should be.
And the reverse problem of a message with date in the extreme future, which always sorts to the end of the mailbox.
On 11/24/2004 12:25, "Terri Oda" terri@zone12.com wrote:
- Helping users sort the email in their inboxes. This can be important to some people, since I know when we had one user posting from 1980, many other list members weren't noticing her posts because their mail clients put those messages where the date: header said they should be.
I seem to be unusual in that a. I prefer to keep my In boxes empty enough that all the messages show b. My attitude about problem mail is that if it's hard to read, toss it out...there's lots of other email in the world. Hard to read would--stretching things--include "hides itself due to faulty date."
--John
"Stephen" == Stephen Pollei stephen_pollei@comcast.net writes:
Stephen> Anyway I think that maybe the existing headers might have
Stephen> enough information to do the job.I agree. The point of adding a new header would be to make the result of the fairly complex analysis required available to other clients (both human and automatic), and to the current "internal pipeline", for that matter.
-- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.
I've been off for a bit, but...
On Sat, Nov 20, 2004 at 10:42:24AM +0100, Brad Knowles wrote:
If there were a way to effectively detect when a "Date:" headerwas wrong and when it was okay, then I might be willing to allow the system to correct the "Date:" header in those particular cases. If you've got a patch or additional code that can do that, I'd like to see it, although I can't promise it would be accepted by the Mailman developers for inclusion in an upcoming version.
As I've said, I don't speak Python. I submitted the hack I created, and while it was rightly criticized for it's lack of subtlety, no one pointed out that those changes would have unintended consequences that would break the software. THAT was of great comfort. It has -intended- consequences that you might not like, but I've been running with it for more than a month now and what had been a regular issue with mis-dated messages is GONE. Completely. There has not been a single complaint, or even a mention, that the dates were changed out from under the author. The date is important, but I don't know any author who considers it part of their composition.
My hope was that other people more fluent in Python might pick up the gauntlet. For me, the hack works and it's simple enough to re-hack once in a while for a new version release. There have been a couple proposals for the algorithm - nothing from the future, messages can't come from before a message they reference... While I could probably learn to do a Python future date compare in a few minutes, I don't know how to look up the date of a referenced message.
I would also be happy with an option to bounce messages from the future
or from too far in the past, but again, I don't know enough Python.
Even if I did I wouldn't want to redo that code for a new release. The
beauty of the simple hack is that it is SIMPLE. An elegant solution
should be part of a release. I can help pseudo-code an algorithm, but
implementation and testing in Python is not in the cards right now.
Ian Eiloart wrote:
Why would anyone want to sort a list by date? The date of a posting isn't really relevant except:
- So that we can understand the currency of the information. For example, I don't want to be reading 1998 postings if I'm looking for information on the latest version of - say - Apache. For that it doesn't matter much that a date might be a few hours out.
This is exactly my main problem - message currency. For my users, if
email is more than two weeks old, it might as well be ancient history.
Threads of discussion (while important) are less important than "what's
new today." I'm a data pack-rat, so I like the archives, but my users
will only glance at them rarely to verify claims. Old newspapers can be
used to line birdcages, old emails are less useful.
A new message with an old date can be important AND disregarded. That's the biggest danger I face. In general, my users:
- Sort their inbox by date (even if their mail client CAN sort by order received).
- Don't read all of the messages that come through, only the ones with subjects relevant to them.
- Don't keep their inboxes (or the destination folders) very clean.
So what happens? They (and sometimes I) don't notice that the "New Message" counter went up by 20 today, but there are only 19 displaying for today. Important messages can slip through the cracks. I'm not using MailMan to write a FAQ or host a conversation thread, but to keep users up to date on NEWS. If the message gets buried for a few days until someone cleans their inbox (if they do it that often) the message may as well not have been sent.
- Location in a conversational thread..... [deleted] I think that development focus should be on displaying threads properly, not on "fixing" dates that the system can't possibly know are right or wrong.
Obviously, we have different priorities. My users aren't "looking for" the latest info on anything, they are sending out notices of the latest info to the rest of the list. This is the latest price on XYZ corp stock! Act on it now. What was it yesterday? Who cares. (No, we're not doing financials - it's just an exaggerated example I think people will grok.)
Terri Oda wrote:
[snip several good reasons] 3. Helping users sort the email in their inboxes. This can be important to some people, since I know when we had one user posting from 1980, many other list members weren't noticing her posts because their mail clients put those messages where the date: header said they should be.
Yep. See?
That said, I think this is something that is better handled by mail clients and the original senders, not Mailman. But in the past, I've been asked for this feature for this reason, so I imagine other people want it for similar reasons. Terri
Well, thanks a lot, Terri :)
Unfortunately for me, I'm volunteering for an organization that is state-wide but county-based. Florida has 67 counties, and every one has their own operation. I cannot even TRY to THINK about enforcing some kind of standard on all those independent (and often volunteer) groups.
Dear So and So, Please fix your clock. "I don't know how." What OS are you using on your computer? "Dell, I think."
-> Implement Hack <-
That's why I chose to come to this forum, where I'm the ignorant rube
hoping to be educated on how to fix my problem. My problem is the 3
item "general user" description above. If ANYONE can tell me how to fix
them, that would be great (but my omnipotency isn't working right now).
Alternately, if enough people have similar problems with similar users,
maybe a solution can be found.
Steven Kuck - scrib@afn.org
On Thu, 2004-12-23 at 13:28, Steven Kuck wrote:
As I've said, I don't speak Python. I submitted the hack I created, and while it was rightly criticized for it's lack of subtlety, no one pointed out that those changes would have unintended consequences that would break the software. THAT was of great comfort. It has -intended- consequences that you might not like, but I've been running with it for more than a month now and what had been a regular issue with mis-dated messages is GONE. Completely. There has not been a single complaint, or even a mention, that the dates were changed out from under the author. The date is important, but I don't know any author who considers it part of their composition.
My hope was that other people more fluent in Python might pick up the gauntlet. For me, the hack works and it's simple enough to re-hack once in a while for a new version release. There have been a couple proposals for the algorithm - nothing from the future, messages can't come from before a message they reference... While I could probably learn to do a Python future date compare in a few minutes, I don't know how to look up the date of a referenced message.
This is the beauty of open source. Personally, I'd rather not have this option in standard Mailman for a number of reasons, but because it's all free software, who cares what I think? :) You don't have to, because you can make this pile of stuff do whatever you want, if the itch is annoying enough!
My reasons include a strong adherence to the least-munge rule, and the aversion to ever more options. Every option or configuration variable you add increases the maintenance burden, opportunity for bugs, and cognitive overhead for understanding and documenting the system. These days I don't think it's particularly difficult to set your computer's clock to a reasonably accurate time.
"Stephen J. Turnbull" stephen@xemacs.org wrote:>>>>> "Steven" == Steven Kuck writes:
At 3:31 AM -0500 2004-11-20, Steven Kuck wrote:
Since all of these messages are, in fact, being sent by my server I think it quite reasonable to change the "Date" to reflect the time that it was processed and changed by the server.
It's equally reasonable to disobey "no trespassing" signs if you plan to avoid making a mess. In both cases, however, you're using a facility that is somebody else's property, and in both cases you may inadvertantly cause problems for the person who has the right to use it.
Steven> Can you point me to the definitive text for mail header Steven> definitions? RFC 733 only defines the format - it doesn't Steven> say they are any more inviolate than the "Subject" line.
That's true. In fact, they're equally property of the author(s) (in the wording of RFC 822 and reaffirmed by 2822), and I detest mailing lists that insist on munging the Subject header, too. Even this one. If an MUA can't do the right thing based on the List-* headers, I want no part of it, not even to share a mailing list with it.
Of course, unlike "no trespassing", there are no courts to enforce the RFCs. But don't let that fool you. They are "legislation", the product of the skull sweat of dozens of "representatives" far more competent and hardworking than my Congresszombie, tested in practice, introduced and maintained through the cooperation and hard work of hundreds or thousands of programmers, distributors, and sys admins. The social contract says "These headers are for the use of authors", and you (as a list admin) want to hijack the "Date" header.
You say "well, it would be very convenient if I could use these facilities for my own purposes". I can't blame you for _wishing_; still, an honest man would create an appropriate header _for the use of the mailing list manager_, and then lobby and write the software to get it accepted and diffused through the community.
Specifically, you could use either an X-List-Sequence-Number header or an X-List-Receipt-Date header for your purpose. Teach the archivers about them (and the versions without "X-", since of course you'll submit an RFC, right?) for the benefit of web readers.
Re your examples:
Steven> Why do I care if the message was stuck (unless it's my Steven> server) or if the user's clock is wrong?
You don't. You've already proved that. But authors will care, if they are not responsible for the stuckage, and they look like idiots because their post was written without the benefit of a week's worth of further information, but nonetheless appears with a current date.
Steven> As I said, I can guarantee messages from the future are Steven> wrong. Disagree?
Depends on what you mean by "wrong". Your server may very well change the interval between, or even the order of, two posts from the same (ie, consistently translated) future. Whether this information is interesting or not is another question; you are, however, destroying it, and there's nothing in the RFCs that permits you to do that.
-- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/vaez_ha%40yahoo.co...
Do you Yahoo!? Send holiday email and support a worthy cause. Do good.
"Stephen J. Turnbull" stephen@xemacs.org wrote:>>>>> "Steven" == Steven Kuck writes:
At 3:31 AM -0500 2004-11-20, Steven Kuck wrote:
Since all of these messages are, in fact, being sent by my server I think it quite reasonable to change the "Date" to reflect the time that it was processed and changed by the server.
It's equally reasonable to disobey "no trespassing" signs if you plan to avoid making a mess. In both cases, however, you're using a facility that is somebody else's property, and in both cases you may inadvertantly cause problems for the person who has the right to use it.
Steven> Can you point me to the definitive text for mail header Steven> definitions? RFC 733 only defines the format - it doesn't Steven> say they are any more inviolate than the "Subject" line.
That's true. In fact, they're equally property of the author(s) (in the wording of RFC 822 and reaffirmed by 2822), and I detest mailing lists that insist on munging the Subject header, too. Even this one. If an MUA can't do the right thing based on the List-* headers, I want no part of it, not even to share a mailing list with it.
Of course, unlike "no trespassing", there are no courts to enforce the RFCs. But don't let that fool you. They are "legislation", the product of the skull sweat of dozens of "representatives" far more competent and hardworking than my Congresszombie, tested in practice, introduced and maintained through the cooperation and hard work of hundreds or thousands of programmers, distributors, and sys admins. The social contract says "These headers are for the use of authors", and you (as a list admin) want to hijack the "Date" header.
You say "well, it would be very convenient if I could use these facilities for my own purposes". I can't blame you for _wishing_; still, an honest man would create an appropriate header _for the use of the mailing list manager_, and then lobby and write the software to get it accepted and diffused through the community.
Specifically, you could use either an X-List-Sequence-Number header or an X-List-Receipt-Date header for your purpose. Teach the archivers about them (and the versions without "X-", since of course you'll submit an RFC, right?) for the benefit of web readers.
Re your examples:
Steven> Why do I care if the message was stuck (unless it's my Steven> server) or if the user's clock is wrong?
You don't. You've already proved that. But authors will care, if they are not responsible for the stuckage, and they look like idiots because their post was written without the benefit of a week's worth of further information, but nonetheless appears with a current date.
Steven> As I said, I can guarantee messages from the future are Steven> wrong. Disagree?
Depends on what you mean by "wrong". Your server may very well change the interval between, or even the order of, two posts from the same (ie, consistently translated) future. Whether this information is interesting or not is another question; you are, however, destroying it, and there's nothing in the RFCs that permits you to do that.
-- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/vaez_ha%40yahoo.co...
Do you Yahoo!? Yahoo! Mail - You care about security. So do we.
"Stephen J. Turnbull" stephen@xemacs.org wrote: >>>>> "Steven" == Steven Kuck writes:
At 3:31 AM -0500 2004-11-20, Steven Kuck wrote:
Since all of these messages are, in fact, being sent by my server I think it quite reasonable to change the "Date" to reflect the time that it was processed and changed by the server.
It's equally reasonable to disobey "no trespassing" signs if you plan to avoid making a mess. In both cases, however, you're using a facility that is somebody else's property, and in both cases you may inadvertantly cause problems for the person who has the right to use it.
Steven> Can you point me to the definitive text for mail header Steven> definitions? RFC 733 only defines the format - it doesn't Steven> say they are any more inviolate than the "Subject" line.
That's true. In fact, they're equally property of the author(s) (in the wording of RFC 822 and reaffirmed by 2822), and I detest mailing lists that insist on munging the Subject header, too. Even this one. If an MUA can't do the right thing based on the List-* headers, I want no part of it, not even to share a mailing list with it.
Of course, unlike "no trespassing", there are no courts to enforce the RFCs. But don't let that fool you. They are "legislation", the product of the skull sweat of dozens of "representatives" far more competent and hardworking than my Congresszombie, tested in practice, introduced and maintained through the cooperation and hard work of hundreds or thousands of programmers, distributors, and sys admins. The social contract says "These headers are for the use of authors", and you (as a list admin) want to hijack the "Date" header.
You say "well, it would be very convenient if I could use these facilities for my own purposes". I can't blame you for _wishing_; still, an honest man would create an appropriate header _for the use of the mailing list manager_, and then lobby and write the software to get it accepted and diffused through the community.
Specifically, you could use either an X-List-Sequence-Number header or an X-List-Receipt-Date header for your purpose. Teach the archivers about them (and the versions without "X-", since of course you'll submit an RFC, right?) for the benefit of web readers.
Re your examples:
Steven> Why do I care if the message was stuck (unless it's my Steven> server) or if the user's clock is wrong?
You don't. You've already proved that. But authors will care, if they are not responsible for the stuckage, and they look like idiots because their post was written without the benefit of a week's worth of further information, but nonetheless appears with a current date.
Steven> As I said, I can guarantee messages from the future are Steven> wrong. Disagree?
Depends on what you mean by "wrong". Your server may very well change the interval between, or even the order of, two posts from the same (ie, consistently translated) future. Whether this information is interesting or not is another question; you are, however, destroying it, and there's nothing in the RFCs that permits you to do that.
-- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/vaez_ha%40yahoo.co...
Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses.
On Fri, 2004-12-31 at 15:17, VAEZ(daneshjooye sharif) wrote:
That's true. In fact, they're equally property of the author(s) (in the wording of RFC 822 and reaffirmed by 2822), and I detest mailing lists that insist on munging the Subject header, too. Even this one. If an MUA can't do the right thing based on the List-* headers, I want no part of it, not even to share a mailing list with it.
Subject munging pre-dates RFC 2369, and today I think it's mostly a hold over. It also causes headaches with RFC 2047. Fortunately, you can turn this off in Mailman.
-Barry
One's detesting is another man's love. I absolutely love having the subject line prefix, as do virtually all of my mailing list owners. I also vote for having an option to "fix" the dates, also. Its a headache when emailers don't have their date set properly.
Bob
---------- Original Message ----------- From: Barry Warsaw barry@python.org To: "VAEZ(daneshjooye sharif)" vaez_ha@yahoo.com Cc: mailman-developers@python.org, "Stephen J. Turnbull" stephen@xemacs.org Sent: Sat, 01 Jan 2005 13:04:51 -0500 Subject: Re: [Mailman-Developers] Dates again
On Fri, 2004-12-31 at 15:17, VAEZ(daneshjooye sharif) wrote:
That's true. In fact, they're equally property of the author(s) (in the wording of RFC 822 and reaffirmed by 2822), and I detest mailing lists that insist on munging the Subject header, too. Even this one. If an MUA can't do the right thing based on the List-* headers, I want no part of it, not even to share a mailing list with it.
Subject munging pre-dates RFC 2369, and today I think it's mostly a hold over. It also causes headaches with RFC 2047. Fortunately, you can turn this off in Mailman.
-Barry ------- End of Original Message -------
"Bob" == Bob Puff bob@nleaudio.com writes:
Bob> One's detesting is another man's love. I absolutely love
Bob> having the subject line prefix, as do virtually all of my
Bob> mailing list owners.Having thought about it a bit, the RFC definitely reserves those headers to the authors, but if it's switched ON by the list owner and documented somewhere, it's reasonable to interpret that the authors have given (implicit) permission.
Bob> I also vote for having an option to "fix" the dates, also.
Bob> Its a headache when emailers don't have their date set
Bob> properly.Well, I just hope that (1) a better job will be done on it than was done on the CC-cleaning code, and (2) all the author-header-munging will default to off.
-- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.
participants (13)
-
Barry Warsaw -
Bob Puff -
Brad Knowles -
Ian Eiloart -
Jim Tittsler -
Joe Rhett -
John W. Baxter -
Kenneth Porter -
Stephen J. Turnbull -
Stephen Pollei -
Steven Kuck -
Terri Oda -
VAEZ(daneshjooye sharif)