[Mailman-Users] Archives and bad timestamps

[Brad Knowles]

At 7:47 PM -0700 2006-08-11, G. Armour Van Horn wrote:

>  I've just noticed that one of the lists I'm moderating shows messages
>  from next Monday in the private archive. The last message is timestamped
>  for 8:08 (Pacific, +0700) on 14 Aug, but was actually distributed by
>  Mailman at 08:51 on 11 Aug.

Yup.  Some users have screwed-up machines where the clock is way off.

>  Unfortunately, the archive seems to be ordered by timestamp rather than
>  the time the message hit the server, which leads to some odd sequences
>  of messages. It also makes me really wonder about messages that have
>  timestamps that are not in the current month and where they will be
>  archived. Will they show up in the archive for the month they were
>  timestamped, when that month is archived?

In the web archive, they will be put into the appropriate month and 
year, according to the "Date:" header on the message.  If that user 
has their clock off in 2063 La-La Land, then their message will show 
up in the archives for 2063.

>  There doesn't seem to be an option for archive sequence in the admin
>  pages, is there an option at the shell level to use the transit time
>  instead of the original timestamp? Should there be?

IIRC, if you select the appropriate option in the mm_cfg.py file, 
what you get by default is a certain time window around what the 
Mailman server thinks is the current date/time, and if the incoming 
message has a "Date:" header outside of that window, then it gets 
"corrected" to be the date/time stamp as of when the server received 
the message.

I thought there was some discussion of this issue in the FAQ, but I'm 
not seeing anything obvious come up.  Searching the archives, I found 
a few articles at 
<http://mail.python.org/pipermail/mailman-users/2003-August/031063.html>, 
<http://mail.python.org/pipermail/mailman-users/2003-May/028679.html>, 
<http://mail.python.org/pipermail/mailman-users/2003-November/032764.html>, 
and 
<http://mail.python.org/pipermail/mailman-users/2004-January/033907.html> 
which are related to this issue.  See also my canned Google search at 
<http://tinyurl.com/jsqsu>.


 From Defaults.py on a 2.1.5 installation that I manage, I have:

# This sets the default `clobber date' policy for the archiver.  When a
# message is to be archived either by Pipermail or an external archiver,
# Mailman can modify the Date: header to be the date the message was received
# instead of the Date: in the original message.  This is useful if you
# typically receive messages with outrageous dates.  Set this to 0 to retain
# the date of the original message, or to 1 to always clobber the date.  Set
# it to 2 to perform `smart overrides' on the date; when the date is outside
# ARCHIVER_ALLOWABLE_SANE_DATE_SKEW (either too early or too late), then the
# received date is substituted instead.
ARCHIVER_CLOBBER_DATE_POLICY = 2
ARCHIVER_ALLOWABLE_SANE_DATE_SKEW = days(15)

Of course, if you decide that you want a different choice, you should 
change this in your mm_cfg.py file, and not in Defaults.py -- the 
former will not get replaced if/when you upgrade your Mailman 
installation, while the latter is guaranteed to get replaced on 
upgrade/reinstall.


Now, one thing I'd like to see is related to that last message I 
referenced above -- the archive date handling process should be able 
to look at the last (most recent) "Received:" header and pull out a 
useful date from there, if the date on the message is too far 
out-of-whack.

Hopefully, this is the kind of thing that we could get relatively 
quickly fixed and incorporated into an upcoming release of Mailman 
2.1.x, because I know that this entire system is being completely 
re-done for Mailman 2.2 and this should no longer be a problem, 
although the author for that code should also be made aware of this 
issue.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.