[Mailman-Users] Incoming patch to fix a few Mailman bugs
Lenny Foner
foner at media.mit.edu
Thu Jul 6 08:48:24 CEST 2000
I just sent this to the developer list, but figured that the bugs and
(in some cases) patches might be useful to others as well, before they
can be looked at and/or incorporated into the current release. Enjoy.
[I'm sending this as a separate message to keep discussion from
bleeding from one list to another; note that I'm not on -either- list
and that you need to CC me if you want me to see anything before I
read it out of the archive maybe a month from now.]
- - - Begin forwarded message - - -
Date: Thu, 6 Jul 2000 02:43:19 -0400 (EDT)
From: Lenny Foner <foner at media.mit.edu>
To: mailman-developers at python.org
Subject: Incoming patch to fix a few Mailman bugs
[Please keep me CC'ed on any response you'd like me to see; I will
occasionally read the developer & user archives from the website, but
they're both too high-traffic for me to be subscribed to, so I'm only
on the announce list.]
Hi. On 18 June 2000, I sent three messages detailing about 14, uh,
"issues" I had with Mailman 2.0beta2 as a brand-new user of it to
administer some lists of mine. I've since discovered another serious
one (#0 below). Included below that is a patch which fixes #0, #3, #7
(badly---it just removes the text, even if reminder messages -are-
being sent!), #8, #9, #10, and #11 (but note that #11 is untested in
the umbrella-list configuration).
Fixing #0 actually required me to learn Python and a lot of the
(confusingly-implemented) design of how Mailman archives messages
(totally different control flows for real-time archiving vs rebulding
the archive, and none of this is documented in the source code); I did
both simply by reading the code on Saturday and Sunday. But be aware,
though, that the resulting code may have better solutions for a more
experienced Python programmer; I'm really a Common Lisp programmer at
heart. :)
Bugs #1 and #2 still look reasonably critical to me but are the ones
I'm least likely to be fixing, since they may require more familiarity
with the source than I'll have time to spend.
I'm including all of the original items after the patch, including #0
(which I haven't reported before), so you can figure out what I'm
talking about without digging up my old mail.
Note also that this patch is for 2.0beta2, not 2.0beta3---beta3 looks
a little too unstable for me to use right now, based on the mail I saw
on the users list immediately after its release. However, except
perhaps for line numbers, it doesn't look like much (or any) of my
patches conflict with anything that was done for beta3, so it should
be relatively easy to merge this code w/beta3, especially if you're
using reasonable merge tools. [If this code will definitely be
included in the next release, I might be convinced to do the merge
for you, but I'm not going to do so unless I get some feedback that
I'm even doing the right thing here to begin with...]
- - - Begin patch - - -
[This patch was made via "diff -ur" between my modified directory and
the virgin 2.0beta2 release, rather than via "patch" or "cvs diff" or
anything like that. This means that each file's differences are
separated by a line beginning with "diff -ur". I may put Mailman
under CVS control at my site if I wind up having to track a lot of
divergence between my patches and Mailman, but I'm hoping they'll be
folded in instead. Obviously, all the "Foner" comments should be
removed; they're there mostly to make it obvious to me, and maybe you,
what got changed.]
Also, -beware- of indentation! "diff -ur" inserted a leading space in
front of -every- line, in theory so that those with +'s and -'s in
them would line up with those that had none, but some of the lines
were indented with tabs before, hence -those- lines didn't have their
alignment changed because SPC TAB is exactly as wide as just TAB. So
the 1-character-variances in indentation you see below are spurious;
it'll be obvious when merging the patch back into the real sources.
diff -ur mailman-2.0beta2.virgin/Mailman/Archiver/Archiver.py mailman-2.0beta2/Mailman/Archiver/Archiver.py
--- mailman-2.0beta2.virgin/Mailman/Archiver/Archiver.py Sun Apr 2 20:34:01 2000
+++ mailman-2.0beta2/Mailman/Archiver/Archiver.py Mon Jul 3 02:07:00 2000
@@ -208,6 +208,7 @@
# from this point on, we're doing all the expensive archiving
# work.
txt = msg.unixfrom
+ self.unixfrom = msg.unixfrom ### Foner
for h in msg.headers:
txt = txt + h
if not msg.body or msg.body[0] <> '\n':
diff -ur mailman-2.0beta2.virgin/Mailman/Archiver/HyperArch.py mailman-2.0beta2/Mailman/Archiver/HyperArch.py
--- mailman-2.0beta2.virgin/Mailman/Archiver/HyperArch.py Fri Apr 7 15:06:58 2000
+++ mailman-2.0beta2/Mailman/Archiver/HyperArch.py Mon Jul 3 03:12:53 2000
@@ -62,16 +62,73 @@
def url_quote(s):
return urllib.quote(s)
-
+### +++ Foner.
+# [This template is used when generating archives/private/<listname>/<time>.txt
+# files, and -not- when generating archives/private/<listname.mbox>/<listname.mbox>
+# files!]
+# Changes from the original:
+# (a) Use unpruned_processed_headers so we can get most of the original
+# message headers. This isn't perfect, because the headers have already
+# been processed by Mailman on the way in, and hence the Subject: line
+# may have [listname] in it, multiple Subject: fields have been eliminated
+# (only the last one remains), the anti-looping header fields have been
+# added, and so forth. But it's far, far better than what it used to be.
+# (b) The original kluged its own From_ line from secondary sources. No, no, no!
+# You -cannot- do this---you just can't screw around with the From_ line
+# and put it into different formats or many, many mailreaders will suddenly
+# fail to recognize it as delimiting messages. Emacs RMAIL, in particular,
+# won't figure out that
+# From foner at media.mit.edu Sat, 1 Jul 2000 18:51:22 -0400 (EDT)
+# is the same thing as
+# From foner at out-of-band.media.mit.edu Sat Jul 1 18:51:24 2000
+# and hence will think that <listname>/<time>.txt files consist of a
+# single huge message. (I should also note that not even the UnixMailbox
+# class in Python-1.5.2/Lib/mailbox.py thinks that the former is valid:
+# _fromlinepattern = r"From \s*[^\s]+\s+\w\w\w\s+\w\w\w\s+\d?\d\s+" \
+# r"\d?\d:\d\d(:\d\d)?(\s+[^\s]+)?\s+\d\d\d\d\s*$"
+# pretty clearly rules it out, because it's got everything in the order
+# used in the latter case. (A quick way to notice this is that no commas
+# appear in that regexp...:)) The <listname.mbox>/<listname.mbox> files
+# are fine, since they still have their original From_ lines, but users
+# can't retrieve those from the webserver, so they're stuck. And if they
+# can't use their mail tools on the files they -can- retrieve, why would
+# they bother? So, to fix this, we carefully hand the original unixfrom
+# value all the way down from the call chain, in TWO different ways (!)
+# because real-time archiving has a completely different flow of control
+# than bin/arch uses when regenerating the archives.
+# (c) The original caused all messages to end with an extra TWO blank lines.
+# I changed this to zero (by putting the """ right after "%(body)s"), but
+# unfortunately this runs into a peculiar corner case: when bin/arch is
+# run, the very last message in the archive doesn't have a blank line
+# after it, which means that the next time a realtime archive is done
+# for an incoming message, its From_ line winds up butting up against
+# end the of the last message, with no intervening blank line. This
+# is probably legal, but I don't want to push it---I'm sure there is
+# -some- mailreader out there that will choke. And there's -really-
+# no easy way to make the pipermail.py's processUnixMailbox and
+# HyperArch.py's add_article (which overrides pipermail.py's) handle
+# the situation without doing something really gross, like having
+# processUnixMailbox hand add_article a special flag that says, "This
+# is the last article I'm giving you---put an extra newline on the end
+# of the file." So instead, we put an extra newline on -every- message.
+# Also gross (many other programs on the net likewise play it safe,
+# adding an extra newline "just in case" and hence causing messages to
+# grow trailing padding indefinitely), but oh well. I'm tired.
article_text_template="""\
-From %(email)s %(datestr)s
-Date: %(datestr)s
-From: %(author)s %(email)s
-Subject: %(subject)s
-
+%(unixfrom)s%(unpruned_processed_headers)s
%(body)s
-
"""
+###
+# article_text_template="""\
+# From %(email)s %(datestr)s
+# Date: %(datestr)s
+# From: %(author)s %(email)s
+# Subject: %(subject)s
+#
+# %(body)s
+#
+# """
+### ---
article_template='''\
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
@@ -118,8 +175,6 @@
</body></html>
'''
-
-
def CGIescape(arg):
s=cgi.escape(str(arg))
s=re.sub('"', '"', s)
@@ -202,9 +257,12 @@
def as_text(self):
d = self.__dict__.copy()
d["body"] = string.join(self.body, "")
+### +++ Foner
+ d["unixfrom"] = self.unixfrom
+ d["unpruned_processed_headers"] = string.join(self.headers, "")
+### ---
return self.text_tmpl % d
-
def __init__(self, message=None, sequence=0, keepHeaders=[]):
if message==None: return
self.sequence=sequence
@@ -273,9 +331,55 @@
string.split(references))
# Save any other interesting headers
- self.headers={}
- for i in keepHeaders:
- if message.has_key(i): self.headers[i]=message[i]
+### +++ Foner
+ # For the text-format file, save them -all-! This way, those
+ # who need to look at all headers have them available.
+ # Too bad that these headers are -after- Mailman's munging; thus things
+ # like Sender: are meaningless, and Mailman's anti-loop headers are
+ # present, etc. Really, we should preserve the exact original headers
+ # and use them here; perhaps they're already preserved and I simply
+ # haven't found them yet.
+ self.headers=message.headers
+###
+# self.headers={}
+# for i in keepHeaders:
+# if message.has_key(i): self.headers[i]=message[i]
+### ---
+
+### +++ Foner
+ # For the HTML-format file, only save ones that are generally useful,
+ # so it doesn't get too cluttered. Anyone who wants the rest can
+ # download the text-format file and peruse them with their favorite
+ # mailreading tools. The old code didn't bother to put -any- of the
+ # headers into the HTML. That, combined with the text-mode doing
+ # likewiae, made it totally impossible for anyone reading the archives
+ # to have any idea what other recipients (for instance) might have
+ # gotten the message---and hence might not know who else has been in
+ # on the discussion and whether the message should be send to them
+ # -again-.
+ #
+ # Note that, unlike self.headers and message.headers, this is a sequence,
+ # not a dictionary! Note also that you'll only get one Subject:, if the
+ # message had more than one, and it'll be the last one. This is due to
+ # the munging Mailman does to insert an optional [listname], which probably
+ # uses getheader and not getheaders. We use getallmatchingheaders here
+ # instead of getheaders because (a) the latter mysteriously gives us an
+ # AttributeError, and (b) it doesn't really matter whether whitespace has
+ # been removed, especially since this is for a web page. Finally, note
+ # that order matters in the list of headers below---it determines the
+ # order in which they wind up on the page. One could obviously add a
+ # large number of other fields (Resent-<foo>? Organization?) to this
+ # list; perhaps this should be parameterized per mailing list (uh oh...).
+ # Too bad that these headers are -after- Mailman's munging; thus things
+ # like Sender: are meaningless, and Mailman's anti-loop headers are
+ # present, etc. Really, we should preserve the exact original headers
+ # and use them here; perhaps they're already preserved and I simply
+ # haven't found them yet.
+ self.useful_headers_for_HTML=[]
+ for i in ['Date','From','To','CC','Subject']:
+ for j in message.getallmatchingheaders(i):
+ self.useful_headers_for_HTML.append(j)
+### ---
# Read the message body
self.body=[]
@@ -356,9 +460,11 @@
</ul>
<p>
<hr>
- <i>This archive was generated by
- <a href="http://starship.skyport.net/crew/amk/maintained/pipermail.html">
- Pipermail %(version)s</a>.</i>
+ <!-- This link seems to be 404 [Foner 6/17/00]
+ <i>This archive was generated by
+ <a href="http://starship.skyport.net/crew/amk/maintained/pipermail.html">
+ Pipermail %(version)s</a>.</i>
+ -->
</BODY>
</HTML>
'''
@@ -533,7 +639,6 @@
self._unlocklist=unlock
self._lock_file=None
-
#
# can't init the database while other
# processes are writing to it!
@@ -634,8 +739,10 @@
and the empty list is legal."""
if article.subject in ['subscribe', 'unsubscribe']: return None
res = self.dateToVolName(string.atof(article.date))
- self.message("figuring article archives\n")
- self.message(res + "\n")
+### +++ This might be useful to the author, but not to me. --- Foner
+# self.message("figuring article archives\n")
+# self.message(res + "\n")
+### ---
return res
@@ -784,7 +891,6 @@
# If a string was returned, convert to a list:
if type(archives)==type(''): archives=[archives]
if archives==[]: return # Ignore the article
-
# Add the article to each archive in turn
article.filename=filename=self.get_filename(article)
article_text=article.as_text()
@@ -817,6 +923,10 @@
f.close()
# Write the text article to the text archive.
+ # [Note that this is what writes a 2000-July.txt type file,
+ # and -not- a <listname>.mbox/<listname>.mbox type file!
+ # This code runs both during real-time archiving -and-
+ # when bin/arch is run. --- Foner
archivetextfile=os.path.join(self.basedir,"%s.txt" % i)
f=open(archivetextfile, 'a+')
@@ -1000,7 +1110,15 @@
if i<l: source[i]=None ; i=i+1
def format_article(self, article):
- source=article.body ; dest=[None]*len(source)
+### +++ Foner
+ # Actually make the message in the HTML archives show some useful
+ # headers, instead of losing them all and making people guess about
+ # whether the message went anywhere else (for instance).
+ source=article.useful_headers_for_HTML + ['\n'] + article.body
+ dest=[None]*len(source)
+###
+# source=article.body ; dest=[None]*len(source)
+### ---
# Handle <HTML> </HTML> directives
if self.ALLOWHTML:
self.__processbody_HTML(source, dest)
@@ -1011,9 +1129,34 @@
s=source[i]
if s==None: continue
dest[i]=CGIescape(s) ; source[i]=None
- if len(dest) > 0:
- dest.insert(0, '<PRE>')
- dest.append('</pre>')
+ if len(dest) > 0:
+### +++ Foner
+ # The <PRE></PRE> construction wants to get wrapped around
+ # the message text -with no leading or trailing whitespace!-
+ # In particular, not having a trailing newline before the
+ # </PRE> is -essential- in making the <HR> after it not have
+ # an unsightly extra gob of vertical space between it and the
+ # end of the message (at least in NS 3.0x, and probably in
+ # other browsers, too). While we're at it, make it really
+ # be </PRE> and not </pre>; this doesn't matter to the browser,
+ # but it's gratuitously inconsistent with the way all the other
+ # markup is done in this program. Note that the method below
+ # is horribly inefficient, but it sure was easy to write; a
+ # more-efficient approach would scan the text and snip off
+ # only those lines & characters that are necessary, rather
+ # than doing a list->string->list transformation. This would
+ # typically mean that we scan at most a line or two forwards
+ # from the beginning and backwards from the end, rather than
+ # copying all of a possibly-large message, twice, and then
+ # having to add the newlines back onto each line, which no
+ # doubt means consing yet a -third- set of strings! Oh, well.
+ dest=string.split(('<PRE>' + string.strip(string.join(dest, "")) + '</PRE>'), '\n')
+ for i in range(0, len(dest)):
+ dest[i]=dest[i]+'\n'
+###
+# dest.insert(0, '<PRE>')
+# dest.append('</pre>')
+### ---
else:
# Do fancy formatting here
if self.SHOWBR:
diff -ur mailman-2.0beta2.virgin/Mailman/Archiver/pipermail.py mailman-2.0beta2/Mailman/Archiver/pipermail.py
--- mailman-2.0beta2.virgin/Mailman/Archiver/pipermail.py Sat Aug 21 01:28:32 1999
+++ mailman-2.0beta2/Mailman/Archiver/pipermail.py Mon Jul 3 02:43:14 2000
@@ -16,7 +16,10 @@
VERSION=__version__
CACHESIZE=100 # Number of slots in the cache
-
+### +++ Foner
+from Mailman import Mailbox
+import rfc822
+### ---
msgid_pat=re.compile(r'(<.*>)')
def strip_separators(s):
@@ -155,6 +158,7 @@
INDEX_EXT = ".html" # Extension for indexes
def __init__(self, basedir=None, reload=1, database=None):
# If basedir isn't provided, assume the current directory
if basedir==None: self.basedir=os.getcwd()
else:
@@ -378,11 +382,22 @@
def processUnixMailbox(self, input, articleClass=Article):
import mailbox
- mbox=mailbox.UnixMailbox(input)
+### +++ Foner
+ # We want Mailbox.Mailbox here, not mailbox.UnixMailbox,
+ # because the latter uses the broken _search_start method
+ # that starts just -after- the From_ line and hence can't
+ # possibly get unixfrom bound to a non-null string. So
+ # inhert a _searchstart and .next method that works right.
+ mbox=Mailbox.Mailbox(input)
+# mbox=mailbox.UnixMailbox(input)
+### ---
while (1):
m=mbox.next()
if not m: break # End of file reached
a=articleClass(m, self.sequence) # Create an article object
+### +++ Foner
+ a.unixfrom = m.unixfrom # Must copy this from the message into the article.
+### ---
self.sequence=self.sequence+1 # Increment the archive's sequence number
self.add_article(a) # Add the article
diff -ur mailman-2.0beta2.virgin/Mailman/Cgi/admindb.py mailman-2.0beta2/Mailman/Cgi/admindb.py
--- mailman-2.0beta2.virgin/Mailman/Cgi/admindb.py Tue Apr 4 19:12:50 2000
+++ mailman-2.0beta2/Mailman/Cgi/admindb.py Sun Jun 18 13:32:19 2000
@@ -222,10 +222,10 @@
t.AddRow([
Bold('If you reject this post, explain (optional):'),
TextArea('comment-%d' % id, rows=4, cols=60,
- text=("Please do *not* post administrative requests"
- " to the mailing list! If you wish to subscribe,"
- " visit %s or send a 'help' message to the"
- " the request address, %s , for instructions"
+ text=("Please do not post administrative requests"
+ " to the mailing list. If you wish to subscribe,"
+ " visit %s or send a 'help' message to"
+ " the request address, %s, for instructions."
% (mlist.GetAbsoluteScriptURL('listinfo'),
mlist.GetRequestEmail())))
])
diff -ur mailman-2.0beta2.virgin/Mailman/Handlers/Hold.py mailman-2.0beta2/Mailman/Handlers/Hold.py
--- mailman-2.0beta2.virgin/Mailman/Handlers/Hold.py Tue Mar 21 01:25:51 2000
+++ mailman-2.0beta2/Mailman/Handlers/Hold.py Sun Jun 18 13:10:22 2000
@@ -69,7 +69,7 @@
pass
class Administrivia(HandlerAPI.MessageHeld):
- "Message may contain adbministrivia"
+ "Message may contain administrivia"
pass
class SuspiciousHeaders(HandlerAPI.MessageHeld):
diff -ur mailman-2.0beta2.virgin/Mailman/Mailbox.py mailman-2.0beta2/Mailman/Mailbox.py
--- mailman-2.0beta2.virgin/Mailman/Mailbox.py Tue Mar 21 01:24:59 2000
+++ mailman-2.0beta2/Mailman/Mailbox.py Mon Jul 3 02:36:46 2000
@@ -19,9 +19,19 @@
import mailbox
+### +++ Foner
+import sys
+import rfc822
+import Message
+### ---
class Mailbox(mailbox.UnixMailbox):
# msg should be an rfc822 message or a subclass.
+ #
+ # Note that AppendMessage gets called by the realtime archiver to stick a
+ # message into the <listname>.mbox/<listname>.mbox file, and -not- to stick
+ # a message into, e.g., 2000-July.txt! --- Foner
+ #
def AppendMessage(self, msg):
# seek to the last char of the mailbox
self.fp.seek(1,2)
@@ -34,3 +44,51 @@
self.fp.write('\n')
self.fp.write(msg.body)
+### +++ Foner
+ # Mailman/pythonlib/rfc822.py assumes that a message starts ON the From_ line,
+ # while Python-1.5.2/Lib/mailbox.py's mailbox.UnixMailbox (via its _search_start
+ # and _search_end) thinks that it starts AFTER the From_ line. Unfortunately, the
+ # latter interpretation means that when pipermail.py's processUnixMailbox calls
+ # mailbox.next (or, rather, our overridden Mailbox.next which runs rfc822.__init__,
+ # which the mailbox class somehow doesn't---see comments at "def next" below), the
+ # file position is already AFTER the From_ line, so readheaders in the __init__
+ # method can't possibly see it. Thus, we must override the _search_start|end
+ # methods with ones that think that a message starts at its From_ line and ends
+ # immediately before the next one.
+
+ def _search_start(self):
+ while 1:
+ _saved_from_line_pos = self.fp.tell() ### Foner
+ line = self.fp.readline()
+ if not line:
+ raise EOFError
+ if line[:5] == 'From ' and self._isrealfromline(line):
+ self.fp.seek(_saved_from_line_pos) ### Foner
+ return
+
+ def next(self):
+ while 1:
+ self.fp.seek(self.seekp)
+ try:
+ self._search_start()
+ except EOFError:
+ self.seekp = self.fp.tell()
+ return None
+ start = self.fp.tell()
+ self.fp.readline() # Skip this From_ so we can find the next one. Must work, 'cause we found a From_ w/o EOF.... --- Foner
+ self._search_end()
+ self.seekp = stop = self.fp.tell()
+ if start <> stop:
+ break
+### +++ Foner
+ # We must instantiate Message.Message here, not rfc822.Message!
+ # For some reason I can't figure out, the latter never runs its
+ # __init__ method, which means it never does readheaders, which
+ # means it never sets unixfrom! (Even if the filepos is correct,
+ # failing to run the init means we can't ever win.) I don't know
+ # why it doesn't init, but Message.Message explicitly runs
+ # rfc822.Message.__init__, so we win.
+ return Message.Message(mailbox._Subfile(self.fp, start, stop))
+###
+# return rfc822.Message(mailbox._Subfile(self.fp, start, stop))
+### ---
diff -ur mailman-2.0beta2.virgin/Mailman/Utils.py mailman-2.0beta2/Mailman/Utils.py
--- mailman-2.0beta2.virgin/Mailman/Utils.py Sun Apr 2 20:30:22 2000
+++ mailman-2.0beta2/Mailman/Utils.py Sat Jun 17 01:52:47 2000
@@ -55,7 +55,7 @@
# a much more naive implementation than say, Emacs's fill-paragraph!
-def wrap(text, column=70):
+def wrap(text, column=78):
"""Wrap and fill the text to the specified column.
Wrapping is always in effect, although if it is not possible to wrap a
diff -ur mailman-2.0beta2.virgin/templates/convert.txt mailman-2.0beta2/templates/convert.txt
--- mailman-2.0beta2.virgin/templates/convert.txt Fri Aug 14 12:17:26 1998
+++ mailman-2.0beta2/templates/convert.txt Sun Jun 18 12:47:20 2000
@@ -9,9 +9,8 @@
2) You have been given an arbitrary password to prevent others from
unsubscribing you without your knowledge. It will be mailed to you in
-a separate email, which you may have already received. Don't worry if
-you forget this password; a reminder will be sent to you via email
-every month.
+a separate email, which you may have already received. You can ask
+the server to send it to you again if you forget it.
3) If you have World Wide Web access, you can use it any time to
unsubscribe from this list, to switch to and from digest mode, to
diff -ur mailman-2.0beta2.virgin/templates/subscribeack.txt mailman-2.0beta2/templates/subscribeack.txt
--- mailman-2.0beta2.virgin/templates/subscribeack.txt Wed Dec 16 19:14:00 1998
+++ mailman-2.0beta2/templates/subscribeack.txt Sun Jun 18 23:36:59 2000
@@ -1,4 +1,4 @@
-Welcome to the %(real_name)s@%(host_name)s mailing list!
+Welcome to the %(real_name)s@%(host_name)s mailing list.
%(welcome)s
To post to this list, send your email to:
@@ -12,8 +12,7 @@
from digest mode, change your password, etc.), visit your subscription
page at:
- %(optionsurl)s
-%(umbrella)s
+ %(optionsurl)s %(umbrella)s
You can also make such adjustments via email by sending a message to:
%(real_name)s-request@%(host_name)s
@@ -26,11 +25,5 @@
%(password)s
-If you forget your password, don't worry, you will receive a monthly
-reminder telling you what all your %(host_name)s mailing list passwords
-are, and how to unsubscribe or change your options. There is also a
-button on your options page that will email your current password to
-you.
-
-You may also have your password mailed to you automatically off of the
-Web page noted above.
+If you forget your password, you can ask the server to send it to you
+again.
- - - End patch - - -
New #0:
(0) Mailman doesn't keep enough original headers around in its message
archives to make it apparent to anyone where the message went, because it
doesn't keep -any- of them, really. So if the message went to secondary
addresses, there's no way for the reader of the archives to know, yet it's
often critical to know where -else- a message went, etc . Also, its
flat-text archive version tries to roll its own version of the From_ (unix
envelope) line, and gets it wrong, to the point where neither the Python
libraries nor other mail-handling programs (e.g., Emacs Rmail) even
recognize them as From_ lines in the first place--- hence they treat the
entire archive as one big message, which entirely defeats the purpose of
making a flat-text version of the archive available, because it breaks
users' ability to read them except by paging through them as a single big
file. I've fixed the former by keeping -all- headers in the flat-text
file (since users downloading that will presumably use their normal
mailreading tools on them, which will no doubt prune some), and by keeping
some of the most useful headers in the HTML archives (few enough to not
clutter up the page too much, but enough to figure out where else the
message went). I've fixed the latter by carefully preserving the From_
line by taking pains to pass it down the call stack (in two different
ways, because Mailman has two entirely different archiving strategies).
Fixing these was remarkably difficult, because (a) real-time vs after-the-
fact archive rebuilding use -completely- different call paths and very
different code, it's not documented anywhere which methods run in each
case, and they're a tangled mess of Python libraries (in the distribution),
other libraries in pythonlib, and then patches to -those- methods elsewhere
in Mailman to override bugs in them---can this maybe be cleaned up and all
put in a single file, please?--and (b) Mailman/pythonlib/rfc822.py assumes
that a message starts ON the From_ line, while Python-1.5.2/Lib/
mailbox.py's mailbox.UnixMailbox (via its_search_start) assumes it starts
AFTER the From_ line, so readheaders in the __init__ method can't possibly
see it (and besides, rfc822.Message never seems to -run- its __init__
method, which doesn't help much). Finally, the HTML version of the
messages isn't formatted very well, because of carelessness with the stuff
inside <PRE></PRE>.
Original items (from 3 messages), concatenated into one:
Date: Sun, 18 Jun 2000 15:01:38 -0400 (EDT)
From: Lenny Foner <foner at media.mit.edu>
To: mailman-developers at python.org
Subject: Various mailman bugs (all in 2.0beta2 under HPUX 10.20)
I've just started using Mailman, and, while it's quite a nice little
package, I've already tripped over a number of bugs. I've listed them
approximately in order from most severe to least severe below; I've
fixed 3 & 7-11 myself, but 1 & 2 require someone with at least a
passing familiarity with Python and Mailman's internals, and of course
they should all wind up in the current source tree.
I also spent a while poking at the Jitterbug interface, and concluded
that it wasn't likely that I'd be able to figure out which category
each individual bug should go into, or that it would save anybody any
time for me to try to stick them in the interface in the first place.
It's also incredibly painful to submit bugs by filling in web forms,
because the editing environment available is so poor. Having the
forms used by Jitterbug also allow file uploading (when supported by
the browser, which most do) would help, but aren't a complete solution.
(1) Mailman doesn't send error reports to the right place. Our setup
here has individual workstations, such as foo.media.mit.edu,
masquerade such that their mail headers are rewritten to appear to
come from media.mit.edu (which forwards mail to our main mail-handling
machine, and also has an MX pointing there). This is usually how
large sets of machines are set up everywhere. But when I was trying
to configure Mailman (with the wrong compiled-in GID [it wasn't clear
to me whether it was the webserver's GID or sendmail's GID that was
the right one], hence any request caused the wrapper to err), Mailman
evidently went out of its way to attempt to send the error mail to the
individual workstation, e.g., foo.media.mit.edu, and -not- to
media.mit.edu, which is what the From: address in the incoming mail to
Mailman specified and which should be respected in any event---From:
should -always- take precedence over some attempt to divine a better
address. After all, Mailman cannot possibly have a better idea of
which machine is handling mail for the sender than the sender itself
does. (This meant, btw, that the error mail simply got wedged in the
sendmail queue of the machine I'm running Mailman on, since individual
workstations here don't listen to SMTP requests.)
(2) Sending a message with no subject and no body to listname-request
causes the blank message to simply arrive at the admin address. I
contend that it would be far nicer to the users -and- the administrator
if a blank message acted like one with "help" in it. Otherwise, users
see a black hole (until the admin responds), and admins need to deal
with responding in a way that gets users to ask "help" anyway, most
likely. It's just a pain all around.
(3) mailman-2.0beta2/Mailman/Archiver/HyperArch.py puts the URL
http://starship.skyport.net/crew/amk/maintained/pipermail.html
into the footer of its page describing the message archives,
but this is a dead link. (I just commented out the footer entirely.)
(4) Feature request: Mailman should support a --with-default-url as a
build argument so it can use cnames and/or virtual hosts. Ditto for
the FQDN of the host. I had to go in and screw around with various
options files to get this set up correctly after Mailman guessed; why
make the user edit this instead of taking it as an arg? (Yes, I know
you can change it one list at a time in the admin web interface, but
the default should be right, too.)
(5) The doc says to run "make install" in at least 2 places, but never
says to actually run "make" anywhere. But you -do- have to do a make :)
(6) Should I be creating mailman-owner or owner-mailman as aliases?
The doc is inconsistent, so I did both. Is this specific to the MTA?
If so, you should figure it out and document it for common ones. (I'm
using sendmail 8.9.3.)
(7) Even if the list administrator has disabled monthly password
mailings, templates/convert.txt and templates/subscribeack.txt both
claim that users' passwords will be mailed to them monthly (in fact,
the latter seems to say the same thing twice, in both the last and
penultimate paragraphs). There should be some way for these templates
to check the current setting for the list, and to fail to include this
text if monthly reminders are turned off. Mailman/HTMLFormatter.py
appears to make this check correctly, though I'm not positive that it's
generating the page I think it is. [I find monthly password reminders
absolutely pernicious---after all, if I forget the password, I can
-ask- for it and get an instant response from the server anyway. And
this way, I don't have the extra traffic, my password doesn't get
sent more than it has to over insecure channels, and very low-volume
lists don't send me mostly password-reminder traffic instead of
blessed silence.]
(8) Three's some bad formatting in the help page returned by sending a
"help" request to listname-request, apparently caused by linewrapping
performed by Utils.wrap (in Mailman/Utils.py)---Utils.wrap defaults to
70 columns wide, but the template text in template/help.txt has lines
longer than this (as do a couple of other calls from elsewhere in the
source code which call Utils.wrap on a fixed string; these are less
objectionable, in general, because the constant text isn't also
indented). The end result looks pretty awful:
end or --
Stop processing commands (good to do if your mailer
automatically
adds a signature file - it'll save you from a lot of cruft).
should be
end or --
Stop processing commands (good to do if your mailer automatically
adds a signature file - it'll save you from a lot of cruft).
and
unsubscribe from a different address than the one you
subscribed
from, you may specify it in the 'address' field.
should be
unsubscribe from a different address than the one you subscribed
from, you may specify it in the 'address' field.
I've fixed this by changing the default in Utils.wrap to 78
characters, which is half-baked but works.
(9) Mailman/Handlers/Hold.py spells administrivia as adbministrivia instead
(there shouldn't be a "b" in that word...)
(10) Mailman/Cgi/admindb.py's default textbox message that says
"Please do *not* post administrative requests to the list!" has a
repeated "the" in the next sentence, so it talks about "the the
request address". (The two "the"'s are separated by a newline in the
source, which makes this harder to see.) I removed the repeat and
also changed the first sentence to "Please do not post administrative
requests to the list." (no *, no !) so it seems less like the admin is
yelling at the poor user.
(11) templates/subscribeack.txt puts an extra newline before "You" in
the normal (non-umbrella) case in:
%(optionsurl)s
%(umbrella)s
You can also make such adjustments via email by sending a message to:
which I've fixed simply by doing
%(optionsurl)s %(umbrella)s
You can also make such adjustments via email by sending a message to:
but this also requires putting the right number of newlines into the
string in Deliverer.py which generates the string in umbrella. Since
I can't easily test it, I didn't bother fixing that.
(12) It would be less confusing if the "details" page for various
configuration stuff didn't say "don't change it here---change it on
the previous page" or whatever. Either make the form active on the
details page, or don't make it a form at all. This wasn't a problem
for me, but I imagine it will be for others. (Yes, I understand it's
easier to be able to reuse the code that generates the form this way,
but surely it can't be that hard to modularize.)
(13) Things that get bounced to the admin address because they look
like administrivia (e.g., sending "help" to the main list address)
have -two- X-BeenThere: message headers. Since this header is
presumably there to stop loops, it strikes me as odd that two of them
could be in a message under any circumstances. (Maybe they're only
checked for messages intended to go all the way out through the main
list---the two of them -do- seem harmless so far---but it's still odd.)
(14) When setting archival options, the form specifies the time period as
yearly/monthly/quarterly/weekly/daily. This is an odd order. How about
yearly/quarterly/monthly/weekly/daily, which at least leaves them in size
order?
- - - End forwarded message - - -
More information about the Mailman-Users
mailing list