Mailman 3 July 2000 - Mailman-Developers

code documentation
by Kathryn Hill July 6, 2000

July 6, 2000

Does anyone have documentation for all the different tokens (%(listname)s, %(real_name)s, etc) in the Mailman program? Some of them are obvious, but I would like to customize some of the templates but it's not proving to easy. Thanks for any assistance. -Kathryn Hill

1 0

Re: [Mailman-Developers] Incoming patch to fix a few Mailman bugs
by Nigel Metheringham July 6, 2000

July 6, 2000

Comments on a couple of these issues... foner(a)media.mit.edu said: > Bugs #1 and #2 still look reasonably critical to me but are the ones > I'm least likely to be fixing, since they may require more familiarity > with the source than I'll have time to spend. [from earlier message...] foner(a)media.mit.edu said: > (1) Mailman doesn't send error reports to the right place. Our setup > here has individual workstations, such as foo.media.mit.edu, > masquerade such that their mail headers are rewritten to appear to > come from media.mit.edu (which forwards mail to our main mail-handling > machine, and also has an MX pointing there). This is usually how > large sets of machines are set up everywhere. But when I was trying > to configure Mailman (with the wrong compiled-in GID [it wasn't clear > to me whether it was the webserver's GID or sendmail's GID that was > the right one], hence any request caused the wrapper to err), Mailman > evidently went out of its way to attempt to send the error mail to the > individual workstation, e.g., foo.media.mit.edu, and -not- to > media.mit.edu, which is what the From: address in the incoming mail to > Mailman specified and which should be respected in any event---From: > should -always- take precedence over some attempt to divine a better > address. After all, Mailman cannot possibly have a better idea of > which machine is handling mail for the sender than the sender itself > does. (This meant, btw, that the error mail simply got wedged in the > sendmail queue of the machine I'm running Mailman on, since individual > workstations here don't listen to SMTP requests.) No no no a thousand time no! Error reports must always (thats always) be sent to the envelope sender. If your systems send out mail with a broken envelope sender that is *your* problem and not mailman's. [That error mail is also being generated by the MTA not mailman]. If your machines do not accept SMTP to machine name, and machine name does not have an MX to somewhere that handles mail for machine name, then your mail config is terminally broken and should be barred from the net. foner(a)media.mit.edu said: > (4) Feature request: Mailman should support a --with-default-url as a > build argument so it can use cnames and/or virtual hosts. Ditto for > the FQDN of the host. I had to go in and screw around with various > options files to get this set up correctly after Mailman guessed; why > make the user edit this instead of taking it as an arg? (Yes, I know > you can change it one list at a time in the admin web interface, but > the default should be right, too.) Config is best done by configuration and not by hardcoding to a build. Most people will end up installing this as a package, ie rpm. foner(a)media.mit.edu said: > (6) Should I be creating mailman-owner or owner-mailman as aliases? > The doc is inconsistent, so I did both. Is this specific to the MTA? > If so, you should figure it out and document it for common ones. (I'm > using sendmail 8.9.3.) mailman (and listname) -owner is what mailman uses. sendmail from memory has some magic related to owner-<name> related to some oddity in list handling. Personally I intercept both in my MTA rules. Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham(a)VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ]

1 0

another couple mailman bugs (still 2.0beta2)
by Lenny Foner July 6, 2000

July 6, 2000

(15) If someone who is not currently subscribed to a list sends a "subscribe" message to it, and if the list rejects messages from people who aren't subscribed, they get a message like the following: Your mail to 'FOO' with the subject subscribe Is being held until the list moderator can review it for approval. The reason it is being held: Post by non-member to a members-only list But this is stupid. What -should- happen is, if the list is currently set to notice things that look like administrivia, the message should be internally bounced to the -request address, so they user gets to participate in the confirmation and/or approval dialog. In other words, Mailman should check administrivia-to-list -first-, (and reflect it to -request), -then- check for non-subscriber posts. Perhaps it's a bad idea to do this for -all- things that look like administrivia, because then the moderator might not get to approve something that really wasn't administrivia (because it spuriously wound up at the -request address)---but if so, Mailman should at least check the special case we have above. Once again, that's: (a) administrivia to main list (b) administrivia looks like "subscribe" (c) uername is not already subscribed if so, then bounce to -request address and do whatever the -request address would do with a subscription request [By the way, in the bounce message above, that "Is" should be "is", no doubt.] (16) Mailman can't make up its mind about capitalization. If I've told it that a list should be called FOO (e.g., via the first entry in the General Options configuration page), then the welcome message when someone subscribes says "Welcome to the FOO(a)some.domain mailing list." and then continues with the welcome text I've configured via the web interface. But the "confirmation of subscription" message the user gets first refers to the list as "foo(a)some.domain" and "foo-request(a)some.domain" and so forth. I haven't checked all the other various messages and pages, but I'll bet they're equally inconsistent. This is (a) ugly and (b) potentially a problem, because not all mailsystems treat stuff to the left of the @ as case-insensitive. The -right- approach is actually to have -two- configurable things in the web interface: the "human-readable name" and the "mailsystem name", just in case the former wants to be FOO and the latter wants to be foo. And if this is done, Mailman needs to use the right one at each point in its templates. (Even if this -isn't- done, Mailman needs to be consistent about whichever one it chooses. But it should really allow configuring both.)

1 0

Incoming patch to fix a few Mailman bugs
by Lenny Foner July 6, 2000

July 6, 2000

[Please keep me CC'ed on any response you'd like me to see; I will occasionally read the developer & user archives from the website, but they're both too high-traffic for me to be subscribed to, so I'm only on the announce list.] Hi. On 18 June 2000, I sent three messages detailing about 14, uh, "issues" I had with Mailman 2.0beta2 as a brand-new user of it to administer some lists of mine. I've since discovered another serious one (#0 below). Included below that is a patch which fixes #0, #3, #7 (badly---it just removes the text, even if reminder messages -are- being sent!), #8, #9, #10, and #11 (but note that #11 is untested in the umbrella-list configuration). Fixing #0 actually required me to learn Python and a lot of the (confusingly-implemented) design of how Mailman archives messages (totally different control flows for real-time archiving vs rebulding the archive, and none of this is documented in the source code); I did both simply by reading the code on Saturday and Sunday. But be aware, though, that the resulting code may have better solutions for a more experienced Python programmer; I'm really a Common Lisp programmer at heart. :) Bugs #1 and #2 still look reasonably critical to me but are the ones I'm least likely to be fixing, since they may require more familiarity with the source than I'll have time to spend. I'm including all of the original items after the patch, including #0 (which I haven't reported before), so you can figure out what I'm talking about without digging up my old mail. Note also that this patch is for 2.0beta2, not 2.0beta3---beta3 looks a little too unstable for me to use right now, based on the mail I saw on the users list immediately after its release. However, except perhaps for line numbers, it doesn't look like much (or any) of my patches conflict with anything that was done for beta3, so it should be relatively easy to merge this code w/beta3, especially if you're using reasonable merge tools. [If this code will definitely be included in the next release, I might be convinced to do the merge for you, but I'm not going to do so unless I get some feedback that I'm even doing the right thing here to begin with...] - - - Begin patch - - - [This patch was made via "diff -ur" between my modified directory and the virgin 2.0beta2 release, rather than via "patch" or "cvs diff" or anything like that. This means that each file's differences are separated by a line beginning with "diff -ur". I may put Mailman under CVS control at my site if I wind up having to track a lot of divergence between my patches and Mailman, but I'm hoping they'll be folded in instead. Obviously, all the "Foner" comments should be removed; they're there mostly to make it obvious to me, and maybe you, what got changed.] Also, -beware- of indentation! "diff -ur" inserted a leading space in front of -every- line, in theory so that those with +'s and -'s in them would line up with those that had none, but some of the lines were indented with tabs before, hence -those- lines didn't have their alignment changed because SPC TAB is exactly as wide as just TAB. So the 1-character-variances in indentation you see below are spurious; it'll be obvious when merging the patch back into the real sources. diff -ur mailman-2.0beta2.virgin/Mailman/Archiver/Archiver.py mailman-2.0beta2/Mailman/Archiver/Archiver.py --- mailman-2.0beta2.virgin/Mailman/Archiver/Archiver.py Sun Apr 2 20:34:01 2000 +++ mailman-2.0beta2/Mailman/Archiver/Archiver.py Mon Jul 3 02:07:00 2000 @@ -208,6 +208,7 @@ # from this point on, we're doing all the expensive archiving # work. txt = msg.unixfrom + self.unixfrom = msg.unixfrom ### Foner for h in msg.headers: txt = txt + h if not msg.body or msg.body[0] <> '\n': diff -ur mailman-2.0beta2.virgin/Mailman/Archiver/HyperArch.py mailman-2.0beta2/Mailman/Archiver/HyperArch.py --- mailman-2.0beta2.virgin/Mailman/Archiver/HyperArch.py Fri Apr 7 15:06:58 2000 +++ mailman-2.0beta2/Mailman/Archiver/HyperArch.py Mon Jul 3 03:12:53 2000 @@ -62,16 +62,73 @@ def url_quote(s): return urllib.quote(s) - +### +++ Foner. +# [This template is used when generating archives/private/<listname>/<time>.txt +# files, and -not- when generating archives/private/<listname.mbox>/<listname.mbox> +# files!] +# Changes from the original: +# (a) Use unpruned_processed_headers so we can get most of the original +# message headers. This isn't perfect, because the headers have already +# been processed by Mailman on the way in, and hence the Subject: line +# may have [listname] in it, multiple Subject: fields have been eliminated +# (only the last one remains), the anti-looping header fields have been +# added, and so forth. But it's far, far better than what it used to be. +# (b) The original kluged its own From_ line from secondary sources. No, no, no! +# You -cannot- do this---you just can't screw around with the From_ line +# and put it into different formats or many, many mailreaders will suddenly +# fail to recognize it as delimiting messages. Emacs RMAIL, in particular, +# won't figure out that +# From foner(a)media.mit.edu Sat, 1 Jul 2000 18:51:22 -0400 (EDT) +# is the same thing as +# From foner(a)out-of-band.media.mit.edu Sat Jul 1 18:51:24 2000 +# and hence will think that <listname>/<time>.txt files consist of a +# single huge message. (I should also note that not even the UnixMailbox +# class in Python-1.5.2/Lib/mailbox.py thinks that the former is valid: +# _fromlinepattern = r"From \s*[^\s]+\s+\w\w\w\s+\w\w\w\s+\d?\d\s+" \ +# r"\d?\d:\d\d(:\d\d)?(\s+[^\s]+)?\s+\d\d\d\d\s*$" +# pretty clearly rules it out, because it's got everything in the order +# used in the latter case. (A quick way to notice this is that no commas +# appear in that regexp...:)) The <listname.mbox>/<listname.mbox> files +# are fine, since they still have their original From_ lines, but users +# can't retrieve those from the webserver, so they're stuck. And if they +# can't use their mail tools on the files they -can- retrieve, why would +# they bother? So, to fix this, we carefully hand the original unixfrom +# value all the way down from the call chain, in TWO different ways (!) +# because real-time archiving has a completely different flow of control +# than bin/arch uses when regenerating the archives. +# (c) The original caused all messages to end with an extra TWO blank lines. +# I changed this to zero (by putting the """ right after "%(body)s"), but +# unfortunately this runs into a peculiar corner case: when bin/arch is +# run, the very last message in the archive doesn't have a blank line +# after it, which means that the next time a realtime archive is done +# for an incoming message, its From_ line winds up butting up against +# end the of the last message, with no intervening blank line. This +# is probably legal, but I don't want to push it---I'm sure there is +# -some- mailreader out there that will choke. And there's -really- +# no easy way to make the pipermail.py's processUnixMailbox and +# HyperArch.py's add_article (which overrides pipermail.py's) handle +# the situation without doing something really gross, like having +# processUnixMailbox hand add_article a special flag that says, "This +# is the last article I'm giving you---put an extra newline on the end +# of the file." So instead, we put an extra newline on -every- message. +# Also gross (many other programs on the net likewise play it safe, +# adding an extra newline "just in case" and hence causing messages to +# grow trailing padding indefinitely), but oh well. I'm tired. article_text_template="""\ -From %(email)s %(datestr)s -Date: %(datestr)s -From: %(author)s %(email)s -Subject: %(subject)s - +%(unixfrom)s%(unpruned_processed_headers)s %(body)s - """ +### +# article_text_template="""\ +# From %(email)s %(datestr)s +# Date: %(datestr)s +# From: %(author)s %(email)s +# Subject: %(subject)s +# +# %(body)s +# +# """ +### --- article_template='''\ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> @@ -118,8 +175,6 @@ </body></html> ''' - - def CGIescape(arg): s=cgi.escape(str(arg)) s=re.sub('"', '"', s) @@ -202,9 +257,12 @@ def as_text(self): d = self.__dict__.copy() d["body"] = string.join(self.body, "") +### +++ Foner + d["unixfrom"] = self.unixfrom + d["unpruned_processed_headers"] = string.join(self.headers, "") +### --- return self.text_tmpl % d - def __init__(self, message=None, sequence=0, keepHeaders=[]): if message==None: return self.sequence=sequence @@ -273,9 +331,55 @@ string.split(references)) # Save any other interesting headers - self.headers={} - for i in keepHeaders: - if message.has_key(i): self.headers[i]=message[i] +### +++ Foner + # For the text-format file, save them -all-! This way, those + # who need to look at all headers have them available. + # Too bad that these headers are -after- Mailman's munging; thus things + # like Sender: are meaningless, and Mailman's anti-loop headers are + # present, etc. Really, we should preserve the exact original headers + # and use them here; perhaps they're already preserved and I simply + # haven't found them yet. + self.headers=message.headers +### +# self.headers={} +# for i in keepHeaders: +# if message.has_key(i): self.headers[i]=message[i] +### --- + +### +++ Foner + # For the HTML-format file, only save ones that are generally useful, + # so it doesn't get too cluttered. Anyone who wants the rest can + # download the text-format file and peruse them with their favorite + # mailreading tools. The old code didn't bother to put -any- of the + # headers into the HTML. That, combined with the text-mode doing + # likewiae, made it totally impossible for anyone reading the archives + # to have any idea what other recipients (for instance) might have + # gotten the message---and hence might not know who else has been in + # on the discussion and whether the message should be send to them + # -again-. + # + # Note that, unlike self.headers and message.headers, this is a sequence, + # not a dictionary! Note also that you'll only get one Subject:, if the + # message had more than one, and it'll be the last one. This is due to + # the munging Mailman does to insert an optional [listname], which probably + # uses getheader and not getheaders. We use getallmatchingheaders here + # instead of getheaders because (a) the latter mysteriously gives us an + # AttributeError, and (b) it doesn't really matter whether whitespace has + # been removed, especially since this is for a web page. Finally, note + # that order matters in the list of headers below---it determines the + # order in which they wind up on the page. One could obviously add a + # large number of other fields (Resent-<foo>? Organization?) to this + # list; perhaps this should be parameterized per mailing list (uh oh...). + # Too bad that these headers are -after- Mailman's munging; thus things + # like Sender: are meaningless, and Mailman's anti-loop headers are + # present, etc. Really, we should preserve the exact original headers + # and use them here; perhaps they're already preserved and I simply + # haven't found them yet. + self.useful_headers_for_HTML=[] + for i in ['Date','From','To','CC','Subject']: + for j in message.getallmatchingheaders(i): + self.useful_headers_for_HTML.append(j) +### --- # Read the message body self.body=[] @@ -356,9 +460,11 @@ </ul> <hr> - This archive was generated by - <a href="http://starship.skyport.net/crew/amk/maintained/pipermail.html"> - Pipermail %(version)s</a>. +  </BODY> </HTML> ''' @@ -533,7 +639,6 @@ self._unlocklist=unlock self._lock_file=None - # # can't init the database while other # processes are writing to it! @@ -634,8 +739,10 @@ and the empty list is legal.""" if article.subject in ['subscribe', 'unsubscribe']: return None res = self.dateToVolName(string.atof(article.date)) - self.message("figuring article archives\n") - self.message(res + "\n") +### +++ This might be useful to the author, but not to me. --- Foner +# self.message("figuring article archives\n") +# self.message(res + "\n") +### --- return res @@ -784,7 +891,6 @@ # If a string was returned, convert to a list: if type(archives)==type(''): archives=[archives] if archives==[]: return # Ignore the article - # Add the article to each archive in turn article.filename=filename=self.get_filename(article) article_text=article.as_text() @@ -817,6 +923,10 @@ f.close() # Write the text article to the text archive. + # [Note that this is what writes a 2000-July.txt type file, + # and -not- a <listname>.mbox/<listname>.mbox type file! + # This code runs both during real-time archiving -and- + # when bin/arch is run. --- Foner archivetextfile=os.path.join(self.basedir,"%s.txt" % i) f=open(archivetextfile, 'a+') @@ -1000,7 +1110,15 @@ if i<l: source[i]=None ; i=i+1 def format_article(self, article): - source=article.body ; dest=[None]*len(source) +### +++ Foner + # Actually make the message in the HTML archives show some useful + # headers, instead of losing them all and making people guess about + # whether the message went anywhere else (for instance). + source=article.useful_headers_for_HTML + ['\n'] + article.body + dest=[None]*len(source) +### +# source=article.body ; dest=[None]*len(source) +### --- # Handle <HTML> </HTML> directives if self.ALLOWHTML: self.__processbody_HTML(source, dest) @@ -1011,9 +1129,34 @@ s=source[i] if s==None: continue dest[i]=CGIescape(s) ; source[i]=None - if len(dest) > 0: - dest.insert(0, '<PRE>') - dest.append('</pre>') + if len(dest) > 0: +### +++ Foner + # The <PRE></PRE> construction wants to get wrapped around + # the message text -with no leading or trailing whitespace!- + # In particular, not having a trailing newline before the + # </PRE> is -essential- in making the <HR> after it not have + # an unsightly extra gob of vertical space between it and the + # end of the message (at least in NS 3.0x, and probably in + # other browsers, too). While we're at it, make it really + # be </PRE> and not </pre>; this doesn't matter to the browser, + # but it's gratuitously inconsistent with the way all the other + # markup is done in this program. Note that the method below + # is horribly inefficient, but it sure was easy to write; a + # more-efficient approach would scan the text and snip off + # only those lines & characters that are necessary, rather + # than doing a list->string->list transformation. This would + # typically mean that we scan at most a line or two forwards + # from the beginning and backwards from the end, rather than + # copying all of a possibly-large message, twice, and then + # having to add the newlines back onto each line, which no + # doubt means consing yet a -third- set of strings! Oh, well. + dest=string.split(('<PRE>' + string.strip(string.join(dest, "")) + '</PRE>'), '\n') + for i in range(0, len(dest)): + dest[i]=dest[i]+'\n' +### +# dest.insert(0, '<PRE>') +# dest.append('</pre>') +### --- else: # Do fancy formatting here if self.SHOWBR: diff -ur mailman-2.0beta2.virgin/Mailman/Archiver/pipermail.py mailman-2.0beta2/Mailman/Archiver/pipermail.py --- mailman-2.0beta2.virgin/Mailman/Archiver/pipermail.py Sat Aug 21 01:28:32 1999 +++ mailman-2.0beta2/Mailman/Archiver/pipermail.py Mon Jul 3 02:43:14 2000 @@ -16,7 +16,10 @@ VERSION=__version__ CACHESIZE=100 # Number of slots in the cache - +### +++ Foner +from Mailman import Mailbox +import rfc822 +### --- msgid_pat=re.compile(r'(<.*>)') def strip_separators(s): @@ -155,6 +158,7 @@ INDEX_EXT = ".html" # Extension for indexes def __init__(self, basedir=None, reload=1, database=None): # If basedir isn't provided, assume the current directory if basedir==None: self.basedir=os.getcwd() else: @@ -378,11 +382,22 @@ def processUnixMailbox(self, input, articleClass=Article): import mailbox - mbox=mailbox.UnixMailbox(input) +### +++ Foner + # We want Mailbox.Mailbox here, not mailbox.UnixMailbox, + # because the latter uses the broken _search_start method + # that starts just -after- the From_ line and hence can't + # possibly get unixfrom bound to a non-null string. So + # inhert a _searchstart and .next method that works right. + mbox=Mailbox.Mailbox(input) +# mbox=mailbox.UnixMailbox(input) +### --- while (1): m=mbox.next() if not m: break # End of file reached a=articleClass(m, self.sequence) # Create an article object +### +++ Foner + a.unixfrom = m.unixfrom # Must copy this from the message into the article. +### --- self.sequence=self.sequence+1 # Increment the archive's sequence number self.add_article(a) # Add the article diff -ur mailman-2.0beta2.virgin/Mailman/Cgi/admindb.py mailman-2.0beta2/Mailman/Cgi/admindb.py --- mailman-2.0beta2.virgin/Mailman/Cgi/admindb.py Tue Apr 4 19:12:50 2000 +++ mailman-2.0beta2/Mailman/Cgi/admindb.py Sun Jun 18 13:32:19 2000 @@ -222,10 +222,10 @@ t.AddRow([ Bold('If you reject this post, explain (optional):'), TextArea('comment-%d' % id, rows=4, cols=60, - text=("Please do *not* post administrative requests" - " to the mailing list! If you wish to subscribe," - " visit %s or send a 'help' message to the" - " the request address, %s , for instructions" + text=("Please do not post administrative requests" + " to the mailing list. If you wish to subscribe," + " visit %s or send a 'help' message to" + " the request address, %s, for instructions." % (mlist.GetAbsoluteScriptURL('listinfo'), mlist.GetRequestEmail()))) ]) diff -ur mailman-2.0beta2.virgin/Mailman/Handlers/Hold.py mailman-2.0beta2/Mailman/Handlers/Hold.py --- mailman-2.0beta2.virgin/Mailman/Handlers/Hold.py Tue Mar 21 01:25:51 2000 +++ mailman-2.0beta2/Mailman/Handlers/Hold.py Sun Jun 18 13:10:22 2000 @@ -69,7 +69,7 @@ pass class Administrivia(HandlerAPI.MessageHeld): - "Message may contain adbministrivia" + "Message may contain administrivia" pass class SuspiciousHeaders(HandlerAPI.MessageHeld): diff -ur mailman-2.0beta2.virgin/Mailman/Mailbox.py mailman-2.0beta2/Mailman/Mailbox.py --- mailman-2.0beta2.virgin/Mailman/Mailbox.py Tue Mar 21 01:24:59 2000 +++ mailman-2.0beta2/Mailman/Mailbox.py Mon Jul 3 02:36:46 2000 @@ -19,9 +19,19 @@ import mailbox +### +++ Foner +import sys +import rfc822 +import Message +### --- class Mailbox(mailbox.UnixMailbox): # msg should be an rfc822 message or a subclass. + # + # Note that AppendMessage gets called by the realtime archiver to stick a + # message into the <listname>.mbox/<listname>.mbox file, and -not- to stick + # a message into, e.g., 2000-July.txt! --- Foner + # def AppendMessage(self, msg): # seek to the last char of the mailbox self.fp.seek(1,2) @@ -34,3 +44,51 @@ self.fp.write('\n') self.fp.write(msg.body) +### +++ Foner + # Mailman/pythonlib/rfc822.py assumes that a message starts ON the From_ line, + # while Python-1.5.2/Lib/mailbox.py's mailbox.UnixMailbox (via its _search_start + # and _search_end) thinks that it starts AFTER the From_ line. Unfortunately, the + # latter interpretation means that when pipermail.py's processUnixMailbox calls + # mailbox.next (or, rather, our overridden Mailbox.next which runs rfc822.__init__, + # which the mailbox class somehow doesn't---see comments at "def next" below), the + # file position is already AFTER the From_ line, so readheaders in the __init__ + # method can't possibly see it. Thus, we must override the _search_start|end + # methods with ones that think that a message starts at its From_ line and ends + # immediately before the next one. + + def _search_start(self): + while 1: + _saved_from_line_pos = self.fp.tell() ### Foner + line = self.fp.readline() + if not line: + raise EOFError + if line[:5] == 'From ' and self._isrealfromline(line): + self.fp.seek(_saved_from_line_pos) ### Foner + return + + def next(self): + while 1: + self.fp.seek(self.seekp) + try: + self._search_start() + except EOFError: + self.seekp = self.fp.tell() + return None + start = self.fp.tell() + self.fp.readline() # Skip this From_ so we can find the next one. Must work, 'cause we found a From_ w/o EOF.... --- Foner + self._search_end() + self.seekp = stop = self.fp.tell() + if start <> stop: + break +### +++ Foner + # We must instantiate Message.Message here, not rfc822.Message! + # For some reason I can't figure out, the latter never runs its + # __init__ method, which means it never does readheaders, which + # means it never sets unixfrom! (Even if the filepos is correct, + # failing to run the init means we can't ever win.) I don't know + # why it doesn't init, but Message.Message explicitly runs + # rfc822.Message.__init__, so we win. + return Message.Message(mailbox._Subfile(self.fp, start, stop)) +### +# return rfc822.Message(mailbox._Subfile(self.fp, start, stop)) +### --- diff -ur mailman-2.0beta2.virgin/Mailman/Utils.py mailman-2.0beta2/Mailman/Utils.py --- mailman-2.0beta2.virgin/Mailman/Utils.py Sun Apr 2 20:30:22 2000 +++ mailman-2.0beta2/Mailman/Utils.py Sat Jun 17 01:52:47 2000 @@ -55,7 +55,7 @@ # a much more naive implementation than say, Emacs's fill-paragraph! -def wrap(text, column=70): +def wrap(text, column=78): """Wrap and fill the text to the specified column. Wrapping is always in effect, although if it is not possible to wrap a diff -ur mailman-2.0beta2.virgin/templates/convert.txt mailman-2.0beta2/templates/convert.txt --- mailman-2.0beta2.virgin/templates/convert.txt Fri Aug 14 12:17:26 1998 +++ mailman-2.0beta2/templates/convert.txt Sun Jun 18 12:47:20 2000 @@ -9,9 +9,8 @@ 2) You have been given an arbitrary password to prevent others from unsubscribing you without your knowledge. It will be mailed to you in -a separate email, which you may have already received. Don't worry if -you forget this password; a reminder will be sent to you via email -every month. +a separate email, which you may have already received. You can ask +the server to send it to you again if you forget it. 3) If you have World Wide Web access, you can use it any time to unsubscribe from this list, to switch to and from digest mode, to diff -ur mailman-2.0beta2.virgin/templates/subscribeack.txt mailman-2.0beta2/templates/subscribeack.txt --- mailman-2.0beta2.virgin/templates/subscribeack.txt Wed Dec 16 19:14:00 1998 +++ mailman-2.0beta2/templates/subscribeack.txt Sun Jun 18 23:36:59 2000 @@ -1,4 +1,4 @@ -Welcome to the %(real_name)s@%(host_name)s mailing list! +Welcome to the %(real_name)s@%(host_name)s mailing list. %(welcome)s To post to this list, send your email to: @@ -12,8 +12,7 @@ from digest mode, change your password, etc.), visit your subscription page at: - %(optionsurl)s -%(umbrella)s + %(optionsurl)s %(umbrella)s You can also make such adjustments via email by sending a message to: %(real_name)s-request@%(host_name)s @@ -26,11 +25,5 @@ %(password)s -If you forget your password, don't worry, you will receive a monthly -reminder telling you what all your %(host_name)s mailing list passwords -are, and how to unsubscribe or change your options. There is also a -button on your options page that will email your current password to -you. - -You may also have your password mailed to you automatically off of the -Web page noted above. +If you forget your password, you can ask the server to send it to you +again. - - - End patch - - - New #0: (0) Mailman doesn't keep enough original headers around in its message archives to make it apparent to anyone where the message went, because it doesn't keep -any- of them, really. So if the message went to secondary addresses, there's no way for the reader of the archives to know, yet it's often critical to know where -else- a message went, etc . Also, its flat-text archive version tries to roll its own version of the From_ (unix envelope) line, and gets it wrong, to the point where neither the Python libraries nor other mail-handling programs (e.g., Emacs Rmail) even recognize them as From_ lines in the first place--- hence they treat the entire archive as one big message, which entirely defeats the purpose of making a flat-text version of the archive available, because it breaks users' ability to read them except by paging through them as a single big file. I've fixed the former by keeping -all- headers in the flat-text file (since users downloading that will presumably use their normal mailreading tools on them, which will no doubt prune some), and by keeping some of the most useful headers in the HTML archives (few enough to not clutter up the page too much, but enough to figure out where else the message went). I've fixed the latter by carefully preserving the From_ line by taking pains to pass it down the call stack (in two different ways, because Mailman has two entirely different archiving strategies). Fixing these was remarkably difficult, because (a) real-time vs after-the- fact archive rebuilding use -completely- different call paths and very different code, it's not documented anywhere which methods run in each case, and they're a tangled mess of Python libraries (in the distribution), other libraries in pythonlib, and then patches to -those- methods elsewhere in Mailman to override bugs in them---can this maybe be cleaned up and all put in a single file, please?--and (b) Mailman/pythonlib/rfc822.py assumes that a message starts ON the From_ line, while Python-1.5.2/Lib/ mailbox.py's mailbox.UnixMailbox (via its_search_start) assumes it starts AFTER the From_ line, so readheaders in the __init__ method can't possibly see it (and besides, rfc822.Message never seems to -run- its __init__ method, which doesn't help much). Finally, the HTML version of the messages isn't formatted very well, because of carelessness with the stuff inside <PRE></PRE>. Original items (from 3 messages), concatenated into one: Date: Sun, 18 Jun 2000 15:01:38 -0400 (EDT) From: Lenny Foner <foner(a)media.mit.edu> To: mailman-developers(a)python.org Subject: Various mailman bugs (all in 2.0beta2 under HPUX 10.20) I've just started using Mailman, and, while it's quite a nice little package, I've already tripped over a number of bugs. I've listed them approximately in order from most severe to least severe below; I've fixed 3 & 7-11 myself, but 1 & 2 require someone with at least a passing familiarity with Python and Mailman's internals, and of course they should all wind up in the current source tree. I also spent a while poking at the Jitterbug interface, and concluded that it wasn't likely that I'd be able to figure out which category each individual bug should go into, or that it would save anybody any time for me to try to stick them in the interface in the first place. It's also incredibly painful to submit bugs by filling in web forms, because the editing environment available is so poor. Having the forms used by Jitterbug also allow file uploading (when supported by the browser, which most do) would help, but aren't a complete solution. (1) Mailman doesn't send error reports to the right place. Our setup here has individual workstations, such as foo.media.mit.edu, masquerade such that their mail headers are rewritten to appear to come from media.mit.edu (which forwards mail to our main mail-handling machine, and also has an MX pointing there). This is usually how large sets of machines are set up everywhere. But when I was trying to configure Mailman (with the wrong compiled-in GID [it wasn't clear to me whether it was the webserver's GID or sendmail's GID that was the right one], hence any request caused the wrapper to err), Mailman evidently went out of its way to attempt to send the error mail to the individual workstation, e.g., foo.media.mit.edu, and -not- to media.mit.edu, which is what the From: address in the incoming mail to Mailman specified and which should be respected in any event---From: should -always- take precedence over some attempt to divine a better address. After all, Mailman cannot possibly have a better idea of which machine is handling mail for the sender than the sender itself does. (This meant, btw, that the error mail simply got wedged in the sendmail queue of the machine I'm running Mailman on, since individual workstations here don't listen to SMTP requests.) (2) Sending a message with no subject and no body to listname-request causes the blank message to simply arrive at the admin address. I contend that it would be far nicer to the users -and- the administrator if a blank message acted like one with "help" in it. Otherwise, users see a black hole (until the admin responds), and admins need to deal with responding in a way that gets users to ask "help" anyway, most likely. It's just a pain all around. (3) mailman-2.0beta2/Mailman/Archiver/HyperArch.py puts the URL http://starship.skyport.net/crew/amk/maintained/pipermail.html into the footer of its page describing the message archives, but this is a dead link. (I just commented out the footer entirely.) (4) Feature request: Mailman should support a --with-default-url as a build argument so it can use cnames and/or virtual hosts. Ditto for the FQDN of the host. I had to go in and screw around with various options files to get this set up correctly after Mailman guessed; why make the user edit this instead of taking it as an arg? (Yes, I know you can change it one list at a time in the admin web interface, but the default should be right, too.) (5) The doc says to run "make install" in at least 2 places, but never says to actually run "make" anywhere. But you -do- have to do a make :) (6) Should I be creating mailman-owner or owner-mailman as aliases? The doc is inconsistent, so I did both. Is this specific to the MTA? If so, you should figure it out and document it for common ones. (I'm using sendmail 8.9.3.) (7) Even if the list administrator has disabled monthly password mailings, templates/convert.txt and templates/subscribeack.txt both claim that users' passwords will be mailed to them monthly (in fact, the latter seems to say the same thing twice, in both the last and penultimate paragraphs). There should be some way for these templates to check the current setting for the list, and to fail to include this text if monthly reminders are turned off. Mailman/HTMLFormatter.py appears to make this check correctly, though I'm not positive that it's generating the page I think it is. [I find monthly password reminders absolutely pernicious---after all, if I forget the password, I can -ask- for it and get an instant response from the server anyway. And this way, I don't have the extra traffic, my password doesn't get sent more than it has to over insecure channels, and very low-volume lists don't send me mostly password-reminder traffic instead of blessed silence.] (8) Three's some bad formatting in the help page returned by sending a "help" request to listname-request, apparently caused by linewrapping performed by Utils.wrap (in Mailman/Utils.py)---Utils.wrap defaults to 70 columns wide, but the template text in template/help.txt has lines longer than this (as do a couple of other calls from elsewhere in the source code which call Utils.wrap on a fixed string; these are less objectionable, in general, because the constant text isn't also indented). The end result looks pretty awful: end or -- Stop processing commands (good to do if your mailer automatically adds a signature file - it'll save you from a lot of cruft). should be end or -- Stop processing commands (good to do if your mailer automatically adds a signature file - it'll save you from a lot of cruft). and unsubscribe from a different address than the one you subscribed from, you may specify it in the 'address' field. should be unsubscribe from a different address than the one you subscribed from, you may specify it in the 'address' field. I've fixed this by changing the default in Utils.wrap to 78 characters, which is half-baked but works. (9) Mailman/Handlers/Hold.py spells administrivia as adbministrivia instead (there shouldn't be a "b" in that word...) (10) Mailman/Cgi/admindb.py's default textbox message that says "Please do *not* post administrative requests to the list!" has a repeated "the" in the next sentence, so it talks about "the the request address". (The two "the"'s are separated by a newline in the source, which makes this harder to see.) I removed the repeat and also changed the first sentence to "Please do not post administrative requests to the list." (no *, no !) so it seems less like the admin is yelling at the poor user. (11) templates/subscribeack.txt puts an extra newline before "You" in the normal (non-umbrella) case in: %(optionsurl)s %(umbrella)s You can also make such adjustments via email by sending a message to: which I've fixed simply by doing %(optionsurl)s %(umbrella)s You can also make such adjustments via email by sending a message to: but this also requires putting the right number of newlines into the string in Deliverer.py which generates the string in umbrella. Since I can't easily test it, I didn't bother fixing that. (12) It would be less confusing if the "details" page for various configuration stuff didn't say "don't change it here---change it on the previous page" or whatever. Either make the form active on the details page, or don't make it a form at all. This wasn't a problem for me, but I imagine it will be for others. (Yes, I understand it's easier to be able to reuse the code that generates the form this way, but surely it can't be that hard to modularize.) (13) Things that get bounced to the admin address because they look like administrivia (e.g., sending "help" to the main list address) have -two- X-BeenThere: message headers. Since this header is presumably there to stop loops, it strikes me as odd that two of them could be in a message under any circumstances. (Maybe they're only checked for messages intended to go all the way out through the main list---the two of them -do- seem harmless so far---but it's still odd.) (14) When setting archival options, the form specifies the time period as yearly/monthly/quarterly/weekly/daily. This is an odd order. How about yearly/quarterly/monthly/weekly/daily, which at least leaves them in size order?

1 0

Getting Latest Mailman CVS Snapshot
by John MacKenzie July 6, 2000

July 6, 2000

HEy guys, Where can i get the latest CVS Snapshot.... Since upgrading to B3 , The lists aren't working... approved messages aren't sent etc, and unless I get these lists out fast someones gonna kill me. Also couldn't downgrade to 1.1 for some reason ? Hope you can help - John -- John MacKenzie | Unix Systems Admin | e: mailto:jmackenzie@local.ie ___________________________________________________________________ Stay in touch your local area through Local Ireland http://chat.local.ie/chat/index.html ___________________________________________________________________ local ireland | dublin | new york | http://www.local.ie t: +353 1 676 8996 f: +353 1 283 9988

5 8

WebArchives with replies from the archives under MHonArc
by J C Lawrence July 4, 2000

July 4, 2000

A while back I referred to a project I was working on of getting MHonArc and PHP4/PHPLib to produce web archives for mailing lists that support replying to the archived messages with proper quoting, In-Reply-To/References headers and all the rest, right there from the web page. v1.0 is now up and can be seen at the pages under: http://www.kanga.nu/archives/ Note that some list archives support replying, and some don't (simple per-list configuration). eg: http://www.kanga.nu/archives/IRead-L/2000Q2/msg00001.php (replyable) http://www.kanga.nu/archives/ICGnu-L/1998Q3/msg00783.php (no reply) Software requirements: PHP4 (not PHP3, it needs the <<<herefile supports from PHP4) PHPLib (uses PHPLib templates (abuses them more)) MHonArc (of course) This is not finished but does in general work. Some of the data abstractions need cleaning such as the use of variables.rc vs thislist.inc (pleasantly thislist.inc overrides variables.rc) and the PHP-based SMTP engine should be extracted into a standalone function that can talk to any named SMTP server, and there are many other crudities, but all the basics are there ATM. You can find the relevant sources under: ftp://ftp.kanga.nu/pub/Kanga.Nu/WebArchives/ An advantage of this approach in general is that the archive pages are now templatised and you can change the entire presentation of your archive pages as your site changes or even dynamically tailoring them to the browser's login or how he got to that page, without having to regen your archives. If anyone decides to use or build something off this setup, I'd appreciate an email. Aside: It would take little work to go from here to having all the message data in a DB. With the current setup MHonArc produces PHP files which consist only of variable assignments for the various extracted message sections. As PHP files can be setup to execute from the command line, it would be fairly simple to post-process those files to insert the variable values into a DB and to then have the new equivalent of msg_page.inc do the appropriate SQL queries based on the $msg_num as extracted from the URL. -- "Show me the way to go $HOME" Home: claw(a)kanga.nu J C Lawrence Other: coder(a)kanga.nu ----------(*) Keys etc: finger claw(a)kanga.nu --=| A man is as sane as he is dangerous to his environment |=--

1 0

almost found the b3 bug?
by Ricardo Kustner July 2, 2000

July 2, 2000

Hi, i posted earlier that i thought i fixed my b3 approval trouble by completely reinstalling mailman and all the list... unfortunately the approvals still don't work correctly... after doing some testing with qrunner I think I've tracked down part of the problem: i wrote a post to the list... it arrived on the approval page and i did a 'approve'... when I run qrunner by hand with some added syslogs, i noticed it ends up thinking the message is an '-request' post... so the only thing it does with the message is mlist.ParseMailCommands(msg)... I'm going to see if I can track it down more precisely... Ricardo. --

3 6

Re: ML replies (was: CVS: python/dist/src/PCbuild readme.txt,1.5,1.6)
by bwarsaw＠beopen.com July 2, 2000

July 2, 2000

[Please remove python-dev from followups, I'm sure most of those folks don't care about this... -BAW] MM-Devers: on python-dev there's been some discussion about the Reply-to: munging that you can select on Mailman lists. Say you've got a python-checkins list that receives CVS log messages automatically. But you don't want discussions about the changes on python-checkins, you want them on python-dev. How can Mailman set up the headers on outgoing messages to get this behavior? It can't. So it fakes it by setting Reply-to: to the address of the discussion list. We now join our conversation already in progress... >>>>> "GS" == Greg Stein <gstein(a)lyra.org> writes: GS> What about the Mail-Followup-To header? That can be helpful GS> for some newsreaders (e.g. Mutt). Dunno how many observe it. It looks like VM/Emacs doesn't (which is what I use), but I'm sure that'd be easy to add. I would be most interested to know what Outlook and NS do with it. However Mail-Followup-To: isn't in any RFC that I can find, not even 2076. It can be found in a discussion by Dan Bernstein[1], but that document specifically says that list managers should /not/ set it because this defeats the intended purpose. I agree - given Dan's recommendation. So there is clearly no header that will support the functionality we want. And all this crud has been debated ad-nauseum in several working groups, MUA forums, etc. with no agreement. I almost hesitate even bringing it up here. Few of the discussions that I've read (in a morning's worth of archive surfing) even mention the situation we're trying to address. One potential solution for Mailman might be to lie about the To: address. So in the above scenario, the message would have a To: field set to python-dev even though the message would actually be sent to the python-checkins membership. An elaboration on that would be to set Resent-To: to python-checkins. But those are all hacks, nearly as disgusting as what Mailman currently does. -Barry [1] http://cr.yp.to/proto/replyto.html

4 4

Re: [Mailman-Developers] Re: ML replies
by Greg Stein July 2, 2000

July 2, 2000

Um... problem reset... My real point wasn't to have *Mailman* set any of these headers, but the script that generates the message(!). Mailman can comfortably ignore any Reply-To or Mail-Followup-To message. Just operate it like a normal list. The script that gathers up the CVS data and ships it off can insert the two headers into the outbound messages. Cheers, -g p.s. yes, maybe Mailman can also set those headers for normal operation to capture the bozos that send mail to the list; but that is a separate question and (obviously from the responses) not a straight-forward one. On Sat, Jul 01, 2000 at 09:34:19AM -0700, Chuq Von Rospach wrote: > At 12:16 PM -0400 7/1/00, Barry A. Warsaw wrote: > > > GS> What about the Mail-Followup-To header? That can be helpful > > GS> for some newsreaders (e.g. Mutt). Dunno how many observe it. > > > >It looks like VM/Emacs doesn't (which is what I use), but I'm sure > >that'd be easy to add. I would be most interested to know what > >Outlook and NS do with it. > > > >However Mail-Followup-To: isn't in any RFC that I can find, not even > >2076. > > It's non-standard, and in fact, arguably non-compliant, since any > unofficial header should be in the X-headername form. > > >So there is clearly no header that will support the functionality we > >want. > > This is a case where Reply-To is the correct header to set, or at > least, least incorrect. > > >One potential solution for Mailman might be to lie about the To: > >address. So in the above scenario, the message would have a To: field > >set to python-dev even though the message would actually be sent to > >the python-checkins membership. > > um, uh, well... I don't like it. Even if you had Sender: and List-ID > set properly, it still seems wrong. > > For this case, I think the best setup is Reply-to, because that's > waht you want: it came FROM this place, but responses go to this > other place. The proper answer is setting reply-to, not attempting to > rearrange the concept of "from this place". > > As a strong proponent of "don't use reply-to!" -- this is a case > where it's the proper answer. It's not perfect by any means, since > reply-to coerces someone away from replying privately, but in this > situation, that's better than not doing it. > > -- > Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) > Apple Mail List Gnome (mailto:chuq@apple.com) > > And they sit at the bar and put bread in my jar > and say 'Man, what are you doing here?'" -- Greg Stein, http://www.lyra.org/

1 0

b3 list working again
by Ricardo Kustner July 1, 2000

July 1, 2000

After doing a fresh re-install of mailman, the approval of posts seems to be working correctly again. I still don't know what went wrong in my upgrade progress that made it impossible to work... I'll try to find out what went wrong to see if other people can be warned about this... oh well, maybe it was time to restart with a clean config anyway... it's almost 3am... i need to get some sleep soon :) Ricardo. --

1 0