Mailman 3 February 2000 - Mailman-Developers

Development reference manual?
by Gerrit Holl Feb. 7, 2000

Feb. 7, 2000

Hello, I wonder if there is any reference manual to all of Mailmans classes, functions and variables. Like www.python.org/doc/current/lib. I'd like such a feature, since I'm interested in developing Mailman. I've almost a year experience in Python, but I get confused my the Mailman core package. I've read the available documentaion but I get confused by some of the code. regards, Gerrit. P.S. Is the majority of people in here also Dutch, like on c.l.py? -- homepage: http://www.nl.linux.org/~gerrit -----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com Version: 3.12 GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O !M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y -----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth

1 0

Quoted printable umlaut quick hack
by pf＠artcom-gmbh.de Feb. 7, 2000

Feb. 7, 2000

I'm running mailman 1.1 on our intranet web server. Since our lists are in German language, umlauts occure quite often in mail bodies and subjects: äöü ÄÖÜ ß. Depending on the used MUA these are often encoded as quoted printables and end up as ugly trash in the pipermail HTML archives: F=FCr instead of Für ... Unfortunately this dooes not only look ugly but also prevents HTdig (the search engine) to find any words with umlauts in the pipermail archives. So today I did some quick hacks to HyperArch.py which at least work for me. (Spend too much time on this detail already) My dirty patch against HyperArch.py is appended below. May be this will help somebody to do this right. Regards, Peter -- Peter Funk, Oldenburger Str.86, 27777 Ganderkesee, Tel: 04222 9502 70, Fax: -60 ---- 8< ---- 8< ---- cut here ---- 8< ---- schnipp ---- 8< ---- schnapp ---- --- /home/pf/freeware/mailman-1.1/Mailman/Archiver/HyperArch.py Sat Aug 21 07:13:23 1999 +++ HyperArch.py Mon Feb 7 17:36:07 2000 @@ -34,6 +34,7 @@ import sys import re, cgi, urllib, string import time, pickle, os, posixfile +import quopri, cStringIO import HyperDatabase import pipermail from Mailman import mm_cfg @@ -49,6 +50,7 @@ except ImportError: pass +umlaut_in_subj = re.compile(r"=\?iso-8859-1\?Q\?(.*)\?=", re.IGNORECASE) def html_quote(s): repls = ( ('&', '&'), @@ -228,6 +230,16 @@ self.subject=self.subject[i:] else: i=-1 if self.subject=="": self.subject='No subject' + # --Pefus quick Hack: + umls = umlaut_in_subj.search(self.subject) + if umls: + umlconvin = cStringIO.StringIO( + umlaut_in_subj.sub(r"\1", self.subject)) + umlconvout = cStringIO.StringIO() + quopri.decode(umlconvin, umlconvout) + umlconvout.seek(0) + self.subject = string.join(umlconvout.read(), "") + # -- if message.has_key('Date'): self.datestr=str(message['Date']) @@ -239,7 +251,12 @@ date, tzoffset=date[:9], date[-1] if not tzoffset: tzoffset = 0 - date=time.mktime(date)-tzoffset + try: + date=time.mktime(date)-tzoffset + except ValueError: + print "illegal date discovered:", date + date=time.time() + else: date=self.__last_article_time+1 @@ -276,8 +293,13 @@ # Read the message body self.body=[] message.rewindbody() + converterInput=cStringIO.StringIO( + string.join(message.fp.readlines(), "")) + converterOutput=cStringIO.StringIO() + quopri.decode(converterInput, converterOutput) + converterOutput.seek(0) while (1): - line=message.fp.readline() + line=converterOutput.readline() if line=="": break self.body.append(line)

1 0

Re: [Mailman-Developers] Re: [Mailman-Users] Can't change with admin pages
by Harald Meland Feb. 7, 2000

Feb. 7, 2000

[Sean Reifschneider] > On Mon, Oct 11, 1999 at 08:59:17PM +0200, Per Starback wrote: > > path = urlparse(self.web_page_url)[2] # '/mailman' > > Is that comment supposed to mean something? Only that the typical value of `path' after that line would be "/mailman", but might be different if the list has a "exotic" (which does not imply "invalid") `web_page_url'. > Should the above code be changed to something more like: > > path = urlparse(self.web_page_url)[2] # '/mailman' > if not path: path = self.web_page_url Hmm, I think it'd probably be better to fix problems with invalid `web_page_url's in MailList.CheckValues(), which is called every time the list is loaded (and which already contains code to ensure that `web_page_url' ends in a "/"). -- Harald

1 0

Re: [Mailman-Developers] Re: Mailman and your code for online posting
by Harald Meland Feb. 7, 2000

Feb. 7, 2000

[Steven Hazel] > I hope you don't mind if I cc my reply to the Mailman developers list. > > Is the code that you submitted to the developers list going to be > incorporated into a subsequent release of Mailman? > > I hope so. It got no response, so I'm not quite sure what's up. Just to let you have a response (finally :): I've marked the messages you sent as "look at these when there's time". As you can infer from the time between your posting them and this message, my time for Mailman has been rather sparse for some time :(, but I'm hoping that will improve now. -- Harald

1 0

Re: [Mailman-Developers] feature requests
by Harald Meland Feb. 7, 2000

Feb. 7, 2000

Yes, the message this is a reply to is pretty old by now -- I'm quite some messages behind on -developers, not to mention -users :( I'll try to get up to date (and stay there :) now... [Christopher Kolar] > While beating on the documentation a few things came to mind. > > 1. Any chance that the monthly reminder can be used by a bounce > handler script as a membership probe (good for low-volume lists). > Right now I just get N bounce messages as mailman-owner but I do not > see that the list manager ever gets the information. With the currently released Mailman, where only the list objects themselves know which addresses are members, trying to implement something like that would probably scale pretty badly. However, here at uio.no, we've now been running a Mailman with a user database -- mainly for avoiding the pains of having separate passwords for separate mailing lists -- and I guess one could pull of something like what you suggest with the help of the user db. I'll try getting around to post some details about what (rather substantial) changes I've made to our Mailman within the next few days -- but for now, to -developers only, I guess. > 2. How about adding an "invite" feature like on some of the > commercial list providers. The invite would be non-binding, but a > simple reply would add the person to the list (I suppose that it > would use most of the same code that the sub-confirmation bits use). > See listbot and egroups if you do not know what I am talking about. This would just be a slightly differently worded "subscription confirmation" message and something to avoid "open subscribes" turning into "open invites", I guess. > 3. How about a chunk of html code that can be dropped into a web > page that would allow a person to just type in their e-mail and > click to make a sub request (also something that listbot/egroups > do). Mailman does, to my knowledge, not do any kind of Referer checking or somesuch, so the only problem is the (current) need to enter a subscription password when posting a sub request. This _could_ be hacked around in various (ugly) ways by generating this form (dynamically, to avoid everyone having the same password) with HIDDEN fields for passwords. > The request could be handled like a normal sub request with a > confirmation request being sent out to the e-mail address that was > entered. Is this already possible by hacking the listinfo page to > remove everything but the field for e-mail address and the submit > button? I deal with a lot of novice users and I would like to take > out a lot of the verbiage and the initial password setting. How do others feel about this? Should Mailman allow users subscribing without explicitly setting their member password (and instead have Mailman set a random password for them)? -- Harald

1 0

integrating changes into distribution
by Bill Bumgarner Feb. 6, 2000

Feb. 6, 2000

For the Mailman-decode-mime-file-via-webdav modifications, I have a relatively significant set of addtions and patches to Mailman. I would like to intregrate these into the core distribution. What is the best way to procede? Barry? b.bum

3 2

locking.
by Thomas Wouters Feb. 6, 2000

Feb. 6, 2000

I've found some small buglets in my locking patch, mostly causing leftover files and missing logging... if anyone is integrating my patches, drop me a line so I can send an updated version ;) -- Thomas Wouters <thomas(a)xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

1 0

race condition in locking ?
by Thomas Wouters Feb. 5, 2000

Feb. 5, 2000

I've finally found the time to do some serious testing of mailman (that is, set up a mailman list and play with it ;) and I was hit by what looks like a race condition in the locking. I was sending a lot of small mails to the mailman-test list, using echo "foobar foobar baz baz baz" | mail -s "foobar" mailman-test(a)xs4all.nl and just manually hitting arrow-up, enter, arrow-up, enter, etc. I'd subscribed a bouncing email address and wanted to see what would happen ;) In any case, the machine mailman runs on is also used for some other things; majordomo and our 'batched-smtp' product. Both services generate very variable loads, majordomo because of some very large lists, and batched-smtp due to customers with very large mailqueues dialing in and starting a queuerun. I dont send the mail from the same machine, so the mails I sent probably arrived in one sendmail session, and then were executed by the queueing sendmail right after another. Some of the messages (around 10/15 per incident) I'd sent then bounced, most with the following traceback: Traceback (innermost last): File "/usr/local/mailman/scripts/mailowner", line 69, in ? main() File "/usr/local/mailman/scripts/mailowner", line 48, in main mlist = MailList.MailList(sys.argv[1]) File "/usr/local/mailman/Mailman/MailList.py", line 69, in __init__ self.Load() File "/usr/local/mailman/Mailman/MailList.py", line 824, in Load self.Lock() File "/usr/local/mailman/Mailman/MailList.py", line 1255, in Lock self.__lock.lock() File "/usr/local/mailman/Mailman/LockFile.py", line 209, in lock os.link(self.__lockfile, self.__tmpfname) OSError: [Errno 2] No such file or directory And a couple with the following traceback: Traceback (innermost last): File "/usr/local/mailman/scripts/post", line 86, in ? main() File "/usr/local/mailman/scripts/post", line 62, in main mlist.Post(msg) File "/usr/local/mailman/Mailman/MailList.py", line 1249, in Post self.Save() File "/usr/local/mailman/Mailman/MailList.py", line 814, in Save Utils.reraise() File "/usr/local/mailman/Mailman/MailList.py", line 811, in Save os.link(fname, fname_last) OSError: [Errno 17] File exists The first traceback is probably because one process does a kickstart, erasing the lockfile and starting a new one, while a lot of other processes are trying to link() to it. The second traceback looks like a lock was stolen from an active process, in other words a real locking failure. The lock-logfile suggests some inefficient locking, too; long pauses between unlocks and locks: (18502) mailman-test.lock waiting for claim (18492) mailman-test.lock unlocked (18513) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim Feb 01 18:53:08 2000 (18622) mailman-test.lock kickstarted (18425) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18467) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim (18443) mailman-test.lock waiting for claim (18500) mailman-test.lock waiting for claim (18502) mailman-test.lock waiting for claim (18513) mailman-test.lock waiting for claim (18526) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18425) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18443) mailman-test.lock waiting for claim (18513) mailman-test.lock waiting for claim (18467) mailman-test.lock waiting for claim (18584) mailman-test.lock waiting for claim (18500) mailman-test.lock waiting for claim (18502) mailman-test.lock waiting for claim (18526) mailman-test.lock waiting for claim (18440) mailman-test.lock waiting for claim (18504) mailman-test.lock waiting for claim (18425) mailman-test.lock waiting for claim (18456) mailman-test.lock waiting for claim (18622) mailman-test.lock waiting for claim (18443) mailman-test.lock got the lock (This is with the default sleep time of .25) This all led me to believe there was something wrong with locking, and I spent the mornings' traintrip from home to work (1.5 hours) reading LockFile.py and thinking about the locking. And I think the logic of LockFile.lock() is flawed :P lock() seems to rely on two things: os.stat() returning the right number of links for the lockfile (which is a fairly safe assumption) and the contents of the lockfile, which should give an indication as to who locked it and for how long. But there is at least one situation possible where one process thinks it has the lock, and another thinks the lock is faulty and does a forced kickstart: process 1 process 2 os.link(lockfile,tmpfile) os.stat(lockfile) == 2 os.link(lockfile, tmpfile) os.stat(lockfile) == 3 self.__write(): open(tmpfile, "w") self.__read() write(os.getpid(), etc) __kickstart(force=1) return I think I was able to reproduce this on my laptop by stressing it fairly hard (load of 15 and rising.) The testcode only acquired and released locks, with some small sleep inbetween, but I saw several double-locks, and one OSError on the os.link(). I also think I see a way to fix this problem, by checking the mtime of the file if the __read() fails. I think we can safely say that if the __read() fails AND the mtime is recent, someone else is either assuming they have the lock, or is kickstart()ing, in both of which cases we should step back and start over. Comments ? Am I trespassing on anyones' work here ? Is this part of mailman still being looked at ? -- Thomas Wouters <thomas(a)xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

3 8

SourceForge
by Barry A. Warsaw Feb. 2, 2000

Feb. 2, 2000

What would you folks think about moving the CVS repository for Mailman over to SourceForge.net? They even /use/ Mailman over there. Our CVS repository is running on a pokey Sparc5 and getting others write access to the tree is a pain (not that anybody's clamoring for that). I like the idea of SourceForge and we had a bunch of people raving about it at the Python conference last week. Anybody have any negative experiences with it? -Barry

2 1

Re: [Mailman-Developers] Bugs in weekly archiving - mailman 1.1?
by Nigel Metheringham Feb. 2, 2000

Feb. 2, 2000

[copied into mailman-users since the followup question is more appropriate there... please keep discussion onto users from now on] I've built a patch to fix the weekly archive bug I see in mailman 1.1 - this fixes the problem of abusing strftime and hoping it works (it does mostly but I have exceptions). Anyone know a neat way of rearchiving the misarchived messages (I can do this by manually pulling them out the mbox archive) and losing the broken week archives - ie I currently have a Week-of-Mon-20000200 which is no use to man nor beast, but needs more than just a subtle delete to get rid of... Nigel. [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham(a)VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ]

1 0