Mailman 3 February 2001 - Mailman-Developers

Mailman + PostgreSQL
by Chris Ryan 21 Feb '01

21 Feb '01

Hello all, I've been using Mailman on our web site for open source development of "serious business software." http://www.greatbridge.org/. We are interested in seeing Mailman have a PostgreSQL database back end (and further in the future PHP front end web tools.) I have been given permission to start working on this as soon as I have completed my current project (should be done this week). I've started to familiarize myself with the general flow of Mailman. I have not had a chance to get into the current data handling classes but, correct me if i'm wrong, I believe modifying the code to use a database should be fairly contained. In the process of doing this I want to do everything in a such a manner that is best for contributing back to the Mailman project. Any feedback anyone can offer on this subject would be wonderful. Thanks in advance. Chris Ryan chris(a)greatbridge.com

4 3

[Fwd: URGENT: Something in message is breaking mailman!]
by Kambiz Aghaiepour 21 Feb '01

21 Feb '01

I sent this to mailman-users and now I'm sending this to you to see if anyone has any insight into this. Actually, I've been able to track this down to a problem in the python library mimetools.py. The message that causes problems has a header as such: Content-Type: multipart/report; report-type=delivery-status boundary="-------=_2EF51FF1.3FFF45E0" And if you notice, the report-type value is not terminated with a ';' character. As a result, the value gets set to: "delivery-status boundary=...." So when ~mailman/Mailman/Bouncers/Netscape.py (or whatever) goes to run (in the process routine): boundary = msg.getparam('boundary') it gets back None which gets pushed onto the stack. Later, when str is popped off the stack python fails because it can't do a string operation on None. If you look at the mimetools.py in the python library, and look at the parsetype method of the Message class, you'll see that the problem is in the assumption that each of the variable declarations in the Content-Type: header of the mime encoded message are always terminated by a semicolon. Just as a test, I hand modified the message in the qfiles directory and change the above Content-Type: header to read: Content-Type: multipart/report; report-type=delivery-status; boundary="-------=_2EF51FF1.3FFF45E0" and the message was processed. So, what is the fix? Occasionally, a message will come back and end up in the qfiles directory that has the header as described above. As a result, qrunner fails (errors out) and NONE of the email on the system is processed. This is a critical problem. Could you please help out with this? Thanks Kambiz ps. Please reply directly to me, as I am not on this list. Thanks -- \o__O o -=< Kambiz Aghaiepour - (919)593-1964 >=- o o \_ /|\ -=< Senior Consultant - The Collective >=- //\ //\ |\ |\ -=< mailto:kambiz@collectivetech.com >=- // // / / |/ -=< http://www.collectivetech.com/ >=- |\ ||

2 3

Direct injection into hold queue
by J C Lawrence 21 Feb '01

21 Feb '01

I've come up with a slightly non-standard need: I need to able to side-line messages in the hold queue (basically take them out of the queue without approving/rejecting/discarding them), and to later put them back into the hold queue so I can then do whatever to them. Is there an elegant way to do this (I didn't see anything obvious)? -- J C Lawrence claw(a)kanga.nu ---------(*) http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: [Mailman-Developers] print >> sys.stderr does not compile
by Ralph Corderoy 21 Feb '01

21 Feb '01

Hi Fil, > You recently changed all calls like > write('(locked)', file=sys.stderr) > to > print >> sys.stderr, _('(locked)') > > but the latter will not compile on my machine. Am I missing something? I'd guess that your version of Python isn't recent enough to have the new `>>' for I/O redirection. Sorry, but I don't know what version you'd require. Ralph.

6 18

COMPLETELY Virtual mailing lists....
by Scott Brown 21 Feb '01

21 Feb '01

Has anyone been able to completely virtualize the mailing lists under mailman v2? I'm not talking about the improvements made in v2.... I mean having multiple domains be able to have the SAME list name. Unless I've missed something, right now they're limited to being unique per machine. I know I can do this now with foo_LISTA and bar_LISTA - but thats just cludgy when the users want LISTA as the name... My gut instinct is to have all the file access routines store their info under a directory named after the virtual host. I'm not sure how simply this could be done as I've not looked tooo deeply into the guts of it, but if the file references are all dealt with via common routines, it couldnt be that hard could it? so in ($prefix)/mailman/lists there would be foo.com/somelistname foo.com/LISTA bar.com/someotherlistname bar.com/LISTA Obviously, the mail receiver post script would need to look at who the mail was addressed to in order to be able to deal with the appropriate list.... probably some extra things in the webpage generation as well. Is this a pipe dream??

4 5

-checkins broken?
by Ron Jarrell 20 Feb '01

20 Feb '01

Something wrong with mailman-checkins? A whole buncha stuff got checked in, which fixed several of the things I'd noticed (although I'd already made those changes locally), but I didn't see a thing from the checkins list...

1 1

Archiver options
by Gary Palmer 20 Feb '01

20 Feb '01

Hi, Is there any way of turning off specific archive index options? e.g. keep threads and author, but disable date and subject? Thanks, Gary

1 0

Re: [Mailman-Developers] print >> sys.stderr does not compile
by Ralph Corderoy 20 Feb '01

20 Feb '01

Hi Fil, > Well: I've symlinked /usr/bin/python to /usr/bin/python2, and another error > pops up : does mailman need some python libraries which are not in > python2-base ? > > ... > ImportError: No module named mimelib.address Yep, Barry's new mimelib. http://www.wooz.org/users/barry/software/mimelib/index.html Ralph.

1 0

Re: [Mailman-Developers] print >> sys.stderr does not compile
by J C Lawrence 20 Feb '01

20 Feb '01

On Tue, 20 Feb 2001 12:50:13 +0100 Fil <fil(a)rezo.net> wrote: > All right. But. I've installed python2 with Debian, and mailman > does not know how to use it (Debian installs python2 as > /usr/bin/python2, and /usr/bin/env python points to the python1x > version.) > But this must be a debian issue ;) Move your /etc/alternatives pointer. -- J C Lawrence claw(a)kanga.nu ---------(*) http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Big checkins a'comin'!
by barry＠digicool.com 20 Feb '01

20 Feb '01

Folks, As you've probably guessed, I've been quite busy with other things[*] lately. But I /have/ been working on Mailman in my spare time. I'm now ready to check in some significant changes. They seem to work although they've only gone through limited testing, and I have not even converted my own sites to using this snapshot. Mostly I want to checkpoint my CVS and give others a chance to play with the new stuff and give feedback. These are mostly architectural changes to the mail delivery subsystem, along many of the lines that were discussed last year. My next step is to finish integrating the I18N stuff. I'll probably work toward a release after that and leave rewriting the web subsystem until the following release. I'm shooting for a working alpha 2.1 before the Python conference early next month. First, all messages internally are now mimelib.Message objects instead of rfc822.Message objects. In case you don't know, I've been working on the side on a spanking new, from scratch Python package to better handle RFC1341 (MIME) and RFC822 style messages. I'm pretty happy with it, and just released version 0.2. If you plan to run the newest snapshot of Mailman, you will need to download and install mimelib-0.2. For more information see http://barry.wooz.org/software/pyware.html Installation is very easy due to Greg Ward's wonderful distutils package. Don't worry though, Mailman 2.1 will come bundled with mimelib. As an aside, take a look at ToDigest.py to see how much cleaner creating MIME and RFC1153 digests is now! Oh yeah, did I mention our plain text digests are now RFC1153 compliant? :) The second major change is the splitting of the qfiles queue into several sub-queues. Right now I've got incoming, outgoing, news, archiver, and `virgin' queues (the latter being for messages conceived internally by Mailman). Here's how messages flow in the system: (mailman) ------> qfiles/virgin -------+ | v smtpd ----------+--> qfiles/in -+-> qfiles/out -> smtp | | cron/gate_news -+ +-> qfiles/news -> nntpd | +-> qfiles/arch -> (archiver) Each qfile directory has an associated qrunner (e.g. IncomingRunner, OutgoingRunner, NewsRunner, ArchRunner, VirginRunner) which is managed by a master watchdog (the transformed cron/qrunner). This is the third major change: qrunner is now a long running process. If it finds that one of its subrunners has exited, it will restart it. Thus each subrunner needs no lock, although the master does hold a lock. Further, it is possible (although as yet not fully tested), to create more than one qrunner per qfile subdirectory. It is best to create 2^N such qrunners, since the hash space will be divided up among the parallel runners. This should be random enough to allow for improved performance on clusters or multiproc machines without a lot of complicated coordination machinery. This arrangement has another benefit: you could replace any of the qrunners with your own processes to clear the queues. The only one that probably /has/ to be the Mailman qrunner is qfiles/in since that's what performs the "moderate and munge" phase of message disposal, and it is the only queue which requires a list lock (well, ArchRunner does currently, but that's an artifact of the implementation). E.g. I could see a completely independent runner for qfiles/arch to interface with external archivers, or a better, more highly parallelized runner for the qfiles/out queue. Notes: the NewsRunner may not work. I can't tell if my new ISP is just refusing postings, requires some authorization I'm not aware of, or if the runner is actually broken. I'll fix that later. Also, there is no separate BounceRunner yet, although there will be. I know some of you have thought about other queues, and I don't think it would be too difficult to squeeze those in. Each message in the queue is represented by two files, a .msg file which is simply the plain text of the message in RFC822+envelope format, and a .db file, the format of which is configurable on a per-site basis. Three formats are currently supported, Python marshals, `ascii' (plaintext key/value pairs), and "bsddb native", which are hash files as written by the default Python bsddb module. It isn't difficult to add additional file formats. Hmm, other than that, there's a few more bounce detectors. Also, I'm ditching the crufty md5/crypt munging of passwords and opting for an sha1 hash always. However, to support backwards compatibility (i.e. the list passwords are not kept in plain text), if the sha hash of the response doesn't match the challenge, we try crypt as a fallback. Remember, Python 2.0 is required! Enjoy, -Barry [*] Primarily the Python 2.1 alpha releases, spending a week on a company retreat, getting ready for the Python conference in Long Beach March 5-8, and working on "paying gig" stuff, primarily a Berkeley DB storage implementation for the Zope Object Database. Anybody else coming to the Python conference?

7 27