Mailman 3 April 2001 - Mailman-Developers

broken HTML
by Tollef Fog Heen April 17, 2001

April 17, 2001

It seems to be a small bug in mailman 2.0.3, in Mailman/htmlformat.py, line 281, this _will_ generate broken html like: <HTML> <HEAD> <TITLE>soci Admindb Results</TITLE> </HEAD> <BODY <BODY bgcolor="#ffffff"> <h2>Database Updated...</h2><h2>Administrative requests for mailing list: <em>so Note the double (and first unfinished) <BODY> If this is still in cvs, somebody please fix. :) -- Tollef Fog Heen Unix _IS_ user friendly... It's just selective about who its friends are.

2 1

Multilanguage support in Mailman.
by Conny Andersson April 17, 2001

April 17, 2001

Hi, I just entered the list and thought I could share a beta release of a Swedish translation of the templates and HTMLFormatter.py files. I'm not sure how to incorporate the changes to Mailman (I have just made symbolic links in my setup) but one suggestion could be to have an option for every user to choose their desired language. You can find the tar file at http://conny.nu/mailman-swe.tar.gz Best, Conny .................................................... Conny Andersson Work Area: Embedded Systems Cendio Systems AB Direct: +46(0)13 290882 Teknikringen 3 Mobile: +46(0)707 452066 583 30 Linköping Fax: +46(0)13 214700 Sweden Internet: www.cendio.com

2 1

Urgent: Duplicates with 2.0 final
by Todd Joseph April 17, 2001

April 17, 2001

Sorry for sending this to developers and users. I'm not sure which group is more appropriate. I've been using 2.0 final for a couple of months now without a problem. I've got roughly 60,000 people on my mailing list and I'm using sendmail as the MTA on RH6.2. The list is moderated and there is only 1-2 messages a month. The last message sent to the list was sent to some subscribers twice. I had seen the problem before with 2.0rc1. From the release notes for 2.0rc2, that is documented as fixed (bug #117015) (see below). Anyone know why I would see it happen again in 2.0 Final? Here's what I've done: (1) Scanned the Archives. Found a post to mailman-users with no reply that seems related. http://mail.python.org/pipermail/mailman-users/2001-April/010625.html (2) Looked through Syslog logs for System and Sendmail errors. Nothing unusual reported. (3) Looked through the Mailman logs dir (/home/mailman/logs). Nothing out of the orinary except in the errors file (see below). What's strange about this is the duplicate message had already been sent by this time (the few duplicates checked showed 1st delivery at ~8PM April 6 and subsequent delivery at ~2:30AM April 7 -- all times synced with NTP on systems tested). Excerpt from /home/mailman/logs/errors: Apr 07 04:09:18 2001 qrunner(16818): Traceback (innermost last): Apr 07 04:09:18 2001 qrunner(16818): File "/home/mailman/cron/qrunner", line 278, in ? Apr 07 04:09:18 2001 qrunner(16818): kids = main(lock) Apr 07 04:09:18 2001 qrunner(16818): File "/home/mailman/cron/qrunner", line 256, in main Apr 07 04:09:18 2001 qrunner(16818): dequeue(root) Apr 07 04:09:18 2001 qrunner(16818): File "/home/mailman/cron/qrunner", line 184, in dequeue Apr 07 04:09:18 2001 qrunner(16818): os.unlink(root + '.db') Apr 07 04:09:18 2001 qrunner(16818): OSError : [Errno 2] No such file or directory: '/home/mailman/qfiles/c879896f57e6392365c42f995f13c09de378bbcd.db' (cut-n-paste for 2.0rc2 release notes)... >SourceForge bugs fixed: 116615 (README.BSD update), 117015 (duplicate >messages on moderated posts), 117548 (exception in HyperArch.py), >117682 (typos), 121185 (vsnprintf signature), 121591 and 122017 (bogus >link after web unsubscribe), 121811 (`subscribe' in Subject: doesn't >get archived) Help...

3 8

Re: [Mailman-Developers] mailman misparses non RFC 822 compliant Email addresses
by Dan Mick April 13, 2001

April 13, 2001

> > I'm coming to this late, but: we all agree that > > > > David E.Fox <dfox(a)m206-157.dsl.tsoft.com> > > > > is not a legal RFC822 address, right? > > I can't read this into the RFC document. OK, I retract my claim. I was misled by address = mailbox ; one addressee / group ; named list mailbox = addr-spec ; simple address / phrase route-addr ; name & addr-spec route-addr = "<" [route] addr-spec ">" phrase = 1*word ; Sequence of words word = atom / quoted-string atom = 1*<any CHAR except specials, SPACE and CTLs> which seems to lead to "phrase can't include spaces without being quoted". But I missed this disclaimer in 3.4.4: The one exception to this rule is that a single SPACE is assumed to exist between contiguous words in a phrase, Sigh. This, of course, makes me wonder why phrase can't include that in its definition, rather than saying "1*word", but there you have it.

2 1

Re: [Mailman-Developers] Documentation for announce only lists
by Ben Burnett April 13, 2001

April 13, 2001

Thanks for the feedback. It went to mailman-DEVLOPERS (no E) so I've posted it in its entirety below. I think the screen shots are a good idea. I'll try to Gimp some together this weekend if I can't get our marketing guys to tackle it. I'm not running any lists with high membership currently, so if anyone else has any tips on running large audience announce lists I'd like to include them in the documentation. -Ben ------- Original Copy ------- Subject: Re: [Mailman-Developers] Documentation for announce only lists Date: 04/12/2001 10:40 PM From: Todd Joseph <todd(a)connactivity.connactivity.com> To: benwa(a)ocentrix.com Cc: mailman-devlopers(a)python.org Looks good to me. Some screen shots of the Web pages would help break up the descriptions (draw interest). I've been doing much the same myself. I'm frustrated with duplicate messages having been sent with 2.0 Final (IE I sent an announcement and it went to some email addresses twice). I'm not confident how to resolve the problem. I think announce only lists can be much larger than regular lists (one of mine is 60,000). Something that would be nice to have accompany your document is primer on what to watch for. (IE if you are running a large moderated lists with low volume, you may see duplicates when...)

1 0

Re: [Mailman-Developers] mailman misparses non RFC 822 compliant Email addresses
by Dan Mick April 13, 2001

April 13, 2001

> > >>>>> "MM" == Marc MERLIN <marc_news(a)valinux.com> writes: > > MM> Correct. I haven't found the right piece of code in mailman > MM> yet, but it smells like a regular expression that grabs the > MM> wrong pattern and ends up with davide.fox instead of > MM> dfox(a)m206-157.dsl.tsoft.com > > I suspect the problem is in Python's rfc822 module: > > >>> a = rfc822.AddrlistClass('David E.Fox <dfox(a)m206-157.dsl.tsoft.com>') > >>> a.getaddrlist() > [('', 'DavidE.Fox'), ('', 'dfox(a)m206-157.dsl.tsoft.com')] > > Mailman, interestingly enough, has Utils.ParseAddrs() which appears to > try to work around problems in rfc822: > > >>> ParseAddrs('David E.Fox <dfox(a)m206-157.dsl.tsoft.com>') > 'dfox(a)m206-157.dsl.tsoft.com' > > which seems to suck out the right address in this case. What should > probably in MM2.1 is for MailList.HasExplicitDest() to fallback on the > output of ParseAddrs() if the rfc822 method doesn't match. I'm coming to this late, but: we all agree that David E.Fox <dfox(a)m206-157.dsl.tsoft.com> is not a legal RFC822 address, right? Strings containing spaces must-repeat-must be enclosed in doublequotes: "David E.Fox" <dfox(a)m206-157.dsl.tsoft.com> I see the subject, but: why is this considered to be a "problem" in rfc822, rather than a problem in the mail format? I would expect other MTAs and MUAs to barf on this too.

2 1

Documentation for announce only lists
by Ben Burnett April 13, 2001

April 13, 2001

Hey all, My company is in the process of implementing Mailman to provide announcement lists for our hosting clients. I have been playing with 2.0.1 to 2.0.3 for the past two months and have come up with what I think is a pretty good (and very verbose) description of how to set up an announce only list. Judging by the amount of traffic on this topic here and on the mailman-users list I though it would be a good idea to submit it. Please excuse the verbosity of my description as I was trying to make the description easy to understand by those who are new to Mailman. I was also trying to take into account the different ways that I thought someone would want to run an announcement list. I have tested each of the methods outlined in the description and it all seems to work well, but I would like to get some more opinions before I post it to the mailman-users list. If anyone finds the descritption usefull please feel free to use it however you like. instructions follow: -------------------- Managing announce only lists. INTRODUCTION Keep in mind that some of the techniques below require making changes that will affect how your Mailman installation will manage all of its lists. Wherever this is an issue it has been pointed out. If you choose to use any of these techniques It might be a good idea to run separate installations of Mailman for discussion lists and announcement lists. If you would like to do this there are some additional tips listed at the end of the instructions. In order to run an announcement list with Mailman you'll need to configure a regular list to allow only certain addresses to post to the list. You should create the list as you would any other list. and make a few changes to the it through the web based administrative interface. An announce only list may require some additional maintenance if you frequently have unauthorized posters attempting to post to the list. BASIC CHANGES In the administrative interface under Privacy options set the following fields as shown. Must posts be approved by an administrator? No Restrict posting privilege to list members? No Then place the e-mail addresses of all of those who are authorized to post announcements in the field "Addresses of members accepted for posting to this list without implicit approval requirement." This will limit all posts to the list to those who are listed. Unfortunately(?) all posts to the list by unauthorized posters will still show up in the "Tend to pending administrative requests" area of the administrative interface. List administrators will also receive an email for each of these posts telling them that they have an administrative request pending. This email can be disabled under General options by setting the field "Should administrator get immediate notice of new requests, as well as daily notices about collected ones?" to "No". This might be a bad idea because this also turns off emails that refer to other types of administrative requests that may need approval such as subscribes and unsubscribes. However, administrators will still receive a notice of any pending administrative requests at 5PM every day. MAINTAINING ANNOUNCE ONLY LISTS Mailman does not come with any easy way to automate tending to the administrative requests that are holding unauthorized posters' emails for approval. I have found that there are different ways to get rid of them manually. Each method has its pros and it's cons. In deciding how to get rid of these unauthorized posts one should consider whether they want the unauthorized poster to know anything about the fact that their post has been held. Under General options the field "Send mail to poster when their posting is held for approval?" will determine... exactly that. If you set this to "Yes" the poster will receive an email based on the contents of the template: (/your_mailman_install_directory/templates/postheld.txt). This template can be modified to explain that they have posted to a list that is for announcements only. And that they have posted from an address that is not authorized for posting. This is a site-wide template, so any changes you make to it will affect all lists. The method used to actually tend to the unauthorized post will also affect how much the poster knows about the posting process. If you choose "reject" in handling the post the poster will receive an email based on the contents of the template: (/your_mailman_install_directory/templates/refuse.txt) combined with the text explanation offered in the text box marked "If you reject this post, please explain (optional)". However, if you choose "discard" the poster will not receive an email at all. Make sure that you tend to these requests frequently because if they are not dealt with they can build up to a very long list on the administrative request management page. These requests will be mixed together with any other types of requests that may be pending. CLOSING These are the basic steps in managing an announcement list with Mailman. There are many other tweaks and changes that you can make to optimize your Mailman install for announcement only lists, such as modifying the templates for each list or for the site. And setting some default values for list creation in mm_cfg.py. And of course since Mailman is an open source project written in Python you could always sit down and learn some Python. Python is a great language that is being used in more and more places for all kinds of projects. With basic to intermediate Python skills you can make some other changes to the Mailman scripts that will make managing announcement only lists a lot easier. As your skills progress you can even help to develop Mailman into exactly the mailing list manager that you are looking for. Wouldn't it be great if you could help to contribute to this wonderful list manager that was given to you freely? RUNNING A SEPERATE ANNOUNCE ONLY INSTALLATION If you are running a separate installation of Mailman for announcement lists only you might make a couple of the changes below. 1. In the file mm_cfg.py, set the variable "DEFAULT_DIGESTABLE = 0" set to make list digests unavailable since announce only lists will not have much traffic. 2. If your announcement lists have very low traffic you might consider redefining the variable "DEFAULT_ARCHIVE_VOLUME_FREQUENCY" in mm_cfg.py. Setting it to "2" will create quarterly archives and setting it to "0" will create yearly archives. 3. If your list admins really don't like the 5PM reminders of requests pending authorization These emails can be disabled by removing or commenting out the second line below in the mailman user's crontab. # At 5PM every day, mail reminders to admins as to pending requests 2 0 17 * * * /usr/bin/python -S /your_mailman_install_directory/cron/checkdbs I *strongly* recommend against doing this however because this will turn off ALL 5PM pending request reminders for ALL lists managed by this Mailman installation. As mentioned above This might be a bad idea because this also turns off emails that refer to other types of administrative requests that need approval such as subscribes and unsubscribes. -------------------- Thank you for your help. - Ben Burnett

1 0

Mailman 2.0 usage notes...
by chuq von rospach April 11, 2001

April 11, 2001

Been whapping at my big mailman machine the last week, because it's been slowly falling behind and never quite catching up. Unfortunately, it seems like I've simply hit capacity for now, so I'm looking for ways to extend that or at least minimize the damage until 2.1's multi-threading queueing comes online and I can use it (my big problem is the single-threading....) One thing that would help would be if the Sendmail module got productionalized so it could be used instead of SMTPDirect, because then you could use the -deliverymode=defer option, which you can't on SMTPDirect because it disables some spamchecking. But going elbow-deep into the server for a few days under a constant grinding load has brought forward a few things I thought I'd pass on... First, there's a problem with the way queues are processed. qrunners uses: for file in os.listdir(xxxx) to read the queue. The order is undefined, but in practice, it's basically blatting it out how it's stored in the inode. If you're not overly busy, that's fine. But If you start hitting the point where you're backing up, it means you process "N" slots into the directory, then qrunner exits and starts again, and it then re-covers the same directory slots. Things that go in and don't get processed simply NEVER get processed, unless the system quiets down enough to let qrunner to catch up. Qrunner *really* needs to process the queue FIFO. Unfortunately, teaching qrunner to go FIFO is a bit complicated. you'd have to pull all of the filenames out, stat them all, and then sort that. There's a much easier solution, though -- In Mailman/Message.py where the filename si created, mailman uses time.time() and some other values to create a filename, which is then converted to hex. the idea is to create a unique filename. But in fact, time.time() should be unique (to be paranoid, one could grab it and then check to see if the filename exists and loop), and if you stored the queue files as "time.time()".msg/.db, then qrunner could sort the queue trivially, guaranteeing that the oldest messages are always processed in each queue run. This creates some interesting race conditions, where an item is added to the queue and never comes out -- which causes people to think it's lose and repost it, adding to the queue clogging, which slows stuff, which... until Saturday, when they all go home, the system slows down, and qrunner catches up and posts three day old messages.... And since the system seems to be working just fine, tracking it down is fun... This also led to finding a problem in the bounce processing area. The bouncer works pretty well, but it has one flaw for which I don't have an easy answer. If I'm subscribed as "chuq(a)plaidworks.com", but for some reason the bounce comes back as "chuq(a)mail.plaidworks.com" (or vice versa, or if I'm forwarding mail in some other name), the bouncer will catch the bounce and try to process it, not find me, and log it as a "user not subscribed". Unless the admin is somehow post-processing the bounce logs, though, that bounce is never REALLY handled, so it bounces indefinitely, and the admin never knows. This also over time encourages queue clogging and wastes bandwidth and CPU and all of that -- and worse, list admins and site admins probably think everything is fine because the bouncing system is working and these "not a member" bounces are never reported anywhere. On the other, other hand, you probably don't want to just blat all these at admins, since they'll tune them out. But some kind of nightly report of some sort is the tradeoff I'd make, I think, so admins could see continual bounce problems that need to be manually investigated. And I strongly recommend all site admins watch the bounce logs and look for these "missing" bounces, so they can be manually tracked. I found on my busy site this made a HUGE difference in my queue backlogs, too; these things were silently contributing a significant amount of traffic to the queue system and exacerbating my capacity issues. I really think the qrunner issue needs to be dealt with; it only shows up on fairly busy sites, but it's a definite bug for folks like me. The bouncer issue is less nasty, but in a "good citizens clean up their trash" attitude, I think we need to at least make sure list/site admins are aware of these bouncers, unless someone can figure out a way to automate fixing them (this is a place where VERP type things could help, but I'm not going there, honest... giggle) chuq -- Chuq Von Rospach, Internet Gnome <http://www.chuqui.com> [<chuqui(a)plaidworks.com> = <me(a)chuqui.com> = <chuq(a)apple.com>] Yes, yes, I've finally finished my home page. Lucky you. The first rule of holes: If you are in one, stop digging.

4 8

Patch for non-ASCII characters in Subject line
by Johan Holmberg April 11, 2001

April 11, 2001

Hello mailman-developers ! I have for a long time been annoyed by the fact that Mailman adds more than one copy of the mailing-list-prefix to the Subject line in replies sent to a list. This can occur if the Subject line contains non-ASCII characters (this has been reported in the bug-database as bug #230400: "German Umlaute mess up subject-line and body"). It looks something like (if you can view this in your mailers :-)): Subject: Re: [Test] Re: [Test] räkmsörgås The reason seems to be that Mailman has the following "algorithm" to decide what to do: - add list-name-prefix (if it isn't already there) instead of the more correct: - MIME-decode the subject - add list-name-prefix (if it isn't already there) - MIME-reencode the subject If no MIME-decoding is done, Mailman can fail to detect the precense of a mailing-list-prefix, since the prefix can be "buried" in the MIME-encoded stuff. I have made a patch to "CookHeaders.py" that fixes this problem, and implements the second alternative above (see the attached patch). Could something like this be added to future versions of Mailman ?? (I don't know if I have made the change the right way (I'm not a Python programmer)). /Johan Holmberg

1 0

mailman misparses non RFC 822 compliant Email addresses
by Marc MERLIN April 11, 2001

April 11, 2001

Apparently, this old bug is still here (I remember seeing it in the mailman 1.0rc1 days, and back then we solved it by having the user change his mail name so as not to have a dot) My mailman is configured to look at the header from (not the envelope from), and the header from says: From: David E.Fox <dfox(a)m206-157.dsl.tsoft.com> It seems that mailman gets confused by the dot in "David E.Fox" and then fails to handle the Email address correctly, and decides that the user isn't in the list of subscribers, so he can't post, or it may just be confused by the fact that there are no quotes around the name. (notice that mailman thinks the Email is davide.fox and not dfox(a)m206-157.dsl.tsoft.com) Did I miss something, is this actually a buglet that's still here? (Note that this From field is invalid as per RFC 822, because it's missing the double quotes. The user uses Kmail and it's a kmail bug). If you concur that this is a problem, please let me know, and I can open a bug on sourceforge. Thanks, Marc ----- Forwarded message from svlug-admin(a)lists.svlug.org ----- From: svlug-admin(a)lists.svlug.org Subject: svlug post from davide.fox requires approval To: svlug-admin(a)lists.svlug.org X-Ack: no Date: Sat, 07 Apr 2001 14:25:01 -0700 As list administrator, your authorization is requested for the following mailing list posting: List: svlug(a)lists.svlug.org From: davide.fox Subject: Re: [svlug] DSL vedor shakeout II: PSINet Reason: Post by non-member to a members-only list At your convenience, visit: http://lists.svlug.org/mailman/admindb/svlug to approve or deny the request. ----- End forwarded message ----- -- Microsoft is to operating systems & security .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | Finger marc_f(a)merlins.org for PGP key

3 3