Mailman 3 September 2004 - Mailman-Developers

HTML email
by Marci Abraham O'Daffer 26 Sep '04

26 Sep '04

I am having a hard time understanding how mm handles HTML mail. I realize the digest pretty much trashes it, and the archive does not preserve it, but is there any way to make either of them behave so that the end user has a digest or an archive that is more readable? Thanks. -- Marci Abraham O'Daffer marci(a)kalico.net

3 2

Convert from 2.1b2+ and move to another host
by Bob Puff＠NLE 26 Sep '04

26 Sep '04

Hi Gang, I've got to move a few lists from a machine running "2.1b2+" to one running the latest "stable" 2.1.5 version. No archives, just basic lists. Can I just create the lists on the new machine, and then copy over the config.pck file, or is there more "data massaging" that needs to be done? Has the format of this file changed any since the beta 2 version of 2.1? Bob

2 1

Re: [Mailman-Developers] HTML email
by J C Lawrence 26 Sep '04

26 Sep '04

On Sat, 25 Sep 2004 15:25:18 -0700 Marci Abraham O'Daffer <marci(a)kalico.net> wrote: > I am having a hard time understanding how mm handles HTML mail. In generaly, especially in v2.1, it handles it just fine by not doing anything special with it. > I realize the digest pretty much trashes it... That's only true of the text digest. MIME digests handle HTML (and MIME) just fine. You may wish to disable text digests if your lists have frequent HTML messages.. > .. and the archive does not preserve it... Actually, it can. You can also use an external archiver such as MHonArc which is particularly capable in this areas. > ... but is there any way to make either of them behave so that the end > user has a digest or an archive that is more readable? Yup. See above and make sure you are running v2.1 for starters. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] Foreign Lang Posts
by J C Lawrence 26 Sep '04

26 Sep '04

On Sat, 25 Sep 2004 13:14:46 -0700 Marci Abraham O'Daffer <marci(a)kalico.net> wrote: > We have a user who experiences intermittent problems with MM -- > sometimes it sends, sometimes it does not. She has multiple lists, and > all the same problem. The only consistent issue I can see is that she > sends a lot of messages in French. Are there known issues with MM > dealing with non-US characters? Or could her ISP be messing with the > formatting somehow? I can't think of anything else. Any suggestions > would be most appreciated. What do the Mailman and MTA logs say? -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

limt people sending to list to only people who have @pps.k12.or.us in their email address
by Bev Scott 24 Sep '04

24 Sep '04

Hi Jim Thank for your help that works great now. Another question is there away to limit email coming to the list to a certain domain for example Only except email coming from @pps.k12.or.us with out adding the person email id. Bev On Wed, Sep 22, 2004 at 12:59:48PM -0700, Bev Scott wrote: > The list are moderated list. Only certain staff members can send to > them. I add all-list(a)pps.k12.or.us as a admin and a moderator of the > list. > > list1 > list2 > list3 > > We would like to make an alias called all-list > all-list: list1(a)mailman.pps.k12.or.us, list2(a)mailman.pps.k12.or.us, > list3(a)mailman.pps.k12.or.us > > what happens when I send to the all-list(a)pps.k12.or.us > > the message gets sent to the vette file in the logs (message > is "has implicit destination") and get saved in the Tend to > pending moderator requests You can add 'all-list(a)pps.k12.or.us' to the acceptable_aliases option on the Privacy options... Recipient filters... page of each of list1, list2, and list3 to have 'all-list' be recognized as an acceptable destination. -- Jim Tittsler http://www.OnJapan.net/ GPG: 0x01159DB6 Python Starship http://Starship.Python.net/ Ringo MUG Tokyo http://www.ringo.net/rss.html Bev Scott Web Database Portland Public Schools 503-916-2000 4966 bscott(a)pps.k12.or.us Bev Scott Web Database Portland Public Schools 503-916-2000 4966 bscott(a)pps.k12.or.us

2 1

Re: [Mailman-Developers] 8bit and NNTP gateway
by J C Lawrence 23 Sep '04

23 Sep '04

On Wed, 22 Sep 2004 17:57:03 -0700 John W Baxter <jwblist(a)olympus.net> wrote: > On 9/21/2004 1:14, "Brad Knowles" <brad(a)stop.mail-abuse.org> wrote: >> At 11:24 PM -0400 2004-09-20, J C Lawrence wrote: >>> Are there any extant tools to detect 8bit mail (identified by a >>> Content-Transfer-Encoding: 8bit or not) and to suitably transform >>> the affected MIME parts (or base message if no MIME) to >>> quoted-printable? >> The easy way to handle this is to configure your MTA so that it >> thinks that it is 7-bit only, and it should do the conversion for >> you. Sendmail certainly does this, and I'm pretty sure postfix does >> too. Not sure about any of the others. > Exim by default is configured to not advertise 8BITMIME and to reject > offered 8 bit messages. It doesn't try to transcode. The actual background for the problem is that I have 8bit messages in my archives that I need to import into newsgroups. > And since it doesn't, it can't accept 8 bit messages for relay since > it would have to do the conversion if the relay target doesn't accept > 8 bit. Much as you identify, I can prevent the problem from occurring via Exim in future. My problem is the archives and any other potential message acquisition paths which don't pass through Exim, and yet which will need both NNTP injection and broadcast via Mailman and Exim. > If that's an issue in a given installation, then Exim is the wrong MTA > choice. Exim is unlikely to be taught to do the conversions. That's where filters come in. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

combine list
by Bev Scott 22 Sep '04

22 Sep '04

Hello, I have three lists. The list are moderated list. Only certain staff members can send to them. I add all-list(a)pps.k12.or.us as a admin and a moderator of the list. list1 list2 list3 We would like to make an alias called all-list all-list: list1(a)mailman.pps.k12.or.us, list2(a)mailman.pps.k12.or.us, list3(a)mailman.pps.k12.or.us what happens when I send to the all-list(a)pps.k12.or.us the message gets sent to the vette file in the logs (message is "has implicit destination") and get saved in the Tend to pending moderator requests Is there anyway that I can get this to work? Bev Bev Scott Web Database Portland Public Schools 503-916-2000 4966 bscott(a)pps.k12.or.us

3 2

8bit and NNTP gateway
by J C Lawrence 22 Sep '04

22 Sep '04

Are there any extant tools to detect 8bit mail (identified by a Content-Transfer-Encoding: 8bit or not) and to suitably transform the affected MIME parts (or base message if no MIME) to quoted-printable? I'm running an NNTP gateway to CNEWS and am running into problems with 8bit mail being rejected by the NNTP gateway after being accepted and otherwise broadcast by Mailman. I can write a script to do the needed transformations, but handling the general MIME case is a bit messy and something I'd like to avoid. Has anybody resolved this area? I note that Mailman itself doesn't seem to touch it.xc -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

3 2

Re: [Mailman-Developers] 8bit and NNTP gateway
by J C Lawrence 21 Sep '04

21 Sep '04

On Tue, 21 Sep 2004 11:00:22 -0400 John Dennis <jdennis(a)redhat.com> wrote: > On Tue, 2004-09-21 at 09:38, J C Lawrence wrote: >> Exim alas does not fall in that camp, and I need to use Exim as >> versus Postfix due to the fine grained control it offers over >> executing UID/GID for filters. Sendmail and QMail are not a >> reasonable consideration for this case. > Out of curiosity could you elaborate on UID/GID constraints which > prevent you from using a postfix/mailman combination? While things have moved on slightly, the details are still pretty similar to the details I posted to the User FAQ a few years back (see the stuff about TMDA etc). Essentially I run a moderately complex filter system in front of Mailman and various other accounts which requires explicitly defining and controlling the UID and GID as well as the assumed environment (${HOME}, shell context etc) that procmail and other scripts are executed under. Commonly that UID/GID is not an obvious function of the LHS of the address, but rather an arbitrary special case mapping. I can reach the same end under Postfix as of last I checked via vhosts and maps, but that would require also changing my MX and address architecture which I'm not prepared to do. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] 8bit and NNTP gateway
by J C Lawrence 21 Sep '04

21 Sep '04

On Tue, 21 Sep 2004 10:14:10 +0200 Brad Knowles <brad(a)stop.mail-abuse.org> wrote: > At 11:24 PM -0400 2004-09-20, J C Lawrence wrote: >> Are there any extant tools to detect 8bit mail (identified by a >> Content-Transfer-Encoding: 8bit or not) and to suitably transform the >> affected MIME parts (or base message if no MIME) to quoted-printable? > The easy way to handle this is to configure your MTA so that it thinks > that it is 7-bit only, and it should do the conversion for you. Exim alas does not fall in that camp, and I need to use Exim as versus Postfix due to the fine grained control it offers over executing UID/GID for filters. Sendmail and QMail are not a reasonable consideration for this case. > Sendmail certainly does this, and I'm pretty sure postfix does too. > Not sure about any of the others. The closest I've found for the Exim world is: http://www.exim.org/pipermail/exim-users/Week-of-Mon-20020916/043866.html <sigh> > The only other alternative I know of would be to hack the gateway code > in Mailman so that it does the conversion. The Mailman code for news injection is rather generous in terms of what it accepts on the inbound side, and does remarkably little to fix errors or enforce correctness. My current hack for properly handling NNTP injection looks something like: --<cut>-- #!/usr/bin/python import os, email, email.Utils, re, sys, string, time, nntplib, optparse from StringIO import StringIO _error = """ Exception: %s Error: %s -=-=-=-=-=-=-=-=-=-=-= %s -=-=-=-=-=-=-=-=-=-=-= """ parser = optparse.OptionParser () parser.add_option ("-s", "--server", dest = "nntp_server", help = "NNTP server to post to", metavar = "SERVER") parser.add_option ("-p", "--port", type = 'int', dest = "nntp_port", help = "Port on NNTP server to post to", metavar = "PORT") parser.add_option ("-U", "--user", dest = "nntp_user", help = "Authentication user on NNTP server", metavar = "USER" ) parser.add_option ("-P", "--password", dest = "nntp_password", help = "Authentication password on NNTP server", metavar = "PASSWORD") parser.add_option ("-m", "--mode", type = 'int', default = 0, dest = "nntp_mode", help = "Set reader mode on NNTP server?", metavar = "MODE") parser.add_option ("-a", "--approved", dest = "approved", help = "Value of Approved: header to add", metavar = "APPROVE D") parser.add_option ("-g", "--group", dest = "newsgroup", help = "Newsgroup[s] to post to", metavar = "NEWSGROUP") parser.add_option("-d", "--debug", action = "store_true", dest = "debug", default = False, help = "Print debug messages") #parser.add_option("-v", "--verbose", # action = "store_false", dest = "verbose", default = False, # help = "Verbose operation") (options, args) = parser.parse_args() def debug ( opt, str): """ Output if in debug mode. """ if opt.debug: print str #debug (options, options) re_continuation = re.compile ('^\s+(\S.*)$') re_empty = re.compile ('^\s*$') re_envelope = re.compile ('^From ') ndx = 0 headers = [] body = [] current = headers inbody = 0 # Read message in headers line and body list for line in sys.stdin.readlines (): rc = re_empty.search (line) # End of headers or hit separator if inbody or rc != None: # End of headers or hit separator inbody = True body.append (line) continue rc = re_continuation.search (line) # Header continuation if rc: headers[-1] += ' ' + rc.group (1).strip () continue if line.startswith ('From '): headers.append ("X-Envelope-From: " + line[4:]) continue headers.append (line.strip ()) # Fix up headers to be suitable for NNTP headers_out = [] headers_noted = {} for hdr in headers: # Extract name/value ndx = hdr.find (':') # Can't use split as there may be other colons n = hdr[:ndx].strip () v = hdr[ndx + 1:].strip () # Some header names require special casing per the RFC. n = string.capitalize (n) cased_headers = { 'Message-id' : 'Message-ID', 'Relay-version' : 'Relay-Version', 'Posting-version' : 'Posting-Version', 'Reply-to' : 'Reply-To', 'In-reply-to' : 'In-Reply-To', 'Followup-to' : 'Followup-To', 'Date-received' : 'Date-Received', 'newsgroups' : 'newsgroups', 'Content-type' : 'Content-Type', 'Content-transfer-encoding' : 'Content-Transfer-Encoding', } if n in cased_headers.keys (): n = cased_headers[n] # Remove illegal headers that just shouldn't be there. bad_headers = ['NNTP-Posting-Host', 'NNTP-Posting-Date',] if n in bad_headers: continue # Prepend X- to unprotected headers protected_headers = ['From', 'Date', 'Subject', 'Path', 'Relay-version', 'Posting-version', 'Newsgroups', 'Message-id', 'Reply-to', 'Sender', 'Followup-to', 'Date-received', 'Expires', 'References', 'Distribution', 'Organization', 'Approved', 'Mime-Version' ] if n not in protected_headers and not n.startswith ('X-'): n = "X-" + n # Remember the values of special headers for later processing special_headers = [ 'From', 'Message-ID', 'Path', 'References', 'In-Reply-To', 'Date', 'newsgroups', 'Subject', 'Content-Transfer-Encoding', ] if n in special_headers: headers_noted[n] = v # We need to suppress some headers so that we can have a chance to fix # them up later. suppressed_headers = [ 'Message-ID', 'Content-Transfer-Encoding' ] # We need to specially process Message-ID later (format fixups), # so don't add it to the headers out yet. if n in suppressed_headers: continue # Already stashed in headers_noted # Leave everything else untouched headers_out.append ("%s: %s\n" % (n ,v)) # Missing From? if not headers_noted.has_key ('From'): v = 'unknown@somewhere' headers_noted['From'] = v headers_out.append ('From: ' + v + '\n') # Make a (new) Path as needed if not headers_noted.has_key ('Path'): v = headers_noted['From'] a, b = email.Utils.parseaddr (v)[1].split ('@') headers_out.append ('Path: %s!%s\n' % (b, a)) # Fix References header from In-Reply-To if not headers_noted.has_key ('References') \ and headers_noted.has_key ('In-Reply-To'): v = headers_noted['In-Reply-To'] v = email.Utils.parseaddr (v)[1] headers_out.append ('References: <' + v + '>\n') # Missing Date? if not headers_noted.has_key ('Date'): debug (options, 'Missing date fixed.') headers_out.append ('Date: ' + email.Utils.formatdate (time.time (), 1) + '\n' ) # Missing Subject? if not headers_noted.has_key ('Subject'): debug (options, 'Missing subject fixed.') headers_out.append ('Subject: (subject missing)\n') # Fix Message-ID and fieldname case (adding if needed) if not headers_noted.has_key ('Message-ID'): headers_noted['Message-ID'] = email.Utils.make_msgid () v = headers_noted['Message-ID'] if not v.find ('<') < v.find ('>'): # Message-ID: <id_string> headers_out.append ('X-orig-Message-ID: ' + v + '\n') v = email.Utils.parseaddr (v)[1] headers_out.append ('Message-ID: <' + v + '>\n') # Fix or add newsgroups header v = None if headers_noted.has_key ('newsgroups'): v = headers_noted['newsgroups'] if not v: # Just add the one from the CLI headers_out.append ('newsgroups: ' + options.newsgroup + '\n') else: # Check the newsgroups header for the group already being present groups = v.split (',') found = 0 for group in groups: group = group.strip () if options.newsgroup == group: found = 1 if not found: headers_out.append ('newsgroups: ' + v.strip () + ', ' + options.newsgroup + '\n') # Add an approved header if requested if hasattr (options, 'approved'): headers_out.append ('Approved: ' + options.approved + '\n') # Re-assemble message out_f = StringIO () for line in headers_out: out_f.write (line) out_f.write ('\n') for line in body: out_f.write (line) out_f.seek (0) if options.debug: debug (options, "Not sending.") debug (options, out_f.getvalue ()) sys.exit (0) try: debug (options, 'Server: %s Port: %s User: %s Password: %s Mode: %s' % (options.nntp_server, options.nntp_port, options.nntp_user, options.nntp_password, options.nntp_mode)) server = nntplib.NNTP (options.nntp_server, options.nntp_port, options.nntp_user, options.nntp_password, options.nntp_mode) server.post (out_f) server.quit () except (nntplib.NNTPError), message: debug (options, 'Message: ' + repr (message) + '\n' + _error % sys.exc_info () ) raise message sys.exit (2) sys.exit (0) --<cut>-- This was originally written to inject old archives into a news spool, for which it works rather well modulo a few caveats discussed below. About the only other thing which ended up needing to be done for archive injection was to comment out the date check in your copy of inews so that the old mail being freshly injected wasn't bounced for having a stale date. Strengths of the script are that it enforces and fixes/generates header correctness in multiple areas (case, presence etc), preserves all prior headers (esp Received:) for debugging/tracing downstream, unfolds continued header lines for older/more fragile News servers downstream, and generally tries to be very generous in terms of what it will accept and rather pedantically picky in terms of what it will emit. Known weaknesses are that it's fragile on incorrect or missing CLI arguments, email.Utils.parseaddr() has known problems with addresses which contain parenthesis in the GECOS field, and it doesn't begin to handle the 7/8bit problem yet. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

4 3