Mailman 3 September 2006 - Mailman-Developers

Debugging mailman (resolving inline signature attachments)
by Sven Ingebrigt Ulland 04 Oct '06

04 Oct '06

Hello there. How would I go about debugging mailman, by -- for example -- polluting the .py libraries with print or assert statements? If I do that (which seems like a pretty thoughtless idea), the output would end up .. in /dev/null? It would probably be better to write to file in stead? Right now I'm having a specific problem, but it would be nice to have some general debugging methods to use in the future for other problems. I'm using version 2.1.5-8sarge2 (yes, Debian sarge). The current problem is that list msg_footer, that is list signatures, are inline attached as base64- encoded UTF-8 mime parts, but only in some cases: Original message: iso8859-15 text/plain. Result: Mailman distributes the message to the list members as mime multipart, first part is the original iso8859-15 message, and the list footer is appended like this: --===============0410419818== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline WWVzLCB0aGlzIGlzIG5vdCB0aGUgcmVhbCBsaXN0IHNpZ25hdHVyZSwgeW91IGhheG9yIG NyYXhvcg== --===============0410419818==-- The same happens for charset 'us-ascii' and 'iso8859-1', which were some examples I could find now after a quick look. When the original message is UTF-8 text/plain, however, the body and signature are merged together as they should be. The footer contains *only* standard 7-bit ascii values.. no 8-bit stuff or strangeness. Now, mailman determines this footer business in Handlers/Decorate.py on lines 80 to 110 (roughly). It calls the following to determine list charset: lcset = Utils.GetCharSet(mlist.preferred_language) GetCharSet calls mm_cfg.LC_DESCRIPTIONS, which imports Defaults.py, where -- since my lists have lang set to 'en' -- 'us-ascii' is returned. I also tried mapping 'en' to 'utf-8', without success. This is the snippet that determines if the message should be multipart or not (linenums included): 86 if not msg.is_multipart() and msgtype == 'text/plain' and \ 87 msg.get('content-transfer-encoding', '').lower() <> 'base64' and \ 88 (lcset == 'us-ascii' or mcset == lcset): The incoming messages from users are: - not multipart - charset is text/plain - content-transfer-encoding is mostly '7bit' (not base64) - lcset *should* be 'us-ascii', but it appears it might not be - mcset (message charset) is what the user used, normally iso8859-1, iso8859-15 or utf-8. I'd really like to debug that code passage to verify what variables are screwed up, and which are not. Any tips? best regards, sven

4 3

Re: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [8041] trunk/mailman/Mailman
by Tokio Kikuchi 03 Oct '06

03 Oct '06

bwarsaw(a)users.sourceforge.net wrote: > Revision: 8041 > http://svn.sourceforge.net/mailman/?rev=8041&view=rev > Author: bwarsaw > Date: 2006-09-25 00:53:58 -0700 (Mon, 25 Sep 2006) > > Log Message: > ----------- > Another milestone: you can now post to lists. Converted the following to use > the new configuration object: admin, admindb, bounces, confirm, inject, join, > leave, owner, post, request, unshunt, version. > > Also change MailList.GetScriptURL() to return the list's fully qualified name > in links. > I've tested a bit. Postfix.py: got duplicate warning while creating virtual-mailman. Shouldn't we include hostname in aliases? Otherwise, true virtual hosting breaks. MailList.py: got error when name is None. Looks like full_path is hostname@listname order. Is this name + '@' + self.host_name? I needed this patch to create new lists: mailman@colinux:~/src/svn.trunk$ svn diff Mailman Index: Mailman/MTA/Postfix.py =================================================================== --- Mailman/MTA/Postfix.py (revision 8041) +++ Mailman/MTA/Postfix.py (working copy) @@ -142,8 +142,9 @@ print >> fp, '# CREATED:', time.ctime(time.time()) # Now add all the standard alias entries for k, v in makealiases(mlist): + fqdnaddr = '%s@%s' % (k, hostname) # Format the text file nicely - print >> fp, mlist.fqdn_listname, ((fieldsz - len(k)) * ' '), k + print >> fp, fqdnaddr, ((fieldsz - len(k)) * ' '), k # Finish the text file stanza print >> fp, '# STANZA END:', listname print >> fp Index: Mailman/MailList.py =================================================================== --- Mailman/MailList.py (revision 8041) +++ Mailman/MailList.py (working copy) @@ -287,14 +287,17 @@ os.path.join(config.LOCK_DIR, name or '<site>') + '.lock', lifetime=config.LIST_LOCK_LIFETIME) # XXX FIXME Sometimes name is fully qualified, sometimes it's not. - if '@' in name: - self._internal_name, self.host_name = name.split('@', 1) - self._full_path = os.path.join(config.LIST_DATA_DIR, name) - else: - self._internal_name = name - self.host_name = config.DEFAULT_EMAIL_HOST - self._full_path = os.path.join(config.LIST_DATA_DIR, - self.host_name + '@' + name) + if name: + if '@' in name: + self._internal_name, self.host_name = name.split('@', 1) + self._full_path = os.path.join(config.LIST_DATA_DIR, name) + else: + self._internal_name = name + self.host_name = config.DEFAULT_EMAIL_HOST + self._full_path = os.path.join(config.LIST_DATA_DIR, + self.host_name + '@' + name) + else: + self._full_path = '' # Only one level of mixin inheritance allowed for baseclass in self.__class__.__bases__: if hasattr(baseclass, 'InitTempVars'): -- Tokio Kikuchi, tkikuchi(a)is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/

12 46

Re: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
by Barry Warsaw 29 Sep '06

29 Sep '06

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sep 28, 2006, at 8:45 PM, Larry Stone wrote: > This all made me curious. I'm just a user of Mailman on Mac OS X - no > development of any sort by me - so I'm good with 2.1.9 and Python > 2.3.5 on > 10.4.7 - but this topic made me look at the Python 2.5 package at > python.org. I finally figured out that the MacOS X python installer > installs > a new version in /usr/local/bin (as well as other places) separate > from the > Tiger provided version in /usr/bin. But how would you get mailman > to use the > user installed version? mailmanctl has #! /usr/bin/python at the > top which > will send it to the Tiger provided version. Of course you could modify > mailmanctl but that would be subject to being overwritten when a > new version > of mailman is installed. Or is that the way it would need to be done? I'm assuming you built Mailman from source. If so, run configure with --with-python=/usr/local/bin/python or put that python on your $PATH first, and Mailman will use that one. If you look at the source, you'll see that the #! line is actually @PYTHON@ which gets substituted by configure at build time. I forget exactly why, but the standard #! /usr/bin/env python invocation caused problems for people, so now we hardcode it via configure. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRRyCCnEjvBPtnXfVAQKYxAP+Ibg1hUcx6nE/7/hH8g6sbJ/m82+ApBl9 WkSNuR3e01gyLm0ogIu3npk0pQeflVrmsvjqSCP6iBw81gaO+oeo7zkrIyFag7Ck bqMCPIBkSDzblEUrueyTPhBvlpDlP4+vaaQSHe/EGDYSz+r/nzY/PJR6PU+lLgn4 c3SF+iHwsWU= =FnoT -----END PGP SIGNATURE-----

2 1

Patches in mandriva package
by Guillaume Rousse 29 Sep '06

29 Sep '06

Hello. I'm the mailman maintainer for mandriva Linux. I'm currently upgrading 2.1.8 to 2.1.9rc1, due to the recent security fixes therein. I'd like to use this occasion to drop a maximum of patches we still have: - is 2.1.9 still vulnearble to CVE-2005-3573 ? I didn't found any reference to it in the release notes, and the patch [1] still apply - the default build procedure is not suited to package building: it check target directory directly (which doesn't exist), and leave reference to package build root in python bytecode files. The patches [2] and [3] fixes those issues, maybe they could get integrated. - we have a patch for the embeded email module that just fix an encoding name [4]. I didn't found reference to a website or a standalone distribution of this module elsewhere. Could you please transmit the patch to its authors ? [1] CVE-2005-3573 fix: http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-2.1.6-CVE-… [2] buildroot references in bytecode fix: http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-2.1.6-CVE-… [3] buildroot check fix: http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-buildroot-…

6 11

Re: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
by Barry Warsaw 29 Sep '06

29 Sep '06

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sep 28, 2006, at 11:03 PM, Larry Stone wrote: > On 9/28/06 9:16 PM, Barry Warsaw at barry(a)python.org wrote: > > So, that leads to the question, is there any reason to install > python 2.5 > while running 2.1.9 or are we fine with 2.3.5 if we aren't doing > anything > else that would benefit from 2.5? If it ain't broke... :) You're probably fine with 2.3.5. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRRyg9nEjvBPtnXfVAQLEhgP/dGuGbr+fzIzaw7/GZbZFtbXW3DfUUbed luVI4Rhm4fgBjFbqh35d1fu12/qpIPbLWv4TKFVVV522obDJaXnY9o2sxF7H/bkF rUrXIEIVOe5usd1jq1btZlrgHrsZUOwZ64MBaGekXOhjp4XS/zTsSgex1clGeQSp ZBvXFiNSKYs= =ec3Q -----END PGP SIGNATURE-----

1 0

Re: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
by Barry Warsaw 28 Sep '06

28 Sep '06

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sep 28, 2006, at 2:13 PM, Stubbs Jeff wrote: > Barry - thanks for the advice > > I just wanted to report that Tiger (10.4.7 : ppc), Python 2.5 (from > the OS X installer), and Mailman 2.1.9 works perfectly. Install > went without a hitch. > > Stumbled a little, setting up virtual domains, but, I've been down > that road before, so I knew what to fix. Excellent, thanks for the feedback Jeff! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRRwYCnEjvBPtnXfVAQIsQAQAqlgmUL/iZuOa+Fxy0e8b/BCkrKOqvujX fHzLs3nvozDXVu2+FZ6bPOJ89nVdSchIJiUjpVfUvQxmSu3NKy8Dn6szfIgg1zuT Bk/2gkl8jMPlgh6aknxSkaNKMUJut8P74z/pjbfbBYgdm36AS9K5v1aLBvVhIx5E sB8vEZ4HP04= =vN/W -----END PGP SIGNATURE-----

1 0

Re: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
by Barry Warsaw 28 Sep '06

28 Sep '06

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm including mailman-developers on this message, because I want to discuss the issue of which Python versions to support. On Sep 27, 2006, at 11:04 AM, Stubbs Jeff wrote: > Got a question. I picked up another Mac, so I'm going to rebuild my > list server from scratch, using Postfix 2.3.3 and Mailman 2.1.9. The > default python install supplied by Apple is version 2.3.5. Last week > or so , I noticed the version 2.4.3 was available from the Python > website in an OS X installer. This morning, I noticed that the 2.5 > version was available. (whew, the developers must be sucking in the > coffee). > > All things being equal, would Mailman benefit from using the 2.5 > version as opposed to Apple's 2.3.5? > > The old server seemed to run fine, but I wouldn't mind upgrading > python, if it wasn't going to be problematic. My recommendation would be to run at least Python 2.4. Mailman 2.1.9 should run on Python 2.5, but earlier versions of Mailman won't. While we still claim to be able to run Mailman 2.1 on versions of Python back to 2.1, I believe we broke that claim in Mailman 2.1.9. Does anybody care? I would dearly love to drop support for Python 2.1 and Python 2.2 even in the Mailman 2.1 series. As I'm developing on OSX myself these days, I can't even build these earlier versions of Python and they haven't been supported by the PSF for years. In fact, Python 2.3 is no longer supported and I would even love to drop support for Python 2.3 in Mailman 2.1.x, although I know we can't. Then there is the question of what versions we support for Mailman 2.2, which is currently under development. Previously we've said we'll support Python 2.3 but I think we should revisit that decision. In summary my preferences would be: Mailman 2.1.x supported on Python 2.3, 2.4, and 2.5. Drop support for Python 2.1 and 2.2. We've done this accidentally in Mailman 2.1.9, so let's make it official. Mailman 2.2 supported on Python 2.4 and 2.5. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRRqeyHEjvBPtnXfVAQLYNwQAmaO7uxgkORkfy0ebOYU2Ez8jgSlcg4uT 8kz26FBgN0sKzp4hPN/kFK2Mqtao0FFdWxDfzUvb0f96o0moh71yqyUa3lfsoW0Y eTXXGMyACtjFasCYE2IcFuhHCeNHZlxu/yvlfznFQ6RXMi4c83AC/qeutY9o3Jl9 xTdNyISdvmc= =/bCq -----END PGP SIGNATURE-----

5 9

Re: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [8043] trunk/mailman/Mailman
by Tokio Kikuchi 28 Sep '06

28 Sep '06

Hi Barry and all, I've checked in this patch today. Barry, please fix or backout them if you don't like them. It looked like that MaildirRunner.py couldn't deal with the list name like 'mailman-users', so I rewrote it with a primitive method instead of using the re module. One problem: I manage a list called 'system-admin@...'; this can't go well with this method. Isn't it time to get rid of the -admin address in mailman-2.2? ;-) tkikuchi(a)users.sourceforge.net wrote: > Revision: 8043 > http://svn.sourceforge.net/mailman/?rev=8043&view=rev > Author: tkikuchi > Date: 2006-09-28 00:54:21 -0700 (Thu, 28 Sep 2006) > > Log Message: > ----------- > Here are the patches needed in order to create new lists on my test > installation. I've tested four cases of combination of > POSTFIX_STYLE_VIRTUAL_DOMAINS (Any/None) and USE_MAIL_DIR (Yes/No). > Also, I've added POSTFIX_VIRTUAL_SEPARATOR = '_at_' for unique mapping of > local part in the virtual-mailman/aliases files. > > Modified Paths: > -------------- > trunk/mailman/Mailman/Defaults.py.in > trunk/mailman/Mailman/MTA/Postfix.py > trunk/mailman/Mailman/MailList.py > trunk/mailman/Mailman/Queue/MaildirRunner.py -- Tokio Kikuchi, tkikuchi(a)is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/

2 1

Re: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [8036]trunk/mailman/Mailman/loginit.py
by Mark Sapiro 25 Sep '06

25 Sep '06

>Revision: 8036 > http://svn.sourceforge.net/mailman/?rev=8036&view=rev >Author: tkikuchi >Date: 2006-09-22 06:05:30 -0700 (Fri, 22 Sep 2006) > >Log Message: >----------- >I needed this for web ui to work. > >Modified Paths: >-------------- > trunk/mailman/Mailman/loginit.py > >Modified: trunk/mailman/Mailman/loginit.py >=================================================================== >--- trunk/mailman/Mailman/loginit.py 2006-09-21 18:23:48 UTC (rev 8035) >+++ trunk/mailman/Mailman/loginit.py 2006-09-22 13:05:30 UTC (rev 8036) >@@ -25,6 +25,7 @@ > import logging > > from Mailman.configuration import config >+config.load() > > FMT = '%(asctime)s (%(process)d) %(message)s' > DATEFMT = '%b %d %H:%M:%S %Y' > > Yes, but this is not the proper fix. I think the place to put config.load() for the web interface is in scripts/driver. I sent the following to Barry a few weeks ago, but to date have seen no reply. ----------- excerpt from message to Barry ------------------- Unfortunately, I neglected to report something else I saw because I didn't have time to figure it out. I'm still not sure about this, but here's the problem: [msapiro@msapiro ...f/test-mailman]$ scripts/post Traceback (most recent call last): File "scripts/post", line 48, in ? main() File "scripts/post", line 39, in main status = sys.modules[module_name].main() File "/cygdrive/f/test-mailman/Mailman/bin/post.py", line 44, in main loginit.initialize(propagate=True) File "/cygdrive/f/test-mailman/Mailman/loginit.py", line 119, in initialize handler = ReopenableFileHandler(os.path.join(config.LOG_DIR, logger)) AttributeError: 'Configuration' object has no attribute 'LOG_DIR' In this case, nothing has called config.load(). The same problem exists in the web interface. That can be fixed with --- f:/MM-Trunk/mailman/scripts/driver 2006-05-16 20:09:56.437500000 -0700 +++ scripts/driver 2006-08-29 09:51:32.375000000 -0700 @@ -68,6 +68,8 @@ log = None try: import paths + from Mailman.configuration import config + config.load() # When running in non-stealth mode, we need to escape entities, # otherwise we're vulnerable to cross-site scripting attacks. try: but this then leads to other places where we try to sort the set returned by Utils.list_names() I can work on cleaning this up, but I'd like confirmation that scripts/driver is the right place for config.load() and also what to do about the various scripts invoked by the mail wrapper. --------------------end excerpt------------------------- The problem with calling config.load() from loginit is I think it will override the loading of a alternate config which may have been specified as a command line option for those commands that support it (at least the ones that do logging too). -- Mark Sapiro <msapiro(a)value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 4

Re: [Mailman-Developers] Patches in mandriva package
by Bob Puff 21 Sep '06

21 Sep '06

(Reposting this, since Python.org's mail servers are having issues) > BTW, what do you think about changing the way we hold messages for > digests? E.g. instead of putting them in an mbox file, where it's > more difficult to skip bad messages, stick them in a queue-like > directory and pull them from there. Any messages that can't be read > and scrubbed would just be removed. Hey Barry, Do you mean as mailman builds the file during the day for the digest? If so, why would there be a bad message in there? If mailman's already processed a message and blasted it out to all the normal delivery people, why would there be a problem with it? Also, while we are sort of on the topic, would it be possible to put the message count back in the subject line? I recall there was a problem with the count being off from the actual number of messages... is this what you were getting at? One thing I hacked into a 2.0.x version was a "brief" digest - one that only gave the message subject list. The changes in ToDigest in 2.1.x were such that it made it a lot more difficult to make this happen. It would be great to get it back. I can revisit the code and see if I can come up with a patch, but other things need to stablize a bit. Bob

1 0