Mailman 3 September 2005 - Mailman-Developers

structural problem with MemberAdapter
by John Dennis 12 Oct '05

12 Oct '05

Good news, bad news: good news is I've implemented a LDAP based membership adapter that implements the complete MemberAdapter interface (the previous LDAP adapter was incomplete and limited in application). In stand alone testing my adapter seems to work quite nicely. Skip to the last paragraph if you don't need or want the details. Now the bad news, I'm having trouble integrating it because of either structural problems with adapters or a lack of understanding of how adapters are supposed to work. I'm looking for guidance or insight. Here are the issues I've run into: * Adapters are discovered by the presence of extend.py in the list's directory. How does extend.py get into the list directory? The only way I could figure was manual copying after list creation, this isn't a viable mechanism for a running site. Also, while I can see the value of per list adapters that seems like a niche case, a more common scenario would be every list resident at a site would use the same adapter module. To solve this problem I added code in the MailList constructor to look for a mm_cfg variable defining a global adapter module. But list creation/instantiation is a two step process, MailList object construction in __init__, followed by MailList.Create(). The adapter is not loaded in __init__ unless a name is provided, which is only done after Create() has been called. Most importantly an adapter cannot correctly initialize itself until the work of Create() is complete. This is very important, but more on this later. This sequence in TestBase.py leaves the adapter unloaded and uninitialized, it does not work. mlist = MailList.MailList() mlist.Create('_xtest', 'test(a)dom.ain', 'xxxxx') This idiom below which is found in most of the code does work because the list has already been created, but now we're back to the chicken and egg problem of when does extend.py get into the list directory. mlist = MailList.MailList(listname, lock=0) This is a good moment to go back to the issue of per list adapter initialization (or rather the lack of it). After the mlist is instantiated as above it might very well call an adapter method, for example mlist.addNewMember(). But the adapter has never been called as part of list creation. If the adapter must create entities in a backend database to service the list data it has never been given that opportunity. Thus the adapter is dependent on asking the question "does this list exist in my database, if not create it" every time the adapter object is instantiated, this is not only a big performance problem but it completely fails with the calling sequence in TestBase.py. If you've followed along so far pat yourself on the back :-) here's the payoff :-) Summary: -------- Unless I'm out to lunch with a horrible misunderstanding I propose 3 changes: 1) Provide a site wide variable that specifies an adapter for every list which is overridden by the presence of extend.py in the list directory. If the site wide adapter is used then MailList.__init__() loads it and calls an initialization function in the module (extend()??, __init__()??) 2) At the end of MailList.Create() it should test for the existence of an adapter and if so it should attempt to call the adapters Create() method. 3) Just as the adapter is not informed of list creation is is not informed of list destruction either, calls need to be added to invoke Remove() as well. I'm happy to provide patches for all this, I'm not interested in suggesting anyone else do more work, I'm mostly interested in knowing if I'm on the right track or if I have a misunderstanding. PostScript (or a few more questions): ------------------------------------- Now after having fully implemented the adapter interface I have to admit I don't really understand what its buying you over the existing OldMemberAdapter. My initial thought was to capitalize on existing user information at a site, but given the way mailman data is structured (a set of lists, each list may contain both local and unknown foreign users, and user properties are per list) then there seems to be little value in intermingling site user data and mailman list data. Also, it was not clear how an adapter might implement just a subset of the methods via inheritance, I suppose it would copy the function pointers from the mlist._memberadapter into its own methods before resetting mlist._memberadapter to itself. yes/no? -- John Dennis <jdennis(a)redhat.com>

2 2

Patch submitted
by Bjorn-Erik Townsend 25 Sep '05

25 Sep '05

Hello all, I submitted a small patch just now for Bug 1212066 (lack of date header in new list notification emails), a cosmetic issue which was annoying me (and setting off my spam filters). The website suggests sending an email to this list to let the developers know when a patch is submitted, so I thought I'd send this along. Here's a link to the patch info on Sourceforge: http://sourceforge.net/tracker/index.php?func=detail&aid=1301983&group_id=1… I have it running in production on a site running Mailman 2.1.6+Courier-MTA and it works as expected. Thanks for a fine list manager! -- Bjorn Townsend | bjorn(a)sourcelabs.com

2 1

Patch: Select language(s) when install
by Tokio Kikuchi 22 Sep '05

22 Sep '05

Hi Developers, I've uploaded a patch to select languages when installing mailman. Here is a excerpt from SF tracker: ------------------------------------------------------- Subject: [ mailman-Patches-1298355 ] Select language(s) when install Date: Wed, 21 Sep 2005 22:35:37 -0700 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1298355&group_… Initial Comment: While the number of translated languages is growing, most users are using only a few languages. With this patch, the site admin can select his/her required languages by an option to the configure script by `--with-languages' and can considerably reduce the install size. This patch is for 2.2.0a0 (most recent CVS as of writing) but also applicable to 2.1.6 except that you may have to regenerate configure script by invoing autoconf command. Patch usage: % cd mailman (src directory) % patch -p0 < /path/to/install_selected_language.patch.txt Configure usage: % ./configure --with-languages="ja fr" ... en+ja+fr % ./configure --with-languages="" ... all available languages % ./configure .... all available languages % ./configure --with-languages="none" ... English only Japanese and Korean codecs are no longer installed by `make install' but availability is checked during the configure run and urge to install if they are absent when CJK languages are included in the language list. I hope skilled developers can review this patch before I check in to the CVS. -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/

1 0

Mal Formed MIME post leaked through to list
by Mark Sapiro 18 Sep '05

18 Sep '05

I just had a problem on a Mailman 2.1.5 list although I think it's the Python email library - Python is 2.3.3. A mal formed MIME message was posted to a list. The message was much larger than max_message_size yet it wasn't held, and several parts came through that weren't in pass_mime_types. The basic post was multipart/mixed with a multipart/alternative sub part, a message/rfc822 sub part and the final text/plain msg_footer. The message/rfc822 sub part was multipart/mixed with 3 subparts of type multipart/alternative, application/pdf and multipart/appledouble. The problem with the MIME structure is that the boundary for the multipart/alternative and multipart/appledouble sub parts of the multipart/mixed message/rfc822 was identical to the boundary of the multipart/mixed part. I suspect the original message/rfc822 attached message was malformed, but it could have been broken in the attaching process. The original attached message was created by User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.0.6 and the post was sent by Yahoo mail. I suspect what happened is that the Python email library saw the end of part boundary for the second multipart/alternative part and treated it as the end of the message/rfc822 part since the boundary was the same. Thus the big parts didn't get counted in the message size nor did they get filtered by content filtering. Is this analysis correct? Is it fixed in later versions of the email library? Here is an annotated copy of the received post with all content and non-relevant headers removed. Received: from [63.201.34.79] by web81402.mail.yahoo.com via HTTP; Fri, 16 Sep 2005 16:32:37 PDT MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="0-95218806-1126913557=:91474" Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 X-BeenThere: gpc-talk(a)grizz.org X-Mailman-Version: 2.1.5 -------------------above from the headers of the received post --0-95218806-1126913557=:91474 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit ------------------above are first part headers - I think this was originally multipart/alternative and a text/html part was stripped --0-95218806-1126913557=:91474 Content-Type: message/rfc822 ------------------part headers from the attached message/rfc822 part User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.0.6 Mime-version: 1.0 Content-type: multipart/mixed; boundary="MS_Mac_OE_3209732028_3749601_MIME_Part" Content-Length: 175299 ------------------from the headers of the attached message --MS_Mac_OE_3209732028_3749601_MIME_Part Content-type: multipart/alternative; boundary="MS_Mac_OE_3209732028_3749601_MIME_Part" ---------------------part headers for multipart/alternative sub part of attached message. Note that the boundary is the same as that of the containing multipart/mixed part. --MS_Mac_OE_3209732028_3749601_MIME_Part Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit -------------------------part headers for text/plain alternative --MS_Mac_OE_3209732028_3749601_MIME_Part-- -----------------------end of multipart/alternative part but also looks like end of multipart/mixed part. I think a text/html part may have been stripped, but the multipart/alternative part isn't collapsed. --MS_Mac_OE_3209732028_3749601_MIME_Part Content-type: application/pdf; name="Recruitment.pdf"; x-mac-creator="4341524F"; x-mac-type="50444620" Content-disposition: attachment Content-transfer-encoding: base64 -----------------------------------another sub part of multipart/mixed - should have been filtered --MS_Mac_OE_3209732028_3749601_MIME_Part Content-type: multipart/appledouble; boundary="MS_Mac_OE_3209732024_3737509_MIME_Part" -----------------------------------another sub part of multipart/mixed - should have been filtered. Still same boundary --MS_Mac_OE_3209732024_3737509_MIME_Part Content-type: application/applefile; name="High Sierra Trip 2003 Me" Content-transfer-encoding: base64 Content-disposition: attachment ----------------------sub part of multipart/appledouble should have been filtered --MS_Mac_OE_3209732024_3737509_MIME_Part Content-type: image/jpeg; name="High Sierra Trip 2003 Me"; x-mac-creator="6F676C65"; x-mac-type="4A504547" Content-disposition: attachment Content-transfer-encoding: base64 ----------------------sub part of multipart/appledouble should have been filtered --MS_Mac_OE_3209732024_3737509_MIME_Part-- ----------------------- end of multipart/appledouble --MS_Mac_OE_3209732028_3749601_MIME_Part-- ------------------------ end of multipart/mixed --0-95218806-1126913557=:91474 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------ part headers for list footer --0-95218806-1126913557=:91474-- ------------------------ end of outermost message -- Mark Sapiro <msapiro(a)value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 5

Dealing with DomainKeys and DKIM
by Joe Peterson 12 Sep '05

12 Sep '05

I've recently been testing DomainKeys (http://antispam.yahoo.com/domainkeys) and DKIM (which is supposedly a merging of DomainKeys with Cisco's scheme. I am using dk-milter and dkim-milter with sendmail. What this does is add two header lines to outgoing email that allow the receiver to determine the authenticity of the sender... Anyway, since I run a Mailman system too, I figured this might be a problem. Indeed it is, since the header lines get passed through, and when the check is done, it indicates a failure. DomainKeys recommends mail lists regenerate the keys rather than pass them through. What I tried was pretty simple: Mailman doesn't have to deal with these things itself, but if it strips the old keys from the header, the keys will be regenerated on the way out by the MTA, thereby making the whole process clean. So the receiver of the email can at least verify that the mail came from the host hosting Mailman. I suppose Mailman could also check email on the way in for valid keys if it wanted, but that's another subject... I patched Handlers/Cleanse.py as follows: 49a50,55 > # JGP: Remove all "DomainKeys" type header lines, since we want these > # to be regenerated by the MTA when this message is sent out. We need > # to let new such keys be generated because Mailman alters the message, > # and the old keys would be seen as invalid by the receiver. > del msg['domainkey-signature'] > del msg['dkim-signature'] I wanted to pass this by the developers here and see if: 1) This is a reasonable thing to do (or maybe have an option, or even a way to strip selected headers in the config?) 2) If this is the right place to do it. -Thanks, Joe

3 7

LDAP-based list
by Patrice Albaret 05 Sep '05

05 Sep '05

Hi, I m currently using the LDAP-connector developped by Martin : http://webserver.offal.homelinux.org/LDAPMemberAdaptor/V3.0/LDAPMemberAdapt… I've modified it for being usable with very large LDAP directories (rebuilding an adress instead of making an ldap request for each member). That's not the point. My problem : the member list seems just right through the web interface or even the bin/list_members script at all time (updating when the ldap group is updated etc...). But in fact, the changes in the delivery list are effectives only after a restart of the mailman processes ... The list seems cached... maybe in the config.pck file ... Any hint to solve this problem ? Regards P

3 2

Public archives
by Bob Puff＠NLE 02 Sep '05

02 Sep '05

Hi Gang, I've got one mailing list for a customer who wants public archives. The box hosts many sites, and he wants his domain in the URL for the archives. This isn't a problem for private archives, but there doesn't seem to be a way to set the URL for public archives. I recall we hashed this one over a while ago, but was there ever a solution? Bob

2 1

Mailman 2.X CVS MAIN is back
by Tokio Kikuchi 01 Sep '05

01 Sep '05

Hi Developers, I've done backporting the changes in Release_2_1-maint into the MAIN branch which was not updated more than a year and this has caused confusion for the people who want to try CVS checkout. For I was a little bit tired of maintaining two branches (or of the pressure to do so) I want to check in to the MAIN branch only hereafter, although some serious security bug fix (if any) should be committed in the 2.1 branch. I believe the MAIN CVS checkout is now ready to be tested but plese let me know if you notice something. Cheers, -- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/

11 16

Mailman-Developers September 2005