Mailman 3 July 2002 - Mailman-Developers

Off topic: a cautionary tale.
by Chuq Von Rospach 30 Jul '02

30 Jul '02

Just because it's too much fun to not pass along, and a useful cautionary tale. Fortunately, the person this happened to has a sense of humor about it... A friend of mine was working on his home web site. It was a bunch of stuff, back-ended by mySQL. He was, for instance, installing a search engine (using htDig) for the content, but it wasn't cooperating and he was trying to figure out why. Suddenly, the site goes dark. He can't log into it. He starts snooping, he can't get into MySQL. Uh, oh. (oh. No backups, either). The entire site implodes. He finally gives up, goes into the web logs to see what happens, thinking he got hacked. Well, not quite. One of the tools he installed was phpMyAdmin to administer the MySQL stuff. He installed it behind a .htaccess file like you're supposed to. But what he didn't realize was the .htaccess file wasn't working right, letting anything in. What got in was -- htDig, the search engine. Which happily follows all links, including, if you let it spider phpMyAdmin, the "delete this database" links. Including the database holding all of the MySQL configuration and account info. Which causes MySQL to die. Which... You get the picture. His search engine got into his database and deleted all of his data, because while it wasn't working, it COULD spider. And it got into an area it shouldn't have gotten into, even though it wasn't linked on the web site. How's that, you ask? How did HtDIG find it? Well -- one of the other things he'd added was log processing. Including referer tracking. And one of the links the logs showed was a referer link back to the phpMyAdmin pages. So by spidering the web log data, HtDIG found a link into phpMyAdmin, which was supposed to be password protected but wasn't, which let htDig become a DBA, which let HtDIG delete all of the data... When you build systems, do you think through the side effects of what you're doing? What are you missing? (and would you have ever figured out why this happened to you? I wonder if I would have.... ) Happy hacking... But be careful out there. -- Chuq Von Rospach, Architech chuqui(a)plaidworks.com -- http://www.chuqui.com/ The Cliff's Notes Cliff's Notes on Hamlet: And they all died happily ever after

3 3

Patch to add general regex filtering module, hooks to archive
by Laurence Berland 30 Jul '02

30 Jul '02

All, I've written and now tested a new Handler module, Filter.py, and I've added the various hooks it needs to filter on messages before they are archived. There are about 70billion different uses for this sort of thing, but the one I'm after is replacing a string (say :1:) with an image tag (say <img src="/icons/green.png">). My two questions are thus: 1. Is there a way to turn off the conversion of < and > to %lt; etc in the archive for a specific list? 2. Where should I submit my patches? They're against a cvs checkout from yesterday afternoon, so likely will patch completely cleanly into the code. Laurence

1 0

Re: [Mailman-Developers] Re: Integrating TMDA (was Re: Cute TMDA use)
by J C Lawrence 30 Jul '02

30 Jul '02

On Mon, 29 Jul 2002 10:47:02 -0600 Jason R Mastaler <Jason> wrote: > J C Lawrence <claw(a)kanga.nu> writes: >> Given that the TMDA python product carries almost all the needed >> payload, I wonder if a TMDA-plugin rooted in the TMDA product doesn't >> make more sense. > I'm comfortable with this as well, and don't mind making modifications > to TMDA that would make this easier if that proves necessary. <nod> Not speaking for Barry, but there are large advantages to leveraging code that other people maintain vs increasing the code and maintenance load of Mailman. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

4 11

[ mailman-Patches-444884 ] Integration of Mailman & htdig for archi
by noreply＠sourceforge.net 30 Jul '02

30 Jul '02

Patches item #444884, was opened at 2001-07-26 18:27 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=444884&group_i… Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Richard Barrett (ppsys) Assigned to: Nobody/Anonymous (nobody) Summary: Integration of Mailman & htdig for archi Initial Comment: This patch is applicable to Mailman 2.0.6 release that has had search enhancement patch 444879 patch installed - if your Defaults.py has the ARCHIVE_INDEXING_ENABLE and ARCHIVE_INDEXING_DISABLE in it then you've got that patch. It replaces earlier patches 401670 and 402423 and is mainly to correct some problems arising from fixes introduced into Mailman by bug fix releases since the 402423 patch. This patch integrates htdig with Mailman and provides: 1. per list search facility with a search form on the list's TOC page. 2. maintenance of privacy of private archives which requires the user to establish their credentials via the normal private archive access before any access via htdig is allowed. 3. a common base URL for both public and private archive access via htsearch results so that htdig indices are unaffected by changingan archive from private to public and vice versa. All access to archives via htdig is controlled by a new wrapped cgi- bin script called htdig.py. 4. a new cron activated script and extra crontab entry which runs htdig regularly to maintain the per list search indices. 5. automatic creation, deletion and maintenance of htdig configuration files and such. Beyond installing htdig and telling Mailman where it is via mm_cfg you do not have to do any other setup. Well not quite you do have to set up a single per installation symlink to allow htdig to find the automatically generated per list htdig configuration files. You probably want to run this patch as follows: cd <mailman 2.0.6 untarred and unzipped directory> patch -p1 < <this patch file> ---------------------------------------------------------------------- >Comment By: Richard Barrett (ppsys) Date: 2002-07-30 11:25 Message: Logged In: YES user_id=75166 htdig-2.0.13-0.1.patch is purely cosmetic to get no mumble application to MM 2.0.13 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-07-25 15:07 Message: Logged In: YES user_id=75166 Do not use htdig-2.0.12-0.1.patch there is an error in it. Use htdig-2.0.12-0.2.patch instead ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-07-25 14:10 Message: Logged In: YES user_id=75166 htdig-2.0.12-0.1.patch is a revised version of the patch that applies without complaint to MM 2.0.12. It also add a facility for adding site wide htdig configuration attributes to all list specific htdig configuration files. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-05-23 09:59 Message: Logged In: YES user_id=75166 htdig-2.0.11-0.1.patch is a revised version of the patch that is compatible with MM 2.0.11 This version removes an incompatibility with Python 2.2 which caused warning messages to be generated when any of the family cron/nightly_htdig scripts were run. Some guidance on file access permissions for some htdig database files needed by rundig have been added to installation notes. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-04-19 10:59 Message: Logged In: YES user_id=75166 htdig-2.0.10-0.1.patch is a revised version of the patch that is compatible with MM 2.0.10 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-04-08 17:46 Message: Logged In: YES user_id=75166 htdig-2.0.9-0.1.patch is a revised version of the patch that is compatible with MM 2.0.9 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-03-06 16:22 Message: Logged In: YES user_id=75166 htdig-2.1cvs-20020306.patch is a revised version of the patch that is compatible with the code published in mailman CVS on sourceforge as 12:30 GMT 6 Mar 2002 Known deficiency is that the non-English versions of files under $build/templates still contain text in English and need translations I cannot do. Also the necessary pygettext activity and subsequent translations in files under $build/messages remain to be done. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-12-17 16:56 Message: Logged In: YES user_id=75166 htdig-2.1cvs-20011217.patch is a revised version of the patch that is compatible with the code published in mailman CVS on sourceforge as 11:50 GMT 17 Dec 2001 The only known deficiency is that the non-English versions of files under $build/templates still contain text in English and need translations I cannot do. Also the necessary pygettext activity and subsequent translations in files under $build/messages remain to be done. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-12-13 16:58 Message: Logged In: YES user_id=75166 htdig-2.1a3-0.1.patch is a revised version of the patch that is compatible with the code published in mailman-2.1a3.tgz on sourceforge. The only known deficiency is that the non-English versions of files under $build/templates still contain text in English and need translations I cannot do. Also the necessary pygettext activity and subsequent translations in files under $build/messages remain to be done. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-11-28 17:33 Message: Logged In: YES user_id=75166 The htdig-2.0.8-0.1.patch version of the patch resolves a problem that can arise with htdig indexing if the web_page_url for a list uses other than the http addressing (some folks want to use https). While specified as for MM 2.0.8 the revised patch should work OK with 2.0.7, 2.0.6 and probably back as far as 2.0.3. If you do not have the requirement for using other than http addressing in you lists web_page_urls it probably isn't worth the trouble of upgrading to this patch level. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-11-28 11:08 Message: Logged In: YES user_id=75166 This patch should also apply without problems to MM 2.0.8 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-11-27 12:00 Message: Logged In: YES user_id=75166 This patch should also apply without problems to Mm 2.0.7 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-11-09 11:54 Message: Logged In: YES user_id=75166 The htdig-2.0.6-03.patch version of the patch makes some previously hard-coded things configurable and enhances the capability to run the htdig searches and indexing on a different machine to the one delivering Mailman and Mailman's web UI. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=444884&group_i…

1 0

[ mailman-Patches-444879 ] Archive indexer control to improve index
by noreply＠sourceforge.net 30 Jul '02

30 Jul '02

Patches item #444879, was opened at 2001-07-26 18:01 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=444879&group_i… Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Richard Barrett (ppsys) Assigned to: Nobody/Anonymous (nobody) Summary: Archive indexer control to improve index Initial Comment: This patch is applicable to Mailman 2.0.6 release and supercedes ealier patches 401669 and 402422. This patch should improve the quality of search results returned by search engines such as htdig (http://www.htdig.org) where the seach engine's index builder responds to strings embedded in the html pages that are the subject of the indexing. The changes in this patch: 1. allow strings for enabling and disabling indexing to be defined in mm_cfg.py. 2. embeds those strings in the pages generated as the html version of a list's archive. By default nothing in the html changes. To get the desired effect, you must define ARCHIVE_INDEXING_ENABLE and ARCHIVE_INDEXING_DISABLE in mm_cfg.py You probably want to run this patch as follows: cd <mailman 2.0.6 untarred and unzipped directory> patch -p1 < <this patch file> See also the associated patch for integrating the htdig search software with mailman's internal archiver ouput. ---------------------------------------------------------------------- >Comment By: Richard Barrett (ppsys) Date: 2002-07-30 11:23 Message: Logged In: YES user_id=75166 indexing-2.0.13-0.1.patch is purely cosmetic to get no mumble application to MM 2.0.13 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-07-25 14:11 Message: Logged In: YES user_id=75166 indexing-2.0.11-0.1.patch should apply without problems to MM 2.0.12 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-05-23 09:50 Message: Logged In: YES user_id=75166 indexing-2.0.11-0.1.patch is a revised version of the patch that is compatible with MM 2.0.11 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-04-19 10:53 Message: Logged In: YES user_id=75166 indexing-2.0.9-0.1.patch should apply without problems to MM 2.0.10 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-04-08 17:43 Message: Logged In: YES user_id=75166 indexing-2.0.9-0.1.patch is a revised version of the patch that is compatible with MM 2.0.9 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2002-03-06 16:14 Message: Logged In: YES user_id=75166 indexing-2.1cvs-20020306.patch is a revised version of the patch that is compatible with the code published in mailman CVS on sourceforge as 12:30 GMT 6 Mar 2002. Corrects problem noted or 5 Mar 2002 by nobody ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2002-03-05 21:41 Message: Logged In: NO When applying this patch I get an error with Hunk 1 and Defaults.py is not updated. This happens with the a clean download of the latest cvs installation (5 Mar 2002). Any ideas what the problem is? ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-12-17 16:53 Message: Logged In: YES user_id=75166 indexing-2.1cvs-20011217.patch is a revised version of the patch that is compatible with the code published in mailman CVS on sourceforge as 11:50 GMT 17 Dec 2001 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-12-13 16:48 Message: Logged In: YES user_id=75166 indexing-2.1a3-0.1.patch is a revised version of the patch that is compatible with the code published in mailman-2.1a3.tgz on sourceforge. ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-11-28 11:07 Message: Logged In: YES user_id=75166 This patch should also apply without problems to MM 2.0.8 ---------------------------------------------------------------------- Comment By: Richard Barrett (ppsys) Date: 2001-11-27 12:03 Message: Logged In: YES user_id=75166 This patch should also apply without problems to MM 2.0.7 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=444879&group_i…

1 0

Re: [Mailman-Developers] Re: Integrating TMDA (was Re: Cute TMDA use)
by J C Lawrence 30 Jul '02

30 Jul '02

On Mon, 29 Jul 2002 21:02:25 -0600 Jason R Mastaler <Jason> wrote: > barry(a)python.org (Barry A. Warsaw) writes: >> If what you say is the case -- builtin support isn't terribly useful >> without the shared whitelist -- then maybe it does make sense to >> defer until a later Mailman version. > Some users will not want to use a shared whitelist, but in my > experience, the vast majority will. I predict that if you add the > feature without this sub-feature, it will become a nagging FAQ. At its root its a privacy and entrance notification problem. Its not so much that the whitelist is private, but that entering it for discrete list populations is a hard edged and published (to the list owner or subscriber base) event, as distinct from the global whitelist. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] Re: Cute TMDA use
by J C Lawrence 30 Jul '02

30 Jul '02

On Mon, 29 Jul 2002 20:55:27 -0600 Jason R Mastaler <Jason> wrote: > barry(a)python.org (Barry A. Warsaw) writes: >> So if we /don't/ build the functionality into MM2.1 (as Chuq is so >> sensibly arguing :) is there anything we need to do to implement the >> whitelisting TMDA fronted MM that is being proposed here? > I think we just need documentation. I've only ever combined MM and > TMDA under qmail, and it's quite easy. Given .qmail-list which posts > to the list, you just need to edit and add the following to the top of > the file: I have TMDA+Mailman (with MIME filters and other niceties) working under Exim. Doing the same under Postfix is a bit of a bitch due to effective UID/GID problems for executing pipe, but can be solved with the vtable trick detailed in the Postfix README. If you decide _NOT_ to roll TMDA-like features into v2.1 I'll continue with my project and should post documentation, scripts, and other relevant bits here in the next week or so. > Non-qmail MTAs aren't as flexible, so perhaps JC can comment on his > experiences? Mostly its as commented here and on the TMDA lists (macro limitations etc). There's a little problem with effective UID/GID for gaining access to the Mailman config.db, but that's a trivial problem under Exim if you follow the Exim HowTo as a base model. > In any case, I think the necessary changes (if any) would go into TMDA > and not MM. So, perhaps you are done. Yeah, plus a little plugin logic (eg web UI). -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Cute TMDA use
by J C Lawrence 30 Jul '02

30 Jul '02

You can front a list with TMDA, pointing TMDA at your list's subscriber base and a local whitelist. TMDA then passes all subscriber mails straight thru to the list, but mail from non-subscribers is held until confirmed (at which point you can whitelist or not). The TMDA lists do this to decent effect. Nice side effects: No more problems with posters from non-subscribed addresses. They have a subscription to _GET_ mail and to define an allowed posting address. They can (trivially) define additional allowed posting addresses by just posting from them and confirming the post to get on the whitelist. You're now essentially back in the old days of the 1980s when we could safely run open lists. Almost all SPAM disappears. They don't confirm. Ditto viruses. Well some viruses might learn confirming down the road, but none do now and the confirmation steps can be trivially changed. -- J C Lawrence - ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

5 25

[ mailman-Bugs-588373 ] bug with 2.1b2 (something with cookies?)
by noreply＠sourceforge.net 29 Jul '02

29 Jul '02

Bugs item #588373, was opened at 2002-07-29 23:33 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=588373&group_i… Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Paul Marshall (paulmarshll) Assigned to: Nobody/Anonymous (nobody) Summary: bug with 2.1b2 (something with cookies?) Initial Comment: This is hopefully my third and final stupid question involving mailman. I have mailman 2.1b2 setup and mostly working (I can create lists, and post, etc...). However whenever I try to log into a list via the admin page to edit/delete/whatever to it, it comes up with this bug: Traceback (most recent call last): File "/var/mailman/scripts/driver", line 82, in run_main main() File "/var/mailman/Mailman/Cgi/admin.py", line 82, in main cgidata.getvalue('adminpw', '')): File "/var/mailman/Mailman/SecurityManager.py", line 208, in WebAuthenticate print self.MakeCookie(ac, user) File "/var/mailman/Mailman/SecurityManager.py", line 223, in MakeCookie c = Cookie.SimpleCookie() AttributeError: 'module' object has no attribute 'SimpleCookie' Is this a permissions problem? something to do with creating cookies...Honestly, I have no idea... I've searched some bug databases and most likely passed right over it, however I can't seem to find anything dealing with this error. Thanks for all the help. Paul ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=588373&group_i…

1 0

Re: [Mailman-Developers] Re: Integrating TMDA (was Re: Cute TMDA use)
by Dan Mick 29 Jul '02

29 Jul '02

> JRM> This is surely better than nothing, but the shared list is a > JRM> pretty integral feature IMO. I don't think I'd use the MM > JRM> built-in support until this was available in some form. > > I have to defer to you and others who have actually started using this > arrangement. If what you say is the case -- builtin support isn't > terribly useful without the shared whitelist -- then maybe it does > make sense to defer until a later Mailman version. If MM+TMDA does > the job, then that's probably good enough. Why does it matter? Doesn't that just cause the user to go through N confirmations per server instead of 1, but since the confirmations are nearly free, who cares?... Probably missing something again. But I'm on very few lists that share a server, so I'd expect the normal behavior of "one confirm per list per alien address" behavior anyway (I think the only multilist server I communicate with is python.org).

3 2