Search tool for Mailman Archives
Hello;
I am trying to implement a search tool for Mailman Archives.
In our system, there are more than 400 lists, and the size of their archives vary, since there are more than 10 years old lists addition to new lists.
There are several alternatives. htdig, xapian etc.
Which one do you recommend?
Is there a practical tool which I can just install and implement, or is there an additional patch need?
Thank you very much. Evrim AKMAN
On Thu, Dec 9, 2010 at 10:29 AM, Liste Yoneticisi listeyon@metu.edu.trwrote:
Hello;
I am trying to implement a search tool for Mailman Archives.
In our system, there are more than 400 lists, and the size of their archives vary, since there are more than 10 years old lists addition to new lists.
There are several alternatives. htdig, xapian etc.
There is no harm in installing and testing all of them until you find one you are happy with.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223
Damn!!
On Thu, Dec 9, 2010 at 2:29 AM, Liste Yoneticisi listeyon@metu.edu.tr wrote:
In our system, there are more than 400 lists, and the size of their archives vary, since there are more than 10 years old lists addition to new lists.
There are several alternatives. htdig, xapian etc.
If you find one that allows you to still use the mailman login system so that only members of a list can view the archives, please let me know. I looked at a few archive searchers, but it appears they all either make the list archives public, or you can do a htaccess thing which will require you to change the password every time somebody leaves the list and tell all the current members the new password, etc.
-- http://www.linkedin.com/in/paultomblin http://careers.stackoverflow.com/ptomblin
I meant private archives, which can be reached after submiting an e-mail and password.
09.12.2010 14:49, Paul Tomblin yazmış:
On Thu, Dec 9, 2010 at 2:29 AM, Liste Yoneticisilisteyon@metu.edu.tr wrote:
In our system, there are more than 400 lists, and the size of their archives vary, since there are more than 10 years old lists addition to new lists.
There are several alternatives. htdig, xapian etc.
If you find one that allows you to still use the mailman login system so that only members of a list can view the archives, please let me know. I looked at a few archive searchers, but it appears they all either make the list archives public, or you can do a htaccess thing which will require you to change the password every time somebody leaves the list and tell all the current members the new password, etc.
On Thu, Dec 09, 2010 at 07:49:47AM -0500, Paul Tomblin wrote:
I looked at a few archive searchers, but it appears they all either make the list archives public, or you can do a htaccess thing which will require you to change the password every time somebody leaves the list and tell all the current members the new password, etc.
One could, always, re-implement the login system &c, but there's still the scope that authentication methods probably won't prevent (authorized) people from taking copies of the data, for example.
-- "Tony Blair has made 'morale boosting' visits to the wives of servicemen serving in the Gulf." -- BBC News
On Thu, 2010-12-09 at 07:49 -0500, Paul Tomblin wrote:
If you find one that allows you to still use the mailman login system so that only members of a list can view the archives, please let me know. I looked at a few archive searchers, but it appears they all either make the list archives public, or you can do a htaccess thing which will require you to change the password every time somebody leaves the list and tell all the current members the new password, etc.
About three years ago I worked out a complete system for integrating Namazu with Mailman, and it's still successfully in use on my production servers. You can find it at http://www.fmp.com/namazu. This is the guts of the system. I've subsequently added the ability to mailman to read the index.html file in the Pipermail archive in a PHP context which allows the use of an accessory list of lists which are authorized to use the namazu search, while others don't offer this ability.
There are a few glitches in this, since the integration of namazu and mailman is imperfect by nature. Keep in mind that all this was written three years ago, for a previous version of Mailman. Additionally, Namazu's date sort is, or was, dependent on the file dates in the pipermail structure, rather than parsing dates from files, so if you move your archives to a new system the date sort becomes quite broken if the file creation dates are munged. This is a namazu issue. The author of Namazu struggles with English, and I have no knowledge of Japanese, so I wasn't able to get very far in resolving this.
I put quite a lot of work into this project, but have moved on and am doing other things, so if someone with the knowledge and skills to do so wants to grab it and run with it, updating it and improving it, please have at. I've forgotten a lot of the details that went into the design of this, although the project contains a lot of documentation which I wrote at the time, so if you have problems, read the code and figure it out if you can. Don't ask me for support. Just make any necessary changes or improvements, send them to me, and I'll post them to the collection on the web server.
-- Lindsay Haisley | "In an open world, | PGP public key FMP Computer Services | who needs Windows | available at 512-259-1190 | or Gates" | http://pubkeys.fmp.com http://www.fmp.com | |
Lindsay Haisley wrote:
Additionally, Namazu's date sort is, or was, dependent on the file dates in the pipermail structure, rather than parsing dates from files, so if you move your archives to a new system the date sort becomes quite broken if the file creation dates are munged.
See the script at http://www.msapiro.net/scripts/update_archive_mtime (mirrored at http://fog.ccsf.edu/~msapiro/scripts/update_archive_mtime) for a way to set the file times by parsing dates from the files.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Paul Tomblin wrote:
If you find one that allows you to still use the mailman login system so that only members of a list can view the archives, please let me know.
Lindsay Haisley has replied with one. See the patches at https://bugs.launchpad.net/mailman/+bug/266553 and https://bugs.launchpad.net/mailman/+bug/266554 for another.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 9 Dec 2010, Liste Yoneticisi wrote:
Hello;
I am trying to implement a search tool for Mailman Archives.
I've always liked swish for indexing web sites and mailing list archives
========================================================== Chris Candreva -- chris@westnet.com -- (914) 948-3162 WestNet Internet Services of Westchester http://www.westnet.com/
Liste Yoneticisi wrote:
Is there a practical tool which I can just install and implement, or is there an additional patch need?
See the FAQ at http://wiki.list.org/x/MoA9
For public archives you could just install a search engine along side Mailman or use one of the public archiving sites with search capability.
To privately search private archves or to integrate searching (e.g. a search form on the archive TOC pages) requires patches in Mailman 2.1.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (7)
-
Adam McGreggor -
Christopher X. Candreva -
Lindsay Haisley -
Liste Yoneticisi -
Mark Sapiro -
Odhiambo Washington -
Paul Tomblin