Just thought I'd point you'll to something I've done. While not completely integrated yet, I've got a preliminary system where mailman is used as the mailing list system, and namazu is used to index and search the archives.
You can see it at work at http://mm.ilug-bom.org.in/mailman/search/linuxers/ and http://mm.ilug-bom.org.in/mailman/search/life/
If you think it's a good idea, and useful, I could tell you how I did it.
Am still working on getting automatic index updates everytime the archive is updated.
Philip
-- I can't believe I ate the whole thing.
-- Homer Simpson
The FrontVisit my webpage at http://www.ncst.ernet.in/~philip/ Read my writings at http://www.ncst.ernet.in/~philip/writings/
MSN philiptellis Yahoo! philiptellis AIM philiptellis ICQ 129711328
Sometime on Nov 29, Jon Carnes assembled some asciibets to say:
Looks cool. How did you do it, and what are the current problems with the way it's setup/working?
How I did it:
Namazu
There's this product called namazu (http://www.namazu.org/). It's a search engine that uses a full text index. Can index local paths, not remote paths.
Put the namazu cgi in the mailman/cgi-bin/ directory, and called it search. Also had to hack namazu a bit to get it to read the index from the PATH_INFO from the url (/linuxers/, /life/, etc.).
Also changed the namazu templates to look like mailman's listinfo pages.
The last thing was to index the pages.
So far, no changes were required to mailman, but the products aren't fully integrated yet.
The only thing that isn't done is automatic reindexing everytime a new message is added. I thought it may make sense to do the reindexing at the same time that a new archive is created (ie, weekly or monthly), but couldn't find the place to put that.
The next best idea I had was to put it the reindexing as a cron job. Problem here is that there is a chance that the reindexer would run while a new message was only partially written to disk.
Mailman
I also had to add either a link to the search page on the listinfo page, and a search box or link on the archive pages. While this isn't up on the urls yet, it is done by hacking Mailman/Archiver/HyperArch.py.
The real problem is getting the link to show up only on pages that have actually been indexed.
How this can be fully integrated
A config variable stating whether the list needs to be indexed for searching, and based on that, these links can be written to the archive pages. The reindexer can be run through mailman or through cron, depending on what others think is a better solution.
Philip
-- Darth Vader: The force is with you young Skywalker, but you are not a Jedi yet.
Visit my webpage at http://www.ncst.ernet.in/~philip/ Read my writings at http://www.ncst.ernet.in/~philip/writings/
MSN philiptellis Yahoo! philiptellis AIM philiptellis ICQ 129711328
Sometime on Nov 29, Jon Carnes assembled some asciibets to say:
Looks cool. How did you do it, and what are the current problems with the way it's setup/working?
How I did it:
Namazu
There's this product called namazu (http://www.namazu.org/). It's a search engine that uses a full text index. Can index local paths, not remote paths.
Put the namazu cgi in the mailman/cgi-bin/ directory, and called it search. Also had to hack namazu a bit to get it to read the index from the PATH_INFO from the url (/linuxers/, /life/, etc.).
Also changed the namazu templates to look like mailman's listinfo pages.
The last thing was to index the pages.
So far, no changes were required to mailman, but the products aren't fully integrated yet.
The only thing that isn't done is automatic reindexing everytime a new message is added. I thought it may make sense to do the reindexing at the same time that a new archive is created (ie, weekly or monthly), but couldn't find the place to put that.
The next best idea I had was to put it the reindexing as a cron job. Problem here is that there is a chance that the reindexer would run while a new message was only partially written to disk.
Mailman
I also had to add either a link to the search page on the listinfo page, and a search box or link on the archive pages. While this isn't up on the urls yet, it is done by hacking Mailman/Archiver/HyperArch.py.
The real problem is getting the link to show up only on pages that have actually been indexed.
How this can be fully integrated
A config variable stating whether the list needs to be indexed for searching, and based on that, these links can be written to the archive pages. The reindexer can be run through mailman or through cron, depending on what others think is a better solution.
Philip
-- Darth Vader: The force is with you young Skywalker, but you are not a Jedi yet.
Visit my webpage at http://www.ncst.ernet.in/~philip/ Read my writings at http://www.ncst.ernet.in/~philip/writings/
MSN philiptellis Yahoo! philiptellis AIM philiptellis ICQ 129711328
participants (2)
-
Jon Carnes -
Philip S Tellis