What is the simplest way to add a searchbox for the archives?
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
Odhiambo Washington writes:
What is the simplest way to add a searchbox for the archives?
If you're talking about about Mailman 2's Pipermail, you need to install and configure a separate application. I think HT:Dig is most popular, but I've only used Namazu, which worked well for me. There's a detailed FAQ on it: https://wiki.list.org/x/4030514, which says there are maintained branches on LaunchPad with patches to integrate HT:Dig. That may be easiest. Mark will know more when he gets back.
Another formerly somewhat popular approach is to substitute MHonArc for Pipermail, although that software is quite old now, and I'm not sure about its security status.
Steve
On Sun, Sep 11, 2022 at 11:01 AM Stephen J. Turnbull < stephenjturnbull@gmail.com> wrote:
Odhiambo Washington writes:
What is the simplest way to add a searchbox for the archives?
If you're talking about about Mailman 2's Pipermail, you need to install and configure a separate application. I think HT:Dig is most popular, but I've only used Namazu, which worked well for me. There's a detailed FAQ on it: https://wiki.list.org/x/4030514, which says there are maintained branches on LaunchPad with patches to integrate HT:Dig. That may be easiest. Mark will know more when he gets back.
Another formerly somewhat popular approach is to substitute MHonArc for Pipermail, although that software is quite old now, and I'm not sure about its security status.
Steve
Hi Steve,
Great that you are using namazu. I hope you can help me with the bits I am stuck about using it.
I have found a detailed HOWTO http://bakacsin.ki.iif.hu/~kissg/project/mailman+namazu/#nmzproc for installing it on Linux, but I use FreeBSD. I have installed namazu on FreeBSD and configured it using the guide (while conforming to the FreeBSD file system layout), but I have two problems:
- I am getting too many messages from cron which I am unable to address because I am not a perl programmer:
Useless use of greediness modifier '?' in regex; marked by <-- HERE in m/^\w+:{1,1}? <-- HERE / at /usr/local/share/namazu/filter/mp3.pl line 155.
I wonder if doing away with the "?" on that regex is sufficient. I have done that, but I am unsure how it affects the indexing.
- The next step that I am not getting clear is how to now include the search box in each list's archive. In the guide, there is a section that talks about having /etc/mailman/*/{archtoc.html | archtocnombox.html} and I get lost right there. My mailman is installed in /usr/local/mailman2/ and nothing exists as /etc/mailman/ (which I believe is Linux file system layout). Where should I place these two files - archtoc.html and archtocnombox.html
- and how do I now integrate them in the whole setup to get the searchbox?
Thanks in advance.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On Sun, Sep 11, 2022 at 12:26 PM Odhiambo Washington odhiambo@gmail.com wrote:
On Sun, Sep 11, 2022 at 11:01 AM Stephen J. Turnbull < stephenjturnbull@gmail.com> wrote:
Odhiambo Washington writes:
What is the simplest way to add a searchbox for the archives?
If you're talking about about Mailman 2's Pipermail, you need to install and configure a separate application. I think HT:Dig is most popular, but I've only used Namazu, which worked well for me. There's a detailed FAQ on it: https://wiki.list.org/x/4030514, which says there are maintained branches on LaunchPad with patches to integrate HT:Dig. That may be easiest. Mark will know more when he gets back.
Another formerly somewhat popular approach is to substitute MHonArc for Pipermail, although that software is quite old now, and I'm not sure about its security status.
Steve
Hi Steve,
Great that you are using namazu. I hope you can help me with the bits I am stuck about using it.
I have found a detailed HOWTO http://bakacsin.ki.iif.hu/~kissg/project/mailman+namazu/#nmzproc for installing it on Linux, but I use FreeBSD. I have installed namazu on FreeBSD and configured it using the guide (while conforming to the FreeBSD file system layout), but I have two problems:
- I am getting too many messages from cron which I am unable to address because I am not a perl programmer:
Useless use of greediness modifier '?' in regex; marked by <-- HERE in m/^\w+:{1,1}? <-- HERE / at /usr/local/share/namazu/filter/mp3.pl line 155.
I wonder if doing away with the "?" on that regex is sufficient. I have done that, but I am unsure how it affects the indexing.
- The next step that I am not getting clear is how to now include the search box in each list's archive. In the guide, there is a section that talks about having /etc/mailman/*/{archtoc.html | archtocnombox.html} and I get lost right there. My mailman is installed in /usr/local/mailman2/ and nothing exists as /etc/mailman/ (which I believe is Linux file system layout). Where should I place these two files - archtoc.html and archtocnombox.html
- and how do I now integrate them in the whole setup to get the searchbox?
Thanks in advance.
Answering myself on #2, I did figure out that the two files should go to my templates/{language}/ as drop-in replacements. I have done that but accessing the archives of my list still do not bring the search box as shown here - https://lists.kictanet.or.ke/pipermail/kictanet/ - and I am thinking that there's something minor that I am still missing.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
Answering myself on #2, I did figure out that the two files should go to my templates/{language}/ as drop-in replacements. I have done that but accessing the archives of my list still do not bring the search box as shown here - https://lists.kictanet.or.ke/pipermail/kictanet/ - and I am thinking that there's something minor that I am still missing.
Once you've swapped in the new archtoc.html and archtocnombox.html files you need to run bin/arch --wipe <LISTNAME>
The renewed archive will then have the search box.
If your archive is private you have to be logged as a member to use the search. It won't allow you to search if you're logged in as an administrator.
And you may first want to edit those new archtoc.html and archtocnombox.html files. By default the search box is at the bottom of the page. I think it's better at the top.
On Sun, Sep 11, 2022 at 3:14 PM Mark Dale via Mailman-Users < mailman-users@python.org> wrote:
Answering myself on #2, I did figure out that the two files should go to my templates/{language}/ as drop-in replacements. I have done that but accessing the archives of my list still do not bring the search box as shown here - https://lists.kictanet.or.ke/pipermail/kictanet/ - and I am thinking that there's something minor that I am still missing.
Once you've swapped in the new archtoc.html and archtocnombox.html files you need to run bin/arch --wipe <LISTNAME>
The renewed archive will then have the search box.
Thank you. That's what I was missing. However, looking at the two files, I see references to:
<form method="get" action="/cgi-bin/mailman/search">
I suppose in my case this too has to be modified to reflect the correct path. With my MM2 installed in /usr/local/mailman2/ the above path then should become "/mailman/cgi-bin/search" ??
When I do a search now, I end up with an error saying "The requested URL was not found on this server." and the resulting URL in the address bar is:
If your archive is private you have to be logged as a member to use the search. It won't allow you to search if you're logged in as an administrator.
And you may first want to edit those new archtoc.html and archtocnombox.html files. By default the search box is at the bottom of the page. I think it's better at the top.
Thanks for that hint. It is not documented in the HOWTO.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 2022-09-11 8:58 AM, Odhiambo Washington wrote:
However, looking at the two files, I see references to:
<form method="get" action="/cgi-bin/mailman/search">
I suppose in my case this too has to be modified to reflect the correct path. With my MM2 installed in /usr/local/mailman2/ the above path then should become "/mailman/cgi-bin/search" ??
It references cgi-bin url in your httpd config. Or possibly DocumentRoot if you don't have a separate cgi-bin configured. Either way, url -> filesystem path mapping is in your httpd config.
Dima
On Sun, 2022-09-11 at 16:58 +0300, Odhiambo Washington wrote:
However, looking at the two files, I see references to:
<form method="get" action="/cgi-bin/mailman/search">
I suppose in my case this too has to be modified to reflect the correct path. With my MM2 installed in /usr/local/mailman2/ the above path then should become "/mailman/cgi-bin/search" ??
When I do a search now, I end up with an error saying "The requested URL was not found on this server." and the resulting URL in the address bar is:
Firstly, check your mm_cfg.py file. In my case it is:
DEFAULT_URL_PATTERN = 'https://%s/mailman/'
Note that there is no "cgi-bin" in the above URL.
If that's the same for you, then yes you need to edit the new archtoc.html and archtocnombox.html files accordingly.
There are 2 strings in each file that would need to be edited so they look like:
<form method="get" action="/mailman/search">
<a href="/mailman/search?idxname=%(listname)s">[How to search]</a>
Then rebuild the archive again:
bin/arch --wipe <LISTNAME>
And it won't hurt to tell Namazu to re-index the archive(s):
ls /var/lib/namazu/mailman | xargs $HOME/bin/mailman_index
On Mon, Sep 12, 2022 at 12:54 AM Mark Dale via Mailman-Users < mailman-users@python.org> wrote:
On Sun, 2022-09-11 at 16:58 +0300, Odhiambo Washington wrote:
However, looking at the two files, I see references to:
<form method="get" action="/cgi-bin/mailman/search">
I suppose in my case this too has to be modified to reflect the correct path. With my MM2 installed in /usr/local/mailman2/ the above path then should become "/mailman/cgi-bin/search" ??
When I do a search now, I end up with an error saying "The requested URL was not found on this server." and the resulting URL in the address bar is:
Firstly, check your mm_cfg.py file. In my case it is:
DEFAULT_URL_PATTERN = 'https://%s/mailman/'
Note that there is no "cgi-bin" in the above URL.
If that's the same for you, then yes you need to edit the new archtoc.html and archtocnombox.html files accordingly.
There are 2 strings in each file that would need to be edited so they look like:
<form method="get" action="/mailman/search">
<a href="/mailman/search?idxname=%(listname)s">[How to search]</a>
Then rebuild the archive again:
bin/arch --wipe <LISTNAME>
And it won't hurt to tell Namazu to re-index the archive(s):
ls /var/lib/namazu/mailman | xargs $HOME/bin/mailman_index
Thank you very much for this explanation. As a matter of fact, I had missed seeing the 2nd string (perhaps out of anxiety). It is now working quite well, and yes, I also put the searchbox at the top instead of the bottom.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On Sun, 2022-09-11 at 16:58 +0300, Odhiambo Washington wrote:
And you may first want to edit those new archtoc.html and archtocnombox.html files. By default the search box is at the bottom of the page. I think it's better at the top.
Thanks for that hint. It is not documented in the HOWTO.
It is well documented in the How-To:
http://bakacsin.ki.iif.hu/~kissg/project/mailman+namazu/
"Search ... In case of private lists this program checks if user is authorized to access archive content. (Note: authorization is based on regular membership only. Server and list administrator access rights are disregarded. Enabling admin staff to search archives may be subject of further development.) "
The HOW-TO is very good. It was written in 2007 so a couple of things have changed (like package names). It needs VERY close attention to be paid to the detail that the author has taken the trouble to write. I wish all documentation was as good as this.
On Mon, Sep 12, 2022 at 1:06 AM Mark Dale via Mailman-Users < mailman-users@python.org> wrote:
On Sun, 2022-09-11 at 16:58 +0300, Odhiambo Washington wrote:
And you may first want to edit those new archtoc.html and archtocnombox.html files. By default the search box is at the bottom of the page. I think it's better at the top.
Thanks for that hint. It is not documented in the HOWTO.
It is well documented in the How-To:
The documentation is Linux-centric, I should have said, and mostly related to an installation from the distro packages. I am not so used to running applications from distro packages as sometimes they are quite limiting.
"Search ... In case of private lists this program checks if user is authorized to access archive content. (Note: authorization is based on regular membership only. Server and list administrator access rights are disregarded. Enabling admin staff to search archives may be subject of further development.) "
The HOW-TO is very good. It was written in 2007 so a couple of things have changed (like package names). It needs VERY close attention to be paid to the detail that the author has taken the trouble to write. I wish all documentation was as good as this.
I agree with that.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
Odhiambo Washington writes:
- I am getting too many messages from cron which I am unable to address because I am not a perl programmer:
I'm not, either.
Useless use of greediness modifier '?' in regex; marked by <-- HERE in m/^\w+:{1,1}? <-- HERE / at /usr/local/share/namazu/filter/mp3.pl line 155.
I wonder if doing away with the "?" on that regex is sufficient. I have done that, but I am unsure how it affects the indexing.
That's a very strange RE. It appears say "you should match at least one ':' and at most one ':' and if there is more than one way to match that many ':' you should choose the shortest one that allows the whole pattern to match". A normal person would just write ':' instead of ':{1,1}?' but this is Perl code so.... ;-)
Where should I place these two files - archtoc.html and archtocnombox.html
- and how do I now integrate them in the whole setup to get the searchbox?
I haven't done this is a very long time. I assume those two files go in the template directory, and probably you need to put a link to them in each template where you a search function. But if you want a search box in the thread page, for example, you're going to need to extract the search form from one of those search pages and add it to the thread template, the date template, and the author template.
On 9/11/22 2:00 AM, Odhiambo Washington wrote:
What is the simplest way to add a searchbox for the archives?
Because my list archives are open and regularly crawled, I tell my user to just use their favorite search engine, enter the list name, and the term they're looking for.
You could probably code a simple form to do that for you.
I had considered MonHarc back in the day, but never got around to implementing it. And since it's probably not secure these days (see previous message in this thread) I'm glad I didn't.
-- Bill Christensen http://SustainableSources.com http://LinkedIn.com/in/billc108
On Sat, Oct 8, 2022 at 3:28 AM Bill Christensen < billc_lists@greenbuilder.com> wrote:
On 9/11/22 2:00 AM, Odhiambo Washington wrote:
What is the simplest way to add a searchbox for the archives?
Because my list archives are open and regularly crawled, I tell my user to just use their favorite search engine, enter the list name, and the term they're looking for.
You could probably code a simple form to do that for you.
I had considered MonHarc back in the day, but never got around to implementing it. And since it's probably not secure these days (see previous message in this thread) I'm glad I didn't.
I ended up using Namazu and it's working fine.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
participants (5)
-
Bill Christensen -
dmitri maziuk -
Mark Dale -
Odhiambo Washington -
Stephen J. Turnbull