[Moin-user] Moin1.6a with XAPIAN (help and attachment support)

[mail at heavy.ch]

Thanks for your Tipps! But still the Xapian Search get's disabled while
using FullSearch Macro with Regex.

***************************************************
Steps I did:
***************************************************

1. Ok, I installed the PyStemmer and also Xpdf (for pdftotext) 

2. Now I changed my syntax and used the --mode-add switch or later the
rebuild. The syntax looks now like this:

[root at marius server]# sudo -u www-data moin
--config-dir=/var/www/moin/config --wiki-url=Home.Heavy.ch:8080/ index
build --mode=add

2. while building the index I got this error message

WARNING:root:Filter application_octet_stream threw error '[Errno 21] Is
a directory' for
file /var/www/moin/data/pages/MoinMoinWiki(2f)Gallery2/attachments/img

well, i just remove the img directory and rebuild the index (guess it
was from extracting a zip file)

3. And now the SystemInfo shows index is available!
Xapian search
        Enabled, index available, last modified: 2007-08-06 11:59:11

***************************************************
==> Everything seems to work, but...
***************************************************

4. From Time to Time the Xapain get's disabled... Here's an example

a) When I start a fulltext search for "xml"
http://home.heavy.ch:8080/CategoryHomepage?action=fullsearch&context=180&value=xml&fullsearch=Text
--> The output shows that "_xapianProcess = 0.229s" xapian is used. 

b) Then I call a Category like
http://home.heavy.ch:8080/CategoryHomepage 
--> with a [[FullSearch]] Macro (regex)

and after that I research for "xml"
http://home.heavy.ch:8080/CategoryHomepage?action=fullsearch&context=180&value=xml&fullsearch=Text

and then the xapian get disabled
_moinSearch = 1.102s

Also some CategoryPages who not Use FullSearchCached doesn't show any
entries.

==> temporary solution is to restart the moin application.

***************************************************

==> So believe the FullSearch Macro disable in someway the xapian
search. If 'im trying this on a "CategoryLinux" it doesn't happen. So
it's maybe the  "regex" parameter who breaks the xapian search.

http://home.heavy.ch:8080/CategoryLinux
[[FullSearch(linkto:CategoryLinux -title:Category)]]

http://home.heavy.ch:8080/CategoryHomepage
[[FullSearch(regex:(----(-*)(\r)?\n)(.*)CategoryHomepage\b)]]

Could you confirm this? or any other ideas?



cya
Marcel




Am Montag, den 06.08.2007, 11:05 +0200 schrieb Thomas Waldmann:
> > b) I also build the index with 
> > moin --config-dir=/var/www/moin/config --wiki-url=Home.Heavy.ch index
> > build
> > --> and it shows up with no error messages
> 
> Make very sure --wiki-url is correct and it will match whatever you 
> configured in wikiconfig/farmconfig.
> 
>  From the stuff you posted below, it rather looks like it is:
> 
> --wiki-url=home.heavy.ch:8080/
> 
> But usually you get some error if it does not match.
> 
> > 
> > (Note: To get a few pages I just copied my data dir from my moin 1.5.7
> > (http://lotek.heavy.ch) to my new 1.6a (http://home.heavy.ch);
> > everything seems to work...)
> 
> Most pages will work, but some stuff can break.
> 
> > c) I also changed the wikiconfig.py and yes it shows up under:
> > 
> > http://home.heavy.ch:8080/SystemInfo
> > Xapian search
> >         Enabled, index unavailable, last modified: N/A
>                     ^^^^^^^^^^^^^^^^^
> 
> This is your problem.
> 
> > Xapian Version
> >         1.0.2
> > Stemming for Xapian
> >         Disabled
>  > PyStemmer Version
>  >         PyStemmer not installed
>  > PyStemmer stems
>  >         NONE
> 
> Maybe installing PyStemmer is a good idea (but not related to your 
> trouble, I guess).
> 
> > d) but if I'm starting to search it changes to:
> > Xapian search
> >         Disabled
> 
> If moin doesn't find a xapian index, moin will auto-disable xapian_search.
> 
> 
> > ==> WHY? Are there any Configuration-Hints Online?
> > The HelpOnXapian is still empty :(
> > http://moinmoin.wikiwikiweb.de/HelpOnXapian
> 
> Look there: http://master.moinmo.in/HelpOnXapian
> 
> (master has 1.6 content, everything else is still 1.5)
> 
> BTW, if you get it working, you can help us by doing ACL tests with xapian.
> 
> Just index some pages and attachments that are not readable for user X.
> 
> Then search for stuff on those pages with user X.
> 
> Does it find pages or attachments it should not find for X because ACLs 
> normally limit reading of the stuff for user X?
> 
> > ****************************************
> > 2. Attachment Support
> > ****************************************
> > What makes me wonder, if it would be possible to search also in
> > attachments (word, openoffice, xls, pdf, zip, etcetera).
> 
> Yes, everything we have a filter for (see MoinMoin/filter/ - some stuff 
> there [esp. the M$ stuff] needs additional software installed).
> 
> Some other stuff (*.iso, *.zip, *.tar.gz, ...) is excluded from indexing.
> 
> Also look into docs/CHANGES, some of the stuff should be documented there.
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> Moin-user mailing list
> Moin-user at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/moin-user