[Mailman-Users] Indexing mail right after delivery
cedric.jeanneret at camptocamp.com
Thu Mar 4 07:43:07 CET 2010
On Wed, 03 Mar 2010 10:04:31 -0800
Mark Sapiro <mark at msapiro.net> wrote:
> On 3/3/2010 9:20 AM, Cédric Jeanneret wrote:
> > Maybe a python version? What is really strange is that it works inside
> > the archiver.... I tried to NOT use email.message_from_file (so use
> > directly StringIO on sys.stdin), and it worked fine. In fact, the
> > error was that "Message doesn't have "tell()" method"...
> Which says you are passing a Message object, not a StringIO or file
> object. I considered at one point just passing sys.stdin directly, but
> that won't work because sys.stdin does not have seek() or tell() methods.
> > Another error was really annoying : ALL worked. almost. I couldn't do
> > my mlist.Save(), as there was an error for the lockfile.
> > I did :
> > mlist = MailList.MailList('toto', lock=False)
> > # other code
> > mlist.Save()
> Right. I overlooked the fact that you can't Save() an unlocked list.
> But, I don't think you need to. I don't think the archiver actually
> updates your list instance in it's processing, so you should be OK if
> you just remove the Save() from your code.
> > -> crashed. After poking into MailList code, I saw that it refreshes
> > the lockfile. Commenting out this line made it work again.... more or
> > less : message was in mbox, but wasn't in pipermail archives....
> Don't do that. It won't work anyway because the locked list object in
> ArchRunner will be saved after you're done and will undo any changes you
> made to your list object. But, as I say, you shouldn't need to save your
> list object. It is only passed to the HyperArch.HyperArchive()
> constructor so the archiver knows where to find the archive. I don't
> think it is updated.
> > Poking on the Net, I found this post
> > http://email@example.com/msg47499.html you
> > answered some months (well, years) ago. I tried this way :
> > applying the patch, so that it uses mailman internal archiver, and it
> > calls my indexer right after.
> > That's not really clean, it's not really a portable way, but it works.
> > The fact that I have to patch a file from mailman package annoy me a
> > bit, but... I didn't have any success with the ways you showed me :(
> > To be honnest, maybe I'll try to put a handler (like XapianIndexer.py)
> > for this. As I saw how to debug my scripts (thank you for the tip), I
> > guess it would be the best way, instead of patching a code (which will
> > be overriden on the next update).
> > Or maybe there's a variable in mm_config (or defaults) which tell
> > mailman to call a script after archiving ? I didn't see such a thing,
> > I guess that's the role a the GLOBAL_PIPELINE and its handlers
> > chain...
> As I tried to point out in my initial reply
> that won't work.
> The pipeline includes ToArchive which only queues the message in the
> archive queue for ArchRunner. Then IncomingRunner continues processing
> the pipeline. When it gets to your handler, there's no guarantee that
> ArchRunner has yet archived the message so how do you index something
> that may not yet even be there.
> We were almost there with the external archiver method. Let's try to
> make that work.
> What do you have now in the external archiver code and in the
> PUBLIC_EXTERNAL_ARCHIVER and PRIVATE_EXTERNAL_ARCHIVER strings and what
> is the problem?
First of all, I want to thank you for the time you spend on my case. I really appreciate.
Now, for my code:
I attached the latest (buggy) version of my archive-and-index.py script. I've done a rollback to the way you told me, so that we won't go in all directions.
You'll find anotther attachment : debug file I added in this way :
PUBLIC_EXTERNAL_ARCHIVER = '/root/archive-and-index.py %(hostname)s %(listname)s &>/var/log/mailman/archiver'
It seems that the Message.Message stays, even if we create a new StringIO variable... weird.
Just in case :
Maybe there's a problem with this version... ? If so, it will be a "little" problem, as it's the lenny version.
I'll keep on trying, and keep you updated as soon as I have some new things.
Cédric Jeanneret | System Administrator
021 619 10 32 | Camptocamp SA
cedric.jeanneret at camptocamp.com | PSE-A / EPFL
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 197 bytes
Desc: not available
More information about the Mailman-Users