[Mailman-Users] Indexing mail right after delivery

[Cedric Jeanneret]

On Tue, 02 Mar 2010 11:34:25 -0800
Mark Sapiro <mark at msapiro.net> wrote:

> On 3/2/2010 3:41 AM, Cedric Jeanneret wrote:
> > On Fri, 26 Feb 2010 10:15:13 -0800
> > Mark Sapiro <mark at msapiro.net> wrote:
> >>
> >> At this point, you have a list object (locked) and a message object. You
> >> might think you could just do
> >>
> >> mlist.ArchiveMail(msg)
> >>
> >> to archive the mail to the listname.mbox file and the pipermail archive,
> >> but that wouldn't quite work because that method would re-invoke the
> >> external archiver. Also, you don't need to worry about the listname.mbox
> >> file because the ArchiveMail() method already did that before invoking
> >> the external archiver, so what you would need is
> >>
> >> from Mailman.Archiver import HyperArch
> >> from cStringIO import StringIO
> >> f = StringIO(str(msg))
> >> h = HyperArch.HyperArchive(mlist)
> >> h.processUnixMailbox(f)
> >> h.close()
> >> f.close()
> >>
> >> Which is what the ArchiveMail() method would do. Now you still have the
> >> mlist and msg objects, and you need to save and unlock the list at some
> >> point
> >>
> >> mlist.Save()
> >> mlist.Unlock()
> >>
> >> and the message is now in the pipermail archive and can be indexed.
> >>
> > 
> > Hello again,
> > 
> > I'm having some troubles with my code. According to what Mark said, I've done this :
> > 
> > #!/usr/bin/env python
> > import sys
> > sys.path.insert(0,'/usr/lib/mailman')
> > 
> > import syslog
> > 
> > syslog.syslog('begin script')
> > 
> > import email
> > from Mailman import MailList
> > from Mailman import Message
> > ## archive part
> > from Mailman.Archiver import HyperArch
> > from cStringIO import StringIO
> > 
> > maillist = sys.argv[2]
> > hostname = sys.argv[1]
> > 
> > msg = email.message_from_file(sys.stdin, Message.Message)
> > syslog.syslog(maillist)
> > 
> > mlist = MailList.MailList(maillist, lock=True)
> > 
> > syslog.syslog('processing archiver')
> > ## let archive it
> > f = StringIO(str(msg))
> > h = HyperArch.HyperArchive(mlist)
> > h.processUnixMailbox(f)
> > h.close()
> > f.close()
> > mlist.Save()
> > mlist.Unlock()
> > 
> > mlist.ArchiveMail(msg)
> 
> 
> Here is one problem. Remove the above line. As I tried to say above you
> can't do this. The lines above from "f = StringIO(str(msg))" through
> "f.close()" archive the message. When you call mlist.ArchiveMail(msg),
> it reinvokes your external archiver in an endless loop.
> 
> You need to remove the mlist.ArchiveMail(msg).
> 
> The locking problem is something else. The external archiver is called
> with the list locked, thus when we try to instantiate the list 'locked',
> we have a deadlock. Thus, you never saw the loop because of the deadlock.
> 
> The good news is we don't have to pass a locked list instance to
> HyperArch.HyperArchive() as it uses a special archiver lock.
> 
> So, replace
> 
> mlist = MailList.MailList(maillist, lock=True)
> 
> with
> 
> mlist = MailList.MailList(maillist, lock=False)
> 
> and remove the "mlist.Unlock()" as your instance isn't locked, and
> ArchRunner will unlock its list instance when you exit.
> 
> 
> > syslog.syslog('processing indexer')
> > ### coming soon
> > 
> > syslog.syslog('exiting - all ok')
> > sys.exit(0)
> > 
> > "syslog" is for debug purpose only.
> > 
> > And if I send an email on my ML, I have this kind of error:
> > 
> > Mar 02 12:38:33 2010 (28380) toto.lock lifetime has expired, breaking
> 
> 

Hmm, it seems it crashes in pipermail.py, in function processUnixMailbox:
we have a
pos = input.tell() on line 564, but unfortunately "input" does NOT have any "tell()" method...
It returns a "41" status.

-- 
Cédric Jeanneret                 |  System Administrator
021 619 10 32                    |  Camptocamp SA
cedric.jeanneret at camptocamp.com  |  PSE-A / EPFL