[Mailman-Users] running External Archiver on background/foreground

[Mohamed CHAARI]

Mark Sapiro wrote:
> Mohamed CHAARI wrote:
> 
>> Mark Sapiro wrote:
>>> See
>>> <http://mail.python.org/pipermail/mailman-users/2005-August/045994.html>.
>>>
>> you mentioned, in the thread above (Aug 2005), that it would be better 
>> to have a unique temporary file, to avoid conflict problems ...
>> I've tried this, in mm_cfg.py, using unix timestamp
>>
>> PUBLIC_EXTERNAL_ARCHIVER = 'cat > /var/run/mailman/mail_tmp.$(date +%s); 
>> /usr/local/bin/external_arch.pl %(listname)s &'
>>
>> but without success.
>> is there a way to do it ? should I use Python functions to get unix 
>> timestamp for current date ?
> 
> 
> It looks to me as if you have inherited something from Jean-Philippe GIOLA and
> it is now way more complicated than it needs to be.
> 
> Here are my observations on the above:
> 
> 1) The name "mail_tmp.$(date +%s)" may still not be unique. What if two
> messages arrive to be archived within the same second?
> 
> 2)How does /usr/local/bin/external_arch.pl determine what file to read? And, if
> it just picks one, it could be for the wrong list.
> 
> 3)The reason the above doesn't work at all is mailman interpolates into the
> PUBLIC_EXTERNAL_ARCHIVER string from a dictionary in order to replace
> %(listname)s with the actual listname, but this interpolation sees the %s date
> format and replaces it with the entire dictionary. You have to double the % to
> avoid this. E.g.
> 
> 'cat > /var/run/mailman/mail_tmp.$(date +%%s); /usr/local/bin/external_arch.pl
> %(listname)s &'
> 
> 4) The normal way to do this is to not bother with the tempfile stuff at all,
> but rather to just pipe the message to the external archiver as in
> 
> PUBLIC_EXTERNAL_ARCHIVER = '/usr/local/bin/external_arch.pl %(listname)s'
> 
> Then external_arch.pl can just read it's standard input for the message. If you
> do it this way, you can't run it in the background because it won't get
> standard input from the pipe, but the only reason it might need to run in the
> background is if it does something which locks the list.
> 

Hello,

I come back to this topic ...
As I really need to keep possibility to have both MM archiving and 
external one, I've made a small patch on Mailman, so that it accepts 
both internal archiving (pipermail) and external archiving (if enabled).
In this way, there is no longer need to call 'arch' in 
'external_arch.pl' script, and thus, this script doesn't need to be run 
in background.

But I would like to have your opinion (Mark and others), about this patch:
- is there any side effect ?
- can there be problems (or conflicts) when receiving and archiving many 
mails ?

here's the patch (on 'Archiver.py'):

--- /home/mailman/Mailman/Archiver/Archiver.orig.py	2008-01-11 
19:40:44.000000000 +0100
+++ /home/mailman/Mailman/Archiver/Archiver.py	2008-01-11 
19:44:54.000000000 +0100
@@ -201,20 +201,22 @@
                  # Archive to mbox only.
                  return
          txt = str(msg)
-        # should we use the internal or external archiver?
+
+        # keep using the internal archiver
+        f = StringIO(txt)
+        import HyperArch
+        h = HyperArch.HyperArchive(self)
+        h.processUnixMailbox(f)
+        h.close()
+        f.close()
+
+	# now, use the external archiver (if enabled)
          private_p = self.archive_private
          if mm_cfg.PUBLIC_EXTERNAL_ARCHIVER and not private_p:
              self.ExternalArchive(mm_cfg.PUBLIC_EXTERNAL_ARCHIVER, txt)
          elif mm_cfg.PRIVATE_EXTERNAL_ARCHIVER and private_p:
              self.ExternalArchive(mm_cfg.PRIVATE_EXTERNAL_ARCHIVER, txt)
-        else:
-            # use the internal archiver
-            f = StringIO(txt)
-            import HyperArch
-            h = HyperArch.HyperArchive(self)
-            h.processUnixMailbox(f)
-            h.close()
-            f.close()
+

      #
      # called from MailList.MailList.Save()


Thanks & best regards

-- 
---
--Mohamed CHAARI   (mailto:mohamed.chaari at st.com)