[Mailman-Developers] Patch for HyperArch

[Mark Sapiro]

On 03/11/2016 06:39 AM, Sebastian Hagedorn wrote:
> --On 10. März 2016 um 17:00:18 -0800 Mark Sapiro <mark at msapiro.net> wrote:
> 
>> I have reported this bug and fixed it. The bug is
>> <https://bugs.launchpad.net/mailman/+bug/1555798> and the fix is
>> <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1633>
...
> 
> Thanks. I applied the patch and tried ro re-archive the list in
> question. Now I get this:
> 
> Schreibe Archivzustand in Datei
> /var/lib/mailman/archives/private/linux-users/pipermail.pck
> Traceback (most recent call last):
>  File "./bin/arch", line 201, in <module>
>    main()
>  File "./bin/arch", line 189, in main
>    archiver.processUnixMailbox(fp, start, end)
>  File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 597, in
> processUnixMailbox
>    a = self._makeArticle(m, self.sequence)
>  File "/usr/lib/mailman/Mailman/Archiver/HyperArch.py", line 688, in
> _makeArticle
>    mlist=self.maillist)
>  File "/usr/lib/mailman/Mailman/Archiver/HyperArch.py", line 264, in
> __init__
>    self.__super_init(message, sequence, keepHeaders)
>  File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 187, in
> __init__
>    self._set_date(message)
>  File "/usr/lib/mailman/Mailman/Archiver/HyperArch.py", line 600, in
> _set_date
>    self.__super_set_date(message)
>  File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 256, in
> _set_date
>    message.get('received'), flags=re.S))
>  File "/usr/lib/python2.7/re.py", line 155, in sub
>    return _compile(pattern, flags).sub(repl, string, count)
> TypeError: expected string or buffer
> 
> For the time being, I have reverted the patch.


Thanks for testing and thanks for the report.

I have fixed this and pushed rev 1634. The entire patch combining revs
1633 and 1644 is attached.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
-------------- next part --------------
=== modified file 'Mailman/Archiver/pipermail.py'
--- Mailman/Archiver/pipermail.py	2013-12-14 00:53:13 +0000
+++ Mailman/Archiver/pipermail.py	2016-03-11 17:30:47 +0000
@@ -16,6 +16,7 @@
 VERSION = __version__
 CACHESIZE = 100    # Number of slots in the cache
 
+from Mailman import mm_cfg
 from Mailman import Errors
 from Mailman.Mailbox import ArchiverMailbox
 from Mailman.Logging.Syslog import syslog
@@ -230,21 +231,30 @@
         self.body = s.readlines()
 
     def _set_date(self, message):
-        def floatdate(header):
-            missing = []
-            datestr = message.get(header, missing)
-            if datestr is missing:
+        def floatdate(datestr):
+            if not datestr:
                 return None
             date = parsedate_tz(datestr)
             try:
-                return mktime_tz(date)
+                date = mktime_tz(date)
+                if (date < 0 or
+                    date - time.time() >
+                        mm_cfg.ARCHIVER_ALLOWABLE_SANE_DATE_SKEW
+                   ):
+                    return None
+                return date
             except (TypeError, ValueError, OverflowError):
                 return None
-        date = floatdate('date')
-        if date is None:
-            date = floatdate('x-list-received-date')
-        if date is None:
-            # What's left to try?
+        date = floatdate(message.get('date'))
+        if date is None:
+            date = floatdate(message.get('x-list-received-date'))
+        if date is None:
+            date = floatdate(re.sub(r'^.*;\s*', '',
+                                    message.get('received', ''), flags=re.S))
+        if date is None:
+            date = floatdate(re.sub(r'From \s*\S+\s+', '',
+                                    message.get_unixfrom() or '' ))
+        if date is None:
             date = self._last_article_time + 1
         self._last_article_time = date
         self.date = '%011i' % date