[Mailman-Users] Memory error when running arch

[James Madill]

Last night, when running arch against one of the mailing lists on my 
Mailman server, the following error was displayed on the terminal:

        Updating HTML for article 12815
        Updating HTML for article 12816
        Updating HTML for article 12817
        Traceback (most recent call last):
          File "./arch", line 129, in ?
            main()
          File "./arch", line 119, in main
            archiver.close()
          File "/mailman/Mailman/Archiver/pipermail.py", line 304, in 
close
            self.update_dirty_archives()
          File "/mailman/Mailman/Archiver/HyperArch.py", line 949, in 
update_dirty_archives
            self.__super_update_dirty_archives()
          File "/mailman/Mailman/Archiver/pipermail.py", line 513, in 
update_dirty_archives
            self.update_archive(i)
          File "/mailman/Mailman/Archiver/HyperArch.py", line 952, in 
update_archive
            self.__super_update_archive(archive)
          File "/mailman/Mailman/Archiver/pipermail.py", line 422, in 
update_archive
            self._update_thread_index(archive, arcdir)
          File "/mailman/Mailman/Archiver/pipermail.py", line 479, in 
_update_thread_index
            self.update_article(arcdir, article, a1, a3) 
          File "/mailman/Mailman/Archiver/HyperArch.py", line 1115, in 
update_article
            f.write(article.as_html())
          File "/mailman/Mailman/Archiver/HyperArch.py", line 298, in 
as_html
            return self.html_tmpl % d
        MemoryError
        $

When I reran it this morning after checking the OS for errors, the same 
problem occured:

        Updating HTML for article 12490
        Updating HTML for article 12491
        Updating HTML for article 12492
        Updating index files for archive [2001-February]
          Date
        no mem for new parser
        Traceback (most recent call last):
          File "./arch", line 129, in ?
            main()
          File "./arch", line 119, in main
            archiver.close()
          File "/mailman/Mailman/Archiver/pipermail.py", line 304, in 
close
            self.update_dirty_archives()
          File "/mailman/Mailman/Archiver/HyperArch.py", line 949, in 
update_dirty_archives
            self.__super_update_dirty_archives()
          File "/mailman/Mailman/Archiver/pipermail.py", line 513, in 
update_dirty_archives
            self.update_archive(i)
          File "/mailman/Mailman/Archiver/HyperArch.py", line 952, in 
update_archive
            self.__super_update_archive(archive)
          File "/mailman/Mailman/Archiver/pipermail.py", line 420, in 
update_archive
            self._update_simple_index(hdr, archive, arcdir)
          File "/mailman/Mailman/Archiver/pipermail.py", line 437, in 
_update_simple_index
            article = self.database.getArticle(self.archive, msgid)
          File "/mailman/Mailman/Archiver/HyperDatabase.py", line 279, in 
getArticle
            article = self.__cache[msgid] = pickle.loads(buf)
        MemoryError
        $ 

The three cron jobs also appeared to fail today as well:

        Your "cron" job on arachnia
        /usr/local/bin/python -S /mailman/cron/qrunner

        produced the following output:

        Traceback (most recent call last):
          File "/mailman/cron/qrunner", line 85, in ?
            from Mailman import MailList
          File "/mailman/Mailman/MailList.py", line 43, in ?
            from Mailman.MailCommandHandler import MailCommandHandler 
        MemoryError

--

        Your "cron" job on arachnia
        /usr/local/bin/python -S /mailman/cron/gate_news

        produced the following output:

        Traceback (most recent call last):
          File "/mailman/cron/gate_news", line 40, in ?
            from Mailman import MailList
          File "/mailman/Mailman/MailList.py", line 44, in ?
            from Mailman.HTMLFormatter import HTMLFormatter 
          File "/mailman/Mailman/HTMLFormatter.py", line 27, in ?
            from Mailman.htmlformat import *
        MemoryError

--

        Your "cron" job on arachnia
        /usr/local/bin/python -S /mailman/cron/senddigests

        produced the following output:

        Traceback (most recent call last):
          File "/mailman/cron/senddigests", line 65, in ?
            main()
          File "/mailman/cron/senddigests", line 42, in main
            send_list_digest(mlist)
          File "/mailman/cron/senddigests", line 46, in send_list_digest
            mlist.Lock()
          File "/mailman/Mailman/MailList.py", line 1343, in Lock
            self.Load()
          File "/mailman/Mailman/MailList.py", line 903, in Load
            raise Errors.MMCorruptListDatabaseError, e
        Mailman.Errors.MMCorruptListDatabaseError: bad marshal data

The machine is a Solaris 7 box with 256Meg of RAM
Mailman version is 2.0.6
Python version is 2.1
The list in question has about 16,000 archived messages

arch has no problem running against lists with fewer messages in the 
archives.  It looks like the arch script might be overflowing some kind of 
Python memory buffer.

Any ideas, solutions?

-- James

      o o o o o o o . . .   _______________________ _______=======_T___
    o      _____            |James Madill         | |Duke Univ Med Ctr|
 >.][__n_n_| D[  ====|____  |james.madill at duke.edu| | (919) 286-6384  |
  (________|__|_[____/____]_|_____________________|_|_________________|
 _/oo  O-O-O  `  oo     oo  'o^o^o           o^o^o` 'o^o           o^o`
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
<http://www.duke.edu/~madil001/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/mailman-users/attachments/20010727/593bb494/attachment.html