[Mailman-Developers] Memory error when running arch

[Thomas Wouters]

On Mon, Jul 30, 2001 at 01:18:23PM -0400, James Madill wrote:

> Last night, when running arch against one of the mailing lists on my 
> Mailman server, the following error was displayed on the terminal:
>             return self.html_tmpl % d
>         MemoryError

> When I reran it this morning after checking the OS for errors, the same 
> problem occured:

> The three cron jobs also appeared to fail today as well:

>         /usr/local/bin/python -S /mailman/cron/qrunner

>         Traceback (most recent call last):
>           File "/mailman/cron/qrunner", line 85, in ?
>             from Mailman import MailList
>           File "/mailman/Mailman/MailList.py", line 43, in ?
>             from Mailman.MailCommandHandler import MailCommandHandler 
>         MemoryError


These MemoryErrors are either Python running out of stackspace, or your
process/machine running out of memory (malloc() returning NULL.) Actually,
under most operating systems, I think the first doesn't even raise a
MemoryError, so the most likely cause is just out of memory. I doubt the
machine itself is running out of memory (most operating systems
overallocate, anyway, so it should just crash when it does, not return a
'nice' error.) Instead, you probably have memory limits on your machine. Not
sure about Solaris, but on BSDI you tweak limits in /etc/login.conf, and on
Linux /etc/login.defs (I think -- I've never actually had to do it :). You
can also use 'limits -a' (bash) or 'limit' (tcsh) to see the memory limits
of your current process (though Cron jobs may be started with different
process limits) and 'ulimit' or 'unlimit' to remove/raise limits.

>         /usr/local/bin/python -S /mailman/cron/senddigests

>         Traceback (most recent call last):
>           File "/mailman/cron/senddigests", line 65, in ?
>             main()
>           File "/mailman/cron/senddigests", line 42, in main
>             send_list_digest(mlist)
>           File "/mailman/cron/senddigests", line 46, in send_list_digest
>             mlist.Lock()
>           File "/mailman/Mailman/MailList.py", line 1343, in Lock
>             self.Load()
>           File "/mailman/Mailman/MailList.py", line 903, in Load
>             raise Errors.MMCorruptListDatabaseError, e
>         Mailman.Errors.MMCorruptListDatabaseError: bad marshal data

This is probably not the cause of the problems, though it *could* be...
hmmm...
It looks like the config db of one of your lists is broken. Perhaps you had
a disk full or quota problems ? The previous crashes shouldn't have caused
this, since they didn't crash Mailman during the write of the marshal. If
you'll check your Mailman logs ($prefix/logs/errors, to be exact), you
should see a message like

<listname> db file was corrupt, using fallback: <filename>

That's your broken list. To see if the marshal is really broken, use 

$prefix/bin/dumpdb lists/<listname>/config.db

If that dumps a lot of info like

{   'acceptable_aliases': '',
    'admin_immed_notify': 1,
    'admin_member_chunksize': 30,
    'admin_notify_mchanges': 1,
[... lots more ....]

your config.db is not broken, but rather something goes wrong when you try
to read it in... Probably the same MemoryError being masked somewhere... Ah,
yeah, now that I check it, that is definately possible.

If your config.db *is* broken, your best chance is to restore a backup,
since the 'previous version' Mailman keeps around is also broken (it tries
to load it automatically.) If all else fails, you'll have to recreate the
list :-( But it's probably a MemoryError just like the rest.

> arch has no problem running against lists with fewer messages in the 
> archives.  It looks like the arch script might be overflowing some kind of 
> Python memory buffer.

Python doesn't have that many (limited) memory buffers... As long as the OS
allows it to grow, it keeps on growing. :)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!