marshal.load() and error handling
I'd been getting EOF errors for some time now, both in log/error and from cron/checkdbs, e.g:
Traceback (most recent call last): File "/local/mailman/cron/checkdbs", line 92, in ? main() File "/local/mailman/cron/checkdbs", line 43, in main count = mlist.NumRequestsPending() File "/local/mailman/Mailman/ListAdmin.py", line 96, in NumRequestsPending self.__opendb() File "/local/mailman/Mailman/ListAdmin.py", line 69, in __opendb self.__db = marshal.load(fp) EOFError: EOF read where object expected
So I decided to track it down: the problem was lists with corrupt request.dbs. I think the solution would be to handle the EOFError exception in ListAdmin.py in __opendb. I don't know Python, but perhaps something along the lines of:
def __opendb(self):
if self.__db is None:
assert self.Locked() and self.__filename
try:
fp = open(self.__filename)
self.__db = marshal.load(fp)
fp.close()
except IOError, e:
if e.errno <> errno.ENOENT: raise
self.__db = {}
except EOFError, e:
self.__db = {}
In addition to this, it may be worthwhile handling other errors (ValueError, TypeError etc.) which indicate corrupt databases. Some of the other mailman code which uses marshal.load does this. Perhaps all uses of marshal.load should be checked for extra error handling?
There was no easy way to find out which request database was corrupt (there doesn't appear to be any script similar to check_db to do the same for the request.db file) so I just wrote a script based on check_db to help find corrupt request.dbs. I think something like this (cleaned up of course) should be added to the distribution..
bin/check_request_db ------------8<------------ #! /usr/bin/env python
"""Check the raw request.db for a mailing list.
Usage: %(program)s listname """
import sys import os import string import marshal
import paths from Mailman import mm_cfg
program = sys.argv[0]
def testfile(filename): try: fp = open(filename) except IOError, (code, msg): print filename, 'cannot be opened:\n\t', msg return 1 else: try: d = marshal.load(fp) except (EOFError, ValueError, TypeError), msg: print filename, 'is corrupted:\n\t', msg return 1 else: print filename, 'is fine' return 0
def main(): if len(sys.argv) == 2: listname = string.lower(sys.argv[1]) else: print __doc__ % globals() sys.exit(1)
listpath = os.path.join(mm_cfg.LIST_DATA_DIR, listname)
configdb = os.path.join(listpath, 'request.db')
testfile(configdb)
if __name__ == '__main__': main()
------------>8------------
Regards,
-- Cillian
[PS: please CC me on any replies]
participants (1)
-
Cillian Sharkey