[Mailman-Users] Spurious ImportError stopping the qrunner
R.Barrett at ftel.co.uk
Tue Nov 19 18:55:50 CET 2002
I have a hitherto stable Mailman server currently running MM 2.0.13 with
Python 2.2.1, both built from source, under Suse 7.3 (kernel 2.4.10-4GB)
When I say stable I mean the server has been running without problems for
over twelve months, handling light/medium levels of traffic for 450 lists.
The only interruptions have been for the odd MM or Python or OS update.
Also an occasional rehearsal of up primary-to-backup switchover procedure..
The current configuration has been untouched since August and the only
recent interruption of service was due to an untidy reboot about 10 days
ago after we had a total loss of mains power and the UPS batteries ran out
before the mains came back.
At around 01:30 this morning, the qrunner began failing. It was reported as
a python ImportError exception when $prefix/Mailman/Utils called form
$prefix/cron/qrunner executed the code line 'from UserDict import UserDict'
This was a showstopper and not the sort of problem that passes unnoticed
I spent several frantic hours of trying figure out what had gone wrong. I
switched service to our identical backup server (the MM python code and
data are held on NFS mounts from our Auspex server and we have a rehearsed
and fairly quick procedure for swapping the primary and backup machine
identities using DHCP) . No avail. A final last ditch reinstall of both
Python and Mailman from source code on the current primary server and the
qrunner was still failing in the same way.
I then stumbled across a UserDict.pyc file in the $prefix/Mailman
directory; just a .pyc, no matching .py
But, I thought, UserDict.pyc resides in /usr/local/lib/python2.2/, it being
a standard library module. That UserDict.pyc was in its proper place.
After removing the 'extra' UserDict.py from $prefix/Mailman the qrunner
started working normally.
During the investigation of the problem I found that, when running python
from the command line, executing 'from UserDict import UserDict' as
expected worked if and only if it had not been preceded by an import of
MM's paths.py which inserts $prefix at the start of the sys.path. Which I
thought was pretty weird but maybe the 'extra' UserDict.pyc has that effect.
What I cannot explain is why a UserDict.pyc file popped up in the
$prefix/Mailman directory, presumably in the middle of last night, when the
service interruption started. Unfortunately, I deleted rather than renamed
that 'extra' UserDict.pyc so the evidence is gone; my justification is that
I was in something of a rush to get Mailman back on air before our user
community lynched me.
I'm inclined to rule out some malicious modification of the system. The MM
server is dedicated to running Mailman, resides in a locked computer room,
has a private IP number, resides inside a well structured firewall, only
gets its mail from our primary mail server after virus checking and I can
find no sign of tampering with the system from inside or outside our domain.
Has anybody else experienced anything similar to this or have any idea how
this problem arose?
Any advice or help welcome. I just want to avoid another morning of blind
panic resulting from this sort of problem.
More information about the Mailman-Users