[Mailman-Users] unable to access admin interface
Chad Rebuck
chadrebuck at gmail.com
Wed Jul 23 17:05:59 CEST 2014
Here is additional information. The log entries for "lifetime has expired"
started happening much more frequently after upgrading to 2.1.18-1.
This is the most recent complete log entry for the "lifetime has expired":
Jul 23 03:47:28 2014 (21881) a2-16v-list.lock lifetime has expired, breaking
Jul 23 03:47:28 2014 (21881) File "/usr/lib/mailman/bin/qrunner", line
278, in <module>
Jul 23 03:47:28 2014 (21881) main()
Jul 23 03:47:28 2014 (21881) File "/usr/lib/mailman/bin/qrunner", line
238, in main
Jul 23 03:47:28 2014 (21881) qrunner.run()
Jul 23 03:47:28 2014 (21881) File
"/usr/lib/mailman/Mailman/Queue/Runner.py", line 70, in run
Jul 23 03:47:28 2014 (21881) filecnt = self._oneloop()
Jul 23 03:47:28 2014 (21881) File
"/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
Jul 23 03:47:28 2014 (21881) self._onefile(msg, msgdata)
Jul 23 03:47:28 2014 (21881) File
"/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
Jul 23 03:47:28 2014 (21881) keepqueued = self._dispose(mlist, msg,
msgdata)
Jul 23 03:47:28 2014 (21881) File
"/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 115, in _dispose
Jul 23 03:47:28 2014 (21881)
mlist.Lock(timeout=mm_cfg.LIST_LOCK_TIMEOUT)
Jul 23 03:47:28 2014 (21881) File "/usr/lib/mailman/Mailman/MailList.py",
line 161, in Lock
Jul 23 03:47:28 2014 (21881) self.__lock.lock(timeout)
Jul 23 03:47:28 2014 (21881) File "/usr/lib/mailman/Mailman/LockFile.py",
line 306, in lock
Jul 23 03:47:28 2014 (21881) important=True)
Jul 23 03:47:28 2014 (21881) File "/usr/lib/mailman/Mailman/LockFile.py",
line 416, in __writelog
Jul 23 03:47:28 2014 (21881) traceback.print_stack(file=logf)
[root at pogo chadr]# grep lifetime /var/lib/mailman/logs/locks
Feb 01 23:05:41 2012 (19141) master-qrunner lifetime has expired, breaking
Dec 19 08:48:06 2012 (25356) master-qrunner lifetime has expired, breaking
Dec 28 17:54:14 2012 (588) master-qrunner lifetime has expired, breaking
Jun 05 03:02:35 2013 (249) master-qrunner lifetime has expired, breaking
Sep 09 09:25:40 2013 (235) a2-16v-list.lock lifetime has expired, breaking
Dec 14 06:30:53 2013 (251) a2-16v-list.lock lifetime has expired, breaking
Feb 05 22:49:50 2014 (247) a2-16v-list.lock lifetime has expired, breaking
Mar 02 22:31:50 2014 (189) master-qrunner lifetime has expired, breaking
Jun 27 17:05:45 2014 (344) master-qrunner lifetime has expired, breaking
--> this is when mailman was upgraded to 2.1.18-1
Jul 20 18:08:00 2014 (365) a2-16v-list.lock lifetime has expired, breaking
Jul 20 23:15:50 2014 (346) a2-16v-list.lock lifetime has expired, breaking
Jul 21 13:58:47 2014 (13825) a2-16v-list.lock lifetime has expired, breaking
Jul 22 22:01:47 2014 (21735) a2-16v-list.lock lifetime has expired, breaking
Jul 23 03:47:28 2014 (21881) a2-16v-list.lock lifetime has expired, breaking
[root at pogo chadr]# grep mailman /var/log/pacman.log
[2011-08-04 23:11] Running 'pacman -S mailman'
[2011-08-04 23:12] Running 'pacman -S mailman'
[2011-08-04 23:15] installed mailman (2.1.14-1)
[2012-04-09 18:04] upgraded mailman (2.1.14-1 -> 2.1.14-3)
[2012-04-30 15:56] Running 'pacman -S mailman'
[2012-04-30 15:57] upgraded mailman (2.1.14-3 -> 2.1.14-3)
[2012-04-30 16:01] Running 'pacman -U
/var/cache/pacman/pkg/mailman-2.1.14-1-arm.pkg.tar.xz'
[2012-04-30 16:02] upgraded mailman (2.1.14-3 -> 2.1.14-1)
[2012-04-30 18:03] Running 'pacman -S mailman'
[2012-04-30 18:14] Running 'pacman -Syu mailman'
[2012-04-30 18:15] Running 'pacman -S mailman'
[2012-04-30 18:16] upgraded mailman (2.1.14-1 -> 2.1.14-4)
[2012-08-14 09:37] upgraded mailman (2.1.14-4 -> 2.1.15-2)
[2013-02-05 03:06] upgraded mailman (2.1.15-2 -> 2.1.15-3)
[2014-03-06 12:37] [PACMAN] upgraded mailman (2.1.15-3 -> 2.1.17-1)
[2014-07-20 17:37] [PACMAN] Running 'pacman -Syu mailman'
[2014-07-20 17:37] [PACMAN] Running 'pacman -S mailman'
[2014-07-20 17:38] [PACMAN] upgraded mailman (2.1.17-1 -> 2.1.18.1-1)
[root at pogo chadr]#
On Wed, Jul 23, 2014 at 9:54 AM, Chad Rebuck <chadrebuck at gmail.com> wrote:
> I installed it with a package on my arch system using
>
> pacman -Syu
>
> [root at pogo chadr]# ll /var/lib/mailman/locks/
> total 16K
> -rw-rw-r-- 1 http mailman 52 Jul 23 03:52 a2-16v-list.lock.pogo.22018.0
> -rw-rw-r-- 1 http mailman 52 Jul 23 04:45 a2-16v-list.lock.pogo.22205.0
> -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner
> -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner.pogo.341
>
> Then I tried to access
>
> http://mail.a2-16v.com/mailman/admindb/a2-16v-list and waited for the 504
> error to appear in the browser
>
>
> [root at pogo chadr]# ll /var/lib/mailman/locks/
> total 24K
> -rw-rw-r-- 2 http mailman 52 Jul 23 2014 a2-16v-list.lock
> -rw-rw-r-- 1 http mailman 52 Jul 23 03:52 a2-16v-list.lock.pogo.22018.0
> -rw-rw-r-- 1 http mailman 52 Jul 23 04:45 a2-16v-list.lock.pogo.22205.0
> -rw-rw-r-- 2 http mailman 52 Jul 23 2014 a2-16v-list.lock.pogo.25275.0
> -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner
> -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner.pogo.341
>
> No recent updates to the locks or error log
>
> [root at pogo chadr]# tail /var/lib/mailman/logs/locks
> Jul 23 03:47:28 2014 (21881) File
> "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
> Jul 23 03:47:28 2014 (21881) keepqueued = self._dispose(mlist, msg,
> msgdata)
> Jul 23 03:47:28 2014 (21881) File
> "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 115, in _dispose
> Jul 23 03:47:28 2014 (21881)
> mlist.Lock(timeout=mm_cfg.LIST_LOCK_TIMEOUT)
> Jul 23 03:47:28 2014 (21881) File
> "/usr/lib/mailman/Mailman/MailList.py", line 161, in Lock
> Jul 23 03:47:28 2014 (21881) self.__lock.lock(timeout)
> Jul 23 03:47:28 2014 (21881) File
> "/usr/lib/mailman/Mailman/LockFile.py", line 306, in lock
> Jul 23 03:47:28 2014 (21881) important=True)
> Jul 23 03:47:28 2014 (21881) File
> "/usr/lib/mailman/Mailman/LockFile.py", line 416, in __writelog
> Jul 23 03:47:28 2014 (21881) traceback.print_stack(file=logf)
>
> [root at pogo chadr]# tail /var/lib/mailman/logs/error
> Jun 20 06:32:44 2014 mailmanctl(382): lock = acquire_lock_1(force)
> Jun 20 06:32:44 2014 mailmanctl(382): File
> "/usr/lib/mailman/bin/mailmanctl", line 203, in acquire_lock_1
> Jun 20 06:32:44 2014 mailmanctl(382): if not force or qrunner_state():
> Jun 20 06:32:44 2014 mailmanctl(382): File
> "/usr/lib/mailman/bin/mailmanctl", line 187, in qrunner_state
> Jun 20 06:32:44 2014 mailmanctl(382): os.kill(pid, 0)
> Jun 20 06:32:44 2014 mailmanctl(382): OSError : [Errno 1] Operation not
> permitted
> Jun 21 01:24:28 2014 (3941) No such list "a2-16v-list<":
> Jul 20 17:47:52 2014 mailmanctl(20690): The master qrunner lock could not
> be acquired because it appears as if another
> master qrunner is already running.
> Jul 20 17:47:52 2014 mailmanctl(20690):
>
> [root at pogo chadr]# date
> Wed Jul 23 09:51:56 EDT 2014
>
>
>
> On Wed, Jul 23, 2014 at 12:00 AM, Mark Sapiro <mark at msapiro.net> wrote:
>
>> On 07/22/2014 07:25 PM, Chad Rebuck wrote:
>> > I didn't remove it until now, but I saw it seemed to update itself when
>> I
>> > tried to access the admin page.
>> >
>> > [root at pogo locks]# ll
>> > total 16K
>> > -rw-rw-r-- 1 http mailman 52 Jul 23 2014 a2-16v-list.lock
>> > -rw-rw-r-- 1 http mailman 52 Jul 23 2014
>> a2-16v-list.lock.pogo.21765.0
>> > -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner
>> > -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner.pogo.341
>> >
>> > [root at pogo locks]# rm -rf a2-16v-list.lock*
>> >
>> > [root at pogo locks]# ll
>> > total 8.0K
>> > -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner
>> > -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner.pogo.341
>> >
>> >
>> > I was then able to access the admin page, but when I clicked on "tend to
>> > pending requests" I got the 504 error after a couple minutes
>> >
>> > [root at pogo locks]# tail -2 /var/log/httpd/error_log
>> > [Tue Jul 22 22:22:30.721677 2014] [cgi:error] [pid 378:tid 1143579696]
>> > [client 144.160.5.25:34507] Script timed out before returning headers:
>> > admindb, referer: http://mail.a2-16v.com/mailman/admin/a2-16v-list
>> > [Tue Jul 22 22:23:30.781402 2014] [cgi:warn] [pid 378:tid 1143579696]
>> > [client 144.160.5.25:34507] AH01220: Timeout waiting for output from
>> CGI
>> > script /usr/lib/mailman/cgi-bin/admindb, referer:
>> > http://mail.a2-16v.com/mailman/admin/a2-16v-list
>> > [root at pogo locks]# tail -2 /var/log/httpd/access_log
>> > 144.160.5.25 - - [22/Jul/2014:22:21:14 -0400] "GET
>> > /mailman/admin/a2-16v-list HTTP/1.1" 200 20748
>> > 144.160.5.25 - - [22/Jul/2014:22:21:30 -0400] "GET
>> > /mailman/admindb/a2-16v-list HTTP/1.1" 504 247
>> >
>> > now the lock folder is as follows
>> >
>> > [root at pogo locks]# ll
>> > total 16K
>> > -rw-rw-r-- 2 http mailman 52 Jul 23 2014 a2-16v-list.lock
>> > -rw-rw-r-- 2 http mailman 52 Jul 23 2014
>> a2-16v-list.lock.pogo.21828.0
>> > -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner
>> > -rw-rw-r-- 2 mailman mailman 46 Jul 24 2014 master-qrunner.pogo.341
>> > [root at pogo locks]#
>>
>>
>> So PID 21828 has the list lock. It looks like the admindb CGI, PID 378
>> timed out, presumably because it couldn't get the lock, but its request
>> would be a file named a2-16v-list.lock.pogo.378.0, and that doesn't seem
>> to be there.
>>
>> What if anything is in Mailman's 'error' log and 'locks' log?
>>
>> Locking may be the underlying issue, but it's not just stale locks
>> because even after they're cleared, you still have issues.
>>
>> How did you install 2.1.18-1?
>>
>> --
>> Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
>> San Francisco Bay Area, California better use your sense - B. Dylan
>>
>
>
More information about the Mailman-Users
mailing list