Debugging mass invitations using web page interface...
![](https://secure.gravatar.com/avatar/968b4f8351b892470cf934cde87db134.jpg?s=120&d=mm&r=g)
Dear Mailman Cognoscenti,
I'm helping one of my list owners send out 5K plus invitations to students to subscribe to his mailing list. Our current configuration:
Mailman v2.1.20
RHEL v5.11
Semdmail v8.13.8
Apache v2.2.3
Since this was the first time doing this, I suggested breaking the batch input into 3 groups, 50, 500, and the rest. The 50 went fine, as did the 500, but the largest batch gave him a generic web server error:
I looked at the logs and I couldn't find anything that hinted at what went wrong. So I asked the owner to send me the last back and I'd give it a try. I wrote a script that removed folks already subscribed to his list and split the remaining subscribers up into 6 files with a thousand records each. I just tried uploading the 1st batch of 1K, with the following options:
Subscribe these users now... (*) Invite
Send welcome message... (*) No
Send notifications... (*) No
And entered a 7 line paragraph explaining the invitation.
I ended up having the same error happen. Looking at the Mailman logs, I can't see any difference before or after my submission. In the HTTPD logs, I see:
So is there an inherent limit to the number of invites that can be submitted via the web form?
As a work around, how would I do large invites on behalf of the owner from the command line, including the 'extra text' that is allowed via the web interface?
-- Cordially, the UNH Mailing List Server Admins Bill Costa, senior admin (603) 862-3056
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 08/26/2016 05:24 PM, The Mailing List Server Admin wrote:
The admin CGI is processing the mass subscribe/invite list. While it is doing this it is not generating any output. If it takes 'too long' the web server will time out waiting for output from the CGI. That's what's happening here.
The amount of time is set by apache's TimeOut directive <http://httpd.apache.org/docs/current/mod/core.html#timeout>
You'd need a custom script to do the 'extra text'. I have added an invite_members script I use at <https://www.msapiro.net/scripts/invite_members> and <https://fog.ccsf.edu/~msapiro/scripts/invite_members>. This can send invitations including the 'extra text'.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/968b4f8351b892470cf934cde87db134.jpg?s=120&d=mm&r=g)
Mark, once again thank you for your valued expertise...
With regards to the failed thousand invitations submission via the Web UI:
Makes perfect sense. (I wonder if MM3 uses Ajax or other web techniques to provide some sort of progress indicator to prevent this from happening.)
So when Apache timed out, would it have killed the admin CGI on the way out? In other words, would the admin script still run to completion and send out all the invitations from that batch? Or is it most likely to have been a partial run? Am I correct that the only place I will find a record of the invitations sent will be in the Sendmail mail logs? I'd like to avoid sending dup invites if possible.
Thanks for pointer to the command line invitation script. I'll add it to my growing arsenal.
-- Cordially, the UNH Mailing List Server Admins Bill Costa, senior admin (603) 862-3056
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 08/27/2016 03:16 AM, The Mailing List Server Admin wrote:
When Apache times out, it kills the CGI process. The process meanwhile has been queueing the invitations to be sent, so a bunch will be sent.
You could use the script at <https://www.msapiro.net/scripts/list_pending> to list the pending subscriptions. Some may have been accepted already, so the fact that one isn't there doesn't mean it wasn't sent, but if you know the order from the original input, you can probably figure how far it got.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/968b4f8351b892470cf934cde87db134.jpg?s=120&d=mm&r=g)
Mark Sapiro recently wrote (in part):
Between being able to see who is now already subscribed from my original list, and looking at the pending subscription requests, I should be able to figure out who we've missed. Thanks for pointing out yet another useful MM2 utility!
-- Cordially, the UNH Mailing List Server Admins Bill Costa, senior admin (603) 862-3056
![](https://secure.gravatar.com/avatar/968b4f8351b892470cf934cde87db134.jpg?s=120&d=mm&r=g)
Way way back in August 2016, we had a problem with submitting too many members via the web interface. As a solution, the following was suggested:
We used this script with great success in Aug 2017 and 2018. But this year we've run into a problem. When I submit the script it appears to hang waiting for some kind of lock. I can see this when I eventually CTRL-C out.
Traceback (most recent call last):
File "/home/mladmin/Util/invite_members", line 236, in ?
main()
File "/home/mladmin/Util/invite_members", line 197, in main
mlist = MailList.MailList(listname)
File "/usr/local/mailman/Mailman/MailList.py", line 129, in
__init__
self.Lock()
File "/usr/local/mailman/Mailman/MailList.py", line 162, in
Lock
self.__lock.lock(timeout)
File "/usr/local/mailman/Mailman/LockFile.py", line 312, in
lock
self.__sleep()
File "/usr/local/mailman/Mailman/LockFile.py", line 496, in
__sleep
time.sleep(interval)
KeyboardInterrupt
In the past we've used the script from a non-priv'd account with no problems. When it failed this time, I tried running it as 'mailman' but it still hangs on the lock. Any suggestions? (BTW, the line numbers will be different from the original script because of some debug print statements and additional comments added.)
-- Cordially, the UNH Mailing List Server Admins Bill Costa, Adjunct Admin (603) 862-3056
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 8/26/19 3:28 PM, The Mailing List Server Admin wrote:
The list is locked. See <https://wiki.list.org/x/17891756> for more about locks and removing stale locks.
I have looked at the script and I see an issue. It can leave the list locked if you specify a message file and that file cannot be read.
I have updated the script to fix this.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 08/26/2016 05:24 PM, The Mailing List Server Admin wrote:
The admin CGI is processing the mass subscribe/invite list. While it is doing this it is not generating any output. If it takes 'too long' the web server will time out waiting for output from the CGI. That's what's happening here.
The amount of time is set by apache's TimeOut directive <http://httpd.apache.org/docs/current/mod/core.html#timeout>
You'd need a custom script to do the 'extra text'. I have added an invite_members script I use at <https://www.msapiro.net/scripts/invite_members> and <https://fog.ccsf.edu/~msapiro/scripts/invite_members>. This can send invitations including the 'extra text'.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/968b4f8351b892470cf934cde87db134.jpg?s=120&d=mm&r=g)
Mark, once again thank you for your valued expertise...
With regards to the failed thousand invitations submission via the Web UI:
Makes perfect sense. (I wonder if MM3 uses Ajax or other web techniques to provide some sort of progress indicator to prevent this from happening.)
So when Apache timed out, would it have killed the admin CGI on the way out? In other words, would the admin script still run to completion and send out all the invitations from that batch? Or is it most likely to have been a partial run? Am I correct that the only place I will find a record of the invitations sent will be in the Sendmail mail logs? I'd like to avoid sending dup invites if possible.
Thanks for pointer to the command line invitation script. I'll add it to my growing arsenal.
-- Cordially, the UNH Mailing List Server Admins Bill Costa, senior admin (603) 862-3056
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 08/27/2016 03:16 AM, The Mailing List Server Admin wrote:
When Apache times out, it kills the CGI process. The process meanwhile has been queueing the invitations to be sent, so a bunch will be sent.
You could use the script at <https://www.msapiro.net/scripts/list_pending> to list the pending subscriptions. Some may have been accepted already, so the fact that one isn't there doesn't mean it wasn't sent, but if you know the order from the original input, you can probably figure how far it got.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/968b4f8351b892470cf934cde87db134.jpg?s=120&d=mm&r=g)
Mark Sapiro recently wrote (in part):
Between being able to see who is now already subscribed from my original list, and looking at the pending subscription requests, I should be able to figure out who we've missed. Thanks for pointing out yet another useful MM2 utility!
-- Cordially, the UNH Mailing List Server Admins Bill Costa, senior admin (603) 862-3056
![](https://secure.gravatar.com/avatar/968b4f8351b892470cf934cde87db134.jpg?s=120&d=mm&r=g)
Way way back in August 2016, we had a problem with submitting too many members via the web interface. As a solution, the following was suggested:
We used this script with great success in Aug 2017 and 2018. But this year we've run into a problem. When I submit the script it appears to hang waiting for some kind of lock. I can see this when I eventually CTRL-C out.
Traceback (most recent call last):
File "/home/mladmin/Util/invite_members", line 236, in ?
main()
File "/home/mladmin/Util/invite_members", line 197, in main
mlist = MailList.MailList(listname)
File "/usr/local/mailman/Mailman/MailList.py", line 129, in
__init__
self.Lock()
File "/usr/local/mailman/Mailman/MailList.py", line 162, in
Lock
self.__lock.lock(timeout)
File "/usr/local/mailman/Mailman/LockFile.py", line 312, in
lock
self.__sleep()
File "/usr/local/mailman/Mailman/LockFile.py", line 496, in
__sleep
time.sleep(interval)
KeyboardInterrupt
In the past we've used the script from a non-priv'd account with no problems. When it failed this time, I tried running it as 'mailman' but it still hangs on the lock. Any suggestions? (BTW, the line numbers will be different from the original script because of some debug print statements and additional comments added.)
-- Cordially, the UNH Mailing List Server Admins Bill Costa, Adjunct Admin (603) 862-3056
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 8/26/19 3:28 PM, The Mailing List Server Admin wrote:
The list is locked. See <https://wiki.list.org/x/17891756> for more about locks and removing stale locks.
I have looked at the script and I see an issue. It can leave the list locked if you specify a message file and that file cannot be read.
I have updated the script to fix this.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Mark Sapiro
-
The Mailing List Server Admin