![](https://secure.gravatar.com/avatar/3e6a60db6f6233bdfaf25099b95218ec.jpg?s=120&d=mm&r=g)
Hello,
I just found this very useful looking script but am getting the following errors when I try running it.
./prune_arch -l mgsa-l -d 1095 -n
Traceback (most recent call last):
File "./prune_arch", line 191, in ?
main()
File "./prune_arch", line 156, in main
mdate = mktime_tz(parsedate_tz(msg['date']))
File "/usr/local/mailman/pythonlib/email/_parseaddr.py", line 140, in mktime_tz
if data[9] is None:
TypeError: unsubscriptable object
Any help would be appreciated
Con Wieland Office of Information Technology University of California at Irvine
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Con Wieland wrote:
The issue is the script didn't take into account messages in the mailbox without Date: headers. The scripts at <http://www.msapiro.net/scripts/prune_arch> and <http://fog.ccsf.cc.ca.us/~msapiro/scripts/prune_arch> have been updated.
The fix is to change line 156 from
mdate = mktime_tz(parsedate_tz(msg['date']))
to
mdate = msg['date'] and
mktime_tz(parsedate_tz(msg['date']))
(watch out for wrapping above)
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/3e6a60db6f6233bdfaf25099b95218ec.jpg?s=120&d=mm&r=g)
Now it processes some but I now get:
./prune_arch -l mgsa-l -d 1095 -bpnv Processing mgsa-l mailbox Traceback (most recent call last): File "./prune_arch", line 191, in ? main() File "./prune_arch", line 156, in main mdate = msg['date'] and mktime_tz(parsedate_tz(msg['date'])) TypeError: string indices must be integers
My "date' command gives this format Wed Sep 26 08:21:50 PDT 2012
con
On Sep 25, 2012, at 2:07 PM, Mark Sapiro wrote:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 9/26/2012 8:27 AM, Con Wieland wrote:
I don't know what the issue could be. The code is doing
for msg in mbox:
msgs += 1
mdate = msg['date'] and mktime_tz(parsedate_tz(msg['date']))
And the above error seems to be saying that msg is not a mailbox.Message object, but I don't know what that could be. I have made a change in the script changing line 144 from
mbox = Mailbox.Mailbox(mboxf)
to
mbox = Mailbox.mbox(mboxf)
The former was not strictly correct, but worked in my tests. The online scripts have been updated.
If this doesn't fix it, I think I'll need access to the actual mailbox to debug this.
My "date' command gives this format Wed Sep 26 08:21:50 PDT 2012
That's not relevant to this problem.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/3e6a60db6f6233bdfaf25099b95218ec.jpg?s=120&d=mm&r=g)
The mbox variable changed seemed to take care of that error. Now after running for about 4 hours I get this
./prune_arch -l mgsa-l -d 1095 -bpnv Traceback (most recent call last): File "./prune_arch", line 191, in ? main() File "./prune_arch", line 173, in main os.umask(omask) UnboundLocalError: local variable 'omask' referenced before assignment
any ideas, thanks
On Sep 26, 2012, at 8:53 AM, Mark Sapiro wrote:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Con Wieland wrote:
The mbox variable changed seemed to take care of that error.
Actually, it didn't. That change was totally wrong, my bad.
I think it ran for 4 hours because the list was left locked by the previous failure and at least part of that time, it was waiting for the list lock to expire. I don't know how that would have happened though because the list should have been unlocked in the finally: clause of the try: in which the error occurred.
The above error is because of the incorrect change of line 144 from
mbox = Mailbox.Mailbox(mboxf)
to
mbox = Mailbox.mbox(mboxf)
This threw an exception because of the erroneous change.
I have changed the script again to move the
omask = os.umask(2)
line ahead of the try: finally: to protect against the above error and to restore the
mbox = Mailbox.Mailbox(mboxf)
line.
I also added a test for an unparseable message being returned as a null string by Mailman's Mailbox class which was the cause of the
mdate = msg['date'] and mktime_tz(parsedate_tz(msg['date']))
TypeError: string indices must be integers
exception.
The current script is at <http://www.msapiro.net/scripts/prune_arch> and <http://fog.ccsf.cc.ca.us/~msapiro/scripts/prune_arch>.
See the FAQ at <http://wiki.list.org/x/noA9> and remove the stale locks for the mgsa-l list.
Then, before running prune_arch again, try checking the archives/private/mgsa-l.mbox/mgsa-l.mbox file for unescaped From_ lines with bin/cleanarch and fix any problems.
This may or may not fix the unparseable message issue, but now the script should just report it and continue.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
I have updated the scripts at <http://www.msapiro.net/scripts/prune_arch> and <http://fog.ccsf.cc.ca.us/~msapiro/scripts/prune_arch> again. The script will now skip unparseable messages and messages with missing or unparseable Date: headers and report the same.
It only reports the sequence number of the message in the mbox, but if you run the script with the -b or --backup option, you can find the skipped message(s) in the listname.mbox.bak file with something like
grep "^From " listname.mbox.bak | grep -n . | grep ^xxx:
where xxx is the sequence number of a skipped message.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/3e6a60db6f6233bdfaf25099b95218ec.jpg?s=120&d=mm&r=g)
back at trying this. I now get:
./prune_arch -l mgsa-l -d 1095 -bpnv Processing mgsa-l mailbox Traceback (most recent call last): File "./prune_arch", line 191, in ? main() File "./prune_arch", line 156, in main mdate = mktime_tz(parsedate_tz(msg['date'])) File "/usr/local/mailman/pythonlib/email/_parseaddr.py", line 140, in mktime_tz if data[9] is None: TypeError: unsubscriptable object
thanks for the help
con
On Sep 27, 2012, at 7:27 AM, Mark Sapiro wrote:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Con Wieland wrote:
You do not have the latest version of the script. The version I installed on my web site on Sept 27 at 06:52 PDT (before posting the message you quote) catches and displays information about this exception. Either you got the script before that or you got an older, browser cached version.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/268485a169c06ee9994c34e6d8ac5f47.jpg?s=120&d=mm&r=g)
I seem to be getting a lot of "connection timeout" messages and I'd like to change my time out settings....and get some advice from the list on reducing the number of time outs.
I'm running Mailman 2.1.14 on OSX 10.6 client (NOT server) with MailServe for snow leopard
It seems like when I did this installation, there were tuning instructions I found that had information about changing time out settings, the number of threads and such, but I can't seem to find that information now and I use terminal so little that I don't want to play around and screw something up that's working fairly good.
thanks W
Wayne Cook wcook@mycoachonline.com http://www.mycoachonline.com/
I've flown faster than the speed of sound, stared down a shark while standing on the ocean floor and launched men into space. Care to join me tomorrow?
![](https://secure.gravatar.com/avatar/1c4a4096ba25b08fbf1a8cb1c4bf9fcb.jpg?s=120&d=mm&r=g)
I'd like to suggest that no one provide Mr. Cook any assistance. Since I helped answer a question of his some months ago, mycoachonline.com has been a persistent spam source on my server. The most recent piece of spam from his company was received 11 hours ago.
-- Larry Stone lstone19@stonejongleux.com
On Sep 26, 2012, at 1:57 PM, Wayne Cook <wcook@mycoachonline.com> wrote:
![](https://secure.gravatar.com/avatar/268485a169c06ee9994c34e6d8ac5f47.jpg?s=120&d=mm&r=g)
And I'd like to know what address you claim is being spammed? I can't find anything in any of my databases that contain "stonejongleux.com"
w
Wayne Cook wcook@mycoachonline.com http://www.mycoachonline.com/
I live in a world where mistakes are evil, accidents are newsworthy and being late is allowed once.
On Sep 26, 2012, at 6:39 PM, Larry Stone wrote:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Wayne Cook wrote:
I seem to be getting a lot of "connection timeout" messages and I'd like to change my time out settings....and get some advice from the list on reducing the number of time outs.
Please don't hijack threads. Compost a new message to the list rather than replying to an unrelated message.
Is this even a Mailman question? Where do these "connection timeout" messages occur? If on a web access, it would seem to be a web server or DNS issue. If on a connect to the MTA, a MTA issue?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Con Wieland wrote:
The issue is the script didn't take into account messages in the mailbox without Date: headers. The scripts at <http://www.msapiro.net/scripts/prune_arch> and <http://fog.ccsf.cc.ca.us/~msapiro/scripts/prune_arch> have been updated.
The fix is to change line 156 from
mdate = mktime_tz(parsedate_tz(msg['date']))
to
mdate = msg['date'] and
mktime_tz(parsedate_tz(msg['date']))
(watch out for wrapping above)
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/3e6a60db6f6233bdfaf25099b95218ec.jpg?s=120&d=mm&r=g)
Now it processes some but I now get:
./prune_arch -l mgsa-l -d 1095 -bpnv Processing mgsa-l mailbox Traceback (most recent call last): File "./prune_arch", line 191, in ? main() File "./prune_arch", line 156, in main mdate = msg['date'] and mktime_tz(parsedate_tz(msg['date'])) TypeError: string indices must be integers
My "date' command gives this format Wed Sep 26 08:21:50 PDT 2012
con
On Sep 25, 2012, at 2:07 PM, Mark Sapiro wrote:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 9/26/2012 8:27 AM, Con Wieland wrote:
I don't know what the issue could be. The code is doing
for msg in mbox:
msgs += 1
mdate = msg['date'] and mktime_tz(parsedate_tz(msg['date']))
And the above error seems to be saying that msg is not a mailbox.Message object, but I don't know what that could be. I have made a change in the script changing line 144 from
mbox = Mailbox.Mailbox(mboxf)
to
mbox = Mailbox.mbox(mboxf)
The former was not strictly correct, but worked in my tests. The online scripts have been updated.
If this doesn't fix it, I think I'll need access to the actual mailbox to debug this.
My "date' command gives this format Wed Sep 26 08:21:50 PDT 2012
That's not relevant to this problem.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/3e6a60db6f6233bdfaf25099b95218ec.jpg?s=120&d=mm&r=g)
The mbox variable changed seemed to take care of that error. Now after running for about 4 hours I get this
./prune_arch -l mgsa-l -d 1095 -bpnv Traceback (most recent call last): File "./prune_arch", line 191, in ? main() File "./prune_arch", line 173, in main os.umask(omask) UnboundLocalError: local variable 'omask' referenced before assignment
any ideas, thanks
On Sep 26, 2012, at 8:53 AM, Mark Sapiro wrote:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Con Wieland wrote:
The mbox variable changed seemed to take care of that error.
Actually, it didn't. That change was totally wrong, my bad.
I think it ran for 4 hours because the list was left locked by the previous failure and at least part of that time, it was waiting for the list lock to expire. I don't know how that would have happened though because the list should have been unlocked in the finally: clause of the try: in which the error occurred.
The above error is because of the incorrect change of line 144 from
mbox = Mailbox.Mailbox(mboxf)
to
mbox = Mailbox.mbox(mboxf)
This threw an exception because of the erroneous change.
I have changed the script again to move the
omask = os.umask(2)
line ahead of the try: finally: to protect against the above error and to restore the
mbox = Mailbox.Mailbox(mboxf)
line.
I also added a test for an unparseable message being returned as a null string by Mailman's Mailbox class which was the cause of the
mdate = msg['date'] and mktime_tz(parsedate_tz(msg['date']))
TypeError: string indices must be integers
exception.
The current script is at <http://www.msapiro.net/scripts/prune_arch> and <http://fog.ccsf.cc.ca.us/~msapiro/scripts/prune_arch>.
See the FAQ at <http://wiki.list.org/x/noA9> and remove the stale locks for the mgsa-l list.
Then, before running prune_arch again, try checking the archives/private/mgsa-l.mbox/mgsa-l.mbox file for unescaped From_ lines with bin/cleanarch and fix any problems.
This may or may not fix the unparseable message issue, but now the script should just report it and continue.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
I have updated the scripts at <http://www.msapiro.net/scripts/prune_arch> and <http://fog.ccsf.cc.ca.us/~msapiro/scripts/prune_arch> again. The script will now skip unparseable messages and messages with missing or unparseable Date: headers and report the same.
It only reports the sequence number of the message in the mbox, but if you run the script with the -b or --backup option, you can find the skipped message(s) in the listname.mbox.bak file with something like
grep "^From " listname.mbox.bak | grep -n . | grep ^xxx:
where xxx is the sequence number of a skipped message.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/3e6a60db6f6233bdfaf25099b95218ec.jpg?s=120&d=mm&r=g)
back at trying this. I now get:
./prune_arch -l mgsa-l -d 1095 -bpnv Processing mgsa-l mailbox Traceback (most recent call last): File "./prune_arch", line 191, in ? main() File "./prune_arch", line 156, in main mdate = mktime_tz(parsedate_tz(msg['date'])) File "/usr/local/mailman/pythonlib/email/_parseaddr.py", line 140, in mktime_tz if data[9] is None: TypeError: unsubscriptable object
thanks for the help
con
On Sep 27, 2012, at 7:27 AM, Mark Sapiro wrote:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Con Wieland wrote:
You do not have the latest version of the script. The version I installed on my web site on Sept 27 at 06:52 PDT (before posting the message you quote) catches and displays information about this exception. Either you got the script before that or you got an older, browser cached version.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/268485a169c06ee9994c34e6d8ac5f47.jpg?s=120&d=mm&r=g)
I seem to be getting a lot of "connection timeout" messages and I'd like to change my time out settings....and get some advice from the list on reducing the number of time outs.
I'm running Mailman 2.1.14 on OSX 10.6 client (NOT server) with MailServe for snow leopard
It seems like when I did this installation, there were tuning instructions I found that had information about changing time out settings, the number of threads and such, but I can't seem to find that information now and I use terminal so little that I don't want to play around and screw something up that's working fairly good.
thanks W
Wayne Cook wcook@mycoachonline.com http://www.mycoachonline.com/
I've flown faster than the speed of sound, stared down a shark while standing on the ocean floor and launched men into space. Care to join me tomorrow?
![](https://secure.gravatar.com/avatar/1c4a4096ba25b08fbf1a8cb1c4bf9fcb.jpg?s=120&d=mm&r=g)
I'd like to suggest that no one provide Mr. Cook any assistance. Since I helped answer a question of his some months ago, mycoachonline.com has been a persistent spam source on my server. The most recent piece of spam from his company was received 11 hours ago.
-- Larry Stone lstone19@stonejongleux.com
On Sep 26, 2012, at 1:57 PM, Wayne Cook <wcook@mycoachonline.com> wrote:
![](https://secure.gravatar.com/avatar/268485a169c06ee9994c34e6d8ac5f47.jpg?s=120&d=mm&r=g)
And I'd like to know what address you claim is being spammed? I can't find anything in any of my databases that contain "stonejongleux.com"
w
Wayne Cook wcook@mycoachonline.com http://www.mycoachonline.com/
I live in a world where mistakes are evil, accidents are newsworthy and being late is allowed once.
On Sep 26, 2012, at 6:39 PM, Larry Stone wrote:
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
Wayne Cook wrote:
I seem to be getting a lot of "connection timeout" messages and I'd like to change my time out settings....and get some advice from the list on reducing the number of time outs.
Please don't hijack threads. Compost a new message to the list rather than replying to an unrelated message.
Is this even a Mailman question? Where do these "connection timeout" messages occur? If on a web access, it would seem to be a web server or DNS issue. If on a connect to the MTA, a MTA issue?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (4)
-
Con Wieland
-
Larry Stone
-
Mark Sapiro
-
Wayne Cook