MM feature request - spread out the pain
Perhaps this is on the feature request list already, or better yet, Barry borrowed Guido's time machine when I wasn't looking, however...
I help administer a number of python.org mailing lists. MM sends out subscription reminders on the first of each month. This inevitably leads to a huge number of mail bounces many of which MM doesn't understand. The python.org postmaster folk then get to go in and manually remove a ton of email addresses. (Looks like about 100 this month. I'm sure other sites have to handle many more such bounces.) I don't know how to automate this. The best thing I can think of to do is to write an Emacs macro which copies a highlighted email address, pops over to a shell mode window ssh'd into mail.python.org and runs remove_members with the given address. That's still more manual than I would like, but given the variability in bounce formats that seems about as good as I can do without expending a lot of effort.
I realize that getting Mailman to grok more rejection notice formats is a worthwhile goal, however I think it would also be helpful to spread the pain of invalid email addresses more evenly through the month, at least across mailing lists but ideally within mailing lists. Notification day for any given email might be
ord(email[0]) % 28 + 1
This would be close enough for gummint work (the 29th, 30th and 31st would be declared to be postmaster holidays).
-- Skip Montanaro - skip@pobox.com - http://www.smontanaro.net/
On Thu, Apr 01, 2010 at 11:59:05AM -0500, skip@pobox.com wrote:
Perhaps this is on the feature request list already, or better yet, Barry borrowed Guido's time machine when I wasn't looking, however...
I help administer a number of python.org mailing lists. MM sends out subscription reminders on the first of each month. This inevitably leads to a huge number of mail bounces many of which MM doesn't understand. The python.org postmaster folk then get to go in and manually remove a ton of email addresses. (Looks like about 100 this month. I'm sure other sites have to handle many more such bounces.) I don't know how to automate this. The best thing I can think of to do is to write an Emacs macro which copies a highlighted email address, pops over to a shell mode window ssh'd into mail.python.org and runs remove_members with the given address. That's still more manual than I would like, but given the variability in bounce formats that seems about as good as I can do without expending a lot of effort.
shove all the postmaster@ mails into one mailbox, and...
awk/grep the email addresses and list name as a (c|t)sv'd line; select a list, and feed the email addresses to remove_members ?
(maybe doing a check on the body content, too)
cron that to happen at $interval, on Mailman-day +n, perhaps?
I don't really have this problem, but very few of my lists (I think it may be ~300 now) send reminders.
-- ``Should not the Society of Indexers be known as Indexers, Society of, The?'' (Keith Waterhouse)
On Apr 01, 2010, at 11:59 AM, skip@pobox.com wrote:
I realize that getting Mailman to grok more rejection notice formats is a worthwhile goal, however I think it would also be helpful to spread the pain of invalid email addresses more evenly through the month, at least across mailing lists but ideally within mailing lists. Notification day for any given email might be
ord(email[0]) % 28 + 1
This would be close enough for gummint work (the 29th, 30th and 31st would be declared to be postmaster holidays).
Or how about we just get rid of them? :)
-Barry
skip@pobox.com wrote:
Perhaps this is on the feature request list already, or better yet, Barry borrowed Guido's time machine when I wasn't looking, however...
As barry already noted, there will be no reminders in MM 3.
I help administer a number of python.org mailing lists. MM sends out subscription reminders on the first of each month. This inevitably leads to a huge number of mail bounces many of which MM doesn't understand.
MM does not try to understand them because it can't handle them automatically. See the FAQ at http://wiki.list.org/x/aICB
The python.org postmaster folk then get to go in and manually remove a ton of email addresses. (Looks like about 100 this month. I'm sure other sites have to handle many more such bounces.) I don't know how to automate this. The best thing I can think of to do is to write an Emacs macro which copies a highlighted email address, pops over to a shell mode window ssh'd into mail.python.org and runs remove_members with the given address. That's still more manual than I would like, but given the variability in bounce formats that seems about as good as I can do without expending a lot of effort.
I realize that getting Mailman to grok more rejection notice formats is a worthwhile goal,
Mailman probably already can recognize almost all of these. The problem is not in the recognition, it's in what to do about them. In addition to the info in the above FAQ, there's lots of detail about this in comments in Mailman/Queue/BounceRunner.py.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Barry> Or how about we just get rid of them? :)
I wasn't sure what you meant until I saw Mark's reply. I guess I wasn't listening to the silence between the notes. ;-)
At any rate, that solves one problem but creates another, right? In MM3 how will you weed out bad email addresses? Again, this may already be accounted for somehow. If so, I'd be happy to learn how you do it without attempting to send email to an address then process the bounces.
Skip
skip@pobox.com wrote:
At any rate, that solves one problem but creates another, right? In MM3 how will you weed out bad email addresses? Again, this may already be accounted for somehow. If so, I'd be happy to learn how you do it without attempting to send email to an address then process the bounces.
If the list is active and the user's delivery is enabled, The list's bounce processing should take care of ultimately removing bouncing members. If it doesn't, the list's bounce processing needs to be adjusted.
The dead addresses of concern, are those for which the user or admin disabled delivery before the address died. We have talked about some kind of periodic mailing from the list to all members. If we have such a thing, a bounce of that mail could identify such addresses.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
At 03:01 PM 4/1/2010, Mark Sapiro wrote:
skip@pobox.com wrote:
At any rate, that solves one problem but creates another, right? In MM3 how will you weed out bad email addresses? Again, this may already be accounted for somehow. If so, I'd be happy to learn how you do it without attempting to send email to an address then process the bounces.
If the list is active and the user's delivery is enabled, The list's bounce processing should take care of ultimately removing bouncing members. If it doesn't, the list's bounce processing needs to be adjusted.
The dead addresses of concern, are those for which the user or admin disabled delivery before the address died. We have talked about some kind of periodic mailing from the list to all members. If we have such a thing, a bounce of that mail could identify such addresses.
Some other list software, I believe, has a feature that sends out a canned message once a month, such as list rules. I would like to see this in MM3 as I would like to send people the rules once a month, and not all my lists are that active, but still need to be in place and used occasionally.
Dave
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/dandrews%40visi.co...
Security Policy: http://wiki.list.org/x/QIA9
__________ Information from ESET Smart Security, version of virus signature database 4993 (20100401) __________
The message was checked by ESET Smart Security.
David Andrews writes:
Some other list software, I believe, has a feature that sends out a canned message once a month, such as list rules. I would like to see this in MM3
-1. This is mission creep. A cron job will serve perfectly well.
Something that assists in setting up such cron jobs (and/or generalizes to whatever weirdness Windows does instead of cron jobs) would be good as a contrib module, of course.
Stephen> David Andrews writes:
>> Some other list software, I believe, has a feature that sends out a
>> canned message once a month, such as list rules. I would like to see
>> this in MM3
Stephen> -1. This is mission creep. A cron job will serve perfectly
Stephen> well.
Stephen> Something that assists in setting up such cron jobs (and/or
Stephen> generalizes to whatever weirdness Windows does instead of cron
Stephen> jobs) would be good as a contrib module, of course.
I assume the monthly reminder emails have be canned for a reason. They do have the nice property of sending something to every subscribed address, even those who are set NOMAIL. I can do something from the command line (list_members? I use the command line so infrequently that I always have to look), but many list admins don't have shell access to their lists.
I hope I didn't open a can of worms. MM2 doesn't seem to have a through-the-web way to gather all list members from the admin's web interface. Will MM3? That would at least allow admins to assemble a list for sending all-hands or canned messages manually.
Skip
skip@pobox.com writes:
I assume the monthly reminder emails have be canned for a reason. They do have the nice property of sending something to every subscribed address, even those who are set NOMAIL.
That's a good point. I think a better way to handle this might be an Urgent flag (I guess we can't use Precedence for this, unfortunately, list mail should still be bulk in most cases) accompanied by an appropriate authentication.
This could eventually be generalized so that users could filter (some users might want only Announce or above, others of more catholic tasts would go for Spam or better<wink>). That might (or might not) be a better idea than having PROJECT-discuss subscribed to PROJECT-announce.
MM2 doesn't seem to have a through-the-web way to gather all list members from the admin's web interface. Will MM3?
I hope so[1]; this is a FAQ anyway.
Footnotes: [1] Ie, this should be done by Mailman 3's RESTful API, not an end-around by using an LDAP server or whatnot to hold the address db, and accessing the LDAP server outside of Mailman.
On Apr 02, 2010, at 09:30 PM, Stephen J. Turnbull wrote:
That's a good point. I think a better way to handle this might be an Urgent flag (I guess we can't use Precedence for this, unfortunately, list mail should still be bulk in most cases) accompanied by an appropriate authentication.
Which MM2 already has. :) MM3 currently uses the same mechanism, but I'm open to other ideas.
[1] Ie, this should be done by Mailman 3's RESTful API, not an end-around by using an LDAP server or whatnot to hold the address db, and accessing the LDAP server outside of Mailman.
Yep. -Barry
At 11:25 PM 4/1/2010, Stephen J. Turnbull wrote:
David Andrews writes:
Some other list software, I believe, has a feature that sends out a canned message once a month, such as list rules. I would like to see this in MM3
-1. This is mission creep. A cron job will serve perfectly well.
Something that assists in setting up such cron jobs (and/or generalizes to whatever weirdness Windows does instead of cron jobs) would be good as a contrib module, of course.
I disagree, not everyone who runs a list has access to the command line, and/or the means or ability to set up a Cron Job, so something through the Web UI would be useful!
Dave
David Andrews writes:
I disagree, not everyone who runs a list has access to the command line, and/or the means or ability to set up a Cron Job, so something through the Web UI would be useful!
Of course it would be useful to people who are using platforms that were designed to make Mailman use painful. Nevertheless, Mailman's mission is "providing the best possible mailing list manager", not "providing poor emulations of everything that Unix does that Windows and cPanel do not". Supporting these Misfeatured Platforms already takes up a lot of time. It's like sharpening the claw end of a hammer head: the more such features we add, the more people will take up up using hammers to drive screws. It's still better to spend the $1 (or $15, if you go Snap-On) for the screwdriver.
If Mailman can do it *better* than cron, that would be another matter. That point is arguable, but my personal preference would be to provide a generic feature that both cron and other possible schedulers could take advantage of.
Alternatively, if this feature can be provided easily as a side effect of something Mailman already has to do, that would be good, too -- as long as it's not a step onto the slippery slope of mission creep.
[ on the topic of the demise of the monthly mailing ]
I haven't followed MM3 discussions closely, but if there is some sort of extension capability or programmatic API it might be useful to provide a demonstration of how to reproduce the defunct monthly mailing using such an interface.
Is MM3 far enough along to be running somewhere that one could experiment?
Skip
Stephen> If Mailman can do it *better* than cron, that would be another
Stephen> matter. That point is arguable, but my personal preference
Stephen> would be to provide a generic feature that both cron and other
Stephen> possible schedulers could take advantage of.
In fact, it is quite easy to argue that with a programmatic interface and access to cron my "spread out the pain" request is better implemented there. The monthly mailing becomes a daily mailing which only sends reminder notices to a fraction of the users.
Skip
On Apr 04, 2010, at 06:34 PM, David Andrews wrote:
At 11:25 PM 4/1/2010, Stephen J. Turnbull wrote:
David Andrews writes:
Some other list software, I believe, has a feature that sends out a canned message once a month, such as list rules. I would like to see this in MM3
-1. This is mission creep. A cron job will serve perfectly well.
Something that assists in setting up such cron jobs (and/or generalizes to whatever weirdness Windows does instead of cron jobs) would be good as a contrib module, of course.
I disagree, not everyone who runs a list has access to the command line, and/or the means or ability to set up a Cron Job, so something through the Web UI would be useful!
This could potentially be a third-party add-on to the wui, but I don't think it belongs in the core Mailman project. Mailman certainly shouldn't be adding any functionality that duplicates cron.
-Barry
On Apr 05, 2010, at 06:15 AM, skip@pobox.com wrote:
I haven't followed MM3 discussions closely, but if there is some sort of extension capability or programmatic API it might be useful to provide a demonstration of how to reproduce the defunct monthly mailing using such an interface.
Yes to both, though the passwords are no longer stored in plaintext so the monthly reminder can't be a *password* reminder. You could definitely do a monthly subscription reminder slash membership probe. The nice thing is that because folks can register multiple emails under one account, you should be able to send one message w/CCs I think to each person.
Is MM3 far enough along to be running somewhere that one could experiment?
Yes, certainly for this. Let us know how it goes!
-Barry
skip@pobox.com writes:
Stephen> If Mailman can do it *better* than cron, that would be another Stephen> matter. That point is arguable, but my personal preference Stephen> would be to provide a generic feature that both cron and other Stephen> possible schedulers could take advantage of.
In fact, it is quite easy to argue that with a programmatic interface and access to cron my "spread out the pain" request is better implemented there.
"There" being in Mailman itself? Or in cron? (It's not obvious to me whether Mailman could do this better internally or not; of course I'd try to do it in cron before looking for a feature in Mailman, but Jack's mileage would vary, I'm sure. :-)
The monthly mailing becomes a daily mailing which only sends reminder notices to a fraction of the users.
Indeed, or weekly, or even 4x daily, and providing that kind of flexibility of timing is something that cron has a lot of existing support for. OTOH, somebody still has to write the code to manage the partial lists. However, that code might be useful for other purposes (eg, if a user who refers to the member list page often wants it split into 4 equal-sized pages rather than 27 pages of wildly varying size). (The above is what I mean by "generic features", and an example of how they might be applied.)
>> In fact, it is quite easy to argue that with a programmatic interface
>> and access to cron my "spread out the pain" request is better
>> implemented there.
Stephen> "There" being in Mailman itself? Or in cron? (It's not
Stephen> obvious to me whether Mailman could do this better internally
Stephen> or not; of course I'd try to do it in cron before looking for a
Stephen> feature in Mailman, but Jack's mileage would vary, I'm
Stephen> sure. :-)
In cron via a web services api of some sort.
>> The monthly mailing becomes a daily mailing which only sends reminder
>> notices to a fraction of the users.
Stephen> Indeed, or weekly, or even 4x daily, and providing that kind of
Stephen> flexibility of timing is something that cron has a lot of
Stephen> existing support for. OTOH, somebody still has to write the
Stephen> code to manage the partial lists. However, that code might be
Stephen> useful for other purposes (eg, if a user who refers to the
Stephen> member list page often wants it split into 4 equal-sized pages
Stephen> rather than 27 pages of wildly varying size). (The above is
Stephen> what I mean by "generic features", and an example of how they
Stephen> might be applied.)
In my original post I wrote the selector I would use to determine which day of the month an email was to get a notice:
ord(email[0]) % 28 + 1
Assuming you can pull the current list members from an API you select those whose first letter matches the current day of the month. (*)
Now, as to actually performing this experiment, I find starting up Mailman a rather high barrier simply because you have to create and populate a mailing list. How do people generally go about doing that? Do you just create a bunch of dummy aliases which map to a single address?
Skip
(*) In the Apocryphal Bible we find these extra two verses:
Genesis 2
...
4 And on the 29th, 30th and 31st days of the month the Moderator rested from all the labors to which he had been assigned by the Administrator.
5 And the Administrator blessed the 29th, 30th and 31st days of the month and sanctified them by pulling a beer from the cooler.
participants (6)
-
Adam McGreggor
-
Barry Warsaw
-
David Andrews
-
Mark Sapiro
-
skip@pobox.com
-
Stephen J. Turnbull