multiple footers accumulating on messages as discussion progresses - how to eliminate all but the last mailman footer?
I'm not sure what you call the long tail of footers that accumulates on messages, but I would like to eliminate all prior footers (from the same list). When the new reply goes out, it should include only one footer. BTW, we're using full personalization and one link in the footer is like this: %(user_optionsurl)s
We could wrap the footer in a tag of some type if that would help. Any suggestions? I would think this is a common request, but I can't find any mention of it anywhere.
Regards, Dave
David <dave@fiteyes.com> wrote:
I'm not sure what you call the long tail of footers that accumulates on messages, but I would like to eliminate all prior footers (from the same list). When the new reply goes out, it should include only one footer. BTW, we're using full personalization and one link in the footer is like this: %(user_optionsurl)s
We could wrap the footer in a tag of some type if that would help. Any suggestions? I would think this is a common request, but I can't find any mention of it anywhere.
The short answer is train your users to not top-post and to quote minimally.
Given that that is impossible, doing this would be best accomplished with a custom handler. See the FAQ at <http://wiki.list.org/x/l4A9> for information on installing custom handlers.
You could implement a handler that does something along the lines of the following. It's tricky because the footer may be quoted and we don't know what the quote characters are. It's even trickier if the list allows HTML alternatives, and trickier still because of personalized replacements. A basic idea is:
def process(mlist, msg, msgdata): for part in msg.walk(): if part.get_content_maintype == 'text': clean_footers(mlist, part)
def clean_footers(mlist, part): txt = part.get_payload(decode=True) newtxt = [] changed = False for bodyline in txt.splitlines(keepends=True): for footline in mlist.msg_footer.splitlines(): footline = footline.strip() if not footline: continue if bodyline.find(footline) >= 0: continue changed = True else: newtxt.append(bodyline) if changed: part.set_payload(''.join(newtxt)
What this will essentially do is remove from all text parts every line that contains as a substring any non-empty line of the list's msg_footer, but it doesn't handle replacements, and HTML makes it tricky because there may be added HTML tags that would cause the 'find' to fail, and there may be non-footer text on the same HTML source line as footer text.
To do replacements, instead of
for footline in mlist.msg_footer.splitlines():you'd need something like:
d = SafeDict({})
for user in msg.get_senders():
if mlist.is_member(user):
d['user_optionsurl'] = mlist.GetOptionsURL(user)
break
d.update({'real_name' : mlist.real_name,
'list_name' : mlist.internal_name(),
# For backwards compatibility
'_internal_name': mlist.internal_name(),
'host_name' : mlist.host_name,
'web_page_url' : mlist.web_page_url,
'description' : mlist.description,
'info' : mlist.info,
'cgiext' : mm_cfg.CGIEXT,
})
footlines = mlist.msg_footer % d
for footline in footlines:plus
from Mailman import mm_cfg from Mailman.SafeDict import SafeDict
at the beginning of the module.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Fri, Apr 20, 2012 at 5:56 AM, David <dave@fiteyes.com> wrote:
I'm not sure what you call the long tail of footers that accumulates on messages,
Collateral damage from MUAs That Only A Professional Programmer Who Hates His Job (And His Users) Could Love[tm]. Because of the potential for damaging author's content, this really should be handled by the author's MUA, but the only ones I know of that do are based on Emacs, which sort of rules out usage by all but a miniscule fraction of the user population.
but I would like to eliminate all prior footers (from the same list).
As Mark points out, this is hard to do well, and impossible to do right (where we define "right" to mean "cannot possibly damage the author's intended content").
However, the code he suggests should do very well (up to 100% for lists where top-posting is the rule, at least if you're able and willing to look at any posts where it doesn't work and tweak the regexps a bit).
If users are really bothered by it, Gmail does a very good job of eliding quoted content, including footers. (A user has to be pretty bothered to even think about changing his MUA, of course. But it's possible some are looking for alternatives anyway.)
I would think this is a common request, but I can't find any mention of it anywhere.
Does this actually bother your users? IME lists where this is a problem, it's a much smaller problem than top-posting itself, and the users aren't going to notice because they don't read any of the included content anyway. I suspect that's why nobody (that I can remember) has ever asked for this before.
On Thu, Apr 19, 2012 at 10:56 PM, Stephen J. Turnbull <stephen@xemacs.org>wrote:
On Fri, Apr 20, 2012 at 5:56 AM, David <dave@fiteyes.com> wrote:
I would think this is a common request, but I can't find any mention of it anywhere.
Does this actually bother your users? IME lists where this is a problem, it's a much smaller problem than top-posting itself, and the users aren't going to notice because they don't read any of the included content anyway. I suspect that's why nobody (that I can remember) has ever asked for this before.
Stephen and Mark - you guys have given me a lot of good advice. I appreciate it.
Maybe I can narrow my problem and find a better solution without the challenges of a general solution.
The main issue is that we include a personalized "instant unsubscribe" link in the footer, no password required. Our users seem to really want/need this. Many times they cannot unsubscribe themselves with any other method we have tried. However, we did not initially consider that any one person can easily unsubscribe another person in the thread because of the accumulated footers.
Here are our current personalized footers:
%(real_name)s mailing list Reply to: %(real_name)s@%(host_name)s Your settings: %(user_optionsurl)s Remove me from this list: %(user_optionsurl)s?unsub=1&unsubconfirm=1&password=%(user_password)s
It would probably be satisfactory to remove any url containing the string "unsubconfirm". Removing the user's user_optionsurl link would be nice, but not as absolutely necessary.
I can probably find a good regex that will recognize a URL and I can tailor that to match only URLs like the above. Given that, is there a custom module that is built to apply regular expressions for search/replace? I don't know Python. (But I would love to learn it when time permits.)
I reviewed http://wiki.list.org/pages/viewpage.action?pageId=4030615.
Coding the handler module itself requires some knowledge of Python and Mailman and won't be discussed further here except that there is a simple template/example at MyHandler.py<http://wiki.list.org/download/attachments/4030615/MyHandler.py?version=1&modificationDate=1283485784289> .
Using that template for my purposes would require much more than changing a couple lines of code, so I don't think I can adapt/use it. I'm hoping someone has already created a handler for the purpose of doing a regex-based search/replace. Searching Google on "mailman search replace module OR handler" didn't turn up anything, however.
Regards, David
On Fri, Apr 20, 2012 at 1:06 PM, David <dave@fiteyes.com> wrote:
The main issue is that we include a personalized "instant unsubscribe" link in the footer, no password required. Our users seem to really want/need this.
Aha. Now this all makes sense.
An alternative to requiring a password is to require confirmation via their email address. This does have the very undesirable feature that users can still really annoy others by banging on their unsubscribe link, thus resulting in spam to the targeted user. But if it's not deliberate harassment, this might help. (It would also be possible, but I suspect it's unimplemented, to refuse to send more than one unsubscribe confirmation per time period.)
Remove me from this list: %(user_optionsurl)s?unsub=1&unsubconfirm=1&password=%(user_password)s
Ouch ... this presents a lot of opportunity for harrassment, since the user's password will be sent around in clear.
I can probably find a good regex that will recognize a URL and I can tailor that to match only URLs like the above. Given that, is there a custom module that is built to apply regular expressions for search/replace? I don't know Python. (But I would love to learn it when time permits.)
import re cre = re.compile(r'some regex|other') result = cre.sub('', 'A string containing some other') print result 'A string containing '
The idiom is a little unintuitive; there is also a function re.sub you could use
result = re.sub(r'some regex|other', '', 'A string containing some other')
which gives the same result, marginally less efficiently. (It used to be that the regular expression was compiled fairly frequently if you used the same regexp repeatedly, but now a fairly large number of compiled regular expressions are cached, so the overhead is limited to finding the right compiled regexp.)
For URLs in particular, you might want to look at the urlparse module. I don't remember how it's implemented but there's probably a regexp for URLs in the source.
Using that template for my purposes would require much more than changing a couple lines of code, so I don't think I can adapt/use it. I'm hoping someone has already created a handler for the purpose of doing a regex-based search/replace. Searching Google on "mailman search replace module OR handler" didn't turn up anything, however.
Do you understand how Handlers work? What you do is put the code that Mark suggested in a separate file in the Handlers directory, figure out where you want the Handler to run, and then put
GLOBAL_PIPELINE.insert(position, 'ExcessFooterTrimmer')
in mm_cfg.py.
It's also possible to do this per list (eg, for testing on a single list) by creating the Handler file, and instead of using the GLOBAL_PIPELINE above, use
$ cd /var/lib/mailman # or whereever mailman's library code is $ bin/withlist -i test-list
m.lock() m.pipeline = GLOBAL_PIPELINE m.pipeline.insert(position, 'ExcessFooterTrimmer') m.save() quit() $
If that sounds more reasonable, feel free to ask for more detailed and accurate information (I'm just going off the top of my head here, but the kind and amount of work involved is correct).
David <dave@fiteyes.com> wrote:
The main issue is that we include a personalized "instant unsubscribe" link in the footer, no password required. Our users seem to really want/need this. Many times they cannot unsubscribe themselves with any other method we have tried. However, we did not initially consider that any one person can easily unsubscribe another person in the thread because of the accumulated footers.
This is the danger of including an 'instant, no confirmation required' unsubscribe link in list messages.
My experience based on running Topica.com lists for some time is that you can mung or remove the instant unsub URL in messages that pass back through the list (Topica does this), but this is only a partial solution. Your users will still forward list posts to others 'off list' and the others will click the link thinking they are unsubscribing themselves or just out of curiousity or maliciously to 'teach a lesson'.
You've probably thought about this and feel that even the simple confirmation email from an unsub without password is too high a hurdle for some of your users, but I'm surprised that other users are not complaining about your mailing passwords in the clear with every list post.
Stephen has just replied with most of what I was going to add to this, so I'll let you digest that, and if you need more specific coding or regexp help, let us know.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Fri, Apr 20, 2012 at 2:38 AM, Mark Sapiro <mark@msapiro.net> wrote:
Stephen has just replied with most of what I was going to add to this, so I'll let you digest that, and if you need more specific coding or regexp help, let us know.
Mark, Stephen: yes, you both gave me a lot of useful info again. It was
very tempting to pursue the regex solution, but I think we are going to try the simplest solution first. We will eliminate the 'instant, no confirmation required' unsubscribe link in footer and then see how our members deal with other unsubscribe options.
I like the following Mailman options page:
%(web_page_url)soptions/%(list_name)s
A footer link to this page may work well because it has a clearly visible unsubscribe option.
This leads to two questions:
Where is the html template for this options page? It is not one of the items found at "Edit the public HTML pages and text files". (The only related template I see is for the page that is shown *after* logging in to the above page.)
Where is the text for the unsubscribe confirmation message so we can customize that?
Thank you for all the help.
David <dave@fiteyes.com> wrote:
I like the following Mailman options page:
%(web_page_url)soptions/%(list_name)s
If the list is personalized, it may be better to use just
%(user_optionsurl)s
The difference is that user_optionsurl is equivalent to
%(web_page_url)soptions/%(list_name)s/%(user_address)s
This has the advantage of pre-providing the users email address which avoids the user's having to type it and the issues of the user not knowing which of her many addresses is actually subscribed to the list.
I don't think there's normally an issue around the address being sensitive information since if the user replies to or forwards the message, her address is in the From: header of the reply/forward anyway.
This might be a concern for an anonymous list, but not otherwise.
A footer link to this page may work well because it has a clearly visible unsubscribe option.
This leads to two questions:
- Where is the html template for this options page? It is not one of the items found at "Edit the public HTML pages and text files". (The only related template I see is for the page that is shown *after* logging in to the above page.)
There is no template for the options login page. It is built on the fly by the loginpage() function in the Mailman/Cgi/options.py module.
You can edit the strings in that module if your lists are English language only, but changing those strings breaks i18n translations of them.
A more robust way to do this is to create the path messages/en/LC_MESSAGES/ and create a mailman.po in that directory with just the "translations" you need, and then use msgfmt to cerate the corresponding mailman.mo file.
- Where is the text for the unsubscribe confirmation message so we can customize that?
See the FAQ at <http://wiki.list.org/x/jYA9>. The name of the template is unsub.txt.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
David -
Mark Sapiro -
Stephen J. Turnbull