Feature Requests item #1822565, was opened at 2007-10-30 00:33 Message generated for change (Comment added) made by msapiro You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=350103&aid=1822565&group_id=103 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Cyndi (cyndi) Assigned to: Nobody/Anonymous (nobody) Summary: Plain text option for nondigest Initial Comment: I searched this to the best of my ability and there seems to be no current method to do it. I found this: http://www.mail-archive.com/mailman-users@python.org/msg36881.html and searched the feature requests, but it seems no one has submitted this yet, as far as I can tell. You have a plain-text option for subscribers to select if they receive the digest. I've tested it and it works perfectly. Messages that came through as HTML gobblygook in the nondigest will display as lovely plain text in the digest. What I would love is a plain-text option for the nondigest. Just exactly what you do for the digests, only post by post. Yahoogroups does it, as do many other mailing list sites/programs. It seems that currently I have only three choices: 1) Convert all posts to plain text. I can't get this to work right though (I am using a hosting service via my ISP, so they may not have all the prereqs, or I need to tweak it more myself). But this means people who want HTML can't get it. 2) Get MM to hold all HTML posts for moderation, then I send rejection notices insisting that people post in plain text only. This "works" but disenfranchises people with poor email skills or who use mailers/ISP's like AOL. I run a support group filled with people who have brain processing deficits, as well as those who simply don't understand computers. I've tried for years to get folks to post in plain text and most just can't. I finally gave up and hand-edited the HTML posts, but this isn't possible on MM (and I hate doing it). 3) Live with it. Most people have HTML-capable mailers (I'm an oldtimer holdout, but can switch). The problem is that some of my subscribers are blind or otherwise can't deal with HTML-formatted email (not all have converting software). A choice #4 (user option for nondigest plain text) would be wonderful. Thanks. ----------------------------------------------------------------------

Comment By: Mark Sapiro (msapiro) Date: 2007-10-31 21:02

Message: Logged In: YES user_id=1123998 Originator: NO As far as I can tell, sonic.net runs a more or less standard 2.1.9 version. At least it doesn't have a 'package identifier'. <quote> When someone posts to the test list in HTML, I get gobbygook in the individual posts. The readability varies, depending on their mailer and settings. </quote> Is this using the non MIME/HTML aware reader? Is mail from the list any different from mail you would see if they sent the mail to you directly in the same way? Does this list mail look 'normal' if you view it in a MIME/HTML aware reader? <quote> So the same emails are coming through differently in nondigest (no conversion) and digest (plain text conversion). </quote> In standard mailman, no conversion whatsoever is done for the plain text digest. The messages are passed through exactly the same scrubber process as is used for scrub_nondigest. This process removes every non text/plain MIME part and every text/plain MIME part with an unknown character set and stores these parts separately and replaces them with links to the stored parts exactly as you indicate. What I'm trying to tell you is this process is not different for individual messages with scrub_nondigest vs. the plain text digest. That's the part where I have a problem with what you are saying, since you are saying the individual message received with scrub_nondigest is unacceptable but a similar message in the plain text digest is acceptable, but I am saying the messages are produced in exactly the same way unless sonic.net's Mailman has modifications in the production of plain text digests of which I am not aware. Regarding the scrubbed HTML attachment, I can't view it because it is in a private archive, but I imagine your complaint is because you are seeing 'html escaped' text rather than rendered HTML. It is possible for a site to choose to save this as HTML rather than 'escaped' HTML, but it is not recommended because it exposes the site to injection of any and all types of malicious HTML via emails to a list which the site will then serve to web visitors - not a good thing. <quote> My plain text posts came through fine (just like they did before using the scrub_nondigest setting), but anything with HTML or a graphic got the treatment above. </quote> And are not posts with HTML and/or graphics given exactly the same treatment in the plain text digest? <quote> So, I turned the scrub_nondigest setting back off and went to the Content Filtering page. I turned it on and removed the attachment types but otherwise left defaults as is. I got errors similar to those with scrub_nondigest. Perhaps I need to put more of the defaults back in; I am willing to play with it if I can find some clear directions. </quote> Are you saying that now the messages in the plain text digest look like the individual scrub-nondigest messages did? If so, I'm not surprised because that's what I think the case should be. If this appeared to not be the case before, perhaps it is as I said yesterday. <quote> Perhaps when you tested the digest, you looked at a multipart/alternative message and when you tested scrub_nondigest, you looked at an HTML only message. </quote> <quote> If it would be helpful, I would be happy to forward emails to you from my tests. Or I could run new tests at your request. </quote> What I would like to see is a plain text digest that contains a "converted" HTML message, the original of that HTML message as posted to the list, and a copy of all the content filtering settings that were in effect at the time the message was posted. A copy of the original HTML message as received from the list as an individual message might also be interesting. You can put the raw text of those messages including headers in one or more files and attach them here or you can email the messages to me in any way that preserves the MIME structure of the messages. ---------------------------------------------------------------------- Comment By: Jim Popovitch (jimpop) Date: 2007-10-30 23:39 Message: Logged In: YES user_id=3142 Originator: NO Hi Cyndi, First, I am a huge fan of Mailman as well as a long time administrator of a Mailman system. Now that I have said that, let me say that I agree with you that scrub_nondigest doesn't work the way you (nor I) feel it should. I leave it disabled on our systems. That said, what I think you need is something like demime before the email hits Mailman. I used that for years before deciding that HTML email would be ok for our lists. I do understand your situation is such that you don't have console access to the system, but perhaps you can ask your hosting provider to supply if for you. Again, I don't think that scrub_nondigest will do what you need, and there is little else in Mailman to change the overall format of incoming email messages. Best wishes. -Jim P. ---------------------------------------------------------------------- Comment By: Cyndi (cyndi) Date: 2007-10-30 23:23 Message: Logged In: YES user_id=1925111 Originator: YES If I miss a question, or don't give the answer you need, please ask again. Note that I say "HTML" when I really mean "HTML or MIME or related encodings." I am using Mailman on sonic.net. Version 2.1.9. I can't find more information than that...my guess is that it's being run on a unix box (or some flavor of linux). If it is modified, I don't know where or how. I started a mailman test list with 4 other volunteers from my current (majordomo) list so we can work out the kinks. I am subscribed under two addresses, so I have digest and nondigest. I use a non-HTML capable mailer for most of my work, though I have access to HTML/MIME-capable ones. My volunteers use various ISP's and mailers and mostly post in HTML. When someone posts to the test list in HTML, I get gobbygook in the individual posts. The readability varies, depending on their mailer and settings. Then I did my new subscription address and chose digest with plain text. The digests are completely readable with no HTML code or anything else like that. This includes digests made up of posts that were HTML/junk in nondigest mail. So the same emails are coming through differently in nondigest (no conversion) and digest (plain text conversion). When I tested the scrub_nondigest setting, the posts still went through, but the content was removed from any that were not sent in plain text. For example, this was the complete body of one message (the sender posted in HTML and included a small graphic as part of the test; the MM footer was appended correctly at the bottom, which I have removed here): An HTML attachment was scrubbed... URL: http://lists.sonic.net/mailman/private/immune-test/attachments/20071028/4b520cb9/attachment.ht\ ml If you go to that URL, you'll see that the message is a complete mess. My plain text posts came through fine (just like they did before using the scrub_nondigest setting), but anything with HTML or a graphic got the treatment above. So, I turned the scrub_nondigest setting back off and went to the Content Filtering page. I turned it on and removed the attachment types but otherwise left defaults as is. I got errors similar to those with scrub_nondigest. Perhaps I need to put more of the defaults back in; I am willing to play with it if I can find some clear directions. If it would be helpful, I would be happy to forward emails to you from my tests. Or I could run new tests at your request. I've already posted to sonic.help.lists which is my ISP's internal newsgroup for MM issues (no replies on that topic yet). Thanks very much for your discussion and let me know if I failed to answer a question fully. Cyndi ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2007-10-30 20:15 Message: Logged In: YES user_id=1123998 Originator: NO Sorry. I didn't read the message you referenced in your original description. My bad. However, I don't understand why you say scrub_nondigest "was a total disaster" and regarding plain digests "I've tested it and it works perfectly." Exactly the same scrubber process is applied in both cases. Perhaps when you tested the digest, you looked at a multipart/alternative message and when you tested scrub_nondigest, you looked at an HTML only message. Or perhaps you are using someone else's modified Mailman. If so, perhaps you could convince them to abide by the terms of the GPL and submit their patches. As far as the scrubber links not working well is concerned, the only issue with current Mailman project Mailman that I am aware of is you have to log in if your archives are private. There are other issues I am aware of with some cPanel versions, but those are beyond our control. What problems did you have, and what Mailman version is this? HTML to plain text conversion relies on an external program (default lynx). In older versions of Mailman there were issues because quoted-printable and base64 encoded HTML was passed to the external program without decoding, but this was fixed in (I think) 2.1.7. As far as the request for an individual option is concerned, it is a valid request, but even though you seem convinced that we know how to do what you want for digests, I'm not convinced, so we need to resolve that first. Otherwise you may get a user option for something that you consider unsatisfactory. ---------------------------------------------------------------------- Comment By: Cyndi (cyndi) Date: 2007-10-30 19:24 Message: Logged In: YES user_id=1925111 Originator: YES Well, you yourself said that anyone wanting this ability to be per-user should submit a feature request here (see http://www.mail-archive.com/mailman-users@python.org/msg36881.html ). So I did. If it can be per user for digests, I don't see why not for nondigests, but you know the limitations of the software. I already tested the scrub_nondigest option and it was a total disaster. It didn't do any conversion. All it did was take all the text (along with any attachments) in the message and remove it, then put a link to where to find it. Even that didn't work well. I also tested the Content Filtering to do HTML conversion to plain text and it was even more of a disaster. I found some instructions that indicate it is very complex to setup, but my browser crashed so I'll have to look them up again. I think it might be possible to do HTML conversion on all the posts but only if my ISP agrees to do some stuff on their end. And it's not straightforward. For right now, I would be happy with a bug-free way to do conversions for the entire list. But I still think a feature where the user can choose HTML or plain text individual posts is a good thing. ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2007-10-30 18:48 Message: Logged In: YES user_id=1123998 Originator: NO Perhaps you didn't search well enough, or perhaps we just hide or don't describe the option in these terms, but the Non-digest options->scrub_nondigest option in the list admin interface will do what you want. It will apply the same process that is applied to the plain digest to individual messages. The only problem is this is a list option, not an individual subscriber option. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=350103&aid=1822565&group_id=103