Header & Footer for non-ASCII messages
We have run into a problem which is probably not uncommon for international mailing lists: whenever a message is sent that does not have US-ASCII as content type, header and footer are attached to this message via additional mime parts.
This doesn't really look nice in the mail applications.
Wouldn't it be wise to decode the message payload and prepend/ append the header and footer properly encoded to the payload instead ?
-- Marc-Andre Lemburg eGenix.com
Professional Python Software directly from the Source (#1, Apr 22 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
EuroPython 2003, Charleroi, Belgium: 63 days left
-----Original Message----- From: M.-A. Lemburg Sent: Tuesday, April 22, 2003 7:54 AM Subject: [Mailman-Developers] Header & Footer for non-ASCII messages
We have run into a problem which is probably not uncommon for international mailing lists: whenever a message is sent that does not have US-ASCII as content type, header and footer are attached to this message via additional mime parts.
Unfortunately this is a problem here in the US as well. Various Microsoft clients and other clients as well create character set problems, especially when using Mailman content filtering to convert mail to plain test.
The only solution we've come up with is to pre-process list mail through demime to convert mail to plain text before being process by mailman. I realize that mailman content filtering is a more "general purpose" filter that has other jobs besides plain text conversions, but I would really like to avoid pre-processing list mail.
Demime converts mail to 'Content-Type: text/plain; charset="us-ascii"' and at this point headers and footers are no problem. I realize that this won't work on international lists.
I don't really have a suggestion for the solution.
-Jeff
M.-A. Lemburg wrote:
We have run into a problem which is probably not uncommon for international mailing lists: whenever a message is sent that does not have US-ASCII as content type, header and footer are attached to this message via additional mime parts.
This doesn't really look nice in the mail applications.
I've done a lot of looking in to this, and "the mail applications" that you mention are pretty much just Outlook, which is pretty broken with inline attachments. Just about every other mail reader will inline the header/footer MIME parts, but Outlook shows them as separate attachments for no good reason.
Mailman does everything it can do to hint to the MUA to keep the header and footer MIME parts inline with the body, but Outlook just ignores this.
Wouldn't it be wise to decode the message payload and prepend/ append the header and footer properly encoded to the payload instead ?
This just doesn't work. We have to assume the header and footer are in the same charset as the language of the mailing list (by default, us-ascii).
If a message posted is in a language that doesn't match the language of the mailing list (say, UTF-8), and the header and footer of the list are ISO-8859-1, you can't just "decode" them both and concatenate them, as this will make a message with two unrelated character sets combined.
If you can come up with any hack to make Outlook include MIME attachments inline, it would be appreciated. That's really all we need, because then we'll have the best of all worlds.
Ben
Ben Gertzfield wrote:
M.-A. Lemburg wrote:
We have run into a problem which is probably not uncommon for international mailing lists: whenever a message is sent that does not have US-ASCII as content type, header and footer are attached to this message via additional mime parts.
This doesn't really look nice in the mail applications.
I've done a lot of looking in to this, and "the mail applications" that you mention are pretty much just Outlook, which is pretty broken with inline attachments. Just about every other mail reader will inline the header/footer MIME parts, but Outlook shows them as separate attachments for no good reason.
Hmm, I'm using Mozilla's mail app and it doesn't look right in there either.
Mailman does everything it can do to hint to the MUA to keep the header and footer MIME parts inline with the body, but Outlook just ignores this.
Wouldn't it be wise to decode the message payload and prepend/ append the header and footer properly encoded to the payload instead ?
This just doesn't work. We have to assume the header and footer are in the same charset as the language of the mailing list (by default, us-ascii).
Right.
If a message posted is in a language that doesn't match the language of the mailing list (say, UTF-8), and the header and footer of the list are ISO-8859-1, you can't just "decode" them both and concatenate them, as this will make a message with two unrelated character sets combined.
Uhm, you can decode both into Unicode, paste them together and then encode the result again in either the mailing lists default charset or UTF-8.
If you can come up with any hack to make Outlook include MIME attachments inline, it would be appreciated. That's really all we need, because then we'll have the best of all worlds.
Not really -- inline MIME parts simply don't look right for things like headers and footers. They are fine for images and the like, but headers and footers really belong right into the main message text.
-- Marc-Andre Lemburg eGenix.com
Professional Python Software directly from the Source (#1, Apr 22 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
EuroPython 2003, Charleroi, Belgium: 63 days left
M.-A. Lemburg wrote:
Uhm, you can decode both into Unicode, paste them together and then encode the result again in either the mailing lists default charset or UTF-8.
This is a definite possibility. But then we risk changing the actual contents of the message -- transforming everything to UTF-8 and back is not an operation that's guaranteed one-to-one.
If I get some time, I'll try to code this up.
Ben
On Tue, Apr 22, 2003 at 03:21:11PM -0700, Ben Gertzfield wrote:
Uhm, you can decode both into Unicode, paste them together and then encode the result again in either the mailing lists default charset or UTF-8.
I'd suggest trying this sequence for charset down-conversion:
- plaintext (e.g. us-ascii)
- the list's default charset
- utf-8
maybe just before utf-8 we could have a per-language (or per-list) configurable array of preferred fallbacks, just to kill every other complaint :)
This is a definite possibility. But then we risk changing the actual contents of the message -- transforming everything to UTF-8 and back is not an operation that's guaranteed one-to-one.
Absolutely agreed, so the whole procedure should be a per-list option, defaulting to off.
-- Simone Piunno -- http://members.ferrara.linux.it/pioppo .------- Adde parvum parvo magnus acervus erit -------. Ferrara Linux Users Group - http://www.ferrara.linux.it Deep Space 6, IPv6 on Linux - http://www.deepspace6.net GNU Mailman, Mailing List Manager - http://www.list.org `-------------------------------------------------------'
Hi,
Have you tried this patch ?
http://sourceforge.net/tracker/?func=detail&aid=664209&group_id=103&atid=300103
Good luck.
M.-A. Lemburg wrote:
We have run into a problem which is probably not uncommon for international mailing lists: whenever a message is sent that does not have US-ASCII as content type, header and footer are attached to this message via additional mime parts.
This doesn't really look nice in the mail applications.
Wouldn't it be wise to decode the message payload and prepend/ append the header and footer properly encoded to the payload instead ?
-- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/
Tokio Kikuchi wrote:
Hi,
Have you tried this patch ?
http://sourceforge.net/tracker/?func=detail&aid=664209&group_id=103&atid=300103
Thanks. This looks like a start, but I'd add your new if case as elif to the existing one; you'll also have to deal with errors from trying to decode the payload and have to make sure that in case of a conversion error, the default action (add MIME parts) is chosen as fallback.
Good luck.
M.-A. Lemburg wrote:
We have run into a problem which is probably not uncommon for international mailing lists: whenever a message is sent that does not have US-ASCII as content type, header and footer are attached to this message via additional mime parts.
This doesn't really look nice in the mail applications.
Wouldn't it be wise to decode the message payload and prepend/ append the header and footer properly encoded to the payload instead ?
-- Marc-Andre Lemburg eGenix.com
Professional Python Software directly from the Source (#1, Apr 23 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
EuroPython 2003, Charleroi, Belgium: 62 days left
participants (5)
-
Ben Gertzfield -
Jeff Hahn -
M.-A. Lemburg -
Simone Piunno -
Tokio Kikuchi