Headers and footers for MIME messages
I've got a bit of a dilemma that I'd like to get your opinion on. In MM2.1, I can be more sane about adding headers and footers to messages that flow through a list, especially when it comes to MIME messages. Here are the cases I've identified:
A message comes in that is either text/plain or has no Content-Type. In that case, I can just textually prepend any header and append any footer. This is essentially what MM2.0.x does for all messages.
A message comes in that's multipart/mixed. That's easy too, since we can just add the header and footers as additional text/plain subparts.
A message comes in that has some other main Content-Type than "multipart", e.g. image/gif. This is a little tricker, but what I think you can do is transform the container message into a multipart/mixed, add the header and footers as subparts, and create a new subpart for the original message payload. You'd need to copy the body of the original message, plus any Content-* headers, into the subpart, but I think you leave all other headers in the container alone.
A message comes in that is multipart/* but not multipart/mixed. What to do here? I suppose we could play the same trick as #3, but it seems like an awful lot of work and potentially error prone. Maybe not though. Alternatively, we could add the header and footer to any existing preamble and epilogue, although they'll get hidden by most MIME-aware MUAs.
Or we could simply not add headers and footers for #3 and/or #4.
Let's assume for the moment <wink> that MM2.1 will have facilities to hold, discard, chop apart, etc. MIME messages and that we're talking about messages that are allowed to go though the list untouched.
Any other suggestions? Any preferences? -Barry
On 28 Mar 23:15, Barry A. Warsaw wrote:
- A message comes in that's multipart/mixed. That's easy too, since we can just add the header and footers as additional text/plain subparts.
If the first body part is text/plain, would you prefer to add the header or footer directly to it, instead of using separate parts?
Michael
Any other suggestions? Any preferences? -Barry
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers
"MY" == Michael Yount csf@moscow.com writes:
>> On 28 Mar 23:15, Barry A. Warsaw wrote:
>> 2. A message comes in that's multipart/mixed. That's easy too,
>> since we can just add the header and footers as additional
>> text/plain subparts.
MY> If the first body part is text/plain, would you prefer to add
MY> the header or footer directly to it, instead of using separate
MY> parts?I don't know. I'm a bit loathe to drill down too far into the subparts, but just because I think it'll make the algorithm for searching where to add the headers/footers more complicated than it needs to be.
-Barry
- A message comes in that has some other main Content-Type than "multipart", e.g. image/gif. This is a little tricker, but what I think you can do is transform the container message into a multipart/mixed, add the header and footers as subparts, and create a new subpart for the original message payload. You'd need to copy the body of the original message, plus any Content-* headers, into the subpart, but I think you leave all other headers in the container alone.
I'd consider it slight offensive to send random binary objects to a mailing list without any lead-in text/plain explanation, but I can imagine a special purpose list that would want to do this (but would they want headers and footers attached?).
A straight text/html message is probably the most common problem case - the swiss-army-knife of list software would probably have the header and footer as html, and convert to text/plain when necessary (yuk).
There are another of other options people will ask for: convert text/html to text/plain, bounce text/html back to sender, etc.
You may think this is well outside the scope of what you were intending to implement, but the worm-can has been opened by adding mime support and people won't let you rest until your software makes breakfast in bed for them.
You'll soon end up with a complex language for specifying things like "accept binary attachments only if preceeded by a text/plain", etc.. 8-)
BTW, I think there's a special mime type for headers and footers - text/plain isn't the right one. Can't remember off the top of my head.
- A message comes in that is multipart/* but not multipart/mixed. What to do here? I suppose we could play the same trick as #3, but it seems like an awful lot of work and potentially error prone. Maybe not though. Alternatively, we could add the header and footer to any existing preamble and epilogue, although they'll get hidden by most MIME-aware MUAs.
The big one being multipart/alternative, of course, which typically contains a text/plain and a text/html.
The safest rules would be:
- if it's not a mime message, do the old thing.
- if it's mime, and it's not already a multipart/mixed, create a text/multipart with header/original mime/footer.
- it it's mime, and and multipart/mixed, simply insert and append the header and footer parts.
Messing with the text/plain part is vaguely attractive, although conceivably it could break things (smime? dunno).
Andrew McNamara (System Architect)
connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111
"AM" == Andrew McNamara andrewm@connect.com.au writes:
AM> You'll soon end up with a complex language for specifying
AM> things like "accept binary attachments only if preceeded by a
AM> text/plain", etc.. 8-)Ug. It's probably inevitable, but I really don't want Mailman to become a hypergeneralized MIME slicer-n-dicer.
AM> BTW, I think there's a special mime type for headers and
AM> footers - text/plain isn't the right one. Can't remember off
AM> the top of my head.I'm not aware of it, so I'd appreciate pointers. Maybe you're thinking of multipart/digest or message/* types? Neither is exactly appropriate in this context.
AM> The safest rules would be:
AM> - if it's not a mime message, do the old thing.Agreed.
AM> - if it's
AM> mime, and it's not already a multipart/mixed, create a
AM> text/multipart with header/original mime/footer.I think you mean "create a multipart/mixed with 3 subparts, the header, the original article, and the footer".
AM> - it it's
AM> mime, and and multipart/mixed, simply insert and append the
AM> header and footer parts.Agreed.
My main objective is above all "to do no harm", so for now, I'm going to adopt these rules:
- if it's not MIME or it's text/plain, do the old thing
- if it's multipart/mixed, add the header and footers as
additional subparts
- anything else, pass the message through unchangedWe can elaborate on that last one later if necessary.
-Barry
Ug. It's probably inevitable, but I really don't want Mailman to become a hypergeneralized MIME slicer-n-dicer.
Yes - I think it's sort of inevitable. If you structure your MIME parser correctly, it may be relatively painless.
BTW, you should look at the perl MIME module (as used by Majordomo 2) - I understand that it's one of the best MIME interfaces around (although reading other people's perl is futile, it's interface doco might suggest some ideas for your code).
AM> BTW, I think there's a special mime type for headers and AM> footers - text/plain isn't the right one. Can't remember off AM> the top of my head.
I'm not aware of it, so I'd appreciate pointers. Maybe you're thinking of multipart/digest or message/* types? Neither is exactly appropriate in this context.
I had a quick look when I posted, but couldn't find it. I think I was probably thinking of the multipart/digest and message/* stuff, as you suggest.
AM> - if it's AM> mime, and it's not already a multipart/mixed, create a AM> text/multipart with header/original mime/footer.
I think you mean "create a multipart/mixed with 3 subparts, the header, the original article, and the footer".
Yes - wrap the original mime message in a new multipart/mixed with 3 subparts. Mime is intended to be a hierarchy, so this is safe.
It will also be a good test for your Python MIME code - if it's going to cope with the random crap that people mail around, it shouldn't have much trouble producing this.
Personally, I find MIME problematic - the basic idea was okay, but when people start nesting objects several levels deep the utility starts to disappear. Also, it assumes that SMTP is a file transport protocol, which it isn't (if it was, it would support restarting failed transfers). I also turn it off in my MUA (exmh) as it slows it down considerably.
My main objective is above all "to do no harm", so for now, I'm going to adopt these rules:
Yes.
if it's not MIME or it's text/plain, do the old thing
if it's multipart/mixed, add the header and footers as additional subparts
anything else, pass the message through unchanged
We can elaborate on that last one later if necessary.
Yes.
Andrew McNamara (System Architect)
connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111
"AM" == Andrew McNamara andrewm@connect.com.au writes:
>> Ug. It's probably inevitable, but I really don't want Mailman
>> to become a hypergeneralized MIME slicer-n-dicer.
AM> Yes - I think it's sort of inevitable. If you structure your
AM> MIME parser correctly, it may be relatively painless.I think mimelib is pretty good, certainly better than anything else I've seen for Python. I've got some nits about its interface, which I'll need to clean up for any 1.0 release (or for inclusion in Python, and I know Guido has some nits).
AM> BTW, you should look at the perl MIME module (as used by
AM> Majordomo 2) - I understand that it's one of the best MIME
AM> interfaces around (although reading other people's perl is
AM> futile, it's interface doco might suggest some ideas for your
AM> code).Do you have a reference? I gave up Perl almost 7 years ago, so I know I couldn't read it, but if the interface spec is online, I'd love to take a look.
AM> It will also be a good test for your Python MIME code - if
AM> it's going to cope with the random crap that people mail
AM> around, it shouldn't have much trouble producing this.It's actually fairly easy with mimelib.
AM> Personally, I find MIME problematic - the basic idea was okay,
AM> but when people start nesting objects several levels deep the
AM> utility starts to disappear. Also, it assumes that SMTP is a
AM> file transport protocol, which it isn't (if it was, it would
AM> support restarting failed transfers). I also turn it off in my
AM> MUA (exmh) as it slows it down considerably.I agree. Also, I've find that generating MIME correctly really requires holding the entire message hierarchy in memory and making several passes over the generated text before you can really start outputing correctly. That sucks too.
Thanks for the feedback.
-Barry
I think mimelib is pretty good, certainly better than anything else I've seen for Python. I've got some nits about its interface, which I'll need to clean up for any 1.0 release (or for inclusion in Python, and I know Guido has some nits).
I had a quick look a few weeks back - my impression was that it didn't entirely embrace the OO'edness, although it can be a big mistake to go too far with objects anyway, and I can't provide any sensible comments on what you could do to make it more abstract (armchair critic). If it works, that's the most important goal. 8-)
Do you have a reference? I gave up Perl almost 7 years ago, so I know I couldn't read it, but if the interface spec is online, I'd love to take a look.
It's the MIME::Tools module on cpan, I think this is the author's site (and it includes online class doco):
http://www.zeegee.com/code/perl/MIME-tools/
They also have a MIME::Lite.
I agree. Also, I've find that generating MIME correctly really requires holding the entire message hierarchy in memory and making several passes over the generated text before you can really start outputing correctly. That sucks too.
This is definitely the biggest shortcoming with MIME - for things like MTA's, there's a certain amount of presure to include MIME support (encoding 8-bit messages when they encounter a 7 bit path, DSN, etc), but the performance implications just suck too hard.
Andrew McNamara (System Architect)
connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111
On Tue, Apr 03, 2001 at 01:04:57PM +1000, Andrew McNamara wrote:
BTW, you should look at the perl MIME module (as used by Majordomo 2) - I understand that it's one of the best MIME interfaces around (although reading other people's perl is futile, it's interface doco might suggest some ideas for your code).
But, Andrew...
didn't you read the press release?
Cheers, -- jra
Jay R. Ashworth jra@baylink.com Member of the Technical Staff Baylink The Suncoast Freenet The Things I Think Tampa Bay, Florida http://baylink.pitas.com +1 727 804 5015
barry@digicool.com (Barry A. Warsaw) writes:
- A message comes in that is multipart/* but not multipart/mixed. What to do here?
Wrap the whole thing as a multipart/mixed:
- text/plain: header
- multipart/blah: original message
- text/plain: footer
Of course, I'm demiming all my lists anyway, so it would probably have gotten bounced before this point. Speaking of demime, and offtopic for this list, I just worked out a simple yet wonderful procmail recipe:
:0 fw
- ^Content-Type:.*(multipart/alternative|text/html) | /news/local/demime - | formail -a 'From '
Brian.
participants (5)
-
Andrew McNamara -
barry＠digicool.com -
Brian Edmonds -
Jay R. Ashworth -
Michael Yount