modifying payload using same encoding and Content-Transfer-Encoding
![](https://secure.gravatar.com/avatar/786a78a3216d110a6714efd5dd9b114b.jpg?s=120&d=mm&r=g)
Hi,
I don't manage to figure out how to keep the same Content-Transfer-Encoding, while rewriting the content?
msg.set_payload() doesn't seems to trigger any encoding, base64 for example was set.
not using decode=True, return a base64 bloc… I just want to write it back. I should be able to know that the payload was encoded…
related = MIMEMultipart('related')
html_footer = HTML_ATTACHMENT_HOLDER % {'HTML_HERE':
html_attach} html_footer += '</body>' old_content = msg.get_payload(decode=True) new_content = re.sub(r'</body>', html_footer, old_content) if old_content != new_content: syslog('debug', 'add html footer') else: syslog('debug', 'no html footer added')
charset = msg.get_content_charset()
syslog('debug', 'get_content_charset: %s,
Content-Transfer-Encoding: %s', charset, msg['Content-Transfer-Encoding'])
msg.set_payload(new_content, charset)
related.attach(msg)
related.attach(clip_payload)
The output produced must be wrong:
Content-Transfer-Encoding: base64 MIME-Version: 1.0 Content-Type: text/html; charset="utf-8"
<html> <head> <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000">
sysloged output: May 12 17:45:32 2014 (1795) get_content_charset: utf-8, Content-Transfer-Encoding: base64
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 05/12/2014 09:03 AM, Sylvain Viart wrote:
I don't manage to figure out how to keep the same Content-Transfer-Encoding, while rewriting the content?
Do you really want that? That's not what your example code does.
msg.set_payload() doesn't seems to trigger any encoding, base64 for example was set.
It will if there is a charset parameter and the message contains no Content-Transfer-Encoding: header.
See <https://docs.python.org/2/library/email.message.html#email.message.Message.set_payload> and the implicit <https://docs.python.org/2/library/email.message.html#email.message.Message.set_charset>.
not using decode=True, return a base64 bloc… I just want to write it back. I should be able to know that the payload was encoded…
related = MIMEMultipart('related') html_footer = HTML_ATTACHMENT_HOLDER % {'HTML_HERE':
html_attach} html_footer += '</body>' old_content = msg.get_payload(decode=True) new_content = re.sub(r'</body>', html_footer, old_content) if old_content != new_content: syslog('debug', 'add html footer') else: syslog('debug', 'no html footer added')
charset = msg.get_content_charset() syslog('debug', 'get_content_charset: %s,
Content-Transfer-Encoding: %s', charset, msg['Content-Transfer-Encoding'])
Here, add del msg['content-transfer-encoding'] Don't delete Content-Type or it will get set to text/plain.
msg.set_payload(new_content, charset) related.attach(msg) related.attach(clip_payload)
The output produced must be wrong:
...
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/786a78a3216d110a6714efd5dd9b114b.jpg?s=120&d=mm&r=g)
Hi,
Thanks for your reply. Unfortunately it doesn't work. I don't know why. I did read the doc many times, without the expected result.
See <https://docs.python.org/2/library/email.message.html#email.message.Message.set_payload> and the implicit <https://docs.python.org/2/library/email.message.html#email.message.Message.set_charset>.
My code is inside this custom handler. It works, but not for keeping the encoding. https://github.com/Sylvain303/mailman-AttachmentMove/blob/master/AttachmentM...
It could be related to this specific usage inside the recursive change…
To solve it, I've forced the encoding explicitly:
msg.set_payload(new_content)
del msg['content-transfer-encoding']
encoders.encode_7or8bit(msg)
I hope it will handle it correctly.
This code outside mailman is working: took from http://bugs.python.org/issue1525919
from email.Message import Message from email.Charset import Charset, QP text = "=" msg = Message() charset = Charset("utf-8") charset.header_encoding = QP charset.body_encoding = QP msg.set_charset(charset) msg.set_payload(text) print msg.as_string()
I've tried with no success:
from email.Charset import Charset, QP, BASE64 […] del msg['content-transfer-encoding'] charset = Charset("utf-8") charset.header_encoding = BASE64 charset.body_encoding = BASE64 msg.set_charset(charset) msg.set_payload(new_content)
I got the output below. But it's not base64… just the plain new_content I've produced. I don't need the base64 encoding specially, just that the content match what the header is saying.
MIME-Version: 1.0
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64
<html>
<head>[…]
If I call the encoding explicitly it works, in conjunction with msg.set_charset(charset), I got header Content-Transfer-Encoding: twice.
msg.set_payload(new_content)
encoders.encode_base64(msg)
Regards, Sylvain.
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 05/13/2014 07:14 AM, Sylvain Viart wrote:
Hi,
Thanks for your reply. Unfortunately it doesn't work. I don't know why. I did read the doc many times, without the expected result.
It works for me. The following withlist interaction based on a stripped down version of what you originally posted shows it works.
I create a message with utf-8, base64 encoded content. I get its decoded payload, delete the existing Content-Transfer-Encoding: header and set a new payload (not actually different, but could be) with the charset from the original message and the body is base64 encoded and the Content-Transfer-Encoding: header is set appropriately. And the Python email package's encoding for utf-8 is base64, so the encoding will be base64 if you just include the utf-8 charset argument in the call to set_payload().
This
del msg['content-transfer-encoding'] charset = Charset("utf-8") charset.header_encoding = BASE64 charset.body_encoding = BASE64 msg.set_charset(charset)
All gets undone when you do this
msg.set_payload(new_content)
$ /var/MM/21/bin/withlist -i No list name supplied. Python 2.7.5+ (default, Feb 27 2014, 19:37:08) [GCC 4.8.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole)
import email from Mailman.Message import Message msg = email.message_from_string("""From: me ... To: you ... Subject: test message ... Content-Transfer-Encoding: base64 ... MIME-Version: 1.0 ... Content-Type: text/html; charset="utf-8" ... ... PGh0bWw+CjxoZWFkPgo8L2hlYWQ+Cjxib2R5PgpUaGlzIGlzIGEgdGVzdAo8L2JvZHk+CjwvaHRt ... bD4K ... """, Message) msg From nobody Tue May 13 17:30:12 2014 From: me To: you Subject: test message Content-Transfer-Encoding: base64 MIME-Version: 1.0 Content-Type: text/html; charset="utf-8"
PGh0bWw+CjxoZWFkPgo8L2hlYWQ+Cjxib2R5PgpUaGlzIGlzIGEgdGVzdAo8L2JvZHk+CjwvaHRt bD4K
old_content = msg.get_payload(decode=True) old_content '<html>\n<head>\n</head>\n<body>\nThis is a test\n</body>\n</html>\n' new_content = old_content charset = msg.get_content_charset() del msg['content-transfer-encoding'] msg.set_payload(new_content, charset) msg From nobody Tue May 13 17:32:32 2014 From: me To: you Subject: test message MIME-Version: 1.0 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64
PGh0bWw+CjxoZWFkPgo8L2hlYWQ+Cjxib2R5PgpUaGlzIGlzIGEgdGVzdAo8L2JvZHk+CjwvaHRt bD4K
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/786a78a3216d110a6714efd5dd9b114b.jpg?s=120&d=mm&r=g)
Hi,
Le 14/05/2014 02:45, Mark Sapiro a écrit :
It works for me. The following withlist interaction based on a stripped down version of what you originally posted shows it works.
Thanks a lot, it finally works as you said…
I was pretty sure to have tested that code already, but I should have missed a small part…
This code works:
charset = msg.get_content_charset()
old_content = msg.get_payload(decode=True)
new_content = re.sub(r'</body>', html_footer, old_content)
del msg['content-transfer-encoding']
msg.set_payload(new_content, charset)
See the full code in the repository: https://github.com/Sylvain303/mailman-AttachmentMove/blob/master/AttachmentM...
Regards, Sylvain.
participants (2)
-
Mark Sapiro
-
Sylvain Viart