Problems with email.Generator.Generator

[Peter Otten]

Chris Withers wrote:

> The following piece of code is giving me issues:
> 
> from email.Charset import Charset,QP
> from email.MIMEText import MIMEText
> charset = Charset('utf-8')
> charset.body_encoding = QP
> msg = MIMEText(
>      u'Some text with chars that need encoding: \xa3',
>      'plain',
>      )
> msg.set_charset(charset)
> print msg.as_string()
> 
> Under Python 2.4.2, this produces the following output, as I'd expect:
> 
> MIME-Version: 1.0
> Content-Transfer-Encoding: 8bit
> Content-Type: text/plain; charset="utf-8"
> 
> Some text with chars that need encoding: =A3
> 
> However, under Python 2.4.3, I now get:
> 
> Traceback (most recent call last):
>    File "test_encoding.py", line 14, in ?
>      msg.as_string()
>    File "c:\python24\lib\email\Message.py", line 129,
> in
> as_string
>      g.flatten(self, unixfrom=unixfrom)
>    File "c:\python24\lib\email\Generator.py", line 82,
> in flatten
>      self._write(msg)
>    File "c:\python24\lib\email\Generator.py", line 113,
> in _write
>      self._dispatch(msg)
>    File "c:\python24\lib\email\Generator.py", line 139,
> in
> _dispatch
>      meth(msg)
>    File "c:\python24\lib\email\Generator.py", line 182,
> in
> _handle_text
>      self._fp.write(payload)
> UnicodeEncodeError: 'ascii' codec can't encode
> character
> u'\xa3' in position 41:
>   ordinal not in range(128)
> 
> This seems to be as a result of this change:
> 
>
http://svn.python.org/view/python/branches/release24-maint/Lib/email/Generator.py?rev=42272&r1=37910&r2=42272
> 
> ...which is referred to as part of a fix for this bug:
> 
>
http://sourceforge.net/tracker/?func=detail&aid=1409455&group_id=5470&atid=105470
> 
> Now, is this change to Generator.py in error or am I doing something
> wrong?

I'm not familiar enough with the email package to answer that.
 
> If the latter, how can I change my code such that it works as I'd expect?

email.Generator and email.Message use cStringIO.StringIO internally, which
can't cope with unicode. A quick fix might be to monkey-patch:

from StringIO import StringIO
from email import Generator, Message
Generator.StringIO = Message.StringIO = StringIO
# your code here

Peter