[New-bugs-announce] [issue16564] email.generator.BytesGenerator fails with bytes payload

Alexander Kruppa report at bugs.python.org
Tue Nov 27 15:38:50 CET 2012


New submission from Alexander Kruppa:

I'm trying to use the email.* functions to craft HTTP POST data for file upload. Trying something like

filedata = open("data", "rb").read()
postdata = MIMEMultipart()
fileattachment = MIMEApplication(filedata, _encoder=email.encoders.encode_noop)
postdata.attach(fileattachment)
fp = BytesIO()
g = BytesGenerator(fp)
g.flatten(postdata, unixfrom=False)

fails with 

Traceback (most recent call last):
  File "./minetest.py", line 30, in <module>
    g.flatten(postdata, unixfrom=False)
  File "/usr/lib/python3.2/email/generator.py", line 91, in flatten
    self._write(msg)
  File "/usr/lib/python3.2/email/generator.py", line 137, in _write
    self._dispatch(msg)
  File "/usr/lib/python3.2/email/generator.py", line 163, in _dispatch
    meth(msg)
  File "/usr/lib/python3.2/email/generator.py", line 224, in _handle_multipart
    g.flatten(part, unixfrom=False, linesep=self._NL)
  File "/usr/lib/python3.2/email/generator.py", line 91, in flatten
    self._write(msg)
  File "/usr/lib/python3.2/email/generator.py", line 137, in _write
    self._dispatch(msg)
  File "/usr/lib/python3.2/email/generator.py", line 163, in _dispatch
    meth(msg)
  File "/usr/lib/python3.2/email/generator.py", line 192, in _handle_text
    raise TypeError('string payload expected: %s' % type(payload))
TypeError: string payload expected: <class 'bytes'>

This is because BytesGenerator._handle_text() expects str payload in which byte values that are non-printable in ASCII have been replaced by surrogates. The example above creates a bytes payload, however, for which super(BytesGenerator,self)._handle_text(msg) = Generator._handle_text(msg) throws the exception.

Note that using any email.encoders other than encode_noop does not really fit the HTTP POST bill, as those define a Content-Transfer-Encoding which HTTP does not know.

It would seem better to me to let BytesGenerator accept a bytes payload and just copy that to the output, rather than making the application encode the bytes as a string, hopefully in a way that s.encode('ascii', 'surrogateescape') can invert.

E.g., a workaround class I use now does

class FixedBytesGenerator(BytesGenerator):
    def _handle_bytes(self, msg):  
        payload = msg.get_payload()
        if payload is None:
            return
        if isinstance(payload, bytes):
            self._fp.write(payload)   
        elif isinstance(payload, str):
            super(FixedBytesGenerator,self)._handle_text(msg)
        else:
            # Payload is neither bytes not string - this can't be right
            raise TypeError('bytes or str payload expected: %s' % type(payload))
    _writeBody = _handle_bytes

----------
components: Library (Lib)
messages: 176476
nosy: Alexander.Kruppa
priority: normal
severity: normal
status: open
title: email.generator.BytesGenerator fails with bytes payload
type: behavior
versions: Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16564>
_______________________________________


More information about the New-bugs-announce mailing list