[New-bugs-announce] [issue20531] TypeError in e-mail.parser when non-ASCII is present

[Jason R. Coombs]

New submission from Jason R. Coombs:

As reported in https://bitbucket.org/dholth/wheel/issue/104, the email.parser no longer accepts Unicode content as it did in 3.3. I searched the What's New and module documentation, but found no indication that this behavior is no longer supported, so it appears to be a regression. If it's an intentional change, the behavior should be documented in one of the aforementioned documents.

Consider this simple test case:

# -*- coding: utf-8 -*-
import email.parser
meta = """
Header: ☃
"""
email.parser.Parser().parsestr(meta)

Run that on Python 3.3.3 or Python 2 and it executes silently. Run it on Python 3.4.0b3 and it produces this traceback:

Traceback (most recent call last):
  File "C:\Users\jaraco\projects\public\wheel\test.py", line 6, in <module>
    email.parser.Parser().parsestr(meta)
  File "C:\Program Files\Python34\lib\email\parser.py", line 70, in parsestr
    return self.parse(StringIO(text), headersonly=headersonly)
  File "C:\Program Files\Python34\lib\email\parser.py", line 60, in parse
    return feedparser.close()
  File "C:\Program Files\Python34\lib\email\feedparser.py", line 170, in close
    self._call_parse()
  File "C:\Program Files\Python34\lib\email\feedparser.py", line 163, in _call_parse
    self._parse()
  File "C:\Program Files\Python34\lib\email\feedparser.py", line 449, in _parsegen
    self._cur.set_payload(EMPTYSTRING.join(lines))
  File "C:\Program Files\Python34\lib\email\message.py", line 311, in set_payload
    " payload") from None
TypeError: charset argument must be specified when non-ASCII characters are used in the payload

----------
components: email
keywords: 3.4regression
messages: 210390
nosy: barry, jason.coombs, r.david.murray
priority: normal
severity: normal
status: open
title: TypeError in e-mail.parser when non-ASCII is present
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20531>
_______________________________________