[New-bugs-announce] [issue30788] email.policy.SMTP.fold() issue with long filenames with spaces

[Joel Hillacre]

New submission from Joel Hillacre:

Found an issue in email.policy.SMTP.fold on "Content-Disposition" headers for long filenames with spaces. Managed to simply the issue to a test case as follows:

>>> from email.policy import SMTP; SMTP.fold('Content-Disposition', 'attachment; filename="{}"'.format('1234 67890' + '1234567890' * 6))

Below are the tracebacks various python versions produces for this code. I think that the difference between python 3.5 - 3.6 and 3.7 are related to the fix in bpo-30532.

Python 3.6.1 (default, Apr  7 2017, 09:32:32) & Python 3.5.3 (default, Jan 17 2017, 14:34:36)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.6/email/policy.py", line 183, in fold
    return self._fold(name, value, refold_binary=True)
  File "/usr/lib64/python3.6/email/policy.py", line 213, in _fold
    return self.header_factory(name, ''.join(lines)).fold(policy=self)
  File "/usr/lib64/python3.6/email/headerregistry.py", line 255, in fold
    return header.fold(policy=policy)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 300, in fold
    self._fold(folded)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 1228, in _fold
    rest._fold(folded)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 347, in _fold
    if folded.append_if_fits(part):
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 149, in append_if_fits
    token._fold(self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 338, in _fold
    if folded.append_if_fits(part, tstr):
... last 4 lines repeated 482 more times ...
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 149, in append_if_fits
    token._fold(self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 325, in _fold
    tstr = str(part)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in __str__
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in <genexpr>
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in __str__
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in <genexpr>
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in __str__
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in <genexpr>
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 633, in __str__
    return quote_string(''.join(str(x) for x in self))
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 633, in <genexpr>
    return quote_string(''.join(str(x) for x in self))
RecursionError: maximum recursion depth exceeded while getting the str of an object

Python 3.7.0a0 (heads/master:e613e6add5, Jun 27 2017, 12:17:18) 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/joel/PycharmProjects/cpython/Lib/email/policy.py", line 183, in fold
    return self._fold(name, value, refold_binary=True)
  File "/home/joel/PycharmProjects/cpython/Lib/email/policy.py", line 213, in _fold
    return self.header_factory(name, ''.join(lines)).fold(policy=self)
  File "/home/joel/PycharmProjects/cpython/Lib/email/headerregistry.py", line 255, in fold
    return header.fold(policy=policy)
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 300, in fold
    self._fold(folded)
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 1226, in _fold
    rest._fold(folded)
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 345, in _fold
    if folded.append_if_fits(part):
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 145, in append_if_fits
    ws = token.pop_leading_fws()
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 272, in pop_leading_fws
    return self[0].pop_leading_fws()
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 272, in pop_leading_fws
    return self[0].pop_leading_fws()
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 270, in pop_leading_fws
    if self[0].token_type == 'fws':
IndexError: list index out of range

Note that the following example without a space does not cause a traceback:

Python 3.7.0a0 (heads/master:e613e6add5, Jun 27 2017, 12:17:18) 
>>> from email.policy import SMTP; SMTP.fold('Content-Disposition', 'attachment; filename="{}"'.format('1234567890' + '1234567890' * 6))
'Content-Disposition: attachment;\r\n filename="1234567890123456789012345678901234567890123456789012345678901234567890"\r\n'

----------
components: email
messages: 297072
nosy: barry, jhillacre, r.david.murray
priority: normal
severity: normal
status: open
title: email.policy.SMTP.fold() issue with long filenames with spaces
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue30788>
_______________________________________