[New-bugs-announce] [issue46462] Email Header Folding Converts Non-CRLF Newlines to CRLFs

J. Walter Clark report at bugs.python.org
Fri Jan 21 13:21:38 EST 2022

New submission from J. Walter Clark <jwalterclark at gmail.com>:

In various places in the email library `str.splitlines` is used to split up a message where folding might take place in the original message source. This appears to be a bug because when these split parts are re-joined they are joined by a CRLF.

`str.splitlines` splits on "universal newlines" which can include newlines other than the CRLF.

However, the email RFCs define folding whitespace with CRLF as the only possible newline type (optionally surrounded by WSP (SP/HTAB) and/or comments).

The end result is that a message making a roundtrip through the email parser/generator is mangled because it has any non-CRLF "universal newlines" converted to CRLFs. Anything in the header after the non-CRLF "universal newline" appears on it's own line with no preceding whitespace. This appears to happen with all of the stock policies.

from email import message_from_bytes
from email.policy import SMTPUTF8

eml_bytes = b'Header-With-FS-Char: BEFORE\x1cAFTER\r\n\r\nBody\r\n'

message = message_from_bytes(eml_bytes, policy=SMTPUTF8)

b'Header-With-FS-Char: BEFORE\x1cAFTER\r\n\r\nBody\r\n'
b'Header-With-FS-Char: BEFORE\r\nAFTER\r\n\r\nBody\r\n'

The operational impact of this mangling is that the "AFTER" text now makes the message format invalid because it is neither a valid header (no ": ") nor the valid start of a message body (only one CRLF). Common MIME-viewers (e.g. Thunderbird/Outlook) appear to interpret it as a body anyway and any subsequent headers become part of the body.

components: Library (Lib)
messages: 411171
nosy: jwalterclark
priority: normal
severity: normal
status: open
title: Email Header Folding Converts Non-CRLF Newlines to CRLFs
type: behavior
versions: Python 3.11

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list