[New-bugs-announce] [issue45299] SMTP.send_message() does from mangling when it should not

Grant Edwards report at bugs.python.org
Mon Sep 27 01:02:29 EDT 2021

New submission from Grant Edwards <grant.b.edwards at gmail.com>:

SMTP.send_message() does from mangling even when the message's policy
has that disabled. The problem is in the send_messsage() function
shown below:

   912      def send_message(self, msg, from_addr=None, to_addrs=None,
   913                       mail_options=(), rcpt_options=()):
   914          """Converts message to a bytestring and passes it to sendmail.
   963          # Make a local copy so we can delete the bcc headers.
   964          msg_copy = copy.copy(msg)
   977          with io.BytesIO() as bytesmsg:
   978              if international:
   979                  g = email.generator.BytesGenerator(
   980                      bytesmsg, policy=msg.policy.clone(utf8=True))
   981                  mail_options = (*mail_options, 'SMTPUTF8', 'BODY=8BITMIME')
   982              else:
   983                  g = email.generator.BytesGenerator(bytesmsg)
   984              g.flatten(msg_copy, linesep='\r\n')
   985              flatmsg = bytesmsg.getvalue()

If 'international' is True, then the BytesGenerator is passed
msg.policy with utf8 added, and from mangling is only done if the
message's policy has from mangling enabled. This is correct behavior.

If 'international' is False, then the generator always does from
mangling regardless of the message'spolicy. From mangling is only used
when writing message to mbox format files. When sending a message via
SMTP It is wrong to do it when the message's policy says not to.

This needs to be fixed by passing the message's policy to BytesGenerator()
in the 'else' clause also.  I would suggest changing

   983                  g = email.generator.BytesGenerator(bytesmsg)

   983                  g = email.generator.BytesGenerator(bytesmsg, policy=msg.policy)

The problem is that if neither mangle_from_ nor policy arguments are
passed to email.generator.BytesGenerator(), then mangle_from_ defaults
to True, and the mangle_from_ setting in the message is ignored. This is
not really documented:


    If optional mangle_from_ is True, put a > character in front of
    any line in the body that starts with the exact string "From ",
    that is From followed by a space at the beginning of a
    line. mangle_from_ defaults to the value of the mangle_from_
    setting of the policy.
Where "the policy" refers to the one passed to BytesGenerator(). Note
that this section does _not_ state what happens if neither
mangle_from_ nor policy are passed to BytesGenerator(). What actually
happens is that in that case mangle_from_ defaults to True. (Not a
good choice for default, since that's only useful for the one case
where you're writing to an mbox file.)

However, there is also a misleading sentence later in that same

    If policy is None (the default), use the policy associated with
    the Message or EmailMessage object passed to flatten to control
    the message generation.

That's only partially true. If you don't pass a policy to
BytesGenerator(), only _some_ of the settings from the message's
policy will be used. Some policy settings (e.g. mangle_from_) are
obeyed when passed to BytesGenerator(), but ignored in the message's
policy even if there was no policy passed to BytesGenerator().

The documentation needs to be changed to state that mangle_from_
defaults to True if neitehr mangle_from_ nor policy are passed to
BytesGenerator(), and that last sentence needs to be changed to state
that when no policy is passed to BytesGenerator() only _some_ of the
fields in the message's policy are used to control the message
generation. (An actual list of policy fields are obeyed from the
message's policy would be nice.)

components: Library (Lib)
messages: 402690
nosy: grant.b.edwards
priority: normal
severity: normal
status: open
title: SMTP.send_message() does from mangling when it should not
versions: Python 3.9

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list