[docs] [Roto Rooters] Multiple bugs in email package and documentation - Python 3.4 (fwd)

R. David Murray rdmurray at bitdance.com
Mon Nov 23 10:52:14 EST 2015 

The first bug has already been reported.  I just need to get around to
addressing it :(.

The report should be posted to bugs.python.org, otherwise it is unlikely
to get addressed.  It looks to be several bugs with one reproducer,
but it's fine to start with just a bug report with the reproducer and
a description of the issues encountered.

Ervin: FYI the api being provisional means it is not yet subject to our
full backward compatibility policies.  The current plan is for it to become
so in 3.6 (to no longer be provisional).  Obviously whatever these
issues are they should be dealt with before then :)

--David

On Mon, 23 Nov 2015 14:59:34 +0100, Laura Creighton <lac at openend.se> wrote:
> Thought this would be of interest to those not reading the docs
> mailing list.  If all of you already read that list, then I
> apologise for wasting your time.
> 
> Laura
> 
> ------- Forwarded Message
> 
> From:    Ervin Hegedüs <airween at gmail.com>
> Subject: [docs] Multiple bugs in email package and documentation - Python 3.4
> 
> 
> Hi,
> 
> looks like I've found a complex, and connected bug in 'email' package.
> 
> Bug 1:
> =====
> 
> I've read the documentation and examples:
> 
> [1]https://docs.python.org/3.4/library/email-examples.html
> 
> I've looked up this example (copy and paste, replace the addresses):
> 
> [2]https://docs.python.org/3.4/library/email-examples.html#
> examples-using-the-provisional-api
> 
> and this example gives a bug:
> 
> (after several lines, the end of the stack is this)
> 
>   File "/usr/lib/python3.4/email/_header_value_parser.py", line 629, in
> __str__
>     return quote_string(''.join(str(x) for x in self))
>   File "/usr/lib/python3.4/email/_header_value_parser.py", line 629, in
> <genexpr>
>     return quote_string(''.join(str(x) for x in self))
> RuntimeError: maximum recursion depth exceeded while getting the str of an
> object
> 
> Okay, I've commented out the file write method, in line 50. and 51 - then
> script terminated normally.
> 
> 
> Bug 2:
> =====
> 
> I've received the e-mail, but the MIME tree of message is completely wrong. I
> think the original author would like to make a html mail (multipart/
> alternative), with embedded image. The regular tree of a html mail is this:
> 
>   I     1 <no description>      [multipa/alternativ, 7bit, 0,5K]
>   I     2 ├─><nodescription>   [text/plain, quoted, utf-8, 0,1K]
>   I     3 └─><nodescription>    [text/html, quoted, utf-8, 0,2K]
>   I     4 attached_image.png            [image/png, base64, 43K]
> 
> but the script in the example gives this tree:
> 
>   I     1 <no description>      [multipa/alternativ, 7bit, 101K]
>   I     2 ├─><no description>    [text/plain, 8bit, utf-8, 0,1K]
>   I     3 └─><no description>      [multipa/related, 7bit, 101K]
>   I     4   ├─><no description> [text/html, quoted, utf-8, 0,3K]
>   I     5   └─><no description>       [image/jpeg, base64, 100K]
> 
> So, the problem is, in a general MUA (Thunderbird, any webmail client...) the
> image is not visible, but mail contains it.
> 
> 
> Bug 3:
> =====
> 
> The attached image is "pseduo" base64 encoded. The encode function is defined
> in email/contenmanager.py:
> 
>    129  # XXX: This is a cleaned-up version of base64mime.body_encode.  It
> would
>    130  # be nice to drop both this and quoprimime.body_encode in favor of
>    131  # enhanced binascii routines that accepted a max_line_length
> parameter.
>    132  def _encode_base64(data, max_line_length):
>    133      encoded_lines = []
>    134      unencoded_bytes_per_line = max_line_length * 3 // 4
>    135      for i in range(0, len(data), unencoded_bytes_per_line):
>    136          thisline = data[i:i+unencoded_bytes_per_line]
>    137          encoded_lines.append(binascii.b2a_base64(thisline).decode
> ('ascii'))
>    138      return ''.join(encoded_lines)
> 
> But this code generates this form of base64:
> 
> --===============8598821442385181731==
> Content-Type: image/jpeg
> Content-Transfer-Encoding: base64
> Content-ID: <[3]20151122160934.12051.88999 at myhost.com>
> MIME-Version: 1.0
> Content-Disposition: inline
> 
> /9j/
> 4R1zRXhpZgAASUkqAAgAAAAMAAABAwABAAAAWQIAAAEBAwABAAAA3AAAAAIBAwADAAAAngAAAA==
> BgEDAAEAAAACAAAAEgEDAAEAAAABAAAAFQEDAAEAAAADAAAAGgEFAAEAAACkAAAAGwEFAAEAAACsAA
> ==
> 
> 
> I don't know, why had been created a new encoder here, because there is the
> email/encoders.py module, which contains a correct base64 encoder. With that,
> I've got this form (with same image):
> 
> --===============6414765018955620772==
> Content-Type: image/jpeg
> Content-Transfer-Encoding: base64
> Content-ID: <[4]20151122161400.14276.87008 at myhost.com>
> MIME-Version: 1.0
> Content-Disposition: inline
> 
> /9j/4R1zRXhpZgAASUkqAAgAAAAMAAABAwABAAAAWQIAAAEBAwABAAAA3AAAAAIBAwADAAAAngAA
> AAYBAwABAAAAAgAAABIBAwABAAAAAQAAABUBAwABAAAAAwAAABoBBQABAAAApAAAABsBBQABAAAA
> 
> 
> I've checked this routine with another MIME tree, where image should be
> visible, but the user agent could't show that.
> 
> Whit the correct tree and correct encoding, the e-mail visible as correctly,
> with inline image.
> 
> 
> I don't know, how can I report this connected bugs? I have a good and working
> example - but it uses not exactly modules, than in example, however it builds
> a correct e-mail message.
> (I don't know, what does it mean the "Provisional API" - and what is the
> eventual?)
> 
> 
> Thanks,
> 
> 
> a.
> 
> References:
> 
> [1] https://docs.python.org/3.4/library/email-examples.html
> [2] https://docs.python.org/3.4/library/email-examples.html#examples-using-the-p
> rovisional-api
> [3] mailto:20151122160934.12051.88999 at myhost.com
> [4] mailto:20151122161400.14276.87008 at myhost.com
> _______________________________________________
> docs mailing list
> docs at python.org
> https://mail.python.org/mailman/listinfo/docs
> lac at fido:~/Mail/webmaster$ show
> show
> (Message inbox:4120)
> Date:    Sun, 22 Nov 2015 18:32:54 +0100
> To:      docs at python.org
> From:    Ervin Hegedüs <airween at gmail.com>
> Subject: [docs] Multiple bugs in email package and documentation - Python 3.4
> 
> 
> Hi,
> 
> looks like I've found a complex, and connected bug in 'email' package.
> 
> Bug 1:
> =====
> 
> I've read the documentation and examples:
> 
> [1]https://docs.python.org/3.4/library/email-examples.html
> 
> I've looked up this example (copy and paste, replace the addresses):
> 
> [2]https://docs.python.org/3.4/library/email-examples.html#
> examples-using-the-provisional-api
> 
> and this example gives a bug:
> 
> (after several lines, the end of the stack is this)
> 
>   File "/usr/lib/python3.4/email/_header_value_parser.py", line 629, in
> __str__
>     return quote_string(''.join(str(x) for x in self))
>   File "/usr/lib/python3.4/email/_header_value_parser.py", line 629, in
> <genexpr>
>     return quote_string(''.join(str(x) for x in self))
> RuntimeError: maximum recursion depth exceeded while getting the str of an
> object
> 
> Okay, I've commented out the file write method, in line 50. and 51 - then
> script terminated normally.
> 
> 
> Bug 2:
> =====
> 
> I've received the e-mail, but the MIME tree of message is completely wrong. I
> think the original author would like to make a html mail (multipart/
> alternative), with embedded image. The regular tree of a html mail is this:
> 
>   I     1 <no description>      [multipa/alternativ, 7bit, 0,5K]
>   I     2 ├─><nodescription>   [text/plain, quoted, utf-8, 0,1K]
>   I     3 └─><nodescription>    [text/html, quoted, utf-8, 0,2K]
>   I     4 attached_image.png            [image/png, base64, 43K]
> 
> but the script in the example gives this tree:
> 
>   I     1 <no description>      [multipa/alternativ, 7bit, 101K]
>   I     2 ├─><no description>    [text/plain, 8bit, utf-8, 0,1K]
>   I     3 └─><no description>      [multipa/related, 7bit, 101K]
>   I     4   ├─><no description> [text/html, quoted, utf-8, 0,3K]
>   I     5   └─><no description>       [image/jpeg, base64, 100K]
> 
> So, the problem is, in a general MUA (Thunderbird, any webmail client...) the
> image is not visible, but mail contains it.
> 
> 
> Bug 3:
> =====
> 
> The attached image is "pseduo" base64 encoded. The encode function is defined
> in email/contenmanager.py:
> 
>    129  # XXX: This is a cleaned-up version of base64mime.body_encode.  It
> would
>    130  # be nice to drop both this and quoprimime.body_encode in favor of
>    131  # enhanced binascii routines that accepted a max_line_length
> parameter.
>    132  def _encode_base64(data, max_line_length):
>    133      encoded_lines = []
>    134      unencoded_bytes_per_line = max_line_length * 3 // 4
>    135      for i in range(0, len(data), unencoded_bytes_per_line):
>    136          thisline = data[i:i+unencoded_bytes_per_line]
>    137          encoded_lines.append(binascii.b2a_base64(thisline).decode
> ('ascii'))
>    138      return ''.join(encoded_lines)
> 
> But this code generates this form of base64:
> 
> --===============8598821442385181731==
> Content-Type: image/jpeg
> Content-Transfer-Encoding: base64
> Content-ID: <[3]20151122160934.12051.88999 at myhost.com>
> MIME-Version: 1.0
> Content-Disposition: inline
> 
> /9j/
> 4R1zRXhpZgAASUkqAAgAAAAMAAABAwABAAAAWQIAAAEBAwABAAAA3AAAAAIBAwADAAAAngAAAA==
> BgEDAAEAAAACAAAAEgEDAAEAAAABAAAAFQEDAAEAAAADAAAAGgEFAAEAAACkAAAAGwEFAAEAAACsAA
> ==
> 
> 
> I don't know, why had been created a new encoder here, because there is the
> email/encoders.py module, which contains a correct base64 encoder. With that,
> I've got this form (with same image):
> 
> --===============6414765018955620772==
> Content-Type: image/jpeg
> Content-Transfer-Encoding: base64
> Content-ID: <[4]20151122161400.14276.87008 at myhost.com>
> MIME-Version: 1.0
> Content-Disposition: inline
> 
> /9j/4R1zRXhpZgAASUkqAAgAAAAMAAABAwABAAAAWQIAAAEBAwABAAAA3AAAAAIBAwADAAAAngAA
> AAYBAwABAAAAAgAAABIBAwABAAAAAQAAABUBAwABAAAAAwAAABoBBQABAAAApAAAABsBBQABAAAA
> 
> 
> I've checked this routine with another MIME tree, where image should be
> visible, but the user agent could't show that.
> 
> Whit the correct tree and correct encoding, the e-mail visible as correctly,
> with inline image.
> 
> 
> I don't know, how can I report this connected bugs? I have a good and working
> example - but it uses not exactly modules, than in example, however it builds
> a correct e-mail message.
> (I don't know, what does it mean the "Provisional API" - and what is the
> eventual?)
> 
> 
> Thanks,
> 
> 
> a.
> 
> References:
> 
> [1] https://docs.python.org/3.4/library/email-examples.html
> [2] https://docs.python.org/3.4/library/email-examples.html#examples-using-the-p
> rovisional-api
> [3] mailto:20151122160934.12051.88999 at myhost.com
> [4] mailto:20151122161400.14276.87008 at myhost.com
> _______________________________________________
> docs mailing list
> docs at python.org
> https://mail.python.org/mailman/listinfo/docs
> 
> 
> ------- End of Forwarded Message
> _______________________________________________
> Roto-Rooters mailing list
> Roto-Rooters at wooz.org
> Your options: http://www.wooz.org/mailman/options/roto-rooters/rdmurray%40bitdance.com
> Subscribed as: rdmurray at bitdance.com

More information about the docs mailing list