[issue13832] tokenization assuming ASCII whitespace; missing multiline case

[Jim Jewett]

Jim Jewett <jimjjewett at gmail.com> added the comment:

Ignoring non-ascii whitespace is defensible, and I agree that it should match the rest of the parser.  Ignoring 2nd lines is still a problem, and supposedly part of what got fixed.  Test case:

s="""x=5  # comment
x=6
"""
compile(s, "<testbadsingle>", 'single')

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13832>
_______________________________________