[issue2182] tokenize: does not allow CR for a newline

[Jared Grubb]

Jared Grubb <python at jaredgrubb.com> added the comment:

This is not a report on a bug in exec(), but rather a bug in the
tokenize module -- the behavior between the CPython tokenizer and the
tokenize module is not consistent. If you look in the tokenize.py
source, it contains code to recognize both \n and \r\n as newlines, but
it ignores the possibility that \r could be the line ending character
(as the Python reference says).

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue2182>
__________________________________