[docs] [issue33766] Grammar Incongruence

[Ammar Askar]

Ammar Askar <ammar at ammaraskar.com> added the comment:

Sorry, I was already working on the patch by the time you posted the comment. If we see above, it seems like the tokenize module doesn't correctly mirror the behavior of the C tokenizer. Do you want to try fixing that as a bug? That would involve making a new bpo ticket and submitting a PR there.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33766>
_______________________________________