[docs] [issue8968] token type constants are not documented

[Ilya Sandler]

Ilya Sandler <ilya.sandler at gmail.com> added the comment:

I'm attaching a documentation patch.

Do note that there is also a bit of code-level inconsistency: a few tokens (COMMENT, NL) are defined in tokenize module which is strange and inconvenient.

Should that be fixed too? (by moving token definitions into token module)

----------
keywords: +patch
Added file: http://bugs.python.org/file17641/token.rst.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue8968>
_______________________________________