[New-bugs-announce] [issue25324] Importing tokenize modifies token

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Importing the tokenize modifies the content of the token module.

>>> import token
>>> len(token.tok_name)
59
>>> import tokenize
>>> len(token.tok_name)
61

Such side effect looks as bad design to me. Either token.tok_name should contain elements needed for tokenize, or tokenize should not modify token.tok_name.

----------
components: Library (Lib)
messages: 252397
nosy: georg.brandl, meador.inge, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Importing tokenize modifies token
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25324>
_______________________________________