[New-bugs-announce] [issue31394] Ellipsis token.type != token.ELLIPSIS

[Aivar Annamaa]

New submission from Aivar Annamaa:

Type code for ellipsis token doesn't match with the constant token.ELLIPSIS:
---------------------------------------
import io
import token
import tokenize

source = "..."

tokens = list(tokenize.tokenize(io.BytesIO(source.encode('utf-8')).readline))
ellipsis = tokens[1]

print(ellipsis)
print(token.ELLIPSIS) 
-----------------------------------------

This code outputs following in Python 3.5 and 3.6:

> TokenInfo(type=53 (OP), string='...', start=(1, 0), end=(1, 3), line='...')
> 52

and following in Python 3.4

> TokenInfo(type=52 (OP), string='...', start=(1, 0), end=(1, 3), line='...')
> 51

----------
components: Interpreter Core
messages: 301687
nosy: Aivar.Annamaa
priority: normal
severity: normal
status: open
title: Ellipsis token.type != token.ELLIPSIS
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue31394>
_______________________________________