[Python-checkins] [3.6] bpo-31394: Clarify documentation about token type attribute (GH-3469) (GH-3525)

[Mariatta]

https://github.com/python/cpython/commit/5513e888e9a742156c35ce7ab628407d8cf9e1f0
commit: 5513e888e9a742156c35ce7ab628407d8cf9e1f0
branch: 3.6
author: Mariatta <Mariatta at users.noreply.github.com>
committer: GitHub <noreply at github.com>
date: 2017-09-12T20:43:04-07:00
summary:

[3.6] bpo-31394: Clarify documentation about token type attribute (GH-3469) (GH-3525)

Make it clear that Ellipsis tokens also have type attribute set to token.OP..
(cherry picked from commit 5f8fbf917ebf2398aa75a1f271617e2e50ab7c88)

files:
M Doc/library/tokenize.rst

diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst
index ff55aacbd44..afed36b9e46 100644
--- a/Doc/library/tokenize.rst
+++ b/Doc/library/tokenize.rst
@@ -16,8 +16,9 @@ implemented in Python.  The scanner in this module returns comments as tokens
 as well, making it useful for implementing "pretty-printers," including
 colorizers for on-screen displays.
 
-To simplify token stream handling, all :ref:`operators` and :ref:`delimiters`
-tokens are returned using the generic :data:`token.OP` token type.  The exact
+To simplify token stream handling, all :ref:`operator <operators>` and
+:ref:`delimiter <delimiters>` tokens and :data:`Ellipsis` are returned using
+the generic :data:`~token.OP` token type.  The exact
 type can be determined by checking the ``exact_type`` property on the
 :term:`named tuple` returned from :func:`tokenize.tokenize`.