[Python-checkins] bpo-33260: Regenerate token.py after removing ASYNC and AWAIT. (GH-6447)
Serhiy Storchaka
webhook-mailer at python.org
Wed Apr 11 12:16:05 EDT 2018
https://github.com/python/cpython/commit/d08972fdb92cad6b813d22205752c97ea18df060
commit: d08972fdb92cad6b813d22205752c97ea18df060
branch: master
author: Serhiy Storchaka <storchaka at gmail.com>
committer: GitHub <noreply at github.com>
date: 2018-04-11T19:15:51+03:00
summary:
bpo-33260: Regenerate token.py after removing ASYNC and AWAIT. (GH-6447)
files:
M Doc/reference/expressions.rst
M Lib/token.py
M Lib/tokenize.py
diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst
index 151062ba1753..e5af927da1ea 100644
--- a/Doc/reference/expressions.rst
+++ b/Doc/reference/expressions.rst
@@ -172,7 +172,7 @@ Common syntax elements for comprehensions are:
.. productionlist::
comprehension: `expression` `comp_for`
- comp_for: [ASYNC] "for" `target_list` "in" `or_test` [`comp_iter`]
+ comp_for: ["async"] "for" `target_list` "in" `or_test` [`comp_iter`]
comp_iter: `comp_for` | `comp_if`
comp_if: "if" `expression_nocond` [`comp_iter`]
diff --git a/Lib/token.py b/Lib/token.py
index 091f80bf191b..ba132059abf5 100644
--- a/Lib/token.py
+++ b/Lib/token.py
@@ -65,14 +65,12 @@
ELLIPSIS = 52
# Don't forget to update the table _PyParser_TokenNames in tokenizer.c!
OP = 53
-AWAIT = 54
-ASYNC = 55
-ERRORTOKEN = 56
+ERRORTOKEN = 54
# These aren't used by the C tokenizer but are needed for tokenize.py
-COMMENT = 57
-NL = 58
-ENCODING = 59
-N_TOKENS = 60
+COMMENT = 55
+NL = 56
+ENCODING = 57
+N_TOKENS = 58
# Special definitions for cooperation with parser
NT_OFFSET = 256
#--end constants--
diff --git a/Lib/tokenize.py b/Lib/tokenize.py
index f5c6ac7f5e05..6528b9006128 100644
--- a/Lib/tokenize.py
+++ b/Lib/tokenize.py
@@ -284,7 +284,7 @@ def compat(self, token, iterable):
self.encoding = tokval
continue
- if toknum in (NAME, NUMBER, ASYNC, AWAIT):
+ if toknum in (NAME, NUMBER):
tokval += ' '
# Insert a space between two consecutive strings
More information about the Python-checkins
mailing list