[issue25580] async and await missing from token list
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
New submission from SilentGhost: With introduction of async and await tokens in 3.5 the token documentation needs updating. ---------- assignee: docs@python components: Documentation files: token.diff keywords: patch messages: 254281 nosy: SilentGhost, docs@python priority: normal severity: normal stage: patch review status: open title: async and await missing from token list versions: Python 3.5, Python 3.6 Added file: http://bugs.python.org/file40974/token.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Stéphane Wirtel added the comment: With python 3.5, async is a token with the ASYNC type.
tokens = tokenize.generate_tokens(io.StringIO('async def foo').readline) pprint.pprint(list(tokens)) [TokenInfo(type=55 (ASYNC), string='async', start=(1, 0), end=(1, 5), line='async def foo'), TokenInfo(type=1 (NAME), string='def', start=(1, 6), end=(1, 9), line='async def foo'), TokenInfo(type=1 (NAME), string='foo', start=(1, 10), end=(1, 13), line='async def foo'), TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]
---------- nosy: +matrixise _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Martin Panter added the comment: I wonder if the new tokens need a “versionadded” notice. They were added in revision eeeb666a5365 for 3.5. ---------- nosy: +martin.panter, yselivanov _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
SilentGhost added the comment: I've added versionchanged, since this is what's done in os, for similar lists. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Martin Panter added the comment: Do you have a new version of the patch? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Yury Selivanov added the comment: ASYNC/AWAIT tokens are temporary and will be removed in 3.7 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
SilentGhost added the comment: Hm, not sure why the file didn't get uploaded, I clearly remember attaching it. Yury, what is the implication for this issue? Are you suggesting that the note should state that these two are temporary? Or is this information available somewhere else, in What's new, for example? ---------- Added file: http://bugs.python.org/file41028/issue25580.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Yury Selivanov added the comment:
Are you suggesting that the note should state that these two are temporary?
Yes, that would be great.
Or is this information available somewhere else, in What's new, for example?
Not directly, I think. It was detailed to some extent in PEP 492. In what's new of 3.5 & 3.6 it's stated that async/await will be keywords in 3.7. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
SilentGhost added the comment: Here is the updated wording. I'm not familiar with other such cases in the stdlib, if any one knows of another similar case, I'd be glad to correct the language to match previous usage. ---------- Added file: http://bugs.python.org/file41031/issue25580_2.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Martin Panter added the comment: Will they really be removed? What are the compatibility implications? Maybe it would be better to alias them to NAME, or keep them as unused values. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Martin Panter added the comment: My only background here is via the tokenize module, which seems to currently behave as described in <https://www.python.org/dev/peps/pep-0492/#transition-plan>: async and await are either NAME tokens (like ordinary identifiers and other reserved keywords) outside a coroutine definition, but they become special ASYNC and AWAIT tokens inside a coroutine. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Yury Selivanov added the comment:
My only background here is via the tokenize module, which seems to currently behave as described in <https://www.python.org/dev/peps/pep-0492/#transition-plan>: async and await are either NAME tokens (like ordinary identifiers and other reserved keywords) outside a coroutine definition, but they become special ASYNC and AWAIT tokens inside a coroutine.
Exactly. And in 3.7 they will be NAME tokens in any context. That's why I'm reluctant to document them atm. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Yury Selivanov added the comment: Ah, I see that issue25580_2.diff does exactly what I proposed. I'm committing it. ---------- keywords: +easy _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Roundup Robot added the comment: New changeset aa79b2a5b2e1 by Yury Selivanov in branch '3.5': docs: Document ASYNC/AWAIT tokens (issue #25580) https://hg.python.org/cpython/rev/aa79b2a5b2e1 ---------- nosy: +python-dev _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
![](https://secure.gravatar.com/avatar/fa0f7819f1825f596b384c19aa7dcf33.jpg?s=120&d=mm&r=g)
Yury Selivanov added the comment: Thanks for the patch! ---------- resolution: -> fixed stage: patch review -> resolved status: open -> closed type: -> enhancement _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue25580> _______________________________________
participants (5)
-
Martin Panter
-
Roundup Robot
-
SilentGhost
-
Stéphane Wirtel
-
Yury Selivanov