[Python-checkins] [3.6] bpo-33899: Mention tokenize behavior change in What's New (GH-10073) (GH-10075)

Miss Islington (bot) webhook-mailer at python.org
Wed Oct 24 03:32:41 EDT 2018

commit: 9a0476283393f9988d0946491052d7724a7f9d21
branch: 3.6
author: Tal Einat <taleinat+github at gmail.com>
committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
date: 2018-10-24T00:32:37-07:00

[3.6] bpo-33899: Mention tokenize behavior change in What's New (GH-10073) (GH-10075)

(cherry picked from commit dfba1f67e7f1381ceb7cec8fbcfa37337620a9b0)


M Doc/whatsnew/3.6.rst

diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
index 96a8831cd6ff..009584428fc1 100644
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -2433,3 +2433,8 @@ Notable changes in Python 3.6.7
 :mod:`xml.dom.minidom` and mod:`xml.sax` modules no longer process
 external entities by default. See also :issue:`17239`.
+In 3.6.7 the :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token
+when provided with input that does not have a trailing new line.  This behavior
+now matches what the C tokenizer does internally.
+(Contributed by Ammar Askar in :issue:`33899`.)

More information about the Python-checkins mailing list