[Python-checkins] [3.9] bpo-44409: Fix error location in tokenizer errors that happen during initialization (GH-26712). (GH-26723)
pablogsal
webhook-mailer at python.org
Mon Jun 14 13:08:00 EDT 2021
https://github.com/python/cpython/commit/0d0a9eaa822658679cc2b65f125ab74bfd4aedfe
commit: 0d0a9eaa822658679cc2b65f125ab74bfd4aedfe
branch: 3.9
author: Pablo Galindo <Pablogsal at gmail.com>
committer: pablogsal <Pablogsal at gmail.com>
date: 2021-06-14T18:07:51+01:00
summary:
[3.9] bpo-44409: Fix error location in tokenizer errors that happen during initialization (GH-26712). (GH-26723)
(cherry picked from commit 507ed6fa1d6661e0f8e6d3282764aa9625a99594)
Co-authored-by: Pablo Galindo <Pablogsal at gmail.com>
files:
A Misc/NEWS.d/next/Core and Builtins/2021-06-13-23-12-18.bpo-44409.eW4LS-.rst
M Lib/test/test_exceptions.py
M Parser/pegen/pegen.c
diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py
index 8d125b57ad6d5a..0f7b7f84809d96 100644
--- a/Lib/test/test_exceptions.py
+++ b/Lib/test/test_exceptions.py
@@ -235,6 +235,7 @@ def baz():
""", 9, 20)
check("pass\npass\npass\n(1+)\npass\npass\npass", 4, 4)
check("(1+)", 1, 4)
+ check(b"\xef\xbb\xbf#coding: utf8\nprint('\xe6\x88\x91')\n", 0, -1)
# Errors thrown by symtable.c
check('x = [(yield i) for i in range(3)]', 1, 5)
diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-06-13-23-12-18.bpo-44409.eW4LS-.rst b/Misc/NEWS.d/next/Core and Builtins/2021-06-13-23-12-18.bpo-44409.eW4LS-.rst
new file mode 100644
index 00000000000000..0f204ed812b27a
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2021-06-13-23-12-18.bpo-44409.eW4LS-.rst
@@ -0,0 +1,2 @@
+Fix error location information for tokenizer errors raised on initialization
+of the tokenizer. Patch by Pablo Galindo.
diff --git a/Parser/pegen/pegen.c b/Parser/pegen/pegen.c
index 111009af63e22a..bf53214b4d0637 100644
--- a/Parser/pegen/pegen.c
+++ b/Parser/pegen/pegen.c
@@ -269,6 +269,7 @@ static void
raise_tokenizer_init_error(PyObject *filename)
{
if (!(PyErr_ExceptionMatches(PyExc_LookupError)
+ || PyErr_ExceptionMatches(PyExc_SyntaxError)
|| PyErr_ExceptionMatches(PyExc_ValueError)
|| PyErr_ExceptionMatches(PyExc_UnicodeDecodeError))) {
return;
More information about the Python-checkins
mailing list