[Python-checkins] r73439 - in python/trunk: Lib/test/test_coding.py Misc/NEWS Parser/tokenizer.c

benjamin.peterson python-checkins at python.org
Tue Jun 16 02:29:31 CEST 2009


Author: benjamin.peterson
Date: Tue Jun 16 02:29:31 2009
New Revision: 73439

Log:
don't mask encoding errors when decoding a string #6289

Modified:
   python/trunk/Lib/test/test_coding.py
   python/trunk/Misc/NEWS
   python/trunk/Parser/tokenizer.c

Modified: python/trunk/Lib/test/test_coding.py
==============================================================================
--- python/trunk/Lib/test/test_coding.py	(original)
+++ python/trunk/Lib/test/test_coding.py	Tue Jun 16 02:29:31 2009
@@ -21,6 +21,18 @@
         fp.close()
         self.assertRaises(SyntaxError, compile, text, filename, 'exec')
 
+    def test_error_from_string(self):
+        # See http://bugs.python.org/issue6289
+        input = u"# coding: ascii\n\N{SNOWMAN}".encode('utf-8')
+        try:
+            compile(input, "<string>", "exec")
+        except SyntaxError as e:
+            expected = "'ascii' codec can't decode byte 0xe2 in position 16: " \
+                "ordinal not in range(128)"
+            self.assertTrue(str(e).startswith(expected))
+        else:
+            self.fail("didn't raise")
+
 def test_main():
     test.test_support.run_unittest(CodingTest)
 

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS	(original)
+++ python/trunk/Misc/NEWS	Tue Jun 16 02:29:31 2009
@@ -12,6 +12,8 @@
 Core and Builtins
 -----------------
 
+- Issue #6289: Encoding errors from compile() were being masked.
+
 - When no module is given in a relative import, the module field of the
   ImportFrom AST node is now None instead of an empty string.
 

Modified: python/trunk/Parser/tokenizer.c
==============================================================================
--- python/trunk/Parser/tokenizer.c	(original)
+++ python/trunk/Parser/tokenizer.c	Tue Jun 16 02:29:31 2009
@@ -619,11 +619,8 @@
 	if (tok->enc != NULL) {
 		assert(utf8 == NULL);
 		utf8 = translate_into_utf8(str, tok->enc);
-		if (utf8 == NULL) {
-			PyErr_Format(PyExc_SyntaxError,
-				"unknown encoding: %s", tok->enc);
+		if (utf8 == NULL)
 			return error_ret(tok);
-		}
 		str = PyString_AsString(utf8);
 	}
 #endif


More information about the Python-checkins mailing list