[issue4626] compile() doesn't ignore the source encoding when a string is passed in

[STINNER Victor]

STINNER Victor <victor.stinner at haypocalc.com> added the comment:

@benjamin.peterson: I don't see you changes... I read both patches 
carrefuly and I saw:
 - source_as_string(): remove "cf.cf_flags = supplied_flags | 
PyCF_SOURCE_IS_UTF8;"
 - rename PyTokenizer_FromUnicode() to PyTokenizer_FromUTF8()
 - update NEWS

I don't understand the change in source_as_string(). Except of that, 
it looks correct.

> The one problem is that it causes test_coding to fail because 
> coding cookies are ignored and not checked to see if they
> are the same as the encoding of the file.

The test have to fail, but the error is not the the compile() patch, 
but in the test. Input file is opened as unicode instead of bytes. A 
propose this patch to fix the test:

Index: Lib/test/test_coding.py
===================================================================
--- Lib/test/test_coding.py     (révision 69723)
+++ Lib/test/test_coding.py     (copie de travail)
@@ -17,7 +17,7 @@

         path = os.path.dirname(__file__)
         filename = os.path.join(path, module_name + '.py')
-        fp = open(filename, encoding='utf-8')
+        fp = open(filename, 'rb')
         text = fp.read()
         fp.close()
         self.assertRaises(SyntaxError, compile, text, 
filename, 'exec')

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4626>
_______________________________________