[issue10335] tokenize.open_python(): open a Python file with the right encoding

Alexander Belopolsky report at bugs.python.org
Sat Nov 6 17:00:15 CET 2010

Alexander Belopolsky <belopolsky at users.sourceforge.net> added the comment:

+1 on adding the function.

Note that it is useful for opening any text file with an encoding cookie, not only python source code, so "tokenize.open()" sounds attractive.

Once we are at it, I would like to factor out and document code that extracts the cookie from a decoded string.  This functionality is useful for tools that produce annotated source code and want to preserve source encoding in their output.  See issue10329.

Also, to answer Victor's comment on issue10329, msg120601, while using this function simplifies dealing with source files in trace module, it does not solve all the issues: source file will still be opened separately in linecache.gettines() and find_executable_linenos() and  find_executable_linenos() still does not respect __loader__ interface.  See issue 10342.

nosy: +belopolsky

Python tracker <report at bugs.python.org>

More information about the Python-bugs-list mailing list