[Tutor] myown.getfilesystemencoding()

[eryksun]

On Sat, Aug 31, 2013 at 9:16 AM, Oscar Benjamin
<oscar.j.benjamin at gmail.com> wrote:
> Spyder has both an internal interpreter and an external interpreter.
> One is the same interpreter process that runs the Spyder GUI. The
> other is run in a subprocess which keeps the GUI safe but reduces your
> ability to inspect the workspace data via the GUI. So presumable
> Albert means the "external" interpreter here.

I installed Spyder on Windows to look into this. It's using Qt
QProcess to run the external interpreter in a child process.
sys.stdin.isatty() confirms it's not a tty, and Process Explorer
confirms that all 3 standard I/O handles (from msvcrt.get_osfhandle())
are pipes.

The file encoding is None for piped standard I/O, so printing unicode
falls back to the default encoding. Normally this is ASCII in 2.x, but
Spyder uses sitecustomize to set the default encoding based on the
default locale. It also sets the hidden console's codepage:

    if os.name == 'nt': # Windows platforms

        # Setting console encoding (otherwise Python does not
        # recognize encoding)
        try:
            import locale, ctypes
            _t, _cp = locale.getdefaultlocale('LANG')
            try:
                _cp = int(_cp[2:])
                ctypes.windll.kernel32.SetConsoleCP(_cp)
                ctypes.windll.kernel32.SetConsoleOutputCP(_cp)
            except (ValueError, TypeError):
                # Code page number in locale is not valid
                pass
        except ImportError:
            pass

http://code.google.com/p/spyderlib/source/browse/spyderlib/
widgets/externalshell/sitecustomize.py?name=v2.2.0#74

Probably this was added for a good reason, but I don't grok the point.
Python isn't interested in the hidden console window at this stage,
and the standard handles are all pipes. I didn't notice any difference
with these lines commented out, running with Python 2.7.5. YMMV

There's a design flaw here since sys.stdin.encoding is used by the
parser in single-input mode. With it set to None, Unicode literals
entered in the REPL will be incorrectly parsed if they use non-ASCII
byte values. For example, given the input is Windows 1252, then u'€'
will be parsed as u'\x80' (i.e. PAD, a C1 Control code).

Here's an alternative to messing with the default encoding -- at least
for the new version of Spyder that doesn't have to support 2.5. Python
2.6+ checks for the PYTHONIOENCODING environment variable. This
overrides the encoding/errors values in Py_InitializeEx():

http://hg.python.org/cpython/file/70274d53c1dd/Python/pythonrun.c#l265

You can test setting PYTHONIOENCODING without restarting Spyder. Just
bring up Spyder's "Internal Console" and set
os.environ['PYTHONIOENCODING']. The change applies to new interpreters
started from the "Interpreters" menu. Spyder could set this itself in
the environment that gets passed to the QProcess object.