[New-bugs-announce] [issue16129] No good way to set 'PYTHONIOENCODING' when embedding python.

[Campbell Barton]

New submission from Campbell Barton:

note, I was asked to report this issue, posted on the py dev mailing list: see - http://code.activestate.com/lists/python-dev/118015/

---

We've run into an issue recently with blender3d on ms-windows where we
want to enforce the encoding is UTF-8 with the embedded python
interpreter.
(the encoding defaults to cp437).

I naively thought setting the environment variable before calling
Py_Initialize() would work, but the way python DLL loads, it gets its
own environment variables that cant be modified directly [1].
eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");

We had bug reports by windows users not able to export files because
the stdout errors on printing paths with unsupported encoding. [2],[3]

---

Of course we could distribute blender with a bat file launcher that
sets env variables, or ask the user to set their env variable - but I
dont think this is really a good option.

I tried overriding the stderr & stdout, but this caused another bug on exiting, giving an assert in MSVCR90.DLL's write.c (called from python32_d.dll):
_VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1);

import sys, io
sys.__stdout__ = sys.stdout =
io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1),
encoding='utf-8', errors='surrogateescape', newline="\n",
line_buffering=True)
sys.__stderr__ = sys.stderr =
io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1),
encoding='utf-8', errors='surrogateescape', newline="\n",
line_buffering=True)



IMHO either of these solutions would be fine.

* have a PyOS_PutEnv() function, gettext has gettext_putenv() to
workaround this problem.

* manage this the same as Py_GetPythonHome(), which can be defined by
the embedding application to override the default.


[1] http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
[2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
[3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555

----------
components: Windows
messages: 171938
nosy: ideasman42
priority: normal
severity: normal
status: open
title: No good way to set 'PYTHONIOENCODING' when embedding python.
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16129>
_______________________________________