[issue9122] Problems with multiprocessing, Python embedding and Windows

[Alexander Myodov]

New submission from Alexander Myodov <amyodov at gmail.com>:

Hello,

I am using Python 2.6.5 and Cython (to embed the Python interpreter) to create a simple launcher.exe launcher which then imports and uses the remaining logic from the other modules, conveniently stored in a single modules.zip file (i.e. importing from inside zip file). Such setup causes some issues when it comes to the multiprocessing.

First of all, the __main__ module is built as a binary (via Cython->mingw), but the stuff like freeze is not used. In multiprocessing/forking.py, when it comes to determination of the primary path, there is a WINEXE variable which affects the further processing. In its assignment, getattr(sys, 'frozen', False) obviously results in False, causing the WINEXE variable be False too (though in my case it is obviously wrong, the stuff like imp.is_builtin("__main__") shows that the main module is indeed builtin). This is the first problem which probably could be solved by adding an additional check for something like imp.is_builtin("__main__") to WINEXE assignment.
Further, as the WINEXE is wrongly False, it causes the "if not WINEXE:" branch to be processed, and adds the 'main_path' to the data, containing the name of the .exe file. 
On the forked side, d['main_path'] is received and parsed, and the name of the .exe file is split out of its extension, and such name is attempted to be imported, using imp.find_module/imp.load_module. The import fails, as there is indeed no such files wildly lying around in the installation; on the other hand, to get a quick fix of the problem, I thought of adding the appropriate launcher.py(.pyo/.pyc) files into the modules.zip, so that they still could be externally imported by the "launcher" name, even though it is already builtin under the __main__ name; but the second problem is that the imp.find_module/imp.load_module functions do not find the module inside the zip files, like the import statement does.

----------
components: Extension Modules, Windows
messages: 108953
nosy: honeyman
priority: normal
severity: normal
status: open
title: Problems with multiprocessing, Python embedding and Windows
versions: Python 2.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9122>
_______________________________________