[Numpy-discussion] multiprocessing, numpy and 32-64 bit cohabitation

[Antony Lee]

Hi all,

While I realize that this is certainly tweaking multiprocessing beyond its
specifications, I would like to use it on Windows to start a 32-bit Python
process from a 64-bit Python process (use case: I need to interface with a
64-bit DLL and use an extension (pyFFTW) for which I can only find a 32-bit
compiled version (yes, I could try to install MSVC and compile it myself
but I'm trying to avoid that...))

In fact, this is "easy" to do by using multiprocessing.set_executable
(...while that may not be its original role):

import multiprocessing as mp
import imp, site, sys
if "32" in sys.executable: # checking for my 32-bit Python install
    del sys.path[1:] # recompute sys.path
    print(sys.path)
    site.main()
    print(sys.path) # now points to the 32bit site-packages

import numpy

if __name__ == '__main__':
    mp.set_executable(sys.executable.replace("33", "33-32")) # path of my
32-bit Python install
    mp.Process(target=lambda: None).start()

The sys.path modifications are needed as otherwise the child process
inherits the parent's sys.path and importing numpy (from the 64-bit path)
fails as it is "not a valid Win32 application", complains Python (rightly).

However, even after the sys.path modifications, the numpy import fails with
the error message (that I had never seen before):

<sorry, I can't copy paste from the Windows command prompt...>
    from . import multiarray # <- numpy/core/__init__.py, line 5
SystemError: initialization of multiarray raised an unreported exception

Any hints as to how this could be fixed would be most welcome.

Thanks in advance,

Antony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130917/dc265d17/attachment.html>