[Numpy-discussion] multiprocessing, numpy and 32-64 bit cohabitation

[Robert Kern]

On Wed, Sep 18, 2013 at 1:51 AM, Antony Lee <antony.lee at berkeley.edu> wrote:
>
> Hi all,
>
> While I realize that this is certainly tweaking multiprocessing beyond
its specifications, I would like to use it on Windows to start a 32-bit
Python process from a 64-bit Python process (use case: I need to interface
with a 64-bit DLL and use an extension (pyFFTW) for which I can only find a
32-bit compiled version (yes, I could try to install MSVC and compile it
myself but I'm trying to avoid that...))

Just use subprocess to start up the 32-bit Python. If you want to use the
multiprocessing tools for communicating data, use a Manager server in the
32-bit Python to communicate over a socket.

  http://docs.python.org/2/library/multiprocessing#managers
  http://docs.python.org/2/library/multiprocessing#using-a-remote-manager

It is possible that this won't work if the protocol assumes that the
bitness is the same between server and client (e.g. struct.pack('Q', ...)),
but I suspect this is not the case.

You may also consider writing a small server using pyzmq or similar. I am
guessing that you are just calling one function from pyFFTW and getting the
result back. A simple REQ/REP server is easy to write with pyzmq. Do you
need to use pyFFTW for some specific functionality that is not available in
numpy.fft or scipy.fftpack?

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130918/442afbad/attachment.html>