[Numpy-discussion] multiprocessing, numpy and 32-64 bit cohabitation
antony.lee at berkeley.edu
Wed Sep 18 21:40:44 EDT 2013
Thanks, I didn't know that multiprocessing Managers could be used with
processes not started by multiprocessing itself... I will give them a try.
I just need to compute FFTs, but speed is a real issue for me (I am using
the results for real-time feedback). To be honest I don't know yet if the
FFTs are going to be the limiting step but I thought I may as well give
pyFFTW a try and ran into that issue...
2013/9/18 Robert Kern <robert.kern at gmail.com>
> On Wed, Sep 18, 2013 at 1:51 AM, Antony Lee <antony.lee at berkeley.edu>
> > Hi all,
> > While I realize that this is certainly tweaking multiprocessing beyond
> its specifications, I would like to use it on Windows to start a 32-bit
> Python process from a 64-bit Python process (use case: I need to interface
> with a 64-bit DLL and use an extension (pyFFTW) for which I can only find a
> 32-bit compiled version (yes, I could try to install MSVC and compile it
> myself but I'm trying to avoid that...))
> Just use subprocess to start up the 32-bit Python. If you want to use the
> multiprocessing tools for communicating data, use a Manager server in the
> 32-bit Python to communicate over a socket.
> It is possible that this won't work if the protocol assumes that the
> bitness is the same between server and client (e.g. struct.pack('Q', ...)),
> but I suspect this is not the case.
> You may also consider writing a small server using pyzmq or similar. I am
> guessing that you are just calling one function from pyFFTW and getting the
> result back. A simple REQ/REP server is easy to write with pyzmq. Do you
> need to use pyFFTW for some specific functionality that is not available in
> numpy.fft or scipy.fftpack?
> Robert Kern
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion