[Numpy-discussion] multiprocessing, numpy and 32-64 bit cohabitation

[Antony Lee]

Thanks, I didn't know that multiprocessing Managers could be used with
processes not started by multiprocessing itself...  I will give them a try.
I just need to compute FFTs, but speed is a real issue for me (I am using
the results for real-time feedback).  To be honest I don't know yet if the
FFTs are going to be the limiting step but I thought I may as well give
pyFFTW a try and ran into that issue...
Antony


2013/9/18 Robert Kern <robert.kern at gmail.com>

> On Wed, Sep 18, 2013 at 1:51 AM, Antony Lee <antony.lee at berkeley.edu>
> wrote:
> >
> > Hi all,
> >
> > While I realize that this is certainly tweaking multiprocessing beyond
> its specifications, I would like to use it on Windows to start a 32-bit
> Python process from a 64-bit Python process (use case: I need to interface
> with a 64-bit DLL and use an extension (pyFFTW) for which I can only find a
> 32-bit compiled version (yes, I could try to install MSVC and compile it
> myself but I'm trying to avoid that...))
>
> Just use subprocess to start up the 32-bit Python. If you want to use the
> multiprocessing tools for communicating data, use a Manager server in the
> 32-bit Python to communicate over a socket.
>
>   http://docs.python.org/2/library/multiprocessing#managers
>   http://docs.python.org/2/library/multiprocessing#using-a-remote-manager
>
> It is possible that this won't work if the protocol assumes that the
> bitness is the same between server and client (e.g. struct.pack('Q', ...)),
> but I suspect this is not the case.
>
> You may also consider writing a small server using pyzmq or similar. I am
> guessing that you are just calling one function from pyFFTW and getting the
> result back. A simple REQ/REP server is easy to write with pyzmq. Do you
> need to use pyFFTW for some specific functionality that is not available in
> numpy.fft or scipy.fftpack?
>
> --
> Robert Kern
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130918/4414bff0/attachment.html>