Hi, I see that in the new scipy (0.7) only fftpack (NETLIB) remains for fft. I understand the reasons for this, but for my application (filtering gigabytes of neurophysiological data) fft speed differences really lead to very significant speed differences. What would the best way of using fftw, now that the wrappers are gone. Cython? Weave? I haven't used either of these yet, but would look into them if someone could confirm that that would be a good idea in order to solve my problem. I vaguely remember having seen an example somewhere, but I can't find it anymore. Best, Gabriel
Gabriel Beckers wrote:
Hi,
I see that in the new scipy (0.7) only fftpack (NETLIB) remains for fft. I understand the reasons for this, but for my application (filtering gigabytes of neurophysiological data) fft speed differences really lead to very significant speed differences.
What would the best way of using fftw, now that the wrappers are gone. Cython? Weave? I haven't used either of these yet, but would look into them if someone could confirm that that would be a good idea in order to solve my problem. I vaguely remember having seen an example somewhere, but I can't find it anymore.
Best, Gabriel
I have a boost::python wrapper
On Thu, Dec 11, 2008 at 10:55 PM, Gabriel Beckers <beckers@orn.mpg.de> wrote:
Hi,
I see that in the new scipy (0.7) only fftpack (NETLIB) remains for fft. I understand the reasons for this, but for my application (filtering gigabytes of neurophysiological data) fft speed differences really lead to very significant speed differences.
What it the dimensions of your fourier transforms ? The use of fftw in scipy was far from optimal for various reasons, so I am a bit surprised about the very significant part.
What would the best way of using fftw, now that the wrappers are gone.
Reusing the removed code from scipy into your own package would be one solution. Ideally, I should have put this code into a scikit, but this would have required some work to be in an acceptable shape. If you are willing to spend some time on it, I would certainly welcome a good wrapper around fftw put into a scikit.
Cython?
Cython would be a bit difficult because it lacks complex support (more exactly, it cannot translate yet into C99 complex numbers; it should be possible to circumvent the problems). If you only need real to real transforms, I think cython would be a quick way to do it. That's certainly how I would do it if I had to. If you really care about speed, you should tweak you allocation scheme to force aligned allocation: it will give you a factor 2 speed increase in many cases - it is difficult to do in a cross platform manner, but trivial in a particular case/platform (since fftw has such an allocator). cheers, David
Hello All, I've just noticed Python 3.0 final is now available. Curious if there are any expectations or thoughts about SciPy and Python 3.0? My best, Jo ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Joseph Anderson Lecturer in Music School of Arts and New Media University of Hull, Scarborough Campus, Scarborough, North Yorkshire, YO11 3AZ, UK T: +44.(0)1723.357341 T: +44.(0)1723.357370 F: +44.(0)1723.350815 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ***************************************************************************************** To view the terms under which this email is distributed, please go to http://www.hull.ac.uk/legal/email_disclaimer.html *****************************************************************************************
NumPy is going to have to move over to Python 3.0 before the whole of SciPy can, and as far as I know that's not going to happen until early 2009. I'd hold off on Python 3.x for the time being. Josh On Thu, Dec 11, 2008 at 7:52 AM, Joseph Anderson <j.anderson@hull.ac.uk> wrote:
Hello All,
I've just noticed Python 3.0 final is now available. Curious if there are any expectations or thoughts about SciPy and Python 3.0?
My best, Jo
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Joseph Anderson Lecturer in Music
School of Arts and New Media University of Hull, Scarborough Campus, Scarborough, North Yorkshire, YO11 3AZ, UK
T: +44.(0)1723.357341 T: +44.(0)1723.357370 F: +44.(0)1723.350815 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
***************************************************************************************** To view the terms under which this email is distributed, please go to http://www.hull.ac.uk/legal/email_disclaimer.html ***************************************************************************************** _______________________________________________ SciPy-user mailing list SciPy-user@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user
On Thu, Dec 11, 2008 at 9:12 AM, Joshua Lippai <discerptor@gmail.com> wrote:
NumPy is going to have to move over to Python 3.0 before the whole of SciPy can, and as far as I know that's not going to happen until early 2009. I'd hold off on Python 3.x for the time being.
It definitely will not happen in early 2009. Early 2010 would be a better estimate. We are aiming to fully support Python 2.6 in early 2009. -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/
Thanks for the update. I'd already made the mistake of rolling to Python 2.6, finding I was just a bit too ambitious, then rolling back to 2.5. My best, ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Joseph Anderson Lecturer in Music School of Arts and New Media University of Hull, Scarborough Campus, Scarborough, North Yorkshire, YO11 3AZ, UK T: +44.(0)1723.357341 T: +44.(0)1723.357370 F: +44.(0)1723.350815 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ On 12/12/2008 5:24 am, "Jarrod Millman" <millman@berkeley.edu> wrote:
On Thu, Dec 11, 2008 at 9:12 AM, Joshua Lippai <discerptor@gmail.com> wrote:
NumPy is going to have to move over to Python 3.0 before the whole of SciPy can, and as far as I know that's not going to happen until early 2009. I'd hold off on Python 3.x for the time being.
It definitely will not happen in early 2009. Early 2010 would be a better estimate. We are aiming to fully support Python 2.6 in early 2009.
***************************************************************************************** To view the terms under which this email is distributed, please go to http://www.hull.ac.uk/legal/email_disclaimer.html *****************************************************************************************
thanks David, On Thu, 2008-12-11 at 23:23 +0900, David Cournapeau wrote:
What it the dimensions of your fourier transforms ? The use of fftw in scipy was far from optimal for various reasons, so I am a bit surprised about the very significant part.
The dimension varies, but I must immediately admit that I didn't make any comparisons between the different fft libraries from within scipy. I just used fftw a lot before, in plain C programs, and then it was very fast and gave significant speed increases. I was assuming that it would do the same if I could use it from within scipy. But apparently that may not be the case. Perhaps I should try the boost::python wrapper that Neal mentions, since it already exists, and see if I get an improvement that is worth the trouble or not. Cheers, Gabriel
We use Cython to call the parallel version of FFTW and its worked fine.
Cython would be a bit difficult because it lacks complex support (more exactly, it cannot translate yet into C99 complex numbers; it should be possible to circumvent the problems). If you only need real to real transforms, I think cython would be a quick way to do it. That's certainly how I would do it if I had to.
This was not an issue for us. We just created our complex arrays as numpy arrays and then passed the memory pointers into FFTW and it went fine. But, we weren't building the actual arrays in cython so we didn't run into the the lack of C99 complex numbers. Here is our parallel FFTW wrapper for Cython: http://bazaar.launchpad.net/~ipython-dev/ipython/ipythondistarray/files/35?f... Just a warning. This code has bugs in it and doesn't yet have a test suite, so I don't recommend copying this verbatim. But, it at least shows that it can be done. Depending on what exactly you want to do, you could get Cython+FFTW working in a *very* short amount or time. Cheers, Brian
participants (7)
-
Brian Granger -
David Cournapeau -
Gabriel Beckers -
Jarrod Millman -
Joseph Anderson -
Joshua Lippai -
Neal Becker