ANN: MiniFFTW 1.0.4, a FFTW wrapping C-extension

Hi! I hereby want to announce a small C-Extension we created for an internal research project. It might be useful to scientists who use 1- dimensional fourier transforms frequently. It wrapps the "Fastest Fourier Transformation in the West", or at least a very small subset of it (currently only the complex 1D transform, both serial and with MPI, and wisdom handling). You can find the source code, licensed under GPLv3, here: https://git.sr.ht/~imperator/mini-mpi-fftw I have a sourcehut mailing list for discussions, bug reports etc. ~imperator/mini-mpi-fftw-discuss@lists.sr.ht https://lists.sr.ht/~imperator/mini-mpi-fftw-discuss ...and might in the future announce new versions on this mailing list: https://lists.sr.ht/~imperator/mini-mpi-fftw-announce Minifftw's main design goal is to make the FFTW on Linux clusters with MPI very easy to use for people without much experience on MPI. You can build two versions of the module * one without MPI, wrapping the serial FFTW for usage i.e. on desktop computers * one with MPI, wrapping the MPI-FFTW for distributed computing. Both versions behave exactly the same from the perspective of the Python program. Therefore, the programmer can run the same simulation program without changing the code on his laptop (within hardware resource limits, of course) or on a cluster. Another advantage is that it directly writes to and reads from the memory of passed numpy arrays, without copying memory when entering or leaving the API. MFFTW aims at mostly offering the FFTW API to the Python world, so you have the same functions (plan, execute) available and don't really need to relearn anything when you already know FFTW. MFFTW manually uses MPI to distribute and collect parts of a whole contiguous array behind the scenes. That's necessary because FFTW only supports pure distributed memory, where each process allocates and fills its memory slice locally. Therefore, MFFTW is intended for comparatively small transforms (currently maximum size 2^31-1 complex numbers), which you want to speed up using distributed computing. Our tests showed that, with the right configuration, you can gain significant performance, even with arrays much smaller to what distributed memory FFTW seems to be intended for. The disadvantage of this easy-to-use approach is that the control over MPI is taken away from the user, therefore I don't think it will be possible to use MFFTW and other MPI-requiring functions in the same simulation. Also, currently, the wrapper is very small according the implemented functionality. We maybe will implement more transforms and would accept patches implementing those, but our own simulations only need the DFT- 1D. The README should answer most questions about the internals, I hope. Next steps might include packaging for Debian and Arch-Linux. I am unsure, though, how much sense this makes, because you'd either have to provide two packages (w/ & ^w/ MPI), or needed a runtime-flag to use serial FFTW in a package build for MPI. Also, I expect the wrapper to be most useful on clusters, where you will likely have a custom build process without standard package managers. I hope it will be helpful to some of you. Greetings, P. <P><A HREF="https://git.sr.ht/~imperator/mini-mpi-fftw ">MiniFFTW 1.0.4</A> -short C-Extension for easy FFTW-MPI usage (11-April-21)
participants (1)
-
Philipp Stanner