Robert Kern wrote:

Can you be more specific?

Sure: in my branch to refactor fftpack, every non default backend (that is everything but fftpack) is a separate python module, which implements some fft functions, and is 'importable'. So in scipy.fftpack, I have a function which: - tries to find one of the backend from a list (by using __import__) - if no backend is found, just use the default backend - if one is found, pull the functions from the found backend, and use the default backend as a fallback for non available functions (each backend can implement a different subset of fftpack). This is done in the following prototype: http://projects.scipy.org/scipy/scipy/browser/branches/refactor_fft/scipy/ff... I would like to add the possibility to control which backend is used: the problem is how to communicate the name of the backend to scipy.fftpack in a persistent way (such as if you want to use mkl, it will remember it at the next python session), and in a dynamic way (such as you can decide which one to use before importing scipy.fftpack, or even changing the backend on the fly). thanks, David

On Mon, May 26, 2008 at 11:01 PM, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:

Lisandro Dalcin wrote:

...

David, I've implemented something like this for petsc4py.

Basically, I want to be able to build extension modules accessing PETSc for different configuration options (for example, optimized versus debug), when using PETSc, it is normal to select the build variant by a $PETSC_ARCH environ var.

Yes, that's another solution I forgot to mention.

...

If you think this fits your needs, then I'll help you. Buth then I'll need to know a bit more about the directory tree structure of your package and your extension modules.

Thanks, but I already have the code to dynamically load a module from a string. My problem is how to control it, and how we want to do this kind of things. Up to know, scipy have never depended on env variables or configuration files, so I don't want to introduce this kind of things without discussing it first.

There are actually a couple of places, like scipy.misc.imshow(). I tend to dislike libraries automatically reading configuration (including environment variables). However, imshow() is not really a library function so much as a function for the interactive interpreter, so I don't mind so much. I would definitely *not* use environment variables as the sole control of FFT backend selection. If I have to restart my IPython session just because I forgot to set the right environment variable, I will be very unhappy. For FFTs, I would probably keep the backend functions in a module-level list. On import, the package would try to import the backends it knows could be bundled in scipy.fftpack and insert them into the list. Probably, these would just be the optimized versions; the default FFTPACK versions would be kept separate. If an import fails, it would be a good idea to format the traceback into a string and store it in an accessible location for debugging unintentional ImportErrors. Each API function (e.g. rfft()) would check its list for an optimized implementation and use it or else fall back to the default. Each implementation module (e.g. scipy.fftpack._fftw3, or whatever you have named them; I haven't reviewed your code, yet) would have an explicit registration function that puts its implementations (all or a subset, e.g. _fftw3.register() or _fftw3.register('fft', 'rfft')) at the top of the lists. One needs to think about how to handle power-of-two only implementations. This is probably the only real special case we have to handle, so we can probably do it explicitly instead of coming up with an overly generic solution. I wouldn't bother trying to persist this information. If anyone wants to persist their preferences, then can write a utility function that does exactly what they need. That's the five-minute overview of what I would try. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

David Cournapeau wrote:

Thanks, but I already have the code to dynamically load a module from a string. My problem is how to control it, and how we want to do this kind of things. Up to know, scipy have never depended on env variables or configuration files, so I don't want to introduce this kind of things without discussing it first.

Both matplotlib and ipython are using rc files already. For your fftpack work, it would be great to be able to compare different backends within the same session, both for benchmarking and accuracy checking. In that case the user would probably want to control what is loaded at runtime. Thanks! Jon