On 5/1/08, David Cournapeau <cournapeau@cslab.kecl.ntt.co.jp> wrote:
On Wed, 2008-04-30 at 16:44 -0300, Lisandro Dalcin wrote:
David, in order to put clear what I was proposing to you in previous mail regarding to implementing plugin systems for numpy, please take a look at the attached tarball.
Thanks for looking at this Lisandro.
The problem I see with the approach of struct vs direct function pointers is of ABI compatibility: it is easy to mess things up with structures.
That's a very valid point.
There is the advantage of using only one dlsym (or equivalent) with the struct, which may be much faster than using hundreds of dlsym for each function. Linux does not seem to have problems with that, but mac os X for example felt slow when I tried doing this for several thousand functions. I did not go really far on that, though. I don't really see why using a struct is cleaner, though. That's really the same thing (function pointers), in both cases the name namespace pollution will be the same, and in both cases there will be a need to generate source code.
Perhaps cleaner is not the right word. I actually believe that is far more portable, regarding the oddities of dlopening in different platforms.
Concerning the loading mechanism, I don't understand the point of using PyCObject_Import. By quickly looking at the code of the function, the plugin needs to be a python module in that case, which is not needed in our case; I don't like the fact that it is not documented either.
Well, that is the same mechanism NumPy uses for exporting its C API to extensions modules. Look at generated header '__multiarray_api.h' in function '_import_array()' in your numpy installation. This is the standard, portable, and (as Python doc says) recommended way to expose C API's from extensions modules.
The code for dlopen/dlclose is really short. For each platform, it is like 50 lines of code, and we have a better control on what we can do (which may be needed; for example, you want want to load symbols from a dll A, but the symbols are only in dll B, which dll A depends on; that's a problem that does not happen for python extensions, I think; I don't really know to be honest).
Please note that all my concerns about recommending you not to use dlopen, is just to save you from future headaches!!!. See yourself at row 17 of table here <http://www.fortran-2000.com/ArnaudRecipes/sharedlib.html> . And that table does not cover stuff like RTLD_GLOBAL flags to dlopen (or equivalent). -- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594