Compile extension modules with Visual Studio 2005
Hi, I am about to write a C extension module. C functions in the module will take and return numpy arrays. I found a tutorial online, but I am not sure about the following: 1. Can I compile my extension with Visual Studio 2005? My impression is that I will have to link with numpy libraries, and, if numpy was compiled with a different compiler, I might have problems. However, if numpy is a DLL, maybe there is a way that I can build a LIB file based on the DLL and link with the LIB file. Does anyone have experience in doing this? 2. I am new to writing python extensions. The tutorial is doing things by hand. Does anyone know what is the best way to do this? How about SWIG? Thanks, Geoffrey
Geoffrey Zhu wrote:
Hi,
I am about to write a C extension module. C functions in the module will take and return numpy arrays. I found a tutorial online, but I am not sure about the following:
1. Can I compile my extension with Visual Studio 2005? My impression is that I will have to link with numpy libraries, and, if numpy was compiled with a different compiler, I might have problems. However, if numpy is a DLL, maybe there is a way that I can build a LIB file based on the DLL and link with the LIB file. Does anyone have experience in doing this?
numpy isn't the issue. The main Windows Python distribution requires Visual Studio 2003 for building extensions. One can do it with mingw, though, with care.
2. I am new to writing python extensions. The tutorial is doing things by hand. Does anyone know what is the best way to do this? How about SWIG?
There isn't a single best way. They all have tradeoffs. It might be easiest for you to actually just write your C functions into a DLL without referencing numpy or Python at all and call those functions using ctypes. That avoids needing a specific compiler and is a pretty handy tool to learn. If you do want to write an extension, I think I might suggest starting with writing one by hand. It helps with the other techniques to know what's going on underneath. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Hi Robert, On 7/24/07, Robert Kern <robert.kern@gmail.com> wrote:
Geoffrey Zhu wrote:
Hi,
I am about to write a C extension module. C functions in the module will take and return numpy arrays. I found a tutorial online, but I am not sure about the following:
1. Can I compile my extension with Visual Studio 2005? My impression is that I will have to link with numpy libraries, and, if numpy was compiled with a different compiler, I might have problems. However, if numpy is a DLL, maybe there is a way that I can build a LIB file based on the DLL and link with the LIB file. Does anyone have experience in doing this?
numpy isn't the issue. The main Windows Python distribution requires Visual Studio 2003 for building extensions. One can do it with mingw, though, with care.
2. I am new to writing python extensions. The tutorial is doing things by hand. Does anyone know what is the best way to do this? How about SWIG?
There isn't a single best way. They all have tradeoffs. It might be easiest for you to actually just write your C functions into a DLL without referencing numpy or Python at all and call those functions using ctypes. That avoids needing a specific compiler and is a pretty handy tool to learn.
If you do want to write an extension, I think I might suggest starting with writing one by hand. It helps with the other techniques to know what's going on underneath.
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Thanks for your help. Do you know what exactly is the issue of having to use VS2003 to build extensions? If the interactions are done at DLL level, shouldn't call compilers that can generate DLLs work? It doesn't look like using ctypes would be an option, as my goal is to 'vectorize' some operations. Thanks a lot, Geoffrey
Geoffrey Zhu wrote:
Thanks for your help. Do you know what exactly is the issue of having to use VS2003 to build extensions? If the interactions are done at DLL level, shouldn't call compilers that can generate DLLs work?
Mostly it's an issue of the C runtime that is used for either compiler. C extensions need to use the same runtime as Python itself. Mostly.
It doesn't look like using ctypes would be an option, as my goal is to 'vectorize' some operations.
You mean you need to use PyMultiIter objects for broadcasting? Yeah, that would require an extension. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Hi Robert, On 7/24/07, Robert Kern <robert.kern@gmail.com> wrote:
Geoffrey Zhu wrote:
Thanks for your help. Do you know what exactly is the issue of having to use VS2003 to build extensions? If the interactions are done at DLL level, shouldn't call compilers that can generate DLLs work?
Mostly it's an issue of the C runtime that is used for either compiler. C extensions need to use the same runtime as Python itself. Mostly.
If it is a problem of the runtime library conflict, I can probably statically link the VS2005 runtime into my extension DLL and there would be no conflict. Do you see any problems with this plan? :-)
It doesn't look like using ctypes would be an option, as my goal is to 'vectorize' some operations.
You mean you need to use PyMultiIter objects for broadcasting? Yeah, that would require an extension.
I haven't looked at PyMultilter objects. I am just trying to build a 'vector version' of my C function so that it can do batch calculations. For example, for a vector X, I can do for x in X: y=my_func(x) Or I can do Y=my_vector_func(X). The latter is probably much more efficient. That's why I need the extension. Thanks, Geoffrey
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Geoffrey Zhu wrote:
Hi Robert,
On 7/24/07, Robert Kern <robert.kern@gmail.com> wrote:
Geoffrey Zhu wrote:
Thanks for your help. Do you know what exactly is the issue of having to use VS2003 to build extensions? If the interactions are done at DLL level, shouldn't call compilers that can generate DLLs work? Mostly it's an issue of the C runtime that is used for either compiler. C extensions need to use the same runtime as Python itself. Mostly.
If it is a problem of the runtime library conflict, I can probably statically link the VS2005 runtime into my extension DLL and there would be no conflict. Do you see any problems with this plan? :-)
If it works, then no, no problem. distutils may balk at building your extension, though.
It doesn't look like using ctypes would be an option, as my goal is to 'vectorize' some operations. You mean you need to use PyMultiIter objects for broadcasting? Yeah, that would require an extension.
I haven't looked at PyMultilter objects. I am just trying to build a 'vector version' of my C function so that it can do batch calculations. For example, for a vector X, I can do
for x in X: y=my_func(x)
Or I can do Y=my_vector_func(X).
The latter is probably much more efficient. That's why I need the extension.
ctypes would be an option, then. You would just do the loop in C. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Tue, Jul 24, 2007 at 02:32:10PM -0500, Robert Kern wrote:
I haven't looked at PyMultilter objects. I am just trying to build a 'vector version' of my C function so that it can do batch calculations. For example, for a vector X, I can do
for x in X: y=my_func(x)
Or I can do Y=my_vector_func(X).
The latter is probably much more efficient. That's why I need the extension.
ctypes would be an option, then. You would just do the loop in C.
I agree with Robert that ctypes is probably the simplest option. Have a look at http://scipy.org/Cookbook/Ctypes HTH, Gaël
participants (3)
-
Gael Varoquaux -
Geoffrey Zhu -
Robert Kern