Suggested additions to function argument list
All, In trying to implement the compiler class for Borland's C++ compiler I've run up against a minor snag or two. First, some background. For extension modules on Windows, we want to support the specification of an optional module definition file (a.k.a. a .DEF file). On Unix I believe it's true that all of the symbols in a shared library are "exported" so this is a non-issue. On Windows, however, you must specify in one way or another which symbols in a DLL are to be exported. One way to do this is to provide the exported symbols' names in a .DEF file which you must write (it's a specially-formatted ASCII text file). Another way to do this is list each of the exported symbols' names on the linker command line using a linker-specific switch. Currently, a module developer can tell the Distutils which DEF file to use for each extension module with the "def_file" key, e.g.: setup (..., ext_modules = [ ('module1', { 'sources' : ['module1.c'], 'def_file' : ['module1.def'] } ), ('module2', { 'sources' : ['module2.c'], 'def_file' : ['module2.def'] } ), ] ) If the "def_file" information for an extension module is not explicitly specified, there is code in the build_ext command which guesses a likely name for the .DEF file (basically module_name + ".def") and sees if that file exists. If so, it uses that file. Otherwise, we adopt a reasonable fallback position and guess the initialization function's name to be "init" + module_name and ask the linker to export that symbol for us. OK, we have now arrived at the problem part ;) The way to specify either the .DEF file name or the exported symbol name(s) is linker-specific. For Microsoft's linker they are passed along as straightforward arguments, either: /DEF:module1.def or /EXPORTS:initmodule1 but for Borland's linker this information comes in very differently -- not as simple arguments which you can just append to the linker's argument list. Since only the compiler class knows for sure how to specify the .DEF file name (or alternatively the exported symbols list) it seems that we should pass this information directly to the compiler through the link_shared_object() function's argument list, instead of cooking up those arguments in the build_ext command. I'd propose adding an additional argument to the function's signature (its default could be None and could be ignored by non-Windows compilers). Changing the subject, it might also be appropriate to pass down the temporary files directory for the current extension's build, unless this kind of code: build_ext = self.find_peer ('build_ext') build_temp = build_ext.build_temp is considered safe. If this doesn't violate encapsulation (for the compiler object to know that the build_ext object has an attribute named "build_temp") then I'll just do this and sleep soundly ;) Sorry for the rambling e-mail! Lyle
From: "Lyle Johnson"
The way to specify either the .DEF file name or the exported symbol
is linker-specific. For Microsoft's linker they are passed along as straightforward arguments, either:
/DEF:module1.def
or
/EXPORTS:initmodule1
but for Borland's linker this information comes in very differently -- not as simple arguments which you can just append to the linker's argument
name(s) list. How ? Thomas
but for Borland's linker this information comes in very differently -- not as simple arguments which you can just append to the linker's argument list.
How ?
The command-line syntax for Borland's linker (called ILINK32) is: ILINK32 [options] objects, [exe], [mapfile], [libs], [deffile] The name of the module definition file, if specified, must appear in exactly the position shown. It's not considered one of the "options" which can appear in any order at the beginning of the command line. And after a second look at the documentation for ILINK32 it doesn't appear that they have anything analogous to the /EXPORTS switch for naming the exported symbol(s) directly on the command line. Lyle
The command-line syntax for Borland's linker (called ILINK32) is:
ILINK32 [options] objects, [exe], [mapfile], [libs], [deffile]
The name of the module definition file, if specified, must appear in exactly the position shown. It's not considered one of the "options" which can appear in any order at the beginning of the command line. And after a second look at the documentation for ILINK32 it doesn't appear that they have anything analogous to the /EXPORTS switch for naming the exported symbol(s) directly on the command line.
Looks much like the MSVC 1.5 (the 16-bit compiler) linker command line. [Just curious (and unrelated to python extensions): What about resources? How are they built into the executable?] So what to do? - REQUIRE a def file? (Extension writers on unix would like not to have to care about this) - Trust the extension writer to use the (compiler specific?) pragmas? (Same as above) - Dynamically build a def-file and delete it after the build? Maybe the (distutils-generated) temporary def-file would be the best.(?) Greg? Thomas
[Just curious (and unrelated to python extensions): What about resources? How are they built into the executable?]
Sorry, left that off by accident; the resources file is the last argument. The full command line would be: ILINK32 [options] objects, [exe], [mapfile], [libs], [deffile], [resfile]
So what to do?
- REQUIRE a def file? (Extension writers on unix would like not to have to care about this) - Trust the extension writer to use the (compiler specific?) pragmas? (Same as above) - Dynamically build a def-file and delete it after the build?
Maybe the (distutils-generated) temporary def-file would be the best.(?)
I'd prefer not to *require* a .DEF file, since by convention most people will name their module initialization function "init" + modulename. The compiler class can then generate the def-file on the fly (if needed) and use it appropriately. The terminally clever developer who wants the (private) name of his module initialization function to be something else would be required to provide a .DEF file which exports that function as "init" + module, though -- there's obviously no way for distutils to guess that. By the way: again unlike Microsoft, Borland's linker prepends an underscore to exported functions' names, so the automatically-generated .DEF file for BorlandCCompiler will need to look something like this: EXPORTS initmodule=_initmodule --Lyle
I'd prefer not to *require* a .DEF file, since by convention most people will name their module initialization function "init" + modulename. The compiler class can then generate the def-file on the fly (if needed) and
use
it appropriately. The terminally clever developer who wants the (private) name of his module initialization function to be something else would be required to provide a .DEF file which exports that function as "init" + module, though -- there's obviously no way for distutils to guess that.
Is this possible? I think when you 'import extension' python looks for extension.dll or extension.pyd, then tries to locate and call 'void initextension (void)' and will fail if this is not found.
By the way: again unlike Microsoft, Borland's linker prepends an
underscore
to exported functions' names, so the automatically-generated .DEF file for BorlandCCompiler will need to look something like this:
EXPORTS initmodule=_initmodule More reason to generate the def-file.
BTW: If you change the CCompiler (and derived classes) function signatures I'm willing to test this for the MS case. Another reason to update my distutils... Thomas
BTW: If you change the CCompiler (and derived classes) function signatures I'm willing to test this for the MS case. Another reason to update my distutils...
Sounds good to me. I would like to wait until the CVS link to python.org is working again so that I can be sure I'm up-to-date (see Greg's e-mail from Thursday). I also want to hear back from Greg himself and anyone else who might have an opinion on the subject before I make too many changes.
Changing the subject, it might also be appropriate to pass down the temporary files directory for the current extension's build, unless this kind of code:
build_ext = self.find_peer ('build_ext') build_temp = build_ext.build_temp
is considered safe. If this doesn't violate encapsulation (for the compiler object to know that the build_ext object has an attribute named "build_temp") then I'll just do this and sleep soundly ;) As far as I understood distutils design philosophie (sp?) Greg will not like this.
The CCompiler classes should remain usable outside distutils. Thomas
As far as I understood distutils design philosophie (sp?) Greg will not like this.
The CCompiler classes should remain usable outside distutils.
Yes, as I indicated I'm not too comfortable with the CCompiler classes knowing that much about the internals of the "build_ext" command either. The issue of knowing a "safe" place to dump temporary files is again somewhat compiler (and linker) specific. We know that for Microsoft's linker, it's going to automatically create an import library when it creates the DLL. Currently, the decision about where to dump that DLL is made in the build_ext command, thereby making "build_ext" aware of the MSVCCompiler class details. If we could instead just tell the (abstract) compiler where the build_temp directory is, it could do the right thing without build_ext's intervention. Similarly, Borland's linker wants to create a lot of temporary files during the link process. I've found linker options to suppress the creation of most of these files but at least one (the "TDS" file) I don't seem to be able to kill off. But I can tell the linker *where* to put it -- and that's where it would again be useful for my subclass of CCompiler to know where the temporary files should go.
participants (2)
-
Lyle Johnson -
Thomas Heller