
Hi all -- this is a followup to Lyle and Thomas' exchange earlier in the week; sorry for the delay. 1. First topic: list of symbols to export. [Lyle]
Since only the compiler class knows for sure how to specify the .DEF file name (or alternatively the exported symbols list) it seems that we should pass this information directly to the compiler through the link_shared_object() function's argument list, instead of cooking up those arguments in the build_ext command. I'd propose adding an additional argument to the function's signature (its default could be None and could be ignored by non-Windows compilers).
Yep, I've been thinking this was needed for a while. Here's a proposed new signature for the 'link_shared_lib()' method: def link_shared_lib (self, objects, output_libname, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, exported_symbols=None, debug=0, extra_preargs=None, extra_postargs=None): 'exported_symbols' would be a list of strings; it's up to the compiler class to translate that into command-line options, a file (in the appropriate format for that compiler) listing the names, or whatever. For MSVCCompiler, we would add "/EXPORT:" options to the command line, one for each string in 'exported_symbols'. For BorlandCCompiler (or whatever Lyle's calling it -- I suppose I should have veto power on the name, but I still can't make up my mind), the class would have to generate a .def file (in Borland style with underscores added/removed as appropriate) and pass that in just the right place on the compiler command-line. 'link_shared_object()', of course, would be adjusted similarly (since 'link_shared_lib()' is just a wrapper around 'link_shared_object()' on both Unix and Windows.) 2. Variation on first topic: allowing sneaky symbol aliasing [Lyle]
The terminally clever developer who wants the (private) name of his module initialization function to be something else would be required to provide a .DEF file which exports that function as "init" + module, though -- there's obviously no way for distutils to guess that.
[Thomas echoes my sentiments]
Is this possible? I think when you 'import extension' python looks for extension.dll or extension.pyd, then tries to locate and call 'void initextension (void)' and will fail if this is not found.
More importantly, is this *desirable* or *useful*? This doesn't sound like a useful thing to support; in fact, it sounds like a downright *bad* idea to let developers name their module initializer anything other than "init" + extension_name. (Assuming that it even works -- if the symbols exported from a DLL really can be aliased in the way that Lyle's "initmodule = _initmodule" example implies, then that demolishes Thomas' argument... but it doesn't make this kind of trickery a good idea!) 3. Second topic: temporary files [Lyle]
Changing the subject, it might also be appropriate to pass down the temporary files directory for the current extension's build, unless this kind of code:
build_ext = self.find_peer ('build_ext') build_temp = build_ext.build_temp
is considered safe. If this doesn't violate encapsulation (for the compiler object to know that the build_ext object has an attribute named "build_temp") then I'll just do this and sleep soundly ;)
If you're talking about putting this code in one (or more) of the CCompiler classes, then Thomas was right: I don't like this. Furthermore, it won't work -- 'find_peer()' is a method of Command, and CCompiler is independent of that class hierarchy. I think the right thing to do is add a 'temp_dir' attribute to CCompiler. (I see no reason to allow overriding this in every method.) Currently this would not gain much, just the ability to move a bit of the MSVC-specific code out of build_ext.py (specifically, the stuff that generates the /IMPLIB: option). But if it's necessary to support Borland's compiler cleanly, I'm willing to do it. Here are proposed semantics for 'temp_dir': * files that are obviously compiler/linker turds (like the stuff Lyle is talking about, and the MSVC turds that Thomas took care of several months ago by generating that /IMPLIB: option) would unconditionally go in 'temp_dir' * intermediate products, specifically object files, would go under 'temp_dir' unless you pass 'output_dir' to 'compile()' * 'temp_dir' would really be the root of a temporary tree, to avoid possible name collisions. Eg. if you compile "src/foo/a.c" and "src/bar/a.c", you'd get "<temp_dir>/src/foo/a.o" and "<temp_dir>/src/bar/a.o" * in the distutils context (ie. the "build_ext" command), temp_dir would be build/temp.<plat> -- ie. build_ext would do something like self.compiler.temp_dir = self.build_temp and that's the only use it would have for 'self.build_temp' * the compiler classes would be responsible for making sure any output directories (temp_dir or otherwise) exist before writing to them * the compiler classes are *not* responsible for cleaning up the temp dir -- that way, we can reuse .o files across builds. In the Distutils context, the user is responsible for cleaning via the "clean" command Sound reasonable? Greg -- Greg Ward - Unix geek gward@python.net http://starship.python.net/~gward/ "Passionate hatred can give meaning and purpose to an empty life." -- Eric Hoffer

Yep, I've been thinking this was needed for a while. Here's a proposed new signature for the 'link_shared_lib()' method:
def link_shared_lib (self, objects, output_libname, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, exported_symbols=None, debug=0, extra_preargs=None, extra_postargs=None):
This works for me. I'm also not attached to the "BorlandCCompiler" name, I promise!
2. Variation on first topic: allowing sneaky symbol aliasing
More importantly, is this *desirable* or *useful*? This doesn't sound like a useful thing to support; in fact, it sounds like a downright *bad* idea to let developers name their module initializer anything other than "init" + extension_name. (Assuming that it even works -- if the symbols exported from a DLL really can be aliased in the way that Lyle's "initmodule = _initmodule" example implies, then that demolishes Thomas' argument... but it doesn't make this kind of trickery a good idea!)
You can definitely alias exported symbol names; look up the MSDN documentation for the /EXPORT switch if you are still a doubter! But I meant for my use of the phrase "terminally clever" to indicate that developers who do this for Python extensions should be, um, terminated ;) So I agree with both you and Thomas that we shouldn't give them this hook.
3. Second topic: temporary files
I think the right thing to do is add a 'temp_dir' attribute to CCompiler. (I see no reason to allow overriding this in every method.) Currently this would not gain much, just the ability to move a bit of the MSVC-specific code out of build_ext.py (specifically, the stuff that generates the /IMPLIB: option). But if it's necessary to support Borland's compiler cleanly, I'm willing to do it.
Here are proposed semantics for 'temp_dir': * files that are obviously compiler/linker turds (like the stuff Lyle is talking about, and the MSVC turds that Thomas took care of several months ago by generating that /IMPLIB: option) would unconditionally go in 'temp_dir' * intermediate products, specifically object files, would go under 'temp_dir' unless you pass 'output_dir' to 'compile()' * 'temp_dir' would really be the root of a temporary tree, to avoid possible name collisions. Eg. if you compile "src/foo/a.c" and "src/bar/a.c", you'd get "<temp_dir>/src/foo/a.o" and "<temp_dir>/src/bar/a.o" * in the distutils context (ie. the "build_ext" command), temp_dir would be build/temp.<plat> -- ie. build_ext would do something like self.compiler.temp_dir = self.build_temp and that's the only use it would have for 'self.build_temp' * the compiler classes would be responsible for making sure any output directories (temp_dir or otherwise) exist before writing to them * the compiler classes are *not* responsible for cleaning up the temp dir -- that way, we can reuse .o files across builds. In the Distutils context, the user is responsible for cleaning via the "clean" command
Sound reasonable?
Yes, this sounds great! Anxiously awaiting the next CVS update...

[me, proposing a new 'temp_dir' attribute for CCompiler and friends]
I think the right thing to do is add a 'temp_dir' attribute to CCompiler. (I see no reason to allow overriding this in every method.) Currently this would not gain much, just the ability to move a bit of the MSVC-specific code out of build_ext.py (specifically, the stuff that generates the /IMPLIB: option). But if it's necessary to support Borland's compiler cleanly, I'm willing to do it. [...]
[Lyle enthusiastically agrees]
Yes, this sounds great! Anxiously awaiting the next CVS update...
OK, but it'll have to wait -- I'm currently in the throes of making config files a reality at last. Lots of upheaval involved, surprise surprise. Also, I started implementing my proposed 'temp_file' semantics only to discover a fly in the ointment. Now I can't even remember if it was just a harmless little housefly or a great bug-eyed blood-sucking poisonous monster. I'll get back to it when the config file stuff is done... in the meantime, keep on "cvs up"'ing ... Greg -- Greg Ward - "always the quiet one" gward@python.net http://starship.python.net/~gward/ MTV -- get off the air! -- Dead Kennedys
participants (2)
-
Greg Ward
-
Lyle Johnson