Oh yeah! one more feature

Whoops, I totally forgot to mention the other potentially useful feature I added this weekend. Now, entries in the 'libraries' list can include a directory component, which will tell the compiler interface to instruct the C compiler to look only in that directory for that particular library. Umm, was that too opaque? Here's an example. Before this change, the example pil_setup.py included with the Distutils had this: ext_modules = \ [('_imaging', { 'sources': # ... # ... 'library_dirs': ['libImaging', '/usr/local/lib'], 'libraries': ['Imaging', 'jpeg', 'z', 'tcl8.0', 'tk8.0'] } This resulted (on Unix) in a link command like this: gcc -shared ... -LlibImaging -L/usr/local/lib \ -lImaging -ljpeg -lz -ltcl8.0 -ltk8.0 Now, we can specify that the "Imaging" library must come from the "libImaging" subdirectory, rather than being searched for all over the place. The current pil_setup.py has this: ext_modules = \ [('_imaging', { 'sources': # ... # ... 'library_dirs': ['/usr/local/lib'], 'libraries': ['libImaging/Imaging', 'jpeg', 'z', 'tcl8.0', 'tk8.0'] } Now the link command looks like this: gcc -shared ... -L/usr/local/lib \ libImaging/Imaging.a -ljpeg -lz -ltcl8.0 -ltk8.0 Note that this has the unpleasant consequence that the Distutils compiler class (UnixCCompiler, in this case) has to second-guess what the linker does to find a particular library. This isn't too hard for me using GCC, since GCC is fairly well documented in this regard. However I'm sure there are Unices where my rash assumptions (if "dir/foo" found in 'libraries' list, look for "dir/libfoo.so", then "dir/libfoo.a") will not hold. Of course, the directory can be absolute -- this shoots portability to hell, but should be welcome if you're using the Distutils as a private build tool and just want to be sure you're linking in *exactly* such-and-such version of a library. I know at least one person (hi Paul!) who will like this. And the implementation of this feature for MSVC is a wild guess on my part. Again, Windows people are kindly requested to take a look and see how close to reality my guess is. (See the 'gen_lib_options()' function in distutils.ccompiler, and then the 'find_library_file()' method in both distutils.unixccompiler and distutils.msvccompiler.) This code is all in yesterday's snapshot. Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

This can lead to some real problems. When given a shared object, most link editors (ld) will use that exact pathname to find the object at runtime, ignoring the -R (-rpath) and LD_LIBRARY_PATH (LIB_PATH) variables. The system had better determine when and how shared objects (.sl/.so) should be handled differently from archive files (.a). Also, considering that most testing is handled in the build environment (with those same pathnames), this type of problem might not be found until after installation. (Just as a rule, think of how the executable search path rules work. The library search path rules are similar: if there is a '/' in the filename, then do not search with LD_LIBRARY_PATH.) Does anyone know if this might happen with DLLs too? -Arcege -- ------------------------------------------------------------------------ | Michael P. Reilly, Release Engineer | Email: arcege@shore.net | | Salem, Mass. USA 01970 | | ------------------------------------------------------------------------

On 04 October 1999, Michael P. Reilly said:
Hmmm, good point -- I hadn't considered this possibility. One possible "solution" (ahem) is to make the "libraries with a directory component" feature only look for static libraries. If you want to use a shared library, it had better work with -l/-L/-R. This is, I think, compatible with the two uses I see for this feature. The first use is how I demonstrated in pil_setup.py -- just link in a C library built for and with this extension module. In that case it'd be silly to make the C library shared, as we're building a shared object to load at runtime anyway. No point in having a small .so with the Python glue, which then loads a large .so with all the real C code. Just load one .so with both of them. The other use is to appease Paul Dubois and anyone else using the Distutils as a local build tool: Paul needs to be sure that he links with a specific development version of some library. Again, I don't see it as a big loss to say "if you want to link with library foo in /some/devel/dir, then it had better be a static library". Comments? Windows/DLL perspectives? Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

Comments? Windows/DLL perspectives?
DLLs are normally searched for in this order (from memory so I may have left one out...): 1. the directory where the .exe file is located 2. the windows system directory 3. the windows directory 4. the directories on the PATH For DLLs that are dynamically loaded, like Python extension modules, you specify the whole path to the DLL. -- Robin Dunn Software Craftsman robin@AllDunn.com http://AllDunn.com/robin/ http://AllDunn.com/wxPython/ Check it out!

This can lead to some real problems. When given a shared object, most link editors (ld) will use that exact pathname to find the object at runtime, ignoring the -R (-rpath) and LD_LIBRARY_PATH (LIB_PATH) variables. The system had better determine when and how shared objects (.sl/.so) should be handled differently from archive files (.a). Also, considering that most testing is handled in the build environment (with those same pathnames), this type of problem might not be found until after installation. (Just as a rule, think of how the executable search path rules work. The library search path rules are similar: if there is a '/' in the filename, then do not search with LD_LIBRARY_PATH.) Does anyone know if this might happen with DLLs too? -Arcege -- ------------------------------------------------------------------------ | Michael P. Reilly, Release Engineer | Email: arcege@shore.net | | Salem, Mass. USA 01970 | | ------------------------------------------------------------------------

On 04 October 1999, Michael P. Reilly said:
Hmmm, good point -- I hadn't considered this possibility. One possible "solution" (ahem) is to make the "libraries with a directory component" feature only look for static libraries. If you want to use a shared library, it had better work with -l/-L/-R. This is, I think, compatible with the two uses I see for this feature. The first use is how I demonstrated in pil_setup.py -- just link in a C library built for and with this extension module. In that case it'd be silly to make the C library shared, as we're building a shared object to load at runtime anyway. No point in having a small .so with the Python glue, which then loads a large .so with all the real C code. Just load one .so with both of them. The other use is to appease Paul Dubois and anyone else using the Distutils as a local build tool: Paul needs to be sure that he links with a specific development version of some library. Again, I don't see it as a big loss to say "if you want to link with library foo in /some/devel/dir, then it had better be a static library". Comments? Windows/DLL perspectives? Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

Comments? Windows/DLL perspectives?
DLLs are normally searched for in this order (from memory so I may have left one out...): 1. the directory where the .exe file is located 2. the windows system directory 3. the windows directory 4. the directories on the PATH For DLLs that are dynamically loaded, like Python extension modules, you specify the whole path to the DLL. -- Robin Dunn Software Craftsman robin@AllDunn.com http://AllDunn.com/robin/ http://AllDunn.com/wxPython/ Check it out!
participants (3)
-
Greg Ward
-
Michael P. Reilly
-
Robin Dunn