Mailman 3 May 2000 - Distutils-SIG

Re: [Distutils] handling complex link/library specifications
by andrew.macintyre＠aba.gov.au 25 May '00

25 May '00

[repost as my original was misdirected - apologies to -admin. am] Thanks for your response. the -R option can certainly be handled that way, but the toggling of the static/dynamic library loading I don't think can be. Mind you, I'm no guru on the subject. ------------------------------------------------------------- Andrew MacIntyre \ andrew.macintyre(a)aba.gov.au Planning & Licensing Branch \ Tel: +61 2 6256 2812 Australian Broadcasting Authority \ Fax: +61 2 6253 3277 ---------- Original Text ---------- From: Harry Henry Gebel <hgebel(a)inet.net>, on 24/05/00 16:35: On Wed, May 24, 2000 at 02:28:36PM +0000, Andrew MacIntyre wrote: > Unfortunately, the commercial package I'm dealing with has a long list of > libraries required to be linked, with several specified static and the rest > dynamic. > > The recommended library spec (for Solaris, which is what I'm working with at > the moment) looks something like this: > > -l<lib> ... -l<lib> -Bstatic -l<lib>... -l<lib> -Bdynamic -l<lib>...-l<lib> > -R<path> > > What would be the recommended way to deal with this, if Distutils in its > current form can handle this situation? The CVS version (I think you have to get it from SourceForge) should except the CFLAGS environment variable, I don't know if this will help you or not. -- Harry Henry Gebel, Senior Developer, Landon House SBS West Dover Hundred, Delaware _______________________________________________ Distutils-SIG maillist - Distutils-SIG(a)python.org http://www.python.org/mailman/listinfo/distutils-sig

1 0

handling complex link/library specifications
by andrew.macintyre＠aba.gov.au 25 May '00

25 May '00

In the process of creating a Python module for a commercial package, I have built a Distutils package for it (using Distutils-0.8.2). Unfortunately, the commercial package I'm dealing with has a long list of libraries required to be linked, with several specified static and the rest dynamic. The recommended library spec (for Solaris, which is what I'm working with at the moment) looks something like this: -l<lib> ... -l<lib> -Bstatic -l<lib>... -l<lib> -Bdynamic -l<lib>...-l<lib> -R<path> My setup.py (built from the distutils template) has all the libraries listed in a 'libraries' record against the module, and things appear to link OK as is though I'm still chasing down a couple of things which might turn out to be related to the static/dynamic stuff above. I tried looking in the documentation, but the bits that might answer my question haven't been written yet :-( What would be the recommended way to deal with this, if Distutils in its current form can handle this situation? (BTW http://www.python.org/sigs/distutils-sig/doc/dist/source-dist.html has an HTML problem, at least with IE 3.02) ------------------------------------------------------------- Andrew MacIntyre \ andrew.macintyre(a)aba.gov.au Planning & Licensing Branch \ Tel: +61 2 6256 2812 Australian Broadcasting Authority \ Fax: +61 2 6253 3277

3 3

the build_scripts patch
by Bastian Kleineidam 25 May '00

25 May '00

I submitted a patch to Greg against the latest Snapshot release. It is too big to include it here, but a list of changes follows: - new command build_scripts: copy all scripts in build/scripts. If the first line of a script begins with "#!" and ends with "python", write the path to the current Python interpreter in this line. - bug fixed in dist.py: import sysconfig for win32 platforms. - new function Command.create_tree which calls util.create_tree. - install_misc is now the base class for all install commands - install_data now creates leading directories of the files you supply and copies the files in these directories. So for example pixmaps/open.xpm gets installed in <self.install_data>/pixmaps/open.xpm and not in <self.install_data>/open.xpm I think the last point is a convenient way to install data in different directories. Another way (like in automake) is to supply for each set of filenames a different directory in which the files are going to be installed, but I like my solution better. Bastian Kleineidam

2 5

Config files working (I hope)
by Greg Ward 23 May '00

23 May '00

Hi all -- I seem to have got the Distutils config file mechanism working. Please check out the latest code from the CVS archive (SourceForge!) and see how many bugs I left in. The basic idea is this: you can create 1 .. 3 config files, depending on your platform. On Unix: <prefix>/lib/python1.x/site-packages/distutils/pydistutils.cfg [1] $HOME/.pydistutils.cfg setup.cfg [1] this is really just the directory of the distutils package -- so it could be the distutils development directory if you haven't installed the distutils, or it could be lib/python1.6/distutils -- you get the idea. I'm not entirely thrilled with putting a config file in the library directory, but I was even less thrilled with any of the alternatives. ;-( On Windows and Mac OS: <prefix>/pydistutils.cfg setup.cfg Config files are read in the order listed above, ie. setup.cfg in the current directory takes precedence over ~/.pydistutils.cfg, which take precedence over the system pydistutils.cfg. Config files are in turn overridden by the command line. Syntax is as follows: [command] option=value ...as implemented by the standard ConfigParser module. Thus, [build] build_base=blib in setup.cfg is equivalent to python setup.py build --build-base=blib except, of course, that the config file is entirely passive: no commands are run based on what's found in the config file! Error-checking is done partly by the Distribution class and partly by the individual command classes. In particular, Distribution.get_command_obj() is responsible for ensuring that config files don't contain invalid options, ie. instance attributes not set in the command's 'initialize_options()' method. The command class itself is responsible for all other error-checking -- whether the content of an option is valid, whether certain options conflict, etc. This is the same mechanism as has worked for command-line options for ages now. There is a problem, though, which is that commands don't know *where* the options they are error-checking originate. The Distribution class keeps track of that information, in the 'command_options' dictionary, but it's currently not shared with the command classes. There are a couple of ways to do this, and I'm mulling the alternatives before diving in. Until that's fixed, you may get error messages that don't tell you if the offending option value was in a config file, on the command line, or what. Annoying but not fatal. Well, that's enough for tonight. Happy hacking! Greg -- Greg Ward - Linux geek gward(a)python.net http://starship.python.net/~gward/ The real Nazis run your schools / They're coaches, businessmen and cops -- Dead Kennedys

1 0

CVS access
by Thomas Heller 23 May '00

23 May '00

I have successfully accessed the python source on sourceforge, including the lib/distutils subtree. How would I access the distutils (complete) tree? Thomas Heller

4 3

Supporting Borland's compiler
by Greg Ward 23 May '00

23 May '00

Hi all -- this is a followup to Lyle and Thomas' exchange earlier in the week; sorry for the delay. 1. First topic: list of symbols to export. [Lyle] > Since only the compiler class knows for sure how to specify the .DEF file > name (or alternatively the exported symbols list) it seems that we should > pass this information directly to the compiler through the > link_shared_object() function's argument list, instead of cooking up those > arguments in the build_ext command. I'd propose adding an additional > argument to the function's signature (its default could be None and could be > ignored by non-Windows compilers). Yep, I've been thinking this was needed for a while. Here's a proposed new signature for the 'link_shared_lib()' method: def link_shared_lib (self, objects, output_libname, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, exported_symbols=None, debug=0, extra_preargs=None, extra_postargs=None): 'exported_symbols' would be a list of strings; it's up to the compiler class to translate that into command-line options, a file (in the appropriate format for that compiler) listing the names, or whatever. For MSVCCompiler, we would add "/EXPORT:" options to the command line, one for each string in 'exported_symbols'. For BorlandCCompiler (or whatever Lyle's calling it -- I suppose I should have veto power on the name, but I still can't make up my mind), the class would have to generate a .def file (in Borland style with underscores added/removed as appropriate) and pass that in just the right place on the compiler command-line. 'link_shared_object()', of course, would be adjusted similarly (since 'link_shared_lib()' is just a wrapper around 'link_shared_object()' on both Unix and Windows.) 2. Variation on first topic: allowing sneaky symbol aliasing [Lyle] > The terminally clever developer who wants the (private) > name of his module initialization function to be something else would be > required to provide a .DEF file which exports that function as "init" + > module, though -- there's obviously no way for distutils to guess that. [Thomas echoes my sentiments] > Is this possible? I think when you 'import extension' python looks > for extension.dll or extension.pyd, then tries to locate and call > 'void initextension (void)' and will fail if this is not found. More importantly, is this *desirable* or *useful*? This doesn't sound like a useful thing to support; in fact, it sounds like a downright *bad* idea to let developers name their module initializer anything other than "init" + extension_name. (Assuming that it even works -- if the symbols exported from a DLL really can be aliased in the way that Lyle's "initmodule = _initmodule" example implies, then that demolishes Thomas' argument... but it doesn't make this kind of trickery a good idea!) 3. Second topic: temporary files [Lyle] > Changing the subject, it might also be appropriate to pass down the > temporary files directory for the current extension's build, unless this > kind of code: > > build_ext = self.find_peer ('build_ext') > build_temp = build_ext.build_temp > > is considered safe. If this doesn't violate encapsulation (for the compiler > object to know that the build_ext object has an attribute named > "build_temp") then I'll just do this and sleep soundly ;) If you're talking about putting this code in one (or more) of the CCompiler classes, then Thomas was right: I don't like this. Furthermore, it won't work -- 'find_peer()' is a method of Command, and CCompiler is independent of that class hierarchy. I think the right thing to do is add a 'temp_dir' attribute to CCompiler. (I see no reason to allow overriding this in every method.) Currently this would not gain much, just the ability to move a bit of the MSVC-specific code out of build_ext.py (specifically, the stuff that generates the /IMPLIB: option). But if it's necessary to support Borland's compiler cleanly, I'm willing to do it. Here are proposed semantics for 'temp_dir': * files that are obviously compiler/linker turds (like the stuff Lyle is talking about, and the MSVC turds that Thomas took care of several months ago by generating that /IMPLIB: option) would unconditionally go in 'temp_dir' * intermediate products, specifically object files, would go under 'temp_dir' unless you pass 'output_dir' to 'compile()' * 'temp_dir' would really be the root of a temporary tree, to avoid possible name collisions. Eg. if you compile "src/foo/a.c" and "src/bar/a.c", you'd get "<temp_dir>/src/foo/a.o" and "<temp_dir>/src/bar/a.o" * in the distutils context (ie. the "build_ext" command), temp_dir would be build/temp.<plat> -- ie. build_ext would do something like self.compiler.temp_dir = self.build_temp and that's the only use it would have for 'self.build_temp' * the compiler classes would be responsible for making sure any output directories (temp_dir or otherwise) exist before writing to them * the compiler classes are *not* responsible for cleaning up the temp dir -- that way, we can reuse .o files across builds. In the Distutils context, the user is responsible for cleaning via the "clean" command Sound reasonable? Greg -- Greg Ward - Unix geek gward(a)python.net http://starship.python.net/~gward/ "Passionate hatred can give meaning and purpose to an empty life." -- Eric Hoffer

2 2

distutils & Cygwin compiler on Win32
by Rene Liebscher (Rene.Liebscher＠htw-dresden.de) 22 May '00

22 May '00

Hi, did anyone ever try to use the Cygwin compiler on Win32 with distutils? If not, then I'm the first. You can find a compiler class for this purpose at http://www.informatik.htw-dresden.de/~htw7192/distutils-cygwin/ there is also some additional information about some problem it still has. It is not completly tested, so if anyone want to test, use or debug it, she is free to do so. (I used it to build some dll's, the complete rest was not tested.) (PS: It works probably with mingw32, too. I used cygwin in no-cygwin mode, which is same as mingw32, I think.) Kind regards Rene Liebscher

3 2

Optimized bytecode patch
by Harry Henry Gebel 20 May '00

20 May '00

I have to go out, when I get back I will send explanations -- Harry Henry Gebel, Senior Developer, Landon House SBS West Dover Hundred, Delaware PyNcurses ncurses binding for Python http://pyncurses.sourceforge.net

2 4

util.copy_file and --dry-run
by Bastian Kleineidam 20 May '00

20 May '00

I found a misleading behaviour with the --dry-run option. Suppose you create a directory and then you copy a file in this directory. With --dry-run the directory is not created and this makes the copy_file function to announce the renaming of the file rather than copying it into the new directory. code snippet: ./setup.py --dry-run build [...] creating build/scripts copying linkchecker -> build should be: creating build/scripts copying linkchecker -> build/scripts I do not have a fix for this. I think this is a feature, not a bug. Bastian -- Bastian Kleineidam -- Just do it. Use Linux. .~. http://fsinfo.cs.uni-sb.de/~calvin/official/ /V\ // \\ /( )\ ^`~'^

2 1

do-nothing fails
by Paul F. Dubois 20 May '00

20 May '00

Allow me to confess that the following low-priority bug actually bit me: You can't have packages be an empty list. The build directory does not get created and the installer complains it can't copy the tree. Why might I be running distutils to install "nothing"? The CXX package consists of header files and some C and C++ source files; no python files or extension modules at the moment. I propose to put the source files (needed by a builder of a CXX extension) into PYTHON/etc/CXX. The include files get treated like Numeric's. I can obviously work around this but it is a generally nice idea that end cases work.

2 1

Distutils-SIG May 2000