Mailman 3 June 2000 - Distutils-SIG

Behaviour of "install_data" changed
by Greg Ward 26 Jun '00

26 Jun '00

I've just fixed the behaviour of the "install_data" command: the default directory for installating data files is now just the installation base, which is usually sys.prefix. Thus, if you put data_files = ["my_data"] in your setup script, you will wind up installing (eg.) /usr/local/my_data -- definitely the wrong thing on Unix, and probably wrong on Windows too. (Except for applications that have their own prefix, but that's not really dealt with yet.) This is the Right Thing now because of the change to "install_data" that lets you specify where to put data files; the above should be spelled data_files = [("share", ["my_data"])] which will install /usr/local/share/my_data. Of course, this still doesn't solve the problem of, "How does my application/module know where Distutils installed this data file if the user did some wild funky custom installation?". Oh well, better than nothing. Greg -- Greg Ward - Unix nerd gward(a)python.net http://starship.python.net/~gward/ Vote anarchist.

3 3

building extensions with AIX
by Rene Liebscher 26 Jun '00

26 Jun '00

Hi, I have seen in the TODO file that there was an entry about AIX. I tried it and found some little problems. It seems some paths in the Makefile are wrong. After playing a little bit around with it, I wrote a new compiler class. (I think, these changes could be included in unixccompiler, but to test it, it is nice to have a own file for it.) Maybe we should also change the default compiler table. I think we should have entries for sys.platform in it, too. So we could try a more specific name first ('aix4') and if there is no entry the usual os.name ('posix'.) The AIXCCompiler class is attached, maybe someone wants to try it, and find out if this is really all we have to change against the standard unixccompiler. (Don't forget to make an entry for it in ccompiler.py.) kind regards Rene Liebscher PS: I used AIX 4.3 and its 'cc' compiler. """distutils.aixccompiler Contains the AIXCCompiler class, a subclass of UnixCCompiler that handles some path problems with AIX. The makefile isn't correct for distutils' standard compiler. """ # created 2000/06/23, Rene Liebscher __revision__ = "$Id: aixccompiler.py $" from distutils import sysconfig from distutils.unixccompiler import UnixCCompiler # XXX Things not currently handled: # * see UnixCCompiler class AIXCCompiler (UnixCCompiler): compiler_type = 'aix' def __init__ (self, verbose=0, dry_run=0, force=0): UnixCCompiler.__init__ (self, verbose, dry_run, force) # we have to correct some paths self.ld_shared=sysconfig.get_python_lib(standard_lib=1)+'/config/ld_so_aix' self.ldflags_shared.append('-bI:'+sysconfig.get_python_lib(standard_lib=1)+'/config/python.exp') # __init__ () # class AIXCCompiler

2 3

Updated TODO list
by Greg Ward 24 Jun '00

24 Jun '00

Hey, this to-do list is looking pretty good: lots of "done" items as of today. Highest priority: * bdist_wininst * auto-configuration; currently has enough to support mxDateTime, probably needs some more to support PIL, and then I need to find some other distributions with auto-configuration needs to play with Medium priority: * Cygwin GCC support, Borland C++ support * AIX, OSF/1 support Here's the list... GENERAL ------- * think about how to support distribution-specific "configure" commands -- eg. how can they push option values (include/library directories, that sort of thing) onto the "build_ext" command? [see also "AUTO-CONFIGURATION" section below, and the "config" command] * need a good, general-purpose, portable-if-you-want-it, unportable-if- you-need-control, way for the builder/installer to specify compiler and linker flags (lots of people expect CFLAGS and LDFLAGS to work, so they probably should; but options to either command and/or build are the right way, so those should be preferred) * install_data should install to $prefix by default, not $prefix/share (and should warn if developer doesn't supply any dir component: installing right to $prefix is almost always the wrong thing to do) [done 2000/06/24 GPW] DOCS ---- * write write write * include a standard blurb for README files somewhere PATCH REVIEW/INTEGRATION ------------------------ * review Rene Liebscher's patch for help options: he changed archive_util.py ccompiler.py command/bdist.py command/build.py command/build_clib.py command/build_ext.py command/sdist.py dist.py ...and I checked 'em all in without review because I want other people to try it out and I don't have time to review it right now (and he fixed a stupid bug in bdist.py for me) [done 2000/06/23 GPW] * review Rene Liebscher's cygwinccompiler.py contribution [started and aborted, 2000/06/24 GPW; asked Rene to resubmit patch with stylistic changes] COMMAND-LINE PARSING -------------------- * I think fancy_getopt needs to get fancier to properly support the -I, -D, -l, -L, etc. options of "build_ext": need to be able to accumulate multiple options in a list, should be able to split a string on a given delimiter, probably want to specify if repetitions of the same option will accumulate or replace, etc. * do the above options even work at all? seem to recall hearing reports of dismal failure, but I never looked into it [knowing how FancyGetopt works, there's no way these options can work: damn] COMPILER MODEL & CLASSES ------------------------ * add pre-processor interface to CCompiler (needed to support Autoconf-style 'try_cpp', 'search_cpp', 'search_headers' in config commands) [done, but only UnixCCompiler implements that interface] * fix UnixCCompiler so it doesn't depend on sysconfig (ie. cc must be passed in) [done 2000/06/24 GPW] * allow user to supply cc (compiler executable) in addition to compiler type [done 2000/06/24 GPW] * radically simplify CCompiler method signatures: drop most of the per-invocation customization, and just use the instance attributes for eg. include_dirs, libraries, ... [NB. should probably hold off integrating Cygwin and Borland support until this is done, *if* that is it is the right thing to do...] [update 2000/06/24: I'm cooling to this idea; it turns out the extra complex interface that's been there all along is useful for the "config" command, which has to do lots of little temporary compile and link steps] * Cygwin/Mingw32 support [Rene Liebscher is working on this] * Borland C++ support [Lyle Johnson is working on this] AUTO-CONFIGURATION ------------------ * add more goodies to the standard "config" command, and finish hacking up the example mxDateTime setup script to take advantage of them [partly done: at least enough is there to auto-configure mxDateTime; need to work on PIL's auto-configuration next] EXTENSION BUILDING ------------------ * extension building on AIX [update 2000/06/24: Rene Liebscher has a patch for this, which I have asked him to refine] * support for SWIG -- should just have to specify the .i file and have Distutils take care of running SWIG and knowing what the output is (to be really clever: make sure SWIG's output is included in source distributions, so builders-from-source don't need to have SWIG installed) [update 2000/06/24: Thomas Heller and I have gone back and forth on this a couple times: sounds like Thomas has the right idea, I'll let him work on it] * support for PyFort (lower priority than SWIG!) * OSF/1 problem: ld command has "*" in it, which is appropriate when a shell is involved, but there isn't one here, so we need to strip the quotes (and, ideally, put them back on again when spawn() prints out the command run!) INSTALLATION ------------ * if Distutils installs the first third-party modules in an installation (and creates site-packages), then "install" needlessly warns about having installed to a location not on sys.path -- presumably because site-packages isn't in sys.path at startup, since it doesn't exist until we create it! (I think this is only a problem with 1.5.2, since site-packages is now created when Python is installed -- check!) * need a mechanism for specifying pre-install and post-install hooks, which will be run when installing from a smart built distribution (RPM, wininst, etc.); also, "install" needs to run these hooks *except* for "fake" installs done solely to create a built distribution * bdist_dumb should grow a little intelligence: let packager choose whether to make archive relative to prefix or the root (prefix is essential for proper Windows support ) DISTRIBUTION FORMATS -------------------- * figure out why bdist_rpm doesn't work with RPM 2.x, and make it work if possible [punt: I don't care anymore, if anyone else does, let them fix it] * make "bdist" take multiple formats (both for convenience and consistency with "sdist"); should be doable now that we can reinitialize, refinalize, and (presumably) rerun commands [done 2000/06/06 GPW] * Solaris pkgtool, HP depot [Mark Alexander is working on these] * minimal Windows installer ("bdist_wininst") [Thomas Heller is working on this] * Wise installer for Windows ("bdist_wise") [Thomas Heller said he will work on this when "bdist_wininst" is done] $Id: TODO,v 1.5 2000/06/25 02:37:09 gward Exp $ -- Greg Ward - Linux geek gward(a)python.net http://starship.python.net/~gward/ No problem is so formidable that you can't just walk away from it.

1 0

bdist_rpm and RPM 2.x
by Greg Ward 24 Jun '00

24 Jun '00

The "bdist_rpm" doesn't work with RPM 2.x. This means that building RPMs will only work on Red Hat 6.x -- and presumably any version of Mandrake (since Mandrake started as a fork of Red Hat 6.0), and recent SuSE versions as well, but I don't know how recent. Does anyone care? I don't anymore, since I finally got around to upgrading to Red Hat 6. If compatibility with RPM 2.x matters to you, then please dig in to the bdist_rpm command, figure out why it doesn't work, and submit a patch. Otherwise Distutils will only be able to build RPMs with RPM 3. Greg -- Greg Ward - Linux nerd gward(a)python.net http://starship.python.net/~gward/ In space, no one can hear you fart.

1 0

RC and MC patch for msvccompiler
by Thomas Heller 23 Jun '00

23 Jun '00

This is the patch for msvccompiler factored out of my previous swig patch, which allows compiling resource scripts (.rc) to .res files, and messages (.mc) to .res files. The resulting .res files are passed to the linker. Additionally .cxx is now recognized as C++ source extension. Thomas

2 1

distutil install
by Michel Sanner 23 Jun '00

23 Jun '00

Hello, I got the OpenDX Python wrapper from Randall Hopper and asked him why the .so wrapping the DX library ends up not int he package directory but in $exec_prefix/lib/python/site-packages. His answer was that this is where distutil puts it. I feel that it would nicer to have the .so along with all other files of the package in $exec_prefix/lib/python/site-packages/DX. Is there a way to specify this in setup.py ? -Michel -- ----------------------------------------------------------------------- >>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!! Michel F. Sanner Ph.D. The Scripps Research Institute Assistant Professor Department of Molecular Biology 10550 North Torrey Pines Road Tel. (858) 784-2341 La Jolla, CA 92037 Fax. (858) 784-2860 sanner(a)scripps.edu http://www.scripps.edu/sanner -----------------------------------------------------------------------

2 1

ccompiler problem
by Thomas Heller 23 Jun '00

23 Jun '00

Consider the object_filenames method: def object_filenames (self, source_filenames, strip_dir=0, output_dir=''): if output_dir is None: output_dir = '' obj_names = [] for src_name in source_filenames: (base, ext) = os.path.splitext (src_name) if ext not in self.src_extensions: continue ^^^^^^^^^ Problem is here if strip_dir: base = os.path.basename (base) obj_names.append (os.path.join (output_dir, base + self.obj_extension)) return obj_names # object_filenames () If you specify a source-file with an unknown extension in the sources list, you later get a traceback later saying something like 'sources and targets must have the same length'. IMO we have two possibilities here: - assume that 'ext' is something which should be passed to the linker - raise an error like 'Don't know how to compile this file' Thomas Heller ION-TOF GmbH

2 1

Re: [Distutils] Duelling SWIG patches
by Michel Sanner 23 Jun '00

23 Jun '00

I have been following the SWIG support discussion for a while. I am not sure I see the point in having distutil run SWIG. Why not simply provide the result or running swig as part of the source code that get's compiled. There are a number of versions of SWIG that have different types of arguments, in addition the interface might use user defined typemaps ! It seems to me that this would be very difficult to get to work properly and I cannot see any real benefit. -Michel On Jun 23, 10:42am, Thomas Heller wrote: > Subject: Re: [Distutils] Duelling SWIG patches > > Well, I was inspired by Thomas' patch for SWIG support to come up with > > my own patch. Here it is: completely untested, so I haven't checked it > > in yet. (I don't have SWIG installed at home, so I figured I'd let > > someone Out There who actually uses it test this for me. Remember: > > laziness is a virtue!) > > > > Note that Thomas' patch was Windows/C++-centric; mine tries to be > > OS-agnostic, but is C-centric. Is it possible to tell from a SWIG .i > > file whether it is destined to be C or C++? I can't think of a good way > > to do it, but then my knowledge if SWIG is roughly epsilon. (Out of > > curiosity, I took the manual home and read it about three years ago. It > > struck me as being *way* to easy, and I wanted to write Perl extensions > > like a *real* man. Ahh, the folly of youth...) > > > I don't like this patch. > > While I won't argue whether ccompiler is an appropriate superclass > for swig, swig shares some important properties: > > Swig has command line flags to specify include directories -I<dir>, > libraries -l<ifile>, define symbols -Dsymbol like C-compilers. > Other options which are maybe usefull for python extensions > are -t <typemap file>, -shadow (create shadow classes), > -module name (set module name) and maybe more. > > >From this I have the impression that swig should be a separate class > implemented in swig.py, and there should be distutils command line > flags for setting the above options (and also the -c++ option). > Although I did not implement them so far. > > (Sidenote: I also needed a swig-object in build_ext for my setup-script > for win32all. I had to extend distutils in the setup-script > because there are also exe-files to be built from C++ > and swig sources). > > As I pointed out in a separate post, there should also be an option > to specify C or C++ processing. > > If I find time, I will submit a new patch. > > Thomas > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG(a)python.org > http://www.python.org/mailman/listinfo/distutils-sig >-- End of excerpt from Thomas Heller -- ----------------------------------------------------------------------- >>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!! Michel F. Sanner Ph.D. The Scripps Research Institute Assistant Professor Department of Molecular Biology 10550 North Torrey Pines Road Tel. (858) 784-2341 La Jolla, CA 92037 Fax. (858) 784-2860 sanner(a)scripps.edu http://www.scripps.edu/sanner -----------------------------------------------------------------------

2 1

Fw: [Distutils] Duelling SWIG patches
by Thomas Heller 23 Jun '00

23 Jun '00

Thomas Heller ION-TOF GmbH ----- Original Message ----- From: "Thomas Heller" <thomas.heller(a)ion-tof.com> To: <distutils-sig(a)python.org> Sent: Friday, June 23, 2000 9:08 AM Subject: Re: [Distutils] Duelling SWIG patches > > > > Greg Ward wrote: > > [snip] > > > OS-agnostic, but is C-centric. Is it possible to tell from a SWIG .i > > > file whether it is destined to be C or C++? I can't think of a good way > > [snip] > > and Michael wrote: > > > > I'm pretty sure that the only way to do this short of parsing the file is > to > > run "swig" with no options and then (if it fails) run "swig -c++" > (enabling > > the C++ processing features). This seems more like something you'd want > to > > do during configuration then during the build. > Unfortunately swig does not fail in C-mode on an I-file > which is destined for C++: The C-compiler will fail with syntax > errors. > > Thomas > > >

1 0

Re: [Distutils] Duelling SWIG patches
by Michael Muller 22 Jun '00

22 Jun '00

Greg Ward wrote: [snip] > OS-agnostic, but is C-centric. Is it possible to tell from a SWIG .i > file whether it is destined to be C or C++? I can't think of a good way [snip] I'm pretty sure that the only way to do this short of parsing the file is to run "swig" with no options and then (if it fails) run "swig -c++" (enabling the C++ processing features). This seems more like something you'd want to do during configuration then during the build. Maybe it would be best to adopt a convention for SWIG files in distutils (uppercase ".I" in C++, maybe?) or find some other way to allow the user to explicitly indicate a C++ file. ============================================================================= michaelMuller = mmuller(a)enduden.com | http://www.cloud9.net/~proteus ----------------------------------------------------------------------------- We are explorers in the further reaches of experience: demons to some, angels to others. - "Pinhead" from "Hellraiser" =============================================================================

1 0