From seberino at spawar.navy.mil Mon Nov 3 15:30:23 2003 From: seberino at spawar.navy.mil (seberino@spawar.navy.mil) Date: Mon Nov 3 15:30:41 2003 Subject: [Distutils] Problems installing data directories if has SUBDIRECTORIES Message-ID: <20031103203023.GA10767@spawar.navy.mil> My setup.py can create the tarball fine but when I try to install then it chokes when it gets to a subdirectory. It gives error about subdirectory not being a "regular file". It obviously expects the directory to be a file and gets confused when it is not. How fix? Chris -- _______________________________________ Christian Seberino, Ph.D. SPAWAR Systems Center San Diego Code 2872 49258 Mills Street, Room 158 San Diego, CA 92152-5385 U.S.A. Phone: (619) 553-9973 Fax : (619) 553-6521 Email: seberino@spawar.navy.mil _______________________________________ From seberino at spawar.navy.mil Wed Nov 5 17:05:44 2003 From: seberino at spawar.navy.mil (seberino@spawar.navy.mil) Date: Wed Nov 5 17:05:50 2003 Subject: [Distutils] How add "-L path" switch when building C extension? Message-ID: <20031105220544.GA17763@spawar.navy.mil> How add "-L path" switch when building C extension? I want compilation to use a DIFFERENT version of a library in a DIFFERENT directory than usual. Thanks. Chris -- _______________________________________ Christian Seberino, Ph.D. SPAWAR Systems Center San Diego Code 2872 49258 Mills Street, Room 158 San Diego, CA 92152-5385 U.S.A. Phone: (619) 553-9973 Fax : (619) 553-6521 Email: seberino@spawar.navy.mil _______________________________________ From theller at python.net Thu Nov 6 14:53:57 2003 From: theller at python.net (Thomas Heller) Date: Thu Nov 6 14:54:16 2003 Subject: [Distutils] Proposed change to ccompiler classes Message-ID: While hacking on a setup script for the win32all extensions, I encountered a problem with the distutils' ccompiler class(es). The compile() method determines which files have to be compiled, and constructs a dictionary 'build'. This contains as keys the object filename, the corresponding values are 2-tuples containing the source filename and the extension of the source filename. Later, the files are compiled using this loop: for obj, (src, ext) in build.items(): .... So, the files are compiled in a random order. For Windows, however, there may be the requirement that a certain build order is required, because the compilation of some files creates other files, which may be included by the following compilation steps. To give an example, there are '.mc' files which are first compiled by the so called messagecompiler. A '.h' file is also created, which is included by other source files. The essence is that a certain compile order is required. I can see two solutions: - Enhance the compiler classes with a method which will bring the files into the correct order, then make sure that they are actually compiled in this order. - Change the code so that the files are compiled in the order of the sequence passed to the compiler, and let the setup script writer take care that the order is correct. The latter is the easiest option, and this change would enable this - although it must be done in each compiler class: #for obj, (src, ext) in build.items(): for obj in objects: try: src, ext = build[obj] except KeyError: continue .... Any thoughts? Thomas From seefeld at sympatico.ca Thu Nov 6 15:57:54 2003 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Thu Nov 6 15:59:37 2003 Subject: [Distutils] Proposed change to ccompiler classes References: Message-ID: <6d32192921ca28f82f2a12a545d9bdab3faab367@Orthosoft.ca> Thomas Heller wrote: > I can see two solutions: > - Enhance the compiler classes with a method which will bring the files > into the correct order, then make sure that they are actually compiled > in this order. > > - Change the code so that the files are compiled in the order of the > sequence passed to the compiler, and let the setup script writer take > care that the order is correct. > > The latter is the easiest option, and this change would enable this - > although it must be done in each compiler class: > > #for obj, (src, ext) in build.items(): > for obj in objects: > try: > src, ext = build[obj] > except KeyError: > continue > .... > > Any thoughts? Yes, the second solution looks like the right thing to do. May I add another RFE here that somewhat relates to your proposal: Right now an 'Extension' is basically a flat list of files, to which some parameters are associated, specifying the build procedure. I'v had to work with code (all belonging to the same 'extension module'), where some files needed other compiler options than the rest. Would it be possible to allow 'Extensions' to be composed not only of files, but of 'sub extensions' ? Each such 'sub extension' would contain its own parameters (possibly inheriting default values from their parent), and the final link stage would just combine all the thusly generated .o files into one module. That seems completely doable, and even fully backward compatible. Of course, there, too, the order of execution may be important. Comments ? Regards, Stefan From Jack.Jansen at cwi.nl Fri Nov 7 04:29:43 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Fri Nov 7 04:29:39 2003 Subject: [Distutils] Proposed change to ccompiler classes In-Reply-To: References: Message-ID: On 6 Nov 2003, at 20:53, Thomas Heller wrote: > I can see two solutions: > - Enhance the compiler classes with a method which will bring the files > into the correct order, then make sure that they are actually compiled > in this order. > > - Change the code so that the files are compiled in the order of the > sequence passed to the compiler, and let the setup script writer take > care that the order is correct. I would prefer a way to specify the dependency, similar to "make". But distutils design is already tather top-heavy as it is, so practicality may dictate your second solution... -- Jack Jansen http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From theller at python.net Fri Nov 7 11:14:19 2003 From: theller at python.net (Thomas Heller) Date: Fri Nov 7 11:14:41 2003 Subject: [Distutils] Proposed change to ccompiler classes In-Reply-To: (Jack Jansen's message of "Fri, 7 Nov 2003 10:29:43 +0100") References: Message-ID: <1xsk5f6s.fsf@python.net> Jack Jansen writes: > On 6 Nov 2003, at 20:53, Thomas Heller wrote: > >> I can see two solutions: >> - Enhance the compiler classes with a method which will bring the files >> into the correct order, then make sure that they are actually compiled >> in this order. >> >> - Change the code so that the files are compiled in the order of the >> sequence passed to the compiler, and let the setup script writer take >> care that the order is correct. > > I would prefer a way to specify the dependency, similar to "make". > But distutils design is already tather top-heavy as it is, so > practicality may dictate your second solution... Well, I would *like* to view the second option as a bug fix, especially because it doesn't solve the original problem. You specify a sequence of files, and they should not be compiled in random order. Andrew, what do you think? Oh, I wanted to mention this: It's possible to work around a *lot* of problems or limitiations in distutils with command subclasses in the setup script. But it's a pain to change the behaviour of the compiler instance. Thomas From amk at amk.ca Sat Nov 8 17:00:55 2003 From: amk at amk.ca (A.M. Kuchling) Date: Sat Nov 8 17:01:18 2003 Subject: [Distutils] Proposed change to ccompiler classes In-Reply-To: References: Message-ID: <20031108220055.GA19008@rogue.amk.ca> On Thu, Nov 06, 2003 at 08:53:57PM +0100, Thomas Heller wrote: > The latter is the easiest option, and this change would enable this - > although it must be done in each compiler class: Good suggestion. Feel free to check it in immediately. --amk From theller at python.net Wed Nov 19 13:49:56 2003 From: theller at python.net (Thomas Heller) Date: Wed Nov 19 13:50:10 2003 Subject: [Distutils] Prepatory snapshot before Distutils 1.1 In-Reply-To: <20031025020337.GA16992@rogue.amk.ca> (amk@amk.ca's message of "Fri, 24 Oct 2003 22:03:37 -0400") References: <20031025020337.GA16992@rogue.amk.ca> Message-ID: amk@amk.ca writes: > I want to make a Distutils 1.1 release that wraps up the version of the code > in Python 2.3. This would be the last 1.5.2-compatible release, giving > people the chance to use the code with versions of Python from 1.5.2 to 2.2. > > The README and related bits have been updated, and I've made a snapshot > release that can be downloaded from > http://www.python.org/sigs/distutils-sig/files/ > > If someone wants to try it with an older version of Python, please do and > report any problems. If no bugs turn up, I'll wrap it up as the 1.1 final > release, tag the CVS tree, and then we can start making 1.5.2-incompatible > changes. I haven't tried it (which is a shame) but isn't there the problem that wininst.exe will not be installed in the distutils/command directory by setup.py install? Having said that, IMO there should be a distutils command to do this... Has someone lying around a command to do this? Thomas From theller at python.net Wed Nov 19 15:06:41 2003 From: theller at python.net (Thomas Heller) Date: Wed Nov 19 15:06:56 2003 Subject: [Distutils] Installing (data-)files in package directories Message-ID: It seems installing data-files in package directories is a common requirement - at least for me. Readme files, data files needed by the packages - the bdist_wininst.exe file is such an example. It needs to be in the same directory as the distutils.command.bdist_wininst module. So here is a build_py subclass (contained in a setup script) which does this. Any comments? Thomas -------------- next part -------------- import sys if len(sys.argv) == 1: sys.argv.append("install") ################ from distutils.core import setup ################ from distutils.command import build_py import glob, os class my_build_py(build_py.build_py): """This build_py replacement not only builds Python packages and modules, it also builds (copies) all other files in the packages into the build tree.""" # This does the right thing with the 'install' command as well as # the bdist_wininst command - eventually install all these files # into the final destination directories. # A possible improvement would be to be be able to specify which # files are installed, but, as I see it, this would require # changes to distutils' distribution class as well - how could the # filenames or wildcards be specified otherwise? def run(self): build_py.build_py.run(self) for package in self.packages: package_dir = self.get_package_dir(package) files = self.find_package_files(package, package_dir) for (dest_base, package_, file_path) in files: assert package == package_ self.build_file(dest_base, package, file_path) def find_package_files(self, package, package_dir): # return a list of triples, the first item is the basename # of the destination file, the second is the package name, # and the third item is the source filename result = [] for fname in glob.glob(os.path.join(package_dir, "*.*")): base, ext = os.path.splitext(fname) if ext in (".py", ".pyc", ".pyo"): continue result.append((os.path.basename(fname), package, fname)) return result def build_file(self, dest_base, package, file_path): from types import StringType, ListType, TupleType if type(package) is StringType: package = package.split('.') elif type(package) not in (ListType, TupleType): raise TypeError, \ "'package' must be a string (dot-separated), list, or tuple" parts = [self.build_lib] + list(package) + [dest_base] outfile = os.path.join(*parts) dir = os.path.dirname(outfile) self.mkpath(dir) return self.copy_file(file_path, outfile, preserve_mode=0) ################ setup(name="test", packages = ["my_package"], cmdclass = {"build_py": my_build_py}, ) From bob at redivi.com Wed Nov 19 15:26:19 2003 From: bob at redivi.com (Bob Ippolito) Date: Wed Nov 19 15:26:19 2003 Subject: [Distutils] Installing (data-)files in package directories In-Reply-To: References: Message-ID: On Nov 19, 2003, at 3:06 PM, Thomas Heller wrote: > It seems installing data-files in package directories is a common > requirement - at least for me. Me too > Readme files, data files needed by the packages - the bdist_wininst.exe > file is such an example. It needs to be in the same directory as the > distutils.command.bdist_wininst module. > > So here is a build_py subclass (contained in a setup script) which does > this. Any comments? It's also often necessary to recursively include data files and preserve symlinks, especially on OS X. I'm currently using a (modified?) version of Rene Liebscher's "install_data" setup extensions. I believe I borrowed them from PyXML, but I'm not quite sure. -bob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2357 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20031119/9f5e1a5f/smime.bin From mal at lemburg.com Wed Nov 19 15:37:13 2003 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Nov 19 15:36:40 2003 Subject: [Distutils] Installing (data-)files in package directories In-Reply-To: References: Message-ID: <3FBBD479.9020808@lemburg.com> Thomas Heller wrote: > It seems installing data-files in package directories is a common > requirement - at least for me. > > Readme files, data files needed by the packages - the bdist_wininst.exe > file is such an example. It needs to be in the same directory as the > distutils.command.bdist_wininst module. > > So here is a build_py subclass (contained in a setup script) which does > this. Any comments? Does this only copy the files into the build directory or also install them ? It's a bit too unflexible for my taste (I wouldn't want *all* files to get copied into the package tree). Here's what I use for install_data to achieve more or less the same with more control: class mx_install_data(install_data): """ Rework the install_data command to something more useful. """ def finalize_options(self): if self.install_dir is None: installobj = self.distribution.get_command_obj('install') self.install_dir = installobj.install_data if _debug: print 'Installing data files to %s' % self.install_dir install_data.finalize_options(self) def run (self): if not self.dry_run: self.mkpath(self.install_dir) data_files = self.get_inputs() for entry in data_files: if type(entry) == types.StringType: if 1 or os.name != 'posix': # Unix- to platform-convention conversion entry = string.join(string.split(entry, '/'), os.sep) filenames = glob.glob(entry) for filename in filenames: dst = os.path.join(self.install_dir, filename) dstdir = os.path.split(dst)[0] if not self.dry_run: self.mkpath(dstdir) outfile = self.copy_file(filename, dst)[0] else: outfile = dst self.outfiles.append(outfile) else: raise ValueError, 'tuples in data_files not supported' -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Nov 19 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC.Zope Database Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From theller at python.net Wed Nov 19 15:41:11 2003 From: theller at python.net (Thomas Heller) Date: Wed Nov 19 15:43:01 2003 Subject: [Distutils] Installing (data-)files in package directories In-Reply-To: (Bob Ippolito's message of "Wed, 19 Nov 2003 15:26:19 -0500") References: Message-ID: <1xs4jdlk.fsf@python.net> Bob Ippolito writes: > On Nov 19, 2003, at 3:06 PM, Thomas Heller wrote: > >> It seems installing data-files in package directories is a common >> requirement - at least for me. > > Me too > >> Readme files, data files needed by the packages - the bdist_wininst.exe >> file is such an example. It needs to be in the same directory as the >> distutils.command.bdist_wininst module. >> >> So here is a build_py subclass (contained in a setup script) which does >> this. Any comments? > > It's also often necessary to recursively include data files and > preserve symlinks, especially on OS X. I'm currently using a > (modified?) version of Rene Liebscher's "install_data" setup > extensions. I believe I borrowed them from PyXML, but I'm not quite > sure. Ah yes, this looks quite nice (I'm looking into PyXML-0.8.3.tar.gz). And the trick is to reuse distutils' useless (?) install_data command ;-). Thanks, Thomas From seefeld at sympatico.ca Wed Nov 19 15:44:40 2003 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Wed Nov 19 15:48:32 2003 Subject: [Distutils] Installing (data-)files in package directories References: <1xs4jdlk.fsf@python.net> Message-ID: <29884005069fe43b61f40208b5e42a893fbbd3a4@Orthosoft.ca> Thomas Heller wrote: > Ah yes, this looks quite nice (I'm looking into PyXML-0.8.3.tar.gz). > And the trick is to reuse distutils' useless (?) install_data command > ;-). talking about data, I believe there's still a bug in the sdist command that causes data files not to be put into a distro by default, i.e. when no MANIFEST[.in] is provided. Regards, Stefan From mohammad_saaqib at yahoo.com Sat Nov 22 22:41:58 2003 From: mohammad_saaqib at yahoo.com (Mohammad) Date: Sat Nov 22 22:42:01 2003 Subject: [Distutils] test Message-ID: <20031123034158.6418.qmail@web60609.mail.yahoo.com> test __________________________________ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree From mohammad_saaqib at yahoo.com Sat Nov 22 22:42:05 2003 From: mohammad_saaqib at yahoo.com (Mohammad) Date: Sat Nov 22 22:42:08 2003 Subject: [Distutils] test Message-ID: <20031123034205.61021.qmail@web60608.mail.yahoo.com> test __________________________________ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree From seefeld at sympatico.ca Tue Nov 25 15:36:58 2003 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Tue Nov 25 15:38:59 2003 Subject: [Distutils] building documentation Message-ID: <4cf28e63afa3636dd1d8d1f454d528703fc3bbae@Orthosoft.ca> hi there, I'm trying to integrate document generation into my distutils- based build system, and I'm not sure how to do it best. Ideally, I'd like to provide a new command 'build_doc' that does all the work, and then let the other commands install the documentation (tutorial, manual). Where should I build the docs ? 'inplace' ? Or somewhere under build/ ? If I put it into build/lib.../ it gets installed as part of the library, i.e. under site-packages. 'setup.py clean' currently doesn't care about it either. In a nutshell, it seems I have to modify *all* commands that participate in the build/installation to take my docs into account. The shortest I can come up with is to build them inplace, and then treat them as 'data', as there is already a set of commands for that category. Is there anything I overlooked ? Any advice would be highly welcome, Stefan From seefeld at sympatico.ca Sun Nov 30 18:15:09 2003 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Sun Nov 30 17:18:12 2003 Subject: [Distutils] build_clib usage Message-ID: <3FCA79FD.6010003@sympatico.ca> hi there, I'v discovered almost by accident the presence of the build_clib command, and it seems to be what I'v been looking for for a while. Where is this documented ? The (online) distutils docs don't seem to talk about it anywhere. What keywords can I pass to setup() to generate a library via build_clib ? Thanks, Stefan