Mailman 3 January 2000 - Distutils-SIG

Pre-0.1.3 snapshot available
by Greg Ward 18 Jan '00

18 Jan '00

Hi all -- I've finally caught up with the last couple weeks' traffic on the sig, and fixed many of the bugs reported. There's a new code snapshot available: http://www.python.org/sigs/distutils-sig/distutils-20000117.tar.gz Please give it a whirl, especially if you're using Windows. This mostly includes Windows portability fixes. It does *not* include Finn Bock's fixes for JPython; I haven't had time to look closely at those. They might have to wait for version 0.2. There *might* be a 0.1.3 release before IPC8, but I really should work on my slides for the conference first... so please give this snapshot a try. Just download it and run the setup script as usual -- the main difference between the snapshot and a real release is no doc updates (hence README, USAGE, and CHANGES are not included in the snapshot). Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 2

More (Windows) comments on distutils-0.1.2
by Thomas Heller 18 Jan '00

18 Jan '00

1. Python extensions for Windows MUST be linked with the /MD switch to use the multithreaded runtime library. Optimizations are also enabled by the following line: > class MSVCCompiler (CCompiler) : > def __init__ (self, > [...] > self.compile_options = [ '/nologo', '/Ox', '/MD', '/GD' ] 2. In build_ext.py I find the following code: > def build_extensions (self, extensions): > if self.compiler.compiler_type == 'msvc': > def_file = build_info.get ('def_file') > if def_file is None: > source_dir = os.path.dirname (sources[0]) > ext_base = (string.split (extension_name, '.'))[-1] > def_file = os.path.join (source_dir, "%s.def" % ext_base) > if not os.path.exists (def_file): > self.warn ("file '%s' not found: " % def_file + > "might have problems building DLL") > def_file = None > > if def_file is not None: > extra_args.append ('/DEF:' + def_file) Doesn't this belong into msvccompiler.py? We do not need any stinkin' DEF-files, if we use the link options /EXPORT and /BASE. The following code builds the required arguments: > if def_file is not None: > extra_args.append ('/DEF:' + def_file) > else: > # if no def file is found, export the init<name> function > # and calculate a random base address > extra_args.append ('/EXPORT:init' + extension_name) > import whrandom > base = whrandom.randint (0x1000, 0x8000) * 0x10000 > extra_args.append ('/BASE:0x%x' % base) Regards, Thomas Heller ION-TOF GmbH

4 3

wish list
by Paul F. Dubois 17 Jan '00

17 Jan '00

Here's my wish list after some use of Distutils-0.1.2 1. The object files should go in some subdirectory of build so that I can clean up by just blowing away that directory. 2. Sometimes I eliminate a .py file but there is a copy in build that still gets installed. 3. If you just type python setup.py the information you get back is not helpful, since you don't know what the commands are. I think it should list all the legal commands.

4 3

bugs in MSVCCompiler ?
by Robin Becker 17 Jan '00

17 Jan '00

I've just found what appears to be a bug in 0.1.2 in the MSVCCompiler class. at line 267 the library_filename functions should look like def library_filename (self, libname): """Return the static library filename corresponding to the specified library name.""" return "%s%s" %( libname, self._static_lib_ext ) def shared_library_filename (self, libname): """Return the shared library filename corresponding to the specified library name.""" return "%s%s" %( libname, self._shared_lib_ext ) the original format strings were "lib%s%s" leading to errors such as trying to use libpython15.lib during the build. -- Robin Becker

2 1

Linux Journal Python Supplement
by Greg Ward 17 Jan '00

17 Jan '00

Hi all -- the folks at Linux Journal asked me to post this to the sig. Free magazine, woo hoo! ----- Forwarded message from Rebecca Cassity <rebecca(a)ssc.com> ----- [...] The May 2000 issue of Linux Journal is going to be accompanied by a special edition Python Supplement! Python users can sign up for their free copy at http://www.linuxjournal.com/python/. ----- End forwarded message ----- -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

1 0

Re: [Distutils] Comments on Distutils-0.1.2 on Windows
by Thomas Heller 17 Jan '00

17 Jan '00

> On 12 January 2000, Thomas Heller said: > > 1. There is no standard zip-utility under windows, > > but zip.exe from info-zip works right out of the box. > > Oh good! I've changed the comment in the source now. > > > I've fixed the source to use James C. Ahlstrom's zipfile.py > > when zip.exe is not available (Should this always be used?) > > Do you mean should zipfile.py be the default rather than the standalone > zip utility? Not yet -- the latter is far more widespread than the > former! In Python 1.6, maybe. Maybe you could include zipfile.py into the distutils package? > > Anyways, I've incorporated your support for zipfile.py into the dist > command, but I don't have that module installed to try it out. The code > is slightly different from yours, so would you mind trying out the > attached version of dist.py? Well, you didn't expect that I would really test it ;-) ? This code try: self.spawn (["zip", "-r", base_dir + ".zip", base_dir]) except DistutilsExecError: try: import zipfile except ImportError: raise DistutilsExecError, \ fails for two simple reasons: - You are not doing 'from distutils.errors import *' in dist.py - self.spawn() raises an OSError, not an DistutilsExecError. Otherwise your code looks fine. (Except that my patch for using shutils.copyfile is missing (if os.link fails). > > I'll get to the bugs you found in MSVCCompiler shortly. Thanks! > Thank you for distutils! Thomas

1 0

distutils and JPython
by Finn Bock 17 Jan '00

17 Jan '00

Hi, Generally I'm looking at distutils to see if it can help me simplify the installation of some of the JPython modules I have created. A JPython modules could typically consists of: - .py files - .jar file or .class files - .dll files Lets first assume that all these files have been compiled and built in some traditional way. In order to get started with distutils, I will avoid any attempt to compile any of the C/C++ source at the target machine. So this is windows only, at least to begin with. My primary target user is a java/jpython developer. Copying the .py files works like a dream. For .jar files, the situation get more complicated. A jar file could: a) be copied to an well-known directory f.ex. a subdir to sys.prefix. The JPython startup script would then have to be changed. I.e. inserting the new .jar in the -classpath or /cp option to the java command. b) If using Sun-JDK, the jar could be copied to the "$java.ext.dir" directory in the JDK/JRE installation. c) If using MS-jview, the jar could be extracted into "$java.home\Lib" or "$java.home\TrustLib" directory. java.home is narmmaly something like "c:\windows\Java" d) Some other directory, leaving it entirely up to the user to fiddle with startup scripts and CLASSPATH environment variables. Other JVMs could possible have different default places where class files are located. Even if distutils will support all methods, I think the user must be queried which strategy she want to use. For .dll files, the situation get similar. A dll file could: a) be copied to one of the directories on the current PATH (only easily available in Sun-JDK as $java.library.path) b) be copied to JVM specific exe directory. $java.home/bin in the JDK/JRE for Sun-JDK, $com.ms.sysdir in c:\windows\system32 for MS-jview. c) Some other directory, leaving it up the user to change the $PATH / $LD_LIBRARY_PATH(?) environment variables. Again, I think the user must be queried before .dll files can be copied. These are normal installation issues. I am a bit unsure on how much I can reasonably expect distutils to deal with this. From my very limited experience with distutils, I think distutils is exactly the right place to put some of the logic that deals with JVM and OS dependencies as described above. Any thoughs? Buglets and suggestions: a) Documentation buglet. In the USAGE file for "dist", -f is described as an alias for --formats. -f does not work. b) How about allowing for some alias names for the README file. Some of us lazy double-clicking windows types prefer names like README.txt. c) Why are "packages" and "py_module" mutually exclusive. One of my products happens to have a toplevel module which contains "from pck.Main import *". I think that I find the current behavior limiting. regards, finn

2 1

Disposition of C extensions and packages
by A.M. Kuchling 17 Jan '00

17 Jan '00

[Crossposted to xml-sig, distutils-sig] I'm working on getting the XML-SIG's CVS tree to install using the current version of the Distutils. Right now there are two C extensions, sgmlop.so and pyexpat.so, and they're installed under xml/parsers/ . It's hard to handle this case using the distutils code as it stands, because it expects to put extensions into a build/platlib/ directory, from which they'll be installed into site-packages. I can coerce setup.py into installing them into xml/parsers/, by subclassing the BuildExt command and setting build_dir myself: from distutils.command.build_ext import BuildExt class XMLBuildExt(BuildExt): def set_default_options (self): BuildExt.set_default_options( self ) self.build_dir = 'lib/build/xml/parser' setup (name = "PyXML", cmdclass = {'build_ext':XMLBuildExt}, ...) You also have to subclass the Install command and set build_dir there; I've trimmed that code. It's really clunky.\ Note that this scheme will break once there are C modules that need to be installed anywhere other than xml/parsers/, because build_dir is being hardwired without knowledge of what module is being compiled. Questions: 1) A general Python question about packaging style: Is mixing C extensions and Python modules in one package tree a bad idea? It makes the whole tree platform-dependent, which is probably annoying for sites maintaining Python installation for different architectures. 2) Another general question, this time o: how should this be handled? Should C extensions always be effectively top-level, and therefore go into site-packages? Should there be an xml package holding .py files, and an X package holding all the C extensions? (X = 'plat_xml', 'xml_binary', or something like that) 3) XML-SIG question: should I go ahead and change it (since I first changed it to use xml.parsers.sgmlop)? 4) Distutils question: is this a problem with the Distutils code that needs fixing? I suspect not; if the tools make it difficult to do stupid things like mix .py and .so files, that's a good thing. -- A.M. Kuchling http://starship.python.net/crew/amk/ The Kappamaki, a whaling research ship, was currently researching the question: How many whales can you catch in one week? -- Terry Pratchett & Neil Gaiman, _Good Omens_

4 3

Comments on Distutils-0.1.2 on Windows
by Thomas Heller 12 Jan '00

12 Jan '00

Well, I finally got a chance to test distutils 0.1.2 under Windows. 1. There is no standard zip-utility under windows, but zip.exe from info-zip works right out of the box. I've fixed the source to use James C. Ahlstrom's zipfile.py when zip.exe is not available (Should this always be used?) class Dist (Command): ... def make_zipfile (self, base_dir): # This assumes the Unix 'zip' utility -- it could be easily recast # to use pkzip (or whatever the command-line zip creation utility # on Redmond's archaic CP/M knockoff is nowadays), but I'll let # someone who can actually test it do that. try: self.spawn (["zip", "-r", base_dir + ".zip", base_dir]) except OSError: import zipfile z = zipfile.ZipFile (base_dir + ".zip", "wb", compression=zipfile.ZIP_DEFLATED) def visit (z, dirname, names): for name in names: path = os.path.join (dirname, name) if os.path.isfile (path): z.write (path, path) os.path.walk (base_dir, visit, z) z.close() 2. Libraries for VC++ are named python15.lib, not libpython15.lib. I had to fix this in msvccompiler.py: class MSVCCompiler (CCompiler): ... def library_filename (self, libname): """Return the static library filename corresponding to the specified library name.""" # return "lib%s%s" %( libname, self._static_lib_ext ) return "%s%s" %( libname, self._static_lib_ext ) def shared_library_filename (self, libname): """Return the shared library filename corresponding to the specified library name.""" # return "lib%s%s" %( libname, self._shared_lib_ext ) return "%s%s" %( libname, self._shared_lib_ext ) 3. os.link does not exist under windows, use shutil.copyfile for that: class Dist (Command): def make_release_tree (self, base_dir, files): ... self.announce ("making hard links in %s..." % base_dir) for file in files: dest = os.path.join (base_dir, file) if not os.path.exists (dest): try: self.execute (os.link, (file, dest), "linking %s -> %s" % (file, dest)) except AttributeError: import shutil self.execute (shutil.copyfile, (file, dest), "copying %s -> %s" % (file, dest)) Regards Thomas Heller ION-TOF GmbH

1 0

Possible bug in Distutils-0.1.2 on Windows
by Geza Groma 12 Jan '00

12 Jan '00

Trying to install the new Numerical package by Distutils-0.1.2 on Windows NT with MSVC 6.0, I got the following error: D:\Archives\Python\Numerical-14>python setup.py build running build running build_py not copying Lib\ArrayPrinter.py (output up-to-date) not copying Lib\FFT.py (output up-to-date) not copying Lib\LinearAlgebra.py (output up-to-date) not copying Lib\Matrix.py (output up-to-date) not copying Lib\MLab.py (output up-to-date) not copying Lib\Numeric.py (output up-to-date) not copying Lib\Precision.py (output up-to-date) not copying Lib\RandomArray.py (output up-to-date) not copying Lib\UserArray.py (output up-to-date) running build_ext C:\Program Files\Microsoft Visual Studio\VC98\bin\cl.exe /nologo -Ic:\Program Fi les\Python\include\python1.5 -Ic:\Program Files\Python\include -IInclude /c /FoS rc/_numpymodule.obj /TcSrc/_numpymodule.c Command line warning D4024 : unrecognized source file type 'Files\Python\include \python1.5', object file assumed Command line warning D4027 : source file 'Files\Python\include\python1.5' ignore d Command line warning D4024 : unrecognized source file type 'Files\Python\include ', object file assumed Command line warning D4027 : source file 'Files\Python\include' ignored _numpymodule.c Src/_numpymodule.c(1) : fatal error C1083: Cannot open include file: 'Python.h': No such file or directory Traceback (innermost last): File "setup.py", line 29, in ? ext_modules = [('_numpy', File "c:\Program Files\Python\distutils\core.py", line 96, in setup dist.run_commands () File "c:\Program Files\Python\distutils\core.py", line 442, in run_commands self.run_command (cmd) File "c:\Program Files\Python\distutils\core.py", line 491, in run_command cmd_obj.run () File "c:\Program Files\Python\distutils\command\build.py", line 52, in run self.run_peer ('build_ext') File "c:\Program Files\Python\distutils\core.py", line 802, in run_peer self.distribution.run_command (command) File "c:\Program Files\Python\distutils\core.py", line 491, in run_command cmd_obj.run () File "c:\Program Files\Python\distutils\command\build_ext.py", line 146, in ru n self.build_extensions (self.extensions) File "c:\Program Files\Python\distutils\command\build_ext.py", line 209, in bu ild_extensions include_dirs=include_dirs) File "c:\Program Files\Python\distutils\msvccompiler.py", line 103, in compile self.spawn ([self.cc] + cc_args) File "C:\Program Files\Python\distutils\ccompiler.py", line 421, in spawn spawn (cmd, verbose=self.verbose, dry_run=self.dry_run) File "c:\Program Files\Python\distutils\spawn.py", line 37, in spawn _spawn_nt (cmd, search_path, verbose, dry_run) File "c:\Program Files\Python\distutils\spawn.py", line 68, in _spawn_nt raise DistutilsExecError("command failed: %d" % rc) distutils.errors.DistutilsExecError: command failed: 2 I guess the problem is that cl.exe does not accept a directory name including spaces in its argument list unless in quotation marks. In the above example -Ic:\Program Files\Python\include\python1.5 should be written as -I"c:\Program Files\Python\include\python1.5" I am not a member of this list, so please, cc all responses to my private e-mail address. Thanks, Geza Groma

1 0