Mailman 3 June 2000 - Distutils-SIG

Win32+Numpy+distutils 0.8 problem
by Robin Becker 02 Jun '00

02 Jun '00

For some reason I'm getting this with the latest 15.3 tgz. This is with distutils 0.8. Seems as though the search for the python include directory has again failed somewhere. C:\Program Files\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 -I""\include\python1.5 -IInclude -I""\Include /TcSrc/_numpymodule.c /Fobuild\tem p.win32\Release\Src/_numpymodule.obj _numpymodule.c Src/_numpymodule.c(1) : fatal error C1083: Cannot open include file: 'Python.h': No such file or directory error: command '"C:\Program Files\Microsoft Visual Studio\VC98\BIN\cl.exe"' fail ed with exit status 2 -- Robin Becker

1 1

Suggested directory convention
by Greg Ward 31 May '00

31 May '00

Lots of distribution formats need a place to put files that aren't really temporary, can often be generated by the Distutils, but may be tweaked by the developer (and put under source control). I'm thinking of RPM's spec files, Debian's directory of package meta-data, Wise scripts, and similar stuff. Seems like a good idea to me to put this all in a directory "dist": the first time you use the Distutils to generate a .spec file, or a Wise script, or what-have-you, it would create "dist" and would always put those sort of files in there. Naturally, this would be a configurable option to the bdist_* commands, but "dist" strikes me as a sensible default. This is distinct from the "build" directory for two reasons: 1) it's for distribution, not building -- hence the domain of the module developer rather than the installer/packager; 2) it's not temporary -- it's always safe to blow away the "build" directory, but could be foolish to ditch "dist". (Eg. if someone has started with Distutils-generated files and customized them to suit.) In addition to the bdist_* commands, the sdist command could possibly use the dist tree for creating source archives. (Currently, it creates <distname>-<version> (eg. "Distutils-0.8.3") in the distribution root, fill it up with copies or hardlinks to all the files in MANIFEST, and creates an archive rooted at <distname>-<version>. If we move that source distribution tree down a level, we'd have to chdir to create the source archive.) Opinions? Sound like a good idea or just more complexity? Greg -- Greg Ward - nerd gward(a)python.net http://starship.python.net/~gward/ When the ship lifts, all bills are paid. No regrets.

3 4

Pondering multi-package packages
by Mark W. Alexander 31 May '00

31 May '00

I apologize of this is documented somewhere. If it is, I haven't found it. I've done lots of python modules, but not any packages, so if I'm just package ignorant feel free to slap me..... It all started in the early Zope days when I loaded Zope only to discover that all my scripts using Marc Lemburg's mxDateTime bombed because Zope's DateTime produced a namespace collision. The solution then was to move Zope, since I didn't 'need' it at the time. Now I do. So, I contacted Marc who, of course, knew about the collision and has already started looking at packaging all his stuff under an umbrella "mx" package. When I asked about Distutils, he hadn't really looked at it so I volunteered to see what it would take. Since Distutils includes a sample mxDateTime setup.py, it seemed fairly easy. I hacked it up to push it down a level (see attempt below). No problem except, of course, that it doesn't import because there's no mx.__init__.py. Ok, so I add an mx.__init__.py with __all__ defined and appropriate imports. Everything's happy and Distutils does fine. Now, I add in mx.TextTools and things get murkier. How does mx.__init__.py determine what __all__ should be and what to import? How do I tell Distutils about the other packages? I tried a single all-purpose Distutils setup.py that searches the mx package directory and adds in the sub-packages that it finds. So far so good. Now, what about mx.__init__.py? Should it: 1) Search it's package directory at runtime to determine what __all__ should be and what subpackages to import? If it does that, then mx.__init__.py will exist in all possible mx packages. Distutils probably won't care and will happily copy the lastest __init__.py into place (since they "should" all be the same, that's not a problem). BUT -- what if I make packages with --bdist-WHATEVER. Package manager WHATEVER is going to not be pleased with multiple packages that all provide mx.__init__.py. Some type of package forcing is going to have to occur for all except the first subpackage installed. - OR - 2) Be created dynamically at "install" time. The all-purpose setup.py can scan when Distutils runs and programatically create an __init__.py at installation time that meets the needs of the currently installed subpacakges. BUT -- that doesn't help when I used --bdist-WHATEVER again, because the resulting package is going to be installed on a system that probably does not have the same set of subpackages installed. I'll have to provide a postinstall script to WHATEVER to create __init__.py on the install host. Do-able, but now I'm manually tacking stuff on to the package that --bdist-WHATEVER can't (unless I missed it) handle. And, mx.__init__.py is not registered to the package somewhat defeating the purpose. It looks like --bdist-WHATEVER needs information about pre- and post-installation processing. This would have to be included in setup.py somewhere. But what should it be? Package managers (that support pre/post install) expect shell scripts. It would be really nice if --bdist took a python function and wrapped it for us in a shell wrapper, e.g. #!/bin/sh python -i <<EOF def postinstall(): ... postinstall EOF So, am I missing something or are these issues real? FWIW, I've included the directory scanning setup.py below. If you try it, it "almost" works for mxDateTime, mxTextTools, and mxNewBuiltins. There's still some issues with mx sub-modules that I'm ignorant of, but they're not related to this discussion. Mark mwa(a)gate.net ############## mx.DateTime setup.py ################ #!/usr/bin/env python import os """ setup.py for Marc-Andr� Lemburg's mx Extension modules Will scan the 'mx' directory for Marc's packages (just DateTime, TextTools, and NewBuiltins so far....) and provide the Distutils infor for all found packages. .""" # created 1999/09/19, Greg Ward # pushed to mx package by Mark Alexander __revision__ = "$Id: mxdatetime_setup.py,v 1.4 2000/03/02 01:49:46 gward Exp $" from distutils.core import setup DateTimePackages=['mx.DateTime', 'mx.DateTime.Examples', 'mx.DateTime.mxDateTime'] DateTimeExtensions=[('mx.DateTime.mxDateTime.mxDateTime', { 'sources': ['mx/DateTime/mxDateTime/mxDateTime.c'], 'include_dirs': ['mxDateTime'], 'macros': [('HAVE_STRFTIME', None), ('HAVE_STRPTIME', None), ('HAVE_TIMEGM', None)], } )] TextToolsPackages=['mx.TextTools','mx.TextTools','mx.TextTools.Constants','mx.TextTools.Examples'] TextToolsExtensions=[('mx.TextTools.mxTextTools.mxTextTools', { 'sources': ['mx/TextTools/mxTextTools/mxTextTools.c'], 'include_dirs': ['mxTextTools'], 'macros': [('HAVE_STRFTIME', None), ('HAVE_STRPTIME', None), ('HAVE_TIMEGM', None)], } )] NewBuiltinsPackages=['mx.NewBuiltins','mx.NewBuiltins.Examples','mx.NewBuiltins.mxTools'] NewBuiltinsExtensions=[('mx.NewBuiltins.mxTools.mxTools', { 'sources': ['mx/NewBuiltins/mxTools/mxTools.c'], 'include_dirs': ['NewBuiltins'], 'macros': [('HAVE_STRFTIME', None), ('HAVE_STRPTIME', None), ('HAVE_TIMEGM', None)], } ), ('mx.NewBuiltins.mxTools.xmap', { 'sources': ['mx/NewBuiltins/mxTools/xmap.c'], 'include_dirs': ['NewBuiltins'], 'macros': [('HAVE_STRFTIME', None), ('HAVE_STRPTIME', None), ('HAVE_TIMEGM', None)], } )] mxPackages=['mx'] mxExtensions=[] if os.path.isdir('mx/DateTime'): mxPackages=mxPackages+DateTimePackages mxExtensions=mxExtensions+DateTimeExtensions if os.path.isdir('mx/TextTools'): mxPackages=mxPackages+TextToolsPackages mxExtensions=mxExtensions+TextToolsExtensions if os.path.isdir('mx/NewBuiltins'): mxPackages=mxPackages+NewBuiltinsPackages mxExtensions=mxExtensions+NewBuiltinsExtensions setup (name = "mxDateTime", version = "1.3.0", description = "", author = "Marc-Andr� Lemburg", author_email = "mal(a)lemburg.com", url = "http://starship.python.net/~lemburg/mxDateTime.html", packages = mxPackages, # XXX user might have to edit the macro definitions here: yuck! # Probably do need to support 'Setup' file or something similar. ext_modules = mxExtensions, )

6 22