Mailman 3 September 2000 - Distutils-SIG

Re: [Python-checkins] CVS: distutils/distutils/command bdist_wininst.py,1.12,1.13
by Greg Ward Sept. 21, 2000

Sept. 21, 2000

On 19 September 2000, Thomas Heller said: > Modified Files: > bdist_wininst.py > Log Message: > Set the 'nt' installation scheme for the install command even if run > on other systems, so that data, headers, scripts are included in > the installer. Ahh good, I was hoping you'd take care of this. But: > *** bdist_wininst.py 2000/09/16 15:56:32 1.12 > --- bdist_wininst.py 2000/09/19 11:10:23 1.13 > *************** > *** 75,78 **** > --- 75,89 ---- > install = self.reinitialize_command('install') > install.root = self.bdist_dir > + if os.name != 'nt': > + # must force install to use the 'nt' scheme > + install.select_scheme ('nt') > + # change the backslash to the current pathname separator > + for key in ('purelib', 'platlib', 'headers', 'scripts', > + 'data'): > + attrname = 'install_' + key > + attr = getattr (install, attrname) > + if attr: > + attr = string.replace (attr, '\\', os.sep) > + setattr (install, attrname, attr) I think we can get rid of this hack by changing the INSTALL_SCHEMES dictionary in install.py to store paths in Unix form, and then convert to native when they're needed. That is, the install path "templates" in install.py would be treated just like install paths in the setup script (or on the command line) -- convert from foo/bar/baz to foo\bar\baz on Windows, and foo:bar:baz on Mac OS. (Modulo Mac OS' peculiar convention for absolute/relative directories: not sure if 'convert_path()' handles that completely right.) Greg Greg -- Greg Ward gward(a)python.net http://starship.python.net/~gward/

2 2

Re: [Distutils] test command for setup.py
by Berthold Höllmann Sept. 21, 2000

Sept. 21, 2000

hoel(a)germanlloyd.org (Berthold Höllmann) writes: > Hello, > > I'm thinking about providing a framework for pre-install testing for > distutils enabled Python packages. I think about a new "test" command, > which inserts the lib build path into PYTHONPATH and runs specified > tests. Is anyone else working on something like this, and what are > your opinions and requirements on this? Attatched you find a first version of test.py. This copied into ../distutils/command allows python setup.py test First this command calls "build" to enshure a correctly build disttribution. It then modifies "sys.path" to point it to the build directory, imports files from a subdirectory, and calls a function named "test" in the imported module. The default name of the subdirectory is "test". From this directory by default all modules named "test_*.py" are imported. Additional options are: - test_dir (test-dir): name of the directory containg the test files (default: "test") - test_prefix (test-prefix): prefix for the test files. (default: "test_") - test_suffixes (test-suffixes): suffixes used to generate filenames for the tests (default: None, which means: look for all files in test_dir named test_prefix<something>.py and list all <somethings>s for test_suffixes). I took build.py as a template for writing this. Berthold -- email: hoel(a)GermanLloyd.org ) tel. : +49 (40) 3 61 49 - 73 74 ( C[_] These opinions might be mine, but never those of my employer.

2 1

How I sorted things out
by Uche Ogbuji Sept. 20, 2000

Sept. 20, 2000

I ended up doing things by hand. I'd like to have distutils help automate things, and if it's any help, here's what I ended up doing. I had to do all the following as root, which I know distutils tries to avoid. Maybe there's another way around. I copied the spec file from build/bdist.linux-i686/rpm/SOURCES/4Suite-0.9.0.tar.gz to /usr/src/redhat/SOURCES and used the following spec file (mods from distutils) -----------------------------------%-------------------------------------- %define name 4Suite %define version 0.9.0 %define release 1 Summary: Tools for XML, DOM, XPath, XSLT, RDF and object database Name: %{name} Version: %{version} Release: %{release} Source0: %{name}-%{version}.tar.gz Copyright: FourThought, Inc Group: Development/Languages/Python Vendor: Fourthought, Inc. <support(a)4suite.org> Url: http://www.4suite.org %description 4Suite is a collection of Python tools for XML processing and object-databases. 4Suite is an integrated packaging of 4DOM, 4XPath, 4XSLT, 4RDF, and 4ODS. %prep %setup %build env CFLAGS="$RPM_OPT_FLAGS" python setup.py build %install python setup.py install --record=INSTALLED_FILES #%clean %files -f INSTALLED_FILES /usr/lib/python1.5/site-packages/xml/xpath/XPath.py /usr/lib/python1.5/site-packages/xml/xslt/XPattern.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Odl/Odl.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oql/Oql.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oif/Oif.py /usr/lib/python1.5/site-packages/xml/xpath/XPathParserBase.py /usr/lib/python1.5/site-packages/xml/xslt/XPatternParserBase.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Odl/OdlParserBase.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Odl/OdlParseTreeMapping.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oql/OqlParserBase.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oql/OqlParseTreeTuple.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oif/OifParserBase.py /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oif/OifParseTreeMapping.py /usr/lib/python1.5/site-packages/xml/xpath/XPath.py? /usr/lib/python1.5/site-packages/xml/xslt/XPattern.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Odl/Odl.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oql/Oql.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oif/Oif.py? /usr/lib/python1.5/site-packages/xml/xpath/XPathParserBase.py? /usr/lib/python1.5/site-packages/xml/xslt/XPatternParserBase.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Odl/OdlParserBase.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Odl/OdlParseTreeMapping.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oql/OqlParserBase.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oql/OqlParseTreeTuple.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oif/OifParserBase.py? /usr/lib/python1.5/site-packages/Ft/Ods/Parsers/Oif/OifParseTreeMapping.py? %defattr(-,root,root) %changelog * Wed Sep 20 2000 Uche Ogbuji <uche.ogbuji(a)fourthought.com> - initial RPM release -----------------------------------%-------------------------------------- Note that we have to manually list out more files than setup.py recognizes. This is because of the limitation of distutils with regard to extensions with multiple output files (see Mike Olson's message to distutils-sig today). I don't know if things can be made smoother. Are we missing something? Thanks. -- Uche Ogbuji Principal Consultant uche.ogbuji(a)fourthought.com +1 303 583 9900 x 101 Fourthought, Inc. http://Fourthought.com 4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA Software-engineering, knowledge-management, XML, CORBA, Linux, Python

1 0

test command for setup.py
by Berthold Höllmann Sept. 19, 2000

Sept. 19, 2000

Hello, I'm thinking about providing a framework for pre-install testing for distutils enabled Python packages. I think about a new "test" command, which inserts the lib build path into PYTHONPATH and runs specified tests. Is anyone else working on something like this, and what are your opinions and requirements on this? Thanks Berthold -- email: hoel(a)GermanLloyd.org ) tel. : +49 (40) 3 61 49 - 73 74 ( C[_] These opinions might be mine, but never those of my employer.

3 3

Brewing controversy over header file installation
by Greg Ward Sept. 19, 2000

Sept. 19, 2000

Jack Jansen has been hacking on the Mac OS support in the Distutils, and brings up an interesting -- but annoying point: installing extension-related header files under Python's include directory could be perceived as interfering with Python's source. I disagree, but let me see if I can summarize what's been going on in email between Jack, Corran Webster (the other person working on Distutils Mac support), and me. I mentioned how files are installed on Mac OS (by quoting the source, of course): > 'mac': { > 'purelib': '$base:Lib:site-packages', > 'platlib': '$base:Lib:site-packages', > 'headers': '$base:Include:$dist_name', > 'scripts': '$base:Scripts', Regarding the 'headers' install dir -- where extension-related header files go -- Jack says: > This one is wrong. Where do you put these things on Unix? I don't > particularly like it if distutils scripts go mucking about in my > source directories (which is what $base:Include would be). On Unix, the answer is: 'unix_prefix': { 'purelib': '$base/lib/python$py_version_short/site-packages', 'platlib': '$platbase/lib/python$py_version_short/site-packages', 'headers': '$base/include/python$py_version_short/$dist_name', 'scripts': '$base/bin', 'data' : '$base', }, In the "prefix" installation scheme, $base expands to sys.prefix, so in a standard Python 2.0 installation on Unix, the 'headers' install dir is /usr/local/include/python2.0/$dist_name. $dist_name in turn expands to the name of the Distribution -- "Numerical", "Imaging", whatever. So yes, we are installing files *under* one of Python's directories, but not *into* it -- the include/python2.0 directory continues to provide just the header files installed by Python, namely Python.h and friends (abstract.h, bitset.h, ..., unicodeobject.h). And that's why I think this isn't a bad thing to do. The real reason it was done this way, of course, was laziness. Any time you compile a Python extension, you have to include "-I/usr/local/include/python2.0" (or local equivalent). With Numerical Python's headers installed under this location, an extension that needs access to Numerical's C structures just has to say #include <Numerical/arrayobject.h> and it will Just Work. Jack had an interesting suggestion: > Moreover, for packages to use one another I would guess that more than just > include files is needed (like the location of the shared libraries). I think > that my preference (but I haven't thought about it very long) would be to put > "distribution data for other distributions" all together in a single place, > something like > $base/Extensions/packagename/Include > $base/Extensions/packagename/Lib If the shared libraries that go in Extensions/$dist_name/Lib (Jack misspelled it ;-) are C libraries, that's fine -- if they're Python extensions, I'm confused. This makes sense, but it does require adding another directory to the standard include search path when building Python extensions. And the directory would probably be named something different on Unix. But it's in interesting idea, and I'm not going to throw it out willy-nilly. What does everyone else think? Corran's response to this was: > I agree, though, that this would be a natural way to go. It might > make more sense to have it all inside the site-packages directory, > though: headers live in > > $base:Lib:site-packages:$package:Include > > while modules (both python and C shared libraries) live in > > $base:Lib:site-packages:$package: I disagree: site-packages should be for Python modules and extensions. If we have to install C headers and libraries -- and I suspect we probably do -- then they should find their own home. On Unix, perhaps another directory under <exec-prefix>/lib/python2.0, but I'm not sure what to call it that makes it clear these are for *C* programming, not Python programming. (Granted it's the specialized domain of C programming for developing Python extensions, but still.) On Mac OS and Windows, it's easy -- I'd say put 'em in Extensions/Includes and Extensions/Lib under <prefix> (where that / is really a : or a \, of course). Jack later follows up: > Ok, so then we have (assuming we're running in a source tree) an "Include" > which contains standard Python include files and "includes" which contains > package-deposited include files. I'm not too thrilled with the similar names, > but I guess I can live with this. Yechh -- I would be *very* wary of depending on case distinction in filenames! I still don't see what's so wrong with Include/*.h being standard Python headers, and Include/*/*.h being headers provided by whatever extensions are installed on the system. Greg -- Greg Ward gward(a)python.net http://starship.python.net/~gward/

3 6

bdist_wininst: data and scripts
by Thomas Heller Sept. 19, 2000

Sept. 19, 2000

I just check in the following patch for bdist_wininst: (This should fix Bastian's complaints about not including data and scripts into the windows installer) Modified Files: bdist_wininst.py Log Message: Set the 'nt' installation scheme for the install command even if run on other systems, so that data, headers, scripts are included in the installer. Index: bdist_wininst.py =================================================================== RCS file: /cvsroot/python/distutils/distutils/command/bdist_wininst.py,v retrieving revision 1.12 retrieving revision 1.13 diff -C2 -r1.12 -r1.13 *** bdist_wininst.py 2000/09/16 15:56:32 1.12 --- bdist_wininst.py 2000/09/19 11:10:23 1.13 *************** *** 75,78 **** --- 75,89 ---- install = self.reinitialize_command('install') install.root = self.bdist_dir + if os.name != 'nt': + # must force install to use the 'nt' scheme + install.select_scheme ('nt') + # change the backslash to the current pathname separator + for key in ('purelib', 'platlib', 'headers', 'scripts', + 'data'): + attrname = 'install_' + key + attr = getattr (install, attrname) + if attr: + attr = string.replace (attr, '\\', os.sep) + setattr (install, attrname, attr) install_lib = self.reinitialize_command('install_lib') *************** *** 100,111 **** "%s.win32" % fullname) ! # XXX hack! Our archive MUST be relative to sys.prefix ! # XXX What about .install_data, .install_scripts, ...? ! # [Perhaps require that all installation dirs be under sys.prefix ! # on Windows? this will be acceptable until we start dealing ! # with Python applications, at which point we should zip up ! # the application directory -- and again everything can be ! # under one dir --GPW] ! root_dir = install.install_lib arcname = self.make_archive (archive_basename, "zip", root_dir=root_dir) --- 111,128 ---- "%s.win32" % fullname) ! # Our archive MUST be relative to sys.prefix, which is the ! # same as install_lib in the 'nt' scheme. ! root_dir = os.path.normpath (install.install_lib) ! ! # Sanity check: Make sure everything is included ! for key in ('purelib', 'platlib', 'headers', 'scripts', 'data'): ! attrname = 'install_' + key ! install_x = getattr (install, attrname) ! # (Use normpath so that we can string.find to look for ! # subdirectories) ! install_x = os.path.normpath (install_x) ! if string.find (install_x, root_dir) != 0: ! raise DistutilsInternalError \ ! ("'%s' not included in install_lib" % key) arcname = self.make_archive (archive_basename, "zip", root_dir=root_dir) Thomas

1 0

PyOpenGL -- setup files checked into repository, demos added
by Mike Fletcher Sept. 19, 2000

Sept. 19, 2000

Thanks to Rene and Thomas, the current PyOpenGL (CVS) now builds the extensions under the latest official distutils for Python 1.5.2 under VC++ *yay*. [Note: I made a very minor change to the setup script to build the latest openglutils module (requires glu)] I don't have time to get the .py stuff working just now (from what I can see it looks simple, but I'm on a short timeline, and I spent my time this evening trying to figure out what's going on with the PyOpenGL CVS tree (it's a mess!)). I've checked the 1.5.5 demos into the repository. If any of the authors would like them _not_ included, let someone know and we'll pull them out again. If someone who knows wants to let people know what all those directories are about it might be advisable... I'm thinking of just deleting most of them if they're not needed. Things to do (for PyOpenGL people): Add the .py files to the setup scripts (I assume you just specify the directory, at least, that's what it looks like) Build and test the results. Check back into the repository. Start creating distributables. I'll be busy until next week, so no chance I'll get to this before then. Thanks distutils people, this is cool stuff. Enjoy yourselves, Mike

1 0

CCompiler interfaces
by Rene Liebscher Sept. 18, 2000

Sept. 18, 2000

Last weekend I tried to use Distutils' compiler classes to build an executable on Win32. Beside some problems with the compilers, I found that parameters of some methods of CCompiler are a little bit confusing. Output filenames: * create_static_lib needs a library name, which becomes intern a real file name. (Is anyone using this method?) * link_shared_library is similar to create_static_lib. * link_shared_object needs a real file name. * link_executable wants the basename of its output file. We have two methods shared_object_filename() and library_filename() (for static and shared libraries). Wouldn't it be better to add a method executable_filename() and use for all of the linker methods the real filename as parameter? For link_executable() we also need a parameter export_symbols. (An executable could export symbols to be able to be extended by plug-ins.) If we would make both changes I mentioned above, the interfaces of the three link_* methods would be the same. Also the implementation is almost the same. ( link_shared_library() calls link_shared_object(), and link_executable() differs only in one line in UnixCCompiler. The other compilers are similar, they only change some options to compile an executable instead a shared library.) The best way would to merge these three methods into one new method and remove the old ones. (Or let the old ones call the new and give a warning the user should use the new method.) This new method would have following parameters: (It is better to see a shared libray as an half-build executable than to compare it with static libraries.) def link (self, target, objects, output_libname, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None): target would be one of 'shared_object', 'shared_library' or 'executable'. The compiler classes would then be much simpler and smaller. For example UnixCCompiler. * Remove link_shared_library() and link_executable() * Rename link_shared_object() to link() and add target to the parameter list. * Change the spawn() call in it to the following piece of code. try: if target == 'executable': self.spawn (self.linker_exe + ld_args) else: self.spawn (self.linker_so + ld_args) except DistutilsExecError, msg: raise LinkError, msg This saves about 70 lines of redundant code. For BCPPCompiler and MSVCCompiler it is similar. Are there any objections? If not, I would send Greg a patch for it. ########################################################### And now a short piece of Changes.txt: > Release 0.9 (29 June, 2000): > ---------------------------- ... > * added the ability to compile and link in resource files with > Visual C++ on Windows (Thomas Heller) However, I can't find the place where it is done. Maybe it wasn't checked in ? I can do the same using BCPPCompiler and CygwinCCompiler, and will send a patch after reorganizing the compiler classes. Kind regards Rene Liebscher

3 5

Help: request for assistance with PyOpenGL distribution using dis tutils on NT
by Mike Fletcher Sept. 18, 2000

Sept. 18, 2000

Background: While stuck on my real work, I'm trying to get a PyOpenGL distribution (1.5.6alpha1 from CVS) working using distutils. Rene Liebscher kindly created an installation script that seems to work for him on Win32 for the 1.5.5 PyOpenGL, and for Jason Petrone (on Linux) using the 1.5.6 source from CVS. I'm working on NT 4.0 sp3, using VC++ 6.0sp3, (I get similar effects on Cygnus, but since I'm not sure the package builds under that compiler I'm not focussing on it yet). When trying to build, I get a long set of errors (see below), the elimination of which is well beyond my meagre VC++ or distutils skills to fix. According to MSDN, the problems I'm getting are normally caused by having a C++ file compiled as a C file. To get around it, you apparently put an ifdef at the top and bottom of the file, but the file concerned (GL.h) has this check, and still reports the errors. I'm assuming there's some flag that needs to be set somewhere, but I don't know what it is. Note: everything builds fine using the old setup script for the latest CVS version on this machine (the one for which I'm trying to get a distutils script). D:\bin\lang\studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX -Isrc -ID:\bin\lang\Python\Include /Tcsrc/_glumodule.c /Fobuild\temp.win32\Release\src/_glumodule.obj Doesn't work. The GUI under the old BUILD.py uses: /nologo /MD /W3 /GX /O2 /Ob2 /I "D:\bin\lang\Python\include" /D "WIN32" /D "_WINDOWS" /D "NDEBUG" /Fp"tmp\./_glu.pch" /YX /Fo"tmp\./" /Fd"tmp\./" /FD /c and works fine. GL.h has the following code near the top (and close at the bottom): #ifdef __cplusplus extern "C" { #endif Which, according to MSDN, should avoid the errors, but doesn't seem to. Here's the output from running setup.py build... S:\PyOpenGL>setup build running build running build_py warning: build_py: package init file 'OpenGL\__init__.py' not found (or not a re gular file) warning: build_py: package init file 'OpenGL\Demo\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\Demo\da\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\Demo\dek\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\Demo\dek\OglSurface\__init__.py' no t found (or not a regular file) warning: build_py: package init file 'OpenGL\Demo\srenner\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\Demo\srenner\Images\__init__.py' no t found (or not a regular file) warning: build_py: package init file 'OpenGL\Demo\tom\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\GL\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\GLU\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\GLUT\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\Tk\__init__.py' not found (or not a regular file) warning: build_py: package init file 'OpenGL\shared\__init__.py' not found (or n ot a regular file) running build_ext skipping '_opengl' extension (up-to-date) skipping 'openglutil' extension (up-to-date) building '_glu' extension D:\bin\lang\studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX -Isrc -ID:\bin\lan g\Python\Include /Tcsrc/_glumodule.c /Fobuild\temp.win32\Release\src/_glumodule. obj _glumodule.c src\GL/gl.h(1135) : error C2054: expected '(' to follow 'WINGDIAPI' src\GL/gl.h(1135) : error C2085: 'APIENTRY' : not in formal parameter list src\GL/gl.h(1135) : error C2146: syntax error : missing ',' before identifier 'g lAccum' src\GL/gl.h(1135) : error C2143: syntax error : missing ';' before '(' src\GL/gl.h(1135) : error C2059: syntax error : ')' src\GL/gl.h(1136) : error C2054: expected '(' to follow 'WINGDIAPI' src\GL/gl.h(1136) : error C2085: 'APIENTRY' : not in formal parameter list src\GL/gl.h(1136) : error C2146: syntax error : missing ',' before identifier 'g lAlphaFunc' src\GL/gl.h(1136) : error C2143: syntax error : missing ';' before '(' src\GL/gl.h(1136) : error C2059: syntax error : ')' src\GL/gl.h(1137) : error C2054: expected '(' to follow 'WINGDIAPI' ... (100 errors, then stops due to having so many) Does anyone have any suggestions on how to fix this (keeping in mind that I'm not a C/C++ programmer, so the instructions should be pretty basic)? Thoughts appreciated, Mike __________________________________ Mike C. Fletcher Designer, VR Plumber http://members.home.com/mcfletch

3 2

More minor issues
by Rene Liebscher Sept. 18, 2000

Sept. 18, 2000

There are two features which could be very useful. 1) If the install command had a '--force' option, which lets install ignore file dates, you could overwrite manipulated installations with the original files. Currently if there are changed some files for any reason, you have to find and delete them and then reinstall the package. (Distutils doesn't overwrite newer files.) (Using this feature you could also install an older version of the package.) 2) If you are using two or more different python versions on your computer, and you want to install a package which contains C-compiled extensions, then you have to delete the build directory before you can install the package with another version of python or you use build with the '--force' option. In any case you have to force distutils to rebuild all extensions because they were compiled using a wrong version of python. If you only build this package once, this may be not a problem, but if you develop a package on your own, you have to compile it more often, and then you don't want to compile everything new to test it. You could use here different build directories, this is not too much work. But you could forget these options, so why doesn't let distutils do it for us. We simply have to change the default build directory to something like 'build_' + sys.version[0:3] There is attached a patch for it. (build.patch) (Using this, I found 'build_rpm' using a different version of python than I have used to start the setup. I probably never had spotted it, if both had used the same build directory.) Kind regards Rene Liebscher diff -BurN --exclude=*.pyc --minimal distutils.orig/distutils/command/build.py distutils.patched/distutils/command/build.py --- distutils.orig/distutils/command/build.py Sat Jun 24 03:23:37 2000 +++ distutils.patched/distutils/command/build.py Tue Sep 12 12:05:46 2000 @@ -48,7 +48,7 @@ ] def initialize_options (self): - self.build_base = 'build' + self.build_base = 'build_' + sys.version[0:3] # these are decided only after 'build_base' has its final value # (unless overridden by the user or client) self.build_purelib = None

3 6