Mailman 3 October 2000 - Distutils-SIG

Installation directory
by Moore, Paul 24 Oct '00

24 Oct '00

I haven't been involved with Python for quite a while now, having been doing work with Perl instead. I was recently prompted to have another look at Python. (I don't have strong enough views on scripting languages to be described as a proper convert, though...) I was fairly quickly drawn into a discussion on 3rd party modules, and on the back of a comment that Python really needs a repository of packages like Perl's CPAN, I said that what was really important was that Python had a standardised build and distribution process, like Perl's MakeMaker and the standard Perl directory structure. As usual when I make comments like this, I immediately discovered distutils, which seems almost exactly what I was thinking of... :-) So I printed out the documentation, downloaded and skimmed the mailing list archive (not had time to read all 1600 messages yet!) and had a go with it. So far, I've only played with "pure python" modules, but it looks quite nice. I do, however, have one problem with it, which may be more a Python issue than a distutils issue, but I thought I'd raise it here anyway. I'm working on Win32, with Python 2.0 (the ActiveState distribution, although I don't believe there is any fundamental difference between this and the BeOpen distribution). When I package a module, and then unpack it, it is installed directly into the C:\Applications\Python20 directory, which is the Python installation directory. This is not what I would want, especially as the default action. (Excuse the mention of Perl here...) When Perl is built and installed, it creates a directory structure under its installation directory, consisting of a "bin" directory (executables and DLLs), a "lib" directory (library modules) and a "site" directory (locally installed modules). The "site" directory is empty by default, but is where all MakeMaker-installed modules are placed. I would like it if Python worked similarly. Clearly, there isn't any point in changing the existing directory structure, but adding a ...\Python20\Site directory, and having distutils install modules there by default, would help to separate components of the installation from local additions. I guess this requires co-operation from the Python core, as it would need ...\Python20\Site adding by default into the sys.path list. Does this idea make sense? Is it worth doing, or have I missed something fundamental? (BTW, I have no idea if Python on Unix is set up differently - maybe this is already in place there...? Thanks for your time, Paul Moore.

2 1

Some suggested additions for distutils
by Moore, Paul 24 Oct '00

24 Oct '00

I've just started looking at distutils. There are two things which I feel would be nice to add to the existing functionality. 1. A --dry-run option to the install action. This could list what was going to be done, but not actually do it. This would be useful for testing installations, or for systems admins who want to be sure what an installer is going to do before starting. 2. An uninstall action. This can be simulated by doing a python setup.py install --record installation.log and then using the list of files in install.log as a list of files to delete. However, a proper uninstall action would be nice. (And I don't know if there are more complex installs where the above workaround would be inadequate...) Thanks, Paul Moore

3 2

Can distutils handle PYTHONPATH?
by Mike Coleman 20 Oct '00

20 Oct '00

Hi, I've been looking at distutils to package my python project (see subterfugue.org, if curious), and I think I've stumbled across a problem. Two parts of my project are the main python code, which resides in the "root" (no) package, and one or more directories of "tricks" (like plugins), which are python modules, also in the "root" package, which the main code imports. The tricks get imported by virtue of trick directories being on PYTHONPATH. End users can also write their own tricks, and add them by putting their directory on PYTHONPATH. I don't want to put the tricks in a package, because that would complicate the steps for end users wanting to add their own tricks. It appears, though, that distutils wants all python code for the "root" package to live in a single directory. As far as I can tell, the only way I could use distutils is to break the tricks into one or more separate pieces, which I'm pretty reluctant to do. Any suggestions? Thanks, --Mike (please Cc me on this thread) -- [O]ne of the features of the Internet [...] is that small groups of people can greatly disturb large organizations. --Charles C. Mann

1 0

Doc tweak: home dirs on Windows
by Greg Ward 19 Oct '00

19 Oct '00

Hi all -- I'm working on polishing and finishing up the "Installing Python Modules" manual, and I notice the following sentence: Since Windows has no conception of a user's home directory, and since the standard Python installation under Windows is simpler than that under Unix, there's no point in having separate \longprogramopt{prefix} and \longprogramopt{home} options. (Please excuse the LaTeX markup!) My understanding, gleaned from a rather involved discussion on python-dev some months ago is that this is not entirely true. Thus, I propose changing this to sentence to: Since Windows has only a vague and fuzzy notion of a user's home directory, and the exact definition varies across different Windows versions, the \command{install} command makes no attempt to figure out a home directory. There's no point in having separate \longprogramopt{prefix} and \longprogramopt{home} options, so only \longprogramopt{prefix} is supported. That's my understanding of the situation. Does this sound right to everyone? Greg -- Greg Ward gward(a)python.net http://starship.python.net/~gward/

1 0

Factoring out WindowsCCompiler?
by Greg Ward 19 Oct '00

19 Oct '00

Rene -- now that the fuss over Distutils 1.0/Python 2.0 has died down, where does your work on factoring out WindowsCCompiler stand? Now would be a good time to get that checked in, so it can be well tested before Distutils 1.1. Greg -- Greg Ward gward(a)python.net http://starship.python.net/~gward/

2 1

Hacks and bugs for distutilizing wxPython
by Robin Dunn 18 Oct '00

18 Oct '00

Looks like I've got wxPython building with Distutils now on win32, Linux and Solaris. I thought I'd pass along the things I had to hack around so maybe some official solutions could be put into Distutils 1.1... Win32 (MSCVCompiler) -------------------- 1. As mentioned previously I need to pass -I flags to the resource compiler for my .rc file. I just hacked it to use the same flags passed to the C compiler but it might make sense to do it with a new attribute of Extension. 2. My secondary extensions need to link with the import lib produced from building the first extension. The default location of this file is dynamic based on the python version, debug status, and also the relative path to the first source file. Since there was no reliable way to determine before setup() was called where that directory would be I hacked it to just use a fixed location for the implib (build\ilib\). 3. I implemented these two hacks by deriving from MSCVCompiler and replacing the whole compile and link methods, each with only a one or two line change. It would be nice if they were more modular so derived classes could just override little bits here and there instead of huge methods. 4. I noticed this one on win32 but it may be a general problem. If relative path names containing '..' are used for extension source files then the objects can be placed in odd locations, possibly not even under ./build. For example, '../../wx/contrib/src/stc/stc.cpp' will have its object file put in build\temp.win32-1.5\Release\../../wx/contrib/src/stc/stc.obj which defeats the purpose of having the protection of temp.win32-1.5\Release! Related to this, if a full path is given to a source file, then the object will be put in the same directory as the source. I think object files should always be put under the build dir. Unix ---- 5. On my Linux system (RH 6.1, RPM 3.0.3) bdist_rpm failed with this message: Could not open /home/rd/wx/wxPython/build/bdist.linux-i686/rpm/RPMS/i386/wxPython- 2.2.2-1.i386.rpm because bdist_rpm didn't make the i386 directory. 6. Why is -g used by the compiler when --debug is not given to distutils? 7. It would be nice to be able to override CC, LDSAHRED, etc. (by setting environment variables or something) instead of just blindly taking them from Python's Makefile. For example I would like to be able to use "gcc -G" instead of the default (with Py-1.5.2) of "ld -G" on Solaris so that the static C++ objects will actually get initialized instead of being full of garbage and causing the application to segfault. SWIG ---- 8. Neither the SWIG support in 1.0 nor the recently contributed patches would meet my needs so I rolled my own. It's not integrated with build_ext but is run before setup() while I am constructing my Extension objects, and the list of generated sources is returned so it can be used to build the Extension object. It's very specific to my needs but others may find it useful too. I was able to reuse a lot of utility functions from distutils so much of the behaviour is similar to the way the rest of it works. You can see my setup.py and support files at: http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/wxWindows/wxPython/Attic/?cvsr oot=wxwindows&only_with_tag=WX_2_2_BRANCH It may be educational to take a look at it, even if it's just to point out the mistakes that a Distutils newbie like me makes! <grin> BTW, does anybody else have a 436 line setup.py? (Plus a coulple hundred for my MSVCCompiler hacks and other helper functions.) -- Robin Dunn Software Craftsman robin(a)AllDunn.com http://wxPython.org Java give you jitters? http://wxPROs.com Relax with wxPython!

2 2

Watch those tabs!
by Greg Ward 18 Oct '00

18 Oct '00

Hi all -- when submitting patches, please be sure that no evil tabs sneak in! I've just noticed that there are several Distutils source files with the dreaded ^I in them. I need to be more vigilant in fixing tabs when I apply patches, and patch authors need to be more vigilant about not letting them into their patches in the first place. Remember, (setq-default indent-tabs-mode nil) is your friend! Greg (What, you mean there are editors other than Emacs in the world? Nawwww...) -- Greg Ward gward(a)python.net http://starship.python.net/~gward/

3 3

Distutilizing wxPython
by Robin Dunn 18 Oct '00

18 Oct '00

I'm finally going to the effort to convert to using distutils for wxPython. So far, so good, I have the core extension module being built for win32. One problem that I ran into is that the resource compiler needs to have the -I and -D flags passed to it. The little patch below did the trick for me. One problem/question I have at this point is that the secondary extension modules need to link with the implib produced when linking the core extension module. Is there a way to determine before setup() is called where that library will be put so I can add that directory to the library_dirs for the other Extensions? (For example, "build/temp.win32-1.5/Release/src/") Index: msvccompiler.py =================================================================== RCS file: /cvsroot/python/distutils/distutils/msvccompiler.py,v retrieving revision 1.42 diff -c -r1.42 msvccompiler.py *** msvccompiler.py 2000/09/27 02:08:14 1.42 --- msvccompiler.py 2000/10/07 20:00:47 *************** *** 313,319 **** input_opt = src output_opt = "/fo" + obj try: ! self.spawn ([self.rc] + [output_opt] + [input_opt]) except DistutilsExecError, msg: raise CompileError, msg --- 313,319 ---- input_opt = src output_opt = "/fo" + obj try: ! self.spawn ([self.rc] + pp_opts + [output_opt] + [input_opt]) except DistutilsExecError, msg: raise CompileError, msg -- Robin Dunn Software Craftsman robin(a)AllDunn.com http://wxPython.org Java give you jitters? http://wxPROs.com Relax with wxPython!

4 7

extra files into the package dir
by Pete Shinners 18 Oct '00

18 Oct '00

i have some extra files i'd like to have copied into the destination package directory when installing. it seems that distutils only installs the .PY(C) files when installing a package directory. how can i tell distutils to copy my other files too? does this involve 'extending' the install command? i could probably do that if there were docs or an example of something like that. thanks

2 1

problems with "runtime_library_dirs" option
by Berthold Höllmann 14 Oct '00

14 Oct '00

Hello, I tried to use the "runtime_library_dirs" option when compiling the experimental "py-bsddb3-2.2.0" package. I had two problems with it. First I tried replace "library_dirs" by "runtime_library_dirs", but that failed, because the linker needed both, -L... and -R... , so I would suggest to change the method "runtime_library_dir_option" in unixcompiler.py from def runtime_library_dir_option (self, dir): return "-R" + dir to def runtime_library_dir_option (self, dir): return "-R" + dir + " -L" + dir The second problem is, that my gcc 2.95.2 does not propagate unknown options to the linker, so I get: gcc -shared build/temp.linux-i686-2.0/db.o -L/usr/local/BerkeleyDB.3.1/lib -R/usr/local/BerkeleyDB.3.1/lib -ldb -o build/lib.linux-i686-2.0/dbc.so gcc: unrecognized option `-R/usr/local/BerkeleyDB.3.1/lib' The module will be written, but it won't find the specified library, because the path is not in LD_LIBRARY_PATH. I see two ways to cure this. Both are unsatisfactory to me. The first way is to use "ld" instead of "gcc" or "cc" for linking, thus changing the lines 'linker_so' : ["cc", "-shared"], 'linker_exe' : ["cc"], to 'linker_so' : ["ld", "-shared"], 'linker_exe' : ["ld"], but, as I understand it the value for 'linker_so' is eventually taken from the LDSHARED value in the python Makefile stored in python?.?/config/Makefile and that value had to be changed. So the changed value will be overwritten from there. Second gcc has ha flag for options to be given to the linker, thus "-Wl,-R/usr/local/BerkeleyDB.3.1/lib" gives no warning and the wanted effect, but certainly only when you use gcc, other compiler will give an error with this. Is there an other way to solve this than to ask everyone to change python?.?/config/Makefile ? Greetings Berthold -- bhoel(a)starship.python.net / http://starship.python.net/crew/bhoel/ It is unlawful to use this email address for unsolicited ads (USC Title 47 Sec.227). I will assess a US$500 charge for reviewing and deleting each unsolicited ad.

1 0