Mailman 3 July 2001 - Distutils-SIG

Seeking technical reviewers for Python
by Kristen_Blanco＠prenhall.com 24 Jul '01

24 Jul '01

am writing from Prentice Hall publishing and I am seeking reviewers for an upcoming publication. We are one of the largest college textbook publishers in the US. We are publishing a book entitled "Python How to Program" by Harvey and Paul Deitel. They are premier programming language authors, with the best-selling C++ and Java books in the college market place. More information on their suite of publications can be found here: http://www.prenhall.com/deitel We are presently seeking qualified technical reviewers to verify that the Deitels' coverage of Python in their forthcoming book is accurate. In return, we are offering a token honorarium. Might you be willing to participate? If not, could you perhaps suggest a colleague? If you are interested, or have any questions, please contact my colleague, Crissy Statuto, at Crissy_Statuto(a)prenhall.com Thank you in advance for your assistance and consideration. Sincerely, Crissy Statuto Crissy Statuto Project Manager, Computer Science Prentice Hall One Lake Street- #3F54 Upper Saddle River, NJ 07458 Tel: 201-236-6695 Fax: 201-236-7170 Email: crissy_statuto(a)prenhall.com

1 0

directory for applications
by Gernot Jander 23 Jul '01

23 Jul '01

Is there any known "standard" directory location for complete applications written in Python? I think a typical internationalized application contains at least four different file types: 1. an executable (Python-)script 2. one or more (Python-)modules (specific for this application) 3. one or more subdirectories containing xxx.mo files for gettext 4. a xxx.rc file for gtk-styles Splitting this up in /usr/local/bin, /usr/python/site-packages and /usr/local/share makes it hard to de-install the application. Furthermore I think that application-specific modules should not go to site-packages, since this is the place for global packages/modules and not for modules that are only meaningfull in the context of one application. Therefore I decided to have one /usr/local/python/xxx directory for each of my applications and install all files there (besides the scripts). But this solution implies to have these directories in the $PYTHONPATH and makes the distribution not portable to other OS. Are there any intentions to improve the support of applications in distutils? Asks Gernot

2 1

The build_ext command has -I and -L options, why don't others???
by Paul Moore 23 Jul '01

23 Jul '01

I think this may be a fundamental flaw in the way distutils works... (By that, I mean that it's something that "obviously" should work, but it's going to be massively hard to implement that way :-() Let me know what you think. I'm still hacking on my PIL setup.py file. My latest thought was that, given that PIL needs the ZLib, JPEG and TCL libraries, and given that I don't have a "standard" location to look in for them, what command line options might help. And sure enough, I found the -I and -L options for the build_ext command. Great! That's just what I want. Obviously (!), I'm using the "build" command, but that executed build_ext behind the scenes for my extensions, so I'll just pass -I and -L, and let build pass them on to build_ext as needed. Oh, no... The build command complains that the -I and -L options are invalid. Grr. I guess this means that I either have to run build_ext first, by hand, and then run build (this works - well, it would if the build_clib command had an -I option, but that's a separate issue...), or I have to write a custom build command which accepts -I and -L and passes them onto build_ext. Surely, commands (like build and install) which run other commands should take *all* of the options allowed by any of their sub-commands, and pass them onto those sub-commands. As I say, this seems obvious to me, but (having looked briefly at the internals) annoyingly hard to do, in practice. As I say, any thoughts? Paul. PS I think this pretty much kills any hope of writing a setup.py which usefully implements command line argument extensions. You'd have to write custom commands for every command, from the lowest one which needs the extra option, all the way up the tree (install must be customised to take the same options as the custom build, which must be customised to take the same options as the custom build_ext, ...)

2 3

Distutils question
by Sophie Coon 18 Jul '01

18 Jul '01

I want to distribute the library wrapped (gle) for Python for different platform. We already have the .py and glecmodule.so for the different platform can I use the following setup.py and MANIFEST.in and then do a python2.0 setup.py sdist instead of listing the c extensions and c files to build the libraries gle/ setup.py: #!/usr/bin/env python2.0 from distutils.core import setup, Extension import os, sys, string setup( name = "gle", version = "3.0", description = "GLE Tubing and Extrusion Library", author = "Linas Vepstas", author_email = "linas(a)linas.org", url = "http://www.linas.org/gle/", packages = [''] ) MANIFEST.in include glecmodule.so Any information or suggestions will be great ! Thanks Sophie -- ############################################################ Sophie COON The Scripps Research Institute Research Programmer III Molecular Graphics Laboratory 10550 North Torrey Pines Road Phone: (858) 784-9556 La Jolla, CA 92037 Fax : (858) 784-2860 ############################################################

1 0

RE: [Distutils] Re: [Python-Dev] PEP 250: Summary of comments
by Moore, Paul 18 Jul '01

18 Jul '01

From: M.-A. Lemburg [mailto:mal@lemburg.com] > (I believe that install.py would have to be told about > sys.extinstallpath too and that it should fallback to the > defaults given in the install schemes if it is not set.) Hmm, browsing this a bit more, I'm getting further confused. The cause of this is the INSTALL_SCHEMES stuff, which has a purelib/platlib distinction, which is only used on unix_prefix (all other cases use the same value for both of these). I can't see how sys.extinstallpath relates - I could use it as default for both purelib and platlib, but that somewhat defeats the point of having the two. Does this imply that sys.extinstallpath should be split into two parts (pure & plat)? I can't comment, as this is a Unix-only thing. This is getting silly. I feel that the correct approach is to go back to my original stance, of *only* changing Windows behaviour - leave the Unix and Mac camps as they are. With that in mind, sys.extinstallpath seems like an overgeneralisation, and the attached patch does everything bar handle bdist_wininst. The Windows Installer should then do the same thing - load Python, and generate os.path.join(sys.prefix, "lib", "site-packages") as the destination directory. OK, so the same thing is hard-coded in four places, but this whole area is rife with duplicated code, and fixing that issue is way outside the scope of PEP 250. For the limited purpose of making site-packages appear in sys.path, and making python setup.py install install to site-packages, the attached patch works. I've only tested it on a simple Python module, but that's all I have to hand. I can try some C modules tonight when I get home, but I see no reason why they wouldn't work as well. The patch is pretty much trivial, which (IMHO) is very much in its favour as Python 2.2a1 is already out... Unless someone comes up with a *very strong* argument as to why I should be going further than this, I would like to request that this goes into Python as it stands. If someone can supply the source of the bdist_wininst installer, I will make a corresponding change to that. I will NOT make any changes which affect Unix, or Mac platforms. I don't know the issues. If someone wants to supply a patch which does this, I'll be happy to see it go in, and I am quite comfortable with it going under the banner of PEP 250, but I will not get involved in the issues - I simply am not qualified to comment. Paul. ------------------------------------------------------ diff -u site.py.orig site.py --- site.py.orig Tue Jun 26 10:07:06 2001 +++ site.py Wed Jul 18 16:33:37 2001 @@ -143,7 +143,7 @@ elif os.sep == ':': sitedirs = [makepath(prefix, "lib", "site-packages")] else: - sitedirs = [prefix] + sitedirs = [prefix, makepath(prefix, "lib", "site-packages")] for sitedir in sitedirs: if os.path.isdir(sitedir): addsitedir(sitedir) diff -u distutils\sysconfig.py.orig distutils\sysconfig.py --- distutils\sysconfig.py.orig Thu Apr 19 10:24:24 2001 +++ distutils\sysconfig.py Wed Jul 18 16:20:20 2001 @@ -87,7 +87,7 @@ elif os.name == "nt": if standard_lib: - return os.path.join(PREFIX, "Lib") + return os.path.join(PREFIX, "Lib", "site-packages") else: return prefix diff -u distutils\command\install.py.orig distutils\command\install.py --- distutils\command\install.py.orig Thu Apr 19 10:24:24 2001 +++ distutils\command\install.py Wed Jul 18 16:29:29 2001 @@ -31,8 +31,8 @@ 'data' : '$base', }, 'nt': { - 'purelib': '$base', - 'platlib': '$base', + 'purelib': '$base/Lib/site-packages', + 'platlib': '$base/Lib/site-packages', 'headers': '$base/Include/$dist_name', 'scripts': '$base/Scripts', 'data' : '$base',

3 2

PEP 250: Summary of comments
by Moore, Paul 18 Jul '01

18 Jul '01

Having waited a few days to let the dust settle, I believe that the following is the current state of affairs: 1. The change to site.py, to include site-packages in sys.path, is in. 2. The change to distutils.sysconfig to change to site-packages, is in. 3. The Windows Installer still needs changes: a) site.py should change to export a "sys.extinstallpath" which points to site-packages b) the Windows Installer should use this, rather than the registry I see no great issue with 3a - it should be a pretty trivial change. Can someone with access to the sources make it? I attach a suggested patch. Item 3b is the key point - it's pretty critical that the Windows Installer change to use the new directory, otherwise, most of the work is a waste. I can't do much about this, as I haven't even seen the source yet. Can someone do something about this? On point 3a, sys.extinstallpath should be set for all platforms, but I have to admit that I don't know what to do for non-Windows platforms. The best I can suggest is that we do something like if os.sep == '/': sys.extinstallpath = os.path.join(sys.prefix, "lib", "python" + sys.version[:3], "site-packages") else: sys.extinstallpath = os.path.join(sys.prefix, "lib", "site-packages") which matches the sys.path setting for Unix - but I couldn't really offer this as a patch, as I don't understand the issues around site-packages vs site-python on Unix. All I could say is that it's better than leaving sys.extinstallpath unset on some platforms. To summarise the summary: 1. The patch to site.py to expose sys.extinstallpath should be made, at least for Windows. 2. The Windows Installer needs to be updated to use sys.extinstallpath for Python 2.2 and greater. 3. If the Mac people want, the same can be done for Mac. 4. If the Unix people have a consensus, that should go in too (affects site.py, bdist_rpm, at least). As a side benefit, if this goes in, bdist_wininst will start working for Pythons which don't use the registry (such as the PythonLabs ones). Sadly, I note that I've just missed the 2.2a1 release. Is anyone likely to be able to do anything about this prior to 2.2a2? (If someone sends me a pointer to the wininst sources, I'll look into what's involved in a patch, assuming no-one else has adequate time). Paul. PS As documentation of sys.extinstallpath, I'd suggest something like: sys.extinstallpath: The directory into which Python extensions should be installed. This is merely a recommendation - Python will pick up extensions which are located anywhere along sys.path. However, extension installers should use this directory by default. The distutils package (and installers built with it) will use this directory (XXX - currently not trie for bdist_rpm, I guess). Patch for site.py (point 3a - Windows only, from Thomas Heller) --- \Applications\Python\lib\site.py.orig Tue Jun 26 10:07:06 2001 +++ \Applications\Python\lib\site.py Wed Jul 18 14:43:54 2001 @@ -148,6 +148,9 @@ if os.path.isdir(sitedir): addsitedir(sitedir) +if os.sep == '\\': # != '/' if you want to do all except Unix like this... + sys.extinstallpath = os.path.join(sys.prefix, "lib", "site-packages") + # Define new built-ins 'quit' and 'exit'. # These are simply strings that display a hint on how to exit. if os.sep == ':':

3 4

RE: [Distutils] Re: [Python-Dev] PEP 250: Summary of comments
by Moore, Paul 18 Jul '01

18 Jul '01

From: Thomas Heller [mailto:thomas.heller@ion-tof.com] >> 2. I can't do anything about bdist_wininst. I hope Thomas is listening, and >> can offer some help here :-) > I am, and I bdist_wininst *will* be ready when the rest is complete. > > I already have a version which retrieves sys.extinstallpath at run time. Great! Thanks for that. In recent messages, I've reached the point where I can no longer see much use for sys.extinstallpath, unless the scope is extended to Unix and Mac. In the absence of a suitable Unix/Mac expert, that's not likely to happen... Would you be willing to change to os.path.join(sys.prefix, "lib", "site-packages") (as this is windows-specific, just sys.prefix + "\\lib\\site-packages" would be fine)? Alternatively, I could expose sys.extinstallpath from site.py solely for NT - I don't mind either way. > I already posted the link to access Python's cvs repository via http. > (Can easily be found starting at http://python.sourceforge.net/). > This is implemented via Greg Stein's ViewCVS - check it out, it is > very powerful! I *thought* you had, but I couldn't find the message. Sorry - I'll have a look later on. Sorry - must rush now - hope this mail makes sense... Paul.

1 0

RE: [Distutils] Re: [Python-Dev] PEP 250: Summary of comments
by Moore, Paul 18 Jul '01

18 Jul '01

From: M.-A. Lemburg [mailto:mal@lemburg.com] > Paul, note that your patches don't even touch install.py -- are > your sure that the patch to sysconfig.py suffices to have distutils > install the extensions into site-packages on Windows ? AAARGH! I guess you are right - I missed this somehow. I thought I'd checked, but the grep I did must have missed this one. I will attempt to sort out a patch which actually works. However, I should point out: 1. I've never looked at the internals code for distutils in any detail, so I can't be sure that what I'm doing is complete. Reviewers would be gratefully received (Marc, both your comments and Thomas' have been very helpful in this regard so far). 2. I can't do anything about bdist_wininst. I hope Thomas is listening, and can offer some help here :-) 3. I'll have to do a patch against Python 2.1. I don't have 2.2a1 installed yet. 4. I don't have a particularly large testbed of applications to check all this - I'm working on my main machine, and I'll be in trouble if I break the setup by indiscriminate hacking... 5. I'm totally reliant on someone else feeding my changes into the current CVS. This is frustrating, to say the least. (I understand that people are very busy - but I've already sent in one complete patch, along with the final PEP, which didn't get applied - instead, the site.py part was reimplemented, and the sysconfig.py part missed completely :-() Anyway, I'll report back ASAP... Paul.

2 1

RE: [Python-Dev] PEP 250: Summary of comments
by Moore, Paul 18 Jul '01

18 Jul '01

From: Fred L. Drake, Jr. [mailto:fdrake@acm.org] > There's one aspect that doesn't appear to have been addressed for > Unix: there are two reasonable values for extinstallpath. In > multi-architecture installations, where the Python portions of the > library are shared among architectures, there are two site-packages > directories: I agree entirely. I have no knowledge of the issues on Unix, and hence I can make no comment. As far as I am concerned, I feel very strongly that this should go in for Windows (where I believe that the current use of a "bare" sys.prefix is wrong), but I have no view at all on other platforms. My PEP was originally entitled "Using site-packages on Windows" - I am happy if the scope gets extended, but don't rely on me to do it - and please don't let the key point (for me) which is fixing Windows, get sidetracked by issues for Unix, where (as far as I know) the current status quo is perfectly acceptable to the majority of users. So I'd vote to leave sys.extinstallpath undefined except on Windows, and leave the other platforms for a new PEP. Paul.

1 0

RE: [Distutils] PEP 250: Summary of comments
by Moore, Paul 18 Jul '01

18 Jul '01

From: Moore, Paul [mailto:Paul.Moore@atosorigin.com] > Having waited a few days to let the dust settle, I believe that the > following is the current state of affairs: > > 1. The change to site.py, to include site-packages in sys.path, is in. > 2. The change to distutils.sysconfig to change to site-packages, is in. Urk. I just downloaded 2.2a1, and the sysconfig.py change isn't in :-( Attached are a couple of possible patches - one which just tweaks Windows (os.name is 'nt' for Win9x???) and another which tries to slim down on the platform-specific special casing, but which may have larger effects on non-Windows platforms. Can someone put one of these in? Thanks, Paul. Trivial version of the change: --- sysconfig.py.orig Sat Jul 07 23:55:28 2001 +++ sysconfig.py Wed Jul 18 15:13:45 2001 @@ -89,7 +89,7 @@ if standard_lib: return os.path.join(PREFIX, "Lib") else: - return prefix + return os.path.join(libpython, "site-packages") elif os.name == "mac": if plat_specific: Better version, which covers all platforms (but which removes a couple of "OK, where DO site-specific modules go on the Mac?" errors, which may be glossing over an issue...) is --- sysconfig.py.orig Sat Jul 07 23:55:28 2001 +++ sysconfig.py Wed Jul 18 15:18:55 2001 @@ -80,35 +80,23 @@ if os.name == "posix": libpython = os.path.join(prefix, "lib", "python" + sys.version[:3]) - if standard_lib: - return libpython - else: - return os.path.join(libpython, "site-packages") - elif os.name == "nt": - if standard_lib: - return os.path.join(PREFIX, "Lib") - else: - return prefix - + libpython = os.path.join(PREFIX, "Lib") elif os.name == "mac": if plat_specific: - if standard_lib: - return os.path.join(EXEC_PREFIX, "Mac", "Plugins") - else: - raise DistutilsPlatformError, \ - "OK, where DO site-specific extensions go on the Mac?" + libpython = os.path.join(EXEC_PREFIX, "Mac", "Plugins") else: - if standard_lib: - return os.path.join(PREFIX, "Lib") - else: - raise DistutilsPlatformError, \ - "OK, where DO site-specific modules go on the Mac?" + libpython = os.path.join(PREFIX, "Lib") else: raise DistutilsPlatformError, \ ("I don't know where Python installs its library " + "on platform '%s'") % os.name + if standard_lib: + return libpython + else: + return os.path.join(libpython, "site-packages") + # get_python_lib()

2 1