Distutils-SIG February 2000

distutils-sig@python.org

23 participants
41 discussions

a patch :)
by est＠hyperreal.org 23 Feb '00

23 Feb '00

Could be old news, but I had to do this to 0.1.3: --- command/dist.py~ Sun Jan 30 15:22:27 2000 +++ command/dist.py Wed Feb 23 13:07:50 2000 @@ -223,6 +223,7 @@ for fn in standards: if type (fn) is TupleType: alts = fn + got_it = 0 for fn in alts: if os.path.exists (fn): got_it = 1

1 0

Distutils patch for Python 1.5.1 compatibility
by Greg Ward 23 Feb '00

23 Feb '00

Hi all -- I've gotten enough bug reports about Distutils not working with Python 1.5.1 to finally do something about it. The result is the attached patch. Please give it a try, *especially* if you use Python 1.5.1 (or older, I suppose). If this works, I'll release it as Distutils 0.1.4 so the NumPy and PyFort people can get their jobs done without stumbling across my bugs. Oh yeah, you can apply the patch either in the Distutils source tree (right in Distutils-0.1.3, wherever you put it) or to the installed version, from the "site-packages" directory. In either case, apply it with "patch -p1". Eg. if Distutils is in the normal Unix "site-packages", and the patch is in /tmp/distutils-apply.patch, you can patch the installed version of Distutils with: cd /usr/local/lib/python1.5/site-packages patch -p1 < /tmp/distutils-apply.patch And then try building and installing NumPy, or PIL (using the setup script included with Distutils, that is), or something generated by PyFort -- whatever. Please let me know how it goes! Thanks -- Greg

1 0

Re: [Distutils] Installing Numeric and linking against libpython.so
by Greg Ward 23 Feb '00

23 Feb '00

[Jim -- this discussion deserves to be on the SIG, so I'm cc'ing it there; SIG readers -- if it looks like you missed a message you did: Jim replied straight to me, but I'm putting this thread back on the SIG.] [Jim Preston explains the problem in the first place] > I am trying to upgrade my Numeric, etc. and I seem to have a problem > with the distutils > setup. Everything appears to install fine, but when I import Numeric, I > get the message > >> import Numeric > Traceback (innermost last): > File "<stdin>", line 1, in ? > File "/usr/local/lib/python1.5/site-packages/Numeric/Numeric.py", line > 76, in ? > import multiarray > ImportError: ld.so.1: python: fatal: relocation error: file > /usr/local/lib/python1.5/site-packages/Numeric/multiarray.so: symbol > PySequence_Length: referenced symbol not found > >>> [me, bugging Jim for more info] > What version of Python is this? (And, for that matter, which version of > NumPy and Distutils? I'm assuming 15.2 and 0.1.3, but you never > know...) [Jim coughs up the details] > Python 1.5.2 (latest, AFAIK), Numeric 15.2 and distutils 0.1.3. [me again] > How was your libpython built -- is it static or dynamic? (Look in > $exec_prefix/lib/python1.5/config to see -- you're looking for either > libpython.a or libpython.so.) [Jim] > RIght. The default configuration was static, I tried to create libpython.so > to try and get the linkage going, but no luck there either. [me] > Are you able to import any of the other NumPy extensions? Have you run > "ldd" on them to see if they were linked against libpython.so (assuming > libpython.so is what you have)? Have you run "nm" or "elfdump" to see > if you can figure out what symbols from libpython are expected by the > extensions? [Jim] > ldd tells me nothing, and nm tells me that all of the Py_Sequence functions, > etc. are undefined. [me] > Did you try hacking (just cut and paste it) the link command used by > Distutils to create multiarray.so to add > "-L$exec_prefix/lib/python1.5/config -lpython" (where $exec_prefix is > Python's prefix, which of course you'll have to interpolate manually), > and see if that helps? [Jim] > No change. OK, I'm stumped. For those of you who missed his first post, Jim is running Solaris 2.6, which just happens to be what I have on my machine it work. So I can swear up and down that NumPy 15.2 builds just fine with Distutils 0.1.3 under Python 1.5.2 and Solaris 2.6. Something seems to be wrong with multiarray.so (one of the extensions installed by NumPy into site-packages/Numeric) in Jim's installation, but for me it works great: $ python Python 1.5.2 (#3, Feb 11 2000, 11:14:47) [GCC 2.8.1] on sunos5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import multiarray >>> dir(multiarray) ['__doc__', '__file__', '__name__', '__version__', 'argmax', 'argsort', 'array', 'arraytype', 'binarysearch', 'choose', 'concatenate', 'cross_correlate', 'error', 'fromstring', 'innerproduct', 'repeat', 'reshape', 'set_string_function', 'sort', 'take', 'transpose', 'zeros'] See, It Just Works (TM). IOW, when my Python loads multiarray.so, it has no problem finding symbols like 'PySequence_Length'. My copy of multiarray.so can't possibly have all of Python linked into it -- it's only 70k. Anyways, that would be so colossally stupid it doesn't even bear thinking about. "ldd" on multiarray.so reveals that it depends on the usual Solaris shared libs -- libc.so, libelf.so, libucb.so, libsocket.so, and a few others. All of them are in /usr/lib, so clearly nothing Python-specific there. But Jim gives one last hint: > I keep thinking that there is some old stuff hanging around from earlier > python releases that is messing this up, but I have tried quite hard to get > rid of everything and build a clean installation and I am still having the same > problems. OK, have you done this, with a *fresh*, brand-new Python 1.5.2 source distribution: $ ./configure --prefix=/tmp/usr $ [edit the setup file, you know the routine] $ make $ make install and then unpack a fresh, brand-new Distutils 0.1.3 distribution and install it to the temporary Python installation $ /tmp/usr/bin/python setup.py install # in the Distutils-0.1.3 directory and then unpack a fresh, brand-new NumPy 15.2 distribution ... $ /tmp/usr/bin/python setup.py install # in the Numerical-15.2 directory I hate having to say "reinstall Python is the answer", but I don't see what could be going wrong except for a screwed up Python installation. Please try this and let us know if it works or not -- either way, I want to know! [Jim asks a relevant question] > I am not a shared library guru, by any means. How is it supposed to > work, anyway? Is it the case that the multiarraymodule should be able > to call functions from libpython,a without any such linkage specified, > simply because it is being imported into a program (python) that is > already linked against libpython.a ? I must say that I am quite > baffled here. That's a good question, and I'm not entirely sure how it works. My gut-level intuition says that because python is the binary that's loading multiarray.so into memory, all of Python's API is already there -- so it Just Works. (At least, for *me* it just works.) IOW, if you tried to load multiarray.so into Tcl or Perl or another C program, then of course it wouldn't be able to resolve PySequence_Length. But I would like to hear from someone who really understands this what's going on. I'm curious too! > I am using gcc-2.95.1, with gnu ld, etc. Hmmm, there's one difference between Jim's setup and mine -- we're still using gcc 2.8.1, and the linker used to create shared objects is the system linker -- /usr/ucb/ld. This comes from Python's Makefile, ie. this is how Python was built, so that's how Distutils builds extensions. If you build Python with a different linker, Distutils will build extensions with that linker. This might be key! Greg -- Greg Ward - all-purpose geek gward(a)python.net http://starship.python.net/~gward/ God is real, unless declared integer.

1 0

problem compiling on SGI IRIX 6.5
by Mona Wong 18 Feb '00

18 Feb '00

Hi: I've just downloaded Distutils 0.1.3 and tried to compile it on my SGI IRIX 6.5 machine. I am using python 1.5.1. Here is what I got: proteus:/p12/src/Distutils-0.1.3) python setup.py install running install Traceback (innermost last): File "setup.py", line 13, in ? setup (name = "Distutils", File "distutils/core.py", line 102, in setup raise SystemExit, "error: %s: %s" % (exc.filename, exc.strerror) AttributeError: filename What is wrong and how do I fix this? By the way, I just signed up to be on this list but have not received the confirmation yet so please reply back to me personally to be sure that I am getting your responses. Cheers, Mona

3 2

Installing Numeric and linking against libpython.so
by James L. Preston 18 Feb '00

18 Feb '00

I am trying to upgrade my Numeric, etc. and I seem to have a problem with the distutils setup. Everything appears to install fine, but when I import Numeric, I get the message >> import Numeric Traceback (innermost last): File "<stdin>", line 1, in ? File "/usr/local/lib/python1.5/site-packages/Numeric/Numeric.py", line 76, in ? import multiarray ImportError: ld.so.1: python: fatal: relocation error: file /usr/local/lib/python1.5/site-packages/Numeric/multiarray.so: symbol PySequence_Length: referenced symbol not found >>> I believe that this is the result of the multiarray.so, not being linked with libpython. I have tried to understand where I can put this option in the setup.py, etc. but I must say that I am a bit baffled. I am using Solaris 2.6 on an UltraSPARC. I am not a big shared library expert, but I think that on Solaris, it is necessary to tell one .so file where to find the functions from the other .so files that it may be calling, if you know what I mean. Any suggestions. Thanks, jim preston jlp(a)apti.com

2 1

Second incompatible change: command class renamings
by Greg Ward 18 Feb '00

18 Feb '00

Hi again -- I've just commited a second batch of incompatible changes: the renaming of several command class attributes and methods has been done. In particular, options -> user_options (class attribute) set_default_options -> initialize_options (method) set_final_options -> finalize_options (method) I posted on Jan 30 about the reasons for doing this -- if you're curious, I'm sure you can find it in the archive. This will break code for anyone who's defining their own command classes, in particular the NumPy setup script. This just means NumPy users should stick to 0.1.3 and not follow the CVS archive, which I expect they're doing anyways. It does mean the NumPy setup script will have to change for the next release of Distutils -- which I expect to be 1.0, bundled with Python 1.6. Oh, I also bumped the version in the Distutils setup script to "0.2pre", not because there is going to be a release 0.2, but just to remind everyone of these incompatible changes. And I cvs-tagged the pre-incompatibility versions of everything with "pre_0_2_breakage", just in case you need the absolute latest code that still works with the NumPy setup script. Greg

1 0

First incompatible change: 'dist' command is dead
by Greg Ward 18 Feb '00

18 Feb '00

Hi all -- those of you following the Distutils CVS archive should be aware that I've just checked in the first incompatible change: the "dist" command is gone, replaced by "sdist". Like its predecessor, "sdist" exists to create a source distribution. Unlike its predecessor, there is now an explicit MANIFEST.in/MANIFEST model, meaning that any developers using Distutils now will have to rewrite their old MANIFEST files as a MANIFEST.in. The syntax is *not* what I proposed last week here; I think I've come up with something better. (I.e. I implemented my proposal and it didn't work too well. I came up with something new and a went ahead and implemented before I could poke any holes in it.) For now, the documentation for the MANIFEST.in (aka "the manifest template file") syntax consist of the Distutils manifest template and the 'read_template()' method in distutils/command/sdist.py. Use the Source, Luke! ...but seriously, I'll try to write up some notes on how to write manifest templates soon. It's a small but important feature. Greg

1 0

Re: Where is Imaging.h?
by Greg Ward 17 Feb '00

17 Feb '00

On 17 February 2000, Fred L. Drake, Jr. said: > I think this would be a good idea. There are actually two Python > include directories, one for $prefix and one for $exec_prefix; > extensions would have to "do the right thing" here. > Greg Ward, I presume you've already thought of this and distutils > supports it? ;-) Nope, I blew it and had to hack the NumPy setup script to install the C header files. Both Paul Dubois and I totally forgot about this need until a NumPy user pointed it out. Oops. You're right though, Distutils will need to be fixed to do this properly for other module distributions than NumPy. For the record, NumPy installs its headers to $prefix/include/python1.5/Numeric which I guess makes sense. C headers aren't normally platform-specific, except for those of the config.h variety -- and in fact I believe that most of Python's own headers are installed to $prefix, but config.h goes to $exec_prefix. Wouldn't it be nice if we could just say "Extensions with platform-specific headers are EVIL". Doubt it though. ;-( Greg

1 0

install w/o build, spaces in directory names
by David Ascher 16 Feb '00

16 Feb '00

1) While building Numerical on NT last night w/ the latest Distutils CVS snapshot, I found that it refused to build because the directory names had spaces in them which were not quoted. In other words, the code being executed when something like cl .... -IC:\Program Files\Python... which broke. To correct it, I patched ccompiler.py and msvccompiler.py to turn those kinds of things into cl .... -I"C:\Program Files\Python..." ... but Greg says that that will break on Unix -- I'm surprised by that claim, since IIRC quoting directory names is OK, but it's been a while since I really dealt with Unix seriously. FWIW, so far the problem occurs for -I in ccompiler.py and /LIBPATH in msvccompiler.py. The patch is listed at the end of this message. 2) Again on Windows, if I do python setup.py build python setup.py install the second line causes the code to be recompiled, even though the compilation is not needed. Greg assures me that that is not the case on Unix. Anyone have the time to tackle this? --david ascher diff -c distutils//distutils/ccompiler.py C:\Program Files\Python\distutils/ccompiler.py *** distutils//distutils/ccompiler.py Wed Feb 09 02:16:14 2000 --- C:\Program Files\Python\distutils/ccompiler.py Thu Feb 10 19:44:37 2000 *************** *** 563,569 **** pp_opts.append ("-D%s=%s" % macro) for dir in include_dirs: ! pp_opts.append ("-I%s" % dir) return pp_opts --- 563,569 ---- pp_opts.append ("-D%s=%s" % macro) for dir in include_dirs: ! pp_opts.append ('-I"%s"' % dir) return pp_opts diff -c distutils//distutils/msvccompiler.py C:\Program Files\Python\distutils/msvccompiler.py *** distutils//distutils/msvccompiler.py Thu Feb 10 02:52:42 2000 --- C:\Program Files\Python\distutils/msvccompiler.py Thu Feb 10 19:45:05 2000 *************** *** 369,375 **** def library_dir_option (self, dir): ! return "/LIBPATH:" + dir def library_option (self, lib): return self.library_filename (lib) --- 369,375 ---- def library_dir_option (self, dir): ! return "/LIBPATH:" + '"' + dir + '"' def library_option (self, lib): return self.library_filename (lib)

4 9

Should I rename the "dist" command?
by Greg Ward 15 Feb '00

15 Feb '00

Hi all -- it just occurred to me that since the "dist" command is only responsible for creating *source* distributions, it might make sense to rename it to "sdist". That way, there would be two parallel, complementary "make a distribution" commands: sdist - make a source distribution bdist - make a "built" distribution (frequently a binary distribution) (Well, there will be two parallel, complementary commands anyways: this just makes the names consistent.) Also, it reduces the small potential for confusion between the Distribution class and the Dist class. (The former is the grand central station of the Distutils; the latter is just another command class that happens to implement the "dist" command.) I don't see any big problems with this; it just means that the developers currently using Distutils would have to adjust their habits by one letter. Any thoughts? Greg -- Greg Ward - Linux weenie gward(a)python.net http://starship.python.net/~gward/ No problem is so formidable that you can't just walk away from it.

2 1

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG February 2000