Hi all,
I've been aware that the distutils sig has been simmerring away, but
until recently it has not been directly relevant to what I do.
I like the look of the proposed api, but have one question. Will this
support an installed system that has multiple versions of the same
package installed simultaneously? If not, then this would seem to be a
significant limitation, especially when dependencies between packages
are considered.
Assuming it does, then how will this be achieved? I am presently
managing this with a messy arrangement of symlinks. A package is
installed with its version number in it's name, and a separate
directory is created for an application with links from the
unversioned package name to the versioned one. Then I just set the
pythonpath to this directory.
A sample of what the directory looks like is shown below.
I'm sure there is a better solution that this, and I'm not sure that
this would work under windows anyway (does windows have symlinks?).
So, has this SIG considered such versioning issues yet?
Cheers,
Tim
--------------------------------------------------------------
Tim Docker timd(a)macquarie.com.au
Quantative Applications Division
Macquarie Bank
--------------------------------------------------------------
qad16:qad $ ls -l lib/python/
total 110
drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1
-r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so
drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1
drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1
dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8
drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1
dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1
dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3
drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7
dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1
qad16:qad $ ls -l lib/python/1.1
total 30
lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so
lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so
lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so
lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so
lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so
lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so
lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1
lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8
lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1
lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1
lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so
lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so
lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3
lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7
lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1
Here's a patch that seems to get sysconfig minimally working under NT for
me. Has anyone else done better?
I noticed that distutils installs itself under 'lib\site-packages' which
does not seem to be a convention under NT, at least the directory does not
exist by default, and is not in the Python path by default. It seems that
Windows packages normally go in the top level Python directory--at least
that's where the win32 packages install themselves, and it *is* in the
Python path by default...
Anyway here's the simple-minded patch.
-Amos
RCS file: /projects/cvsroot/distutils/distutils/sysconfig.py,v
retrieving revision 1.5
diff -r1.5 sysconfig.py
130a131,139
> def _init_nt():
> """Initialize the module as appropriate for NT"""
> g=globals()
> # load config.h, though I don't know how useful this is
> parse_config_h(open(
> os.path.join(sys.exec_prefix, "include", "config.h")), g)
> # set basic install directories
> g['LIBDEST']=os.path.join(sys.exec_prefix, "Lib")
> g['BINLIBDEST']=os.path.join(sys.exec_prefix, "Lib")
132c141
<
---
>
141a151
> del _init_nt
As I've been going on about, I've got the changes made so that
GNU automake supports Python. Because it is the FSF, I had to
sign paperwork to disclaim copyright interest in our contributions
before they would accept it. I mailed them in today.
Thus, I've been touching up my changes before submitting them.
I wanted to make sure I was using a sensible (or at least
reasonable) set of names before I did that.
Here are the names for which I am uncertain:
o PYTHON_SITE
This is the value
sys.prefix + "/python" + sys.version[:3] + "/site-packages"
It is the location where platform independent modules should
be installed. It can be overridden on the configure line with
the parameter --with-python-site=DIR
o PYTHON_SITE_PACKAGE
automake defines the variable PACKAGE, which I am taking also
as the module name (nothing unusual about that, I hope).
o PYTHON_SITE_EXEC
This is the value
sys.exec_prefix + "/python" + sys.version[:3] + "/site-packages"
It can be overridded on the configure line with the parameter
--with-python-site-exec=DIR.
I played around with a few names. These seem to make sense to
me (eg, PYTHON_SITE_EXEC is better than PYTHON_SITE_SHARED which I
had). I cannot find if the standard Python libraries refer to
these directories in some other fashion. The closest I can find
is "site.py" which combines the first and last directory names
directly into "sitedirs".
*** question 1: Do these names make sense to others?
o I added a new m4 macro for "aclocal" called "AM_CHECK_PYTHON".
This is the command which goes in "configure.in" to tell autoconf
that it needs to generate Python information.
The macro takes one optional parameter. If the value is "classes"
the Python files in the package are installed directly in
PYTHON_SITE. If the value is "module" then they are installed
into PYTHON_SITE_PACKAGE. The installation directory name is
stored in the variable PYTHON_SITE_INSTALL.
Eg:
If you have the file "Vector.py" in the autoconf package "vecmath"
with the PYTHON_SITE set to /usr/local/lib/python1.5/site-packages
AM_CHECK_PYTHON(module) puts it in
/usr/local/lib/python1.5/site-packages/vecmath/Vector.py
AM_CHECK_PYTHON(classes) puts it in
/usr/local/lib/python1.5/site-packages/Vector.py
*** question 2: Should it be "classes" or something else?
Since there's no existing python command-line program to compile
a *single* python program (as compared to compileall which
compiles, well, all :), I'm using the following in the Makefile
(the value of PYTHON is the full location of the python executable
and was found during the configure step):
PYTHON_COMPILE_CMD = "-c 'import py_compile, sys; \
py_compile.compile(sys.argv[1])'
PYTHON_OCOMPILE = $(PYTHON) $(PYTHON_COMPILE_CMD)
PYTHON_OCOMPILE = $(PYTHON) -c $(PYTHON_COMPILE_CMD)
.py.pyc:
@echo "Compiling python file $< "
@$(PYTHON_COMPILE) $<
.py.pyo:
@echo "Compiling python file $< (optimized)"
@$(PYTHON_OCOMPILE) $<
The idea is that I hope distutils will have a command-line way to
do this, so I can replace all that with:
.py.pyc:
@$(PYTHON) $pythondir/distutils/compile.pr $<
.py.pyo:
@$(PYTHON) -O $pythondir/distutils/compile.py $<
(I hide the actual compile with @ because otherwise you just see
the full "import .... argv[1])" which is hard to read. By having
a program to byte compile a file directly I can get rid of the noise.)
*** question 3: Can distutils provide this sort of service?
*** question 4: Anyone interested in getting the latest CVS
distribution of automake (after I get the changes in) to play^U
test things out?
To help out, I'm also working on documentation so people can use
it without reading the full automake documentation (which has a
strong emphasis on building C programs). Not suprisingly, this
email will be a hefty part of said documentation :)
Andrew
dalke(a)bioreason.com
P.S.
Here's the list of other definitions created by configure.in.
I cannot see how there's any real problem with these names, but
who knows ....
PYTHON = full name of the python executable
PYTHON_PREFIX = sys.prefix
PYTHON_EXEC_PREFIX = sys.exec_prefix
PYTHON_VERSION = sys.version
PYTHON_PLATFORM = sys.platform
pythondir = sys.prefix + "/python" + sys.version[:3]
Hi all --
as promised, I have finally checked in some real Distutils code.
There's just enough functionality for the Distutils to build and install
itself. To try it out, you'll need to download the code from the
anonymous CVS archive at cvs.python.org; see
http://www.python.org/sigs/distutils-sig/cvs.html
for instructions.
If you just want to try it out, then from the top-level Distutils
directory (the one that has setup.py), run:
./setup.py build install
This will copy all the .py files to a mockup installation tree in
'build', compile them (to .pyc -- don't run setup.py with "python -O",
as I don't handle that yet and it will get confused), and then copy the
whole mockup tree to the 'site-packages' directory under your Python
library directory. Figuring the installation directory relies on Fred
Drake's 'sysconfig' module -- I don't recall what the status of
sysconfig was on non-Unix systems; hope someone out there can try it and
let us know! Come to think of it, I'd like to hear how it works on
*any* system apart from my Red Hat Linux 5.2, stock Python 1.5.1, home
PC. ;-)
If you want to build in a different directory, use the '--basedir'
option to the 'build' command; to install in a different place, you can
use either the '--prefix' or '--install-lib' options to the 'install'
command. Here are a couple of samples to demonstrate:
./setup.py build --basedir=/tmp/build install --prefix=/tmp/usr/local
or
./setup.py build --basedir=/tmp/build
./setup.py install --build-base=/tmp/build --prefix=/tmp/usr/local
Go crazy. No documentation yet -- please read the code! Start with
distutils/core.py which, as its name implies, is the start of
everything. I went nuts with docstrings in that module last night, so
hopefully there's just enough there that you can figure things out in
the absence of the "Distutils Implementation Notes" document that I'd
like to write.
Greg
--
Greg Ward - software developer gward(a)cnri.reston.va.us
Corporation for National Research Initiatives
1895 Preston White Drive voice: +1-703-620-8990 x287
Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913
Hi all --
I've just fixed the bug that Michel Sanner inadvertently pointed out --
the "-n" ("--dry-run") flag was universally ignored. This was solved by
adding several methods to the Command base class through which all
"external actions" (anything that affects the outside world, most
particularly the filesystem) should be channelled. Now the verbose and
dry-run flags can be respected in a consistently painless way by *all*
command classes.
In addition to "cvs update", you can now download a snapshot tarball:
http://www.python.org/sigs/distutils-sig/distutils-19990403.tar.gz
I make no promises about the longevity or reliability of that URL --
it'll probably go away the next time I make a snapshot. Let me know if
you like the snapshot, but would prefer a zip file to a tarball; under
duress, I could hack the 'mksnap' script to make both. ;-)
Greg
--
Greg Ward - software developer gward(a)cnri.reston.va.us
Corporation for National Research Initiatives
1895 Preston White Drive voice: +1-703-620-8990
Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913
Just to chime in about the unmentionable, I have a tool for connecting
Fortran and Python (as do others; mine isn't ready for the light of day yet)
and unfortunately the existing Setup scheme makes me compile my Fortran
separately in a library. I don't have any way to piggyback on what Python
has learned about the platform. This is too bad. I am sure it isn't the
Python community's job to solve the Fortran 90 make problem but there are
days when I dream about it.
The "right" object-oriented approach should allow some way to extend the
system, e.g. adding suffices or file names together with info about how to
compile them.
-----Original Message-----
From: Mark Hammond <MHammond(a)skippinet.com.au>
To: distutils-sig(a)python.org <distutils-sig(a)python.org>
Date: Tuesday, March 30, 1999 3:50 PM
Subject: RE: [Distutils] Compiler abstractiom model
>> > Quoth David Ascher, on 29 March 1999:
>> > > On windows, it is sometimes needed to specify other files
>> which aren't .c
>> > > files, but .def files (possibly all can be done with
>> command line options,
>> > > but might as well build this in). I don't know how these
>> should fit in...
>> >
>> > Are they just listed in the command line like .c files? Or
>are they
>> > specified by a command-line option? Would you use these in
>> code that's
>> > meant to be portable to other platforms?
>>
>> Yes, no, and yes. =)
>>
>> E.g. Python extensions need to declare the 'exported' entry
>> point. This
>> can be done either by modifying the source code (bad for
>> portable code,
>> requires #ifdef's etc.), by specifying a command-line option,
>or by
>> including a .DEF file.
>
>Just to follow up on this, many obscure options may need to be
>passed to the Windows linker, but they do require their own
>option - they are not passed as normal files. Examples are
>/DEF: - the .def file David mentions, /NOD:lib - to prevent a
>default library from being linked, /implib:lib to name the built
>.lib file, etc.
>
>It wasnt clear from David's post that these need their own
>option, and are not passed as a simple filename param to the
>linker....
>
>Building from what Greg said, I agree that _certain_ command-line
>params can be mandated - they are designed not to be
>configurable, as messing them will likely break the build. But
>there also needs to be a secondary class that are virtually
>unconstrained.
>
>[Sorry - as usual, speaking having kept only a quick eye on the
>posts, and not having seen the latest code drop...]
>
>Mark.
>
>
>_______________________________________________
>Distutils-SIG maillist - Distutils-SIG(a)python.org
>http://www.python.org/mailman/listinfo/distutils-sig
>
>
