Hi all --
I've just put out a new code snapshot that needs some widespread
testing. (More so than usual, I mean.) This is mainly because I
tweaked the sysconfig module to parse Python's makefile lazily rather
than eagerly, and to store what it finds in a dictionary rather than
global variables. This facility of sysconfig was only used in a few
places throughout the Distutils, and I'm pretty sure I got all of them
-- as usual, It Works For Me (TM). But I just have my one Linux box at
home; at least now I test with Python 1.6 as well as 1.5.2. (And I have
that enormous RPM of 2.0b1 ready to install one of these days...) So
please, download tonight's snapshot and give it a good workout -- build
stuff, lots of stuff, on lots of platforms. Please!
Oh, a neat feature added today: I devised a brilliant and entirely
original new method for describing extensions to build. I call it "the
Setup file". This amazingly innovative mechanism lets you describe the
extension name, source files, include paths, macros to define/undefine,
and a few other goodies in a simple, compact, familiar (to Unix geeks at
least) notation.
For example, you could describe the "_tkinter" extension as follows:
_tkinter _tkinter.c tkappinit.c -DWITH_APPINIT -UFOO \
-I/usr/include \
-I/usr/X11R6/include \
-DWITH_TIX -ltix4.1.8.0 \
-L/usr/lib \
-ltk8.0 -ltcl8.0 \
-L/usr/X11R6/lib \
-lX11
Then, a setup script to build this extension is quite simple:
from distutils.core import setup
from distutils.extension import read_setup_file
exts = read_setup_file("Setup.tk")
setup(name = "Tkinter",
ext_modules = exts)
Hmmm... on second thought, this syntax seems vaguely familiar... where
*have* I seen it before? ;-)
Greg
P.S. yes, it does support comments just like in Modules/Setup!
--
Greg Ward gward(a)python.net
http://starship.python.net/~gward/
This may not be directly related to distutils,
it may also be a bug in 1.6 and 2.0b1 re implementation.
'setup.py sdist' with the current distutils CVS version
hangs while parsing MANIFEST.in,
executing the re.sub command in these lines in text_file.py:
# collapse internal whitespace (*after* joining lines!)
if self.collapse_ws:
line = re.sub (r'(\S)\s+(\S)', r'\1 \2', line)
Has anyone else noticed this, or is smething wrong on my side?
Thomas
I am working on a setup script for Tkinter, which requires a
-DWITH_APPINIT flag at compile time. The docs suggest that I should
add the following definition to my Extension constructor call:
define_macros=[('WITH_APPINIT', None),],
I did this, but it had no effect. It looks like there is no where
that the define_macros attribute of the Extension object is actually
used.
I assumed this is a bug and tried to figure out where the macros
definitions ought to be processed. I'm not familiar with the code, so
I may have put it in the wrong place. A patch is included below,
which allows _tkinter to build.
Jeremy
*** build_ext.py Tue Sep 12 20:58:48 2000
--- /home/jeremy/src/python/dist/src/Lib/distutils/command/build_ext.py Thu Sep 7 11:09:18 2000
***************
*** 401,407 ****
# any sensible compiler will give precedence to later
# command line args. Hence we combine them in order:
extra_args = ext.extra_compile_args or []
- macros = ext.define_macros or []
# XXX and if we support CFLAGS, why not CC (compiler
# executable), CPPFLAGS (pre-processor options), and LDFLAGS
--- 401,406 ----
***************
*** 410,419 ****
if os.environ.has_key('CFLAGS'):
extra_args.extend(string.split(os.environ['CFLAGS']))
!
objects = self.compiler.compile (sources,
output_dir=self.build_temp,
! macros=macros,
include_dirs=ext.include_dirs,
debug=self.debug,
extra_postargs=extra_args)
--- 409,418 ----
if os.environ.has_key('CFLAGS'):
extra_args.extend(string.split(os.environ['CFLAGS']))
!
objects = self.compiler.compile (sources,
output_dir=self.build_temp,
! #macros=macros,
include_dirs=ext.include_dirs,
debug=self.debug,
extra_postargs=extra_args)
1) Why is the --formats list comma-separated and the --include-dirs and
--library-dirs lists not? They are separated by os.pathsep.
Well, at least its documented :)
2) The os.pathsep separator is not documented in the help text:
> python setup.py build_ext --help
...
--library-dirs (-L) directories to search for external C libraries
This should be:
...
--library-dirs (-L) colon-separated list of directories to search
for external C libraries
or whatever os.pathsep is on the current platform.
3) bdist_wininst does not seem to install data files and scripts.
If you execute "strings" with the .exe, you will only see Python module
files.
And I did not found a file with the install log. The installer just
copied the files in the Python dir.
Bastian
[cc'ing the SIG again, for the same reason]
On 13 September 2000, Corran Webster said:
> At the moment to install you have to use the command line interface -
> you can't just double click on setup.py. What Greg is saying is that
> one way to avoid this on the mac side is to have some version of the
> bdist command which builds an intelligent installer of some sort (for
> example, it may bundle everything up into a VISE installer) which
> then puts everything in the right place.
That only helps installers -- see my previous post for a possible way to
help all Distutils users (developers and packagers too).
> I can see two slightly simpler ways of dealing with this. The
> easiest to implement is to have an applet which you can just drop
> setup.py files on and it handles everything from there (easy to do
> thanks to the "run_setup" function - I have a 5 line applet which
> does this now). This "Install package" applet could be added to the
> standard mac distribution of Python much like the "Build applet" and
> "Build application" applets are now.
Hmm, neat idea. You could even make your "Install package" applet smart
enough to unpack a Distutils-style archive and dig up setup.py there --
just download a source release off the net, drop it onto the applet, and
away-we-go. How's *that* for ease of use? ;-)
> A slightly more sophisticated approach would be to have a bdist-type
> command which creates an installer applet for the package, and then
> bundles everything up in a zip/tar/stuffit file. All the end users
> would just unpack the archive and run the applet and it would take
> care of everything automatically.
That might be more sophisticated, but it sounds inferior to the previous
option in usability terms: the user has to unpack the archive and "do
something" with what he finds there.
> An even more sophisticated approach would bundle everything up using
> zip/tar/stuffit and then store the data inside a python script
> somehow (a really big string?). This script would know how to unpack
> and install the archived data. This script would then be turned into
> an applet (or even a standalone application) and distributed -
> essentially making a pure python poor man's version of VISE.
This is essentially what Thomas Heller's bdist_wininst does, except the
wrapper is a C program -- really a self-expanding ZIP file with a bit of
GUI to make it look pretty. Despite the fact that I have to carry
around C source and a Windows .exe in the Distutils CVS and source
distribution, this strikes me as more elegant than making a big Python
script with a little bit of code and a lot of encoded
ZIP/StuffIt/whatever data.
> The most sophisticated would be to package everything up in a VISE
> Installer. Jack would know far more than I about how
> possible/practical that would be.
The original idea for Windows was to generate WISE installers. (Any
relation to VISE?) The impetus for this has dropped off with the
success of the bdist_wininst command; it sounds like not many module
distributions need the sophistication of a full-blown commercial
installer. I suspect the same would hold on Mac OS.
However, when it comes time to deal with full-blown Python applications,
we may have to reexamine the issue.
Again, I'll deal with header file installation in a separate post.
Greg
--
Greg Ward gward(a)python.net
http://starship.python.net/~gward/
On 14 September 2000, Jack Jansen said:
> is there a reason that simply starting setup.py doesn't do an install?
> That may be the easiets way to go...
On 13 September 2000, Jack Jansen said:
> Greg,
> I've cc-d Corran on this reply, it's a bit silly to keep two conversations
> going with similar content. Don't worry, we'll keep our esoteric Mac-talk to
> ourselves:-)
And I'm bringing the whole SIG in on this -- surely you two aren't the
only Mac hackers in the world!
[I opined:]
> I like the model we've used for Windows installers: developers and
> packagers have to put up with a command line interface -- ie. open a
> "DOS box" and run "python setup.py bdist_wininst" -- but then end users
> have a nice little Windows-friendly executable installer. They're not
> even aware of such a thing as the Distutils, which is how it should be
> for end-users.
[Jack replies]
> Ok, that's fine. From reading the document I got the impression that the
> various command-line switches to install (in the end-user situation) would
> actually be necessary relatively often, but you're saying that they're rare,
> and that the end user will normally just do "python setup.py". Right?
Not quite. You're right that the options to install will not be needed
very often, especially on single-user systems with no file permissions.
But the normal case is "python setup.py install" -- you CANNOT use the
Distutils without providing at least one argument to the setup script.
The rationale for this reveals my deep-seated Unix geek prejudice: how
am I supposed to find out what this program does? Well, I run it
without arguments and see what it says. As long as it's not a filter
(remember trying to learn how to use grep before you understood what a
filter was?), you're fine. Well-behaved command-line programs give you
a usage summary when run with bad args or no args, and any Distutils
setup script automatically plays by those rules.
This is bad on Mac OS, where providing *any* command-line args is a
pain. However, I have a cunning plan...
When a setup script is run under Mac OS, instead of printing a usage
summary to stdout and exiting, throw up a dialog with the user's options
-- ie. what commands they can run. You can get a barebones list of
commands from distutils.command.__all__, and a list of all commands
(including non-standard commands provided by the current setup script)
by poking around in the Distribution object. See 'print_commands()' in
dist.py for an example, and 'print_command_list()' for how to get the
description of each command in turn. (The command module has to be
imported, which is why running "setup.py --help-commands" is slow and
chunky: it has to import an entire module for each line of help text!)
However, the barebones list is probably sub-optimal; you'd want to
shuffle it around a bit so "install", "build", "clean", "sdist", and
"bdist" are up at the top. Then the sub-commands can come below,
grouped as they already are (which is a function of the ordering of
distutils.command.__all__). This reshuffling is purely a usability
issue; 90% of users will just run "install" 90% of the time, so it
should be first, and checked by default. Eg. here's a rough sketch of
the dialog I have in mind:
+-------------------------------------------------------------+
| (x) install install everything from build directory |
| ( ) build build everything needed to install |
| ( ) clean clean up output of 'build' command |
| ( ) sdist create a source distribution ... |
| ( ) bdist create a built (binary) distribution |
| ----------------------------------------------------------- |
| ( ) build_py ... |
| ( ) build_ext ... |
| ( ) build_clib ... |
| ( ) build_scripts ... |
| ( ) install_lib ... |
| ( ) install_headers |
| ( ) ... |
| ( ) bdist_dumb |
| ( ) ... |
| |
| [ Ok ] [ Cancel ] |
+-------------------------------------------------------------+
(Memo to myself: if this ordering makes more sense in a GUI dialog, it
makes more sense in the output of --help-commands. Hmmm.)
I *think* the commands should be radiobuttons, even though you can run
multiple commands in one go, because of the need to associate
command-line options with each command.
Hmmm, maybe not. If you pick multiple commands *and* choose to "Supply
additional command-line arguments" (this would probably have to be
another checkbox, off by default), then you could get a dialog with
space for multiple argument lists:
Arguments for:
build_clib ____________________________
install_headers ____________________________
if the user selected to run "build_clib" and "install_headers".
You might want to use distutils.util.split_quoted() to parse those
argument strings into words, in case -- gasp! -- people use directories
with spaces in them. (Which will have to be quoted -- ugh! I can just
see Joe Mac User saying, "Whaddya mean, the standard shell quoting
convention?!??!")
Of course, if the user doesn't check the "Supply additional command-line
arguments" dialog, then the requested command(s) immediately run.
stdout should be preserved and presented to the user in a big scrolling
window -- yeah it's ugly, yeah it'll look like someone took a big ol'
Unix program and ported to Mac OS as quickly and cheaply as possible.
But that's what (will have) happened!
So whaddya think? How outrageously horrible and Unix-oid is that? I
figure it's not too bad as long as the user doesn't want to supply any
extra args, and I don't think that'll be necessary for most end-user
installation situations.
> Mac installers don't have to worry about HKEY_USER and HKEY_MACHINE and the
> difference between OS version X and OS version Y and such: there's always a
> single place to put things, and if that somehow changes "the old way"
> continues to work. Usually through Apple-supplied magic, sometimes through
> VISE-supplied magic.
What, you mean you have a *sensible* operating system? I thought they
were banned by the secret Microsoft World Domination Act of 1994 (passed
by the hidden One World Government, of course). (*wink*... I think.)
The issue of where to install header files affects other platforms as
well, so I think I'll bring that up in a new thread on the SIG.
Greg
--
Greg Ward gward(a)python.net
http://starship.python.net/~gward/
Hi all --
just noticed the BeOS-specific code in sysconfig._init_posix():
g['LDSHARED'] = ("%s -L%s/lib -lpython%s" %
(linkerscript, PREFIX, sys.version[0:3]))
Should that be EXEC_PREFIX? I dunno in one would ever do a prefix !=
exec-prefix installation on BeOS, but it seems like good practice to
respect the "exec-prefix for binaries" convention in any event.
Greg
--
Greg Ward gward(a)python.net
http://starship.python.net/~gward/
I've written a script to apply a command across a bunch of setup.py
scripts in a subdirectory, and found a problem:
distutils.dir_utils._path_cache stores a list of directories that have
already been created. The directory name is just stored as 'dist',
not a full path name, so if two subdirectories need to create a 'dist'
directory, the second one thinks it's already been created, and dies.
Fix: storing full path names in the cache seems reasonable.
(The setup.py script is included below, since it may be instructive; feel free
to use it as an example.)
--amk
# Top-level script for running a command-line over all the subpackages
import sys, os, glob
from distutils.core import run_setup
files = os.listdir('.')
dirs = filter(os.path.isdir, files)
packages = []
for dir in dirs:
if os.path.exists( os.path.join( dir, 'setup.py') ):
packages.append( dir )
top_dir = os.getcwd()
for package_dir in packages:
print 'Running', package_dir
os.chdir( package_dir )
dist = run_setup('setup.py', script_args=sys.argv[1:], stop_after="run")
os.chdir( top_dir )
Hello,
I just discovered the "bug" which prevents bdist_wininst from installing
data files and scripts.
bdist_wininst.py:109: root_dir = install.install_lib
I noticed this comment in the source:
# XXX hack! Our archive MUST be relative to sys.prefix
# XXX What about .install_data, .install_scripts, ...?
So what about data and scripts? Why dont they get in the archive?
Bastian
