On 04 August 2000, Amos Latteier said:
> I'm sending this to you directly since there seems to be some problem
> with the distutils sig -- my mail bounced with a mailman error. Anyway
> if you have any suggestions I'd love to hear them. If you want, I'll
> summarize your response to the distutils sig later when the mailinglist
> is fixed.
Odd. I'll try cc'ing the list and see what happens. I did notice http
connections to python.org were appallingly slow yesterday -- and I still
work in the same building (same LAN, modulo firewalls and so forth) as
that machine!
> -----Original Message-----
> From: Amos Latteier
> Sent: Friday, August 04, 2000 2:17 PM
> To: 'distutils-sig(a)python.org'
> Subject: Extracting meta-data from distributions
>
> I've been thinking about how to query distutils packages. One thing I'd
> like to do is download archives from the net and programmatically find
> meta-data about the archive.
$ ./setup.py --help
[...]
Information display options (just display information, ignore any commands)
--help-commands list all available commands
--name print package name
--version (-V) print package version
--fullname print <package name>-<version>
--author print the author's name
--author-email print the author's email address
--maintainer print the maintainer's name
--maintainer-email print the maintainer's email address
--contact print the maintainer's name if known, else the author's
--contact-email print the maintainer's email address if known, else the
author's
--url print the URL for this package
--licence print the licence of the package
--license alias for --licence
--description print the package description
--long-description print the long package description
[...]
$ ./setup.py --name --version --author-email --url
Distutils
0.9.1pre
gward(a)python.net
http://www.python.org/sigs/distutils-sig/
Clear enough?
> * It seems that I must execute potentially untrusted code to get the
> meta-data. Is there anyway around this? I guess RExec is the answer...
Curses. I knew someone would get paranoid about writing dist-bots at
some point. I have no answers to this... however, the above
"informational" command shouldn't be doing any filesystem access apart
from importing stuff, so it's probably "securable". Obviously, general
Distutils usage makes extensive use of the filesystem, so writing
general dist-bots securely will probably be tricky. Sigh.
> * It seems that there is no distutils command to return all the
> meta-data that I can poke into sys.argv before importing setup.py.
> Should I replace distutils.core.setup with my own function before
> importing the archive's setup.py. Or is there a less hackish way to get
> back the meta-data all in one piece?
There's no "give me all meta-data" option -- you have to "just know" the
options listed in the --help command. Also, obviously, the output
assumes you "just know" the order of options as you supplied them on the
command line. Oh well, it's a heck of a lot better than nothing.
> Perhaps there are other better strategies for getting at distutils
> archive meta-data, since I'm pretty sure that this scenario has been
> already thought through by someone smart.
*blush* Actually, I can only take credit for knowing a good idea when I
see it (the meta-data display options were Bastian Kleineidam's idea),
and for tearing the command-line parsing code to shreds in order to
allow the above semantics, where multiple options can be used together
and their order is preserved.
Greg
--
Greg Ward - just another /P(erl|ython)/ hacker gward(a)python.net
http://starship.python.net/~gward/
I invented skydiving in 1989!
[cc'd to the distutils-sig: Corran and I have exchanged private email
about Distutils on the Mac in the past; in fact, all the Mac-supporting
code currently in the Distutils is thanks to Corran. I think it's worth
violating netiquette a bit to inform the rest of the sig that Corran is
working on Distutils-on-Mac support, to see if there are any other Mac
people lurking out there, wishing Distutils worked on Mac OS. Please
speak up!]
On 31 July 2000, Corran Webster said:
> I've finally gotten back to a position where I can spend a little time
> looking at distutils for the macintosh again. I took this snapshot an
> dtried a simple install as a first step to getting things going again.
Excellent!
> There were a few bugs, one of which is serious, and I don't know what the
> solution is.
>
> Firstly an easy bug: on line 80 of sysconfig.py, "platform_specific" needs
> to be changed to "plat_specific".
Yep, too easy. Fixed -- thanks.
> The major bug has to do with macintosh paths. The notation to indicate
> that you move up one level in the directory heirarchy on Mac OS is a double
> colon, ie.
>
> dir1:dir2::dir3:file
>
> is the same as "dir1:dir3:file" in the same way that
> "dir1/dir2/../dir3/file" is the same as "dir1/dir3/file" on posix systems.
> This is important, because sometimes macintosh paths are specified with a
> trailing ":" if it is a directory, and sometimes they are not. For
> example, on my system,
Interesting. "Trailing slash means directory" is a (weak) convention on
Unix as well, but I've avoided it in the Distutils because it breaks on
Windows. (In particular: if os.path.isdir("\\foo\\bar") is true,
os.path.isdir("\\foo\\bar\\") is not. ISTR that this may be fixed in
the current CVS Python, but don't remember for sure.)
> >>> import sys
> >>> sys.prefix
> 'Macintosh HD:Applications:Python 1.5.2c1:'
> >>> sys.path
> ['Macintosh HD:Applications:Python 1.5.2c1:', 'Macintosh
> HD:Applications:Python 1.5.2c1:', 'Macintosh HD:Applications:Python
> 1.5.2c1:Lib', 'Macintosh HD:Applications:Python 1.5.2c1:Lib:site-packages',
> 'Macintosh HD:Applications:Python 1.5.2c1:Lib:lib-tk', ... etc ]
Well, one thing I learn from this is that you really ought to upgrade to
1.5.2final. ;-)
> Notice how sometimes the paths have trailing ":" and sometimes they don't.
I guess the convention is a weak one in Mac OS too.
> While os.path.join does the right thing no matter how the paths end, but
> os.path.normpath does not do anything, one way or the other. This leads to
> two problems:
>
> 1. with the install schemes of the install.py module, you get something like:
>
> "$base:Libs" expanding to "Macintosh HD:Applications:Python 1.5.2c1::Libs"
>
> which is the same as "Macintosh HD:Applications:Libs", ie. very much the
> wrong place.
Ahh, I see. You can be lazy on Unix, because "foo//bar" is the same as
"foo/bar" (except in Emacs!). If I understand correctly, on Mac OS it's
as though "foo//bar" was the same as "foo/../bar", which makes life a
bit harder. I try hard not to rely on the collapsibility of "//" though
-- just seems a bit dodgy (and kernel-dependent).
Note that on Unix:
>>> os.path.normpath("foo//bar")
'foo/bar'
>>> os.path.normpath("foo//bar/")
'foo/bar'
which argues in favour of fixing normpath to strip trailing colons on
Mac OS.
> 2. I hacked a fix for 1 (see below), but this led to a second problem: just
> because you do os.path.normpath, you still can't assume that two different
> paths aren't really the same. The result of this was having install.py
> tell me that distutils wasn't installed somewhere in sys.path, when in fact
> it was. This may have other consequences where distutils compares paths to
> each other.
Ick! The whole point of normpath, as I see it, is to guarantee that
string equality means filesystem equality. If it doesn't do that on Mac
OS, then it's broken.
Offhand, I can't think of other places where Distutils compares paths.
Maybe the code that excludes setup.py from the modules to install --
that's in build_py.py; see the 'find_package_modules()' method.
> (a) The simple hack I used to get around problem 1 was simply to expand the
> variable substitution system to accept "${base}Lib". The problem with this
> is that we can't guarantee that base will end with ":", so we could still
> get mangled pathnames.
Yup.
> (b) A more robust solution is probebly to write a normpath for distutils
> which guarantees that pathnames either do or do not end in a ":" on the
> mac. Not sure the best place to put it, or how much code this is going to
> affect, but it could be a real pain to get right.
>
> (c) Lobby for normpath to be "fixed" on the mac - almost certainly doable,
> but leaves the problem of 1.5.* versions not working properly.
I think we can do both (b) and (c). Eg. somewhere we'd say:
if python < 1.6 and Mac OS: # maybe 2.0
def normpath (...):
...
os.path.normpath = normpath
and hopefully the "Distutils-specific" normpath would be right there in
macpath.py in 2.0 (maybe 1.6).
Can you whip up a patch to macpath.py and submit it to the SourceForge
patch manager? (Or just mail it to me if you don't want to deal with
SF.) Make sure you add a test too!
> (d) Some change to the install scheme mechanism so that it uses
> os.path.join(). For example, use a list or tuple of strings, rather than a
> single string:
>
> 'mac': {
> 'purelib': ('$base', 'Lib'),
> 'platlib': ('$base', 'Mac', 'PlugIns'),
> 'headers': ('$base', 'Include', '$dist_name'),
> 'scripts': ('$base', 'Scripts'),
> 'data' : ('$base',),
> }
I'm not keen on that -- getting that stuff working was fairly tricky
(although not nearly as tricky as designing it!), and I really don't
want to revisit it.
Greg
--
Greg Ward - Unix bigot gward(a)python.net
http://starship.python.net/~gward/
I'd like some JUNK FOOD ... and then I want to be ALONE --
New code snapshot available at
http://starship.python.net/~gward/python/distutils.html
This adds Rene Liebscher's patches to the DOS/Windows compiler classes
and build_ext.py. Please try these out -- there are some small but
far-reaching changes in there! Also a few tiny bugs fixed.
Greg
--
Greg Ward - Linux weenie gward(a)python.net
http://starship.python.net/~gward/
Could I have a drug overdose?
