[Distutils] New code snapshot

[Greg Ward]

[cc'd to the distutils-sig: Corran and I have exchanged private email
about Distutils on the Mac in the past; in fact, all the Mac-supporting
code currently in the Distutils is thanks to Corran.  I think it's worth
violating netiquette a bit to inform the rest of the sig that Corran is
working on Distutils-on-Mac support, to see if there are any other Mac
people lurking out there, wishing Distutils worked on Mac OS.  Please
speak up!]

On 31 July 2000, Corran Webster said:
>     I've finally gotten back to a position where I can spend a little time
> looking at distutils for the macintosh again.  I took this snapshot an
> dtried a simple install as a first step to getting things going again.

Excellent!

> There were a few bugs, one of which is serious, and I don't know what the
> solution is.
> 
> Firstly an easy bug: on line 80 of sysconfig.py, "platform_specific" needs
> to be changed to "plat_specific".

Yep, too easy.  Fixed -- thanks.

> The major bug has to do with macintosh paths.  The notation to indicate
> that you move up one level in the directory heirarchy on Mac OS is a double
> colon, ie.
> 
> dir1:dir2::dir3:file
> 
> is the same as "dir1:dir3:file" in the same way that
> "dir1/dir2/../dir3/file" is the same as "dir1/dir3/file" on posix systems.
> This is important, because sometimes macintosh paths are specified with a
> trailing ":" if it is a directory, and sometimes they are not.  For
> example, on my system,

Interesting.  "Trailing slash means directory" is a (weak) convention on
Unix as well, but I've avoided it in the Distutils because it breaks on
Windows.  (In particular: if os.path.isdir("\\foo\\bar") is true,
os.path.isdir("\\foo\\bar\\") is not.  ISTR that this may be fixed in
the current CVS Python, but don't remember for sure.)

> >>> import sys
> >>> sys.prefix
> 'Macintosh HD:Applications:Python 1.5.2c1:'
> >>> sys.path
> ['Macintosh HD:Applications:Python 1.5.2c1:', 'Macintosh
> HD:Applications:Python 1.5.2c1:', 'Macintosh HD:Applications:Python
> 1.5.2c1:Lib', 'Macintosh HD:Applications:Python 1.5.2c1:Lib:site-packages',
> 'Macintosh HD:Applications:Python 1.5.2c1:Lib:lib-tk', ... etc ]

Well, one thing I learn from this is that you really ought to upgrade to 
1.5.2final.  ;-)

> Notice how sometimes the paths have trailing ":" and sometimes they don't.

I guess the convention is a weak one in Mac OS too.

> While os.path.join does the right thing no  matter how the paths end, but
> os.path.normpath does not do anything, one way or the other.  This leads to
> two problems:
> 
> 1. with the install schemes of the install.py module, you get something like:
> 
> "$base:Libs"    expanding to   "Macintosh HD:Applications:Python 1.5.2c1::Libs"
> 
> which is the same as "Macintosh HD:Applications:Libs", ie. very much the
> wrong place.

Ahh, I see.  You can be lazy on Unix, because "foo//bar" is the same as
"foo/bar" (except in Emacs!).  If I understand correctly, on Mac OS it's
as though "foo//bar" was the same as "foo/../bar", which makes life a
bit harder.  I try hard not to rely on the collapsibility of "//" though
-- just seems a bit dodgy (and kernel-dependent).

Note that on Unix:

>>> os.path.normpath("foo//bar")
'foo/bar'
>>> os.path.normpath("foo//bar/")
'foo/bar'

which argues in favour of fixing normpath to strip trailing colons on
Mac OS.  

> 2. I hacked a fix for 1 (see below), but this led to a second problem: just
> because you do os.path.normpath, you still can't assume that two different
> paths aren't really the same.  The result of this was having install.py
> tell me that distutils wasn't installed somewhere in sys.path, when in fact
> it was.  This may have other consequences where distutils compares paths to
> each other.

Ick!  The whole point of normpath, as I see it, is to guarantee that
string equality means filesystem equality.  If it doesn't do that on Mac 
OS, then it's broken.

Offhand, I can't think of other places where Distutils compares paths.
Maybe the code that excludes setup.py from the modules to install --
that's in build_py.py; see the 'find_package_modules()' method.

> (a) The simple hack I used to get around problem 1 was simply to expand the
> variable substitution system to accept "${base}Lib".  The problem with this
> is that we can't guarantee that base will end with ":", so we could still
> get mangled pathnames.

Yup.

> (b) A more robust solution is probebly to write a normpath for distutils
> which guarantees that pathnames either do or do not end in a ":" on the
> mac.  Not sure the best place to put it, or how much code this is going to
> affect, but it could be a real pain to get right.
> 
> (c) Lobby for normpath to be "fixed" on the mac - almost certainly doable,
> but leaves the problem of 1.5.* versions not working properly.

I think we can do both (b) and (c).  Eg. somewhere we'd say:

    if python < 1.6 and Mac OS:              # maybe 2.0
        def normpath (...):
            ...
        os.path.normpath = normpath

and hopefully the "Distutils-specific" normpath would be right there in
macpath.py in 2.0 (maybe 1.6).

Can you whip up a patch to macpath.py and submit it to the SourceForge
patch manager?  (Or just mail it to me if you don't want to deal with
SF.)  Make sure you add a test too!

> (d) Some change to the install scheme mechanism so that it uses
> os.path.join().  For example, use a list or tuple of strings, rather than a
> single string:
> 
>     'mac': {
>         'purelib': ('$base', 'Lib'),
>         'platlib': ('$base', 'Mac', 'PlugIns'),
>         'headers': ('$base', 'Include', '$dist_name'),
>         'scripts': ('$base', 'Scripts'),
>         'data'   : ('$base',),
>         }

I'm not keen on that -- getting that stuff working was fairly tricky
(although not nearly as tricky as designing it!), and I really don't
want to revisit it.

        Greg
-- 
Greg Ward - Unix bigot                                  gward@python.net
http://starship.python.net/~gward/
I'd like some JUNK FOOD ... and then I want to be ALONE --