[Pythonmac-SIG] Some thoughts on Python and OS X
Kevin Ollivier
kevino at tulane.edu
Mon Jun 14 11:33:14 EDT 2004
Hi Kenneth,
On Jun 14, 2004, at 1:07 AM, Kenneth McDonald wrote:
> Bob,
>
> Thanks for the responses. Sorry I was wrong on some points: I'd
> received
> the impression that Apple's Python was somehow "special", and for that
> I apologise. In my own defense, I definitely received that impression
> on this group due to messages from folks at Apple; they weren't
> being misleading, they simply said, "Don't muck around with the
> Apple installation, because replacing it with your own Python
> distro will do bad things", or words to that effect. Without further
> details,
> it's not difficult to conclude that this must mean Apple distro !=
> Python distro.
The reason people say this is that Apple not only includes Python in OS
X, they also *use* it at the system level, and they may do so even more
in the future now that they can use CoreGraphics from Python. So if you
replace Apple's Python with something else, you run the risk of
breaking any OS X system scripts/utilities that haven't been tested
with 2.3.3, for example, which definitely would not be a good thing.
> And while I haven't actually used Linux in a long time (and never
> particularly enthusiastically), you are partly right about expecting
> certain things to be done in a UNIXish manner. There never was
> a real "standard" for the UNIX file structure, but Apple's structure
> is, uh, somewhat less standard than standard :-).
Just remember that Apple's top priority was to make a Mac desktop that
Mac users could be comfortable with - and then to provide UNIX
compatibility. The UNIX filesystem is cryptic enough that the average
user would be intimidated by it, and it doesn't fit well at all with
Apple's concept of self-contained applications. (BTW, small rant, I
think it's a semi-standard standard that *needs* to be broken for *nix
to target the Desktop.)
> What would really help--for me at least, and perhaps for others--
> is a good document explaining how and why Apple has implemented
> certain structures, how they map to "old" structures. I've looked
> through a number of the OS X books available, and though I can
> find bits and pieces here and there, I've never seen a good,
> coherent overview. For example, I'd love to see a good explanation
> as to why Frameworks exist, why to put things into a framework
> structure rather than placing libraries, binaries, etc. into
> /.../lib, /.../bin, how to deal with Makefiles that don't know about
> frameworks (most of them), and so on.
See:
http://www.macdevcenter.com/pub/a/mac/2002/10/22/macforunix.html
for some answers to FAQs from UNIX users, including the filesystem
issue (you may want to consider picking up the book, O'Reilly books are
usually a good investment), and see here:
http://developer.apple.com/unix/index.html
For more information about OS X UNIX support from the source. There's
documentation on how to port UNIX apps to OS X, and even a mailing list
for it, among other things here.
> I also apologise for getting /usr/local wrong. As I say, I've ended up
> creating a number of 'old-style' UNIX directories. I guess it must've
> been another one.
>
> Definitely, the fact that /Library/Python/2.3 and
> /Library/Frameworks/Python/...{somewhere}
> are supposed to be the same place is news to me. They aren't the same
> on my system, I have no idea whether due to me or to some installer
> script, and now I'm scared to try to fix this :-)
/System/Library/Frameworks/Python.Framework/Versions/2.3/lib/site-
packages should just be a symlink to /Library/Python/2.3. All packages
should install to /Library/Python/2.3. Any packages that install
directory into /System/Library have broken installers and you should
report the problem to the package maintainer. (Note that old versions
of wxPython did break the symlink, but recent ones have fixed the
problems.)
> As regards to 'distutils', well first, I'll absolutely claim stupidity
> on my
> part about distutils always installing into a 'versioned' subdirectory,
> that's useful knowledge, and I'm certainly turning bright red.
> However, simply saying that "distutils understands OS X, so anything
> written properly using distutils should work on OS X" is a little too
> facile. I've spent too many hours trying to tell Python (and other
> things)
> where to find (non-Python) libs, files, etc. because the OS X default
> paths/file structure didn't seem adequate for 'legacy' UNIX software.
Well, there are two types of non-Python libs - UNIX libs and Mac
Frameworks. The UNIX libs should work fine as 'legacy' UNIX software,
and any Makefile that doesn't work (as Bob said) is a poorly written
makefile that probably isn't even compatible with all variants of
UNIX/Linux. As for Frameworks, they are linked with "-framework
FrameworkName" and headers are included by using the #include
<FrameworkName/FrameworkName.h> syntax, but the person who authored the
Python extension should know this and deal with this. The end result is
that for the most part, if the developer wrote their distutils
scripts/makefiles correctly, things should just "work" out of the box
on OS X.
Is it SPE that's giving you so much trouble, or another package? Are
several of them doing this?
> And again, this was not intended to be criticism, at least not
> in the derogatory sense. I like OS X, I like programming on it, and
> I'm extremely glad to be back using Python after several years away
> from it. However, I have spent some time wrestling with problems
> largely caused by the "Apple-ness" of OS X at the UNIX level, and
> I know others have had similar problems, and I thought it would be
> a good idea to talk about some of these. I've certainly learned some
> useful things from your response.
It is a "learning experience", but just be careful not to assume it's
always due to it being a problem with Apple's UNIX distro. ;-) The fact
is that developing truly "portable" makefiles, for example, is not an
easy task and some UNIX software may break if ported to Solaris or any
other platform the developer didn't test on, unless they created their
makefiles in a very specific manner. This can also apply to distutils
scripts as well. There are, of course, some issues specific to OS X,
but the problems you're having sound like there's something else going
on here.
HTH,
Kevin
> Thanks,
> Ken
>
>
> On Jun 14, 2004, at 12:54 AM, Bob Ippolito wrote:
>
>>
>> On Jun 14, 2004, at 1:28 AM, Kenneth McDonald wrote:
>>
>>> The following comes from some experiences I've had recently whilst
>>> wrangling with wxPython/SPE, plus some others encountered in the
>>> past while doing custom installs of Tk and tkinter. I don't want to
>>> start
>>> any flame wars, and I certainly don't want to hurt any feelings--the
>>> Python
>>> people at Apple are doing a great job, and I really appreciate their
>>> effort. However, the "Apple Philosophy" as applied to Python (and
>>> indeed as applied to much of the UNIX subsurface in OS X) is making
>>> things painfully, and in my opinion unnecessarily, difficult to use.
>>>
>>> First, the obvious: Having a "required, custom" build of Python on
>>> OS X is a Bad Idea. Last I heard, this was the official OS X way of
>>> doing things, i.e. the Python you got was what you should use, don't
>>> try replacing it, and no, it's not a stock Python. I believe this
>>> has been
>>> recognized to be a problem and is being worked on--sorry if I'm
>>> ignorant of recent fixes, I "threw out" the stock OS X Python some
>>> time ago, as I needed my own custom compilations.
>>
>> It's not custom. It's highly suggested not to replace it because
>> there are various bugs here and there on the Python side that make it
>> a bad idea to do so.
>>
>>> In general (and I'm sure the Apple techs understand this, but perhaps
>>> you could pass this on to those in management who don't?), DON'T
>>> EVER take a community standard tool such as Python and "improve"
>>> it in such a manner as to make it incompatible with what people have
>>> come to expect. If you must produce an Apple-specific version which
>>> requires special consideration when being used with other packages,
>>> then observe the following:
>>>
>>> 1) The _standard_ version should be the standard; even on OS X.
>>> People who want to use Python without caring about the OS its
>>> on will then be able to use it properly. Those who want to make use
>>> of the os-enhanced version will know what to do.
>>>
>>> 2) Make sure that an 'enhanced' version can coexist peacefully with
>>> the standard version.
>>
>> It *is* standard. It's not "improved". The only difference between
>> what Apple distributes and what was in 2.3.0 is the location that it
>> is installed in (/System rather than /) and the additional
>> CoreGraphics extension, which has nothing to do with how it was
>> compiled.
>>
>>> One of the things that is expected is that I can download, compile,
>>> and install a more recent version of Python without problem (assuming
>>> the newer version itself doesn't have problems.) Another is that
>>> add-ons for Python which have been written to install nicely in a
>>> generic UNIX environment should continue to do so under OS X.
>>
>> It will work just fine, but it must supersede the system installation
>> due to bugs that were in 2.3.0 that we can not fix. Once you install
>> your own Python, you can no longer expect to be able to compile
>> software for the stock 2.3.0.
>>
>> Any Python extensions that have been written properly already use
>> distutils, and thus already work fine on OS X. If anything is
>> designed to work in a "generic UNIX environment" and does NOT work on
>> OS X is simply not done correctly. Things should work on Windows
>> too, and that means using distutils.
>>
>>> Next, paths. I can understand why Apple (actually, probably NeXT)
>>> decided to come up with improvements on the UNIX file structure.
>>> But why, oh why, couldn't they have maintained some sort of backward
>>> compatibility using links? I've effectively recreated my own UNIX
>>> file
>>> structure using links, to make up for this, and would be surprised
>>> if others
>>> haven't as well. In particular, why the heck isn't there the
>>> equivalent of
>>> /usr/local? Having this basic de facto standard sort of file
>>> structure in
>>> place would save us UNIX software users countless hours in headaches.
>>> Yes, it's easy to argue that the problems are the fault of bad
>>> installation
>>> scripts and makefiles. Well, yes, in theory that's probably true;
>>> but so
>>> what? The point of software is to be a useful tool, not to drive the
>>> customer
>>> crazy by insisting that somebody else, in assuming a de facto
>>> standard of
>>> the time would continue into the future, made a mistake.
>>
>> Uh, there is a /usr/local.
>>
>>> I find this a particular problem when it comes to Frameworks and
>>> Libraries.
>>> Here's a typical example of installing a couple of Python modules
>>> recently:
>>>
>>> 1) Compile module A--works, great. Install it. Where the heck are the
>>> files? Do a search on the disk. Ah, there they are in
>>> /Library/Frameworks/Python/.../.../... and so on
>>> .../2.3/site-packages.
>>> Create a link from a more reasonable location so they'll be on the
>>> python path.
>>
>> /Library/Python/2.3 is the more reasonable link, it already exists.
>>
>>> 2) Compile and install module B. Where is it? Disk search. Ah, it's
>>> in
>>> /Library/Python/2.3. Create link. Why did it end up here, vs. Module
>>> A in Frameworks? Beats me.
>>
>> They're the SAME PLACE unless you or some broken installation script
>> screwed that up.
>>
>>> And why, for that matter, are all of these modules always ending up
>>> in
>>> a '2.3' directory? I happen to know that some of them aren't
>>> dependent
>>> on the Python binary, and it looks to me as if, when 2.3 changes to
>>> 2.4,
>>> I'm gonna have some unnecessary work, again. Trust me, if a module
>>> is incompatible with 2.4, I'll know it soon enough; you don't have to
>>> force me to move everything manually to the 2.4 path.
>>
>> This happens on EVERY Python platform. If you want to install
>> "version independent" Python files somewhere else, feel free to do
>> so. There are many options that you may pass to distutils to control
>> this behavior. Then you simply need to make a pth file that points
>> to your "version independent" site directory.
>>
>>> None of this is going to make me move off of OS X, of course; it
>>> still
>>> is, by far, the best OS for most users IMHO, myself included. But
>>> this
>>> situation will cost Apple some money; I generally keep fairly
>>> up-to-date
>>> with hardware, and was planning on getting a new Powerbook when
>>> they hit 2 GHz. However, I've had to do so much manual tweaking to my
>>> system that the effort needed to recreate that on a new drive is a
>>> little too
>>> much to want to repeat in the forseeable future, so Apple's probably
>>> going
>>> to have to hit 3GHz for me to decide it's worth the pain. (Yes, I
>>> know
>>> I don't have to do a "clean" install of everything when I go to a
>>> new system,
>>> but that's one of the things I do to help me clean all the cruft
>>> out.)
>>>
>>> I'm looking forward to what comes out in the future, I have a lot of
>>> confidence in the Python people at Apple, and I have no problems at
>>> all in continuing to pay for new versions of OS X which bring new
>>> features and improvements. I just hope that the above is of some help
>>> in deciding how some of those features should be implemented in the
>>> future.
>>
>> There aren't really "Python people at Apple", at least not that I'm
>> aware of. Any problems with the Python distribution for OS X are at
>> least partially our fault. Start filing bugs and feature requests,
>> and making patches if you REALLY want a particular issue fixed in a
>> way that suits you.
>>
>> I think a lot of your troubles are just because you expect things to
>> work as they do in Linux, and that's just not the case. There are
>> equivalent or better ways of doing things on OS X, that you either
>> don't accept or haven't figured out yet.
>>
>> -bob
>
>
> _______________________________________________
> Pythonmac-SIG maillist - Pythonmac-SIG at python.org
> http://mail.python.org/mailman/listinfo/pythonmac-sig
>
More information about the Pythonmac-SIG
mailing list