[Pythonmac-SIG] Versions, Frameworks, Linking, PantherPythonFix

[Bob Ippolito]

On Feb 22, 2005, at 1:25, Roger Binns wrote:

> [BTW I apologise for not updating the subject.  My questions are
> in the bigger picture context of Python on Mac, not the need for
> PantherPythonFix]
>
>> You quoted something about link-time linker support for frameworks 
>> and then talk about runtime linker support..
>
> On other platforms they are related.  On Windows the contents of a .lib
> file at link time determines what .dll is linked to at runtime.  On 
> Linux/
> ELF, rpath information (if used) is embedded at link time and affects
> runtime.

Yes, of course, they're related but not in the context you quoted.

 From the perspective of the runtime linker (dyld), there is really no 
difference between dylibs in a framework or outside of one except under 
exceptional conditions.  They're both MH_DYLIB Mach-O files with no 
difference in link commands or anything else.

Frameworks are about 98% link time and 2% runtime (the runtime handling 
for frameworks only kicks into effect upon certain environment 
variables or to handle  frameworks).  It's been a couple months since 
I've read the source to dyld, but I'm pretty sure that *versioned* 
frameworks have zero runtime support, they're entirely a link time 
"feature".  The deficiencies of versioned frameworks are in the tools, 
not the implementation.

>> If you have Python 2.X and Python 2.Y both installed, as frameworks,
>
> But earlier messages said that only Apple does the frameworks, and
> they don't even bother updating the one they did install let alone
> install other concurrent versions.

Earlier message says that Apple only does the frameworks, not only 
Apple does the frameworks.  MacPython 2.3.5 is a framework build, for 
example.

>> ... (10.2 compatible) ...
>
>> From this statement and various others it would appear that there
> were significant changes in the underlying mechanisms for deploying
> Python between 10.2 and 10.3.  I am quite happy to ignore the existence
> of 10.2 :-)

I would love to ignore the existence of 10.2, but that's not really 
even an option until the release of 10.4 at the earliest.

>> Things aren't done differently..
>
> Frameworks don't exist on the other platforms, nor did changing symbol
> resolution, the Mach-O format, just putting all Python stuff in
> /usr/lib/python-version or c:\python-version.  There are many
> similarities, but it is different.  (If it wasn't I wouldn't be
> asking all these questions :-)

 From the perspective of a Python developer, when you are using a 
version of Python without distutils bugs, then things are not done 
differently.  The implementation details are not the same, of course.

>> py2app of course prefers relocatable extensions, but it can solve the 
>> issue the hard way by rewriting the extension's Mach-O load commands 
>> to relocate it at build time, but there are some edge cases where it 
>> won't work (if there isn't enough room in the header to accommodate 
>> the new load commands, which has a very, very, very low chance of 
>> occurring -- I have never seen it nor heard any reports of it).
>
> I have to do something similar with wxPython on Linux.  It uses rpath
> to hard code the location of the wxgtk libraries in the C interface
> modules.  That means things just work for users without having to set
> their LD_LIBRARY_PATH.  However it is really bad news for distributing
> binary apps.  I use a program named chrpath to strip out the rpath's.
> The cx-Freeze initialisation code modifies the LD_LIBRARY_PATH to
> point to the applications directory first, and so everything works.

Yeah, py2app does all of this automatically, including building a 
dependency graph of everything you use, etc.

>> What's hard to understand about "Install this fix if you use Python 
>> at all on Mac OS X 10.3"?  If you don't do it, you will probably run 
>> across a problem eventually.
>
> The PantherPythonFix is obvious.  The other cases are more interesting,
> especially once OS X 10.4 comes out.  At that point I will want to 
> build
> on 10.3 using a Python 2.3 interpretter and extensions, and include 
> that interpretter and extensions in the app bundle so that 10.4 users
> will use the exact same Python as everyone else.
>
> Since you won't make py2app redistribute the system Python, it means
> I have to use some other one and then deal with PATH and whatever
> else so that the correct one is picked up by default.

Well as I said in the py2app documentation, if you want maximum future 
compatibility you're going to have to use your own installation of 
Python and let py2app embed it into your application bundles.

Yes, you will have to set your PATH if you want this one to be default.

For my commercial application, I have a Python framework built for OS X 
10.2 with a set of extensions built for OS X 10.2, and my build scripts 
make sure to invoke this particular Python (because it's not for 
general use) when doing the build with py2app.  py2app doesn't 
introduce any versioning issues, so if you build all of your extensions 
on a 10.2 environment and successfully distribute it to another machine 
then you can use it to assemble it with py2app at a later date on any 
version of Mac OS X and it will remain 10.2 compatible.

It is not yet public knowledge as to whether applications built with 
the vendor Python 2.3.0 on Mac OS X 10.3 will work on Mac OS X 10.4.  
Given that Python 2.3.x was in the WWDC sources and setup to build as a 
framework in the same place, then signs point to yes.  However, if 
Apple changes their minds and go with 2.4, then the answer will 
probably be no.

> And then when we start using Python 2.4, OS X 10.3 will probably still
> be the base OS so more interesting version issues will arise.

Assuming you are all patched up (PantherPythonFix'ed Python 2.3.0, 
MacPython 2.3.5+, MacPython 2.4.1+ when it exists) then all three 
flavors of framework Python will live together just fine with no issues 
at all.  You will not be able to reliably share extensions between 
interpreters because:
- You are likely to have extensions compiled prior to PantherPythonFix, 
which are tightly bound to the location of that interpreter and can not 
be shared
- MacPython 2.3.5 as built by Jack is 10.2 compatible, so its distutils 
will build extensions that can not be shared as they are tightly bound 
to that interpreter
- Python 2.4 and Python 2.3 are not API compatible

In other words for a maximally redistributable application:
- Install PantherPythonFix immediately, on any Mac OS X 10.3 machine 
that you will use distutils on, without hesitation
- Build your application and all extensions on the lowest common 
denominator version of Mac OS X *not* using the vendor Python.
- Run py2app on your application with this environment (though it may 
be on another machine on a newer version of Mac OS X if you can figure 
out how to copy the environment to another machine).

>> This is *not* a Mac-ism.
>
> It is certainly a Mac-ism that the system Python is not 
> redistributable :-)

On Windows there is no vendor Python.  I don't really care about Linux 
-- I'm sure it's possible to bundle the vendor Python with your 
application, but I'm not sure why you would do such a thing.  Is that 
default behavior for cx_Freeze?

Anyway, there is no technical reason that py2app can't bundle the 
vendor Python, it is my decision, and I'm not going to change my mind.  
I'm not going to make py2app build applications that are illegal to 
redistribute by default.  Additionally, until the release of Mac OS X 
10.4, there is absolutely no reason whatsoever to bundle the vendor 
Python, because the resultant application bundle will simply be larger. 
  It won't work on Mac OS X 10.2 or earlier.  As py2app is liberally 
licensed, if you'd like to fork it to allow you to redistribute Apple 
files, you can do it.  Who knows, it might be a good way to meet some 
of the folks at Apple's legal department ;)

-bob