[Pythonmac-SIG] Python 2.3.4 Installation

[tmk]

Thanks a lot for the thorough (and quick!) explanation Bob.

This is much appreciated.

= tmk =

On Nov 28, 2004, at 17:45, Bob Ippolito wrote:

> On Nov 28, 2004, at 10:53 AM, tmk wrote:
>
>> First I must confess that I don't really understand the details of  
>> the problem of building extensions with multiple versions of Python  
>> installed on Panther...
>
> Very few people do :)
>
>> Out of curiosity I just checked and the lib/python2.3/config/Makefile  
>> file from my compiled from source python 2.3.4 don't appear to be  
>> patched with the patch below.
>
> The correct linker settings are not adopted by any mainline version of  
> Python 2.3.  I had thought they were in 2.3.4, but they're not.  Only  
> Python 2.4 builds correctly by default.  Also, Apple's build of Python  
> 2.3.3 in the WWDC 2004 preview  
> <http://www.opensource.apple.com/darwinsource/WWDC2004/python-11/>  
> adopts a similar method to this patch, but slightly modified so that  
> it does not depend on MACOSX_DEPLOYMENT_TARGET being set:
>
> (this is what a good  
> /System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
> python2.3/config/Makefile should look like, starting on or around line  
> 98)
>
> LDSHARED=   env MACOSX_DEPLOYMENT_TARGET=10.3 $(CC) $(LDFLAGS) -bundle  
> -undefined dynamic_lookup
> BLDSHARED=  env MACOSX_DEPLOYMENT_TARGET=10.3 $(CC) $(LDFLAGS) -bundle  
> -undefined dynamic_lookup
>
>> Am I correct in assuming that this is ok and that the patch below is  
>> only meant for the Apple-installed python 2.3?
>
> Unfortunately not, it recently came to my attention that it is VERY  
> IMPORTANT TO PATCH THIS MAKEFILE.  Here is some more information  
> (listing what will happen with various permutations of common Python  
> installations).  I hope to never have to repeat this again by email :)
>
> Versions of Mac OS X prior to 10.3: This patch isn't at all compatible  
> because dyld did not support this feature until Darwin 7.  That sucks,  
> but there's little that can be done about that!
>
> = Scenario 1 =
>
> [a] Python 2.3.0 in /System/Library, unpatched
> [b] Python 2.3.x in /Library, unpatched
>
> Building extensions with [a] no longer works, because -framework  
> Python locates [b].  Therefore, it actually ends up building  
> extensions only compatible with [b].
> Building extensions with [b] correctly creates extensions, but are  
> only compatible with [b].
> Loading correctly built extensions built for [a] from [b] or vice  
> versa will crash the interpreter.
>
> = Scenario 2 =
>
> [a] Python 2.3.0 in /System/Library, patched
> [b] Python 2.3.x in /Library, unpatched
>
> Building extensions with [a] will build extensions compatible with [b]  
> (or any other Python 2.3.x, even non-framework builds).
> Building extensions with [b] correctly creates extensions, but are  
> only compatible with [b].
> Loading correctly built extensions built for [b] from [a] will crash  
> the interpreter.
>
> = Scenario 3 =
>
> [a] Python 2.3.0 in /System/Library, unpatched
> [b] Python 2.3.x in /Library, patched
>
> Building extensions with [a] no longer works, because -framework  
> Python locates [b].  Therefore, it actually ends up building  
> extensions only compatible with [b].
> Building extensions with [b] will build extensions compatible with [a]  
> (or any other Python 2.3.x, even non-framework builds).
> Loading correctly built extensions built for [a] from [b] will crash  
> the interpreter.
>
> = Scenario 4 =
>
> [a] Python 2.3.0 in /System/Library, patched
> [b] Python 2.3.x in /Library, patched
>
> Building extensions with [a] will build extensions compatible with [b]  
> (or any other Python 2.3.x, even non-framework builds) and vice versa.
>
> = Scenario 5 =
>
> [a] Python 2.3.0 in /System/Library, unpatched
> [b] Python 2.3.x in /Library, unpatched
> [c] Python 2.4.x in /Library <no patch necessary>
>
> Building extensions with [a] no longer works, because -framework  
> Python locates [c].  These extensions are not compatible with ANY  
> Python (built with 2.3 headers, linked with 2.4).
> Building extensions with [b] no longer works, because -framework  
> Python locates [c].  These extensions are not compatible with ANY  
> Python (built with 2.3 headers, linked with 2.4).
> Loading correctly built extensions built for [a] from [b] or vice  
> versa will crash the interpreter.
>
> = Scenario 6 =
>
> [a] Python 2.3.0 in /System/Library, patched
> [b] Python 2.3.x in /Library, unpatched
> [c] Python 2.4.x in /Library <no patch necessary>
>
> Building extensions with [b] no longer works, because -framework  
> Python locates [c].  These extensions are not compatible with ANY  
> Python (built with 2.3 headers, linked with 2.4).
> Loading correctly built extensions built for [b] from [a] will crash  
> the interpreter.
>
> = Scenario 7 =
>
> [a] Python 2.3.0 in /System/Library, unpatched
> [b] Python 2.3.x in /Library, patched
> [c] Python 2.4.x in /Library <no patch necessary>
>
> Building extensions with [a] no longer works, because -framework  
> Python locates [c].  These extensions are not compatible with ANY  
> Python (built with 2.3 headers, linked with 2.4).
> Loading correctly built extensions built for [a] from [b] will crash  
> the interpreter.
>
> = Scenario 8 =
>
> [a] Python 2.3.0 in /System/Library, patched
> [b] Python 2.3.x in /Library, patched
> [c] Python 2.4.x in /Library <no patch necessary>
>
> Building extensions with [b] will build extensions compatible with [a]  
> (or any other Python 2.3.x, even non-framework builds) and vice versa.
>
> ...
>
> So obviously, it's very important to patch your Python!  The sooner  
> you patch, the less extensions you will build that are tied to a  
> particular installation location.  Note that nearly all precompiled  
> extensions you will find for Python 2.3.0 are generally of the  
> unpatched variety, so they aren't compatible with other installations  
> of Python 2.3.x.
>
> Considering [c] can screw up the whole mix, even though it doesn't  
> have it's own problems... it turns out that there is a fundamental  
> flaw in the way gcc's compiler and linker finds headers and dylibs  
> from frameworks!  It can't take version numbers into account!  The  
> runtime does just fine, but link time does not.. This is pretty bad!   
> It's not solvable without patching gcc either, because from the source  
> files you say <FrameworkName/HeaderName.h> but HeaderName.h doesn't  
> live in a directory named FrameworkName, it lives in a directory named  
> Headers, so it's not possible to specify an -I that works correctly if  
> you use this feature.
>
> There are a few workarounds to versioned framework issues, and they  
> all suck.
>
> [1] Keep versioned frameworks in separate places and use -F  
> appropriately.
> [2] Use -I with only "inside-framework" header names (i.e. <Python.h>  
> instead of <Python/Python.h>) and link directly to the framework's  
> dylib as if it were an .o file (i.e.  
> /Library/Frameworks/Python.framework/Versions/2.3/Python instead of  
> -framework Python)
> [3] Name your frameworks with the version number in them instead of  
> using the versioning capability of frameworks (i.e.  
> Python23.framework)
>
> Note that Python 2.4 uses a hybrid of [1] and [2].  When building the  
> Python framework, it uses -F into the build directory, so it always  
> links the interpreter correctly.  All sources expect the headers to be  
> available via "Python.h" and it uses an appropriate -I to make this  
> work.  When building extensions, it uses -undefined dynamic_lookup, so  
> it doesn't link directly to any Python.
>
>> Also in a separate but related e-mail you also suggested that one  
>> would "also need to set the "MACOSX_DEPLOYMENT_TARGET" environment  
>> variable to "10.3" when using this patch."
>>
>> Could you please elaborate on how/where one should set that  
>> environment variable (I set it in ~/.MacOSX/environment.plist).
>
> Setting it there is fine.  You could also, instead, adopt a better  
> patch like Apple has as stated above.
>
> -bob
>