[Pythonmac-SIG] building universal binaries

[Bob Ippolito]

On Feb 3, 2006, at 10:59 AM, Ronald Oussoren wrote:

>
> On 3-feb-2006, at 18:28, Bob Ippolito wrote:
>
>> On Jan 27, 2006, at 12:19 AM, Ronald Oussoren wrote:
>>
>>>
>>> On 27-jan-2006, at 8:46, Kevin Ollivier wrote:
>>>
>>>> On Jan 26, 2006, at 10:55 PM, Bob Ippolito wrote:
>>>>
>>>>> Some extensions aren't going to build cleanly universal, and  
>>>>> most users probably aren't going to have all the SDKs installed  
>>>>> so if it shipped with universal flags then it wouldn't be able  
>>>>> to build extensions.
>>>>
>>>> Unless we had distutils check for the existence of the Universal  
>>>> SDK before setting the flags, or probably more accurately,  
>>>> remove them if the SDK isn't available. Then perhaps add a  
>>>> distutils 'flag' to manually turn off Universal building for  
>>>> extensions that can't be fixed. This might be a bit of a hack,  
>>>> but I don't imagine it would be too painful. The question is how  
>>>> many extensions would have troubles... But I guess the only way  
>>>> we'll know is to test the waters. In any case, even if it  
>>>> wouldn't be reasonable to make a Universal build the "official"  
>>>> build, it would be very useful to people packaging apps, even if  
>>>> it were Tiger+ only.
>>>
>>> I know of one popular extension (cElementTree) that won't build  
>>> with '-arch i386 -arch ppc' because it calcutes a #define based  
>>> on the current byte-order. There might be others, I haven't done  
>>> a thorough search yet.
>>
>> In my python24-fat branch, I made the following change to  
>> pyconfig.h.in which I believe will resolve that issue in any  
>> extension that doesn't do something dumb like trying to grep the  
>> endianness out of pyconfig.h:
>>
>> http://svn.red-bean.com/bob/python24-fat/pyconfig.h.in
>>
>> /* Define to 1 if your processor stores words with the most  
>> significant byte
>>    first (like Motorola and SPARC, unlike Intel and VAX). */
>> #ifdef __BIG_ENDIAN__
>> #define WORDS_BIGENDIAN 1
>> #else
>> #ifndef __LITTLE_ENDIAN__
>> #undef WORDS_BIGENDIAN
>> #endif
>> #endif
>>
>> GCC defines __BIG_ENDIAN__ to 1 when it's compiling a big endian  
>> arch... and __LITTLE_ENDIAN__ to 1 when it's compiling on a little  
>> endian arch.  Configured on i386, this turns into:
>>
>> #ifdef __BIG_ENDIAN__
>> #define WORD_BIGENDIAN 1
>> #else
>> #ifndef __LITTLE_ENDIAN__
>> /* #undef WORDS_BIGENDIAN */
>> #endif
>> #endif
>>
>> When configured on PPC, it turns into:
>>
>> #ifdef __BIG_ENDIAN__
>> #define WORD_BIGENDIAN 1
>> #else
>> #ifndef __LITTLE_ENDIAN__
>> #define WORDS_BIGENDIAN 1
>> #endif
>> #endif
>>
>> Since this is GCC, the #ifndef __LITTLE_ENDIAN__ branch is always  
>> false... so I'm making this a compile-time not configure-time  
>> value, but I left in the configure-time stuff for other platforms.
>
> Did you test this? I'd be surprised if distutils honours the ifdef  
> statement. Luckily the easiest way to find the byteorder is  
> sys.byteorder.

Yes, I ran the test suite on PPC with a PPC-built build and on i386  
with and without Rosetta on a i386 build... I didn't test the PPC- 
built version on an i386, but I'm going to assume it works for now  
since the two possible outputs above are logically equivalent.

distutils does know how to parse and find pyconfig.h  
(sysconfig.parse_config_h, get_config_h_filename), but Python itself  
only uses that functionality in the main setup.py.  Surprisingly, the  
main setup.py parses pyconfig.h twice (once with distutils, once  
without -- good job there), but it of course doesn't go anywhere near  
WORDS_BIGENDIAN.

Packages that use WORDS_BIGENDIAN from distutils are broken.  There's  
nothing we can do to fix that short of patching the package.   
Fortunately I don't think any package is dumb enough to do that.

Packages that do something with sys.byteorder at setup.py time are  
probably going to be broken also... but I've never seen that either.

> I have a patched version of pyconfig.h that #includes "macosx- 
> config.h" at the end. This file redefines a number of values based  
> on the cpu-type. Anyone that grabs CPU information from pyconfig.h  
> will get the values from the platform that was used to generate  
> pyconfig.h (an intel system in my case). I like your trick better,  
> mine is a bit too elaborate and not needed until someone wants to  
> do a ppc64 build.

sys.byteorder is actually determined at runtime:

         unsigned long number = 1;
         char *value;

         s = (char *) &number;
         if (s[0] == 0)
             value = "big";
         else
             value = "little";
         PyDict_SetItemString(sysdict, "byteorder",
                      v = PyString_FromString(value));
         Py_XDECREF(v);

-bob