[New-bugs-announce] [issue4060] PyUnicode_DecodeUTF16(..., byteorder=0) gets it wrong on Mac OS X/PowerPC

[Trent Mick]

New submission from Trent Mick <trentm at gmail.com>:

Revision 63955 removed a block from configure.in (and effectively from
pyconfig.h.in) having to do with endianness that results in an incorrect
setting for "WORDS_BIGENDIAN" in Universal builds on Mac OS X.

The removed part was this:

> AH_VERBATIM([WORDS_BIGENDIAN], 	 
> [ 	 
>  /* Define to 1 if your processor stores words with the most
significant byte 	 
>     first (like Motorola and SPARC, unlike Intel and VAX). 	 
>  	 
>     The block below does compile-time checking for endianness on
platforms 	 
>     that use GCC and therefore allows compiling fat binaries on OSX by
using 	 
>     '-arch ppc -arch i386' as the compile flags. The phrasing was
choosen 	 
>     such that the configure-result is used on systems that don't use
GCC. 	 
>   */ 	 
> #ifdef __BIG_ENDIAN__ 	 
> #define WORDS_BIGENDIAN 1 	 
> #else 	 
> #ifndef __LITTLE_ENDIAN__ 	 
> #undef WORDS_BIGENDIAN 	 
> #endif 	 
> #endif])

This used to allow "WORDS_BIGENDIAN" to be correct for all parts of a
universal Python build done via `gcc -arch i386 -arch ppc ...`.

This was originally added for issue 1471883 (see msg50040 for a
discussion of this particular bit).


The result of this bug is that Python extensions using either of the
following to get native byte ordering for UTF-16 decoding:

   PyUnicode_DecodeUTF16(..., byteorder=0);
   PyUnicode_DecodeUTF16Stateful(..., byteorder=0, ...);

on Mac OS X/PowerPC with a universal build built on Intel hardware (most
such builds) will get the wrong byte-ordering.


The fix is to restore that section to configure.in and re-run autoconf
and autoheader.


Ronald,

Was there are particular reason that this block was removed from
configure.in (and pyconfig.h.in)?

I'd like to hear comments from either Ronald or Martin, and then I can
commit the fix.

----------
components: Unicode
messages: 74398
nosy: loewis, ronaldoussoren, trentm
severity: normal
status: open
title: PyUnicode_DecodeUTF16(..., byteorder=0) gets it wrong on Mac OS X/PowerPC
type: behavior
versions: Python 2.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4060>
_______________________________________