[Python-Dev] test_gzip/test_tarfile failure om AMD64

[Bob Ippolito]

On May 29, 2006, at 1:18 PM, Bob Ippolito wrote:

>
> On May 29, 2006, at 12:44 PM, Guido van Rossum wrote:
>
>> On 5/29/06, Tim Peters <tim.peters at gmail.com> wrote:
>>>> I think we should do as Thomas proposes: plan to make it an  
>>>> error in
>>>> 2.6 (or 2.7 if there's a big outcry, which I don't expect) and
>>>> accept
>>>> it with a warning in 2.5.
>>>
>>> That's what I arrived at, although 2.4.3's checking behavior is
>>> actually so inconsistent that "it" needs some defining (what exactly
>>> are we trying to still accept?  e.g., that -1 doesn't trigger "I"
>>> complaints but that -1L does above?  that one's surely a bug).
>>
>> No, it reflects that (up to 2.3 I believe) 0xffffffff was -1 but
>> 0xffffffffL was 4294967295L.
>
> Python 2.3 did a FutureWarning on 0xffffffff but its value was -1.
>
> Anyway, my plan is to make it such that all non-native format codes
> will behave exactly like C casting, but will do a DeprecationWarning
> for input numbers that were initially out of bounds. This behavior
> will be consistent across (python) int and long, and will be easy
> enough to explain in the docs (but still more complicated than
> "values not representable by this data type will raise struct.error").
>
> This means that I'm also changing it so that struct.pack will not
> raise OverflowError for some longs, it will always raise struct.error
> or do a warning (as long as the input is int or long).
>
> Pseudocode looks kinda like this:
>
> def wrap_unsigned(x, CTYPE):
> 	if not (0 <= x <= CTYPE_MAX):
> 		DeprecationWarning()
> 		x &= CTYPE_MAX
> 	return x
>
> Actually, should this be a FutureWarning or a DeprecationWarning?

OK, this behavior is implemented in revision 46537:

(this is from ./python.exe -Wall)

 >>> import struct
 >>> struct.pack('B', 256)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/Users/bob/src/python/Lib/struct.py", line 63, in pack
     return o.pack(*args)
struct.error: ubyte format requires 0 <= number <= 255
 >>> struct.pack('B', -1)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/Users/bob/src/python/Lib/struct.py", line 63, in pack
     return o.pack(*args)
struct.error: ubyte format requires 0 <= number <= 255
 >>> struct.pack('<B', 256)
/Users/bob/src/python/Lib/struct.py:63: DeprecationWarning: 'B'  
format requires 0 <= number <= 255
   return o.pack(*args)
'\x00'
 >>> struct.pack('<B', -1)
/Users/bob/src/python/Lib/struct.py:63: DeprecationWarning: struct  
integer wrapping is deprecated
   return o.pack(*args)
/Users/bob/src/python/Lib/struct.py:63: DeprecationWarning: 'B'  
format requires 0 <= number <= 255
   return o.pack(*args)
'\xff'
 >>> struct.pack('<B', 256L)
/Users/bob/src/python/Lib/struct.py:63: DeprecationWarning: 'B'  
format requires 0 <= number <= 255
   return o.pack(*args)
'\x00'
 >>> struct.pack('<B', -1L)
/Users/bob/src/python/Lib/struct.py:63: DeprecationWarning: struct  
integer wrapping is deprecated
   return o.pack(*args)
/Users/bob/src/python/Lib/struct.py:63: DeprecationWarning: 'B'  
format requires 0 <= number <= 255
   return o.pack(*args)
'\xff'

In _struct.c, getting rid of the #define PY_STRUCT_WRAPPING 1 will  
turn off this warning+wrapping nonsense and just raise errors for out  
of range values. It'll also enable some additional performance hacks  
(swapping out the host-endian table's pack and unpack functions with  
the faster native versions).

-bob