array and struct 64-bit Linux change in behavior Python 3.7 and 2.7
Chris Angelico
rosuav at gmail.com
Wed Dec 4 00:48:42 EST 2019
On Wed, Dec 4, 2019 at 4:16 PM Chris Clark <Chris.Clark at actian.com> wrote:
> I think the consensus from the various threads is that the docs are either lacking or misleading.
>
> I mentioned that this impacts bytes and the problem there is more telling as it hard fails (this is how I first discovered this was an issue):
>
> >>> array.array('L', b'\0\0\0\0')
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> ValueError: string length not a multiple of item size
>
> I don't believe the documentation is accurate by using the word "minimum". Minimum would suggest that it would accept a 4-byte value as a minimum and on 64-bit it does *not*, it hard fails. If it were to document that, "the sizes are native integer types for the platform, the table documents some typical but *not* guaranteed sizes", that would be more clear.
>
I think array.array() is possibly the wrong tool for this job. If you
have a collection of bytes from some well-defined source (eg you're
parsing a file in a known format), struct is better suited to it,
because it's easy to define both the size and byte order.
> For my uses case I'm seriously thinking about not using array moving forward and only using struct. I briefly wondered about ctypes (it has nice names, e.g. c_int64 that are unambiguous) but then I remembered it is not available in Jython).
>
I wouldn't bother with ctypes for this type of job.
> Regarding Barry's comment, yep size consistency with array is a pain - what I implemented as workaround is below (and likely to be my solution going forward):
>
> x = array.array('L', [0])
> if x.itemsize == 4:
> FMT_ARRAY_4BYTE = 'L'
> FMT_STRUCT_4BYTE = '<L'
> else:
> x = array.array('I', [0])
> if x.itemsize == 4:
> FMT_ARRAY_4BYTE = 'I'
> FMT_STRUCT_4BYTE = '<L'
> del(x)
>
> and then use the constants in array/struct calls where (binary) file IO is happening.
Yep, looks like struct is the way to go here. (Especially since you
don't have a final 'else'.)
ChrisA
More information about the Python-list
mailing list