struct module docs vs reality
The documentation for the struct module says: http://docs.python.org/dev/library/struct.html#module-struct "short is 2 bytes; int and long are 4 bytes; long long (__int64 on Windows) is 8 bytes" and lists 'l' and 'L' as the pack code for a C long. As its implemented today, the documentation is incorrect. On an LP64 host (pretty much any 64-bit linux, bsd or unixish thing) a long is 8 bytes. I assume this means there is existing code out there that expects the current not-as-documented behavior. There is also code out there that expects the documented behavior but behaves wrong when a 64bit Python is used. I assume I should just fix the documentation and anything in Lib that uses the struct module incorrectly (zipfile for example) rather than change the behavior? This is for http://bugs.python.org/issue1789
Gregory P. Smith schrieb:
The documentation for the struct module says:
http://docs.python.org/dev/library/struct.html#module-struct
"short is 2 bytes; int and long are 4 bytes; long long (__int64 on Windows) is 8 bytes"
and lists 'l' and 'L' as the pack code for a C long.
As its implemented today, the documentation is incorrect. On an LP64 host (pretty much any 64-bit linux, bsd or unixish thing) a long is 8 bytes.
I assume this means there is existing code out there that expects the current not-as-documented behavior. There is also code out there that expects the documented behavior but behaves wrong when a 64bit Python is used.
I assume I should just fix the documentation and anything in Lib that uses the struct module incorrectly (zipfile for example) rather than change the behavior?
+1 (actually +100) from me. Thomas
On 1/23/08, Thomas Heller
The documentation for the struct module says:
http://docs.python.org/dev/library/struct.html#module-struct
"short is 2 bytes; int and long are 4 bytes; long long (__int64 on Windows) is 8 bytes"
and lists 'l' and 'L' as the pack code for a C long.
As its implemented today, the documentation is incorrect. On an LP64 host (pretty much any 64-bit linux, bsd or unixish thing) a long is 8 bytes.
I assume this means there is existing code out there that expects the current not-as-documented behavior. There is also code out there that expects the documented behavior but behaves wrong when a 64bit Python is used.
I assume I should just fix the documentation and anything in Lib that uses the struct module incorrectly (zipfile for example) rather than change
Gregory P. Smith schrieb: the
behavior?
+1 (actually +100) from me.
Thomas
Ok, its a pretty big diff. Much of the standard library is completely broken when used on LP64 platforms. I added it to the bug. I'm running unit tests now. These fixes should go into 2.5.2. The diff could use some eyes on it, especially the Mac API stuff that I know nothing about but am assuming relies on 32bit values. http://bugs.python.org/issue1789 Greg
Gregory P. Smith wrote:
The documentation for the struct module says:
http://docs.python.org/dev/library/struct.html#module-struct
" short is 2 bytes; int and long are 4 bytes; long long ( __int64 on Windows) is 8 bytes"
and lists 'l' and 'L' as the pack code for a C long.
As its implemented today, the documentation is incorrect. On an LP64 host (pretty much any 64-bit linux, bsd or unixish thing) a long is 8 bytes.
You overlooked the words "Standard size and alignment are as follows" that start the quoted paragraph. It's a little confusing because standard size is not the default. The default is platform-specific sizes. Only if you start the format string with >, <, ! or = do you get standard sizes. The reference documentation is correct as it stands, and, I suspect, so is the LP64 implementation. Doesn't struct.pack('>l',42) produce a 4-byte string on LP64? The tutorial at http://docs.python.org/tut/node13.html#SECTION0013300000000000000000%3E has a bug though: the format string should use '<'. I believe zipfile.py correctly uses '<' throughout. regards, Anders
Oh good. Reading the Modules/_struct.c code I see that is indeed what happens. There are still several instances of misused struct pack and unpack strings in Lib but the problem is less serious, I'll make a new patch that just addresses those.
participants (3)
-
Anders J. Munch
-
Gregory P. Smith
-
Thomas Heller