before I open an issue re: posix.stat and/or os.stat
import sys sys.maxsize 9223372036854775807 import posix posix.stat("/tmp/xxx")
My focus is AIX - and I believe I found a bug in AIX include files in 64-bit mode. I'll take that up with IBM and AIX support. However, this issue might also be valid in Python3. The following is from Centos, not AIX Python 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. posix.stat_result(st_mode=33188, st_ino=33925869, st_dev=64768L, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1550742595, st_mtime=1550742595, st_ctime=1550742595)
st=posix.stat("/tmp/xxx") dev=st.st_dev min=posix.minor(dev) maj=posix.major(dev) min,max (0, <built-in function max>) min 0 max <built-in function max> maj 253 posix.minor(dev) 0 posix.major(655536) 2560 posix.major(65536) 256 posix.major(256) 1 import os os.major(256) 1
In AIX - 64-bit mode Python 3.8.0a1+ (heads/master:e7a4bb554e, Feb 20 2019, 18:40:08) [C] on aix7 Type "help", "copyright", "credits" or "license" for more information.
import sys,os,posix sys.maxsize 9223372036854775807 posix.major(256) 0 posix.major(65536) 1 posix.stat("/tmp/xxx") os.stat_result(st_mode=33188, st_ino=12, st_dev=-9223371993905102841, st_nlink=1, st_uid=202, st_gid=1954, st_size=0, st_atime=1550690105, st_mtime=1550690105, st_ctime=1550690105)
AIX 32-bit: root@x066:[/data/prj/python/git/python3-3.8.0.66]./python Python 3.8.0a1+ (heads/master:e7a4bb554e, Feb 19 2019, 11:22:56) [C] on aix6 Type "help", "copyright", "credits" or "license" for more information.
import os,sys,posix sys.maxsize 2147483647 posix.major(65536) 1 posix.stat("/tmp/xxx") os.stat_result(st_mode=33188, st_ino=149, st_dev=655367, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1550743517, st_mtime=1550743517, st_ctime=1550743517)
To make it easier to view: buildbot@x064:[/home/buildbot]cat osstat.c #include <sys/types.h> #include <sys/stat.h> #include <sys/sysmacros.h> #include <stdio.h> main() { dev_t dev; char *path = "/tmp/xxx"; struct stat st; int minor,major; lstat(path,&st); printf("size: %d\n", sizeof(st.st_dev)); dev = st.st_dev; minor = minor(dev); major = major(dev); printf("%016lx %ld %ld\n",dev,dev, (unsigned) dev); printf("%d,%d\n",major,minor); } buildbot@x064:[/home/buildbot]OBJECT_MODE=32 cc osstat.c -o osstat-32 && ./osstat-32 size: 4 00000000000a0007 655367 655367 10,7 And here is the AIX behavior (and bug - major() macro!) buildbot@x064:[/home/buildbot]OBJECT_MODE=64 cc osstat.c -o osstat-64 && ./osstat-64 size: 8 8000000a00000007 -9223371993905102841 7 0,7 The same on AIX 6 (above is AIX7) - and also with gcc: root@x068:[/data/prj]gcc -maix64 osstat.c -o osstat-64 && ./osstat-64 size: 8 8000000a00000007 -9223371993905102841 42949672967 0,7 root@x068:[/data/prj]gcc -maix32 osstat.c -o osstat-32 && ./osstat-32 size: 4 00000000000a0007 655367 0 10,7 root@x068:[/data/prj] So, the AIX 'bug' with the macro major() has been around for ages - but ALSO setting the MSB of the st_dev. +++++ Now my question: Will this continue to be enough space - i.e., is the Dev size going to be enough? +2042 #ifdef MS_WINDOWS +2043 PyStructSequence_SET_ITEM(v, 2, PyLong_FromUnsignedLong(st->st_dev)); +2044 #else +2045 PyStructSequence_SET_ITEM(v, 2, _PyLong_FromDev(st->st_dev)); +2046 #endif +711 #define _PyLong_FromDev PyLong_FromLongLong It seems so - however, Is there something such as PyUnsignedLong and is that large enough for a "long long"? and if it exists, would that make the value positive (for the first test). posix.major and os.major will need to mask away the MSB and posix.makedev and os.makedev will need to add it back. OR - do I need to make the PyStat values "the same" in both 32-bit and 64-bit? Puzzled on what you think is the correct approach. Michael
On 2/21/19 2:26 AM, Michael wrote:
Will this continue to be enough space - i.e., is the Dev size going to be enough?
+2042 #ifdef MS_WINDOWS +2043 PyStructSequence_SET_ITEM(v, 2, PyLong_FromUnsignedLong(st->st_dev)); +2044 #else +2045 PyStructSequence_SET_ITEM(v, 2, _PyLong_FromDev(st->st_dev)); +2046 #endif
+711 #define _PyLong_FromDev PyLong_FromLongLong
It seems so - however, Is there something such as PyUnsignedLong and is that large enough for a "long long"? and if it exists, would that make the value positive (for the first test).
Surely you can answer this second question yourself? You do have full source to the CPython interpreter. To answer your question: there is no PyUnsignedLong. Python 2 has a "native int" PyIntObject, which is for most integers you see in a program, and a "long" PyLongObject which is those integers that end in L. Python 3 only has the PyLongObject, which is used for all integers. The PyLongObject is an "arbitrary-precision integer": https://en.wikipedia.org/wiki/Arbitrary-precision_arithmetic and can internally expand as needed to accommodate any size integer, assuming you have enough heap space. Any "long long", unsigned or not, on any extant AIX platform, would be no problem to represent in a PyLongObject. You should use PyLong_FromLongLong or PyLong_FromUnsignedLongLong to create your PyLong object and populate the st_dev field of the os.stat() structsequence. //arry/
Shortening the original mail to something shorter. The reason I am starting here, in -dev, rather than as an issue directly, is because I would like some direction/recommendation from concerned individuals before I take a "outsider" approach. Too often I have learned that I guessed wrong how the Python community "sees the world".
On 2/21/2019 11:26 AM, Michael wrote: It seems so - however, Is there something such as PyUnsignedLong and is that large enough for a "long long"? and if it exists, would that make the value positive (for the first test).
posix.major and os.major will need to mask away the MSB and posix.makedev and os.makedev will need to add it back.
OR - do I need to make the PyStat values "the same" in both 32-bit and 64-bit? I think I already have the answer to this - follow the platform (as far as size goes).
On 32-bit max(1) == 256 (2^^8) and on 64-bit 65536 (2^^16). However, I am still puzzeled re: the idiosyncratic MSB addition - that the value for st.st_dev on AIX 64-bit has always (as far back as I could look at least) had the MSB set so that, as a signed number st.st_dev is always negative. And MSB is not part of the major device number. It's just there. So, it has been a few days since I studied this (waiting for reactions) - but is the return suppossed to always be a value returned by the posixmodule.c (if so, can strip the MSB bit and make everything positive - even for st.st_dev (but that would break under a direct comparision with a C program that does not strip the MSB). Currently, the macros on AIX for 64-bit are broken, so I cannot guess how they will decide to implement major() and mkdevice() as far as striping (and adding) the MSB when "constructing" the st_dev number from two "regular" major and minor numbers. Parked for now, Best wishes, Michael
Puzzled on what you think is the correct approach.
Michael
participants (3)
-
Larry Hastings -
Michael -
Michael Felt