Python-Dev February 2019

python-dev@python.org

86 participants
48 discussions

by Facundo Batista

Hello! This list (which I co-admin, with Georg) is getting less and less traffic as months pass by. See: https://mail.python.org/pipermail/python-porting/ The interwebs has been collecting ton of resources about porting py2 to 3 during these years. Any not-yet-answered question surely can be done in a list with more participants. Can we kill this list? Thanks! Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org.ar/ Twitter: @facundobatista

8 months

congrats on 3.5! Alas, windows 7 users are having problems installing it

by Laura Creighton

webmaster has already heard from 4 people who cannot install it. I sent them to the bug tracker or to python-list but they seem not to have gone either place. Is there some guide I should be sending them to, 'how to debug installation problems'? Laura

1 year

why is not 64-bit installer the default download link for Windows?

by Cosimo Lupo

If one goes to httWhps://www.python.org/downloads <https://www.python.org/downloads> from a Windows browser, the default download URL is for the 32-bit installer instead of the 64-bit one. I wonder why is this still the case? Shouldn't we encourage new Windows users (who may not even know the distinction between the two architectures) to use the 64-bit version of Python, since most likely they can? If this is not the correct forum for this, please let me know where I can direct my question/feature request, thanks. Cosimo -- Cosimo Lupo

1 year, 3 months

int() and math.trunc don't accept objects that only define __index__

by Rémi Lapeyre

Hi, I open this thread to discuss the proposal by Nick Coghlan in https://bugs.python.org/issue33039 to add __int__ and __trunc__ to a type when __index__ is defined. Currently __int__ does not default to __index__ during class initialisation so both must be defined to get a coherant behavior: (cpython-venv) ➜ cpython git:(add-key-argument-to-bisect) ✗ python3 Python 3.8.0a1+ (heads/add-key-argument-to-bisect:b7aaa1adad, Feb 18 2019, 16:10:22) [Clang 10.0.0 (clang-1000.10.44.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import math >>> class MyInt: ... def __index__(self): ... return 4 ... >>> int(MyInt()) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: int() argument must be a string, a bytes-like object or a number, not 'MyInt' >>> math.trunc(MyInt()) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: type MyInt doesn't define __trunc__ method >>> hex(MyInt()) '0x4' >>> len("a"*MyInt()) 4 >>> MyInt.__int__ = MyInt.__index__ >>> int(MyInt()) 4 The difference in behavior is espacially weird in builtins like int() and hex(). The documentation mentions at https://docs.python.org/3/reference/datamodel.html#object.__index__ the need to always define both __index__ and __int__: Note: In order to have a coherent integer type class, when __index__() is defined __int__() should also be defined, and both should return the same value. Nick Coghlan proposes to make __int__ defaults to __index__ when only the second is defined and asked to open a discussion on python-dev before making any change "as the closest equivalent we have to this right now is the "negative" derivation, where overriding __eq__ without overriding __hash__ implicitly marks the derived class as unhashable (look for "type->tp_hash = PyObject_HashNotImplemented;").". I think the change proposed makes more sense than the current behavior and volunteer to implement it if it is accepted. What do you think about this?

1 year, 6 months

Possible performance regression

by Raymond Hettinger

I'll been running benchmarks that have been stable for a while. But between today and yesterday, there has been an almost across the board performance regression. It's possible that this is a measurement error or something unique to my system (my Mac installed the 10.14.3 release today), so I'm hoping other folks can run checks as well. Raymond -- Yesterday ------------------------------------------------------------------------ $ ./python.exe Tools/scripts/var_access_benchmark.py Variable and attribute read access: 4.0 ns read_local 4.5 ns read_nonlocal 13.1 ns read_global 17.4 ns read_builtin 17.4 ns read_classvar_from_class 15.8 ns read_classvar_from_instance 24.6 ns read_instancevar 19.7 ns read_instancevar_slots 18.5 ns read_namedtuple 26.3 ns read_boundmethod Variable and attribute write access: 4.6 ns write_local 4.8 ns write_nonlocal 17.5 ns write_global 39.1 ns write_classvar 34.4 ns write_instancevar 25.3 ns write_instancevar_slots Data structure read access: 17.5 ns read_list 18.4 ns read_deque 19.2 ns read_dict Data structure write access: 19.0 ns write_list 22.0 ns write_deque 24.4 ns write_dict Stack (or queue) operations: 55.5 ns list_append_pop 46.3 ns deque_append_pop 46.7 ns deque_append_popleft Timing loop overhead: 0.3 ns loop_overhead -- Today --------------------------------------------------------------------------- $ ./python.exe py Tools/scripts/var_access_benchmark.py Variable and attribute read access: 5.0 ns read_local 5.3 ns read_nonlocal 14.7 ns read_global 18.6 ns read_builtin 19.9 ns read_classvar_from_class 17.7 ns read_classvar_from_instance 26.1 ns read_instancevar 21.0 ns read_instancevar_slots 21.7 ns read_namedtuple 27.8 ns read_boundmethod Variable and attribute write access: 6.1 ns write_local 7.3 ns write_nonlocal 18.9 ns write_global 40.7 ns write_classvar 36.2 ns write_instancevar 26.1 ns write_instancevar_slots Data structure read access: 19.1 ns read_list 19.6 ns read_deque 20.6 ns read_dict Data structure write access: 22.8 ns write_list 23.5 ns write_deque 27.8 ns write_dict Stack (or queue) operations: 54.8 ns list_append_pop 49.5 ns deque_append_pop 49.4 ns deque_append_popleft Timing loop overhead: 0.3 ns loop_overhead

1 year, 6 months

[bpo-35155] Requesting a review

by Denton Liu

Hello all, A couple months back, I reported bpo-35155[1] and I submitted a PR for consideration[2]. After a couple of reviews, it seems like progress has stalled. Would it be possible for someone to review this? Thanks, Denton [1]: https://bugs.python.org/issue35155 [2]: https://github.com/python/cpython/pull/10313

1 year, 6 months

Compact ordered set

by INADA Naoki

Hello, folks. I'm working on compact and ordered set implementation. It has internal data structure similar to new dict from Python 3.6. It is still work in progress. Comments, tests, and documents should be updated. But it passes existing tests excluding test_sys and test_gdb (both tests checks implementation detail) https://github.com/methane/cpython/pull/16 Before completing this work, I want to evaluate it. Following is my current thoughts about the compact ordered set. ## Preserving insertion order Order is not fundamental for set. There are no order in set in the math world. But it is convenient sometime in real world. For example, it makes doctest easy. When writing set to logs, we can use "grep" command if print order is stable. pyc is stable without PYTHONHASHSEED=0 hack. Additionally, consistency with dict is desirable. It removes one pitfall for new Python users. "Remove duplicated items from list" idiom become `list(set(duplicated))` from `list(dict.fromkeys(duplicated))`. ## Memory efficiency Hash table has dilemma. To reduce collision rate, hash table should be sparse. But it wastes memory. Since current set is optimized for both of hit and miss cases, it is more sparse than dict. (It is bit surprise that set typically uses more memory than same size dict!) New implementation partially solve this dilemma. It has sparse "index table" which items are small (1byte when table size <= 256, 2bytes when table size <= 65536), and dense entry table (each item has key and hash, which is 16bytes on 64bit system). I use 1/2 for capacity rate for now. So new implementation is memory efficient when len(s) <= 32768. But memory efficiency is roughly equal to current implementation when 32768 < len(s) <= 2**31, and worse than current implementation when len(s) > 2**31. Here is quick test about memory usage. https://gist.github.com/methane/98b7f43fc00a84964f66241695112e91 # Performance pyperformance result: $ ./python -m perf compare_to master.json oset2.json -G --min-speed=2 Slower (3): - unpickle_list: 8.48 us +- 0.09 us -> 12.8 us +- 0.5 us: 1.52x slower (+52%) - unpickle: 29.6 us +- 2.5 us -> 44.1 us +- 2.5 us: 1.49x slower (+49%) - regex_dna: 448 ms +- 3 ms -> 462 ms +- 2 ms: 1.03x slower (+3%) Faster (4): - meteor_contest: 189 ms +- 1 ms -> 165 ms +- 1 ms: 1.15x faster (-13%) - telco: 15.8 ms +- 0.2 ms -> 15.3 ms +- 0.2 ms: 1.03x faster (-3%) - django_template: 266 ms +- 6 ms -> 259 ms +- 3 ms: 1.03x faster (-3%) - unpickle_pure_python: 818 us +- 6 us -> 801 us +- 9 us: 1.02x faster (-2%) Benchmark hidden because not significant (49) unpickle and unpickle_list shows massive slowdown. I suspect this slowdown is not caused from set change. Linux perf shows many pagefault is happened in pymalloc_malloc. I think memory usage changes hit weak point of pymalloc accidentally. I will try to investigate it. On the other hand, meteor_contest shows 13% speedup. It uses set. Other doesn't show significant performance changes. I need to write more benchmarks for various set workload. I expect new set is faster on simple creation, iteration and destruction. Especially, sequential iteration and deletion will reduce cache misses. (e.g. https://bugs.python.org/issue32846 ) On the other hand, new implementation will be slow on complex (heavy random add & del) case. ----- Any comments are welcome. And any benchmark for set workloads are very welcome. Regards, -- INADA Naoki <songofacandy(a)gmail.com>

1 year, 6 months

Summary of Python tracker Issues

by Python tracker

ACTIVITY SUMMARY (2019-02-01 - 2019-02-08) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6998 (+13) closed 40696 (+47) total 47694 (+60) Open issues with patches: 2783 Issues opened (43) ================== #35885: configparser: indentation https://bugs.python.org/issue35885 opened by mrs.red #35886: Move PyInterpreterState into Include/internal/pycore_pystate.h https://bugs.python.org/issue35886 opened by eric.snow #35887: Doc string for updating the frozen version of importlib in _bo https://bugs.python.org/issue35887 opened by nnja #35888: ssl module - could not get the server certificate w/o complete https://bugs.python.org/issue35888 opened by Lee Eric #35889: sqlite3.Row doesn't have useful repr https://bugs.python.org/issue35889 opened by vlad #35891: urllib.parse.splituser has no suitable replacement https://bugs.python.org/issue35891 opened by jaraco #35892: Fix awkwardness of statistics.mode() for multimodal datasets https://bugs.python.org/issue35892 opened by rhettinger #35893: distutils fails to build extension on windows when it is a pac https://bugs.python.org/issue35893 opened by ronaldoussoren #35898: The TARGETDIR variable must be provided when invoking this ins https://bugs.python.org/issue35898 opened by Thomas Trummer #35899: '_is_sunder' function in 'enum' module fails on empty string https://bugs.python.org/issue35899 opened by Maxpxt #35900: Add pickler hook for the user to customize the serialization o https://bugs.python.org/issue35900 opened by pierreglaser #35901: json.dumps infinite recurssion https://bugs.python.org/issue35901 opened by MultiSosnooley #35903: Build of posixshmem.c should probe for required OS functions https://bugs.python.org/issue35903 opened by nascheme #35904: Add statistics.fmean(seq) https://bugs.python.org/issue35904 opened by rhettinger #35905: macOS build docs need refresh (2019) https://bugs.python.org/issue35905 opened by jaraco #35906: Header Injection in urllib https://bugs.python.org/issue35906 opened by push0ebp #35907: Unnecessary URL scheme exists to allow file:// reading file i https://bugs.python.org/issue35907 opened by push0ebp #35912: _testembed.c fails to compile when using --with-cxx-main in th https://bugs.python.org/issue35912 opened by pablogsal #35913: asyncore: allow handling of half closed connections https://bugs.python.org/issue35913 opened by Isaac Boukris #35915: re.search extreme slowness (looks like hang/livelock), searchi https://bugs.python.org/issue35915 opened by benspiller #35918: multiprocessing's SyncManager.dict.has_key() method is broken https://bugs.python.org/issue35918 opened by giampaolo.rodola #35919: multiprocessing: shared manager Pool fails with AttributeError https://bugs.python.org/issue35919 opened by giampaolo.rodola #35920: Windows 10 ARM32 platform support https://bugs.python.org/issue35920 opened by steve.dower #35921: Use ccache by default https://bugs.python.org/issue35921 opened by pitrou #35922: robotparser crawl_delay and request_rate do not work with no m https://bugs.python.org/issue35922 opened by joseph_myers #35923: Update the BuiltinImporter in importlib to use loader._ORIGIN https://bugs.python.org/issue35923 opened by nnja #35924: curses segfault resizing window https://bugs.python.org/issue35924 opened by Josiah Ulfers #35925: test_httplib test_nntplib test_ssl fail on ARMv7 Debian buster https://bugs.python.org/issue35925 opened by pablogsal #35926: Need openssl 1.1.1 support on Windows for ARM and ARM64 https://bugs.python.org/issue35926 opened by Paul Monson #35927: Intra-package References Documentation Incomplete https://bugs.python.org/issue35927 opened by ADataGman #35928: socket makefile read-write discards received data https://bugs.python.org/issue35928 opened by pravn #35930: Raising an exception raised in a "future" instance will create https://bugs.python.org/issue35930 opened by jcea #35931: pdb: "debug print(" crashes with SyntaxError https://bugs.python.org/issue35931 opened by blueyed #35933: python doc does not say that the state kwarg in Pickler.save_r https://bugs.python.org/issue35933 opened by pierreglaser #35934: Add socket.bind_socket() utility function https://bugs.python.org/issue35934 opened by giampaolo.rodola #35935: threading.Event().wait() not interruptable with Ctrl-C on Wind https://bugs.python.org/issue35935 opened by Chris Billington #35936: Give modulefinder some much-needed updates. https://bugs.python.org/issue35936 opened by brandtbucher #35937: Add instancemethod to types.py https://bugs.python.org/issue35937 opened by bup #35939: Remove urllib.parse._splittype from mimetypes.guess_type https://bugs.python.org/issue35939 opened by corona10 #35940: multiprocessing manager tests fail in the Refleaks buildbots https://bugs.python.org/issue35940 opened by pablogsal #35941: ssl.enum_certificates() regression https://bugs.python.org/issue35941 opened by schlenk #35942: posixmodule.c:path_converter() returns an invalid exception me https://bugs.python.org/issue35942 opened by lukasz.langa #35943: PyImport_GetModule() can return partially-initialized module https://bugs.python.org/issue35943 opened by pitrou Most recent 15 issues with no replies (15) ========================================== #35942: posixmodule.c:path_converter() returns an invalid exception me https://bugs.python.org/issue35942 #35940: multiprocessing manager tests fail in the Refleaks buildbots https://bugs.python.org/issue35940 #35939: Remove urllib.parse._splittype from mimetypes.guess_type https://bugs.python.org/issue35939 #35936: Give modulefinder some much-needed updates. https://bugs.python.org/issue35936 #35934: Add socket.bind_socket() utility function https://bugs.python.org/issue35934 #35931: pdb: "debug print(" crashes with SyntaxError https://bugs.python.org/issue35931 #35930: Raising an exception raised in a "future" instance will create https://bugs.python.org/issue35930 #35928: socket makefile read-write discards received data https://bugs.python.org/issue35928 #35927: Intra-package References Documentation Incomplete https://bugs.python.org/issue35927 #35926: Need openssl 1.1.1 support on Windows for ARM and ARM64 https://bugs.python.org/issue35926 #35924: curses segfault resizing window https://bugs.python.org/issue35924 #35920: Windows 10 ARM32 platform support https://bugs.python.org/issue35920 #35919: multiprocessing: shared manager Pool fails with AttributeError https://bugs.python.org/issue35919 #35918: multiprocessing's SyncManager.dict.has_key() method is broken https://bugs.python.org/issue35918 #35912: _testembed.c fails to compile when using --with-cxx-main in th https://bugs.python.org/issue35912 Most recent 15 issues waiting for review (15) ============================================= #35936: Give modulefinder some much-needed updates. https://bugs.python.org/issue35936 #35934: Add socket.bind_socket() utility function https://bugs.python.org/issue35934 #35931: pdb: "debug print(" crashes with SyntaxError https://bugs.python.org/issue35931 #35926: Need openssl 1.1.1 support on Windows for ARM and ARM64 https://bugs.python.org/issue35926 #35922: robotparser crawl_delay and request_rate do not work with no m https://bugs.python.org/issue35922 #35921: Use ccache by default https://bugs.python.org/issue35921 #35920: Windows 10 ARM32 platform support https://bugs.python.org/issue35920 #35913: asyncore: allow handling of half closed connections https://bugs.python.org/issue35913 #35906: Header Injection in urllib https://bugs.python.org/issue35906 #35903: Build of posixshmem.c should probe for required OS functions https://bugs.python.org/issue35903 #35900: Add pickler hook for the user to customize the serialization o https://bugs.python.org/issue35900 #35887: Doc string for updating the frozen version of importlib in _bo https://bugs.python.org/issue35887 #35886: Move PyInterpreterState into Include/internal/pycore_pystate.h https://bugs.python.org/issue35886 #35878: ast.c: end_col_offset may be used uninitialized in this functi https://bugs.python.org/issue35878 #35876: test_start_new_session for posix_spawnp fails https://bugs.python.org/issue35876 Top 10 most discussed issues (10) ================================= #35813: shared memory construct to avoid need for serialization betwee https://bugs.python.org/issue35813 19 msgs #35904: Add statistics.fmean(seq) https://bugs.python.org/issue35904 11 msgs #35913: asyncore: allow handling of half closed connections https://bugs.python.org/issue35913 11 msgs #35921: Use ccache by default https://bugs.python.org/issue35921 9 msgs #35706: Make it easier to use a venv with an embedded Python interpret https://bugs.python.org/issue35706 7 msgs #35893: distutils fails to build extension on windows when it is a pac https://bugs.python.org/issue35893 7 msgs #30670: pprint for dict in sorted order or insert order? https://bugs.python.org/issue30670 6 msgs #35907: Unnecessary URL scheme exists to allow file:// reading file i https://bugs.python.org/issue35907 6 msgs #34572: C unpickling bypasses import thread safety https://bugs.python.org/issue34572 5 msgs #35933: python doc does not say that the state kwarg in Pickler.save_r https://bugs.python.org/issue35933 5 msgs Issues closed (46) ================== #20001: pathlib inheritance diagram too large https://bugs.python.org/issue20001 closed by inada.naoki #22474: No explanation of how a task gets destroyed in asyncio 'task' https://bugs.python.org/issue22474 closed by cheryl.sabella #24087: Documentation doesn't explain the term "coroutine" (PEP 342) https://bugs.python.org/issue24087 closed by paul.moore #24209: Allow IPv6 bind in http.server https://bugs.python.org/issue24209 closed by jaraco #26256: Fast decimalisation and conversion to other bases https://bugs.python.org/issue26256 closed by skrah #27344: zipfile *does* support utf-8 filenames https://bugs.python.org/issue27344 closed by cheryl.sabella #29734: os.stat handle leak https://bugs.python.org/issue29734 closed by steve.dower #30130: array.array is not an instance of collections.MutableSequence https://bugs.python.org/issue30130 closed by cheryl.sabella #32560: [EASY C] inherit the py launcher's STARTUPINFO https://bugs.python.org/issue32560 closed by steve.dower #33316: Windows: PyThread_release_lock always fails https://bugs.python.org/issue33316 closed by steve.dower #33895: LoadLibraryExW called with GIL held can cause deadlock https://bugs.python.org/issue33895 closed by steve.dower #34691: _contextvars missing in xmaster branch Windows build? https://bugs.python.org/issue34691 closed by steve.dower #35299: LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found https://bugs.python.org/issue35299 closed by steve.dower #35358: Document that importlib.import_module accepts names that are n https://bugs.python.org/issue35358 closed by matrixise #35485: tkinter windows turn black while resized using Tk 8.6.9.1 on m https://bugs.python.org/issue35485 closed by ned.deily #35605: backported patch requires new sphinx, minimum sphinx version w https://bugs.python.org/issue35605 closed by ned.deily #35606: Add prod() function to the math module https://bugs.python.org/issue35606 closed by rhettinger #35615: "RuntimeError: Dictionary changed size during iteration" when https://bugs.python.org/issue35615 closed by pitrou #35642: _asynciomodule.c compiled in both pythoncore.vcxproj and _asyn https://bugs.python.org/issue35642 closed by steve.dower #35686: BufferError with memory.release() https://bugs.python.org/issue35686 closed by skrah #35692: pathlib.Path.exists() on non-existent drive raises WinError in https://bugs.python.org/issue35692 closed by steve.dower #35758: Disable x87 control word for MSVC ARM compiler https://bugs.python.org/issue35758 closed by Minmin.Gong #35851: Make search result in online docs keep their position when sea https://bugs.python.org/issue35851 closed by xtreak #35861: test_named_expressions raises SyntaxWarning https://bugs.python.org/issue35861 closed by emilyemorehouse #35862: Change the environment for a new process https://bugs.python.org/issue35862 closed by steve.dower #35872: Creating venv from venv no longer works in 3.7.2 https://bugs.python.org/issue35872 closed by steve.dower #35873: Controlling venv from venv no longer works in 3.7.2 https://bugs.python.org/issue35873 closed by steve.dower #35877: parenthesis is mandatory for named expressions in while statem https://bugs.python.org/issue35877 closed by emilyemorehouse #35879: test_type_comments leaks references https://bugs.python.org/issue35879 closed by gvanrossum #35884: Add variable access benchmark to Tools/Scripts https://bugs.python.org/issue35884 closed by rhettinger #35890: Cleanup some non-consistent API callings https://bugs.python.org/issue35890 closed by steve.dower #35894: Apparent regression in 3.8-dev: 'TypeError: required field "ty https://bugs.python.org/issue35894 closed by gvanrossum #35895: the test suite of pytest failed with 3.8.0a1 https://bugs.python.org/issue35895 closed by gvanrossum #35896: sysconfig.get_platform returns wrong value when Python 32b is https://bugs.python.org/issue35896 closed by steve.dower #35897: Support list as argument to .startswith() https://bugs.python.org/issue35897 closed by rhettinger #35902: Forking from background thread https://bugs.python.org/issue35902 closed by pitrou #35908: build with building extension modules as builtins is broken in https://bugs.python.org/issue35908 closed by doko #35909: Zip Slip Vulnerability https://bugs.python.org/issue35909 closed by christian.heimes #35910: Curious problem with my choice of variables https://bugs.python.org/issue35910 closed by matrixise #35911: add a cell construtor, and expose the cell type in Lib/types.p https://bugs.python.org/issue35911 closed by pitrou #35914: [2.7] PyStructSequence objects not behaving like nametuple https://bugs.python.org/issue35914 closed by eric.snow #35916: 3.6.5 try/except/else/finally block executes code with typos, https://bugs.python.org/issue35916 closed by SilentGhost #35917: multiprocessing: provide unit-tests for manager classes and sh https://bugs.python.org/issue35917 closed by pitrou #35929: Spam https://bugs.python.org/issue35929 closed by Mariatta #35932: Interpreter gets stuck while applying a regex pattern https://bugs.python.org/issue35932 closed by tim.peters #35938: crash of METADATA file cannot be fixed by reinstall of python https://bugs.python.org/issue35938 closed by steven.daprano

1 year, 6 months

Another update for PEP 394 -- The "python" Command on Unix-Like Systems

by Petr Viktorin

PEP 394 says: > This recommendation will be periodically reviewed over the next few > years, and updated when the core development team judges it > appropriate. As a point of reference, regular maintenance releases > for the Python 2.7 series will continue until at least 2020. I think it's time for another review. I'm especially worried about the implication of these: - If the `python` command is installed, it should invoke the same version of Python as the `python2` command - scripts that are deliberately written to be source compatible with both Python 2.x and 3.x [...] may continue to use `python` on their shebang line. So, to support scripts that adhere to the recommendation, Python 2 needs to be installed :( Please see this PR for details and a suggested change: https://github.com/python/peps/pull/893

1 year, 6 months

before I open an issue re: posix.stat and/or os.stat

by Michael

My focus is AIX - and I believe I found a bug in AIX include files in 64-bit mode. I'll take that up with IBM and AIX support. However, this issue might also be valid in Python3. The following is from Centos, not AIX Python 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.maxsize 9223372036854775807 >>> import posix >>> posix.stat("/tmp/xxx") posix.stat_result(st_mode=33188, st_ino=33925869, st_dev=64768L, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1550742595, st_mtime=1550742595, st_ctime=1550742595) >>> st=posix.stat("/tmp/xxx") >>> dev=st.st_dev >>> min=posix.minor(dev) >>> maj=posix.major(dev) >>> min,max (0, <built-in function max>) >>> min 0 >>> max <built-in function max> >>> maj 253 >>> posix.minor(dev) 0 >>> posix.major(655536) 2560 >>> posix.major(65536) 256 >>> posix.major(256) 1 >>> import os >>> os.major(256) 1 >>> In AIX - 64-bit mode Python 3.8.0a1+ (heads/master:e7a4bb554e, Feb 20 2019, 18:40:08) [C] on aix7 Type "help", "copyright", "credits" or "license" for more information. >>> import sys,os,posix >>> sys.maxsize 9223372036854775807 >>> posix.major(256) 0 >>> posix.major(65536) 1 >>> posix.stat("/tmp/xxx") os.stat_result(st_mode=33188, st_ino=12, st_dev=-9223371993905102841, st_nlink=1, st_uid=202, st_gid=1954, st_size=0, st_atime=1550690105, st_mtime=1550690105, st_ctime=1550690105) AIX 32-bit: root@x066:[/data/prj/python/git/python3-3.8.0.66]./python Python 3.8.0a1+ (heads/master:e7a4bb554e, Feb 19 2019, 11:22:56) [C] on aix6 Type "help", "copyright", "credits" or "license" for more information. >>> import os,sys,posix >>> sys.maxsize 2147483647 >>> posix.major(65536) 1 >>> posix.stat("/tmp/xxx") os.stat_result(st_mode=33188, st_ino=149, st_dev=655367, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1550743517, st_mtime=1550743517, st_ctime=1550743517) To make it easier to view: buildbot@x064:[/home/buildbot]cat osstat.c #include <sys/types.h> #include <sys/stat.h> #include <sys/sysmacros.h> #include <stdio.h> main() { dev_t dev; char *path = "/tmp/xxx"; struct stat st; int minor,major; lstat(path,&st); printf("size: %d\n", sizeof(st.st_dev)); dev = st.st_dev; minor = minor(dev); major = major(dev); printf("%016lx %ld %ld\n",dev,dev, (unsigned) dev); printf("%d,%d\n",major,minor); } buildbot@x064:[/home/buildbot]OBJECT_MODE=32 cc osstat.c -o osstat-32 && ./osstat-32 size: 4 00000000000a0007 655367 655367 10,7 And here is the AIX behavior (and bug - major() macro!) buildbot@x064:[/home/buildbot]OBJECT_MODE=64 cc osstat.c -o osstat-64 && ./osstat-64 size: 8 8000000a00000007 -9223371993905102841 7 0,7 The same on AIX 6 (above is AIX7) - and also with gcc: root@x068:[/data/prj]gcc -maix64 osstat.c -o osstat-64 && ./osstat-64 size: 8 8000000a00000007 -9223371993905102841 42949672967 0,7 root@x068:[/data/prj]gcc -maix32 osstat.c -o osstat-32 && ./osstat-32 size: 4 00000000000a0007 655367 0 10,7 root@x068:[/data/prj] So, the AIX 'bug' with the macro major() has been around for ages - but ALSO setting the MSB of the st_dev. +++++ Now my question: Will this continue to be enough space - i.e., is the Dev size going to be enough? +2042 #ifdef MS_WINDOWS +2043 PyStructSequence_SET_ITEM(v, 2, PyLong_FromUnsignedLong(st->st_dev)); +2044 #else +2045 PyStructSequence_SET_ITEM(v, 2, _PyLong_FromDev(st->st_dev)); +2046 #endif +711 #define _PyLong_FromDev PyLong_FromLongLong It seems so - however, Is there something such as PyUnsignedLong and is that large enough for a "long long"? and if it exists, would that make the value positive (for the first test). posix.major and os.major will need to mask away the MSB and posix.makedev and os.makedev will need to add it back. OR - do I need to make the PyStat values "the same" in both 32-bit and 64-bit? Puzzled on what you think is the correct approach. Michael

1 year, 6 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev February 2019