Mailman 3 January 2022 - Python-Dev

Python no longer leaks memory at exit
by Victor Stinner 28 Jan '22

28 Jan '22

Hi, tl; dr Python no longer leaks memory at exit on the "python -c pass" command ;-) == Bug report == In 2007, the bpo-1635741 issue was reported on SourceForge: "Interpreter seems to leak references after finalization". This bug is 15 years old. It saw the bugs migration from SourceForge to Roundup (bugs.python.org) in 2007, the code migration from Subversion to Mercurial in 2009, and the second code migration from Mercurial to Git in 2016! Link to the main issue: https://bugs.python.org/issue1635741 Some people reported similar issues seen by WinCRT debug (bpo-6741), MSVC Debug C Runtime (bpo-32026), GCC AddressSanitzer (bpo-26888), or LeakSanitizer and Valgrind (bpo-21387). In general, "leaking memory" at Python exit doesn't matter, since the operating system releases all memory when a process completes. It matters when Python is embedded in an application. For example, Python can be used by a plugin which is unloaded and reloaded multiple times. It also matters for sub-interpreters. == Tedious task == In the last 3 years, many people helped fixing this old issue by converting static types to heap types, adding a module state to C extension modules, converting extensions to the multi-phase initialization API (PEP 489), fix many memory leaks, fix bugs, etc. When it became possible to cleanly unload an extension module (thanks for the multi-phase init), tests on sub-intepreters (which load and unload extension modules) showed many *old* reference leaks: all of them have been fixed! When a test is run in sub-interpreters, Python is able to detect leaks, whereas currently it doesn't check for memory leaks at Python exit. == Python 3.10 regressions == During the Python 3.10 development, we identified and fixed 3 major regressions caused by this work: * Converting static types to heap types make them mutable: the Py_TPFLAGS_IMMUTABLETYPE flag was added; 68 types use it in Python 3.11 (bpo-43908). * It became possible to create uninitialized objects by instanciating types (which should not be instanciated directly): the Py_TPFLAGS_DISALLOW_INSTANTIATION was added; 41 types use it in Python 3.11 (bpo-43916). For example, you cannot create an instance of type(sys.flags). * Heap types must implement the full GC protocol: Py_TPFLAGS_HAVE_GC flag, traverse and clear functions. Otherwise, the GC is unable to break reference cycles, whereas a type contains (multiple) strong references to itself (in the MRO tuple and in methods). All heap types have been fixed to fully implement the GC protocol (bpo-40217). == PEPs == The work relies on multiple PEPs: * PEP 489: Multi-phase extension module initialization * PEP 573: Module State Access from C Extension Methods * PEP 630: Isolating Extension Modules == Persons who helped fixing the issue == Incomplete list of people who helped to fix this issue: * Christian Heimes (modules: symtable, _hashlib, ,_random, grp, pwd, _queue, spwd, _struct, gc, _posixshmem, _posixsubprocess, select, resource) * Dong-hee Na (modules: _statistics, itertools, _heapq, _collections, _uuid, math, _stat, syslog, errno, fcntl, mmap, _dbm, _gdbm, _lzma, faulthandler, _bisect) * Eric Snow (PEP 573) * Erlend Egeberg Aasland (modules: _sqlite3, _sre, _multibytecodec) * Hai Shi (modules: _json, _codecs, _crypt, _contextvars, _abc, _bz2, _locale, audioop, _ctypes_test, _weakref, _locale) * Marcel Plch (PEP 573) * Martin von Löwis (PEP 3121) * Mohamed Koubaa (modules: sha256, multiprocessing, _winapi, _blake2, _sha3, _signal, _sha1, _sha512, _md5, zlib, _opcode, _overlapped, _curses_panel, termios, _sha256, scproxy, cmath, _lsprof, unicodedata) * Nick Coghlan (PEP 489, PEP 573) * Paulo Henrique Silva (modules: time, operator, _functools) * Petr Viktorin (PEP 489, PEP 573, PEP 630) * Stefan Behnel (PEP 489) * Victor Stinner (modules _string, mashal,_imp, _warnings, _thread) The work was scatted into many sub-issues, it was hard for me to track all persons who contributed, sorry about that! I only searched for "New changeset" in bpo-1635741. For me, it was a very pleasant collaborative work :-) Contributors wrote pull requests, I reviewed and merged them. Slowly, contributors started to review each others and shared some recipes for these tasks. == Close the very old bpo-1635741 issue == Today, in the main development branch, Python no longer leaks memory at exit for the simplest command: "python3 -c pass"! Using a Python debug build, you can check the "python -I -X showrefcount -c pass" command (if you get a negative reference count, see bpo-46449), or you can use a memory debugger like Valgrind. While the work is not 100% done, it's a great milestone (at least for me ;-)! 15 years after bpo-1635741 was reported, finally I can close it! Sadly, this bug will no see the bugs migration from Roundup (bugs.python.org) to GitHub ;-) There are still some static types which should be converted to heap types and some extensions which should be be ported to the multi-phase initialization API. But this work can be done in existing or new specific issues. Again, a *big* thanks to every single person who helped directly and indirectly on fixing this issue! Victor -- Night gathers, and now my watch begins. It shall not end until my death.

4 3

RFC on PEP 680 -- tomllib: Support for Parsing TOML in the Standard Library
by Shantanu Jain 27 Jan '22

27 Jan '22

Hello all, Link to the PEP: https://www.python.org/dev/peps/pep-0680/ Please redirect discussion to: https://discuss.python.org/t/13040 PEP 680 proposes adding the tomllib module to the standard library for parsing TOML <https://toml.io>. The main motivation is to address bootstrapping problems with pyproject.toml based builds. Thank you to everyone who has already left comments, suggestions and reacts on the Discuss thread! We welcome any further suggestions before submitting to the Steering Council :-) Thanks, Shantanu Jain Taneli Hukkinen

1 0

Dropping AC_C_CHAR_UNSIGNED from configure.ac?
by Miro Hrončok 25 Jan '22

25 Jan '22

Hello Pythonistas. In (development branch of) Fedora, we have juts upgraded to GCC 12. It seems that the presence of AC_C_CHAR_UNSIGNED in Python's autotools files (configure.ac?) is causing the __CHAR_UNSIGNED__ symbol to be defined in pyconfig.h and that breaks some other packages with GCC 12. The GCC maintainers told us it is a reserved symbol, see https://bugzilla.redhat.com/2043555 for details. It seems that using AC_C_CHAR_UNSIGNED is not recommended and also not required, but I must confess that I am pretty much horrified by autotools and I don't really know if we can get rid of AC_C_CHAR_UNSIGNED or not. I can test this in Fedora and I am quite sure it'll work, but I don't know the impact on all the other environments where CPython can be compiled. Is there anybody on this list who knows a reason we need to keep AC_C_CHAR_UNSIGNED around in 2022? Or is it safe to get rid of it? Thanks, -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 2

SC Acceptance: PEP 673 -- Self Type
by Petr Viktorin 24 Jan '22

24 Jan '22

Hello Pradeep, James and Jelle, The Steering Council discussed PEP 673 -- Self Type, and unanimously decided to accept it. Congratulations! Please change the PEP status to Accepted, and merge the change to Python 3.11, at your convenience. Happy typing! - Petr (on behalf of the Python Steering Council)

1 0

The current state of typing PEPs
by Barry Warsaw 24 Jan '22

24 Jan '22

Does PEP 563 or 649 satisfy static and dynamic typing needs? In the interest of full transparency, we want to let the Python community know that the Steering Council continues to discuss PEP 563 (Postponed Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations Using Descriptors). We have concluded that we lack enough detailed information to make a decision in favor of either PEP. As we see it, adopting either PEP 563 or PEP 649 as the default would be insufficient. They will not fully resolve the existing problems these PEPs intend to fix, will break some existing code, and likely don’t address all the use cases and requirements across the static and dynamic typing constituents. We are also uncertain as to the best migration path from the current state of affairs. Defer decision on PEP 563 and 649 in 3.11 As such, at this time, the only reasonable path forward that the SC sees is to defer the decision in Python 3.11 again, essentially keeping the 3.10 status quo. We know that this is far from ideal, but it’s also the safest path since we can’t clearly make the situation better, and we don’t have confidence that either PEP solves the problems once and for all. Pragmatically, we don’t want to make the situation worse, and we really don’t want to find ourselves back here again in a couple of releases because we overlooked an important requirement for a set of users. Calling for help from static and dynamic typing users This is also a call to you, the folks who both care deeply about typing in Python, and have a solid understanding of the subject matter (or willingness to learn) to help us! There are use cases we don’t know about, and unclear requirements from both the static and dynamic typing users. We need help in defining those requirements clearly, in uncovering the use cases we’re not aware of, and most importantly, being an interface to typing enthusiasts to help build consensus with either of the proposed PEPs, or some other solution that doesn’t yet have a PEP (such as suggestions made by Łukasz in his blog post here: https://lukasz.langa.pl/61df599c-d9d8-4938-868b-36b67fdb4448/). Volunteer to be a typing PEP shepherd We’re looking for someone(s) who can be neutral as to the solution, knowledgeable enough to understand all sides of the problem, willing to serve as not quite a PEP author, not a PEP Delegate, but more than a PEP shepherd. If you are that person, please let us know! Cheers, -Barry (on behalf of the Python Steering Council)

29 81

Summary of Python tracker Issues
by Python tracker 21 Jan '22

21 Jan '22

ACTIVITY SUMMARY (2022-01-14 - 2022-01-21) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7175 (-27) closed 51014 (+108) total 58189 (+81) Open issues with patches: 2878 Issues opened (59) ================== #24711: Document getpass.getpass behavior on ^C https://bugs.python.org/issue24711 reopened by iritkatriel #44133: Some C-API symbols (e.g. Py_FrozenMain) are not always exporte https://bugs.python.org/issue44133 reopened by vstinner #45522: Allow to build Python without freelists https://bugs.python.org/issue45522 reopened by vstinner #46035: mimetypes.guess_type returns deprecated mimetype application/x https://bugs.python.org/issue46035 reopened by iritkatriel #46133: Feature request: allow mechanism for creator of exec-generated https://bugs.python.org/issue46133 reopened by posita #46381: Improve documentation of CFLAGS_NODIST, LDFLAGS_NODIST https://bugs.python.org/issue46381 opened by matthiaskoeppe #46382: dataclass(slots=True) does not account for slots in base class https://bugs.python.org/issue46382 opened by ariebovenberg #46383: _zoneinfo module_free has invalid function signature https://bugs.python.org/issue46383 opened by christian.heimes #46384: Request: make lzma._(encode|decode)_filter_properties public https://bugs.python.org/issue46384 opened by miurahr #46389: 3.11: unused generator comprehensions cause f_lineno==None https://bugs.python.org/issue46389 opened by nedbat #46390: Multiple test failures on Alpine 3.15 / musl-1.2.2-r7 https://bugs.python.org/issue46390 opened by christian.heimes #46391: Library multiprocess leaks named resources. https://bugs.python.org/issue46391 opened by milestonejxd #46392: MessageIDHeader is too strict for message-id https://bugs.python.org/issue46392 opened by bpoaugust #46393: Generate frozenset constants when explicitly appropriate https://bugs.python.org/issue46393 opened by terry.reedy #46396: Invalid usage of `Concatenate` is not covered at all https://bugs.python.org/issue46396 opened by sobolevn #46397: urllib.parse.urlencode() return wrong character https://bugs.python.org/issue46397 opened by scratch #46398: posixshmem module shm_rename freebsd support. https://bugs.python.org/issue46398 opened by devnexen #46399: Addition of `mapping` attribute to dict views classes has inad https://bugs.python.org/issue46399 opened by AlexWaygood #46400: Please update bundled libexpat to 2.4.3 with security fixes https://bugs.python.org/issue46400 opened by sping #46404: 3.11a4: a small attrs regression https://bugs.python.org/issue46404 opened by tinchester #46406: optimize int division https://bugs.python.org/issue46406 opened by gregory.p.smith #46407: optimizing `1 << n` or `2 ** n` and modulo-only operations https://bugs.python.org/issue46407 opened by February291948 #46410: TypeError when parsing regexp with unicode named character seq https://bugs.python.org/issue46410 opened by jirkamarsik #46414: Add typing.reveal_type https://bugs.python.org/issue46414 opened by Jelle Zijlstra #46416: Direct invocation of `Lib/test/test_typing.py` fails https://bugs.python.org/issue46416 opened by sobolevn #46417: Clear static types in Py_Finalize() for embedded Python https://bugs.python.org/issue46417 opened by vstinner #46419: Incomplete Comparison to C++ Methods https://bugs.python.org/issue46419 opened by jharmse #46420: Use NOTRACE_DISPATCH in specialized opcodes https://bugs.python.org/issue46420 opened by Dennis Sweeney #46421: unittest ValueError when invoking as module https://bugs.python.org/issue46421 opened by BaderSZ #46422: Why do we need `dis.Positions`? https://bugs.python.org/issue46422 opened by sobolevn #46425: Multiple test modules fail to run if invoked directly https://bugs.python.org/issue46425 opened by sobolevn #46426: Improve tests for the dir_fd argument https://bugs.python.org/issue46426 opened by serhiy.storchaka #46429: Merge all deepfrozen files into one https://bugs.python.org/issue46429 opened by kumaraditya303 #46430: intern strings in deepfrozen modules https://bugs.python.org/issue46430 opened by kumaraditya303 #46431: Trouble subclassing ExceptionGroup https://bugs.python.org/issue46431 opened by petr.viktorin #46432: AMD64 FreeBSD Shared 3.x buildbot fails to build: error: error https://bugs.python.org/issue46432 opened by vstinner #46433: _PyType_GetModuleByDef optimization is incorrect https://bugs.python.org/issue46433 opened by petr.viktorin #46434: pdb help fails with AttributeError when using Windows embeddab https://bugs.python.org/issue46434 opened by sparrowt #46435: MessageID parser can crash with IndexError: string index out o https://bugs.python.org/issue46435 opened by bpoaugust #46436: Pass the -d/--directory command-line option to http.server.CGI https://bugs.python.org/issue46436 opened by maggyero #46438: Static linking _decimal module breaks test_freeze https://bugs.python.org/issue46438 opened by steve.dower #46439: Clarify urllib.request.add_header documentation https://bugs.python.org/issue46439 opened by epicfaace #46440: ArgumentParser.parse_args exits on missing required argument w https://bugs.python.org/issue46440 opened by trilader #46442: testExceptionCleanupNames doesn't test anything? https://bugs.python.org/issue46442 opened by yellowdusk1590 #46445: Multiple inheritance of TypedDict is not covered in `test_typi https://bugs.python.org/issue46445 opened by sobolevn #46446: OpenBSD not MULTIARCH https://bugs.python.org/issue46446 opened by kmosiejczuk #46447: datetime.isoformat() documentation does not point to the risk https://bugs.python.org/issue46447 opened by jeanCarloMachado #46448: TypedDict inspect.signature error https://bugs.python.org/issue46448 opened by jhwang #46449: Deep-freezed modules create inconsistency in sys.gettotalrefco https://bugs.python.org/issue46449 opened by vstinner #46451: Possibly bad interaction with tracing and cython? https://bugs.python.org/issue46451 opened by seberg #46452: Possible false detection of Windows LZMA library as a malware https://bugs.python.org/issue46452 opened by thfetoile #46453: argparse subparser help text is not escaped before string form https://bugs.python.org/issue46453 opened by pbsds #46455: Deprecate / remove os.name=java https://bugs.python.org/issue46455 opened by xmorel #46456: Add mime type "image/avif" https://bugs.python.org/issue46456 opened by wangjiahua #46457: test_unittest: TestAsyncCase.test_debug_cleanup_same_loop() ha https://bugs.python.org/issue46457 opened by vstinner #46458: Optimise try-except code generation for the happy path https://bugs.python.org/issue46458 opened by iritkatriel #46459: Installing python to external drive (USB) using python install https://bugs.python.org/issue46459 opened by oMqngo #46460: Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCE https://bugs.python.org/issue46460 opened by sobolevn #46461: Kodi crashing https://bugs.python.org/issue46461 opened by mrpg Most recent 15 issues with no replies (15) ========================================== #46461: Kodi crashing https://bugs.python.org/issue46461 #46460: Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCE https://bugs.python.org/issue46460 #46457: test_unittest: TestAsyncCase.test_debug_cleanup_same_loop() ha https://bugs.python.org/issue46457 #46456: Add mime type "image/avif" https://bugs.python.org/issue46456 #46453: argparse subparser help text is not escaped before string form https://bugs.python.org/issue46453 #46447: datetime.isoformat() documentation does not point to the risk https://bugs.python.org/issue46447 #46446: OpenBSD not MULTIARCH https://bugs.python.org/issue46446 #46445: Multiple inheritance of TypedDict is not covered in `test_typi https://bugs.python.org/issue46445 #46440: ArgumentParser.parse_args exits on missing required argument w https://bugs.python.org/issue46440 #46439: Clarify urllib.request.add_header documentation https://bugs.python.org/issue46439 #46438: Static linking _decimal module breaks test_freeze https://bugs.python.org/issue46438 #46436: Pass the -d/--directory command-line option to http.server.CGI https://bugs.python.org/issue46436 #46435: MessageID parser can crash with IndexError: string index out o https://bugs.python.org/issue46435 #46420: Use NOTRACE_DISPATCH in specialized opcodes https://bugs.python.org/issue46420 #46419: Incomplete Comparison to C++ Methods https://bugs.python.org/issue46419 Most recent 15 issues waiting for review (15) ============================================= #46460: Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCE https://bugs.python.org/issue46460 #46458: Optimise try-except code generation for the happy path https://bugs.python.org/issue46458 #46446: OpenBSD not MULTIARCH https://bugs.python.org/issue46446 #46445: Multiple inheritance of TypedDict is not covered in `test_typi https://bugs.python.org/issue46445 #46442: testExceptionCleanupNames doesn't test anything? https://bugs.python.org/issue46442 #46439: Clarify urllib.request.add_header documentation https://bugs.python.org/issue46439 #46434: pdb help fails with AttributeError when using Windows embeddab https://bugs.python.org/issue46434 #46433: _PyType_GetModuleByDef optimization is incorrect https://bugs.python.org/issue46433 #46430: intern strings in deepfrozen modules https://bugs.python.org/issue46430 #46429: Merge all deepfrozen files into one https://bugs.python.org/issue46429 #46426: Improve tests for the dir_fd argument https://bugs.python.org/issue46426 #46425: Multiple test modules fail to run if invoked directly https://bugs.python.org/issue46425 #46422: Why do we need `dis.Positions`? https://bugs.python.org/issue46422 #46421: unittest ValueError when invoking as module https://bugs.python.org/issue46421 #46420: Use NOTRACE_DISPATCH in specialized opcodes https://bugs.python.org/issue46420 Top 10 most discussed issues (10) ================================= #46417: Clear static types in Py_Finalize() for embedded Python https://bugs.python.org/issue46417 20 msgs #46399: Addition of `mapping` attribute to dict views classes has inad https://bugs.python.org/issue46399 19 msgs #46376: PyMapping_Check returns 1 for list https://bugs.python.org/issue46376 14 msgs #40066: Enum: modify __repr__, __str__; update docs https://bugs.python.org/issue40066 11 msgs #46071: Graphlib documentation (edge direction) https://bugs.python.org/issue46071 11 msgs #46393: Generate frozenset constants when explicitly appropriate https://bugs.python.org/issue46393 9 msgs #46425: Multiple test modules fail to run if invoked directly https://bugs.python.org/issue46425 9 msgs #46382: dataclass(slots=True) does not account for slots in base class https://bugs.python.org/issue46382 8 msgs #46434: pdb help fails with AttributeError when using Windows embeddab https://bugs.python.org/issue46434 8 msgs #46392: MessageIDHeader is too strict for message-id https://bugs.python.org/issue46392 7 msgs Issues closed (101) =================== #13649: termios.ICANON is not documented https://bugs.python.org/issue13649 closed by iritkatriel #20281: [doc] time.strftime %z format specifier is the same as %Z https://bugs.python.org/issue20281 closed by iritkatriel #23162: collections.abc sequences don't check identity before equality https://bugs.python.org/issue23162 closed by iritkatriel #23557: Misc/SpecialBuilds.txt contains outdated information about PYM https://bugs.python.org/issue23557 closed by iritkatriel #25652: collections.UserString.__rmod__() raises NameError https://bugs.python.org/issue25652 closed by iritkatriel #27302: csv.Sniffer guesses wrong when unquoted fields contain quotes https://bugs.python.org/issue27302 closed by iritkatriel #28169: shift exponent overflow https://bugs.python.org/issue28169 closed by iritkatriel #28186: Autogenerated tabs / trailing whitespace https://bugs.python.org/issue28186 closed by iritkatriel #28206: signal.Signals not documented https://bugs.python.org/issue28206 closed by iritkatriel #28477: Add optional user argument to pathlib.Path.home() https://bugs.python.org/issue28477 closed by iritkatriel #29541: Python3 error while building on Alt-F https://bugs.python.org/issue29541 closed by iritkatriel #29860: smtplib.py doesn't capitalize EHLO. https://bugs.python.org/issue29860 closed by iritkatriel #29964: [doc] %z directive has no effect on the output of time.strptim https://bugs.python.org/issue29964 closed by iritkatriel #30268: Make mimetypes.guess_type accept path-like objects https://bugs.python.org/issue30268 closed by iritkatriel #30492: 'make clinic' does not work for out of tree builds / clinic.py https://bugs.python.org/issue30492 closed by iritkatriel #30512: CAN Socket support for NetBSD https://bugs.python.org/issue30512 closed by serhiy.storchaka #30819: Linking with 'ld -b' fails with 64-bit using Itanium HP compil https://bugs.python.org/issue30819 closed by iritkatriel #31876: python363.chm includes gibberish https://bugs.python.org/issue31876 closed by iritkatriel #32976: linux/random.h present but cannot be compiled https://bugs.python.org/issue32976 closed by iritkatriel #33286: Conflict between tqdm and multiprocessing on windows https://bugs.python.org/issue33286 closed by iritkatriel #33994: python build egg fails with error while compiling test cases https://bugs.python.org/issue33994 closed by iritkatriel #34672: '%Z' strftime specifier never works with musl https://bugs.python.org/issue34672 closed by iritkatriel #34924: inspect.signature isn't aware that types.MethodType can wrap a https://bugs.python.org/issue34924 closed by iritkatriel #35107: untokenize() fails on tokenize output when a newline is missin https://bugs.python.org/issue35107 closed by iritkatriel #35974: os.DirEntry.inode() returns invalid value within Docker contai https://bugs.python.org/issue35974 closed by iritkatriel #36369: test_weakref super slow on RPi Zero https://bugs.python.org/issue36369 closed by iritkatriel #37084: _ctypes not failing, can't find reason https://bugs.python.org/issue37084 closed by iritkatriel #37310: Solaris 11.3 w/ Studio 12.6 test_ctypes fail https://bugs.python.org/issue37310 closed by iritkatriel #37311: Solaris 11.3 w/ Studio 12.6 test_support fail https://bugs.python.org/issue37311 closed by iritkatriel #37776: [subinterpreters] Test Py_Finalize() from a subinterpreter https://bugs.python.org/issue37776 closed by vstinner #37800: Clean up the documentation on module attributes https://bugs.python.org/issue37800 closed by iritkatriel #38550: hashlib missing algorithms https://bugs.python.org/issue38550 closed by iritkatriel #38963: multiprocessing processes seem to "bleed" user information (GI https://bugs.python.org/issue38963 closed by iritkatriel #39147: using zipfile with root privilege shows FileNotFoundError https://bugs.python.org/issue39147 closed by iritkatriel #39276: type() cause segmentation fault in callback function called f https://bugs.python.org/issue39276 closed by iritkatriel #39663: IDLE: Add additional tests for pyparse https://bugs.python.org/issue39663 closed by iritkatriel #39819: NULL pointer crash in Modules/_cursesmodule.c in PyInit__curse https://bugs.python.org/issue39819 closed by iritkatriel #39952: Using VS2019 to automatically build Python3 and it failed to b https://bugs.python.org/issue39952 closed by iritkatriel #40116: Regression in memory use of shared key dictionaries for "compa https://bugs.python.org/issue40116 closed by Mark.Shannon #40729: Update the list of auto-generated files in .gitattributes https://bugs.python.org/issue40729 closed by BTaskaya #41034: test_builtin: PtyTests fail when run twice https://bugs.python.org/issue41034 closed by vstinner #41857: Document timeout arguments to poll() in select module https://bugs.python.org/issue41857 closed by taleinat #42028: Regression in mimetypes for image/bmp https://bugs.python.org/issue42028 closed by iritkatriel #42199: bytecode_helper assertNotInBytecode fails too eagerly https://bugs.python.org/issue42199 closed by iritkatriel #42548: debugger stops at breakpoint of `pass` that is not actually re https://bugs.python.org/issue42548 closed by iritkatriel #42712: Segmentation fault in running ast.literal_eval() with large ex https://bugs.python.org/issue42712 closed by iritkatriel #43430: Exception raised when attempting to create Enum via functional https://bugs.python.org/issue43430 closed by iritkatriel #43476: Enabling access to showsyntaxerror for IDLE's shell https://bugs.python.org/issue43476 closed by terry.reedy #43869: Fix documentation of epoch/time.time https://bugs.python.org/issue43869 closed by vstinner #43975: Incorrect MIME type returned for .js files Windows 10. https://bugs.python.org/issue43975 closed by iritkatriel #44024: Improve the TypeError message for non-string second arguments https://bugs.python.org/issue44024 closed by serhiy.storchaka #44901: Info about used pickle protocol used by multiprocessing.Queue https://bugs.python.org/issue44901 closed by iritkatriel #44934: Windows installer: Append Python to PATH instead of prepending https://bugs.python.org/issue44934 closed by steve.dower #45113: [subinterpreters][C API] Add a new function to create PyStruct https://bugs.python.org/issue45113 closed by shihai1991 #45535: Enum's dir() does not contain inherited members https://bugs.python.org/issue45535 closed by ethan.furman #45554: multiprocessing exitcode is insufficiently documented https://bugs.python.org/issue45554 closed by pitrou #45680: Documentation on `GenericAlias` objects and `__class_getitem__ https://bugs.python.org/issue45680 closed by kj #45834: Move runtime except: check to the parser https://bugs.python.org/issue45834 closed by iritkatriel #45877: Inconsistency in minimal supported version of Microsoft Visual https://bugs.python.org/issue45877 closed by arhadthedev #45947: Place dict (and values) pointers at a fixed (negative) offset https://bugs.python.org/issue45947 closed by Mark.Shannon #46017: Tutorial incorrectly refers to skits rather than sketches. https://bugs.python.org/issue46017 closed by iritkatriel #46038: Mark /configure file as generated in .gitattributes https://bugs.python.org/issue46038 closed by arhadthedev #46080: argparse.BooleanOptionalAction with default=argparse.SUPPRESS https://bugs.python.org/issue46080 closed by taleinat #46086: Add ratio_min() function to the difflib library https://bugs.python.org/issue46086 closed by taleinat #46108: Enum repr() incorrect when mixed with non-__new__ data types https://bugs.python.org/issue46108 closed by ethan.furman #46242: Improve error message when attempting to extend an enum with ` https://bugs.python.org/issue46242 closed by AlexWaygood #46255: Remove unnecessary check in _IOBase._check*() methods https://bugs.python.org/issue46255 closed by malin #46258: Minor algorithmic improvements for math.isqrt https://bugs.python.org/issue46258 closed by mark.dickinson #46316: Optimize pathlib.Path.iterdir() https://bugs.python.org/issue46316 closed by zach.ware #46339: PEG parser segfault from ast.literal_eval https://bugs.python.org/issue46339 closed by pablogsal #46361: Small ints aren't always cached properly https://bugs.python.org/issue46361 closed by mark.dickinson #46363: Two typos in versions 3.7 document translation of zh_CN https://bugs.python.org/issue46363 closed by iritkatriel #46379: itertools.product reference implementation creates temporaries https://bugs.python.org/issue46379 closed by rhettinger #46380: `test_functools.TestLRU` must not use `functools` module dire https://bugs.python.org/issue46380 closed by rhettinger #46385: Remove parenthetical symbols for readability and nlp https://bugs.python.org/issue46385 closed by Dennis Sweeney #46386: Improve `test_typing::test_immutability_by_copy_and_pickle` https://bugs.python.org/issue46386 closed by corona10 #46387: `test_field_descriptor` in `test_collections` should test all https://bugs.python.org/issue46387 closed by rhettinger #46388: Improve test coverage of functools.total_ordering https://bugs.python.org/issue46388 closed by rhettinger #46394: setuptools' _distutils_hack breaks Python build https://bugs.python.org/issue46394 closed by christian.heimes #46395: `Concatenate` is not covered to be a return type in `test_typi https://bugs.python.org/issue46395 closed by sobolevn #46401: Setuptools 60.0 breaks building from source on Debian 11: No m https://bugs.python.org/issue46401 closed by i-Ching #46402: Enhance sqlite3 to avoid implicit creation? https://bugs.python.org/issue46402 closed by erlendaasland #46403: Unhelpful SyntaxError https://bugs.python.org/issue46403 closed by terry.reedy #46405: Warning compiling main on Windows https://bugs.python.org/issue46405 closed by terry.reedy #46408: signal module wrongly relies on small int singletons https://bugs.python.org/issue46408 closed by christian.heimes #46409: Add a new bytecode instruction to create generators https://bugs.python.org/issue46409 closed by Mark.Shannon #46411: modernize exception handling in tests https://bugs.python.org/issue46411 closed by iritkatriel #46412: PyQT6 projects crashes with python 3.10 https://bugs.python.org/issue46412 closed by ronaldoussoren #46413: `__or__` is not covered for `_GenericAlias` and `_SpecialGener https://bugs.python.org/issue46413 closed by kj #46415: ipaddress.ip_{address,network,interface} raises TypeError inst https://bugs.python.org/issue46415 closed by Jelle Zijlstra #46418: Simplify `MODULE` variable in `test_enum.py` https://bugs.python.org/issue46418 closed by sobolevn #46423: CLI: Addition assignment for tuples https://bugs.python.org/issue46423 closed by mark.dickinson #46424: `typing.Annotated` one type argument usage is not covered in t https://bugs.python.org/issue46424 closed by sobolevn #46427: Correct MSBuild's configuration for _freeze_module.exe https://bugs.python.org/issue46427 closed by steve.dower #46428: Cython Build: '_PyErr_StackItem�� has no member named ��exc_ https://bugs.python.org/issue46428 closed by YoSTEALTH #46437: Non-required `hasattr` checks in `test_typing` https://bugs.python.org/issue46437 closed by AlexWaygood #46441: Caret points to wrong line on 'return yield 42' in REPL https://bugs.python.org/issue46441 closed by pablogsal #46443: Deepfreeze use preallocated small ints https://bugs.python.org/issue46443 closed by kumaraditya303 #46444: Wrong value of pi for larger values using math.cos() function https://bugs.python.org/issue46444 closed by serhiy.storchaka #46450: namedtuple leaks data between instances when field's default v https://bugs.python.org/issue46450 closed by eric.smith #46454: '0 -> /dev/null' is lost https://bugs.python.org/issue46454 closed by vstinner

1 0

Discussion about crash tolerance feature for gdbm module
by Dong-hee Na 21 Jan '22

21 Jan '22

Hi folks, From gdbm 1.21, gdbm supports the crash tolerance feature. see: https://www.gnu.org.ua/software/gdbm/manual/Crash-Tolerance.html I would like to introduce this feature since python standard library is the only gdbm binding library that is available for end-users. And this is also effort for end-users to provide the latest new features. If this feature is added following methods will be added. - 'x' flag will be added for extended gdbm format. - gdbm.gdbm_failure_atomic('path1', 'path2') API will be added for snapshot paths. - gdbm.gdbm_latest_snapshot('path1', 'path2') API will be added for getting latest valid snapshot path. The above APIs will be used for people who need to recover their gdbm file if disaster situations happen. However, this feature is not yet decided to land to CPython. and we have already discussed this issue at https://bugs.python.org/issue45452 but I would like to hear other devs opinions. I cc the original authors of this feature and Serhiy who thankfully already participated in this discussion too :) Warm Regards, Dong-hee

2 6

Adding an `inspect` function to return the signature of a type?
by Steven Troxler 20 Jan '22

20 Jan '22

I've been wondering whether it would make sense to have a function in `inspect` that returns the signature of a type, rather than the signature of a specific callable object. I'm not attached to any name for such a function, two ideas are `inspect.signature_of` or `inspect.signature_of_type`. ## Context: the behavior of `inspect.signature` The existing `inspect.signature` function specified in PEP 362 [0] returns the call signature of a particular object, in other words given ``` class Dog: def __init__(self, color: str): self.color = color def __call__(self, command: str): print(command) ``` we will get ``` >>> import inspect >>> inspect.signature(Dog) <Signature (color: str)> >>> inspect.signature(Dog("brown")) <Signature (command: str)> ``` ## Outline of the idea In some cases it might be nice to be able to access the signature of an instance of `Dog` given the type `Dog`, without actually needing to instantiate it. Moreover, there are some cases where it isn't even possible to get a callable object, but it still might be nice to have access to an `inspect.Signature` object: - protocols cannot be instantiated. This is particularly relevant to getting Signatures because of callback protocols [1] - typing.Callable types describe a signature, but again can't be instantiated If if we had a function to operate on types, it would treat normal classes as if calling `inspect.signature` on the `__call__` method of an instance. For callable types we'd want to do something similar, although this brings up some issues: - the existing Signature requires parameter names, but a `typing.Callable` doesn't have any. - One option would be to invent names `__0`, `__1,` etc. Otherwise we'd have to make the name optional. - it's not obvious how to support PEP 612 ParamSpecs, and even less obvious how to support PEP 646 in full generality I have some code snippets running through how simple cases might work, as well as the edge cases I've thought of thus far: https://gist.github.com/stroxler/4760cbc5e49df2295cd0b524328b1c73 I'm undecided about whether the edge cases are an indication that this idea isn't worth considering. It's worth noting that most of the edge cases involve PEP 612 / 646 generics which are pretty rare. ## Why did I start thinking about this? In discussions of PEP 677 [2], a few folks suggested using `inspect.Signature` as inspiration for the runtime API (which we did). This got me thinking that it might be nice for code that uses runtime types, regardless of what we decide about PEP 677, to have an API that makes accessing signature information from a type easier. I don't personally have much need for this, but - when I chatted with Carl Meyer from the static python team at Meta he thought it could be handy - I could imagine it being useful for tools like ML frameworks, which I've worked on in the past, that interpret type annotations to auto-generate logic -- [0] PEP 362 Function Signatures https://www.python.org/dev/peps/pep-0362/ [1] Callback protocols https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols [2] PEP 677 Callable Type Syntax https://www.python.org/dev/peps/pep-0677/

1 0

RFC on PEP 673: Self Type
by Pradeep Kumar Srinivasan 20 Jan '22

20 Jan '22

This PEP [1] introduces a simple and intuitive way to annotate methods and classmethods that return an instance of their class. Such methods and classmethods occur quite frequently, but the existing way to annotate them correctly is quite arcane and error-prone. The PEP introduces a special type `Self` to represent the type of the `self` parameter, similar to the `this` type in TypeScript and the `Self` type in Rust. We have implementations for mypy and pyright. The PEP does not affect CPython directly except for the addition of one special form (Self) to typing.py [2]. Since we have reached consensus on the PEP in typing-sig [3], we wanted to get your comments and suggestions before submitting to the Steering Council. Thanks, Pradeep Kumar Srinivasan James Hilton-Balfe [1]: https://www.python.org/dev/peps/pep-0673/ [2]: Adding `Self` to typing_extensions.py: https://github.com/python/typing/pull/933 [3]: See the comments from typing-sig members on the Google doc: https://docs.google.com/document/d/1ujuSMXDmSIOJpiZyV7mvBEC8P-y55AgSzXcvhrZ…

4 4

3.11 enhanced error location - can it be smarter?
by Mats Wichmann 20 Jan '22

20 Jan '22

"When printing tracebacks, the interpreter will now point to the exact expression that caused the error instead of just the line." I get the motivation for better error messages, but there are scenarios where you can't provide useful new information. I've been lax in getting to testing the project I work on with 3.11, but here's some spew from one testcase (this is an intentionally triggered error - the testcase is testing predictable behavior in the face of this error, 3.11 didn't *cause* this, as can be fairly easily seen): Traceback (most recent call last): File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1403, in main _exec_main(parser, values) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1366, in _exec_main _main(parser) ^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1034, in _main SCons.Script._SConscript._SConscript(fs, script) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\SConscript.py", line 285, in _SConscript exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\AppData\Local\Temp\testcmd.16164.fpv9xt5b\SConstruct", line 4, in <module> raise InternalError('error inside') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ SCons.Errors.InternalError: error inside The grump here is all those pointy hats added absolutely zero new information, because in each case the entire line is underlined. So the "benefit" of the change is the traceback went from 12 lines to 17. In many cases tracebacks are much longer and adding lots of not-useful lines is not a great thing. A thought - how about omitting the underline line if the to-be-underlined part would be the whole line?

10 16