Mailman 3 January 2022 - Python-Dev

PEP 674: Disallow using macros as l-value
by Victor Stinner 12 Jan '22

12 Jan '22

Hi, I propose to disallow using macros as l-value. Read and comment the plain text below, or read the PEP 674 online, https://python.github.io/peps/pep-0674/ While I'm not sure that the proposed changes are really controversial, I decided to write a formal PEP since the incompatible changes are not following the PEP 387 deprecation process and so a format PEP 387 exception is better. Well, the list of modified macros is also quite long. Moreover, a PEP is a way to document and announce the changes ;-) The Py_TYPE() and Py_SIZE() changes are already approved the Steering Council, but I prefer to list them in the PEP: https://github.com/python/steering-council/issues/79 In practice, I'm only aware of 4 projects impacted by these changes, and I wrote the pythoncapi_compat project which updates automatically C extensions: add Python 3.11 support, without losing support for older Python versions. I already prepared major projects like Cython and numpy for these changes (in total, 14 impacted projects have already been updated). Victor --- PEP: 674 Title: Disallow using macros as l-value Author: Victor Stinner <vstinner(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30-Oct-2021 Python-Version: 3.11 Abstract ======== Incompatible C API change disallowing using macros as l-value to allow evolving CPython internals and to ease the C API implementation on other Python implementation. In practice, the majority of projects impacted by these incompatible changes should only have to make two changes: * Replace ``Py_TYPE(obj) = new_type;`` with ``Py_SET_TYPE(obj, new_type);``. * Replace ``Py_SIZE(obj) = new_size;`` with ``Py_SET_SIZE(obj, new_size);``. Rationale ========= Using a macro as a l-value -------------------------- In the Python C API, some functions are implemented as macro because writing a macro is simpler than writing a regular function. If a macro exposes directly a struture member, it is technically possible to use this macro to not only get the structure member but also set it. Example with the Python 3.10 ``Py_TYPE()`` macro:: #define Py_TYPE(ob) (((PyObject *)(ob))->ob_type) This macro can be used as a **r-value** to **get** an object type:: type = Py_TYPE(object); It can also be used as **l-value** to **set** an object type:: Py_TYPE(object) = new_type; It is also possible to set an object reference count and an object size using ``Py_REFCNT()`` and ``Py_SIZE()`` macros. Setting directly an object attribute relies on the current exact CPython implementation. Implementing this feature in other Python implementations can make their C API implementation less efficient. CPython nogil fork ------------------ Sam Gross forked Python 3.8 to remove the GIL: the `nogil branch <https://github.com/colesbury/nogil/>`_. This fork has no ``PyObject.ob_refcnt`` member, but a more elaborated implementation for reference counting, and so the ``Py_REFCNT(obj) = new_refcnt;`` code fails with a compiler error. Merging the nogil fork into the upstream CPython main branch requires first to fix this C API compatibility issue. It is a concrete example of a Python optimization blocked indirectly by the C API. This issue was already fixed in Python 3.10: the ``Py_REFCNT()`` macro has been already modified to disallow using it as a l-value. HPy project ----------- The `HPy project <https://hpyproject.org/>`_ is a brand new C API for Python using only handles and function calls: handles are opaque, structure members cannot be accessed directly,and pointers cannot be dereferenced. Disallowing the usage of macros as l-value helps the migration of existing C extensions to HPy by reducing differences between the C API and the HPy API. PyPy cpyext module ------------------ In PyPy, when a Python object is accessed by the Python C API, the PyPy ``cpyext`` module has to convert PyPy object to a CPython object. While PyPy objects are designed to be efficient with the PyPy JIT compiler, CPython objects are less efficient and increase the memory usage. This PEP alone is not enough to get rid of the CPython objects in the PyPy ``cpyext`` module, but it is a step towards this long term goal. PyPy already supports HPy which is a better solution in the long term. Specification ============= Disallow using macros as l-value -------------------------------- PyObject and PyVarObject macros ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * ``Py_TYPE()``: ``Py_SET_TYPE()`` must be used instead * ``Py_SIZE()``: ``Py_SET_SIZE()`` must be used instead "GET" macros ^^^^^^^^^^^^ * ``PyByteArray_GET_SIZE()`` * ``PyBytes_GET_SIZE()`` * ``PyCFunction_GET_CLASS()`` * ``PyCFunction_GET_FLAGS()`` * ``PyCFunction_GET_FUNCTION()`` * ``PyCFunction_GET_SELF()`` * ``PyCell_GET()`` * ``PyCode_GetNumFree()`` * ``PyDict_GET_SIZE()`` * ``PyFunction_GET_ANNOTATIONS()`` * ``PyFunction_GET_CLOSURE()`` * ``PyFunction_GET_CODE()`` * ``PyFunction_GET_DEFAULTS()`` * ``PyFunction_GET_GLOBALS()`` * ``PyFunction_GET_KW_DEFAULTS()`` * ``PyFunction_GET_MODULE()`` * ``PyHeapType_GET_MEMBERS()`` * ``PyInstanceMethod_GET_FUNCTION()`` * ``PyList_GET_SIZE()`` * ``PyMemoryView_GET_BASE()`` * ``PyMemoryView_GET_BUFFER()`` * ``PyMethod_GET_FUNCTION()`` * ``PyMethod_GET_SELF()`` * ``PySet_GET_SIZE()`` * ``PyTuple_GET_SIZE()`` * ``PyUnicode_GET_DATA_SIZE()`` * ``PyUnicode_GET_LENGTH()`` * ``PyUnicode_GET_LENGTH()`` * ``PyUnicode_GET_SIZE()`` * ``PyWeakref_GET_OBJECT()`` "AS" macros ^^^^^^^^^^^ * ``PyByteArray_AS_STRING()`` * ``PyBytes_AS_STRING()`` * ``PyFloat_AS_DOUBLE()`` * ``PyUnicode_AS_DATA()`` * ``PyUnicode_AS_UNICODE()`` PyUnicode macros ^^^^^^^^^^^^^^^^ * ``PyUnicode_1BYTE_DATA()`` * ``PyUnicode_2BYTE_DATA()`` * ``PyUnicode_4BYTE_DATA()`` * ``PyUnicode_DATA()`` * ``PyUnicode_IS_ASCII()`` * ``PyUnicode_IS_COMPACT()`` * ``PyUnicode_IS_READY()`` * ``PyUnicode_KIND()`` * ``PyUnicode_READ()`` * ``PyUnicode_READ_CHAR()`` PyDateTime "GET" macros ^^^^^^^^^^^^^^^^^^^^^^^ * ``PyDateTime_DATE_GET_FOLD()`` * ``PyDateTime_DATE_GET_HOUR()`` * ``PyDateTime_DATE_GET_MICROSECOND()`` * ``PyDateTime_DATE_GET_MINUTE()`` * ``PyDateTime_DATE_GET_SECOND()`` * ``PyDateTime_DATE_GET_TZINFO()`` * ``PyDateTime_DELTA_GET_DAYS()`` * ``PyDateTime_DELTA_GET_MICROSECONDS()`` * ``PyDateTime_DELTA_GET_SECONDS()`` * ``PyDateTime_GET_DAY()`` * ``PyDateTime_GET_MONTH()`` * ``PyDateTime_GET_YEAR()`` * ``PyDateTime_TIME_GET_FOLD()`` * ``PyDateTime_TIME_GET_HOUR()`` * ``PyDateTime_TIME_GET_MICROSECOND()`` * ``PyDateTime_TIME_GET_MINUTE()`` * ``PyDateTime_TIME_GET_SECOND()`` * ``PyDateTime_TIME_GET_TZINFO()`` PyDescr macros ^^^^^^^^^^^^^^ * ``PyDescr_NAME()`` * ``PyDescr_TYPE()`` Port C extensions to Python 3.11 -------------------------------- In practice, the majority of projects impacted by these PEP incompatible changes should only have to make two changes: * Replace ``Py_TYPE(obj) = new_type;`` with ``Py_SET_TYPE(obj, new_type);``. * Replace ``Py_SIZE(obj) = new_size;`` with ``Py_SET_SIZE(obj, new_size);``. The `pythoncapi_compat project <https://github.com/pythoncapi/pythoncapi_compat>`_ can be used to update automatically C extensions: add Python 3.11 support without losing support with older Python versions. The project provides a header file which provides ``Py_SET_REFCNT()``, ``Py_SET_TYPE()`` and ``Py_SET_SIZE()`` functions to Python 3.8 and older. PyTuple_GET_ITEM() and PyList_GET_ITEM() ---------------------------------------- The ``PyTuple_GET_ITEM()`` and ``PyList_GET_ITEM()`` macros are left unchanged. The code pattern ``&PyTuple_GET_ITEM(tuple, 0)`` and ``&PyList_GET_ITEM(list, 0)`` is still commonly used to get access to the inner ``PyObject**`` array. Changing these macros would require to add a new API to get access to the inner array which is out of the scope of this PEP. Backwards Compatibility ======================= The proposed C API changes are backward incompatible on purpose. In practice, only a minority of third party projects are affected (16 projects are known to be broken) and `most of them have already been updated for these changes <https://bugs.python.org/issue39573#msg401378>`__ (12 on 16). Most projects are broken by ``Py_TYPE()`` and ``Py_SIZE()`` changes. These two macros have been converted to static inline macro in Python 3.10 alpha versions, but the change had to be reverted since it broke too many projects. In the meanwhile, many projects, like Cython, have been prepared for this change by using ``Py_SET_TYPE()`` and ``Py_SET_SIZE()``. For example, projects using Cython only have to regenerate their outdated generated C code to become compatible. For the "GET" functions like ``PyDict_GET_SIZE()``, no project in the PyPI top 5000 projects use these functions as l-value. The ``PyFloat_AS_DOUBLE()`` function is not used as a l-value in the PyPI top 5000 projects. The ``PyBytes_AS_STRING()`` and ``PyByteArray_AS_STRING()`` are used as l-value but only to modify string characters, not to override the ``PyBytesObject.ob_sval`` or ``PyByteArrayObject.ob_start`` member. For example, Cython uses the following code which remains valid:: PyByteArray_AS_STRING(string)[i] = (char) v; This change does not follow the PEP 387 deprecation process. There is no known way to emit a deprecation warning when a macro is used as a l-value, but not when it's used differently (ex: r-value). Rejected Idea: Leave the macros as they are =========================================== The documentation of each function can discourage developers to use macros to modify Python objects. If these is a need to make an assignment, a setter function can be added and the macro documentation can require to use the setter function. For example, a ``Py_SET_TYPE()`` function has been added to Python 3.9 and the ``Py_TYPE()`` documentation now requires to use the ``Py_SET_TYPE()`` function to set an object type. If developers use macros as l-value, it's their responsibility when their code breaks, not the Python responsibility. We are operating under the consenting adults principle: we expect users of the Python C API to use it as documented and expect them to take care of the fallout, if things break when they don't. This idea was rejected because only few developers read the documentation, and only a minority is tracking changes of the Python C API documentation. The majority of developers are only using CPython and so are not aware of compatibility issues with other Python implementations. Moreover, continuing to allow using macros as l-value does not solve issues of the nogil, PyPy and HPy projects. Macros already modified ======================= The following C API macros have already been modified to disallow using them as l-value: * ``PyCell_SET()`` * ``PyList_SET_ITEM()`` * ``PyTuple_SET_ITEM()`` * ``Py_REFCNT()`` (Python 3.10): ``Py_SET_REFCNT()`` must be used * ``_PyGCHead_SET_FINALIZED()`` * ``_PyGCHead_SET_NEXT()`` * ``asdl_seq_GET()`` * ``asdl_seq_GET_UNTYPED()`` * ``asdl_seq_LEN()`` * ``asdl_seq_SET()`` * ``asdl_seq_SET_UNTYPED()`` For example, ``PyList_SET_ITEM(list, 0, item) < 0`` now fails with a compiler error as expected. References ========== * `Python C API: Add functions to access PyObject <https://vstinner.github.io/c-api-abstract-pyobject.html>`_ (October 2021) article by Victor Stinner * `[C API] Disallow using PyFloat_AS_DOUBLE() as l-value <https://bugs.python.org/issue45476>`_ (October 2021) * `[capi-sig] Py_TYPE() and Py_SIZE() become static inline functions <https://mail.python.org/archives/list/capi-sig@python.org/thread/WGRLTHTHC3…>`_ (September 2021) * `[C API] Avoid accessing PyObject and PyVarObject members directly: add Py_SET_TYPE() and Py_IS_TYPE(), disallow Py_TYPE(obj)=type <https://bugs.python.org/issue39573>`__ (February 2020) * `bpo-30459: PyList_SET_ITEM could be safer <https://bugs.python.org/issue30459>`_ (May 2017) Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

10 25

Status of Python 3.11.0a4
by Pablo Galindo Salgado 11 Jan '22

11 Jan '22

Hi everyone, I want to report on the status of Python 3.11.0a4. We had a ton of release blockers (some extra ones since I reported the last time) and it seems that we managed to fix them all (thanks to Mark Shannon, Christian Heimes, Gregory P. Smith, Neil Schemenauer, Steve Dower and many others that helped with the blockers. Unfortunately it seems that the release is cursed and we are having some problems with the certificates to create and sign the Windows artefacts so we are waiting for that to unblock us. I will keep you posted on new developments. Regards from rainy London, Pablo Galindo Salgado

1 0

PEP 676 -- PEP Infrastructure Process
by python＠quite.org.uk 10 Jan '22

10 Jan '22

Hi, I would like to announce PEP 676 to python-dev. It is a meta-PEP focussed on modernising the PEP build infrastructure. From the abstract, "This PEP addresses the infrastructure around rendering PEP files from reStructuredText files to HTML webpages. We aim to specify a self-contained and maintainable solution for PEP readers, authors, and editors." Link: https://www.python.org/dev/peps/pep-0676/ Rendered through the PEP 676 system: https://python.github.io/peps/pep-0676/ Please see https://discuss.python.org/t/10774 for prior discussion and to give any feedback. Thanks, Adam Turner

4 3

Core review request: bpo-33178: Add BigEndianUnion, LittleEndianUnion classes to ctypes #25480
by Dave Goncalves 10 Jan '22

10 Jan '22

Apologies for one of these review request emails. Anybody with ctypes experience in the core reviewer team available to review and comment/merge? This has gotten reviewed and passes checks. Python bug: https://bugs.python.org/issue33178 Current PR: https://github.com/python/cpython/pull/25480 Thanks, David Goncalves

1 0

PEP 679 – Allow parentheses in assert statements
by Pablo Galindo Salgado 10 Jan '22

10 Jan '22

Hi everyone, I would like to start a discussion about a small PEP proposal to allow parentheses in assert statements to fix a common gotcha with assert statements. Link to the PEP: https://www.python.org/dev/peps/pep-0679/ *Please, redirect all discussions to: * https://discuss.python.org/t/pep-679-allow-parentheses-in-assert-statements… as I will not be monitoring answers to this thread. Thanks, everyone for your time! Regards from cloudy London, Pablo Galindo Salgado

6 7

About vulnerabilities in Cpython native code
by lxr1210＠mail.ustc.edu.cn 09 Jan '22

09 Jan '22

Hi all, I am currently doing some research on the security of CPython. I used the open source vulnerability analysis engine, Infer(https://fbinfer.com/), to scan the native code of CPython 3.10.0. The scan results show that there are still a number of vulnerabilities in the CPython native code, such as Null dereference, Uninitialized variable, Resource/Memory leak, etc. Moreover, I found that some of the vulnerabilities are related to Python/C API. I enclose the vulnerability report for your reference. Based on the research of the result, I tried to design a tool to automatically detect and repair vulnerabilities in CPython and make this tool available. See: https://github.com/PVMPATCH/PVMPatch Python is my favourite programming language. I sincerely hope that I can help Python become stronger and safer. I hope this discovery can be useful for you to develop Python in the future. Thank you for your time and consideration! Lin

6 10

Python 3.6 rides into the sunset
by Ned Deily 08 Jan '22

08 Jan '22

While there are many reasons to welcome the end of 2021, I would like to give a shout-out to Python 3.6 which officially reached end-of-life on 2021-12-23, 6.5 years after its development began and exactly five years after its initial release. Building on the success of previous Python 3 releases, 3.6 added many new features and improvements too numerous to list: over 10000 commits by some 160 contributors, including one of the most popular features in recent Python releases, f-strings (thanks, EVS!). I think it is fair to say that, with Python 3.6, the long transition from Python 2 was finally settled. As release manager for 3.6, I would like to personally thank all those contributors, and mostly volunteers: you made my job an easy one with your overwhelmingly positive attitude and support. I would also like to thank the authors and maintainers of the many third-party packages that were updated to support 3.6 as well as the downstream distributors of Python whose rapid uptake and release of 3.6 in their distributions was crucial to its success. I would also like to thank those who helped get the 3.6 releases out the door, in particular, Steve Dower for manufacturing the Windows packages, Julien Palard for managing our on-line documentation build process, Elvis Pranskevichus and Yury Selivanov for taking on the thankless task of assembling and editing the 3.6 "What's New" document, my fellow release managers for their encouragement and support from the start to finish of 3.6's life, the Steering Councils, the PSF Infrastructure Team, those individuals and organizations who contribute resources (money, people, time, facilities, services) to the PSF, making Python development possible. And, I suppose I should thank that git who produces the macOS packages. Thanks again to you all for making 3.6 so successful! --Ned P.S. As a reminder, with 3.6 having reached end-of-life, we no longer accept bug reports of any type against 3.6 and the 3.6 source code is now frozen. There is no longer a 3.6 branch in the GitHub cpython repository; the final state of the branch is captured in the repo as tag "3.6" and, as always, the source code for any release can be checked out using its tag; for example, the source for the final release of 3.6 can be obtained with "git checkout v3.6.15". Pro tip: if you haven't already, you may want to update your repo clones with "git fetch --tags upstream" (if you use the recommended naming convention) to get the latest tags and branches. -- Ned Deily nad(a)python.org -- []

1 0

Summary of Python tracker Issues
by Python tracker 07 Jan '22

07 Jan '22

ACTIVITY SUMMARY (2021-12-31 - 2022-01-07) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7199 ( +4) closed 50825 (+78) total 58024 (+82) Open issues with patches: 2886 Issues opened (50) ================== #34931: os.path.splitext with more dots https://bugs.python.org/issue34931 reopened by xnovakj #46009: sending non-None values makes generator raise StopIteration on https://bugs.python.org/issue46009 reopened by christian.heimes #46110: compile("-"*3000000 + "4", '', mode) causes hard crash https://bugs.python.org/issue46110 reopened by pablogsal #46216: spurious link to os.system() from os.times() documentation ent https://bugs.python.org/issue46216 opened by emery.berger #46217: 3.11 build failure on Win10: new _freeze_module changes? https://bugs.python.org/issue46217 opened by terry.reedy #46220: imaplib.py "select" mailbox names containing spaces. https://bugs.python.org/issue46220 opened by mckenzm #46223: asyncio cause infinite loop during debug https://bugs.python.org/issue46223 opened by zillionare #46225: f_lasti behaves differently for lambdas returned from loops https://bugs.python.org/issue46225 opened by nedbat #46226: User specific paths added to System PATH environment variable https://bugs.python.org/issue46226 opened by akrymskiy #46227: add pathlib.Path.walk method https://bugs.python.org/issue46227 opened by Ovsyanka #46232: Client certificates with UniqueIdentifier in the subject break https://bugs.python.org/issue46232 opened by kacper #46234: 3.11: Tracing of decorators now visits the decorator line befo https://bugs.python.org/issue46234 opened by nedbat #46235: Do all ref-counting at once for sequence multiplication https://bugs.python.org/issue46235 opened by Dennis Sweeney #46237: Incorrect line reported in syntax error https://bugs.python.org/issue46237 opened by arian-f #46242: Improve error message when attempting to extend an enum with ` https://bugs.python.org/issue46242 opened by sobolevn #46244: typing._TypeVarLike missing __slots__ https://bugs.python.org/issue46244 opened by ariebovenberg #46245: Add support for dir_fd in shutil.rmtree() https://bugs.python.org/issue46245 opened by serhiy.storchaka #46246: importlib.metadata.DeprecatedList appears to be missing __slot https://bugs.python.org/issue46246 opened by ariebovenberg #46247: in xml.dom.minidom, Node and DocumentLS appear to be missing _ https://bugs.python.org/issue46247 opened by ariebovenberg #46249: [sqlite3] move set lastrowid out of the query loop and enable https://bugs.python.org/issue46249 opened by erlendaasland #46250: 3.10 docs responsive design removes navigation buttons (next, https://bugs.python.org/issue46250 opened by vkvanjavk #46252: SSLWantReadError causes _SelectorSocketTransport to close https://bugs.python.org/issue46252 opened by matan1008 #46253: C API documentation of Py_UNICODE_* character properties macro https://bugs.python.org/issue46253 opened by juliangilbey #46254: Better fitting type for iterating in the trace_init C function https://bugs.python.org/issue46254 opened by uriya1998 #46255: Remove unnecessary check in _IOBase._check*() methods https://bugs.python.org/issue46255 opened by malin #46258: Minor algorithmic improvements for math.isqrt https://bugs.python.org/issue46258 opened by mark.dickinson #46261: [doc] fix inaccuracies in sqlite3.Cursor.lastrowid docs https://bugs.python.org/issue46261 opened by erlendaasland #46264: 'I'.lower() should give non dotted i for LANG=tr_TR https://bugs.python.org/issue46264 opened by fbacher #46265: Error when cross compiling for hardfloat MIPS https://bugs.python.org/issue46265 opened by jefferyto #46267: gzip.compress incorrectly ignores level parameter https://bugs.python.org/issue46267 opened by rhpvorderman #46269: '__new__' is never shown in `dir(SomeEnum)` https://bugs.python.org/issue46269 opened by sobolevn #46270: Comparison operators in Python Tutorial 5.7 https://bugs.python.org/issue46270 opened by realjanpaulus #46271: frozen modules are not regenerated on bytecode magic change wh https://bugs.python.org/issue46271 opened by christian.heimes #46272: Fix bitwise and logical terminology in python.gram https://bugs.python.org/issue46272 opened by rhettinger #46273: Document what asyncio.wait() and asyncio.as_completed() do if https://bugs.python.org/issue46273 opened by termim #46274: Tokenizer module does not handle backslash characters correctl https://bugs.python.org/issue46274 opened by ucodery #46275: caret location for syntax error pointing with f-strings https://bugs.python.org/issue46275 opened by williamnavaraj #46276: ImportError: DLL load failed while importing https://bugs.python.org/issue46276 opened by 89z #46279: [docs] Minor information-ordering issue in __main__ doc https://bugs.python.org/issue46279 opened by ferdnyc #46280: About vulnerabilities in Cpython native code https://bugs.python.org/issue46280 opened by urnotmax #46282: print() docs do not indicate its return value https://bugs.python.org/issue46282 opened by SnoopJeDi #46283: [subinterpreters] Unicode interned strings must not be shared https://bugs.python.org/issue46283 opened by vstinner #46284: DirEntry returns invalid values for is_dir and is_file on NFS https://bugs.python.org/issue46284 opened by Nicolas SURRIBAS #46285: http/server.py wont respect its protocol_version https://bugs.python.org/issue46285 opened by openalmeida #46287: UNC path normalisation issues on Windows https://bugs.python.org/issue46287 opened by steve.dower #46289: AST: FomattedValue conversion's default value should be -1 https://bugs.python.org/issue46289 opened by iyume #46290: Parameter names are inaccurate in dataclasses docs https://bugs.python.org/issue46290 opened by zsol #46291: [doc] First argument to raise can also be BaseException https://bugs.python.org/issue46291 opened by gtitze #46292: Add microseconds to logging.LogRecord https://bugs.python.org/issue46292 opened by jamescasbon #46296: Unreachable condition: `if enum_class._member_type_ is object` https://bugs.python.org/issue46296 opened by sobolevn Most recent 15 issues with no replies (15) ========================================== #46296: Unreachable condition: `if enum_class._member_type_ is object` https://bugs.python.org/issue46296 #46292: Add microseconds to logging.LogRecord https://bugs.python.org/issue46292 #46290: Parameter names are inaccurate in dataclasses docs https://bugs.python.org/issue46290 #46284: DirEntry returns invalid values for is_dir and is_file on NFS https://bugs.python.org/issue46284 #46283: [subinterpreters] Unicode interned strings must not be shared https://bugs.python.org/issue46283 #46279: [docs] Minor information-ordering issue in __main__ doc https://bugs.python.org/issue46279 #46274: Tokenizer module does not handle backslash characters correctl https://bugs.python.org/issue46274 #46273: Document what asyncio.wait() and asyncio.as_completed() do if https://bugs.python.org/issue46273 #46271: frozen modules are not regenerated on bytecode magic change wh https://bugs.python.org/issue46271 #46267: gzip.compress incorrectly ignores level parameter https://bugs.python.org/issue46267 #46265: Error when cross compiling for hardfloat MIPS https://bugs.python.org/issue46265 #46258: Minor algorithmic improvements for math.isqrt https://bugs.python.org/issue46258 #46255: Remove unnecessary check in _IOBase._check*() methods https://bugs.python.org/issue46255 #46254: Better fitting type for iterating in the trace_init C function https://bugs.python.org/issue46254 #46253: C API documentation of Py_UNICODE_* character properties macro https://bugs.python.org/issue46253 Most recent 15 issues waiting for review (15) ============================================= #46296: Unreachable condition: `if enum_class._member_type_ is object` https://bugs.python.org/issue46296 #46292: Add microseconds to logging.LogRecord https://bugs.python.org/issue46292 #46290: Parameter names are inaccurate in dataclasses docs https://bugs.python.org/issue46290 #46282: print() docs do not indicate its return value https://bugs.python.org/issue46282 #46280: About vulnerabilities in Cpython native code https://bugs.python.org/issue46280 #46269: '__new__' is never shown in `dir(SomeEnum)` https://bugs.python.org/issue46269 #46267: gzip.compress incorrectly ignores level parameter https://bugs.python.org/issue46267 #46261: [doc] fix inaccuracies in sqlite3.Cursor.lastrowid docs https://bugs.python.org/issue46261 #46258: Minor algorithmic improvements for math.isqrt https://bugs.python.org/issue46258 #46255: Remove unnecessary check in _IOBase._check*() methods https://bugs.python.org/issue46255 #46254: Better fitting type for iterating in the trace_init C function https://bugs.python.org/issue46254 #46253: C API documentation of Py_UNICODE_* character properties macro https://bugs.python.org/issue46253 #46252: SSLWantReadError causes _SelectorSocketTransport to close https://bugs.python.org/issue46252 #46249: [sqlite3] move set lastrowid out of the query loop and enable https://bugs.python.org/issue46249 #46246: importlib.metadata.DeprecatedList appears to be missing __slot https://bugs.python.org/issue46246 Top 10 most discussed issues (10) ================================= #46280: About vulnerabilities in Cpython native code https://bugs.python.org/issue46280 21 msgs #46070: [subinterpreters] asyncio crash when importing _asyncio in sub https://bugs.python.org/issue46070 17 msgs #45665: Problems caused by isinstance(list[int], type) returning True https://bugs.python.org/issue45665 10 msgs #46249: [sqlite3] move set lastrowid out of the query loop and enable https://bugs.python.org/issue46249 9 msgs #34931: os.path.splitext with more dots https://bugs.python.org/issue34931 8 msgs #45925: Upgrade macOS and Windows installers to use SQLite 3.37.2 https://bugs.python.org/issue45925 7 msgs #46006: [subinterpreter] _PyUnicode_EqualToASCIIId() issue with subint https://bugs.python.org/issue46006 7 msgs #46142: python --help output is too long https://bugs.python.org/issue46142 7 msgs #46187: Optionally support rounding for math.isqrt() https://bugs.python.org/issue46187 6 msgs #46198: Duplicate and unused code in tests https://bugs.python.org/issue46198 6 msgs Issues closed (75) ================== #9937: _winreg.EnumValue causes MemoryError https://bugs.python.org/issue9937 closed by iritkatriel #17951: TypeError during gdb backtracing https://bugs.python.org/issue17951 closed by iritkatriel #20369: concurrent.futures.wait() blocks forever when given duplicate https://bugs.python.org/issue20369 closed by AlexWaygood #25478: Consider adding a normalize() method to collections.Counter() https://bugs.python.org/issue25478 closed by rhettinger #30337: Vague wording of pkgutil.walk_packages parameter 'prefix' https://bugs.python.org/issue30337 closed by iritkatriel #31137: Add a path attribute to NamedTemporaryFile https://bugs.python.org/issue31137 closed by iritkatriel #32696: Fix pickling exceptions with multiple arguments https://bugs.python.org/issue32696 closed by iritkatriel #33130: functools.reduce signature uses `iterable`, documentation shou https://bugs.python.org/issue33130 closed by iritkatriel #33252: [doc] Clarify ResourceWarning documentation https://bugs.python.org/issue33252 closed by iritkatriel #33857: python exception on Solaris : code for hash blake2b was not fo https://bugs.python.org/issue33857 closed by iritkatriel #34538: Remove encouragement to author a base class for all Exception https://bugs.python.org/issue34538 closed by iritkatriel #37489: pickling instance which inherited from Exception with keyword https://bugs.python.org/issue37489 closed by iritkatriel #39181: Add `os.makedirs()` as `Path.mkdir()` equivalent in correspond https://bugs.python.org/issue39181 closed by jack1142 #40222: "Zero cost" exception handling https://bugs.python.org/issue40222 closed by Mark.Shannon #40477: Python Launcher app on macOS 10.15+ fails to run scripts https://bugs.python.org/issue40477 closed by ned.deily #40545: Expose _PyErr_GetTopmostException https://bugs.python.org/issue40545 closed by iritkatriel #42032: Setting PYTHONPYCACHEPREFIX using ~ (tilde) creates a "~" fold https://bugs.python.org/issue42032 closed by petr.viktorin #42657: Python 3.9.1 building process could not use local standard lib https://bugs.python.org/issue42657 closed by ned.deily #43122: Python Launcher doesn't open a terminal window https://bugs.python.org/issue43122 closed by ned.deily #44092: [sqlite3] Remove special rollback handling https://bugs.python.org/issue44092 closed by pablogsal #44669: TypeError: 'type' object is not subscriptable https://bugs.python.org/issue44669 closed by iritkatriel #45256: Remove the usage of the C stack in Python to Python calls https://bugs.python.org/issue45256 closed by Mark.Shannon #45292: Implement PEP 654: Exception Groups https://bugs.python.org/issue45292 closed by iritkatriel #45615: Missing test for type of error when printing traceback for non https://bugs.python.org/issue45615 closed by iritkatriel #45711: Simplify the interpreter's (type, val, tb) exception represent https://bugs.python.org/issue45711 closed by iritkatriel #45903: What��s New In Python 3.11: wrong reference to Signature.from https://bugs.python.org/issue45903 closed by Mariatta #46031: add POP_JUMP_IF_NOT_NONE and POP_JUMP_IF_NONE https://bugs.python.org/issue46031 closed by Mark.Shannon #46039: Break up the YIELD_FROM instruction. https://bugs.python.org/issue46039 closed by Mark.Shannon #46079: [doc] Broken URL in "Brief Tour of the Standard Library" https://bugs.python.org/issue46079 closed by rhettinger #46095: Improve SeqIter documentation https://bugs.python.org/issue46095 closed by rhettinger #46148: Optimize pathlib https://bugs.python.org/issue46148 closed by kumaraditya303 #46178: Remove `.travis.yml`? https://bugs.python.org/issue46178 closed by sobolevn #46181: Destroying an expaned Combobox prevents Entry focus until Alt+ https://bugs.python.org/issue46181 closed by serhiy.storchaka #46182: `super` and descriptor clarification https://bugs.python.org/issue46182 closed by rhettinger #46184: Remove `netlify.toml`? https://bugs.python.org/issue46184 closed by sobolevn #46196: documentation for cmd library should include columnize() funct https://bugs.python.org/issue46196 closed by eric.araujo #46199: Calculation influenced by print https://bugs.python.org/issue46199 closed by mark.dickinson #46202: remove opcode POP_EXCEPT_AND_RERAISE https://bugs.python.org/issue46202 closed by iritkatriel #46206: Crash when editing emoji containing strings https://bugs.python.org/issue46206 closed by pablogsal #46208: os.path.normpath change between 3.11.0a2 and 3.11.0a3+ https://bugs.python.org/issue46208 closed by steve.dower #46215: Wrong error line after using pdb https://bugs.python.org/issue46215 closed by bedapisl #46218: Change long_pow() to sliding window algorithm https://bugs.python.org/issue46218 closed by tim.peters #46219: except* assumes that an exception group is truthy https://bugs.python.org/issue46219 closed by iritkatriel #46221: Simplify except* implementation and move helpers to exceptions https://bugs.python.org/issue46221 closed by iritkatriel #46222: posixmodule sendfile FreeBSD's constants update https://bugs.python.org/issue46222 closed by asvetlov #46224: doc: Fix bisect example using mutable function default https://bugs.python.org/issue46224 closed by rhettinger #46228: argparse docs: default for prog= in ArgumentParser() should be https://bugs.python.org/issue46228 closed by rhettinger #46229: CODE_OF_CONDUCT shadowing org default https://bugs.python.org/issue46229 closed by Mariatta #46230: NEWLINE* is missing in 'file' rule in PEG grammar https://bugs.python.org/issue46230 closed by pablogsal #46231: grammar rule definition is missing: invalid_double_starred_kvp https://bugs.python.org/issue46231 closed by pablogsal #46233: Minor speedup for bigint squaring https://bugs.python.org/issue46233 closed by tim.peters #46236: PyFunction_GetAnnotations returning Tuple vs Dict https://bugs.python.org/issue46236 closed by pablogsal #46238: Improve constants usage / definition in asyncio.windows_events https://bugs.python.org/issue46238 closed by asvetlov #46239: Incosistent error message in asyncio: windows_events / windows https://bugs.python.org/issue46239 closed by asvetlov #46240: Incorrect hint about forgetting a comma https://bugs.python.org/issue46240 closed by pablogsal #46241: importlib.utils.find_spec() has issues with "tests" folder https://bugs.python.org/issue46241 closed by brett.cannon #46243: Spam https://bugs.python.org/issue46243 closed by christian.heimes #46248: Compilation errors on macOS https://bugs.python.org/issue46248 closed by ned.deily #46251: logger.config.configure_formatter executes arbitrary code https://bugs.python.org/issue46251 closed by vinay.sajip #46256: Objects __del__ called after module have been removed https://bugs.python.org/issue46256 closed by pablogsal #46257: Convert statistics sum of squares to a single pass algorithm https://bugs.python.org/issue46257 closed by rhettinger #46259: float formatting error? https://bugs.python.org/issue46259 closed by rhettinger #46260: Misleading SyntaxError on f-string https://bugs.python.org/issue46260 closed by eric.smith #46262: Enum tests: Error path in `_missing_()` is not covered for `Fl https://bugs.python.org/issue46262 closed by ethan.furman #46263: FreeBSD buildbots cannot compile Python https://bugs.python.org/issue46263 closed by christian.heimes #46266: Improve day constants (`MONDAY` ... `SUNDAY`) in `calendar.py` https://bugs.python.org/issue46266 closed by rhettinger #46268: Buildbot failure for buildbot/AMD64 FreeBSD Non-Debug / Shared https://bugs.python.org/issue46268 closed by erlendaasland #46277: '''...''' error https://bugs.python.org/issue46277 closed by vstinner #46278: AbstractEventLoop.call_* should reflect the 'context' argument https://bugs.python.org/issue46278 closed by asvetlov #46281: Python 3.10.1 build errors on Ubuntu 18.04 https://bugs.python.org/issue46281 closed by christian.heimes #46286: use the new POP_JUMP_IF_NOT_NONE to simplify except* a bit https://bugs.python.org/issue46286 closed by iritkatriel #46288: Migrating python 2.7 to 3.6 using 2to3 tool https://bugs.python.org/issue46288 closed by eric.smith #46293: Typo in specifying escape sequence https://bugs.python.org/issue46293 closed by steven.daprano #46294: Integer overflow & Int values loaded into Bool detected via Li https://bugs.python.org/issue46294 closed by swirsz #46295: Subinterpreters hang in GIL adquisition if an extension module https://bugs.python.org/issue46295 closed by pablogsal

1 0

Sanity check about ctypes
by Yonatan Zunger 06 Jan '22

06 Jan '22

Hey everyone. Quick sanity check: The ctypes docs <https://docs.python.org/3.10/library/ctypes.html#ctypes._CData> refer to _CData as a non-public class which is in the module, but _ctypes.c doesn't actually export it <https://github.com/python/cpython/blob/main/Modules/_ctypes/_ctypes.c#L5680>. (I discovered this because it turns out that typeshed *is* referencing _CData, e.g. in its type annotations for RawIOBase <https://github.com/python/typeshed/blob/master/stdlib/_typeshed/__init__.py…> ) Is this intended behavior in CPython (in which case the docs are a bit off and typeshed has a bug), or is it unexpected to people on this list (in which case it's an issue in _ctypes.c)? Yonatan

2 2

Python 3.11.0a4 is blocked
by Pablo Galindo Salgado 06 Jan '22

06 Jan '22

Hi everyone, I am writing this to notify you that unfortunately the release of 3.11.0a4 is blocked as there are a bunch of release blockers (some of them affect Python 3.10): https://bugs.python.org/issue46263 https://bugs.python.org/issue46208 https://bugs.python.org/issue46006 https://bugs.python.org/issue43683 If this was a single release blocker I would think about moving forward but unfortunately, there are several of them and one of them is that Python fails to compile FreeBSD, so I am halting the release until these are fixed. Regards from rainy London, Pablo Galindo Salgado

3 3