Mailman 3 January 2022 - Python-Dev

Proposal: Allow non-default after default arguments
by Sebastian Rittau Oct. 16, 2023

Oct. 16, 2023

Currently, Python doesn't allow non-default arguments after default arguments: >>> def foo(x=None, y): pass File "<stdin>", line 1 def foo(x=None, y): pass ^ SyntaxError: non-default argument follows default argument I believe that at the time this was introduced, no use cases for this were known and this is is supposed to prevent a source of bugs. I have two use cases for this, one fringe, but valid, the other more important: The fringe use case: Suppose you have a function that takes a 2D coordinate value as separate "x" and "y" arguments. The "x" argument is optional, the "y" argument isn't. Currently there are two ways to do this, none of them particularly great: def foo(y, x): # reverse x and y arguments, confusing ... def foo(x, y=None): # Treat the x argument as y if only one argument is provided if y is None: x, y = y, x ... To me, the "natural" solution looks like this: def foo(x=None, y): ... # Called like this: foo(1, 2) foo(y=2) This could also be useful when evolving APIs. For example there is a function "bar" that takes two required arguments. In a later version, the first argument gains a useful default, the second doesn't. There is no sensible way to evolve the API at the moment. The more important use case involves @overloads. A condensed example of a function where the return type depends on an "encoding" parameter, followed by further parameters that could be called like this: foo(123, "utf-8") # would return bytes foo(encoding="utf-8") foo(123, None) # would return str foo(encoding=None) foo(x=123) # would return str foo() This could ideally be written as: @overload def foo(x: int = ..., encoding: None = ...) -> str: ... @overload def foo(x: int = ..., encoding: str) -> bytes: ... # plus the actual implementation But due to the syntax constraint, this needs to be hacked around with a third overload: @overload def foo(x: int = ... encoding: None = ...) -> str: ... @overload def foo(x: int, encoding: str) -> bytes: ... # for foo(123, "utf-8") @overload def foo(*, encoding: str) -> bytes: ... # for foo(encoding="utf-8") Not only is this hard to read, real examples in typeshed are usually more complex, with many arguments before or after the affected argument or even multiple affected arguments. This often becomes too complex to write or maintain. Here is one example from the wild: https://github.com/python/typeshed/blob/b95b729b9e07ab21d252701af0f5b740467… Allowing non-default arguments after default arguments would solve both use cases above and eliminates a special case. I'm also not sure what exactly the current SyntaxError really protects us from. Adding a non-default after a default argument can't really lead bugs. - Sebastian

11 14

Python multithreading without the GIL
by Sam Gross July 31, 2023

July 31, 2023

Hi, I've been working on changes to CPython to allow it to run without the global interpreter lock. I'd like to share a working proof-of-concept that can run without the GIL. The proof-of-concept involves substantial changes to CPython internals, but relatively few changes to the C-API. It is compatible with many C extensions: extensions must be rebuilt, but usually require small or no modifications to source code. I've built compatible versions of packages from the scientific Python ecosystem, and they are installable through the bundled "pip". Source code: https://github.com/colesbury/nogil Design overview: https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlej… My goal with the proof-of-concept is to demonstrate that removing the GIL is feasible and worthwhile, and that the technical ideas of the project could serve as a basis of such an effort. I'd like to start a discussion about these ideas and gauge the community's interest in this approach to removing the GIL. Regards, Sam Gross colesbury(a)gmail.com / sgross(a)fb.com

24 48

Mixed Python/C debugging
by Skip Montanaro April 26, 2023

April 26, 2023

Having tried comp.lang.python with no response, I turn here... After at least ten years away from Python's run-time interpreter & byte code compiler, I'm getting set to familiarize myself with that again. This will, I think, entail debugging a mixed Python/C environment. I'm an Emacs user and am aware that GDB since 7.0 has support for debugging at the Python code level. Is Emacs+GDB my best bet? Are there any Python IDEs which support C-level breakpoints and debugging? Thanks, Skip

5 6

PEP 448 review
by Guido van Rossum March 29, 2023

March 29, 2023

I'm back, I've re-read the PEP, and I've re-read the long thread with "(no subject)". I think Georg Brandl nailed it: """ *I like the "sequence and dict flattening" part of the PEP, mostly because itis consistent and should be easy to understand, but the comprehension syntaxenhancements seem to be bad for readability and "comprehending" what the codedoes.The call syntax part is a mixed bag on the one hand it is nice to be consistent with the extended possibilities in literals (flattening), but on the other hand there would be small but annoying inconsistencies anyways (e.g. the duplicate kwarg case above).* """ Greg Ewing followed up explaining that the inconsistency between dict flattening and call syntax is inherent in the pre-existing different rules for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1, a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1, **{'a': 1}) is a TypeError.) For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations for function calls proposed by the PEP is an easy +1 -- it's a straightforward extension of the existing pattern, and anybody who knows what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d, **e) means shouldn't be hard either. Understanding the edge case for duplicate keys with f(**d, **e) is a little harder, but the error messages are pretty clear, and it is not a new edge case. The sequence and dict flattening syntax proposals are also clean and logical -- we already have *-unpacking on the receiving side, so allowing *x in tuple expressions reads pretty naturally (and the similarity with *a in argument lists certainly helps). From here, having [a, *x, b, *y] is also natural, and then the extension to other displays is natural: {a, *x, b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me. So that leaves comprehensions. IIRC, during the development of the patch we realized that f(*x for x in xs) is sufficiently ambiguous that we decided to disallow it -- note that f(x for x in xs) is already somewhat of a special case because an argument can only be a "bare" generator expression if it is the only argument. The same reasoning doesn't apply (in that form) to list, set and dict comprehensions -- while f(x for x in xs) is identical in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x for x in xs)] (that's a list of one element, and the element is a generator expression). The basic premise of this part of the proposal is that if you have a few iterables, the new proposal (without comprehensions) lets you create a list or generator expression that iterates over all of them, essentially flattening them: >>> xs = [1, 2, 3] >>> ys = ['abc', 'def'] >>> zs = [99] >>> [*xs, *ys, *zs] [1, 2, 3, 'abc', 'def', 99] >>> But now suppose you have a list of iterables: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xss[0], *xss[1], *xss[2]] [1, 2, 3, 'abc', 'def', 99] >>> Wouldn't it be nice if you could write the latter using a comprehension? >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs for xs in xss] [1, 2, 3, 'abc', 'def', 99] >>> This is somewhat seductive, and the following is even nicer: the *xs position may be an expression, e.g.: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs[:2] for xs in xss] [1, 2, 'abc', 'def', 99] >>> On the other hand, I had to explore the possibilities here by experimenting in the interpreter, and I discovered some odd edge cases (e.g. you can parenthesize the starred expression, but that seems a syntactic accident). All in all I am personally +0 on the comprehension part of the PEP, and I like that it provides a way to "flatten" a sequence of sequences, but I think very few people in the thread have supported this part. Therefore I would like to ask Neil to update the PEP and the patch to take out the comprehension part, so that the two "easy wins" can make it into Python 3.5 (basically, I am accepting two-thirds of the PEP :-). There is some time yet until alpha 2. I would also like code reviewers (Benjamin?) to start reviewing the patch <http://bugs.python.org/issue2292>, taking into account that the comprehension part needs to be removed. -- --Guido van Rossum (python.org/~guido)

7 17

PEP 638: Syntactic macros
by Mark Shannon Feb. 5, 2023

Feb. 5, 2023

Hi everyone, I've submitted my PEP on syntactic macros as PEP 638. https://www.python.org/dev/peps/pep-0638/ All comments and suggestions are welcome. Cheers, Mark

16 25

How about using modern C++ in development of CPython ?
by redradist＠gmail.com Oct. 19, 2022

Oct. 19, 2022

Hi all, What about of using modern C++ in developing CPython ? With C++ we can get the following benefits: 1) Readability - less code, most code is hidden by abstraction without losing performance 2) Maintainability - no code duplication in favor of using reusable classes 3) RAII - Resource Acquisition Is Initialization, predictable allocation and free resources ...

23 51

PEP 505 (None-aware operators) for Python 3.11
by Doug Swarin Sept. 21, 2022

Sept. 21, 2022

Hello, I've been following PEP 505 (https://www.python.org/dev/peps/pep-0505/) since it was first proposed for Python 3.8. It's been deferred for some time and I'm interested in seeing it in Python 3.11, but I know there were also a number of objections which resulted in it being deferred (including by one of the original authors, Mr. Dower). I did email both Mr. Dower and Mr. Haase and they graciously gave me their permission to bring it up on this list for discussion and hopefully final pronouncement one way or the other. I personally believe that the PEP will result in a significant reduction in boilerplate code, and it is substantially similar to the same operators now found in a number of other languages, especially C# and JavaScript (https://wikipedia.org/wiki/Null_coalescing_operator and https://wikipedia.org/wiki/Safe_navigation_operator). I believe strong and valid arguments can be made about the use of None being a fundamental flaw in some types of coding (and that adding additional support for it to the language will increase the use of None in this way), but I also believe there are many use cases in programming where it is by far the simplest way to express various semantics, and the fact exists that None is already used extensively in large quantities of code, and further that there is already a great deal of code written to constantly test against None and break out of a statement without throwing an error. I also understand the argument that especially the maybe-dot (?.) and maybe-subscript (?[) operators can decrease readability of code and also believe these are valid arguments against it. While I believe the existence and use of these operators in other languages definitely helps the case that these can be used and understood successfully, I think it is entirely valid to either consider other syntax (though I prefer the chosen syntax of PEP 505), or even to reduce PEP 505 to having only the coalesce operator (??) and the maybe-assign operator (??=). Separately, I have implemented a pure-Python solution for PEP505 (which is definitely rather beta) which might help test the waters for a final implementation in CPython (though the CPython implementation would of course be much more efficient). It can be found at https://pypi.org/project/pep505/ Thanks, Doug

22 53

PEP 563 and Python 3.10.
by Thomas Wouters July 22, 2022

July 22, 2022

(Starting a new thread so as not to derail any of the ongoing discussions.) Thanks, everyone, for your thoughts on Python 3.10 and the impact of PEP 563 (postponed evaluation of annotations) becoming the default. The Steering Council has considered the issue carefully, along with many of the proposed alternatives and solutions, and we’ve decided that at this point, we simply can’t risk the compatibility breakage of PEP 563. We need to roll back the change that made stringified annotations the default, at least for 3.10. (Pablo is already working on this.) To be clear, we are not reverting PEP 563 itself. The future import will keep working like it did since Python 3.7. We’re delaying making PEP 563 string-based annotations the default until Python 3.11. This will give us time to find a solution that works for everyone (or to find a feasible upgrade path for users who currently rely on evaluated annotations). Some considerations that led us to this decision: - PEP 563’s default change is clearly too disruptive to downstream users and third-party libraries to happen right now. We can’t risk breaking even a small subset of the FastAPI/pydantic users, not to mention other uses of evaluated type annotations that we’re not aware of yet. - PEP 563 provides no warning to users of the feature it’s disabling. Without that, we can’t expect users to be aware of the upcoming breakage. The lack of a warning was by design, and made sense in a world where type annotations were only consumed by static type checkers --- but that’s not actually the situation we’re in. There are clearly existing real-world, run-time uses of type annotations that would be adversely affected by this change. - Originally, PEP 563 was scheduled to take effect in Python 4, and this changed recently (after the discussion in the Language Summit of 2020). It's possible that third-party libraries and users didn’t plan to react in the current time frame as they were not aware of this change in timing. - There isn’t enough time to properly discuss PEP 649 or any of the alternatives before the beta 1 deadline, and we really need to make sure we don’t compound errors here. We need to look for a long term solution, which isn’t possible while still maintaining the release deadlines of Python 3.10. That means we’re also deferring PEP 649 to Python 3.11. In the Steering Council’s unanimous opinion, rolling back the default flip for stringified annotations in Python 3.10 is the least disruptive of all the options. We need to continue discussing the issue and potential solutions, since this merely postpones the problem until 3.11. (For the record, postponing the change further is not off the table, either, for example if the final decision is to treat evaluated annotations as a deprecated feature, with warnings on use.) For what it’s worth, the SC is also considering what we can do to reduce the odds of something like this happening again, but that’s a separate consideration, and a multi-faceted one at that. For the Steering Council, Thomas. -- Thomas Wouters <thomas(a)python.org> Hi! I'm an email virus! Think twice before sending your email to help me spread!

6 5

compiled python3.10 is unable to find _ssl
by Sandeep Gupta May 29, 2022

May 29, 2022

I am having compilation issues again with python3.10 with ssl . The ./configure was invoked with ssl options and ssl modules seems to be build successfully. """ The following modules found by detect_modules() in setup.py, have been built by the Makefile instead, as configured by the Setup files: _abc _hashlib _ssl pwd time """ However, when I do import ssl from python, I get the error: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/shared/Builds/Python-3.10.0/lib/python3.10/ssl.py", line 98, in <module> import _ssl # if we can't import it, let the error propagate ModuleNotFoundError: No module named '_ssl' I can't find _ssl.so in build or install directory. Thanks -S

6 12

Slowly bend the C API towards the limited API to get a stable ABI for everyone
by Victor Stinner April 20, 2022

April 20, 2022

Hi, There is a reason why I'm bothering C extensions maintainers and Python core developers with my incompatible C API changes since Python 3.8. Let me share my plan with you :-) In 2009 (Python 3.2), Martin v. Löwis did an amazing job with the PEP 384 "Defining a Stable ABI" to provide a "limited C API" and a "stable ABI" for C extensions: build an extension once, use it on multiple Python versions. Some projects like PyQt5 and cryptograpy use it, but it is just a drop in the PyPI ocean (353,084 projects). I'm trying to bend the "default" C API towards this "limited C API" to make it possible tomorrow to build *more* C extensions for the stable ABI. My goal is that the stable ABI would be the default, and only a minority of C extensions would opt-out because they need to access to more functions for best performance. The basic problem is that at the ABI level, C extensions must only call functions, rather than getting and setting directly to structure members. Structures changes frequently in Python (look at changes between Python 3.2 and Python 3.11), and any minor structure change breaks the ABI. The limited C API hides structures and only use function calls to solve this problem. Since 2020, I'm modifying the C API, one function by one, to slowly hide implementations (prepare the API to make strutures opaque). I focused on the following structures: * PyObject and PyVarObject (bpo-39573) * PyTypeObject (bpo-40170) * PyFrameObject (bpo-40421) * PyThreadState (bpo-39947) The majority of C extensions use functions and macros, they don't access directly structure members. There are a few members which are sometimes accessed directly which prevents making these structures opaque. For example, some old C extensions use obj->ob_type rather than Py_TYPE(obj). Fixing the minority of C extensions should benefit to the majority which may become compatible with the stable ABI. I am also converting macros to static inline functions to fix their API: define parameter types, result type and avoid surprising macros side effects ("macro pitfalls"). I wrote the PEP 670 "Convert macros to functions in the Python C API" for these changes. I wrote the upgrade_pythoncapi.py tool in my pythoncapi_project (*) which modify C code to use Py_TYPE(), Py_SIZE() and Py_REFCNT() rather than accessing directly PyObject and PyVarObject members. (*) https://github.com/pythoncapi/pythoncapi_compat In this tool, I also added "Borrow" variant of functions like PyFrame_GetCode() which returns a strong reference, to replace frame->f_code with _PyFrame_GetCodeBorrow(). In Python 3.11, you cannot use the frame->f_code member anymore, since it has been removed! You must call PyFrame_GetCode() (or pythoncapi_compat _PyFrame_GetCodeBorrow() variant). There are also a few macros which can be used as l-values like Py_TYPE(): "Py_TYPE(type1) = type2" must now be written "Py_SET_TYPE(type1, type2)" to avoid setting directly the tp_type type at the ABI level. I proposed the PEP 674 "Disallow using Py_TYPE() and Py_SIZE() macros as l-values" to solve these issues. Currently, many "functions" are still implemented as macros or static inline functions, so C extensions still access structure members at the ABI level for best Python performance. Converting these to regular functions has an impact on performance and I would prefer to first write a PEP giving the rationale for that. Today, it is not possible yet to build numpy for the stable ABI. The gap is just too large for this big C extension. But step by step, the C API becomes closer to the limited API, and more and more code is ready to be built for the stable ABI. Well, these C API changes have other advantages, like preparing Python for further optimizations, ease Python maintenance, clarify the seperation between the limited C API and the default C API, etc. ;-) Victor -- Night gathers, and now my watch begins. It shall not end until my death.

12 38

PEP 678: Enriching Exceptions with Notes
by Zac Hatfield Dodds April 12, 2022

April 12, 2022

Hi all, I've written PEP 678, proposing the addition of a __note__ attribute which can be used to enrich displayed exceptions. This is particularly useful with ExceptionGroup, or where exception chaining is unsuitable, and ensures that 'orphan' output is not produced if the annotated exception is caught. Link to the PEP: https://www.python.org/dev/peps/pep-0678/ *Please, redirect all discussions to:* https://discuss.python.org/t/pep-678-enriching-exceptions-with-notes/13374 Thanks for your time, and to those who have already made (very helpful!) comments! Best wishes from just-moved-to San Francisco, Zac Hatfield-Dodds

2 1

RFC on PEP 655: Required[] and NotRequired[] for TypedDict
by David Foster March 10, 2022

March 10, 2022

This PEP [1] introduces syntax to mark individual keys of a TypedDict as either required or potentially-missing. Previously the only way to have a TypedDict with mixed required and non-required keys was to define two TypedDicts - one with total=True and one with total=False - and have one of those TypedDicts inherit from the other. This PEP introduces special forms Required[] and NotRequired[] to the "typing" module that can be used to directly mark individual items of a TypedDict as required or not-required. We have implementations for mypy, pyright, and pyanalyze. Consensus on the PEP has been reached in typing-sig. My understanding of the PEP process as described in PEP 1 [2] is that content review should be requested of core developers here in python-dev. So consider this my official request for comments! ^_^ I believe that after review & feedback from python-dev the next step would be to submit this PEP to the Steering Council. However it's not clear to me from [2] where I should actually do that when the time comes. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy [1]: https://www.python.org/dev/peps/pep-0655/ [2]: https://www.python.org/dev/peps/pep-0001/#pep-review-resolution

4 6

Suggestion: a little language for type definitions
by jack.jansen＠cwi.nl Feb. 10, 2022

Feb. 10, 2022

I posted this suggestion earlier in the callable type syntax discussion, at which point it was completely ignored. Possibly because it’s a really stupid idea, but let me post it again on the off chance that it isn’t a stupid idea but was overlooked. > If I can make a wild suggestion: why not create a little language for type specifications? > > If you look at other programming languages you’ll see that the “type definition sub-language” is often completely different from the “execution sub-language”, with only some symbols in common and used in vaguely related ways. `bool (*myfuncptr)(int, float*)` uses a completely different set of syntactic rules than `rv = (*myfunptr)(*myintptr, &myfloat)`. So with some grains of salt you could say that C is comprised of a declarative typing sublanguage and an imperative execution sublanguage. And an even better example is Pascal, which uses a set of syntactic constructs for typing that are completely different from the execution statement syntax: `var a : array[1..10] of real` looks very different from `a[1]`, where C `float a[10]` looks pretty similar to `a[10]`. The next bit of my original email is another wild idea, the previous bit doesn’t depend on it really. I can imagine completely different ways of doing a typing sublanguage: > Python typing uses basically a subset of the execution expression syntax as its declarative typing language. > > What if we created a little language that is clearly flagged, for example as t”….” or t’….’? Then we could simply define the typestring language to be readable, so you could indeed say t”(int, str) -> bool”. And we could even allow escapes (similar to f-strings) so that the previous expression could also be specified, if you really wanted to, as t”{typing.Callable[[int, str], bool}”. -- Jack Jansen, <Jack.Jansen(a)cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman

15 35

"immortal" objects and how they would help per-interpreter GIL
by Eric Snow Feb. 8, 2022

Feb. 8, 2022

Most of the work toward interpreter isolation and a per-interpreter GIL involves moving static global variables to _PyRuntimeState or PyInterpreterState (or module state). Through the effort of quite a few people, we've made good progress. However, many globals still remain, with the majority being objects and most of those being static strings (e.g. _Py_Identifier), static types (incl. exceptions), and singletons. On top of that, a number of those objects are exposed in the public C-API and even in the limited API. :( Dealing with this specifically is probably the trickiest thing I've had to work through in this project. There is one solution that would help both of the above in a nice way: "immortal" objects. The idea of objects that never get deallocated isn't new and has been explored here several times. Not that long ago I tried it out by setting the refcount really high. That worked. Around the same time Eddie Elizondo at Facebook did something similar but modified Py_INCREF() and Py_DECREF() to keep the refcount from changing. Our solutions were similar but with different goals in mind. (Facebook wants to avoid copy-on-write in their pre-fork model.) A while back I concluded that neither approach would work for us. The approach I had taken would have significant cache performance penalties in a per-interpreter GIL world. The approach that modifies Py_INCREF() has a significant performance penalty due to the extra branch on such a frequent operation. Recently I've come back to the idea of immortal objects because it's much simpler than the alternate (working) solution I found. So how do we get around that performance penalty? Let's say it makes CPython 5% slower. We have some options: * live with the full penalty * make other changes to reduce the penalty to a more acceptable threshold than 5% * eliminate the penalty (e.g. claw back 5% elsewhere) * abandon all hope Mark Shannon suggested to me some things we can do. Also, from a recent conversation with Dino Viehland it sounds like Eddie was able to reach performance-neutral with a few techniques. So here are some things we can do to reduce or eliminate that penalty: * reduce refcount operations on high-activity objects (e.g. None, True, False) * reduce refcount operations in general * walk the heap at the end of runtime initialization and mark all objects as immortal * mark all global objects as immortal (statics or in _PyRuntimeState; for PyInterpreterState not needed) What do you think? Does this sound realistic? Are there additional things we can do to counter that penalty? -eric

16 35

PEP 674 – Disallow using macros as l-value (version 2)
by Victor Stinner Feb. 7, 2022

Feb. 7, 2022

Hi, I made the following changes to the PEP since the version 1 (November 30): * Elaborate the HPy section and add a new "Relationship with the HPy project" section * Elaborate which projects are impacted and which changes are needed * Remove the PyPy section The PEP can be read online: https://python.github.io/peps/ep-0674/ And you can find the plain text below. Victor PEP: 674 Title: Disallow using macros as l-value Author: Victor Stinner <vstinner(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30-Nov-2021 Python-Version: 3.11 Abstract ======== Incompatible C API change disallowing using macros as l-value to: * Allow evolving CPython internals; * Ease the C API implementation on other Python implementation; * Help migrating existing C extensions to the HPy API. On the PyPI top 5000 projects, only 14 projects (0.3%) are affected by 4 macro changes. Moreover, 24 projects just have to regenerate their Cython code to use ``Py_SET_TYPE()``. In practice, the majority of affected projects only have to make two changes: * Replace ``Py_TYPE(obj) = new_type;`` with ``Py_SET_TYPE(obj, new_type);``. * Replace ``Py_SIZE(obj) = new_size;`` with ``Py_SET_SIZE(obj, new_size);``. Rationale ========= Using a macro as a l-value -------------------------- In the Python C API, some functions are implemented as macro because writing a macro is simpler than writing a regular function. If a macro exposes directly a structure member, it is technically possible to use this macro to not only get the structure member but also set it. Example with the Python 3.10 ``Py_TYPE()`` macro:: #define Py_TYPE(ob) (((PyObject *)(ob))->ob_type) This macro can be used as a **r-value** to **get** an object type:: type = Py_TYPE(object); It can also be used as **l-value** to **set** an object type:: Py_TYPE(object) = new_type; It is also possible to set an object reference count and an object size using ``Py_REFCNT()`` and ``Py_SIZE()`` macros. Setting directly an object attribute relies on the current exact CPython implementation. Implementing this feature in other Python implementations can make their C API implementation less efficient. CPython nogil fork ------------------ Sam Gross forked Python 3.9 to remove the GIL: the `nogil branch <https://github.com/colesbury/nogil/>`_. This fork has no ``PyObject.ob_refcnt`` member, but a more elaborated implementation for reference counting, and so the ``Py_REFCNT(obj) = new_refcnt;`` code fails with a compiler error. Merging the nogil fork into the upstream CPython main branch requires first to fix this C API compatibility issue. It is a concrete example of a Python optimization blocked indirectly by the C API. This issue was already fixed in Python 3.10: the ``Py_REFCNT()`` macro has been already modified to disallow using it as a l-value. These statements are endorsed by Sam Gross (nogil developer). HPy project ----------- The `HPy project <https://hpyproject.org/>`_ is a brand new C API for Python using only handles and function calls: handles are opaque, structure members cannot be accessed directly, and pointers cannot be dereferenced. Searching and replacing ``Py_SET_SIZE()`` is easier and safer than searching and replacing some strange macro uses of ``Py_SIZE()``. ``Py_SIZE()`` can be semi-mechanically replaced by ``HPy_Length()``, whereas seeing ``Py_SET_SIZE()`` would immediately make clear that the code needs bigger changes in order to be ported to HPy (for example by using ``HPyTupleBuilder`` or ``HPyListBuilder``). The fewer internal details exposed via macros, the easier it will be for HPy to provide direct equivalents. Any macro that references "non-public" interfaces effectively exposes those interfaces publicly. These statements are endorsed by Antonio Cuni (HPy developer). GraalVM Python -------------- In GraalVM, when a Python object is accessed by the Python C API, the C API emulation layer has to wrap the GraalVM objects into wrappers that expose the internal structure of the CPython structures (PyObject, PyLongObject, PyTypeObject, etc). This is because when the C code accesses it directly or via macros, all GraalVM can intercept is a read at the struct offset, which has to be mapped back to the representation in GraalVM. The smaller the "effective" number of exposed struct members (by replacing macros with functions), the simpler GraalVM wrappers can be. This PEP alone is not enough to get rid of the wrappers in GraalVM, but it is a step towards this long term goal. GraalVM already supports HPy which is a better solution in the long term. These statements are endorsed by Tim Felgentreff (GraalVM Python developer). Specification ============= Disallow using macros as l-value -------------------------------- PyObject and PyVarObject macros ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * ``Py_TYPE()``: ``Py_SET_TYPE()`` must be used instead * ``Py_SIZE()``: ``Py_SET_SIZE()`` must be used instead "GET" macros ^^^^^^^^^^^^ * ``PyByteArray_GET_SIZE()`` * ``PyBytes_GET_SIZE()`` * ``PyCFunction_GET_CLASS()`` * ``PyCFunction_GET_FLAGS()`` * ``PyCFunction_GET_FUNCTION()`` * ``PyCFunction_GET_SELF()`` * ``PyCell_GET()`` * ``PyCode_GetNumFree()`` * ``PyDict_GET_SIZE()`` * ``PyFunction_GET_ANNOTATIONS()`` * ``PyFunction_GET_CLOSURE()`` * ``PyFunction_GET_CODE()`` * ``PyFunction_GET_DEFAULTS()`` * ``PyFunction_GET_GLOBALS()`` * ``PyFunction_GET_KW_DEFAULTS()`` * ``PyFunction_GET_MODULE()`` * ``PyHeapType_GET_MEMBERS()`` * ``PyInstanceMethod_GET_FUNCTION()`` * ``PyList_GET_SIZE()`` * ``PyMemoryView_GET_BASE()`` * ``PyMemoryView_GET_BUFFER()`` * ``PyMethod_GET_FUNCTION()`` * ``PyMethod_GET_SELF()`` * ``PySet_GET_SIZE()`` * ``PyTuple_GET_SIZE()`` * ``PyUnicode_GET_DATA_SIZE()`` * ``PyUnicode_GET_LENGTH()`` * ``PyUnicode_GET_LENGTH()`` * ``PyUnicode_GET_SIZE()`` * ``PyWeakref_GET_OBJECT()`` "AS" macros ^^^^^^^^^^^ * ``PyByteArray_AS_STRING()`` * ``PyBytes_AS_STRING()`` * ``PyFloat_AS_DOUBLE()`` * ``PyUnicode_AS_DATA()`` * ``PyUnicode_AS_UNICODE()`` PyUnicode macros ^^^^^^^^^^^^^^^^ * ``PyUnicode_1BYTE_DATA()`` * ``PyUnicode_2BYTE_DATA()`` * ``PyUnicode_4BYTE_DATA()`` * ``PyUnicode_DATA()`` * ``PyUnicode_IS_ASCII()`` * ``PyUnicode_IS_COMPACT()`` * ``PyUnicode_IS_READY()`` * ``PyUnicode_KIND()`` * ``PyUnicode_READ()`` * ``PyUnicode_READ_CHAR()`` PyDateTime "GET" macros ^^^^^^^^^^^^^^^^^^^^^^^ * ``PyDateTime_DATE_GET_FOLD()`` * ``PyDateTime_DATE_GET_HOUR()`` * ``PyDateTime_DATE_GET_MICROSECOND()`` * ``PyDateTime_DATE_GET_MINUTE()`` * ``PyDateTime_DATE_GET_SECOND()`` * ``PyDateTime_DATE_GET_TZINFO()`` * ``PyDateTime_DELTA_GET_DAYS()`` * ``PyDateTime_DELTA_GET_MICROSECONDS()`` * ``PyDateTime_DELTA_GET_SECONDS()`` * ``PyDateTime_GET_DAY()`` * ``PyDateTime_GET_MONTH()`` * ``PyDateTime_GET_YEAR()`` * ``PyDateTime_TIME_GET_FOLD()`` * ``PyDateTime_TIME_GET_HOUR()`` * ``PyDateTime_TIME_GET_MICROSECOND()`` * ``PyDateTime_TIME_GET_MINUTE()`` * ``PyDateTime_TIME_GET_SECOND()`` * ``PyDateTime_TIME_GET_TZINFO()`` PyDescr macros ^^^^^^^^^^^^^^ * ``PyDescr_NAME()`` * ``PyDescr_TYPE()`` Port C extensions to Python 3.11 -------------------------------- In practice, the majority of projects affected by these PEP only have to make two changes: * Replace ``Py_TYPE(obj) = new_type;`` with ``Py_SET_TYPE(obj, new_type);``. * Replace ``Py_SIZE(obj) = new_size;`` with ``Py_SET_SIZE(obj, new_size);``. The `pythoncapi_compat project <https://github.com/pythoncapi/pythoncapi_compat>`_ can be used to update automatically C extensions: add Python 3.11 support without losing support with older Python versions. The project provides a header file which provides ``Py_SET_REFCNT()``, ``Py_SET_TYPE()`` and ``Py_SET_SIZE()`` functions to Python 3.8 and older. PyTuple_GET_ITEM() and PyList_GET_ITEM() ---------------------------------------- The ``PyTuple_GET_ITEM()`` and ``PyList_GET_ITEM()`` macros are left unchanged. The code pattern ``&PyTuple_GET_ITEM(tuple, 0)`` and ``&PyList_GET_ITEM(list, 0)`` is still commonly used to get access to the inner ``PyObject**`` array. Changing these macros is out of the scope of this PEP. Backwards Compatibility ======================= The proposed C API changes are backward incompatible on purpose. At December 1, 2021, a code search on the PyPI top 5000 projects (4760 projects in practice, others don't have a source archive) found that `only 14 projects are affected <https://bugs.python.org/issue45476#msg407456>`_ (0.3%): * datatable (1.0.0) * frozendict (2.1.1) * guppy3 (3.1.2) * M2Crypto (0.38.0) * mecab-python3 (1.0.4) * mypy (0.910) * Naked (0.1.31) * pickle5 (0.0.12) * pysha3 (1.0.2) * python-snappy (0.6.0) * recordclass (0.16.3) * scipy (1.7.3) * zodbpickle (2.2.0) * zstd (1.5.0.2) These 14 projects only use 4 macros as l-value: * ``PyDescr_NAME()`` and ``PyDescr_TYPE()`` (2 projects) * ``Py_SIZE()`` (8 projects) * ``Py_TYPE()`` (4 projects) Moreover, `24 projects just have to regenerate their Cython code <https://bugs.python.org/issue45476#msg407416>`_ to use ``Py_SET_TYPE()``. This change does not follow the PEP 387 deprecation process. There is no known way to emit a deprecation warning only when a macro is used as a l-value, but not when it's used differently (ex: as a r-value). Relationship with the HPy project ================================= The HPy project --------------- The hope with the HPy project is indeed to provide a C API that is close to the original API to make porting easy and have it perform as close to the existing API as possible. At the same time, HPy is sufficently removed to be a good "C extension API" (as opposed to a stable subset of the CPython implementation API) that does not leak implementation details. To ensure this latter property is why the HPy project tries to develop everything in parallel for CPython, PyPy, and GraalVM Python. HPy is still evolving very fast. Issues are still being solved while migrating NumPy, and work has begun on adding support for HPy to Cython. Work on pybind11 is starting soon. Tim Felgentreff believes by the time HPy has these users of the existing C API working, HPy should be in a state where it is generally useful and can be deemed stable enough that further development can follow a more stable process. In the long run the HPy project would like to become a promoted API to write Python C extensions. The HPy project is a good solution for the long term. It has the advantage of being developed outside Python and it doesn't require any C API change. The C API is here is stay for a few more years ---------------------------------------------- The first concern about HPy is that right now, HPy is not mature nor widely used, and CPython still has to continue supporting a large amount of C extensions which are not likely to be ported to HPy soon. The second concern is the inability to evolve CPython internals to implement new optimizations, and the inefficient implementation of the current C API in PyPy, GraalPython, etc. Sadly, HPy will only solve these problems when most C extensions will be fully ported to HPy: when it will become reasonable to consider dropping the "legacy" Python C API. While porting a C extension to HPy can be done incrementally on CPython, it requires to modify a lot of code and takes time. Porting most C extensions to HPy is expected to take a few years. This PEP proposes to make the C API "less bad" by fixing one problem which is clearily identified as causing practical issues: macros used as l-values. This PEP only requires to update a minority of C extensions, and usually only a few lines need to be changed in impacted extensions. For example, numpy 1.22 is made of 307,300 lines of C code, and adapting numpy to the this PEP only modified 11 lines (use Py_SET_TYPE and Py_SET_SIZE) and adding 4 lines (to define Py_SET_TYPE and Py_SET_SIZE for Python 3.8 and older). Porting numpy to HPy is expected to require modifying more lines than that. Right now, it's hard to bet which approach is the best: fixing the current C API, or focusing on HPy. It would be risky to only focus on HPy. Rejected Idea: Leave the macros as they are =========================================== The documentation of each function can discourage developers to use macros to modify Python objects. If these is a need to make an assignment, a setter function can be added and the macro documentation can require to use the setter function. For example, a ``Py_SET_TYPE()`` function has been added to Python 3.9 and the ``Py_TYPE()`` documentation now requires to use the ``Py_SET_TYPE()`` function to set an object type. If developers use macros as l-value, it's their responsibility when their code breaks, not the Python responsibility. We are operating under the consenting adults principle: we expect users of the Python C API to use it as documented and expect them to take care of the fallout, if things break when they don't. This idea was rejected because only few developers read the documentation, and only a minority is tracking changes of the Python C API documentation. The majority of developers are only using CPython and so are not aware of compatibility issues with other Python implementations. Moreover, continuing to allow using macros as l-value does not help the HPy project and leaves the burden of emulating them on GraalVM's Python implementation. Macros already modified ======================= The following C API macros have already been modified to disallow using them as l-value: * ``PyCell_SET()`` * ``PyList_SET_ITEM()`` * ``PyTuple_SET_ITEM()`` * ``Py_REFCNT()`` (Python 3.10): ``Py_SET_REFCNT()`` must be used * ``_PyGCHead_SET_FINALIZED()`` * ``_PyGCHead_SET_NEXT()`` * ``asdl_seq_GET()`` * ``asdl_seq_GET_UNTYPED()`` * ``asdl_seq_LEN()`` * ``asdl_seq_SET()`` * ``asdl_seq_SET_UNTYPED()`` For example, ``PyList_SET_ITEM(list, 0, item) < 0`` now fails with a compiler error as expected. Discussion ========== * `PEP 674: Disallow using macros as l-value <https://mail.python.org/archives/list/python-dev@python.org/thread/KPIJPPJ6…>`_ (Nov 30, 2021) References ========== * `Python C API: Add functions to access PyObject <https://vstinner.github.io/c-api-abstract-pyobject.html>`_ (October 2021) article by Victor Stinner * `[C API] Disallow using PyFloat_AS_DOUBLE() as l-value <https://bugs.python.org/issue45476>`_ (October 2021) * `[capi-sig] Py_TYPE() and Py_SIZE() become static inline functions <https://mail.python.org/archives/list/capi-sig@python.org/thread/WGRLTHTHC3…>`_ (September 2021) * `[C API] Avoid accessing PyObject and PyVarObject members directly: add Py_SET_TYPE() and Py_IS_TYPE(), disallow Py_TYPE(obj)=type <https://bugs.python.org/issue39573>`__ (February 2020) * `bpo-30459: PyList_SET_ITEM could be safer <https://bugs.python.org/issue30459>`_ (May 2017) Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

3 7

Increase of Spammy PRs and PR reviews
by Lrupert Feb. 1, 2022

Feb. 1, 2022

As someone who is watching the python/cpython repository, I'm very used to see lots of traffic. But lately there have been a surge of spammy PRs which are about the same, generally very trivial subject but individually fixing each occurrence one by one: - Add coverage to X (tens of them, as separate PRs) - Make `test_xxx` executable with direct invocation (tens of them, as separate PRs) - Lint source with flake8, fix linting errors (stylistic changes) And lots of non-committer PR reviews that only approve. Am I the only one who feels like this is only done to 'grind' commits (get a higher number of commits into the repository, and boast about it)? In the past there were one or two people who would submit typo fixes, but most of them weren't making it continuously. The situation right now feels much worse than those.

21 29

PEP 651, Robust Stack Overflow Handling, Rejection notice
by Python Steering Council Jan. 31, 2022

Jan. 31, 2022

Hi Mark, Thank you for submitting PEP 651. The Steering Council has spent the past two weeks reviewing PEP 651. After careful consideration, we have decided to reject the PEP. The following were the key points that led us to this decision: * The benefits are not compelling enough. Deep recursion is not a common tool in Python, and even with PEP 651 it would not be efficient enough to make it a common tool. * The benefit of PEP 651 is negated as soon as a non-Python function is involved in the recursion, making the likelihood of it being useful even smaller. It also creates easy pitfalls for users who do end up relying on recursion. * We believe the PEP understates the disruption created by the technical solution of multiple Python stack frames per C call. Although this may be solvable, it will certainly cause substantial disruption to existing debuggers, tracers, and state inspection tools as they need to adapt to this change (which may not be trivial). * As the way to approach this will be platform-specific (as some parts of the proposal are not portable), this can cause generic Python code to behave differently on different platforms, making this kind of code less portable and less predictable. In the end, the benefit of the PEP does not outweigh the cost of the potential breakage, confusion, and unpredictability of the feature. With our appreciation, The Python Steering Council

6 8

Blocking the main branch due to too many buildbots failing
by Pablo Galindo Salgado Jan. 29, 2022

Jan. 29, 2022

Hi everyone, We have a huge amount of buildbots failing and seems to be related to different issues that keep piling up. To prevent this from going worse,* I am blocking the main branch* *until these issues are resolved.* Only release managers and the developer in residence will be able to merge pull requests until these issues are fixed. Please, ping me if you have a pull request for fixing any of these issues so we can merge. I apologize for the inconvenience. Thanks for your understanding, Regards from rainy London, Pablo Galindo Salgado

2 1

Summary of Python tracker Issues
by Python tracker Jan. 28, 2022

Jan. 28, 2022

ACTIVITY SUMMARY (2022-01-21 - 2022-01-28) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7157 (-18) closed 51138 (+124) total 58295 (+106) Open issues with patches: 2888 Issues opened (71) ================== #33205: GROWTH_RATE prevents dict shrinking https://bugs.python.org/issue33205 reopened by rhettinger #38195: A bug in the multiprocessing module https://bugs.python.org/issue38195 reopened by eshkrig #44733: Feature request: maxtasksperchild for ProcessPoolExecutor https://bugs.python.org/issue44733 reopened by gregory.p.smith #44734: turtle: tests for Vec2D.__abs__ are too strict https://bugs.python.org/issue44734 reopened by petr.viktorin #45162: Remove old deprecated unittest features https://bugs.python.org/issue45162 reopened by gregory.p.smith #45200: Address Sanitizer: libasan dead lock in pthread_create() (test https://bugs.python.org/issue45200 reopened by vstinner #45680: Documentation on `GenericAlias` objects and `__class_getitem__ https://bugs.python.org/issue45680 reopened by kj #46240: Incorrect hint about forgetting a comma https://bugs.python.org/issue46240 reopened by vstinner #46285: protocol_version in http.server.test can be ignored https://bugs.python.org/issue46285 reopened by eric.araujo #46462: Email Header Folding Converts Non-CRLF Newlines to CRLFs https://bugs.python.org/issue46462 opened by jwalterclark #46464: concurrent.futures.ProcessPoolExecutor can deadlock when tcmal https://bugs.python.org/issue46464 opened by yilei #46465: Regression caused by CALL_FUNCTION specialization for C functi https://bugs.python.org/issue46465 opened by vstinner #46472: A option that choose between single quote and double quote in https://bugs.python.org/issue46472 opened by I-love-study #46475: typing.Never and typing.assert_never https://bugs.python.org/issue46475 opened by Jelle Zijlstra #46477: Enum: ensure bitwise operators on subclasses are correct https://bugs.python.org/issue46477 opened by ethan.furman #46479: Implement typing.reveal_locals https://bugs.python.org/issue46479 opened by Jelle Zijlstra #46480: Implement typing.assert_type https://bugs.python.org/issue46480 opened by Jelle Zijlstra #46482: `typing.Annotation.__new__` is not covered https://bugs.python.org/issue46482 opened by sobolevn #46483: `pathlib.PurePath.__class_getitem__` does not return `GenericA https://bugs.python.org/issue46483 opened by sobolevn #46484: Add test for Calendar().iterweekdays() https://bugs.python.org/issue46484 opened by wangjiahua #46487: `_SSLProtocolTransport` doesn't have the `get_write_buffer_lim https://bugs.python.org/issue46487 opened by mooncell07 #46489: webbrowser crashes Ubuntu kernel https://bugs.python.org/issue46489 opened by dizcza #46490: Add "follow_symlinks=False" support for "os.utime()" on Window https://bugs.python.org/issue46490 opened by Delgan #46493: Add an API to indicate if the process might have multiple thre https://bugs.python.org/issue46493 opened by gregory.p.smith #46494: Mention typing_extensions in the typing documentation https://bugs.python.org/issue46494 opened by Jelle Zijlstra #46495: IDLE subsection of What's New 3.11 https://bugs.python.org/issue46495 opened by terry.reedy #46496: idlelib/NEWS.txt for 3.11.0 and backports https://bugs.python.org/issue46496 opened by terry.reedy #46497: IDLE macOS shortcut ctrl+S doesn��t work for show completions https://bugs.python.org/issue46497 opened by dvd101x #46498: Add new triplets for loongarch64 https://bugs.python.org/issue46498 opened by loongson-zn #46500: make timeit module accept files https://bugs.python.org/issue46500 opened by CCLDArjun #46501: Windows 10, turtle left right not working https://bugs.python.org/issue46501 opened by wizprokidz #46506: [Windows] wrap CreateFile to support follow_symlinks https://bugs.python.org/issue46506 opened by eryksun #46507: enabling cProfile to profile code given as an argument "�� la" https://bugs.python.org/issue46507 opened by jul2 #46508: codec name acceptance became way too lenient in 3.9 https://bugs.python.org/issue46508 opened by gregory.p.smith #46509: Type-specialized Py_DECREF https://bugs.python.org/issue46509 opened by Dennis Sweeney #46511: dataclasses: Allow typing.Annotated to wrap dataclasses-specif https://bugs.python.org/issue46511 opened by GBeauregard #46512: Explicit or correct behavior of filecmp.cmpfiles w/ absolute p https://bugs.python.org/issue46512 opened by bers #46518: SSL socket timeout not set after client connects via accept https://bugs.python.org/issue46518 opened by tomazas #46520: `ast.unparse` produces syntactically illegal code for identifi https://bugs.python.org/issue46520 opened by Kodiologist #46521: compile_command not raising syntax error when command ends wit https://bugs.python.org/issue46521 opened by tusharsadhwani #46522: concurrent.futures.__getattr__ raises the wrong AttributeError https://bugs.python.org/issue46522 opened by graingert #46523: Test suite skips failing tests when setUp[Class] fails https://bugs.python.org/issue46523 opened by sobolevn #46524: test_peg_generator takes 8 minutes on Windows https://bugs.python.org/issue46524 opened by vstinner #46525: datetime.timestamp() lose precision when the time is too large https://bugs.python.org/issue46525 opened by yilei #46526: iglob exclude patterns https://bugs.python.org/issue46526 opened by hleth #46528: Simplify the VM's stack manipulations https://bugs.python.org/issue46528 opened by brandtbucher #46532: Improve effeciency of PRECALL/CALL instructions https://bugs.python.org/issue46532 opened by Mark.Shannon #46533: Specialize for staticmethods and classmethods https://bugs.python.org/issue46533 opened by Mark.Shannon #46534: Implementing PEP 673 (Self type) https://bugs.python.org/issue46534 opened by Gobot1234 #46535: Possible bug: pdb causes exception https://bugs.python.org/issue46535 opened by cklb #46540: dylibs not loading properly from NFS mounts https://bugs.python.org/issue46540 opened by embassy_vfx #46541: Replace _Py_IDENTIFIER() with statically initialized objects. https://bugs.python.org/issue46541 opened by eric.snow #46542: test_json and test_lib2to3 crash on s390x Fedora Clang 3.x bui https://bugs.python.org/issue46542 opened by vstinner #46543: Add sys._getfunc https://bugs.python.org/issue46543 opened by Jelle Zijlstra #46544: `textwrap.TextWrapper` leaks two intermediate vars into class https://bugs.python.org/issue46544 opened by sobolevn #46547: `pydoc.Helper` leaks several `for` loop variables https://bugs.python.org/issue46547 opened by sobolevn #46548: macOS installers cannot be signed on Monterey https://bugs.python.org/issue46548 opened by nneonneo #46550: __slots__ updates despite being read-only https://bugs.python.org/issue46550 opened by IanLee1521 #46553: typing: get_type_hints on stringified lone ClassVar raises Typ https://bugs.python.org/issue46553 opened by GBeauregard #46554: Add append keyword argument to Path.write_text() and Path.writ https://bugs.python.org/issue46554 opened by keelung-yang #46555: Unicode-mangled names refer inconsistently to constants https://bugs.python.org/issue46555 opened by Kodiologist #46556: pathlib.Path.__enter__() should emit DeprecationWarning https://bugs.python.org/issue46556 opened by barneygale #46557: Logging captured warnings with a format string unnecessarily g https://bugs.python.org/issue46557 opened by mnito #46559: base64 documentation is out-dated; altchars has to have the ex https://bugs.python.org/issue46559 opened by ynn #46560: ParamSpec: Typo in doc string https://bugs.python.org/issue46560 opened by ESchalton #46561: Descriptor resolution should own arguments passed to descripto https://bugs.python.org/issue46561 opened by tekknolagi #46563: Add tests and documentation regarding context of exception in https://bugs.python.org/issue46563 opened by iritkatriel #46564: Near zero-cost super().meth() calls via adaptive superinstruct https://bugs.python.org/issue46564 opened by kj #46565: Multiple modules leak `for` loop variables into module's names https://bugs.python.org/issue46565 opened by sobolevn #46566: Support -3.11-arm64 in py.exe launcher https://bugs.python.org/issue46566 opened by steve.dower #46567: Add Tcl/Tk builds for ARM64 https://bugs.python.org/issue46567 opened by steve.dower Most recent 15 issues with no replies (15) ========================================== #46567: Add Tcl/Tk builds for ARM64 https://bugs.python.org/issue46567 #46566: Support -3.11-arm64 in py.exe launcher https://bugs.python.org/issue46566 #46564: Near zero-cost super().meth() calls via adaptive superinstruct https://bugs.python.org/issue46564 #46563: Add tests and documentation regarding context of exception in https://bugs.python.org/issue46563 #46559: base64 documentation is out-dated; altchars has to have the ex https://bugs.python.org/issue46559 #46556: pathlib.Path.__enter__() should emit DeprecationWarning https://bugs.python.org/issue46556 #46555: Unicode-mangled names refer inconsistently to constants https://bugs.python.org/issue46555 #46547: `pydoc.Helper` leaks several `for` loop variables https://bugs.python.org/issue46547 #46543: Add sys._getfunc https://bugs.python.org/issue46543 #46540: dylibs not loading properly from NFS mounts https://bugs.python.org/issue46540 #46535: Possible bug: pdb causes exception https://bugs.python.org/issue46535 #46534: Implementing PEP 673 (Self type) https://bugs.python.org/issue46534 #46525: datetime.timestamp() lose precision when the time is too large https://bugs.python.org/issue46525 #46518: SSL socket timeout not set after client connects via accept https://bugs.python.org/issue46518 #46512: Explicit or correct behavior of filecmp.cmpfiles w/ absolute p https://bugs.python.org/issue46512 Most recent 15 issues waiting for review (15) ============================================= #46565: Multiple modules leak `for` loop variables into module's names https://bugs.python.org/issue46565 #46564: Near zero-cost super().meth() calls via adaptive superinstruct https://bugs.python.org/issue46564 #46563: Add tests and documentation regarding context of exception in https://bugs.python.org/issue46563 #46561: Descriptor resolution should own arguments passed to descripto https://bugs.python.org/issue46561 #46560: ParamSpec: Typo in doc string https://bugs.python.org/issue46560 #46557: Logging captured warnings with a format string unnecessarily g https://bugs.python.org/issue46557 #46556: pathlib.Path.__enter__() should emit DeprecationWarning https://bugs.python.org/issue46556 #46554: Add append keyword argument to Path.write_text() and Path.writ https://bugs.python.org/issue46554 #46553: typing: get_type_hints on stringified lone ClassVar raises Typ https://bugs.python.org/issue46553 #46547: `pydoc.Helper` leaks several `for` loop variables https://bugs.python.org/issue46547 #46544: `textwrap.TextWrapper` leaks two intermediate vars into class https://bugs.python.org/issue46544 #46543: Add sys._getfunc https://bugs.python.org/issue46543 #46542: test_json and test_lib2to3 crash on s390x Fedora Clang 3.x bui https://bugs.python.org/issue46542 #46541: Replace _Py_IDENTIFIER() with statically initialized objects. https://bugs.python.org/issue46541 #46534: Implementing PEP 673 (Self type) https://bugs.python.org/issue46534 Top 10 most discussed issues (10) ================================= #45382: platform() is not able to detect windows 11 https://bugs.python.org/issue45382 17 msgs #46465: Regression caused by CALL_FUNCTION specialization for C functi https://bugs.python.org/issue46465 13 msgs #41370: PEP 585 and ForwardRef https://bugs.python.org/issue41370 9 msgs #46528: Simplify the VM's stack manipulations https://bugs.python.org/issue46528 9 msgs #46553: typing: get_type_hints on stringified lone ClassVar raises Typ https://bugs.python.org/issue46553 9 msgs #46425: Multiple test modules fail to run if invoked directly https://bugs.python.org/issue46425 8 msgs #46524: test_peg_generator takes 8 minutes on Windows https://bugs.python.org/issue46524 8 msgs #34680: asyncio event_loop fails when accessed from multiple threads https://bugs.python.org/issue34680 7 msgs #38472: setup.py: GCC detection is broken when cross-compiling with a https://bugs.python.org/issue38472 7 msgs #45476: [C API] PEP 674: Disallow using macros (Py_TYPE and Py_SIZE) a https://bugs.python.org/issue45476 7 msgs Issues closed (125) =================== #5884: pydoc to return error status code https://bugs.python.org/issue5884 closed by iritkatriel #7057: tkinter doc: more 3.x updates https://bugs.python.org/issue7057 closed by zach.ware #10943: abitype: Need better support to port C extension modules to th https://bugs.python.org/issue10943 closed by zach.ware #21987: TarFile.getmember on directory requires trailing slash iff ove https://bugs.python.org/issue21987 closed by serhiy.storchaka #22045: Python make issue https://bugs.python.org/issue22045 closed by iritkatriel #22211: Remove VMS specific code in expat.h & xmlrole.h https://bugs.python.org/issue22211 closed by iritkatriel #23289: concurrent.futures.Executor.map is not equivalent to map. https://bugs.python.org/issue23289 closed by iritkatriel #23556: [doc] Scope for raise without argument is different in Python https://bugs.python.org/issue23556 closed by iritkatriel #23769: valgrind reports leaks for test_zipimport https://bugs.python.org/issue23769 closed by vstinner #24046: Incomplete build on AIX https://bugs.python.org/issue24046 closed by iritkatriel #29844: Windows Python installers not installing DLL to System32/SysWO https://bugs.python.org/issue29844 closed by zach.ware #30053: Problems building with --enable-profiling on macOS using homeb https://bugs.python.org/issue30053 closed by iritkatriel #30424: make pydoc-topics fails https://bugs.python.org/issue30424 closed by iritkatriel #31011: Users (except from the one who installed) not able to see pyt https://bugs.python.org/issue31011 closed by iritkatriel #31305: 'pydoc -w import' report "no Python documentation found for 'i https://bugs.python.org/issue31305 closed by iritkatriel #31698: Add REQ_NAME to the node.h API https://bugs.python.org/issue31698 closed by Jelle Zijlstra #31991: Race condition in wait with timeout for multiprocessing.Event https://bugs.python.org/issue31991 closed by iritkatriel #32202: [ctypes] all long double tests fail on android-24-x86_64 https://bugs.python.org/issue32202 closed by iritkatriel #32203: [ctypes] test_struct_by_value fails on android-24-arm64 https://bugs.python.org/issue32203 closed by iritkatriel #32343: Leak Sanitizer reports memory leaks while building using ASAN https://bugs.python.org/issue32343 closed by iritkatriel #33822: IDLE subsection of What's New 3.8 https://bugs.python.org/issue33822 closed by terry.reedy #33936: OPENSSL_VERSION_1_1 never defined in _hashopenssl.c https://bugs.python.org/issue33936 closed by christian.heimes #33983: unify types for lib2to3.pytree.Base.children https://bugs.python.org/issue33983 closed by Jelle Zijlstra #35774: ASAN, memory leak https://bugs.python.org/issue35774 closed by vstinner #36257: configure with --with-icc --with-cxx-main=icpc https://bugs.python.org/issue36257 closed by iritkatriel #37080: Cannot compile Python3.7.3 on Alt-F (ARM) https://bugs.python.org/issue37080 closed by iritkatriel #37886: PyStructSequence_UnnamedField not exported https://bugs.python.org/issue37886 closed by iritkatriel #38039: Segfault when pickling dictionary with large pandas dataframes https://bugs.python.org/issue38039 closed by iritkatriel #38665: Crash when running SQLAlchemy with pyodbc https://bugs.python.org/issue38665 closed by iritkatriel #39708: final link stage in compile fails for 3.8.1 with missing CFLAG https://bugs.python.org/issue39708 closed by iritkatriel #39731: ModuleNotFoundError: No module named '_ctypes' https://bugs.python.org/issue39731 closed by iritkatriel #39854: f-strings with format specifiers have wrong col_offset https://bugs.python.org/issue39854 closed by iritkatriel #40170: [C API] Prepare PyTypeObject structure for a stable ABI: avoid https://bugs.python.org/issue40170 closed by vstinner #40250: unable to comment out r'\u' string with triple quote marks https://bugs.python.org/issue40250 closed by iritkatriel #40356: OverflowError: mktime argument out of range https://bugs.python.org/issue40356 closed by iritkatriel #41209: Scripts Folder is Empty https://bugs.python.org/issue41209 closed by iritkatriel #41403: Uncaught AttributeError in unittest.mock._get_target https://bugs.python.org/issue41403 closed by iritkatriel #41603: Compilation issue 3.8.5 with Redhat 7.8 and gcc version 4.8.5 https://bugs.python.org/issue41603 closed by iritkatriel #41682: [Windows] : Proactor test_sendfile_close_peer_in_the_middle_of https://bugs.python.org/issue41682 closed by vstinner #41906: logging.config.dictConfig does not work with callable filters https://bugs.python.org/issue41906 closed by vinay.sajip #42091: strftime returns empty string for -d, -I 3.8.3 https://bugs.python.org/issue42091 closed by iritkatriel #42585: Segmentation fault on Linux with multiprocess queue https://bugs.python.org/issue42585 closed by iritkatriel #42982: Update suggested number of iterations for pbkdf2_hmac() https://bugs.python.org/issue42982 closed by gregory.p.smith #43536: 3.9.2 --without-pymalloc --with-pydebug --with-valgrind: test https://bugs.python.org/issue43536 closed by vstinner #44642: Union of a type and the typing module function https://bugs.python.org/issue44642 closed by sobolevn #45578: Missing tests for the dis module https://bugs.python.org/issue45578 closed by taleinat #45868: xattr support missing in os module under cygwin https://bugs.python.org/issue45868 closed by iritkatriel #45945: compileall.py throws a traceback when using -j0 and thus 'make https://bugs.python.org/issue45945 closed by iritkatriel #46071: Graphlib documentation (edge direction) https://bugs.python.org/issue46071 closed by dam1784 #46091: IndendationError from multi-line indented statements https://bugs.python.org/issue46091 closed by pablogsal #46098: Missing test for multi-line errors in traceback https://bugs.python.org/issue46098 closed by iritkatriel #46124: Deprecation warning in zoneinfo module https://bugs.python.org/issue46124 closed by jaraco #46157: Typo in JSON documentation https://bugs.python.org/issue46157 closed by serhiy.storchaka #46204: Graphlib documentation (general cleanup) https://bugs.python.org/issue46204 closed by dam1784 #46214: Remove unused opcode ROT_FOUR https://bugs.python.org/issue46214 closed by iritkatriel #46249: [sqlite3] move set lastrowid out of the query loop and enable https://bugs.python.org/issue46249 closed by corona10 #46254: Better fitting type for iterating in the trace_init C function https://bugs.python.org/issue46254 closed by iritkatriel #46340: DeprecationWarning emitted when running asyncio tests https://bugs.python.org/issue46340 closed by kumaraditya303 #46378: Experiment with LLVM BOLT binary optimizer https://bugs.python.org/issue46378 closed by corona10 #46383: _zoneinfo module_free has invalid function signature https://bugs.python.org/issue46383 closed by christian.heimes #46396: `Concatenate` should not raise any semantic errors https://bugs.python.org/issue46396 closed by gvanrossum #46399: Addition of `mapping` attribute to dict views classes has inad https://bugs.python.org/issue46399 closed by AlexWaygood #46406: optimize int division https://bugs.python.org/issue46406 closed by mark.dickinson #46407: optimizing `1 << n` or `2 ** n` and modulo-only operations https://bugs.python.org/issue46407 closed by tim.peters #46416: Direct invocation of `Lib/test/test_typing.py` fails https://bugs.python.org/issue46416 closed by sobolevn #46417: Clear static types in Py_Finalize() for embedded Python https://bugs.python.org/issue46417 closed by vstinner #46419: Incomplete Comparison to C++ Methods https://bugs.python.org/issue46419 closed by gvanrossum #46420: Use NOTRACE_DISPATCH in specialized opcodes https://bugs.python.org/issue46420 closed by Dennis Sweeney #46422: Why do we need `dis.Positions`? https://bugs.python.org/issue46422 closed by sobolevn #46426: Improve tests for the dir_fd argument https://bugs.python.org/issue46426 closed by serhiy.storchaka #46429: Merge all deepfrozen files into one https://bugs.python.org/issue46429 closed by kumaraditya303 #46431: Trouble subclassing ExceptionGroup https://bugs.python.org/issue46431 closed by iritkatriel #46434: pdb help fails with AttributeError when using Windows embeddab https://bugs.python.org/issue46434 closed by steve.dower #46442: testExceptionCleanupNames doesn't test anything? https://bugs.python.org/issue46442 closed by eric.smith #46445: Multiple inheritance of TypedDict is not covered in `test_typi https://bugs.python.org/issue46445 closed by sobolevn #46449: Deep-freezed modules create inconsistency in sys.gettotalrefco https://bugs.python.org/issue46449 closed by vstinner #46452: Possible false detection of Windows LZMA library as a malware https://bugs.python.org/issue46452 closed by terry.reedy #46453: argparse subparser help text is not escaped before string form https://bugs.python.org/issue46453 closed by iritkatriel #46460: Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCE https://bugs.python.org/issue46460 closed by iritkatriel #46463: escape4chm.py script causing AttributeError in docs build https://bugs.python.org/issue46463 closed by steve.dower #46466: help function reads comments https://bugs.python.org/issue46466 closed by steven.daprano #46467: Rounding 5,50,500 behaves differently depending on preceding v https://bugs.python.org/issue46467 closed by steven.daprano #46468: http.server documentation missing default value for port https://bugs.python.org/issue46468 closed by asvetlov #46469: asyncio Generic classes should return GenericAlias in __class_ https://bugs.python.org/issue46469 closed by asvetlov #46470: `_remove_dups_flatten` in `typing.py` contains dead branch https://bugs.python.org/issue46470 closed by kj #46471: use global singletons for single byte bytes objects in deepfre https://bugs.python.org/issue46471 closed by erlendaasland #46473: importlib.metadata does not honor .egg-link files https://bugs.python.org/issue46473 closed by onlinespending #46474: Inefficient regular expression complexity in EntryPoint.patter https://bugs.python.org/issue46474 closed by jaraco #46476: Not all memory allocated by _Py_Quicken() is released at Pytho https://bugs.python.org/issue46476 closed by vstinner #46478: DirEntry.stat() of os.scandir() has no dir_fd parameter https://bugs.python.org/issue46478 closed by vstinner #46481: Implement vectorcall protocol for weakref_call() https://bugs.python.org/issue46481 closed by corona10 #46485: ~ PEG operator in the grammar https://bugs.python.org/issue46485 closed by pablogsal #46486: Rename DesciptorClassification => DescriptorClassification in https://bugs.python.org/issue46486 closed by rhettinger #46488: listsort.txt wrongly assumes you cannot calculate leading zero https://bugs.python.org/issue46488 closed by rhettinger #46491: typing: allow Annotated in outermost scope https://bugs.python.org/issue46491 closed by kj #46492: BrokenPipeError when piping to head (linux) https://bugs.python.org/issue46492 closed by royroy #46499: circular impoty https://bugs.python.org/issue46499 closed by iritkatriel #46502: Py_CompileString no longer allows to tell "incomplete input" f https://bugs.python.org/issue46502 closed by pablogsal #46503: assertion failure in Parser/string_parser.c https://bugs.python.org/issue46503 closed by eric.smith #46504: Faster code for trial quotient in x_divrem https://bugs.python.org/issue46504 closed by tim.peters #46505: Simplify exception handling code in py_compile https://bugs.python.org/issue46505 closed by iritkatriel #46510: Update Python2-style exception handling https://bugs.python.org/issue46510 closed by iritkatriel #46513: AC_C_CHAR_UNSIGNED from configure.ac confuses GCC 12+ by defin https://bugs.python.org/issue46513 closed by christian.heimes #46514: Pathlib Path.touch() seems to ignore groups write bit for mode https://bugs.python.org/issue46514 closed by ronaldoussoren #46515: SPAM https://bugs.python.org/issue46515 closed by mrabarnett #46516: Use existing unbound_local_error label in DELETE_FAST opcode https://bugs.python.org/issue46516 closed by kj #46517: Review exception handling in urllib https://bugs.python.org/issue46517 closed by iritkatriel #46519: test_typing failing on branch 3.10 https://bugs.python.org/issue46519 closed by kj #46527: enumerate no longer accepts iterable keyword argument https://bugs.python.org/issue46527 closed by corona10 #46529: Improve test coverage of `Union` and `Optional` repr() https://bugs.python.org/issue46529 closed by kj #46530: `'thread_time'` is missing from `test_get_clock_info` https://bugs.python.org/issue46530 closed by sobolevn #46531: Simplify exception handling in `doctest.py` https://bugs.python.org/issue46531 closed by sobolevn #46536: Better for loop https://bugs.python.org/issue46536 closed by eric.smith #46537: zipfile crash on windows, detected during pip install pyodbc- https://bugs.python.org/issue46537 closed by Martin-Forster #46538: [C API] Make the PyDescrObject structure opaque: PyDescr_NAME( https://bugs.python.org/issue46538 closed by vstinner #46539: typing: forward references don't understand special type forms https://bugs.python.org/issue46539 closed by AlexWaygood #46545: `textwrap.TextWrapper` leaks two intermediate vars into class https://bugs.python.org/issue46545 closed by sobolevn #46546: `importlib.metadata.DeprecatedList` leaks `method_name` variab https://bugs.python.org/issue46546 closed by sobolevn #46549: Error in list comprehension conditionals when used in class at https://bugs.python.org/issue46549 closed by Dennis Sweeney #46551: Provide number of workers option for fast PGO build time https://bugs.python.org/issue46551 closed by corona10 #46552: typing: get_type_hints can't handle stringified annotations wi https://bugs.python.org/issue46552 closed by gvanrossum #46558: Quadratic time internal base conversions https://bugs.python.org/issue46558 closed by tim.peters #46562: Add typeof or enum behavior in the typing module https://bugs.python.org/issue46562 closed by Jelle Zijlstra #1635741: Py_Finalize() doesn't clear all Python objects at exit https://bugs.python.org/issue1635741 closed by vstinner #1471934: Python libcrypt build problem on Solaris 8, 9, 10 and OpenSola https://bugs.python.org/issue1471934 closed by iritkatriel

1 0

Request to revert unittest and configparser incompatible changes in Python 3.11
by Victor Stinner Jan. 28, 2022

Jan. 28, 2022

Hi, My colleagues Tomáš Hrnčiar and Miro Hrončok made good progress on updating Python 3.10 to Python 3.11 in Fedora, but some specific Python 3.11 incompatible changes are causing more troubles than others: https://discuss.python.org/t/experience-with-python-3-11-in-fedora/12911 We propose to revert the following 2 changes in Python 3.11 and postpone them in a later Python version, once most projects will be compatible with these changes: * Removal of unittest aliases (bpo-45162): it broke 61 Fedora packages * Removals from configparser module (bpo-45173) - broke 28 Fedora packages -- We reported the issue to many affected projects, or when possible, we also wrote a pull request to make the code compatible (lot of those were made by others, e.g. Hugo van Kemenade, thank you!). The problem is that fixing a Fedora package requires multiple steps: * (1) Propose a pull request upstream * (2) Get the pull request merged upstream * (3) Wait for a new release upstream * (4) Update the Fedora package downstream, or backport the change in Fedora (only needed by Fedora) Identifying the Python incompatible changes causing most troubles took us a lot of time, but we did this work. Reverting the two Python 3.11 incompatible changes (causing most troubles) will save us "bug triage" time, to get more time on updating projects to Python 3.11 for the other remaining incompatible changes. We are not saying that these incompatible changes are bad, it's just a matter of getting most projects ready before Python 3.11 final will be released. -- By the way, before making a change known to be incompatible, it would be nice to run a code search on PyPI top 5000 projects to estimate how many projects will be broken, and try to update these projects *before* making the change. For example, only introduce an incompatible change into Python once less than 15 projects are affected. Getting a pull request merged is nice, but a release including the fix is way better for practical reasons: people use "pip install <project name>". -- Fedora work on Python 3.11 is public and tracked at: https://bugzilla.redhat.com/show_bug.cgi?id=PYTHON3.11 Click on "depends on" to see current issues: https://bugzilla.redhat.com/buglist.cgi?bug_id=2016048&bug_id_type=anddepen… Example of bz #2025600: mom fails to build with Python 3.11: AttributeError: module 'configparser' has no attribute 'SafeConfigParser'. Victor Stinner -- in the name of the Python Red Hat / Fedora maintenance team -- Night gathers, and now my watch begins. It shall not end until my death.

12 16

Python no longer leaks memory at exit
by Victor Stinner Jan. 28, 2022

Jan. 28, 2022

Hi, tl; dr Python no longer leaks memory at exit on the "python -c pass" command ;-) == Bug report == In 2007, the bpo-1635741 issue was reported on SourceForge: "Interpreter seems to leak references after finalization". This bug is 15 years old. It saw the bugs migration from SourceForge to Roundup (bugs.python.org) in 2007, the code migration from Subversion to Mercurial in 2009, and the second code migration from Mercurial to Git in 2016! Link to the main issue: https://bugs.python.org/issue1635741 Some people reported similar issues seen by WinCRT debug (bpo-6741), MSVC Debug C Runtime (bpo-32026), GCC AddressSanitzer (bpo-26888), or LeakSanitizer and Valgrind (bpo-21387). In general, "leaking memory" at Python exit doesn't matter, since the operating system releases all memory when a process completes. It matters when Python is embedded in an application. For example, Python can be used by a plugin which is unloaded and reloaded multiple times. It also matters for sub-interpreters. == Tedious task == In the last 3 years, many people helped fixing this old issue by converting static types to heap types, adding a module state to C extension modules, converting extensions to the multi-phase initialization API (PEP 489), fix many memory leaks, fix bugs, etc. When it became possible to cleanly unload an extension module (thanks for the multi-phase init), tests on sub-intepreters (which load and unload extension modules) showed many *old* reference leaks: all of them have been fixed! When a test is run in sub-interpreters, Python is able to detect leaks, whereas currently it doesn't check for memory leaks at Python exit. == Python 3.10 regressions == During the Python 3.10 development, we identified and fixed 3 major regressions caused by this work: * Converting static types to heap types make them mutable: the Py_TPFLAGS_IMMUTABLETYPE flag was added; 68 types use it in Python 3.11 (bpo-43908). * It became possible to create uninitialized objects by instanciating types (which should not be instanciated directly): the Py_TPFLAGS_DISALLOW_INSTANTIATION was added; 41 types use it in Python 3.11 (bpo-43916). For example, you cannot create an instance of type(sys.flags). * Heap types must implement the full GC protocol: Py_TPFLAGS_HAVE_GC flag, traverse and clear functions. Otherwise, the GC is unable to break reference cycles, whereas a type contains (multiple) strong references to itself (in the MRO tuple and in methods). All heap types have been fixed to fully implement the GC protocol (bpo-40217). == PEPs == The work relies on multiple PEPs: * PEP 489: Multi-phase extension module initialization * PEP 573: Module State Access from C Extension Methods * PEP 630: Isolating Extension Modules == Persons who helped fixing the issue == Incomplete list of people who helped to fix this issue: * Christian Heimes (modules: symtable, _hashlib, ,_random, grp, pwd, _queue, spwd, _struct, gc, _posixshmem, _posixsubprocess, select, resource) * Dong-hee Na (modules: _statistics, itertools, _heapq, _collections, _uuid, math, _stat, syslog, errno, fcntl, mmap, _dbm, _gdbm, _lzma, faulthandler, _bisect) * Eric Snow (PEP 573) * Erlend Egeberg Aasland (modules: _sqlite3, _sre, _multibytecodec) * Hai Shi (modules: _json, _codecs, _crypt, _contextvars, _abc, _bz2, _locale, audioop, _ctypes_test, _weakref, _locale) * Marcel Plch (PEP 573) * Martin von Löwis (PEP 3121) * Mohamed Koubaa (modules: sha256, multiprocessing, _winapi, _blake2, _sha3, _signal, _sha1, _sha512, _md5, zlib, _opcode, _overlapped, _curses_panel, termios, _sha256, scproxy, cmath, _lsprof, unicodedata) * Nick Coghlan (PEP 489, PEP 573) * Paulo Henrique Silva (modules: time, operator, _functools) * Petr Viktorin (PEP 489, PEP 573, PEP 630) * Stefan Behnel (PEP 489) * Victor Stinner (modules _string, mashal,_imp, _warnings, _thread) The work was scatted into many sub-issues, it was hard for me to track all persons who contributed, sorry about that! I only searched for "New changeset" in bpo-1635741. For me, it was a very pleasant collaborative work :-) Contributors wrote pull requests, I reviewed and merged them. Slowly, contributors started to review each others and shared some recipes for these tasks. == Close the very old bpo-1635741 issue == Today, in the main development branch, Python no longer leaks memory at exit for the simplest command: "python3 -c pass"! Using a Python debug build, you can check the "python -I -X showrefcount -c pass" command (if you get a negative reference count, see bpo-46449), or you can use a memory debugger like Valgrind. While the work is not 100% done, it's a great milestone (at least for me ;-)! 15 years after bpo-1635741 was reported, finally I can close it! Sadly, this bug will no see the bugs migration from Roundup (bugs.python.org) to GitHub ;-) There are still some static types which should be converted to heap types and some extensions which should be be ported to the multi-phase initialization API. But this work can be done in existing or new specific issues. Again, a *big* thanks to every single person who helped directly and indirectly on fixing this issue! Victor -- Night gathers, and now my watch begins. It shall not end until my death.

4 3

RFC on PEP 680 -- tomllib: Support for Parsing TOML in the Standard Library
by Shantanu Jain Jan. 28, 2022

Jan. 28, 2022

Hello all, Link to the PEP: https://www.python.org/dev/peps/pep-0680/ Please redirect discussion to: https://discuss.python.org/t/13040 PEP 680 proposes adding the tomllib module to the standard library for parsing TOML <https://toml.io>. The main motivation is to address bootstrapping problems with pyproject.toml based builds. Thank you to everyone who has already left comments, suggestions and reacts on the Discuss thread! We welcome any further suggestions before submitting to the Steering Council :-) Thanks, Shantanu Jain Taneli Hukkinen

1 0

Dropping AC_C_CHAR_UNSIGNED from configure.ac?
by Miro Hrončok Jan. 25, 2022

Jan. 25, 2022

Hello Pythonistas. In (development branch of) Fedora, we have juts upgraded to GCC 12. It seems that the presence of AC_C_CHAR_UNSIGNED in Python's autotools files (configure.ac?) is causing the __CHAR_UNSIGNED__ symbol to be defined in pyconfig.h and that breaks some other packages with GCC 12. The GCC maintainers told us it is a reserved symbol, see https://bugzilla.redhat.com/2043555 for details. It seems that using AC_C_CHAR_UNSIGNED is not recommended and also not required, but I must confess that I am pretty much horrified by autotools and I don't really know if we can get rid of AC_C_CHAR_UNSIGNED or not. I can test this in Fedora and I am quite sure it'll work, but I don't know the impact on all the other environments where CPython can be compiled. Is there anybody on this list who knows a reason we need to keep AC_C_CHAR_UNSIGNED around in 2022? Or is it safe to get rid of it? Thanks, -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 2

SC Acceptance: PEP 673 -- Self Type
by Petr Viktorin Jan. 24, 2022

Jan. 24, 2022

Hello Pradeep, James and Jelle, The Steering Council discussed PEP 673 -- Self Type, and unanimously decided to accept it. Congratulations! Please change the PEP status to Accepted, and merge the change to Python 3.11, at your convenience. Happy typing! - Petr (on behalf of the Python Steering Council)

1 0

The current state of typing PEPs
by Barry Warsaw Jan. 24, 2022

Jan. 24, 2022

Does PEP 563 or 649 satisfy static and dynamic typing needs? In the interest of full transparency, we want to let the Python community know that the Steering Council continues to discuss PEP 563 (Postponed Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations Using Descriptors). We have concluded that we lack enough detailed information to make a decision in favor of either PEP. As we see it, adopting either PEP 563 or PEP 649 as the default would be insufficient. They will not fully resolve the existing problems these PEPs intend to fix, will break some existing code, and likely don’t address all the use cases and requirements across the static and dynamic typing constituents. We are also uncertain as to the best migration path from the current state of affairs. Defer decision on PEP 563 and 649 in 3.11 As such, at this time, the only reasonable path forward that the SC sees is to defer the decision in Python 3.11 again, essentially keeping the 3.10 status quo. We know that this is far from ideal, but it’s also the safest path since we can’t clearly make the situation better, and we don’t have confidence that either PEP solves the problems once and for all. Pragmatically, we don’t want to make the situation worse, and we really don’t want to find ourselves back here again in a couple of releases because we overlooked an important requirement for a set of users. Calling for help from static and dynamic typing users This is also a call to you, the folks who both care deeply about typing in Python, and have a solid understanding of the subject matter (or willingness to learn) to help us! There are use cases we don’t know about, and unclear requirements from both the static and dynamic typing users. We need help in defining those requirements clearly, in uncovering the use cases we’re not aware of, and most importantly, being an interface to typing enthusiasts to help build consensus with either of the proposed PEPs, or some other solution that doesn’t yet have a PEP (such as suggestions made by Łukasz in his blog post here: https://lukasz.langa.pl/61df599c-d9d8-4938-868b-36b67fdb4448/). Volunteer to be a typing PEP shepherd We’re looking for someone(s) who can be neutral as to the solution, knowledgeable enough to understand all sides of the problem, willing to serve as not quite a PEP author, not a PEP Delegate, but more than a PEP shepherd. If you are that person, please let us know! Cheers, -Barry (on behalf of the Python Steering Council)

29 81

Summary of Python tracker Issues
by Python tracker Jan. 21, 2022

Jan. 21, 2022

ACTIVITY SUMMARY (2022-01-14 - 2022-01-21) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7175 (-27) closed 51014 (+108) total 58189 (+81) Open issues with patches: 2878 Issues opened (59) ================== #24711: Document getpass.getpass behavior on ^C https://bugs.python.org/issue24711 reopened by iritkatriel #44133: Some C-API symbols (e.g. Py_FrozenMain) are not always exporte https://bugs.python.org/issue44133 reopened by vstinner #45522: Allow to build Python without freelists https://bugs.python.org/issue45522 reopened by vstinner #46035: mimetypes.guess_type returns deprecated mimetype application/x https://bugs.python.org/issue46035 reopened by iritkatriel #46133: Feature request: allow mechanism for creator of exec-generated https://bugs.python.org/issue46133 reopened by posita #46381: Improve documentation of CFLAGS_NODIST, LDFLAGS_NODIST https://bugs.python.org/issue46381 opened by matthiaskoeppe #46382: dataclass(slots=True) does not account for slots in base class https://bugs.python.org/issue46382 opened by ariebovenberg #46383: _zoneinfo module_free has invalid function signature https://bugs.python.org/issue46383 opened by christian.heimes #46384: Request: make lzma._(encode|decode)_filter_properties public https://bugs.python.org/issue46384 opened by miurahr #46389: 3.11: unused generator comprehensions cause f_lineno==None https://bugs.python.org/issue46389 opened by nedbat #46390: Multiple test failures on Alpine 3.15 / musl-1.2.2-r7 https://bugs.python.org/issue46390 opened by christian.heimes #46391: Library multiprocess leaks named resources. https://bugs.python.org/issue46391 opened by milestonejxd #46392: MessageIDHeader is too strict for message-id https://bugs.python.org/issue46392 opened by bpoaugust #46393: Generate frozenset constants when explicitly appropriate https://bugs.python.org/issue46393 opened by terry.reedy #46396: Invalid usage of `Concatenate` is not covered at all https://bugs.python.org/issue46396 opened by sobolevn #46397: urllib.parse.urlencode() return wrong character https://bugs.python.org/issue46397 opened by scratch #46398: posixshmem module shm_rename freebsd support. https://bugs.python.org/issue46398 opened by devnexen #46399: Addition of `mapping` attribute to dict views classes has inad https://bugs.python.org/issue46399 opened by AlexWaygood #46400: Please update bundled libexpat to 2.4.3 with security fixes https://bugs.python.org/issue46400 opened by sping #46404: 3.11a4: a small attrs regression https://bugs.python.org/issue46404 opened by tinchester #46406: optimize int division https://bugs.python.org/issue46406 opened by gregory.p.smith #46407: optimizing `1 << n` or `2 ** n` and modulo-only operations https://bugs.python.org/issue46407 opened by February291948 #46410: TypeError when parsing regexp with unicode named character seq https://bugs.python.org/issue46410 opened by jirkamarsik #46414: Add typing.reveal_type https://bugs.python.org/issue46414 opened by Jelle Zijlstra #46416: Direct invocation of `Lib/test/test_typing.py` fails https://bugs.python.org/issue46416 opened by sobolevn #46417: Clear static types in Py_Finalize() for embedded Python https://bugs.python.org/issue46417 opened by vstinner #46419: Incomplete Comparison to C++ Methods https://bugs.python.org/issue46419 opened by jharmse #46420: Use NOTRACE_DISPATCH in specialized opcodes https://bugs.python.org/issue46420 opened by Dennis Sweeney #46421: unittest ValueError when invoking as module https://bugs.python.org/issue46421 opened by BaderSZ #46422: Why do we need `dis.Positions`? https://bugs.python.org/issue46422 opened by sobolevn #46425: Multiple test modules fail to run if invoked directly https://bugs.python.org/issue46425 opened by sobolevn #46426: Improve tests for the dir_fd argument https://bugs.python.org/issue46426 opened by serhiy.storchaka #46429: Merge all deepfrozen files into one https://bugs.python.org/issue46429 opened by kumaraditya303 #46430: intern strings in deepfrozen modules https://bugs.python.org/issue46430 opened by kumaraditya303 #46431: Trouble subclassing ExceptionGroup https://bugs.python.org/issue46431 opened by petr.viktorin #46432: AMD64 FreeBSD Shared 3.x buildbot fails to build: error: error https://bugs.python.org/issue46432 opened by vstinner #46433: _PyType_GetModuleByDef optimization is incorrect https://bugs.python.org/issue46433 opened by petr.viktorin #46434: pdb help fails with AttributeError when using Windows embeddab https://bugs.python.org/issue46434 opened by sparrowt #46435: MessageID parser can crash with IndexError: string index out o https://bugs.python.org/issue46435 opened by bpoaugust #46436: Pass the -d/--directory command-line option to http.server.CGI https://bugs.python.org/issue46436 opened by maggyero #46438: Static linking _decimal module breaks test_freeze https://bugs.python.org/issue46438 opened by steve.dower #46439: Clarify urllib.request.add_header documentation https://bugs.python.org/issue46439 opened by epicfaace #46440: ArgumentParser.parse_args exits on missing required argument w https://bugs.python.org/issue46440 opened by trilader #46442: testExceptionCleanupNames doesn't test anything? https://bugs.python.org/issue46442 opened by yellowdusk1590 #46445: Multiple inheritance of TypedDict is not covered in `test_typi https://bugs.python.org/issue46445 opened by sobolevn #46446: OpenBSD not MULTIARCH https://bugs.python.org/issue46446 opened by kmosiejczuk #46447: datetime.isoformat() documentation does not point to the risk https://bugs.python.org/issue46447 opened by jeanCarloMachado #46448: TypedDict inspect.signature error https://bugs.python.org/issue46448 opened by jhwang #46449: Deep-freezed modules create inconsistency in sys.gettotalrefco https://bugs.python.org/issue46449 opened by vstinner #46451: Possibly bad interaction with tracing and cython? https://bugs.python.org/issue46451 opened by seberg #46452: Possible false detection of Windows LZMA library as a malware https://bugs.python.org/issue46452 opened by thfetoile #46453: argparse subparser help text is not escaped before string form https://bugs.python.org/issue46453 opened by pbsds #46455: Deprecate / remove os.name=java https://bugs.python.org/issue46455 opened by xmorel #46456: Add mime type "image/avif" https://bugs.python.org/issue46456 opened by wangjiahua #46457: test_unittest: TestAsyncCase.test_debug_cleanup_same_loop() ha https://bugs.python.org/issue46457 opened by vstinner #46458: Optimise try-except code generation for the happy path https://bugs.python.org/issue46458 opened by iritkatriel #46459: Installing python to external drive (USB) using python install https://bugs.python.org/issue46459 opened by oMqngo #46460: Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCE https://bugs.python.org/issue46460 opened by sobolevn #46461: Kodi crashing https://bugs.python.org/issue46461 opened by mrpg Most recent 15 issues with no replies (15) ========================================== #46461: Kodi crashing https://bugs.python.org/issue46461 #46460: Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCE https://bugs.python.org/issue46460 #46457: test_unittest: TestAsyncCase.test_debug_cleanup_same_loop() ha https://bugs.python.org/issue46457 #46456: Add mime type "image/avif" https://bugs.python.org/issue46456 #46453: argparse subparser help text is not escaped before string form https://bugs.python.org/issue46453 #46447: datetime.isoformat() documentation does not point to the risk https://bugs.python.org/issue46447 #46446: OpenBSD not MULTIARCH https://bugs.python.org/issue46446 #46445: Multiple inheritance of TypedDict is not covered in `test_typi https://bugs.python.org/issue46445 #46440: ArgumentParser.parse_args exits on missing required argument w https://bugs.python.org/issue46440 #46439: Clarify urllib.request.add_header documentation https://bugs.python.org/issue46439 #46438: Static linking _decimal module breaks test_freeze https://bugs.python.org/issue46438 #46436: Pass the -d/--directory command-line option to http.server.CGI https://bugs.python.org/issue46436 #46435: MessageID parser can crash with IndexError: string index out o https://bugs.python.org/issue46435 #46420: Use NOTRACE_DISPATCH in specialized opcodes https://bugs.python.org/issue46420 #46419: Incomplete Comparison to C++ Methods https://bugs.python.org/issue46419 Most recent 15 issues waiting for review (15) ============================================= #46460: Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCE https://bugs.python.org/issue46460 #46458: Optimise try-except code generation for the happy path https://bugs.python.org/issue46458 #46446: OpenBSD not MULTIARCH https://bugs.python.org/issue46446 #46445: Multiple inheritance of TypedDict is not covered in `test_typi https://bugs.python.org/issue46445 #46442: testExceptionCleanupNames doesn't test anything? https://bugs.python.org/issue46442 #46439: Clarify urllib.request.add_header documentation https://bugs.python.org/issue46439 #46434: pdb help fails with AttributeError when using Windows embeddab https://bugs.python.org/issue46434 #46433: _PyType_GetModuleByDef optimization is incorrect https://bugs.python.org/issue46433 #46430: intern strings in deepfrozen modules https://bugs.python.org/issue46430 #46429: Merge all deepfrozen files into one https://bugs.python.org/issue46429 #46426: Improve tests for the dir_fd argument https://bugs.python.org/issue46426 #46425: Multiple test modules fail to run if invoked directly https://bugs.python.org/issue46425 #46422: Why do we need `dis.Positions`? https://bugs.python.org/issue46422 #46421: unittest ValueError when invoking as module https://bugs.python.org/issue46421 #46420: Use NOTRACE_DISPATCH in specialized opcodes https://bugs.python.org/issue46420 Top 10 most discussed issues (10) ================================= #46417: Clear static types in Py_Finalize() for embedded Python https://bugs.python.org/issue46417 20 msgs #46399: Addition of `mapping` attribute to dict views classes has inad https://bugs.python.org/issue46399 19 msgs #46376: PyMapping_Check returns 1 for list https://bugs.python.org/issue46376 14 msgs #40066: Enum: modify __repr__, __str__; update docs https://bugs.python.org/issue40066 11 msgs #46071: Graphlib documentation (edge direction) https://bugs.python.org/issue46071 11 msgs #46393: Generate frozenset constants when explicitly appropriate https://bugs.python.org/issue46393 9 msgs #46425: Multiple test modules fail to run if invoked directly https://bugs.python.org/issue46425 9 msgs #46382: dataclass(slots=True) does not account for slots in base class https://bugs.python.org/issue46382 8 msgs #46434: pdb help fails with AttributeError when using Windows embeddab https://bugs.python.org/issue46434 8 msgs #46392: MessageIDHeader is too strict for message-id https://bugs.python.org/issue46392 7 msgs Issues closed (101) =================== #13649: termios.ICANON is not documented https://bugs.python.org/issue13649 closed by iritkatriel #20281: [doc] time.strftime %z format specifier is the same as %Z https://bugs.python.org/issue20281 closed by iritkatriel #23162: collections.abc sequences don't check identity before equality https://bugs.python.org/issue23162 closed by iritkatriel #23557: Misc/SpecialBuilds.txt contains outdated information about PYM https://bugs.python.org/issue23557 closed by iritkatriel #25652: collections.UserString.__rmod__() raises NameError https://bugs.python.org/issue25652 closed by iritkatriel #27302: csv.Sniffer guesses wrong when unquoted fields contain quotes https://bugs.python.org/issue27302 closed by iritkatriel #28169: shift exponent overflow https://bugs.python.org/issue28169 closed by iritkatriel #28186: Autogenerated tabs / trailing whitespace https://bugs.python.org/issue28186 closed by iritkatriel #28206: signal.Signals not documented https://bugs.python.org/issue28206 closed by iritkatriel #28477: Add optional user argument to pathlib.Path.home() https://bugs.python.org/issue28477 closed by iritkatriel #29541: Python3 error while building on Alt-F https://bugs.python.org/issue29541 closed by iritkatriel #29860: smtplib.py doesn't capitalize EHLO. https://bugs.python.org/issue29860 closed by iritkatriel #29964: [doc] %z directive has no effect on the output of time.strptim https://bugs.python.org/issue29964 closed by iritkatriel #30268: Make mimetypes.guess_type accept path-like objects https://bugs.python.org/issue30268 closed by iritkatriel #30492: 'make clinic' does not work for out of tree builds / clinic.py https://bugs.python.org/issue30492 closed by iritkatriel #30512: CAN Socket support for NetBSD https://bugs.python.org/issue30512 closed by serhiy.storchaka #30819: Linking with 'ld -b' fails with 64-bit using Itanium HP compil https://bugs.python.org/issue30819 closed by iritkatriel #31876: python363.chm includes gibberish https://bugs.python.org/issue31876 closed by iritkatriel #32976: linux/random.h present but cannot be compiled https://bugs.python.org/issue32976 closed by iritkatriel #33286: Conflict between tqdm and multiprocessing on windows https://bugs.python.org/issue33286 closed by iritkatriel #33994: python build egg fails with error while compiling test cases https://bugs.python.org/issue33994 closed by iritkatriel #34672: '%Z' strftime specifier never works with musl https://bugs.python.org/issue34672 closed by iritkatriel #34924: inspect.signature isn't aware that types.MethodType can wrap a https://bugs.python.org/issue34924 closed by iritkatriel #35107: untokenize() fails on tokenize output when a newline is missin https://bugs.python.org/issue35107 closed by iritkatriel #35974: os.DirEntry.inode() returns invalid value within Docker contai https://bugs.python.org/issue35974 closed by iritkatriel #36369: test_weakref super slow on RPi Zero https://bugs.python.org/issue36369 closed by iritkatriel #37084: _ctypes not failing, can't find reason https://bugs.python.org/issue37084 closed by iritkatriel #37310: Solaris 11.3 w/ Studio 12.6 test_ctypes fail https://bugs.python.org/issue37310 closed by iritkatriel #37311: Solaris 11.3 w/ Studio 12.6 test_support fail https://bugs.python.org/issue37311 closed by iritkatriel #37776: [subinterpreters] Test Py_Finalize() from a subinterpreter https://bugs.python.org/issue37776 closed by vstinner #37800: Clean up the documentation on module attributes https://bugs.python.org/issue37800 closed by iritkatriel #38550: hashlib missing algorithms https://bugs.python.org/issue38550 closed by iritkatriel #38963: multiprocessing processes seem to "bleed" user information (GI https://bugs.python.org/issue38963 closed by iritkatriel #39147: using zipfile with root privilege shows FileNotFoundError https://bugs.python.org/issue39147 closed by iritkatriel #39276: type() cause segmentation fault in callback function called f https://bugs.python.org/issue39276 closed by iritkatriel #39663: IDLE: Add additional tests for pyparse https://bugs.python.org/issue39663 closed by iritkatriel #39819: NULL pointer crash in Modules/_cursesmodule.c in PyInit__curse https://bugs.python.org/issue39819 closed by iritkatriel #39952: Using VS2019 to automatically build Python3 and it failed to b https://bugs.python.org/issue39952 closed by iritkatriel #40116: Regression in memory use of shared key dictionaries for "compa https://bugs.python.org/issue40116 closed by Mark.Shannon #40729: Update the list of auto-generated files in .gitattributes https://bugs.python.org/issue40729 closed by BTaskaya #41034: test_builtin: PtyTests fail when run twice https://bugs.python.org/issue41034 closed by vstinner #41857: Document timeout arguments to poll() in select module https://bugs.python.org/issue41857 closed by taleinat #42028: Regression in mimetypes for image/bmp https://bugs.python.org/issue42028 closed by iritkatriel #42199: bytecode_helper assertNotInBytecode fails too eagerly https://bugs.python.org/issue42199 closed by iritkatriel #42548: debugger stops at breakpoint of `pass` that is not actually re https://bugs.python.org/issue42548 closed by iritkatriel #42712: Segmentation fault in running ast.literal_eval() with large ex https://bugs.python.org/issue42712 closed by iritkatriel #43430: Exception raised when attempting to create Enum via functional https://bugs.python.org/issue43430 closed by iritkatriel #43476: Enabling access to showsyntaxerror for IDLE's shell https://bugs.python.org/issue43476 closed by terry.reedy #43869: Fix documentation of epoch/time.time https://bugs.python.org/issue43869 closed by vstinner #43975: Incorrect MIME type returned for .js files Windows 10. https://bugs.python.org/issue43975 closed by iritkatriel #44024: Improve the TypeError message for non-string second arguments https://bugs.python.org/issue44024 closed by serhiy.storchaka #44901: Info about used pickle protocol used by multiprocessing.Queue https://bugs.python.org/issue44901 closed by iritkatriel #44934: Windows installer: Append Python to PATH instead of prepending https://bugs.python.org/issue44934 closed by steve.dower #45113: [subinterpreters][C API] Add a new function to create PyStruct https://bugs.python.org/issue45113 closed by shihai1991 #45535: Enum's dir() does not contain inherited members https://bugs.python.org/issue45535 closed by ethan.furman #45554: multiprocessing exitcode is insufficiently documented https://bugs.python.org/issue45554 closed by pitrou #45680: Documentation on `GenericAlias` objects and `__class_getitem__ https://bugs.python.org/issue45680 closed by kj #45834: Move runtime except: check to the parser https://bugs.python.org/issue45834 closed by iritkatriel #45877: Inconsistency in minimal supported version of Microsoft Visual https://bugs.python.org/issue45877 closed by arhadthedev #45947: Place dict (and values) pointers at a fixed (negative) offset https://bugs.python.org/issue45947 closed by Mark.Shannon #46017: Tutorial incorrectly refers to skits rather than sketches. https://bugs.python.org/issue46017 closed by iritkatriel #46038: Mark /configure file as generated in .gitattributes https://bugs.python.org/issue46038 closed by arhadthedev #46080: argparse.BooleanOptionalAction with default=argparse.SUPPRESS https://bugs.python.org/issue46080 closed by taleinat #46086: Add ratio_min() function to the difflib library https://bugs.python.org/issue46086 closed by taleinat #46108: Enum repr() incorrect when mixed with non-__new__ data types https://bugs.python.org/issue46108 closed by ethan.furman #46242: Improve error message when attempting to extend an enum with ` https://bugs.python.org/issue46242 closed by AlexWaygood #46255: Remove unnecessary check in _IOBase._check*() methods https://bugs.python.org/issue46255 closed by malin #46258: Minor algorithmic improvements for math.isqrt https://bugs.python.org/issue46258 closed by mark.dickinson #46316: Optimize pathlib.Path.iterdir() https://bugs.python.org/issue46316 closed by zach.ware #46339: PEG parser segfault from ast.literal_eval https://bugs.python.org/issue46339 closed by pablogsal #46361: Small ints aren't always cached properly https://bugs.python.org/issue46361 closed by mark.dickinson #46363: Two typos in versions 3.7 document translation of zh_CN https://bugs.python.org/issue46363 closed by iritkatriel #46379: itertools.product reference implementation creates temporaries https://bugs.python.org/issue46379 closed by rhettinger #46380: `test_functools.TestLRU` must not use `functools` module dire https://bugs.python.org/issue46380 closed by rhettinger #46385: Remove parenthetical symbols for readability and nlp https://bugs.python.org/issue46385 closed by Dennis Sweeney #46386: Improve `test_typing::test_immutability_by_copy_and_pickle` https://bugs.python.org/issue46386 closed by corona10 #46387: `test_field_descriptor` in `test_collections` should test all https://bugs.python.org/issue46387 closed by rhettinger #46388: Improve test coverage of functools.total_ordering https://bugs.python.org/issue46388 closed by rhettinger #46394: setuptools' _distutils_hack breaks Python build https://bugs.python.org/issue46394 closed by christian.heimes #46395: `Concatenate` is not covered to be a return type in `test_typi https://bugs.python.org/issue46395 closed by sobolevn #46401: Setuptools 60.0 breaks building from source on Debian 11: No m https://bugs.python.org/issue46401 closed by i-Ching #46402: Enhance sqlite3 to avoid implicit creation? https://bugs.python.org/issue46402 closed by erlendaasland #46403: Unhelpful SyntaxError https://bugs.python.org/issue46403 closed by terry.reedy #46405: Warning compiling main on Windows https://bugs.python.org/issue46405 closed by terry.reedy #46408: signal module wrongly relies on small int singletons https://bugs.python.org/issue46408 closed by christian.heimes #46409: Add a new bytecode instruction to create generators https://bugs.python.org/issue46409 closed by Mark.Shannon #46411: modernize exception handling in tests https://bugs.python.org/issue46411 closed by iritkatriel #46412: PyQT6 projects crashes with python 3.10 https://bugs.python.org/issue46412 closed by ronaldoussoren #46413: `__or__` is not covered for `_GenericAlias` and `_SpecialGener https://bugs.python.org/issue46413 closed by kj #46415: ipaddress.ip_{address,network,interface} raises TypeError inst https://bugs.python.org/issue46415 closed by Jelle Zijlstra #46418: Simplify `MODULE` variable in `test_enum.py` https://bugs.python.org/issue46418 closed by sobolevn #46423: CLI: Addition assignment for tuples https://bugs.python.org/issue46423 closed by mark.dickinson #46424: `typing.Annotated` one type argument usage is not covered in t https://bugs.python.org/issue46424 closed by sobolevn #46427: Correct MSBuild's configuration for _freeze_module.exe https://bugs.python.org/issue46427 closed by steve.dower #46428: Cython Build: '_PyErr_StackItem�� has no member named ��exc_ https://bugs.python.org/issue46428 closed by YoSTEALTH #46437: Non-required `hasattr` checks in `test_typing` https://bugs.python.org/issue46437 closed by AlexWaygood #46441: Caret points to wrong line on 'return yield 42' in REPL https://bugs.python.org/issue46441 closed by pablogsal #46443: Deepfreeze use preallocated small ints https://bugs.python.org/issue46443 closed by kumaraditya303 #46444: Wrong value of pi for larger values using math.cos() function https://bugs.python.org/issue46444 closed by serhiy.storchaka #46450: namedtuple leaks data between instances when field's default v https://bugs.python.org/issue46450 closed by eric.smith #46454: '0 -> /dev/null' is lost https://bugs.python.org/issue46454 closed by vstinner

1 0

Discussion about crash tolerance feature for gdbm module
by Dong-hee Na Jan. 21, 2022

Jan. 21, 2022

Hi folks, From gdbm 1.21, gdbm supports the crash tolerance feature. see: https://www.gnu.org.ua/software/gdbm/manual/Crash-Tolerance.html I would like to introduce this feature since python standard library is the only gdbm binding library that is available for end-users. And this is also effort for end-users to provide the latest new features. If this feature is added following methods will be added. - 'x' flag will be added for extended gdbm format. - gdbm.gdbm_failure_atomic('path1', 'path2') API will be added for snapshot paths. - gdbm.gdbm_latest_snapshot('path1', 'path2') API will be added for getting latest valid snapshot path. The above APIs will be used for people who need to recover their gdbm file if disaster situations happen. However, this feature is not yet decided to land to CPython. and we have already discussed this issue at https://bugs.python.org/issue45452 but I would like to hear other devs opinions. I cc the original authors of this feature and Serhiy who thankfully already participated in this discussion too :) Warm Regards, Dong-hee

2 6

Adding an `inspect` function to return the signature of a type?
by Steven Troxler Jan. 21, 2022

Jan. 21, 2022

I've been wondering whether it would make sense to have a function in `inspect` that returns the signature of a type, rather than the signature of a specific callable object. I'm not attached to any name for such a function, two ideas are `inspect.signature_of` or `inspect.signature_of_type`. ## Context: the behavior of `inspect.signature` The existing `inspect.signature` function specified in PEP 362 [0] returns the call signature of a particular object, in other words given ``` class Dog: def __init__(self, color: str): self.color = color def __call__(self, command: str): print(command) ``` we will get ``` >>> import inspect >>> inspect.signature(Dog) <Signature (color: str)> >>> inspect.signature(Dog("brown")) <Signature (command: str)> ``` ## Outline of the idea In some cases it might be nice to be able to access the signature of an instance of `Dog` given the type `Dog`, without actually needing to instantiate it. Moreover, there are some cases where it isn't even possible to get a callable object, but it still might be nice to have access to an `inspect.Signature` object: - protocols cannot be instantiated. This is particularly relevant to getting Signatures because of callback protocols [1] - typing.Callable types describe a signature, but again can't be instantiated If if we had a function to operate on types, it would treat normal classes as if calling `inspect.signature` on the `__call__` method of an instance. For callable types we'd want to do something similar, although this brings up some issues: - the existing Signature requires parameter names, but a `typing.Callable` doesn't have any. - One option would be to invent names `__0`, `__1,` etc. Otherwise we'd have to make the name optional. - it's not obvious how to support PEP 612 ParamSpecs, and even less obvious how to support PEP 646 in full generality I have some code snippets running through how simple cases might work, as well as the edge cases I've thought of thus far: https://gist.github.com/stroxler/4760cbc5e49df2295cd0b524328b1c73 I'm undecided about whether the edge cases are an indication that this idea isn't worth considering. It's worth noting that most of the edge cases involve PEP 612 / 646 generics which are pretty rare. ## Why did I start thinking about this? In discussions of PEP 677 [2], a few folks suggested using `inspect.Signature` as inspiration for the runtime API (which we did). This got me thinking that it might be nice for code that uses runtime types, regardless of what we decide about PEP 677, to have an API that makes accessing signature information from a type easier. I don't personally have much need for this, but - when I chatted with Carl Meyer from the static python team at Meta he thought it could be handy - I could imagine it being useful for tools like ML frameworks, which I've worked on in the past, that interpret type annotations to auto-generate logic -- [0] PEP 362 Function Signatures https://www.python.org/dev/peps/pep-0362/ [1] Callback protocols https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols [2] PEP 677 Callable Type Syntax https://www.python.org/dev/peps/pep-0677/

1 0

RFC on PEP 673: Self Type
by Pradeep Kumar Srinivasan Jan. 20, 2022

Jan. 20, 2022

This PEP [1] introduces a simple and intuitive way to annotate methods and classmethods that return an instance of their class. Such methods and classmethods occur quite frequently, but the existing way to annotate them correctly is quite arcane and error-prone. The PEP introduces a special type `Self` to represent the type of the `self` parameter, similar to the `this` type in TypeScript and the `Self` type in Rust. We have implementations for mypy and pyright. The PEP does not affect CPython directly except for the addition of one special form (Self) to typing.py [2]. Since we have reached consensus on the PEP in typing-sig [3], we wanted to get your comments and suggestions before submitting to the Steering Council. Thanks, Pradeep Kumar Srinivasan James Hilton-Balfe [1]: https://www.python.org/dev/peps/pep-0673/ [2]: Adding `Self` to typing_extensions.py: https://github.com/python/typing/pull/933 [3]: See the comments from typing-sig members on the Google doc: https://docs.google.com/document/d/1ujuSMXDmSIOJpiZyV7mvBEC8P-y55AgSzXcvhrZ…

4 4

3.11 enhanced error location - can it be smarter?
by Mats Wichmann Jan. 20, 2022

Jan. 20, 2022

"When printing tracebacks, the interpreter will now point to the exact expression that caused the error instead of just the line." I get the motivation for better error messages, but there are scenarios where you can't provide useful new information. I've been lax in getting to testing the project I work on with 3.11, but here's some spew from one testcase (this is an intentionally triggered error - the testcase is testing predictable behavior in the face of this error, 3.11 didn't *cause* this, as can be fairly easily seen): Traceback (most recent call last): File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1403, in main _exec_main(parser, values) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1366, in _exec_main _main(parser) ^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1034, in _main SCons.Script._SConscript._SConscript(fs, script) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\SConscript.py", line 285, in _SConscript exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\AppData\Local\Temp\testcmd.16164.fpv9xt5b\SConstruct", line 4, in <module> raise InternalError('error inside') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ SCons.Errors.InternalError: error inside The grump here is all those pointy hats added absolutely zero new information, because in each case the entire line is underlined. So the "benefit" of the change is the traceback went from 12 lines to 17. In many cases tracebacks are much longer and adding lots of not-useful lines is not a great thing. A thought - how about omitting the underline line if the to-be-underlined part would be the whole line?

10 16

Restated typing issue for class fields.
by Vincent Risi Jan. 19, 2022

Jan. 19, 2022

I have written an xsd parser to generate python classes. I also have written a utility to read in an xml file to populate the python classes. As well as a utility to write xml from the python classes thus generated. For example the code below that I include here shows a small sample of the generated code. The __annotations__ for these are very easy to use by the utility code for doing their job. Using the typing list[x] and tuple[a,b] do not work as well as - [x] instead of list[x] - (a, b) instead of tuple(a, b) I seem to feel list[x] and tuple[a,b] are far more non Python, requiring having to parse the string they return in a complicated way. Looping through the annotations to get the field types it's easy to test for list or tuple and then get the field type for the list or tuple and the usage for the tuple. I have been using Python from version 1.6 and have been involved in writing and generating tons of lines of Python. ATTRIB, PSEUDO, ASIS = range(3) class RecordType: isNew: str column: (str,ATTRIB) whizz: (str,PSEUDO) def __init__(self): self.isNew = '' self.whizz = '' self.column = '' class MsgTableType: record: [RecordType] def __init__(self): self.record = [] rt = RecordType() for annote in rt.__annotations__: print (annote, type(rt.__annotations__[annote])) mtt = MsgTableType() for annote in mtt.__annotations__: print (annote, type(mtt.__annotations__[annote]))

2 1

SC Acceptance: PEP 646 -- Variadic Generics
by Barry Warsaw Jan. 19, 2022

Jan. 19, 2022

Hello Mark, Matthew, Pradeep, Vincent, and Guido, The Python Steering Council discussed the latest version of PEP 646 (Variadic Generics) at our last meeting, and have unanimously decided to accept the PEP. Congratulations! We want to specifically mention that we appreciate the way you called out the Python grammar changes required by the typing features you proposed. As we’ve said before, the Steering Council strongly believes that the typing language and the “general” Python programming language should remain aligned, so the implications of syntax change proposed in the PEP for typing needed to be addressed for non-typed Python as well. The PEP explains this change well, and does a good job of justifying the semantics and usefulness of the change for non-type related purposes. Please feel free to change the PEP status to Accepted, and to merge your changes to Python 3.11 at your convenience. With our appreciation, -Barry (on behalf of the Python Steering Council)

3 2

Minor inconvenience: f-string not recognized as docstring
by Antoine Pitrou Jan. 19, 2022

Jan. 19, 2022

Hello, Currently, a f-string is not recognized as a docstring: >>> class C: f"foo" >>> C.__doc__ >>> This means you need to use a (admittedly easy) workaround: >>> class C: __doc__ = f"foo" >>> C.__doc__ 'foo' Shouldn't the former be allowed for convenience? Regards Antoine.

9 11

PEP 646 (Variadic Generics): final call for comments
by Matthew Rahtz Jan. 18, 2022

Jan. 18, 2022

Hi everyone, We've got to the stage now with PEP 646 that we're feeling pretty happy with it. So far though we've mainly been workshopping it in typing-sig, so as PEP 1 requires we're asking for some feedback here too before submitting it to the steering council. If you have time over the next couple of weeks, please take a look at the current draft and let us know your thoughts: https://www.python.org/dev/peps/pep-0646/ (Note that the final couple of sections are out of date; https://github.com/python/peps/pull/1880 clarifies which grammar changes would be required, now that PEP 637 has been rejected. We also have a second PR in progress at https://github.com/python/peps/pull/1881 clarifying some of the motivation.) Thanks! Matthew and Pradeep

9 19

PEP 677 (Callable Type Syntax): Final Call for Comments
by Steven Troxler Jan. 18, 2022

Jan. 18, 2022

Hello all, We’d like to make a last call for discussion on PEP 677: Callable Type Syntax [1] before sending this to the Steering Council. For context, PEP 677 proposes an arrow-like callable syntax for python: ``` (int, str) -> bool # equivalent to Callable[[int, str], bool] ``` Thanks everyone for comments on our earlier thread [2]. We made some changes to the PEP, but not the proposal itself, in response to that discussion: - More discussion about readability of return types and parentheses - Noting that complex code might be improved by using TypeAliases - Discussion of additional ideas in the Rejected Alternatives section Many of the concerns in the previous thread are about whether new callable syntax is worth the implementation complexity, which is an important question. We’ve seen demand for this from regular typed Python users. And given that Callable is one of the most used and complicated type annotations, we at typing-sig favor new syntax. We have considered the feedback here and in typing-sig and feel that the syntax in the PEP is a reasonable tradeoff between usability and simplicity. One concern raised was that the `async` portion of the proposed syntax may not be worthwhile, even if the rest of the proposal is. We considered removing it, but several heavy users of async pointed out [3] that: - For authors of large, typed async codebases, this syntax would be very helpful - For everyone else there’s minimal readability cost, because async callables will almost never come up. As a result, we decided to leave it in for now, but would be happy to remove it if the SC would prefer that. If there are no other concerns, we'd like to move forward and submit it to the SC. Cheers, Steven ------- [1] PEP 677: https://www.python.org/dev/peps/pep-0677/ [2] Previous RFC thread in python-dev: https://mail.python.org/archives/list/python-dev@python.org/thread/OGACYN2X… [3] Discussion in typing-sig about removing `async` keyword: https://mail.python.org/archives/list/typing-sig@python.org/thread/RFZOSCJH…

7 10

Is anyone using 15-bit PyLong digits (PYLONG_BITS_IN_DIGIT=15)?
by Mark Dickinson Jan. 18, 2022

Jan. 18, 2022

tl;dr: I'd like to deprecate and eventually remove the option to use 15-bit digits in the PyLong implementation. Before doing so, I'd like to find out whether there's anyone still using 15-bit PyLong digits, and if so, why they're doing so. History: the use of 30-bit digits in PyLong was introduced for Python 3.1 and Python 2.7, to improve performance of int (Python 3) / long (Python 2) arithmetic. At that time, we retained the option to use 15-bit digits, for two reasons: - (1) use of 30-bit digits required C99 features (uint64_t and friends) at a time when we hadn't yet committed to requiring C99 - (2) it wasn't clear whether 30-bit digits would be a performance win on 32-bit operating systems Twelve years later, reason (1) no longer applies, and I suspect that: - No-one is deliberately using the 15-bit digit option. - There are few machines where using 15-bit digits is faster than using 30-bit digits. But I don't have solid data on either of these suspicions, hence this post. Removing the 15-bit digit option would simplify the code (there's significant mental effort required to ensure we don't break things for 15-bit builds when modifying Objects/longobject.c, and 15-bit builds don't appear to be exercised by the buildbots), remove a hidden compatibility trap (see b.p.o. issue 35037), widen the applicability of the various fast paths for arithmetic operations, and allow for some minor fast-path small-integer optimisations based on the fact that we'd be able to assume that presence of *two* extra bits in the C integer type rather than just one. As an example of the latter: if `a` and `b` are PyLongs that fit in a single digit, then with 15-bit digits and a 16-bit `digit` and `sdigit` type, `a + b` can't currently safely (i.e., without undefined behaviour from overflow) be computed with the C type `sdigit`. With 30-bit digits and a 32-bit `digit` and `sdigit` type, `a + b` is safe. Mark *References* Related b.p.o. issue: https://bugs.python.org/issue45569 MinGW compatibility issue: https://bugs.python.org/issue35037 Introduction of 30-bit digits: https://bugs.python.org/issue4258

10 32

Changing PySequence and PyMapping checks
by Bar Harel Jan. 15, 2022

Jan. 15, 2022

There's a long time issue of trying to differentiate mappings and sequences in the C-API in a fast and reliable way. Due to recent changes, we might be able to do so at last, by checking tp_flags + str/bytes/bytearray which are considered unique. This might be a breaking change in the Stable ABI promise but one that can be considered a bug fix as the current behavior is lacking. What do you think? bpo: https://bugs.python.org/issue46376 Best regards, Bar Harel

3 2

[RELEASE] Python 3.10.2, 3.9.10, and 3.11.0a4 are now available
by Łukasz Langa Jan. 14, 2022

Jan. 14, 2022

Hi there, before we begin the usual round of release notes, please do note that the three new versions of Python released today do not contain Windows installers yet. This is temporary, due to a more complex than expected code signing certificate renewal. We’ve held the releases all week while the situation is getting resolved but the urgency of 3.10.2 in particular made us release without the Windows installers after all. We apologize for the inconvenience and are doing everything we can to put the Windows installer in place as soon as possible. We’re rooting for both Ee Durbin and Steve Dower who are helping us resolve this. Thanks for your hard work! Hopefully, by this time next week, this will only be a footnote in release management history. The releases you’re looking at were all cursed in some way. What a way to start 2022! Besides the certificate hold up, Python 3.10.2 is an expedited release (you’ll want to upgrade, read below!), Python 3.11.0a4 had almost 20 (sic, twenty!) release blockers before being finally green, and Python 3.9.10 was made from a new M1 Mac on macOS Monterey which made the usually boring process quite a ride. We’re hoping 2022 won’t be this intense all year! <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-avai…>Python 3.10.2 Get it here: https://www.python.org/downloads/release/python-3102/ <https://www.python.org/downloads/release/python-3102/> <https://www.blogger.com/#> This is a special bugfix release ahead of schedule to address a memory leak that was happening on certain function calls when using Cython <https://github.com/cython/cython>. The memory leak consisted of a small constant amount of bytes in certain function calls from Cython code. Although in most cases this was not very noticeable, it was very impactful for long-running applications and certain usage patterns. Check bpo-46347 <https://bugs.python.org/issue46347> for more information. Upgrading existing Python 3.10 installations is highly recommended. Even though this is an expedited release, it still contains over 100 other bug fixes. See the change log <https://docs.python.org/release/3.10.2/whatsnew/changelog.html> for details. The next Python 3.10 maintenance release will be 3.10.3, currently scheduled for 2022-04-04. <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-avai…>Python 3.9.10 Get it here: https://www.python.org/downloads/release/python-3910/ <https://www.blogger.com/#> Python 3.9.10 is the ninth maintenance release of the legacy 3.9 series. Note: Python 3.10 is now the latest feature release series of Python 3. Python 3.9 micro-releases enter double digits! There’s been 130 commits since 3.9.9 which is a higher number of fixes for this stage of the life cycle compared to 3.8. See the changelog <https://docs.python.org/release/3.9.10/whatsnew/changelog.html> for details on what changed. As a reminder, on macOS, the default installer is now the new universal2 variant. It’s compatible with Mac OS X 10.9 and newer, including macOS 11 Big Sur and macOS 12 Monterey. Python installed with this variant will work natively on Apple Silicon processors. The next Python 3.9 maintenance release will be 3.9.11, currently scheduled for Pi Day '22 (2022-03-14). <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-avai…>Python 3.11.0a4 Get it here: https://www.python.org/downloads/release/python-3110a4/ <https://www.blogger.com/#> Python 3.11 is still in development. This release, 3.11.0a4, is the fourth of seven planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes by the community, as well as to test the release process. During the alpha phase, features may be added up until the start of the beta phase (2022-05-06) and, if necessary, may be modified or deleted up until the release candidate phase (2022-08-01). Please keep in mind that this is a preview release and its use is not recommended for production environments. Many new features for Python 3.11 are still being planned and written. Among the new major new features and changes so far: PEP 657 <https://www.python.org/dev/peps/pep-0657/> – Include Fine-Grained Error Locations in Tracebacks PEP 654 <https://www.python.org/dev/peps/pep-0654/> – Exception Groups and except* The Faster CPython Project <https://github.com/faster-cpython> is already yielding some exciting results: this version of CPython 3.11 is ~ 19% faster on the geometric mean of the PyPerformance benchmarks <>, compared to 3.10.0. (Hey, fellow core developer, if a feature you find important is missing from this list, let Pablo know <mailto:pablogsal@python.org>.) The next pre-release of Python 3.11 will be 3.11.0a5, currently scheduled for Wednesday, 2022-02-02. <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-avai…>Python 3.6 is pining for the fjords Python 3.6 is no more. It’s an ex-Python. It has ceased to be. On December 23rd 2021 is has reached its end-of-life phase <https://www.python.org/dev/peps/pep-0494/> after five successful years. It’s been the first truly popular Python 3 release, introducing f-strings to the world and making big improvements to both asyncio (async generators!) and typing (variable annotations!). We’d like to congratulate Ned Deily @nad <https://discuss.python.org/u/nad> on successfully driving the 3.6 series to completion as Release Manager. He’s not fully retired yet, as 3.7, which he is also managing, is still receiving security patches until June 2023. <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-avai…>We hope you enjoy the new releases Your friendly release team, Pablo Galindo Salgado @pablogsal <https://discuss.python.org/u/pablogsal> Łukasz Langa @ambv <https://discuss.python.org/u/ambv>

1 0

Summary of Python tracker Issues
by Python tracker Jan. 14, 2022

Jan. 14, 2022

ACTIVITY SUMMARY (2022-01-07 - 2022-01-14) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7202 ( +3) closed 50906 (+81) total 58108 (+84) Open issues with patches: 2886 Issues opened (48) ================== #32876: HTMLParser raises exception on some inputs https://bugs.python.org/issue32876 reopened by iritkatriel #46298: Automatically check for __slots__-mistakes in Lib https://bugs.python.org/issue46298 opened by ariebovenberg #46304: Unable to iterate over lines in a file without a block of code https://bugs.python.org/issue46304 opened by jaraco #46309: Task created by StreamReaderProtocol gets garbage collected. https://bugs.python.org/issue46309 opened by simwr872 #46311: Clean up PyLong_FromLong and PyLong_FromLongLong https://bugs.python.org/issue46311 opened by mark.dickinson #46312: range() function could accept slice() objects as parameters https://bugs.python.org/issue46312 opened by yota moteuchi #46313: SSLObject does not raise SSLEOFError on OpenSSL 3 https://bugs.python.org/issue46313 opened by alex.gronholm #46315: Add support for WebAssembly System Interface (wasm32-wasi) https://bugs.python.org/issue46315 opened by christian.heimes #46316: Optimize pathlib.Path.iterdir() https://bugs.python.org/issue46316 opened by barneygale #46317: Pathlib.rename isn't robust https://bugs.python.org/issue46317 opened by Oz.Tiram #46318: asyncio and ssl: ResourceWarning: unclosed transport https://bugs.python.org/issue46318 opened by mdk #46323: Use _PyObject_Vectorcall in Modules/_ctypes/callbacks.c https://bugs.python.org/issue46323 opened by hydroflask #46325: Documentation for SubprocessTransport.get_pipe_transport retur https://bugs.python.org/issue46325 opened by xx11mz #46326: 'venv --clear' should prompt user before nuking entire directo https://bugs.python.org/issue46326 opened by alimpfard #46329: Split up the CALL_NO_KW and CALL_KW instructions. https://bugs.python.org/issue46329 opened by Mark.Shannon #46330: Simplify the signature of __exit__ https://bugs.python.org/issue46330 opened by Jelle Zijlstra #46333: ForwardRef.__eq__ does not respect module parameter https://bugs.python.org/issue46333 opened by andreash #46334: Glossary URLs with anchor link no longer jump to definitions https://bugs.python.org/issue46334 opened by trey #46335: asyncio.create_subprocess_exec throws RuntimeError yet still e https://bugs.python.org/issue46335 opened by Clint Olsen #46336: Sixth element of tuple from __reduce__(), inconsistency betwee https://bugs.python.org/issue46336 opened by lev.bishop #46337: urllib.parse: Allow more flexibility in schemes and URL resolu https://bugs.python.org/issue46337 opened by lincolnauster #46338: libc_ver() runtime error when sys.executable is empty https://bugs.python.org/issue46338 opened by allie.hammond #46339: PEG parser segfault from ast.literal_eval https://bugs.python.org/issue46339 opened by gregory.p.smith #46340: DeprecationWarning emitted when running asyncio tests https://bugs.python.org/issue46340 opened by kumaraditya303 #46341: duplicate paragraphs - asyncio Coroutines and Tasks file https://bugs.python.org/issue46341 opened by davem #46343: Add PyErr_GetActiveException and PyErr_SetActiveException https://bugs.python.org/issue46343 opened by iritkatriel #46349: inspect.getdoc() should append parent class method docs when e https://bugs.python.org/issue46349 opened by gregory.p.smith #46350: re.sub, re.Match.expand, etc doesn't allow x, u, U, or N escap https://bugs.python.org/issue46350 opened by bup #46351: Makefile missing '/' for some path names https://bugs.python.org/issue46351 opened by gwolski #46353: 'pydoc -k' fails when some module's loader is not found https://bugs.python.org/issue46353 opened by dlax #46356: [C API] Enforce usage of PyFrame_GetBack() https://bugs.python.org/issue46356 opened by vstinner #46360: Inconsistent import behavior for (unusual) submodules https://bugs.python.org/issue46360 opened by eric.snow #46361: Small ints aren't always cached properly https://bugs.python.org/issue46361 opened by brandtbucher #46363: Two typos in versions 3.7 document translation of zh_CN https://bugs.python.org/issue46363 opened by perlang #46364: asyncio subprocess cannot read from /dev/stdin https://bugs.python.org/issue46364 opened by xoph #46367: multiprocessing's "spawn" doesn't actually use spawn https://bugs.python.org/issue46367 opened by jakirkham #46368: faulthandler: add the ability to dump all interpreters, not on https://bugs.python.org/issue46368 opened by vstinner #46369: get_type_hints does not evaluate ForwardRefs inside NewType https://bugs.python.org/issue46369 opened by andreash #46371: A better way to resolve ForwardRefs in type aliases across mod https://bugs.python.org/issue46371 opened by andreash #46372: int/float specializations should mutate the LHS in-place when https://bugs.python.org/issue46372 opened by brandtbucher #46373: TypedDict and NamedTuple do not evaluate cross-module ForwardR https://bugs.python.org/issue46373 opened by andreash #46374: Assertion failed in ceval.c https://bugs.python.org/issue46374 opened by kumaraditya303 #46375: io.BytesIO does not have peek() https://bugs.python.org/issue46375 opened by marcelm #46376: PyMapping_Check returns 1 for list https://bugs.python.org/issue46376 opened by aviramha #46377: TimedRotatingFileHandler "midnight" misleading when interval > https://bugs.python.org/issue46377 opened by mschiess #46378: Experiment with LLVM BOLT binary optimizer https://bugs.python.org/issue46378 opened by corona10 #46379: itertools.product reference implementation creates temporaries https://bugs.python.org/issue46379 opened by mwallerb #46380: `test_functools.TestLRU` must not use `functools` module dire https://bugs.python.org/issue46380 opened by sobolevn Most recent 15 issues with no replies (15) ========================================== #46380: `test_functools.TestLRU` must not use `functools` module dire https://bugs.python.org/issue46380 #46378: Experiment with LLVM BOLT binary optimizer https://bugs.python.org/issue46378 #46375: io.BytesIO does not have peek() https://bugs.python.org/issue46375 #46373: TypedDict and NamedTuple do not evaluate cross-module ForwardR https://bugs.python.org/issue46373 #46372: int/float specializations should mutate the LHS in-place when https://bugs.python.org/issue46372 #46371: A better way to resolve ForwardRefs in type aliases across mod https://bugs.python.org/issue46371 #46367: multiprocessing's "spawn" doesn't actually use spawn https://bugs.python.org/issue46367 #46341: duplicate paragraphs - asyncio Coroutines and Tasks file https://bugs.python.org/issue46341 #46337: urllib.parse: Allow more flexibility in schemes and URL resolu https://bugs.python.org/issue46337 #46329: Split up the CALL_NO_KW and CALL_KW instructions. https://bugs.python.org/issue46329 #46325: Documentation for SubprocessTransport.get_pipe_transport retur https://bugs.python.org/issue46325 #46317: Pathlib.rename isn't robust https://bugs.python.org/issue46317 #46316: Optimize pathlib.Path.iterdir() https://bugs.python.org/issue46316 #46311: Clean up PyLong_FromLong and PyLong_FromLongLong https://bugs.python.org/issue46311 #46298: Automatically check for __slots__-mistakes in Lib https://bugs.python.org/issue46298 Most recent 15 issues waiting for review (15) ============================================= #46380: `test_functools.TestLRU` must not use `functools` module dire https://bugs.python.org/issue46380 #46379: itertools.product reference implementation creates temporaries https://bugs.python.org/issue46379 #46377: TimedRotatingFileHandler "midnight" misleading when interval > https://bugs.python.org/issue46377 #46376: PyMapping_Check returns 1 for list https://bugs.python.org/issue46376 #46372: int/float specializations should mutate the LHS in-place when https://bugs.python.org/issue46372 #46368: faulthandler: add the ability to dump all interpreters, not on https://bugs.python.org/issue46368 #46364: asyncio subprocess cannot read from /dev/stdin https://bugs.python.org/issue46364 #46361: Small ints aren't always cached properly https://bugs.python.org/issue46361 #46353: 'pydoc -k' fails when some module's loader is not found https://bugs.python.org/issue46353 #46343: Add PyErr_GetActiveException and PyErr_SetActiveException https://bugs.python.org/issue46343 #46339: PEG parser segfault from ast.literal_eval https://bugs.python.org/issue46339 #46337: urllib.parse: Allow more flexibility in schemes and URL resolu https://bugs.python.org/issue46337 #46333: ForwardRef.__eq__ does not respect module parameter https://bugs.python.org/issue46333 #46316: Optimize pathlib.Path.iterdir() https://bugs.python.org/issue46316 #46315: Add support for WebAssembly System Interface (wasm32-wasi) https://bugs.python.org/issue46315 Top 10 most discussed issues (10) ================================= #46070: [subinterpreters] crash when importing _sre in subinterpreters https://bugs.python.org/issue46070 18 msgs #46291: [doc] First argument to raise can also be BaseException https://bugs.python.org/issue46291 13 msgs #37295: Possible optimizations for math.comb() https://bugs.python.org/issue37295 10 msgs #39298: add BLAKE3 to hashlib https://bugs.python.org/issue39298 10 msgs #46249: [sqlite3] move set lastrowid out of the query loop and enable https://bugs.python.org/issue46249 8 msgs #46279: [docs] Minor information-ordering issue in __main__ doc https://bugs.python.org/issue46279 8 msgs #40479: Port _hashlib to OpenSSL 3.0.0 https://bugs.python.org/issue40479 7 msgs #46304: Unable to iterate over lines in a file without a block of code https://bugs.python.org/issue46304 7 msgs #46313: SSLObject does not raise SSLEOFError on OpenSSL 3 https://bugs.python.org/issue46313 7 msgs #46142: python --help output is too long https://bugs.python.org/issue46142 6 msgs Issues closed (75) ================== #23183: timeit CLI best of 3: undocumented output format https://bugs.python.org/issue23183 closed by iritkatriel #23819: test_asyncio fails when run under -O https://bugs.python.org/issue23819 closed by asvetlov #24650: Error in yield expression documentation https://bugs.python.org/issue24650 closed by iritkatriel #27273: subprocess.run(cmd, input='text') should pass universal_newlin https://bugs.python.org/issue27273 closed by gregory.p.smith #28546: [doc] Clarify setting pdb breakpoints https://bugs.python.org/issue28546 closed by iritkatriel #31472: "Emulating callable objects" documentation misleading https://bugs.python.org/issue31472 closed by iritkatriel #32679: concurrent.futures should store full sys.exc_info() https://bugs.python.org/issue32679 closed by iritkatriel #32996: Improve What's New in 3.7 https://bugs.python.org/issue32996 closed by iritkatriel #37591: test_concurrent_future failed https://bugs.python.org/issue37591 closed by iritkatriel #37595: Python 3.7.4 does not build on Raspbian Buster with optimizati https://bugs.python.org/issue37595 closed by christian.heimes #38204: Cannot compile on RPi with optimizations https://bugs.python.org/issue38204 closed by iritkatriel #39316: settrace skips lines when chaining methods without arguments https://bugs.python.org/issue39316 closed by iritkatriel #40064: py38: document xml.etree.cElementTree will be removed in 3.9 https://bugs.python.org/issue40064 closed by iritkatriel #44133: Some C-API symbols (e.g. Py_FrozenMain) are not always exporte https://bugs.python.org/issue44133 closed by vstinner #44577: Probably defect in Python 3.7.11 https://bugs.python.org/issue44577 closed by iritkatriel #44594: AsyncExitStack.enter_async_context() is mishandling exception https://bugs.python.org/issue44594 closed by iritkatriel #45331: Can create enum of ranges, cannot create range enum. Range sho https://bugs.python.org/issue45331 closed by AlexWaygood #45729: [doc] "history and license" link has wrong target https://bugs.python.org/issue45729 closed by mdk #45786: Avoid allocating when exiting frame; it may be unsafe. https://bugs.python.org/issue45786 closed by Mark.Shannon #45797: AMD64 Arch Linux Asan Debug buildbot sometimes hangs before te https://bugs.python.org/issue45797 closed by vstinner #46006: [subinterpreter] _PyUnicode_EqualToASCIIId() issue with subint https://bugs.python.org/issue46006 closed by vstinner #46162: Make `builtins.property` generic https://bugs.python.org/issue46162 closed by sobolevn #46203: Add timeout for Windows build steps https://bugs.python.org/issue46203 closed by mark.dickinson #46205: test.libregrtest: Race condition in runtest_mp leads to hangs https://bugs.python.org/issue46205 closed by corona10 #46216: spurious link to os.system() from os.times() documentation ent https://bugs.python.org/issue46216 closed by iritkatriel #46217: 3.11 build failure on Win10: new _freeze_module changes? https://bugs.python.org/issue46217 closed by steve.dower #46235: Do all ref-counting at once for sequence multiplication https://bugs.python.org/issue46235 closed by tim.peters #46237: Incorrect line reported in syntax error https://bugs.python.org/issue46237 closed by pablogsal #46244: typing._TypeVarLike missing __slots__ https://bugs.python.org/issue46244 closed by AlexWaygood #46253: C API documentation of Py_UNICODE_* character properties macro https://bugs.python.org/issue46253 closed by benjamin.peterson #46261: [doc] fix inaccuracies in sqlite3.Cursor.lastrowid docs https://bugs.python.org/issue46261 closed by erlendaasland #46269: '__new__' is never shown in `dir(SomeEnum)` https://bugs.python.org/issue46269 closed by sobolevn #46270: Comparison operators in Python Tutorial 5.7 https://bugs.python.org/issue46270 closed by rhettinger #46272: Fix bitwise and logical terminology in python.gram https://bugs.python.org/issue46272 closed by rhettinger #46285: http/server.py wont respect its protocol_version https://bugs.python.org/issue46285 closed by openalmeida #46289: AST: FomattedValue conversion's default value should be -1 https://bugs.python.org/issue46289 closed by BTaskaya #46290: Parameter names are inaccurate in dataclasses docs https://bugs.python.org/issue46290 closed by taleinat #46296: Unreachable condition: `if enum_class._member_type_ is object` https://bugs.python.org/issue46296 closed by ethan.furman #46297: Python interpreter crashes on bootup with multiple PythonPaths https://bugs.python.org/issue46297 closed by steve.dower #46299: Improve several exception handling practices in `test_descr.py https://bugs.python.org/issue46299 closed by corona10 #46300: zlib.compress level and wbits args are shown as keyword-only i https://bugs.python.org/issue46300 closed by andrei.avk #46301: Enum tests: One branch is not covered in `Enum._convert_` https://bugs.python.org/issue46301 closed by ethan.furman #46302: IndexError inside list comprehension + workaround https://bugs.python.org/issue46302 closed by steven.daprano #46303: _Py_stat and _Py_wstat using incorrect type for status argumen https://bugs.python.org/issue46303 closed by vstinner #46305: SyntaxError when using dict key in fstring https://bugs.python.org/issue46305 closed by Samisafool #46306: Suspicious operation in `doctest.py`: `None - 1` https://bugs.python.org/issue46306 closed by eric.smith #46307: string.Template should allow inspection of identifiers https://bugs.python.org/issue46307 closed by AlexWaygood #46308: Unportable test(1) operator in configure script https://bugs.python.org/issue46308 closed by vstinner #46310: Simplify `for` loop in `.close()` method in `asyncio/windows_e https://bugs.python.org/issue46310 closed by asvetlov #46314: Stray RESUME opcode for unused lambda https://bugs.python.org/issue46314 closed by Mark.Shannon #46319: datetime.utcnow() should return a timezone aware datetime https://bugs.python.org/issue46319 closed by p-ganssle #46320: runtime/interp/thread state refactoring leads to segmentation https://bugs.python.org/issue46320 closed by Quentin.Pradet #46321: Don't need delineators on lists https://bugs.python.org/issue46321 closed by Dennis Sweeney #46322: Invalid file path of SSLKEYLOGFILE throw FileNotFoundError https://bugs.python.org/issue46322 closed by ghaaf #46324: 'import traceback' Causes a Crash https://bugs.python.org/issue46324 closed by steven.daprano #46327: `test_enum` contains tests for older versions of python https://bugs.python.org/issue46327 closed by sobolevn #46328: add sys.exception() https://bugs.python.org/issue46328 closed by iritkatriel #46331: 3.11: tracing revisits class line after class docstring https://bugs.python.org/issue46331 closed by erlendaasland #46332: Use raise..from in logging/config instead of assigning __cause https://bugs.python.org/issue46332 closed by iritkatriel #46342: Make @final introspectable at runtime https://bugs.python.org/issue46342 closed by AlexWaygood #46344: 3.11: except/else/if/raise traced incorrectly https://bugs.python.org/issue46344 closed by iritkatriel #46345: Add an explicit test for `get_type_hints` for a class field wi https://bugs.python.org/issue46345 closed by sobolevn #46346: New compilation warnings on Windows https://bugs.python.org/issue46346 closed by vstinner #46347: memory leak in PyEval_EvalCodeEx https://bugs.python.org/issue46347 closed by yselivanov #46348: Modernize `test_typing` https://bugs.python.org/issue46348 closed by AlexWaygood #46352: Spam https://bugs.python.org/issue46352 closed by Dennis Sweeney #46354: AttributeError: module 'collections' has no attribute 'Mapping https://bugs.python.org/issue46354 closed by AlexWaygood #46355: [C API] Document PyFrameObject and PyThreadState changes and e https://bugs.python.org/issue46355 closed by vstinner #46357: socket module fix warning build on FreeBSD https://bugs.python.org/issue46357 closed by devnexen #46358: Modernize `test_asyncio/test_base_events.py` https://bugs.python.org/issue46358 closed by AlexWaygood #46359: Continue to modernize `test_typing.py` https://bugs.python.org/issue46359 closed by sobolevn #46362: os.path.abspath() needs more normalization on Windows https://bugs.python.org/issue46362 closed by steve.dower #46365: _ curses module is not installed https://bugs.python.org/issue46365 closed by simon28li #46366: Recursively propagate the mode, in os.makedirs https://bugs.python.org/issue46366 closed by dcasier #46370: Move runtime static init to its own header file. https://bugs.python.org/issue46370 closed by eric.snow

1 0

[PSA] OpenSSL 3.0 support is preliminary and experimental
by Christian Heimes Jan. 12, 2022

Jan. 12, 2022

Hi, I would like to remind everybody that Python's support for OpenSSL 3.0 is preliminary [1]. Python compiles with OpenSSL 3.0.0 and simple code kinda works. However there are known performance regressions, missing features (e.g. usedforsecurity flag), and potential bugs cause by API incompatibilities. Due to the experimental state I advise against using Python with OpenSSL 3.0 in production. It may take a while until Python gains full support for the next version of OpenSSL. I have shifted my personal OSS time to more fun topics like performance and WASM. My work time is currently limited, too. Christian [1] https://docs.python.org/3/whatsnew/3.10.html#ssl

4 5

PEP 674: Disallow using macros as l-value
by Victor Stinner Jan. 12, 2022

Jan. 12, 2022

Hi, I propose to disallow using macros as l-value. Read and comment the plain text below, or read the PEP 674 online, https://python.github.io/peps/pep-0674/ While I'm not sure that the proposed changes are really controversial, I decided to write a formal PEP since the incompatible changes are not following the PEP 387 deprecation process and so a format PEP 387 exception is better. Well, the list of modified macros is also quite long. Moreover, a PEP is a way to document and announce the changes ;-) The Py_TYPE() and Py_SIZE() changes are already approved the Steering Council, but I prefer to list them in the PEP: https://github.com/python/steering-council/issues/79 In practice, I'm only aware of 4 projects impacted by these changes, and I wrote the pythoncapi_compat project which updates automatically C extensions: add Python 3.11 support, without losing support for older Python versions. I already prepared major projects like Cython and numpy for these changes (in total, 14 impacted projects have already been updated). Victor --- PEP: 674 Title: Disallow using macros as l-value Author: Victor Stinner <vstinner(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30-Oct-2021 Python-Version: 3.11 Abstract ======== Incompatible C API change disallowing using macros as l-value to allow evolving CPython internals and to ease the C API implementation on other Python implementation. In practice, the majority of projects impacted by these incompatible changes should only have to make two changes: * Replace ``Py_TYPE(obj) = new_type;`` with ``Py_SET_TYPE(obj, new_type);``. * Replace ``Py_SIZE(obj) = new_size;`` with ``Py_SET_SIZE(obj, new_size);``. Rationale ========= Using a macro as a l-value -------------------------- In the Python C API, some functions are implemented as macro because writing a macro is simpler than writing a regular function. If a macro exposes directly a struture member, it is technically possible to use this macro to not only get the structure member but also set it. Example with the Python 3.10 ``Py_TYPE()`` macro:: #define Py_TYPE(ob) (((PyObject *)(ob))->ob_type) This macro can be used as a **r-value** to **get** an object type:: type = Py_TYPE(object); It can also be used as **l-value** to **set** an object type:: Py_TYPE(object) = new_type; It is also possible to set an object reference count and an object size using ``Py_REFCNT()`` and ``Py_SIZE()`` macros. Setting directly an object attribute relies on the current exact CPython implementation. Implementing this feature in other Python implementations can make their C API implementation less efficient. CPython nogil fork ------------------ Sam Gross forked Python 3.8 to remove the GIL: the `nogil branch <https://github.com/colesbury/nogil/>`_. This fork has no ``PyObject.ob_refcnt`` member, but a more elaborated implementation for reference counting, and so the ``Py_REFCNT(obj) = new_refcnt;`` code fails with a compiler error. Merging the nogil fork into the upstream CPython main branch requires first to fix this C API compatibility issue. It is a concrete example of a Python optimization blocked indirectly by the C API. This issue was already fixed in Python 3.10: the ``Py_REFCNT()`` macro has been already modified to disallow using it as a l-value. HPy project ----------- The `HPy project <https://hpyproject.org/>`_ is a brand new C API for Python using only handles and function calls: handles are opaque, structure members cannot be accessed directly,and pointers cannot be dereferenced. Disallowing the usage of macros as l-value helps the migration of existing C extensions to HPy by reducing differences between the C API and the HPy API. PyPy cpyext module ------------------ In PyPy, when a Python object is accessed by the Python C API, the PyPy ``cpyext`` module has to convert PyPy object to a CPython object. While PyPy objects are designed to be efficient with the PyPy JIT compiler, CPython objects are less efficient and increase the memory usage. This PEP alone is not enough to get rid of the CPython objects in the PyPy ``cpyext`` module, but it is a step towards this long term goal. PyPy already supports HPy which is a better solution in the long term. Specification ============= Disallow using macros as l-value -------------------------------- PyObject and PyVarObject macros ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * ``Py_TYPE()``: ``Py_SET_TYPE()`` must be used instead * ``Py_SIZE()``: ``Py_SET_SIZE()`` must be used instead "GET" macros ^^^^^^^^^^^^ * ``PyByteArray_GET_SIZE()`` * ``PyBytes_GET_SIZE()`` * ``PyCFunction_GET_CLASS()`` * ``PyCFunction_GET_FLAGS()`` * ``PyCFunction_GET_FUNCTION()`` * ``PyCFunction_GET_SELF()`` * ``PyCell_GET()`` * ``PyCode_GetNumFree()`` * ``PyDict_GET_SIZE()`` * ``PyFunction_GET_ANNOTATIONS()`` * ``PyFunction_GET_CLOSURE()`` * ``PyFunction_GET_CODE()`` * ``PyFunction_GET_DEFAULTS()`` * ``PyFunction_GET_GLOBALS()`` * ``PyFunction_GET_KW_DEFAULTS()`` * ``PyFunction_GET_MODULE()`` * ``PyHeapType_GET_MEMBERS()`` * ``PyInstanceMethod_GET_FUNCTION()`` * ``PyList_GET_SIZE()`` * ``PyMemoryView_GET_BASE()`` * ``PyMemoryView_GET_BUFFER()`` * ``PyMethod_GET_FUNCTION()`` * ``PyMethod_GET_SELF()`` * ``PySet_GET_SIZE()`` * ``PyTuple_GET_SIZE()`` * ``PyUnicode_GET_DATA_SIZE()`` * ``PyUnicode_GET_LENGTH()`` * ``PyUnicode_GET_LENGTH()`` * ``PyUnicode_GET_SIZE()`` * ``PyWeakref_GET_OBJECT()`` "AS" macros ^^^^^^^^^^^ * ``PyByteArray_AS_STRING()`` * ``PyBytes_AS_STRING()`` * ``PyFloat_AS_DOUBLE()`` * ``PyUnicode_AS_DATA()`` * ``PyUnicode_AS_UNICODE()`` PyUnicode macros ^^^^^^^^^^^^^^^^ * ``PyUnicode_1BYTE_DATA()`` * ``PyUnicode_2BYTE_DATA()`` * ``PyUnicode_4BYTE_DATA()`` * ``PyUnicode_DATA()`` * ``PyUnicode_IS_ASCII()`` * ``PyUnicode_IS_COMPACT()`` * ``PyUnicode_IS_READY()`` * ``PyUnicode_KIND()`` * ``PyUnicode_READ()`` * ``PyUnicode_READ_CHAR()`` PyDateTime "GET" macros ^^^^^^^^^^^^^^^^^^^^^^^ * ``PyDateTime_DATE_GET_FOLD()`` * ``PyDateTime_DATE_GET_HOUR()`` * ``PyDateTime_DATE_GET_MICROSECOND()`` * ``PyDateTime_DATE_GET_MINUTE()`` * ``PyDateTime_DATE_GET_SECOND()`` * ``PyDateTime_DATE_GET_TZINFO()`` * ``PyDateTime_DELTA_GET_DAYS()`` * ``PyDateTime_DELTA_GET_MICROSECONDS()`` * ``PyDateTime_DELTA_GET_SECONDS()`` * ``PyDateTime_GET_DAY()`` * ``PyDateTime_GET_MONTH()`` * ``PyDateTime_GET_YEAR()`` * ``PyDateTime_TIME_GET_FOLD()`` * ``PyDateTime_TIME_GET_HOUR()`` * ``PyDateTime_TIME_GET_MICROSECOND()`` * ``PyDateTime_TIME_GET_MINUTE()`` * ``PyDateTime_TIME_GET_SECOND()`` * ``PyDateTime_TIME_GET_TZINFO()`` PyDescr macros ^^^^^^^^^^^^^^ * ``PyDescr_NAME()`` * ``PyDescr_TYPE()`` Port C extensions to Python 3.11 -------------------------------- In practice, the majority of projects impacted by these PEP incompatible changes should only have to make two changes: * Replace ``Py_TYPE(obj) = new_type;`` with ``Py_SET_TYPE(obj, new_type);``. * Replace ``Py_SIZE(obj) = new_size;`` with ``Py_SET_SIZE(obj, new_size);``. The `pythoncapi_compat project <https://github.com/pythoncapi/pythoncapi_compat>`_ can be used to update automatically C extensions: add Python 3.11 support without losing support with older Python versions. The project provides a header file which provides ``Py_SET_REFCNT()``, ``Py_SET_TYPE()`` and ``Py_SET_SIZE()`` functions to Python 3.8 and older. PyTuple_GET_ITEM() and PyList_GET_ITEM() ---------------------------------------- The ``PyTuple_GET_ITEM()`` and ``PyList_GET_ITEM()`` macros are left unchanged. The code pattern ``&PyTuple_GET_ITEM(tuple, 0)`` and ``&PyList_GET_ITEM(list, 0)`` is still commonly used to get access to the inner ``PyObject**`` array. Changing these macros would require to add a new API to get access to the inner array which is out of the scope of this PEP. Backwards Compatibility ======================= The proposed C API changes are backward incompatible on purpose. In practice, only a minority of third party projects are affected (16 projects are known to be broken) and `most of them have already been updated for these changes <https://bugs.python.org/issue39573#msg401378>`__ (12 on 16). Most projects are broken by ``Py_TYPE()`` and ``Py_SIZE()`` changes. These two macros have been converted to static inline macro in Python 3.10 alpha versions, but the change had to be reverted since it broke too many projects. In the meanwhile, many projects, like Cython, have been prepared for this change by using ``Py_SET_TYPE()`` and ``Py_SET_SIZE()``. For example, projects using Cython only have to regenerate their outdated generated C code to become compatible. For the "GET" functions like ``PyDict_GET_SIZE()``, no project in the PyPI top 5000 projects use these functions as l-value. The ``PyFloat_AS_DOUBLE()`` function is not used as a l-value in the PyPI top 5000 projects. The ``PyBytes_AS_STRING()`` and ``PyByteArray_AS_STRING()`` are used as l-value but only to modify string characters, not to override the ``PyBytesObject.ob_sval`` or ``PyByteArrayObject.ob_start`` member. For example, Cython uses the following code which remains valid:: PyByteArray_AS_STRING(string)[i] = (char) v; This change does not follow the PEP 387 deprecation process. There is no known way to emit a deprecation warning when a macro is used as a l-value, but not when it's used differently (ex: r-value). Rejected Idea: Leave the macros as they are =========================================== The documentation of each function can discourage developers to use macros to modify Python objects. If these is a need to make an assignment, a setter function can be added and the macro documentation can require to use the setter function. For example, a ``Py_SET_TYPE()`` function has been added to Python 3.9 and the ``Py_TYPE()`` documentation now requires to use the ``Py_SET_TYPE()`` function to set an object type. If developers use macros as l-value, it's their responsibility when their code breaks, not the Python responsibility. We are operating under the consenting adults principle: we expect users of the Python C API to use it as documented and expect them to take care of the fallout, if things break when they don't. This idea was rejected because only few developers read the documentation, and only a minority is tracking changes of the Python C API documentation. The majority of developers are only using CPython and so are not aware of compatibility issues with other Python implementations. Moreover, continuing to allow using macros as l-value does not solve issues of the nogil, PyPy and HPy projects. Macros already modified ======================= The following C API macros have already been modified to disallow using them as l-value: * ``PyCell_SET()`` * ``PyList_SET_ITEM()`` * ``PyTuple_SET_ITEM()`` * ``Py_REFCNT()`` (Python 3.10): ``Py_SET_REFCNT()`` must be used * ``_PyGCHead_SET_FINALIZED()`` * ``_PyGCHead_SET_NEXT()`` * ``asdl_seq_GET()`` * ``asdl_seq_GET_UNTYPED()`` * ``asdl_seq_LEN()`` * ``asdl_seq_SET()`` * ``asdl_seq_SET_UNTYPED()`` For example, ``PyList_SET_ITEM(list, 0, item) < 0`` now fails with a compiler error as expected. References ========== * `Python C API: Add functions to access PyObject <https://vstinner.github.io/c-api-abstract-pyobject.html>`_ (October 2021) article by Victor Stinner * `[C API] Disallow using PyFloat_AS_DOUBLE() as l-value <https://bugs.python.org/issue45476>`_ (October 2021) * `[capi-sig] Py_TYPE() and Py_SIZE() become static inline functions <https://mail.python.org/archives/list/capi-sig@python.org/thread/WGRLTHTHC3…>`_ (September 2021) * `[C API] Avoid accessing PyObject and PyVarObject members directly: add Py_SET_TYPE() and Py_IS_TYPE(), disallow Py_TYPE(obj)=type <https://bugs.python.org/issue39573>`__ (February 2020) * `bpo-30459: PyList_SET_ITEM could be safer <https://bugs.python.org/issue30459>`_ (May 2017) Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

10 25

Status of Python 3.11.0a4
by Pablo Galindo Salgado Jan. 11, 2022

Jan. 11, 2022

Hi everyone, I want to report on the status of Python 3.11.0a4. We had a ton of release blockers (some extra ones since I reported the last time) and it seems that we managed to fix them all (thanks to Mark Shannon, Christian Heimes, Gregory P. Smith, Neil Schemenauer, Steve Dower and many others that helped with the blockers. Unfortunately it seems that the release is cursed and we are having some problems with the certificates to create and sign the Windows artefacts so we are waiting for that to unblock us. I will keep you posted on new developments. Regards from rainy London, Pablo Galindo Salgado

1 0

PEP 676 -- PEP Infrastructure Process
by python＠quite.org.uk Jan. 10, 2022

Jan. 10, 2022

Hi, I would like to announce PEP 676 to python-dev. It is a meta-PEP focussed on modernising the PEP build infrastructure. From the abstract, "This PEP addresses the infrastructure around rendering PEP files from reStructuredText files to HTML webpages. We aim to specify a self-contained and maintainable solution for PEP readers, authors, and editors." Link: https://www.python.org/dev/peps/pep-0676/ Rendered through the PEP 676 system: https://python.github.io/peps/pep-0676/ Please see https://discuss.python.org/t/10774 for prior discussion and to give any feedback. Thanks, Adam Turner

4 3

Core review request: bpo-33178: Add BigEndianUnion, LittleEndianUnion classes to ctypes #25480
by Dave Goncalves Jan. 10, 2022

Jan. 10, 2022

Apologies for one of these review request emails. Anybody with ctypes experience in the core reviewer team available to review and comment/merge? This has gotten reviewed and passes checks. Python bug: https://bugs.python.org/issue33178 Current PR: https://github.com/python/cpython/pull/25480 Thanks, David Goncalves

1 0

PEP 679 – Allow parentheses in assert statements
by Pablo Galindo Salgado Jan. 10, 2022

Jan. 10, 2022

Hi everyone, I would like to start a discussion about a small PEP proposal to allow parentheses in assert statements to fix a common gotcha with assert statements. Link to the PEP: https://www.python.org/dev/peps/pep-0679/ *Please, redirect all discussions to: * https://discuss.python.org/t/pep-679-allow-parentheses-in-assert-statements… as I will not be monitoring answers to this thread. Thanks, everyone for your time! Regards from cloudy London, Pablo Galindo Salgado

6 7

About vulnerabilities in Cpython native code
by lxr1210＠mail.ustc.edu.cn Jan. 9, 2022

Jan. 9, 2022

Hi all, I am currently doing some research on the security of CPython. I used the open source vulnerability analysis engine, Infer(https://fbinfer.com/), to scan the native code of CPython 3.10.0. The scan results show that there are still a number of vulnerabilities in the CPython native code, such as Null dereference, Uninitialized variable, Resource/Memory leak, etc. Moreover, I found that some of the vulnerabilities are related to Python/C API. I enclose the vulnerability report for your reference. Based on the research of the result, I tried to design a tool to automatically detect and repair vulnerabilities in CPython and make this tool available. See: https://github.com/PVMPATCH/PVMPatch Python is my favourite programming language. I sincerely hope that I can help Python become stronger and safer. I hope this discovery can be useful for you to develop Python in the future. Thank you for your time and consideration! Lin

6 10

Python 3.6 rides into the sunset
by Ned Deily Jan. 8, 2022

Jan. 8, 2022

While there are many reasons to welcome the end of 2021, I would like to give a shout-out to Python 3.6 which officially reached end-of-life on 2021-12-23, 6.5 years after its development began and exactly five years after its initial release. Building on the success of previous Python 3 releases, 3.6 added many new features and improvements too numerous to list: over 10000 commits by some 160 contributors, including one of the most popular features in recent Python releases, f-strings (thanks, EVS!). I think it is fair to say that, with Python 3.6, the long transition from Python 2 was finally settled. As release manager for 3.6, I would like to personally thank all those contributors, and mostly volunteers: you made my job an easy one with your overwhelmingly positive attitude and support. I would also like to thank the authors and maintainers of the many third-party packages that were updated to support 3.6 as well as the downstream distributors of Python whose rapid uptake and release of 3.6 in their distributions was crucial to its success. I would also like to thank those who helped get the 3.6 releases out the door, in particular, Steve Dower for manufacturing the Windows packages, Julien Palard for managing our on-line documentation build process, Elvis Pranskevichus and Yury Selivanov for taking on the thankless task of assembling and editing the 3.6 "What's New" document, my fellow release managers for their encouragement and support from the start to finish of 3.6's life, the Steering Councils, the PSF Infrastructure Team, those individuals and organizations who contribute resources (money, people, time, facilities, services) to the PSF, making Python development possible. And, I suppose I should thank that git who produces the macOS packages. Thanks again to you all for making 3.6 so successful! --Ned P.S. As a reminder, with 3.6 having reached end-of-life, we no longer accept bug reports of any type against 3.6 and the 3.6 source code is now frozen. There is no longer a 3.6 branch in the GitHub cpython repository; the final state of the branch is captured in the repo as tag "3.6" and, as always, the source code for any release can be checked out using its tag; for example, the source for the final release of 3.6 can be obtained with "git checkout v3.6.15". Pro tip: if you haven't already, you may want to update your repo clones with "git fetch --tags upstream" (if you use the recommended naming convention) to get the latest tags and branches. -- Ned Deily nad(a)python.org -- []

1 0

Summary of Python tracker Issues
by Python tracker Jan. 7, 2022

Jan. 7, 2022

ACTIVITY SUMMARY (2021-12-31 - 2022-01-07) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7199 ( +4) closed 50825 (+78) total 58024 (+82) Open issues with patches: 2886 Issues opened (50) ================== #34931: os.path.splitext with more dots https://bugs.python.org/issue34931 reopened by xnovakj #46009: sending non-None values makes generator raise StopIteration on https://bugs.python.org/issue46009 reopened by christian.heimes #46110: compile("-"*3000000 + "4", '', mode) causes hard crash https://bugs.python.org/issue46110 reopened by pablogsal #46216: spurious link to os.system() from os.times() documentation ent https://bugs.python.org/issue46216 opened by emery.berger #46217: 3.11 build failure on Win10: new _freeze_module changes? https://bugs.python.org/issue46217 opened by terry.reedy #46220: imaplib.py "select" mailbox names containing spaces. https://bugs.python.org/issue46220 opened by mckenzm #46223: asyncio cause infinite loop during debug https://bugs.python.org/issue46223 opened by zillionare #46225: f_lasti behaves differently for lambdas returned from loops https://bugs.python.org/issue46225 opened by nedbat #46226: User specific paths added to System PATH environment variable https://bugs.python.org/issue46226 opened by akrymskiy #46227: add pathlib.Path.walk method https://bugs.python.org/issue46227 opened by Ovsyanka #46232: Client certificates with UniqueIdentifier in the subject break https://bugs.python.org/issue46232 opened by kacper #46234: 3.11: Tracing of decorators now visits the decorator line befo https://bugs.python.org/issue46234 opened by nedbat #46235: Do all ref-counting at once for sequence multiplication https://bugs.python.org/issue46235 opened by Dennis Sweeney #46237: Incorrect line reported in syntax error https://bugs.python.org/issue46237 opened by arian-f #46242: Improve error message when attempting to extend an enum with ` https://bugs.python.org/issue46242 opened by sobolevn #46244: typing._TypeVarLike missing __slots__ https://bugs.python.org/issue46244 opened by ariebovenberg #46245: Add support for dir_fd in shutil.rmtree() https://bugs.python.org/issue46245 opened by serhiy.storchaka #46246: importlib.metadata.DeprecatedList appears to be missing __slot https://bugs.python.org/issue46246 opened by ariebovenberg #46247: in xml.dom.minidom, Node and DocumentLS appear to be missing _ https://bugs.python.org/issue46247 opened by ariebovenberg #46249: [sqlite3] move set lastrowid out of the query loop and enable https://bugs.python.org/issue46249 opened by erlendaasland #46250: 3.10 docs responsive design removes navigation buttons (next, https://bugs.python.org/issue46250 opened by vkvanjavk #46252: SSLWantReadError causes _SelectorSocketTransport to close https://bugs.python.org/issue46252 opened by matan1008 #46253: C API documentation of Py_UNICODE_* character properties macro https://bugs.python.org/issue46253 opened by juliangilbey #46254: Better fitting type for iterating in the trace_init C function https://bugs.python.org/issue46254 opened by uriya1998 #46255: Remove unnecessary check in _IOBase._check*() methods https://bugs.python.org/issue46255 opened by malin #46258: Minor algorithmic improvements for math.isqrt https://bugs.python.org/issue46258 opened by mark.dickinson #46261: [doc] fix inaccuracies in sqlite3.Cursor.lastrowid docs https://bugs.python.org/issue46261 opened by erlendaasland #46264: 'I'.lower() should give non dotted i for LANG=tr_TR https://bugs.python.org/issue46264 opened by fbacher #46265: Error when cross compiling for hardfloat MIPS https://bugs.python.org/issue46265 opened by jefferyto #46267: gzip.compress incorrectly ignores level parameter https://bugs.python.org/issue46267 opened by rhpvorderman #46269: '__new__' is never shown in `dir(SomeEnum)` https://bugs.python.org/issue46269 opened by sobolevn #46270: Comparison operators in Python Tutorial 5.7 https://bugs.python.org/issue46270 opened by realjanpaulus #46271: frozen modules are not regenerated on bytecode magic change wh https://bugs.python.org/issue46271 opened by christian.heimes #46272: Fix bitwise and logical terminology in python.gram https://bugs.python.org/issue46272 opened by rhettinger #46273: Document what asyncio.wait() and asyncio.as_completed() do if https://bugs.python.org/issue46273 opened by termim #46274: Tokenizer module does not handle backslash characters correctl https://bugs.python.org/issue46274 opened by ucodery #46275: caret location for syntax error pointing with f-strings https://bugs.python.org/issue46275 opened by williamnavaraj #46276: ImportError: DLL load failed while importing https://bugs.python.org/issue46276 opened by 89z #46279: [docs] Minor information-ordering issue in __main__ doc https://bugs.python.org/issue46279 opened by ferdnyc #46280: About vulnerabilities in Cpython native code https://bugs.python.org/issue46280 opened by urnotmax #46282: print() docs do not indicate its return value https://bugs.python.org/issue46282 opened by SnoopJeDi #46283: [subinterpreters] Unicode interned strings must not be shared https://bugs.python.org/issue46283 opened by vstinner #46284: DirEntry returns invalid values for is_dir and is_file on NFS https://bugs.python.org/issue46284 opened by Nicolas SURRIBAS #46285: http/server.py wont respect its protocol_version https://bugs.python.org/issue46285 opened by openalmeida #46287: UNC path normalisation issues on Windows https://bugs.python.org/issue46287 opened by steve.dower #46289: AST: FomattedValue conversion's default value should be -1 https://bugs.python.org/issue46289 opened by iyume #46290: Parameter names are inaccurate in dataclasses docs https://bugs.python.org/issue46290 opened by zsol #46291: [doc] First argument to raise can also be BaseException https://bugs.python.org/issue46291 opened by gtitze #46292: Add microseconds to logging.LogRecord https://bugs.python.org/issue46292 opened by jamescasbon #46296: Unreachable condition: `if enum_class._member_type_ is object` https://bugs.python.org/issue46296 opened by sobolevn Most recent 15 issues with no replies (15) ========================================== #46296: Unreachable condition: `if enum_class._member_type_ is object` https://bugs.python.org/issue46296 #46292: Add microseconds to logging.LogRecord https://bugs.python.org/issue46292 #46290: Parameter names are inaccurate in dataclasses docs https://bugs.python.org/issue46290 #46284: DirEntry returns invalid values for is_dir and is_file on NFS https://bugs.python.org/issue46284 #46283: [subinterpreters] Unicode interned strings must not be shared https://bugs.python.org/issue46283 #46279: [docs] Minor information-ordering issue in __main__ doc https://bugs.python.org/issue46279 #46274: Tokenizer module does not handle backslash characters correctl https://bugs.python.org/issue46274 #46273: Document what asyncio.wait() and asyncio.as_completed() do if https://bugs.python.org/issue46273 #46271: frozen modules are not regenerated on bytecode magic change wh https://bugs.python.org/issue46271 #46267: gzip.compress incorrectly ignores level parameter https://bugs.python.org/issue46267 #46265: Error when cross compiling for hardfloat MIPS https://bugs.python.org/issue46265 #46258: Minor algorithmic improvements for math.isqrt https://bugs.python.org/issue46258 #46255: Remove unnecessary check in _IOBase._check*() methods https://bugs.python.org/issue46255 #46254: Better fitting type for iterating in the trace_init C function https://bugs.python.org/issue46254 #46253: C API documentation of Py_UNICODE_* character properties macro https://bugs.python.org/issue46253 Most recent 15 issues waiting for review (15) ============================================= #46296: Unreachable condition: `if enum_class._member_type_ is object` https://bugs.python.org/issue46296 #46292: Add microseconds to logging.LogRecord https://bugs.python.org/issue46292 #46290: Parameter names are inaccurate in dataclasses docs https://bugs.python.org/issue46290 #46282: print() docs do not indicate its return value https://bugs.python.org/issue46282 #46280: About vulnerabilities in Cpython native code https://bugs.python.org/issue46280 #46269: '__new__' is never shown in `dir(SomeEnum)` https://bugs.python.org/issue46269 #46267: gzip.compress incorrectly ignores level parameter https://bugs.python.org/issue46267 #46261: [doc] fix inaccuracies in sqlite3.Cursor.lastrowid docs https://bugs.python.org/issue46261 #46258: Minor algorithmic improvements for math.isqrt https://bugs.python.org/issue46258 #46255: Remove unnecessary check in _IOBase._check*() methods https://bugs.python.org/issue46255 #46254: Better fitting type for iterating in the trace_init C function https://bugs.python.org/issue46254 #46253: C API documentation of Py_UNICODE_* character properties macro https://bugs.python.org/issue46253 #46252: SSLWantReadError causes _SelectorSocketTransport to close https://bugs.python.org/issue46252 #46249: [sqlite3] move set lastrowid out of the query loop and enable https://bugs.python.org/issue46249 #46246: importlib.metadata.DeprecatedList appears to be missing __slot https://bugs.python.org/issue46246 Top 10 most discussed issues (10) ================================= #46280: About vulnerabilities in Cpython native code https://bugs.python.org/issue46280 21 msgs #46070: [subinterpreters] asyncio crash when importing _asyncio in sub https://bugs.python.org/issue46070 17 msgs #45665: Problems caused by isinstance(list[int], type) returning True https://bugs.python.org/issue45665 10 msgs #46249: [sqlite3] move set lastrowid out of the query loop and enable https://bugs.python.org/issue46249 9 msgs #34931: os.path.splitext with more dots https://bugs.python.org/issue34931 8 msgs #45925: Upgrade macOS and Windows installers to use SQLite 3.37.2 https://bugs.python.org/issue45925 7 msgs #46006: [subinterpreter] _PyUnicode_EqualToASCIIId() issue with subint https://bugs.python.org/issue46006 7 msgs #46142: python --help output is too long https://bugs.python.org/issue46142 7 msgs #46187: Optionally support rounding for math.isqrt() https://bugs.python.org/issue46187 6 msgs #46198: Duplicate and unused code in tests https://bugs.python.org/issue46198 6 msgs Issues closed (75) ================== #9937: _winreg.EnumValue causes MemoryError https://bugs.python.org/issue9937 closed by iritkatriel #17951: TypeError during gdb backtracing https://bugs.python.org/issue17951 closed by iritkatriel #20369: concurrent.futures.wait() blocks forever when given duplicate https://bugs.python.org/issue20369 closed by AlexWaygood #25478: Consider adding a normalize() method to collections.Counter() https://bugs.python.org/issue25478 closed by rhettinger #30337: Vague wording of pkgutil.walk_packages parameter 'prefix' https://bugs.python.org/issue30337 closed by iritkatriel #31137: Add a path attribute to NamedTemporaryFile https://bugs.python.org/issue31137 closed by iritkatriel #32696: Fix pickling exceptions with multiple arguments https://bugs.python.org/issue32696 closed by iritkatriel #33130: functools.reduce signature uses `iterable`, documentation shou https://bugs.python.org/issue33130 closed by iritkatriel #33252: [doc] Clarify ResourceWarning documentation https://bugs.python.org/issue33252 closed by iritkatriel #33857: python exception on Solaris : code for hash blake2b was not fo https://bugs.python.org/issue33857 closed by iritkatriel #34538: Remove encouragement to author a base class for all Exception https://bugs.python.org/issue34538 closed by iritkatriel #37489: pickling instance which inherited from Exception with keyword https://bugs.python.org/issue37489 closed by iritkatriel #39181: Add `os.makedirs()` as `Path.mkdir()` equivalent in correspond https://bugs.python.org/issue39181 closed by jack1142 #40222: "Zero cost" exception handling https://bugs.python.org/issue40222 closed by Mark.Shannon #40477: Python Launcher app on macOS 10.15+ fails to run scripts https://bugs.python.org/issue40477 closed by ned.deily #40545: Expose _PyErr_GetTopmostException https://bugs.python.org/issue40545 closed by iritkatriel #42032: Setting PYTHONPYCACHEPREFIX using ~ (tilde) creates a "~" fold https://bugs.python.org/issue42032 closed by petr.viktorin #42657: Python 3.9.1 building process could not use local standard lib https://bugs.python.org/issue42657 closed by ned.deily #43122: Python Launcher doesn't open a terminal window https://bugs.python.org/issue43122 closed by ned.deily #44092: [sqlite3] Remove special rollback handling https://bugs.python.org/issue44092 closed by pablogsal #44669: TypeError: 'type' object is not subscriptable https://bugs.python.org/issue44669 closed by iritkatriel #45256: Remove the usage of the C stack in Python to Python calls https://bugs.python.org/issue45256 closed by Mark.Shannon #45292: Implement PEP 654: Exception Groups https://bugs.python.org/issue45292 closed by iritkatriel #45615: Missing test for type of error when printing traceback for non https://bugs.python.org/issue45615 closed by iritkatriel #45711: Simplify the interpreter's (type, val, tb) exception represent https://bugs.python.org/issue45711 closed by iritkatriel #45903: What��s New In Python 3.11: wrong reference to Signature.from https://bugs.python.org/issue45903 closed by Mariatta #46031: add POP_JUMP_IF_NOT_NONE and POP_JUMP_IF_NONE https://bugs.python.org/issue46031 closed by Mark.Shannon #46039: Break up the YIELD_FROM instruction. https://bugs.python.org/issue46039 closed by Mark.Shannon #46079: [doc] Broken URL in "Brief Tour of the Standard Library" https://bugs.python.org/issue46079 closed by rhettinger #46095: Improve SeqIter documentation https://bugs.python.org/issue46095 closed by rhettinger #46148: Optimize pathlib https://bugs.python.org/issue46148 closed by kumaraditya303 #46178: Remove `.travis.yml`? https://bugs.python.org/issue46178 closed by sobolevn #46181: Destroying an expaned Combobox prevents Entry focus until Alt+ https://bugs.python.org/issue46181 closed by serhiy.storchaka #46182: `super` and descriptor clarification https://bugs.python.org/issue46182 closed by rhettinger #46184: Remove `netlify.toml`? https://bugs.python.org/issue46184 closed by sobolevn #46196: documentation for cmd library should include columnize() funct https://bugs.python.org/issue46196 closed by eric.araujo #46199: Calculation influenced by print https://bugs.python.org/issue46199 closed by mark.dickinson #46202: remove opcode POP_EXCEPT_AND_RERAISE https://bugs.python.org/issue46202 closed by iritkatriel #46206: Crash when editing emoji containing strings https://bugs.python.org/issue46206 closed by pablogsal #46208: os.path.normpath change between 3.11.0a2 and 3.11.0a3+ https://bugs.python.org/issue46208 closed by steve.dower #46215: Wrong error line after using pdb https://bugs.python.org/issue46215 closed by bedapisl #46218: Change long_pow() to sliding window algorithm https://bugs.python.org/issue46218 closed by tim.peters #46219: except* assumes that an exception group is truthy https://bugs.python.org/issue46219 closed by iritkatriel #46221: Simplify except* implementation and move helpers to exceptions https://bugs.python.org/issue46221 closed by iritkatriel #46222: posixmodule sendfile FreeBSD's constants update https://bugs.python.org/issue46222 closed by asvetlov #46224: doc: Fix bisect example using mutable function default https://bugs.python.org/issue46224 closed by rhettinger #46228: argparse docs: default for prog= in ArgumentParser() should be https://bugs.python.org/issue46228 closed by rhettinger #46229: CODE_OF_CONDUCT shadowing org default https://bugs.python.org/issue46229 closed by Mariatta #46230: NEWLINE* is missing in 'file' rule in PEG grammar https://bugs.python.org/issue46230 closed by pablogsal #46231: grammar rule definition is missing: invalid_double_starred_kvp https://bugs.python.org/issue46231 closed by pablogsal #46233: Minor speedup for bigint squaring https://bugs.python.org/issue46233 closed by tim.peters #46236: PyFunction_GetAnnotations returning Tuple vs Dict https://bugs.python.org/issue46236 closed by pablogsal #46238: Improve constants usage / definition in asyncio.windows_events https://bugs.python.org/issue46238 closed by asvetlov #46239: Incosistent error message in asyncio: windows_events / windows https://bugs.python.org/issue46239 closed by asvetlov #46240: Incorrect hint about forgetting a comma https://bugs.python.org/issue46240 closed by pablogsal #46241: importlib.utils.find_spec() has issues with "tests" folder https://bugs.python.org/issue46241 closed by brett.cannon #46243: Spam https://bugs.python.org/issue46243 closed by christian.heimes #46248: Compilation errors on macOS https://bugs.python.org/issue46248 closed by ned.deily #46251: logger.config.configure_formatter executes arbitrary code https://bugs.python.org/issue46251 closed by vinay.sajip #46256: Objects __del__ called after module have been removed https://bugs.python.org/issue46256 closed by pablogsal #46257: Convert statistics sum of squares to a single pass algorithm https://bugs.python.org/issue46257 closed by rhettinger #46259: float formatting error? https://bugs.python.org/issue46259 closed by rhettinger #46260: Misleading SyntaxError on f-string https://bugs.python.org/issue46260 closed by eric.smith #46262: Enum tests: Error path in `_missing_()` is not covered for `Fl https://bugs.python.org/issue46262 closed by ethan.furman #46263: FreeBSD buildbots cannot compile Python https://bugs.python.org/issue46263 closed by christian.heimes #46266: Improve day constants (`MONDAY` ... `SUNDAY`) in `calendar.py` https://bugs.python.org/issue46266 closed by rhettinger #46268: Buildbot failure for buildbot/AMD64 FreeBSD Non-Debug / Shared https://bugs.python.org/issue46268 closed by erlendaasland #46277: '''...''' error https://bugs.python.org/issue46277 closed by vstinner #46278: AbstractEventLoop.call_* should reflect the 'context' argument https://bugs.python.org/issue46278 closed by asvetlov #46281: Python 3.10.1 build errors on Ubuntu 18.04 https://bugs.python.org/issue46281 closed by christian.heimes #46286: use the new POP_JUMP_IF_NOT_NONE to simplify except* a bit https://bugs.python.org/issue46286 closed by iritkatriel #46288: Migrating python 2.7 to 3.6 using 2to3 tool https://bugs.python.org/issue46288 closed by eric.smith #46293: Typo in specifying escape sequence https://bugs.python.org/issue46293 closed by steven.daprano #46294: Integer overflow & Int values loaded into Bool detected via Li https://bugs.python.org/issue46294 closed by swirsz #46295: Subinterpreters hang in GIL adquisition if an extension module https://bugs.python.org/issue46295 closed by pablogsal

1 0

Sanity check about ctypes
by Yonatan Zunger Jan. 6, 2022

Jan. 6, 2022

Hey everyone. Quick sanity check: The ctypes docs <https://docs.python.org/3.10/library/ctypes.html#ctypes._CData> refer to _CData as a non-public class which is in the module, but _ctypes.c doesn't actually export it <https://github.com/python/cpython/blob/main/Modules/_ctypes/_ctypes.c#L5680>. (I discovered this because it turns out that typeshed *is* referencing _CData, e.g. in its type annotations for RawIOBase <https://github.com/python/typeshed/blob/master/stdlib/_typeshed/__init__.py…> ) Is this intended behavior in CPython (in which case the docs are a bit off and typeshed has a bug), or is it unexpected to people on this list (in which case it's an issue in _ctypes.c)? Yonatan

2 2

Python 3.11.0a4 is blocked
by Pablo Galindo Salgado Jan. 6, 2022

Jan. 6, 2022

Hi everyone, I am writing this to notify you that unfortunately the release of 3.11.0a4 is blocked as there are a bunch of release blockers (some of them affect Python 3.10): https://bugs.python.org/issue46263 https://bugs.python.org/issue46208 https://bugs.python.org/issue46006 https://bugs.python.org/issue43683 If this was a single release blocker I would think about moving forward but unfortunately, there are several of them and one of them is that Python fails to compile FreeBSD, so I am halting the release until these are fixed. Regards from rainy London, Pablo Galindo Salgado

3 3

my plans for subinterpreters (and a per-interpreter GIL)
by Eric Snow Jan. 6, 2022

Jan. 6, 2022

Hi all, I'm still hoping to land a per-interpreter GIL for 3.11. There is still a decent amount of work to be done but little of it will require solving any big problems: * pull remaining static globals into _PyRuntimeState and PyInterpreterState * minor updates to PEP 554 * finish up the last couple pieces of the PEP 554 implementation * maybe publish a companion PEP about per-interpreter GIL There are also a few decisions to be made. I'll open a couple of other threads to get feedback on those. Here I'd like your thoughts on the following: Do we need a PEP about per-interpreter GIL? I haven't thought there would be much value in such a PEP. There doesn't seem to be any decision that needs to be made. At best the PEP would be an explanation of the project, where: * the objective has gotten a lot of support (and we're working on addressing the concerns of the few objectors) * most of the required work is worth doing regardless (e.g. improve runtime init/fini, eliminate static globals) * the performance impact is likely to be a net improvement * it is fully backward compatible and the C-API is essentially unaffected So the value of a PEP would be in consolidating an explanation of the project into a single document. It seems like a poor fit for a PEP. (You might wonder, "what about PEP 554?" I purposefully avoided any discussion of the GIL in PEP 554. It's purpose is to expose subinterpreters to Python code.) However, perhaps I'm too close to it all. I'd like your thoughts on the matter. Thanks! -eric

14 29

Function Prototypes
by asleep.cult＠gmail.com Jan. 5, 2022

Jan. 5, 2022

Hello In thread about PEP 677, I (with the help of another proposal) came up with an alternative to typing.Callable called function prototypes. The basic concept is to create a function prototype object when the body of a function is omitted. The thread can be found here: https://mail.python.org/archives/list/python-dev@python.org/thread/OGACYN2X… Mark Shannon initially proposed that functions be used as types and provided this example: @Callable def IntToIntFunc(a:int)->int: pass def flat_map( l: list[int], func: IntToIntFunc ) -> list[int]: .... I further proposed that we make the body of a function non-mandatory and create a function prototype if it is omitted. I provided these examples: import typing @typing.Callable def IntToIntFunc(a: int) -> int def flat_map( l: list[int], func: IntToIntFunc ) -> list[int]: ... import ctypes @ctypes.CFUNCTYPE def f(x: int) -> bool I have since taken it upon myself to implement this in a fork of cpython: https://github.com/asleep-cult/cpython To remain consistent with function definitions, I have also added an alternative lambda syntax that allows you to annotate arguments and the return type. The syntax requires you to parenthesize the argument list: lambda (a: int, b: int) -> int: ... This new lambda syntax also allows you to create a function prototype by omitting the body. The original example can be rewritten as follows: def flat_map( l: list[int], func: lambda (a: int) -> int ) -> list[int]: ... Problems: - It is not possible to use ParamSpec with this - The lambda prototype syntax might be jarring - Some people might accidentally forget the function body Here is some feedback that I have already collected: "Yeah, making the body optional (without looking at decorators) is not acceptable either. Too easy to do by mistake (I still do this All. The. Time. :-)" - Guido van Rossum "i would strongly prefer this over the existing pep" - Ronny Pfannschmidt "I find this unnecessary and unreadable. Python isn't C or Java." - BundleOfJoysticks (Reddit)

14 27

RFC on Callable Syntax PEP
by Steven Troxler Jan. 4, 2022

Jan. 4, 2022

Hello all, Thanks everyone for comments on our earlier thread [1] about callable type syntax. We now have a draft PEP [2] proposing an arrow-based syntax for callable types, for example: ``` (int, str) -> bool # equivalent to Callable[[int, str], bool] ``` In support of the PEP we also have: - a reference implementation of the parser [3] to ensure the grammar is correct (tests [5], [6], [7]) - a detailed specification of planned runtime behavior [4], which is not yet in the reference implementation We'd like to get your feedback about the PEP in general, and especially details and edge cases we need to consider regarding runtime behavior. Cheers, Steven Troxler --------- [1] Earlier python-dev thread https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3L… [2] PEP 677: https://www.python.org/dev/peps/pep-0677/ [3] Reference implementation of Parser: https://github.com/stroxler/cpython/tree/callable-type-syntax--shorthand [4] Details on the runtime behavior: https://docs.google.com/document/d/15nmTDA_39Lo-EULQQwdwYx_Q1IYX4dD5WPnHbFG… [5] Ast tests for parser changes: https://github.com/stroxler/cpython/blob/20eb59fdca0d6d8dbe4efa3b04038c7c22… [6] Easy-read tests of examples from the PEP: https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Li… [7] Test sanity checking hundreds of examples pulled from typeshed: https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Li…

13 22