Mailman 3 - Python-Dev

Python 3.12.0 alpha 2 released
by Thomas Wouters Nov. 15, 2022

Nov. 15, 2022

I'm pleased to announce the release of Python 3.12 alpha 2. https://www.python.org/downloads/release/python-3120a2/ *This is an early developer preview of Python 3.12*. Major new features of the 3.12 series, compared to 3.11 Python 3.12 is still in development. This release, 3.12.0a2 is the second of seven planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process. During the alpha phase, features may be added up until the start of the beta phase (2023-05-08) and, if necessary, may be modified or deleted up until the release candidate phase (2023-07-31). Please keep in mind that this is a preview release and its use is **not** recommended for production environments. Many new features for Python 3.12 are still being planned and written. Among the new major new features and changes so far: - Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user. - Support for the Linux perf profiler to report Python function names in traces. - The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623 <https://peps.python.org/pep-0623>. - In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2). - The deprecated smtpd and distutils modules have been removed (see PEP 594 <https://peps.python.org/pep-0594/> and PEP 632 <https://peps.python.org/pep-0632/>). The setuptools package (installed by default in virtualenvs and many other places) continues to provide the distutils module. - A number of other old, broken and deprecated functions, classes and methods have been removed. - (Hey, **fellow core developer,** if a feature you find important is missing from this list, let Thomas know <thomas(a)python.org>.) For more details on the changes to Python 3.12, see What's new in Python 3.12 <https://docs.python.org/dev/whatsnew/3.12.html>. The next pre-release of Python 3.12 will be 3.12.0a3, currently scheduled for 2022-12-05. More resources Online Documentation <https://docs.python.org/3.12/>. PEP 693 <https://www.python.org/dev/peps/pep-0693/>, the Python 3.12 Release Schedule. Report bugs via GitHub Issues <https://github.com/python/cpython/issues>. Help fund Python and its community <https://www.python.org/psf/donations/>. And now for something completely different Life, believe, is not a dream > So dark as sages say; > Oft a little morning rain > Foretells a pleasant day. > Sometimes there are clouds of gloom, > But these are transient all; > If the shower will make the roses bloom, > O why lament its fall? > > Rapidly, merrily, > Life's sunny hours flit by, > Gratefully, cheerily, > Enjoy them as they fly! > > What though Death at times steps in > And calls our Best away? > What though sorrow seems to win, > O'er hope, a heavy sway? > Yet hope again elastic springs, > Unconquered, though she fell; > Still buoyant are her golden wings, > Still strong to bear us well. > > Manfully, fearlessly, > The day of trial bear, > For gloriously, victoriously, > Can courage quell despair! *Life*, by Charlotte Brontë, from *Poems by Currer, Ellis, and Acton Bell*. Charlotte wrote about the publishing under pseudonyms by her and her sisters, Emily and Anne, in a preface to Emily's *Wuthering Heights*: Averse to personal publicity, we veiled our own names under those of > Currer, Ellis and Acton Bell; the ambiguous choice being dictated by a sort > of conscientious scruple at assuming Christian names positively masculine, > while we did not like to declare ourselves women, because – without at > that time suspecting that our mode of writing and thinking was not what is > called "feminine" – we had a vague impression that authoresses are liable > to be looked on with prejudice; we had noticed how critics sometimes use > for their chastisement the weapon of personality, and for their reward, a > flattery, which is not true praise. Enjoy the new releases Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Regards from chilly Amsterdam, Your release team, Thomas Wouters Ned Deily Steve Dower -- Thomas Wouters <thomas(a)python.org>

1 0

[CVE-2022-37454] SHA3 vulnerability and upcoming Python patches for 3.7 - 3.10
by mark_topham＠rapid7.com Nov. 10, 2022

Nov. 10, 2022

I’m looking for help understanding how Python will release fixes related to the SHA3 critical security vulnerability (CVE-2022-37454). I’ve tried to figure this out myself, but I’m far from a Python expert and I’m not sure where else I should look. Apologies in advance if this is the wrong place to ask - if it is, a redirect to the correct place would be most appreciated. Here’s what I’ve found so far: * Python versions 3.6 through 3.10 appear to be affected * 3.6 is end of life, so no fix is expected * A code fix appears to have been applied to 3.7 through 3.10 https://github.com/python/cpython/issues/98517 * 3.9 and 3.10 by default use OpenSSL1.1.1+ if it’s available, appearing to default to the builtin, vulnerable SHA3 implementation if OpenSSL is not found (if there’s an exception) * 3.9 introduced this change via bpo-37630 in release 3.9.0 beta1 * 3.10 appears to have had this functionality since it was originally released * 3.11 uses tiny_sha3 and AFAICT was never affected by the CVE But what I’m having trouble figuring out is when/how these fixes will become generally available and ready for users of Python to download. * When will patched releases for Python 3.7-3.10 be released? * If pending releases are not in the release pipeline, what other patching opportunities exist? Ultimately I’m trying to set patching expectations for my company’s engineering teams who are still running vulnerable versions of Python. More notes around what i’ve found, in case it helps clarify my questions: From the Python project GitHub I can see gh-98517 to fix the buffer overflow in Python’s internal _sha3 module (CVE-2022-37454) has been committed to the Python 3.7 - 3.10 branches. I understand that for Python releases 3.9 and 3.10 if one is using the OpenSSL 1.1.1+ sha3 modules instead of the internal _sha3 module that is already a mitigation. I also understand that Python 3.11 and later has switched to using tiny_sha3, and no longer relies on the vulnerable _sha3 module. Any information you could point me at would be most helpful. If there is a more ideal forum to raise this question, please redirect me there. Thank you in advance

5 5

August steering council update
by Pablo Galindo Salgado Nov. 9, 2022

Nov. 9, 2022

Below, you can find the just-published Steering Council update for August. September and October coming soon. Apologies for the delay! https://github.com/python/steering-council/blob/main/updates/2022-08-steeri… As a reminder, if you have any questions or concerns, feel free to contact us or open an issue in the SC repo: https://github.com/python/steering-council # SC meeting summary - August 2022 ## 2022-08-01 * The Steering Council met with the Developer-in-Residence (Łukasz) and discussed: * The status of the 3.11 release * the asyncio uncancel issue * The number of last-minute bugs and issues that we are experiencing * The SC discussed the fall Core Dev Sprint with the PSF Executive Director, as well as plans for another Developer-in-Residence and offering mentoring support to core developers. * The SC discussed the Python 3.11 release and the approximate cause of the release blockers, as well as how to improve the situation in the future and how to gather feedback for future release managers. ## 2022-08-08 * The SC discussed the status and next steps for CVE-2020-10735: Prevent DoS by large int<->str conversions. * Pablo asked the Docs working group for assistance with the 3.11 release notes. * The SC discussed the status of the repo for planet.python.org. The SC will ask Ee about moving the planet repo over to the PSF organization. * The SC discussed how to move forward with the future of type annotations ([PEP 563 – Postponed Evaluation of Annotations](https://peps.python.org/pep-0563/) and [PEP 649 – Deferred Evaluation Of Annotations Using Descriptors](https://peps.python.org/pep-0649/)). Several maintenance considerations and deprecation strategies were considered, listing things that would break and ways to overcome the challenges. * The SC discussed the state of [PEP 2 – Procedure for Adding New Modules](https://peps.python.org/pep-0002/), [PEP 4 – Deprecation of Standard Modules](https://peps.python.org/pep-0004/), and [PEP 411 – Provisional packages in the Python standard library](https://peps.python.org/pep-0411/). The SC decided to email the community to see if they think we should have separate policy documents from PEP. * The SC discussed asking the platform release managers (Ned and Steve) to document their side of the release process to reduce the bus factor and search for more automation possibilities. ## 2022-08-15 * The SC met with the Developer in Residence and discussed: * Fixing blockers for the 3.11.0rc1 release * An update of the CLA bot, and a future update to EdgeDB 2.0 * Possible candidates for blog posts for the community * The SC met with the PSF Executive Director and discussed the organization of the 2022 Core Dev sprints. * The SC discussed the logistics of having a meeting with Meta as part of the Visionary sponsor relationship. * The SC discussed encouraging projects to release wheels for Release Candidates of new minor versions and what challenges are being faced by users and maintainers. Several possible proposals were listed, including asking NumFocus and other organizations for assistance. ## 2022-08-22 * The SC briefly discussed [PEP 649 – Deferred Evaluation Of Annotations Using Descriptors](https://peps.python.org/pep-0649/) and what is left to decide. * The SC discussed several aspects of the upcoming core dev sprint. * The SC discussed planning of the meeting with Meta as a PSF Visionary sponsor benefit. * The SC discussed possibly delaying 3.11.0 due to a timing conflict with release managers and the core dev sprint and life in general. It was agreed that it is ok to delay 3.11.0 until the 24th of October. * The SC confirmed that 3.12alpha1 is still intended during the dev sprint. * As a way to improve the testing of new versions, Brett noted he's planning to work to improve GitHub CI configs to make it more likely for users to test against release candidates and betas and to make it obvious how to do it. * The SC discussed the migration concerns raised from the proposed migration for PEP discussion from python-dev to discuss.python.org migration. The SC noted that documentation work remains before making it fully official. Also, we won't shut down the python-dev list, but we expect PEP authors to not pay attention to it when discussing PEPs. * The SC discussed the possibility of inviting some non-core devs to the sprint and we agreed that a goal of focus is key for a successful sprint. * The SC discussed what to do with PEPs that are not regular enhancement proposals nor informative documents (like https://peps.python.org/pep-0659/). The SC decided that for [PEP 659– Specializing Adaptive Interpreter](https://peps.python.org/pep-0659/) in particular the status should be Final. * The SC agreed that we want to avoid having people write a PEP after pushing through an implementation. * The SC discussed CPython's backward compatibility policy and decided that we should gather input from the core devs on that. It was proposed to ask people attending the core dev sprint as a first measure. ## 2022-08-22 * The SC met with the Developer in Residence and discussed: * Bot/CLA hosting and Heroku tiers. * EdgeDB update for the CLA bot. * Blog posts and talk ideas. * Typing PEPs. * The SC discussed details about the organization of the 2022 Core Dev sprints. * The SC discussed resources for mentoring and how to help core devs mentor new contributors. * The SC discussed [PEP 674 – Disallow using macros as l-values](https://peps.python.org/pep-0674/) and the changes that were merged without a PEP accepted in 3.11. Pablo will answer [to the discussion that Victor started in discourse](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-…. * The SC discussed several aspects on how to deal with breaking changes. It was decided that this will be a discussion point with the rest of the core devs in the core dev sprint. In representation of the Steering Council, Pablo Galindo Salgado

1 0

NEWLINE sentinel behavior in CPython's PEG grammar
by David J W Nov. 3, 2022

Nov. 3, 2022

I am writing a Rust version of Python for fun and I am at the parser stage of development. I copied and modified a PEG grammar ruleset from another open source project and I've already noticed some problems (ex Newline vs NL) with how they transcribed things. I am suspecting that CPython's grammar NEWLINE is a builtin rule for the parser that is something like `(Newline+ | NL+ ) {NOP}` but wanted to sanity check if that is right before I figure out how to hack in a NEWLINE rule and update my grammar ruleset.

5 9

TypedMapping structural type
by Alice Purcell Nov. 2, 2022

Nov. 2, 2022

I would like to propose adding a new TypedMapping structural type, which (like Mapping vs dict) functions as a read-only equivalent of TypedDict. (This is a cross-post from typing-sig@, where our original proposal got refined.) # Specification ```python class Request(TypedMapping, total=False): some_field: string | None request: Request = { "some_field": "foo" } request2: Request = {} # fine because total=False request["some_field"] = None # error del request["some_field"] # error ``` TypedMapping is a structural type that mirrors TypedDict except for: 1) instances need not be subclasses of dict; 2) no mutate methods will be generated 3) subclasses can narrow field types (consequence of 2) Rules for 3 mirror Protocols. ## Multiple inheritance and TypedDict A type that inherits from a TypedMapping subclass and from TypedDict (either directly or indirectly): 4) is the structural intersection of its parents, or invalid if no such intersection exists 5) instances must be a dict subclass 6) adds mutate methods only for fields it explicitly (re)declares ```python class A(TypedMapping): field1: int field2: str class B(A, TypedDict): field1: int b: B = { "field1": 5, "field2": "value" } b["field1"] = 6 # Fine, mutate methods added in definition of B b["field2"] = "value2" # Error, field2 mutator not declared ``` # Use case We are making increasing use of TypedDict to type hint code that uses dictionaries to hold requests to and responses from services (both client and server side). However, we run into problems whenever any constraint is loosened, e.g. passing data from a response where a field is guaranteed to a request where it is optional: ```python class Response(TypedDict): some_field: string class Request(TypedDict, total=False): some_field: string | None class Server1: def fetch_data() -> Response: ... class Server2: def accept_data(request: Request): ... def my_code(server1: Server1, server2: Server2): data: Response = server1.fetch_data() server2.accept_data(data) # error: Response and Request are incompatible ```python This last line is not type-safe, as there are mutation operations on Request that aren't valid on Response. If we could remove those mutation operations from the Request type, then it would be a valid subtype of Response. This problem seems quite common, e.g. https://github.com/python/mypy/pull/12142. Pablo Galindo has agreed to sponsor a PEP.

1 0

Possible bug in `re` module in Python 3.11?
by Piotr Waszkiewicz Oct. 26, 2022

Oct. 26, 2022

Hi, I would like to ask your guidance as I'm entirely sure whether the problem I'm experiencing should be posted in CPython's repo as a bug issue. I've tried using newly released Python 3.11 interpreter in some of my projects and one of them failed to start with "RuntimeError: invalid SRE code" error. Looking into implementation of one of the dependencies I've found out that the issue has been found out by a maintainer and fixed (https://github.com/pydicom/pydicom/issues/1658). It looks like this particular regexp caused the `re.compile()` method to raise: ``` re.compile( r"(?P<h>^([01][0-9]|2[0-3]))" r"((?P<m>([0-5][0-9]))?" r"(?(5)(?P<s>([0-5][0-9]|60))?)" r"(?(7)(\.(?P<ms>([0-9]{1,6})?))?))$" ) ``` I've checked and this hasn't been an issue in all previous Python interpreter versions, starting from 3.6 (the oldest I've checked). What's more the regex i correctly recognized and does not cause any issues in other regexp implementations, e.g. the online tool https://regex101.com/ Could somebody help me decide whether this is indeed a bug? Best regards, Piotr Waszkiewicz

3 3

Thank you for your contributions to Python 3.11!
by Pablo Galindo Salgado Oct. 26, 2022

Oct. 26, 2022

Hi everyone, Now that the 3.11.0 release is finally done and I can relax a bit, I just wanted to thank you all for your fantastic work that has made Python 3.11 such a fantastic release. No matter if you committed code to 3.11 or opened a bug, helped with the documentation, reviewed pull requests, participated in discussions, made a PEP or help writing one, fixed a bug or one hundred or make optimizations to the interpreter or any other of the many ways to contribute. Your work makes a huge difference and Python 3.11 is much better because of that :) Also, I want to especially thank all core devs and contributors that have helped me and the release team take care of release blockers, buildbot failures, CVE patches, and any other form of release crisis. Thank you! Finally, a huge thanks to my colleagues in the release team that make these releases possible and help to make sure that my mistakes are not too obvious to end users :P Being your release manager for 3.11 and 3.10 has been a privilege and an honor (and it will continue for a couple of years of bugfixes and security releases, I'm not going anywhere). Regards from rainy London, Pablo Galindo Salgado

1 0

problem with Distributed File System Replication and Namespacing and different versions of Python 3
by rainonthescarecrowhumanwheels＠gmail.com Oct. 26, 2022

Oct. 26, 2022

Hi, I have a question about a problem we are facing in different Python versions, like Python 3.7.7, Python 3.8, Python 3.6 and Python 3.9. This apparently doesnt happen in Python 3.10.2 and we were wondering how we could achieve the same behaviours without going deep into the nuts and bolts of the python import system. We have python libraries that are distributed in a networked environment that when imported for the first time they are correctly imported but something happens in the meantime that prevents it from loading anymore. This networked paths are domain paths that are bound to a final "real" path in a specific file server from a specific site in the world. What happens is, when injecting into the sys.path the domain names it doesnt import but when injecting into the sys.path the "real" file server path it works, generally speaking. We have been facing this issue in such different python 3 versions and i was wondering what makes this work in Python 3.10.2. Apparently, in my opinion it is a mix of two things involved: one, something the DFS-R DFS-N does at a very low level, that python 3 versions dont like when importing and the other one, the fact that is a network path. Can anybody shed some light about the improvements regarding this? We will be tied to all these python versions for a very long time and we need to give solution to this. One of the possibilities would be to distribute the libraries into C:/ local disc, this eliminating the network path variable. But it would be nice to know what is causing this in Python < 3.10.2. Thanks in advance.

6 9

Python 3.12.0 alpha 1 released.
by Thomas Wouters Oct. 24, 2022

Oct. 24, 2022

As Pablo released Python 3.11.0 final earlier today, now it's my turn to release Python 3.12.0 alpha 1. *This is an early developer preview of Python 3.12* Major new features of the 3.12 series, compared to 3.11 Python 3.12 is still in development. This release, 3.12.0a1 is the first of seven planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process. During the alpha phase, features may be added up until the start of the beta phase (2023-05-08) and, if necessary, may be modified or deleted up until the release candidate phase (2023-07-31). Please keep in mind that this is a preview release and its use is *not *recommended for production environments. Many new features for Python 3.12 are still being planned and written. Among the new major new features and changes so far: - The deprecated `wstr` and `wstr_length` members of the C implementation of unicode objects were removed, per PEP 623 <https://peps.python.org/pep-0623/>. - In the `unittest` module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2). - The deprecated `smtpd` module has been removed. - A number of other old, broken and deprecated functions, classes and methods have been removed. - (Hey, **fellow core developer,** if a feature you find important is missing from this list, let Thomas know <thomas(a)python.org>.) The next pre-release of Python 3.12 will be 3.12.0a2, currently scheduled for 2022-11-14. More resources - Online Documentation <https://docs.python.org/3.12> - PEP 693 <https://www.python.org/dev/peps/pep-0693/>, the 3.12 Release Schedule - Report bugs at https://github.com/python/cpython/issues. - Help fund Python and its community at https://www.python.org/psf/donations/. And now for something completely different This is Not the Poem that I Had Hoped to Write <https://twitter.com/brian_bilston/status/1579378460610662401> This is not the poem that I had hoped to write when I sat at my desk and the page was white. You see, there were other words that I’d had in mind, yet this is what I leave behind. I thought it was a poem to eradicate war; one of such power, it would heal all the sores of a world torn apart by conflict and schism. But it isn’t. Lovers, I’d imagined, would quote from it daily, Mothers would sing it to soothe crying babies. And whole generations would be given new hope. Nope. I had grand aspirations. Believe me, I tried. Humanity examined with lessons applied. But the right words escaped me; so often they do. Have these in lieu. Brian Bilston <https://twitter.com/brian_bilston> Enjoy the new releases Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Regards from dusky California, Your release team, Thomas Wouters @Yhg1s Ned Deily @nad Steve Dower @steve.dower -- Thomas Wouters <thomas(a)python.org>

1 0

[RELEASE] Python 3.11 final (3.11.0) is available
by Pablo Galindo Salgado Oct. 24, 2022

Oct. 24, 2022

Python 3.11 is finally released. In the CPython release team, we have put a lot of effort into making 3.11 the best version of Python possible. Better tracebacks, faster Python, exception groups and except*, typing improvements and much more. Get it here: https://www.python.org/downloads/release/python-3110/ ## This is the stable release of Python 3.11.0 Python 3.11.0 is the newest major release of the Python programming language, and it contains many new features and optimizations. # Major new features of the 3.11 series, compared to 3.10 Some of the new major new features and changes in Python 3.11 are: ## General changes * [PEP 657](https://www.python.org/dev/peps/pep-0657/) -- Include Fine-Grained Error Locations in Tracebacks * [PEP 654](https://www.python.org/dev/peps/pep-0654/) -- Exception Groups and `except*` * [PEP 680](https://www.python.org/dev/peps/pep-0680/) -- tomllib: Support for Parsing TOML in the Standard Library * [gh-90908](https://github.com/python/cpython/issues/90908) -- Introduce task groups to asyncio * [gh-34627](https://github.com/python/cpython/issues/34627/) -- Atomic grouping (`(?>...)`) and possessive quantifiers (`*+, ++, ?+, {m,n}+`) are now supported in regular expressions. * The [Faster CPython Project](https://github.com/faster-cpython/) is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See [Faster CPython]( https://docs.python.org/3.11/whatsnew/3.11.html#faster-cpython) for details. ## Typing and typing language changes * [PEP 673](https://www.python.org/dev/peps/pep-0673/) -- Self Type * [PEP 646](https://www.python.org/dev/peps/pep-0646/) -- Variadic Generics * [PEP 675](https://www.python.org/dev/peps/pep-0675/) -- Arbitrary Literal String Type * [PEP 655](https://www.python.org/dev/peps/pep-0655/) -- Marking individual TypedDict items as required or potentially-missing * [PEP 681](https://www.python.org/dev/peps/pep-0681/) -- Data Class Transforms # More resources * [Online Documentation](https://docs.python.org/3.11/) * [PEP 664](https://www.python.org/dev/peps/pep-0664/), 3.11 Release Schedule * Report bugs at [ https://github.com/python/cpython/issues](https://github.com/python/cpython… . * [Help fund Python and its community](/psf/donations/). # And now for something completely different When a spherical non-rotating body of a critical radius collapses under its own gravitation under general relativity, theory suggests it will collapse to a single point. This is not the case with a rotating black hole (a Kerr black hole). With a fluid rotating body, its distribution of mass is not spherical (it shows an equatorial bulge), and it has angular momentum. Since a point cannot support rotation or angular momentum in classical physics (general relativity being a classical theory), the minimal shape of the singularity that can support these properties is instead a ring with zero thickness but non-zero radius, and this is referred to as a ringularity or Kerr singularity. This kind of singularity has the following peculiar property. The spacetime allows a geodesic curve (describing the movement of observers and photons in spacetime) to pass through the center of this ring singularity. The region beyond permits closed time-like curves. Since the trajectory of observers and particles in general relativity are described by time-like curves, it is possible for observers in this region to return to their past. This interior solution is not likely to be physical and is considered a purely mathematical artefact. There are some other interesting free-fall trajectories. For example, there is a point in the axis of symmetry that has the property that if an observer is below this point, the pull from the singularity will force the observer to pass through the middle of the ring singularity to the region with closed time-like curves and it will experience repulsive gravity that will push it back to the original region, but then it will experience the pull from the singularity again and will repeat this process forever. This is, of course, only if the extreme gravity doesn’t destroy the observer first. # We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. https://www.python.org/psf/ If you have any questions, please reach out to me or another member of the release team :) Your friendly release team, Ned Deily @nad https://discuss.python.org/u/nad Steve Dower @steve.dower https://discuss.python.org/u/steve.dower Pablo Galindo Salgado @pablogsal https://discuss.python.org/u/pablogsal

1 0