Mailman 3 - Python-Dev

IMPORTANT: Check the 3.11.0 cherry-picks
by Pablo Galindo Salgado Oct. 24, 2022

Oct. 24, 2022

Hi everyone, I emerged from cherry-picking hell! As mentioned previously, the 3.11.0 final release will be done from the "branch-v3.11.0" branch and will contain a bunch of cherry-picked commits on top of v3.11.0rc2. These commits are: * All documentation commits that **do not touch** any source code (120+ commits). * The following bugfixes: + 6b999982cb8 gh-95027: Fix regrtest stdout encoding on Windows (GH-98492) + 970c10aa6d0 gh-97616: list_resize() checks for integer overflow (GH-97617) + 67f5d24e44c gh-96848: Fix -X int_max_str_digits option parsing (GH-96988) + 9e008fe3519 gh-96821: Fix undefined behaviour in `_testcapimodule.c` (GH-96915) (GH-96927) + bac61bc5b13 gh-95778: Mention sys.set_int_max_str_digits() in error message (GH-96874) + 9cb7324e8fc [3.11] gh-96587: Raise `SyntaxError` for PEP654 on older `feature_version` (GH-96588) (#96591) + 84fd4a54a61 [3.11] gh-97897: Prevent os.mkfifo and os.mknod segfaults with macOS 13 SDK (GH-97944) (#97969) + 1a788914ca6 gh-96865: [Enum] fix Flag to use CONFORM boundary (GH-97528) + c95433573ac [3.11] gh-98331: Update bundled pip to 22.3 (GH-98332) (gh-98400) + fc127628d58 gh-98414: py.exe launcher does not use defaults for -V:company/ option (GH-98460) + 585c95df957 [3.11] GH-97752: Clear the previous member of newly-created generator/coroutine frames (GH-97812) + 4e0fda59f1f gh-98360: multiprocessing now spawns children on Windows with correct argv[0] in virtual environments (GH-98462) + 4c0c1e201a8 [3.11] gh-97514: Don't use Linux abstract sockets for multiprocessing (GH-98501) (GH-98502) + d0ab10f6f01 [3.11] GH-97002: Prevent _PyInterpreterFrames from backing more than one PyFrameObject (GH-98002) + 154b3cd7516 GH-96975: Skip incomplete frames in PyEval_GetFrame (GH-97018) Please *check these commits* and let me know ASAP if we are missing something you would like to include on the 3.11.0 final release. You have until 15:00 UTC+0 today to let me know, otherwise, your changes will need to wait until 3.11.1. Thanks for your help! Regards from sunny London, Pablo Galindo Salgado

1 0

RELEASE MANAGEMENT: Python 3.11 release stream
by Pablo Galindo Salgado Oct. 22, 2022

Oct. 22, 2022

Hi everyone, I am currently swamped fixing merge conflicts from the cherry-picking to the final release candidate but I wanted to write to tell you I'm excited to confirm that we will be haven a Python 3.11 release party as we did last year with 3.10. The event will be co-hosted by the fantastic people of the Python discord channel. The URL for the stream is this one: https://www.youtube.com/watch?v=PGZPSWZSkJI *The release party will start on October 24th at 17:00 UTC+0.* In this stream, some core developers (Irit, Mark, Brandt, Thomas and Łukasz) will be talking about Python 3.11 features, the work on the faster CPython project, some sneak peek on Python 3.12 and beyond, talking about the core Dev in residence role and other things with some time for moderated Q&A. I will be doing the release live and will also cover what I am doing. Is a fantastic opportunity to watch me break everything live :) Who knows, maybe I break GitHub..... again! :D In the end, people from the audience will be able to ask questions in a moderated Q&A, so it would be fantastic if you can be there to maybe answer some questions from the comments or complete the ones we give (or correct us if we say something wrong). Or you can just hang out quietly. Whatever you prefer is good! The main purpose of the stream is to raise excitement about Python 3.11, explain the new features to users, try to get the community closer to the core Dev team and allow them to ask questions to the core dev team. Also, funny party hats will be used! I hope you find the event interesting and consider attending. Python 3.11 is going to be a fantastic release and we want it to be even better :) Please, reach out to me if you have any questions or suggestions. Regards from rainy London, Pablo Galindo Salgado

1 0

How about using modern C++ in development of CPython ?
by redradist＠gmail.com Oct. 19, 2022

Oct. 19, 2022

Hi all, What about of using modern C++ in developing CPython ? With C++ we can get the following benefits: 1) Readability - less code, most code is hidden by abstraction without losing performance 2) Maintainability - no code duplication in favor of using reusable classes 3) RAII - Resource Acquisition Is Initialization, predictable allocation and free resources ...

23 51

[RELEASE] Python versions 3.10.8, 3.9.15, 3.8.15, 3.7.15 now available
by Łukasz Langa Oct. 11, 2022

Oct. 11, 2022

Déjà vu? Right, a month after the expedited releases we are doing the dance again. This coincides with the regular scheduled time for 3.10.8 but since we accrued a few fixes in 3.7 - 3.9 as well, we’re again releasing all four editions at the same time. We’re not promising to continue at this pace 😅 <https://discuss.python.org/t/python-versions-3-10-8-3-9-15-3-8-15-3-7-15-no…>Security content this time CVE-2022-40674: bundled libexpat was upgraded from 2.4.7 to 2.4.9 which fixes a heap use-after-free vulnerability in function doContent gh-97616: a fix for a possible buffer overflow in list *= int gh-97612: a fix for possible shell injection in the example script get-remote-certificate.py (this issue originally had a CVE assigned to it, which its author withdrew) gh-96577: a fix for a potential buffer overrun in msilib <https://discuss.python.org/t/python-versions-3-10-8-3-9-15-3-8-15-3-7-15-no…>Python 3.10.8 Get it here: https://www.python.org/downloads/release/python-3108/ <https://www.python.org/downloads/release/python-3108/> As a bugfix release coming a mere month after an out-of-schedule security release, 3.10.8 is somewhat smaller compared to 3.9.8 released at the same stage of the release cycle a year ago. There’s 151 commits vs 204 in 3.9. It’s still a larger release than 3.10.7 at 113 commits. One way or the other, it’s worth checking out the change log <https://docs.python.org/release/3.10.8/whatsnew/changelog.html>. <https://discuss.python.org/t/python-versions-3-10-8-3-9-15-3-8-15-3-7-15-no…>And now for something completely different Granular convection is a phenomenon where granular material subjected to shaking or vibration will exhibit circulation patterns similar to types of fluid convection. It is sometimes described as the Brazil nut effect when the largest particles end up on the surface of a granular material containing a mixture of variously sized objects; this derives from the example of a typical container of mixed nuts, where the largest will be Brazil nuts. The phenomenon is also known as the muesli effect since it is seen in packets of breakfast cereal containing particles of different sizes but similar densities, such as muesli mix. Under experimental conditions, granular convection of variously sized particles has been observed forming convection cells similar to fluid motion. <https://discuss.python.org/t/python-versions-3-10-8-3-9-15-3-8-15-3-7-15-no…>We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Your friendly release team, Ned Deily @nad <https://discuss.python.org/u/nad> Steve Dower @steve.dower <https://discuss.python.org/u/steve.dower> Pablo Galindo Salgado @pablogsal <https://discuss.python.org/u/pablogsal> Łukasz Langa @ambv <https://discuss.python.org/u/ambv>

1 0

Embedded Python Memory Leaks
by alexei73＠hotmail.com Oct. 8, 2022

Oct. 8, 2022

Hi Python team Is any plan to try and fix memory leaks when embedding Python in a C/C++ application? Thanks in advance, Alexei

3 2

PEP 699: Remove private dict version field added in PEP 509
by Ken Jin Oct. 5, 2022

Oct. 5, 2022

I've created a new PEP to repeal PEP 509. Please post comments to the Discourse thread. Thank you! Discourse thread: https://discuss.python.org/t/pep-699-remove-private-dict-version-field-adde… Link to the PEP: https://github.com/python/peps/blob/main/pep-0699.rst

1 0

Moving to Discourse
by Petr Viktorin Sept. 23, 2022

Sept. 23, 2022

As mentioned previously [0], the Steering Council decided to switch from python-dev to Discourse (discuss.python.org). We're aware that Discourse is not perfect. The previous mail thread [0] lists various shortcomings, as well as some workarounds. However, we don't see anything that would block Python development. Practically, the switch means that: - New PEPs should be announced in the PEPs category on Discourse (rather than on this list), and - The Devguide will list Discourse, rather than mailing lists, as the primary communication channel. Note that you can have development-related discussions anywhere, as long as you (eventually) include all relevant people. You're welcome to continue using python-dev and other mailing lists, IRC, in-person sprints, etc. But for PEP-level changes, we believe python-dev no longer reaches the proper audience. For the related docs changes, see [peps-2775] and [devguide-945]. Note that this is documentation, not law – if something is unclear or doesn't make sense, please ask for clarification or propose edits. [0]: https://mail.python.org/archives/list/python-dev@python.org/thread/VHFLDK43… [peps-2775]: https://github.com/python/peps/pull/2775 [devguide-945]: https://github.com/python/devguide/pull/945 — Petr, on behalf of the Steering Council

4 6

PEP 505 (None-aware operators) for Python 3.11
by Doug Swarin Sept. 21, 2022

Sept. 21, 2022

Hello, I've been following PEP 505 (https://www.python.org/dev/peps/pep-0505/) since it was first proposed for Python 3.8. It's been deferred for some time and I'm interested in seeing it in Python 3.11, but I know there were also a number of objections which resulted in it being deferred (including by one of the original authors, Mr. Dower). I did email both Mr. Dower and Mr. Haase and they graciously gave me their permission to bring it up on this list for discussion and hopefully final pronouncement one way or the other. I personally believe that the PEP will result in a significant reduction in boilerplate code, and it is substantially similar to the same operators now found in a number of other languages, especially C# and JavaScript (https://wikipedia.org/wiki/Null_coalescing_operator and https://wikipedia.org/wiki/Safe_navigation_operator). I believe strong and valid arguments can be made about the use of None being a fundamental flaw in some types of coding (and that adding additional support for it to the language will increase the use of None in this way), but I also believe there are many use cases in programming where it is by far the simplest way to express various semantics, and the fact exists that None is already used extensively in large quantities of code, and further that there is already a great deal of code written to constantly test against None and break out of a statement without throwing an error. I also understand the argument that especially the maybe-dot (?.) and maybe-subscript (?[) operators can decrease readability of code and also believe these are valid arguments against it. While I believe the existence and use of these operators in other languages definitely helps the case that these can be used and understood successfully, I think it is entirely valid to either consider other syntax (though I prefer the chosen syntax of PEP 505), or even to reduce PEP 505 to having only the coalesce operator (??) and the maybe-assign operator (??=). Separately, I have implemented a pure-Python solution for PEP505 (which is definitely rather beta) which might help test the waters for a final implementation in CPython (though the CPython implementation would of course be much more efficient). It can be found at https://pypi.org/project/pep505/ Thanks, Doug

22 53

Annotating pure functions to improve inline caching/optimization
by Philipp Burch Sept. 19, 2022

Sept. 19, 2022

Hello everyone, the docs on the upcoming 3.11 release state > This [specializing adaptive interpreter] also brings in another concept called inline caching, where Python caches the results of expensive operations directly in the bytecode. I wonder how this caching works, given that the dynamic nature means that virtually every operation could have side effects, causing wrong behaviour when cached. The only mitigation for this that I can imagine is that caching just occurs for basic operations defined in the standard library, where it is known that they are free of side effects or "pure". A web search did reveal some discussions[1,2] and a module related to dealing with pure functions, but, as far as I see, not related to optimization. As an example, consider a code like this: ```py @pure def rot_factor(angle_deg: float) -> complex: # This could also be a much more expensive calculation. return cmath.exp(angle_deg / 180 * cmath.pi * 1j) # ... res: List[Tuple(complex, complex, complex, float)] = [] for x in many: res.append(( x * rot_factor(90), x * rot_factor(45), x * rot_factor(-45), x * math.sin(math.pi/8), )) ``` The problem with this code is obvious, every loop iteration calls `rot_factor()` with 90, 45 and -45 and will get exactly the same set of results. The last factor might already be inline cached by the interpreter, since it probably knows that `math.pi` is a constant and `math.sin()` is a pure function. Optimizing this by hand (not considering a list comprehension or other more sophisticated improvements) is easy, but not very pretty: ```py f_p90 = rot_factor(90) f_p45 = rot_factor(45) f_m45 = rot_factor(-45) f_sin = math.sin(math.pi / 8) res: List[Tuple(complex, complex, complex, float)] = [] for x in many: res.append(( x * f_p90, x * f_p45, x * f_m45, x * f_sin, )) ``` I actually find myself often factoring such data out of loops in Python, whereas in C I would just leave that to the optimizer/compiler. An available option would be to use `@lru_cache` for `rot_factor()`, but this will still cause the same dictionary lookups in every iteration and it may not work at all in case the function argument(s) is/are not hashable. Now, if the interpreter understood the `@pure` decorator for `rot_factor()` indicated above would give it the same opportunity to cache the three results throughout the loop, basically creating the manually-optimized code above. For these completely static values, it could even precompute the results and integrate them into the bytecode. Has anything like this been considered already, or is the interpreter itself capable to perform such optimizations? Thanks and best regards, Philipp [1] 'pure' type annotation for mypy: https://github.com/python/mypy/issues/4468 [2] pure-func module: https://pypi.org/project/pure-func/

4 5

glob's new include_hidden parameter
by Mats Wichmann Sept. 13, 2022

Sept. 13, 2022

Just spotted that 3.11 adds an include_hidden. At the moment a little confused because there's an apparent mismatch between docstring and docs. Lib/glob.py: If `include_hidden` is true, the patterns '*', '?', '**' will match hidden directories. Doc/library/glob.rst: If *include_hidden* is true, "``**``" pattern will match hidden directories. ,,, with no mention of the other patterns. Is that just an omission in the rst doc?

3 3