Mailman 3 October 2022 - Python-Dev

Proposal: Allow non-default after default arguments
by Sebastian Rittau Oct. 16, 2023

Oct. 16, 2023

Currently, Python doesn't allow non-default arguments after default arguments: >>> def foo(x=None, y): pass File "<stdin>", line 1 def foo(x=None, y): pass ^ SyntaxError: non-default argument follows default argument I believe that at the time this was introduced, no use cases for this were known and this is is supposed to … [View More]prevent a source of bugs. I have two use cases for this, one fringe, but valid, the other more important: The fringe use case: Suppose you have a function that takes a 2D coordinate value as separate "x" and "y" arguments. The "x" argument is optional, the "y" argument isn't. Currently there are two ways to do this, none of them particularly great: def foo(y, x): # reverse x and y arguments, confusing ... def foo(x, y=None): # Treat the x argument as y if only one argument is provided if y is None: x, y = y, x ... To me, the "natural" solution looks like this: def foo(x=None, y): ... # Called like this: foo(1, 2) foo(y=2) This could also be useful when evolving APIs. For example there is a function "bar" that takes two required arguments. In a later version, the first argument gains a useful default, the second doesn't. There is no sensible way to evolve the API at the moment. The more important use case involves @overloads. A condensed example of a function where the return type depends on an "encoding" parameter, followed by further parameters that could be called like this: foo(123, "utf-8") # would return bytes foo(encoding="utf-8") foo(123, None) # would return str foo(encoding=None) foo(x=123) # would return str foo() This could ideally be written as: @overload def foo(x: int = ..., encoding: None = ...) -> str: ... @overload def foo(x: int = ..., encoding: str) -> bytes: ... # plus the actual implementation But due to the syntax constraint, this needs to be hacked around with a third overload: @overload def foo(x: int = ... encoding: None = ...) -> str: ... @overload def foo(x: int, encoding: str) -> bytes: ... # for foo(123, "utf-8") @overload def foo(*, encoding: str) -> bytes: ... # for foo(encoding="utf-8") Not only is this hard to read, real examples in typeshed are usually more complex, with many arguments before or after the affected argument or even multiple affected arguments. This often becomes too complex to write or maintain. Here is one example from the wild: https://github.com/python/typeshed/blob/b95b729b9e07ab21d252701af0f5b740467… Allowing non-default arguments after default arguments would solve both use cases above and eliminates a special case. I'm also not sure what exactly the current SyntaxError really protects us from. Adding a non-default after a default argument can't really lead bugs. - Sebastian [View Less]

11 14

Python multithreading without the GIL
by Sam Gross July 31, 2023

July 31, 2023

Hi, I've been working on changes to CPython to allow it to run without the global interpreter lock. I'd like to share a working proof-of-concept that can run without the GIL. The proof-of-concept involves substantial changes to CPython internals, but relatively few changes to the C-API. It is compatible with many C extensions: extensions must be rebuilt, but usually require small or no modifications to source code. I've built compatible versions of packages from the scientific Python ecosystem,… [View More]

24 48

Mixed Python/C debugging
by Skip Montanaro April 26, 2023

April 26, 2023

Having tried comp.lang.python with no response, I turn here... After at least ten years away from Python's run-time interpreter & byte code compiler, I'm getting set to familiarize myself with that again. This will, I think, entail debugging a mixed Python/C environment. I'm an Emacs user and am aware that GDB since 7.0 has support for debugging at the Python code level. Is Emacs+GDB my best bet? Are there any Python IDEs which support C-level breakpoints and debugging? Thanks, Skip

5 6

PEP 448 review
by Guido van Rossum March 29, 2023

March 29, 2023

I'm back, I've re-read the PEP, and I've re-read the long thread with "(no subject)". I think Georg Brandl nailed it: """ *I like the "sequence and dict flattening" part of the PEP, mostly because itis consistent and should be easy to understand, but the comprehension syntaxenhancements seem to be bad for readability and "comprehending" what the codedoes.The call syntax part is a mixed bag on the one hand it is nice to be consistent with the extended possibilities in literals (… [View More]flattening), but on the other hand there would be small but annoying inconsistencies anyways (e.g. the duplicate kwarg case above).* """ Greg Ewing followed up explaining that the inconsistency between dict flattening and call syntax is inherent in the pre-existing different rules for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1, a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1, **{'a': 1}) is a TypeError.) For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations for function calls proposed by the PEP is an easy +1 -- it's a straightforward extension of the existing pattern, and anybody who knows what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d, **e) means shouldn't be hard either. Understanding the edge case for duplicate keys with f(**d, **e) is a little harder, but the error messages are pretty clear, and it is not a new edge case. The sequence and dict flattening syntax proposals are also clean and logical -- we already have *-unpacking on the receiving side, so allowing *x in tuple expressions reads pretty naturally (and the similarity with *a in argument lists certainly helps). From here, having [a, *x, b, *y] is also natural, and then the extension to other displays is natural: {a, *x, b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me. So that leaves comprehensions. IIRC, during the development of the patch we realized that f(*x for x in xs) is sufficiently ambiguous that we decided to disallow it -- note that f(x for x in xs) is already somewhat of a special case because an argument can only be a "bare" generator expression if it is the only argument. The same reasoning doesn't apply (in that form) to list, set and dict comprehensions -- while f(x for x in xs) is identical in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x for x in xs)] (that's a list of one element, and the element is a generator expression). The basic premise of this part of the proposal is that if you have a few iterables, the new proposal (without comprehensions) lets you create a list or generator expression that iterates over all of them, essentially flattening them: >>> xs = [1, 2, 3] >>> ys = ['abc', 'def'] >>> zs = [99] >>> [*xs, *ys, *zs] [1, 2, 3, 'abc', 'def', 99] >>> But now suppose you have a list of iterables: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xss[0], *xss[1], *xss[2]] [1, 2, 3, 'abc', 'def', 99] >>> Wouldn't it be nice if you could write the latter using a comprehension? >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs for xs in xss] [1, 2, 3, 'abc', 'def', 99] >>> This is somewhat seductive, and the following is even nicer: the *xs position may be an expression, e.g.: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs[:2] for xs in xss] [1, 2, 'abc', 'def', 99] >>> On the other hand, I had to explore the possibilities here by experimenting in the interpreter, and I discovered some odd edge cases (e.g. you can parenthesize the starred expression, but that seems a syntactic accident). All in all I am personally +0 on the comprehension part of the PEP, and I like that it provides a way to "flatten" a sequence of sequences, but I think very few people in the thread have supported this part. Therefore I would like to ask Neil to update the PEP and the patch to take out the comprehension part, so that the two "easy wins" can make it into Python 3.5 (basically, I am accepting two-thirds of the PEP :-). There is some time yet until alpha 2. I would also like code reviewers (Benjamin?) to start reviewing the patch <http://bugs.python.org/issue2292>, taking into account that the comprehension part needs to be removed. -- --Guido van Rossum (python.org/~guido) [View Less]

7 17

PEP 638: Syntactic macros
by Mark Shannon Feb. 5, 2023

Feb. 5, 2023

Hi everyone, I've submitted my PEP on syntactic macros as PEP 638. https://www.python.org/dev/peps/pep-0638/ All comments and suggestions are welcome. Cheers, Mark

16 25

Switching to Discourse
by Petr Viktorin Jan. 26, 2023

Jan. 26, 2023

Hello, Currently development discussions are split between multiple communication channels, for example: - python-dev and discuss.python.org for design discussions, - GitHub Issues and Pull Requests for specific changes, - IRC, Discord and private chats for real-time discussions, - Topic-specific channels like typing-sig. While most of these serve different needs, there is too much overlap between python-dev and discuss.python.org. It seems that for most people, this situation is worse than … [View More]sticking to either one platform – even if we don't go with that person's favorite. The discuss.python.org experiment has been going on for quite a while, and while the platform is not without its issues, we consider it a success. The Core Development category is busier than python-dev. According to staff, discuss.python.org is much easier to moderate.. If you're following python-dev but not discuss.python.org, you're missing out. The Steering Council would like to switch from python-dev to discuss.python.org. Practically, this means: - Moving the required PEP announcements to discuss.python.org - Moving discuss.python.org up in the devguide communications page (https://devguide.python.org/communication/) - And that's it? I imagine that the mailing list will stay around for continuing past discussion threads and for announcements, eventually switching to auto-reject incoming messages with a pointer to discuss.python.org. To be clear, discuss.python.org allows editing posts, which is frankly handy for typos and clarifications. Editing alone should not be used for adding new info -- we should cultivate a culture of being friendly to mail users & notification watchers. This probably bears repeating in a few places. We're aware not everyone wants to use the discuss.python.org website, but there are some ways to avoid it: - For new PEPs, you can point your RSS client to https://www.python.org/dev/peps/peps.rss – it's not e-mail, but many email clients have RSS support. You can also watch the Steering Council issues on GitHub (https://github.com/python/steering-council/issues/) for important questions and discussions. - You can use discuss.python.org's “mailing list mode” (which subscribes you to all new posts), possibly with filtering and/or categorizing messages locally. However, we would like to know if this will pose an undue burden to anyone, if there are workflows or usage problems that we are not aware of. As mentioned, this is something the Steering Council thinks is a good idea, but we want to make sure we're aware of all the impact when we make the final decision. – Petr, on behalf of the Steering Council [View Less]

43 114

NEWLINE sentinel behavior in CPython's PEG grammar
by David J W Nov. 3, 2022

Nov. 3, 2022

I am writing a Rust version of Python for fun and I am at the parser stage of development. I copied and modified a PEG grammar ruleset from another open source project and I've already noticed some problems (ex Newline vs NL) with how they transcribed things. I am suspecting that CPython's grammar NEWLINE is a builtin rule for the parser that is something like `(Newline+ | NL+ ) {NOP}` but wanted to sanity check if that is right before I figure out how to hack in a NEWLINE rule and update my grammar ruleset.

5 9

Possible bug in `re` module in Python 3.11?
by Piotr Waszkiewicz Oct. 26, 2022

Oct. 26, 2022

Hi, I would like to ask your guidance as I'm entirely sure whether the problem I'm experiencing should be posted in CPython's repo as a bug issue. I've tried using newly released Python 3.11 interpreter in some of my projects and one of them failed to start with "RuntimeError: invalid SRE code" error. Looking into implementation of one of the dependencies I've found out that the issue has been found out by a maintainer and fixed (https://github.com/pydicom/pydicom/issues/1658). It looks like … [View More]

3 3

Thank you for your contributions to Python 3.11!
by Pablo Galindo Salgado Oct. 26, 2022

Oct. 26, 2022

Hi everyone, Now that the 3.11.0 release is finally done and I can relax a bit, I just wanted to thank you all for your fantastic work that has made Python 3.11 such a fantastic release. No matter if you committed code to 3.11 or opened a bug, helped with the documentation, reviewed pull requests, participated in discussions, made a PEP or help writing one, fixed a bug or one hundred or make optimizations to the interpreter or any other of the many ways to contribute. Your work makes a huge … [View More]

1 0

problem with Distributed File System Replication and Namespacing and different versions of Python 3
by rainonthescarecrowhumanwheels＠gmail.com Oct. 26, 2022

Oct. 26, 2022

Hi, I have a question about a problem we are facing in different Python versions, like Python 3.7.7, Python 3.8, Python 3.6 and Python 3.9. This apparently doesnt happen in Python 3.10.2 and we were wondering how we could achieve the same behaviours without going deep into the nuts and bolts of the python import system. We have python libraries that are distributed in a networked environment that when imported for the first time they are correctly imported but something happens in the meantime … [View More]

6 9

Python 3.12.0 alpha 1 released.
by Thomas Wouters Oct. 25, 2022

Oct. 25, 2022

As Pablo released Python 3.11.0 final earlier today, now it's my turn to release Python 3.12.0 alpha 1. *This is an early developer preview of Python 3.12* Major new features of the 3.12 series, compared to 3.11 Python 3.12 is still in development. This release, 3.12.0a1 is the first of seven planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process. During the alpha phase, features may be … [View More]added up until the start of the beta phase (2023-05-08) and, if necessary, may be modified or deleted up until the release candidate phase (2023-07-31). Please keep in mind that this is a preview release and its use is *not *recommended for production environments. Many new features for Python 3.12 are still being planned and written. Among the new major new features and changes so far: - The deprecated `wstr` and `wstr_length` members of the C implementation of unicode objects were removed, per PEP 623 <https://peps.python.org/pep-0623/>. - In the `unittest` module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2). - The deprecated `smtpd` module has been removed. - A number of other old, broken and deprecated functions, classes and methods have been removed. - (Hey, **fellow core developer,** if a feature you find important is missing from this list, let Thomas know <thomas(a)python.org>.) The next pre-release of Python 3.12 will be 3.12.0a2, currently scheduled for 2022-11-14. More resources - Online Documentation <https://docs.python.org/3.12> - PEP 693 <https://www.python.org/dev/peps/pep-0693/>, the 3.12 Release Schedule - Report bugs at https://github.com/python/cpython/issues. - Help fund Python and its community at https://www.python.org/psf/donations/. And now for something completely different This is Not the Poem that I Had Hoped to Write <https://twitter.com/brian_bilston/status/1579378460610662401> This is not the poem that I had hoped to write when I sat at my desk and the page was white. You see, there were other words that I’d had in mind, yet this is what I leave behind. I thought it was a poem to eradicate war; one of such power, it would heal all the sores of a world torn apart by conflict and schism. But it isn’t. Lovers, I’d imagined, would quote from it daily, Mothers would sing it to soothe crying babies. And whole generations would be given new hope. Nope. I had grand aspirations. Believe me, I tried. Humanity examined with lessons applied. But the right words escaped me; so often they do. Have these in lieu. Brian Bilston <https://twitter.com/brian_bilston> Enjoy the new releases Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Regards from dusky California, Your release team, Thomas Wouters @Yhg1s Ned Deily @nad Steve Dower @steve.dower -- Thomas Wouters <thomas(a)python.org> [View Less]

1 0

[RELEASE] Python 3.11 final (3.11.0) is available
by Pablo Galindo Salgado Oct. 24, 2022

Oct. 24, 2022

Python 3.11 is finally released. In the CPython release team, we have put a lot of effort into making 3.11 the best version of Python possible. Better tracebacks, faster Python, exception groups and except*, typing improvements and much more. Get it here: https://www.python.org/downloads/release/python-3110/ ## This is the stable release of Python 3.11.0 Python 3.11.0 is the newest major release of the Python programming language, and it contains many new features and optimizations. # Major … [View More]new features of the 3.11 series, compared to 3.10 Some of the new major new features and changes in Python 3.11 are: ## General changes * [PEP 657](https://www.python.org/dev/peps/pep-0657/) -- Include Fine-Grained Error Locations in Tracebacks * [PEP 654](https://www.python.org/dev/peps/pep-0654/) -- Exception Groups and `except*` * [PEP 680](https://www.python.org/dev/peps/pep-0680/) -- tomllib: Support for Parsing TOML in the Standard Library * [gh-90908](https://github.com/python/cpython/issues/90908) -- Introduce task groups to asyncio * [gh-34627](https://github.com/python/cpython/issues/34627/) -- Atomic grouping (`(?>...)`) and possessive quantifiers (`*+, ++, ?+, {m,n}+`) are now supported in regular expressions. * The [Faster CPython Project](https://github.com/faster-cpython/) is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See [Faster CPython]( https://docs.python.org/3.11/whatsnew/3.11.html#faster-cpython) for details. ## Typing and typing language changes * [PEP 673](https://www.python.org/dev/peps/pep-0673/) -- Self Type * [PEP 646](https://www.python.org/dev/peps/pep-0646/) -- Variadic Generics * [PEP 675](https://www.python.org/dev/peps/pep-0675/) -- Arbitrary Literal String Type * [PEP 655](https://www.python.org/dev/peps/pep-0655/) -- Marking individual TypedDict items as required or potentially-missing * [PEP 681](https://www.python.org/dev/peps/pep-0681/) -- Data Class Transforms # More resources * [Online Documentation](https://docs.python.org/3.11/) * [PEP 664](https://www.python.org/dev/peps/pep-0664/), 3.11 Release Schedule * Report bugs at [ https://github.com/python/cpython/issues](https://github.com/python/cpython… . * [Help fund Python and its community](/psf/donations/). # And now for something completely different When a spherical non-rotating body of a critical radius collapses under its own gravitation under general relativity, theory suggests it will collapse to a single point. This is not the case with a rotating black hole (a Kerr black hole). With a fluid rotating body, its distribution of mass is not spherical (it shows an equatorial bulge), and it has angular momentum. Since a point cannot support rotation or angular momentum in classical physics (general relativity being a classical theory), the minimal shape of the singularity that can support these properties is instead a ring with zero thickness but non-zero radius, and this is referred to as a ringularity or Kerr singularity. This kind of singularity has the following peculiar property. The spacetime allows a geodesic curve (describing the movement of observers and photons in spacetime) to pass through the center of this ring singularity. The region beyond permits closed time-like curves. Since the trajectory of observers and particles in general relativity are described by time-like curves, it is possible for observers in this region to return to their past. This interior solution is not likely to be physical and is considered a purely mathematical artefact. There are some other interesting free-fall trajectories. For example, there is a point in the axis of symmetry that has the property that if an observer is below this point, the pull from the singularity will force the observer to pass through the middle of the ring singularity to the region with closed time-like curves and it will experience repulsive gravity that will push it back to the original region, but then it will experience the pull from the singularity again and will repeat this process forever. This is, of course, only if the extreme gravity doesn’t destroy the observer first. # We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. https://www.python.org/psf/ If you have any questions, please reach out to me or another member of the release team :) Your friendly release team, Ned Deily @nad https://discuss.python.org/u/nad Steve Dower @steve.dower https://discuss.python.org/u/steve.dower Pablo Galindo Salgado @pablogsal https://discuss.python.org/u/pablogsal [View Less]

1 0

IMPORTANT: Check the 3.11.0 cherry-picks
by Pablo Galindo Salgado Oct. 24, 2022

Oct. 24, 2022

Hi everyone, I emerged from cherry-picking hell! As mentioned previously, the 3.11.0 final release will be done from the "branch-v3.11.0" branch and will contain a bunch of cherry-picked commits on top of v3.11.0rc2. These commits are: * All documentation commits that **do not touch** any source code (120+ commits). * The following bugfixes: + 6b999982cb8 gh-95027: Fix regrtest stdout encoding on Windows (GH-98492) + 970c10aa6d0 gh-97616: list_resize() checks for integer overflow (GH-97617) +… [View More]

1 0

RELEASE MANAGEMENT: Python 3.11 release stream
by Pablo Galindo Salgado Oct. 22, 2022

Oct. 22, 2022

Hi everyone, I am currently swamped fixing merge conflicts from the cherry-picking to the final release candidate but I wanted to write to tell you I'm excited to confirm that we will be haven a Python 3.11 release party as we did last year with 3.10. The event will be co-hosted by the fantastic people of the Python discord channel. The URL for the stream is this one: https://www.youtube.com/watch?v=PGZPSWZSkJI *The release party will start on October 24th at 17:00 UTC+0.* In this stream, … [View More]

1 0

How about using modern C++ in development of CPython ?
by redradist＠gmail.com Oct. 19, 2022

Oct. 19, 2022

Hi all, What about of using modern C++ in developing CPython ? With C++ we can get the following benefits: 1) Readability - less code, most code is hidden by abstraction without losing performance 2) Maintainability - no code duplication in favor of using reusable classes 3) RAII - Resource Acquisition Is Initialization, predictable allocation and free resources ...

23 51

[RELEASE] Python versions 3.10.8, 3.9.15, 3.8.15, 3.7.15 now available
by Łukasz Langa Oct. 11, 2022

Oct. 11, 2022

Déjà vu? Right, a month after the expedited releases we are doing the dance again. This coincides with the regular scheduled time for 3.10.8 but since we accrued a few fixes in 3.7 - 3.9 as well, we’re again releasing all four editions at the same time. We’re not promising to continue at this pace 😅 <https://discuss.python.org/t/python-versions-3-10-8-3-9-15-3-8-15-3-7-15-no…>Security content this time CVE-2022-40674: bundled libexpat was upgraded from 2.4.7 to 2.4.9 which fixes a … [View More]heap use-after-free vulnerability in function doContent gh-97616: a fix for a possible buffer overflow in list *= int gh-97612: a fix for possible shell injection in the example script get-remote-certificate.py (this issue originally had a CVE assigned to it, which its author withdrew) gh-96577: a fix for a potential buffer overrun in msilib <https://discuss.python.org/t/python-versions-3-10-8-3-9-15-3-8-15-3-7-15-no…>Python 3.10.8 Get it here: https://www.python.org/downloads/release/python-3108/ <https://www.python.org/downloads/release/python-3108/> As a bugfix release coming a mere month after an out-of-schedule security release, 3.10.8 is somewhat smaller compared to 3.9.8 released at the same stage of the release cycle a year ago. There’s 151 commits vs 204 in 3.9. It’s still a larger release than 3.10.7 at 113 commits. One way or the other, it’s worth checking out the change log <https://docs.python.org/release/3.10.8/whatsnew/changelog.html>. <https://discuss.python.org/t/python-versions-3-10-8-3-9-15-3-8-15-3-7-15-no…>And now for something completely different Granular convection is a phenomenon where granular material subjected to shaking or vibration will exhibit circulation patterns similar to types of fluid convection. It is sometimes described as the Brazil nut effect when the largest particles end up on the surface of a granular material containing a mixture of variously sized objects; this derives from the example of a typical container of mixed nuts, where the largest will be Brazil nuts. The phenomenon is also known as the muesli effect since it is seen in packets of breakfast cereal containing particles of different sizes but similar densities, such as muesli mix. Under experimental conditions, granular convection of variously sized particles has been observed forming convection cells similar to fluid motion. <https://discuss.python.org/t/python-versions-3-10-8-3-9-15-3-8-15-3-7-15-no…>We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Your friendly release team, Ned Deily @nad <https://discuss.python.org/u/nad> Steve Dower @steve.dower <https://discuss.python.org/u/steve.dower> Pablo Galindo Salgado @pablogsal <https://discuss.python.org/u/pablogsal> Łukasz Langa @ambv <https://discuss.python.org/u/ambv> [View Less]

1 0

Embedded Python Memory Leaks
by alexei73＠hotmail.com Oct. 8, 2022

Oct. 8, 2022

Hi Python team Is any plan to try and fix memory leaks when embedding Python in a C/C++ application? Thanks in advance, Alexei

3 2

PEP 699: Remove private dict version field added in PEP 509
by Ken Jin Oct. 5, 2022

Oct. 5, 2022

I've created a new PEP to repeal PEP 509. Please post comments to the Discourse thread. Thank you! Discourse thread: https://discuss.python.org/t/pep-699-remove-private-dict-version-field-adde… Link to the PEP: https://github.com/python/peps/blob/main/pep-0699.rst

1 0