Mailman 3 June 2023 - Python-Dev

Proposal: Allow non-default after default arguments
by Sebastian Rittau 16 Oct '23

16 Oct '23

Currently, Python doesn't allow non-default arguments after default arguments: >>> def foo(x=None, y): pass File "<stdin>", line 1 def foo(x=None, y): pass ^ SyntaxError: non-default argument follows default argument I believe that at the time this was introduced, no use cases for this were known and this is is supposed to prevent a source of bugs. I have two use cases for this, one fringe, but valid, the other more important: The fringe use case: Suppose you have a function that takes a 2D coordinate value as separate "x" and "y" arguments. The "x" argument is optional, the "y" argument isn't. Currently there are two ways to do this, none of them particularly great: def foo(y, x): # reverse x and y arguments, confusing ... def foo(x, y=None): # Treat the x argument as y if only one argument is provided if y is None: x, y = y, x ... To me, the "natural" solution looks like this: def foo(x=None, y): ... # Called like this: foo(1, 2) foo(y=2) This could also be useful when evolving APIs. For example there is a function "bar" that takes two required arguments. In a later version, the first argument gains a useful default, the second doesn't. There is no sensible way to evolve the API at the moment. The more important use case involves @overloads. A condensed example of a function where the return type depends on an "encoding" parameter, followed by further parameters that could be called like this: foo(123, "utf-8") # would return bytes foo(encoding="utf-8") foo(123, None) # would return str foo(encoding=None) foo(x=123) # would return str foo() This could ideally be written as: @overload def foo(x: int = ..., encoding: None = ...) -> str: ... @overload def foo(x: int = ..., encoding: str) -> bytes: ... # plus the actual implementation But due to the syntax constraint, this needs to be hacked around with a third overload: @overload def foo(x: int = ... encoding: None = ...) -> str: ... @overload def foo(x: int, encoding: str) -> bytes: ... # for foo(123, "utf-8") @overload def foo(*, encoding: str) -> bytes: ... # for foo(encoding="utf-8") Not only is this hard to read, real examples in typeshed are usually more complex, with many arguments before or after the affected argument or even multiple affected arguments. This often becomes too complex to write or maintain. Here is one example from the wild: https://github.com/python/typeshed/blob/b95b729b9e07ab21d252701af0f5b740467… Allowing non-default arguments after default arguments would solve both use cases above and eliminates a special case. I'm also not sure what exactly the current SyntaxError really protects us from. Adding a non-default after a default argument can't really lead bugs. - Sebastian

11 14

Python multithreading without the GIL
by Sam Gross 31 Jul '23

31 Jul '23

Hi, I've been working on changes to CPython to allow it to run without the global interpreter lock. I'd like to share a working proof-of-concept that can run without the GIL. The proof-of-concept involves substantial changes to CPython internals, but relatively few changes to the C-API. It is compatible with many C extensions: extensions must be rebuilt, but usually require small or no modifications to source code. I've built compatible versions of packages from the scientific Python ecosystem, and they are installable through the bundled "pip". Source code: https://github.com/colesbury/nogil Design overview: https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlej… My goal with the proof-of-concept is to demonstrate that removing the GIL is feasible and worthwhile, and that the technical ideas of the project could serve as a basis of such an effort. I'd like to start a discussion about these ideas and gauge the community's interest in this approach to removing the GIL. Regards, Sam Gross colesbury(a)gmail.com / sgross(a)fb.com

24 48

[RELEASE] Python 3.12.0 beta 3 released
by Thomas Wouters 20 Jun '23

20 Jun '23

We’re getting close! 3.12.0 beta 3 has been released: https://www.python.org/downloads/release/python-3120b3/ *This is a beta preview of Python 3.12* Python 3.12 is still in development. This release, 3.12.0b3, is the third of four beta release previews of 3.12. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. We *strongly encourage* maintainers of third-party Python projects to* test with 3.12* during the beta phase and report issues found to [the Python bug tracker (https://github.com/python/cpython/issues) as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (Monday, 2023-07-31). Our goal is to have no ABI changes after beta 4 and as few code changes as possible after 3.12.0rc1, the first release candidate. To achieve that, it will be *extremely important* to get as much exposure for 3.12 as possible during the beta phase. Please keep in mind that this is a preview release and its use is *not *recommended for production environments. *Major new features of the 3.12 series, compared to 3.11* Some of the new major new features and changes in Python 3.12 are: - New type annotation syntax for generic classes (PEP 695 <https://peps.python.org/pep-0695/>). - More flexible f-string parsing, allowing many things previously disallowed (PEP 701 <https://peps.python.org/pep-0701/>). - Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user. - Many large and small performance improvements (like PEP 709 <https://peps.python.org/pep-0709/>). - Support for the Linux perf profiler to report Python function names in traces. - The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623 <https://peps.python.org/pep-0623/>. - In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2). - The deprecated smtpd and distutils modules have been removed (see PEP 594 <https://peps.python.org/pep-0594/> and PEP 632 <https://peps.python.org/pep-0632/>. The setuptools package continues to provide the distutils module. - A number of other old, broken and deprecated functions, classes and methods have been removed. - Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.) - The internal representation of integers has changed in preparation for performance enhancements. (This should not affect most users as it is an internal detail, but it may cause problems for Cython-generated code.) - (Hey, fellow core developer, if a feature you find important is missing from this list, let Thomas know <thomas(a)python.org>.) For more details on the changes to Python 3.12, see What's new in Python 3.12 <https://docs.python.org/dev/whatsnew/3.12.html>. The next pre-release of Python 3.12 will be 3.12.0b4, the last beta release, currently scheduled for 2023-07-10. *More resources*Online Documentation <https://docs.python.org/3.12/>. PEP 693 <https://www.python.org/dev/peps/pep-0693/>, the Python 3.12 Release Schedule. Report bugs via GitHub Issues <https://github.com/python/cpython/issues>. Help fund Python and its community <https://www.python.org/psf/donations/>. *We hope you enjoy the new releases!* Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Regards from a suddenly very stormy Amsterdam, Thomas Wouters Your release team, Ned Deily Steve Dower Łukasz Langa -- Thomas Wouters <thomas(a)python.org>

1 0

[RELEASE] Python 3.11.4, 3.10.12, 3.9.17, 3.8.17, 3.7.17, and 3.12.0 beta 2 are now available
by Łukasz Langa 06 Jun '23

06 Jun '23

Greetings! Time for another combined release of six separate versions of Python! <https://discuss.python.org/t/python-3-11-4-3-10-12-3-9-17-3-8-17-3-7-17-and…>Before you scroll away to the download links Please test the 3.12 beta! Downloading it and trying it out helps us a lot in ensuring Python 3.12.0 will be as polished as possible. We welcome 3.10 to the prestigious club of security-only releases. It’s officially an old version of Python now! If you haven’t rewritten all your if:elif:else:s with pattern matching yet, are you even still writing Python? At the same time, it looks like 3.7 is reaching end-of-life. Unless another security release happens in June, 3.7.17 will be the final release of Python 3.7. I mean, now that I typed it out for all you to read, I’m sure I jinxed it. But in case I didn’t, I would like to thank Ned Deily for serving as the release manager of Python 3.6 and Python 3.7. He was my mentor as Release Manager, and continues serving Python as the provider of Mac installers for new releases. Thank you, Ned! Speaking of installers, Steve Dower used to be the sole provider of Windows installers for Python releases for years now. His secret was a well-automated Azure pipeline that let him build, sign, and publish releases with minimal manual effort. Now he extended the power to press the blue “Run pipeline” button to more members of the team. Thank you, Steve! This is an important bus factor increment. In fact, the Windows installers for both 3.12.0b2 and 3.11.4 were made by meinitiated by me <https://dev.azure.com/Python/cpython/_build/results?buildId=129764&view=res…>. If there’s anything wrong with them, well, I guess that means I pressed the button wrong. <https://discuss.python.org/t/python-3-11-4-3-10-12-3-9-17-3-8-17-3-7-17-and…>Security fixes in today’s releases Updating is recommended due to security content: 3.7 - 3.12: gh-103142 <https://github.com/python/cpython/issues/103142>: The version of OpenSSL used in Windows and Mac installers has been upgraded to 1.1.1u to address CVE-2023-2650, CVE-2023-0465, CVE-2023-0466, CVE-2023-0464, as well as CVE-2023-0286, CVE-2022-4303, and CVE-2022-4303 fixed previously in 1.1.1t (gh-101727). 3.7 - 3.11: gh-102153 <https://github.com/python/cpython/issues/102153>: urllib.parse.urlsplit() now strips leading C0 control and space characters following the specification for URLs defined by WHATWG in response to CVE-2023-24329. 3.7 - 3.11: gh-99889 <https://github.com/python/cpython/issues/99889>: Fixed a security in flaw in uu.decode() that could allow for directory traversal based on the input if no out_file was specified. 3.7 - 3.11: gh-104049 <https://github.com/python/cpython/issues/104049>: Do not expose the local on-disk location in directory indexes produced by http.client.SimpleHTTPRequestHandler. 3.7 - 3.11: gh-101283 <https://github.com/python/cpython/issues/101283>: subprocess.Popen now uses a safer approach to find cmd.exe when launching with shell=True. 3.8 - 3.11: gh-103935 <https://github.com/python/cpython/issues/103935>: trace.__main__ now uses io.open_code() for files to be executed instead of raw open(). 3.8 - 3.11: gh-102953 <https://github.com/python/cpython/issues/102953>: The extraction methods in tarfile, and shutil.unpack_archive(), have a new filterargument that allows limiting tar features than may be surprising or dangerous, such as creating files outside the destination directory. See Extraction filters <https://docs.python.org/3/library/tarfile.html#extraction-filters> for details. 3.9: gh-102126 <https://github.com/python/cpython/issues/102126>: Fixed a deadlock at shutdown when clearing thread states if any finalizer tries to acquire the runtime head lock. 3.9: gh-100892 <https://github.com/python/cpython/issues/100892>: Fixed a crash due to a race while iterating over thread states in clearing threading.local. Python 3.12.0 beta 2 Get it here: 3.12.0b2 <https://www.python.org/downloads/release/python-3120b2/> 116 new commits since 3.12.0 beta 1. Python 3.11.4 Get it here: 3.11.4 <https://www.python.org/downloads/release/python-3114/> 233 new commits. Python 3.10.12 Get it here: 3.10.12 <https://www.python.org/downloads/release/python-31012/> Security-only release with no binaries. 20 new commits. Python 3.9.17 Get it here: 3.9.17 <https://www.python.org/downloads/release/python-3917/> Security-only release with no binaries. 26 commits. Python 3.8.17 Get it here: 3.8.17 <https://www.python.org/downloads/release/python-3817/> Security-only release with no binaries. 24 commits. Python 3.7.17 Get it here as it might be the last release of 3.7 ever <https://peps.python.org/pep-0537/>: 3.7.17 <https://www.python.org/downloads/release/python-3717/> Security-only release with no binaries. 21 commits. We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation <https://www.python.org/psf/>. – Łukasz Langa @ambv <https://discuss.python.org/u/ambv> on behalf of your friendly release team, Ned Deily @nad <https://discuss.python.org/u/nad> Steve Dower @steve.dower <https://discuss.python.org/u/steve.dower> Pablo Galindo Salgado @pablogsal <https://discuss.python.org/u/pablogsal> Łukasz Langa @ambv <https://discuss.python.org/u/ambv> Thomas Wouters @thomas <https://discuss.python.org/u/thomas>

1 0