Mailman 3 November 2022 - Python-Dev

Proposal: Allow non-default after default arguments
by Sebastian Rittau Oct. 16, 2023

Oct. 16, 2023

Currently, Python doesn't allow non-default arguments after default arguments: >>> def foo(x=None, y): pass File "<stdin>", line 1 def foo(x=None, y): pass ^ SyntaxError: non-default argument follows default argument I believe that at the time this was introduced, no use cases for this were known and this is is supposed to … [View More]prevent a source of bugs. I have two use cases for this, one fringe, but valid, the other more important: The fringe use case: Suppose you have a function that takes a 2D coordinate value as separate "x" and "y" arguments. The "x" argument is optional, the "y" argument isn't. Currently there are two ways to do this, none of them particularly great: def foo(y, x): # reverse x and y arguments, confusing ... def foo(x, y=None): # Treat the x argument as y if only one argument is provided if y is None: x, y = y, x ... To me, the "natural" solution looks like this: def foo(x=None, y): ... # Called like this: foo(1, 2) foo(y=2) This could also be useful when evolving APIs. For example there is a function "bar" that takes two required arguments. In a later version, the first argument gains a useful default, the second doesn't. There is no sensible way to evolve the API at the moment. The more important use case involves @overloads. A condensed example of a function where the return type depends on an "encoding" parameter, followed by further parameters that could be called like this: foo(123, "utf-8") # would return bytes foo(encoding="utf-8") foo(123, None) # would return str foo(encoding=None) foo(x=123) # would return str foo() This could ideally be written as: @overload def foo(x: int = ..., encoding: None = ...) -> str: ... @overload def foo(x: int = ..., encoding: str) -> bytes: ... # plus the actual implementation But due to the syntax constraint, this needs to be hacked around with a third overload: @overload def foo(x: int = ... encoding: None = ...) -> str: ... @overload def foo(x: int, encoding: str) -> bytes: ... # for foo(123, "utf-8") @overload def foo(*, encoding: str) -> bytes: ... # for foo(encoding="utf-8") Not only is this hard to read, real examples in typeshed are usually more complex, with many arguments before or after the affected argument or even multiple affected arguments. This often becomes too complex to write or maintain. Here is one example from the wild: https://github.com/python/typeshed/blob/b95b729b9e07ab21d252701af0f5b740467… Allowing non-default arguments after default arguments would solve both use cases above and eliminates a special case. I'm also not sure what exactly the current SyntaxError really protects us from. Adding a non-default after a default argument can't really lead bugs. - Sebastian [View Less]

11 14

Python multithreading without the GIL
by Sam Gross July 31, 2023

July 31, 2023

Hi, I've been working on changes to CPython to allow it to run without the global interpreter lock. I'd like to share a working proof-of-concept that can run without the GIL. The proof-of-concept involves substantial changes to CPython internals, but relatively few changes to the C-API. It is compatible with many C extensions: extensions must be rebuilt, but usually require small or no modifications to source code. I've built compatible versions of packages from the scientific Python ecosystem,… [View More]

24 48

Mixed Python/C debugging
by Skip Montanaro April 26, 2023

April 26, 2023

Having tried comp.lang.python with no response, I turn here... After at least ten years away from Python's run-time interpreter & byte code compiler, I'm getting set to familiarize myself with that again. This will, I think, entail debugging a mixed Python/C environment. I'm an Emacs user and am aware that GDB since 7.0 has support for debugging at the Python code level. Is Emacs+GDB my best bet? Are there any Python IDEs which support C-level breakpoints and debugging? Thanks, Skip

5 6

PEP 448 review
by Guido van Rossum March 29, 2023

March 29, 2023

I'm back, I've re-read the PEP, and I've re-read the long thread with "(no subject)". I think Georg Brandl nailed it: """ *I like the "sequence and dict flattening" part of the PEP, mostly because itis consistent and should be easy to understand, but the comprehension syntaxenhancements seem to be bad for readability and "comprehending" what the codedoes.The call syntax part is a mixed bag on the one hand it is nice to be consistent with the extended possibilities in literals (… [View More]flattening), but on the other hand there would be small but annoying inconsistencies anyways (e.g. the duplicate kwarg case above).* """ Greg Ewing followed up explaining that the inconsistency between dict flattening and call syntax is inherent in the pre-existing different rules for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1, a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1, **{'a': 1}) is a TypeError.) For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations for function calls proposed by the PEP is an easy +1 -- it's a straightforward extension of the existing pattern, and anybody who knows what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d, **e) means shouldn't be hard either. Understanding the edge case for duplicate keys with f(**d, **e) is a little harder, but the error messages are pretty clear, and it is not a new edge case. The sequence and dict flattening syntax proposals are also clean and logical -- we already have *-unpacking on the receiving side, so allowing *x in tuple expressions reads pretty naturally (and the similarity with *a in argument lists certainly helps). From here, having [a, *x, b, *y] is also natural, and then the extension to other displays is natural: {a, *x, b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me. So that leaves comprehensions. IIRC, during the development of the patch we realized that f(*x for x in xs) is sufficiently ambiguous that we decided to disallow it -- note that f(x for x in xs) is already somewhat of a special case because an argument can only be a "bare" generator expression if it is the only argument. The same reasoning doesn't apply (in that form) to list, set and dict comprehensions -- while f(x for x in xs) is identical in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x for x in xs)] (that's a list of one element, and the element is a generator expression). The basic premise of this part of the proposal is that if you have a few iterables, the new proposal (without comprehensions) lets you create a list or generator expression that iterates over all of them, essentially flattening them: >>> xs = [1, 2, 3] >>> ys = ['abc', 'def'] >>> zs = [99] >>> [*xs, *ys, *zs] [1, 2, 3, 'abc', 'def', 99] >>> But now suppose you have a list of iterables: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xss[0], *xss[1], *xss[2]] [1, 2, 3, 'abc', 'def', 99] >>> Wouldn't it be nice if you could write the latter using a comprehension? >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs for xs in xss] [1, 2, 3, 'abc', 'def', 99] >>> This is somewhat seductive, and the following is even nicer: the *xs position may be an expression, e.g.: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs[:2] for xs in xss] [1, 2, 'abc', 'def', 99] >>> On the other hand, I had to explore the possibilities here by experimenting in the interpreter, and I discovered some odd edge cases (e.g. you can parenthesize the starred expression, but that seems a syntactic accident). All in all I am personally +0 on the comprehension part of the PEP, and I like that it provides a way to "flatten" a sequence of sequences, but I think very few people in the thread have supported this part. Therefore I would like to ask Neil to update the PEP and the patch to take out the comprehension part, so that the two "easy wins" can make it into Python 3.5 (basically, I am accepting two-thirds of the PEP :-). There is some time yet until alpha 2. I would also like code reviewers (Benjamin?) to start reviewing the patch <http://bugs.python.org/issue2292>, taking into account that the comprehension part needs to be removed. -- --Guido van Rossum (python.org/~guido) [View Less]

7 17

PEP 638: Syntactic macros
by Mark Shannon Feb. 5, 2023

Feb. 5, 2023

Hi everyone, I've submitted my PEP on syntactic macros as PEP 638. https://www.python.org/dev/peps/pep-0638/ All comments and suggestions are welcome. Cheers, Mark

16 25

Switching to Discourse
by Petr Viktorin Jan. 26, 2023

Jan. 26, 2023

Hello, Currently development discussions are split between multiple communication channels, for example: - python-dev and discuss.python.org for design discussions, - GitHub Issues and Pull Requests for specific changes, - IRC, Discord and private chats for real-time discussions, - Topic-specific channels like typing-sig. While most of these serve different needs, there is too much overlap between python-dev and discuss.python.org. It seems that for most people, this situation is worse than … [View More]sticking to either one platform – even if we don't go with that person's favorite. The discuss.python.org experiment has been going on for quite a while, and while the platform is not without its issues, we consider it a success. The Core Development category is busier than python-dev. According to staff, discuss.python.org is much easier to moderate.. If you're following python-dev but not discuss.python.org, you're missing out. The Steering Council would like to switch from python-dev to discuss.python.org. Practically, this means: - Moving the required PEP announcements to discuss.python.org - Moving discuss.python.org up in the devguide communications page (https://devguide.python.org/communication/) - And that's it? I imagine that the mailing list will stay around for continuing past discussion threads and for announcements, eventually switching to auto-reject incoming messages with a pointer to discuss.python.org. To be clear, discuss.python.org allows editing posts, which is frankly handy for typos and clarifications. Editing alone should not be used for adding new info -- we should cultivate a culture of being friendly to mail users & notification watchers. This probably bears repeating in a few places. We're aware not everyone wants to use the discuss.python.org website, but there are some ways to avoid it: - For new PEPs, you can point your RSS client to https://www.python.org/dev/peps/peps.rss – it's not e-mail, but many email clients have RSS support. You can also watch the Steering Council issues on GitHub (https://github.com/python/steering-council/issues/) for important questions and discussions. - You can use discuss.python.org's “mailing list mode” (which subscribes you to all new posts), possibly with filtering and/or categorizing messages locally. However, we would like to know if this will pose an undue burden to anyone, if there are workflows or usage problems that we are not aware of. As mentioned, this is something the Steering Council thinks is a good idea, but we want to make sure we're aware of all the impact when we make the final decision. – Petr, on behalf of the Steering Council [View Less]

43 114

A proposal to modify `None` so that it hashes to a constant
by Yoni Lavi Dec. 3, 2022

Dec. 3, 2022

I wrote a doc stating my case here: https://docs.google.com/document/d/1et5x5HckTJhUQsz2lcC1avQrgDufXFnHMin7GlI… Briefly, 1. The main motivation for it is to allow users to get a predictable result on a given input (for programs that are doing pure compute, in domains like operations research / compilation), any time they run their program. Having stable repro is important for debugging. Notebooks with statistical analysis are another similar case where this is needed: you might want other … [View More]

11 29

RFC: expose ssl internals for use by ctypes/cffi
by christer＠weinigel.se Dec. 1, 2022

Dec. 1, 2022

Hi all, for quite some time I've been working on a Python implementation of a protocol called NTS which requires access to an API in OpenSSL which is not provided by the Python ssl module. I added a patch for that which unfortunately for me the maintainer did not want to accept. Some comments were made of a possible future way to give more generic access to the openssl library via ctypes/cffi but I have been unable to find more information about that. I was home sick last week and decided … [View More]to take a shot at it and have built something that I feel is a bit ugly but does seem to work. I'd like to some feedback on this approach. My patches can be found on github, based on the Python 3.11 tag: https://github.com/python/cpython/compare/3.11...wingel:cpython:main Here's a short description of each patch on this branch: "bpo-37952: SSL: add support for export_keying_material" is my old patch which adds the method I need to the ssl library just for reference. The other commits add the necessary infrastructure with some example code. These commits are not ready for submission but hopefully they show what I have in mind. "Add CRYPTO_DLL_PATH and SSL_DLL_PATH to the _ssl module. " This commit adds two constants to the "_ssl" C module with the paths to libcrypto and libssl respectively. On Linux dladdr and on Windows GetModuleHandle/GetModuleFilename are used on a symbol in each library to find the path to the corresponding DLL. I've verified that this works Debian Bulleye and on Windows 10 with Visual Studio 2017. I don't own a Mac so I haven't been able to test this on macOS, but I believe dladdr is available on modern macOS so it might work out of the box. With the paths it's possible to use ctypes or cffi get a handle to these libraries. "Add API to get the address of the SSL structure" then adds an API to an SSLSocket which returns the address of the corresponding "SSL" C structure. This address can be used by ctypes/cffi. One would probably want to expose SSL_CTX, SSL_SESSION and BIO too but I started with just SSL since that's what my code needs right now. "Add a small test program" is a small test program that uses the infrastructure from the two above commits to call C functions in libssl/libcrypto using both ctypes and cffi. It's a bit ugly but hopefully it's not too hard to understand. "Example of how to extend the ssl library using ctypes" is an example of how a Python module that extends the SSL library using ctypes could look. First get a handle to libssl using ctypes, set up ctypes with the correct API for the export_keying_material function, wrap it in a more Pythonic function and then extend SSLSocket with the new function. A simplified version looks like this: import ssl, ctypes ssl_lib = ctypes.CDLL(ssl._ssl.SSL_DLL_PATH) ssl_lib.SSL_export_keying_material.argtypes = ( ctypes.c_void_p, # SSL pointer ctypes.c_void_p, ctypes.c_size_t, # out pointer, out length ctypes.c_void_p, ctypes.c_size_t, # label buffer, label length ctypes.c_void_p, ctypes.c_size_t, # context, context length ctypes.c_int) # use context flag ssl_lib.SSL_export_keying_material.restype = ctypes.c_int def SSL_export_keying_material(self, label, key_len, context = None): c_key = ctypes.create_string_buffer(key_len) c_label = ctypes.create_string_buffer(len(label)) c_context = ctypes.create_string_buffer(context, len(context)) if ssl_lib.SSL_export_keying_material( self._sslobj.get_internal_addr(), c_key, key_len, c_label, len(label), c_context, len(context), 1); return bytes(c_key) ssl.SSLSocket.export_keying_material = SSL_export_keying_material There's a final commit "Expose more OPENSSL_ variables" which add some more constants to the ssl module which expose the cflags and build information from OpenSSL. This patch is not really necessary, but it might be a good idea to compare these constants with the corresponding constants retrieved using ctypes/cffi to make sure that exactly the same version of the openssl library is used. Does this seem like a good idea? As I said, I feel that it is a bit ugly, but it does mean that if someone wants to use some SSL_really_obscure_function in libcrypto or libssl they can do that without having to rebuild all of CPython themselves. Or if they want to integrate with some other C library that wants a raw pointer to a SSL socket. Hopefully this would reduce the burden on the ssl module maintainers a bit. Anyway, if you think this is a good approach I could clean up my patches, add support for SSL_CTX/SSL_SESSION/BIO, document all of this and make it into a proper pull request. /Christer [View Less]

5 4

Re: Python-Dev Digest, Vol 232, Issue 10
by Yoni Lavi Nov. 29, 2022

Nov. 29, 2022

> I can't see any, but then I couldn't see the security consequences of > predictable string hashes until they were pointed out to me. So it would > be really good to have some security experts comment on whether this is > safe or not. I can't either. I can point out that the complexity attack via hash collisions is not possible on None specifically because it is a singleton, so: 1. There may only be one None in a dict at most, 2. For composite types that depend on None and other … [View More]things, like say a tuple of string and an optional int, they become as resistant to attack as the same types without None, in this case string and int. 3. Python only explicitly added this security (hash randomization) feature to string and bytes hashes, and if your composite key type depends on those types, it will still be protected regardless of what None does > Because this entire discussion is motivated by the OP who wants > consistent set order across multiple runs of his Python application. > That's what he needs; having None hash to a constant value is just a > means to that end. Not entirely. I do explain in my doc why there is a foundational reason why, once the choice was made to have None represent the `None` of Optional[T], it entered the family of types that you would expect to have deterministic behavior. And as the hashing function is intrinsic to types in Python, it is included in that. > Even if we grant None a constant hash, that still does not guarantee > consistent set order across runs. At best, we might get such consistent > order as an undocumented and changeable implementation detail, until we > change the implementation of hashing, or of sets, or of something > seemingly unrelated like address randomisation. ASLR will not cause any trouble so long as I keep objects with identity based hashing out of my sets. Or at least, any sets I later iterate on and run imperative code with side-effects under the loop. The possibility of disabling ASLR is not a reason to dismiss this change. No one guarantees a user of Python is in a position to make infosec related decisions on the computer system they are working on. Regarding the possibilities of hashes changing for the worse (in this regard) - sure. Anything is possible. Regarding sets - the set implementation is deterministic, and always has been (for decades). Yes, in theory it is possible that a new version or implementation of Python will make sets non-deterministic - shuffle themselves in the background independently from their history of operations, or what have you, and then my change loses all of its value. Like I said, anything is possible. But I think these last two points are essentially FUD. I made my proposal because I believe the FUD scenarios are strongly unlikely. (and even then, at worst we end up with a "practically useless" behavior on None, that can also be freely reverted along with such other changes anyway) On Tue, Nov 29, 2022 at 5:34 AM <python-dev-request(a)python.org> wrote: > Send Python-Dev mailing list submissions to > python-dev(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman3/lists/python-dev.python.org/ > or, via email, send a message with subject or body 'help' to > python-dev-request(a)python.org > > You can reach the person managing the list at > python-dev-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-Dev digest..."Today's Topics: > > 1. Re: A proposal to modify `None` so that it hashes to a constant > (Steven D'Aprano) > 2. Re: A proposal to modify `None` so that it hashes to a constant > (Oscar Benjamin) > 3. Re: A proposal to modify `None` so that it hashes to a constant > (Chris Angelico) > 4. Re: A proposal to modify `None` so that it hashes to a constant > (Steven D'Aprano) > > > > ---------- Forwarded message ---------- > From: "Steven D'Aprano" <steve(a)pearwood.info> > To: python-dev(a)python.org > Cc: > Bcc: > Date: Mon, 28 Nov 2022 15:35:44 +1100 > Subject: [Python-Dev] Re: A proposal to modify `None` so that it hashes to > a constant > On Tue, Nov 29, 2022 at 01:34:54PM +1300, Greg Ewing wrote: > > > I got the impression that there were some internal language reasons > > to want stable dicts, e.g. so that the class dict passed to __prepare__ > > preserves the order in which names are assigned in the class body. Are > > there any such use cases for stable sets? > > Some people wanted order preserving kwargs, I think for web frameworks. > There was even a discussion for a while about using OrderedDict for > kwargs and leaving dicts unordered. > > For me, the biggest advantage of preserving input order in dicts is that > it is now easier to write doctests using the dict's repr. It would be > nice to be able to do the same for sets, but not nice enough to justify > making them bigger or slower. > > > -- > Steve > > > > ---------- Forwarded message ---------- > From: Oscar Benjamin <oscar.j.benjamin(a)gmail.com> > To: > Cc: python-dev(a)python.org > Bcc: > Date: Tue, 29 Nov 2022 02:07:34 +0000 > Subject: [Python-Dev] Re: A proposal to modify `None` so that it hashes to > a constant > On Tue, 29 Nov 2022 at 01:33, Steven D'Aprano <steve(a)pearwood.info> wrote: > > > > On Mon, Nov 28, 2022 at 11:13:34PM +0000, Oscar Benjamin wrote: > > > On Mon, 28 Nov 2022 at 22:56, Brett Cannon <brett(a)python.org> wrote: > > > > As I understand it, we could make sets ordered, but only at the cost of > > space (much more memory) or time (slower) or both. > > > > I am sure that Guido is correct that **if** somebody comes up with a > > fast, efficient ordered set implementation that doesn't perform worse > > than the current implementation, we will happily swap to giving sets a > > predictable order, as we did with dicts. (Practicality beats purity -- > > even if sets are *philosophically* unordered, preserving input order is > > too useful to give up unless we gain something in return.) > > Let's split this into two separate questions: > > 1. Is it *innately* good that set order is non-deterministic? > 2. Are there some other reasons why it is good to choose a model that > implies *non-deterministic* set order? > > The answer to 1. is emphatically NO. In fact the question itself is > badly posed: why are we even asking about "set order" rather than the > benefits of determinism in general? If I want my code to be > deterministic then that's just something that I want regardless of > whether sets, dicts, floats etc are involved. > > As for point 2. the fact that sets are currently non-deterministic is > actually a relatively new thing in Python. Before hash-randomisation > set and dict order *was* deterministic but with an arbitrary order. > That was only changed because of a supposed security issue with hash > collisions. Prior to that it was well understood that determinism was > beneficial (honestly I don't understand why I have to state this point > explicitly: determinism is almost always best in our context). > > Please everyone don't confuse arbitrary order, implementation defined > order and non-deterministic order. There is no reason why sets in > Python need to have a *non-deterministic* order or at least why there > shouldn't be a way to control that. There is no performance penalty in > making the order *deterministic*. (If you think that there might be a > performance penalty then you haven't understood the suggestion!) > > > > It would be useful to have a straight-forward way to sort a set into a > > > deterministic ordering but no such feature exists after the Py3K > > > changes (sorted used to do this in Python 2.x). > > > > `sorted()` works fine on homogeneous sets. It is only heterogeneous sets > > that are a problem, and in practice, that usually means None mixed in > > with some other type. > > That is of course precisely the context for this thread! > > -- > Oscar > > > > ---------- Forwarded message ---------- > From: Chris Angelico <rosuav(a)gmail.com> > To: python-dev(a)python.org > Cc: > Bcc: > Date: Tue, 29 Nov 2022 13:20:30 +1100 > Subject: [Python-Dev] Re: A proposal to modify `None` so that it hashes to > a constant > On Tue, 29 Nov 2022 at 13:12, Oscar Benjamin <oscar.j.benjamin(a)gmail.com> > wrote: > > As for point 2. the fact that sets are currently non-deterministic is > > actually a relatively new thing in Python. Before hash-randomisation > > set and dict order *was* deterministic but with an arbitrary order. > > That was only changed because of a supposed security issue with hash > > collisions. Prior to that it was well understood that determinism was > > beneficial (honestly I don't understand why I have to state this point > > explicitly: determinism is almost always best in our context). > > To clarify: The hash collision attack is a very real one, but specific > to dictionaries of string keys, since there are quite a few ways for > an attacker to send a string that gets automatically parsed into such > a dictionary (eg web app frameworks where the request parameters are > made available as a dictionary). But since that attack surface is *so* > specific, randomization of non-string hashes is unimportant. > > ChrisA > > > > ---------- Forwarded message ---------- > From: "Steven D'Aprano" <steve(a)pearwood.info> > To: python-dev(a)python.org > Cc: > Bcc: > Date: Mon, 28 Nov 2022 17:19:22 +1100 > Subject: [Python-Dev] Re: A proposal to modify `None` so that it hashes to > a constant > On Tue, Nov 29, 2022 at 02:07:34AM +0000, Oscar Benjamin wrote: > > > Let's split this into two separate questions: > > Let's not. Your first question about non-deterministic set order being > "innately good" is a straw man: as we've already discussed, set order is > not non-deterministic (except in the informal sense of "hard to > predict") and I don't think anyone is arguing in favour of keeping set > order unpredictable even if there are faster, more compact, simpler > implementations which preserve order. > > Talking about determinism is just muddying the waters. Sets are > deterministic: their order is determinied by the implementation, the set > history, and potentially any environmental sources of entropy used in > address randomisation. Deterministic does not mean predictable. > > If we want to get this discussion onto a more useful path, we should > start with a security question: > > The hash of None changes because of address randomisation. Address > randomisation is enabled as a security measure. Are there any security > consequences of giving None a constant hash? > > I can't see any, but then I couldn't see the security consequences of > predictable string hashes until they were pointed out to me. So it would > be really good to have some security experts comment on whether this is > safe or not. > > > > why are we even asking about "set order" rather than the > > benefits of determinism in general? > > Because this entire discussion is motivated by the OP who wants > consistent set order across multiple runs of his Python application. > That's what he needs; having None hash to a constant value is just a > means to that end. > > His sets contain objects whose hashes depend on `Optional[int]`, which > means sometimes they include None, and when he runs under an interpreter > built with address randomisation, the hash of None can change. > > Even if we grant None a constant hash, that still does not guarantee > consistent set order across runs. At best, we might get such consistent > order as an undocumented and changeable implementation detail, until we > change the implementation of hashing, or of sets, or of something > seemingly unrelated like address randomisation. > > > -- > Steve > _______________________________________________ > Python-Dev mailing list -- python-dev(a)python.org > To unsubscribe send an email to python-dev-leave(a)python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > [View Less]

1 0

Python 3.12.0 alpha 2 released
by Thomas Wouters Nov. 15, 2022

Nov. 15, 2022

I'm pleased to announce the release of Python 3.12 alpha 2. https://www.python.org/downloads/release/python-3120a2/ *This is an early developer preview of Python 3.12*. Major new features of the 3.12 series, compared to 3.11 Python 3.12 is still in development. This release, 3.12.0a2 is the second of seven planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process. During the alpha phase, … [View More]features may be added up until the start of the beta phase (2023-05-08) and, if necessary, may be modified or deleted up until the release candidate phase (2023-07-31). Please keep in mind that this is a preview release and its use is **not** recommended for production environments. Many new features for Python 3.12 are still being planned and written. Among the new major new features and changes so far: - Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user. - Support for the Linux perf profiler to report Python function names in traces. - The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623 <https://peps.python.org/pep-0623>. - In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2). - The deprecated smtpd and distutils modules have been removed (see PEP 594 <https://peps.python.org/pep-0594/> and PEP 632 <https://peps.python.org/pep-0632/>). The setuptools package (installed by default in virtualenvs and many other places) continues to provide the distutils module. - A number of other old, broken and deprecated functions, classes and methods have been removed. - (Hey, **fellow core developer,** if a feature you find important is missing from this list, let Thomas know <thomas(a)python.org>.) For more details on the changes to Python 3.12, see What's new in Python 3.12 <https://docs.python.org/dev/whatsnew/3.12.html>. The next pre-release of Python 3.12 will be 3.12.0a3, currently scheduled for 2022-12-05. More resources Online Documentation <https://docs.python.org/3.12/>. PEP 693 <https://www.python.org/dev/peps/pep-0693/>, the Python 3.12 Release Schedule. Report bugs via GitHub Issues <https://github.com/python/cpython/issues>. Help fund Python and its community <https://www.python.org/psf/donations/>. And now for something completely different Life, believe, is not a dream > So dark as sages say; > Oft a little morning rain > Foretells a pleasant day. > Sometimes there are clouds of gloom, > But these are transient all; > If the shower will make the roses bloom, > O why lament its fall? > > Rapidly, merrily, > Life's sunny hours flit by, > Gratefully, cheerily, > Enjoy them as they fly! > > What though Death at times steps in > And calls our Best away? > What though sorrow seems to win, > O'er hope, a heavy sway? > Yet hope again elastic springs, > Unconquered, though she fell; > Still buoyant are her golden wings, > Still strong to bear us well. > > Manfully, fearlessly, > The day of trial bear, > For gloriously, victoriously, > Can courage quell despair! *Life*, by Charlotte Brontë, from *Poems by Currer, Ellis, and Acton Bell*. Charlotte wrote about the publishing under pseudonyms by her and her sisters, Emily and Anne, in a preface to Emily's *Wuthering Heights*: Averse to personal publicity, we veiled our own names under those of > Currer, Ellis and Acton Bell; the ambiguous choice being dictated by a sort > of conscientious scruple at assuming Christian names positively masculine, > while we did not like to declare ourselves women, because – without at > that time suspecting that our mode of writing and thinking was not what is > called "feminine" – we had a vague impression that authoresses are liable > to be looked on with prejudice; we had noticed how critics sometimes use > for their chastisement the weapon of personality, and for their reward, a > flattery, which is not true praise. Enjoy the new releases Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Regards from chilly Amsterdam, Your release team, Thomas Wouters Ned Deily Steve Dower -- Thomas Wouters <thomas(a)python.org> [View Less]

1 0

[CVE-2022-37454] SHA3 vulnerability and upcoming Python patches for 3.7 - 3.10
by mark_topham＠rapid7.com Nov. 10, 2022

Nov. 10, 2022

I’m looking for help understanding how Python will release fixes related to the SHA3 critical security vulnerability (CVE-2022-37454). I’ve tried to figure this out myself, but I’m far from a Python expert and I’m not sure where else I should look. Apologies in advance if this is the wrong place to ask - if it is, a redirect to the correct place would be most appreciated. Here’s what I’ve found so far: * Python versions 3.6 through 3.10 appear to be affected * 3.6 is end of life, so no … [View More]

5 5

August steering council update
by Pablo Galindo Salgado Nov. 9, 2022

Nov. 9, 2022

Below, you can find the just-published Steering Council update for August. September and October coming soon. Apologies for the delay! https://github.com/python/steering-council/blob/main/updates/2022-08-steeri… As a reminder, if you have any questions or concerns, feel free to contact us or open an issue in the SC repo: https://github.com/python/steering-council # SC meeting summary - August 2022 ## 2022-08-01 * The Steering Council met with the Developer-in-Residence (Łukasz) and … [View More]discussed: * The status of the 3.11 release * the asyncio uncancel issue * The number of last-minute bugs and issues that we are experiencing * The SC discussed the fall Core Dev Sprint with the PSF Executive Director, as well as plans for another Developer-in-Residence and offering mentoring support to core developers. * The SC discussed the Python 3.11 release and the approximate cause of the release blockers, as well as how to improve the situation in the future and how to gather feedback for future release managers. ## 2022-08-08 * The SC discussed the status and next steps for CVE-2020-10735: Prevent DoS by large int<->str conversions. * Pablo asked the Docs working group for assistance with the 3.11 release notes. * The SC discussed the status of the repo for planet.python.org. The SC will ask Ee about moving the planet repo over to the PSF organization. * The SC discussed how to move forward with the future of type annotations ([PEP 563 – Postponed Evaluation of Annotations](https://peps.python.org/pep-0563/) and [PEP 649 – Deferred Evaluation Of Annotations Using Descriptors](https://peps.python.org/pep-0649/)). Several maintenance considerations and deprecation strategies were considered, listing things that would break and ways to overcome the challenges. * The SC discussed the state of [PEP 2 – Procedure for Adding New Modules](https://peps.python.org/pep-0002/), [PEP 4 – Deprecation of Standard Modules](https://peps.python.org/pep-0004/), and [PEP 411 – Provisional packages in the Python standard library](https://peps.python.org/pep-0411/). The SC decided to email the community to see if they think we should have separate policy documents from PEP. * The SC discussed asking the platform release managers (Ned and Steve) to document their side of the release process to reduce the bus factor and search for more automation possibilities. ## 2022-08-15 * The SC met with the Developer in Residence and discussed: * Fixing blockers for the 3.11.0rc1 release * An update of the CLA bot, and a future update to EdgeDB 2.0 * Possible candidates for blog posts for the community * The SC met with the PSF Executive Director and discussed the organization of the 2022 Core Dev sprints. * The SC discussed the logistics of having a meeting with Meta as part of the Visionary sponsor relationship. * The SC discussed encouraging projects to release wheels for Release Candidates of new minor versions and what challenges are being faced by users and maintainers. Several possible proposals were listed, including asking NumFocus and other organizations for assistance. ## 2022-08-22 * The SC briefly discussed [PEP 649 – Deferred Evaluation Of Annotations Using Descriptors](https://peps.python.org/pep-0649/) and what is left to decide. * The SC discussed several aspects of the upcoming core dev sprint. * The SC discussed planning of the meeting with Meta as a PSF Visionary sponsor benefit. * The SC discussed possibly delaying 3.11.0 due to a timing conflict with release managers and the core dev sprint and life in general. It was agreed that it is ok to delay 3.11.0 until the 24th of October. * The SC confirmed that 3.12alpha1 is still intended during the dev sprint. * As a way to improve the testing of new versions, Brett noted he's planning to work to improve GitHub CI configs to make it more likely for users to test against release candidates and betas and to make it obvious how to do it. * The SC discussed the migration concerns raised from the proposed migration for PEP discussion from python-dev to discuss.python.org migration. The SC noted that documentation work remains before making it fully official. Also, we won't shut down the python-dev list, but we expect PEP authors to not pay attention to it when discussing PEPs. * The SC discussed the possibility of inviting some non-core devs to the sprint and we agreed that a goal of focus is key for a successful sprint. * The SC discussed what to do with PEPs that are not regular enhancement proposals nor informative documents (like https://peps.python.org/pep-0659/). The SC decided that for [PEP 659– Specializing Adaptive Interpreter](https://peps.python.org/pep-0659/) in particular the status should be Final. * The SC agreed that we want to avoid having people write a PEP after pushing through an implementation. * The SC discussed CPython's backward compatibility policy and decided that we should gather input from the core devs on that. It was proposed to ask people attending the core dev sprint as a first measure. ## 2022-08-22 * The SC met with the Developer in Residence and discussed: * Bot/CLA hosting and Heroku tiers. * EdgeDB update for the CLA bot. * Blog posts and talk ideas. * Typing PEPs. * The SC discussed details about the organization of the 2022 Core Dev sprints. * The SC discussed resources for mentoring and how to help core devs mentor new contributors. * The SC discussed [PEP 674 – Disallow using macros as l-values](https://peps.python.org/pep-0674/) and the changes that were merged without a PEP accepted in 3.11. Pablo will answer [to the discussion that Victor started in discourse](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-…. * The SC discussed several aspects on how to deal with breaking changes. It was decided that this will be a discussion point with the rest of the core devs in the core dev sprint. In representation of the Steering Council, Pablo Galindo Salgado [View Less]

1 0

NEWLINE sentinel behavior in CPython's PEG grammar
by David J W Nov. 3, 2022

Nov. 3, 2022

I am writing a Rust version of Python for fun and I am at the parser stage of development. I copied and modified a PEG grammar ruleset from another open source project and I've already noticed some problems (ex Newline vs NL) with how they transcribed things. I am suspecting that CPython's grammar NEWLINE is a builtin rule for the parser that is something like `(Newline+ | NL+ ) {NOP}` but wanted to sanity check if that is right before I figure out how to hack in a NEWLINE rule and update my grammar ruleset.

5 9

TypedMapping structural type
by Alice Purcell Nov. 2, 2022

Nov. 2, 2022

I would like to propose adding a new TypedMapping structural type, which (like Mapping vs dict) functions as a read-only equivalent of TypedDict. (This is a cross-post from typing-sig@, where our original proposal got refined.) # Specification ```python class Request(TypedMapping, total=False): some_field: string | None request: Request = { "some_field": "foo" } request2: Request = {} # fine because total=False request["some_field"] = None # error del request["some_field"] # error ``` … [View More]

1 0