Mailman 3 July 2022 - Python-Dev

Presenting PEP 692: Using TypedDict for more precise **kwargs typing
by Franek Magiera 12 Jul '22

12 Jul '22

Hello, PEP 692 is posted. It proposes to use TypedDicts to type **kwargs that consist of different types. This feature has gained a lot of interest from the community (https://github.com/python/mypy/issues/4441). The PEP also proposes a grammar change and a new dunder __unpack__. Here is the link to the PEP itself - https://peps.python.org/pep-0692/ And the discussion on discuss.python.org - https://discuss.python.org/t/pep-692-using-typeddict-for-more-precise-kwarg… Please share your comments, thoughts and feedback! Franek

1 0

Structural pattern matching and mangling private names
by dw-git＠d-woods.co.uk 12 Jul '22

12 Jul '22

For this code: class C: def f(self, x): match x: case D(__something=y): return y It appears that the name "__something" isn't mangled. Under most other circumstances I'd expect this to be mangled to "_C__something". Is this: * intentional, * accidental, but how that it's done it's the defined behaviour, * or a defect? It doesn't seem like it's explicitly tested for in test_patma.py either way. Thanks David

3 5

Question about code in test_email/test_message.py
by Sidney Markowitz 07 Jul '22

07 Jul '22

I'm writing up a PR for an issue I created and adding a test to test/test_email/test_message.py In that file there is a variable named message_params that is initialized with about 15 different sets of test data, in class TestEmailMessageBase, but is never referenced again. I even grepped for the variable name in all test py files to confirm that it isn't somehow imported somewhere. Is there something about the test framework that uses that automatically? I'm hesitant to add any testing code for my issue if I'm missing something so fundamental. Thanks.

2 1

Python 3.11 bytecode and exception table
by Matthieu Dartiailh 05 Jul '22

05 Jul '22

Hi all, I am the current maintainer of bytecode (https://github.com/MatthieuDartiailh/bytecode) which is a library to perform assembly and disassembly of Python bytecode. The library was created by V. Stinner. I started looking in Python 3.11 support in bytecode, I read Objects/exception_handling_notes.txt and I have a couple of questions regarding the exception table: Currently bytecode exposes three level of abstractions: - the concrete level in which one deals with instruction offset for jumps and explicit indexing into the known constants and names - the bytecode level which uses labels for jumps and allow non integer argument to instructions - the cfg level which provides basic blocks delineation over the bytecode level So my first idea was to directly expose the unpacked exception table (start, stop, target, stack_depth, last_i) at the concrete level and use pseudo-instruction and labels at the bytecode level. At this point of my reflections, I saw https://github.com/python/cpython/commit/c57aad777afc6c0b382981ee9e4bc94c03… about adding pseudo-instructionto dis output in 3.12 and though it would line up quite nicely. Reading through, I got curious about how SETUP_WITH handled popping one extra item from the stack so I went to look at dis results on a couple of small examples. I tried on 3.10 and 3.11b3 (for some reasons I cannot compile main at a391b74d on windows). I looked at simple things and got a bit surprised: Disassembling: deff(): try: a= 1 except: raise I get on 3.11: 1 0 RESUME 0 2 2 NOP 3 4 LOAD_CONST 1 (1) 6 STORE_FAST 0 (a) 8 LOAD_CONST 0 (None) 10 RETURN_VALUE >> 12 PUSH_EXC_INFO 4 14 POP_TOP 5 16 RAISE_VARARGS 0 >> 18 COPY 3 20 POP_EXCEPT 22 RERAISE 1 ExceptionTable: 4 to 6 -> 12 [0] 12 to 16 -> 18 [1] lasti On 3.10: 2 0 SETUP_FINALLY 5 (to 12) 3 2 LOAD_CONST 1 (1) 4 STORE_FAST 0 (a) 6 POP_BLOCK 8 LOAD_CONST 0 (None) 10 RETURN_VALUE 4 >> 12 POP_TOP 14 POP_TOP 16 POP_TOP 5 18 RAISE_VARARGS 0 This surprised me on two levels: - first I have never seen the RESUME opcode and it is currently not documented - my second surprise comes from the second entry in the exception table. At first I failed to see why it was needed but writing this I realize it corresponds to the explicit handling of exception propagation to the caller. Since I cannot compile 3.12 ATM I am wondering how this plays with pseudo-instruction: in particular are pseudo-instructions generated for all entries in the exception table ? My initial idea was to have a SETUP_FINALLY/SETUP_CLEANUP - POP_BLOCK pair for each line in the exception table and label for the jump target. But I realize it means we will have many such pairs than in 3.10. It is fine by me but I wondered what choice was made in 3.12 dis and if this approach made sense. Best regards Matthieu

3 4

Announcing the Diátaxis Documentation Workshop
by Mariatta 05 Jul '22

05 Jul '22

Diátaxis Documentation Workshop The Python documentation community <https://github.com/python/docs-community> will be hosting a free two-part workshop by Daniele Procida <https://vurt.org/>, the creator of the Diátaxis framework <https://diataxis.fr/>. Each session will be two hours; including an introduction, discussion/questions, and practical exercises in small groups; and will be hosted on Zoom. There’s an *optional* take-home exercise after the first session, which we’ll review during the second. The second session builds upon the first, so we strongly prefer that you attend both sessions to fully benefit from it. However, we may be able to accommodate a limited number of people that can only make it to one. They will be hands-on, so to get the most out of your attendance, be ready to participate and interact with Daniele and your fellow attendees! While the primary focus of the workshop is on contributors to the Python documentation, they are open to anyone in the open source community interested in writing better documentation, as long as space allows. To attend the workshop, sign up using the registration form <https://forms.gle/xQSZJ2Yf4Jjd27wf6>, and we’ll confirm your attendance by August 1. We will try our best to fill any remaining slots on a rolling space-available basis. The presentation portion of the workshop will be recorded (though the interactive aspect won’t be). <https://discuss.python.org/t/announcing-the-diataxis-documentation-workshop…>Workshop Agenda Session 1: Introduction to Diátaxis Tuesday, August 16 2022, 16:00-18:00 UTC (12 - 2 PM US Eastern / 9 - 11 AM US Pacific) <https://www.timeanddate.com/worldclock/fixedtime.html?msg=Di%C3%A1taxis+Wor…> - The Diátaxis system and how it works - Using it for documentation in general - Applying it to the Python docs specifically Session 2: Getting Things Done Thursday, August 18 2022, 16:00-18:00 UTC (12 - 2 PM US Eastern / 9 - 11 AM US Pacific) <https://www.timeanddate.com/worldclock/fixedtime.html?msg=Di%C3%A1taxis+Wor…> - Authoring practices and approaches - Getting work started and keeping it going - Making life easy for ourselves <https://discuss.python.org/t/announcing-the-diataxis-documentation-workshop…>About Daniele Procida I’m a Director of Engineering at Canonical <https://canonical.com/>, where I work to transform documentation practice across 40-plus engineering teams. Ever since I’ve become involved in software in 2009, I’ve been involved in its documentation too. I found my way to software through Python and Django, so that’s where I’ve made most of my direct contributions: Django, pytest, django CMS and others. Through the international Python community, I’ve had the enormous pleasure of taking part in numerous events and initiatives. The most important one to me is the African Python movement: since 2015, I’ve attended multiple editions of PyCon Namibia, PyCon Ghana and PyCon Africa. In recent years, I’ve had the fortune of seeing the approaches I’ve developed in my work in documentation adopted much more widely across the software industry. I am always thrilled to have the opportunity to contribute back through documentation to the same software communities that have lifted me up in my own software journey. <https://discuss.python.org/t/announcing-the-diataxis-documentation-workshop…>Getting in Touch If you have questions or concerns about the workshop, please post in the Docs Community forum <https://discuss.python.org/c/documentation/26> on Discourse. For private communications, please get in touch with one of the organizers: Mariatta, Ned Batchelder, and C.A.M. Gerlach <https://discuss.python.org/t/announcing-the-diataxis-documentation-workshop…> Acknowledgement Thanks to Canonical for their support and generosity in making Daniele’s participation in this workshop series possible.

1 0

[Release] Status of Python 3.11 release
by Pablo Galindo Salgado 04 Jul '22

04 Jul '22

Hi everyone, # TLDR We may be pushing the final release until December if the stability of Python 3.11 doesn't improve. # Long Explanation Unfortunately, we cannot still release the next Python 3.11 beta release (3.11.0b4) because we still have a bunch of pending release blockers. Unfortunately, this is still after several preexisting release blockers have been fixed, some of them being discovered after I sent my last update. These are the current release blockers: https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+sort%3Aupda… We also have some deferred blockers (some of them should actually be release blockers): https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+sort%3Aupda… Due to this and the fact that we are already 3 weeks delayed in the release schedule, the current stability of Python 3.11 is not as good as it is supposed to be at this stage of the release schedule and more testing from end-users and library authors is required. After discussing with the Steering Council, we are considering delaying the final release until December to allow for two more beta releases. This is how we are going to proceed: * If the current release blockers are not fixed by the end of this week, two more betas will be released (1 month per beta) and we will *definitely* delay the final release until December. * If the current release blockers are fixed we will proceed to release Python 3.11.0b4 on Monday. We will target the current release date (Monday, 2022-10-03) but if more release blockers that affect fundamental parts of the Python interpreter or the standard libraries are raised, the release team will still consider adding two more betas nd pushing the final release to December. One of the goals that we are going to try to achieve from the release team is that no substantial code changes are added between the last beta and the first release candidate. This is so all the fixes that affect fundamental parts of the interpreter or the standard library can be tested by end-users before the first release candidate is released (and not with it). This is also partially because once we release the first release candidate, the ABI will be frozen and certain kinds of fixes will be more complicated. Hopefully, this addresses some of you that have reached out with concerns over the stability of Python 3.11 and the release schedule. I understand that delaying the release until December will complicate things for some Linux distributions and will affect end users and redistributors targeting the original release, but please understand that our responsibility in the release team after all is to guarantee a stable final release above all and unfortunately, we don't currently have the confidence that we would like given the current state of the release process. Please do not hesitate in reaching out if you have any questions or concerns. Thanks, everyone for your help and understanding and thanks a lot to all of you for your great work! Cheers from cloudy London, Pablo Galindo Salgado

1 0

EPOLLEXCLUSIVE and selectors
by David Gilman 04 Jul '22

04 Jul '22

Hello, I'd like to bring up https://bugs.python.org/issue44951 / https://github.com/python/cpython/pull/27819 to the mailing list's consideration as it has idled for a bit. I would appreciate some authoritative feedback on which API design choice is best. I'll also recap the PR quickly: Motivation: - There is community demand for EPOLLEXCLUSIVE in Python (see blog posts in BPO issue) - You can't do this with the existing stdlib code as _BaseSelectorImpl.register() raises ValueError on non-READ/WRITE constants. (This is why _PollLikeSelector.register() has two variables, events and poller_events) - It's not an invasive change. The Python API doesn't change much or at all and the kernel's EPOLLEXCLUSIVE behavior was carefully designed to be backwards compatible with using exclusive and non-exclusive watches on the same file descriptor. I've got two approaches to this. The first extends the EpollSelector class with a property to toggle setting the EPOLLEXCLUSIVE on newly registered file descriptors: https://github.com/dgilman/cpython/commit/43174df5bd7a78eedf0692ebbe63a9b94… The second introduces an entirely new sibling class, EpollExclusiveSelector, that unconditionally sets it on registration: https://github.com/dgilman/cpython/commit/554a5bf9c16b6bd82ce47b2d0dd1833f2… The first one was my first attempt but I am leaning towards the second. It doesn't require any new API surface area. It also gets integrated into the DefaultSelector logic, and even if that shouldn't happen it's still easy to swap out your existing selector class for the EpollExclusiveSelector class. Enabling EPOLLEXCLUSIVE by default: From the research I did last year my understanding is that EPOLLEXCLUSIVE is never a performance drawback on Linux, and in the case of a server that gets fast traffic it's a dramatic improvement. However, I have not done my own benchmarking (with say, gunicorn, which uses the stdlib's selectors module). Note that EPOLLEXCLUSIVE does have one kernel API break: you can no longer use EPOLL_CTL_MOD on an exclusive file descriptor. Python uses the _MOD flag under the hood to implement epoll.modify(), which results in EpollSelector.modify() throwing an OSError if you try to modify an exclusive file descriptor. In the second PR I implemented a EpollExclusiveSelector.modify() which unregisters and reregisters the file descriptor to get around the _MOD behavior. This means no surprise crash when someone updates Python. But there may be other subtle regressions here: the performance of modify() is likely going to regress, and someone could have a dependency on Python actually using _MOD. But as a rebuttal to those, I have a feeling that nobody really depends on the performance of modify(), and even in the case where someone does use it a lot it's likely for the data= path which is completely unchanged here. I also am struggling to think of a place where someone would care about the kernel-level changes between _MOD and _ADD/_DEL but that might be my own lack of imagination or knowledge of epoll techniques. Maybe a compromise is to ship EpollExclusveSelector for a release without it being the default and bump it to the default after seeing if anyone's turned up any incompatibilities. -- David Gilman :DG<

2 1

[Release] Python 3.11.0b4 is still blocked
by Pablo Galindo Salgado 04 Jul '22

04 Jul '22

Hi everyone, A small update since the last communication from the release team regarding the status of Python 3.11.0b4. Unfortunately, even if we have fixed most of the original release blockers and 4 more that appear during this week, we still have a bunch of release blockers to deal with. One of them has been reported today. I would like to release the next beta next week if everything looks good, but there are also some items that need discussion:. https://github.com/python/cpython/issues/93910 https://github.com/python/cpython/issues/93516 Ideally we should reach consensus as a team on how to proceed in these issues. In particular, we should decide collectively what is an acceptable slowdown, specially looking to the release candidate. If releasing the next betas is further delayed, I will consider delaying the full release schedule to accommodate for the delay so users have the appropriate time to test and validate every release. Please do not hesitate in reaching out if you have any questions or concerns or if there is any issue you think we should include in the next release. Thanks everyone for your help and understanding. I'm sure 3.11 is going to be an outstanding release thank to all of you :) Cheers from cloudy London, Pablo Galindo Salgado

3 5