Mailman 3 September 2021 - Python-Dev

Should I care what -3.14 // inf produces?
by Jeff Allen 02 Oct '21

02 Oct '21

Is an implementation of Python free to make up its own answers to division and modulus operatons in the case of inf and nan arguments? The standard library documentation says only that // is "rounded towards minus infinity". The language reference says that ||: 1. |x == (x//y)*y + (x%y)|, 2. the modulus has the same sign as y, and 3. division by (either kind of) zero raises |ZeroDivisionError| . It's consistent, but it doesn't define the operator over the full range of potential arguments. In Python 3.8 and 3.9: >>> from decimal import Decimal >>> Decimal('-3.14') // Decimal('Infinity') Decimal('-0') >>> -3.14 // float("inf") -1.0 >>> import math >>> math.floor(-3.14 / float("inf")) 0 I can see sense in all three answers, as possible interpretations of "rounded towards minus infinity", but I quite like decimal's. There seem to be no regression tests for floor division of floats, and for modulus only with finite arguments, perhaps intentionally. -- Jeff Allen

3 3

Python 3.10 release party
by Pablo Galindo Salgado 28 Sep '21

28 Sep '21

Hi, As it seems that I didn't have enough work dealing with the multiple release blockers that have been raised these weeks I am also preparing a Python 3.10 release party. We will be doing an stream, co-hosted by the good people of Python discord channel. The URL for the stream is this one: https://youtu.be/AHT2l3hcIJg The release party will start on October 4th at 17:00 UTC+0. In this stream, some core developers (Irit, Carol, Brandt and Łukasz) will be talking about Python 3.10 features and some sneak peak on Python 3.11 and beyond, talking about the core Dev in residence role and other things with some time for moderated Q&A. I will be doing the release live and will also cover what I am doing. Is a fantastic opportunity to watch me make mistakes live :) The main purpose of the stream is to raise excitement about Python 3.10, explain the new features to users, trying to get the community closer to the core Dev team and allowing them to meet some of the new core Devs (Brandt and Irit). Also, funny party hats will be used! I hope you find the event interesting. Python 3.10 is going to be a fantastic release and we want it to be even better :) Please, reach out to me if you have any question or suggestion. Regards from very rainy London, Pablo Galindo Salgado

1 0

PEP: Taking the Python C API to the Next Level
by Victor Stinner 28 Sep '21

28 Sep '21

Hi, I would like to change the Python C API. I failed to write a single document listing all constraints and proposing all changes that I would to do. For example, my previous PEP 620 contains too many changes and is too long. Here is my attempt to focus on the bare minimum and (what I consider as) the least controversial part: list current usages of C API and constraints of these usages. This *informal* PEP should be the base of future PEP changing the C API. The current draft lives at: https://github.com/vstinner/misc/blob/main/cpython/pep-c-api-next-level.rst My PEP is based on HPy Next Level Manifesto written by Simon Cross: https://github.com/hpyproject/hpy/wiki/c-api-next-level-manifesto To reach most users of the C API, I cross-posted this email to python-dev, capi-sig and hpy-dev. +++++++++++++++++++++++++++++++++++++++++ Taking the Python C API to the Next Level +++++++++++++++++++++++++++++++++++++++++ :: PEP: xxx Title: Taking the Python C API to the Next Level Author: Victor Stinner <vstinner(a)python.org> Status: Draft Type: Informational Content-Type: text/x-rst Created: 28-Sep-2021 Python-Version: 3.11 While the C API is a key of the Python popularity, it causes multiple subtle and complex issues. There are different ways to use the C API, each usage has its own constraints, and some constraints are exclusive. This document lists constraints but doesn't propose changes, it only gives vague ideas how to solve some issues. More concrete C API changes will require writing separated PEPs. C extensions are a key component of the Python popularity ========================================================= The Python popularity comes from its great programming language and from its wide collection of modules freely available on PyPI. Many of the most popular Python modules rely directly or indirectly on C extensions written with the C API. The Python C API is a key component of the Python popularity. For example, the numpy project is now a common dependency on many scientific projects and a large part of the project is written by hand with the C API. **Abandoning or removing the C API** is out of question. Years ago, the incomplete C API support was the main drawback of PyPy, since PyPy only supported a minority of C extensions. Today, CPython still have a similar issue. **When Cython or numpy don't support a new Python version** (because of incompatible C API changes), many Python projects depending on them are cannot be installed, especially during the development phase of the next Python version. Backward compatibility and unmaintained C extensions ==================================================== One important property of the C API is the backward compatibility. Developers expect that if their C extension works on Python 3.10, it will work unmodified in Python 3.11: building the C extension with Python 3.11 should be enough. This property is even more important for unmaintained C extensions. Sometimes, unmaintained just means that the only maintainer is busy or overwhelmed for a few months. Sometimes, the project has no activity for longer than 5 years. When an incompatible change is introduced in the C API, like removing a function or changing a function behavior, there is a **risk of breaking an unknown number of C extensions**. One option can be to update old C extensions when they are built on recent Python versions, to adapt them to incompatible changes. This conversion is non trivial and cannot handle all kinds of incompatible changes. Migration plan for incompatible changes ======================================= There should be a **sensible migration path** for large C extensions (e.g. numpy) when incompatible changes are introduced. Whenever possible, it should be possible to write a **single code base** compatible with old and new Python versions. A **compatibility layer** can be maintained externally. Cython and numpy have their own internal compatibility layer. There should be a way to easily pick up common errors introduced by migrating. One practical way to **minimize the number of broken projects** is to attempt to check in advance if an incompatible change is going to break popular C extensions. For broken C extensions, propose a fix and wait until a new release includes the fix, before introducing the change in Python. Obviously, it doesn't solve the problem of less popular C extensions and private C extensions. Obtain the best possible performance ==================================== There are two main reasons for writing a C extension: implement a function which cannot be written in pure Python, or write a **C accelerator**: rewrite the 10% of an application in C where 90% of the CPU time is spent. About the former use case, the intent is to obtain the best possible performance. Tradeoffs are made with portability: it is acceptable to only support a limited number of Python versions and to only support a limited number of Python implementations (usually only CPython). Cython is a good example of accelerator. It is able to support a large number of Python versions and multiple Python implementation with compatibility layers and ``#ifdef``. The main drawback is that it is common that Cython is **broken by incompatible changes made at each Python release**. It happens because Cython relies on many implementation details. On the other side, the **limited C API** is a small as possible, excludes implementation details on purpose, and provides a stable ABI. Building a C extension with the limited C API only once produces a binary wheel package usable on many Python versions, but each platform still requires its own binary wheel package. Emulating the current C API is inefficient ========================================== The PyPy project is a Python implementation written from scratch, it was not created as a CPython fork. It made many implementation choices different than CPython: no reference counting, moving garbage collector, JIT compiler, etc. To support C extensions, PyPy emulates the Python C API in its cpyext module. When the C API access an object, cpyext has to convert the PyPy object to a CPython object (``PyObject``). CPython objects are less efficient than PyPy objects with the PyPy JIT compiler and conversions from PyPy objects to CPython objects are also inefficient. PyPy has to reimplement every single detail of the CPython implementation to be as much compatible as possible. The C API exposes multiple implementation details: * Reference counting, borrowed references, stealing references. * Objects location in memory. * Rely on pointers for object identity: Python 3.10 adds the ``Py_Is()`` function to solve this problem. * Expose the memory layout of Python objects as part of the API. * Expose static types. * Implicit execution context. * etc. The C API of Python 3.10 is made of around 15 000 lines of C header files, 1500 functions and 100 structures. Supporting the full C API is a significant amount of work. **Freezing the C API** for a few Python releases would help other Python implementations to catch up with the latest Python version, but it doesn't solve the efficiency problem. Moreover, it is common that adding a new feature to Python requires to change the C API, even if it is just to add new functions. Not adding new features to Python for a few Python releases is out of question. The C API prevents optimizing CPython ===================================== It is challenging to evolve the C API to optimize CPython without breaking the backward compatibility. Emulating the old C API is an option, but it is inefficient. If everything above is achievable -- and we believe it is! -- we'll arrive in a wonderful new future where Python implementations can experiment with all sorts of amazing new features: * tracing garbage collectors; * nurseries for short-lived objects; * sub-interpreters with separate contexts; * specialised implementations of lists; * removing the GIL; * avoiding the boxing of primitive types; * just-in-time compilation; * ... and many other things you can imagine that we haven't! No one can guarantee that a particular new idea will work out, but exposing fewer implementation details via the C API will make it possible to try many new things. -- Night gathers, and now my watch begins. It shall not end until my death.

1 0

The Default for python -X frozen_modules.
by Eric Snow 28 Sep '21

28 Sep '21

We've frozen most of the stdlib modules imported during "python -c pass" [1][2], to make startup a bit faster. Import of those modules is controlled by "-X frozen_modules=[on|off]". Currently it defaults to "off" but we'd like to default to "on". The blocker is the impact on contributors. I expect many will make changes to a stdlib module and then puzzle over why those changes aren't getting used. That's an annoyance we can avoid, which is the point of this thread. Possible solutions: 1. always default to "on" (the annoyance for contributors isn't big enough?) 2. default to "on" if it's a PGO build (and "off" otherwise) 3. default to "on" unless running from the source tree Thoughts? -eric [1] https://bugs.python.org/issue45020 [2] FWIW, we may end up also freezing the modules imported for "python -m ...", along with some other commonly used modules (like argparse). That is a separate discussion.

16 41

Re: The Default for python -X frozen_modules.
by Patrick Reader 27 Sep '21

27 Sep '21

I accidentally a word. "... module's hash and/or *timestamp* and comparing it ..." On 27/09/2021 18:38, Patrick Reader wrote: > How about checking each non-frozen module's hash and/or and comparing it to that of the frozen module? Would that defeat the performance improvement of freezing? Is it just a terrible idea? > > On 27/09/2021 17:51, Eric Snow wrote: >> We've frozen most of the stdlib modules imported during "python -c >> pass" [1][2], to make startup a bit faster. Import of those modules >> is controlled by "-X frozen_modules=[on|off]". Currently it defaults >> to "off" but we'd like to default to "on". The blocker is the impact >> on contributors. I expect many will make changes to a stdlib module >> and then puzzle over why those changes aren't getting used. That's an >> annoyance we can avoid, which is the point of this thread. >> >> Possible solutions: >> >> 1. always default to "on" (the annoyance for contributors isn't big enough?) >> 2. default to "on" if it's a PGO build (and "off" otherwise) >> 3. default to "on" unless running from the source tree >> >> Thoughts? >> >> -eric

2 1

Changing exception text in micro releases
by Eric V. Smith 25 Sep '21

25 Sep '21

This PR https://github.com/python/cpython/pull/28310 changes the message for some exceptions. Currently: >>> format('', '%M') Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: Invalid format specifier With the proposed change: >>> format('', '%M') Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: Invalid format specifier '%M' for object of type 'str' This also applies to str.format() and f-strings. This is clearly an improvement. My question is: is it okay to backport this to 3.9 and 3.10? I think we have a rule that it's okay to change the exception text, but I'm not sure how that applies to micro releases (3.9.x, 3.10.1). Eric

6 7

Accepting PEP 654.
by Thomas Wouters 24 Sep '21

24 Sep '21

Irit, Guido, Yuri, Nathaniel, (In the interest of visibility, this was also posted to discuss.python.org <https://discuss.python.org/t/accepting-pep-654-exception-groups-and-except/…> .) After careful consideration and rather lengthy discussion, the Steering Council has decided to accept PEP 654 (Exception Groups and except*) <https://www.python.org/dev/peps/pep-0654/>. We do have two requests, one of which we mentioned before. We’re very pleased with the amount of care and discussion that went into this PEP, and we’re satisfied that it solves a real-world problem in the least disruptive way. We do remain open for further tweaking of the design, incorporating some of Nathaniel’s ideas, or new ones, but we want to move forward with PEP 654’s proposal and let people get hands-on experience with it. To that end, our request is that we get at least one non-toy use of ExceptionGroups somewhere in the standard library (targeted to 3.11, like PEP 654), and preferably get some third-party experimentation with it as well. The other request is that PEP 654 be updated with a reference to Nathaniel’s ideas (a link to the Discourse discussion), outlining why PEP 654 rejects those ideas (it could just be a copy-paste of some of the discussion, just so long as it’s recorded in the PEP). Regarding Nathaniel’s alternative proposal <https://discuss.python.org/t/flat-exception-groups-alternative-to-pep-654>, I want to be clear on the main reason why we’re accepting PEP 654 rather than waiting for a more complete alternative PEP with concrete examples. The SC agrees that exception groups solve a real problem, albeit a relatively rare and low-level one. We don’t think the problem it solves is big enough that it warrants fundamental changes to the language; changes that would affect the way everyone would write code, or that would make learning Python or reasoning about Python code harder. PEP 654 just about squeaks by those criteria: the new syntax is something new to learn about Python, but it’s not something that’s *required *unless consuming an API that uses ExceptionGroups. ExceptionGroups behave like regular exceptions in code that does not expect them, and do not change the behaviour of existing try/except. In code that potentially produces ExceptionGroups (or its documentation, examples, tutorials, etc) there is enough opportunity to document that, show examples of how to use it, explain the implications and refer to documentation about the concept of ExceptionGroups. By contrast, Nathaniel’s proposal that ExceptionGroups affect how *existing *try/except statements behave, fails those criteria. By sheer nature of doing something meaningful (if confusing) in try/except, it would encourage API designs that let ExceptionGroups escape to unsuspecting code. It would introduce subtle bugs in code that -- entirely legitimately -- expects only one except block to be executed, or only executed once. These problems would be very hard to detect since they would only show up if called code could *in practice* raise ExceptionGroups. We don’t consider this a viable design choice. The SC does remain open to proposals to extend PEP 654 (like adding an API to iterate over ExceptionGroups), or change some of the semantics (like flat versus nested ExceptionGroups), and we do not want to discourage discussion on any of this. We do want to move forward with PEP 654 as-is in the meantime, in particular to get practical experience in the standard library. With thanks for all your hard work, For the whole SC, Thomas. -- Thomas Wouters <thomas(a)python.org> Hi! I'm an email virus! Think twice before sending your email to help me spread!

2 2

Summary of Python tracker Issues
by Python tracker 24 Sep '21

24 Sep '21

ACTIVITY SUMMARY (2021-09-17 - 2021-09-24) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7333 (-48) closed 49684 (+97) total 57017 (+49) Open issues with patches: 2907 Issues opened (37) ================== #34451: docs: tutorial/introduction doesn't mention toggle of prompts https://bugs.python.org/issue34451 reopened by rhettinger #45237: Python subprocess not honoring append mode for stdout on Windo https://bugs.python.org/issue45237 opened by wolfgang-kuehn #45239: email.utils.parsedate_tz raises UnboundLocalError if time has https://bugs.python.org/issue45239 opened by benhoyt #45240: Add +REPORT_NDIFF option to pdb tests that use doctest https://bugs.python.org/issue45240 opened by andrei.avk #45242: test_pdb fails https://bugs.python.org/issue45242 opened by oliphaunt #45243: [sqlite3] add support for changing connection limits https://bugs.python.org/issue45243 opened by erlendaasland #45244: pip not installed with fresh python3.8.10 installation https://bugs.python.org/issue45244 opened by shreyanse081 #45247: Add explicit support for Cython to the C API. https://bugs.python.org/issue45247 opened by Mark.Shannon #45249: SyntaxError location range indicator does not work in doctests https://bugs.python.org/issue45249 opened by andrei.avk #45250: Make sure documentation is accurate for what an (async) iterab https://bugs.python.org/issue45250 opened by brett.cannon #45251: signal.SIGCLD alias is not available on OSX https://bugs.python.org/issue45251 opened by tcaswell #45252: Missing support for Source Specific Multicast https://bugs.python.org/issue45252 opened by ciresnave #45253: mimetypes cannot detect mime of mka files https://bugs.python.org/issue45253 opened by manujchandra #45254: HAS_SHMEM detection logic is duplicated in implementation and https://bugs.python.org/issue45254 opened by sobolevn #45255: sqlite3.connect() should check if the sqlite file exists and t https://bugs.python.org/issue45255 opened by iboates #45256: Remove the usage of the C stack in Python to Python calls https://bugs.python.org/issue45256 opened by pablogsal #45258: sysroot_paths in setup.py does not consider -isysroot for macO https://bugs.python.org/issue45258 opened by isuruf #45260: Implement superinstruction UNPACK_SEQUENCE_ST https://bugs.python.org/issue45260 opened by zhangchaospecial #45261: Unreliable (?) results from timeit (cache issue?) https://bugs.python.org/issue45261 opened by timholy #45262: crash if asyncio is used before and after re-initialization if https://bugs.python.org/issue45262 opened by benjamin-sch #45264: venv: Make activate et al. export custom prompt prefix as an e https://bugs.python.org/issue45264 opened by jwodder #45266: subtype_clear can not be called from derived types https://bugs.python.org/issue45266 opened by Victor Milovanov #45267: New install Python 3.9.7 install of Sphinx Document Generator https://bugs.python.org/issue45267 opened by pbroe #45269: c_make_encoder() has uncovered error: "argument 1 must be dict https://bugs.python.org/issue45269 opened by sobolevn #45272: 'os.path' should not be a frozen module https://bugs.python.org/issue45272 opened by steve.dower #45273: OS-specific frozen modules are built, even on other OSes. https://bugs.python.org/issue45273 opened by eric.snow #45274: Race condition in Thread._wait_for_tstate_lock() https://bugs.python.org/issue45274 opened by vstinner #45275: Make argparse print description of subcommand when invoke help https://bugs.python.org/issue45275 opened by longendu #45276: avoid try 1000 in asyncio all_tasks by making weak collection https://bugs.python.org/issue45276 opened by graingert #45277: typo in codecs documentation https://bugs.python.org/issue45277 opened by Gronahak #45278: RuntimeError on race on weakset concurrent iteration https://bugs.python.org/issue45278 opened by bjs #45279: avoid redundant _commit_removals pending_removals guard https://bugs.python.org/issue45279 opened by graingert #45280: Empty typing.NamedTuple creation is not tested https://bugs.python.org/issue45280 opened by sobolevn #45281: Make `is_attribute` and `module` arguments to `ForwardRef` kw- https://bugs.python.org/issue45281 opened by sobolevn #45282: isinstance(x, typing.Protocol-class) unexpectedly evaluates pr https://bugs.python.org/issue45282 opened by daitakahashi #45283: Top / function argument level ClassVar should not be allowed d https://bugs.python.org/issue45283 opened by sobolevn #45284: Better `TypeError` message when a string is indexed using a no https://bugs.python.org/issue45284 opened by Rin Most recent 15 issues with no replies (15) ========================================== #45283: Top / function argument level ClassVar should not be allowed d https://bugs.python.org/issue45283 #45281: Make `is_attribute` and `module` arguments to `ForwardRef` kw- https://bugs.python.org/issue45281 #45280: Empty typing.NamedTuple creation is not tested https://bugs.python.org/issue45280 #45279: avoid redundant _commit_removals pending_removals guard https://bugs.python.org/issue45279 #45278: RuntimeError on race on weakset concurrent iteration https://bugs.python.org/issue45278 #45277: typo in codecs documentation https://bugs.python.org/issue45277 #45276: avoid try 1000 in asyncio all_tasks by making weak collection https://bugs.python.org/issue45276 #45275: Make argparse print description of subcommand when invoke help https://bugs.python.org/issue45275 #45269: c_make_encoder() has uncovered error: "argument 1 must be dict https://bugs.python.org/issue45269 #45264: venv: Make activate et al. export custom prompt prefix as an e https://bugs.python.org/issue45264 #45260: Implement superinstruction UNPACK_SEQUENCE_ST https://bugs.python.org/issue45260 #45254: HAS_SHMEM detection logic is duplicated in implementation and https://bugs.python.org/issue45254 #45253: mimetypes cannot detect mime of mka files https://bugs.python.org/issue45253 #45252: Missing support for Source Specific Multicast https://bugs.python.org/issue45252 #45247: Add explicit support for Cython to the C API. https://bugs.python.org/issue45247 Most recent 15 issues waiting for review (15) ============================================= #45279: avoid redundant _commit_removals pending_removals guard https://bugs.python.org/issue45279 #45276: avoid try 1000 in asyncio all_tasks by making weak collection https://bugs.python.org/issue45276 #45274: Race condition in Thread._wait_for_tstate_lock() https://bugs.python.org/issue45274 #45269: c_make_encoder() has uncovered error: "argument 1 must be dict https://bugs.python.org/issue45269 #45260: Implement superinstruction UNPACK_SEQUENCE_ST https://bugs.python.org/issue45260 #45258: sysroot_paths in setup.py does not consider -isysroot for macO https://bugs.python.org/issue45258 #45256: Remove the usage of the C stack in Python to Python calls https://bugs.python.org/issue45256 #45254: HAS_SHMEM detection logic is duplicated in implementation and https://bugs.python.org/issue45254 #45251: signal.SIGCLD alias is not available on OSX https://bugs.python.org/issue45251 #45243: [sqlite3] add support for changing connection limits https://bugs.python.org/issue45243 #45240: Add +REPORT_NDIFF option to pdb tests that use doctest https://bugs.python.org/issue45240 #45239: email.utils.parsedate_tz raises UnboundLocalError if time has https://bugs.python.org/issue45239 #45229: Always use unittest for collecting tests in regrtests https://bugs.python.org/issue45229 #45223: test_spawn_doesnt_hang (test.test_pty.PtyTest) fails when stdi https://bugs.python.org/issue45223 #45215: Add docs for Mock name and parent args and deprecation warning https://bugs.python.org/issue45215 Top 10 most discussed issues (10) ================================= #45116: Performance regression 3.10b1 and later on Windows: Py_DECREF( https://bugs.python.org/issue45116 21 msgs #43760: The DISPATCH() macro is not as efficient as it could be (move https://bugs.python.org/issue43760 18 msgs #42969: pthread_exit & PyThread_exit_thread from PyEval_RestoreThread https://bugs.python.org/issue42969 15 msgs #45020: Freeze all modules imported during startup. https://bugs.python.org/issue45020 14 msgs #45229: Always use unittest for collecting tests in regrtests https://bugs.python.org/issue45229 10 msgs #45261: Unreliable (?) results from timeit (cache issue?) https://bugs.python.org/issue45261 10 msgs #44958: [sqlite3] only reset statements when needed https://bugs.python.org/issue44958 9 msgs #45026: More compact range iterator https://bugs.python.org/issue45026 9 msgs #15870: PyType_FromSpec should take metaclass as an argument https://bugs.python.org/issue15870 8 msgs #21302: time.sleep (floatsleep()) should use clock_nanosleep() on Linu https://bugs.python.org/issue21302 7 msgs Issues closed (92) ================== #1170: shlex have problems with parsing unicode https://bugs.python.org/issue1170 closed by vstinner #14408: Support ./python -m unittest in test_socket https://bugs.python.org/issue14408 closed by vstinner #19829: _pyio.BufferedReader and _pyio.TextIOWrapper destructor don't https://bugs.python.org/issue19829 closed by vstinner #20524: format error messages should provide context information https://bugs.python.org/issue20524 closed by eric.smith #24076: sum() several times slower on Python 3 64-bit https://bugs.python.org/issue24076 closed by scoder #27069: webbrowser creates zombi processes in the background mode https://bugs.python.org/issue27069 closed by vstinner #28977: Document PyObject_CallFunction() special case more explicitly https://bugs.python.org/issue28977 closed by vstinner #29502: Should PyObject_Call() call the profiler on C functions, use C https://bugs.python.org/issue29502 closed by vstinner #29811: Avoid temporary method object in PyObject_CallMethod() and PyO https://bugs.python.org/issue29811 closed by vstinner #30637: Syntax error reported on compile(...), but not on compile(..., https://bugs.python.org/issue30637 closed by pablogsal #30856: unittest.TestResult.addSubTest should be called immediately af https://bugs.python.org/issue30856 closed by serhiy.storchaka #30951: Documentation error in inspect module https://bugs.python.org/issue30951 closed by lukasz.langa #32085: [Security] A New Era of SSRF - Exploiting URL Parser in Trendi https://bugs.python.org/issue32085 closed by vstinner #33793: asyncio: _ProactorReadPipeTransport reads by chunk of 32 KiB: https://bugs.python.org/issue33793 closed by vstinner #34090: Python function call optimization: avoid temporary tuple to pa https://bugs.python.org/issue34090 closed by vstinner #35276: Document thread safety https://bugs.python.org/issue35276 closed by vstinner #35333: [C API] Rename private structs to use names closer to types https://bugs.python.org/issue35333 closed by vstinner #35478: multiprocessing: ApplyResult.get() hangs if the pool is termin https://bugs.python.org/issue35478 closed by vstinner #35479: multiprocessing.Pool.join() always takes at least 100 ms https://bugs.python.org/issue35479 closed by vstinner #35518: test_timeout uses blackhole.snakebite.net domain which doesn't https://bugs.python.org/issue35518 closed by vstinner #36136: Windows: python._pth sets isolated mode late during Python ini https://bugs.python.org/issue36136 closed by vstinner #36437: method_descriptor surprising error message when self is passed https://bugs.python.org/issue36437 closed by vstinner #37322: test_ssl: test_pha_required_nocert() emits a ResourceWarning https://bugs.python.org/issue37322 closed by vstinner #37475: What is urllib.request.urlcleanup() function? https://bugs.python.org/issue37475 closed by vstinner #38199: python3.8-config --ldflags must not generate -L/usr/lib64 https://bugs.python.org/issue38199 closed by vstinner #38415: @asynccontextmanager decorated functions are not callable like https://bugs.python.org/issue38415 closed by lukasz.langa #38564: test_asyncio: test_run_coroutine_threadsafe_with_timeout() has https://bugs.python.org/issue38564 closed by vstinner #38620: Shell python-config --includes returns the same path twice https://bugs.python.org/issue38620 closed by vstinner #38623: Python documentation should mention how to find site-packages https://bugs.python.org/issue38623 closed by lukasz.langa #39359: zipfile: add missing "pwd: expected bytes, got str" exception https://bugs.python.org/issue39359 closed by lukasz.langa #39549: The reprlib.Repr type should permit the ��fillvalue�� to be https://bugs.python.org/issue39549 closed by rhettinger #39778: collections.OrderedDict and weakref.ref raises "refcount is to https://bugs.python.org/issue39778 closed by rhettinger #40413: Py_RunMain() crashes on subsequence call https://bugs.python.org/issue40413 closed by iritkatriel #40453: [subinterpreters] Add PyConfig._isolated_interpreter: isolated https://bugs.python.org/issue40453 closed by vstinner #40497: subprocess.check_output() accept the check keyword argument https://bugs.python.org/issue40497 closed by lukasz.langa #40657: Resource leaks with threading.Thread https://bugs.python.org/issue40657 closed by vstinner #40699: Memory leak in threading library with Python 3.6-3.8 https://bugs.python.org/issue40699 closed by vstinner #40951: csv skipinitialspace no longer working https://bugs.python.org/issue40951 closed by andrei.avk #40989: [C API] Remove _Py_NewReference() and _Py_ForgetReference() fr https://bugs.python.org/issue40989 closed by vstinner #41078: [C API] Convert PyTuple_GET_ITEM() macro to a static inline fu https://bugs.python.org/issue41078 closed by vstinner #41203: Replace references to OS X in documentation with macOS https://bugs.python.org/issue41203 closed by serhiy.storchaka #42038: Tracemalloc's format() doc contradictory https://bugs.python.org/issue42038 closed by lukasz.langa #42260: [C API] Add _PyInterpreterState_SetConfig(): reconfigure an in https://bugs.python.org/issue42260 closed by vstinner #42294: [C API] Add new C functions with more regular reference counti https://bugs.python.org/issue42294 closed by vstinner #42389: test_multiprocessing: @requires_hashdigest() prevents test dis https://bugs.python.org/issue42389 closed by serhiy.storchaka #42641: Deprecate os.popen() function https://bugs.python.org/issue42641 closed by vstinner #42648: subprocess: add a helper/parameter to catch exec() OSError exc https://bugs.python.org/issue42648 closed by vstinner #42671: Make the Python finalization more deterministic https://bugs.python.org/issue42671 closed by vstinner #43050: threading timer memory leak https://bugs.python.org/issue43050 closed by vstinner #43770: Rework C types initialization https://bugs.python.org/issue43770 closed by vstinner #44019: operator.call/operator.__call__ https://bugs.python.org/issue44019 closed by mark.dickinson #44203: test__xxsubinterpreters: heap-buffer-overflow in interp_is_run https://bugs.python.org/issue44203 closed by vstinner #44358: AMD64 RHEL8 LTO + PGO 3.x build failed with: /usr/bin/ld: Dwar https://bugs.python.org/issue44358 closed by vstinner #44416: test_threading: test_print_exception() hangs and killed after https://bugs.python.org/issue44416 closed by vstinner #44423: copy2 / sendfile fails on linux with large file https://bugs.python.org/issue44423 closed by kr-g #44480: test_compile killed by signal 9 on AMD64 FreeBSD Non-Debug 3.x https://bugs.python.org/issue44480 closed by vstinner #44531: Add _PyType_AllocNoTrack() function: allocate without tracking https://bugs.python.org/issue44531 closed by vstinner #44598: test_constructor (test.test_ssl.ContextTests) ... Fatal Python https://bugs.python.org/issue44598 closed by sxt1001 #44640: Fix punctuation in isinstance() error message https://bugs.python.org/issue44640 closed by pablogsal #44848: Upgrade macOS and Windows installers to use SQLite 3.36.0 https://bugs.python.org/issue44848 closed by erlendaasland #45002: won't match correct version of tcl/tk https://bugs.python.org/issue45002 closed by jett8998 #45021: concurrent.futures race condition https://bugs.python.org/issue45021 closed by gregory.p.smith #45027: Allow basicConfig to configure any logger, not just root https://bugs.python.org/issue45027 closed by vinay.sajip #45040: [sqlite3] optimise transaction control functions https://bugs.python.org/issue45040 closed by pablogsal #45041: [sqlite3] simplify executescript() https://bugs.python.org/issue45041 closed by erlendaasland #45055: Fresh build on Windows fails the first time for zlib.c https://bugs.python.org/issue45055 closed by lukasz.langa #45121: Calling super().__init__ in subclasses of typing.Protocol rais https://bugs.python.org/issue45121 closed by pablogsal #45128: test_multiprocessing_fork fails if run sequentially after test https://bugs.python.org/issue45128 closed by pablogsal #45134: Protocol dealloc not called if Server is closed https://bugs.python.org/issue45134 closed by MarkReedZ #45170: tarfile missing cross-directory checking https://bugs.python.org/issue45170 closed by martin.panter #45183: Unexpected exception with zip importer https://bugs.python.org/issue45183 closed by brett.cannon #45198: __set_name__ documentation not clear about its usage with non- https://bugs.python.org/issue45198 closed by rhettinger #45200: Address Sanitizer: libasan dead lock in pthread_create() (test https://bugs.python.org/issue45200 closed by pablogsal #45209: multiprocessing tests log: UserWarning: resource_tracker: Ther https://bugs.python.org/issue45209 closed by vstinner #45216: Remove redundant information from difflib docstrings https://bugs.python.org/issue45216 closed by lukasz.langa #45231: make regen-all changes files on Linux https://bugs.python.org/issue45231 closed by vstinner #45234: copy_file raises FileNotFoundError when src is a directory https://bugs.python.org/issue45234 closed by pablogsal #45235: argparse does not preserve namespace with subparser defaults https://bugs.python.org/issue45235 closed by rhettinger #45236: pyperformance fails to build master https://bugs.python.org/issue45236 closed by pablogsal #45238: Fix debug() in IsolatedAsyncioTestCase https://bugs.python.org/issue45238 closed by lukasz.langa #45241: python REPL leaks local variables when an exception is thrown https://bugs.python.org/issue45241 closed by ianh2 #45245: ValueError: check_hostname requires server_hostname while pip https://bugs.python.org/issue45245 closed by christian.heimes #45246: the sorted() documentation should refer to operator < https://bugs.python.org/issue45246 closed by rhettinger #45248: Documentation example in copyreg errors https://bugs.python.org/issue45248 closed by CiaranWelsh #45257: Compiling 3.8 branch on Windows attempts to use incompatible l https://bugs.python.org/issue45257 closed by proffitt #45259: No _heappush_max() https://bugs.python.org/issue45259 closed by rhettinger #45263: round displays 2 instead of 3 digits https://bugs.python.org/issue45263 closed by steven.daprano #45265: Bug in append() method in order to appending a temporary list https://bugs.python.org/issue45265 closed by rhettinger #45268: use multiple "in" in one expression? https://bugs.python.org/issue45268 closed by Dennis Sweeney #45270: Clicking "Add to Custom Colors" adds same colour multiple time https://bugs.python.org/issue45270 closed by terry.reedy #45271: Add a 'find' method (a'la str) to list https://bugs.python.org/issue45271 closed by rhettinger #1514420: Traceback display code can attempt to open a file named "<stdi https://bugs.python.org/issue1514420 closed by lukasz.langa

1 0

RFC: New/update the Docs Sphinx theme: responsive, edit page links,
by Wes Turner 23 Sep '21

23 Sep '21

Where would be the best place to discuss or just improve the CPython docs sphinx theme? What would Python need in a {PyData,} Sphinx theme fork: - [ ] ENH: add <edit on github> links to every docs page - [ ] ENH: responsive breakpoints for mobile devices - [ ] ENH: responsive fonts - [ ] ENH: toggle-able dark theme - [ ] ENH: interactive code examples (in WASM with pyodide like JupyterLite) - [ ] ENH?: Two sidebars: {Global TOC} {Content} {Document TOC} The PyData theme [bandwagon, feature justification, maintenance burden justification] Src: https://github.com/pydata/pydata-sphinx-theme Docs: https://pydata-sphinx-theme.readthedocs.io/en/latest/ ```quote Other sites that are using this theme: - Pandas: https://pandas.pydata.org/docs/ - NumPy: https://numpy.org/doc/stable/ - SciPy: https://scipy.github.io/devdocs/ - NetworkX: https://networkx.org/documentation/latest/ - Bokeh: https://docs.bokeh.org/en/latest/ - JupyterHub and Binder: https://docs.mybinder.org/, http://z2jh.jupyter.org/en/latest/, https://repo2docker.readthedocs.io/en/latest/, https://jupyterhub-team-compass.readthedocs.io/en/latest/ - Jupyter Book beta version uses an extension of this theme: https://beta.jupyterbook.org - Fairlearn: https://fairlearn.org/main/about/ ``` The sphinx-book-theme theme (Jupyter Book theme) has a few possibly necessary features for the {CPython, Devguide, *} docs: https://github.com/executablebooks/sphinx-book-theme https://sphinx-book-theme.readthedocs.io/en/latest/ :: ```quote A Sphinx theme with a clean design, support for interactive content, and a modern book-like look and feel. Flexible content layout: Inspired by beautiful online books, such as the Edward Tufte CSS guide Visual classes designed for Jupyter Notebooks: Cell inputs, outputs, and interactive functionality are all supported. Launch buttons for online interactivity: For pages that are built with computational material, connect your site to an online BinderHub for interactive content. Bootstrap 4: To style visual elements and add functionality. International: All text integrated in the theme is translated to the specified Sphinx language. ``` https://github.com/python/cpython/blob/main/Doc/conf.py#L68 : ```python html_theme = 'python_docs_theme' ``` https://github.com/python/cpython/blob/main/Doc/requirements.txt#L12 ``` # The theme used by the documentation is stored separately, so we need # to install that as well. python-docs-theme ``` Pypi: https://pypi.org/project/python-docs-theme/ Src: https://github.com/python/python-docs-theme/ What are some better ways to add {Source RST, Source, Edit, History, and Annotate} links to every sphinx doc page than sphinxcontrib-srclinks? https://github.com/westurner/sphinxcontrib-srclinks#usage - sphinxcontrib-srclinks: Source RST, Source, Edit, History, and Annotate, Clone URL - [ ] copy clone URL to clipboard button - pydata-theme: 'Edit this page' on the document TOC sidebar - sphinx-book-theme: {gh, bb, gl} icon > 'Suggest edit' - [ ] ENH: add collapsible sidebar to whatever is chosen as a suitable replacement for the current non-responsive theme

2 1

f-strings in the grammar
by Pablo Galindo Salgado 23 Sep '21

23 Sep '21

Hi, I have started a project to move the parsing off-strings to the parser and the grammar. Appart from some maintenance improvements (we can drop a considerable amount of hand-written code), there are some interesting things we **could** (emphasis on could) get out of this and I wanted to discuss what people think about them. * The parser will likely have "\n" characters and backslashes in f-strings expressions, which currently is impossible: >>> f"blah blah {'\n'} blah" File "<stdin>", line 1 f"blah blah {'\n'} blah" ^ SyntaxError: f-string expression part cannot include a backslash * The parser will allow nesting quote characters. This means that we **could** allow reusing the same quote type in nested expressions like this: f"some text { my_dict["string1"] } more text" * The parser will naturally allow more control over error messages and AST positions. * The **grammar** of the f-string will be fully specified without ambiguity. Currently, the "grammar" that we have in the docs ( https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-…) is not really formal grammar because not only is mixing lexing details with grammar details (the definition of " literal_char") but also is not compatible with the current python lexing schema (for instance, it recognizes "{{" as its own token, which the language doesn't allow because something like "{{a:b}:c}" is tokenized as "{", "{", "a" ... not as "{{", "a". Adding a formal grammar could help syntax highlighters, IDEs, parsers and other tools to make sure they properly recognize everything that there is. There may be some other advantages that we have not explored still. The work is at a point where the main idea works (all the grammar is already there and working), but we need to make sure that all existing errors and specifics are properly ported to the new code, which is a considerable amount of work still so I wanted to make sure we are on the same page before we decide to invest more time on this (Batuhan is helping me with this and Lyssandros will likely join us). We are doing this work in this branch: https://github.com/we-like-parsers/cpython/blob/fstring-grammar Tell me what you think. P.S. If you are interested to help with this project, please reach out to me. If we decide to go ahead we can use your help! :) Regards from cloudy London, Pablo Galindo Salgado

15 28