Mailman 3 May 2022 - Python-Dev

Python grammar test cases
by Venkat Ramakrishnan 10 May '22

10 May '22

Greetings! I'm wondering if there's a repository of test cases that test the Python grammar. Any help would be appreciated. Thanks & Best Regards, Venkat.

4 7

Test Python 3.11 beta1 with PYTHONSAFEPATH=1
by Victor Stinner 10 May '22

10 May '22

Hi, I added the -P command line option and the PYTHONSAFEPATH=1 environment variable to Python 3.11 beta 1 to not prepend an "unsafe path" to sys.path such as the script's directory or the current directory: https://docs.python.org/dev/using/cmdline.html#cmdoption-P Example: ---------- $ echo 'Nope!' >random.py # broken module $ echo 'import random; print(random.randint(1, 6))' >dice.py $ python3.11 dice.py # default behavior (...) File "/home/vstinner/random.py", line 1 (...) SyntaxError: invalid syntax $ python3.11 -P dice.py # ignore local random.py 4 ---------- Please test Python 3.11 beta1 with the PYTHONSAFEPATH=1 environment variable set, or at least run python with the -P option. I'm curious which use cases are not affected and which use cases are affected. The PYTHONSAFEPATH=1 environment variable is inherited and so affect child processes. It can break applications relying on Python 3.10 sys.path behavior. I proposed adding -p option which does the opposite of the -P option: opt-in for Python 3.10 "unsafe" sys.path behavior, ignore the PYTHONSAFEPATH environment variable: https://github.com/python/cpython/pull/92361 If PYTHONSAFEPATH=1 only breaks a minority of use cases, maybe we can consider to make it the default, as Perl did in Perl 5.26 (2017) to increase its security: https://perldoc.perl.org/perl5260delta#Removal-of-the-current-directory-(%2… Perl has an environment variable to get the old (Perl 5.24) behavior: PERL_USE_UNSAFE_INC=1. If enough people consider that it would be a good idea to change the default, I can maybe write a full PEP (I already have some notes). Even if the PEP is rejected, it might be a good thing to write down everything about this topic since it's a common issue hit by users learning Python and a common question of people auditing the Python security. I was asked a few months ago about changing the default to increase Python security. Victor -- Night gathers, and now my watch begins. It shall not end until my death.

2 1

[RELEASE] The first Python 3.11 beta (3.11.0b1) is available - Feature freeze is here
by Pablo Galindo Salgado 08 May '22

08 May '22

We did it, team!! After quite a bumpy release process and a bunch of last-time fixes, we have reached **beta 1** and **feature freeze**. What a ride eh? You can get the shiny new release artefacts from here: https://www.python.org/downloads/release/python-3110b1/ ## This is a beta preview of Python 3.11 Python 3.11 is still in development. 3.11.0b1 is the first of four planned beta release previews. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. We **strongly encourage** maintainers of third-party Python projects to **test with 3.11** during the beta phase and report issues found to [the Python bug tracker](https://bugs.python.org) as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (Monday, 2021-08-02). Our goal is to have no ABI changes after beta 4 and as few code changes as possible after 3.11.0rc1, the first release candidate. To achieve that, it will be **extremely important** to get as much exposure for 3.11 as possible during the beta phase. Please keep in mind that this is a preview release and its use is **not** recommended for production environments. # Major new features of the 3.11 series, compared to 3.10 Python 3.11 is still in development. This release, 3.11.0b1 is the **first** of four beta releases. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. Many new features for Python 3.11 are still being planned and written. Among the new major new features and changes so far: * [PEP 657](https://www.python.org/dev/peps/pep-0657/) -- Include Fine-Grained Error Locations in Tracebacks * [PEP 654](https://www.python.org/dev/peps/pep-0654/) -- Exception Groups and except* * [PEP 673](https://www.python.org/dev/peps/pep-0673/) -- Self Type * [PEP 646](https://www.python.org/dev/peps/pep-0646/)-- Variadic Generics * [PEP 680](https://www.python.org/dev/peps/pep-0680/)-- tomllib: Support for Parsing TOML in the Standard Library * [PEP 675](https://www.python.org/dev/peps/pep-0675/)-- Arbitrary Literal String Type * [PEP 655](https://www.python.org/dev/peps/pep-0655/)-- Marking individual TypedDict items as required or potentially-missing * [bpo-46752](https://bugs.python.org/issue46752)-- Introduce task groups to asyncio * The Faster Cpython Project <https://github.com/faster-cpython> is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See Faster CPython <https://docs.python.org/3.11/whatsnew/3.11.html#faster-cpython>for details. * Hey, **fellow core developer,** if a feature you find important is missing from this list, let me know. The next pre-release of Python 3.11 will be 3.11.0b2, currently scheduled for Monday, 2022-05-30. # More resources * [Online Documentation](https://docs.python.org/3.11/) * [PEP 664](https://www.python.org/dev/peps/pep-0664/), 3.11 Release Schedule * Report bugs at [https://bugs.python.org](https://bugs.python.org). * [Help fund Python and its community](/psf/donations/). # And now for something completely different The holographic principle is a tenet of string theories and a supposed property of quantum gravity that states that the description of a volume of space can be thought of as encoded on a lower-dimensional boundary to the region—such as a light-like boundary like a gravitational horizon. First proposed by Gerard 't Hooft, it was given a precise string-theory interpretation by Leonard Susskind, who combined his ideas with previous ones of 't Hooft and Charles Thorn.[ Leonard Susskind said, “The three-dimensional world of ordinary experience––the universe filled with galaxies, stars, planets, houses, boulders, and people––is a hologram, an image of reality cited on a distant two-dimensional (2D) surface." As pointed out by Raphael Bousso, Thorn observed in 1978 that string theory admits a lower-dimensional description in which gravity emerges from it in what would now be called a holographic way. The holographic principle was inspired by black hole thermodynamics, which conjectures that the maximal entropy in any region scales with the radius squared, and not cubed as might be expected. In the case of a black hole, the insight was that the informational content of all the objects that have fallen into the hole might be entirely contained in surface fluctuations of the event horizon. The holographic principle resolves the black hole information paradox within the framework of string theory. However, there exist classical solutions to the Einstein equations that allow values of the entropy larger than those allowed by an area law, hence in principle larger than those of a black hole. These are the so-called "Wheeler's bags of gold". The existence of such solutions conflicts with the holographic interpretation, and their effects in a quantum theory of gravity including the holographic principle are not fully understood yet. # We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. https://www.python.org/psf/ Regards from chilly London, Your friendly release team, Pablo Galindo Salgado

3 4

Re: [RELEASE] The first Python 3.11 beta (3.11.0b1) is available - Feature freeze is here
by Pablo Galindo Salgado 08 May '22

08 May '22

Done in all places that I can edit! Thanks for the catch :) On Sun, 8 May 2022 at 09:14, Oleg Iarygin <oleg(a)arhadthedev.net> wrote: > - python-announce@, python-committers@, python-list@ > > > report issues found to [the Python bug tracker](https://bugs.python.org) > as soon as possible > > The template needs to be updated. >

1 0

Summary of Python tracker Issues
by Python tracker 06 May '22

06 May '22

ACTIVITY SUMMARY (2022-04-29 - 2022-05-06) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7146 ( +0) closed 51841 ( +0) total 58987 ( +0) Open issues with patches: 2890 Most recent 15 issues with no replies (15) ========================================== #47258: Python 3.10 hang at exit in drop_gil() (due to resource warnin https://bugs.python.org/issue47258 #47256: re: limit the maximum capturing group to 1,073,741,823, reduce https://bugs.python.org/issue47256 #47253: LOAD_GLOBAL instruction with wrong source position https://bugs.python.org/issue47253 #47252: socket.makefile documentation is missing data regarding the 'b https://bugs.python.org/issue47252 #47251: Merge BINARY_SUBSCR_LIST_INT with BINARY_SUBSCR_LIST_TUPLE https://bugs.python.org/issue47251 #47244: email.utils.formataddr does not respect double spaces https://bugs.python.org/issue47244 #47242: Annoying white bar in IDLE (line 457 in sidebar.py) https://bugs.python.org/issue47242 #47241: [C API] Move the PyCodeObject structure to the internal C API https://bugs.python.org/issue47241 #47238: Python threading.Event().wait() depends on the system time https://bugs.python.org/issue47238 #47236: Document types.CodeType.replace() changes about co_exceptionta https://bugs.python.org/issue47236 #47228: Document that naïve datetime objects represent local time https://bugs.python.org/issue47228 #47222: subprocess.Popen() should allow capturing output and sending i https://bugs.python.org/issue47222 #47219: asyncio with two interpreter instances https://bugs.python.org/issue47219 #47218: adding name to lzmafile https://bugs.python.org/issue47218 #47217: adding name to BZ2File https://bugs.python.org/issue47217 Most recent 15 issues waiting for review (15) ============================================= #47256: re: limit the maximum capturing group to 1,073,741,823, reduce https://bugs.python.org/issue47256 #47255: Many broken :meth: roles in the docs https://bugs.python.org/issue47255 #47254: enhanced dir? https://bugs.python.org/issue47254 #47251: Merge BINARY_SUBSCR_LIST_INT with BINARY_SUBSCR_LIST_TUPLE https://bugs.python.org/issue47251 #47243: Duplicate entry in 'Objects/unicodetype_db.h' https://bugs.python.org/issue47243 #47233: show_caches option affects code positions reported by dis.get_ https://bugs.python.org/issue47233 #47222: subprocess.Popen() should allow capturing output and sending i https://bugs.python.org/issue47222 #47218: adding name to lzmafile https://bugs.python.org/issue47218 #47217: adding name to BZ2File https://bugs.python.org/issue47217 #47216: adding mtime option to gzip open() https://bugs.python.org/issue47216 #47215: Add "unstable" frame stack api https://bugs.python.org/issue47215 #47208: Support libffi implementations that cannot support invocations https://bugs.python.org/issue47208 #47205: posix.sched_{get|set}affinity(-1) no longer returns ProcessLoo https://bugs.python.org/issue47205 #47200: Add ZipInfo.mode property https://bugs.python.org/issue47200 #47199: multiprocessing: micro-optimize Connection.send_bytes() method https://bugs.python.org/issue47199

1 0

Release of Python 3.11 beta 1 is currently blocked
by Pablo Galindo Salgado 06 May '22

06 May '22

Hi everyone, Today we need to start the release of Python 3.11 beta 1. Currently, we have the following blockers: * https://github.com/python/cpython/issues/91162 (was deferred blocker - but all deferred blockers are bumped to release blockers on beta1). * https://github.com/python/cpython/issues/91350 * We have 3 buildbots failing: https://buildbot.python.org/all/#/release_status (some of the failures are due to flaky tests but others look real). Please, if you are involved in the above issues check if they are resolved or if they are not released blockers (notice any semantic change, API change, public data structure change or grammar change must be done before beta 1 is released). I will look at the buildbots later today but if you have some time to investigate, that would help a lot! *I am blocking the main branch so only the release team can merge changes* until these problems are addressed and we can continue with the release. Please, add me as a reviewer to any PR that needs to be merged to address these issues or any other change that *absolutely needs to go into beta 1*. Thanks for your help! Regards from sunny London, Pablo Galindo Salgado

2 2

Add -P command line option to not add sys.path[0]
by Victor Stinner 05 May '22

05 May '22

Hi, There are 4 main ways to run Python: (1) python -m module [...] (2) python script.py [...] (3) python -c code [...] (4) python [...] (1) and (2) insert the directory of the module/script at sys.path[0]. (3) and (4) insert an empty string at sys.path[0]. This behavior is convenient and is maybe part of Python usability success: importing a module in the current directory is as easy as "import other_module" (load other_module.py). But it's also a threat to security: an attacker can override a stdlib module by creating a Python script with the same name than a stdlib module, like os.py or shutil.py. People learning Python commonly create a file with the same name than a stdlib module (ex: random.py) and then are clueless in face of an ImportError exception. Changing the default behavior was discussed multiple times. No consensus was reached, maybe because most users like the current default behavior and are not affected by corner cases (see below). I propose adding a -P option to Python command line interface to "not add sys.path[0]": https://github.com/python/cpython/pull/31542 See the documentation in the PR for the exact behavior of this option. I prefer to add an environment variable, only pass the option explicitly on the command line. Since Python 3.4, there is already the -I ("isolated mode") option: https://docs.python.org/dev/using/cmdline.html#cmdoption-I The -I option has other effects like disabling user site directories, it option doesn't fit use cases of the -P option. One annoying issue of the Python default behavior is that running a script in /usr/bin/ as root can create or override .pyc files in the /usr directory, even in the /usr/bin/ directory. Example of this surprising and annoying issue: https://github.com/benjaminp/six/issues/359#issuecomment-996159668 The -P option can be used in #!/usr/bin/python shebang to avoid this issue. -- An alternative would be to change the default behavior to not add sys.path[0], and add an option to opt-in for Python 3.10 behavior. Here are my notes about it: https://github.com/vstinner/misc/blob/main/cpython/pep_path0.rst What do you think? Victor -- Night gathers, and now my watch begins. It shall not end until my death.

9 19

Can I ask a real dumb procedural question about GitHub email?
by Skip Montanaro 05 May '22

05 May '22

I subscribe to the python/cpython stuff on GitHub. I find it basically impossible to follow because of the volume. I realize there are probably plenty of extra changes going in based on the recent language summit (and maybe some sprints at PyCon?) as well as the proximity to the beta 1 freeze. Still, does anyone actually try to follow everything that comes out of that firehose? I've received updates to about 125 new GMail conversations (more total messages than that) in the last 24 hours, and that's after using filters to delete Miss-Islington-type messages altogether. As far as I can quickly ascertain, all the messages are from real people, not bots. How (if at all) do people deal with this firehose of email? Am I the only person dumb enough to have tried? I used to scan for csv-module-related messages, but don't even try to do that now. My only real reason for continuing to subscribe is that it feeds into a process that updates a dictionary of "common" words used by my XKCD-936-derived password generator. Thx, Skip

5 4

PEP 690: Lazy Imports
by Germán M. Bravo 03 May '22

03 May '22

Hello, PEP 690 <https://peps.python.org/pep-0690/> is now live! It proposes an exciting new feature to transparently defer the execution of imported modules until the moment when an imported object is used. Since Python programs commonly import many more modules than a single invocation of the program is likely to use in practice, lazy imports can greatly reduce the overall number of modules loaded, improving startup time and memory usage. At Meta, we have implemented this feature in Cinder <https://github.com/facebookincubator/cinder>, and has already demonstrated <https://github.com/facebookincubator/cinder/blob/cinder/3.8/CinderDoc/lazy_…> startup time improvements up to 70% and memory-use reductions up to 40% on real-world Python CLIs, while also eliminating the risk for most import cycles. Please, don't reply to this message. Feel free to comment or share thoughts and feedback in Python Discourse <https://discuss.python.org/t/pep-690-lazy-imports> at https://discuss.python.org/t/pep-690-lazy-imports. Best, Germán --- Relevant links follow: https://peps.python.org/pep-0690 https://github.com/facebookincubator/cinder https://github.com/facebookincubator/cinder/blob/cinder/3.8/CinderDoc/lazy_… https://discuss.python.org/t/pep-690-lazy-imports

1 0

Using the Python C API in C++
by Victor Stinner 03 May '22

03 May '22

Hi, If you got issues with using the Python C API <Python.h> in C++, please speak up! I'm looking for feedback :-) Extending Python by writing C++ code is now easy with the pybind11 project: https://pybind11.readthedocs.io/ It seems like over the last years, building C++ extensions with the Python C API started to emit more C++ compiler warnings. One explanation may be that converting macros to static inline functions (PEP 670) introduce new warnings, even if the old and the new code is exactly the same. I just discover this issue recently. C and C++ compilers treat static inline functions differently. Macros are treated at legacy code which cannot be fixed, like system headers or old C header files, and so many warnings are made quiet. Static inline functions (defined in header files) are treated as regular code and compilers are more eager to emit warnings. I just modified the Python C API to use C++ static_cast<type>(expr) and reinterpret_cast<type>(expr) if it's used with C++. In C, the regular (type)expr cast (called "old-style cast" by C++ compilers ;-)) is still used as before. I'm also working on adding an unit test to make suite that using the Python C API works with a C++ compiler and doesn't emit compiler warnings: * https://github.com/python/cpython/issues/91321 * https://github.com/python/cpython/pull/32175 In terms of C++ version, it was proposed to target C++11. In the pythoncapi-compat project, I got warnings when the NULL constant is used in static inline functions. I modified the pythoncapi_compat.h header file to use nullptr if used with C++ to fix these compiler warnings. So far, I'm unable to reproduce the issue with <Python.h>, and so I didn't try to address this issue in Python. Victor -- Night gathers, and now my watch begins. It shall not end until my death.

5 16

Python-Dev May 2022