Mailman 3 December 2021 - Python-Dev

Should isinstance call __getattribute__?
by Steven D'Aprano 10 Dec '21

10 Dec '21

I'm looking for some guidance on a bug report involving isinstance and __getattribute__` please. The issue is that if your class overloads `__getattribute__`, calling isinstance on an instance will call the overloaded `__getattribute__` method when looking up `__class__`, potentially causing isinstance to fail, or return the wrong result. See b.p.o. #32683 https://bugs.python.org/issue32683 I see no reason why this isn't working as designed, __getattribute__ is intended to overload attribute access, and that could include the `__class__` attribute. Am I wrong? It has been suggested that isinstance should call `object.__getattribute__` and bypass the class' overloaded method, but I expect that would probably break objects which intentionally lie about their class. (Mocks? Stubs? Proxies?) Thanks, Steve

2 3

Explicit markers for special C-API situations (re: Clarification regarding Stable ABI and _Py_*)
by Eric Snow 10 Dec '21

10 Dec '21

(replying to https://mail.python.org/archives/list/python-dev@python.org/message/OJ65FPC…) On Wed, Dec 8, 2021 at 10:06 AM Eric Snow <ericsnowcurrently(a)gmail.com> wrote: > What about the various symbols listed in Misc/stable_abi.txt that were > accidentally added to the limited API? Can we move toward dropping > them from the stable ABI? tl;dr We should consider making classifications related to the stable ABI harder to miss. <context> Knowing what is in the limited API is fairly straightforward. [1] However, it's clear that identifying what is part of the stable ABI, and why, is not so easy. Currently, we must rely on Misc/stable_abi.txt [2] (and the associated Tools/scripts/stable_abi.py). Documentation (C-API docs, PEPs, devguide) help too. Yet, there's a concrete disconnect here: the header files are by definition the authoritative single-source-of-truth for the C-API and it's too easy to forget about supplemental info in another file or document. This out-of-sight-out-of-mind situation is part of how we accidentally added things to the limited API for a while. [3] The stable ABI isn't the only area where we must identify different subsets of the C-API. However, in those other cases we use different structural/naming conventions to explicitly group things. Most importantly, each of those conventions makes the grouping unavoidable when reading the code. [4] For example: * closely related declarations go in the same header file (and then also exposed via Include/Python.h) * prefixes (e.g. Py_, PyDict_) provides similar grouping * an additional underscore prefix identifies "private" C-API * symbols are explicitly identified as part of the C-API via macros (PyAPI_FUNC, PyAPI_DATA) [5] * relatively recently, different directories correspond to different API layers (Include, Include/cpython, Include/internal) [3] </context> Could we take a similar explicit, coupled-to-the-code approach to identify when the different stable ABI situations apply? Here's the specific approach I had in mind, with macros similar to PyAPI_FUNC: * PyAPI_ABI_FUNC - in stable ABI when it wouldn't be normally (e.g. underscore prefix, in Include/internal) * PyAPI_ABI_INDIRECT - exposed in stable ABI due to a macro * PyAPI_ABI_ONLY - it only exists for ABI compatibility and isn't actually used any more * PyAPI_ABI_ACCIDENTAL - unintentionally added to limited API, probably not used there (...or perhaps use a PyABI_ prefix, though that's a bit easy to miss when reading.) As a reader I would find markers like this helpful in recognizing those special situations, as well as the constraints those situations impose on modification. At the least such macros would indicate something different is going on, and the macro name would be something I could look up if I needed more info. I expect others reading the code would get comparable value. I also expect tools like Tools/scripts/stable_abi.py would benefit. -eric [1] in Include/*.h and not #ifndef Py_LIMITED_API (sadly also making it easy to accidentally add things to the limited API, see [3]) [2] Before that you had to rely on comments or external documents or, in the worst case, work it out through careful study of the code, commit history, and mailing list archives. [3] The addition of Include/cpython and Include/internal helped us stop accidentally adding to the limited API. [4] It also makes the groupings deterministically discoverable by tools. [5] explicit use of "extern" indicates a different intent

5 7

Clarification regarding Stable ABI and _Py_*
by Guido van Rossum 09 Dec '21

09 Dec '21

Hi Petr, In PEP 384 it is written that no functions starting with an underscore are part of the stable ABI: PEP 384 -- Defining a Stable ABI | Python.org <https://www.python.org/dev/peps/pep-0384/#excluded-functions> > All functions starting with _Py are not available to applications OTOH there's a data file in the repo, Misc/stabe_abi.txt, which lists many functions starting with _Py_, for example _PyObject_GC_Malloc. Then again, that function is not listed in Doc/data/stable_abi.dat. (I didn't check other functions, but maybe there are others.) So is Misc/stable_abi.txt just out of date? Or how can the discrepancy be explained? -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

6 13

[RELEASE] Python 3.11.0a3 is available
by Pablo Galindo Salgado 09 Dec '21

09 Dec '21

You can tell that we are slowly getting closer to the first beta as the number of release blockers that we need to fix on every release starts to increase [image: :sweat_smile:] But we did it! Thanks to Steve Dower, Ned Deily, Christian Heimes, Łukasz Langa and Mark Shannon that helped get things ready for this release :) Go get the new version here: https://www.python.org/downloads/release/python-3110a3/ **This is an early developer preview of Python 3.11** # Major new features of the 3.11 series, compared to 3.10 Python 3.11 is still in development. This release, 3.11.0a3 is the third of seven planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process. During the alpha phase, features may be added up until the start of the beta phase (2022-05-06) and, if necessary, may be modified or deleted up until the release candidate phase (2022-08-01). Please keep in mind that this is a preview release and its use is **not** recommended for production environments. Many new features for Python 3.11 are still being planned and written. Among the new major new features and changes so far: * [PEP 657](https://www.python.org/dev/peps/pep-0657/) -- Include Fine-Grained Error Locations in Tracebacks * [PEP 654](https://www.python.org/dev/peps/pep-0654/) -- Exception Groups and except* * The [Faster Cpython Project](https://github.com/faster-cpython) is already yielding some exciting results: this version of CPython 3.11 is ~12% faster on the geometric mean of the [PyPerformance benchmarks]( speed.python.org), compared to 3.10.0. * Hey, **fellow core developer,** if a feature you find important is missing from this list, let me know. The next pre-release of Python 3.11 will be 3.11.0a4, currently scheduled for Monday, 2022-01-03. # More resources * [Online Documentation](https://docs.python.org/3.11/) * [PEP 664](https://www.python.org/dev/peps/pep-0664/), 3.11 Release Schedule * Report bugs at [https://bugs.python.org](https://bugs.python.org). * [Help fund Python and its community](/psf/donations/). # And now for something completely different Rayleigh scattering, named after the nineteenth-century British physicist Lord Rayleigh is the predominantly elastic scattering of light or other electromagnetic radiation by particles much smaller than the wavelength of the radiation. For light frequencies well below the resonance frequency of the scattering particle, the amount of scattering is inversely proportional to the fourth power of the wavelength. Rayleigh scattering results from the electric polarizability of the particles. The oscillating electric field of a light wave acts on the charges within a particle, causing them to move at the same frequency. The particle, therefore, becomes a small radiating dipole whose radiation we see as scattered light. The particles may be individual atoms or molecules; it can occur when light travels through transparent solids and liquids but is most prominently seen in gases. The strong wavelength dependence of the scattering means that shorter (blue) wavelengths are scattered more strongly than longer (red) wavelengths. This results in the indirect blue light coming from all regions of the sky. # We hope you enjoy those new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Your friendly release team, Pablo Galindo @pablogsal Ned Deily @nad Steve Dower @steve.dower

3 3

Re: PEP 674: Disallow using macros as l-value
by Tim Felgentreff 08 Dec '21

08 Dec '21

Hi, On Tue, 7 Dec 2021 16:08:27 -0800 Guido van Rossum <guido(a)python.org> wrote: > I wonder if there's a role for HPy in this context? What if instead of > evolving the stable ABI and the limited API, instead we were to focus on > first-class support for HPy? The hope with the HPy project is indeed to provide a C API that is close to the original API to make porting easy and have it perform as close to the existing API as possible. At the same time, HPy is sufficently removed to be a good "C extension API" (as opposed to a stable subset of the CPython implementation API) that does not leak implementation details. To ensure this latter property is why we try to develop everything in parallel for CPython, PyPy, and GraalVM Python. On Wed, 8 Dec 2021 01:47:38 +0100 Victor Stinner <vstinner(a)python.org> wrote: > Providing HPy as a first-class citizen in CPython, as already done in > PyPy, would be great to promote HPy! However, HPy evolves quickly and > so needs to be released more frequently than CPython. At least, we > could promote it more in the C API documentation, as we already > promote Cython. Indeed, at this point HPy is still evolving very fast. We are still solving issues while migrating NumPy, have begun adding support for HPy to Cython, and will start working on pybind11 soon. I believe by the time we have these users of the existing C API working, HPy should be in a state where it is generally useful and can be deemed stable enough that further development can follow a more stable process. > For me, HPy is the only valid stable API and stable ABI in the long > term which is efficient on any Python implementation. Its design is > very different than the C API: HPy avoids all C API design mistakes, > it doesn't leak any implementation detail. In the long run the HPy project would like to become a promoted API to write Python C extensions. Tim Felgentreff (on behalf of the HPy team)

1 0

So many PEPs
by Koos Zevenhoven 08 Dec '21

08 Dec '21

Steering Council, g'day, Hawaii? ("copying" BDFL-emeritus, everyone) I'm sorry you haven't heard from me in a while. Actually, neither have any of my other "girl friends" heard from me. I was worried that they might be sad. Then I found out that one of them was at some kind of party at a club or something, and got in trouble because she couldn't be reached. I learned from the newspaper and other media that she indeed was really really sad, because of this reason. Yesterday was supposed to be a big day for her, as the president was throwing a party. But something was missing – because of COVID-19, i believe. So various kinds of things have been going on, about which I wasn't sure if i could tell you ;) Anyway, I really needed to wrap something for her yesterday, but another day has passed. And that is because of *me*. In Finland, we have this thing "me", but it means "we". Then we also have "hän", which means "he"/"she" etc. But using "hän" may feel needlessly formal, so we often instead use "se", which means "it". And that's that. Lots of these PEPs coming from everywhere and been going on for quite some time now! >>> it = iter(filter(f2, zip(filter(f1, peps), fun(other_peps)))) # ouch! I had a laptop with me the other day when going for a swim and somehow the battery had magically drained while I was swimming, although I thought I didn't swim that long. So I think my mind (or "taka-raivo", meaning back of the head; or "back rage"; or perhaps "muscle memory") is playing tricks on me. Or it could be that I just forgot to charge it. Ok, that's a strange story. There was another story, which is even stranger, but also quite embarrassing. So I told you this boring one instead. Oh, and someone mentioned a frog or prog or something and it seemed it was talking about me, and someTHING reminded me of krog. Maybe I should have rejected PEPs like this. Anyway, today I was reminded of an actual *girl friend* from quite some time ago, whose early PEP i didn't reject. We once went to India. The food was good but she thought it was hot ;) Oh, and there's a girl FRIEND of mine from school. She would call me Koppi, meaning "knock-π", or "election booth" or something. At some point in school we somehow ended up in a conversation about God. I had my own scientific ideas about it, which was very strange in the context so I told her a simplified version. But still, we keep rejecting each other's PEPs ;) Hey wait, i'm talking about girls all the time? Ok, music maybe? I had a two-meter high stack of recyclable paper from over the years. The stack has fallen over and broken my shoe rack, and now i noticed that, for some reason, the score of an often requested song was right there: "I love coffee, i love tea, i love the Java Jive and it loves me", dismissing the cover solo in the key of D. (memories) [.....] (saving the discussions) This reminds me of someone saying "I hope `it` is not very long". In a different context some years back, I tried to make someone longer, though obeying the laws of physics. It was so interesting that even the police got involved :D. However, there was no significant change in length. Just as already predicted from special relativity. There was a PEP about "early binding". That's a bit tricky, but seems to be big in 日本. It's possible that I have taken an early course on such things. I have a friend that could be into such things. Anyway, I'm sure there are special interest groups for "binding" ;) Recently, we drove with my parents to their summer place, which is not that far. On the way there, he told me mostly jokes, but was also looking at the difference between Merkel and Scholz (puntended in here!) PEPs keep coming in from different places. Even coffee machines are playing tricks on me. I can't just reject all the PEPs, can I? So I'm in trouble here! (and I originally wrote this as one of the first sentences in this email) I don't even know how many PEPs there are, and whether I yet need yet another one yet. And whether that's not NaN in some language. Getting more and more random, and pretty long. [....] Gooder like this? Yes me can! Let's C! Started moving my feet. TAU, i just had to dance a bit or two. Tread safe. Santa b*ches, shoe size 45! Need food, wash my clothes. Kiitos ja anteeksi. BTW, tosi hieno E! Ah, yes. Kind of wishing i could just reject all PEPs at once. But instead I'm still in big trouble! Eeeek! Help! Please help! Please, please help me deal with the PEPs! In the name of loves and doves, Signed, "sealed", am delivered, Yours sincerely, Koos PS. I'm getting tired, but you are still welcome to rejoice Finland's 104 years of independence (itsenäisyys, technically meaning "being as yourself") PSS. Currently struggling a bit with it myself; I blame the fact that these were mostly non-standard PEPs ;)

1 0

PEP template file now available
by Brett Cannon 07 Dec '21

07 Dec '21

I just pushed an update to PEP 12 which includes a complete PEP template file. You can see the file at https://github.com/python/peps/blob/main/pep-0012/pep-NNNN.rst and it is also embedded in PEP 12 at https://python.github.io/peps/pep-0012/#template (it will hit the official URL eventually).

1 0

[RELEASE] Python 3.10.1 is available
by Pablo Galindo Salgado 07 Dec '21

07 Dec '21

I hope you like bug fixes, because we have a whole shipment of them! Python 3.10.1 is the first maintenance release of Python 3.10 as we have packed more than 330 commits of fixes and general improvements. You can get it here: https://www.python.org/downloads/release/python-3101/ # This is the first maintenance release of Python 3.10 Python 3.10.1 is the newest major release of the Python programming language, and it contains many new features and optimizations. # Major new features of the 3.10 series, compared to 3.9 Among the new major new features and changes so far: * [PEP 623](https://www.python.org/dev/peps/pep-0623/) -- Deprecate and prepare for the removal of the wstr member in PyUnicodeObject. * [PEP 604](https://www.python.org/dev/peps/pep-0604/) -- Allow writing union types as X | Y * [PEP 612](https://www.python.org/dev/peps/pep-0612/) -- Parameter Specification Variables * [PEP 626](https://www.python.org/dev/peps/pep-0626/) -- Precise line numbers for debugging and other tools. * [PEP 618 ](https://www.python.org/dev/peps/pep-0618/) -- Add Optional Length-Checking To zip. * [bpo-12782](https://bugs.python.org/issue12782): Parenthesized context managers are now officially allowed. * [PEP 632 ](https://www.python.org/dev/peps/pep-0632/) -- Deprecate distutils module. * [PEP 613 ](https://www.python.org/dev/peps/pep-0613/) -- Explicit Type Aliases * [PEP 634 ](https://www.python.org/dev/peps/pep-0634/) -- Structural Pattern Matching: Specification * [PEP 635 ](https://www.python.org/dev/peps/pep-0635/) -- Structural Pattern Matching: Motivation and Rationale * [PEP 636 ](https://www.python.org/dev/peps/pep-0636/) -- Structural Pattern Matching: Tutorial * [PEP 644 ](https://www.python.org/dev/peps/pep-0644/) -- Require OpenSSL 1.1.1 or newer * [PEP 624 ](https://www.python.org/dev/peps/pep-0624/) -- Remove Py_UNICODE encoder APIs * [PEP 597 ](https://www.python.org/dev/peps/pep-0597/) -- Add optional EncodingWarning [bpo-38605](https://bugs.python.org/issue38605): `from __future__ import annotations` ([PEP 563](https://www.python.org/dev/peps/pep-0563/)) used to be on this list in previous pre-releases but it has been postponed to Python 3.11 due to some compatibility concerns. You can read the Steering Council communication about it [here]( https://mail.python.org/archives/list/python-dev@python.org/thread/CLVXXPQ2…) to learn more. # More resources * [Changelog](https://docs.python.org/3.10/whatsnew/changelog.html#changelog ) * [Online Documentation](https://docs.python.org/3.10/) * [PEP 619](https://www.python.org/dev/peps/pep-0619/), 3.10 Release Schedule * Report bugs at [https://bugs.python.org](https://bugs.python.org). * [Help fund Python and its community](/psf/donations/). # And now for something completely different The Meissner effect (or Meissner–Ochsenfeld effect) is the expulsion of a magnetic field from a superconductor during its transition to the superconducting state when it is cooled below the critical temperature. This expulsion will repel a nearby magnet. The German physicists Walther Meissner and Robert Ochsenfeld discovered this phenomenon in 1933 by measuring the magnetic field distribution outside superconducting tin and lead samples. The experiment demonstrated for the first time that superconductors were more than just perfect conductors and provided a uniquely defining property of the superconductor state. The ability for the expulsion effect is determined by the nature of equilibrium formed by the neutralization within the unit cell of a superconductor. You can do [ver cool things](https://www.youtube.com/watch?v=HRLvVkkq5GE) with it! # We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. https://www.python.org/psf/ Your friendly release team, Ned Deily @nad https://discuss.python.org/u/nad Steve Dower @steve.dower https://discuss.python.org/u/steve.dower Pablo Galindo Salgado @pablogsal https://discuss.python.org/u/pablogsal

1 0

Tool to search in the source code of PyPI top 5000 projects
by Victor Stinner 05 Dec '21

05 Dec '21

Hi, I wrote two scripts based on the work of INADA-san's work to (1) download the source code of the PyPI top 5000 projects (2) search for a regex in these projects (compressed source archives). You can use these tools if you work on an incompatible Python or C API change to estimate how many projects are impacted. The HPy project created a Git repository for a similar need (latest update in June 2021): https://github.com/hpyproject/top4000-pypi-packages There are also online services for code search: * GitHub: https://github.com/search * https://grep.app/ (I didn't try it yet) * Debian: https://codesearch.debian.net/ (1) Dowload Script: https://github.com/vstinner/misc/blob/main/cpython/download_pypi_top.py Usage: download_pypi_top.py PATH It uses this JSON file: https://hugovk.github.io/top-pypi-packages/top-pypi-packages-30-days.min.js… From this service: https://hugovk.github.io/top-pypi-packages/ At December 1, on 5000 projects, it only downloads 4760 tarball and ZIP archives: I guess that 240 projects don't provide a source archive. It takes around 5,2 GB of disk space. (2) Code search First, I used the fast and nice "ripgrep" tool with the command "rg -zl REGEX path/*.{zip,gz,bz2,tgz}" (-z searchs in ZIP and tarball archives). But it doesn't show the path inside the archive and it searchs in files generated by Cython whereas I wanted to ignore these files. So I wrote a short Python script which decompress tarball and ZIP archive in memory and looks for a regex: https://github.com/vstinner/misc/blob/main/cpython/search_pypi_top.py Usage: search_pypi_top.py "REGEX" output_filename The code to parse command line option is hardcoded and pypi_dir = "PYPI-2021-12-01-TOP-5000" are hardcoded :-D It ignores files generated by Cython and .so binary files (Linux dynamic libraries). While "rg" is very fast, my script is very slow. But I don't care, once the regex is written, I only need to search for the regex once, I can wait 10-15 min ;-) I prefer to wait longer and have a more accurate result. Also, there is room for enhancement, like running multiple jobs in different processes or threads. Victor -- Night gathers, and now my watch begins. It shall not end until my death.

4 5

peps, devguide, voters repositories: master branch renamed to main
by Victor Stinner 05 Dec '21

05 Dec '21

Hi, The "master" branch of the following Python GitHub repositories have been renamed to "main": * devguide * peps * voters For the rationale of the rename, see: https://sfconservancy.org/news/2020/jun/23/gitbranchname/ If you already have a Git checkout of one of these repositories, you can rename your local master branch with these commands (where "orign" is the github.com/python/... remote): --- git branch -m master main git fetch origin git branch -u origin/main main git remote set-head origin -a --- Tell me if you have any issue with the branch rename. Thanks Brett, Ee, Mariatta and the Steering Council who helped to rename these branches ;-) Victor -- Night gathers, and now my watch begins. It shall not end until my death.

2 1