Mailman 3 March 2021 - Python-Dev

Summary of Python tracker Issues
by Python tracker March 26, 2021

March 26, 2021

ACTIVITY SUMMARY (2021-03-19 - 2021-03-26) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7498 (+17) closed 47871 (+58) total 55369 (+75) Open issues with patches: 2984 Issues opened (54) ================== #33164: Blake 2 module update https://bugs.python.org/issue33164 reopened by christian.heimes #42129: Support resources in namespace … [View More]

1 0

Alternative syntax for Python's lambda
by Dan Stromberg March 26, 2021

March 26, 2021

I posted this to LWN, and thought I'd share it here too: I'm opposed to terse-ifying lambda in Python. Lambda is rarely useful in Python - you're almost always better off using a generator expression, a list comprehension, or something from the operator module. And lambdas tend to give rise to the software development equivalent of a runon sentence. Naming a function in those rare cases that you genuinely need something custom really isn't the end of the world, and is more readable anyway.

3 3

failing doc test
by Ethan Furman March 26, 2021

March 26, 2021

I have this documentation: -------------------------------------------------------------------------------- .. class:: FlagBoundary *FlagBoundary* controls how out-of-range values are handled in *Flag* and its subclasses. .. attribute:: STRICT Out-of-range values cause a :exc:`ValueError` to be raised. This is the default for :class:`Flag`:: >>> from enum import STRICT >>> class StrictFlag(Flag, boundary=STRICT): ..… [View More]. RED = auto() ... GREEN = auto() ... BLUE = auto() >>> StrictFlag(2**2 + 2**4) Traceback (most recent call last): ... ValueError: StrictFlag: invalid value: 20 given 0b0 10100 allowed 0b0 00111 .. attribute:: CONFORM Out-of-range values have invalid values removed, leaving a valid *Flag* value:: >>> from enum import CONFORM >>> class ConformFlag(Flag, boundary=CONFORM): ... RED = auto() ... GREEN = auto() ... BLUE = auto() >>> ConformFlag(2**2 + 2**4) ConformFlag.BLUE .. attribute:: EJECT Out-of-range values lose their *Flag* membership and revert to :class:`int`. This is the default for :class:`IntFlag`:: >>> from enum import EJECT >>> class EjectFlag(Flag, boundary=EJECT): ... RED = auto() ... GREEN = auto() ... BLUE = auto() >>> EjectFlag(2**2 + 2**4) 20 .. attribute:: KEEP Out-of-range values are kept, and the *Flag* membership is kept. This is used for some stdlib flags: >>> from enum import KEEP >>> class KeepFlag(Flag, boundary=KEEP): ... RED = auto() ... GREEN = auto() ... BLUE = auto() >>> KeepFlag(2**2 + 2**4) KeepFlag.BLUE|0x10 -------------------------------------------------------------------------------- and I'm getting this error: -------------------------------------------------------------------------------- Warning, treated as error: ********************************************************************** File "library/enum.rst", line ?, in default Failed example: class KeepFlag(Flag, boundary=KEEP): RED = auto() GREEN = auto() BLUE = auto() Exception raised: Traceback (most recent call last): File "/home/travis/build/python/cpython/Lib/doctest.py", line 1337, in __run exec(compile(example.source, filename, "single", File "<doctest default[1]>", line 1, in <module> class KeepFlag(Flag, boundary=KEEP): NameError: name 'Flag' is not defined -------------------------------------------------------------------------------- `Flag` has been imported before, the first three examples using `Flag` pass... any ideas on why the fourth test cannot find `Flag`? -- ~Ethan~ [View Less]

1 0

New public C API functions must not steal references or return borrowed references
by Victor Stinner March 26, 2021

March 26, 2021

Hi, A new Include/README.rst file was just added to document the 3 C API provided by CPython: * Include/: Limited C API * Include/cpython/: CPython implementation details * Include/internal/: The internal API I would like to note that *new* public C API functions must no longer steal references or return borrowed references. Don't worry, there is no plan to deprecate or remove existing functions which do that, like PyModule_AddObject() (streal a reference) or PyDict_GetItem() (return a … [View More]borrowed reference). The policy is only to *add* new functions. IMO for the *internal* C API, it's fine to continue doing that for best performances. Moreover, the limited C API must not expose "implementation details". For example, structure members must not be accessed directly, because most structures are excluded from the limited C API. A function call hiding implementation details is usually better. Here is a copy of the current Include/README.rst file: The Python C API ================ The C API is divided into three sections: 1. ``Include/`` 2. ``Include/cpython/`` 3. ``Include/internal/`` Include: Limited API ==================== ``Include/``, excluding the ``cpython`` and ``internal`` subdirectories, contains the public Limited API (Application Programming Interface). The Limited API is a subset of the C API, designed to guarantee ABI stability across Python 3 versions, and is defined in :pep:`384`. Guidelines for expanding the Limited API: - Functions *must not* steal references - Functions *must not* return borrowed references - Functions returning references *must* return a strong reference - Macros should not expose implementation details - Please start a public discussion before expanding the API - Functions or macros with a ``_Py`` prefix do not belong in ``Include/``. It is possible to add a function or macro to the Limited API from a given Python version. For example, to add a function to the Limited API from Python 3.10 and onwards, wrap it with ``#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000``. Include/cpython: CPython implementation details =============================================== ``Include/cpython/`` contains the public API that is excluded from the Limited API and the Stable ABI. Guidelines for expanding the public API: - Functions *must not* steal references - Functions *must not* return borrowed references - Functions returning references *must* return a strong reference Include/internal: The internal API ================================== With PyAPI_FUNC or PyAPI_DATA ----------------------------- Functions or structures in ``Include/internal/`` defined with ``PyAPI_FUNC`` or ``PyAPI_DATA`` are internal functions which are exposed only for specific use cases like debuggers and profilers. With the extern keyword ----------------------- Functions in ``Include/internal/`` defined with the ``extern`` keyword *must not and can not* be used outside the CPython code base. Only built-in stdlib extensions (built with the ``Py_BUILD_CORE_BUILTIN`` macro defined) can use such functions. When in doubt, new internal C functions should be defined in ``Include/internal`` using the ``extern`` keyword. Victor -- Night gathers, and now my watch begins. It shall not end until my death. [View Less]

7 7

OpenSSL 1.1.1k CVE fixes
by Christian Heimes March 25, 2021

March 25, 2021

Hi, OpenSSL released 1.1.1k today with two high severity CVEs, https://www.openssl.org/news/vulnerabilities.html The ssl module is not affected by CVE-2021-3450 in its default configuration. Python does not set X509_V_FLAG_X509_STRICT on SSLContext. Only applications that that use ssl.VERIFY_X509_STRICT verify flag are affected. It looks like Python's ssl module is vulnerable to CVE-2021-3449. The crash does not affect pip, requests, or any other client-side socket. Only server-side SSL/TLS … [View More]

1 0

Re: Steering Council reply regarding conduct (was Re: Steering Council update for February)
by Brian Herman March 25, 2021

March 25, 2021

Maybe we can change the discussion to something more productive like the python module system or anything else?

1 0

Steering Council reply regarding conduct (was Re: Steering Council update for February)
by Python Steering Council March 24, 2021

March 24, 2021

From Thomas Wouters, on behalf of and with full support of the Python Steering Council: I’ll post a separate reply about the merits of the decision, but we have to talk about this post from Steven D’Aprano in particular. Steven, this reply -- its tone and its message -- are completely unacceptable. It is offensive, dismissive and insulting. It’s a bad faith argument ridiculing a good faith effort. It is particularly insulting that you take the time to draw false equivalences to arguments that … [View More]weren’t made on this list, and that you’re assuming motivation and ridiculing it, without taking the time to engage in the actual arguments. There are plenty of sources online that you could’ve used to learn more, or you could have just asked in this thread. Instead, you posted this sarcastic, denigrating, mocking reply. Yours isn’t the only post that crosses the line in this thread, but it is the worst. It is an example of the kind of discourse that is too common and that reflects very badly on python-dev. It creates an atmosphere of disrespect and abuse, and actively scares away both experienced contributors and new ones. This is not conjecture; I and others on the SC have heard from more than a dozen people that this is exactly the case. This is not acceptable. It has to stop. The PSF Code of Conduct WG has sent the SC a recommendation on this issue. It recommended a temporary ban from python-dev for Steven, and a warning for someone else. While the SC believes a temporary ban would certainly be a reasonable action, we decided to instead make this public statement, as a warning to everyone. The SC wants to be clear that this statement applies to everyone who has made uncivil and inflammatory posts. This tone is not acceptable from anyone. The SC wants to make this very clear: this is a final warning to remind everyone that this behaviour will result in strong measures, including bans and ejections. If you want to argue something, do so on the merits, in good faith, and with empathy and consideration. If you can only ridicule someone, their motivation or their lived experience, please keep it to yourself. > On Tue, Mar 09, 2021 at 08:27:19PM +0000, Pablo Galindo Salgado wrote: > > > The Steering Council discussed renaming the master branch to main and > > the consensus was that we should do that. > > And about time too. Can we now tackle some of the equally pressing use > of offensive terms that are common in the Python community, starting > with the name of the language itself? > > Pythons are snakes, which is triggering to people with a phobia of > snakes. About one third of all people, or more than two *billion* > people, suffer from some level of phobia towards snakes. > > The popular "nose" testing framework is a blatant antisemetic and > neo-nazi dog whistle. > > "bool" is named after George Boole, a problematic white man who > appropriated the culture of both the Middle East and East Asia. > > "dict" is confusable with, and is often abbreviated to, an offensive > word. And don't even get me started with the obvious sexism of "tty". > > Unicode is racist because it has unified Chinese, Japanese and Korean > characters as if they were the same thing, and relegates non-Western > languages to second class status: > > https://modelviewculture.com/pieces/i-can-text-you-a-pile-of-poo-but-i-cant… > > It also includes a teddy bear symbol, which is named after notorious > racist and imperialist Theodore Roosevelt, and no less than *six* > swastika symbols. Also the Cross of Jerusalem, the symbol of such openly > fascist groups as the Federal State of Austria during the 1930s and the > Russian far-right extremist organisation the People's National Party. > > It even has a symbol for chains, which is associated even more closely > with slavery than "master". > > Speaking of slavery, in the standard library we have ChainMap and > itertools.chain. > > We have the ableist "runpy", and in the random module a function named > after Vilfredo Pareto, who supported the rule of fascist dictator Benito > Mussolini. There are the token and tokenize modules, which are offensive > for their association with both sexist and racist views. > > The tarfile module is associated with the racist Uncle Remus stories, > and a derogatory term for US Blacks. > > https://www.washingtonpost.com/blogs/compost/post/doug-lamborns-tar-baby-qu… > > The textwrap module uses a derogatory racist and fatphobic term dozens > of times. > > http://rsdb.org/slur/chunk > > Each of these issues are just as important as the "master" issue. > > > -- > Steve [View Less]

3 2

Merge request reminder.
by Alfred Perlstein March 24, 2021

March 24, 2021

Hello, a while back we hit a bug with doctest and mock.call that confused a few people on our team for a while. I dug into fixing it and made doctest more defensive around un-unwrappable calls. I submitted a PR (https://github.com/python/cpython/pull/22981) for an Issue (https://bugs.python.org/issue35753) back in October 25, 2020 which got some review, and then have not heard for several months. I am wondering if it can be reviewed and feedback given on a way forward. Thank you, -Alfred

1 0

typo in PEP 636
by codeofdusk＠gmail.com March 23, 2021

March 23, 2021

Hello, The GUI section of PEP 636 reads: "It will also requires that the event has a position attribute that matches the (x, y) pattern". It should read "It will also require that the event has a position attribute that matches the (x, y) pattern". Thanks, Bill

3 4

Tests now start with only 131 imported modules, instead of 233
by Victor Stinner March 23, 2021

March 23, 2021

Hi, tl;dr Tests of the Python Test Suite now start with 131 imported modules, instead of 233. For example, asyncio, logging, multiprocessing and warnings are no longer imported by default. -- The Python test suite is run by a custom test runner called "libregrtest" (test.libregrtest). It can detect reference leaks, write the output as JUnit XML, run tests in parallel with multiple processes, etc. Tests are written with test.support which is a collection of helper functions. Over the years, … [View More]test.support got more and more functions, and libregrtest got more and more features. The problem is that they import more and more modules. For example, "import test.support" imports 173 modules in Python 3.8! $ python3.8 >>> import sys, sys >>> before=set(sys.modules); import test.support; after=set(sys.modules) >>> len(after - before) 173 In these imports, you can find some "heavy" modules like asyncio and multiprocessing. Moreover, some modules have "side effects" on import. For example, "import logging" registers an "at fork" callback. In April 2020, I worked with Hai Shi on Python 3.10 in bpo-40275 to reduce the number of test.support imports from 171 to 25! test.support was splitted into multiple sub-modules: - bytecode_helper - hashlib_helper - import_helper - logging_helper - os_helper - script_helper - socket_helper - threading_helper - warnings_helper I continued the work in bpo-41718 to reduce the number of libregrtest imports, since libregrtest also imported many modules. A dummy test which does nothing in the master branch now only has 131 modules in sys.modules, whereas in Python 3.9 it has 233 (+102) modules! For example, asyncio, logging, multiprocessing and warnings are no longer imported by default. "import test.support" is now faster. master branch compared to Python 3.8 using ./python -c 'import test.support' command and Python built in release mode: [py38] 85.7 ms +- 6.8 ms -> [master] 32.4 ms +- 1.3 ms: 2.64x faster More realistic command running a single test method: ./python -m test test_os -m test_access [py38] 136 ms +- 5 ms -> [master] 97.8 ms +- 2.4 ms: 1.39x faster The only annoying point is that splitting test.support into sub-modules was not backported to Python 3.8 and 3.9, and so it will make backports a little bit more annoying. Sorry about that! -- If you know that two modules should be tested together, please write a dedicated test for that ;-) Victor -- Night gathers, and now my watch begins. It shall not end until my death. [View Less]

2 3