From report at bugs.python.org  Mon Nov  1 14:12:14 2021
From: report at bugs.python.org (James Lawrie)
Date: Mon, 01 Nov 2021 18:12:14 +0000
Subject: [New-bugs-announce] [issue45683] dns.asyncresolver ignores
 nameserver parameter
Message-ID: <1635790334.84.0.995507051537.issue45683@roundup.psfhosted.org>


New submission from James Lawrie <james at silvermouse.net>:

The DNS async resolver allows you to specify a list of nameservers to use, but they are ignored and the system nameservers are used instead.

Test code below demonstrating the issue:

# cat test.py
import dns.asyncresolver
import asyncio
from pprint import pprint
 
 
async def main(domains):
    results = await get_ips_bulk(domains)
    results = [item for sublist in results for item in sublist]
    pprint(results)
 
async def get_ips_bulk(domains):
    output = [get_ips(domain) for domain in domains]
    return await asyncio.gather(*output, return_exceptions=True)
 
async def get_ips(domain):
    res = dns.asyncresolver.Resolver()
    res.nameserver = ["1.1.1.1"]
    results = []
    try:
        ipv4 = await res.resolve(domain, 'A')
        for result in ipv4:
            results.append(['A', domain, result.to_text()])
    except:
        results.append(['A', domain, 'n/a'])
    try:
        ipv6 = await res.resolve(domain, 'AAAA')
        results.append(['AAAA', domain, result.to_text()])
    except:
        results.append(['AAAA', domain, 'n/a'])
 
    return results
  
domains = ['python.org']
asyncio.run(main(domains))
 

It should use 1.1.1.1 as the nameserver but watching tcpdump it's using 8.8.8.8 per /etc/resolv.conf:

# tcpdump -n port 53 &
[1] 16751
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
# python3 test.py
19:05:02.750458 IP x.x.x.x.44173 > 8.8.8.8.53: 46143+ A? python.org. (28)
19:05:02.756265 IP 8.8.8.8.53 > x.x.x.x.44173: 46143 1/0/0 A 138.197.63.241 (44)
19:05:02.757392 IP x.x.x.x.37827 > 8.8.8.8.53: 17493+ AAAA? python.org. (28)
19:05:02.765797 IP 8.8.8.8.53 > x.x.x.x.37827: 17493 0/1/0 (115)
[['A', 'python.org', '138.197.63.241'], ['AAAA', 'python.org', 'n/a']]
# fg
tcpdump -n port 53
^C
# grep -P "^nameserver" /etc/resolv.conf
nameserver 8.8.8.8

----------
components: asyncio
messages: 405460
nosy: asvetlov, james2, yselivanov
priority: normal
severity: normal
status: open
title: dns.asyncresolver ignores nameserver parameter
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45683>
_______________________________________

From report at bugs.python.org  Mon Nov  1 14:23:01 2021
From: report at bugs.python.org (Alex Waygood)
Date: Mon, 01 Nov 2021 18:23:01 +0000
Subject: [New-bugs-announce] [issue45684] `functools.singledispatchmethod`
 does not define `__class_getitem__`
Message-ID: <1635790981.47.0.0130236033988.issue45684@roundup.psfhosted.org>


New submission from Alex Waygood <Alex.Waygood at Gmail.com>:

`functools.singledispatchmethod` is marked as being a generic class in `typeshed`, but does not define `__class_getitem__`, so cannot be parameterized at runtime.

cpython source code: https://github.com/python/cpython/blob/e2063d6a1ebc3568e90a14ed163fa291b5977ae8/Lib/functools.py#L891

typeshed stub: https://github.com/python/typeshed/blob/f4143c40e85db42dc98549e09329e196668395ee/stdlib/functools.pyi#L94

----------
components: Library (Lib)
messages: 405462
nosy: AlexWaygood, lukasz.langa, rhettinger
priority: normal
severity: normal
status: open
title: `functools.singledispatchmethod` does not define `__class_getitem__`
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45684>
_______________________________________

From report at bugs.python.org  Mon Nov  1 19:09:34 2021
From: report at bugs.python.org (giancarlo niccolo virgutto)
Date: Mon, 01 Nov 2021 23:09:34 +0000
Subject: [New-bugs-announce] [issue45685] PuthonLauncher app fails run
 scripts on Mac BigSur
Message-ID: <1635808174.14.0.867223626801.issue45685@roundup.psfhosted.org>


New submission from giancarlo niccolo virgutto <gnvirgutto1990 at gmail.com>:

I can?t figure out how to use Python Launcher on BigSur. Or it is broken. Anyway: It does absolutely nothing. When I double click a script file, I see very shortly the preferences window of Launcher, and this is all. The script is not executed. I made a very simple script which writes a small text file. The script works flawlessly, when I call it in the terminal, but the Launcher seems not even to open it.

Is this a bug with BigSur, or what is to be done?

----------
components: macOS
messages: 405475
nosy: gnvirgutto1990, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: PuthonLauncher app fails run scripts on Mac BigSur
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45685>
_______________________________________

From report at bugs.python.org  Mon Nov  1 20:50:51 2021
From: report at bugs.python.org (Michael Wayne Goodman)
Date: Tue, 02 Nov 2021 00:50:51 +0000
Subject: [New-bugs-announce] [issue45686] ElementTree.Element.extend: bad
 error message when error occurs in generator
Message-ID: <1635814251.62.0.355245938965.issue45686@roundup.psfhosted.org>


New submission from Michael Wayne Goodman <goodman.m.w at gmail.com>:

Both the Python and C versions of xml.etree.ElementTree allow Element.extend() to accept a sequence or an iterable, such as a generator expression (although the docs only mention "sequence"; that's a separate problem):

>>> from xml.etree import ElementTree as ET
>>> elem = ET.Element('root')
>>> elem.extend(ET.Element('child') for _ in range(3))
>>> pyelem = ET._Element_Py('root')
>>> pyelem.extend(ET._Element_Py('child') for _ in range(3))

If the generator expression raises an Exception (for instance, by omitting the tag name as below), then the Python implementation passes up the error:

>>> pyelem.extend(ET._Element_Py() for _ in range(3))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/xml/etree/ElementTree.py", line 253, in extend
    for element in elements:
  File "<stdin>", line 1, in <genexpr>
TypeError: __init__() missing 1 required positional argument: 'tag'

But the C implementation hides it with a misleading exception of its own:

>>> elem.extend(ET.Element() for _ in range(3))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expected sequence, not "generator"

Is it possible to pass up the original exception, as in the Python implementation? In the C code, the exception is raised when PySequence_Fast returns something false:

    seq = PySequence_Fast(elements, "");
    if (!seq) {
        PyErr_Format(
            PyExc_TypeError,
            "expected sequence, not \"%.200s\"", Py_TYPE(elements)->tp_name
            );
        return NULL;
    }

Otherwise, while it would be good if the error message didn't incorrectly state that Element.extend() requires a sequence and not a generator, it currently leads the programmer to the actual error. That is, when the programmer tries to make the argument a sequence by creating a list, the error will be propagated before Element.extend() is called:

>>> elem.extend([ET.Element() for _ in range(3)])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <listcomp>
TypeError: Element() takes at least 1 argument (0 given)

----------
components: XML
messages: 405476
nosy: goodmami
priority: normal
severity: normal
status: open
title: ElementTree.Element.extend: bad error message when error occurs in generator
type: behavior
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45686>
_______________________________________

From report at bugs.python.org  Tue Nov  2 04:21:56 2021
From: report at bugs.python.org (=?utf-8?q?Micha=C5=82_Bartoszkiewicz?=)
Date: Tue, 02 Nov 2021 08:21:56 +0000
Subject: [New-bugs-announce] [issue45687] Infinite recursion in
 Pickler.persistent_id
Message-ID: <1635841316.88.0.961902802593.issue45687@roundup.psfhosted.org>


New submission from Micha? Bartoszkiewicz <embe at navalgo.com>:

The following code, which seems reasonable:
import io
import pickle

class Pickler(pickle.Pickler):
  def persistent_id(self, obj):
    return super().persistent_id(obj)

Pickler(io.BytesIO()).dump(42)

crashes with:
RecursionError: maximum recursion depth exceeded while calling a Python object

It works perfectly when inheriting from pickle._Pickler (the Python implementation).

----------
components: Library (Lib)
files: pickle-bug.py
messages: 405494
nosy: embe-navalgo
priority: normal
severity: normal
status: open
title: Infinite recursion in Pickler.persistent_id
type: behavior
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file50418/pickle-bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45687>
_______________________________________

From report at bugs.python.org  Tue Nov  2 04:24:05 2021
From: report at bugs.python.org (Christian Heimes)
Date: Tue, 02 Nov 2021 08:24:05 +0000
Subject: [New-bugs-announce] [issue45688] stdlib_module_names.h is missing
 _scproxy
Message-ID: <1635841445.61.0.662562018122.issue45688@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

stdlib_module_names.h is missing the macOS specific module _scproxy. 
Guido ran into the problem in PR https://github.com/python/cpython/pull/29118

----------
components: Build
messages: 405495
nosy: christian.heimes, eric.snow, gvanrossum, vstinner
priority: normal
severity: normal
status: open
title: stdlib_module_names.h is missing _scproxy
type: behavior
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45688>
_______________________________________

From report at bugs.python.org  Tue Nov  2 05:15:22 2021
From: report at bugs.python.org (Tangellapalli Sai Hanuma Rahul)
Date: Tue, 02 Nov 2021 09:15:22 +0000
Subject: [New-bugs-announce] [issue45689] Custom Name for ThreadPoolExecutor
Message-ID: <1635844522.2.0.423855204781.issue45689@roundup.psfhosted.org>


New submission from Tangellapalli Sai Hanuma Rahul <saihanumarahul66 at gmail.com>:

Feature Request:

Where we can use custom Names for separate threads submitted in ThreadPoolExecutor. 

It achieves by sending the name string to _work_queue of ThreadPoolExecutor and then in _worker function modifies the name of the current thread (through name property).

----------
components: Library (Lib)
files: test.py
hgrepos: 409
messages: 405496
nosy: RahulARanger
priority: normal
severity: normal
status: open
title: Custom Name for ThreadPoolExecutor
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file50419/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45689>
_______________________________________

From report at bugs.python.org  Tue Nov  2 09:39:20 2021
From: report at bugs.python.org (andrew cooke)
Date: Tue, 02 Nov 2021 13:39:20 +0000
Subject: [New-bugs-announce] [issue45690] Argparse exclusive group inside
 required exclusive group displays incorrectly
Message-ID: <1635860360.92.0.910770346728.issue45690@roundup.psfhosted.org>


New submission from andrew cooke <andrew at acooke.org>:

The code below, when invoked with -h, prints:

(.env) [andrew at localhost py]$ python -m tests_sa.argparse_bug -h
usage: argparse_bug.py [-h] (-a A | [-b B | -c C)]

options:
  -h, --help  show this help message and exit
  -a A
  -b B
  -c C

where the final two characters in the usage line are swapped.  It should be

usage: argparse_bug.py [-h] (-a A | [-b B | -c C])

or maybe even

usage: argparse_bug.py [-h] (-a A | (-b B | -c C))



from argparse import ArgumentParser

def main():
    parser = ArgumentParser()
    outer = parser.add_mutually_exclusive_group(required=True)
    outer.add_argument('-a')
    inner = outer.add_mutually_exclusive_group()
    inner.add_argument('-b')
    inner.add_argument('-c')
    parser.parse_args()


if __name__ == '__main__':
    main()

----------
components: Library (Lib)
messages: 405509
nosy: acooke
priority: normal
severity: normal
status: open
title: Argparse exclusive group inside required exclusive group displays incorrectly
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45690>
_______________________________________

From report at bugs.python.org  Tue Nov  2 12:04:49 2021
From: report at bugs.python.org (Mark Shannon)
Date: Tue, 02 Nov 2021 16:04:49 +0000
Subject: [New-bugs-announce] [issue45691] Partial moving of core objects to
 interpreter state is incorrect at best, unsafe at worse.
Message-ID: <1635869089.92.0.280577291297.issue45691@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

We currently have an unstable state in the VM where some core objects are static and some are per-interpreter.

For example, smalls ints are allocated per-interpreter, but many classes are allocated statically.
This means that if any int is reachable from a class, then references to per-interpreter objects can be left dangling, or be out of date.

E.g. consider this sequence:
1. Create an interpreter
2. Destroy it.
3. Create a new interpreter

`sys.float_info.n_unnamed_fields` causes a memory violation if the per-interpreter allocated 0 held by sys.float_info.n_unnamed_fields is freed.
If it is not freed, then `sys.float_info.n_unnamed_fields is 0` is False, meaning that there are two zeros present.

The above is just an example. Classes have many references to ints, floats, code  objects, etc. Any of those could have the same issue.

All objects that form the core object graph must either be entirely static, or entirely per-interpreter.

We cannot change from static to per-interpreter in a piecemeal fashion. It must be done all at once.

----------
components: Interpreter Core
messages: 405514
nosy: Mark.Shannon, eric.snow, vstinner
priority: normal
severity: normal
status: open
title: Partial moving of core objects to interpreter state is incorrect at best, unsafe at worse.
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45691>
_______________________________________

From report at bugs.python.org  Tue Nov  2 12:45:21 2021
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 02 Nov 2021 16:45:21 +0000
Subject: [New-bugs-announce] [issue45692] IDLE: define word/id chars in one
 place.
Message-ID: <1635871521.54.0.926952310508.issue45692@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

IDLE currently defines the same set of chars in 5 places with 5 names. (Listed by Serhiy Storchaka in #45669.)
Lib/idlelib/autoexpand.py:20:    wordchars = string.ascii_letters + string.digits + "_"
Lib/idlelib/undo.py:254:    alphanumeric = string.ascii_letters + string.digits + "_"
Lib/idlelib/editor.py:809:    IDENTCHARS = string.ascii_letters + string.digits + "_"
Lib/idlelib/hyperparser.py:13:_ASCII_ID_CHARS = frozenset(string.ascii_letters + string.digits + "_")
Lib/idlelib/autocomplete.py:33:ID_CHARS = string.ascii_letters + string.digits + "_"

I suspect that either a string or frozenset would work everywhere (check).  I will pick a name after checking this.  The single definition would go in the proposed utils.py, which is part of another issue and PR.

(Note: the utility tk index functions should also go there.)

----------
assignee: terry.reedy
components: IDLE
messages: 405516
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE: define word/id chars in one place.
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45692>
_______________________________________

From report at bugs.python.org  Tue Nov  2 14:32:35 2021
From: report at bugs.python.org (Jim Crist-Harif)
Date: Tue, 02 Nov 2021 18:32:35 +0000
Subject: [New-bugs-announce] [issue45693] `loop.create_server` with port=0
 uses different ports for ipv4 & ipv6
Message-ID: <1635877955.05.0.115164279782.issue45693@roundup.psfhosted.org>


New submission from Jim Crist-Harif <jiminy.crist at gmail.com>:

To create a new server with `loop.create_server` that listens on all interfaces and a random port, I'd expect passing in `host=""`, `port=0` to work (per the documentation). However, as written this results in 2 different ports being used - one for ipv4 and one for ipv6. Instead I'd expect a single random port be determined once, and reused for all other interfaces.

Running the example test code (attached) results in:

```
$ python test.py
listening on 0.0.0.0:38023
listening on :::40899
Traceback (most recent call last):
  File "/home/jcristharif/Code/distributed/test.py", line 36, in <module>
    asyncio.run(main())
  File "/home/jcristharif/miniconda3/envs/dask/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/jcristharif/miniconda3/envs/dask/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/jcristharif/Code/distributed/test.py", line 30, in main
    assert len(ports) == 1, "Only 1 port expected!"
AssertionError: Only 1 port expected!
```

This behavior can be worked around by manually handling `port=0` outside of asyncio, but as it stands naive use can result in accidentally listening on multiple ports.

----------
components: asyncio
files: test.py
messages: 405530
nosy: asvetlov, jcristharif, yselivanov
priority: normal
severity: normal
status: open
title: `loop.create_server` with port=0 uses different ports for ipv4 & ipv6
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file50421/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45693>
_______________________________________

From report at bugs.python.org  Tue Nov  2 15:01:34 2021
From: report at bugs.python.org (Irit Katriel)
Date: Tue, 02 Nov 2021 19:01:34 +0000
Subject: [New-bugs-announce] [issue45694] Limit the number of chained
 exceptions included in formatted traceback
Message-ID: <1635879694.7.0.702941754246.issue45694@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at gmail.com>:

There is currently a limit on the number of frames in each traceback, but not on the number of chained exceptions displayed. This is why traceback.py avoids recursion (making the code more complex). A limit should probably be added.

----------
components: Interpreter Core, Library (Lib)
messages: 405533
nosy: iritkatriel
priority: normal
severity: normal
status: open
title: Limit the number of chained exceptions included in formatted traceback
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45694>
_______________________________________

From report at bugs.python.org  Tue Nov  2 19:29:45 2021
From: report at bugs.python.org (Eric Snow)
Date: Tue, 02 Nov 2021 23:29:45 +0000
Subject: [New-bugs-announce] [issue45695] Out-of-tree builds are not tested.
Message-ID: <1635895785.09.0.174517598815.issue45695@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

Currently we don't test builds done outside the source tree, neither on GitHub nor the buildbots. [1]  As a result, such builds get broken occasionally.  I've certainly broken then a couple times, inadvertently.  It would be helpful if we tested out-of-tree builds on GitHub, or at least on a stable buildbot.

FTR, to do an out-of-tree build locally and test it:

mkdir ../build
cd ../build
../cpython/configure
make
./python -m test


[1] Actually I recently added test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script which happens to do an out-of-tree build incidentally.  However, we should be testing out-of-tree builds explicitly.

----------
components: Build
messages: 405568
nosy: christian.heimes, eric.snow, gregory.p.smith, gvanrossum, steve.dower
priority: normal
severity: normal
stage: test needed
status: open
title: Out-of-tree builds are not tested.
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45695>
_______________________________________

From report at bugs.python.org  Tue Nov  2 20:16:36 2021
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 03 Nov 2021 00:16:36 +0000
Subject: [New-bugs-announce] [issue45696] "Deep-freeze": skip the marshal
 step by generating C code
Message-ID: <1635898596.2.0.043840127306.issue45696@roundup.psfhosted.org>


New submission from Guido van Rossum <guido at python.org>:

See https://github.com/faster-cpython/ideas/issues/84

This appears to improve startup time by another 10% or more compared to main.

----------
assignee: gvanrossum
components: Interpreter Core
messages: 405570
nosy: gvanrossum
priority: normal
severity: normal
status: open
title: "Deep-freeze": skip the marshal step by generating C code
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45696>
_______________________________________

From report at bugs.python.org  Tue Nov  2 21:12:41 2021
From: report at bugs.python.org (Itamar Ostricher)
Date: Wed, 03 Nov 2021 01:12:41 +0000
Subject: [New-bugs-announce] [issue45697] PyType_IsSubtype is doing
 excessive work in the common case
Message-ID: <1635901961.22.0.404899300991.issue45697@roundup.psfhosted.org>


New submission from Itamar Ostricher <itamarost at gmail.com>:

Based on real world profiling data we collected, a vast amount of `PyType_IsSubtype` calls are coming from `type_call`, when it decides whether `__init__` should run or not.

In the common case, the arguments to this call are identical, but the implementation still walks the MRO.

By returning early for identical types, the common case can be optimized with a non-trivial performance gain.

----------
components: Interpreter Core
messages: 405575
nosy: itamaro
priority: normal
severity: normal
status: open
title: PyType_IsSubtype is doing excessive work in the common case
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45697>
_______________________________________

From report at bugs.python.org  Wed Nov  3 02:00:18 2021
From: report at bugs.python.org (Mgs M Rizqi Fadhlurrahman)
Date: Wed, 03 Nov 2021 06:00:18 +0000
Subject: [New-bugs-announce] [issue45698] Error on importing getopt
Message-ID: <1635919218.86.0.867470836468.issue45698@roundup.psfhosted.org>


New submission from Mgs M Rizqi Fadhlurrahman <rizqirizqi23 at gmail.com>:

It looks like there is a breaking change related to python 3.10.0 implementation that impacted getopt package.

https://docs.python.org/3/library/getopt.html
https://github.com/python/cpython/blob/3.10/Lib/getopt.py

Steps to reproduce:
1. Run python 3.10.0
2. Type `import getopt`
3. The error shows up

I already tried python 3.9.7 and it's still working as expected.

----------
components: Library (Lib)
files: OnPaste.20211103-125049.png
messages: 405582
nosy: rizqirizqi23
priority: normal
severity: normal
status: open
title: Error on importing getopt
type: crash
versions: Python 3.10
Added file: https://bugs.python.org/file50422/OnPaste.20211103-125049.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45698>
_______________________________________

From report at bugs.python.org  Wed Nov  3 03:49:03 2021
From: report at bugs.python.org (Kishor Pawar)
Date: Wed, 03 Nov 2021 07:49:03 +0000
Subject: [New-bugs-announce] [issue45699] AttributeError: 'list' object has
 no attribute 'find'
Message-ID: <1635925743.59.0.349899416416.issue45699@roundup.psfhosted.org>


New submission from Kishor Pawar <krisp1506 at gmail.com>:

```
Traceback (most recent call last):
  File "gpgcheck.py", line 33, in <module>
    if config.get(repoName, 'gpgcheck', fallback='1') != "1":
  File "/usr/lib64/python3.6/configparser.py", line 802, in get
    d)
  File "/usr/lib64/python3.6/configparser.py", line 394, in before_get
    self._interpolate_some(parser, option, L, value, section, defaults, 1)
  File "/usr/lib64/python3.6/configparser.py", line 413, in _interpolate_some
    p = rest.find("%")
AttributeError: 'list' object has no attribute 'find'
```


I checked all the calls made to `_interpolate_some`, and each call seems to pass the `list` type argument in the 4th position, but then function calling a `find` method on that 4th `list` type argument.  Which is causing above error.

----------
components: Parser
messages: 405584
nosy: krisp1506, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: AttributeError: 'list' object has no attribute 'find'
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45699>
_______________________________________

From report at bugs.python.org  Wed Nov  3 04:54:55 2021
From: report at bugs.python.org (Nojima Takahide)
Date: Wed, 03 Nov 2021 08:54:55 +0000
Subject: [New-bugs-announce] [issue45700] Got SEGV in compilation python3.6
 with GCC-11, and please renewal python3.6.
Message-ID: <1635929695.24.0.420047257674.issue45700@roundup.psfhosted.org>


New submission from Nojima Takahide <nozzy123nozzy at gmail.com>:

I found python3.6.15(latest) caused SEGV while installing it using pyenv with GCC-11 on Debian sid. I show the Error below,

---------------here-----------------------
$ pyenv install -k 3.6.15
Downloading Python-3.6.15.tar.xz...
-> https://www.python.org/ftp/python/3.6.15/Python-3.6.15.tar.xz
Installing Python-3.6.15...

BUILD FAILED (Debian unstable using python-build 2.1.0-12-g5963dc4f)

Inspect or clean up the working tree at ~/.anyenv/envs/pyenv/sources/3.6.15
Results logged to /tmp/python-build.20211103135942.18999.log

Last 10 log lines:
if test "xupgrade" != "xno"  ; then \
        case upgrade in \
                upgrade) ensurepip="--upgrade" ;; \
                install|*) ensurepip="" ;; \
        esac; \
         ./python -E -m ensurepip \
                $ensurepip --root=/ ; \
fi
Segmentation fault
make: *** [Makefile:1102: install] Error 139
$ gcc -v
...snip...
gcc version 11.2.0 (Debian 11.2.0-10) 
--------------Here------------------------

This root cause is the same as bpo-27987. In the case of python 3.6.15, the "gcc-11 -O3" applies "movaps" against the non-16bytes alignment address of "PyObject."  I show this clue as below,

--------------Here-----------------------
$ gdb --args ./python -m ensurepip
(gdb) run
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6be957e in PyCFuncPtr_new (type=0x555555c3cde8, args=0x7ffff6c6c630, kwds=0x0)
    at ~/.anyenv/envs/pyenv/sources/3.6.15/Python-3.6.15/Modules/_ctypes/_ctypes.c:3557
3557        self->thunk = thunk;
(gdb) disas PyCFuncPtr_new
   ...snip
   0x00007ffff6be955d <+221>:   je     0x7ffff6be98a0 <PyCFuncPtr_new+1056>
   0x00007ffff6be9563 <+227>:   mov    0x78(%rsp),%rax
   0x00007ffff6be9568 <+232>:   movq   %r15,%xmm0
   0x00007ffff6be956d <+237>:   movq   %rax,%xmm1
   0x00007ffff6be9572 <+242>:   addq   $0x1,(%rax)
   0x00007ffff6be9576 <+246>:   mov    0x10(%r13),%rax
   0x00007ffff6be957a <+250>:   punpcklqdq %xmm1,%xmm0
   0x00007ffff6be957e <+254>:   movaps %xmm0,0x60(%r13) ;<- cause SEGV
   0x00007ffff6be9583 <+259>:   mov    0x20(%r15),%rdx
   0x00007ffff6be9587 <+263>:   mov    %rdx,(%rax)

Checking %r13 at "movaps",it has non-16bytes alignment address as its value.
------------------Here------------------

 Then, I would much appriciate if someone does ``cherry-pick`` the following 2 commits of github.com/python/cpython to its branch 3.6,  
  - 8766cb74e186d3820db0a855ccd780d6d84461f7 
  - f0be4bbb9b3cee876249c23f2ae6f38f43fa7495
and please releases the newer python3.6. I already apply and tested this in my Debian box and I confirmed  it to solve this problem.

Thanks.

 Note: For someone who met the same problem, I show two tiny workarounds as below, if no one can release the newer python3.6 with this remedy, because of near the end of life of 3.6.

Workaround1: Use GCC version 10. The GCC-10 seems not to have this problem.
 command: env CC=gcc-10 pyenv install 3.6.15

Workaround2: Use -O2 level with GCC-11. The "-O2" seems to skip the vectorize optimization so that GCC doesn't apply "movaps" at all. As a result, it can avoid this problem.
 command: env CFLAGS=-O2 pyenv install 3.6.15
---
Takahide Nojima

----------
components: Installation
messages: 405588
nosy: Nojima Takahide
priority: normal
severity: normal
status: open
title: Got SEGV in compilation python3.6 with GCC-11, and please renewal python3.6.
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45700>
_______________________________________

From report at bugs.python.org  Wed Nov  3 06:47:28 2021
From: report at bugs.python.org (Nikita Sobolev)
Date: Wed, 03 Nov 2021 10:47:28 +0000
Subject: [New-bugs-announce] [issue45701] Add tuple tests to
 `functools.lru_cache`
Message-ID: <1635936448.66.0.888386290275.issue45701@roundup.psfhosted.org>


New submission from Nikita Sobolev <mail at sobolevn.me>:

Context: https://bugs.python.org/issue45679

We need to add tests that `tuple` works as expected with `functools.lru_cache`, including `1` / `1.0` / `True` and `0` / `0.0` / `False` special cases.

PR: https://github.com/python/cpython/pull/29339

----------
components: Tests
messages: 405601
nosy: sobolevn
priority: normal
severity: normal
status: open
title: Add tuple tests to `functools.lru_cache`
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45701>
_______________________________________

From report at bugs.python.org  Wed Nov  3 07:13:46 2021
From: report at bugs.python.org (Oleg Iarygin)
Date: Wed, 03 Nov 2021 11:13:46 +0000
Subject: [New-bugs-announce] [issue45702] Python/dtoa.c requires 53 bit
 hardware rounding unavalable on x64
Message-ID: <1635938026.99.0.538390017253.issue45702@roundup.psfhosted.org>


New submission from Oleg Iarygin <oleg at arhadthedev.net>:

File configure.ac:4617 states the following:

> # The short float repr introduced in Python 3.1 requires the
> # correctly-rounded string <-> double conversion functions from
> # Python/dtoa.c, which in turn require that the FPU uses 53-bit
> # rounding; this is a problem on x86, where the x87 FPU has a default
> # rounding precision of 64 bits.  For gcc/x86, we can fix this by
> # using inline assembler to get and set the x87 FPU control word.

However, x64 programs use xmm0-7 SIMD registers instead of a FPU stack, so the requirement of hardware 56 bit rounding gets violated. _Py_SET_53BIT_PRECISION_* is unable to do anything here because SSE neither respects the FPU control word, nor provides its own.

Considering that SSE is supported since Pentium III (1999), we can safely enforce its usage for x32 code via compiler flags as well, getting consistent behavior across builds. However, it requires revision of the requirement.

We need to decide what to do with dtoa.c that supposedly relies on the requirement (providing short floats? It looks like _Py_dg_dtoa just stringifies as many digits as specified in ndigits argument. I didn't collect enough courage on this stage to analyze 484 uncommented lines (2370-2854) of bit manipulation over two-letter variables to find and fix the dependency). By the way, the only place that uses _Py_dg_dtoa is Objects/floatobject.c:949 doing it with an argument mode=3.

I can resolve the issue, but I need some educated opinion on this 13-years old stuff (see git blame for configure.ac:4617) to not break things.

Note: I initially added Mark Dickinson into the notifications as a person who created and maintained Python/dtoa.c until 2013.

----------
components: Interpreter Core
messages: 405603
nosy: arhadthedev, mark.dickinson
priority: normal
severity: normal
status: open
title: Python/dtoa.c requires 53 bit hardware rounding unavalable on x64
type: behavior
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45702>
_______________________________________

From report at bugs.python.org  Wed Nov  3 07:21:56 2021
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Wed, 03 Nov 2021 11:21:56 +0000
Subject: [New-bugs-announce] [issue45703] importlib.invalidate_caches() does
 not invalidate _NamespacePath's _last_parent_path-based cache
Message-ID: <1635938516.86.0.42328943914.issue45703@roundup.psfhosted.org>


New submission from Miro Hron?ok <miro at hroncok.cz>:

Recently, when debugging a weird problem (see https://bugzilla.redhat.com/show_bug.cgi?id=2018551 for details if interested, but not important for this issue), I've realized that the _NamespacePath class (from importlib/_bootstrap_external.py) has a cache that (in some cases) uses tuple(sys.path) as the key. I was expecting importlib.invalidate_caches() to invalidate this cache, but it doesn't.


Consider the following directory structure:

.
??? PATH1
??? ??? namespace
???     ??? sub1
???         ??? __init__.py
??? PATH2
    ??? namespace
        ??? sub2
            ??? __init__.py

Here is a helper to create it (on Linux-ish):

$ mkdir -p PATH1/namespace/sub1
$ mkdir -p PATH2/namespace/sub2
$ touch PATH1/namespace/sub1/__init__.py
$ touch PATH2/namespace/sub2/__init__.py


Run Python with PYTHONPATH=PATH1:PATH2 (output slightly formatted for readability):


$ PYTHONPATH=PATH1:PATH2 python3.11
>>> import namespace
>>> namespace.__path__
_NamespacePath(['.../namespace_path_cache/PATH1/namespace',
                '.../namespace_path_cache/PATH2/namespace'])
>>> import namespace.sub1  # works
>>> import namespace.sub2  # works
>>> exit()

The namespace packages seem to work as expected.


Now move PATH2/namespace out of the way:

$ mv PATH2/namespace PATH2/cant-import-this

Run Python again:


$ PYTHONPATH=PATH1:PATH2 python3.11
>>> import namespace
>>> namespace.__path__
_NamespacePath(['.../namespace_path_cache/PATH1/namespace'])
>>> ...

While this interpreter still runs, move the PATH2/namespace module back in:

$ mv PATH2/cant-import-this PATH2/namespace

>>> ...
>>> namespace.__path__
_NamespacePath(['.../namespace_path_cache/PATH1/namespace'])
>>> import namespace.sub2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'namespace.sub2'

>>> import importlib
>>> importlib.invalidate_caches()  # invalidate the cache, not helpful
>>> namespace.__path__
_NamespacePath(['.../namespace_path_cache/PATH1/namespace'])
>>> import namespace.sub2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'namespace.sub2'


>>> import sys
>>> sys.path.remove('')  # changing sys.path solves this
>>> namespace.__path__
_NamespacePath(['.../namespace_path_cache/PATH1/namespace'])
>>> import namespace.sub2
>>> namespace.__path__
_NamespacePath(['.../namespace_path_cache/PATH1/namespace',
                '.../namespace_path_cache/PATH2/namespace'])



importlib.invalidate_caches() documentation says:

> This function should be called if any modules are created/installed while your program is running to guarantee all finders will notice the new module?s existence.

That makes me think calling importlib.invalidate_caches() should also invalidate the cache of _NamespacePaths.

(This also affects older Pythons, but since it is a behavior change, I've only marked 3.11).

----------
messages: 405606
nosy: hroncok, petr.viktorin
priority: normal
severity: normal
status: open
title: importlib.invalidate_caches() does not invalidate _NamespacePath's _last_parent_path-based cache
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45703>
_______________________________________

From report at bugs.python.org  Wed Nov  3 07:55:47 2021
From: report at bugs.python.org (Sascha Desch)
Date: Wed, 03 Nov 2021 11:55:47 +0000
Subject: [New-bugs-announce] [issue45704] string.Formatter.parse does not
 handle auto-numbered positional fields
Message-ID: <1635940547.79.0.60719103588.issue45704@roundup.psfhosted.org>


New submission from Sascha Desch <sascha.desch at hotmail.com>:

It appears when adding auto-numbered positional fields in python 3.1 `Formatter.parse` was not updated to handle them and currently returns an empty string as the field name.

```
list(Formatter().parse('hello {}'))  # [('hello ', '', '', None)]
```

This does not align with `Formatter.get_field` which according to the docs: "Given field_name as returned by parse() (see above), convert it to an object to be formatted."

When supplying an empty string to `.get_field()` you get a KeyError

```
Formatter().get_field("", [1, 2, 3], {}). # raises KeyError
```

----------
messages: 405610
nosy: SDesch
priority: normal
severity: normal
status: open
title: string.Formatter.parse does not handle auto-numbered positional fields
type: behavior
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45704>
_______________________________________

From report at bugs.python.org  Wed Nov  3 16:24:40 2021
From: report at bugs.python.org (James Bowery)
Date: Wed, 03 Nov 2021 20:24:40 +0000
Subject: [New-bugs-announce] [issue45705] |= set update scoping
Message-ID: <1635971080.35.0.70243315955.issue45705@roundup.psfhosted.org>


New submission from James Bowery <jabowery at gmail.com>:

Comment out the |= line and it prints "{'b':2}" as expected.

$ cat t.py
scoped_dict = {'b':2}
def scoped_def():
    print(scoped_dict)
    scoped_dict |= {'a',1}
scoped_def()

$ p t.py 
Traceback (most recent call last):
  File "/home/jabowery/dev/t.py", line 5, in <module>
    scoped_def()
  File "/home/jabowery/dev/t.py", line 3, in scoped_def
    print(scoped_dict)
UnboundLocalError: local variable 'scoped_dict' referenced before assignment

$ python --version
Python 3.10.0

----------
components: Parser
messages: 405643
nosy: jabowery2, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: |= set update scoping
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45705>
_______________________________________

From report at bugs.python.org  Wed Nov  3 20:29:59 2021
From: report at bugs.python.org (Prem Buczkowski)
Date: Thu, 04 Nov 2021 00:29:59 +0000
Subject: [New-bugs-announce] [issue45706] Add IMAP4.login_plain to imaplib
Message-ID: <1635985799.82.0.278344713307.issue45706@roundup.psfhosted.org>


New submission from Prem Buczkowski <prem at prem.moe>:

Hi everyone and nice to meet you!

I would like to propose adding a method login_plain to Python IMAP4 standard library client.

Currently, one can use login, login_cram_md5 or add their own authentication method using authenticate. While login works great for plain text authentication for English users, it does not support characters out of ASCII, causing errors for users with non-ASCII passwords. 

I personally hit that with radicale-imap. It has been reported in issue34577, and its SMTP counterpart in issue29750.

RFC6855 recommends using AUTHENTICATE for such cases, and so this patch adds login_plain in a manner similar to login_cram_md5. Adding this to a standard library will improve the experience for non-English users not only when programming but also when using other programs written in Python. AUTHENTICATE PLAIN is on by default in Dovecot and most e-mail providers, like Gmail and Outlook, making it more common than, already supported, AUTHENTICATE CRAM-MD5.

Since this is my first patch, let me know if I could change something up - I did my best to RTFM!

----------
components: Library (Lib), email
files: 0107-Add-imaplib.IMAP4.login_plain.patch
keywords: patch
messages: 405659
nosy: barry, przemub, r.david.murray
priority: normal
severity: normal
status: open
title: Add IMAP4.login_plain to imaplib
Added file: https://bugs.python.org/file50424/0107-Add-imaplib.IMAP4.login_plain.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45706>
_______________________________________

From report at bugs.python.org  Wed Nov  3 23:24:22 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Thu, 04 Nov 2021 03:24:22 +0000
Subject: [New-bugs-announce] [issue45707] Variable reassginment triggers
 incorrect behaviors of locals()
Message-ID: <1635996262.23.0.981239272273.issue45707@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

Normally after executing a piece of code in a function, locals() should contain the local variables and these variables can be reassigned next. In the following code, "attr" should be found in locals(). Actually, it can not be found in either locals() or globals() after executing code "attr = 1". This program triggers a keyError. I think something wrong during handling locals().

================================
def foo():
    exec("attr = 1")
    a = locals()['attr']
    attr = 2 
foo()
================================

Reported Error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in foo
KeyError: 'attr'


Expected output:
This test program should work well. The value of a is 1 and the value of attr is 2 after execution. No error is reported.

Python version: python3.10, Ubuntu 16.04

----------
components: Interpreter Core
messages: 405660
nosy: xxm
priority: normal
severity: normal
status: open
title: Variable reassginment triggers incorrect behaviors of locals()
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45707>
_______________________________________

From report at bugs.python.org  Thu Nov  4 05:01:14 2021
From: report at bugs.python.org (Sander Bollen)
Date: Thu, 04 Nov 2021 09:01:14 +0000
Subject: [New-bugs-announce] [issue45708] PEP 515-style formatting with
 underscores does not seem to work for Decimal
Message-ID: <1636016474.79.0.0748645326286.issue45708@roundup.psfhosted.org>


New submission from Sander Bollen <sander at sndrtj.eu>:

Hello,

It appears that Decimal does not support PEP-515 style formatting with underscores as thousands separators. 

```
>>> from decimal import Decimal
>>> f"{Decimal('5000'):,}"
'5,000'
>>> f"{Decimal('5000'):_}"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid format string

```

This does work for all the other types mentioned in PEP515

```
>>> f"{5000:_}"
'5_000'
>>> f"{5000.0:_}"
'5_000.0'
>>> f"{complex(5000, 1):_}"
'(5_000+1j)'
```

I have tried this on python 3.8, 3.9 and 3.10 on a Mac.

----------
components: Library (Lib)
messages: 405667
nosy: sndrtj
priority: normal
severity: normal
status: open
title: PEP 515-style formatting with underscores does not seem to work for Decimal
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45708>
_______________________________________

From report at bugs.python.org  Thu Nov  4 06:08:51 2021
From: report at bugs.python.org (Ned Batchelder)
Date: Thu, 04 Nov 2021 10:08:51 +0000
Subject: [New-bugs-announce] [issue45709] 3.11 regression: tracing
 with-statement on exit from block
Message-ID: <1636020531.52.0.942589294345.issue45709@roundup.psfhosted.org>


New submission from Ned Batchelder <ned at nedbatchelder.com>:

Python 3.11 seems to have reverted a behavior that was new in 3.10.0b1: exiting a with-statement re-visits the with line on the way out.

--- %<  bug2.py ----------------------
import linecache, sys

def trace(frame, event, arg):
    # The weird globals here is to avoid a NameError on shutdown...
    if frame.f_code.co_filename == globals().get("__file__"):
        lineno = frame.f_lineno
        print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, lineno).rstrip()))
    return trace

print(sys.version)
sys.settrace(trace)

import contextlib

def f():
    with contextlib.nullcontext():
        1/0

f()
-------------------------------------

Running with 3.10 shows re-visiting the with:

    $ python3.10 bug2.py
    3.10.0 (default, Oct  4 2021, 17:22:29) [Clang 12.0.0 (clang-1200.0.32.29)]
    call 15: def f():
    line 16:     with contextlib.nullcontext():
    line 17:         1/0
    exce 17:         1/0
    line 16:     with contextlib.nullcontext():
    retu 17:         1/0
    Traceback (most recent call last):
      File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 19, in <module>
        f()
      File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 17, in f
        1/0
    ZeroDivisionError: division by zero

3.11 does not:

    $ python3.11 bug2.py
    3.11.0a1 (default, Oct  6 2021, 07:21:05) [Clang 12.0.0 (clang-1200.0.32.29)]
    call 15: def f():
    line 16:     with contextlib.nullcontext():
    line 17:         1/0
    exce 17:         1/0
    retu 17:         1/0
    Traceback (most recent call last):
      File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 19, in <module>
        f()
        ^^^
      File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 17, in f
        1/0
        ~^~
    ZeroDivisionError: division by zero

Versions before 3.10 also do not visit the with statement:

    $ python3.9 bug2.py
    3.9.7 (default, Sep  7 2021, 22:16:49)
    [Clang 12.0.0 (clang-1200.0.32.29)]
    call 15: def f():
    line 16:     with contextlib.nullcontext():
    line 17:         1/0
    exce 17:         1/0
    line 17:         1/0
    retu 17:         1/0
    Traceback (most recent call last):
      File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 19, in <module>
        f()
      File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 17, in f
        1/0
    ZeroDivisionError: division by zero


Is this a bug in 3.11, or an intentional return to previous behavior?

(BTW: there is no 3.11regression keyword available)

----------
components: Interpreter Core
messages: 405668
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: 3.11 regression: tracing with-statement on exit from block
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45709>
_______________________________________

From report at bugs.python.org  Thu Nov  4 06:57:57 2021
From: report at bugs.python.org (Fabio Storino)
Date: Thu, 04 Nov 2021 10:57:57 +0000
Subject: [New-bugs-announce] [issue45710] Junction/symbolic folder access
 error on Windows 11
Message-ID: <1636023477.57.0.826365620965.issue45710@roundup.psfhosted.org>


New submission from Fabio Storino <fstorino at yahoo.com>:

After upgrading to Windows 11 I can't run Python scripts from a junction folder anymore. Everything else works as before on that folder.

Background: I have Minecraft installed on a second volume (E:\Games\Minecraft). I created a junction folder at %appdata%\.minecraft pointing to that folder. When I try to run a Python script from that folder I get this error:

---
python: can't open file 'C:\Users\xxxxx\AppData\Roaming\.minecraft\test.py': [Errno 22] Invalid argument
---

When trying to debug in VS Code, I get a more detailed error:

---
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2032.0_x64
    __qbz5n2kfra8p0\lib\ntpath.py", line 647, in realpath
    path = _getfinalpathname(path)
OSError: [WinError 649] The create operation failed because the name
    contained at least one mount point which resolves to a volume to
    which the specified device object is not attached:
    'c:\\Users\\xxxxx\\AppData\\Roaming\\.minecraft'
---

When trying to open that junction folder in Python's IDLE I get the following error message:

---
[Window Title] Location is not available

[Content] C:\Users\xxxxx\AppData\Roaming\.minecraft is not accessible.

The create operation failed because the name contained at least one
mount point which resolves to a volume to which the specified device
object is not attached.
---

I tried creating a symbolic directory link instead of a junction, but got the same message.

I can run the same scripts directly from the target folder (E:\Games\Minecraft).

I can also run them from the junction folder from an Ubuntu terminal on WSL2 (Windows Subsystem for Linux).

I tried this with Python 3.9 and 3.10 (WSL2 uses Python 3.8).

----------
components: Windows
messages: 405677
nosy: fstorino, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Junction/symbolic folder access error on Windows 11
type: behavior
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45710>
_______________________________________

From report at bugs.python.org  Thu Nov  4 07:29:47 2021
From: report at bugs.python.org (Irit Katriel)
Date: Thu, 04 Nov 2021 11:29:47 +0000
Subject: [New-bugs-announce] [issue45711] Simplify the interpreter's (type,
 val, tb) exception representation
Message-ID: <1636025387.26.0.346517237471.issue45711@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at gmail.com>:

Exceptions are represented in the interpreter as (type, val, tb) triplets which most of the time contain redundant information (the type is the type of val and the tb is also on the exception). This complicates the code and is inefficient as opcodes that manage exceptions push and pop 3 items for each exception. 

We will change the internal representation to be (1) just the exception value if it is normalised and (2) a tuple of the 3 values for the uncommon case where they are all needed.

See also https://github.com/faster-cpython/ideas/issues/106.

----------
components: Interpreter Core
messages: 405681
nosy: Mark.Shannon, gvanrossum, iritkatriel
priority: normal
severity: normal
status: open
title: Simplify the interpreter's (type, val, tb) exception representation
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45711>
_______________________________________

From report at bugs.python.org  Thu Nov  4 08:17:04 2021
From: report at bugs.python.org (Andreas Ley)
Date: Thu, 04 Nov 2021 12:17:04 +0000
Subject: [New-bugs-announce] [issue45712] so it not allowed
Message-ID: <1636028224.13.0.297564810423.issue45712@roundup.psfhosted.org>


New submission from Andreas Ley <andreas.ley at kit.edu>:

https://docs.python.org/3.11/tutorial/controlflow.html
(and earlier versions) have the sentence:

An unpacking like **rest is also supported. (But **_ would be redundant, so it not allowed.)

Although I'm not a native speaker, I suppose this is missing a verb and should read:

? so it is not allowed

----------
assignee: docs at python
components: Documentation
messages: 405714
nosy: Andy, docs at python
priority: normal
severity: normal
status: open
title: so it not allowed
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45712>
_______________________________________

From report at bugs.python.org  Thu Nov  4 09:38:37 2021
From: report at bugs.python.org (vamsi kalapala)
Date: Thu, 04 Nov 2021 13:38:37 +0000
Subject: [New-bugs-announce] [issue45713] gcc warning when compiling
 Modules/expat/xmltok_ns.c
Message-ID: <1636033117.93.0.543536023335.issue45713@roundup.psfhosted.org>


New submission from vamsi kalapala <vamsi1281977 at gmail.com>:

The code that triggers the compiler warning is:

NS(findEncoding)(const ENCODING *enc, const char *ptr, const char *end) {
#  define ENCODING_MAX 128
  char buf[ENCODING_MAX];  /// <==== THIS GIVES A WARNING.
  // THE FIX IS ====> char buf[ENCODING_MAX] = "";
  char *p = buf;
  int i;
  XmlUtf8Convert(enc, &ptr, end, &p, p + ENCODING_MAX - 1);
  if (ptr != end)
    return 0;
  *p = 0;
  if (streqci(buf, KW_UTF_16) && enc->minBytesPerChar == 2)
    return enc;
  i = getEncodingIndex(buf);
  if (i == UNKNOWN_ENC)
    return 0;
  return NS(encodings)[i];
}

----------
messages: 405717
nosy: vamsi1281977
priority: normal
severity: normal
status: open
title: gcc warning when compiling Modules/expat/xmltok_ns.c
type: compile error
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45713>
_______________________________________

From report at bugs.python.org  Thu Nov  4 10:01:30 2021
From: report at bugs.python.org (Skip Montanaro)
Date: Thu, 04 Nov 2021 14:01:30 +0000
Subject: [New-bugs-announce] [issue45714] test_multiprocessing_spawn hangs
 sometimes
Message-ID: <1636034490.33.0.255156135487.issue45714@roundup.psfhosted.org>


New submission from Skip Montanaro <skip.montanaro at gmail.com>:

I find that test_multiprocessing_spawn frequently hangs. Hitting Ctl-C then rerunning "make test" generally works. Still, this behavior makes it problematic to run testing unattended. I don't think I have an unusual environment (XUbuntu 20.04, GCC 9.3.0). Here's some output from the currently running/hanging process, just after the last other unit test completed.

0:03:42 load avg: 4.96 [421/422/21] test_concurrent_futures passed (2 min 57 sec) -- running: test_multiprocessing_spawn (2 min 59 sec)
0:04:12 load avg: 3.19 running: test_multiprocessing_spawn (3 min 29 sec)
0:04:42 load avg: 1.93 running: test_multiprocessing_spawn (3 min 59 sec)
0:05:12 load avg: 1.23 running: test_multiprocessing_spawn (4 min 29 sec)
0:05:42 load avg: 0.74 running: test_multiprocessing_spawn (4 min 59 sec)
...
0:22:12 load avg: 0.42 running: test_multiprocessing_spawn (21 min 29 sec)
0:22:42 load avg: 0.27 running: test_multiprocessing_spawn (21 min 59 sec)
0:23:12 load avg: 0.37 running: test_multiprocessing_spawn (22 min 29 sec)

I'm currerntly mostly messing around with the nogil code and 3.9 tip, but I've noticed this problem on other branches as well. The fact that this seems not to have been reported before suggests that I'm somehow an outlier. If it was common, my guess is that the buildbots would experience this problem on a regular basis.

Here is the current python process info (output of pgrep -fla python).

1572195 /home/skip/src/python/cpython/python -u -W default -bb -E -m test -r -w -j 0 -u all,-largefile,-audio,-gui
1575193 /home/skip/src/python/cpython/python -bb -E -Wdefault -u -m test.regrtest --worker-args [{"testdir": null, "verbose": 0, "quiet": false, "exclude": false, "single": false, "randomize": true, "fromfile": null, "findleaks": 1, "use_resources": ["curses", "network", "decimal", "cpu", "subprocess", "urlfetch"], "trace": false, "coverdir": "coverage", "runleaks": false, "huntrleaks": false, "verbose2": true, "print_slow": false, "random_seed": 2495022, "use_mp": 10, "verbose3": false, "forever": false, "header": false, "failfast": false, "match_tests": null, "ignore_tests": null, "pgo": false, "timeout": null, "wait": false, "worker_args": null, "start": null, "match_filename": null, "ignore_filename": null, "use": [["all", "-largefile", "-audio", "-gui"]], "memlimit": null, "threshold": null, "nowindows": false, "list_tests": false, "list_cases": false, "pgo_extended": false, "fail_env_changed": false, "xmlpath": null, "tempdir": "/home/skip/src/python/cpython/build/test_python_1572195", "cleanup": false, "args": []}, "test_multiprocessing_spawn"]
1575489 /home/skip/src/python/cpython/python -bb -E -Wdefault -c from multiprocessing.resource_tracker import main;main(3)
1575547 /home/skip/src/python/cpython/python -bb -E -Wdefault -c from multiprocessing.forkserver import main; main(6, 7, ['__main__', 'test.test_multiprocessing_forkserver'], **{'sys_path': ['/home/skip/src/python/cpython', '/home/skip/tmp/nogilpgo_build/lib/python39.zip', '/home/skip/src/python/cpython/Lib', '/home/skip/src/python/cpython/build/lib.linux-x86_64-3.9', '/home/skip/.local/lib/python3.9/site-packages', '/home/skip/tmp/nogilpgo_build/lib/python3.9/site-packages']})
1589033 /home/skip/src/python/cpython/python -bb -E -Wdefault -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=4, pipe_handle=12) --multiprocessing-fork
1589034 /home/skip/src/python/cpython/python -bb -E -Wdefault -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=4, pipe_handle=14) --multiprocessing-fork

At the moment I can't pstack either of the last two processes. Unfortunately, I think I need to tweak ptrace.conf and reboot.

----------
messages: 405718
nosy: skip.montanaro
priority: normal
severity: normal
status: open
title: test_multiprocessing_spawn hangs sometimes

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45714>
_______________________________________

From report at bugs.python.org  Thu Nov  4 11:31:24 2021
From: report at bugs.python.org (Jacek)
Date: Thu, 04 Nov 2021 15:31:24 +0000
Subject: [New-bugs-announce] [issue45715] round(2500, -3)
Message-ID: <1636039884.59.0.199110305946.issue45715@roundup.psfhosted.org>


New submission from Jacek <jacekandrzejblazejewski at gmail.com>:

round(2500, -3) returns 2000 instead of 3000.
I have checked it on 3.7.4 and 3.6.9.

----------
components: Library (Lib)
messages: 405726
nosy: jacekblaz
priority: normal
severity: normal
status: open
title: round(2500, -3)
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45715>
_______________________________________

From report at bugs.python.org  Thu Nov  4 13:45:48 2021
From: report at bugs.python.org (Carl Friedrich Bolz-Tereick)
Date: Thu, 04 Nov 2021 17:45:48 +0000
Subject: [New-bugs-announce] [issue45716] Confusing parsing error message
 when trying to use True as keyword argument
Message-ID: <1636047949.0.0.244366488793.issue45716@roundup.psfhosted.org>


New submission from Carl Friedrich Bolz-Tereick <cfbolz at gmx.de>:

A bit of a nitpick, but the following SyntaxError message is a bit confusing:

>>> f(True=1)
  File "<stdin>", line 1
    f(True=1)
      ^^^^^
SyntaxError: expression cannot contain assignment, perhaps you meant "=="?

The problem with that line is not that it contains an assignment, it's almost a valid keyword argument after all. The problem is that the name of the keyword is True, which is no longer a name you can assign to. It would be better to produce the same error as with __debug__:

>>> f(__debug__=1)
  File "<stdin>", line 1
SyntaxError: cannot assign to __debug__

The latter error message is however produced by the compiler, not the parser I think?

----------
messages: 405741
nosy: Carl.Friedrich.Bolz
priority: normal
severity: normal
status: open
title: Confusing parsing error message when trying to use True as keyword argument

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45716>
_______________________________________

From report at bugs.python.org  Thu Nov  4 15:07:09 2021
From: report at bugs.python.org (Joshua)
Date: Thu, 04 Nov 2021 19:07:09 +0000
Subject: [New-bugs-announce] [issue45717] Bad link to python docs in
 help(_hashlib)
Message-ID: <1636052829.11.0.396753142278.issue45717@roundup.psfhosted.org>


New submission from Joshua <joshua at himmens.com>:

I was attempting to look through hashlib to try and understand more about python's built-in hash functions. As part of this, I ran 'help(_hashlib)' which returns this text: 'MODULE REFERENCE
    https://docs.python.org/3.11/library/_hashlib.html'

This is an invalid link which has been an issue in all versions of the documentation I looked up.

As an aside, where would I find the source for _hashlib?

----------
components: Library (Lib)
messages: 405744
nosy: Joshuah143
priority: normal
severity: normal
status: open
title: Bad link to python docs in help(_hashlib)
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45717>
_______________________________________

From report at bugs.python.org  Thu Nov  4 18:22:39 2021
From: report at bugs.python.org (William Fisher)
Date: Thu, 04 Nov 2021 22:22:39 +0000
Subject: [New-bugs-announce] [issue45718] asyncio: MultiLoopWatcher has a
 race condition (Proposed work-around)
Message-ID: <1636064559.39.0.358204060297.issue45718@roundup.psfhosted.org>


New submission from William Fisher <william.w.fisher at gmail.com>:

Summary: asyncio.MultiLoopChildWatcher has two problems that create a race condition.

1. The SIGCHLD signal handler does not guard against interruption/re-entry.
2. The SIGCHLD signal handler can interrupt add_child_handler's `self._do_waitpid(pid)`.

This is a continuation of bpo-38323. That issue discussed two bugs. This issue proposes a work-around for one of them that may be useful in making build tests more reliable.

I'm reserving discussion to the case of a single asyncio event loop on the main thread. (MultiLoopChildWatcher has a separate "signal-delivery-blocked" problem when used in an event loop that is not in the main thread as mentioned in bpo-38323.)


Symptoms:

Log messages that look like this:

1634935451.761 WARNING Unknown child process pid 8747, will report returncode 255
...
1634935451.762 WARNING Child watcher got an unexpected pid: 8747
Traceback (most recent call last):
  File "/Users/runner/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/asyncio/unix_events.py", line 1306, in _do_waitpid
    loop, callback, args = self._callbacks.pop(pid)
KeyError: 8747


Background:

I've been working on a library to make calling asyncio subprocesses more convenient. As part of my testing, I've been stress testing asyncio using different child watcher policies. My CI build runs more than 200 tests each on macOS, Linux and FreeBSD. I've noticed a small percentage of sporadic failures using MultiLoopChildWatcher.

My understanding of Python signal functions is that:

1. Upon receipt of a signal, the native "C" signal handler sets a flag that indicates the signal arrived.
2. The main thread checks the signal flags after each bytecode instruction. If a signal flag is set, Python saves the call stack, runs the signal handler on the main thread immediately, then pops the stack when it returns (assuming no exception raised by signal handler).
3. If you are in the middle of a signal handler running on the main thread and Python detects another signal flag, your signal handler can be interrupted.
4. Stacked signal handlers run in LIFO order. The last one that enters will run to completion without interruption.


Explanation:

I wrapped MultiLoopChildWatcher's sig_chld function in a decorator that logs when it is entered and exited. This clearly shows that _sig_chld is being re-entered. In the following log snippet, I'm running two commands in a pipeline "tr | cat".

1634935451.743 DEBUG process '/usr/bin/tr' created: pid 8747
...
1634935451.746 DEBUG process '/bin/cat' created: pid 8748
...
1634935451.761 DEBUG enter '_sig_chld' 20
1634935451.761 DEBUG enter '_sig_chld' 20
1634935451.761 WARNING Unknown child process pid 8747, will report returncode 255
1634935451.762 DEBUG process 8748 exited with returncode 0
1634935451.762 DEBUG exit '_sig_chld' 20
1634935451.762 WARNING Child watcher got an unexpected pid: 8747
Traceback (most recent call last):
  File "/Users/runner/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/asyncio/unix_events.py", line 1306, in _do_waitpid
    loop, callback, args = self._callbacks.pop(pid)
KeyError: 8747
1634935451.763 WARNING Unknown child process pid 8748, will report returncode 255
1634935451.763 WARNING Child watcher got an unexpected pid: 8748
Traceback (most recent call last):
  File "/Users/runner/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/asyncio/unix_events.py", line 1306, in _do_waitpid
    loop, callback, args = self._callbacks.pop(pid)
KeyError: 8748
1634935451.763 DEBUG exit '_sig_chld' 20


Here is the breakdown of what happens:

1. Pid 8747 exits and we enter _sig_chld #1.
2. sig_chld #1 calls os.waitpid which gives (pid, status) = (8747, 0).
3. Before sig_chld #1 has a chance to call `self._callbacks.pop(pid)`, it is interrupted.
4. sig_chld #2 calls os.waitpid for pid 8747. We get a ChildProcessError and then "Unknown child process pid 8747, will report returncode 255"
5. sig_chld #2 invokes the callback for pid 8747 saying the returncode=255.
6. sig_chld #2 continues to completion. It reaps pid 8748 normally.
7. sig_chld #1 picks up where it left off. We get an error when we try to pop the callback for 8747.
8. sig_chld #1 calls os.waitpid for pid 8748. This gives us failure messages because it was done by sig_chld #2.

The issue of interruption can also happen in the case of running a single process. If the _sig_chld interrupts the call to `self._do_waitpid(pid)` in add_child_watcher, a similar interleaving can occur.


Work-Around:

In my tests, I patched MultiLoopChildWatcher and so far, it appears to be more reliable. In add_child_handler, I call raise_signal(SIGCHLD) so that all the work is done in the signal handler.


class PatchedMultiLoopChildWatcher(asyncio.MultiLoopChildWatcher):
    "Test race condition fixes in MultiLoopChildWatcher."

    def add_child_handler(self, pid, callback, *args):
        loop = asyncio.get_running_loop()
        self._callbacks[pid] = (loop, callback, args)

        # Prevent a race condition in case signal was delivered before
        # callback added.
        signal.raise_signal(signal.SIGCHLD)

    @_serialize
    def _sig_chld(self, signum, frame):
        super()._sig_chld(signum, frame)


_serialize is a decorator that looks like this:


def _serialize(func):
    """Decorator to serialize a non-reentrant signal function.
    If one client is already in the critical section, set a flag to run the
    section one more time. Testing purposes only.
    """

    lock = threading.Lock()  # Used as atomic test-and-set.
    retry = False

    @functools.wraps(func)
    def _decorator(*args, **kwargs):
        nonlocal retry

        while True:
            if lock.acquire(blocking=False):  # pylint: disable=consider-using-with
                try:
                    retry = False
                    func(*args, **kwargs)
                finally:
                    lock.release()
                if retry:
                    continue
            else:
                # A signal handler that interrupts an existing handler will
                # run to completion (LIFO).
                retry = True
            break

    return _decorator

----------
components: asyncio
messages: 405756
nosy: asvetlov, byllyfish, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: MultiLoopWatcher has a race condition (Proposed work-around)
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45718>
_______________________________________

From report at bugs.python.org  Thu Nov  4 20:42:46 2021
From: report at bugs.python.org (DJ PJs)
Date: Fri, 05 Nov 2021 00:42:46 +0000
Subject: [New-bugs-announce] [issue45719] SubProcess stdin.flush freezes
 when running python interpreter
Message-ID: <1636072966.86.0.9507982269.issue45719@roundup.psfhosted.org>


New submission from DJ PJs <djp1012878 at gmail.com>:

Keeping writing to a subprocess, and then flushing it to get the stdout works for other interpreters, but not python's.

Simplified Example of what I mean:

from subprocess import PIPE, Popen, CREATE_NEW_CONSOLE, run

subProcess = Popen("Python", stdin=PIPE, stdout=PIPE, text=True, universal_newlines=True)

subProcess.stdin.write('Print("HelloWorld")')

subProcess.stdin.flush()

for line in subProcess.stdout:
    print(">>> " + str(line.rstrip()))
    subProcess.stdout.flush()

----------
components: Library (Lib)
files: example.py
messages: 405761
nosy: djp1012878
priority: normal
severity: normal
status: open
title: SubProcess stdin.flush freezes when running python interpreter
type: behavior
versions: Python 3.10
Added file: https://bugs.python.org/file50425/example.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45719>
_______________________________________

From report at bugs.python.org  Thu Nov  4 21:04:25 2021
From: report at bugs.python.org (Steve Dower)
Date: Fri, 05 Nov 2021 01:04:25 +0000
Subject: [New-bugs-announce] [issue45720] Remove shlwapi dependency on
 Windows
Message-ID: <1636074265.43.0.608267631685.issue45720@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

According to https://twitter.com/BruceDawson0xB/status/1455714820485894151?s=20 there are some serious performance implications from referencing shlwapi.dll.

It turns out, we only use it for one native path calculation function, which is easily replaceable (since Windows 8). So we can drop the dependency and get a startup (and shutdown) improvement back to 3.9.

----------
assignee: steve.dower
components: Windows
messages: 405762
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Remove shlwapi dependency on Windows
type: performance
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45720>
_______________________________________

From report at bugs.python.org  Thu Nov  4 21:53:40 2021
From: report at bugs.python.org (Steven D'Aprano)
Date: Fri, 05 Nov 2021 01:53:40 +0000
Subject: [New-bugs-announce] [issue45721] Improve error message when python
 shell command is entered at the REPL prompt
Message-ID: <1636077220.31.0.800245298483.issue45721@roundup.psfhosted.org>


New submission from Steven D'Aprano <steve+python at pearwood.info>:

A frequent newbie mistake is to call shell commands from inside the interactive interpreter. Most common is to call Python itself.

Here is an example where a Python instructor was allegedly unable to diagnose the issue for their students:

https://windowsquestions.com/2021/10/09/syntaxerror-invalid-syntax-perhaps-you-forgot-a-comma/


I think it would be a nice feature if the compiler recognised obvious cases of "user tried to call Python from the Python prompt", and suggested a fix. If the statement matches the regex r"python\s+" the error message might say "it looks like you are trying to run a shell command at the Python prompt" rather than suggest a missing comma.

----------
messages: 405764
nosy: steven.daprano
priority: normal
severity: normal
status: open
title: Improve error message when python shell command is entered at the REPL prompt
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45721>
_______________________________________

From report at bugs.python.org  Fri Nov  5 03:32:21 2021
From: report at bugs.python.org (Jens Rapp)
Date: Fri, 05 Nov 2021 07:32:21 +0000
Subject: [New-bugs-announce] [issue45722] documentation missing information
 on objects in submodules
Message-ID: <1636097541.32.0.940629899159.issue45722@roundup.psfhosted.org>


New submission from Jens Rapp <rapp.jens at googlemail.com>:

Documentation 5.4.2. Submodules tells what happens to modules which are imported inside __init__.py of a package>


from .foo import Foo
from .bar import Bar

then executing the following puts a name binding to foo and bar in the spam module:
>>>

>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>

I miss information on what happes to Foo and Bar. 
is it directly usable under spam.Bar() or does one have to use spam.bar.Bar()?

To my mind, that example should tell this.

----------
assignee: docs at python
components: Documentation
messages: 405769
nosy: docs at python, rapp.jens
priority: normal
severity: normal
status: open
title: documentation missing information on objects in submodules
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45722>
_______________________________________

From report at bugs.python.org  Fri Nov  5 06:05:47 2021
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 05 Nov 2021 10:05:47 +0000
Subject: [New-bugs-announce] [issue45723] Improve and simplify configure.ac
 checks
Message-ID: <1636106747.04.0.649539670364.issue45723@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

The autoconf-based build system has room for improvements. The configure.ac script can be simplified in several places by using AS and AC macros or by defining new custom macros.

For example we have a lot of blocks that look like this:

AC_MSG_CHECKING(for chroot)
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <unistd.h>]], [[void x=chroot]])],
  [AC_DEFINE(HAVE_CHROOT, 1, Define if you have the 'chroot' function.)
   AC_MSG_RESULT(yes)],
  [AC_MSG_RESULT(no)
])

The block has an issue, too. It does not use AC_CACHE to cache the result. The entire block can be replaced by a custom macro that takes care of everything and implements correct caching:

  PY_CHECK_FUNC([chroot], [#include <unistd.h>])

We can also move several library and header checks from setup.py into configure.ac, e.g. check for soundcard.h or gdbm libs and headers.

----------
assignee: christian.heimes
components: Build
messages: 405780
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Improve and simplify configure.ac checks
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45723>
_______________________________________

From report at bugs.python.org  Fri Nov  5 07:16:05 2021
From: report at bugs.python.org (ghost_in _the_wires)
Date: Fri, 05 Nov 2021 11:16:05 +0000
Subject: [New-bugs-announce] [issue45724] Segmentation fault
Message-ID: <1636110965.6.0.294482496197.issue45724@roundup.psfhosted.org>


Change by ghost_in _the_wires <mr.wild17 at gmail.com>:


----------
files: log.txt
nosy: ghost.in.the.wires
priority: normal
severity: normal
status: open
title: Segmentation fault
versions: Python 3.10
Added file: https://bugs.python.org/file50426/log.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45724>
_______________________________________

From report at bugs.python.org  Fri Nov  5 07:17:21 2021
From: report at bugs.python.org (Mark Shannon)
Date: Fri, 05 Nov 2021 11:17:21 +0000
Subject: [New-bugs-announce] [issue45725] test_freeze doesn't clean up after
 itself
Message-ID: <1636111041.41.0.0249123262066.issue45725@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

test_tools leaves a copy of the source in Tools/freeze/test/outdir/

which messes up grep and other tools used for searching the source.

----------
components: Tests
messages: 405784
nosy: Mark.Shannon, eric.snow
priority: normal
severity: normal
status: open
title: test_freeze doesn't clean up after itself
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45725>
_______________________________________

From report at bugs.python.org  Fri Nov  5 08:15:07 2021
From: report at bugs.python.org (Alex Waygood)
Date: Fri, 05 Nov 2021 12:15:07 +0000
Subject: [New-bugs-announce] [issue45726] Documentation for
 `@singledispatch` and `@singledispatchmethod` could be improved
Message-ID: <1636114507.95.0.911356270077.issue45726@roundup.psfhosted.org>


New submission from Alex Waygood <Alex.Waygood at Gmail.com>:

The documentation for `functools.singledispatch` and `functools.singledispatchmethod` contains a few small grammar mistakes and instances of slightly inelegant writing. PR to follow shortly.

----------
assignee: docs at python
components: Documentation
messages: 405788
nosy: AlexWaygood, docs at python, lukasz.langa, rhettinger
priority: normal
severity: normal
status: open
title: Documentation for `@singledispatch` and `@singledispatchmethod` could be improved
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45726>
_______________________________________

From report at bugs.python.org  Fri Nov  5 08:44:09 2021
From: report at bugs.python.org (Carl Friedrich Bolz-Tereick)
Date: Fri, 05 Nov 2021 12:44:09 +0000
Subject: [New-bugs-announce] [issue45727] Parse error when missing commas is
 inconsistent
Message-ID: <1636116249.02.0.999141328872.issue45727@roundup.psfhosted.org>


New submission from Carl Friedrich Bolz-Tereick <cfbolz at gmx.de>:

I found following inconsistency in the error message when there's a missing comma (it behaves that way both on main and 3.10).

Here's what happens with numbers, as expected:

Python 3.11.0a1+ (heads/main:32f55d1a5d, Nov  5 2021, 13:18:52) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 1 2 3 4
  File "<stdin>", line 1
    1 2 3 4
    ^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?

But with names the error is further right in the lines:

>>> a b c d
  File "<stdin>", line 1
    a b c d
      ^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?
>>> a b c d e f g
  File "<stdin>", line 1
    a b c d e f g
              ^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?

That looks potentially quite confusing to me?

(I don't know if these nit-picky parsing issues are too annoying, if they are please tell me to stop filing them).

----------
messages: 405792
nosy: Carl.Friedrich.Bolz, pablogsal
priority: normal
severity: normal
status: open
title: Parse error when missing commas is inconsistent
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45727>
_______________________________________

From report at bugs.python.org  Fri Nov  5 09:46:16 2021
From: report at bugs.python.org (jkrupp)
Date: Fri, 05 Nov 2021 13:46:16 +0000
Subject: [New-bugs-announce] [issue45728] SharedMemory documentation: System
 V vs Posix
Message-ID: <1636119976.18.0.141514451368.issue45728@roundup.psfhosted.org>


New submission from jkrupp <python_cee2df at jkrupp.de>:

Hi,

the documentation for the shared_memory module states that "shared memory refers to ?System V style? shared memory". However, I believe it might be clearer to talk about "POSIX style shared memory" here instead.

Primed by the "System V style"-comment, I had expected that shared memory blocks could be identified through a numeric ID (as used by the system V shared memory APIs `shmget`, `shmat`, ...). After looking at the implementation it became clear though, that the newer POSIX-API (`shm_open`, `shm_unlink`, ...) is used, which explains why shared memory blocks have a *name* instead.

Technically, the documentation only uses "System V style" to distinguish it from "distributed shared memory", and further states that "[it] is not necessarily implemented explicitly as such". Yet, this distinction is particularly relevant if memory is to be shared with other (non-python!) processes (although placement of shared_memory in the multiprocessing package might indicate that that is not one of its expected use-cases...)

----------
assignee: docs at python
components: Documentation
messages: 405797
nosy: docs at python, jkrupp
priority: normal
severity: normal
status: open
title: SharedMemory documentation: System V vs Posix
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45728>
_______________________________________

From report at bugs.python.org  Fri Nov  5 12:06:13 2021
From: report at bugs.python.org (Jacob Lifshay)
Date: Fri, 05 Nov 2021 16:06:13 +0000
Subject: [New-bugs-announce] [issue45729] "history and license" link has
 wrong target
Message-ID: <1636128373.81.0.436284013167.issue45729@roundup.psfhosted.org>


New submission from Jacob Lifshay <programmerjake at gmail.com>:

https://docs.python.org/3.10/library/csv.html
at the bottom the "history and license" link just links back to csv.html, rather than the correct target.

----------
assignee: docs at python
components: Documentation
messages: 405807
nosy: docs at python, programmerjake
priority: normal
severity: normal
status: open
title: "history and license" link has wrong target
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45729>
_______________________________________

From report at bugs.python.org  Fri Nov  5 15:09:52 2021
From: report at bugs.python.org (Joef Huang)
Date: Fri, 05 Nov 2021 19:09:52 +0000
Subject: [New-bugs-announce] [issue45730] ERROR:
 PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
 is not a supported wheel on this platform
Message-ID: <1636139392.76.0.708373821493.issue45730@roundup.psfhosted.org>


New submission from Joef Huang <joefh at ca.ibm.com>:

We have been using this command to install Python to our container:
RUN pip3 install -U pip && \
    pip install wheel && \
    pip wheel -w /tmp/wheels 

It started failing today with the following error:

ERROR: PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl is not a supported wheel on this platform.

It turned out the install now downloads the cp39 wheel: 

12:46:14  Collecting pyyaml>=3.12
12:46:14    Downloading PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)


When this worked a couple of days ago, the install downloaded a different wheel:

15:03:12  Collecting pyyaml>=3.12
15:03:12    Downloading PyYAML-6.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (603 kB)


Why does the python install download the cp39 wheel now?

----------
components: Installation
messages: 405821
nosy: joefh
priority: normal
severity: normal
status: open
title: ERROR: PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl is not a supported wheel on this platform
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45730>
_______________________________________

From report at bugs.python.org  Fri Nov  5 15:55:46 2021
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 05 Nov 2021 19:55:46 +0000
Subject: [New-bugs-announce] [issue45731] Handle
 --enable-loadable-sqlite-extensions in configure
Message-ID: <1636142146.47.0.904831540314.issue45731@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

The --enable-loadable-sqlite-extensions configure option is currently handled by a check in setup.py. The approach is incompatible with Modules/Setup and yields incorrect results for --enable-loadable-sqlite-extensions=no. Instead of reading the value from CONFIG_ARGS, configure should add an AC_DEFINE to pyconfig.h

----------
assignee: christian.heimes
components: Build
messages: 405826
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Handle --enable-loadable-sqlite-extensions in configure
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45731>
_______________________________________

From report at bugs.python.org  Fri Nov  5 19:15:21 2021
From: report at bugs.python.org (Ned Deily)
Date: Fri, 05 Nov 2021 23:15:21 +0000
Subject: [New-bugs-announce] [issue45732] Update python.org Windows and
 macOS installers to use Tk 8.6.12
Message-ID: <1636154121.05.0.278439369914.issue45732@roundup.psfhosted.org>


New submission from Ned Deily <nad at python.org>:

https://sourceforge.net/p/tcl/mailman/message/37380142/

----------
components: Build, Windows, macOS
messages: 405834
nosy: ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Update python.org Windows and macOS installers to use Tk 8.6.12
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45732>
_______________________________________

From report at bugs.python.org  Fri Nov  5 21:23:27 2021
From: report at bugs.python.org (John-Mark Gurney)
Date: Sat, 06 Nov 2021 01:23:27 +0000
Subject: [New-bugs-announce] [issue45733] importlib.abc.Traversable.name
 does not match
Message-ID: <1636161807.52.0.0427102935668.issue45733@roundup.psfhosted.org>


New submission from John-Mark Gurney <jmg at funkthat.com>:

The documentation for Traversable.name says it is a method, not a property:
https://docs.python.org/3/library/importlib.html#importlib.abc.Traversable.name

The issue is that with Python 3.9.7 (default, Nov  1 2021, 11:26:33), using a standard posix module (on MacOSX), it returns a string and not a method.  I have not tested on newer versions, but expect similar issues.  I have tagged the newer versions to get exposure, feel free to untag the versions that are confirmed to be unaffected by this bug.

```
$ python3.9
Python 3.9.7 (default, Nov  1 2021, 11:26:33) 
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import mod
<class 'str'>
>>> ^D
```

This is with a module dir named `mod`, w/ an empty `res.txt` file, and the following in `__init__.py`:
```
from importlib import resources

r = resources.files(__name__) / 'res.txt'

print(repr(type(r.name)))
```

----------
messages: 405837
nosy: jmg
priority: normal
severity: normal
status: open
title: importlib.abc.Traversable.name does not match
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45733>
_______________________________________

From report at bugs.python.org  Fri Nov  5 23:25:02 2021
From: report at bugs.python.org (Skillovilla Official)
Date: Sat, 06 Nov 2021 03:25:02 +0000
Subject: [New-bugs-announce] [issue45734] Data Science and Machine Learning
 Course
Message-ID: <1636169102.83.0.727198950247.issue45734@roundup.psfhosted.org>


New submission from Skillovilla Official <skillovillaofficial at gmail.com>:

best data science courses , best data science courses online , data science beginners , data science certification , data science certification course, data science course fees , data science course with placement guarantee , data science full course, data science online classes, data science online course, data scientist course, data scientist course fees , 
https://www.skillovilla.com/tracks/data-science-and-machine-learning

----------
components: Build
files: logo-thumbnail.png
messages: 405844
nosy: skillovillaofficial
priority: normal
severity: normal
status: open
title: Data Science and Machine Learning Course
type: performance
versions: Python 3.9
Added file: https://bugs.python.org/file50427/logo-thumbnail.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45734>
_______________________________________

From report at bugs.python.org  Sat Nov  6 00:02:01 2021
From: report at bugs.python.org (Tim Peters)
Date: Sat, 06 Nov 2021 04:02:01 +0000
Subject: [New-bugs-announce] [issue45735] Promise that the long-time truth
 that `args=list` works
Message-ID: <1636171321.93.0.670075505099.issue45735@roundup.psfhosted.org>


New submission from Tim Peters <tim at python.org>:

A number of contexts allow specifying a tuple of arguments to be passed later to a function. The Thread constructor is a fine example, and happened to come up (again! for me) here today:

https://stackoverflow.com/questions/69858950/why-do-we-have-to-add-comma-in-args-in-python-multithreading/69859068

This often confuses especially newbies, because the function they intend to parallelize often takes only a single argument, and Python's syntax for a 1-element tuple actually _requires_ parentheses in the context of an argument list, with a naked trailing comma:

t = threading.Thread(target=access, args=(thread_number,))

It "looks weird" to people.

I'm not suggesting to change that, but instead to officially bless the workaround I've seen very often in real code: use a list instead.

t = threading.Thread(target=access, args=[thread_number])

Nobody scratches their head over what that means.

CPython's implementations typically couldn't care less what kind of sequence is used, and none that I'm aware of verify that it's specifically a tuple. The implementations just go on to do some simple variation of

    self.target(*self.args)

Tuple or list makes no real difference. I'm not really keen to immortalize the "any sequence type whatsoever that just happens to work" implementation behavior, but am keen to promise that a list specifically will work. A lot of code already relies on it.

----------
assignee: docs at python
components: Documentation
keywords: easy
messages: 405846
nosy: docs at python, tim.peters
priority: low
severity: normal
stage: needs patch
status: open
title: Promise that the long-time truth that `args=list` works
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45735>
_______________________________________

From report at bugs.python.org  Sat Nov  6 07:50:26 2021
From: report at bugs.python.org (theeshallnotknowethme)
Date: Sat, 06 Nov 2021 11:50:26 +0000
Subject: [New-bugs-announce] [issue45736] 2to3 does not support integer
 division fixing
Message-ID: <1636199426.55.0.893331092894.issue45736@roundup.psfhosted.org>


New submission from theeshallnotknowethme <nohackingofkrowten at gmail.com>:

Right now, 2to3 does not support integer division fixing. Supposing `test.py` is a file with these contents:
x = 2 ** 8 / 5 / 7

Here's an example:
C:\Users\admin> py -m lib2to3 test.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: No files need to be modified.

----------
components: 2to3 (2.x to 3.x conversion tool), Library (Lib)
messages: 405857
nosy: February291948
priority: normal
severity: normal
status: open
title: 2to3 does not support integer division fixing
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45736>
_______________________________________

From report at bugs.python.org  Sat Nov  6 08:08:52 2021
From: report at bugs.python.org (Daniel)
Date: Sat, 06 Nov 2021 12:08:52 +0000
Subject: [New-bugs-announce] [issue45737] assertLogs to optionally not
 disable existing handlers
Message-ID: <1636200532.3.0.526903352151.issue45737@roundup.psfhosted.org>


New submission from Daniel <3daniel at hotmail.com>:

At the moment, assertLogs removes the handlers attached to the logger.
In general this is good, because it reduces message spamming in the test logs.
However, if the code being tested is relying on a handler to do something, then the test fails because the handler is being removed.
This leads to the situation that the same exact test must be run twice:
- first time within the context manager, to assert that specific messages were logged (using `with self.assertLogs()`)
- second time, without the assertLogs to ensure the code that uses a handler does the right thing

The proposal is to have `self.assertLogs()` accept a key word argument such as `keep_handlers=False`, which can be set to True, whenever the handlers should be preserved.
It would probably be also useful to add a note in the documentation that makes users aware that the existing handlers will be removed.

----------
messages: 405858
nosy: dandiez
priority: normal
severity: normal
status: open
title: assertLogs to optionally not disable existing handlers
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45737>
_______________________________________

From report at bugs.python.org  Sat Nov  6 09:10:52 2021
From: report at bugs.python.org (PEW's Corner)
Date: Sat, 06 Nov 2021 13:10:52 +0000
Subject: [New-bugs-announce] [issue45738] 3.11 exec raises SystemError
 instead of SyntaxError on char after line continuation
Message-ID: <1636204252.01.0.178939354477.issue45738@roundup.psfhosted.org>


New submission from PEW's Corner <pewscorner at gmail.com>:

When exec'ing code with an illegal character after the line continuation character '\', python 3.11.0a2 raises this strange exception:

  SystemError: Negative size passed to PyUnicode_New

Here's an example where '#' is the illegal character:

  exec('1,\\#\n2')

I expected the following exception (which is raised by Python 3.10.0, and also by Python 3.11.0a2 when using eval instead of exec, or when the string content is put into its own file and run as a normal script):

  SyntaxError: unexpected character after line continuation character

----------
messages: 405860
nosy: pewscorner
priority: normal
severity: normal
status: open
title: 3.11 exec raises SystemError instead of SyntaxError on char after line continuation
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45738>
_______________________________________

From report at bugs.python.org  Sat Nov  6 09:51:18 2021
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 06 Nov 2021 13:51:18 +0000
Subject: [New-bugs-announce] [issue45739] The Python implementation of
 Decimal does not support the "N" format
Message-ID: <1636206678.41.0.252821341068.issue45739@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The C implementation supports both formats "n" and "N". The Python implementation only supports format "n".

>>> from decimal import Decimal
>>> format(Decimal('1e100'), 'n')
'1e+100'
>>> format(Decimal('1e100'), 'N')
'1E+100'
>>> from _pydecimal import Decimal
>>> format(Decimal('1e100'), 'n')
'1e+100'
>>> format(Decimal('1e100'), 'N')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/_pydecimal.py", line 3766, in __format__
    spec = _parse_format_specifier(specifier, _localeconv=_localeconv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/_pydecimal.py", line 6194, in _parse_format_specifier
    raise ValueError("Invalid format specifier: " + format_spec)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Invalid format specifier: N

----------
components: Library (Lib)
messages: 405861
nosy: facundobatista, mark.dickinson, rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: The Python implementation of Decimal does not support the "N" format
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45739>
_______________________________________

From report at bugs.python.org  Sat Nov  6 11:02:00 2021
From: report at bugs.python.org (Dutcho)
Date: Sat, 06 Nov 2021 15:02:00 +0000
Subject: [New-bugs-announce] [issue45740] StrEnum entry doesn't mention its
 introduction in 3.11
Message-ID: <1636210920.1.0.557884718262.issue45740@roundup.psfhosted.org>


New submission from Dutcho <dutcho at ziggo.nl>:

At https://docs.python.org/3.11/library/enum.html#enum.StrEnum no mention is made of 'New in version 3.11', whereas e.g. https://docs.python.org/3.11/library/enum.html#enum.EnumCheck does.

Note https://docs.python.org/3.11/library/enum.html#module-contents DOES document: 'New in version 3.11: StrEnum, EnumCheck, FlagBoundary'.

----------
assignee: docs at python
components: Documentation
messages: 405863
nosy: Dutcho, docs at python
priority: normal
severity: normal
status: open
title: StrEnum entry doesn't mention its introduction in 3.11
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45740>
_______________________________________

From report at bugs.python.org  Sat Nov  6 19:05:18 2021
From: report at bugs.python.org (Rafael Fontenelle)
Date: Sat, 06 Nov 2021 23:05:18 +0000
Subject: [New-bugs-announce] [issue45741] entry points singular?
Message-ID: <1636239918.89.0.442271633024.issue45741@roundup.psfhosted.org>


New submission from Rafael Fontenelle <rffontenelle at gmail.com>:

whatsnew/3.10 on importlib.metadata has the following paragraph:

"importlib.metadata entry points now provides a nicer experience for selecting entry points by group and name through a new importlib.metadata.EntryPoints class. See the Compatibility Note in the docs for more info on the deprecation and usage."

In the above paragraph should "provides" -> "provide", or "entry points" is somehow considered singular?

[1] https://docs.python.org/pt-br/3/whatsnew/3.10.html#importlib-metadata

----------
assignee: docs at python
components: Documentation
messages: 405888
nosy: docs at python, rffontenelle
priority: normal
severity: normal
status: open
title: entry points singular?
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45741>
_______________________________________

From report at bugs.python.org  Sat Nov  6 23:10:01 2021
From: report at bugs.python.org (jiahua wang)
Date: Sun, 07 Nov 2021 03:10:01 +0000
Subject: [New-bugs-announce] [issue45742] python -h can't  find -R option
Message-ID: <1636254601.16.0.481358926268.issue45742@roundup.psfhosted.org>


New submission from jiahua wang <wjh180909 at gmail.com>:

I input python -h on the command line, and I can't find any -R option.

----------
assignee: docs at python
components: Documentation
messages: 405891
nosy: 180909, docs at python
priority: normal
severity: normal
status: open
title: python -h can't  find -R option
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45742>
_______________________________________

From report at bugs.python.org  Sun Nov  7 11:37:24 2021
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 07 Nov 2021 16:37:24 +0000
Subject: [New-bugs-announce] [issue45743] Cleanup and simplify setup.py
Message-ID: <1636303044.3.0.982065996937.issue45743@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

Motivated by deprecation of distutils, I like to move more logic and checks from setup.py into configure.ac. Eventually I like to get rid of setup.py. The file contains a bunch of complicated checks and macOS-specific adjustments that I cannot verify on Linux.


1) socketmodule setup defines __APPLE_USE_RFC_3542 https://github.com/python/cpython/blob/v3.11.0a2/setup.py#L1229 Can we move the define into the __APPLE__ block of socketmodule.c?


2) -Wl,-search_paths_first linker arg, e.g. https://github.com/python/cpython/blob/v3.11.0a2/setup.py#L1616  Would it be safe to make the option default for all core extensions on $ac_sys_system = Darwin? We could add it to PY_CORE_LDFLAGS or add a new Makefile variable for core extensions.


3) detect_dbm_gdbm has about 200 lines of complicated code to detect old to ancient versions of Berkeley DB (libdb), https://github.com/python/cpython/blob/v3.11.0a2/setup.py#L1233 . Can I remove support for libdb-3 / libdb-4 and only support libdb-5 in standard locations? libdb-5.3 has been around for over a decade.

Note: libdb, gdbm, ndbm, and gdbm-compat don't provide pkg-config .pc files. We cannot use pkg-config to detect them.


4) sqlite's setup https://github.com/python/cpython/blob/v3.11.0a2/setup.py#L1531 does extra work to search for header and library files in non-standard locations. Can we replace the code with pkg-config checks in configure.ac with a fallback to AC_CHECK_LIB() and AC_CHECK_HEADERS()?


5) zlib's setup code https://github.com/python/cpython/blob/v3.11.0a2/setup.py#L1661 has a check for a CVE from 2002 (!). I think we can safely assume that everybody has upgraded to a fixed version. The check is mixed with macOS specific code.

----------
components: Build, macOS
messages: 405908
nosy: christian.heimes, gregory.p.smith, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Cleanup and simplify setup.py
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45743>
_______________________________________

From report at bugs.python.org  Sun Nov  7 14:31:22 2021
From: report at bugs.python.org (Abdur-rahmaan Janhangeer)
Date: Sun, 07 Nov 2021 19:31:22 +0000
Subject: [New-bugs-announce] [issue45744] Fix Flawfinder C Errors
Message-ID: <1636313482.78.0.504082429267.issue45744@roundup.psfhosted.org>


New submission from Abdur-rahmaan Janhangeer <arj.python at gmail.com>:

Greetings all,

I was doing a security audit using https://dwheeler.com/flawfinder/

There are quite some issues.

Before i PR, i am opening an issue as per the dev guide.

Await further comments. Thanks!

----------
messages: 405910
nosy: appinv
priority: normal
severity: normal
status: open
title: Fix Flawfinder C Errors
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45744>
_______________________________________

From report at bugs.python.org  Sun Nov  7 16:39:17 2021
From: report at bugs.python.org (Skip Montanaro)
Date: Sun, 07 Nov 2021 21:39:17 +0000
Subject: [New-bugs-announce] [issue45745] ./python -m test --help output for
 refleaks seems wrong
Message-ID: <1636321157.49.0.174239035992.issue45745@roundup.psfhosted.org>


New submission from Skip Montanaro <skip.montanaro at gmail.com>:

Just preparing to make a refleaks test run, so I ran:

./python -m test --help

The output related to refleaks seemed suspicious:

...
Special runs:
  -l, --findleaks       deprecated alias to --fail-env-changed
...
  --fail-env-changed    if a test file alters the environment, mark the test as failed

It's not obvious that "--fail-env-changed" has anything to do with "--findleaks". Does the -l/--findleaks message need to be corrected?

----------
components: Build
messages: 405916
nosy: skip.montanaro
priority: normal
severity: normal
status: open
title: ./python -m test --help output for refleaks seems wrong
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45745>
_______________________________________

From report at bugs.python.org  Sun Nov  7 20:49:30 2021
From: report at bugs.python.org (Matthew H. McKenzie)
Date: Mon, 08 Nov 2021 01:49:30 +0000
Subject: [New-bugs-announce] [issue45746] ftplib please revisit
 retrlines('RETR as it produces files without newlines
Message-ID: <1636336170.65.0.433543407195.issue45746@roundup.psfhosted.org>


New submission from Matthew H. McKenzie <github1 at swellhunter.org>:

Lib/ftplib.py  function retrlines

Inspired by documentation the following writes a file without line-endings:
 
from ftplib import FTP
ftp=FTP()
ftp.connect('hostname')
ftp.login('user','xxxxxxxx')
ftp.sendcmd('pasv')

with open('crap2.txt', 'w') as fp:
    ftp.retrlines('RETR crap.txt', fp.write)

Code goes to pains to slice off the line endings, and then print_line does not add them back? Apologies if this has been covered before, or I am not following the documentation correctly. Not going to suggest a fix as there may be a reason it is like this.

For RETR.
For ascii

----------
components: Library (Lib)
files: crap2.txt
messages: 405921
nosy: mckenzm
priority: normal
severity: normal
status: open
title: ftplib please revisit retrlines('RETR  as it produces files without newlines
versions: Python 3.11
Added file: https://bugs.python.org/file50429/crap2.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45746>
_______________________________________

From report at bugs.python.org  Mon Nov  8 05:27:28 2021
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 08 Nov 2021 10:27:28 +0000
Subject: [New-bugs-announce] [issue45747] Detect dbm and gdbm dependencies
 in configure.ac
Message-ID: <1636367248.6.0.316133977839.issue45747@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

setup.py jumps through hoops to detect gdbm, gdbm_compat, ndbm, and bdb (libdb) build dependencies. I propose to simplify our support matrix and detect all dependencies in configure.ac.

gdbmmodule.c uses gdbm_open() API, which is provided by gdbm.h and libgdbm

dbmmodule.c uses the dbm_open() API, which can be provided by a combination of headers and shared libraries.

- gdbm/ndbm.h with libgdbm_compat
- gdbm-ndbm.h with libgdbm_compat
- ndbm.h with libgdbm_compat
- ndbm.h with libndbm
- db.h with libdb (lib-5.3 for over a decade)

setup.py makes some wrong assumption in regards of libgdbm. The shared library libgdbm does not provide the dbm_open() API on any system I tested. libgdbm_compat is always required. Linking with libgdbm is not needed. libgdbm_compat depends on and wraps libgdbm.

setup.py also performs additional steps to locate old versions of libdb-3.x and libdb-4.x. libdb-5.3 has been around for a decade. Even CentOS 7 and Debian oldoldstable ship with libdb version 5.3 and don't offer libdb version 4. I propose to remove support for libdb 4 and libdb 3.

----------
assignee: christian.heimes
components: Build
messages: 405940
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Detect dbm and gdbm dependencies in configure.ac
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45747>
_______________________________________

From report at bugs.python.org  Mon Nov  8 08:54:44 2021
From: report at bugs.python.org (Thomas Trummer)
Date: Mon, 08 Nov 2021 13:54:44 +0000
Subject: [New-bugs-announce] [issue45748] "import ctypes" segfaults on
 Python 3.6 and Ubuntu 21.10
Message-ID: <1636379684.65.0.582073248809.issue45748@roundup.psfhosted.org>


New submission from Thomas Trummer <th.trummer at gmail.com>:

3.7 and later are all working fine.

First good commit: 55fe1ae9708d81b902b6fe8f6590e2a24b1bd4b0
First bad commit: fdbd01151dbd5feea3e4c0316d102db3d2a2a412


git checkout v3.6.15

#0  0x00007ffff6cc52a0 in PyCFuncPtr_new (type=0x5555559157f8, args=0x7ffff6ce6dd8, kwds=0x0)

    at /home/tom/pydev/cpython/Modules/_ctypes/_ctypes.c:3557

#1  0x00005555556400f9 in type_call (type=0x5555559157f8, args=0x7ffff6ce6dd8, kwds=0x0) at Objects/typeobject.c:895

#2  0x00005555555db4ca in _PyObject_FastCallDict (func=0x5555559157f8, args=0x7ffff6dc1f48, nargs=1, kwargs=0x0) at Objects/abstract.c:2331

#3  0x00005555556b8e7c in call_function (pp_stack=pp_stack at entry=0x7fffffffcb58, oparg=<optimized out>, kwnames=kwnames at entry=0x0)

    at Python/ceval.c:4875

#4  0x00005555556bc9d3 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3335

#5  0x00005555556b774c in PyEval_EvalFrameEx (throwflag=0, f=0x7ffff6dc1dc8) at Python/ceval.c:754

#6  _PyFunction_FastCall (co=<optimized out>, args=<optimized out>, nargs=nargs at entry=0, globals=<optimized out>) at Python/ceval.c:4933

#7  0x00005555556b9269 in fast_function (kwnames=0x0, nargs=0, stack=<optimized out>, func=0x7ffff6d78378) at Python/ceval.c:4968

#8  call_function (pp_stack=pp_stack at entry=0x7fffffffccf8, oparg=<optimized out>, kwnames=kwnames at entry=0x0) at Python/ceval.c:4872

#9  0x00005555556bc9d3 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3335

#10 0x00005555556b83c7 in PyEval_EvalFrameEx (throwflag=0, f=0x5555558d2248) at Python/ceval.c:754

#11 _PyEval_EvalCodeWithName (_co=_co at entry=0x7ffff6cdf660, globals=globals at entry=0x7ffff6dfa438, locals=locals at entry=0x7ffff6dfa438, 

    args=args at entry=0x0, argcount=argcount at entry=0, kwnames=kwnames at entry=0x0, kwargs=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, 

    closure=0x0, name=0x0, qualname=0x0, kwstep=2) at Python/ceval.c:4166

#12 0x00005555556b9a25 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, 

    locals=0x7ffff6dfa438, globals=0x7ffff6dfa438, _co=0x7ffff6cdf660) at Python/ceval.c:4187

#13 PyEval_EvalCode (co=co at entry=0x7ffff6cdf660, globals=globals at entry=0x7ffff6dfa438, locals=locals at entry=0x7ffff6dfa438)

    at Python/ceval.c:731

#14 0x00005555556b611d in builtin_exec_impl (module=<optimized out>, locals=0x7ffff6dfa438, globals=0x7ffff6dfa438, source=0x7ffff6cdf660)

    at Python/bltinmodule.c:983

#15 builtin_exec (module=<optimized out>, args=<optimized out>) at Python/clinic/bltinmodule.c.h:283

#16 0x000055555562b651 in PyCFunction_Call (func=func at entry=0x7ffff6eb7990, args=args at entry=0x7ffff6cdcd08, kwds=kwds at entry=0x7ffff6d6b288)

    at Objects/methodobject.c:126

#17 0x00005555556c11af in do_call_core (kwdict=0x7ffff6d6b288, callargs=0x7ffff6cdcd08, func=0x7ffff6eb7990) at Python/ceval.c:5116

#18 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3404

#19 0x00005555556b8d17 in PyEval_EvalFrameEx (throwflag=0, f=0x7ffff6dc2930) at Python/ceval.c:754

#20 _PyEval_EvalCodeWithName (_co=0x7ffff6eabdb0, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=3, 

    kwnames=0x0, kwargs=0x7ffff6d913c8, kwcount=0, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7ffff6e53ad0, 

    qualname=0x7ffff6e53ad0) at Python/ceval.c:4166

----------
components: ctypes
messages: 405950
nosy: Thomas Trummer
priority: normal
severity: normal
status: open
title: "import ctypes" segfaults on Python 3.6 and Ubuntu 21.10
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45748>
_______________________________________

From report at bugs.python.org  Mon Nov  8 10:43:49 2021
From: report at bugs.python.org (Ross Burton)
Date: Mon, 08 Nov 2021 15:43:49 +0000
Subject: [New-bugs-announce] [issue45749] Silently fails to build hashlib if
 openssl has disabled algorithms
Message-ID: <1636386229.74.0.966794635117.issue45749@roundup.psfhosted.org>


New submission from Ross Burton <ross.burton at arm.com>:

If my openssl is built with no-scrypt then the Python build of hashlib fails (as EVP_PBE_scrypt isn't present), but the overall compile succeeds.

----------
components: Build
messages: 405954
nosy: rossburton2
priority: normal
severity: normal
status: open
title: Silently fails to build hashlib if openssl has disabled algorithms

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45749>
_______________________________________

From report at bugs.python.org  Mon Nov  8 11:46:42 2021
From: report at bugs.python.org (Jose Ville)
Date: Mon, 08 Nov 2021 16:46:42 +0000
Subject: [New-bugs-announce] [issue45750] "SyntaxError: 'await' outside
 function" in "asyncio-task.html#waiting-primitives" code snippets
Message-ID: <1636390002.55.0.0752334780562.issue45750@roundup.psfhosted.org>


New submission from Jose Ville <joseville>:

https://docs.python.org/3/library/asyncio-task.html#asyncio.wait has the following two code snippets both of which fail with ""SyntaxError: 'await' outside function" when I run them in Python 3.9.7

Snippet 1:

```
async def foo():
    return 42

coro = foo()
done, pending = await asyncio.wait({coro})

if coro in done:
    # This branch will never be run!
    pass # I added this to prevent IndentationError
```

Snippet 2:

```
async def foo():
    return 42

task = asyncio.create_task(foo())
done, pending = await asyncio.wait({task})

if task in done:
    # Everything will work as expected now.
    pass # I added this to prevent IndentationError
```

----------
components: asyncio
messages: 405958
nosy: asvetlov, joseville, yselivanov
priority: normal
severity: normal
status: open
title: "SyntaxError: 'await' outside function" in "asyncio-task.html#waiting-primitives" code snippets
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45750>
_______________________________________

From report at bugs.python.org  Mon Nov  8 11:47:59 2021
From: report at bugs.python.org (Jose Ville)
Date: Mon, 08 Nov 2021 16:47:59 +0000
Subject: [New-bugs-announce] [issue45751] "SyntaxError: 'await' outside
 function" in "asyncio-task.html#waiting-primitives" code snippets
Message-ID: <1636390079.55.0.533458820823.issue45751@roundup.psfhosted.org>


New submission from Jose Ville <joseville>:

https://docs.python.org/3/library/asyncio-task.html#asyncio.wait has the following two code snippets both of which fail with ""SyntaxError: 'await' outside function" when I run them in Python 3.9.7

Snippet 1:

```
async def foo():
    return 42

coro = foo()
done, pending = await asyncio.wait({coro})

if coro in done:
    # This branch will never be run!
    pass # I added this to prevent IndentationError
```

Snippet 2:

```
async def foo():
    return 42

task = asyncio.create_task(foo())
done, pending = await asyncio.wait({task})

if task in done:
    # Everything will work as expected now.
    pass # I added this to prevent IndentationError
```

----------
components: asyncio
messages: 405960
nosy: asvetlov, joseville, yselivanov
priority: normal
severity: normal
status: open
title: "SyntaxError: 'await' outside function" in "asyncio-task.html#waiting-primitives" code snippets
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45751>
_______________________________________

From report at bugs.python.org  Mon Nov  8 11:53:12 2021
From: report at bugs.python.org (Stefan Pochmann)
Date: Mon, 08 Nov 2021 16:53:12 +0000
Subject: [New-bugs-announce] [issue45752] copy module doc wrongly says it
 doesn't copy arrays
Message-ID: <1636390392.43.0.38708438512.issue45752@roundup.psfhosted.org>


New submission from Stefan Pochmann <stefan.pochmann at gmail.com>:

The doc https://docs.python.org/3/library/copy.html says:

"This module does not copy types like module, method, stack trace, stack frame, file, socket, window, array, or any similar types."

But it does copy arrays just fine:

import copy, array
a = array.array('i', [1, 2])
b = copy.copy(a)
a[0] = 3
print(a)
print(b)

Output:

array('i', [3, 2])
array('i', [1, 2])

----------
assignee: docs at python
components: Documentation
messages: 405962
nosy: Stefan Pochmann, docs at python
priority: normal
severity: normal
status: open
title: copy module doc wrongly says it doesn't copy arrays
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45752>
_______________________________________

From report at bugs.python.org  Mon Nov  8 12:17:55 2021
From: report at bugs.python.org (Mark Shannon)
Date: Mon, 08 Nov 2021 17:17:55 +0000
Subject: [New-bugs-announce] [issue45753] Further speed up Python-to-Python
 calls.
Message-ID: <1636391875.36.0.891525405866.issue45753@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

There are three things we want to do:

1. Speed up pushing and popping frames. See https://github.com/faster-cpython/ideas/issues/111 for details.
2. Avoid tracing and other admin overhead on entering and leaving. See https://github.com/faster-cpython/ideas/issues/112.
3. Keep the remaining recursion depth in the cframe, to reduce the work for a recursion check from
`++tstate->recursion_depth > tstate->interp->ceval.recursion_limit`
to
`cframe.recursion_overhead-- > 0`

----------
assignee: Mark.Shannon
components: Interpreter Core
messages: 405970
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Further speed up Python-to-Python calls.
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45753>
_______________________________________

From report at bugs.python.org  Mon Nov  8 12:40:00 2021
From: report at bugs.python.org (Erlend E. Aasland)
Date: Mon, 08 Nov 2021 17:40:00 +0000
Subject: [New-bugs-announce] [issue45754] [sqlite3] SQLITE_LIMIT_LENGTH is
 incorrectly used to check statement length
Message-ID: <1636393200.96.0.803607204185.issue45754@roundup.psfhosted.org>


New submission from Erlend E. Aasland <erlend.aasland at innova.no>:

In Modules/_sqlite/statement.c pysqlite_statement_create() and Modules/_sqlite/cursor.c pysqlite_cursor_executescript_impl(), we incorrectly use SQLITE_LIMIT_LENGTH to check statement length. However, the correct limit is *SQLITE_LIMIT_SQL_LENGTH*.

### Alternative 1:
Quick fix is to check against SQLITE_LIMIT_SQL_LENGTH instead of SQLITE_LIMIT_LENGTH.

### Alternative 2:
Let SQLite do the check for us, and instead add integer overflow check, since Py_ssize_t may be larger than int (sqlite3_prepare_v2() uses an int as the max statement length parameter).

### Alternative 3:
As alternative 2, but alter the sqlite3_prepare_v2() call to accept _any_ length (max statement length = -1).


See also:
- https://sqlite.org/limits.html
- https://sqlite.org/c3ref/c_limit_attached.html
- https://sqlite.org/c3ref/prepare.html

----------
messages: 405975
nosy: erlendaasland, serhiy.storchaka
priority: normal
severity: normal
status: open
title: [sqlite3] SQLITE_LIMIT_LENGTH is incorrectly used to check statement length

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45754>
_______________________________________

From report at bugs.python.org  Mon Nov  8 14:30:49 2021
From: report at bugs.python.org (Kevin Jamieson)
Date: Mon, 08 Nov 2021 19:30:49 +0000
Subject: [New-bugs-announce] [issue45755] Mock spec with a specialized
 generic class does not mock class attributes
Message-ID: <1636399849.8.0.841580145929.issue45755@roundup.psfhosted.org>


New submission from Kevin Jamieson <kjamieson at qumulo.com>:

This worked in Python 3.6, but in Python 3.7 and later creating a mock with a spec specifying a subscripted generic class does not mock any of the attributes of the class, because those attributes are not returned by dir().

For example:

# cat test.py
from typing import Generic, TypeVar
from unittest import mock

T = TypeVar('T')

class Foo(Generic[T]):
    def bar(self) -> None:
        pass

m = mock.MagicMock(spec=Foo[int])
m.bar()


# python3.11 test.py
Traceback (most recent call last):
  File "/root/test.py", line 11, in <module>
    m.bar()
    ^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 635, in __getattr__
    raise AttributeError("Mock object has no attribute %r" % name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: Mock object has no attribute 'bar'

----------
components: Library (Lib)
messages: 405981
nosy: kjamieson
priority: normal
severity: normal
status: open
title: Mock spec with a specialized generic class does not mock class attributes
type: behavior
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45755>
_______________________________________

From report at bugs.python.org  Mon Nov  8 14:47:53 2021
From: report at bugs.python.org (Kevin Jamieson)
Date: Mon, 08 Nov 2021 19:47:53 +0000
Subject: [New-bugs-announce] [issue45756] mock raises exception when using a
 spec with an attribute that raises exception on access
Message-ID: <1636400873.51.0.88200985051.issue45756@roundup.psfhosted.org>


New submission from Kevin Jamieson <kjamieson at qumulo.com>:

In Python 3.8 and later creating a mock with a spec specifying an object containing a property that happens to raise an exception when accessed will fail, because _mock_add_spec calls getattr() on every attribute of the spec. This did not happen in Python 3.6/3.7.

This is likely a fairly unusual scenario (and in the particular case where I encountered this I could just use a class instead of an instance for the spec), but it was surprising.

For example:

# cat test.py
from unittest import mock

class Foo:
    @property
    def bar(self) -> str:
        raise Exception('xxx')

m = mock.MagicMock(spec=Foo())

# python3.11 test.py
Traceback (most recent call last):
  File "/root/test.py", line 8, in <module>
    m = mock.MagicMock(spec=Foo())
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 2069, in __init__
    _safe_super(MagicMixin, self).__init__(*args, **kw)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 1087, in __init__
    _safe_super(CallableMixin, self).__init__(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 442, in __init__
    self._mock_add_spec(spec, spec_set, _spec_as_instance, _eat_self)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 497, in _mock_add_spec
    if iscoroutinefunction(getattr(spec, attr, None)):
                           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test.py", line 6, in bar
    raise Exception('xxx')
    ^^^^^^^^^^^^^^^^^^^^^^
Exception: xxx

----------
messages: 405982
nosy: kjamieson
priority: normal
severity: normal
status: open
title: mock raises exception when using a spec with an attribute that raises exception on access
type: behavior
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45756>
_______________________________________

From report at bugs.python.org  Mon Nov  8 17:12:53 2021
From: report at bugs.python.org (Rok Mandeljc)
Date: Mon, 08 Nov 2021 22:12:53 +0000
Subject: [New-bugs-announce] [issue45757] dis module incorrectly handles
 EXTENDED_ARG + NOP sequence
Message-ID: <1636409573.19.0.31373204469.issue45757@roundup.psfhosted.org>


New submission from Rok Mandeljc <rok.mandeljc at gmail.com>:

dis module incorrectly handles the instruction sequence where EXTENDED_ARG is followed by NOP, e.g.,:

EXTENDED_ARG 0x01
NOP
EXTENDED_ARG 0x01
LOAD_CONST 0x29

The above sequence loads the constant from index 0x0129, whereas dis attempts to load it from index 0x010129, resulting in "IndexError: tuple index out of range error" when attempting to iterate over instructions returned by dis.get_instructions().

Complete example:

# *** example.py begin ***
import types
import dis

# Create a test code object...
constants = [None] * (0x000129 + 1)
constants[0x000129] = "Hello world!"

code = types.CodeType(
    0,  # argcount
    0,  # posonlyargcount
    0,  # kwonlyargcount
    0,  # nlocals
    1,  # stacksize
    64,  # flags
    bytes([
        0x90, 0x01,  # EXTENDED_ARG 0x01
        0x09, 0xFF,  # NOP 0xFF
        0x90, 0x01,  # EXTENDED_ARG 0x01
        0x64, 0x29,  # LOAD_CONST 0x29
        0x53, 0x00,  # RETURN_VALUE 0x00
    ]),  # codestring=
    tuple(constants),  # constants
    (),  # names
    (),  # varnames
    '<no file>',  # filename
    'code',  # name
    1,  # firstlineno
    b''  # linetable
)

# ... and eval it to show that NOP resets EXTENDED_ARG
print("Output:", eval(code))

# Try to list all instructions via dis
print(list(dis.get_instructions(code)))

# *** example.py end ***

Running the example gives us:

Output: Hello world!
Traceback (most recent call last):
  File "/home/rok/example.py", line 35, in <module>
    print(list(dis.get_instructions(code)))
  File "/usr/lib64/python3.10/dis.py", line 338, in _get_instructions_bytes
    argval, argrepr = _get_const_info(arg, constants)
  File "/usr/lib64/python3.10/dis.py", line 292, in _get_const_info
    argval = const_list[const_index]
IndexError: tuple index out of range


To fix the problem on dis side, the extended_arg in dis._unpack_opargs should be reset to 0 when NOP (or perhaps any opcode without argument) is encountered. I.e., by adding "extended_arg = 0" here:
https://github.com/python/cpython/blob/91275207296c39e495fe118019a757c4ddefede8/Lib/dis.py#L525



The problematic behavior was reported by users of PyInstaller under python 3.10; it seems that python 3.10 generates bytecode involving EXTENDED_ARG + NOP for telegram.message.py:
https://raw.githubusercontent.com/python-telegram-bot/python-telegram-bot/v13.7/telegram/message.py

The original PyInstaller issue with preliminary analysis is here: https://github.com/pyinstaller/pyinstaller/issues/6301

----------
components: Library (Lib)
messages: 405985
nosy: rok.mandeljc
priority: normal
severity: normal
status: open
title: dis module incorrectly handles EXTENDED_ARG + NOP sequence
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45757>
_______________________________________

From report at bugs.python.org  Mon Nov  8 18:07:35 2021
From: report at bugs.python.org (Victor Milovanov)
Date: Mon, 08 Nov 2021 23:07:35 +0000
Subject: [New-bugs-announce] [issue45758] Crash on Py_DecRef'ing builtin
 object from previous run
Message-ID: <1636412855.25.0.884802984088.issue45758@roundup.psfhosted.org>


New submission from Victor Milovanov <lostfreeman at gmail.com>:

Trying to Py_DecRef owned reference to builtin "iter" crashes if the reference was alive when runtime was reinitialized.

Py_Initialize();
PyObject* builtins = PyEval_GetBuiltins();
PyObject* iter = PyDict_GetItemString(builtins, "iter");

Py_IncRef(iter);

Py_Finalize();

// ----- new run starts, iter should still be alive

Py_Initialize();

Py_DecRef(iter);

Py_Finalize(); // fails inside PyGC_Collect -> validate_list

Related on StackOverflow: https://stackoverflow.com/questions/69890182/is-it-safe-to-call-py-decref-on-an-object-created-before-the-last-py-initializ

----------
components: C API
messages: 405987
nosy: Victor Milovanov
priority: normal
severity: normal
status: open
title: Crash on Py_DecRef'ing builtin object from previous run
type: crash
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45758>
_______________________________________

From report at bugs.python.org  Mon Nov  8 18:51:38 2021
From: report at bugs.python.org (theeshallnotknowethme)
Date: Mon, 08 Nov 2021 23:51:38 +0000
Subject: [New-bugs-announce] [issue45759] `elif` inside `if` block is a
 `SyntaxError`
Message-ID: <1636415498.31.0.65214367286.issue45759@roundup.psfhosted.org>


New submission from theeshallnotknowethme <nohackingofkrowten at gmail.com>:

While fixing errors in a program, I encountered this:

if a == b + 2:
    c = sqrt(b) + a**2
    elif a == b + 3:
    ^^^^
SyntaxError: Invalid syntax

It should be giving an `IndentationError` or a better error message at least.

----------
components: Parser
messages: 405990
nosy: February291948, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: `elif` inside `if` block is a `SyntaxError`
type: behavior
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45759>
_______________________________________

From report at bugs.python.org  Mon Nov  8 19:49:15 2021
From: report at bugs.python.org (Brandt Bucher)
Date: Tue, 09 Nov 2021 00:49:15 +0000
Subject: [New-bugs-announce] [issue45760] Remove "PyNumber_InMatrixMultiply"
Message-ID: <1636418955.82.0.973177651555.issue45760@roundup.psfhosted.org>


New submission from Brandt Bucher <brandtbucher at gmail.com>:

Here's a weird one: for the last 8 years, we've defined a function called "PyNumber_InMatrixMultiply" in abstract.c. It's a pretty clear misspelling of "PyNumber_InPlaceMatrixMultiply", which is *also* defined in that file. There's no documentation or tests for it, which further suggests that its inclusion was a mistake.

Any new contributors interested in deleting that one line? :)

----------
components: C API
keywords: easy (C)
messages: 405992
nosy: brandtbucher
priority: low
severity: normal
status: open
title: Remove "PyNumber_InMatrixMultiply"
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45760>
_______________________________________

From report at bugs.python.org  Tue Nov  9 03:15:50 2021
From: report at bugs.python.org (Evan Greenup)
Date: Tue, 09 Nov 2021 08:15:50 +0000
Subject: [New-bugs-announce] [issue45761] recursive ProcessPoolExecutor
 invoke unable to return result
Message-ID: <1636445750.09.0.78640644256.issue45761@roundup.psfhosted.org>


New submission from Evan Greenup <evan_greenup at protonmail.com>:

Read the hello.py in attachment.

This simple shallow recursion will never return result.

----------
components: Library (Lib)
files: hello.py
messages: 406003
nosy: evan0greenup
priority: normal
severity: normal
status: open
title: recursive ProcessPoolExecutor invoke unable to return result
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file50431/hello.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45761>
_______________________________________

From report at bugs.python.org  Tue Nov  9 03:18:16 2021
From: report at bugs.python.org (Matteo Bertucci)
Date: Tue, 09 Nov 2021 08:18:16 +0000
Subject: [New-bugs-announce] [issue45762] Missing `list` symbols in the
 object inventory
Message-ID: <1636445896.9.0.441408433097.issue45762@roundup.psfhosted.org>


New submission from Matteo Bertucci <matteobertucci2004 at gmail.com>:

Sphinx generates a quite useful inventory file, listing all the different symbols available inside the documentation. You can find the docs.python.org inventory over at https://docs.python.org/objects.inv. The syntax of this file can be found here https://sphobjinv.readthedocs.io/en/latest/syntax.html.

We use it over at Python Discord to power up our `!docs` command. It allows us to look up symbols such as `asyncio.run`, have nicely formatted documentation and links to the web page. 

The problem is due to where the `list` method documentations are located, inside `/tutorial/datastructures` (https://docs.python.org/3/tutorial/datastructures.html), no symbol is exported for those, making commands such as `!docs list.append` fail, which is quite a bummer. It would be very nice to have access to those.

----------
assignee: docs at python
components: Documentation
messages: 406004
nosy: Akarys, docs at python
priority: normal
severity: normal
status: open
title: Missing `list` symbols in the object inventory
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45762>
_______________________________________

From report at bugs.python.org  Tue Nov  9 04:13:49 2021
From: report at bugs.python.org (Christian Heimes)
Date: Tue, 09 Nov 2021 09:13:49 +0000
Subject: [New-bugs-announce] [issue45763] Detect compression libraries in
 configure
Message-ID: <1636449229.48.0.616836518237.issue45763@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

Detect the presence of header files and development libraries for zlib, bz2, and lzma in configure.ac. Also see bpo-45747

----------
components: Build
messages: 406007
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Detect compression libraries in configure
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45763>
_______________________________________

From report at bugs.python.org  Tue Nov  9 06:15:43 2021
From: report at bugs.python.org (Carl Friedrich Bolz-Tereick)
Date: Tue, 09 Nov 2021 11:15:43 +0000
Subject: [New-bugs-announce] [issue45764] Parse error improvement forgetting
 ( after def
Message-ID: <1636456543.8.0.11859375898.issue45764@roundup.psfhosted.org>


New submission from Carl Friedrich Bolz-Tereick <cfbolz at gmx.de>:

Something I see beginners make occasionally when defining functions without arguments is this:

def f:
    ...

Right now it just gives an "invalid syntax", would be nice to get an "expected '('".

I will try to give this a go! Should be a matter of making the '(' token an expected one.

----------
messages: 406010
nosy: Carl.Friedrich.Bolz
priority: normal
severity: normal
status: open
title: Parse error improvement forgetting ( after def

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45764>
_______________________________________

From report at bugs.python.org  Tue Nov  9 08:54:51 2021
From: report at bugs.python.org (Jason R. Coombs)
Date: Tue, 09 Nov 2021 13:54:51 +0000
Subject: [New-bugs-announce] [issue45765] importlib.metadata fails to find
 distributions in empty path
Message-ID: <1636466091.67.0.318711437429.issue45765@roundup.psfhosted.org>


New submission from Jason R. Coombs <jaraco at jaraco.com>:

Reported in https://github.com/python/importlib_metadata/issues/353, importlib.metadata fails to find distributions in the path "". A fix was applied to importlib_metadata 2.1.2 and 4.8.2. Let's apply that fix and backport it to supported Pythons.

----------
messages: 406022
nosy: jaraco
priority: normal
severity: normal
status: open
title: importlib.metadata fails to find distributions in empty path

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45765>
_______________________________________

From report at bugs.python.org  Tue Nov  9 10:16:23 2021
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 09 Nov 2021 15:16:23 +0000
Subject: [New-bugs-announce] [issue45766] Add direct proportion option to
 statistics.linear_regression()
Message-ID: <1636470983.21.0.286113384782.issue45766@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Signature:

    def linear_regression(x, y, /, *, proportional=False):

Additional docstring with example:

    If *proportional* is true, the independent variable *x* and the
    dependent variable *y* are assumed to be directly proportional.
    The data is fit to a line passing through the origin.

    Since the *intercept* will always be 0.0, the underlying linear
    function simplifies to:

        y = slope * x + noise

    >>> y = [3 * x[i] + noise[i] for i in range(5)]
    >>> linear_regression(x, y, proportional=True)  #doctest: +ELLIPSIS
    LinearRegression(slope=3.0244754248461283, intercept=0.0)

See Wikipedia entry for regression without an intercept term:
https://en.wikipedia.org/wiki/Simple_linear_regression#Simple_linear_regression_without_the_intercept_term_(single_regressor)

Compare with the *const* parameter in MS Excel's linest() function:
https://support.microsoft.com/en-us/office/linest-function-84d7d0d9-6e50-4101-977a-fa7abf772b6d

Compare with the *IncludeConstantBasis* option in Mathematica:
https://reference.wolfram.com/language/ref/IncludeConstantBasis.html

----------
components: Library (Lib)
messages: 406026
nosy: rhettinger, steven.daprano
priority: normal
severity: normal
status: open
title: Add direct proportion option to statistics.linear_regression()
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45766>
_______________________________________

From report at bugs.python.org  Tue Nov  9 12:13:09 2021
From: report at bugs.python.org (Dmitry Marakasov)
Date: Tue, 09 Nov 2021 17:13:09 +0000
Subject: [New-bugs-announce] [issue45767] Fix types for dev_t processing in
 posix module
Message-ID: <1636477989.19.0.839413468229.issue45767@roundup.psfhosted.org>


New submission from Dmitry Marakasov <amdmi3 at amdmi3.ru>:

So, I was investigating a test failure of python 3.11 and 3.10 on FreeBSD (but it likely applies to all python versions):

======================================================================
FAIL: test_makedev (test.test_posix.PosixTester)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/lang/python311/work/Python-3.11.0a1/Lib/test/test_posix.py", line 686, in test_makedev
    self.assertGreaterEqual(dev, 0)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: -5228656221359548839 not greater than or equal to 0

----------------------------------------------------------------------

The test checks that posix.stat(somepath).st_dev >= 0, but negative value was returned.

Python uses PyLong_FromLongLong to convert from dev_t C type which st_dev is:

https://github.com/python/cpython/blob/main/Modules/posixmodule.c#L2410
https://github.com/python/cpython/blob/main/Modules/posixmodule.c#L901

POSIX does not seem to define signedness of dev_t,

https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/basedefs/sys_types.h.html

only saying it shall be an "integer type". However on practice on both FreeBSD and Linux it's unsigned:

FreeBSD:
typedef __dev_t     dev_t;   // sys/types.h
typedef __uint64_t  __dev_t; // sys/_types.h

Linux (Ubuntu 18.04):
typedef __dev_t dev_t;             // sys/stat.h
__STD_TYPE __DEV_T_TYPE __dev_t;   // sys/types.h
#define __DEV_T_TYPE            __UQUAD_TYPE;  // sys/typesizes.h

So I suggest the attached patch to switch _PyLong_FromDev to PyLong_FromUnsignedLongLong which also makes it consistent with _Py_Dev_Converter which converts the other way with PyLong_AsUnsignedLongLong.




This change fixes the mentioned test, but another test failure is unmasked:

======================================================================
ERROR: test_makedev (test.test_posix.PosixTester)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/lang/python311/work/Python-3.11.0a1/Lib/test/test_posix.py", line 704, in test_makedev
    self.assertEqual(posix.makedev(major, minor), dev)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python int too large to convert to C int

----------------------------------------------------------------------

This problem needs couple more trivial changes, but I'm not sure how to make them correctly (and I'm also confused by how this file relates with clinic/posixmodule.c).

The problems are that:
- os_major_impl/os_minor_impl and os_makedev_impl are inconsistent in their argument/return types for major/minor dev numbers: the former use `unsigned int`, while the latter uses `int`.
- the correct type is platform dependent, for instance Linux uses `unsigned int` and FreeBSD uses `int` (from `man makedev`).

I guess that to fix this failure one needs to add a macro/typedef for the type for minor/major dev numbers like

#if defined(__FreeBSD__)
#define DEVICE_MAJORMINOR_T int
#else
#define DEVICE_MAJORMINOR_T unsigned int
#endif

and use it in the named functions:

static DEVICE_MAJORMINOR_T os_major_impl(PyObject *module, dev_t device)
static DEVICE_MAJORMINOR_T os_minor_impl(PyObject *module, dev_t device)

static dev_t os_makedev_impl(PyObject *module, DEVICE_MAJORMINOR_T major, DEVICE_MAJORMINOR_T minor)

----------
components: Extension Modules, FreeBSD
files: posixmodule.c.patch
keywords: patch
messages: 406030
nosy: AMDmi3, koobs
priority: normal
severity: normal
status: open
title: Fix types for dev_t processing in posix module
type: behavior
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file50433/posixmodule.c.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45767>
_______________________________________

From report at bugs.python.org  Tue Nov  9 12:39:33 2021
From: report at bugs.python.org (Jose Ville)
Date: Tue, 09 Nov 2021 17:39:33 +0000
Subject: [New-bugs-announce] [issue45768] SyntaxError: 'await' outside
 function" in "asyncio-task.html#waiting-primitives" code snippets
Message-ID: <1636479573.9.0.0363424339507.issue45768@roundup.psfhosted.org>


New submission from Jose Ville <joseville>:

https://docs.python.org/3/library/asyncio-task.html#asyncio.wait has the following two code snippets both of which fail with ""SyntaxError: 'await' outside function" when I run them in Python 3.9.7

Snippet 1:

```
async def foo():
    return 42

coro = foo()
done, pending = await asyncio.wait({coro})

if coro in done:
    # This branch will never be run!
    pass # I added this to prevent IndentationError
```

Snippet 2:

```
async def foo():
    return 42

task = asyncio.create_task(foo())
done, pending = await asyncio.wait({task})

if task in done:
    # Everything will work as expected now.
    pass # I added this to prevent IndentationError
```

----------
messages: 406032
nosy: joseville
priority: normal
severity: normal
status: open
title: SyntaxError: 'await' outside function" in "asyncio-task.html#waiting-primitives" code snippets

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45768>
_______________________________________

From report at bugs.python.org  Tue Nov  9 12:41:18 2021
From: report at bugs.python.org (Jose Ville)
Date: Tue, 09 Nov 2021 17:41:18 +0000
Subject: [New-bugs-announce] [issue45769] SyntaxError: 'await' outside
 function" in "asyncio-task.html#waiting-primitives" code snippets
Message-ID: <1636479678.52.0.397131484588.issue45769@roundup.psfhosted.org>


New submission from Jose Ville <joseville>:

https://docs.python.org/3/library/asyncio-task.html#asyncio.wait has the following two code snippets both of which fail with ""SyntaxError: 'await' outside function" when I run them in Python 3.9.7

Snippet 1:

```
async def foo():
    return 42

coro = foo()
done, pending = await asyncio.wait({coro})

if coro in done:
    # This branch will never be run!
    pass # I added this to prevent IndentationError
```

Snippet 2:

```
async def foo():
    return 42

task = asyncio.create_task(foo())
done, pending = await asyncio.wait({task})

if task in done:
    # Everything will work as expected now.
    pass # I added this to prevent IndentationError
```

----------
messages: 406033
nosy: joseville
priority: normal
severity: normal
status: open
title: SyntaxError: 'await' outside function" in "asyncio-task.html#waiting-primitives" code snippets
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45769>
_______________________________________

From report at bugs.python.org  Tue Nov  9 12:43:49 2021
From: report at bugs.python.org (Jose Ville)
Date: Tue, 09 Nov 2021 17:43:49 +0000
Subject: [New-bugs-announce] [issue45770] SyntaxError: 'await' outside
 function" in "asyncio-task.html#waiting-primitives" code snippets
Message-ID: <1636479829.16.0.0308838926524.issue45770@roundup.psfhosted.org>


New submission from Jose Ville <joseville>:

https://docs.python.org/3/library/asyncio-task.html#asyncio.wait has the following two code snippets both of which fail with ""SyntaxError: 'await' outside function" when I run them in Python 3.9.7

Snippet 1:

```
async def foo():
    return 42

coro = foo()
done, pending = await asyncio.wait({coro})

if coro in done:
    # This branch will never be run!
    pass # I added this to prevent IndentationError
```

Snippet 2:

```
async def foo():
    return 42

task = asyncio.create_task(foo())
done, pending = await asyncio.wait({task})

if task in done:
    # Everything will work as expected now.
    pass # I added this to prevent IndentationError
```

----------
messages: 406034
nosy: joseville
priority: normal
severity: normal
status: open
title: SyntaxError: 'await' outside function" in "asyncio-task.html#waiting-primitives" code snippets
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45770>
_______________________________________

From report at bugs.python.org  Tue Nov  9 13:39:42 2021
From: report at bugs.python.org (Michael Thorpe)
Date: Tue, 09 Nov 2021 18:39:42 +0000
Subject: [New-bugs-announce] [issue45771] urllib.request.urlopen can leak
 socket.timeout to callers
Message-ID: <1636483182.35.0.0840810707208.issue45771@roundup.psfhosted.org>


New submission from Michael Thorpe <michael at thorpe.dev>:

urllib.request.urlopen can raise socket.timeout, outside of the documented contract, if reading response data times out.

----------
components: Library (Lib)
messages: 406038
nosy: thorpe-dev
priority: normal
severity: normal
status: open
title: urllib.request.urlopen can leak socket.timeout to callers
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45771>
_______________________________________

From report at bugs.python.org  Tue Nov  9 14:55:56 2021
From: report at bugs.python.org (Oleg Iarygin)
Date: Tue, 09 Nov 2021 19:55:56 +0000
Subject: [New-bugs-announce] [issue45772] socket.socket should be a class
 instead of a function
Message-ID: <1636487756.2.0.78548786827.issue45772@roundup.psfhosted.org>


New submission from Oleg Iarygin <oleg at arhadthedev.net>:

Found a twice stale bpo-less but useful pull request 23960, so publish it here:

> Currently socket.socket is documented as a function, but it is really
a class (and thus has function-like usage to construct an object). This
correction would ensure that Python projects that are interlinking
Python's documentation can properly locate socket.socket as a type.

----------
assignee: docs at python
components: Documentation
messages: 406045
nosy: arhadthedev, docs at python, xuhdev
priority: normal
pull_requests: 27751
severity: normal
status: open
title: socket.socket should be a class instead of a function
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45772>
_______________________________________

From report at bugs.python.org  Tue Nov  9 14:56:22 2021
From: report at bugs.python.org (Brandt Bucher)
Date: Tue, 09 Nov 2021 19:56:22 +0000
Subject: [New-bugs-announce] [issue45773] Compiler hangs during jump
 elimination
Message-ID: <1636487782.5.0.379047434297.issue45773@roundup.psfhosted.org>


New submission from Brandt Bucher <brandtbucher at gmail.com>:

The following code hangs during compilation on 3.11 and 3.10:

>>> while True or spam: pass

Our peepholer gets stuck in a loop, repeatedly "optimizing" instruction 4 (POP_JUMP_IF_TRUE -> JUMP_ABSOLUTE):

  1           0 JUMP_ABSOLUTE            0 (to 0)
              2 LOAD_NAME                0 (spam)
              4 POP_JUMP_IF_TRUE         0 (to 0)

After optimizing jumps to jumps like these, we always back up and attempt to optimize the same instruction again (which makes it possible to optimize three or more chained jumps on the same line). The issue is that this particular optimization doesn't actually change the instruction, since both jumps target the same exact block. Since nothing changes, we loop forever.

The fix is really simple: just skip the optimization if instr->i_target == target->i_target. We already do this for several other types of jumps; it just looks like POP_JUMP_IF_TRUE and POP_JUMP_IF_FALSE might have been missed.

I'll have a PR up soon.

----------
assignee: brandtbucher
components: Interpreter Core
messages: 406046
nosy: Mark.Shannon, brandtbucher
priority: high
severity: normal
status: open
title: Compiler hangs during jump elimination
type: crash
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45773>
_______________________________________

From report at bugs.python.org  Tue Nov  9 17:26:46 2021
From: report at bugs.python.org (Erlend E. Aasland)
Date: Tue, 09 Nov 2021 22:26:46 +0000
Subject: [New-bugs-announce] [issue45774] Detect SQLite in configure.ac
Message-ID: <1636496806.29.0.76332999648.issue45774@roundup.psfhosted.org>


New submission from Erlend E. Aasland <erlend.aasland at innova.no>:

"Autoconfiscate" SQLite detection. Plan:

- Detect header/library using AC_CHECK_HEADERS/AC_CHECK_LIB
- Check required version using AC_COMPILE_IFELSE
- Use AC_CHECK_LIB to check if sqlite3_load_extension is present, and if the result harmonises with --enable-loadable-sqlite-extensions; bail if not


Nice side effect: configure will bail if --enable-loadable-sqlite-extensions is used and the target SQLite library was compiled with SQLITE_OMIT_LOAD_EXTENSION, even if we are not on macOS.

----------
components: Build
messages: 406059
nosy: christian.heimes, erlendaasland
priority: normal
severity: normal
status: open
title: Detect SQLite in configure.ac
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45774>
_______________________________________

From report at bugs.python.org  Tue Nov  9 19:06:44 2021
From: report at bugs.python.org (Thomas Stolarski)
Date: Wed, 10 Nov 2021 00:06:44 +0000
Subject: [New-bugs-announce] [issue45775] Implementation of
 colorsys.rgb_to_yuv and vice versa
Message-ID: <1636502804.95.0.337411519987.issue45775@roundup.psfhosted.org>


New submission from Thomas Stolarski <thomas.stolarski at gmail.com>:

Since the implementation of `rgb_to_yiq` roughly 30 years ago now, the advent of HDTV has resulted in most broadcasting and video processing having moved towards Rec 709 (or BT.709).

While I know colorsys has been on the chopping block for a while, it seemed a little silly that what is now arguably the most common color space used in video editors isn't available as part of this module, despite being a matrix mapping like much of the others.

YUV is a bit more contentious on its definition than YIQ is, but it is still widely used to refer to Rec 709 and this is the ATSC standardization for HDTV.

I've written a PR for both conversions and will add it to this ticket once it's up.

----------
components: Library (Lib)
messages: 406067
nosy: thomas.stolarski
priority: normal
severity: normal
status: open
title: Implementation of colorsys.rgb_to_yuv and vice versa
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45775>
_______________________________________

From report at bugs.python.org  Wed Nov 10 01:54:36 2021
From: report at bugs.python.org (Steve Canny)
Date: Wed, 10 Nov 2021 06:54:36 +0000
Subject: [New-bugs-announce] [issue45776] abc submodule not an attribute of
 collections on Python 3.10.0 on Windows
Message-ID: <1636527276.55.0.818316908818.issue45776@roundup.psfhosted.org>


New submission from Steve Canny <stcanny at gmail.com>:

On CPython 3.10.0 on Windows but not MacOS, AttributeError is raised when referencing `abc` on collections (instead of importing `collections.abc`).

>>> import collections
>>> collections.abc.Container
AttributeError: module 'collections' has no attribute 'abc'

This works on MacOS CPython 3.10.0 and also works on Windows CPython 3.9.6.

It's possibly interesting this coincides with collection ABCs being made unavailable directly from `collections` with this 3.9 -> 3.10 version change, but that's just a guess, for whatever it's worth.

----------
components: Windows
messages: 406074
nosy: paul.moore, stcanny, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: abc submodule not an attribute of collections on Python 3.10.0 on Windows
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45776>
_______________________________________

From report at bugs.python.org  Wed Nov 10 04:48:51 2021
From: report at bugs.python.org (Abdullah Alnajim)
Date: Wed, 10 Nov 2021 09:48:51 +0000
Subject: [New-bugs-announce] [issue45777] Issue in reading files with a path
 longer than 256 letters after latest update
Message-ID: <1636537731.96.0.00930160338556.issue45777@roundup.psfhosted.org>


New submission from Abdullah Alnajim <aalnajim1 at hotmail.com>:

After updating python to the latest version, an issue related to reading files in long paths (>256 letters) is arisen. Whenever I try to read such a file in Windows 11, I got an exception telling me that the file is not there, even though it?s there and I allowed windows 11 to accept long paths  (> Max_Path). Absolute and relative paths did not work. Projects that were working before normally, are no longer working. The exception is ?FileNotFoundError: [Errno 2] No such file or directory: The_Long_Path_To_The_File,

----------
messages: 406078
nosy: Alnajim
priority: normal
severity: normal
status: open
title: Issue in reading files with a path longer than 256 letters after latest update
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45777>
_______________________________________

From report at bugs.python.org  Wed Nov 10 07:11:49 2021
From: report at bugs.python.org (Matthias Klose)
Date: Wed, 10 Nov 2021 12:11:49 +0000
Subject: [New-bugs-announce] [issue45778] libpython.so 3.9.8 drops symbol
 used by third party extension module(s)
Message-ID: <1636546309.5.0.657250310979.issue45778@roundup.psfhosted.org>


New submission from Matthias Klose <doko at debian.org>:

[reported in Debian as https://bugs.debian.org/998854]

3.9.8, compared to 3.9.7 introduces a regression, making at least one third party extension fail at runtime. The symbol changes in 3.9.8 are:

- _PyUnicode_DecodeUnicodeEscape at Base
+ _PyUnicode_DecodeRawUnicodeEscapeStateful at Base
+ _PyUnicode_DecodeUnicodeEscapeInternal at Base
+ _PyUnicode_DecodeUnicodeEscapeStateful at Base

Affected at least is the typed-ast extension.  Yes, you can rebuild the extension with 3.9.8, but then it doesn't work with earlier 3.9 versions.  Just dropping the symbol on a stable branch is suboptimal.

----------
components: Interpreter Core
keywords: 3.9regression
messages: 406085
nosy: doko, lukasz.langa
priority: release blocker
severity: normal
status: open
title: libpython.so 3.9.8 drops symbol used by third party extension module(s)
type: crash
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45778>
_______________________________________

From report at bugs.python.org  Wed Nov 10 11:47:54 2021
From: report at bugs.python.org (Zhang Zheng)
Date: Wed, 10 Nov 2021 16:47:54 +0000
Subject: [New-bugs-announce] [issue45779] multiprocessing initializer error
 with CPython 3.9.6 on Apple Silicon
Message-ID: <1636562874.34.0.208720566335.issue45779@roundup.psfhosted.org>


New submission from Zhang Zheng <simsicon at gmail.com>:

Hi, I have encountered an error when working with concurrent.futures.ProcessPoolExecutor API, with CPython 3.9.6 on Apple Silicon, the error can not be reproduced with CPython 3.9.6 Linux/X86 build, so I think this might be related to the arch.

I have created a github repo with the code in poc.py file, and the error message in README, please let me know if there's anything more I can provide to help the investigation to the problem.

----------
components: Library (Lib)
messages: 406108
nosy: zhangzheng
priority: normal
severity: normal
status: open
title: multiprocessing initializer error with CPython 3.9.6 on Apple Silicon
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45779>
_______________________________________

From report at bugs.python.org  Wed Nov 10 16:14:59 2021
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 10 Nov 2021 21:14:59 +0000
Subject: [New-bugs-announce] [issue45780] dict. keys view behavious diverges
 from set()
Message-ID: <1636578899.28.0.498722487366.issue45780@roundup.psfhosted.org>


Change by Gregory P. Smith <greg at krypto.org>:


----------
components: Interpreter Core
nosy: gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: dict. keys view behavious diverges from set()
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45780>
_______________________________________

From report at bugs.python.org  Thu Nov 11 04:45:11 2021
From: report at bugs.python.org (Carl Friedrich Bolz-Tereick)
Date: Thu, 11 Nov 2021 09:45:11 +0000
Subject: [New-bugs-announce] [issue45781] Deleting __debug__ should be an
 SyntaxError
Message-ID: <1636623911.67.0.569601489226.issue45781@roundup.psfhosted.org>


New submission from Carl Friedrich Bolz-Tereick <cfbolz at gmx.de>:

Right now, deleting __debug__ is not prevented:

>>> def f():
...     del __debug__
... 

Of course actually executing it doesn't work:

>>> del __debug__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name '__debug__' is not defined


Compare with assigning to __debug__:

>>> def f():
...     __debug__ = 1
... 
  File "<stdin>", line 2
SyntaxError: cannot assign to __debug__

----------
messages: 406149
nosy: Carl.Friedrich.Bolz
priority: normal
severity: normal
status: open
title: Deleting __debug__ should be an SyntaxError

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45781>
_______________________________________

From report at bugs.python.org  Thu Nov 11 05:32:18 2021
From: report at bugs.python.org (Terje Myklebust)
Date: Thu, 11 Nov 2021 10:32:18 +0000
Subject: [New-bugs-announce] [issue45782] Bug in struct.pack
Message-ID: <1636626738.35.0.997508100791.issue45782@roundup.psfhosted.org>


New submission from Terje Myklebust <terje.myklebust123 at gmail.com>:

>>> struct.pack("i", 119) 
b'w\x00\x00\x00'

"w" in a binary value?

>>> struct.pack("I", 116) 
b't\x00\x00\x00'

"t" in a binary value?

----------
messages: 406150
nosy: terje.myklebust123
priority: normal
severity: normal
status: open
title: Bug in struct.pack
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45782>
_______________________________________

From report at bugs.python.org  Thu Nov 11 05:34:54 2021
From: report at bugs.python.org (Petr Viktorin)
Date: Thu, 11 Nov 2021 10:34:54 +0000
Subject: [New-bugs-announce] [issue45783] test_freeze fails if a file is
 removed
Message-ID: <1636626894.31.0.876516857977.issue45783@roundup.psfhosted.org>


New submission from Petr Viktorin <encukou at gmail.com>:

In Fedora, we remove the bundled wheels; pip & co. are supplied (and kept updated) by the system. I believe this is good practice.

However, removing any file from the CPython source checkout makes test_freeze fail:

Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.11.0a2/Lib/test/test_tools/test_freeze.py", line 25, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script)
                                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0a2/Tools/freeze/test/freeze.py", line 144, in prepare
    git_copy_repo(srcdir, SRCDIR)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0a2/Tools/freeze/test/freeze.py", line 97, in git_copy_repo
    shutil.copy2(srcfile, dstfile)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0a2/Lib/shutil.py", line 436, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0a2/Lib/shutil.py", line 256, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/builddir/build/BUILD/Python-3.11.0a2/Lib/ensurepip/_bundled/pip-21.2.4-py3-none-any.whl'


It looks like this test is recreating uncommited modifications, is it OK to change it to support deletions as well?
Also, `git status --porcelain` should be used to get the output in a stable format.

----------
messages: 406151
nosy: eric.snow, petr.viktorin
priority: normal
severity: normal
status: open
title: test_freeze fails if a file is removed
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45783>
_______________________________________

From report at bugs.python.org  Thu Nov 11 05:37:57 2021
From: report at bugs.python.org (Pavithra)
Date: Thu, 11 Nov 2021 10:37:57 +0000
Subject: [New-bugs-announce] [issue45784] SAP HANA Training in Chennai
Message-ID: <1636627077.73.0.882488207493.issue45784@roundup.psfhosted.org>


New submission from Pavithra <ipavi617 at gmail.com>:

SAP MM Training in Chennai
SAP SD Training in Chennai
SAP FICO Training in Chennai
SAP ARIBA Training in Chennai
SAP ABAP Training in Chennai
SAP HR Training in Chennai
SAP HANA Training in Chennai

----------
messages: 406152
nosy: Pavithra
priority: normal
severity: normal
status: open
title: SAP HANA Training in Chennai
type: security

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45784>
_______________________________________

From report at bugs.python.org  Thu Nov 11 06:38:14 2021
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 11 Nov 2021 11:38:14 +0000
Subject: [New-bugs-announce] [issue45785] Remove asyncore,
 asynchat and smtpd modules
Message-ID: <1636630694.41.0.881800749234.issue45785@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

I propose to remove asyncore, asynchat and smtpd modules from the Python standard library to remove the Python maintenance burden. These modules are deprecated since Python 3.6.

--

The asyncore module is a very old module of the Python stdlib for asynchronous programming, usually to handle network sockets concurrently. It's a common event loop, but its design has many flaws.

The asyncio module was added to Python 3.4 with a well designed architecture. Twisted developers who have like 10 to 20 years of experience in asynchronous programming helped to design the asyncio API.

By design, asyncio doesn't have flaws which would be really hard to fix in asyncore and asynchat.

It was decided to start deprecating the asyncore and asynchat module in Python 3.6 released in 2016, 5 years ago. Open issues in asyncore and asynchat have been closed as "wont fix" because the module is deprecated. The two modules are basically no longer maintained.

Documentation:

* https://docs.python.org/dev/library/asyncore.html
* https://docs.python.org/dev/library/asynchat.html

I propose to remove the two modules right now in the Python stdlib. They were removed for 4 Python releases (3.6-3.10), it's long enough to respect the PEP 387. The PEP requires 2 Python releases at least before considering removing code.

Since there are still 7 tests of the Python test suite still uses asyncore and asynchat, I propose to move these modules in Lib/test/ directory and make them private:

* Rename Lib/asyncore.py to Lib/test/support/_asyncore.py
* Rename Lib/asynchat.py to Lib/test/support/_asynchat.py

Projects using asyncore and asynchat should use asyncio. If someone really wants to continue using asyncore and asynchat, it's trivial to copy asyncore.py and asynchat.py in their project, and maintain these files on their side.

--

About the smtpd module, it is also deprecated since Python 3.6 (deprecated for 4 Python releases). It's used by a single test (test_logging). I also propose to remove it from the stdlib.

I proposed to rename Lib/smtpd.py to Lib/test/support/_smtpd.py.

Projects using smtpd can consider using aiosmtpd which is based on asyncio:
https://aiosmtpd.readthedocs.io/

Or again, they can copy Python 3.10 smtpd.py in their project and maintain this file on their side.

----------
components: Library (Lib)
messages: 406156
nosy: vstinner
priority: normal
severity: normal
status: open
title: Remove asyncore, asynchat and smtpd modules
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45785>
_______________________________________

From report at bugs.python.org  Thu Nov 11 10:04:09 2021
From: report at bugs.python.org (Mark Shannon)
Date: Thu, 11 Nov 2021 15:04:09 +0000
Subject: [New-bugs-announce] [issue45786] Avoid allocating when exiting
 frame; it may be unsafe.
Message-ID: <1636643049.78.0.0321804916993.issue45786@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

We exiting a frame (returning from a Python function) we have to release the stack allocated frame. If a heap-allocated frame object exists, we need to copy the contents of the (stack) frame into the frame object.
However, this involves allocating memory for the copy. Allocating memory can invoke GC, causing arbitrary code to be run, or the allocation can fail. Either leaves us in a precarious state, which may be unsafe.

I haven't been able to produce a crash, but I'm not sure that there isn't a potential crash lurking there either.

The fix is fairly simple. Allocate space for the copy of the frame at the end of the frame object. Then we need to copy the data, space will have already been allocated, and nothing can fail.

Since, in theory, heap-allocated frames are relatively rare, the extra memory used won't be an issue.

----------
assignee: Mark.Shannon
components: Interpreter Core
keywords: 3.11regression
messages: 406163
nosy: Mark.Shannon, pablogsal
priority: normal
severity: normal
status: open
title: Avoid allocating when exiting frame; it may be unsafe.
type: crash
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45786>
_______________________________________

From report at bugs.python.org  Thu Nov 11 12:56:34 2021
From: report at bugs.python.org (Alex Waygood)
Date: Thu, 11 Nov 2021 17:56:34 +0000
Subject: [New-bugs-announce] [issue45787] HOWTO for socket programming and
 select documentation contradict
Message-ID: <1636653394.87.0.117733096007.issue45787@roundup.psfhosted.org>


New submission from Alex Waygood <Alex.Waygood at Gmail.com>:

The HOWTO for socket programming in Python (https://docs.python.org/3/howto/sockets.html#non-blocking sockets) includes the following lines (using triple-quotes here to indicate multiline quotes from the docs):

"""
The major mechanical difference [between blocking and non-blocking sockets] is that [for non-blocking sockets] send, recv, connect and accept can return without having done anything. You have (of course) a number of choices. You can check return code and error codes and generally drive yourself crazy. If you don?t believe me, try it sometime. Your app will grow large, buggy and suck CPU. So let?s skip the brain-dead solutions and do it right.

Use select.
"""

However, if you go to the documentation for the select module (https://docs.python.org/3/library/select html), it pointedly tells you at the top *not* to do exactly what the socket HOWTO tells you to do:

"""
Note The selectors module allows high-level and efficient I/O multiplexing, built upon the select module primitives. Users are encouraged to use the selectors module instead, unless they want precise control over the OS-level primitives used.
"""

----------
assignee: docs at python
components: Documentation
messages: 406167
nosy: AlexWaygood, docs at python, giampaolo.rodola, neologix
priority: normal
severity: normal
status: open
title: HOWTO for socket programming and select documentation contradict
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45787>
_______________________________________

From report at bugs.python.org  Thu Nov 11 13:16:42 2021
From: report at bugs.python.org (Catherine Holloway)
Date: Thu, 11 Nov 2021 18:16:42 +0000
Subject: [New-bugs-announce] [issue45788] sys.prefix include directory on
 Windows
Message-ID: <1636654602.08.0.849024909649.issue45788@roundup.psfhosted.org>


New submission from Catherine Holloway <milankie at gmail.com>:

On the page:

https://docs.python.org/3.10/library/sys.html#sys.prefix

the text says

The main collection of Python library modules is installed in the directory prefix/lib/pythonX.Y while the platform independent header files (all except pyconfig.h) are stored in prefix/include/pythonX.Y, where X.Y is the version number of Python, for example 3.2.

However, this seems to only be true on *nix systems. On Windows, the lib and header files are just in prefix/lib and prefix/include. Maybe the text should be changed to say:

On linux, the main collection of Python library modules is installed in the directory prefix/lib/pythonX.Y while the platform independent header files (all except pyconfig.h) are stored in prefix/include/pythonX.Y, where X.Y is the version number of Python, for example 3.2. On Winodws, there are in prefix/lib and prefix/includes.

----------
assignee: docs at python
components: Documentation
messages: 406168
nosy: CatherineH, docs at python
priority: normal
severity: normal
status: open
title: sys.prefix include directory on Windows
type: enhancement
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45788>
_______________________________________

From report at bugs.python.org  Thu Nov 11 15:17:23 2021
From: report at bugs.python.org (firewave)
Date: Thu, 11 Nov 2021 20:17:23 +0000
Subject: [New-bugs-announce] [issue45789] Python execution broken after
 update via Windows Store
Message-ID: <1636661843.78.0.241514284598.issue45789@roundup.psfhosted.org>


New submission from firewave <oliverst at online.de>:

I have Python 3.9 installed from the Windows Store and use it in CMake scripts.

After the recent update to 3.92288.0 Python executions suddenly started failing with "Access is denied". I assumed I was using a wrong or cached path but that didn't turn out to be the case.

It did find the executable in the (apparently) correct folder "C:/Program Files/WindowsApps/PythonSoftwareFoundation.Python.3.9_3.9.2288.0_x64__qbz5n2kfra8p0/python3.9.exe".

I tried running that via the command-line and it failed with "Access is denied." as well. It appears to be related to "C:/Program Files/WindowsApps" requiring Administrator permissions.

Also just running "python3.9.exe" (which exists in the PATH) no longer works. It fails with "The system cannot find the file C:\Users\<username>\AppData\Local\Microsoft\WindowsApps\python3.9.exe."

So it seems the update messed something up.

I had Python 3.9 installed and working fine for some version now. I ran into something similar in the past and had to uninstall and install it again to fix the problem.

I am using Windows 10 Version 21H1 (OS Build 19043.1348).

----------
components: Windows
messages: 406176
nosy: firewave, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python execution broken after update via Windows Store
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45789>
_______________________________________

From report at bugs.python.org  Thu Nov 11 21:55:53 2021
From: report at bugs.python.org (Rodrigo Tobar)
Date: Fri, 12 Nov 2021 02:55:53 +0000
Subject: [New-bugs-announce] [issue45790] Inaccurate phrasing in
 extending/newtypes_tutorial
Message-ID: <1636685753.29.0.246932963554.issue45790@roundup.psfhosted.org>


New submission from Rodrigo Tobar <rtobarc at gmail.com>:

In `extending/newtypes_tutorial.rst` the following phrase appears:


"[...], containing a pointer to a type object and a reference count (these can be accessed using the macros :c:macro:`Py_REFCNT` and c:macro:`Py_TYPE` respectively)."

I believe it should read "using the macros :c:macro:`Py_TYPE` and c:macro:`Py_REFCNT` respectively" to follow the same order in which the fields are described.

I'll put forward a patch. It seems this phrase goes way back a few python versions, but I'm tagging 3.11 here only.

----------
assignee: docs at python
components: Documentation
messages: 406185
nosy: docs at python, rtobar2
priority: normal
severity: normal
status: open
title: Inaccurate phrasing in extending/newtypes_tutorial
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45790>
_______________________________________

From report at bugs.python.org  Fri Nov 12 00:01:11 2021
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 12 Nov 2021 05:01:11 +0000
Subject: [New-bugs-announce] [issue45791] __instancecheck__ being checked of
 type(cls) instead of cls
Message-ID: <1636693271.06.0.414985323738.issue45791@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Per PEP 3119, "Overloading works as follows: The call isinstance(x, C) first checks whether C.__instancecheck__ exists, and if so, calls C.__instancecheck__(x) instead of its normal implementation."

However, this doesn't work because the isinstance() has a bug introduced in Python 3.1 and no one ever noticed.

In abstract.c::object_recursive_isinstance(), we have:

    PyObject *checker = _PyObject_LookupSpecial(cls, &PyId___instancecheck__);

However, that function expects an instance as an argument rather than a class.

Calling typeobject.c::_PyObject_LookupSpecial() runs:

    res = _PyType_LookupId(Py_TYPE(self), attrid);

Note, the Py_TYPE(self) is intended to move up from an instance to a class, but we're already started with a class, so it moves to the metaclass instead.

This code was correct when originally implemented but it did not have tests:

    if (name == NULL) {
        name = PyString_InternFromString("__instancecheck__");
        if (name == NULL)
            return -1;
    }
    checker = PyObject_GetAttr(cls, name);
    if (checker == NULL && PyErr_Occurred())
        PyErr_Clear();


------- Demonstration code ---------------

class C:
    def __instancecheck__(self, inst):
        raise RuntimeError(f'{self=}  {inst=}')

    
class P:
    def __instancecheck__(self, inst):
        raise RuntimeError(f'{self=}  {inst=}')

    
class C(P):
    pass

>>> isinstance(C(), P)    # Incorrectly fails to invoke __instancecheck__
True

>>> isinstance(C(), P())  # Incorrectly invokes __instancecheck__
Traceback (most recent call last):
  File "<pyshell#10>", line 1, in <module>
    isinstance(C(), P())
  File "<pyshell#5>", line 3, in __instancecheck__
    raise RuntimeError(f'{self=}  {inst=}')
RuntimeError: self=<__main__.P object at 0x107586c80>  inst=<__main__.C object at 0x107587100>

----------
components: Interpreter Core
messages: 406187
nosy: rhettinger
priority: normal
severity: normal
status: open
title: __instancecheck__ being checked of type(cls) instead of cls
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45791>
_______________________________________

From report at bugs.python.org  Fri Nov 12 02:59:35 2021
From: report at bugs.python.org (Hynek Schlawack)
Date: Fri, 12 Nov 2021 07:59:35 +0000
Subject: [New-bugs-announce] [issue45792] contextvars.Token has wrong module
 name in Sphinx's objects.inv
Message-ID: <1636703975.97.0.094417369522.issue45792@roundup.psfhosted.org>


New submission from Hynek Schlawack <hs at ox.cx>:

Doc/library/contextvars.rst defines a module using `.. module:: contextvars` which means that all defined symbols are automatically part of the contextvars module.

The docs added in https://github.com/python/cpython/pull/5685 however explicitly use `.. class:: contextvars.Token` instead of just `.. class:: Token` which means that the recorded intersphinx symbol is `contextvars.contextvars.Token`. I have noticed this because sphinx couldn't find `contextvars.Token` in structlog's docs.

AFAICT, this only affects contextvars.Token.

----------
assignee: hynek
components: Documentation
messages: 406192
nosy: hynek, yselivanov
priority: low
severity: normal
stage: needs patch
status: open
title: contextvars.Token has wrong module name in Sphinx's objects.inv
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45792>
_______________________________________

From report at bugs.python.org  Fri Nov 12 04:08:28 2021
From: report at bugs.python.org (junyixie)
Date: Fri, 12 Nov 2021 09:08:28 +0000
Subject: [New-bugs-announce] [issue45793] execute shell command ./configure
 with python subprocess.popen,
 os.system ... checking build system type...  is wrong
Message-ID: <1636708108.33.0.041846133836.issue45793@roundup.psfhosted.org>


New submission from junyixie <xie.junyi at outlook.com>:

wrong:
checking build system type... x86_64-apple-darwin20.5.0
checking host system type... x86_64-apple-darwin20.5.0


expect:
checking build system type... arm-apple-darwin20.5.0
checking host system type... arm-apple-darwin20.5.0

----------
components: macOS
messages: 406194
nosy: JunyiXie, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: execute shell command ./configure with python subprocess.popen, os.system ... checking build system type...  is wrong
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45793>
_______________________________________

From report at bugs.python.org  Fri Nov 12 06:56:23 2021
From: report at bugs.python.org (Ezekeel)
Date: Fri, 12 Nov 2021 11:56:23 +0000
Subject: [New-bugs-announce] [issue45794] Email policy option for disabling
 RFC2231 Mime parameter folding
Message-ID: <1636718183.28.0.396771366939.issue45794@roundup.psfhosted.org>


New submission from Ezekeel <dierksen at saeder-krupp.de>:

I am using the Python email module extensively and noticed that the Microsoft Outlook client does not support RFC2231 for Mime parameter folding. Since that mail client unfortunately is the de facto standard in the corporate world there should be a policy option to disable it.

I have implemented that option: https://github.com/netcon-consulting/cpython/commit/9f485f0a6a5b7f2c207f1d5db7d1e520c71ac3b7

----------
components: email
messages: 406205
nosy: Ezekeel, barry, r.david.murray
priority: normal
severity: normal
status: open
title: Email policy option for disabling RFC2231 Mime parameter folding
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45794>
_______________________________________

From report at bugs.python.org  Fri Nov 12 10:55:25 2021
From: report at bugs.python.org (Muhammad Farhan)
Date: Fri, 12 Nov 2021 15:55:25 +0000
Subject: [New-bugs-announce] [issue45795] urllib http client vulnerable to
 DOS attack
Message-ID: <1636732525.38.0.485782952474.issue45795@roundup.psfhosted.org>


New submission from Muhammad Farhan <farhankhursheed2018 at gmail.com>:

Hi, During my recent tests I have discovered that the urllib http client (urllib.request.urlopen()) is vulnerable to DOS attack using a simple but effective trick.

I am attaching a file named server.py download it and run it using latest version of python. After running it execute the following python code in python interactive mode. (python -i)

import urllib.request
request = urllib.request.Request('http://127.0.0.1:1338')
response = urllib.request.urlopen(req, timeout=1)

DOS limit: We can achieve DOS for unlimited time.
 
How to fix?
Implement a good logic for timeout in urllib.request.urlopen(url, timeout). Timeout value should not be reset after client receives a data(bytes), because it can easily be abused to achieve DOS.

----------
components: Library (Lib)
files: server.py
messages: 406220
nosy: haqsek2
priority: normal
severity: normal
status: open
title: urllib http client vulnerable to DOS attack
type: security
versions: Python 3.10
Added file: https://bugs.python.org/file50436/server.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45795>
_______________________________________

From report at bugs.python.org  Fri Nov 12 16:51:34 2021
From: report at bugs.python.org (Chris Eykamp)
Date: Fri, 12 Nov 2021 21:51:34 +0000
Subject: [New-bugs-announce] [issue45796] Using tab to cycle through tkinter
 widgets breaks foreground styling
Message-ID: <1636753894.77.0.757469513697.issue45796@roundup.psfhosted.org>


New submission from Chris Eykamp <chris at eykamp.com>:

Attached is a slightly modifed example of the code sample in Bryan Oakley's response to:

https://stackoverflow.com/questions/30337351/how-can-i-ensure-my-ttk-entrys-invalid-state-isnt-cleared-when-it-loses-focus

I have modified it by changing the style.map to change the widget's foreground color (rather than background), and by adding a button below the text input.

Typing a string containing "invalid" into the textbox causes it to fail validation (its state is shown in the bottom label, updated every second).  As expected, the foreground changes to red.

However, if I use [tab] to toggle between the elements, the foreground of the text entry gets changed to black, even if the item is invalid.

Adding         
    style.map("TEntry",  selectforeground=[('invalid', "green")])
creates a new set of weirdness, where the invalid text becomes green when tabbing through the widgets.


I'm running on Windows 10, Python 3.83.

----------
components: Tkinter
files: simple.py
messages: 406242
nosy: watusimoto
priority: normal
severity: normal
status: open
title: Using tab to cycle through tkinter widgets breaks foreground styling
versions: Python 3.8
Added file: https://bugs.python.org/file50437/simple.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45796>
_______________________________________

From report at bugs.python.org  Fri Nov 12 18:21:53 2021
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 12 Nov 2021 23:21:53 +0000
Subject: [New-bugs-announce] [issue45797] AMD64 Arch Linux Asan Debug
 buildbot sometimes hangs before tests complete
Message-ID: <1636759313.91.0.941705839577.issue45797@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

Example:

AMD64 Arch Linux Asan Debug 3.10:
https://buildbot.python.org/all/#/builders/621/builds/351

(...)
1:13:40 load avg: 1.85 [420/421/1] test_reprlib passed
1:13:41 load avg: 1.85 [421/421/1] test_funcattrs passed
command timed out: 1200 seconds without output running [b'make', b'buildbottest', b'TESTOPTS=-j1 -x test_ctypes test_capi test_crypt test_decimal test_faulthandler test_interpreters --junit-xml test-results.xml ${BUILDBOT_TESTOPTS}', b'TESTPYTHONOPTS=', b'TESTTIMEOUT=900'], attempting to kill
make: *** [Makefile:1255: buildbottest] Terminated
process killed by signal 15
program finished with exit code -1
elapsedTime=5623.934168


Can it be related to bpo-45200 "Address Sanitizer: libasan dead lock in pthread_create() (test_multiprocessing_fork.test_get() hangs)"?

----------
components: Tests
messages: 406256
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: AMD64 Arch Linux Asan Debug buildbot sometimes hangs before tests complete
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45797>
_______________________________________

From report at bugs.python.org  Sat Nov 13 04:15:19 2021
From: report at bugs.python.org (Christian Heimes)
Date: Sat, 13 Nov 2021 09:15:19 +0000
Subject: [New-bugs-announce] [issue45798] Move _decimal build setup into
 configure
Message-ID: <1636794919.07.0.823304462548.issue45798@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

Compiler and linker flags for _decimal and internal libmpdec are currently handled by a mix of configure checks and if/else chains in setup.py. The split makes it harder to build _decimal correctly from Modules/Setup. The Modules/Setup file also does not handle --with-system-mpdec.

I have a working PR that moves all logic into configure.ac. The new system:

* sets LIBMPDEC_CFLAGS and LIBMPDEC_LDFLAGS based on --with-system-libmpdec value.

* detects libmpdec_machine by looking at ac_sys_system, MACOSX_DEFAULT_ARCH, ac_cv_sizeof_size_t, ac_cv_gcc_asm_for_x64, ac_cv_type___uint128_t, and ac_cv_gcc_asm_for_x87.

* sets libmpdec compiler args based on libmpdec_machine, have_ipa_pure_const_bug, and have_glibc_memmove_bug.

* if --with-system-libmpdec is not given, then our Makefile compiles libmpdec objects and puts them into a libmpdec.a archive.

* finally it either links _decimal with our libmpdec.a or with system's libmpdec shared library.

I went for libmpdec.a because it makes the logic for the internal path look similar to the logic with linking with an external shared library.

Modules/Setup

----------
assignee: christian.heimes
components: Build
messages: 406271
nosy: christian.heimes, mark.dickinson, pitrou
priority: normal
severity: normal
stage: patch review
status: open
title: Move _decimal build setup into configure
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45798>
_______________________________________

From report at bugs.python.org  Sat Nov 13 09:32:31 2021
From: report at bugs.python.org (Jack DeVries)
Date: Sat, 13 Nov 2021 14:32:31 +0000
Subject: [New-bugs-announce] [issue45799] Fix confusing wording in
 Doc/library/__main__.rst
Message-ID: <1636813951.2.0.152753647597.issue45799@roundup.psfhosted.org>


New submission from Jack DeVries <jdevries3133 at gmail.com>:

I was reading this bit last night and thought it was a typo. In the light of day, I realized it wasn't *technically* a typo, but definitely confusing wording. This PR fixes the confusing sentence.

----------
assignee: docs at python
components: Documentation
messages: 406279
nosy: docs at python, jack__d
priority: normal
severity: normal
status: open
title: Fix confusing wording in Doc/library/__main__.rst
type: enhancement
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45799>
_______________________________________

From report at bugs.python.org  Sat Nov 13 10:37:55 2021
From: report at bugs.python.org (Christian Heimes)
Date: Sat, 13 Nov 2021 15:37:55 +0000
Subject: [New-bugs-announce] [issue45800] Move expat handling into configure
 and Makefile
Message-ID: <1636817875.39.0.335173778255.issue45800@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

Move logic for --with-system-expat out of setup.py into configure and Makefile. This will enable --with-system-expat in Modules/Setup without manual patching.

* Set CFLAGS and LDFLAGS for pyexpat and libexpat in configure.
* Build a static libexpat.a from our copy of expat when --with-system-expat is not given
* Either link pyexpat with libexpat.a or system libexpat.

Basically use the same approach as decimal changeset bpo-45798.

----------
assignee: christian.heimes
components: Build
messages: 406281
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Move expat handling into configure and Makefile
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45800>
_______________________________________

From report at bugs.python.org  Sat Nov 13 12:00:06 2021
From: report at bugs.python.org (Andre Roberge)
Date: Sat, 13 Nov 2021 17:00:06 +0000
Subject: [New-bugs-announce] [issue45801] Incorrect "Perhaps you forgot a
 comma" hint
Message-ID: <1636822806.1.0.227872215707.issue45801@roundup.psfhosted.org>


New submission from Andre Roberge <andre.roberge at gmail.com>:

Python 3.10 and 3.11:

>>> sum[i for i in [1, 2, 3] if i%2==0]
  File "<stdin>", line 1
    sum[i for i in [1, 2, 3] if i%2==0]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?

Furthermore, I don't find that highlighting the entire statement (or parts of it, if we use print(sum[...]) is very useful in attempting to find the source of the error.

In previous versions, we would get the following:

>>> sum[i for i in [1, 2, 3] if i%2==0]
  File "<stdin>", line 1
    sum[i for i in [1, 2, 3] if i%2==0]
          ^
SyntaxError: invalid syntax

----------
components: Parser
messages: 406287
nosy: aroberge, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Incorrect "Perhaps you forgot a comma" hint
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45801>
_______________________________________

From report at bugs.python.org  Sat Nov 13 12:11:09 2021
From: report at bugs.python.org (Akkana Peck)
Date: Sat, 13 Nov 2021 17:11:09 +0000
Subject: [New-bugs-announce] [issue45802] MozillaCookieJar can't read
 cookies, should support cookies.sqlite
Message-ID: <1636823469.07.0.796860437304.issue45802@roundup.psfhosted.org>


New submission from Akkana Peck <akkzilla at shallowsky.com>:

http.cookiejar.MozillaCookieJar only reads from cookies.txt, a format that Mozilla hasn't used in over a decade. It should read the file mozilla actually uses, cookies.sqlite.

Here's some code that works to turn cookies.sqlite into cookies.txt in order to read it in to MozillaCookieJar:
 http://blog.mithis.net/archives/python/90-firefox3-cookies-in-python

This was requested in 2008 in issue 2277, around the time Mozilla made the switch. The issue was rejected back then because it was too late to make the final beta for Python 2.6/3.0. I'd like to bring it up again now.

I can write a patch (since a real fix should read the cookies into the cookiejar directly, not use StringIO to create an intermediate cookies.txt) if there's any chance it would be accepted.

----------
components: Library (Lib)
messages: 406288
nosy: akkana
priority: normal
severity: normal
status: open
title: MozillaCookieJar can't read cookies, should support cookies.sqlite

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45802>
_______________________________________

From report at bugs.python.org  Sat Nov 13 14:57:16 2021
From: report at bugs.python.org (Tsvetan Kintisheff)
Date: Sat, 13 Nov 2021 19:57:16 +0000
Subject: [New-bugs-announce] [issue45803] make_dataclass is missing the
 documented kw_only argument
Message-ID: <1636833436.92.0.166448079237.issue45803@roundup.psfhosted.org>


New submission from Tsvetan Kintisheff <kintisheff at gmail.com>:

According to the 3.10 documentation, make_dataclass includes the kw_only argument: https://github.com/python/cpython/blob/3f15792d60011639d9b170d8a76c6db7f6e83665/Lib/dataclasses.py#L1327

However, the source code referred to by the same doc does not appear to include the kw_only argument:
https://github.com/python/cpython/blob/3f15792d60011639d9b170d8a76c6db7f6e83665/Lib/dataclasses.py#L1327

----------
components: ctypes
messages: 406290
nosy: kintisheff
priority: normal
severity: normal
status: open
title: make_dataclass is missing the documented kw_only argument
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45803>
_______________________________________

From report at bugs.python.org  Sat Nov 13 21:48:01 2021
From: report at bugs.python.org (Roger Serwy)
Date: Sun, 14 Nov 2021 02:48:01 +0000
Subject: [New-bugs-announce] [issue45804] IDLE - faster shell writing
Message-ID: <1636858081.72.0.88336234035.issue45804@roundup.psfhosted.org>


New submission from Roger Serwy <roger.serwy at gmail.com>:

The shell provided by IDLE uses synchronous sys.stdout.write() calls between the subprocess and the front-end, leading to very slow writes. The provided patch proposes buffering the stdout/stderr streams in the subprocess and then sending a single update after 50ms. The patch also provides back pressure on the buffer so that it doesn't grow without bound.

When trying the behavior of the patch, disable the squeezer extension, or set its limit to 1000. Then in the shell, run:

    for i in range(500): print(i)

The output will instantly appear in the shell.

----------
files: idlelib_buffer_output.patch
keywords: patch
messages: 406306
nosy: roger.serwy, terry.reedy
priority: normal
severity: normal
stage: patch review
status: open
title: IDLE - faster shell writing
type: enhancement
versions: Python 3.10
Added file: https://bugs.python.org/file50438/idlelib_buffer_output.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45804>
_______________________________________

From report at bugs.python.org  Sun Nov 14 03:20:37 2021
From: report at bugs.python.org (Saul Shanabrook)
Date: Sun, 14 Nov 2021 08:20:37 +0000
Subject: [New-bugs-announce] [issue45805] CO_FUTURE_ANNOTATIONS flag is not
 documented or in inspect
Message-ID: <1636878037.09.0.00718141853617.issue45805@roundup.psfhosted.org>


New submission from Saul Shanabrook <s.shanabrook at gmail.com>:

The CO_FUTURE_ANNOTATIONS code flag was added in Python 3.7 I believe in this PR https://github.com/python/cpython/pull/4390.

However, it does not appear in dis's COMPILER_FLAG_NAMES map and so is not documented in inspect or pretty printed.

I believe the fix would be to add it to the dict in dis as well as document it in inspect.

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 406309
nosy: docs at python, saulshanabrook
priority: normal
severity: normal
status: open
title: CO_FUTURE_ANNOTATIONS flag is not documented or in inspect
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45805>
_______________________________________

From report at bugs.python.org  Mon Nov 15 00:40:28 2021
From: report at bugs.python.org (Dennis Sweeney)
Date: Mon, 15 Nov 2021 05:40:28 +0000
Subject: [New-bugs-announce] [issue45806] Cannot Recover From StackOverflow
 in 3.9 Tests
Message-ID: <1636954828.71.0.201093900258.issue45806@roundup.psfhosted.org>


New submission from Dennis Sweeney <sweeney.dennis650 at gmail.com>:

In bpo-30570, David Bolen noticed that "py -3.9 -m test test_pickle" consistently crashes on Windows (even though other methods of running that test do not crash, and the test succeeds when failed tests are retried).

Curiously, it seems that adding using support.infinite_recursion *introduced* the crash rather than preventing it.

I'm guessing this would have been fixed by GH-24501, but that fix was rolled back in GH-25179 for the sake of 3.9 ABI compatibility.

As of now, 3.9's pycore_ceval.c reads:

    static inline int _Py_RecursionLimitLowerWaterMark(int limit) {
        if (limit > 200) {
            return (limit - 50);
        }
        else {
            return (3 * (limit >> 2));
        }
    }

But support.infinite_recursion(max_depth=75) has a default 75, leaving a "low-water-mark" of 54, which the recursion apparently never recovers back to in the right way.


A couple of solutions could fix this:

(1) Remove the usage of support.infinite_recursion at the test.pickletester.AbstractPickleTests.test_bad_getattr call site.

(2) Use a different value max_depth. On my machine at least, it seems 75 was a "perfect storm" to cause this issue. Using infinite_recursion(60) or 61 or ... or 74 or 76 or 77 or 78 all pass, but infinite_recursion(75) in particular fails.

(3) Use fewer calls after the overflow by inlining something like assertRaises:

             with support.infinite_recursion():
-                self.assertRaises(RuntimeError, self.dumps, x, proto)
+                try:
+                    self.dumps(x, proto)
+                except RuntimeError:
+                    pass
+                else:
+                    self.fail("RuntimeError not raised")

(5) Re-visit an ABI-compliant version of GH-24501, such as GH-25160



The output I keep getting without any changes:

> py -3.9 -m test test_pickle -v
...
test_attribute_name_interning (test.test_pickle.CPicklerTests) ... ok
test_bad_getattr (test.test_pickle.CPicklerTests) ... Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow.Python runtime state: initialized

Current thread 0x000028b0 (most recent call first):
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
  ...
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\copyreg.py", line 74 in _reduce_ex
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\test_pickle.py", line 65 in dumps
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 201 in handle
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 739 in assertRaises
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 2381 in test_bad_getattr
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\support\__init__.py", line 1770 in wrapper
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 550 in _callTestMethod
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 592 in run
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 651 in __call__
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 122 in run
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 122 in run
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 84 in __call__
  File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 122 in run

----------
components: Tests
messages: 406339
nosy: Dennis Sweeney, Mark.Shannon, lukasz.langa
priority: normal
severity: normal
status: open
title: Cannot Recover From StackOverflow in 3.9 Tests
type: crash
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45806>
_______________________________________

From report at bugs.python.org  Mon Nov 15 09:41:01 2021
From: report at bugs.python.org (Pierre Quentel)
Date: Mon, 15 Nov 2021 14:41:01 +0000
Subject: [New-bugs-announce] [issue45807] Strange SyntaxError message /
 suggestions for "@x = 123"
Message-ID: <1636987261.75.0.479908594511.issue45807@roundup.psfhosted.org>


New submission from Pierre Quentel <pierre.quentel at gmail.com>:

In CPython 3.10 :

Python 3.10.0 (tags/v3.10.0:b494f59, Oct  4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> @x = 123
  File "<stdin>", line 1
    @x = 123
     ^^^^^^^
SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?

but both suggestions lead to a SyntaxError :

>>> @x == 123
...
  File "<stdin>", line 2

    ^
SyntaxError: invalid syntax
>>> @x := 123
...
  File "<stdin>", line 2

    ^
SyntaxError: invalid syntax
>>>


Maybe an error message such as "cannot assign to decorator" would be more appropriate ?

----------
components: Parser
messages: 406351
nosy: lys.nikolaou, pablogsal, quentel
priority: normal
severity: normal
status: open
title: Strange SyntaxError message / suggestions for "@x = 123"
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45807>
_______________________________________

From report at bugs.python.org  Mon Nov 15 10:41:05 2021
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 15 Nov 2021 15:41:05 +0000
Subject: [New-bugs-announce] [issue45808] configure check for HAVE_CRYPT_R
 is wrong
Message-ID: <1636990865.28.0.153956179368.issue45808@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

The configure check for HAVE_CRYPT_R assumes that all platforms have <crypt.h>. Some platforms like FreeBSD don't have <crypt.h> at all. Instead the crypt_r() prototype is defined in <unistd.h>.

----------
messages: 406352
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: configure check for HAVE_CRYPT_R is wrong
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45808>
_______________________________________

From report at bugs.python.org  Mon Nov 15 14:02:38 2021
From: report at bugs.python.org (Sam Gross)
Date: Mon, 15 Nov 2021 19:02:38 +0000
Subject: [New-bugs-announce] [issue45809] Race condition in
 WeakKeyDictionary/WeakKeyDictionary
Message-ID: <1637002958.39.0.243945007085.issue45809@roundup.psfhosted.org>


New submission from Sam Gross <colesbury at gmail.com>:

The issue described issue7105 (and maybe issue7060) still exists due to a race condition in WeakKeyDictionary. This shows up as test failure that looks like:

  test test_weakref failed -- Traceback (most recent call last):
    File "Lib/test/test_weakref.py", line 1960, in test_threaded_weak_value_dict_deepcopy
      self.check_threaded_weak_dict_copy(weakref.WeakValueDictionary, True)
    File "Lib/test/test_weakref.py", line 1940, in check_threaded_weak_dict_copy
      raise exc[0]
    File "Lib/test/test_weakref.py", line 1897, in dict_copy
      _ = copy.deepcopy(d)
    File "Lib/copy.py", line 153, in deepcopy
      y = copier(memo)
    File "Lib/weakref.py", line 189, in __deepcopy__
      for key, wr in self.data.items():
  RuntimeError: dictionary changed size during iteration

The cause is that the check of "self._iterating" and the call to "_atomic_removal" are not performed atomically together. By the time _atomic_removal() is called, an iteration might have already started.

https://github.com/python/cpython/blob/ec382fac0db6d9159c2d3496a70b7a605545957e/Lib/weakref.py#L109-L114

----------
components: Library (Lib)
messages: 406357
nosy: colesbury
priority: normal
severity: normal
status: open
title: Race condition in WeakKeyDictionary/WeakKeyDictionary
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45809>
_______________________________________

From report at bugs.python.org  Mon Nov 15 18:11:24 2021
From: report at bugs.python.org (Steven D'Aprano)
Date: Mon, 15 Nov 2021 23:11:24 +0000
Subject: [New-bugs-announce] [issue45810] Prohibit invisible control
 characters in string literals and comments
Message-ID: <1637017884.02.0.311625820684.issue45810@roundup.psfhosted.org>


New submission from Steven D'Aprano <steve+python at pearwood.info>:

Currently invisible control characters aside from whitespace (tabs, newlines, formfeeds, carriage returns) are prohibited outside of comments and string literals. As discussed in this thread:

https://mail.python.org/archives/list/python-dev at python.org/message/DN24FK3A2DSO4HBGEDGJXERSAUYK6VK6/

we should ban C0 and C1 control characters (aside from \t\n\f\r) in string literals and comments too.

To be clear, the ban is on actual invisible control characters, not escape sequences.

----------
components: Interpreter Core
messages: 406370
nosy: serhiy.storchaka, steven.daprano
priority: normal
severity: normal
status: open
title: Prohibit invisible control characters in string literals and comments
type: security
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45810>
_______________________________________

From report at bugs.python.org  Mon Nov 15 18:52:37 2021
From: report at bugs.python.org (Steven D'Aprano)
Date: Mon, 15 Nov 2021 23:52:37 +0000
Subject: [New-bugs-announce] [issue45811] Improve error message when source
 code contains invisible control characters
Message-ID: <1637020357.3.0.163492557929.issue45811@roundup.psfhosted.org>


New submission from Steven D'Aprano <steve+python at pearwood.info>:

Invisible control characters (aside from white space) are not permitted in source code, but the syntax error we get is confusing and lacks information:

>>> s = 'print\x17("Hello")'
>>> eval(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    print("Hello")
         ^
SyntaxError: invalid syntax


The caret points to an invisible character. The offending control character is not visible in the traceback, or the source code unless you use a hex editor. Copying and pasting the string from the traceback, or the source code, may remove the control character (depending on the tools you use), making it even harder to track down the problem.

I suggest that the syntax error should state that the problem is an invisible control character, and display it as a standard human-readable code together with its hex code:

SyntaxError: invisible control character ^W (0x17)


Just in case it isn't obvious what the mapping between controls and the human visible string is:

def control(char):
    n = ord(char)
    if 0 <= n <= 0x1F:
        # C0 control codes
        return '^' + chr(ord('@')+n)
    elif n == 0x7F:
        # DEL
        return '^?'
    elif 0x80 <= n <= 0x9F:
        # C1 control codes
        return 'Esc+' + chr(ord('@')+n-0x80)
    else:
        raise ValueError('Not a control character.')


https://en.wikipedia.org/wiki/C0_and_C1_control_codes

----------
components: Interpreter Core
messages: 406379
nosy: steven.daprano
priority: normal
severity: normal
status: open
title: Improve error message when source code contains invisible control characters
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45811>
_______________________________________

From report at bugs.python.org  Mon Nov 15 19:47:43 2021
From: report at bugs.python.org (Andre Roberge)
Date: Tue, 16 Nov 2021 00:47:43 +0000
Subject: [New-bugs-announce] [issue45812] SystemError when using
 code.interact
Message-ID: <1637023663.78.0.877430736208.issue45812@roundup.psfhosted.org>


New submission from Andre Roberge <andre.roberge at gmail.com>:

In the following, I execute a single invalid statement in the Python interpreter, which gives the correct error message. Afterwards, I repeat this example using code.interact(), generating a different traceback. This issue affects IDLE differently as shown in the attached file: the statement is never considered to be complete.

===
Python 3.11.0a2 (tags/v3.11.0a2:e2b4e4b, Nov  5 2021, 20:00:05) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> a = 3 \ 4
  File "<stdin>", line 1
    a = 3 \ 4
            ^
SyntaxError: unexpected character after line continuation character
>>> import code
>>> code.interact()
Python 3.11.0a2 (tags/v3.11.0a2:e2b4e4b, Nov  5 2021, 20:00:05) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> a = 3 \ 4
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\andre\AppData\Local\Programs\Python\Python311\lib\code.py", line 301, in interact
    console.interact(banner, exitmsg)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\andre\AppData\Local\Programs\Python\Python311\lib\code.py", line 232, in interact
    more = self.push(line)
           ^^^^^^^^^^^^^^^
  File "C:\Users\andre\AppData\Local\Programs\Python\Python311\lib\code.py", line 258, in push
    more = self.runsource(source, self.filename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\andre\AppData\Local\Programs\Python\Python311\lib\code.py", line 63, in runsource
    code = self.compile(source, filename, symbol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\andre\AppData\Local\Programs\Python\Python311\lib\codeop.py", line 185, in __call__
    return _maybe_compile(self.compiler, source, filename, symbol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\andre\AppData\Local\Programs\Python\Python311\lib\codeop.py", line 96, in _maybe_compile
    code2 = compiler(source + "\n\n", filename, symbol)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\andre\AppData\Local\Programs\Python\Python311\lib\codeop.py", line 150, in __call__
    codeob = compile(source, filename, symbol, self.flags, True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SystemError: Negative size passed to PyUnicode_New

----------
components: Parser
files: idle_problem.png
messages: 406381
nosy: aroberge, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: SystemError when using code.interact
versions: Python 3.11
Added file: https://bugs.python.org/file50441/idle_problem.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45812>
_______________________________________

From report at bugs.python.org  Tue Nov 16 03:44:12 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Tue, 16 Nov 2021 08:44:12 +0000
Subject: [New-bugs-announce] [issue45813] Importing asyncio after deleting a
 coroutine object and before cleaning it up leads to crashing on Python3.11
Message-ID: <1637052252.29.0.850993765277.issue45813@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

The following crashing can only reproduce on Python3.11.  In this case, we import "asyncio" after deleting a coroutine object and before cleaning it up, leading to crashing. 


test.py
=======================
async def f():
    pass
f = f()
frame = f.cr_frame
del f

import asyncio

frame.clear()
======================

>>>Python3.11 -Werror test.py
Exception ignored in: <coroutine object f at 0x7f860ad9cdd0>
Traceback (most recent call last):
  File "python311/Lib/warnings.py", line 506, in _warn_unawaited_coroutine
    warn(msg, category=RuntimeWarning, stacklevel=2, source=coro)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeWarning: coroutine 'f' was never awaited
Segmentation fault (core dumped)



Version: Python 3.11.0a2+ on Ubuntu 16.04

----------
components: asyncio
messages: 406388
nosy: asvetlov, xxm, yselivanov
priority: normal
severity: normal
status: open
title: Importing asyncio after deleting a coroutine object and before cleaning it up leads to crashing on Python3.11
type: crash
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45813>
_______________________________________

From report at bugs.python.org  Tue Nov 16 05:40:29 2021
From: report at bugs.python.org (Yevhenii Hyzyla)
Date: Tue, 16 Nov 2021 10:40:29 +0000
Subject: [New-bugs-announce] [issue45814] Use same function signature for
 datetime.time.strftime
Message-ID: <1637059229.27.0.494506093671.issue45814@roundup.psfhosted.org>


New submission from Yevhenii Hyzyla <hyzyla at gmail.com>:

Method datetime.time.strftime in Python and C implementation has different signature. Python implementation use `fmt` as argument and C implementation use `format` as argument. 

Python implementation:
```python
def strftime(self, fmt):
  ...
```

C implementation
```C
static PyObject *
time_strftime(PyDateTime_Time *self, PyObject *args, PyObject *kw)
{
    PyObject *result;
    PyObject *tuple;
    PyObject *format;
    static char *keywords[] = {"format", NULL};

    if (! PyArg_ParseTupleAndKeywords(args, kw, "U:strftime", keywords,
                                      &format))
        return NULL;
```

My suggestion to use the same argument name for both implementations, for example to use `format` as in C implementation.

----------
components: Library (Lib)
messages: 406395
nosy: hyzyla
priority: normal
severity: normal
status: open
title: Use same function signature for datetime.time.strftime
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45814>
_______________________________________

From report at bugs.python.org  Tue Nov 16 05:46:44 2021
From: report at bugs.python.org (Dimitri Papadopoulos Orfanos)
Date: Tue, 16 Nov 2021 10:46:44 +0000
Subject: [New-bugs-announce] [issue45815] Document exceptions raised by
 fnmtach
Message-ID: <1637059604.78.0.155105501767.issue45815@roundup.psfhosted.org>


New submission from Dimitri Papadopoulos Orfanos <dimitri.papadopoulos at cea.fr>:

The fnmatch documentation should explicitly mention the type of exceptions raised by fnmatch.fnmatch():
https://docs.python.org/3/library/fnmatch.html

In my case it raised sre_constants.error, and it took some time to understand that the proper way to catch this type of exceptions is to catch the re.error superclass, by reading https://bugs.python.org/issue795379.

Actually that would be the case for any module using the re module under the hood, possibly passing an ill-formed regex to a re function.

----------
assignee: docs at python
components: Documentation, Library (Lib), Regular Expressions
files: sre_constants.error_stderr.txt
messages: 406396
nosy: DimitriPapadopoulosOrfanos, docs at python, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Document exceptions raised by fnmtach
versions: Python 3.11
Added file: https://bugs.python.org/file50442/sre_constants.error_stderr.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45815>
_______________________________________

From report at bugs.python.org  Tue Nov 16 09:35:04 2021
From: report at bugs.python.org (theeshallnotknowethme)
Date: Tue, 16 Nov 2021 14:35:04 +0000
Subject: [New-bugs-announce] [issue45816] Python does not support building
 with Visual Studio 2022
Message-ID: <1637073304.41.0.776528442295.issue45816@roundup.psfhosted.org>


New submission from theeshallnotknowethme <nohackingofkrowten at gmail.com>:

Python does not support building with Visual Studio 2022, which is the latest officially released stable version [1].

[1] https://devblogs.microsoft.com/visualstudio/visual-studio-2022-now-available/

----------
components: Build
messages: 406405
nosy: February291948
priority: normal
severity: normal
status: open
title: Python does not support building with Visual Studio 2022
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45816>
_______________________________________

From report at bugs.python.org  Tue Nov 16 10:21:23 2021
From: report at bugs.python.org (Aivar Annamaa)
Date: Tue, 16 Nov 2021 15:21:23 +0000
Subject: [New-bugs-announce] [issue45817] Build Windows and macOS installers
 with Tcl/Tk 8.6.12
Message-ID: <1637076083.27.0.779565986622.issue45817@roundup.psfhosted.org>


New submission from Aivar Annamaa <aivar.annamaa at gmail.com>:

New version contains several bugfixes: https://github.com/tcltk/tk/blob/8baf7d337ca0aab7fafb0e670927ab2c0200e80a/changes#L7869

----------
assignee: terry.reedy
components: IDLE, Tkinter, Windows, macOS
messages: 406409
nosy: aivarannamaa, ned.deily, paul.moore, ronaldoussoren, steve.dower, terry.reedy, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Build Windows and macOS installers with Tcl/Tk 8.6.12
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45817>
_______________________________________

From report at bugs.python.org  Tue Nov 16 11:21:51 2021
From: report at bugs.python.org (Sergey M.)
Date: Tue, 16 Nov 2021 16:21:51 +0000
Subject: [New-bugs-announce] [issue45818] socketserver.BaseRequestHandler
 inherited class
Message-ID: <1637079711.69.0.730324544936.issue45818@roundup.psfhosted.org>


New submission from Sergey M. <seregaxvm.main at gmail.com>:

Due to 
```python
try:
    self.handle()
finally:
    self.finish()
```
construct in `socketserver.BaseRequestHandler.__init__()` method inherited classes with `overrided __init__()` method may suffer from incomplete initialization.
For example, in the following snippet
```python
def __init__(self, request, client_address, server):
    super().__init__(request, client_address, server)
    self.foo = 1
```
in some cases all the code after `super()` call will not be executed.

This is a MWE of the server with partially initialized Handler class
```python
from socketserver import UnixStreamServer, StreamRequestHandler, ForkingMixIn


class Handler(StreamRequestHandler):
    def __init__(self, request, client_address, server):
        super().__init__(request, client_address, server)
        self.foo = 1

    def handle(self):
        print(self.foo)


class ThreadedUnixStreamServer(ForkingMixIn, UnixStreamServer):
    pass


with ThreadedUnixStreamServer("/tmp/test.socket", Handler) as server:
    server.serve_forever()
```

----------
components: Library (Lib)
messages: 406413
nosy: matsievskiysv
priority: normal
severity: normal
status: open
title: socketserver.BaseRequestHandler inherited class
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45818>
_______________________________________

From report at bugs.python.org  Tue Nov 16 13:51:40 2021
From: report at bugs.python.org (Jim Crist-Harif)
Date: Tue, 16 Nov 2021 18:51:40 +0000
Subject: [New-bugs-announce] [issue45819] Avoid releasing the GIL in
 nonblocking socket operations
Message-ID: <1637088700.35.0.697825575179.issue45819@roundup.psfhosted.org>


New submission from Jim Crist-Harif <jcristharif at gmail.com>:

In https://bugs.python.org/issue7946 an issue with how the current GIL interacts with mixing IO and CPU bound work. Quoting this issue:

> when an I/O bound thread executes an I/O call,
> it always releases the GIL.  Since the GIL is released, a CPU bound
> thread is now free to acquire the GIL and run.  However, if the I/O
> call completes immediately (which is common), the I/O bound thread
> immediately stalls upon return from the system call.  To get the GIL
> back, it now has to go through the timeout process to force the
> CPU-bound thread to release the GIL again.

This issue can come up in any application where IO and CPU bound work are mixed (we've found it to be a cause of performance issues in https://dask.org for example). Fixing the general problem is tricky and likely requires changes to the GIL's internals, but in the specific case of mixing asyncio running in one thread and CPU work happening in background threads, there may be a simpler fix - don't release the GIL if we don't have to.

Asyncio relies on nonblocking socket operations, which by definition shouldn't block. As such, releasing the GIL shouldn't be needed for many operations (`send`, `recv`, ...) on `socket.socket` objects provided they're in nonblocking mode (as suggested in https://bugs.python.org/issue7946#msg99477). Likewise, dropping the GIL can be avoided when calling `select` on `selectors.BaseSelector` objects with a timeout of 0 (making it a non-blocking call).

I've made a patch (https://github.com/jcrist/cpython/tree/keep-gil-for-fast-syscalls) with these two changes, and run a benchmark (attached) to evaluate the effect of background threads with/without the patch. The benchmark starts an asyncio server in one process, and a number of clients in a separate process. A number of background threads that just spin are started in the server process (configurable by the `-t` flag, defaults to 0), then the server is loaded to measure the RPS.

Here are the results:

```
# Main branch
$ python bench.py -c1 -t0
Benchmark: clients = 1, msg-size = 100, background-threads = 0
16324.2 RPS
$ python bench.py -c1 -t1
Benchmark: clients = 1, msg-size = 100, background-threads = 1
Spinner spun 1.52e+07 cycles/second
97.6 RPS
$ python bench.py -c2 -t0
Benchmark: clients = 2, msg-size = 100, background-threads = 0
31308.0 RPS
$ python bench.py -c2 -t1
Benchmark: clients = 2, msg-size = 100, background-threads = 1
Spinner spun 1.52e+07 cycles/second
96.2 RPS
$ python bench.py -c10 -t0
Benchmark: clients = 10, msg-size = 100, background-threads = 0
47169.6 RPS
$ python bench.py -c10 -t1
Benchmark: clients = 10, msg-size = 100, background-threads = 1
Spinner spun 1.54e+07 cycles/second
95.4 RPS

# With this patch
$ ./python bench.py -c1 -t0
Benchmark: clients = 1, msg-size = 100, background-threads = 0
18201.8 RPS
$ ./python bench.py -c1 -t1
Benchmark: clients = 1, msg-size = 100, background-threads = 1
Spinner spun 9.03e+06 cycles/second
194.6 RPS
$ ./python bench.py -c2 -t0
Benchmark: clients = 2, msg-size = 100, background-threads = 0
34151.8 RPS
$ ./python bench.py -c2 -t1
Benchmark: clients = 2, msg-size = 100, background-threads = 1
Spinner spun 8.72e+06 cycles/second
729.6 RPS
$ ./python bench.py -c10 -t0
Benchmark: clients = 10, msg-size = 100, background-threads = 0
53666.6 RPS
$ ./python bench.py -c10 -t1
Benchmark: clients = 10, msg-size = 100, background-threads = 1
Spinner spun 5e+06 cycles/second
21838.2 RPS
```

A few comments on the results:

- On the main branch, any GIL contention sharply decreases the number of RPS an asyncio server can handle, regardless of the number of clients. This makes sense - any socket operation will release the GIL, and the server thread will have to wait to reacquire it (up to the switch interval), rinse and repeat. So if every request requires 1 recv and 1 send, a server with background GIL contention is stuck at a max of `1 / (2 * switchinterval)` or 200 RPS with default configuration. This effectively prioritizes the background thread over the IO thread, since the IO thread releases the GIL very frequently and the background thread never does.

- With the patch, we still see a performance degradation, but the degradation is less severe and improves with the number of clients. This is because with these changes the asyncio thread only releases the GIL when doing a blocking poll for new IO events (or when the switch interval is hit). With low load (1 client), the IO thread becomes idle more frequently and releases the GIL. Under higher load though the event loop frequently still has work to do at the end of a cycle and issues a `selector.select` call with a 0 timeout (nonblocking), avoiding releasing the GIL at all during that loop (note the nonlinear effect of adding more clients). Since the IO thread still releases the GIL sometimes, the background thread still holds the GIL a larger percentage of the time than the IO thread, but the difference between them is less severe than without this patch.

I have also tested this patch on a Dask cluster running some real-world problems and found that it did improve performance where IO was throttled due to GIL contention.

----------
components: C API, IO
files: bench.py
messages: 406422
nosy: jcristharif
priority: normal
severity: normal
status: open
title: Avoid releasing the GIL in nonblocking socket operations
type: performance
versions: Python 3.11
Added file: https://bugs.python.org/file50443/bench.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45819>
_______________________________________

From report at bugs.python.org  Tue Nov 16 13:58:10 2021
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Tue, 16 Nov 2021 18:58:10 +0000
Subject: [New-bugs-announce] [issue45820] Parser can segfault if an error
 happens before reading any input
Message-ID: <1637089090.21.0.74869570109.issue45820@roundup.psfhosted.org>


Change by Pablo Galindo Salgado <pablogsal at gmail.com>:


----------
nosy: pablogsal
priority: normal
severity: normal
status: open
title: Parser can segfault if an error happens before reading any input

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45820>
_______________________________________

From report at bugs.python.org  Tue Nov 16 14:21:08 2021
From: report at bugs.python.org (Alex Waygood)
Date: Tue, 16 Nov 2021 19:21:08 +0000
Subject: [New-bugs-announce] [issue45821] Many method parameters in the
 datetime module are positional-only in the C implementation but
 positional-or-keyword in the Python implementation
Message-ID: <1637090468.13.0.603790423029.issue45821@roundup.psfhosted.org>


New submission from Alex Waygood <Alex.Waygood at Gmail.com>:

The following methods have parameters that are positional-only in the C implementation of the `datetime` module, but positional-or-keyword in the pure-Python implementation:

* tzinfo.tzname
* tzinfo.utcoffset
* tzinfo.dst
* tzinfo.fromutc
* date.fromordinal
* date.fromisoformat
* date.__format__
* date.__le__
* date.__lt__
* date.__ge__
* date.__gt__
* date.__add__
* date.__radd__
* date.__sub__
* date.__reduce_ex__
* time.__le__
* time.__lt__
* time.__ge__
* time.__gt__
* time.fromisoformat
* time.__format__
* timedelta.__add__
* timedelta.__radd__
* timedelta.__sub__
* timedelta.__rsub__
* timedelta.__mul__
* timedelta.__rmul__
* timedelta.__floordiv__
* timedelta.__truediv__
* timedelta.__mod__
* timedelta.__divmod__
* timedelta.__le__
* timedelta.__lt__
* timedelta.__ge__
* timedelta.__gt__
* datetime.utcfromtimestamp
* datetime.fromisoformat
* datetime.strptime
* datetime.__le__
* datetime.__lt__
* datetime.__ge__
* datetime.__gt__

These inconsistencies make it extremely difficult to provide an accurate stub for these methods in typeshed.

----------
components: Extension Modules, Library (Lib)
messages: 406423
nosy: AlexWaygood, belopolsky, p-ganssle
priority: normal
severity: normal
status: open
title: Many method parameters in the datetime module are positional-only in the C implementation but positional-or-keyword in the Python implementation
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45821>
_______________________________________

From report at bugs.python.org  Tue Nov 16 14:39:26 2021
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Tue, 16 Nov 2021 19:39:26 +0000
Subject: [New-bugs-announce] [issue45822] Py_CompileString does not respect
 the coding cookie with the new parser if flags are empty
Message-ID: <1637091566.73.0.796016842609.issue45822@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

When executing Py_CompileString with a source string that has a coding cookie, this is not respected as with the old parser.

----------
components: Parser
messages: 406425
nosy: lys.nikolaou, pablogsal, twouters
priority: normal
severity: normal
status: open
title: Py_CompileString does not respect the coding cookie with the new parser if flags are empty
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45822>
_______________________________________

From report at bugs.python.org  Tue Nov 16 14:41:01 2021
From: report at bugs.python.org (Roxana 7)
Date: Tue, 16 Nov 2021 19:41:01 +0000
Subject: [New-bugs-announce] [issue45823] phyton stopped working
Message-ID: <1637091661.28.0.129849432506.issue45823@roundup.psfhosted.org>


New submission from Roxana 7 <vicentpaul869 at gmail.com>:

Hello everyone, I try to program a simple calculator for a university project but the sotfware always get an error window "phyton stopped working" when i add it on a IDE.
I have reinstalled it so many times and it's always the same

----------
components: Windows
messages: 406426
nosy: paul.moore, steve.dower, tim.golden, vicentpaul869, zach.ware
priority: normal
severity: normal
status: open
title: phyton stopped working
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45823>
_______________________________________

From report at bugs.python.org  Tue Nov 16 17:15:53 2021
From: report at bugs.python.org (Wesley Altham)
Date: Tue, 16 Nov 2021 22:15:53 +0000
Subject: [New-bugs-announce] [issue45824] CSV module document does not
 include how to append files
Message-ID: <1637100953.03.0.703206320049.issue45824@roundup.psfhosted.org>


New submission from Wesley Altham <wesley.altham at mga.edu>:

easy- The CSV module does not include how to append a file which is using 'a' and not 'w' or 'r'. There might be more to appending but it is not documented for me to know, I saw a stackoverflow post about it when I looked and I tried it and worked. I hope whoever knows the CSV module could document append at all.

----------
messages: 406438
nosy: wesrl
priority: normal
severity: normal
status: open
title: CSV module document does not include how to append files
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45824>
_______________________________________

From report at bugs.python.org  Tue Nov 16 20:55:46 2021
From: report at bugs.python.org (Bill Borskey)
Date: Wed, 17 Nov 2021 01:55:46 +0000
Subject: [New-bugs-announce] [issue45825] Heap Segmentation Fault
Message-ID: <1637114146.37.0.859649606626.issue45825@roundup.psfhosted.org>


New submission from Bill Borskey <wborskey at gmail.com>:

Dereferencing a python object that does not exist through ctypes caused a segfault in pymalloc_alloc. 

rdi: 0x0000000000000006  rsi: 0x0000000000000006 

0   org.python.python             	0x00000001081ee277 pymalloc_alloc + 74
1   org.python.python             	0x00000001081ed3dc _PyObject_Malloc + 17
2   org.python.python             	0x0000000108296a94 normalizestring + 55
3   org.python.python             	0x0000000108296540 _PyCodec_Lookup + 76

Looks like six was passed to the allocator, not sure exactly what was written to it, or if I can control that. But I guess it was more than six bytes. I don't have the id I used with the debug info above, but: I had a smallish list of characters, overwrote it with zeros, called del on it. I checked the value a couple times with ctypes. What I think happened is the garbage collector finally reclaimed the memory and crashed when I dereferenced again with the bad value.

I checked it out using 0xCC to get an idea of where the value was landing, and I'm overwriting rbx directly. But it doesn't get as far as above and segfaults at O_get rather than the allocator. 

I looked and I see this function:

static PyObject *
O_get(void *ptr, Py_ssize_t size)
{
    PyObject *ob = *(PyObject **)ptr;
    if (ob == NULL) {
        if (!PyErr_Occurred())
            /* Set an error if not yet set */
            PyErr_SetString(PyExc_ValueError,
                            "PyObject is NULL");
        return NULL;
    }
    Py_INCREF(ob);
    return ob;
}

Seems like the code is increasing the reference count on the non-existing python object at 0xCCCCCCCCCCCCCCCC and writing out of bounds. 

$ python3 -X dev
Python 3.9.2 (default, Mar 26 2021, 23:27:12) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> ctypes.cast(0xCCCCCCCCCCCCCCCC, ctypes.py_object).value
Fatal Python error: Segmentation fault

(below does not have the heap debugger on it seemed to mess with results)

Current thread 0x0000000104b83dc0 (most recent call first):
  File "<stdin>", line 1 in <module>
Segmentation fault: 11

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x000000010e1be8c0  rbx: 0xcccccccccccccccc  rcx: 0x000000010e2775c9  rdx: 0x000000010e283890
  rdi: 0x000000010e1be8c0  rsi: 0x0000000000000008  rbp: 0x00007ffee20f24e0  rsp: 0x00007ffee20f24d0
   r8: 0x0000000000000004   r9: 0x6a2bff3e46d2619c  r10: 0x000000010e1d4b80  r11: 0x000000010e1d4bb8
  r12: 0x000000010defc5e0  r13: 0x00007f94edc5c390  r14: 0x000000010e1e1b90  r15: 0x0000000000000000
  rip: 0x000000010e2775d7  rfl: 0x0000000000010286  cr2: 0x000000010e2730f3

----------
components: ctypes
files: debug.txt
messages: 406443
nosy: thewb
priority: normal
severity: normal
status: open
title: Heap Segmentation Fault
type: crash
versions: Python 3.9
Added file: https://bugs.python.org/file50444/debug.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45825>
_______________________________________

From report at bugs.python.org  Tue Nov 16 21:50:58 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Wed, 17 Nov 2021 02:50:58 +0000
Subject: [New-bugs-announce] [issue45826] unittest.assertRaisesRegex is
 broken in Python 3.11 and leading to crashing if tested regex does not
 match name.
Message-ID: <1637117458.73.0.114142270952.issue45826@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

In Python 3.11, unittest.assertRaisesRegex is broken and leading to crashing if tested regex does not match name. See the following example:

test.py
=========================================
import unittest

class uTest(unittest.TestCase):
	pass

uTest = uTest()

with uTest.assertRaisesRegex(Exception, 'aaa'):
     aab
=========================================


Output in Python3.9.2, 3.10:
--------------------------------------
NameError: name 'aab' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xxm/Desktop/test.py", line 29, in <module>
    aab
  File "/usr/local/python310/lib/python3.10/unittest/case.py", line 239, in __exit__
    self._raiseFailure('"{}" does not match "{}"'.format(
  File "/usr/local/python310/lib/python3.10/unittest/case.py", line 163, in _raiseFailure
    raise self.test_case.failureException(msg)
AssertionError: "aaa" does not match "name 'aab' is not defined
--------------------------------------------------

Actual output in Python3.11.0a1,Python3.11.0a2:
Segmentation fault (core dumped)

System: Ubuntu 16.04

----------
components: Library (Lib)
messages: 406445
nosy: xxm
priority: normal
severity: normal
status: open
title: unittest.assertRaisesRegex is broken in Python 3.11 and leading to crashing if tested regex does not match name.
type: crash
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45826>
_______________________________________

From report at bugs.python.org  Tue Nov 16 23:28:37 2021
From: report at bugs.python.org (Jamie Chaisson)
Date: Wed, 17 Nov 2021 04:28:37 +0000
Subject: [New-bugs-announce] [issue45827] Unittest - commenting passing
 tests cause previous failing tests to pass
Message-ID: <1637123317.28.0.291429551369.issue45827@roundup.psfhosted.org>


New submission from Jamie Chaisson <chaissoj at oregonstate.edu>:

Ubuntu Release 20.04.3 LTS (Focal Fossa) 64-bit
Using unittest, testing with assertEqual on int values and known good output. Unittest produces one-off error on handful of edge-case tests causing assert to fail. Commenting out passing assertEquals tests to isolate failing tests causes previous failing tests to pass (reproducible). All input values produce output values when testing by hand, only different when using unittest.

Pardon the homework. Removing the class and moving member variables into my_datetime function resolves the issue; however, the behavior as written should not be happening.

----------
components: Tests
files: tests.py
messages: 406449
nosy: nuse
priority: normal
severity: normal
status: open
title: Unittest - commenting passing tests cause previous failing tests to pass
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file50446/tests.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45827>
_______________________________________

From report at bugs.python.org  Wed Nov 17 04:42:19 2021
From: report at bugs.python.org (Erlend E. Aasland)
Date: Wed, 17 Nov 2021 09:42:19 +0000
Subject: [New-bugs-announce] [issue45828] [sqlite3] use unraisable
 exceptions in callbacks
Message-ID: <1637142139.17.0.325447635638.issue45828@roundup.psfhosted.org>


New submission from Erlend E. Aasland <erlend.aasland at innova.no>:

In order to print tracebacks from exceptions in SQLite callbacks, the sqlite3 extension module provides sqlite3.enable_callback_tracebacks(flag). Setting the flag to True instructs the sqlite3 extension module to PyErr_Print() if an exception occurs during a callback. Else, PyErr_Clear() is called.

>From the sqlite3.enable_callback_tracebacks() docs:

    By default you will not get any tracebacks in user-defined functions,
    aggregates, converters, authorizer callbacks etc. If you want to debug
    them, you can call this function with flag set to True. Afterwards, you
    will get tracebacks from callbacks on sys.stderr. Use False to disable the
    feature again.


Few other exceptions use a similar approach:

    $ grep -r PyErr_Print Modules 
    Modules/_tkinter.c:        PyErr_Print();
    Modules/_testcapimodule.c:        PyErr_Print();
    Modules/main.c:    PyErr_Print();
    Modules/main.c:        PyErr_Print();
    Modules/main.c:        PyErr_Print();
    Modules/_io/bytesio.c:        PyErr_Print();
    Modules/_sqlite/connection.c:        PyErr_Print();
    Modules/_sqlite/cursor.c:                    PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_ctypes/callbacks.c:    PyErr_Print();


We get a higher hit for unraisable exceptions:

    $ grep -r PyErr_WriteUnraisable Modules | wc -l
      45


AFAICS, using unraisable exceptions is a better approach.


Current behaviour:

    Python 3.10.0 (v3.10.0:b494f5935c, Oct  4 2021, 14:59:20) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sqlite3
    >>> cx = sqlite3.connect(":memory:")
    >>> cx.set_trace_callback(lambda stmt: 5/0)
    >>> cx.execute("select 1")
    <sqlite3.Cursor object at 0x109bba2c0>
    >>> sqlite3.enable_callback_tracebacks(True)
    >>> cx.execute("select 1")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <lambda>
    ZeroDivisionError: division by zero
    <sqlite3.Cursor object at 0x1099046c0>


With unraisable exceptions:

    Python 3.11.0a2+ (heads/sqlite-unraisable-exceptions-dirty:de29590d6a, Nov 17 2021, 10:29:19) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sqlite3
    >>> cx = sqlite3.connect(":memory:")
    >>> cx.set_trace_callback(lambda stmt: 5/0)
    >>> cx.execute("select 1")
    <sqlite3.Cursor object at 0x10b1fe720>
    >>> sqlite3.enable_callback_tracebacks(True)
    >>> cx.execute("select 1")
    Exception ignored in: <function <lambda> at 0x10b4e3ee0>
    Traceback (most recent call last):
      File "<stdin>", line 1, in <lambda>
    ZeroDivisionError: division by zero
    <sqlite3.Cursor object at 0x10b1fe840>


The user experience is mostly unchanged; we get one extra line, telling us that the exception was ignored. Also, users can now use sys.unraisablehook:

    >>> sys.unraisablehook = lambda unraisable: print(unraisable)
    >>> cx.execute("select 1")
    UnraisableHookArgs(exc_type=<class 'ZeroDivisionError'>,     exc_value=ZeroDivisionError('division by zero'), exc_traceback=<traceback object at 0x10b559900>, err_msg=None, object=<function <lambda> at 0x10b4e3ee0>)
    <sqlite3.Cursor object at 0x10b1fe840>


The only question I have, is if we should deprecate sqlite3.enable_callback_tracebacks() after switching to unraisable exceptions.

----------
components: Extension Modules
messages: 406459
nosy: erlendaasland
priority: normal
severity: normal
status: open
title: [sqlite3] use unraisable exceptions in callbacks
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45828>
_______________________________________

From report at bugs.python.org  Wed Nov 17 04:50:11 2021
From: report at bugs.python.org (Mark Shannon)
Date: Wed, 17 Nov 2021 09:50:11 +0000
Subject: [New-bugs-announce] [issue45829] Remove C stack use by specializing
 BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR
Message-ID: <1637142611.15.0.521267590311.issue45829@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

We can remove the C stack use and general overhead of calling special methods implemented in Python for attribute access and indexing.

Each operation has a special method that implements it. When that special method is implemented in Python, we should avoid the `tp_xxx` slot machinery and use the same mechanism we use for normal calls to Python functions. 

* BINARY_SUBSCR: `__getitem__`
* STORE_SUBSCR: `__setitem__`
* LOAD_ATTR: `__getattribute__` (and maybe `__getattr__`)
* STORE_ATTR: `__setattr__`

It probably isn't worth bothering with the deletion forms.

The getters (`__getitem__` and `__getattribute__`) are relatively simple, as the call returns the result.

The setters are a bit more complicated as the return value needs to be discarded, so an additional frame which discards the result of the call needs to be inserted.

----------
assignee: Mark.Shannon
components: Interpreter Core
messages: 406461
nosy: Mark.Shannon, brandtbucher, pablogsal
priority: normal
severity: normal
status: open
title: Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45829>
_______________________________________

From report at bugs.python.org  Wed Nov 17 11:42:28 2021
From: report at bugs.python.org (Douglas Raillard)
Date: Wed, 17 Nov 2021 16:42:28 +0000
Subject: [New-bugs-announce] [issue45830] Custom pickler memory leak
Message-ID: <1637167348.82.0.904300950392.issue45830@roundup.psfhosted.org>


New submission from Douglas Raillard <douglas.raillard at arm.com>:

The following script exhibits the memory leak. It only happens if "dispatch_table" is set _before_ calling super().__init__, which is pretty unexpected.


    import pickle
    import io
    import gc
    import tracemalloc

    tracemalloc.start(10)
    snap = tracemalloc.take_snapshot()

    class MyPickler(pickle.Pickler):
        def __init__(self, *args, **kwargs):
            # Swapping the next 2 lines "solves" the memory leak for some reason
            self.dispatch_table = dict()
            super().__init__(*args, **kwargs)

    l=[]
    for i in range(10000):
        if i % 1000 == 0:
            print('='*80)
            snap2 = tracemalloc.take_snapshot()
            stats=snap2.compare_to(snap, 'lineno')
            for s in stats[:10]:
                print(s)
            snap = snap2

        f = io.BytesIO()
        MyPickler(f)
        gc.collect()


The output of the last iteration is as follow. The leak of 562 kiB is apparent:

	testmem.py:12: size=562 KiB (+62.5 KiB), count=9000 (+1000), average=64 B
	/usr/lib/python3.10/tracemalloc.py:125: size=2376 B (-72 B), count=33 (-1), average=72 B
	/usr/lib/python3.10/tracemalloc.py:129: size=72 B (+72 B), count=1 (+1), average=72 B
	/usr/lib/python3.10/tracemalloc.py:502: size=252 B (+28 B), count=9 (+1), average=28 B
	/usr/lib/python3.10/tracemalloc.py:498: size=2104 B (+0 B), count=36 (+0), average=58 B
	/home/dourai01/Work/lisa/lisa/testmem.py:10: size=1844 B (+0 B), count=9 (+0), average=205 B
	/usr/lib/python3.10/tracemalloc.py:193: size=1680 B (+0 B), count=35 (+0), average=48 B
	/usr/lib/python3.10/tracemalloc.py:547: size=1256 B (+0 B), count=3 (+0), average=419 B
	/usr/lib/python3.10/tracemalloc.py:226: size=832 B (+0 B), count=2 (+0), average=416 B
	/usr/lib/python3.10/tracemalloc.py:173: size=800 B (+0 B), count=2 (+0), average=400 B

If "dispatch_table" is set after calling super().__init__, there is no leak anymore:

	/usr/lib/python3.10/tracemalloc.py:135: size=740 B (+740 B), count=7 (+7), average=106 B
	/usr/lib/python3.10/tracemalloc.py:125: size=2088 B (-656 B), count=29 (-4), average=72 B
	/usr/lib/python3.10/tracemalloc.py:136: size=320 B (+320 B), count=1 (+1), average=320 B
	/usr/lib/python3.10/tracemalloc.py:132: size=0 B (-256 B), count=0 (-1)
	/usr/lib/python3.10/tracemalloc.py:129: size=72 B (+72 B), count=1 (+1), average=72 B
	/usr/lib/python3.10/tracemalloc.py:498: size=2008 B (+48 B), count=34 (+1), average=59 B
	/usr/lib/python3.10/tracemalloc.py:193: size=1584 B (+48 B), count=33 (+1), average=48 B
	/usr/lib/python3.10/tracemalloc.py:502: size=196 B (-28 B), count=7 (-1), average=28 B
	/usr/lib/python3.10/tracemalloc.py:126: size=84 B (+28 B), count=3 (+1), average=28 B

----------
components: Library (Lib)
messages: 406478
nosy: douglas-raillard-arm
priority: normal
severity: normal
status: open
title: Custom pickler memory leak
type: resource usage
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45830>
_______________________________________

From report at bugs.python.org  Wed Nov 17 12:59:55 2021
From: report at bugs.python.org (=?utf-8?q?=C5=81ukasz_Langa?=)
Date: Wed, 17 Nov 2021 17:59:55 +0000
Subject: [New-bugs-announce] [issue45831] _Py_DumpASCII() writes characters
 one by one leading to corrupted tracebacks
Message-ID: <1637171995.61.0.0611629637793.issue45831@roundup.psfhosted.org>


New submission from ?ukasz Langa <lukasz at langa.pl>:

This is also a performance issue but I'm mostly concerned about the stdout corruption aspect. When more than one thread aborts at the same time, output ends up looking like the example below on the terminal. Unless you understand the issue is output written out byte-by-byte, you might get misled into seeing memory corruption.


Fatal Python error: Aborted

Fatal Python error: Fatal Python error: Fatal Python error: AbortedAborted



AbortedCurrent thread 0x

0000000110cf4dc0 (most recent call first):
Current thread 0xCurrent thread 0x  File 0000000110cf4dc0Current thread 0x (most recent call first):
0000000110cf4dc00000000110cf4dc0 (most recent call first):
 (most recent call first):
"  File   File   File /tmp"""////tctmtmppmpyp//tcchppoy/nytc-hptrohyonoton-t-rh-ormooaoontt-i--rnmomo/aatiLi-nni/m/baLLriiinbab/rrLraayirr/byyr//FFaFrrrayam/mrFeearwmaowermwoeorkrwkkossr//kPPsyy/ttPhsyo/tnhP.hyotnoh.nof.nrff.arfmarreaammwmeeoewwrowkro/orVrkkek///rVVseVierorsesnirsioson/ns3si.//o331.n1s.111///l13il.bi1/b1//pl/ypiltybht/ihopbno/n3yp3.t.1hy11o1/n/m3mu.tul1lth1ti/oipmnrpu3ol.rc1toe1cs/esmsiunsliignt/pigspr/yorncocscehysnrseocsihnsnirigoz/nesi.yzpnyen."cgph/yrs"oy, line nn95i, line  in c95z_ in e__h.e_pnertnyoet"ernr_i__z_
, line
e  File   File .95p in y_""_//"ttemmnppt//, line ccepp95yy in ttr_h_ho_on_-ner-nr
ooto  File ote-rm"_a/_ti
nm  File p//Lcitbp"yr-ta/hrtym/pFm/racaipmnye/owLtoihbrokrnnas-r/-Pyry/rtFoohooorttn-a.mfma-eirmnwaamoiernwk/osL//rikPL/yiVtbherroansrb.iyfo/nrFasr/ma3emw.e1owr1o/kr/lkiVsbe/r/Pspyyittohhnoosnn/.33f..r11a11m//elmwiuolbrt/kpi/pyVtrehorocsenis3os.n1is1n//g3/m.uq1ul1te/iulpeirsb.o/cppeyys"ts, line h102io in nng3g/.qe1utr
e1u  File a/"rme/uytsl/.mtppFi/yrpc"rpa, line yomt102ehcw in eosgnose-rirtnk
gs  File o//o"Pt/qty-mumpea/ticunpey/hLtosih.bopnrnya-".rr, line yfo/r102oF in argtaemm-temew
woaor  File rikkns/"L//Pi/ybtVtrmhapeor/nyc/.pFfyrrtaahmmoerwensow-rorkorsok/t/Pi-Vymoetarhisonnin.o/sfnLrs//ai33mb.er.1w11o1ar/rk/l/lyViieb/brF/srp/aipyomntyhseo/tn3w3.ho1.r11o1/k/lsnic3ob//n.cpPu1yr1yrt/ehmntuothl/otfinup.rtfourrcaeenss3s/.ip1nr1gom/c/qecusoesne.uwpceyuosr."rp, line ry236ke in /"_nVpt, line re/o102fcr in sueitgsuesr_toewn
...

----------
components: Interpreter Core
messages: 406480
nosy: lukasz.langa
priority: normal
severity: normal
stage: patch review
status: open
title: _Py_DumpASCII() writes characters one by one leading to corrupted tracebacks
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45831>
_______________________________________

From report at bugs.python.org  Wed Nov 17 14:00:34 2021
From: report at bugs.python.org (Harald Husum)
Date: Wed, 17 Nov 2021 19:00:34 +0000
Subject: [New-bugs-announce] [issue45832] Misleading membersip expression
 documentation
Message-ID: <1637175634.4.0.451304822027.issue45832@roundup.psfhosted.org>


New submission from Harald Husum <harald.husum at gmail.com>:

https://docs.python.org/3/reference/expressions.html#membership-test-operations

> For container types such as list, tuple, set, frozenset, dict, or collections.deque, the expression `x in y` is equivalent to `any(x is e or x == e for e in y)`.

Yet:

```py
import pandas as pd
import numpy as np

pd_0_dt = pd.Timedelta(0)
np_0_dt = np.timedelta64(0)

cm = (pd_0_dt, np_0_dt)

d1 = {np_0_dt: pd_0_dt}
d2 = {pd_0_dt: np_0_dt}

def test_membership_doc_claim(candidate_members, dct):
    for m in candidate_members:
        if m in dct:
            assert any(m is e or m == e for e in dct)

        if any(m is e or m == e for e in dct):
            assert m in dct

if __name__ == "__main__":
    test_membership_doc_claim(cm, d1) # Fails
    test_membership_doc_claim(cm, d2) # Fails

```

Not too surprised, given the td.__hash__() implementation differs between these classes, but they are considered equal none the less.

Unsure whether it is the dict implementation or the doc claim that needs to budge here.

----------
assignee: docs at python
components: Documentation
messages: 406485
nosy: docs at python, eric.araujo, ezio.melotti, harahu, mdk, willingc
priority: normal
severity: normal
status: open
title: Misleading membersip expression documentation
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45832>
_______________________________________

From report at bugs.python.org  Wed Nov 17 16:22:11 2021
From: report at bugs.python.org (Brian McCutchon)
Date: Wed, 17 Nov 2021 21:22:11 +0000
Subject: [New-bugs-announce] [issue45833] NamedTemporaryFile deleted before
 enclosing context manager exit
Message-ID: <1637184131.4.0.431691431028.issue45833@roundup.psfhosted.org>


New submission from Brian McCutchon <bmccutchon at google.com>:

Consider the following code:

# Copyright 2021 Google LLC.
# SPDX-License-Identifier: Apache-2.0
import contextlib
import os

@contextlib.contextmanager
def my_tmp_file():
  with tempfile.NamedTemporaryFile('w') as f:
    yield f

os.stat(my_tmp_file().__enter__().name)  # File not found
os.stat(contextlib.ExitStack().enter_context(my_tmp_file()).name)  # Same

I would expect the file to still exist, as __exit__ has not been called and I can't see why the file would have been closed. Also, it performs as expected when using NamedTemporaryFile directly, but not when it is nested in another context manager. It also performs as expected when my_tmp_file() or contextlib.ExitStack() is used in a "with" statement.

----------
components: Library (Lib)
messages: 406494
nosy: Brian McCutchon
priority: normal
severity: normal
status: open
title: NamedTemporaryFile deleted before enclosing context manager exit
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45833>
_______________________________________

From report at bugs.python.org  Wed Nov 17 16:27:57 2021
From: report at bugs.python.org (Irit Katriel)
Date: Wed, 17 Nov 2021 21:27:57 +0000
Subject: [New-bugs-announce] [issue45834] Move runtime except: check to the
 parser
Message-ID: <1637184477.04.0.544016485338.issue45834@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at gmail.com>:

The compiler has a runtime check that an "except:" without type can only be the last in the try block:

https://github.com/python/cpython/blob/15409c720be0503131713e3d3abc1acd0da07378/Python/compile.c#L3251

This could be detected by the parser instead.

----------
components: Parser
messages: 406495
nosy: iritkatriel, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Move runtime except: check to the parser
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45834>
_______________________________________

From report at bugs.python.org  Wed Nov 17 16:57:20 2021
From: report at bugs.python.org (Sam Gross)
Date: Wed, 17 Nov 2021 21:57:20 +0000
Subject: [New-bugs-announce] [issue45835] Race condition in test_queue can
 lead to test failures
Message-ID: <1637186240.11.0.170134157787.issue45835@roundup.psfhosted.org>


New submission from Sam Gross <colesbury at gmail.com>:

The test_queue suite has a race condition that can lead to test failures in test_many_threads, test_many_threads_nonblock, and test_many_threads_timeout. Consumers are signaled to exit by a sentinel value (None). The sentinel values are at the end of the input list, but that doesn't mean they are necessarily enqueued at the end of the inter-thread queue when there are multiple "feeder" threads.

In particular, a feeder thread may be delayed in enqueueing a non-sentinel value. The other feeder threads may finish popping and enqueueing the remaining values including all the sentinels, leading to the delayed non-sentinel value arriving AFTER all the sentinels. The "consumer" threads exit before processing all the values leading to the assertion error in run_threads() in test_queue.py:

  self.assertTrue(q.empty())

I will attach a patch that adds a delay in feed() to make the race condition occur more frequently so that the issue is easier to reproduce.

----------
components: Tests
messages: 406498
nosy: colesbury
priority: normal
severity: normal
status: open
title: Race condition in test_queue can lead to test failures
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45835>
_______________________________________

From report at bugs.python.org  Wed Nov 17 19:47:42 2021
From: report at bugs.python.org (Patrick Bourke)
Date: Thu, 18 Nov 2021 00:47:42 +0000
Subject: [New-bugs-announce] [issue45836] unittest.TestCase.assertWarns
 raises RuntimeEror if sys.modules changes size (Python 3.9)
Message-ID: <1637196462.86.0.173865369445.issue45836@roundup.psfhosted.org>


New submission from Patrick Bourke <pbourke at microsoft.com>:

Hi all,

Forgive me if this is not the correct way to report this, but we have run into the issue from #29620 ( https://bugs.python.org/issue29620 ) on Python 3.9.

The fix appears to be present in the tip of 3.8:

https://github.com/python/cpython/blob/c37a0d9c0ae4aa0d9135fac9a58afc7b34ff71d6/Lib/unittest/case.py#L254

and 3.10:

https://github.com/python/cpython/blob/0ef308a2890571c850c624fb99ac00f8951363c6/Lib/unittest/case.py#L255

but missing from 3.9:

https://github.com/python/cpython/blob/0ef308a2890571c850c624fb99ac00f8951363c6/Lib/unittest/case.py#L255

Here is our test case:

    import unittest
    import graspologic


    class MyTestCase(unittest.TestCase):
        def test_something(self):
            with self.assertWarns(UserWarning):
                pass


    if __name__ == '__main__':
        unittest.main()

fails with:

    ======================================================================
    ERROR: test_something (__main__.MyTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "C:\Users\pbourke\AppData\Roaming\JetBrains\PyCharm2021.2\scratches\test.py", line 7, in test_something
        with self.assertWarns(UserWarning):
      File "C:\Users\pbourke\.pyenv\pyenv-win\versions\3.9.7\lib\unittest\case.py", line 255, in __enter__
        for v in sys.modules.values():
    RuntimeError: dictionary changed size during iteration


The module triggering the error appears to be IPython.utils.io, which is included somewhere in our dependencies.

My CPython is:
Python 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)]

----------
components: Library (Lib)
messages: 406516
nosy: pbourke
priority: normal
severity: normal
status: open
title: unittest.TestCase.assertWarns raises RuntimeEror if sys.modules changes size (Python 3.9)
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45836>
_______________________________________

From report at bugs.python.org  Thu Nov 18 08:43:45 2021
From: report at bugs.python.org (Hugo van Kemenade)
Date: Thu, 18 Nov 2021 13:43:45 +0000
Subject: [New-bugs-announce] [issue45837] Fix turtle deprecations
Message-ID: <1637243025.5.0.541899962975.issue45837@roundup.psfhosted.org>


New submission from Hugo van Kemenade <hugovk+python at gmail.com>:

turtle's settiltangle was deprecated in Python 3.1:

"Deprecated since version 3.1."

https://docs.python.org/3.10/library/turtle.html#turtle.settiltangle says of settiltangle:


And the reason:

"`Turtle.tiltangle()` has been enhanced in functionality: it now can be used to get or set the tiltangle. `Turtle.settiltangle()` has been deprecated."

https://docs.python.org/3.10/library/turtle.html#changes-since-python-3-0


However, in docstrings, tiltangle was accidentally marked as deprecated:

"Deprecated since Python 3.1"

https://github.com/python/cpython/blob/v3.10.0/Lib/turtle.py#L2880


Neither tiltangle nor settiltangle raise DeprecationWarnings.


So let's:

* Correct tiltangle's docstring to say it's not really deprecated
* Update settiltangle's docstring to say it is deprecated
* Add a DeprecationWarning to settiltangle
* Internally call self.tiltangle instead of self.settiltangle


BPO references:

2009 https://bugs.python.org/issue5923 - settiltangle originally deprecated, with rationale.

2010 https://bugs.python.org/issue7888 - the mixup was discovered and apparently corrected in py3k and release31-maint. I've not done the SCM archaeology to discover why this regressed.

2020 https://bugs.python.org/issue41165 - both mentioned as deprecated, mixup not noted.

----------
components: Library (Lib)
messages: 406536
nosy: hugovk
priority: normal
severity: normal
status: open
title: Fix turtle deprecations
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45837>
_______________________________________

From report at bugs.python.org  Thu Nov 18 14:08:45 2021
From: report at bugs.python.org (Sam Gross)
Date: Thu, 18 Nov 2021 19:08:45 +0000
Subject: [New-bugs-announce] [issue45838] Incorrect line numbers in GDB
 Python backtraces [3.9]
Message-ID: <1637262525.44.0.265478658084.issue45838@roundup.psfhosted.org>


New submission from Sam Gross <colesbury at gmail.com>:

Starting in Python 3.6 the line numbers table contains a *signed* byte indicating line delta. The calculation in Tools/gdb/libpython.py was not updated to handle signed bytes leading to incorrect line numbers when running "py-bt" (or printing frames) in GDB.

This issue does not exist in Python 3.10 or later because line number table was changed (and libpython.py was updated) in GH-23113.

----------
components: Demos and Tools
messages: 406560
nosy: colesbury
priority: normal
severity: normal
status: open
title: Incorrect line numbers in GDB Python backtraces [3.9]
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45838>
_______________________________________

From report at bugs.python.org  Thu Nov 18 16:52:09 2021
From: report at bugs.python.org (Muhammad Irfan Asghar)
Date: Thu, 18 Nov 2021 21:52:09 +0000
Subject: [New-bugs-announce] [issue45839] python3 executable is able to
 install pandas
Message-ID: <1637272329.47.0.00758927636582.issue45839@roundup.psfhosted.org>


New submission from Muhammad Irfan Asghar <mirfanasghar at gmail.com>:

pip3.10 install pandas
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))': /simple/pandas/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))': /simple/pandas/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))': /simple/pandas/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))': /simple/pandas/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))': /simple/pandas/
Could not fetch URL https://pypi.org/simple/pandas/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pandas/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))) - skipping
ERROR: Could not find a version that satisfies the requirement pandas (from versions: none)
ERROR: No matching distribution found for pandas
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))) - skipping

----------
components: Extension Modules
messages: 406566
nosy: mirfanasghar
priority: normal
severity: normal
status: open
title: python3 executable is able to install pandas
type: security
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45839>
_______________________________________

From report at bugs.python.org  Thu Nov 18 21:01:26 2021
From: report at bugs.python.org (Alex Waygood)
Date: Fri, 19 Nov 2021 02:01:26 +0000
Subject: [New-bugs-announce] [issue45840] Improve cross-references in the
 data model documentation
Message-ID: <1637287286.96.0.846227383123.issue45840@roundup.psfhosted.org>


New submission from Alex Waygood <Alex.Waygood at Gmail.com>:

The documentation for the data model has a lot of cross-references to other parts of the data model. However, often these cross-references do not have proper hyperlinks online, as :meth:`__iter__` is used (for example), instead of :meth:`~object.__iter__`. The documentation would be much more readable and navigable if this were fixed.

----------
assignee: docs at python
components: Documentation
messages: 406569
nosy: AlexWaygood, docs at python
priority: normal
severity: normal
status: open
title: Improve cross-references in the data model documentation
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45840>
_______________________________________

From report at bugs.python.org  Fri Nov 19 02:04:25 2021
From: report at bugs.python.org (Don Chamberlin)
Date: Fri, 19 Nov 2021 07:04:25 +0000
Subject: [New-bugs-announce] [issue45841] IDLE fails to save files in macO
Message-ID: <1637305465.21.0.876821716058.issue45841@roundup.psfhosted.org>


New submission from Don Chamberlin <chamberlin.don at gmail.com>:

Running Python 3.10.0 on MacOS 12.0.1 (Monterey). Start IDLE app to get an IDLE shell. Click File->New File. Edit window appears. Type 
# This is a comment
in the edit window, then click File->Save or command-S.
Error popup appears: "The save file operation failed to connect to the open and save panel service." (See attached screenshot.)
Click "OK" on the error popup. 
File is now saved in the Documents folder with name "None".

----------
assignee: terry.reedy
components: IDLE
files: IDLE save error.png
messages: 406572
nosy: don.chamberlin, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE fails to save files in macO
type: behavior
versions: Python 3.10
Added file: https://bugs.python.org/file50450/IDLE save error.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45841>
_______________________________________

From report at bugs.python.org  Fri Nov 19 07:06:17 2021
From: report at bugs.python.org (Francesc Elies)
Date: Fri, 19 Nov 2021 12:06:17 +0000
Subject: [New-bugs-announce] [issue45842] AddressSanitizer: bad-free - hello
 world c extension
Message-ID: <1637323577.79.0.578225391074.issue45842@roundup.psfhosted.org>


New submission from Francesc Elies <francesc.elies at gmail.com>:

Hi,

Context
=======
we are compiling a dll with clang and it's address sanitizer and loading it via cffi, at random spots ASAN complains with bad-free.

      ==15100==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x01991a850310 in thread T0
          #0 0x7ffa1dcc7f31  (C:\LLVM-13.0.0-win64\lib\clang\13.0.0\lib\windows\clang_rt.asan_dynamic-x86_64.dll+0x180037f31)
          #1 0x7ffa3aea59ec in _PyObject_Realloc D:\_w\1\s\Objects\obmalloc.c:2011
          #2 0x7ffa3af7f347 in _PyObject_GC_Resize D:\_w\1\s\Modules\gcmodule.c:2309
          #3 0x7ffa3aedeeaa in _PyEval_EvalCode D:\_w\1\s\Python\ceval.c:4101
          ...
          See links below for a full trace

The project where we see this it's quite complex and with many moving parts therefore we made a minimal reproducible example.

Reprex
======

The test boils down to the following:

    win32api.LoadLibrary("LLVM-13.0.0-win64/lib/clang/13.0.0/lib/windows/clang_rt.asan_dynamic-x86_64.dll")
    import hello  # hello is our compiled c extension with clang and ASAN
    print(hello.system())
    import pdb


If if comment the last line (import pdb) ASAN does not complain.
If instead of import pdb you import numpy the same problem can be seen.

See the following failing build https://github.com/FrancescElies/min_reprex_python_c_extension_asan/runs/4263693010?check_suite_focus=true
See here the CI test https://github.com/FrancescElies/min_reprex_python_c_extension_asan/blob/d966d3a472df71977dc6519a76be0120d2d58d39/test.py


We did not try this in linux yet, would that help?

Is this a bug? Or are we doing something wrong?

Thanks in advance for your time.

----------
components: Windows
messages: 406578
nosy: FrancescElies, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: AddressSanitizer: bad-free  -  hello world c extension
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45842>
_______________________________________

From report at bugs.python.org  Fri Nov 19 09:06:30 2021
From: report at bugs.python.org (theeshallnotknowethme)
Date: Fri, 19 Nov 2021 14:06:30 +0000
Subject: [New-bugs-announce] [issue45843] Optimizing LOAD_CONST followed by
 COMPARE_OP (or IS_OP)
Message-ID: <1637330790.91.0.849281651207.issue45843@roundup.psfhosted.org>


New submission from theeshallnotknowethme <nohackingofkrowten at gmail.com>:

Result of `255581293 > 12938373 and 113314 < 2`:
Unoptimized: 5000000 loops, best of 5: 42.6 nsec per loop
Optimized: 20000000 loops, best of 5: 14.4 nsec per loop

----------
components: Interpreter Core
messages: 406582
nosy: February291948
priority: normal
severity: normal
status: open
title: Optimizing LOAD_CONST followed by COMPARE_OP (or IS_OP)
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45843>
_______________________________________

From report at bugs.python.org  Fri Nov 19 11:41:55 2021
From: report at bugs.python.org (JMcB)
Date: Fri, 19 Nov 2021 16:41:55 +0000
Subject: [New-bugs-announce] [issue45844] Dead link 'consolelib' in
 faq/library
Message-ID: <1637340115.5.0.328123119567.issue45844@roundup.psfhosted.org>


New submission from JMcB <joel.mcbride1 at live.com>:

At https://docs.python.org/3/faq/library.html#is-there-a-curses-termcap-package-for-python it says:
For Windows: use the consolelib module.
And links to:
http://effbot.org/zone/console-index.htm
This link is dead. Also, a quick google search for consolelib doesn't yield any good results, and I've never heard of it myself.

----------
assignee: docs at python
components: Documentation
messages: 406594
nosy: JMcB17, docs at python
priority: normal
severity: normal
status: open
title: Dead link 'consolelib' in faq/library
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45844>
_______________________________________

From report at bugs.python.org  Fri Nov 19 11:43:21 2021
From: report at bugs.python.org (JMcB)
Date: Fri, 19 Nov 2021 16:43:21 +0000
Subject: [New-bugs-announce] [issue45845] Dead link
 'pythoncraft.com/OSCON2001' in faq/library
Message-ID: <1637340201.11.0.111443507771.issue45845@roundup.psfhosted.org>


New submission from JMcB <joel.mcbride1 at live.com>:

At https://docs.python.org/3/faq/library.html#how-do-i-program-using-threads it says:
Aahz has a set of slides from his threading tutorial that are helpful; see http://www.pythoncraft.com/OSCON2001/.
This link is dead.

----------
assignee: docs at python
components: Documentation
messages: 406595
nosy: JMcB17, docs at python
priority: normal
severity: normal
status: open
title: Dead link 'pythoncraft.com/OSCON2001'  in faq/library
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45845>
_______________________________________

From report at bugs.python.org  Fri Nov 19 11:45:15 2021
From: report at bugs.python.org (JMcB)
Date: Fri, 19 Nov 2021 16:45:15 +0000
Subject: [New-bugs-announce] [issue45846] Incorrect name capitalisation in
 faq/programming
Message-ID: <1637340315.0.0.653516465145.issue45846@roundup.psfhosted.org>


New submission from JMcB <joel.mcbride1 at live.com>:

At https://docs.python.org/3/faq/programming.html#how-can-i-have-modules-that-mutually-import-each-other it says:
van Rossum doesn?t like this approach much because the imports appear in a strange place, but it does work.
The V should be capitalised.
References:
https://gvanrossum.github.io/
https://en.wikipedia.org/wiki/Guido_van_Rossum#Life_and_education

----------
assignee: docs at python
components: Documentation
messages: 406596
nosy: JMcB17, docs at python
priority: normal
severity: normal
status: open
title: Incorrect name capitalisation in faq/programming
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45846>
_______________________________________

From report at bugs.python.org  Fri Nov 19 11:56:30 2021
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 19 Nov 2021 16:56:30 +0000
Subject: [New-bugs-announce] [issue45847] Port module setup to
 PY_STDLIB_MOD() macro and addext()
Message-ID: <1637340990.31.0.170272925457.issue45847@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

bpo-45573 added the PY_STDLIB_MOD() autoconf macro, Modules/Setup.stdlib.in template, and setup.py helper method addext().

The macro sets up

* MODULE_{NAME}_TRUE/FALSE conditional
* MODULE_{NAME} variable with values yes, disabled, missing, or n/a
* MODULE_{NAME}_CFLAGS
* MODULE_{NAME}_LDFLAGS

Additionally there is a MODULE_{NAME}_DEPS variable with dependency information.

The Modules/Setup.stdlib.in template and addext() method consume the variables and set up extension compilation and linking. There is no need to look for headers or libraries in setup.py any more.

Let's port all modules to the new approach.

----------
components: Build
messages: 406597
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Port module setup to PY_STDLIB_MOD() macro and addext()
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45847>
_______________________________________

From report at bugs.python.org  Fri Nov 19 12:46:25 2021
From: report at bugs.python.org (Petr Viktorin)
Date: Fri, 19 Nov 2021 17:46:25 +0000
Subject: [New-bugs-announce] [issue45848] Pegen's nice error reporting
 crashes non-UTF-8 files
Message-ID: <1637343985.54.0.0156635573413.issue45848@roundup.psfhosted.org>


New submission from Petr Viktorin <encukou at gmail.com>:

Parsing a script with non-UTF-8 encoding and a missing close parenthesis, like a file with the following 2 lines:

# encoding: ascii
(

... crashes with:

python: Parser/pegen.c:408: get_error_line: Assertion `p->tok->fp == NULL || p->tok->fp == stdin' failed.
Aborted (core dumped)

----------
components: Parser
files: missing_paren_ascii.py
messages: 406602
nosy: lys.nikolaou, pablogsal, petr.viktorin
priority: normal
severity: normal
status: open
title: Pegen's nice error reporting crashes non-UTF-8 files
Added file: https://bugs.python.org/file50451/missing_paren_ascii.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45848>
_______________________________________

From report at bugs.python.org  Fri Nov 19 17:50:55 2021
From: report at bugs.python.org (thewh1teagle)
Date: Fri, 19 Nov 2021 22:50:55 +0000
Subject: [New-bugs-announce] [issue45849] Embedded python doesn't recognize
 exit()
Message-ID: <1637362255.56.0.439267672829.issue45849@roundup.psfhosted.org>


New submission from thewh1teagle <moshvilla at yandex.com>:

C:\Users\user\Downloads\python-3.10.0-embed-win32>python
Python 3.10.0 (tags/v3.10.0:b494f59, Oct  4 2021, 18:46:30) [MSC v.1929 32 bit (Intel)] on win32
>>> exit
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'exit' is not defined
>>> exit()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'exit' is not defined
>>> import os;os._exit(0)

C:\Users\user\Downloads\python-3.10.0-embed-win32>


same error when running exit from file.

----------
components: Windows
messages: 406624
nosy: paul.moore, steve.dower, thewh1teagle, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Embedded python doesn't recognize exit()
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45849>
_______________________________________

From report at bugs.python.org  Fri Nov 19 18:44:27 2021
From: report at bugs.python.org (Guido van Rossum)
Date: Fri, 19 Nov 2021 23:44:27 +0000
Subject: [New-bugs-announce] [issue45850] Port deep-freeze to Windows
Message-ID: <1637365467.09.0.176625289906.issue45850@roundup.psfhosted.org>


Change by Guido van Rossum <guido at python.org>:


----------
nosy: gvanrossum
priority: normal
severity: normal
status: open
title: Port deep-freeze to Windows
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45850>
_______________________________________

From report at bugs.python.org  Fri Nov 19 20:42:08 2021
From: report at bugs.python.org (Stefan Pochmann)
Date: Sat, 20 Nov 2021 01:42:08 +0000
Subject: [New-bugs-announce] [issue45851] statistics.multimode is
 inefficient (time and space) (mode somewhat, too)
Message-ID: <1637372528.09.0.770941706263.issue45851@roundup.psfhosted.org>


New submission from Stefan Pochmann <stefan.pochmann at gmail.com>:

The current implementation is:

def multimode(data):
    counts = Counter(iter(data)).most_common()
    maxcount, mode_items = next(groupby(counts, key=itemgetter(1)), (0, []))
    return list(map(itemgetter(0), mode_items))

The most_common() does a complete sort of Counter item tuples, taking O(n log n) time and quite big O(n) extra space (mostly for all those tuples).

When Raymond Hettinger suggested it in https://bugs.python.org/issue35892#msg336338 he said it should have "running speed that is optimal for the desired result". But then he detailed that with "Slow O(n log n), loads all data in memory, full sort".

Which seems like a mistake, as that's not optimal. It's easy to do in O(n) time and O(1) extra memory (in addition to the Counter and result, I mean):

def multimode(data):
    counts = Counter(iter(data))
    if not counts:
        return []
    maxcount = max(counts.values())
    return [value for value, count in counts.items() if count == maxcount]

If there are only very few *different* values then the time/space after creating the Counter is insignificant compared to the Counter creation. But if there are many different values, it can be significant.

statistics.mode takes O(n) time and O(1) space, which is optimal, but I found an apparently faster way anyway (code at end).

For example for data = random.choices(range(n), k=n):

           |     multimode     |       mode
     n     | current  proposal | current  proposal
-----------+-------------------+------------------
         1 |    131%    70%    |    125%   58%
        10 |    144%    73%    |    119%   53%
       100 |    126%    71%    |    108%   29%
     1,000 |    123%    65%    |     62%   22%
    10,000 |    172%    55%    |     53%   18%
   100,000 |    164%    44%    |     55%   20%
 1,000,000 |     85%    20%    |     22%    4%
10,000,000 |     56%    12%    |     11%    4%

All four start with Counter(iter(data)), so I took that as baseline and the above results show relative additional times. For example 55% means if Counter construction alone took 10 seconds, the function took 15.5 seconds.

An extreme case, data = list(range(n)):

           |     multimode    |       mode
     n     | current proposal | current proposal
-----------+------------------+-----------------
         1 |   128%   67%     |   124%   56%
        10 |   187%   93%     |   141%   52%
       100 |   316%  149%     |   181%   45%
     1,000 |   380%  174%     |   213%   46%
    10,000 |   349%  111%     |   146%   30%
   100,000 |   397%  128%     |   159%   34%
 1,000,000 |   336%   95%     |   112%   24%
10,000,000 |   349%   97%     |   109%   23%

I also tried a bunch of other cases, didn't find one where my versions weren't quite a bit faster.

My mode() version:

from operator import indexOf
from itertools import islice

def mode(data):
    counts = Counter(iter(data))
    if not counts:
        raise StatisticsError('no mode for empty data') from None
    maxcount = max(counts.values())
    index = indexOf(counts.values(), maxcount)
    return next(islice(counts, index, None))

----------
components: Library (Lib)
messages: 406638
nosy: Stefan Pochmann
priority: normal
severity: normal
status: open
title: statistics.multimode is inefficient (time and space) (mode somewhat, too)
type: performance
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45851>
_______________________________________

From report at bugs.python.org  Fri Nov 19 21:26:19 2021
From: report at bugs.python.org (Stefan Pochmann)
Date: Sat, 20 Nov 2021 02:26:19 +0000
Subject: [New-bugs-announce] [issue45852] statistics.mode test doesn't test
 what it claims to
Message-ID: <1637375179.63.0.568396800727.issue45852@roundup.psfhosted.org>


New submission from Stefan Pochmann <stefan.pochmann at gmail.com>:

This test:

    def test_counter_data(self):
        # Test that a Counter is treated like any other iterable.
        data = collections.Counter([1, 1, 1, 2])
        # Since the keys of the counter are treated as data points, not the
        # counts, this should return the first mode encountered, 1
        self.assertEqual(self.func(data), 1)

If the mode() code *were* wrong this way (used Counter(data) instead of Counter(iter(data))), then the test wouldn't detect it, as mode() would still return 1. The test data should be [1, 2, 2, 2] instead, in which case such wrong mode() would return 2.

It used to be correct but wasn't adjusted correctly when mode() switched from raising an error for multiple modes to returning the first. The old code was:

    def test_counter_data(self):
        # Test that a Counter is treated like any other iterable.
        data = collections.Counter([1, 1, 1, 2])
        # Since the keys of the counter are treated as data points, not the
        # counts, this should raise.
        self.assertRaises(statistics.StatisticsError, self.func, data)

----------
components: Tests
messages: 406642
nosy: Stefan Pochmann
priority: normal
severity: normal
status: open
title: statistics.mode test doesn't test what it claims to
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45852>
_______________________________________

From report at bugs.python.org  Sat Nov 20 10:14:11 2021
From: report at bugs.python.org (Andrei Kulakov)
Date: Sat, 20 Nov 2021 15:14:11 +0000
Subject: [New-bugs-announce] [issue45853] Misspelled _IGNORED_ERROS in
 Lib/pathlib.py
Message-ID: <1637421251.02.0.184439782821.issue45853@roundup.psfhosted.org>


New submission from Andrei Kulakov <andrei.avk at gmail.com>:

Should be _IGNORED_ERRORS

This name was added 3 years ago: https://github.com/python/cpython/commit/216b745eafa7cd4a683a8405dcfbd7f5567f504c

It's only used in a single place in the module. But I'm not sure if it's worth fixing. It is an internal, undocumented name, and not very likely to be used in user code but it's possible.

----------
messages: 406660
nosy: andrei.avk
priority: normal
severity: normal
status: open
title: Misspelled _IGNORED_ERROS in Lib/pathlib.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45853>
_______________________________________

From report at bugs.python.org  Sat Nov 20 10:43:57 2021
From: report at bugs.python.org (=?utf-8?b?6LCt5Lmd6byO?=)
Date: Sat, 20 Nov 2021 15:43:57 +0000
Subject: [New-bugs-announce] [issue45854] winreg: implement RegGetValue
Message-ID: <1637423037.38.0.511813075404.issue45854@roundup.psfhosted.org>


New submission from ??? <109224573 at qq.com>:

The RegGetValue() API was introduced in Vista.

It's similar to RegOpenKeyEx() but it's more convenient because you don't need OpenKey() and CloseKey() when using predefined HKEY_* constants.

I think it would be good to implement it.

If so, I have implemented its body based on RegOpenKeyEx(), but I'm new to CPython and need some help.

https://github.com/imba-tjd/cpython/tree/winreg

----------
components: Library (Lib)
messages: 406663
nosy: imba-tjd
priority: normal
severity: normal
status: open
title: winreg: implement RegGetValue
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45854>
_______________________________________

From report at bugs.python.org  Sat Nov 20 11:27:18 2021
From: report at bugs.python.org (Georg Brandl)
Date: Sat, 20 Nov 2021 16:27:18 +0000
Subject: [New-bugs-announce] [issue45855] PyCapsule_Import still using
 PyImport_ImportModuleNoBlock
Message-ID: <1637425638.46.0.745441938277.issue45855@roundup.psfhosted.org>


New submission from Georg Brandl <georg at python.org>:

PyImport_ImportModuleNoBlock is the same as PyImport_ImportModule since 3.3, according to the docs.

PyCapsule_Import has a no_block parameter to choose, is still documented as if it is relevant, and the implementation also still uses it.

----------
components: C API
messages: 406671
nosy: georg.brandl
priority: low
severity: normal
stage: needs patch
status: open
title: PyCapsule_Import still using PyImport_ImportModuleNoBlock
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45855>
_______________________________________

From report at bugs.python.org  Sat Nov 20 17:54:23 2021
From: report at bugs.python.org (Thibaut Horel)
Date: Sat, 20 Nov 2021 22:54:23 +0000
Subject: [New-bugs-announce] [issue45856] [doc] map() documentation
 ambiguous about consumption order
Message-ID: <1637448863.69.0.971649126325.issue45856@roundup.psfhosted.org>


New submission from Thibaut Horel <thibaut.horel at gmail.com>:

In cases where multiple iterables are passed to the built-in function map(), the documentation is ambiguous about the order in which they are consumed [1]. Although the order of evaluation of function arguments is documented to be left-to-right in general [2], this does not necessarily imply that the __next__() functions of the underlying iterators are called from left to right *before* passing the returned elements to the function being mapped. This is particularly relevant when the same iterator is passed multiple times, or when there are side effects to consuming the iterables.

I suggest adding the sentence ?The iterables are consumed in left-to-right order at each iteration.?, similar to how it is done for the function zip() [3]. Furthermore, I think providing the following (roughly) equivalent implementation in pure Python might be illuminating:

    def map(function, *iterables):
        iterators = tuple(iter(it) for it in iterables)
        while True:
            try:
                args = [next(it) for it in iterators]
            except StopIteration:
                break
            yield func(*args)


Finally, the following example could be added. ?This makes it possible to apply a function to consecutive groups of elements from the same iterator by passing it multiple times to `map`:

    from itertools import count

    ctr = count()
    # map(func, ctr, ctr) -> func(0, 1), func(2, 3), ...
 
?

I am happy to submit a pull request once we reach a consensus on the formulation.

[1] https://docs.python.org/3/library/functions.html#map
[2] https://docs.python.org/3/reference/expressions.html#evaluation-order
[3] https://docs.python.org/3/library/functions.html#zip

----------
assignee: docs at python
components: Documentation
messages: 406693
nosy: docs at python, eric.araujo, ezio.melotti, mdk, thibaut.horel, willingc
priority: normal
severity: normal
status: open
title: [doc] map() documentation ambiguous about consumption order
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45856>
_______________________________________

From report at bugs.python.org  Sun Nov 21 09:48:31 2021
From: report at bugs.python.org (=?utf-8?b?5rSq5piO6IGW?=)
Date: Sun, 21 Nov 2021 14:48:31 +0000
Subject: [New-bugs-announce] [issue45857] Type hint for methods
Message-ID: <1637506111.5.0.826205176723.issue45857@roundup.psfhosted.org>


New submission from ??? <hung11112002 at gmail.com>:

The class methods have a problem compiling when the type refers to the union of itself and others.


# tmp.py
class Foo:
    def __init__(self, tmp: "Foo"|int):
        pass


# Error
Traceback (most recent call last):
  File "/Project/Mslc/Grammar/tmp.py", line 1, in <module>
    class Foo:
  File "/Project/Mslc/Grammar/tmp.py", line 2, in Foo
    def __init__(self, tmp: "Foo"|int):
TypeError: unsupported operand type(s) for |: 'str' and 'type'

----------
messages: 406720
nosy: TNThung
priority: normal
severity: normal
status: open
title: Type hint for methods
type: compile error
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45857>
_______________________________________

From report at bugs.python.org  Sun Nov 21 11:46:00 2021
From: report at bugs.python.org (Ian Fisher)
Date: Sun, 21 Nov 2021 16:46:00 +0000
Subject: [New-bugs-announce] [issue45858] Deprecate default converters in
 sqlite3
Message-ID: <1637513160.43.0.0570339537667.issue45858@roundup.psfhosted.org>


New submission from Ian Fisher <ian at iafisher.com>:

Per discussion at https://discuss.python.org/t/fixing-sqlite-timestamp-converter-to-handle-utc-offsets/, the default converters in SQLite3 have several bugs and are probably not worth continuing to maintain, so I propose deprecating them and removing them in a later version of Python.

Since the converters are opt-in, this should not affect most users of SQLite3.

----------
components: Library (Lib)
messages: 406727
nosy: erlendaasland, iafisher
priority: normal
severity: normal
status: open
title: Deprecate default converters in sqlite3
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45858>
_______________________________________

From report at bugs.python.org  Sun Nov 21 15:46:31 2021
From: report at bugs.python.org (Carl Friedrich Bolz-Tereick)
Date: Sun, 21 Nov 2021 20:46:31 +0000
Subject: [New-bugs-announce] [issue45859] test_collections has a wrong test
 in case _itemgetter is not available
Message-ID: <1637527591.49.0.376717916242.issue45859@roundup.psfhosted.org>


New submission from Carl Friedrich Bolz-Tereick <cfbolz at gmx.de>:

test_field_descriptor in test_collections tries to pickle the descriptors of a namedtuple's fields, which is _collections._itemgetter on CPython. However, on PyPy that class doesn't exist. The code in collections deals fine with that fact, but the above-mentioned test does not make sense in that situation, since you can't pickle properties.

To test this behaviour, you can replace
"from _collections import _tuplegetter"
in collections/__init__.py with raise ImportError and see the test fail on CPython too.

----------
messages: 406738
nosy: Carl.Friedrich.Bolz
priority: normal
severity: normal
status: open
title: test_collections has a wrong test in case _itemgetter is not available
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45859>
_______________________________________

From report at bugs.python.org  Sun Nov 21 21:24:42 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Mon, 22 Nov 2021 02:24:42 +0000
Subject: [New-bugs-announce] [issue45860] UnboundLocalError leads to Illegal
 instruction crashing CPython
Message-ID: <1637547882.55.0.126109992436.issue45860@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

The following code can lead to a crash and report Illegal instruction (core dumped)(few times) or Trace/breakpoint trap (core dumped) (very few times) or Segmentation fault (core dumped) (most times) on Python 3.11. 

test_free_different_thread.py
========================================================
import inspect
import sys
import threading
import unittest
import weakref
import ctypes
from test.support import run_doctest, run_unittest, cpython_only, check_impl_detail
import _testcapi
from types import FunctionType
from test import test_code
import test_code

def test_free_different_thread():
    f = CoExtra.get_func()

    class ThreadTest(threading.Thread):

        def __init__(CoExtra, f, test):
            super().__init__()
            CoExtra.f = CoExtra
            CoExtra.test = test

        def run(CoExtra):
            del CoExtra.f
            CoExtra.test.assertEqual(test_code.LAST_FREED, 500)
    test_code.SetExtra(f.__code__, test_code.FREE_INDEX, ctypes.c_voidp(500))
    f = ThreadTest(CoExtra, f)
    del tt
    tt.start()
    tt.join()
    CoExtra.assertEqual(test_code.LAST_FREED, 500)
    
CoExtra = test_code.CoExtra()
test_free_different_thread()
=========================================================

-------------------------------------
Traceback (most recent call last):
  File "/home/xxm/Desktop/test_free_different_thread.py", line 33, in <module>
    test_free_different_thread()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxm/Desktop/test_free_different_thread.py", line 28, in test_free_different_thread
    del tt
        ^^
UnboundLocalError: cannot access local variable 'tt' where it is not associated with a value
Illegal instruction (core dumped)/Trace/breakpoint trap (core dumped)/Segmentation fault (core dumped)
----------------------------------------------------------------

Version: python 3.9, python 3.10, python 3.11 on ubuntu 16.04

Reproduce step:
1.download test_code.py and place test_free_different_thread.py and test_code in a same directory.
2. run with "python test_free_different_thread.py"

The test_code.py is from cpython' test. We can also annotate "import test_code" and run test_free_different_thread.py directly. But it seems that Illegal instruction and Trace/breakpoint trap cannot be reproduced.

----------
components: Interpreter Core
files: test_code.py
messages: 406740
nosy: xxm
priority: normal
severity: normal
status: open
title: UnboundLocalError leads to Illegal instruction crashing CPython
type: crash
versions: Python 3.11
Added file: https://bugs.python.org/file50455/test_code.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45860>
_______________________________________

From report at bugs.python.org  Sun Nov 21 23:47:34 2021
From: report at bugs.python.org (heheoa)
Date: Mon, 22 Nov 2021 04:47:34 +0000
Subject: [New-bugs-announce] [issue45861] Can't find a usable init.tcl
Message-ID: <1637556454.49.0.701253039916.issue45861@roundup.psfhosted.org>


New submission from heheoa <cosmicmanuscript at gmail.com>:

_tkinter.TclError: Can't find a usable init.tcl in the following directories: 
    /usr/lib/tcl8.6 /usr/lib/tcl8.6 /lib/tcl8.6 /usr/library /library /tcl8.6.12/library /tcl8.6.12/library

----------
components: Tkinter
messages: 406741
nosy: nobodyatall
priority: normal
severity: normal
status: open
title: Can't find a usable init.tcl
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45861>
_______________________________________

From report at bugs.python.org  Mon Nov 22 02:29:08 2021
From: report at bugs.python.org (David Pratten)
Date: Mon, 22 Nov 2021 07:29:08 +0000
Subject: [New-bugs-announce] [issue45862] Anomaly of eval() of list
 comprehension
Message-ID: <1637566148.02.0.0346956477094.issue45862@roundup.psfhosted.org>


New submission from David Pratten <david at prattenmail.com>:

Hi

Example "eg def2" works but "eg def4" gives an error?

David

```
emp = [
    {
        "empno": 7839,
        "mgr": 0,
        "ename": "KING"
    },
    {
        "empno": 7566,
        "mgr": 7839,
        "ename": "JONES"
    },
    {
        "empno": 7698,
        "mgr": 7839,
        "ename": "BLAKE"
    }
]

a = [e for e in emp if e["mgr"] == 0]
print('eg 1', [b for b in a])
print('eg 2', eval('[b for b in a]'))
print('eg 3', [e for e in emp for b in a if e["mgr"] == b["empno"]])
print('eg 4', eval('[e for e in emp for b in a if e["mgr"] == b["empno"]]'))


def eval_anomaly():
    a_anomaly = [e for e in emp if e["mgr"] == 0]
    print('eg def1', [b for b in a_anomaly])
    print('eg def2', eval('[b for b in a_anomaly]'))
    print('eg def3', [e for e in emp for b in a_anomaly if e["mgr"] == b["empno"]])
    print('eg def4', eval('[e for e in emp for b in a_anomaly if e["mgr"] == b["empno"]]'))

eval_anomaly()
```

----------
messages: 406743
nosy: david2
priority: normal
severity: normal
status: open
title: Anomaly of eval() of list comprehension
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45862>
_______________________________________

From report at bugs.python.org  Mon Nov 22 03:01:20 2021
From: report at bugs.python.org (Joshua Root)
Date: Mon, 22 Nov 2021 08:01:20 +0000
Subject: [New-bugs-announce] [issue45863] tarfile zeroes ustar header fields
 unnecessarily
Message-ID: <1637568080.9.0.260845551853.issue45863@roundup.psfhosted.org>


New submission from Joshua Root <josh+python at root.id.au>:

When using the pax format, tarfile will zero out the field in the ustar header for any values that are represented with a float, setting the correct value only in the pax header. This mainly seems to apply to the mtime. This behaviour doesn't cause problems when using an unarchiver that understands the pax header, but unarchivers that don't will extract incorrect metadata (most obviously all mtimes set to the epoch). Compatibility with such unarchivers can easily be achieved by rounding the float value to int for the ustar header only, thus at least giving mtimes that are accurate to the nearest second instead of nothing.

----------
components: Library (Lib)
messages: 406744
nosy: jmr
priority: normal
severity: normal
status: open
title: tarfile zeroes ustar header fields unnecessarily
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45863>
_______________________________________

From report at bugs.python.org  Mon Nov 22 05:28:02 2021
From: report at bugs.python.org (Adam Johnson)
Date: Mon, 22 Nov 2021 10:28:02 +0000
Subject: [New-bugs-announce] [issue45864] unittest does not discover tests
 in PEP420 packages
Message-ID: <1637576882.84.0.616248029554.issue45864@roundup.psfhosted.org>


New submission from Adam Johnson <me at adamj.eu>:

unittest's test discovery does not descend into directories without `__init__.py`. This avoids discovering test modules that are otherwise valid and importable, after PEP 420.

I've seen this more than once where there were valid looking test files not being discovered, and they bit rot. The tests had been run individually when created but never again.

(I created [flake8-no-pep420](https://pypi.org/project/flake8-no-pep420/) to avoid this problem on my projects.)

For example, take this directory structure:

```
$ tree
.
??? tests
    ??? test_thing.py

1 directory, 1 file

$ cat tests/test_thing.py
1/0
```

It's valid to import the naughty file, which crashes:

```
$ python -c 'import tests.test_thing'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/.../tests/test_thing.py", line 1, in <module>
    1/0
ZeroDivisionError: division by zero
```

But unittest does not discover it:

```
$ python -m unittest

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
```

But, after creating an empty `__init__.py`, the tests doth fail:

```
$ touch tests/__init__.py

$ python -m unittest
E
======================================================================
ERROR: tests.test_thing (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests.test_thing
Traceback (most recent call last):
  File "/.../unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/.../unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/.../tests/test_thing.py", line 1, in <module>
    1/0
ZeroDivisionError: division by zero


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)
```

----------
components: Tests
messages: 406756
nosy: adamchainz
priority: normal
severity: normal
status: open
title: unittest does not discover tests in PEP420 packages
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45864>
_______________________________________

From report at bugs.python.org  Mon Nov 22 05:29:50 2021
From: report at bugs.python.org (Adam Johnson)
Date: Mon, 22 Nov 2021 10:29:50 +0000
Subject: [New-bugs-announce] [issue45865] Old syntax in unittest
Message-ID: <1637576990.34.0.316155860666.issue45865@roundup.psfhosted.org>


New submission from Adam Johnson <me at adamj.eu>:

I often browse the unittest code in order to write extensions. It still uses some Python 2-isms like classes inheriting from object, it would be nice to clean that up.

----------
components: Tests
messages: 406757
nosy: adamchainz
priority: normal
severity: normal
status: open
title: Old syntax in unittest
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45865>
_______________________________________

From report at bugs.python.org  Mon Nov 22 06:32:02 2021
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Mon, 22 Nov 2021 11:32:02 +0000
Subject: [New-bugs-announce] [issue45866] Out of tree build of Python
 3.11.0a2+ breaks regen-frozen
Message-ID: <1637580722.83.0.464484506874.issue45866@roundup.psfhosted.org>


New submission from Miro Hron?ok <miro at hroncok.cz>:

In Fedora, when building Python 3.11.0a2 with Python 3.11.0a2 for regen, we found that regen-frozen fails.

It can be reproduced either on the v3.11.0a2 tag or on the main branch.

On tag v3.11.0a2 with 3.11.0a2 installed in $PATH:

# in root of cpython local clone:
$ git clean -fdx
$ mkdir -p build/optimized
$ cd build/optimized
$ ../../configure
$ make regen-all PYTHON_FOR_REGEN=python3.11
...
ERROR: missing _freeze_module


On the main branch:

# in root of cpython local clone:
$ git clean -fdx
$ mkdir -p build/optimized
$ cd build/optimized
$ ../../configure
$ make
$ make regen-all PYTHON_FOR_REGEN=./python
(success)

However, the working tree is dirty:

$ git diff
diff --git a/Parser/parser.c b/Parser/parser.c
index b3aa35989ed..c26cd6eeb05 100644
--- a/Parser/parser.c
+++ b/Parser/parser.c
@@ -1,4 +1,4 @@
-// @generated by pegen from ./Grammar/python.gram
+// @generated by pegen from ../../Grammar/python.gram
 #include "pegen.h"
 
 #if defined(Py_DEBUG) && defined(Py_BUILD_CORE)
diff --git a/Tools/peg_generator/pegen/grammar_parser.py b/Tools/peg_generator/pegen/grammar_parser.py
index 6e9f7d3d11d..fbbbfad76b0 100644
--- a/Tools/peg_generator/pegen/grammar_parser.py
+++ b/Tools/peg_generator/pegen/grammar_parser.py
@@ -1,5 +1,5 @@
 #!/usr/bin/env python3.8
-# @generated by pegen from ./Tools/peg_generator/pegen/metagrammar.gram
+# @generated by pegen from ../../Tools/peg_generator/pegen/metagrammar.gram
 
 import ast
 import sys


And if we install Python somewhere:

$ make
$ make install DESTDIR=/tmp/python

And use that one again for regen:

$ cd ../..
$ git clean -fdx
$ mkdir -p build/optimized
$ cd build/optimized
$ ../../configure
$ make regen-all PYTHON_FOR_REGEN=/tmp/python/usr/local/bin/python3.11
...
# Regenerate Lib/keyword.py from Grammar/python.gram and Grammar/Tokens
# using Tools/peg_generator/pegen
PYTHONPATH=../../Tools/peg_generator /tmp/python/usr/local/bin/python3.11 -m pegen.keywordgen \
	../../Grammar/python.gram \
	../../Grammar/Tokens \
	../../Lib/keyword.py.new
/tmp/python/usr/local/bin/python3.11 ../../Tools/scripts/update_file.py ../../Lib/keyword.py ../../Lib/keyword.py.new
/tmp/python/usr/local/bin/python3.11 ../../Tools/scripts/freeze_modules.py
ERROR: missing _freeze_module
make: *** [Makefile:1259: regen-frozen] Error 1



It fails. I've isolated the failure to:

$ make regen-frozen PYTHON_FOR_REGEN=/tmp/python/usr/local/bin/python3.11
/tmp/python/usr/local/bin/python3.11 ../../Tools/scripts/freeze_modules.py
ERROR: missing _freeze_module
make: *** [Makefile:1259: regen-frozen] Error 1

----------
components: Build
messages: 406764
nosy: eric.snow, hroncok, petr.viktorin, vstinner
priority: normal
severity: normal
status: open
title: Out of tree build of Python 3.11.0a2+ breaks regen-frozen
type: compile error
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45866>
_______________________________________

From report at bugs.python.org  Mon Nov 22 07:32:27 2021
From: report at bugs.python.org (Muhamed Itani)
Date: Mon, 22 Nov 2021 12:32:27 +0000
Subject: [New-bugs-announce] [issue45867] kwarg default value expression
 incorrectly evaluated
Message-ID: <1637584347.64.0.225027731456.issue45867@roundup.psfhosted.org>


New submission from Muhamed Itani <moefear85 at gmail.com>:

I have an MPYFrame class that inherits from tkinter.Frame. It's init looks like this:

from os.path import normcase,normpath
def __init__(self,master=None,port=None,baudrate=115200,timeout=1,localFolder=normcase(normpath(os.getcwd()))):

The problem is, when I check/access the value of localFolder later in the code (even directly as a first statement within __init__), it is obvious that os.getcwd() is executing, but normcase and/or normpath are not. The resulting path contains capital letters (tested on Windows 11), even though it should not. If I run:
localFolder=normcase(normpath(localFolder))

as a first statement after init, then check the value of localFolder (for example at a debugging breakpoint), it correctly then contains a normalized path in which all letters have been converted to lowercase (but only after the statement itself has executed).

----------
components: Parser
files: mpyFrame.pyw
messages: 406765
nosy: lys.nikolaou, moefear85, pablogsal
priority: normal
severity: normal
status: open
title: kwarg default value expression incorrectly evaluated
type: behavior
versions: Python 3.10
Added file: https://bugs.python.org/file50456/mpyFrame.pyw

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45867>
_______________________________________

From report at bugs.python.org  Mon Nov 22 07:39:12 2021
From: report at bugs.python.org (Oskar Enoksson)
Date: Mon, 22 Nov 2021 12:39:12 +0000
Subject: [New-bugs-announce] [issue45868] xattr support missing in os module
 under cygwin
Message-ID: <1637584752.6.0.681447686617.issue45868@roundup.psfhosted.org>


New submission from Oskar Enoksson <enok at lysator.liu.se>:

With the latest Python 3.8.10 under cygwin the following fails:

>>> import os
>>> os.listxattr('.')
Traceback (most recent call last):
  File "<stdin>", line 1, in < module>
AttributeError: module 'os' has no attribute 'listxattr'

/usr/include/attr/xattr.h declares the corresponding library functions, so something must be wrong in the configuration/compilation of python itself.

----------
components: Build
messages: 406768
nosy: enok2
priority: normal
severity: normal
status: open
title: xattr support missing in os module under cygwin
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45868>
_______________________________________

From report at bugs.python.org  Mon Nov 22 08:27:59 2021
From: report at bugs.python.org (Joran van Apeldoorn)
Date: Mon, 22 Nov 2021 13:27:59 +0000
Subject: [New-bugs-announce] [issue45869] Unicode and acii regular
 expressions do not agree on ascii space characters
Message-ID: <1637587679.33.0.798190519421.issue45869@roundup.psfhosted.org>


New submission from Joran van Apeldoorn <git at blubmail.nl>:

The expectation would be that the re.A (or re.ASCII) flag should not impact the matching behavior of a regular expression on strings consisting only of ASCII characters.  However, for the characters 0x1c till 0x1f, the classes \s and \S differ. For ASCII theses characters are not considered space characters while for unicode they are. 

Note that python strings do consider these characters spaces as '\xc1'.isspace() gives True. 

All other classes and characters stay the same for unicode and ASCII matching.

----------
components: Regular Expressions
files: unicode-ascii-space.py
messages: 406773
nosy: control-k, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Unicode and acii regular expressions do not agree on ascii space characters
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file50457/unicode-ascii-space.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45869>
_______________________________________

From report at bugs.python.org  Mon Nov 22 08:51:22 2021
From: report at bugs.python.org (keeely)
Date: Mon, 22 Nov 2021 13:51:22 +0000
Subject: [New-bugs-announce] [issue45870] There's no readline module on
 Windows Python (cmd.Cmd)
Message-ID: <1637589082.36.0.67630471216.issue45870@roundup.psfhosted.org>


New submission from keeely <smh at sophos.com>:

In the past this was worked around by installing a third-party module:
https://github.com/pyreadline/pyreadline

For Python 3.10, however this module doesn't work.  And pyreadline doesn't seem to be maintained anymore, so it's possible it won't get fixed.
https://github.com/pyreadline/pyreadline/issues/73

Consider the following code:

import cmd
import readline

open("history.txt", "w").write("first line\nsecond line\n")

readline.clear_history()
readline.read_history_file("history.txt")

class MyShell(cmd.Cmd):
    def __init__(self):
        super().__init__()

shell = MyShell()
shell.cmdloop()

This works fine on MacOs Python, also on Linux and on Windows prior to 3.10 with the readline module added.  It won't work going forward.

The Windows cmd implementation clearly uses readline or some compatible lib under the hood, so Python should make it available to developers.

----------
components: Library (Lib)
messages: 406778
nosy: keeely
priority: normal
severity: normal
status: open
title: There's no readline module on Windows Python (cmd.Cmd)
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________

From report at bugs.python.org  Mon Nov 22 10:52:54 2021
From: report at bugs.python.org (Irit Katriel)
Date: Mon, 22 Nov 2021 15:52:54 +0000
Subject: [New-bugs-announce] [issue45871] Refactor exception matcher
 validation out of JUMP_IF_NOT_EXC_MATCH
Message-ID: <1637596374.92.0.314923053971.issue45871@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at gmail.com>:

The validation will be shared with except* so this refactor is prep for implementing PEP-654.

Also need to add the missing unit test for the validation.

----------
messages: 406790
nosy: iritkatriel
priority: normal
severity: normal
status: open
title: Refactor exception matcher validation out of JUMP_IF_NOT_EXC_MATCH
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45871>
_______________________________________

From report at bugs.python.org  Mon Nov 22 12:35:35 2021
From: report at bugs.python.org (Pavel V)
Date: Mon, 22 Nov 2021 17:35:35 +0000
Subject: [New-bugs-announce] [issue45872] Turtle documentation, write()
Message-ID: <1637602535.98.0.579349868944.issue45872@roundup.psfhosted.org>


New submission from Pavel V <willyns59 at gmail.com>:

There are no parentheses for 'font' argument in turtle.write() documentation

https://docs.python.org/3.10/library/turtle.html#turtle.write

----------
assignee: docs at python
components: Documentation
messages: 406795
nosy: docs at python, willyns
priority: normal
severity: normal
status: open
title: Turtle documentation, write()
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45872>
_______________________________________

From report at bugs.python.org  Mon Nov 22 13:51:49 2021
From: report at bugs.python.org (Guido van Rossum)
Date: Mon, 22 Nov 2021 18:51:49 +0000
Subject: [New-bugs-announce] [issue45873] Stop using bootstrap_python for
 deep-freeze in UNIX builds
Message-ID: <1637607109.55.0.706004441577.issue45873@roundup.psfhosted.org>


New submission from Guido van Rossum <guido at python.org>:

Now that we've got a way (for Windows) to deep-freeze without building a bootstrap Python binary, we should do the same for UNIX.

----------
components: Build
messages: 406798
nosy: gvanrossum
priority: normal
severity: normal
stage: needs patch
status: open
title: Stop using bootstrap_python for deep-freeze in UNIX builds
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45873>
_______________________________________

From report at bugs.python.org  Mon Nov 22 17:35:40 2021
From: report at bugs.python.org (Christian Sattler)
Date: Mon, 22 Nov 2021 22:35:40 +0000
Subject: [New-bugs-announce] [issue45874] urllib.parse.parse_qsl does not
 parse empty query string with strict parsing
Message-ID: <1637620540.89.0.438888661904.issue45874@roundup.psfhosted.org>


New submission from Christian Sattler <sattler.christian at web.de>:

Calling
  urllib.parse.parse_qsl('', strict_parsing=True)
yields an error:
  ValueError: bad query field: ''

The empty string '' is produced by
  urllib.parse.urlencode({})
and also as query string by
  urllib.parse.urlsplit('http://example.org/')
so it should be accepted by urllib.parse.parse_qsl with strict parsing. 

The problem is that parse_qsl(qs, ...) cannot distinguish between zero and one query arguments. The call to qs.split(separator) returns the non-empty list [''] for qs empty, which means one query argument. However, in this case, we want the other semantics.

----------
components: Library (Lib)
messages: 406807
nosy: sattler
priority: normal
severity: normal
status: open
title: urllib.parse.parse_qsl does not parse empty query string with strict parsing
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45874>
_______________________________________

From report at bugs.python.org  Tue Nov 23 02:04:34 2021
From: report at bugs.python.org (Ruben Vorderman)
Date: Tue, 23 Nov 2021 07:04:34 +0000
Subject: [New-bugs-announce] [issue45875] gzip.decompress performance can be
 improved with memoryviews
Message-ID: <1637651074.98.0.153102708478.issue45875@roundup.psfhosted.org>


New submission from Ruben Vorderman <r.h.p.vorderman at lumc.nl>:

The current implementation uses a lot of bytestring slicing. While it is much better than the 3.10 and earlier implementations, it can still be further improved by using memoryviews instead.

Possibly. I will check this out.

----------
components: Library (Lib)
messages: 406816
nosy: rhpvorderman
priority: normal
severity: normal
status: open
title: gzip.decompress performance can be improved with memoryviews
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45875>
_______________________________________

From report at bugs.python.org  Tue Nov 23 03:21:50 2021
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 23 Nov 2021 08:21:50 +0000
Subject: [New-bugs-announce] [issue45876] Improve accuracy of stdev
 functions in statistics
Message-ID: <1637655710.57.0.966606985611.issue45876@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

The standard deviation computation in the statistics module is still subject to error even though the mean and sum of square differences are computed exactly using fractions.

The problem is that the exact fraction gets rounded to a float before going into math.sqrt() which also rounds.

It would be better to compute a high precision square root directly from the exact fraction rather than from a fraction rounded to a float.

Here are two ways to do it.  With Cpython, the second way is twice as fast.   With PyPy3, the speed is about the same. 


def frac_sqrt(x: Fraction) -> float:
    'Return sqrt to greater precision than math.sqrt()'
    # Needed because a correctly rounded square root of
    # a correctly rounded input can still be off by 1 ulp.
    # Here we avoid the initial rounding and work directly
    # will the exact fractional input.  The square root
    # is first approximated with math.sqrt() and then
    # refined with a divide-and-average step. Since the
    # Newton-Raphson algorithm has quadratic convergence,
    # one refinement step gives excellent accuracy.
    a = Fraction(math.sqrt(x))
    return float((x / a + a) / 2)


def deci_sqrt(x: Fraction) -> float:
    ratio = Decimal(x.numerator) / Decimal(x.denominator)
    return float(ratio.sqrt())

----------
assignee: rhettinger
components: Library (Lib)
messages: 406824
nosy: rhettinger, steven.daprano
priority: normal
severity: normal
status: open
title: Improve accuracy of stdev functions in statistics
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45876>
_______________________________________

From report at bugs.python.org  Tue Nov 23 04:01:10 2021
From: report at bugs.python.org (Oleg Iarygin)
Date: Tue, 23 Nov 2021 09:01:10 +0000
Subject: [New-bugs-announce] [issue45877] Inconsistency in minimal supported
 version of Microsoft Visual Studio
Message-ID: <1637658070.78.0.512777632516.issue45877@roundup.psfhosted.org>


New submission from Oleg Iarygin <oleg at arhadthedev.net>:

According to PCbuild/readme.txt:

> Using this directory requires an installation of
> Microsoft Visual Studio 2017 (MSVC 14.1) of any edition.

However, Doc/using/windows.rst states that Visual Studio 2017 is not required and 2015 may be used instead:

> The source tree contains a build solution and project files for Microsoft
> Visual Studio 2015, which is the compiler used to build the official Python
> releases.

VS 2017 mention was introduced in GH-3418. So either PCbuild readme needs to relax its requirements back or windows.rst needs to be updated.

----------
assignee: docs at python
components: Build, Documentation
messages: 406826
nosy: arhadthedev, docs at python, steve.dower
priority: normal
severity: normal
status: open
title: Inconsistency in minimal supported version of Microsoft Visual Studio
type: behavior
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45877>
_______________________________________

From report at bugs.python.org  Tue Nov 23 04:32:54 2021
From: report at bugs.python.org (Nikita Sobolev)
Date: Tue, 23 Nov 2021 09:32:54 +0000
Subject: [New-bugs-announce] [issue45878] Use `self.assertRaises` instead of
 `try/except` in `ctypes/test_functions.py::test_mro`
Message-ID: <1637659974.6.0.574362436807.issue45878@roundup.psfhosted.org>


New submission from Nikita Sobolev <mail at sobolevn.me>:

Right now this test uses `try: ... except TypeError: ...` to ensure that mro is consistent. This has a flaw: code in `try` might not ever raise and this test would still pass.

I will refactor it to use `self.assertRaises` to be 100% sure.

----------
components: Tests
messages: 406829
nosy: sobolevn
priority: normal
severity: normal
status: open
title: Use `self.assertRaises` instead of `try/except` in `ctypes/test_functions.py::test_mro`
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45878>
_______________________________________

From report at bugs.python.org  Tue Nov 23 08:49:04 2021
From: report at bugs.python.org (vladexl)
Date: Tue, 23 Nov 2021 13:49:04 +0000
Subject: [New-bugs-announce] [issue45879] Access violation
Message-ID: <1637675344.51.0.888057929265.issue45879@roundup.psfhosted.org>


New submission from vladexl <vladexl at yandex.ru>:

It seems object.c hasn't taken into account possible nullptr:

object.c: ...else if (Py_TYPE(v)->tp_as_number != NULL &&...

It seems Py_TYPE(v) returns null

StackTrace:

> python310.dll!PyObject_IsTrue(_object * v=0x1d91e2d4) Line 1444 C
  python310.dll!_PyEval_EvalFrameDefault(_ts * tstate=0x1db76f50, _frame * f=0x1c064028, int throwflag=0) Line 3793 C
  [Inline Frame] python310.dll!_PyEval_EvalFrame(_ts *) Line 46 C
  python310.dll!_PyEval_Vector(_ts * tstate=0x1db76f50, PyFrameConstructor * con=0x1bd91858, _object * locals=0x00000000, _object * const * args=0x1bdf86a8, unsigned int argcount=1, _object * kwnames=0x00000000) Line 5080 C
  python310.dll!_PyFunction_Vectorcall(_object * func=0x1bd91850, _object * const * stack=0x1bdf86a8, unsigned int nargsf=2147483649, _object * kwnames=0x00000000) Line 347 C
  [Inline Frame] python310.dll!_PyObject_VectorcallTstate(_ts *) Line 114 C
  python310.dll!PyObject_Vectorcall(_object * callable=0x1bd91850, _object * const * args, unsigned int nargsf=2147483649, _object * kwnames=0x00000000) Line 123 C
  python310.dll!call_function(_ts * tstate=0x1db76f50, PyTraceInfo * trace_info=0x1bf3f7a0, _object * * * pp_stack=0x1bf3f774, int oparg=1, _object * kwnames=0x00000000) Line 5888 C
  python310.dll!_PyEval_EvalFrameDefault(_ts * tstate=0x1db76f50, _frame * f=0x1bdf8568, int throwflag=0) Line 4222 C
  [Inline Frame] python310.dll!_PyEval_EvalFrame(_ts *) Line 46 C
  python310.dll!_PyEval_Vector(_ts * tstate=0x1db76f50, PyFrameConstructor * con=0x1bd918a0, _object * locals=0x00000000, _object * const * args=0x1bde5334, unsigned int argcount=2, _object * kwnames=0x00000000) Line 5080 C
  python310.dll!_PyFunction_Vectorcall(_object * func=0x1bd91898, _object * const * stack=0x1bde5334, unsigned int nargsf=2, _object * kwnames=0x00000000) Line 347 C
  python310.dll!PyVectorcall_Call(_object * callable=0x1bd91898, _object * tuple=0x1bde5328, _object * kwargs=0x1ef3d618) Line 272 C
  python310.dll!_PyObject_Call(_ts * tstate=0x1db76f50, _object * callable=0x1bd91898, _object * args=0x1bde5328, _object * kwargs=0x1ef3d618) Line 290 C
  [Inline Frame] python310.dll!PyObject_Call(_object *) Line 317 C
  python310.dll!do_call_core(_ts * tstate=0x1db76f50, PyTraceInfo * trace_info=0x1bf3f93c, _object * func=0x1bd91898, _object * callargs=0x1bde5328, _object * kwdict=0x1ef3d618) Line 5940 C
  python310.dll!_PyEval_EvalFrameDefault(_ts * tstate=0x1db76f50, _frame * f=0x1bd7ab28, int throwflag=0) Line 4286 C
  [Inline Frame] python310.dll!_PyEval_EvalFrame(_ts *) Line 46 C
  python310.dll!_PyEval_Vector(_ts * tstate=0x1db76f50, PyFrameConstructor * con=0x1bdef930, _object * locals=0x00000000, _object * const * args=0x1bdfa164, unsigned int argcount=1, _object * kwnames=0x00000000) Line 5080 C
  python310.dll!_PyFunction_Vectorcall(_object * func=0x1bdef928, _object * const * stack=0x1bdfa164, unsigned int nargsf=2147483649, _object * kwnames=0x00000000) Line 347 C
  [Inline Frame] python310.dll!_PyObject_VectorcallTstate(_ts *) Line 114 C
  python310.dll!PyObject_Vectorcall(_object * callable=0x1bdef928, _object * const * args, unsigned int nargsf=2147483649, _object * kwnames=0x00000000) Line 123 C
  python310.dll!call_function(_ts * tstate=0x1db76f50, PyTraceInfo * trace_info=0x1bf3fa94, _object * * * pp_stack=0x1bf3fa70, int oparg=1, _object * kwnames=0x00000000) Line 5888 C
  python310.dll!_PyEval_EvalFrameDefault(_ts * tstate=0x1db76f50, _frame * f=0x1bdfa028, int throwflag=0) Line 4207 C
  [Inline Frame] python310.dll!_PyEval_EvalFrame(_ts *) Line 46 C
  python310.dll!_PyEval_Vector(_ts * tstate=0x1db76f50, PyFrameConstructor * con=0x1bdefa98, _object * locals=0x00000000, _object * const * args=0x1bc9db04, unsigned int argcount=1, _object * kwnames=0x00000000) Line 5080 C
  python310.dll!_PyFunction_Vectorcall(_object * func=0x1bdefa90, _object * const * stack=0x1bc9db04, unsigned int nargsf=2147483649, _object * kwnames=0x00000000) Line 347 C
  [Inline Frame] python310.dll!_PyObject_VectorcallTstate(_ts *) Line 114 C
  python310.dll!PyObject_Vectorcall(_object * callable=0x1bdefa90, _object * const * args, unsigned int nargsf=2147483649, _object * kwnames=0x00000000) Line 123 C
  python310.dll!call_function(_ts * tstate=0x1db76f50, PyTraceInfo * trace_info=0x1bf3fbec, _object * * * pp_stack=0x1bf3fbc8, int oparg=1, _object * kwnames=0x00000000) Line 5888 C
  python310.dll!_PyEval_EvalFrameDefault(_ts * tstate=0x1db76f50, _frame * f=0x1bc9d9c8, int throwflag=0) Line 4207 C
  [Inline Frame] python310.dll!_PyEval_EvalFrame(_ts *) Line 46 C
  python310.dll!_PyEval_Vector(_ts * tstate=0x1db76f50, PyFrameConstructor * con=0x1bdef978, _object * locals=0x00000000, _object * const * args=0x1bf3fce8, unsigned int argcount=1, _object * kwnames=0x00000000) Line 5080 C
  python310.dll!_PyFunction_Vectorcall(_object * func=0x1bdef970, _object * const * stack=0x1bf3fce8, unsigned int nargsf=1, _object * kwnames=0x00000000) Line 347 C
  python310.dll!_PyObject_VectorcallTstate(_ts * tstate=0x1db76f50, _object * callable=0x1bdef970, _object * const * args=0x1bf3fce8, unsigned int nargsf=1, _object * kwnames=0x00000000) Line 115 C
  python310.dll!method_vectorcall(_object * method=0x1e8f56e8, _object * const * args=0x087b41e4, unsigned int nargsf=0, _object * kwnames=0x00000000) Line 61 C
  python310.dll!PyVectorcall_Call(_object * callable=0x1e8f56e8, _object * tuple=0x087b41d8, _object * kwargs=0x00000000) Line 272 C
  python310.dll!_PyObject_Call(_ts * tstate=0x1db76f50, _object * callable=0x1e8f56e8, _object * args=0x087b41d8, _object * kwargs=0x00000000) Line 290 C
  [Inline Frame] python310.dll!PyObject_Call(_object * callable, _object * args=0x087b41d8, _object * kwargs) Line 317 C
  python310.dll!thread_run(void * boot_raw=0x1bdf12d8) Line 1090 C
  python310.dll!bootstrap(void * call=0x1775b860) Line 183 C
  ucrtbase.dll!thread_start<unsigned int (__stdcall*)(void *),1>() Unknown
  kernel32.dll!@BaseThreadInitThunk at 12() Unknown
  ntdll.dll!__RtlUserThreadStart() Unknown
  ntdll.dll!__RtlUserThreadStart at 8() Unknown

----------
messages: 406841
nosy: vladexl
priority: normal
severity: normal
status: open
title: Access violation
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45879>
_______________________________________

From report at bugs.python.org  Tue Nov 23 10:52:58 2021
From: report at bugs.python.org (Xinhang Xu)
Date: Tue, 23 Nov 2021 15:52:58 +0000
Subject: [New-bugs-announce] [issue45880] Performance regression of Int
 object operators. (Python 3.11)
Message-ID: <1637682778.64.0.753223972817.issue45880@roundup.psfhosted.org>


New submission from Xinhang Xu <xuxinhang4567 at 126.com>:

Hello.

I'm working on a compute-bound project recently, so I tested several Python versions on my PC (Windows 10 64-bit), about Python's performance on operating Int object. All Python binaries are official distributions.


Testcase #1 (simple xor op)

Source:

    import cProfile as profile
    profile.run('for _ in range(500000000): 5 ^ 6')

The given result:

   C:\Users\surface\Desktop\PythonTest>python-3.9.9-embed-amd64\python C:\Users\surface\python_test.py
         3 function calls in 24.398 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   24.398   24.398   24.398   24.398 <string>:1(<module>)
        1    0.000    0.000   24.398   24.398 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}



   C:\Users\surface\Desktop\PythonTest>python-3.10.0-embed-amd64\python C:\Users\surface\python_test.py
         3 function calls in 27.941 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   27.941   27.941   27.941   27.941 <string>:1(<module>)
        1    0.000    0.000   27.941   27.941 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}



   C:\Users\surface\Desktop\PythonTest>python-3.11.0a2-embed-amd64\python C:\Users\surface\python_test.py
         3 function calls in 42.209 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   42.209   42.209   42.209   42.209 <string>:1(<module>)
        1    0.000    0.000   42.209   42.209 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}



Testcase #2 (simple add op)

Source:

    import cProfile as profile
    profile.run('for _ in range(500000000): 5 + 6')

The given result:

   C:\Users\surface\Desktop\PythonTest>python-3.9.9-embed-amd64\python C:\Users\surface\python_test.py
         3 function calls in 24.599 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   24.599   24.599   24.599   24.599 <string>:1(<module>)
        1    0.000    0.000   24.599   24.599 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}



   C:\Users\surface\Desktop\PythonTest>python-3.10.0-embed-amd64\python C:\Users\surface\python_test.py
         3 function calls in 27.414 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   27.414   27.414   27.414   27.414 <string>:1(<module>)
        1    0.000    0.000   27.414   27.414 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}



   C:\Users\surface\Desktop\PythonTest>python-3.11.0a2-embed-amd64\python C:\Users\surface\python_test.py
         3 function calls in 43.675 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   43.675   43.675   43.675   43.675 <string>:1(<module>)
        1    0.000    0.000   43.675   43.675 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}



As you can see, Python 3.11 costs *much more* time to execute Int object operator. I have also tested the same cases on another Windows PC, the result shows the same.

Is it a common thing? What's the reason for this problem?


Thanks.

----------
messages: 406849
nosy: xuxinhang
priority: normal
severity: normal
status: open
title: Performance regression of Int object operators. (Python 3.11)
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45880>
_______________________________________

From report at bugs.python.org  Tue Nov 23 11:37:45 2021
From: report at bugs.python.org (Christian Heimes)
Date: Tue, 23 Nov 2021 16:37:45 +0000
Subject: [New-bugs-announce] [issue45881] Cross compiling on Linux is
 untested, undocumented, and broken
Message-ID: <1637685465.3.0.98765140006.issue45881@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

Cross compiling is an approach to compile a program for a different CPU architecture and platform, e.g. compile for an ARM64 (aarch64) or WASM on a x86_64 build system.

Python configure script, Makefile, and setup.py have multiple references to cross compiling. However I could not find any documentation in the devguide or Python docs how to cross compile. We also lack CI (buildbot) to test cross compiling. This lack of awareness and testing leads to breakage of the feature. For example the design of Programs/_freeze_module in main (3.11-dev) is incompatible with cross compiling.

I kinda got cross compiling working with 3.10, but only with some additional hacks and patches. I also ran into other problems like _PYTHON_HOST_PLATFORM env var is not automatically forwarded to setup.py. The helper functions add_multiarch_paths() and add_cross_compiling_paths() break builds for me, too. Cross compiling only works when the methods are commented out.

----------
assignee: docs at python
components: Build, Cross-Build, Documentation, Tests
messages: 406853
nosy: Alex.Willmer, brett.cannon, christian.heimes, docs at python
priority: normal
severity: normal
status: open
title: Cross compiling on Linux is untested, undocumented, and broken
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45881>
_______________________________________

From report at bugs.python.org  Tue Nov 23 13:18:02 2021
From: report at bugs.python.org (Peter Wurmsdobler)
Date: Tue, 23 Nov 2021 18:18:02 +0000
Subject: [New-bugs-announce] [issue45882] mailbox fails to read message from
 mbox
Message-ID: <1637691482.36.0.788226953049.issue45882@roundup.psfhosted.org>


New submission from Peter Wurmsdobler <peter at wurmsdobler.org>:

When using mailbox to convert mbox files to maildir, I have noticed that it cannot handle messages in the mbox file that start with two subsequent `From` lines in the header like:

```
>From - Fri Feb 20 09:46:18 1998
>From namel at company.com Wed Feb 18 18:38:38 1998
```

A new maildir message will be created, but none of fields following the two header lines such as `Date` etc. will be parsed; the result is a message with header and body inside the message.

----------
components: Library (Lib)
messages: 406873
nosy: peter4
priority: normal
severity: normal
status: open
title: mailbox fails to read message from mbox
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45882>
_______________________________________

From report at bugs.python.org  Tue Nov 23 14:07:56 2021
From: report at bugs.python.org (Jim Crist-Harif)
Date: Tue, 23 Nov 2021 19:07:56 +0000
Subject: [New-bugs-announce] [issue45883] reuse_address mistakenly removed
 from loop.create_server
Message-ID: <1637694476.84.0.389312565282.issue45883@roundup.psfhosted.org>


New submission from Jim Crist-Harif <jcristharif at gmail.com>:

In https://bugs.python.org/issue45129 the deprecated `reuse_address` parameter to `create_datagram_endpoint` was removed. This PR mistakenly removed this parameter from `create_server` as well (where it wasn't deprecated).

----------
components: asyncio
messages: 406876
nosy: asvetlov, jcristharif, yselivanov
priority: normal
severity: normal
status: open
title: reuse_address mistakenly removed from loop.create_server
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45883>
_______________________________________

From report at bugs.python.org  Tue Nov 23 14:11:12 2021
From: report at bugs.python.org (Jon Oxtoby)
Date: Tue, 23 Nov 2021 19:11:12 +0000
Subject: [New-bugs-announce] [issue45884] datetime.strptime incorrectly
 handling hours and minutes with bad format string
Message-ID: <1637694672.86.0.81735779429.issue45884@roundup.psfhosted.org>


New submission from Jon Oxtoby <joxtoby27 at gmail.com>:

Running datetime.strptime against a string containing year, month, day but with a formatter including %H causes a two-digit day to be split across the day and hour fields of the datetime:

datetime.datetime.strptime('20140812', '%Y%m%d%H')
returns: datetime.datetime(2014, 8, 1, 2, 0)
expected: ValueError: time data '20140812' does not match format '%Y%m%d%H'

datetime.datetime.strptime('2014081201', '%Y%m%d%H%M')
returns: datetime.datetime(2014, 8, 12, 0, 1)
expected: ValueError: time data '2014081201' does not match format '%Y%m%d%H%M'

----------
components: Library (Lib)
messages: 406877
nosy: joxtoby27
priority: normal
severity: normal
status: open
title: datetime.strptime incorrectly handling hours and minutes with bad format string
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45884>
_______________________________________

From report at bugs.python.org  Tue Nov 23 16:00:24 2021
From: report at bugs.python.org (Dennis Sweeney)
Date: Tue, 23 Nov 2021 21:00:24 +0000
Subject: [New-bugs-announce] [issue45885] Specialize COMPARE_OP
Message-ID: <1637701224.16.0.881347004039.issue45885@roundup.psfhosted.org>


New submission from Dennis Sweeney <sweeney.dennis650 at gmail.com>:

Some specialization statistics:

https://gist.github.com/sweeneyde/49cc3a9d074d56cf095cb0a42d13d7a4

Including 3 opcodes: COMPARE_OP_INT and COMPARE_OP_FLOAT and COMPARE_OP_STR (equality only) seems to give pretty good specialization numbers, better than just 2.

----------
components: Interpreter Core
messages: 406887
nosy: Dennis Sweeney
priority: normal
severity: normal
status: open
title: Specialize COMPARE_OP
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45885>
_______________________________________

From report at bugs.python.org  Tue Nov 23 17:35:22 2021
From: report at bugs.python.org (Christian Heimes)
Date: Tue, 23 Nov 2021 22:35:22 +0000
Subject: [New-bugs-announce] [issue45886] Fix Program/_freeze_module for
 cross compiling Python
Message-ID: <1637706922.21.0.951800087197.issue45886@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

The build process creates the binary Program/_freeze_module, which is used to create frozen modules. The program takes a Python file, compiles it, and stores its marshalled output in a header file.

The approach does not work when cross compiling Python. In cross building case Program/_freeze_module cannot be executed on the build host. For example a cross build with build host "x86_64" and target host "aarch64" would create a aarch64 Program/_freeze_module. The current x86_64 host cannot executed binary (unless you use qemu and binfmt, which I'm deliberately ignoring here).

To unblock cross building and until we find a better solution, I propose that we allow developers to override the freeze module command on the command line. This allows developers to use a freeze_module program from a non-cross build:

   ../../configure -C --host=aarch64-linux-gnu-gcc --build=x86_64-pc-linux-gnu
   make FREEZE_MODULE=../x86_64/Program/_freeze_module

----------
assignee: christian.heimes
components: Build
messages: 406893
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Fix Program/_freeze_module for cross compiling Python
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45886>
_______________________________________

From report at bugs.python.org  Tue Nov 23 17:40:54 2021
From: report at bugs.python.org (Eric Snow)
Date: Tue, 23 Nov 2021 22:40:54 +0000
Subject: [New-bugs-announce] [issue45887] [subinterpreters] Pull all
 interpreter-global objects into one place.
Message-ID: <1637707254.12.0.17655589949.issue45887@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

Currently there are still a bunch of PyObject static variables in the code that need to become per-interpreter.  This includes quite a few static types (e.g. in Objects/*.c), as well as freelists and caches.  To make the transition easier I'd like to move all those objects under a single struct.

When I started consolidating globals a few years back, my plan was to turn static variables into fields on the _PyRuntimeState struct, where they can later be moved down into PyInterpreterState and become per-interpreter.  That has worked fine but the mental indirection in that process is clunky.  Consequently, in practice we've ended up just moving things directly to PyInterpreterState, even in cases where making something per-interpreter is premature.

So at this point I'm planning on a slightly different approach.  We'll move the objects (and other state) to PyInterpreterState as pointer fields, and then use the main interpreter's pointers in the corresponding fields in all subinterpreters.  Thus it will be equivalent to having a single global state.  However, it will go smoother down the road when we make all that state unique to each interpreter.

----------
assignee: eric.snow
components: Interpreter Core
messages: 406894
nosy: Mark.Shannon, eric.snow, ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: [subinterpreters] Pull all interpreter-global objects into one place.
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45887>
_______________________________________

From report at bugs.python.org  Wed Nov 24 02:37:09 2021
From: report at bugs.python.org (Rob Blackbourn)
Date: Wed, 24 Nov 2021 07:37:09 +0000
Subject: [New-bugs-announce] [issue45888] Revisit: start_tls() difficult
 when using asyncio.start_server()
Message-ID: <1637739429.36.0.799749161441.issue45888@roundup.psfhosted.org>


New submission from Rob Blackbourn <rob.blackbourn at googlemail.com>:

The issue 34975 "start_tls() difficult when using asyncio.start_server()" was closed because streams was being re-written, but it's still a useful enhancement, and a fairly simple change.

Could this be revisited?

I've done a proof of concept here: https://github.com/rob-blackbourn/asyncio-upgradeable-streams

----------
assignee: christian.heimes
components: Library (Lib), SSL
messages: 406906
nosy: christian.heimes, rob-blackbourn
priority: normal
severity: normal
status: open
title: Revisit: start_tls() difficult when using asyncio.start_server()
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45888>
_______________________________________

From report at bugs.python.org  Wed Nov 24 02:54:31 2021
From: report at bugs.python.org (Nick Papior)
Date: Wed, 24 Nov 2021 07:54:31 +0000
Subject: [New-bugs-announce] [issue45889] pathlib: Path.match does not work
 on paths
Message-ID: <1637740471.98.0.448189048749.issue45889@roundup.psfhosted.org>


New submission from Nick Papior <nickpapior at gmail.com>:

The documentation of Path.match only says it will match a pattern.

But quite often this pattern may be desirable to have as a Path as well.

import pathlib as pl

path = pl.Path("foo/bar")
print(path.match("bar"))
print(path.match(pl.Path("bar")))

However, the last one fails and one has to resort to 

print(path.match(str(pl.Path("bar"))))

which in my opinion is a little misleading.

I couldn't find any other bug/enhancement report of this. Also, this probably also targets later versions.

----------
components: Library (Lib)
messages: 406910
nosy: nickpapior
priority: normal
severity: normal
status: open
title: pathlib: Path.match does not work on paths
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45889>
_______________________________________

From report at bugs.python.org  Wed Nov 24 06:23:45 2021
From: report at bugs.python.org (Irit Katriel)
Date: Wed, 24 Nov 2021 11:23:45 +0000
Subject: [New-bugs-announce] [issue45890] Add tests for tracing
 try-except-finally blocks
Message-ID: <1637753025.86.0.0360448888462.issue45890@roundup.psfhosted.org>


Change by Irit Katriel <iritkatriel at gmail.com>:


----------
assignee: iritkatriel
components: Interpreter Core, Tests
nosy: iritkatriel
priority: normal
severity: normal
status: open
title: Add tests for tracing try-except-finally blocks
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45890>
_______________________________________

From report at bugs.python.org  Wed Nov 24 08:04:10 2021
From: report at bugs.python.org (noobie1000)
Date: Wed, 24 Nov 2021 13:04:10 +0000
Subject: [New-bugs-announce] [issue45891] bool variable isinstance of int
Message-ID: <1637759050.81.0.989804439301.issue45891@roundup.psfhosted.org>


New submission from noobie1000 <balajesuri at gmail.com>:

Hello,

Recently I observed that isinstance(x, int) returns True, even when x is defined as a bool. While I understand that internally, a bool is treated as an int with values 0 and 1; to me, this is a bit misleading that the python interpreter returns True when we perform the isinstance() check. May be I'm missing some deeper explanation. Could someone please shed some light on this. Has this been discussed by the community in the past. Thank you very much for reading this ticket.

>>> x = True
>>> isinstance(x, int)
True

----------
components: IO
messages: 406926
nosy: noobie1000
priority: normal
severity: normal
status: open
title: bool variable isinstance of int
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45891>
_______________________________________

From report at bugs.python.org  Wed Nov 24 10:55:50 2021
From: report at bugs.python.org (Alex)
Date: Wed, 24 Nov 2021 15:55:50 +0000
Subject: [New-bugs-announce] [issue45892] Improve REPL with the underscore
 separators for int/float
Message-ID: <1637769350.02.0.325532195868.issue45892@roundup.psfhosted.org>


New submission from Alex <alexprengere at gmail.com>:

I often use to the Python REPL to perform some simple calculations (typically some combinatorial or probabilities computation).
I believe it would be a nice improvement if the number displayed in the REPL would be formatted as if f"{result:_}" was given (with the underscore separators). For example:

 >>> 36 ** 7
 78_364_164_096

As I understand things:
* the REPL always shows the __repr__
* updating the __repr__ for int/float is a no-no for backward compatibility reasons

If these assumptions are correct (please correct me if this is wrong), then I guess this cannot be implemented, unless I missed something?

----------
components: Interpreter Core
messages: 406934
nosy: alexprengere
priority: normal
severity: normal
status: open
title: Improve REPL with the underscore separators for int/float
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45892>
_______________________________________

From report at bugs.python.org  Wed Nov 24 13:50:21 2021
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 24 Nov 2021 18:50:21 +0000
Subject: [New-bugs-announce] [issue45893] Azure Pipelines currently failing
Message-ID: <1637779821.28.0.900585770279.issue45893@roundup.psfhosted.org>


New submission from Guido van Rossum <guido at python.org>:

E.g. https://dev.azure.com/Python/cpython/_build/results?buildId=92084&view=results

----------
assignee: steve.dower
components: Build
messages: 406945
nosy: gvanrossum, steve.dower
priority: high
severity: normal
status: open
title: Azure Pipelines currently failing

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45893>
_______________________________________

From report at bugs.python.org  Wed Nov 24 15:35:15 2021
From: report at bugs.python.org (Amos Anderson)
Date: Wed, 24 Nov 2021 20:35:15 +0000
Subject: [New-bugs-announce] [issue45894] exception lost when loop.stop() in
 finally
Message-ID: <1637786115.42.0.981078468976.issue45894@roundup.psfhosted.org>


New submission from Amos Anderson <nitroamos at gmail.com>:

I found a case where an exception is lost if the loop is stopped in a `finally`.


```
import asyncio
import logging


logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger()


async def method_that_raises():
    loop = asyncio.get_event_loop()
    try:
        logger.info("raising exception")
        raise ValueError("my exception1")
    # except Exception as e:
    #     logger.info("in catcher")
    #     logger.exception(e)
    #     raise
    finally:
        logger.info("stopped")
        loop.stop()
        # await asyncio.sleep(0.5)

        return


async def another_level():
    try:
        await method_that_raises()
    except Exception as e:
        logger.info("trapping from another_level")
        logger.exception(e)


if __name__ == "__main__":
    logger.info("start")
    try:
        asyncio.run(another_level())
    except Exception as e:
        logger.exception(e)
    logger.info("done")
```

gives this output in python 3.10.0 and 3.8.10 (tested in Ubuntu Windows Subsystem Linux) and 3.8.11 in Windows:

```
INFO:root:start
DEBUG:asyncio:Using selector: EpollSelector
INFO:root:raising exception
INFO:root:stopped
INFO:root:done
```
i.e., no evidence an exception was raised (other than the log message included to prove one was raised)

If I remove the `return`, then the exception propagates as expected.

I believe the exception should be propagated regardless of whether there's a `return` in the `finally` block.

----------
components: asyncio
messages: 406957
nosy: Amos.Anderson, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: exception lost when loop.stop() in finally
type: behavior
versions: Python 3.10, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45894>
_______________________________________

From report at bugs.python.org  Wed Nov 24 16:50:04 2021
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 24 Nov 2021 21:50:04 +0000
Subject: [New-bugs-announce] [issue45895] _Py_Sigset_Converter used without
 #ifdef HAVE_SIGSET_T check
Message-ID: <1637790604.74.0.0454243973029.issue45895@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

posixmodule.c defines _Py_Sigset_Converter() only when feature macro HAVE_SIGSET_T is set. Several use of the function miss the feature macro check and fail on platforms without sigset_t (e.g. wasm).

Modules/posixmodule.c:5939:14: error: implicit declaration of function '_Py_Sigset_Converter' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
Modules/posixmodule.c:5952:14: error: implicit declaration of function '_Py_Sigset_Converter' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
Modules/clinic/signalmodule.c.h:385:10: error: implicit declaration of function '_Py_Sigset_Converter' is invalid in C99 [-Werror,-Wimplicit-function-declaration]

----------
components: Build, Cross-Build
messages: 406964
nosy: Alex.Willmer, brett.cannon, christian.heimes
priority: normal
severity: normal
stage: needs patch
status: open
title: _Py_Sigset_Converter used without #ifdef HAVE_SIGSET_T check
type: compile error
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45895>
_______________________________________

From report at bugs.python.org  Wed Nov 24 17:15:22 2021
From: report at bugs.python.org (Rob)
Date: Wed, 24 Nov 2021 22:15:22 +0000
Subject: [New-bugs-announce] [issue45896] Conflicting statements in docs
 about default support for asyncio subprocesses on Windows
Message-ID: <1637792122.45.0.429203503373.issue45896@roundup.psfhosted.org>


New submission from Rob <rob at appkingsoftware.com>:

Hi,

In the docs for the asyncio event loop, it has a note on page:
https://docs.python.org/3/library/asyncio-eventloop.html#running-subprocesses

"Note The default asyncio event loop on Windows does not support subprocesses. See Subprocess Support on Windows for details."

Then following the link in that note to:
https://docs.python.org/3/library/asyncio-platforms.html#subprocess-support-on-windows
Says:
"On Windows, the default event loop ProactorEventLoop supports subprocesses, whereas SelectorEventLoop does not."

So the issue is, there are conflicting statements about default support for asyncio subprocesses on Windows. It seems the first statement listed above, is wrong or outdated since the default event loop on Windows is the ProactorEventLoop which does support subprocesses.

Thank you!

----------
assignee: docs at python
components: Documentation
messages: 406967
nosy: Rob4226, docs at python
priority: normal
severity: normal
status: open
title: Conflicting statements in docs about default support for asyncio subprocesses on Windows
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45896>
_______________________________________

From report at bugs.python.org  Wed Nov 24 20:53:51 2021
From: report at bugs.python.org (Trey Hunner)
Date: Thu, 25 Nov 2021 01:53:51 +0000
Subject: [New-bugs-announce] [issue45897] Frozen dataclasses with slots
 raise TypeError
Message-ID: <1637805231.22.0.691568548115.issue45897@roundup.psfhosted.org>


New submission from Trey Hunner <trey at truthful.technology>:

When making a dataclass with slots=True and frozen=True, assigning to an invalid attribute raises a TypeError rather than a FrozenInstanceError:

>>> from dataclasses import dataclass
>>> @dataclass(frozen=True, slots=True)
... class Vector:
...     x: float
...     y: float
...     z: float
...
>>> v = Vector(1, 2, 3)
>>> v.a = 4
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 5, in __setattr__
TypeError: super(type, obj): obj must be an instance or subtype of type

----------
messages: 406973
nosy: trey
priority: normal
severity: normal
status: open
title: Frozen dataclasses with slots raise TypeError
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45897>
_______________________________________

From report at bugs.python.org  Thu Nov 25 06:57:40 2021
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 25 Nov 2021 11:57:40 +0000
Subject: [New-bugs-announce] [issue45898] ctypes cfield.c defines duplicate
 ffi_type_* symbols
Message-ID: <1637841460.9.0.780352638967.issue45898@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

ctypes's cfield.c redefines a couple of symbols like ffi_type_void and others. The symbols are exported by ffi.h and libffi for more than 12 years: https://github.com/libffi/libffi/blame/e1539266e6c6dde3c99832323586f33f977d1dc0/include/ffi.h.in#L184

I think we can safely remove the symbols from the file. The idea is inspired by pyodide patch https://github.com/pyodide/pyodide/blob/main/cpython/patches/remove-duplicate-symbols-from-cfield.c.patch

----------
components: ctypes
messages: 406991
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: ctypes cfield.c defines duplicate ffi_type_* symbols
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45898>
_______________________________________

From report at bugs.python.org  Thu Nov 25 10:16:34 2021
From: report at bugs.python.org (jhpfjyne)
Date: Thu, 25 Nov 2021 15:16:34 +0000
Subject: [New-bugs-announce] [issue45899] NameError on if clause of
 class-level list comprehension
Message-ID: <1637853394.9.0.0901919134045.issue45899@roundup.psfhosted.org>


New submission from jhpfjyne <jhpfjyne at sharklasers.com>:

Accessing an attribute defined at class-level in the if clause of a list comprehension at class-level throws a NameError.

>>> class Foo:
...     a = ['a', 'b']
...     b = ['b', 'c']
...     c = [x for x in a if x not in b]
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in Foo
  File "<stdin>", line 4, in <listcomp>
NameError: name 'b' is not defined

----------
components: Interpreter Core
messages: 407002
nosy: jhpfjyne
priority: normal
severity: normal
status: open
title: NameError on if clause of class-level list comprehension
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45899>
_______________________________________

From report at bugs.python.org  Thu Nov 25 10:53:03 2021
From: report at bugs.python.org (William George)
Date: Thu, 25 Nov 2021 15:53:03 +0000
Subject: [New-bugs-announce] [issue45900] Type annotations needed for
 convenience functions in ipaddress module
Message-ID: <1637855583.63.0.623282774553.issue45900@roundup.psfhosted.org>


New submission from William George <wrgeorge1983 at gmail.com>:

The convenience factory functions in the ipaddress module each return one of two types (IPv4Network vs IPv6Network, etc).  Modern code wants to be friendly to either stack, and these functions are great at enabling that, but current implementation blocks type inference for most (all?) IDEs.

Proposal is easy enough, specifying return type of e.g. `Union[IPv4Network, IPv6Network]` for these factory functions.  

I believe the rest of the public interface for this module is unambiguous enough that annotations aren't needed, but if others see value they could be added easily enough.

For some of these there exists a version-independent base class that could be referenced instead of a union, but it's not clear to me how well IDEs will actually honor such an annotation referencing an internal class (single-underscore).  My limited testing of that didn't work well and there's no such base class for the Interface classes anyway. 

PR for this incomming.

----------
components: Library (Lib)
messages: 407005
nosy: pmoody, wrgeorge1983
priority: normal
severity: normal
status: open
title: Type annotations needed for convenience functions in ipaddress module
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45900>
_______________________________________

From report at bugs.python.org  Fri Nov 26 01:16:47 2021
From: report at bugs.python.org (Eryk Sun)
Date: Fri, 26 Nov 2021 06:16:47 +0000
Subject: [New-bugs-announce] [issue45901] store app file type ignores
 command-line arguments
Message-ID: <1637907407.05.0.378941758633.issue45901@roundup.psfhosted.org>


New submission from Eryk Sun <eryksun at gmail.com>:

The file association for the store app uses '"%1"' for the command-line parameters. This ignores the rest of the command-line arguments, i.e. '%*'. In PC/layout/support/appxmanifest.py, the add_application() calls that add the "Python" and "PythonW" applications should be changed to use the parameters string '"%1" %*' in the file types.

----------
components: Installation, Windows
messages: 407029
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: store app file type ignores command-line arguments
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45901>
_______________________________________

From report at bugs.python.org  Fri Nov 26 04:36:30 2021
From: report at bugs.python.org (Ruben Vorderman)
Date: Fri, 26 Nov 2021 09:36:30 +0000
Subject: [New-bugs-announce] [issue45902] Bytes and bytesarrays can be
 sorted with a much faster count sort.
Message-ID: <1637919390.64.0.975509345951.issue45902@roundup.psfhosted.org>


New submission from Ruben Vorderman <r.h.p.vorderman at lumc.nl>:

Python now uses the excellent timsort for most (all?) of its sorting. But this is not the fastest sort available for one particular use case.

If the number of possible values in the array is limited, it is possible to perform a counting sort: https://en.wikipedia.org/wiki/Counting_sort. In a counting sort each value maps to an integer corresponding to its relative value. The values are then counted by using key = map_to_key(value); count_array[key]. Then from this count_array a new array of sorted values can be constructed. (See the wikipedia article for a more detailed explanation).

For the python bytes and bytesarray types this is extremely simple to implement. All 256 possible values are can be directly used as keys. 

Rough implementation:
- Use buffer protocol to get pointer to bytes/bytesarray internal c-string
- Declare a count_array: Py_ssize_t[256] count_array . (use memset to set it to 0)
- Iterate over the c-string and add each value to the countarray. count_array[buffer[i]] += 1
- Allocate a new bytes(array) object, or in the case of bytesarray the sorting can be performed inplace when bytesarray.sort() is used. 
- Iterate over the count_array. Get the number of values for each key and use memset to insert the sequences of keys into position.


The most obvious way to implement this algorithm will be as bytesarray.sort() where it is sorted inplace and as bytes.sort() which returns a new sorted bytes object. This is much much faster than using bytes(sorted(bytes)).

I made a quick cython implementation for speed testing here: https://github.com/rhpvorderman/bytes_sort/blob/main/bytes_sort.pyx

Currently to get a sorted bytestring one has to do bytes(sorted(my_bytes)). 

Test results:

# First make sure there is no regression when sorting an empty string
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes(sorted(b''))"
500000 loops, best of 5: 560 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'')"
500000 loops, best of 5: 565 nsec per loop

# Test result for very small strings
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes(sorted(b'abc'))"
500000 loops, best of 5: 628 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'abc')"
500000 loops, best of 5: 578 nsec per loop

# Even on a very small already sorted string, a counting sort is faster.

# Test with a proper string
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes(sorted(b'Let\'s test a proper string now. One that has some value to be sorted.'))"
100000 loops, best of 5: 2.32 usec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'Let\'s test a proper string now. One that has some value to be sorted.')"
500000 loops, best of 5: 674 nsec per loop

More than three times faster!

----------
components: C API
messages: 407032
nosy: rhpvorderman
priority: normal
severity: normal
status: open
title: Bytes and bytesarrays can be sorted with a much faster count sort.
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45902>
_______________________________________

From report at bugs.python.org  Fri Nov 26 09:43:12 2021
From: report at bugs.python.org (Jakub Wilk)
Date: Fri, 26 Nov 2021 14:43:12 +0000
Subject: [New-bugs-announce] =?utf-8?b?W2lzc3VlNDU5MDNdIFdoYXTigJlzIE5l?=
 =?utf-8?q?w_In_Python_3=2E11=3A_wrong_reference_to_Signature=2Efrom=5Fcal?=
 =?utf-8?q?lable?=
Message-ID: <1637937792.91.0.161252797195.issue45903@roundup.psfhosted.org>


New submission from Jakub Wilk <jwilk at jwilk.net>:

<https://docs.python.org/3.11/whatsnew/3.11.html#removed> says:

"Removed from the inspect module: [?] the undocumented Signature.from_callable and Signature.from_function functions, deprecated since Python 3.5; use the Signature.from_callable() method instead."

But Signature.from_callable can't possibly be removed and the suggested replacement at the same time.

I think it should say: "? the undocumented Signature.from_builtin and ?"

----------
assignee: docs at python
components: Documentation
messages: 407054
nosy: docs at python, hugovk, jwilk
priority: normal
severity: normal
status: open
title: What?s New In Python 3.11: wrong reference to Signature.from_callable
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45903>
_______________________________________

From report at bugs.python.org  Fri Nov 26 11:28:02 2021
From: report at bugs.python.org (George King)
Date: Fri, 26 Nov 2021 16:28:02 +0000
Subject: [New-bugs-announce] [issue45904] Pasting the U00FF character into
 Python REPL misinterprets character
Message-ID: <1637944082.95.0.645505652849.issue45904@roundup.psfhosted.org>


New submission from George King <george.w.king at gmail.com>:

Using macOS 11.6 Terminal.app with Python 3.10.0 installed directly from python.org.

I open the REPL. If I enter `char(0xff)` I get back '?' as expected (U00FF LATIN SMALL LETTER Y WITH DIAERESIS).

However, If I copy this character with surrounding quotes, and then paste it into the REPL, it pastes as '' and evaluates to the empty string.

If I copy it without quotes and then paste into the REPL, I see nothing. When I hit return, the prompt renders as `>>> ^M>>>`. This suggests that the character is getting misinterpreted as a control character or something.

If I paste it into the terminal shell when the Python REPL is not running, it appears as the latin1 letter that I expect.

If I run `python3 -c 'print("?")'` the character prints fine.

It seems to me that the python REPL is setting some terminal mode that fails on this particular character. Perhaps this is a problem with the macOS readline/libedit implementation?

It seems that only U00FF is problematic; U00FE and U01000 both paste in just fine.

I verified that my terminal profile is set to UTF-8 encoding. I also repeated this experiment in the Kitty terminal emulator, and got identical results.


Here is the readline version:
>>> readline._READLINE_LIBRARY_VERSION
'EditLine wrapper'
>>> readline._READLINE_RUNTIME_VERSION
1026
>>> readline._READLINE_VERSION
1026

----------
messages: 407065
nosy: gwk
priority: normal
severity: normal
status: open
title: Pasting the U00FF character into Python REPL misinterprets character
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45904>
_______________________________________

From report at bugs.python.org  Fri Nov 26 11:41:33 2021
From: report at bugs.python.org (Mark Shannon)
Date: Fri, 26 Nov 2021 16:41:33 +0000
Subject: [New-bugs-announce] [issue45905] Provide a C API for introspectable
 frames for Cython and similar tools
Message-ID: <1637944893.57.0.969509870806.issue45905@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

See https://github.com/cython/cython/issues/4484

----------
messages: 407069
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Provide a C API for introspectable frames for Cython and similar tools

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45905>
_______________________________________

From report at bugs.python.org  Fri Nov 26 15:07:38 2021
From: report at bugs.python.org (Some User)
Date: Fri, 26 Nov 2021 20:07:38 +0000
Subject: [New-bugs-announce] [issue45906] Python github installatiomn issue
Message-ID: <1637957258.27.0.535906199774.issue45906@roundup.psfhosted.org>


New submission from Some User <mazen001.ahmed001 at gmail.com>:

Whenever i try to setup python3.9 configure via github, It raises a configure error.
``
$ ./configure
checking build system type... type
checking host system type... type
checking for python3.9... no
checking for python3... no
checking for python... no
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... "linux"
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... configure: error: in `/path/folder_which_contains_cpython_folder/cpython-3.9':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
```

----------
components: Installation
messages: 407089
nosy: mazen001.ahmed001
priority: normal
severity: normal
status: open
title: Python github installatiomn issue
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45906>
_______________________________________

From report at bugs.python.org  Fri Nov 26 18:59:07 2021
From: report at bugs.python.org (theeshallnotknowethme)
Date: Fri, 26 Nov 2021 23:59:07 +0000
Subject: [New-bugs-announce] [issue45907] Optimize literal comparisons and
 contains
Message-ID: <1637971147.39.0.0141708155517.issue45907@roundup.psfhosted.org>


New submission from theeshallnotknowethme <nohackingofkrowten at gmail.com>:

Most operations with literals are optimized as well, so why shouldn't the comparison/contain operators be? I created a new bug report for it because of this fact and the TODO in the `fold_compare` function in `Python/ast_opt.c`.

----------
components: Interpreter Core
messages: 407114
nosy: February291948
priority: normal
severity: normal
status: open
title: Optimize literal comparisons and contains
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45907>
_______________________________________

From report at bugs.python.org  Sat Nov 27 03:02:49 2021
From: report at bugs.python.org (=?utf-8?b?VmVkcmFuIMSMYcSNacSH?=)
Date: Sat, 27 Nov 2021 08:02:49 +0000
Subject: [New-bugs-announce] [issue45908] dict.fromkeys insertion order
Message-ID: <1638000169.24.0.785616834749.issue45908@roundup.psfhosted.org>


New submission from Vedran ?a?i? <vedgar at gmail.com>:

I'm sure this is exactly how it should work, I just want to know if you think it is documented properly, so I can rely on it. In my opinion the docs should be more precise.

>>> ''.join(dict.fromkeys('axbxc'))
'axbc'

Is this guaranteed by the documentation? Of course, dict iteration order is now guaranteed to be insertion order, but still, nowhere do the docs say that fromkeys inserts the keys into new dictionary in order in which they appear in its argument.

(Probably the reason for this is that dict iteration order was fixed in 3.7, yet fromkeys existed a long time before that.)

I propose an addition to the documentation:

> Create a new dictionary with keys from iterable (in order) and values set to value.

https://docs.python.org/3/library/stdtypes.html

----------
assignee: docs at python
components: Documentation
messages: 407136
nosy: docs at python, veky
priority: normal
severity: normal
status: open
title: dict.fromkeys insertion order
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45908>
_______________________________________

From report at bugs.python.org  Sat Nov 27 07:05:59 2021
From: report at bugs.python.org (Christian Heimes)
Date: Sat, 27 Nov 2021 12:05:59 +0000
Subject: [New-bugs-announce] [issue45909] sysconfig --generate-posix-vars
 creates wrong file when cross compiling
Message-ID: <1638014759.88.0.771169774023.issue45909@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

"sysconfig --generate-posix-vars" creates pybuilddir.txt and a platform-specific sysconfig data file like build/lib.linux-x86_64-3.11/_sysconfigdata__linux_x86_64-linux-gnu.py

When creating a cross-compile build of Python, sysconfig mixes data from the cross compile build and the host build. It creates a pybuilddir.txt and build/lib directory with correct values (e.g. wasm32-unknown-emscripten) but sysconfigdata file with name values from the host Python PYTHON_FOR_BUILD (e.g x86_64-unknown-linux-gnu). 

$ cat pybuilddir.txt 
build/lib.wasm32-unknown-emscripten-3.11

$ ls build/lib.wasm32-unknown-emscripten-3.11/_sysconfigdata*                           
build/lib.wasm32-unknown-emscripten-3.11/_sysconfigdata__linux_x86_64-linux-gnu.py

$ grep CC build/lib.wasm32-unknown-emscripten-3.11/_sysconfigdata__linux_x86_64-linux-gnu.py 
 'CC': 'gcc',

$ grep ^CC Makefile
CC=             emcc

----------
components: Build, Cross-Build
messages: 407141
nosy: Alex.Willmer, christian.heimes
priority: normal
severity: normal
status: open
title: sysconfig --generate-posix-vars creates wrong file when cross compiling
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45909>
_______________________________________

From report at bugs.python.org  Sat Nov 27 11:51:40 2021
From: report at bugs.python.org (bpoaugust)
Date: Sat, 27 Nov 2021 16:51:40 +0000
Subject: [New-bugs-announce] [issue45910] mailbox should support options for
 calling email parser
Message-ID: <1638031900.48.0.213018487662.issue45910@roundup.psfhosted.org>


New submission from bpoaugust <sebbaz+bpo at gmail.com>:

It looks like mailbox uses email.message_from_... for parsing emails.

However it does not allow for passing any options to the parser.

In particular the policy cannot be provided.

It would be useful if there was a way to pass such options.

----------
components: email
messages: 407154
nosy: barry, bpoaugust, r.david.murray
priority: normal
severity: normal
status: open
title: mailbox should support options for calling email parser

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45910>
_______________________________________

From report at bugs.python.org  Sat Nov 27 21:26:51 2021
From: report at bugs.python.org (xmlsax)
Date: Sun, 28 Nov 2021 02:26:51 +0000
Subject: [New-bugs-announce] [issue45911] SystemError occured while running
 an extention
Message-ID: <1638066411.19.0.560872886041.issue45911@roundup.psfhosted.org>


New submission from xmlsax <1627213803 at qq.com>:

I got an <SystemError> while running openfile.open. 
Python release 3.9.6

----------
components: C API
files: openfile.c
messages: 407182
nosy: xmlsax
priority: normal
severity: normal
status: open
title: SystemError occured while running an extention
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file50460/openfile.c

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45911>
_______________________________________

From report at bugs.python.org  Sat Nov 27 21:53:41 2021
From: report at bugs.python.org (Silvio Clecio)
Date: Sun, 28 Nov 2021 02:53:41 +0000
Subject: [New-bugs-announce] [issue45912] [argparse] Print texts starting
 with capital letters and finish with dot for more formality
Message-ID: <1638068021.43.0.0311621080056.issue45912@roundup.psfhosted.org>


New submission from Silvio Clecio <silvioprog at gmail.com>:

Using a simple program as example, the argparse library prints the following text:

usage: app.py [-h]

options:
  -h, --help  show this help message and exit

However, for more formality, it would be nice to print something like this:

Usage: app.py [-h]

Options:
  -h, --help  Show this help message and exit.

Notice the sentences start as capital letters and the helper string finishes with dot.

----------
components: Parser
messages: 407183
nosy: lys.nikolaou, pablogsal, silvioprog
priority: normal
severity: normal
status: open
title: [argparse] Print texts starting with capital letters and finish with dot for more formality

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45912>
_______________________________________

From report at bugs.python.org  Sat Nov 27 22:53:41 2021
From: report at bugs.python.org (Dan Yeaw)
Date: Sun, 28 Nov 2021 03:53:41 +0000
Subject: [New-bugs-announce] [issue45913] Doctest Seg Fault with Python 3.10
 on CI
Message-ID: <1638071621.44.0.368027831233.issue45913@roundup.psfhosted.org>


New submission from Dan Yeaw <dan at yeaw.me>:

When running pytest --doctest-modules, I am getting seg faults on the GitHub Actions CI when running doctests covering module docstrings.

runner at fv-az177-300:~/work/gaphor/gaphor$ source .venv/bin/activate
(.venv) runner at fv-az177-300:~/work/gaphor/gaphor$ xvfb-run gdb python

(gdb) run -m pytest
Starting program: /home/runner/work/gaphor/gaphor/.venv/bin/python -m pytest
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffecd32700 (LWP 22846)]
[New Thread 0x7fffebd31700 (LWP 22847)]
[New Thread 0x7fffead30700 (LWP 22848)]
[New Thread 0x7fffe9d2f700 (LWP 22849)]
[New Thread 0x7fffdbfff700 (LWP 22850)]
[New Thread 0x7fffdaffe700 (LWP 22851)]
[New Thread 0x7fffd9ffd700 (LWP 22852)]
[New Thread 0x7fffcffff700 (LWP 22853)]
[Detaching after fork from child process 22854]
============================= test session starts ==============================
platform linux -- Python 3.10.0, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/runner/work/gaphor/gaphor, configfile: pyproject.toml, testpaths: gaphor, tests, docs
plugins: mock-3.6.1, cov-3.0.0
collected 1135 items                                                           

gaphor/action.py 
Thread 1 "python" received signal SIGSEGV, Segmentation fault.
__GI___libc_free (mem=0x20) at malloc.c:3102
3102    malloc.c: No such file or directory.
(gdb) source python-gdb.py
(gdb) py-bt
Traceback (most recent call first):
  <built-in method create_dynamic of module object at remote 0x7ffff78ebcb0>
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1176, in create_module
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap>", line 674, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/pdb.py", line 157, in __init__
    import readline
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/doctest.py", line 364, in __init__
    pdb.Pdb.__init__(self, stdout=out, nosigint=True)
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/doctest.py", line 1481, in run
    self.debugger = _OutputRedirectingPdb(save_stdout)
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/doctest.py", line 1856, in run
    r = DocTestRunner.run(self, test, compileflags, out, False)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/doctest.py", line 287, in runtest
    self.runner.run(self.dtest, out=failures)  # type: ignore[arg-type]
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 162, in pytest_runtest_call
    item.runtest()
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 255, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 311, in from_call
    result: Optional[TResult] = func()
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 254, in call_runtest_hook
    return CallInfo.from_call(
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 215, in call_and_report
    call = call_runtest_hook(item, when, **kwds)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 126, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 109, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/main.py", line 348, in pytest_runtestloop
    item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/main.py", line 323, in _main
    config.hook.pytest_runtestloop(session=session)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/main.py", line 269, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pytest/__main__.py", line 5, in <module>
    raise SystemExit(pytest.console_main())
  <built-in method exec of module object at remote 0x7ffff78d3d10>
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,

action.py is part of the Gaphor project here: https://github.com/gaphor/gaphor

The doctest that it is seg faulting on is:

    >>> class A:
    ...     @action(name="my_action", label="my action")
    ...     def myaction(self):
    ...         print("action called")
    >>> a = A()
    >>> a.myaction()
    action called
    >>> is_action(a.myaction)
    True
    >>> for method in dir(A):
    ...     if is_action(getattr(A, method, None)):
    ...         print(method)
    myaction
    >>> A.myaction.__action__.name
    'my_action'
    >>> A.myaction.__action__.label
    'my action'
    """

Running the doctests works fine manually:
dan at localhost:~/Projects/gaphor> python -m doctest -v gaphor/action.py
Trying:
    class A:
        @action(name="my_action", label="my action")
        def myaction(self):
            print("action called")
Expecting nothing
ok
Trying:
    a = A()
Expecting nothing
ok
Trying:
    a.myaction()
Expecting:
    action called
ok
Trying:
    is_action(a.myaction)
Expecting:
    True
ok
Trying:
    for method in dir(A):
        if is_action(getattr(A, method, None)):
            print(method)
Expecting:
    myaction
ok
Trying:
    A.myaction.__action__.name
Expecting:
    'my_action'
ok
Trying:
    A.myaction.__action__.label
Expecting:
    'my action'
ok
5 items had no tests:
    action
    action.action.__call__
    action.action.__init__
    action.action.detailed_name
    action.is_action
1 items passed all tests:
   7 tests in action.action
7 tests in 6 items.
7 passed and 0 failed.
Test passed.

If I disable this doctests, the other doctests will seg fault as well. Using xdoctest instead of doctest also fixes the seg fault.

I compiled Python 3.10.0 with debug enabled on the GitHub Actions runner, so I can provide the full backtrace if that is helpful.

----------
components: Library (Lib)
messages: 407184
nosy: danyeaw
priority: normal
severity: normal
status: open
title: Doctest Seg Fault with Python 3.10 on CI
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45913>
_______________________________________

From report at bugs.python.org  Sat Nov 27 23:37:54 2021
From: report at bugs.python.org (Chang Zhou)
Date: Sun, 28 Nov 2021 04:37:54 +0000
Subject: [New-bugs-announce] [issue45914] Very first multiprocessing example
 not working on Windows 11
Message-ID: <1638074274.33.0.534994157782.issue45914@roundup.psfhosted.org>


New submission from Chang Zhou <quattrozhou at gmail.com>:

Very first multiprocessing example not working on Windows 11

https://docs.python.org/3/library/multiprocessing.html

from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(f, [1, 2, 3]))

Tried Python 3.7, 3.8, 3.9, 3.10
Also tried Ubuntu which works fine.
But on Windows (clean installed python):


>>> from multiprocessing import Pool
>>> def f(x):
...     return x*x
...
>>> with Pool(5) as p:
...   print(p.map(f, [1, 2, 3]))
...
Process SpawnPoolWorker-14:
Process SpawnPoolWorker-13:
Traceback (most recent call last):
Process SpawnPoolWorker-15:
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 114, in worker
    task = get()
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python39\lib\multiprocessing\queues.py", line 368, in get
    return _ForkingPickler.loads(res)
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 114, in worker
    task = get()
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>
  File "C:\Program Files\Python39\lib\multiprocessing\queues.py", line 368, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 114, in worker
    task = get()
  File "C:\Program Files\Python39\lib\multiprocessing\queues.py", line 368, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>

----------
components: Windows
messages: 407186
nosy: paul.moore, quattrozhou, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Very first multiprocessing example not working on Windows 11
type: crash
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45914>
_______________________________________

From report at bugs.python.org  Sun Nov 28 05:13:06 2021
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 28 Nov 2021 10:13:06 +0000
Subject: [New-bugs-announce] [issue45915] Use fcntl(fd,
 F_GETFD) to check whether an fd is valid
Message-ID: <1638094386.96.0.551592531479.issue45915@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

is_valid_fd() uses dup() or fstat() to check whether an fd is valid. Both operations are costly. 

fcntl() with F_GETFD returns the file descriptor flags (e.g. CLOEXEC) and -1 with errno set to EBADF when fd is not an open file descriptor. It's faster than duplicating + closing a fd or calling fstat(). The idea to use fcntl(fd, F_GETFD) is inspired by the patch [1]. According to Stackoverflow [2]:

> fcntl(fd, F_GETFD) is the canonical cheapest way to check that fd is a valid open file descriptor.
> F_GETFD is cheaper in principle since it only dereferences the (process-local) file descriptor in kernel space, not the underlying open file description (process-shared) which it refers to.

[1] https://github.com/singlestore-labs/cpython/commit/0364554615c79b9364a0acf3038147a999ea2219

[2] https://stackoverflow.com/questions/12340695/how-to-check-if-a-given-file-descriptor-stored-in-a-variable-is-still-valid

----------
components: C API
messages: 407197
nosy: christian.heimes, vstinner
priority: normal
severity: normal
status: open
title: Use fcntl(fd, F_GETFD) to check whether an fd is valid
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45915>
_______________________________________

From report at bugs.python.org  Sun Nov 28 07:14:56 2021
From: report at bugs.python.org (Tony Zhou)
Date: Sun, 28 Nov 2021 12:14:56 +0000
Subject: [New-bugs-announce] [issue45916] documentation link error
Message-ID: <1638101696.82.0.501489879762.issue45916@roundup.psfhosted.org>


New submission from Tony Zhou <tony.zhou at hotmail.com>:

3.10.0 Documentation ? The Python Tutorial ? 15. Floating Point Arithmetic: Issues and Limitationsin 
in the link "The Perils of Floating Point" brings user to https://www.hmbags.tw/
I don't think this is right. please check

----------
messages: 407200
nosy: cookiez6
priority: normal
severity: normal
status: open
title: documentation link error
type: security

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45916>
_______________________________________

From report at bugs.python.org  Sun Nov 28 12:47:29 2021
From: report at bugs.python.org (Gideon)
Date: Sun, 28 Nov 2021 17:47:29 +0000
Subject: [New-bugs-announce] [issue45917] Add math.exp2() function: 2^x
Message-ID: <1638121649.37.0.378311866468.issue45917@roundup.psfhosted.org>


New submission from Gideon <gideonmitchell01 at gmail.com>:

Dear Python Support Team,

I was looking through Python?s list of supported methods in the math module, and I noticed that C99?s exp2 method was not implemented. This method raises 2 to the power of the supplied argument. I understand that it?s pretty trivial to so this in Python using 2**x or math.pow(x, 2), but I think there are a few reasons why we might want to incorporate it:

Uniformity: This method exists most other programming languages and libraries, including numpy.

Consistency: Every math method from C99 except exp2 is in python?s math or cmath module (math.cbrt will be added as of python 3.11).

Triviality: this method is a part of C99 and is also supported by Visual Studio, so it?s very easy to implement. 

Accuracy(?): a libm exp2 is supposedly more accurate than pow(2.0, x), though I don?t really see how this would be the case (See https://bugs.python.org/issue31980)


That said, this method is a little redundant, so I completely understand if this request is rejected

Non-exhaustive list of other languages / libraries that use this method:

Rust: https://docs.rs/libm/0.1.1/libm/fn.exp2.html
Javascript: https://github.com/stdlib-js/math-base-special-exp2
Numpy: https://numpy.org/doc/stable/reference/generated/numpy.exp2.html
C++: https://en.cppreference.com/w/cpp/numeric/math/exp2 (Not authoritative)
Ruby: https://www.rubydoc.info/gems/ruby-mpfi/MPFI%2FMath.exp2

Similar Issues:
https://bugs.python.org/issue44357
https://bugs.python.org/issue31980

----------
components: Library (Lib)
messages: 407214
nosy: Gideon
priority: normal
severity: normal
status: open
title: Add math.exp2() function: 2^x
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45917>
_______________________________________

From report at bugs.python.org  Sun Nov 28 13:05:23 2021
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 28 Nov 2021 18:05:23 +0000
Subject: [New-bugs-announce] [issue45918] Possibly use ROUND_05UP in
 decimal's localcontext() example
Message-ID: <1638122723.34.0.989498425912.issue45918@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Candidate example:

    with localcontext() as ctx:
        ctx.prec += 10        # Perform a higher precision calculation
        ctx.rounding = ROUND_05UP # Avoid double rounding of the final calculation step
        s = calculate_something()
    s = +s  # Round the final result back to the default precision

Thoughts:

* This would highlight the intended purpose of ROUND_05UP.
* Usually, it would slightly improve accuracy.
* OTOH, it would rarely make a difference in practice.

https://docs.python.org/3/library/decimal.html#decimal.localcontext

----------
assignee: rhettinger
components: Documentation
messages: 407215
nosy: mark.dickinson, rhettinger, tim.peters
priority: normal
severity: normal
status: open
title: Possibly use ROUND_05UP in decimal's localcontext() example
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45918>
_______________________________________

From report at bugs.python.org  Sun Nov 28 14:21:47 2021
From: report at bugs.python.org (Eryk Sun)
Date: Sun, 28 Nov 2021 19:21:47 +0000
Subject: [New-bugs-announce] [issue45919] Use WinAPI GetFileType() in
 is_valid_fd()
Message-ID: <1638127307.77.0.0584468495635.issue45919@roundup.psfhosted.org>


New submission from Eryk Sun <eryksun at gmail.com>:

During startup, is_valid_fd() in Python/pylifecycle.c is called to validate stdin, stdout, and stderr. Performance isn't critical here, but every bit helps in reducing startup time. In my tests, implementing this check in Windows via GetFileType((HANDLE)_get_osfhandle(fd)) is 5-6 times faster than close(dup(fd)). For example:

#if defined(MS_WINDOWS)
    HANDLE hfile;
    _Py_BEGIN_SUPPRESS_IPH
    hfile = (HANDLE)_get_osfhandle(fd);
    _Py_END_SUPPRESS_IPH
    return (hfile != INVALID_HANDLE_VALUE &&
              GetFileType(hfile) != FILE_TYPE_UNKNOWN);
#endif

----------
components: Interpreter Core, Windows
keywords: easy (C)
messages: 407221
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Use WinAPI GetFileType() in is_valid_fd()
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45919>
_______________________________________

From report at bugs.python.org  Sun Nov 28 17:59:43 2021
From: report at bugs.python.org (Arthur Milchior)
Date: Sun, 28 Nov 2021 22:59:43 +0000
Subject: [New-bugs-announce] [issue45920] make doctest fails
Message-ID: <1638140383.16.0.243032979083.issue45920@roundup.psfhosted.org>


New submission from Arthur Milchior <arthur at milchior.fr>:

On current main,  f87ea0350286837e9e96de03f8bfa215176c2928 , 
```
cd cpython/Doc
make doctest
```
fails. 

Due to:

Document: library/functions
---------------------------

Warning, treated as error:
**********************************************************************
File "library/functions.rst", line ?, in default
Failed example:
    list(zip(range(3), ['fee', 'fi', 'fo', 'fum'], strict=True))
Expected:
    Traceback (most recent call last):
      ...
    ValueError: zip() argument 2 is longer than argument 1
Got:
    Traceback (most recent call last):
      File "/usr/lib/python3.8/doctest.py", line 1336, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest default[0]>", line 1, in <module>
        list(zip(range(3), ['fee', 'fi', 'fo', 'fum'], strict=True))
    TypeError: zip() takes no keyword arguments



This is not surprising since zip didn't take "Strict" kwarg in 3.8. The issue is that 3.10 doc is tested with Python 3.8.


If in Makefile I change Python to "Python3.10" I get a new error, but it still mention "/usr/lib/python3.8/doctest.py" so I guess for some reason it was not using 3.10 everywhere. 
I don't know make enough to have any idea how to correct this one.



By the way, is there a tool to auto-format .rst file? Initially, that was what I was trying to figure out, if I could get automatically warning when a new line is more than 80 chars long for example.

----------
assignee: docs at python
components: Documentation
messages: 407238
nosy: Arthur-Milchior, docs at python
priority: normal
severity: normal
status: open
title: make doctest fails
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45920>
_______________________________________

From report at bugs.python.org  Sun Nov 28 20:38:04 2021
From: report at bugs.python.org (Mark Sapiro)
Date: Mon, 29 Nov 2021 01:38:04 +0000
Subject: [New-bugs-announce] [issue45921] codecs module doesn't support
 iso-8859-6-i, iso-8859-6-e, iso-8859-8-i or iso-8859-8-i
Message-ID: <1638149884.18.0.351231109602.issue45921@roundup.psfhosted.org>


New submission from Mark Sapiro <mark at msapiro.net>:

iso-8859-6-i, iso-8859-6-e, iso-8859-8-i and iso-8859-8-i are all IANA recognized character sets per https://www.iana.org/assignments/character-sets/character-sets.xhtml. These are all unrecognized by codecs.lookup().

----------
components: Library (Lib)
messages: 407240
nosy: msapiro
priority: normal
severity: normal
status: open
title: codecs module doesn't support iso-8859-6-i, iso-8859-6-e, iso-8859-8-i or iso-8859-8-i
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45921>
_______________________________________

From report at bugs.python.org  Sun Nov 28 20:58:29 2021
From: report at bugs.python.org (Arthur Milchior)
Date: Mon, 29 Nov 2021 01:58:29 +0000
Subject: [New-bugs-announce] [issue45922] Many method, function,
 built-in... are not clickable and should be
Message-ID: <1638151109.2.0.445399231307.issue45922@roundup.psfhosted.org>


New submission from Arthur Milchior <arthur at milchior.fr>:

I would like to work on this documentation improvement. Before spending time on it, I would like to know whether it would be accepted in principle and whether the way I would intend to improve the documentation would be accepted.

# Problem:
In a lot of part of the documentation, the text refers to a method, function, built-in, variable, member defined somewhere else in the doc, but does not link to it (I'll call it "the defined thing" in the remaining of this BPO).

## Example

As an example, in https://docs.python.org/3.10/library/imp.html#imp.reload , you'll get link to import from, but no link on __name__. Admittedly, it's probably not a big deal by itself, and I would not have opened a bug report only for this one. But I'm convinced that there are sufficiently many missing links that a PR?that would solve add a lot of them would improve the day to day use of the documentation in a small way for a lot of users. 

This example seems interesting to me because it also shows that creating the links can not be 100% automated (even if most of it can). Indeed, __name__ is a property of modules, but it is also a property of class, function, method, descriptor and generator instances. The former is defined in https://docs.python.org/3.10/library/types.html?highlight=__name__#types.ModuleType.__name__ while the latter is defined in https://docs.python.org/3.10/library/stdtypes.html?highlight=__name__#definition.__name__ . Hence, a commit dealing with __name__ need to check that the correct reference is used each time.


I got the idea from a change requested by Terry J Reedy on another of my documentation PR?a few days ago: https://github.com/python/cpython/pull/29183#pullrequestreview-816414072 and I believe this should be more systematic.

# How I'll solve this issue

## Ensuring the PR is easy to review

For each defined thing, I'll create a commit that simply add links to this refered thing. In the example I gave, it will transform ``__name__`` to :meth:`__name__`. That means that each commit will be very easy to review (even if it will be boring). Even better, any diff tools can show that there are no other difference than the one I claim to make.

### One downside

The downside is that it requires that I do not change the place where line break occurs (otherwise, git default diff tool will show a lot of difference). this mean that some lines will grow larger than 80 characters. In this case, I believe that this is a cost worth paying because:
* there are already a lot of lines with more than 80 characters in the documentation, 
* reformating each paragraph will makes the reviewer work extremely hard, for virtually no benefits
* if required, a last commit can reformat all paragraphs at the end.

## How I'll find missing references

I will NOT find all missing references. I do not believe it is a realistic goal. My goal here is to improve the documentation, I don't care if it is not perfect here. What I do is that I search for lines containing matching the regexp `\`\.{2,}\`\`; they are generally, but not always, code that should be reference. For each missing reference, I'll use search and replace in the documentation codebase to replace the code by the link, I'll compile and check that the link works.

## Future work

I believe ideally a linter should warn if a single word betwen `` and `` could potentially be a reference. This seems harder work to be honest, I don't know how I'd start it. And it seems useless while so many missing references are in the codebase. 

#?Conclusion

My questions are thus:
* If I were to make such a PR, would it be accepted
* is there any feedback, improvement one would suggest that may improve the result or save time

----------
assignee: docs at python
components: Documentation
messages: 407242
nosy: Arthur-Milchior, docs at python
priority: normal
severity: normal
status: open
title: Many method, function, built-in... are not clickable and should be
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45922>
_______________________________________

From report at bugs.python.org  Mon Nov 29 06:37:57 2021
From: report at bugs.python.org (Mark Shannon)
Date: Mon, 29 Nov 2021 11:37:57 +0000
Subject: [New-bugs-announce] [issue45923] Improve performance of
 sys.settracing based tools.
Message-ID: <1638185877.94.0.453545318764.issue45923@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

In our quest for performance, the performance of sys.settracing based tools has probably gotten worse.

1. How do we measure this?
2. How do fix this?

We will initially use coverage.py as proxy for all sys.settracing based tools when measuring  performance.

The fix is probably to use quickening to insert a minimum set of instrumentation instructions required for tracing/profiling.
The existence of `f_trace_opcode` is a bit of a problem however, as we will have to instrument *every* instruction.


Ideally, sys.settracing based tools should be faster on 3.11 than 3.10, but at the least we should provide a simple alternative to sys.settracing that is faster.

----------
assignee: Mark.Shannon
components: Interpreter Core
messages: 407263
nosy: Mark.Shannon, nedbat, pablogsal
priority: normal
severity: normal
status: open
title: Improve performance of sys.settracing based tools.
type: performance
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45923>
_______________________________________

From report at bugs.python.org  Mon Nov 29 06:55:26 2021
From: report at bugs.python.org (Irit Katriel)
Date: Mon, 29 Nov 2021 11:55:26 +0000
Subject: [New-bugs-announce] [issue45924] Incorrect traceback when future's
 exception is raised multiple times
Message-ID: <1638186926.45.0.137493772787.issue45924@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at gmail.com>:

Background: 
https://github.com/python/cpython/pull/29780#issuecomment-981170548
https://github.com/python/cpython/pull/29780#issuecomment-981260365

######################################################

import asyncio, traceback

async def raise_after(fut, delay):
    await asyncio.sleep(delay)
    fut.set_exception(TypeError(42))

async def main():
    loop = asyncio.get_running_loop()
    fut = loop.create_future()
    loop.create_task(
        raise_after(fut, 1))

    print('hello ...')

    for i in range(3):
        try:
            print(await fut)
        except Exception as e:
            traceback.print_exception(e)

asyncio.run(main())
######################################################

Output (traceback accumulates a frame each time):


hello ...
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython-1/as.py", line 17, in main
    print(await fut)
          ^^^^^^^^^
TypeError: 42
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython-1/as.py", line 17, in main
    print(await fut)
          ^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/as.py", line 17, in main
    print(await fut)
          ^^^^^^^^^
TypeError: 42
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython-1/as.py", line 17, in main
    print(await fut)
          ^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/as.py", line 17, in main
    print(await fut)
          ^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/as.py", line 17, in main
    print(await fut)
          ^^^^^^^^^
TypeError: 42

----------
components: asyncio
messages: 407265
nosy: asvetlov, gvanrossum, iritkatriel, yselivanov
priority: normal
severity: normal
status: open
title: Incorrect traceback when future's exception is raised multiple times
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45924>
_______________________________________

From report at bugs.python.org  Mon Nov 29 06:57:19 2021
From: report at bugs.python.org (Erlend E. Aasland)
Date: Mon, 29 Nov 2021 11:57:19 +0000
Subject: [New-bugs-announce] [issue45925] Upgrade macOS and Windows
 installers to use SQLite 3.37.0
Message-ID: <1638187039.58.0.251846461245.issue45925@roundup.psfhosted.org>


New submission from Erlend E. Aasland <erlend.aasland at innova.no>:

SQLite 3.37.0 was released a couple of days ago: https://sqlite.org/releaselog/3_37_0.html

Given that 3.11 feature freeze is approx. May 2022, and that it took approx. 5 months between SQLite 3.36.0 and 3.37.0, I'd say we aim for a field tested SQLite 3.37.0 instead of a possibly fresh-out-of-the-box 3.38.0 in 3.11.

Let's wait a couple of weeks before changing the macOS/Windows installers.

----------
components: Windows, macOS
messages: 407266
nosy: erlendaasland, ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Upgrade macOS and Windows installers to use SQLite 3.37.0
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45925>
_______________________________________

From report at bugs.python.org  Mon Nov 29 08:15:13 2021
From: report at bugs.python.org (Alisson Oliveira)
Date: Mon, 29 Nov 2021 13:15:13 +0000
Subject: [New-bugs-announce] [issue45926] singledispatchmethod doesn't
 handle named arguments
Message-ID: <1638191713.81.0.622193348112.issue45926@roundup.psfhosted.org>


New submission from Alisson Oliveira <a2r7o6 at gmail.com>:

`functools.singledispatchmethod` doesn't handle named arguments. Ex:

```python
from functools import singledispatchmethod

class Test:
    def __init__(self):
        ...

    @singledispatchmethod
    def get(self, filters):
        return 'Not Implemented!'

    @get.register
    def _(self, filters: dict):
        for k, v in filters.items():
            print(k, ' => ' ,  v)

    @get.register
    def _(self, filters: list):
        for f in filters:
            print(f)

if __name__ == '__main__':
    t = Test()
    t.get({'a': 'A'})
    t.get(['a'])
    t.get(filters={'a': 'A'})
```

This will raise an error:
```
a  =>  A
a
Traceback (most recent call last):
  File "/Users/alisson.oliveira/Farfetch/Git/blueprints-package/main.py", line 33, in <module>
    t.get(filters={'a': 'A'})
  File "/usr/local/Cellar/python at 3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/functools.py", line 926, in _method
    method = self.dispatcher.dispatch(args[0].__class__)
IndexError: tuple index out of range
```

the problem is the lib always assume positional arguments in line 194: https://github.com/python/cpython/blob/main/Lib/functools.py#L914

----------
components: Library (Lib)
messages: 407273
nosy: alisson276
priority: normal
severity: normal
status: open
title: singledispatchmethod doesn't handle named arguments
type: behavior
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45926>
_______________________________________

From report at bugs.python.org  Mon Nov 29 09:15:45 2021
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Mon, 29 Nov 2021 14:15:45 +0000
Subject: [New-bugs-announce] [issue45927] timeit accepts -c/--clock and
 -t/--time without any functionality
Message-ID: <1638195345.59.0.368504787144.issue45927@roundup.psfhosted.org>


New submission from Marc-Andre Lemburg <mal at egenix.com>:

>From the code:

        opts, args = getopt.getopt(args, "n:u:s:r:tcpvh",
                                   ["number=", "setup=", "repeat=",
                                    "time", "clock", "process",
                                    "verbose", "unit=", "help"])

but the options -c and -t are not used in the subsequent code.

This can lead to situations where if you mistype e.g. -s as -c, you get completely wrong results (see https://bugs.python.org/issue45902 for an example).

----------
components: Library (Lib)
messages: 407276
nosy: lemburg
priority: normal
severity: normal
status: open
title: timeit accepts -c/--clock and -t/--time without any functionality
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45927>
_______________________________________

From report at bugs.python.org  Mon Nov 29 10:30:48 2021
From: report at bugs.python.org (penguin_wwy)
Date: Mon, 29 Nov 2021 15:30:48 +0000
Subject: [New-bugs-announce] [issue45928] Set up file stream for redirecting
 GC logs
Message-ID: <1638199848.02.0.0303380360366.issue45928@roundup.psfhosted.org>


New submission from penguin_wwy <940375606 at qq.com>:

Set up file stream for redirecting logs by adding a parameter in `set_debug`.

The stderr is used by default.

----------
components: Library (Lib)
messages: 407286
nosy: penguin_wwy
priority: normal
severity: normal
status: open
title: Set up file stream for redirecting GC logs
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45928>
_______________________________________

From report at bugs.python.org  Mon Nov 29 10:49:18 2021
From: report at bugs.python.org (Vito De Tullio)
Date: Mon, 29 Nov 2021 15:49:18 +0000
Subject: [New-bugs-announce] [issue45929] extend json.tool --json-lines to
 ignore empty rows
Message-ID: <1638200958.63.0.173460715414.issue45929@roundup.psfhosted.org>


New submission from Vito De Tullio <vito.detullio at gmail.com>:

It would be useful to let json.tool support empty rows during handling of json lines
generally speaking, this tolerance is already present in parsers like srsly and jsonlines

actual behavior:

# happy scenario
$ echo -e '{"foo":1}\n{"bar":2}' | python3.10 -mjson.tool --json-lines
{
    "foo": 1
}
{
    "bar": 2
}
$

# spurious EOL at EOF
$ echo -e '{"foo":1}\n{"bar":2}\n' | python3.10 -mjson.tool --json-lines
{
    "foo": 1
}
{
    "bar": 2
}
Expecting value: line 2 column 1 (char 1)
$

# two groups of "rows" in jsonl <- my current usecase
$ echo -e '{"foo":1}\n\n{"bar":2}' | python3.10 -mjson.tool --json-lines
{
    "foo": 1
}
Expecting value: line 2 column 1 (char 1)
$


my desired outcome is to preserve the EOLs, so to have something like:

# happy scenario
$ echo -e '{"foo":1}\n{"bar":2}' | python3.10 -mjson.tool --json-lines
{
    "foo": 1
}
{
    "bar": 2
}
$

# spurious EOL at EOF
$ echo -e '{"foo":1}\n{"bar":2}\n' | python3.10 -mjson.tool --json-lines
{
    "foo": 1
}
{
    "bar": 2
}

$

# two groups of "rows" in jsonl
$ echo -e '{"foo":1}\n\n{"bar":2}' | python3.10 -mjson.tool --json-lines
{
    "foo": 1
}

{
    "bar": 2
}
$

----------
components: Library (Lib)
messages: 407289
nosy: ZeD
priority: normal
severity: normal
status: open
title: extend json.tool --json-lines to ignore empty rows
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45929>
_______________________________________

From report at bugs.python.org  Mon Nov 29 11:29:31 2021
From: report at bugs.python.org (Valery Lovchikov)
Date: Mon, 29 Nov 2021 16:29:31 +0000
Subject: [New-bugs-announce] [issue45930] Lambda function bug
Message-ID: <1638203371.42.0.53122628449.issue45930@roundup.psfhosted.org>


New submission from Valery Lovchikov <lvch at spb.edu>:

Python 3.9.7 (default, Sep 10 2021, 14:59:43) 
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f=[lambda x: y**x for y in [2,3]]
>>> f[0](2)
9
>>> f[1](2)
9
=============
I expected 4 as a result of f[0](2), but got 9

----------
messages: 407293
nosy: lvch
priority: normal
severity: normal
status: open
title: Lambda function bug
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45930>
_______________________________________

From report at bugs.python.org  Mon Nov 29 14:43:33 2021
From: report at bugs.python.org (David Federman)
Date: Mon, 29 Nov 2021 19:43:33 +0000
Subject: [New-bugs-announce] [issue45931] Directory.Build.props/targets
 leaks from folders above the repo
Message-ID: <1638215013.82.0.388546987279.issue45931@roundup.psfhosted.org>


New submission from David Federman <david.federman at outlook.com>:

When building with Visual Studio 2017+, any Directory.Build.props/targets above the repo in the file structure (eg. in the parent repo when the python repo is a submodule) will be imported automatically.

----------
components: Build
messages: 407320
nosy: dfederm
priority: normal
severity: normal
status: open
title: Directory.Build.props/targets leaks from folders above the repo
type: compile error
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45931>
_______________________________________

From report at bugs.python.org  Mon Nov 29 17:10:40 2021
From: report at bugs.python.org (Alexander Mohr)
Date: Mon, 29 Nov 2021 22:10:40 +0000
Subject: [New-bugs-announce] [issue45932] EmailMessage incorrectly splits
 reply-to header
Message-ID: <1638223840.93.0.45573972976.issue45932@roundup.psfhosted.org>


New submission from Alexander Mohr <thehesiod at gmail.com>:

If you have a reply-to list that contains a name with a comma it must be quoted, however if the line length is just right python with split the name incorrectly and not keep the quote.  Note that the CC line keeps the quote in the name but the reply-to does not, presumably since the line is slightly longer.

example:
from email.message import EmailMessage


def main():
    message = EmailMessage()
    message['From'] = 'no-reply at farmersbusinessnetwork.com'
    message['Reply-To'] = """MEGAN FOOOBAAAAAR <abcfghijkladbfrg at akdja.com>,"KDJEHGI, SCOTT KJUYT" <abcfghijkladbfrg at akdja.com>"""
    message['To'] = """"KDJEHGI, SCOTT KJUYT" <SCOTT.KDJEHGI at MYFNBBANK.COM>"""
    message['Subject'] = "does not matter"
    message['CC'] = """MEGAN FOOOBAAAAAR <abcfghijkladbfrg at akdja.com>,"KDJEHGI, SCOTT KJUYT" <abcfghijkladbfrg at akdja.com>"""
    message.set_content('foo bar')
    print(message.as_string())


if __name__ == '__main__':
    main()

----------
components: email
messages: 407329
nosy: barry, r.david.murray, thehesiod
priority: normal
severity: normal
status: open
title: EmailMessage incorrectly splits reply-to header
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45932>
_______________________________________

From report at bugs.python.org  Mon Nov 29 17:25:10 2021
From: report at bugs.python.org (Tom E)
Date: Mon, 29 Nov 2021 22:25:10 +0000
Subject: [New-bugs-announce] [issue45933] Illegal Instrution (Core Dumped)
Message-ID: <1638224710.52.0.0477949346292.issue45933@roundup.psfhosted.org>


New submission from Tom E <guacaplushy at icloud.com>:

When compiling CPython 3.10 on Ubuntu 22.04, with GCC 11.2.0, it compiles successfully, but when trying to run it it just gives Illegal Instrution (Core Dumped). But when I build 3.9.9 its just fine... CPU is Intel Core i5-10400.

----------
messages: 407330
nosy: guacaplushy
priority: normal
severity: normal
status: open
title: Illegal Instrution (Core Dumped)
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45933>
_______________________________________

From report at bugs.python.org  Mon Nov 29 17:46:54 2021
From: report at bugs.python.org (NIKOLA DRAGANIC)
Date: Mon, 29 Nov 2021 22:46:54 +0000
Subject: [New-bugs-announce] [issue45934] python curses newterm
 implementation
Message-ID: <1638226015.0.0.416996876875.issue45934@roundup.psfhosted.org>


Change by NIKOLA DRAGANIC <draganic1 at llnl.gov>:


----------
components: C API, Windows
nosy: draganic1, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: python curses newterm implementation
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45934>
_______________________________________

From report at bugs.python.org  Mon Nov 29 18:38:11 2021
From: report at bugs.python.org (Irit Katriel)
Date: Mon, 29 Nov 2021 23:38:11 +0000
Subject: [New-bugs-announce] [issue45935] Add test for Issue11109:
 socketserver.ForkingMixIn leaves zombies,
 also fails to reap all zombies in one pass.
Message-ID: <1638229091.88.0.545801814049.issue45935@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at gmail.com>:

As mentioned in https://bugs.python.org/issue11109#msg136869, test_socketserver needs to be updated to serve as a regression test for Issue11109.

----------
components: Tests
messages: 407335
nosy: giampaolo.rodola, gregory.p.smith, iritkatriel, jwarkentin, orsenthil, python-dev, vstinner
priority: normal
severity: normal
status: open
title: Add test for Issue11109: socketserver.ForkingMixIn leaves zombies, also fails to reap all zombies in one pass.
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45935>
_______________________________________

From report at bugs.python.org  Mon Nov 29 21:43:03 2021
From: report at bugs.python.org (Richard Decal)
Date: Tue, 30 Nov 2021 02:43:03 +0000
Subject: [New-bugs-announce] [issue45936] collections.Counter drops key if
 value is 0 and updating using += operator
Message-ID: <1638240183.83.0.336191695201.issue45936@roundup.psfhosted.org>


New submission from Richard Decal <richard.decal at gmail.com>:

In brief:

```
from collections import Counter
x = Counter({'a': 0, 'b': 1})
x.update(x)  # works: Counter({'a': 0, 'b': 2})
x += x  # expected: Counter({'a': 0, 'b': 3}) actual: Counter({'b': 3})
```

I expect `+=` and `.update()` to be synonymous. However, the += operator is deleting keys if the source Counter has a zero count to begin with:

```
x = Counter({'a': 1})
x += Counter({'a': 0})  # ok: Counter({'a': 1})

y = Counter({'a': 0})
y += y  # expected: Counter({'a': 0}) actual: Counter()
```

----------
messages: 407348
nosy: crypdick
priority: normal
severity: normal
status: open
title: collections.Counter drops key if value is 0 and updating using += operator
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45936>
_______________________________________

From report at bugs.python.org  Mon Nov 29 22:36:50 2021
From: report at bugs.python.org (Hozoy)
Date: Tue, 30 Nov 2021 03:36:50 +0000
Subject: [New-bugs-announce] [issue45937] Pdb can't use the commands through
 -c or .pdbrc files
Message-ID: <1638243410.6.0.709504814494.issue45937@roundup.psfhosted.org>


New submission from Hozoy <Zrincet at gmail.com>:

I have a python file:

def do_something():
    pass

def main():
    a = 1
    b = 2
    c = a+b
    print(f"c:{c}")
    pass

if __name__ == '__main__':
    main()

I run this command:

python -m pdb -c "b 5" -c "commands 1;;do_something();;end" -c "c" main.py

It doesn't work.
And I have a .pdbrc file

b 5
commands 1
do_something()
end
c

It doesn't work either.

----------
components: Library (Lib)
messages: 407350
nosy: Zrincet
priority: normal
severity: normal
status: open
title: Pdb can't use the commands through -c or .pdbrc files
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45937>
_______________________________________

From report at bugs.python.org  Tue Nov 30 10:18:50 2021
From: report at bugs.python.org (Marc Villain)
Date: Tue, 30 Nov 2021 15:18:50 +0000
Subject: [New-bugs-announce] [issue45938] EmailMessage as_bytes
Message-ID: <1638285530.74.0.285035322318.issue45938@roundup.psfhosted.org>


New submission from Marc Villain <marc.villain at epita.fr>:

I am parsing an email with a subject header where the encoding of a unicode character happens to be cut in half. When a second encoded unicode character is encountered, we get the following error:

> 'utf-8' codec can't encode characters in position 1-2: surrogates not allowed

This error can be reproduced using the following:
>>> from email.message import EmailMessage
>>> msg = EmailMessage()
>>> msg.add_header('subject', '=?UTF-8?Q?a=C3?= =?UTF-8?Q?=B1o_a=C3=B1o?=')
>>> print(str(msg))         # This will succeed
>>> print(msg.as_bytes())   # This will fail
>>> print(msg.as_string())  # This will fail

After a bit of investigations, it appears the library is at some poing trying to concatenate 'a\udcc3\udcb1o ' and 'c?mo'. It then proceeds to try to call _ew.encode in email._header_value_parser._fold_as_ew on that. This obviously fails as '\udcc3\udcb1o' is not utf-8, whereas 'c?mo' is.

More tests:
[OK] '=?UTF-8?Q?a=C3?= =?UTF-8?Q?=B1o_a=C3=B1o?='
     > b' subject: =?utf-8?q?a=C3=B1o_c=C3=B3mo?=\n\n'
[OK] '=?UTF-8?Q?a=C3?= =?UTF-8?Q?=B1o_cmo?='
     > b' subject: =?unknown-8bit?q?a=C3=B1o?= cmo\n\n'
[OK] '=?UTF-8?Q?a=C3?= =?UTF-8?Q?=B1o?= =?UTF-8?Q?a=C3?= =?UTF-8?Q?=B1o?='
     > b' subject: =?unknown-8bit?q?a=C3=B1oa=C3=B1o?=\n\n'
[KO] '=?UTF-8?Q?a=C3?= =?UTF-8?Q?=B1o_a=C3=B1o?='
     > 'utf-8' codec can't encode characters in position 1-2: surrogates not allowed

Not sure what is the best way to fix that.

----------
components: Library (Lib)
messages: 407379
nosy: marc.villain
priority: normal
severity: normal
status: open
title: EmailMessage as_bytes
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45938>
_______________________________________

From report at bugs.python.org  Tue Nov 30 10:37:16 2021
From: report at bugs.python.org (Eric Blake)
Date: Tue, 30 Nov 2021 15:37:16 +0000
Subject: [New-bugs-announce] [issue45939] PyErr_SetObject documentation
 lacks mention of reference counting
Message-ID: <1638286636.68.0.690482450072.issue45939@roundup.psfhosted.org>


New submission from Eric Blake <eblake at redhat.com>:

While PyErr_SetString is obvious that you do not have to worry about the reference count of 'exception', the documentation for PyErr_SetObject is silent on whether it steals or adds a reference to 'value'.  This is particularly confusing, since other functions on the page (like PyErr_Restore) are explicit about stealing references.  Looking at non-canonical documentation, I found https://docstore.mik.ua/orelly/other/python/0596001886_pythonian-chp-24-sect-1.html which claims PyErr_SetObject steals a reference to 'value'.  But that is wrong; looking at the source code for _PyErr_SetString, it becomes obvious that once you use SetObject, the reference count will be increased, and the caller can release its own reference because nothing was stolen.

I noticed this lack of documentation when trying to debug a memory leak in nbdsh: https://listman.redhat.com/archives/libguestfs/2021-November/msg00280.html

----------
assignee: docs at python
components: Documentation
messages: 407381
nosy: docs at python, eblake
priority: normal
severity: normal
status: open
title: PyErr_SetObject documentation lacks mention of reference counting

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45939>
_______________________________________

From report at bugs.python.org  Tue Nov 30 12:07:07 2021
From: report at bugs.python.org (Ethan Smith)
Date: Tue, 30 Nov 2021 17:07:07 +0000
Subject: [New-bugs-announce] [issue45940] add_multiarch_paths breaks cross
 compilation to Emscripten
Message-ID: <1638292027.25.0.771827901631.issue45940@roundup.psfhosted.org>


New submission from Ethan Smith <ethan at ethanhs.me>:

When I cross compile on an Ubuntu system, the "PyBuildExt.add_multiarch_paths" method seems to add system includes, even when cross compiling for Emscripten.

Adding the system includes breaks Emscripten and causes several extensions to fail to build.

I have a patch which fixes this that I will be submitting shortly to Github.

----------
components: Build
messages: 407387
nosy: christian.heimes, ethan smith
priority: normal
severity: normal
status: open
title: add_multiarch_paths breaks cross compilation to Emscripten
type: compile error
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45940>
_______________________________________

From report at bugs.python.org  Tue Nov 30 17:45:55 2021
From: report at bugs.python.org (Irit Katriel)
Date: Tue, 30 Nov 2021 22:45:55 +0000
Subject: [New-bugs-announce] [issue45941] help("modules") segfaults on 3.11
Message-ID: <1638312355.41.0.606974934326.issue45941@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at gmail.com>:

Python 3.11.0a2+ (heads/main:8a45ca542a, Nov 30 2021, 22:40:56) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> help("modules")

Please wait a moment while I gather a list of all available modules...

/Users/iritkatriel/src/cpython/Lib/pkgutil.py:92: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  __import__(info.name)
/Users/iritkatriel/src/cpython/Lib/pkgutil.py:92: DeprecationWarning: lib2to3 package is deprecated and may not be able to parse Python 3.10+
  __import__(info.name)
test_sqlite3: testing with version '2.6.0', sqlite_version '3.32.3'
/Users/iritkatriel/src/cpython/Lib/distutils/command/build_ext.py:13: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
  from distutils.sysconfig import customize_compiler, get_python_version
zsh: segmentation fault  ./python.exe

----------
components: Interpreter Core
messages: 407403
nosy: iritkatriel
priority: normal
severity: normal
status: open
title: help("modules") segfaults on 3.11
type: crash
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45941>
_______________________________________

From report at bugs.python.org  Tue Nov 30 18:54:43 2021
From: report at bugs.python.org (Daisy Choi)
Date: Tue, 30 Nov 2021 23:54:43 +0000
Subject: [New-bugs-announce] [issue45942] shutil.copytree can raise OSErrors
 not wrapped in shutil.Error
Message-ID: <1638316483.94.0.0446181658365.issue45942@roundup.psfhosted.org>


New submission from Daisy Choi <me at floozutter.site>:

shutil.copytree's docstring [1] and documentation [2] states that, "If exception(s) occur, an Error is raised with a list of reasons," but it can raise OSErrors at the top-level call. For example:

>>> import shutil
>>> shutil.copytree(nonexistent_dir, dst)
FileNotFoundError: [WinError 3] The system cannot find the path specified: '...'

>>> shutil.copytree(actually_a_file, dst)
NotADirectoryError: [WinError 267] The directory name is invalid: '...'

>>> shutil.copytree(src, dst_already_exists, dirs_exist_ok=False)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '...'

The errors above happen because shutil.copytree and shutil._copytree call os.scandir and os.makedir respectively without wrapping any exceptions inside shutil.Error. [3][4]


It seems like shutil.copytree(src, dst) only raises a shutil.Error if an OSError or shutil.Error was raised when copying over a file in src or during a recursive call to shutil.copytree for a subdirectory of src:

>>> shutil.copytree(dir_with_broken_symblink, dst)
shutil.Error: [('a\\symlink', 'b\\symlink', "[Errno 2] No such file or directory: 'a\\\\symlink'")]


Is this behavior intended? Should shutil.copytree be changed so that exceptions for the top level are wrapped inside shutil.Error, or should the documentation for shutil.copytree mention that it can raise exceptions that aren't shutil.Error? (Or is this just a non-issue?)


[1]: https://github.com/python/cpython/blob/b494f5935c92951e75597bfe1c8b1f3112fec270/Lib/shutil.py#L522
[2]: https://docs.python.org/3.10/library/shutil.html?highlight=shutil#shutil.copytree
[3]: https://github.com/python/cpython/blob/b494f5935c92951e75597bfe1c8b1f3112fec270/Lib/shutil.py#L554
[4]: https://github.com/python/cpython/blob/b494f5935c92951e75597bfe1c8b1f3112fec270/Lib/shutil.py#L457

----------
components: Library (Lib)
messages: 407413
nosy: Floozutter
priority: normal
severity: normal
status: open
title: shutil.copytree can raise OSErrors not wrapped in shutil.Error
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45942>
_______________________________________

From report at bugs.python.org  Tue Nov 30 19:39:20 2021
From: report at bugs.python.org (Qualyn Richard)
Date: Wed, 01 Dec 2021 00:39:20 +0000
Subject: [New-bugs-announce] [issue45943] Kids10yrsapart@gmail.com
Message-ID: <1638319160.38.0.564015282647.issue45943@roundup.psfhosted.org>


Change by Qualyn Richard <oktaine57 at gmail.com>:


----------
components: email
files: PSX_20210903_080553.jpg
nosy: barry, oktaine57, r.david.murray
priority: normal
severity: normal
status: open
title: Kids10yrsapart at gmail.com
type: behavior
versions: Python 3.11
Added file: https://bugs.python.org/file50463/PSX_20210903_080553.jpg

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45943>
_______________________________________