From report at bugs.python.org  Fri Jan  1 00:16:27 2021
From: report at bugs.python.org (Steven D'Aprano)
Date: Fri, 01 Jan 2021 05:16:27 +0000
Subject: [New-bugs-announce] [issue42801] Exception catching function
 crashes on recursive list
Message-ID: <1609478187.83.0.886724654093.issue42801@roundup.psfhosted.org>


New submission from Steven D'Aprano <steve+python at pearwood.info>:

This function crashes on the following recursive list:


def length(x):
    try:
        return sum(length(i) for i in x)
    except Exception:
        return 1


a = [[1, 2, 3], [4, 5, 6]]
a.append(a)
length(a)


Crashes:


Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow.
Python runtime state: initialized

Current thread 0x00007eff18d77740 (most recent call first):
  File "<stdin>", line 3 in length
  File "<stdin>", line 3 in <genexpr>
  File "<stdin>", line 3 in length
  File "<stdin>", line 3 in <genexpr>
  ...
Aborted (core dumped)




For brevity I've cut some of the output. There are about fifty pairs of "line 3 in length"/line 3 in <genexpr>" lines, all identical.

----------
components: Interpreter Core
messages: 384150
nosy: steven.daprano
priority: normal
severity: normal
status: open
title: Exception catching function crashes on recursive list
type: crash
versions: Python 3.7, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42801>
_______________________________________

From report at bugs.python.org  Fri Jan  1 09:28:27 2021
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 01 Jan 2021 14:28:27 +0000
Subject: [New-bugs-announce] [issue42802] distutils: Remove bdist_wininst
 command
Message-ID: <1609511307.58.0.913169413627.issue42802@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

Attached PR removes it.

The bdist_wininst command was deprecated in Python 3.8:

commit 1da4462765b084dfa8d869b6cb5855e8f6014a11
Author: Victor Stinner <vstinner at redhat.com>
Date:   Fri Jul 5 10:44:12 2019 +0200

    bpo-37481: Deprecate distutils bdist_wininst command (GH-14553)
    
    The distutils bdist_wininst command is now deprecated, use
    bdist_wheel (wheel packages) instead.


"What?s New In Python 3.9: Build Changes" also announced:

* On non-Windows platforms, creating ``bdist_wininst`` installers is now
  officially unsupported.  (See :issue:`10945` for more details.)

https://docs.python.org/dev/whatsnew/3.9.html#build-changes


See bpo-37481 and:
https://discuss.python.org/t/deprecate-bdist-wininst/1929

----------
components: Distutils
messages: 384173
nosy: dstufft, eric.araujo, vstinner
priority: normal
severity: normal
status: open
title: distutils: Remove bdist_wininst command
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42802>
_______________________________________

From report at bugs.python.org  Fri Jan  1 09:43:03 2021
From: report at bugs.python.org (Ned Batchelder)
Date: Fri, 01 Jan 2021 14:43:03 +0000
Subject: [New-bugs-announce] [issue42803] Traced line number is wrong for
 "if not __debug__"
Message-ID: <1609512183.17.0.454209386156.issue42803@roundup.psfhosted.org>


New submission from Ned Batchelder <ned at nedbatchelder.com>:

(Using CPython commit 6b1ac809b9)

This program never executes line 4, but the "if not __debug__" is partly attributed to line 4, giving an incorrect trace:

    for value in [True, False]:
        if value:
            if not __debug__:
                1/0 # line 4
        else:
            x = 6

Using a simple trace program (https://github.com/nedbat/coveragepy/blob/master/lab/run_trace.py), it produces this output:

    call <string> 1 @-1
        line <string> 1 @0
        line <string> 2 @8
        line <string> 3 @12
        line <string> 4 @14
        line <string> 1 @4
        line <string> 2 @8
        line <string> 6 @16
        line <string> 1 @4
        return <string> 1 @24

Various simplifications of the program make the problem go away.

----------
components: Interpreter Core
messages: 384178
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Traced line number is wrong for "if not __debug__"
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42803>
_______________________________________

From report at bugs.python.org  Fri Jan  1 13:00:07 2021
From: report at bugs.python.org (Bharath Posa)
Date: Fri, 01 Jan 2021 18:00:07 +0000
Subject: [New-bugs-announce] [issue42804] Unable to compile the code x86
 windows
Message-ID: <1609524007.28.0.23660757957.issue42804@roundup.psfhosted.org>


New submission from Bharath Posa <nmania03 at gmail.com>:

Hi Team,

While I am trying to build the  cpython in windows 10. I am getting tcl.h file not found error.

You can find the screen shot of the error along with the bug.

regards
Sunny

----------
components: Build
files: error_python_compilation.JPG
messages: 384194
nosy: pbharat
priority: normal
severity: normal
status: open
title: Unable to compile the code x86 windows
versions: Python 3.8
Added file: https://bugs.python.org/file49715/error_python_compilation.JPG

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42804>
_______________________________________

From report at bugs.python.org  Fri Jan  1 14:39:57 2021
From: report at bugs.python.org (monchy)
Date: Fri, 01 Jan 2021 19:39:57 +0000
Subject: [New-bugs-announce] [issue42805] broken pygame integration into
 wxpython
Message-ID: <1609529997.71.0.350416816551.issue42805@roundup.psfhosted.org>


New submission from monchy <monchytales at gmail.com>:

in previous python verions a pygame window could be embedded into wxpython by assigning the SDL_WINDOWID to the current wx.Panel.GetHandle() and the SDL_VIDEODRIVER to 'windib' before importing python
the same code creates a separate pygame window in the newest python version
a basic implementation can be found here:
https://github.com/Monchytales/Puppet-Theatre/blob/main/pygame_panel.py

----------
messages: 384203
nosy: MonchyTales
priority: normal
severity: normal
status: open
title: broken pygame integration into wxpython
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42805>
_______________________________________

From report at bugs.python.org  Fri Jan  1 19:00:22 2021
From: report at bugs.python.org (Anthony Sottile)
Date: Sat, 02 Jan 2021 00:00:22 +0000
Subject: [New-bugs-announce] [issue42806] Incorrect offsets in new parser
 for f-string substitutions
Message-ID: <1609545622.39.0.162230247041.issue42806@roundup.psfhosted.org>


New submission from Anthony Sottile <asottile at umich.edu>:

here's a sample file.  this is distilled down from an example which broke a code formatter here: https://github.com/asottile/add-trailing-comma/issues/106

def f():
    x = (
            'PERL_MM_OPT', (
                f'INSTALL-BASE={shlex.quote(venv)} '
                f'wat'
            ),
    )


A diff from astpretty (https://github.com/asottile/astpretty) between the old and new parser:

```console
$ diff -u <(.tox/py39/bin/python -X oldparser -m astpretty perl.py) <(.tox/py39/bin/python -m astpretty perl.py)
--- /dev/fd/63	2021-01-01 15:58:09.108060968 -0800
+++ /dev/fd/62	2021-01-01 15:58:09.108060968 -0800
@@ -35,19 +35,19 @@
                                         end_col_offset=22,
                                         value=Call(
                                             lineno=4,
-                                            col_offset=32,
+                                            col_offset=16,
                                             end_lineno=4,
-                                            end_col_offset=49,
+                                            end_col_offset=33,
                                             func=Attribute(
                                                 lineno=4,
-                                                col_offset=32,
+                                                col_offset=16,
                                                 end_lineno=4,
-                                                end_col_offset=43,
-                                                value=Name(lineno=4, col_offset=32, end_lineno=4, end_col_offset=37, id='shlex', ctx=Load()),
+                                                end_col_offset=27,
+                                                value=Name(lineno=4, col_offset=16, end_lineno=4, end_col_offset=21, id='shlex', ctx=Load()),
                                                 attr='quote',
                                                 ctx=Load(),
                                             ),
-                                            args=[Name(lineno=4, col_offset=44, end_lineno=4, end_col_offset=48, id='venv', ctx=Load())],
+                                            args=[Name(lineno=4, col_offset=28, end_lineno=4, end_col_offset=32, id='venv', ctx=Load())],
                                             keywords=[],
                                         ),
                                         conversion=-1,
```

the old parser is correct here, and the new parser is wrong

notably it thinks that the call inside the f-string substitution is positioned at column 16 which is the position of the f-string start

this bug is also present in 3.10:

```console
$ venv/bin/python3.10 --version
Python 3.10.0a3
$ diff -u <(.tox/py39/bin/python -X oldparser -m astpretty perl.py) <(venv/bin/python3.10 -m astpretty perl.py)
--- /dev/fd/63	2021-01-01 15:59:54.000059968 -0800
+++ /dev/fd/62	2021-01-01 15:59:54.000059968 -0800
@@ -35,19 +35,19 @@
                                         end_col_offset=22,
                                         value=Call(
                                             lineno=4,
-                                            col_offset=32,
+                                            col_offset=16,
                                             end_lineno=4,
-                                            end_col_offset=49,
+                                            end_col_offset=33,
                                             func=Attribute(
                                                 lineno=4,
-                                                col_offset=32,
+                                                col_offset=16,
                                                 end_lineno=4,
-                                                end_col_offset=43,
-                                                value=Name(lineno=4, col_offset=32, end_lineno=4, end_col_offset=37, id='shlex', ctx=Load()),
+                                                end_col_offset=27,
+                                                value=Name(lineno=4, col_offset=16, end_lineno=4, end_col_offset=21, id='shlex', ctx=Load()),
                                                 attr='quote',
                                                 ctx=Load(),
                                             ),
-                                            args=[Name(lineno=4, col_offset=44, end_lineno=4, end_col_offset=48, id='venv', ctx=Load())],
+                                            args=[Name(lineno=4, col_offset=28, end_lineno=4, end_col_offset=32, id='venv', ctx=Load())],
                                             keywords=[],
                                         ),
                                         conversion=-1,
```

----------
components: Interpreter Core
messages: 384208
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: Incorrect offsets in new parser for f-string substitutions
type: behavior
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42806>
_______________________________________

From report at bugs.python.org  Fri Jan  1 23:23:16 2021
From: report at bugs.python.org (oon arfiandwi)
Date: Sat, 02 Jan 2021 04:23:16 +0000
Subject: [New-bugs-announce] [issue42807] smtplib send_message should gives
 more clear exception if the msg is None
Message-ID: <1609561396.29.0.629926386929.issue42807@roundup.psfhosted.org>


New submission from oon arfiandwi <oon.arfiandwi at gmail.com>:

I want to enhance the error message on the send_message method of class SMTP, in module smtplib.

One of the parameters on send_message is msg which is email.message.Message object, this is a required argument. If the msg is None, currently it will raise "AttributeError: 'NoneType' object has no attribute 'get_all'".
This message is not straight forward and a bit hard to debug.

As the easy fix with additional check if msg is None in send_message method.

I want to add PR on this, also with the additional test(s). This is my first issue, appreciate any feedback. thank you.

----------
components: Library (Lib)
messages: 384216
nosy: oon
priority: normal
severity: normal
status: open
title: smtplib send_message should gives more clear exception if the msg is None
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42807>
_______________________________________

From report at bugs.python.org  Sat Jan  2 05:00:06 2021
From: report at bugs.python.org (Dennis Sweeney)
Date: Sat, 02 Jan 2021 10:00:06 +0000
Subject: [New-bugs-announce] [issue42808] Add PyType_Type.tp_vectorcall for
 type(obj) performance
Message-ID: <1609581606.85.0.894764244692.issue42808@roundup.psfhosted.org>


New submission from Dennis Sweeney <sweeney.dennis650 at gmail.com>:

# common case
.\python.bat -m pyperf timeit "type(17)"
Master: 49.9 ns +- 1.8 ns ---> PR: 33.3 ns +- 1.4 ns

# uncommon case
.\python.bat -m pyperf timeit "type('A', (object, ), {})"
Master: 5.14 us +- 0.04 us ---> PR: 5.43 us +- 0.06 us

----------
components: Interpreter Core
messages: 384218
nosy: Dennis Sweeney
priority: normal
severity: normal
status: open
title: Add PyType_Type.tp_vectorcall for type(obj) performance
type: performance
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42808>
_______________________________________

From report at bugs.python.org  Sat Jan  2 05:47:08 2021
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 02 Jan 2021 10:47:08 +0000
Subject: [New-bugs-announce] [issue42809] Improve pickle tests for recursive
 data
Message-ID: <1609584428.87.0.716102907179.issue42809@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The proposed PR extends tests for pickling recursive structures. The code is now more generic and covers more corner cases (found during work on issue36694).

----------
components: Tests
messages: 384220
nosy: alexandre.vassalotti, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Improve pickle tests for recursive data
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42809>
_______________________________________

From report at bugs.python.org  Sat Jan  2 10:09:25 2021
From: report at bugs.python.org (Ned Batchelder)
Date: Sat, 02 Jan 2021 15:09:25 +0000
Subject: [New-bugs-announce] [issue42810] Nested if/else gets phantom else
 trace (3.10)
Message-ID: <1609600165.12.0.126191975832.issue42810@roundup.psfhosted.org>


New submission from Ned Batchelder <ned at nedbatchelder.com>:

(Using CPython commit 6b1ac809b9)

This program never executes line 6, but tracing it claims that it does:

    a = b = x = y = z = 1
    if a == 1:
        if b == 1:
            x = 4
        else:
            y = 6
    else:
        z = 8
    assert (a, b, x, y, z) == (1, 1, 4, 1, 1)

Using a simple trace program (https://github.com/nedbat/coveragepy/blob/master/lab/run_trace.py), it produces this output:

    call <string> 1 @-1
        line <string> 1 @0
        line <string> 2 @20
        line <string> 3 @28
        line <string> 4 @36
        line <string> 6 @40
        line <string> 9 @52
        return <string> 9 @76

----------
components: Interpreter Core
messages: 384222
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Nested if/else gets phantom else trace (3.10)
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42810>
_______________________________________

From report at bugs.python.org  Sat Jan  2 15:27:59 2021
From: report at bugs.python.org (Brett Cannon)
Date: Sat, 02 Jan 2021 20:27:59 +0000
Subject: [New-bugs-announce] [issue42811] Update docs for
 importlib.util.resolve_name() to use __spec__ instead of __package__
Message-ID: <1609619279.01.0.578638244778.issue42811@roundup.psfhosted.org>


New submission from Brett Cannon <brett at python.org>:

Since importlib now prefers `__spec__` over `__package__`, https://docs.python.org/3/library/importlib.html#importlib.util.resolve_name should be updated to use `__spec__.parent` instead in the. example.

----------
assignee: brett.cannon
components: Documentation
keywords: easy
messages: 384238
nosy: brett.cannon
priority: low
severity: normal
stage: needs patch
status: open
title: Update docs for importlib.util.resolve_name() to use __spec__ instead of __package__
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42811>
_______________________________________

From report at bugs.python.org  Sun Jan  3 00:51:32 2021
From: report at bugs.python.org (Chaim Gewirtz)
Date: Sun, 03 Jan 2021 05:51:32 +0000
Subject: [New-bugs-announce] [issue42812] @overload-ing method of parent
 class without actual implementation
Message-ID: <1609653092.18.0.893619520042.issue42812@roundup.psfhosted.org>


New submission from Chaim Gewirtz <chaim422 at gmail.com>:

Why should @overload need to be followed by an implementation when an implementation already exists in the parent class?

Illustrative example:

class Parent:
    def foo(**kwargs):
        """Argument names of foo vary depending on the child class."""

class Child(Parent):
    @overload foo(a, b): ...
    
Raises:

"NotImplementedError: You should not call an overloaded function. A series of @overload-decorated functions outside a stub module should always be followed by an implementation that is not @overload-ed."

----------
messages: 384255
nosy: chaim422
priority: normal
severity: normal
status: open
title: @overload-ing method of parent class without actual implementation
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42812>
_______________________________________

From report at bugs.python.org  Sun Jan  3 03:33:08 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Sun, 03 Jan 2021 08:33:08 +0000
Subject: [New-bugs-announce] [issue42813] Extra spaces cause unexpected EOF
 error in "compile" function with mode "single"
Message-ID: <1609662788.81.0.389452774776.issue42813@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

Running the following program will lead to an unexpected EOF error. If we delete the space before  triple-quotation mark("""), the error will gone. If we replace the content of snippet with "a = 1", the program will work well. However, if we replace the content of snippet with "if a:pass" in a single line, the program cause an error again. This is weird. 

Spaces in the end of a statement should not affect the compilation of program in "single" mode. Besides, the error messages are different in Python2 and Python3. Indentation error are reported in Python 2 for snippet" if a:pass" while SyntaxError are reported in Python 3. 
  
======================================================
import math
def check_stack_size( code):
    # To assert that the alleged stack size is not O(N), we
    # check that it is smaller than log(N).
    if isinstance(code, str):
        code = compile(code, "<foo>", "single")
    max_size = math.ceil(math.log(len(code.co_code)))
    # self.assertLessEqual(code.co_stacksize, max_size)

def test_if_else():
    snippet ="""
if x:
    a
elif y:
    b
else:
    c     
    """
    check_stack_size(snippet)

test_if_else()
=====================================================
Behavior in Python 3.10:
Traceback (most recent call last):
  File "/home/xxm/Desktop/nameChanging/report/temp.py", line 30, in <module>
    test_if_else()
  File "/home/xxm/Desktop/nameChanging/report/temp.py", line 20, in test_if_else
    check_stack_size(snippet)
  File "/home/xxm/Desktop/nameChanging/report/temp.py", line 6, in check_stack_size
    code = compile(code, "<foo>", "single")
  File "<foo>", line 8
    
SyntaxError: unexpected EOF while parsing

Behaviors in Python2: 
Traceback (most recent call last):
  File "/home/xxm/Desktop/nameChanging/report/temp.py", line 30, in <module>
    test_if_else()
  File "/home/xxm/Desktop/nameChanging/report/temp.py", line 27, in test_if_else
    check_stack_size(snippet)
  File "/home/xxm/Desktop/nameChanging/report/temp.py", line 6, in check_stack_size
    code = compile(code, "<foo>", "single")
  File "<foo>", line 3
    
    ^
IndentationError: unexpected indent

----------
components: Interpreter Core
messages: 384257
nosy: xxm
priority: normal
severity: normal
status: open
title: Extra spaces cause unexpected EOF error in "compile" function with mode "single"
type: compile error
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42813>
_______________________________________

From report at bugs.python.org  Sun Jan  3 06:40:24 2021
From: report at bugs.python.org (Zackery Spytz)
Date: Sun, 03 Jan 2021 11:40:24 +0000
Subject: [New-bugs-announce] [issue42814] Undefined behavior in
 Objects/genericaliasobject.c
Message-ID: <1609674024.61.0.74243567004.issue42814@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

In is_typing_name(), va_end() is not always called before the function returns.  It is undefined behavior to call va_start() without also calling va_end().

----------
components: Interpreter Core
messages: 384261
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: Undefined behavior in Objects/genericaliasobject.c
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42814>
_______________________________________

From report at bugs.python.org  Sun Jan  3 06:43:34 2021
From: report at bugs.python.org (Yurii Karabas)
Date: Sun, 03 Jan 2021 11:43:34 +0000
Subject: [New-bugs-announce] [issue42815] new thread doesn't copy context of
 the parent thread
Message-ID: <1609674214.71.0.941381845328.issue42815@roundup.psfhosted.org>


New submission from Yurii Karabas <1998uriyyo at gmail.com>:

New thread doesn't copy context of the parent thread.

The minimal example to reproduce the issue:
```
from threading import Thread
from contextvars import ContextVar

foo: ContextVar[str] = ContextVar("foo")


def temp():
    print(foo.get())


foo.set("bar")

t = Thread(target=temp)
t.start()
t.join()
```

Is it expected behavior?

PEP 567 I didn't find anything regarding this case.

----------
components: Library (Lib)
messages: 384262
nosy: uriyyo
priority: normal
severity: normal
status: open
title: new thread doesn't copy context of the parent thread
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42815>
_______________________________________

From report at bugs.python.org  Sun Jan  3 08:09:42 2021
From: report at bugs.python.org (Martin Winks)
Date: Sun, 03 Jan 2021 13:09:42 +0000
Subject: [New-bugs-announce] [issue42816] Add str.split_iter function
Message-ID: <1609679382.47.0.0845431201313.issue42816@roundup.psfhosted.org>


New submission from Martin Winks <cat at snejugal.ru>:

Split string by given separator and return iterator as a result.

The naive and not very efficient solution would be using current str.split:

def split_iter(self: str, sep: str) -> 'Iterator[str]':
    return iter(self.split(sep))

Probably, need we'll some better solution without allocating new list.

----------
components: Unicode
messages: 384270
nosy: ezio.melotti, uwinx, vstinner
priority: normal
severity: normal
status: open
title: Add str.split_iter function
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42816>
_______________________________________

From report at bugs.python.org  Sun Jan  3 08:19:56 2021
From: report at bugs.python.org (Kevin Rasmussen)
Date: Sun, 03 Jan 2021 13:19:56 +0000
Subject: [New-bugs-announce] [issue42817] timedelta zeropadding hh
Message-ID: <1609679996.96.0.581038745817.issue42817@roundup.psfhosted.org>


New submission from Kevin Rasmussen <kj_529 at hotmail.com>:

It looks like hh should be zeropadded to 2 and isn't for timedelta.

----------
messages: 384273
nosy: krasmussen
priority: normal
severity: normal
status: open
title: timedelta zeropadding hh
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42817>
_______________________________________

From report at bugs.python.org  Sun Jan  3 09:09:35 2021
From: report at bugs.python.org (Klaus Ethgen)
Date: Sun, 03 Jan 2021 14:09:35 +0000
Subject: [New-bugs-announce] [issue42818] AttributeError: 'zipimporter'
 object has no attribute 'find_spec'
Message-ID: <1609682975.59.0.559934544299.issue42818@roundup.psfhosted.org>


New submission from Klaus Ethgen <Klaus at Ethgen.de>:

I encountered the bug when installing a package with `python3 setup.py install --user`. After that, a second installation failed with this stack trace:
```
Traceback (most recent call last):                                                                                                                            
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2191, in _handle_ns                                                                   
    loader = importer.find_spec(packageName).loader                                                                                                           
AttributeError: 'zipimporter' object has no attribute 'find_spec'                                                                                             

During handling of the above exception, another exception occurred:                                                                                           

Traceback (most recent call last):                                                                                                                            
  File "/home/klaus/devel/upstream/pass-git-helper/setup.py", line 1, in <module>                                                                             
    from setuptools import setup                                                                                                                              
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 16, in <module>                                                                          
    import setuptools.version                                                                                                                                 
  File "/usr/lib/python3/dist-packages/setuptools/version.py", line 1, in <module>                                                                            
    import pkg_resources                                                                                                                                      
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3239, in <module>                                                                     
    def _initialize_master_working_set():                                                                                                                     
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3222, in _call_aside                                                                  
    f(*args, **kwargs)                                                                                                                                        
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set                                               
    tuple(                                                                                                                                              
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3265, in <genexpr>                                                                    
    dist.activate(replace=False)                                                                                                                              
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2780, in activate                                                                     
    declare_namespace(pkg)                                                                                                                                    
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2279, in declare_namespace                                                            
    _handle_ns(packageName, path_item)                                                                                                                        
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2196, in _handle_ns                                                                   
    loader = importer.find_spec(packageName)                                                                                                                  
AttributeError: 'zipimporter' object has no attribute 'find_spec'
```

This error happens now also with different tools.

Solution is to `rm -fr ~/.local/lib`. But the next installation triggers the bug again.

The only solution is to install via `python3 setup.py install --user --old-and-unmanageable`.

When I look into the code of zipimport(er), I do not see any find_spec only the old find_loader.

Despite it is a core library, this must be fixed! And with that the name should match the name of the py file to make it able to find the relevant file.

----------
components: Library (Lib)
messages: 384279
nosy: mowgli
priority: normal
severity: normal
status: open
title: AttributeError: 'zipimporter' object has no attribute 'find_spec'
type: crash
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42818>
_______________________________________

From report at bugs.python.org  Sun Jan  3 15:52:35 2021
From: report at bugs.python.org (Dustin Rodrigues)
Date: Sun, 03 Jan 2021 20:52:35 +0000
Subject: [New-bugs-announce] [issue42819] readline 8.1 bracketed paste
Message-ID: <1609707155.89.0.22999518491.issue42819@roundup.psfhosted.org>


New submission from Dustin Rodrigues <dust.rod at gmail.com>:

Readline 8.1 enables bracketed paste by default. Package managers like Homebrew for macOS and distributions like Arch Linux which use the latest version of readline (released December 2020) now have new behavior when pasting multiline strings into the python REPL. Disabling bracketed paste on 8.1 reverts to the expected behavior, and enabling bracketed paste on 8.0 also reproduces the behavior. Further information in https://github.com/Homebrew/homebrew-core/issues/68193

Example with bracketed paste on:
$ cat in
1+2
3+4
5+6

$ pbcopy < in
$ /usr/local/Cellar/python\@3.9/3.9.1_3/bin/python3
Python 3.9.1 (default, Dec 28 2020, 11:22:14) 
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+2
3+4
5+6

  File "<stdin>", line 1
    1+2
3+4
5+6


       ^
SyntaxError: multiple statements found while compiling a single statement
>>> 



Example with it off:
$ /usr/local/bin/python3
Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec  7 2020, 12:10:52) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+2
3
>>> 3+4
7
>>> 5+6
11
>>>

----------
messages: 384297
nosy: dtrodrigues
priority: normal
severity: normal
status: open
title: readline 8.1 bracketed paste
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42819>
_______________________________________

From report at bugs.python.org  Sun Jan  3 20:28:14 2021
From: report at bugs.python.org (Tim Hoffmann)
Date: Mon, 04 Jan 2021 01:28:14 +0000
Subject: [New-bugs-announce] [issue42820] Sphinx conf.py needs_version entry
 is outdated
Message-ID: <1609723694.82.0.961390909546.issue42820@roundup.psfhosted.org>


New submission from Tim Hoffmann <tim.hoffmann at posteo.de>:

The sphinx conf.py entry `needs_version=1.8` (https://github.com/python/cpython/blob/1470edd6131c29b8a09ce012cdfee3afa269d553/Doc/conf.py#L48)

is not in sync with the doc build requirements (https://github.com/python/cpython/blob/1470edd6131c29b8a09ce012cdfee3afa269d553/Doc/requirements.txt#L6)

Possible solutions:
a) Drop the `needs_version` config value. It's optional.
b) Make sure to update both places.
c) Or at minimum add a comment that `needs_version` is only a safeguard against very old sphinx versions but is not necessarily sufficient to build.

----------
assignee: docs at python
components: Documentation
messages: 384305
nosy: docs at python, timhoffm
priority: normal
severity: normal
status: open
title: Sphinx conf.py needs_version entry is outdated

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42820>
_______________________________________

From report at bugs.python.org  Mon Jan  4 03:00:54 2021
From: report at bugs.python.org (karl)
Date: Mon, 04 Jan 2021 08:00:54 +0000
Subject: [New-bugs-announce] [issue42821] HTMLParser: subsequent duplicate
 attributes should be ignored
Message-ID: <1609747254.8.0.505282394857.issue42821@roundup.psfhosted.org>


New submission from karl <karl+pythonbugs at la-grange.net>:

This comes up while working on issue 41748


browser input 
data:text/html,<!doctype html><div class="bar" class="foo">text</div>

browser output
<div class="bar">text</div>

Actual HTMLParser output

see https://github.com/python/cpython/pull/24072#discussion_r551158342
('starttag', 'div', [('class', 'bar'), ('class', 'foo')])]

Expected HTMLParser output
('starttag', 'div', [('class', 'bar')])]

----------
components: Library (Lib)
messages: 384308
nosy: karlcow
priority: normal
severity: normal
status: open
title: HTMLParser: subsequent duplicate attributes should be ignored
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42821>
_______________________________________

From report at bugs.python.org  Mon Jan  4 08:03:42 2021
From: report at bugs.python.org (Julien Palard)
Date: Mon, 04 Jan 2021 13:03:42 +0000
Subject: [New-bugs-announce] [issue42822] Missing backslashes in
 documentation for 3.8 and 3.9
Message-ID: <1609765422.49.0.483236804978.issue42822@roundup.psfhosted.org>


New submission from Julien Palard <julien+python at palard.fr>:

This is just to track the issue, initially reported here:

https://github.com/python/python-docs-theme/issues/63

Then discussed here:

https://github.com/python/cpython/pull/23827#issuecomment-753598860

Then reverted:

- in 3.8 https://github.com/python/cpython/pull/24093
- in 3.9 https://github.com/python/cpython/pull/24092

I edited the revert to keep only relevant parts, as only a few bits of the commit were causing troubles.

Issue is: Sphinx 2 and Sphinx 3 have incompatible handling of backslahes, fixing them for Sphinx 3 in Python 3.10, then backporting to 3.8 and 3.9 which use Sphinx 2 caused an issue: `\n` for example are rendered as `n`, easy to notice in the documentaion of print:

    print(*objects, sep=' ', end='n', file=sys.stdout, flush=False)?

As migration to Sphinx 3 was hard (done for Python 3.10 documentation), it won't be backported to 3.8 and 3.9, which will still be built using Sphinx 2, so some bits had to be reverted for the doc to display the backslash of `\n`.

----------
assignee: mdk
components: Documentation
messages: 384324
nosy: adelfino, mdk
priority: normal
severity: normal
stage: commit review
status: open
title: Missing backslashes in documentation for 3.8 and 3.9
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42822>
_______________________________________

From report at bugs.python.org  Mon Jan  4 11:32:45 2021
From: report at bugs.python.org (Mark Shannon)
Date: Mon, 04 Jan 2021 16:32:45 +0000
Subject: [New-bugs-announce] [issue42823] Incorrect frame.f_lineno when
 frame.f_trace is set
Message-ID: <1609777965.06.0.667792191004.issue42823@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

The logic for frame.f_lineno assumes that the internal C field will be updated when f_trace is set, but that is incorrect.
Consequently, the following code

import sys

def print_line():
    print(sys._getframe(1).f_lineno)

def test():
    print_line()
    sys._getframe(0).f_trace = True
    print_line()
    print_line()

test()

prints
7
8
8

it should print
7
9
10


Fixing this is required by PEP 626, but it should be fixed anyway.

----------
assignee: Mark.Shannon
messages: 384331
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Incorrect frame.f_lineno when frame.f_trace is set

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42823>
_______________________________________

From report at bugs.python.org  Mon Jan  4 11:55:18 2021
From: report at bugs.python.org (Fabian.M)
Date: Mon, 04 Jan 2021 16:55:18 +0000
Subject: [New-bugs-announce] [issue42824] get_type_hints throws for local
 class reference
Message-ID: <1609779318.04.0.39059957238.issue42824@roundup.psfhosted.org>


New submission from Fabian.M <f.mehmke at gmail.com>:

The following code throws with "NameError: name 'Nested' is not defined".

For reference, it works well when moving class definitions out of the local scope.


import typing
T = typing.TypeVar('T')

def test():
    class Nested(typing.Generic[T]):
        pass

    class Test(typing.Generic[T]):
      nested: Nested[T]

    typing.get_type_hints(Test) # this throws

test()

----------
components: Library (Lib)
messages: 384332
nosy: Fabian.M
priority: normal
severity: normal
status: open
title: get_type_hints throws for local class reference
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42824>
_______________________________________

From report at bugs.python.org  Mon Jan  4 12:42:40 2021
From: report at bugs.python.org (Austin Lamb)
Date: Mon, 04 Jan 2021 17:42:40 +0000
Subject: [New-bugs-announce] [issue42825] Optimization opportunity on Windows
Message-ID: <1609782160.13.0.0224895080942.issue42825@roundup.psfhosted.org>


New submission from Austin Lamb <austin.lamb at microsoft.com>:

I noticed that CPython and the various libraries built out of the repo aren't using the "/OPT:REF" linker optimization on Windows.  This optimization allows the linker to throw away dead/unreachable code, resulting in a substantial decrease in binary size.  That in turn also reduces the amount of Disk I/O that must be done to get these binaries in memory (which can be meaningful on spinning hard drives that are still rather common), and the reference set of applications using Python.

I'll send a PR for this shortly, just filing the issue to be able to reference it in the PR.

----------
components: Windows
messages: 384333
nosy: Austin-Lamb, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Optimization opportunity on Windows
type: performance
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42825>
_______________________________________

From report at bugs.python.org  Mon Jan  4 12:47:07 2021
From: report at bugs.python.org (=?utf-8?q?Peter_Pavlini=C4=8D?=)
Date: Mon, 04 Jan 2021 17:47:07 +0000
Subject: [New-bugs-announce] [issue42826] typing.Iterable does not
 need__getitem__() method
Message-ID: <1609782427.22.0.495333991832.issue42826@roundup.psfhosted.org>


New submission from Peter Pavlini? <peter.pavlinic at trilogic.si>:

https://docs.python.org/3.10/glossary.html#term-iterable

----------
assignee: docs at python
components: Documentation
messages: 384334
nosy: docs at python, peter.pavlinic
priority: normal
severity: normal
status: open
title: typing.Iterable does not need__getitem__() method
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42826>
_______________________________________

From report at bugs.python.org  Mon Jan  4 15:48:09 2021
From: report at bugs.python.org (Julien Palard)
Date: Mon, 04 Jan 2021 20:48:09 +0000
Subject: [New-bugs-announce] [issue42827] Multiline eval with assignment to
 function call segfaults
Message-ID: <1609793289.79.0.762652539307.issue42827@roundup.psfhosted.org>


New submission from Julien Palard <julien+python at palard.fr>:

Tested in 3.9.0 and master (127dde5916):

$ cat test.py
eval(
    """[file for str(file) in []
]
"""
)

$ ./python test.py
python: Parser/pegen.c:149: byte_offset_to_character_offset: Assertion `col_offset >= 0 && (unsigned long)col_offset <= strlen(str)' failed.
Fatal Python error: Aborted

Current thread 0x00007f6454cfe280 (most recent call first):
  File "/home/mdk/clones/python/cpython/test.py", line 1 in <module>
Aborted (core dumped)

(Adding Guido and Pablo to nosy as, IIRC, you're interested in parser things :))

----------
messages: 384348
nosy: gvanrossum, mdk, pablogsal
priority: normal
severity: normal
status: open
title: Multiline eval with assignment to function call segfaults
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42827>
_______________________________________

From report at bugs.python.org  Tue Jan  5 05:19:01 2021
From: report at bugs.python.org (Marco Franzo)
Date: Tue, 05 Jan 2021 10:19:01 +0000
Subject: [New-bugs-announce] [issue42828] Python readline module
Message-ID: <1609841941.92.0.341483279239.issue42828@roundup.psfhosted.org>


New submission from Marco Franzo <frenzisys at gmail.com>:

It would be better to write at the end of the program this:

os.system('stty sane')

because when you import readline, at the end of program, the console remains unusable

----------
assignee: docs at python
components: Documentation
messages: 384379
nosy: docs at python, frenzisys
priority: normal
severity: normal
status: open
title: Python readline module
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42828>
_______________________________________

From report at bugs.python.org  Tue Jan  5 06:18:32 2021
From: report at bugs.python.org (Svyatoslav)
Date: Tue, 05 Jan 2021 11:18:32 +0000
Subject: [New-bugs-announce] [issue42829] get_type_hints throws for nested
 class with __future__ annotations
Message-ID: <1609845512.57.0.0433631135188.issue42829@roundup.psfhosted.org>


New submission from Svyatoslav <prometheus3375 at gmail.com>:

If a class X has nested class Y and class X has annotation Y AFTER the definition of Y, get_type_hints(X) throws NameError if import from future annotations is present.

This is because get_type_hints looks only at mro of the class. Its namespace must be included too as locals.

----------
files: annotations.py
messages: 384381
nosy: Prometheus3375
priority: normal
severity: normal
status: open
title: get_type_hints throws for nested class with __future__ annotations
versions: Python 3.9
Added file: https://bugs.python.org/file49719/annotations.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42829>
_______________________________________

From report at bugs.python.org  Tue Jan  5 06:28:47 2021
From: report at bugs.python.org (=?utf-8?b?TWllY3p5c8WCYXcgVG9yY2hhxYJh?=)
Date: Tue, 05 Jan 2021 11:28:47 +0000
Subject: [New-bugs-announce] [issue42830] tempfile mkstemp() leaks file
 descriptors if os.close() is not called
Message-ID: <1609846127.44.0.860192251042.issue42830@roundup.psfhosted.org>


New submission from Mieczys?aw Torcha?a <mieczyslaw.torchala at gmail.com>:

tempfile mkstemp() documentation says: "Unlike TemporaryFile(), the user of mkstemp() is responsible for deleting the temporary file when done with it."

mkstemp() returns a tuple:

file_descriptor, file_path = mkstemp()

Calling only 

os.unlink(file_path) 

removes the file, but causes leaking file descriptors and when the number of temporary files created is higher than `ulimit -n`, the process crashes (see /proc/$pid/fd in real time until crash).

The solution I found is to also call on descriptor:

os.close(file_descriptor)

but the documentation doesn't mention that (i.e. releasing file descriptor in addition to removing temporary file).

For many users it doesn't matter as they create a few files and when the process finishes, leaking file descriptors are released. 

However, when a lot of files is created during the execution, it will finally crash (unless someone has a huge ulimit -n setting).

If this is not a bug, at least the documentation should mention that both the temp file needs to be removed and the file descriptor released. However, this means calling two commands when only one command was used to create the temporary file. Therefore, maybe adding a function to tempfile library to fully remove a file without a leaking file descriptor is a solution.

----------
components: Library (Lib)
messages: 384384
nosy: mieczyslaw.torchala
priority: normal
severity: normal
status: open
title: tempfile mkstemp() leaks file descriptors if os.close() is not called
type: resource usage
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42830>
_______________________________________

From report at bugs.python.org  Tue Jan  5 07:54:54 2021
From: report at bugs.python.org (E. Paine)
Date: Tue, 05 Jan 2021 12:54:54 +0000
Subject: [New-bugs-announce] [issue42831] IDLE fix colours for MacOS dark
 mode
Message-ID: <1609851294.14.0.191191657845.issue42831@roundup.psfhosted.org>


New submission from E. Paine <paineelisha at gmail.com>:

Currently IDLE, in some places, uses predefined colours which assume it is in light mode. However, on the new Universal2 installers (I tested using 3.10.0a4) this causes issues, where this can no longer be assumed.

Taking the "About IDLE" dialogue as an example, it appears in a light theme and most notably makes button text very hard to read (as this is changed to white). I propose, on macOS systems running Tk >=8.6.10, we try to switch these colours to the system colours described in https://github.com/tcltk/tk/blob/main/macosx/README (IMO, we would test colour availability and catch a TclError rather than specifying conditions ourselves). For the about dialogue I think we would, for example, replace `#bbbbbb` with one of the `systemWindowBackgroundColor`s.

----------
assignee: terry.reedy
components: IDLE
messages: 384391
nosy: epaine, taleinat, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE fix colours for MacOS dark mode
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42831>
_______________________________________

From report at bugs.python.org  Tue Jan  5 10:07:33 2021
From: report at bugs.python.org (Irit Katriel)
Date: Tue, 05 Jan 2021 15:07:33 +0000
Subject: [New-bugs-announce] [issue42832] classmethod change in 3.9 needs
 better documentation
Message-ID: <1609859253.42.0.13260337218.issue42832@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at yahoo.com>:

The change in issue 19072 broke our code where a decorator assumes that __get__ is only called for instance methods.

It might be worth adding this to What's new in 3.9.

Also in the documentation:
https://docs.python.org/3/library/functions.html#classmethod
It implies that before 3.9 it was not possible to wrap, but it was possible, it just behaved differently.

----------
assignee: docs at python
components: Documentation
messages: 384404
nosy: docs at python, iritkatriel
priority: normal
severity: normal
status: open
title: classmethod change in 3.9 needs better documentation
type: behavior
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42832>
_______________________________________

From report at bugs.python.org  Tue Jan  5 10:35:06 2021
From: report at bugs.python.org (Pierre Tardy)
Date: Tue, 05 Jan 2021 15:35:06 +0000
Subject: [New-bugs-announce] [issue42833] Lib/urllib/request.py: digest
 algorithm should be case insensitive
Message-ID: <1609860906.08.0.978981588338.issue42833@roundup.psfhosted.org>


New submission from Pierre Tardy <tardyp at gmail.com>:

original bug report: https://github.com/buildbot/buildbot/issues/5743

Twisted by default advertises its algorithm in lowercase, which is uncommon, but allowed by the spec.

https://tools.ietf.org/html/rfc3230#section-4.1.1


python's request.py is only supporting MD5 or SHA as algorithm, and not lowercase equivalent.

This is an easy and harmless fix.

----------
messages: 384409
nosy: Pierre.Tardy
priority: normal
severity: normal
status: open
title: Lib/urllib/request.py: digest algorithm should be case insensitive
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42833>
_______________________________________

From report at bugs.python.org  Tue Jan  5 10:54:05 2021
From: report at bugs.python.org (Ken Jin)
Date: Tue, 05 Jan 2021 15:54:05 +0000
Subject: [New-bugs-announce] [issue42834] [subinterpreters] Convert "global"
 static variable caches in _json to heap variables
Message-ID: <1609862045.06.0.177130417143.issue42834@roundup.psfhosted.org>


New submission from Ken Jin <kenjin4096 at gmail.com>:

_json currently uses a few static locals for caching of PyUnicode and one Exception object.  This patch aims to move them to the heap.

Apologies if this isn't the right way to track subinterpreter issues, I took a look at issue36876 but it seemed like a meta issue to me.

----------
components: Library (Lib), Subinterpreters
messages: 384411
nosy: eric.snow, kj, vstinner
priority: normal
severity: normal
status: open
title: [subinterpreters] Convert "global" static variable caches in _json to heap variables
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42834>
_______________________________________

From report at bugs.python.org  Tue Jan  5 18:53:31 2021
From: report at bugs.python.org (Michael L. Boom)
Date: Tue, 05 Jan 2021 23:53:31 +0000
Subject: [New-bugs-announce] [issue42835] Can't Built libxml2 When Using
 Python 3.9.x
Message-ID: <1609890811.68.0.419297459642.issue42835@roundup.psfhosted.org>


New submission from Michael L. Boom <mikeboom at mikeboom.com>:

When building libxml2 using Python 3.9.0 or Python 3.9.1 and gcc 10.2.0 I get the below error.  Others are having this problem according to my Google search.
make[4]: Entering directory '/local/users/michael.l.boom/gits/scoring_engine/git/aae_tmp_dir/libxml2-2.9.10/python'
  CC       libxml.lo
  CC       libxml2-py.lo
  CC       types.lo
In file included from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/pytime.h:6,
                 from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/Python.h:85,
                 from libxml_wrap.h:1,
                 from types.c:9:
types.c: In function ?libxml_xmlXPathObjectPtrConvert?:
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/object.h:633:41: error: expected ?(? before ?PyType_HasFeature?
  633 | #define PyType_FastSubclass(type, flag) PyType_HasFeature(type, flag)
      |                                         ^~~~~~~~~~~~~~~~~
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/longobject.h:15:9: note: in expansion of macro ?PyType_FastSubclass?
   15 |         PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_LONG_SUBCLASS)
      |         ^~~~~~~~~~~~~~~~~~~
types.c:607:15: note: in expansion of macro ?PyLong_Check?
  607 |     } else if PyLong_Check(obj) {
      |               ^~~~~~~~~~~~
types.c:607:33: error: expected ?;? before ?{? token
  607 |     } else if PyLong_Check(obj) {
      |                                 ^
libxml.c: In function ?xmlPythonFileCloseRaw?:
libxml.c:259:5: warning: ?PyEval_CallMethod? is deprecated [-Wdeprecated-declarations]
  259 |     ret = PyEval_CallMethod(file, (char *) "close", (char *) "()");
      |     ^~~
In file included from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/Python.h:145,
                 from libxml.c:14:
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/ceval.h:28:43: note: declared here
   28 | Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallMethod(
      |                                           ^~~~~~~~~~~~~~~~~
libxml.c: In function ?xmlPythonFileReadRaw?:
libxml.c:289:5: warning: ?PyEval_CallMethod? is deprecated [-Wdeprecated-declarations]
  289 |     ret = PyEval_CallMethod(file, (char *) "read", (char *) "(i)", len);
      |     ^~~
In file included from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/Python.h:145,
                 from libxml.c:14:
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/ceval.h:28:43: note: declared here
   28 | Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallMethod(
      |                                           ^~~~~~~~~~~~~~~~~
In file included from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/pytime.h:6,
                 from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/Python.h:85,
                 from libxml.c:14:
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/object.h:633:41: error: expected ?(? before ?PyType_HasFeature?
  633 | #define PyType_FastSubclass(type, flag) PyType_HasFeature(type, flag)
      |                                         ^~~~~~~~~~~~~~~~~
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/unicodeobject.h:115:18: note: in expansion of macro ?PyType_FastSubclass?
  115 |                  PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_UNICODE_SUBCLASS)
      |                  ^~~~~~~~~~~~~~~~~~~
libxml.c:297:15: note: in expansion of macro ?PyUnicode_Check?
  297 |     } else if PyUnicode_Check (ret) {
      |               ^~~~~~~~~~~~~~~
libxml.c:297:37: error: expected ?;? before ?{? token
  297 |     } else if PyUnicode_Check (ret) {
      |                                     ^
libxml.c: In function ?xmlPythonFileRead?:
libxml.c:354:5: warning: ?PyEval_CallMethod? is deprecated [-Wdeprecated-declarations]
  354 |     ret = PyEval_CallMethod(file, (char *) "io_read", (char *) "(i)", len);
      |     ^~~
In file included from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/Python.h:145,
                 from libxml.c:14:
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/ceval.h:28:43: note: declared here
   28 | Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallMethod(
      |                                           ^~~~~~~~~~~~~~~~~
In file included from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/pytime.h:6,
                 from /local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/Python.h:85,
                 from libxml.c:14:
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/object.h:633:41: error: expected ?(? before ?PyType_HasFeature?
  633 | #define PyType_FastSubclass(type, flag) PyType_HasFeature(type, flag)
      |                                         ^~~~~~~~~~~~~~~~~
/local/users/michael.l.boom/gits/scoring_engine/git/aae_build_python_3.9.1/include/python3.9/unicodeobject.h:115:18: note: in expansion of macro ?PyType_FastSubclass?
  115 |                  PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_UNICODE_SUBCLASS)
      |                  ^~~~~~~~~~~~~~~~~~~
libxml.c:362:15: note: in expansion of macro ?PyUnicode_Check?
  362 |     } else if PyUnicode_Check (ret) {
      |               ^~~~~~~~~~~~~~~
libxml.c:362:37: error: expected ?;? before ?{? token
  362 |     } else if PyUnicode_Check (ret) {
      |                                     ^

----------
components: C API
messages: 384443
nosy: boom0192
priority: normal
severity: normal
status: open
title: Can't Built libxml2 When Using Python 3.9.x
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42835>
_______________________________________

From report at bugs.python.org  Wed Jan  6 04:05:31 2021
From: report at bugs.python.org (Sandeep Subramanian)
Date: Wed, 06 Jan 2021 09:05:31 +0000
Subject: [New-bugs-announce] [issue42836] docs: struct: clarify struct
 caching behaviour
Message-ID: <1609923931.89.0.0226570295473.issue42836@roundup.psfhosted.org>


New submission from Sandeep Subramanian <sandeep at mercari.com>:

As per docs in :https://docs.python.org/3.7/library/struct.html#struct.Struct

> The compiled versions of the most recent format strings passed to Struct and the module-level functions are cached, so programs that use only a few format strings needn?t worry about reusing a single Struct instance.

This statement seems to imply that the format string passed as `struct.Struct(format)` is also cached. This doesn't appear to be the case.

Only format strings passed to the module level functions seems to be cached.

`cache_struct_converter` at https://github.com/python/cpython/blob/v3.7.7/Modules/_struct.c#L2052 
appears to be used only in the top module level functions `pack` and `pack_into` and is *not* used in `Struct.__init__`.


Would it be better to change the doc to:

> The compiled versions of the most recent format strings passed to the module-level functions are cached, so programs that use only a few format strings needn?t worry about reusing a single Struct instance.

This issue was introduced by https://github.com/python/cpython/commit/3666b3c1f695a145adab1bf644c22e564e8eb0ee

and is present in every version since.

Refs:
https://github.com/python/cpython/pull/7700
https://bugs.python.org/issue33032

----------
assignee: docs at python
components: Documentation
messages: 384476
nosy: docs at python, sandeep2
priority: normal
severity: normal
status: open
title: docs: struct: clarify struct caching behaviour
type: enhancement
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42836>
_______________________________________

From report at bugs.python.org  Wed Jan  6 04:51:20 2021
From: report at bugs.python.org (Mark Shannon)
Date: Wed, 06 Jan 2021 09:51:20 +0000
Subject: [New-bugs-announce] [issue42837] Symbol table incorrectly
 identifies code as a generator, when 'yield' occurs in an annotation
Message-ID: <1609926680.76.0.190133632062.issue42837@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

This is an internal inconsistency. I have not identified any surface level bugs, but it is a trap for future compiler work.

----------
assignee: Mark.Shannon
messages: 384479
nosy: Mark.Shannon, larry
priority: normal
severity: normal
status: open
title: Symbol table incorrectly identifies code as a generator, when 'yield' occurs in an annotation

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42837>
_______________________________________

From report at bugs.python.org  Wed Jan  6 08:56:08 2021
From: report at bugs.python.org (xloem)
Date: Wed, 06 Jan 2021 13:56:08 +0000
Subject: [New-bugs-announce] [issue42838] Wait for cleanup coroutines before
 event loop is closed.
Message-ID: <1609941368.33.0.950412728581.issue42838@roundup.psfhosted.org>


New submission from xloem <0xloem at gmail.com>:

To handle destruction of resources especially during exceptions, it would be nice if there were some way to provide coroutines/tasks that run at the termination or closure of an event loop.  There are a lot of api options here.  Maybe a simple one would be to have close() call a run_until_closed() function that starts all of these and steps the loop to wait for them, before closing, and provide run_close() functions to queue them for delay.

----------
components: asyncio
messages: 384503
nosy: asvetlov, xloem, yselivanov
priority: normal
severity: normal
status: open
title: Wait for cleanup coroutines before event loop is closed.
type: enhancement
versions: Python 3.10, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42838>
_______________________________________

From report at bugs.python.org  Wed Jan  6 09:32:12 2021
From: report at bugs.python.org (favonia)
Date: Wed, 06 Jan 2021 14:32:12 +0000
Subject: [New-bugs-announce] [issue42839] SourceFileLoader does not (fully)
 accept path-like objects
Message-ID: <1609943532.78.0.260162878801.issue42839@roundup.psfhosted.org>


New submission from favonia <favonia at gmail.com>:

If one uses the loader created by importlib.machinery.SourceFileLoader(module_name, path) in a meta path finder, where path is not a str, then the following error would happen at the moment the module is being imported. Note that, the error would not occur if the corresponding bytecode cache (__pycache__/*.pyc) is present.

  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 786, in exec_module
  File "<frozen importlib._bootstrap_external>", line 932, in get_code
  File "<frozen importlib._bootstrap_external>", line 604, in _code_to_timestamp_pyc
ValueError: unmarshallable object

Here is an example that could trigger the error. The package unipath is not very important except that unipath.Path is not marshallable. (Any library with a fancy, unmarshallable path-like object would work.) The choice of 'local' and the use of 'os.getcwd()' are also irrelevant. The actual production code is quite complicated, using different path-like objects and convoluted logic. I made up this example from scratch.

import os
import sys
import importlib
import importlib.abc
import importlib.util
from unipath import Path # just an example

class DummyFinder(importlib.abc.MetaPathFinder):
    def __init__(self):
        self.cwd = os.getcwd()
    def find_spec(self, fullname, path, target=None):
        if fullname == 'local':
            initpath = os.path.join(self.cwd, '__init__.py')
            if os.path.isfile(initpath):
                loader = importlib.machinery.SourceFileLoader(fullname, Path(initpath)) # some non-str Path-like object here
                return importlib.util.spec_from_file_location(fullname, initpath,
                        loader = loader, submodule_search_locations=[self.cwd])
        else:
            return None

sys.meta_path.append(DummyFinder())
importlib.import_module("local.test2")

If this is indeed a bug, there might be other classes and functions in importlib that share the same problem.

----------
components: Library (Lib)
messages: 384504
nosy: favonia
priority: normal
severity: normal
status: open
title: SourceFileLoader does not (fully) accept path-like objects
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42839>
_______________________________________

From report at bugs.python.org  Wed Jan  6 10:21:21 2021
From: report at bugs.python.org (Erik Soma)
Date: Wed, 06 Jan 2021 15:21:21 +0000
Subject: [New-bugs-announce] [issue42840] `type` takes **kwargs for
 __init_subclass__
Message-ID: <1609946481.51.0.225785046936.issue42840@roundup.psfhosted.org>


New submission from Erik Soma <stillusingirc at gmail.com>:

The documentation (https://docs.python.org/3/library/functions.html#type) shows type's signature as:

class type(object)
class type(name, bases, dict)


But the "actual" 2nd signature in CPython 3.6+ is:

class type(name, bases, dict, **kwargs)


**kwargs here gets passed to __init_subclass__ in the same way that keywords in a class statement do so that:

type("Bar", (Foo,), {}, spam='ham')

is equivalent to

class Bar(Foo, spam='ham'): pass


It's not clear to me whether this is behavior to rely on. I started using this intuitively, but found that my type checker reasonably complained.

Looking through the commit that implemented PEP 487 (d78448e9), it seems this may have been incidental. Additionally I haven't found mention of this in PEP 487 or the documentation and I can't seem to find any tests for it.

----------
messages: 384506
nosy: esoma
priority: normal
severity: normal
status: open
title: `type` takes **kwargs for __init_subclass__
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42840>
_______________________________________

From report at bugs.python.org  Wed Jan  6 10:27:26 2021
From: report at bugs.python.org (Yurii Karabas)
Date: Wed, 06 Jan 2021 15:27:26 +0000
Subject: [New-bugs-announce] [issue42841] Add bitwise or operator to
 collections.abc Mapping and MutableMapping
Message-ID: <1609946846.04.0.311249244168.issue42841@roundup.psfhosted.org>


New submission from Yurii Karabas <1998uriyyo at gmail.com>:

In python3.9 `dict` `bitwise or` operator was introduced.

I think it will be great if `collections.abc.Mapping` and `collections.abc.MutableMapping` will have a default implementation of `bitwise or` operator.

----------
messages: 384507
nosy: uriyyo
priority: normal
severity: normal
status: open
title: Add bitwise or operator to collections.abc Mapping and MutableMapping

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42841>
_______________________________________

From report at bugs.python.org  Wed Jan  6 11:30:16 2021
From: report at bugs.python.org (sasquatchplatypus)
Date: Wed, 06 Jan 2021 16:30:16 +0000
Subject: [New-bugs-announce] [issue42842] import logging.handlers shows
 weird behavior
Message-ID: <1609950616.34.0.697361127063.issue42842@roundup.psfhosted.org>


New submission from sasquatchplatypus <maxiwei at aol.com>:

when trying to import a handler from logging.handlers, you have to explicitely import logging.handlers, because trying to get it from logging directly (for example import logging, logging.handlers.RotatingFileHandlers)
raises this error: AttributeError: module 'logging' has no attribute 'handlers'

----------
messages: 384509
nosy: sasquatchplatypus
priority: normal
severity: normal
status: open
title: import logging.handlers shows weird behavior
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42842>
_______________________________________

From report at bugs.python.org  Wed Jan  6 11:52:46 2021
From: report at bugs.python.org (Julien Palard)
Date: Wed, 06 Jan 2021 16:52:46 +0000
Subject: [New-bugs-announce] [issue42843] What min_sphinx for Python 3.10
Message-ID: <1609951966.97.0.867485424131.issue42843@roundup.psfhosted.org>


New submission from Julien Palard <julien+python at palard.fr>:

According to [1] there's no concensus on which minimum Sphinx version we should use for Python 3.10.

"sadly" since https://github.com/python/cpython/pull/23620 which make use of a Sphinx 3.2.0 only feature, we should bump it to 3.2, but it may not work for everybody.

[1]: https://github.com/python/cpython/pull/23620#issuecomment-755420205

----------
assignee: mdk
components: Documentation
messages: 384512
nosy: mdk
priority: normal
severity: normal
status: open
title: What min_sphinx for Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42843>
_______________________________________

From report at bugs.python.org  Wed Jan  6 12:00:12 2021
From: report at bugs.python.org (Stephen)
Date: Wed, 06 Jan 2021 17:00:12 +0000
Subject: [New-bugs-announce] [issue42844] Turtle Module -- "onclick"
 arguments enchancement
Message-ID: <1609952412.13.0.0764333219073.issue42844@roundup.psfhosted.org>


New submission from Stephen <carlson.stephen13 at gmail.com>:

I have created an enhancement in the Turtle module. When a programmer wants to have an action performed after clicking on a Turtle object, the programmer is currently unable to supply any arguments into the method that is run when "on_clicked" which is extremely limiting, especially to beginners who are looking to modify multiple objects on the screen at one time, such as in a game. I have modified the implementation of the ?on_clicked? method to be able to provide keyword arguments into the method through a dictionary that is later unpacked into the target method. Attached is an example of the benefits of this enhancement to the turtle module.

----------
components: Library (Lib)
files: on_click_arguments_example.py
messages: 384513
nosy: sc1341
priority: normal
severity: normal
status: open
title: Turtle Module -- "onclick" arguments enchancement
type: enhancement
versions: Python 3.10
Added file: https://bugs.python.org/file49723/on_click_arguments_example.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42844>
_______________________________________

From report at bugs.python.org  Wed Jan  6 12:08:08 2021
From: report at bugs.python.org (Shreyan Avigyan)
Date: Wed, 06 Jan 2021 17:08:08 +0000
Subject: [New-bugs-announce] [issue42845] Third Party Module Installation
 using pip fails in Python 3.10.0a3
Message-ID: <1609952888.82.0.0280575677398.issue42845@roundup.psfhosted.org>


New submission from Shreyan Avigyan <shreyan.avigyan at gmail.com>:

A few days ago, I tried installing Python 3.10.a3 for testing. After installing, I tried using the Python built-in modules, they were working fine. Then I tried going for installing third party modules using pip from the command prompt. While installing I noticed that some modules I could install perfectly but for most of the modules, they failed to install. After trying to install them, I tried looking for the full error message. After a lot of searching I found out that it is looking for Microsoft Visual C++ and couldn't find it because I haven't installed it and gave me that error. I haven't tried installing Python 3.10.0a4 yet and I don't know if it's already solved or not but this problem is continuously occurring in Python 3.10.0a3.

----------
components: Installation
messages: 384515
nosy: shreyanavigyan
priority: normal
severity: normal
status: open
title: Third Party Module Installation using pip fails in Python 3.10.0a3
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42845>
_______________________________________

From report at bugs.python.org  Wed Jan  6 17:38:22 2021
From: report at bugs.python.org (neonene)
Date: Wed, 06 Jan 2021 22:38:22 +0000
Subject: [New-bugs-announce] [issue42846] Using _multibytecodec module on
 Windows, test_threading/embed get failure
Message-ID: <1609972702.01.0.789985510588.issue42846@roundup.psfhosted.org>


New submission from neonene <nicesalmon at gmail.com>:

After https://github.com/python/cpython/commit/0b858cdd5d114f0890b11b6c4d6559d0ceb468ab
(bpo-1635741: Convert _multibytecodec to multi-phase init),

On Windows x64/x86 with chinese/japanese/korean system-locale,
MultibyteCodec_Check() in multibytecodec.c returns false and
PyExc_TypeError follows. This affects some tests and PGO training.



1) python -m test --verbose test_threading

======================================================================
FAIL: test_daemon_threads_fatal_error (test.test_threading.SubinterpThreadi
ngTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\cpython-0b858\lib\test\test_threading.py", line 1124, in test_da
emon_threads_fatal_error
    self.assertIn("Fatal Python error: Py_EndInterpreter: "
AssertionError: 'Fatal Python error: Py_EndInterpreter: not the last thread
' not found in 'TypeError: codec is unexpected type\nFatal Python error: _P
yThreadState_Delete: tstate 00000000003FF980 is still current\nPython runti
me state: initialized\n\nThread 0x00000710 (most recent call first):\n<no P
ython frame>\n'



2) python -m test --verbose test_embed

======================================================================
FAIL: test_audit_subinterpreter (test.test_embed.AuditingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\cpython-0b858\lib\test\test_embed.py", line 1433, in test_audit_
subinterpreter
    self.run_embedded_interpreter("test_audit_subinterpreter")
  File "C:\cpython-0b858\lib\test\test_embed.py", line 104, in run_embedded
_interpreter
    self.assertEqual(p.returncode, returncode,
AssertionError: 3221225477 != 0 : bad returncode 3221225477, stderr is 'Typ
eError: codec is unexpected type\nFatal Python error: _PyThreadState_Delete
: tstate 000000000050CAF0 is still current\nPython runtime state: initializ
ed\n\nThread 0x000009d8 (most recent call first):\n<no Python frame>\n'

======================================================================
FAIL: test_subinterps_different_ids (test.test_embed.EmbeddingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\cpython-0b858\lib\test\test_embed.py", line 169, in test_subinte
rps_different_ids
    for run in self.run_repeated_init_and_subinterpreters():
  File "C:\cpython-0b858\lib\test\test_embed.py", line 110, in run_repeated
_init_and_subinterpreters
    out, err = self.run_embedded_interpreter("test_repeated_init_and_subint
erpreters")
  File "C:\cpython-0b858\lib\test\test_embed.py", line 104, in run_embedded
_interpreter
    self.assertEqual(p.returncode, returncode,
AssertionError: 3221225477 != 0 : bad returncode 3221225477, stderr is 'Typ
eError: codec is unexpected type\nFatal Python error: _PyThreadState_Delete
: tstate 000000000041C960 is still current\nPython runtime state: initializ
ed\n\nThread 0x00000a40 (most recent call first):\n<no Python frame>\n'

======================================================================
FAIL: test_subinterps_distinct_state (test.test_embed.EmbeddingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\cpython-0b858\lib\test\test_embed.py", line 177, in test_subinte
rps_distinct_state
    for run in self.run_repeated_init_and_subinterpreters():
  File "C:\cpython-0b858\lib\test\test_embed.py", line 110, in run_repeated
_init_and_subinterpreters
    out, err = self.run_embedded_interpreter("test_repeated_init_and_subint
erpreters")
  File "C:\cpython-0b858\lib\test\test_embed.py", line 104, in run_embedded
_interpreter
    self.assertEqual(p.returncode, returncode,
AssertionError: 3221225477 != 0 : bad returncode 3221225477, stderr is 'Typ
eError: codec is unexpected type\nFatal Python error: _PyThreadState_Delete
: tstate 000000000047C960 is still current\nPython runtime state: initializ
ed\n\nThread 0x00000b34 (most recent call first):\n<no Python frame>\n'

======================================================================
FAIL: test_subinterps_main (test.test_embed.EmbeddingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\cpython-0b858\lib\test\test_embed.py", line 163, in test_subinte
rps_main
    for run in self.run_repeated_init_and_subinterpreters():
  File "C:\cpython-0b858\lib\test\test_embed.py", line 110, in run_repeated
_init_and_subinterpreters
    out, err = self.run_embedded_interpreter("test_repeated_init_and_subint
erpreters")
  File "C:\cpython-0b858\lib\test\test_embed.py", line 104, in run_embedded
_interpreter
    self.assertEqual(p.returncode, returncode,
AssertionError: 3221225477 != 0 : bad returncode 3221225477, stderr is 'Typ
eError: codec is unexpected type\nFatal Python error: _PyThreadState_Delete
: tstate 000000000032C960 is still current\nPython runtime state: initializ
ed\n\nThread 0x00000bf0 (most recent call first):\n<no Python frame>\n'

----------
components: Extension Modules, Subinterpreters, Tests, Windows
messages: 384541
nosy: corona10, erlendaasland, neonene, paul.moore, petr.viktorin, shihai1991, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: Using _multibytecodec module on Windows, test_threading/embed get failure
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42846>
_______________________________________

From report at bugs.python.org  Wed Jan  6 18:41:04 2021
From: report at bugs.python.org (Erlend Egeberg Aasland)
Date: Wed, 06 Jan 2021 23:41:04 +0000
Subject: [New-bugs-announce] [issue42847] Normalise file encodings in
 Lib/sqlite3/test/
Message-ID: <1609976464.16.0.541873091964.issue42847@roundup.psfhosted.org>


New submission from Erlend Egeberg Aasland <erlend.aasland at innova.no>:

GitHub does not play well with other file encodings than UTF-8. For example, when the "suggestion" in https://github.com/python/cpython/pull/20538#discussion_r552926126 was applied, GitHub silently converted Lib/sqlite3/test/types.py from ISO-8859 to UTF-8, as seen in the resulting commit fd4f65141925f2d84e0efd17ca4b9395c171d7ba. Apart from being annoying, this created an inconsistency between the `#-*- coding: iso-8859-1 -*-` header in Lib/sqlite3/test/types.py and its actual encoding (now UTF-8).

I suggest to convert Lib/sqlite3/test/* to UTF-8 encoding for consistency.

$ file Lib/sqlite3/test/*.py
Lib/sqlite3/test/backup.py:        Python script text executable, ASCII text
Lib/sqlite3/test/dbapi.py:         Python script text executable, ISO-8859 text
Lib/sqlite3/test/dump.py:          Python script text executable, ASCII text
Lib/sqlite3/test/factory.py:       Python script text executable, ISO-8859 text
Lib/sqlite3/test/hooks.py:         Python script text executable, ISO-8859 text
Lib/sqlite3/test/regression.py:    Python script text executable, ISO-8859 text
Lib/sqlite3/test/transactions.py:  Python script text executable, ISO-8859 text
Lib/sqlite3/test/types.py:         Python script text executable, ISO-8859 text
Lib/sqlite3/test/userfunctions.py: Python script text executable, UTF-8 Unicode text

----------
components: Library (Lib)
messages: 384543
nosy: berker.peksag, erlendaasland
priority: normal
severity: normal
status: open
title: Normalise file encodings in Lib/sqlite3/test/
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42847>
_______________________________________

From report at bugs.python.org  Wed Jan  6 21:10:12 2021
From: report at bugs.python.org (asleep-cult)
Date: Thu, 07 Jan 2021 02:10:12 +0000
Subject: [New-bugs-announce] [issue42848] asyncio produces an unexpected
 traceback with recursive __getattribute__
Message-ID: <1609985412.09.0.749639106788.issue42848@roundup.psfhosted.org>


New submission from asleep-cult <asleep.cult at gmail.com>:

The code below produces an unexpected traceback format that makes the source of an error very hard to find, this more than likely happens in less specific situations but this is how I came across it. I'm also not sure if this is an issue with asyncio, logging or the traceback module.


import asyncio

class Bar:
    def __init__(self):
        self.y = None

class Foo:
    def __init__(self):
        self._bar = Bar()
        self.y

    def __getattribute__(self, name):
        try:
            attr = super().__getattribute__(name)
        except AttributeError as e:
            try:
                attr = self.ooops_spelled_bar_wrong.__getattribute__(name)
            except AttributeError:
                raise e
        return attr

async def foo():
    Foo()

loop = asyncio.get_event_loop()
loop.create_task(foo())
loop.run_forever()


Traceback:


Exception in default exception handler
Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 1733, in call_exception_handler    self.default_exception_handler(context)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 1707, in default_exception_handler
    logger.error('\n'.join(log_lines), exc_info=exc_info)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 1471, in error
    self._log(ERROR, msg, args, **kwargs)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 1585, in _log
    self.handle(record)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 1595, in handle
    self.callHandlers(record)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 1665, in callHandlers
    lastResort.handle(record)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 950, in handle
    self.emit(record)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 1081, in emit
    msg = self.format(record)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 925, in format
    return fmt.format(record)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 672, in format
    record.exc_text = self.formatException(record.exc_info)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\logging\__init__.py", line 622, in formatException
    traceback.print_exception(ei[0], ei[1], tb, None, sio)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 103, in print_exception
    for line in TracebackException(
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 493, in __init__
    context = TracebackException(
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 493, in __init__
    context = TracebackException(
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 493, in __init__
    context = TracebackException(
  [Previous line repeated 488 more times]
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 476, in __init__
    _seen.add(id(exc_value))
RecursionError: maximum recursion depth exceeded while calling a Python object
``` 

The code below procures the expected traceback

 
import traceback

class Bar:
    def __init__(self):
        self.y = None

class Foo:
    def __init__(self):
        self._bar = Bar()
        self.y

    def __getattribute__(self, name):
        try:
            attr = super().__getattribute__(name)
        except AttributeError as e:
            try:
                attr = self.ooops_spelled_bar_wrong.__getattribute__(name)
            except AttributeError:
                raise e
        return attr

try:
    Foo()
except:
    traceback.print_exc()


Traceback:


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:/Development/test/test.py", line 15, in __getattribute__
    attr = super().__getattribute__(name)
AttributeError: 'Foo' object has no attribute 'ooops_spelled_bar_wrong'

# The part above is repeated hundreds of times

Traceback (most recent call last):
  File "c:/Development/test/test.py", line 24, in <module>
    Foo()
  File "c:/Development/test/test.py", line 11, in __init__
    self.y
  File "c:/Development/test/test.py", line 18, in __getattribute__
    attr = self.ooops_spelled_bar_wrong.__getattribute__(name)
  File "c:/Development/test/test.py", line 18, in __getattribute__
    attr = self.ooops_spelled_bar_wrong.__getattribute__(name)
  File "c:/Development/test/test.py", line 18, in __getattribute__
    attr = self.ooops_spelled_bar_wrong.__getattribute__(name)
  [Previous line repeated 992 more times]
  File "c:/Development/test/test.py", line 15, in __getattribute__
    attr = super().__getattribute__(name)
RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:/Development/test/test.py", line 26, in <module>
    traceback.print_exc()
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 163, in print_exc
    print_exception(*sys.exc_info(), limit=limit, file=file, chain=chain)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 103, in print_exception
    for line in TracebackException(
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 493, in __init__
    context = TracebackException(
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 493, in __init__
    context = TracebackException(
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\traceback.py", line 493, in __init__
    context = TracebackException(
  [Previous line repeated 495 more times]
RecursionError: maximum recursion depth exceeded

----------
components: Library (Lib)
messages: 384551
nosy: asleep-cult
priority: normal
severity: normal
status: open
title: asyncio produces an unexpected traceback with recursive __getattribute__
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42848>
_______________________________________

From report at bugs.python.org  Wed Jan  6 23:00:06 2021
From: report at bugs.python.org (Zhesi Huang)
Date: Thu, 07 Jan 2021 04:00:06 +0000
Subject: [New-bugs-announce] [issue42849] pool worker can't be terminated
Message-ID: <1609992006.83.0.897606551243.issue42849@roundup.psfhosted.org>


New submission from Zhesi Huang <huangzhesi at gmail.com>:

i see a case, the worker proc of pool can't be terminated, i try kill -SIGTERM or -SIGINT, but it still can't terminated the worker proc, then the pool __exit__ will hang forever

```
class NonDaemonProcess(multiprocessing.Process):
    # https://stackoverflow.com/questions/6974695/python-process-pool-non-daemonic

    # make 'daemon' attribute always return False
    def _get_daemon(self):
        return False

    def _set_daemon(self, value):
        pass

    daemon = property(_get_daemon, _set_daemon)


def wrapper_copy_parallel(output_name, local_output_path, obs_path):
    log_thread('upload the content of [%s] outputs' % output_name)
    try:
        if os.path.exists(local_output_path):
            log_thread('%s has %d files to be uploaded' % (output_name, len(file_list)))
            
            xxxxxx

            log_thread('upload the content of [%s] outputs successfully' % output_name)
            log_thread('it can be accessed at obs path [%s]' % obs_path)
        else:
            log_thread('local output path is not found, skip upload the content of [%s] outputs' % output_name)
    except Exception as upload_exception:
        err_thread('upload the content of [%s] outputs failed: %s' % (output_name, str(upload_exception)))
        return 255

    return 0


def upload_to_s3():
    """
    upload the content of local path to s3, handle action [on_completed]
    :return:
    """
    outputs = []
    for local_output_path, (output_name, obs_path, action, _, _) in local_to_target.items():
        if action == ACTION_ON_COMPLETED:
            outputs.append((output_name, local_output_path, obs_path))

    if len(outputs) == 0:
        return 0

    with NonDaemonPool(processes=len(outputs)) as pool:
        results = pool.starmap(wrapper_copy_parallel, outputs)

    for result in results:
        if result != 0:
            return result

    return 0
```

stdout/stderr log

[ma-test Service Log][INFO][2021/01/05 03:07:49,218]: registered signal handler
[ma-test Service Log][INFO][2021/01/05 03:09:40,299]: output-handler finalizing
[ma-test Service Log][INFO][2021-01-05 03:09:40,309][NonDaemonPoolWorker-1]: child process calling self.run()
[ma-test Service Log][INFO][2021-01-05 03:09:40,311][NonDaemonPoolWorker-1]: upload the content of [] outputs
[ma-test Service Log][INFO][2021-01-05 03:09:41,331][Process-1:1]: child process calling self.run()
[ma-test Service Log][INFO][2021-01-05 03:09:41,333][Process-1:2]: child process calling self.run()
[ma-test Service Log][INFO][2021-01-05 03:09:41,338][Process-1:3]: child process calling self.run()
[ma-test Service Log][INFO][2021-01-05 03:09:41,351][Process-1:1]: process shutting down
[ma-test Service Log][INFO][2021-01-05 03:09:41,351][Process-1:1]: process exiting with exitcode 0
[ma-test Service Log][INFO][2021-01-05 03:09:41,386][Process-1:2]: process shutting down
[ma-test Service Log][INFO][2021-01-05 03:09:41,386][Process-1:2]: process exiting with exitcode 0
[ma-test Service Log][INFO][2021-01-05 03:09:41,410][Process-1:3]: process shutting down
[ma-test Service Log][INFO][2021-01-05 03:09:41,410][Process-1:3]: process exiting with exitcode 0
[ma-test Service Log][INFO][2021-01-05 03:09:41,415][NonDaemonPoolWorker-1]: upload the content of [] outputs successfully
[ma-test Service Log][INFO][2021-01-05 03:09:41,415][NonDaemonPoolWorker-1]: it can be accessed at obs path [s3://ma-test-algorancher-intel/model_evaluation/6e5746ff-2839-400a-ba93-df38311415f4/dac957b0-b43b-43e2-ab19-0b45672a7ea0/]

18 (pid) proc python stacktrace

>>>
Interrupting process at following point:
  File "/home/ma-user/runtime-scripts-v2/init-container/outputs-handler.py", line 396, in <module>
    ret_code = upload_to_obs()
  File "/home/ma-user/runtime-scripts-v2/init-container/outputs-handler.py", line 287, in upload_to_obs
    results = pool.starmap(wrapper_copy_parallel, outputs)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/pool.py", line 623, in __exit__
    self.terminate()
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/pool.py", line 548, in terminate
    self._terminate()
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/util.py", line 201, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/pool.py", line 617, in _terminate_pool
    p.join()
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/process.py", line 140, in join
    res = self._popen.wait(timeout)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/popen_fork.py", line 48, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)

18 (pid) proc gdb bt

#0  0x00007f15c2c5ff7b in waitpid () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x000055f54a8a5180 in os_waitpid_impl.isra.78 (options=0, pid=28) at /tmp/build/80754af9/python_1588882889832/work/Modules/posixmodule.c:7066
#2  os_waitpid () at /tmp/build/80754af9/python_1588882889832/work/Modules/clinic/posixmodule.c.h:3049
#3  0x000055f54a7f1b19 in _PyMethodDef_RawFastCallKeywords () at /tmp/build/80754af9/python_1588882889832/work/Objects/call.c:655
#4  0x000055f54a7f1db1 in _PyCFunction_FastCallKeywords (func=0x7f15c1f175f0, args=<optimized out>, nargs=<optimized out>, kwnames=<optimized out>)
    at /tmp/build/80754af9/python_1588882889832/work/Objects/call.c:734

28 (pid) proc python stacktrace

Interrupting process at following point:
  File "/home/ma-user/runtime-scripts-v2/init-container/outputs-handler.py", line 396, in <module>
    ret_code = upload_to_obs()
  File "/home/ma-user/runtime-scripts-v2/init-container/outputs-handler.py", line 286, in upload_to_obs
    with NonDaemonPool(processes=len(outputs)) as pool:
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/pool.py", line 176, in __init__
    self._repopulate_pool()
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/pool.py", line 241, in _repopulate_pool
    w.start()
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/process.py", line 112, in start
    self._popen = self._Popen(self)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
    code = process_obj._bootstrap()
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/pool.py", line 110, in worker
    task = get()
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/queues.py", line 351, in get
    with self._rlock:
  File "/home/ma-user/miniconda3/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()

28 (pid) proc gdb bt

#0  0x00007f15c2c5e827 in do_futex_wait.constprop () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007f15c2c5e8d4 in __new_sem_wait_slow.constprop.0 () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00007f15c2c5e97a in sem_wait@@GLIBC_2.2.5 () from /lib/x86_64-linux-gnu/libpthread.so.0
#3  0x00007f15a98058b1 in semlock_acquire () at /usr/local/src/conda/python-3.7.7/Modules/_multiprocessing/semaphore.c:319
#4  0x000055f54a7f1c94 in _PyMethodDef_RawFastCallKeywords () at /tmp/build/80754af9/python_1588882889832/work/Objects/call.c:694
#5  0x000055f54a7f8aef in _PyMethodDescr_FastCallKeywords () at /tmp/build/80754af9/python_1588882889832/work/Objects/descrobject.c:288

proc list

ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
ma-user      1     0  0 Jan05 ?        00:00:00 /bin/bash /home/ma-user/sidecar.sh
ma-user     16     1  0 Jan05 ?        00:00:00 tail -n 0 -F /home/ma-user/ma-test/log/-0---ma-platform-sidecar.log
ma-user     18     1  0 Jan05 ?        00:00:00 python /home/ma-user/runtime-scripts-v2/init-container/outputs-handler.py
ma-user     28    18  0 Jan05 ?        00:00:00 python /home/ma-user/runtime-scripts-v2/init-container/outputs-handler.py
ma-user    102     0  0 Jan05 pts/0    00:00:00 bash
ma-user    911     0  0 14:56 pts/1    00:00:00 bash
ma-user    931   911  0 14:57 pts/1    00:00:00 ps -ef
ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ 

28 fd

ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ ls -alh /proc/28/fd
total 0
dr-x------ 2 ma-user ma-group  0 Jan  6 20:23 .
dr-xr-xr-x 9 ma-user ma-group  0 Jan  5 17:27 ..
lr-x------ 1 ma-user ma-group 64 Jan  6 20:23 0 -> /dev/null
l-wx------ 1 ma-user ma-group 64 Jan  6 20:23 1 -> /home/ma-user/ma-test/log/-0---ma-platform-sidecar.log
l-wx------ 1 ma-user ma-group 64 Jan  6 20:23 2 -> /home/ma-user/ma-test/log/-0---ma-platform-sidecar.log
lr-x------ 1 ma-user ma-group 64 Jan  6 20:23 3 -> /dev/urandom
lr-x------ 1 ma-user ma-group 64 Jan  6 20:23 4 -> pipe:[952852668]
lrwx------ 1 ma-user ma-group 64 Jan  6 20:23 5 -> socket:[952857499]
lrwx------ 1 ma-user ma-group 64 Jan  6 20:23 6 -> /home/ma-user/conf/scc_logger.conf
l-wx------ 1 ma-user ma-group 64 Jan  6 20:23 7 -> pipe:[952852671]
lr-x------ 1 ma-user ma-group 64 Jan  6 20:23 8 -> /dev/null
l-wx------ 1 ma-user ma-group 64 Jan  6 20:23 9 -> pipe:[952852675]

18 fd

ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ ls -alh /proc/18/fd
total 0
dr-x------ 2 ma-user ma-group  0 Jan  6 20:26 .
dr-xr-xr-x 9 ma-user ma-group  0 Jan  5 03:07 ..
lr-x------ 1 ma-user ma-group 64 Jan  6 20:26 0 -> /dev/null
l-wx------ 1 ma-user ma-group 64 Jan  6 20:26 1 -> /home/ma-user/ma-test/log/-0---ma-platform-sidecar.log
l-wx------ 1 ma-user ma-group 64 Jan  6 20:26 2 -> /home/ma-user/ma-test/log/-0---ma-platform-sidecar.log
lr-x------ 1 ma-user ma-group 64 Jan  6 20:26 3 -> /dev/urandom
lr-x------ 1 ma-user ma-group 64 Jan  6 20:26 4 -> pipe:[952852668]
l-wx------ 1 ma-user ma-group 64 Jan  6 20:26 5 -> pipe:[952852668]
lr-x------ 1 ma-user ma-group 64 Jan  6 20:26 6 -> pipe:[952852671]
l-wx------ 1 ma-user ma-group 64 Jan  6 20:26 7 -> pipe:[952852671]
lr-x------ 1 ma-user ma-group 64 Jan  6 20:26 8 -> pipe:[952852675]

SIGTERM / SIGINT not work

ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ ps -ef 
UID        PID  PPID  C STIME TTY          TIME CMD
ma-user      1     0  0 Jan05 ?        00:00:00 /bin/bash /home/ma-user/sidecar.sh
ma-user     16     1  0 Jan05 ?        00:00:00 tail -n 0 -F /home/ma-user/ma-test/log/-0---ma-platform-sidecar.log
ma-user     18     1  0 Jan05 ?        00:00:00 python /home/ma-user/runtime-scripts-v2/init-container/outputs-handler.py
ma-user     28    18  0 Jan05 ?        00:00:00 python /home/ma-user/runtime-scripts-v2/init-container/outputs-handler.py
ma-user    102     0  0 Jan05 pts/0    00:00:00 bash
ma-user   1166     0  0 Jan06 pts/1    00:00:00 bash
ma-user   1216  1166  0 10:51 pts/1    00:00:00 ps -ef
ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ kill -SIGTERM 28
ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ 
ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ kill -SIGINT 28
ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ 
ma-user at ma-test-eval-dac957b0-b43b-43e2-ab19-0b45672a7ea0-worker-0:/$ 

finally, pool hang

----------
components: Library (Lib)
messages: 384560
nosy: huangzhesi
priority: normal
severity: normal
status: open
title: pool worker can't be terminated
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42849>
_______________________________________

From report at bugs.python.org  Thu Jan  7 01:13:56 2021
From: report at bugs.python.org (Robin Scheibler)
Date: Thu, 07 Jan 2021 06:13:56 +0000
Subject: [New-bugs-announce] [issue42850] Process hangs when calling
 urllib.request in a multiprocessing.Process with import of sounddevice
 package
Message-ID: <1610000036.98.0.246954881352.issue42850@roundup.psfhosted.org>


New submission from Robin Scheibler <fakufaku at gmail.com>:

I am having an issue with using urllib in a multiprocessing.Process when the package sounddevice (https://github.com/spatialaudio/python-sounddevice) has been already imported.

The sub-process hangs upon calling urllib.request.urlopen (and methods from the requests package too).

1. If the package sounddevice is not imported, things work as expected.
2. If the package sounddevice is imported, but urllib.request.urlopen is called once prior to launching thee sub-process, then things also work as expected.

I have attached a file containing code to reproduce the issue.

I have run this example file in:
- Python 3.7.9 (conda distribution)
- Mac OS X 10.15.7

# Running the code as is reproduces the problem
> python ./test_mp_req.py
Nothing in the queue. Leave.
wait for process to terminate...
finished

# Running the code with either line 8 commented *or* line 32 uncommented
# gives the expected output
> python ./test_mp_req.py
Received https://api.github.com. Send GET request... done.
Received https://www.google.com. Send GET request... done.
https://api.github.com: ok
https://www.google.com: ok
Nothing in the queue. Leave.
wait for process to terminate...
finished

----------
components: Interpreter Core, Library (Lib), ctypes
files: test_mp_req.py
messages: 384562
nosy: fakufaku
priority: normal
severity: normal
status: open
title: Process hangs when calling urllib.request in a multiprocessing.Process with import of sounddevice package
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file49724/test_mp_req.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42850>
_______________________________________

From report at bugs.python.org  Thu Jan  7 01:39:43 2021
From: report at bugs.python.org (Mariusz Felisiak)
Date: Thu, 07 Jan 2021 06:39:43 +0000
Subject: [New-bugs-announce] [issue42851] Subclassing Enum with
 ipaddress.IPv4Network/IPv6Network raises TypeError.
Message-ID: <1610001583.18.0.772940964765.issue42851@roundup.psfhosted.org>


New submission from Mariusz Felisiak <felisiak.mariusz at gmail.com>:

We noticed a behavior change in Python3.10.0a4 (it works properly in Python3.10.0a3) when running Django's test suite[1]. Subclassing `enum.Enum` with `ipaddress.IPv4Network/IPv6Network` raises `TypeError`, e.g.

>>> import enum
>>> import ipaddress
>>> class NewEnum(ipaddress.IPv4Address, enum.Enum):
...     pass
... 
>>> class NewEnum(ipaddress.IPv4Network, enum.Enum):
...     pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/enum.py", line 239, in __new__
    enum_class.__bases__ = enum_class.__bases__[1:] #or (object, )
  File "/usr/local/lib/python3.10/enum.py", line 483, in __setattr__
    super().__setattr__(name, value)
TypeError: __bases__ assignment: 'IPv4Network' object layout differs from '_NoInitSubclass'

It still works for `ipaddress.IPv4Address/IPv6Address`.

[1] https://github.com/django/django/blob/102d92fc09849e1a9004dd3f9a14a0ea9ca392cd/tests/model_enums/tests.py#L225-L235

----------
components: Library (Lib)
messages: 384563
nosy: barry, eli.bendersky, ethan.furman, felixxm
priority: normal
severity: normal
status: open
title: Subclassing Enum with ipaddress.IPv4Network/IPv6Network raises TypeError.
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42851>
_______________________________________

From report at bugs.python.org  Thu Jan  7 01:47:34 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Thu, 07 Jan 2021 06:47:34 +0000
Subject: [New-bugs-announce] [issue42852] pprint fails in transformming
 non-breaking space
Message-ID: <1610002054.28.0.212080312966.issue42852@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

"pprint" can transform unicode like "print". However, pprint fails to transform non-breaking space('\240') . See the following example:

Python 3.10.0a2 (default, Nov 24 2020, 14:18:46) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

Example 1(Results as expected):
---------------------------------------------------------------------
>>> import pprint
>>> print(u'\041 hello')
! hello
>>> pprint.pprint(u'\041 hello')
'! hello'

Example 2(Results not as expected):
---------------------------------------------------------------------
>>> print(u'\240 hello')
  hello
>>> pprint.pprint(u'\240 hello')
'\xa0 hello'

Expected output: the output of pprint.pprint(u'\240 hello') should be consistent with output of print(u'\240 hello')

----------
components: Unicode
messages: 384564
nosy: ezio.melotti, vstinner, xxm
priority: normal
severity: normal
status: open
title: pprint fails in transformming non-breaking space
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42852>
_______________________________________

From report at bugs.python.org  Thu Jan  7 03:03:09 2021
From: report at bugs.python.org (Andrew MacDonald)
Date: Thu, 07 Jan 2021 08:03:09 +0000
Subject: [New-bugs-announce] [issue42853] `OverflowError: signed integer is
 greater than maximum` in ssl.py for files larger than 2GB
Message-ID: <1610006589.59.0.311109371799.issue42853@roundup.psfhosted.org>


New submission from Andrew MacDonald <andrew at maccas.net>:

When attempting to read a large file (> 2GB) over HTTPS the read fails with "OverflowError: signed integer is greater than maximum".

This occurs with Python >=3.8 and I've been able to reproduce the problem with the below snippet of code on Linux, Mac OS X, and Windows (the remote file can be any HTTPS hosted file larger than 2GB, e.g. an empty file generated with `dd if=/dev/zero of=2g.img bs=1 count=0 seek=2G` will also do the job.).

```
import http.client
connection = http.client.HTTPSConnection("mirror.aarnet.edu.au")
connection.request("GET", "/pub/centos/8/isos/x86_64/CentOS-8.3.2011-x86_64-dvd1.iso")
response = connection.getresponse()
data = response.read()
```

Doing a git bisect it looks like this is the result of a change in commit d6bf6f2d0c83f0c64ce86e7b9340278627798090 (https://github.com/python/cpython/commit/d6bf6f2d0c83f0c64ce86e7b9340278627798090). Looking over the associated issue and commit message it seems like this was not an intended outcome for the change.

----------
assignee: christian.heimes
components: SSL
messages: 384565
nosy: amacd31, christian.heimes
priority: normal
severity: normal
status: open
title: `OverflowError: signed integer is greater than maximum` in ssl.py for files larger than 2GB
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42853>
_______________________________________

From report at bugs.python.org  Thu Jan  7 03:23:09 2021
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 07 Jan 2021 08:23:09 +0000
Subject: [New-bugs-announce] [issue42854] OpenSSL 1.1.1: use SSL_write_ex()
 and SSL_read_ex()
Message-ID: <1610007789.81.0.109119647565.issue42854@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

SSL_read() and SSL_write() are limited to int. The new SSL_write_ex() and SSL_read_ex() APIs support size_t just like read(2) and recv(2). Also see bpo-42853.

int SSL_write_ex(SSL *s, const void *buf, size_t num, size_t *written);
int SSL_read_ex(SSL *ssl, void *buf, size_t num, size_t *readbytes);

Both functions return 1 for success or 0 for failure.

----------
assignee: christian.heimes
components: SSL
messages: 384567
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: OpenSSL 1.1.1: use SSL_write_ex() and SSL_read_ex()
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42854>
_______________________________________

From report at bugs.python.org  Thu Jan  7 04:23:21 2021
From: report at bugs.python.org (gaborjbernat)
Date: Thu, 07 Jan 2021 09:23:21 +0000
Subject: [New-bugs-announce] [issue42855] pathlib.exists on Windows raises
 an exception on URL like/bad input
Message-ID: <1610011401.69.0.814272139734.issue42855@roundup.psfhosted.org>


New submission from gaborjbernat <gaborjbernat at gmail.com>:

? py -c "from pathlib import Path; Path('http://w.org').exists()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python39\lib\pathlib.py", line 1407, in exists
    self.stat()
  File "C:\Python39\lib\pathlib.py", line 1221, in stat
    return self._accessor.stat(self)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'http:\\w.org'

The above code returns correctly False on UNIX systems.

----------
components: Windows
messages: 384569
nosy: gaborjbernat, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: pathlib.exists on Windows raises an exception on URL like/bad input
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42855>
_______________________________________

From report at bugs.python.org  Thu Jan  7 06:21:06 2021
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 07 Jan 2021 11:21:06 +0000
Subject: [New-bugs-announce] [issue42856] ensurepip: add configure
 --with-wheel-pkg-dir=PATH to get wheel packages from a system directory
Message-ID: <1610018466.49.0.695886938784.issue42856@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

The Fedora packaging policy recommends to "unbundle" bundled dependencies.

"Fedora packages SHOULD make every effort to avoid having multiple, separate, upstream projects bundled together in a single package."
https://docs.fedoraproject.org/en-US/packaging-guidelines/#bundling

The main motivation is to ease updates when fix serious vulnerabilities (less packaging work).

The ensurepip package contains bundled wheel packages of setuptools and pip:

$ ls Lib/ensurepip/_bundled/
pip-20.2.3-py2.py3-none-any.whl
setuptools-47.1.0-py3-none-any.whl

The Fedora python3 package doesn't contain the ensurepip._bundled package:

$ python3
Python 3.9.1 (default, Dec  8 2020, 00:00:00) 
>>> import ensurepip._bundled
ModuleNotFoundError: No module named 'ensurepip._bundled'

Instead, a separated RPM package python-pip-wheel provides wheel packages in /usr/share/python-wheels/ directory:

$ ls /usr/share/python-wheels/
pip-20.2.2-py2.py3-none-any.whl*
setuptools-49.1.3-py3-none-any.whl*
wheel-0.34.2-py2.py3-none-any.whl*

Fedora has a downstream patch on ensurepip (written by Miro Hron?ok) to always use /usr/share/python-wheels/:

https://src.fedoraproject.org/rpms/python3.10/blob/master/f/00189-use-rpm-wheels.patch

Fedora has packages of 9 CPython versions: 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10.

https://developer.fedoraproject.org/tech/languages/python/multiple-pythons.html

Having a separated package for wheel packages allows us to upgrade a single package (python-pip-wheel) for setuptools/pip bugfix or security vulnerability.

--

I propose to add a new --with-wheel-pkg-dir=PATH option to the ./configure script. If used, ensurepip will only use wheel packages from this directory. Otherwise, the existing code is unchanged. In short, the behavior is unchanged, unless the option is used explicitly.

If a directory is specified but wheel packages are missing, ensurepip fails.

If the directory contains multiple wheel packages of different versions, the most recent version is used of each package.

Note: In practice, the Fedora package only provides a single wheel package of each Python module. But I propose to make the Python upstream code as generic as possible.

I'm working on a pull request to implement this.

Downstream Fedora issue: https://bugzilla.redhat.com/show_bug.cgi?id=1874803

--

Fedora (Miro) already contributed to ensurepip to make ensurepip less dependent on pip internals:

commit 88f82b2b9ea3514359cb6e3218121f75334063ac
Author: Miro Hron?ok <miro at hroncok.cz>
Date:   Tue Mar 10 22:16:28 2020 +0100

    bpo-38662: ensurepip invokes pip via runpy (GH-18901)
    
    The ensurepip module now invokes pip via the runpy module.
    Hence it is no longer tightly coupled with the internal API of the bundled
    pip version, allowing easier updates to a newer pip version both
    internally and for distributors.
    
    This way, any changes to the internal pip API won't mean ensurepip needs to be
    changed as well. Also, distributors can update their pip wheels independent on
    CPython release schedule.
    
    Co-Authored-By: Pradyun Gedam <pradyunsg at gmail.com>
    Co-Authored-By: Miro Hron?ok <miro at hroncok.cz>

This change was already related to Fedora downstream change to get wheel packages from a different directory. Fedora can use a different pip version (older or more recent) than ensurepip._bundled.

----------
components: Library (Lib)
messages: 384577
nosy: vstinner
priority: normal
severity: normal
status: open
title: ensurepip: add configure --with-wheel-pkg-dir=PATH to get wheel packages from a system directory
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42856>
_______________________________________

From report at bugs.python.org  Thu Jan  7 06:49:48 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Thu, 07 Jan 2021 11:49:48 +0000
Subject: [New-bugs-announce] [issue42857] Fails to construct paths lead to
 "python is likely shutting down"
Message-ID: <1610020188.09.0.60258603326.issue42857@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

The attached file "test_zipfile.py" is from test directory in cPython. We delete irrelevant code. Running this code on Python 3.10 will lead to the following error messages.
----------------------------------------------------------------------------------------
Exception ignored in: <generator object temp_dir at 0x7f13f30ff2e0>
Traceback (most recent call last):
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 401, in temp_dir
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 358, in rmtree
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 322, in _rmtree
ImportError: sys.meta_path is None, Python is likely shutting down
Exception ignored in: <generator object temp_dir at 0x7f13f30bd5f0>
Traceback (most recent call last):
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 401, in temp_dir
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 358, in rmtree
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 322, in _rmtree
ImportError: sys.meta_path is None, Python is likely shutting down
----------------------------------------------------------------------------------------
The expected output should be something like" ImportError: No module named XXX"  rather than "Python is likely shutting down"

----------
components: Interpreter Core
files: test_zipfile.py
messages: 384581
nosy: xxm
priority: normal
severity: normal
status: open
title: Fails to construct paths lead to "python is likely shutting down"
type: crash
versions: Python 3.10
Added file: https://bugs.python.org/file49725/test_zipfile.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42857>
_______________________________________

From report at bugs.python.org  Thu Jan  7 06:54:25 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Thu, 07 Jan 2021 11:54:25 +0000
Subject: [New-bugs-announce] [issue42858] Incorrect return value for
 os.system() in recognizing import error
Message-ID: <1610020465.58.0.34787573132.issue42858@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

Running attached "test_zipfile.py" on Python 3.10 will lead to the following error messages:
----------------------------------------------------------------------------------------------------
Exception ignored in: <generator object temp_dir at 0x7f13f30ff2e0>
Traceback (most recent call last):
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 401, in temp_dir
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 358, in rmtree
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 322, in _rmtree
ImportError: sys.meta_path is None, Python is likely shutting down
Exception ignored in: <generator object temp_dir at 0x7f13f30bd5f0>
Traceback (most recent call last):
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 401, in temp_dir
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 358, in rmtree
  File "/usr/local/python310/lib/python3.10/test/support/os_helper.py", line 322, in _rmtree
ImportError: sys.meta_path is None, Python is likely shutting down
----------------------------------------------------------------------------------------------------

However, if we run this program with os.system, the return value is 0 instead of 256. 0 represent no fails in this running! This is obviously incorrect.

Reproduce:
=========================================
import os
scode = os.system('python310  test_zipfile.py')
print("The system code of this execution is:", scode)
=========================================

The expected output: "The system code of this execution is: 256"
The actual output: "The system code of this execution is: 0"

Version info:
>>python310 -V
Python 3.10.0a2
>>uname -v
16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019

----------
components: Library (Lib)
files: test_zipfile.py
messages: 384582
nosy: xxm
priority: normal
severity: normal
status: open
title: Incorrect return value for os.system() in recognizing import error
versions: Python 3.10
Added file: https://bugs.python.org/file49726/test_zipfile.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42858>
_______________________________________

From report at bugs.python.org  Thu Jan  7 14:28:50 2021
From: report at bugs.python.org (Dmitry Goldenberg)
Date: Thu, 07 Jan 2021 19:28:50 +0000
Subject: [New-bugs-announce] [issue42859] smtplib: recipe for certifying
 message delivery
Message-ID: <1610047730.69.0.693075661816.issue42859@roundup.psfhosted.org>


New submission from Dmitry Goldenberg <dgoldenberg123 at gmail.com>:

Is there a reliable way to certify (ensure, track) the delivery of an email message when using smtplib?

For example:
Disposition-Notification-To message header which enables the e-mail reader of the recipient to send confirmation e-mail when the recipient opens the message for the first time. That e-mail reader program must support this functionality.

Return-Receipt-To message header which enables the e-mail server of the recipient to send confirmation e-mail when the original e-mail gets received by that server. However, most newer e-mail servers do not support Return-Receipt-To header.

Are these message headers at all reliable?

The request is to enhance the documentation to provide an example of how to track a message delivery. Conversely, if it's plain not possible, the request is to state that explicitly in the docs.

----------
assignee: docs at python
components: Documentation
messages: 384600
nosy: dgoldenberg123, docs at python
priority: normal
severity: normal
status: open
title: smtplib: recipe for certifying message delivery
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42859>
_______________________________________

From report at bugs.python.org  Thu Jan  7 16:09:27 2021
From: report at bugs.python.org (Tobias Kohn)
Date: Thu, 07 Jan 2021 21:09:27 +0000
Subject: [New-bugs-announce] [issue42860] Incompatible types in Python
 grammar
Message-ID: <1610053767.4.0.390149770714.issue42860@roundup.psfhosted.org>


New submission from Tobias Kohn <kohnt at tobiaskohn.ch>:

There seems to be a small type error in the Python grammar in the rule `invalid_parameters`:
```
invalid_parameters:
    | param_no_default* (slash_with_default | param_with_default+) param_no_default
```
While the `slash_with_default` returns a single element, the `param_with_default` returns a list/sequence.

----------
messages: 384603
nosy: tobias.kohn
priority: normal
severity: normal
status: open
title: Incompatible types in Python grammar
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42860>
_______________________________________

From report at bugs.python.org  Thu Jan  7 19:30:51 2021
From: report at bugs.python.org (Faisal Mahmood)
Date: Fri, 08 Jan 2021 00:30:51 +0000
Subject: [New-bugs-announce] [issue42861] ipaddress - add ability to get
 next closet subnet of any prefix size
Message-ID: <1610065851.06.0.929191666424.issue42861@roundup.psfhosted.org>


New submission from Faisal Mahmood <fasial.mahmood94 at gmail.com>:

The ipaddress module in Python is an excellent tool, but I noticed it is missing a feature that I needed several months ago, which is the ability to find the next closest subnet with a specific prefix length.

For example, imagine I had a IPv4Network("10.10.0.72/30"), how would I find the next possible /25 address? It is not the most straightforward thing to do, so think it would be a great enhancement to the ipaddress library.

I think this can be achieved by adding in a new method to the BaseNetwork class, the method could be defined like "next_prefix(next_prefix=None)".  Calling this method would return an IPv4/v6 address that is the closest possible match with the new prefix (defined as next_prefix).

Example calls:
v4 = IPv4Network("10.10.0.72/30")
next_network = v4.next_subnet(next_prefix=25)
# Output: next_network = IPv4Network("10.10.0.128/25")

v4 = IPv4Network("10.10.0.72/30")
next_network = v4.next_subnet(next_prefix=30)
# Output: next_network = IPv4Network("10.10.0.76/30")

v4 = IPv4Network("10.10.0.72/30")
next_network = v4.next_subnet() # if next_prefix is not defined it will use the existing prefix of /30, so this call is exactly the same as the previous
# Output: next_network = IPv4Network("10.10.0.76/30")

v6 = IPv6Network("2001:db8:aaaa:aaaa:aaaa:aaaa:aaaa:0000/112")
next_network = v6.next_subnet()
# Output: next_network = IPv6Network("2001:db8:aaaa:aaaa:aaaa:aaaa:aaab:0/112")

v6 = IPv6Network("2001:db8:aaaa:aaaa:aaaa:aaaa:aaaa:0000/112")
next_network = v6.next_subnet(next_prefix=64)
# Output: next_network = IPv6Network("2001:db8:aaaa:aaab::/64")

I am going to be working on this and plan to raise a PR soon.  This is my first time contributing to Python, so I appreciate your help / comments / suggestions / guidance as I go along.

----------
components: Library (Lib)
messages: 384623
nosy: fasial.mahmood94
priority: normal
severity: normal
status: open
title: ipaddress - add ability to get next closet subnet of any prefix size
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42861>
_______________________________________

From report at bugs.python.org  Thu Jan  7 19:39:03 2021
From: report at bugs.python.org (Erlend Egeberg Aasland)
Date: Fri, 08 Jan 2021 00:39:03 +0000
Subject: [New-bugs-announce] [issue42862] Use functools.lru_cache iso.
 _sqlite.Cache in sqlite3 module
Message-ID: <1610066343.94.0.466682345605.issue42862@roundup.psfhosted.org>


New submission from Erlend Egeberg Aasland <erlend.aasland at innova.no>:

Pro: less code, less complexity, improved maintainability
Con: minor performance hit

PoC here: https://github.com/erlend-aasland/cpython/commits/sqlite-cache

$ ./python.exe
>>> import sqlite3
>>> con = sqlite3.connect(":memory:")
>>> con.execute("select * from sqlite_master")
>>> con.execute("select * from sqlite_master")
>>> c = con.cache()
>>> c.cache_info()
CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)

The test suite runs approx. 10-20 ms slower with this change. Using _functools._lru_cache_wrapper iso. functools.lru_cache almost removes this performance regression.


Berker, is it worth pursuing?

----------
components: Library (Lib)
messages: 384625
nosy: berker.peksag, erlendaasland, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Use functools.lru_cache iso. _sqlite.Cache in sqlite3 module
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42862>
_______________________________________

From report at bugs.python.org  Thu Jan  7 20:10:38 2021
From: report at bugs.python.org (Jeff Moguillansky)
Date: Fri, 08 Jan 2021 01:10:38 +0000
Subject: [New-bugs-announce] [issue42863] Python venv inconsistent folder
 structure on windows
Message-ID: <1610068238.28.0.631501714908.issue42863@roundup.psfhosted.org>


New submission from Jeff Moguillansky <jmoguill2 at gmail.com>:

When creating a virtual environment on windows using venv, the folder structure: "Scripts", "Include", "Lib", is inconsistent with other platforms (e.g. "include", "lib", "bin", etc).  

This causes various integration issues.  

For example, suppose we want to build a native C library, and install it to the folder structure generated by the virtual environment.  
The pkg-config file assumes a folder structure of "include", "lib", "bin", and the generated pkg-config files are inconsistent with the python virtual environment folder structure.  

Can we have a consistent folder structure across platforms?

----------
components: Windows
messages: 384628
nosy: jmoguill2, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python venv inconsistent folder structure on windows
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42863>
_______________________________________

From report at bugs.python.org  Thu Jan  7 22:03:30 2021
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Fri, 08 Jan 2021 03:03:30 +0000
Subject: [New-bugs-announce] [issue42864] Improve error messages regarding
 unclosed parentheses
Message-ID: <1610075010.91.0.684383999677.issue42864@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

Consider this file:

```
x = (


```

The error that we get is:

? python ../a.py
  File "/home/pablogsal/github/python/master/../a.py", line 6

    ^
SyntaxError: unexpected EOF while parsin

This is quite uninformative of the actual problem, which is the closed parentheses.

The same happens with something like this:

( 1+2

  File "/home/pablogsal/github/python/master/lel.py", line 3

    ^
SyntaxError: unexpected EOF while parsing

With some effort, we can include the location of the unclosed parentheses.

----------
messages: 384630
nosy: pablogsal
priority: normal
severity: normal
status: open
title: Improve error messages regarding unclosed parentheses
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42864>
_______________________________________

From report at bugs.python.org  Thu Jan  7 23:00:38 2021
From: report at bugs.python.org (Victor Lopez)
Date: Fri, 08 Jan 2021 04:00:38 +0000
Subject: [New-bugs-announce] [issue42865] sysconfig appends CFLAGS to LD
Message-ID: <1610078438.84.0.384790848261.issue42865@roundup.psfhosted.org>


New submission from Victor Lopez <development at greentwip.xyz>:

This should not happen as some CFLAGS are exclusive to the compiler and it will break cross-compiling linkers

The proposed sysconfig is attached

----------
components: Distutils
files: sysconfig.py
messages: 384632
nosy: Greentwip, dstufft, eric.araujo
priority: normal
pull_requests: 22989
severity: normal
status: open
title: sysconfig appends CFLAGS to LD
versions: Python 3.9
Added file: https://bugs.python.org/file49728/sysconfig.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42865>
_______________________________________

From report at bugs.python.org  Fri Jan  8 04:28:39 2021
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 08 Jan 2021 09:28:39 +0000
Subject: [New-bugs-announce] [issue42866] test test_multibytecodec:
 Test_IncrementalEncoder.test_subinterp() leaks references
Message-ID: <1610098119.51.0.27659926103.issue42866@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

$ ./python -m test test_multibytecodec -m test.test_multibytecodec.Test_IncrementalEncoder.test_subinterp -R 3:3
(...)
test_multibytecodec leaked [258, 258, 258] references, sum=774

I simplified the code. The following test leaks references:

    def test_subinterp(self):
        import _testcapi
        code = textwrap.dedent("""
            import _codecs_jp
            codec = _codecs_jp.getcodec('cp932')
            codec = None
        """)
        _testcapi.run_in_subinterp(code)

_codecs_jp.getcodec() is defined in Modules/cjkcodecs/cjkcodecs.h. Extract:

    cofunc = getmultibytecodec();
    ...
    codecobj = PyCapsule_New((void *)codec, PyMultibyteCodec_CAPSULE_NAME, NULL);
    if (codecobj == NULL)
        return NULL;

    r = PyObject_CallOneArg(cofunc, codecobj);
    Py_DECREF(codecobj);

getmultibytecodec() is the _multibytecodec.__create_codec() which is defined in Modules/cjkcodecs/multibytecodec.c. Simplified code:

    codec = PyCapsule_GetPointer(arg, PyMultibyteCodec_CAPSULE_NAME);
    _multibytecodec_state *state = _multibytecodec_get_state(module);
    self = PyObject_New(MultibyteCodecObject, state->multibytecodec_type);
    self->codec = codec;
    return (PyObject *)self;

----------
components: Library (Lib)
messages: 384643
nosy: vstinner
priority: normal
severity: normal
status: open
title: test test_multibytecodec: Test_IncrementalEncoder.test_subinterp() leaks references
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42866>
_______________________________________

From report at bugs.python.org  Fri Jan  8 07:46:13 2021
From: report at bugs.python.org (John McCabe)
Date: Fri, 08 Jan 2021 12:46:13 +0000
Subject: [New-bugs-announce] [issue42867] Entry Widget not editable on
 Windows 10, but is on Linux Ubuntu 16.04
Message-ID: <1610109973.36.0.703695549775.issue42867@roundup.psfhosted.org>


New submission from John McCabe <john at mccabe.org.uk>:

I've built an application using tkinter (see below). I'm fairly new to tkinter, despite having substantial software experience (33 years), so the layout might be a bit odd/wrong, and the example obviously doesn't follow PEP-8 guidelines, however...

Basically this application (which is a bit more than minimal, but should be easy to follow) is designed to manage pairs of text values as JSON, saving/loading from a file. When it starts, the user is asked if they want to read in an existing file of data; if they select NO, a fairly empty frame with 3 buttons showing "+", "Save" and "Quit" is displayed.

At this point, if "+" is pressed, a new row with two labels and two Entry widgets is added to the frame.

However (and this is the problem which appears to be identical to that reported in Issue #9673), it is not possible to set focus into either of the Entry widgets on Windows 10; there is no problem doing this on Ubuntu 16.04 (although I've only got Python 3.5 on there)

If the "Save" button is then pressed, a message box pops up telling the use that no changes have been saved. Once OK has been pressed on that, it becomes possible to set focus into the Entry widgets.

One of the problems with Issue #9673 was that no 'minimal' example was provided showing this behaviour, and the very minimal example given in https://bugs.python.org/issue9673#msg218765 doesn't exhibit the problem, hence the example below being a bit more than minimal, while still not being particularly complicated.

====
#!/usr/bin/python3

import os
import tkinter as tk
from tkinter import filedialog, messagebox
import json

class Application(tk.Frame):

    def __init__(self, master):
        super().__init__(master)
        self.master = master
        self.grid()
        self.originalJson = {}
        self.inFileName = ""
        self.leftRightEntries = []
        self.fileDlgOpts = { "initialdir"       : os.getcwd(),
                             "initialfile"      : "file.json",
                             "filetypes"        : (("JSON File", "*.json"), ("All Files","*.*")),
                             "defaultextension" : '.json',
                             "title"            : "Select A File" }
        self.createWidgets()

    def openInFile(self):
        fileName = ""
        reuse = tk.messagebox.askquestion("Use An Existing File", "Do you want to load and use an existing file?")
        if reuse == "yes":
            fileName = tk.filedialog.askopenfilename(**self.fileDlgOpts)
            if fileName is not "":
                try:
                    with open(fileName, 'r') as json_file:
                        self.originalJson = json.load(json_file)
                        json_file.close()
                except Exception:
                    tk.messagebox.showerror("Use An Existing File", "File could not be loaded; continuing without one.")
                    fileName = ""
            else:
                tk.messagebox.showwarning("Use An Existing File", "No existing file specified; continuing without one.")

        return fileName

    def createWidgets(self):
        self.inFileName = self.openInFile()

        # We add the buttons to some huge numbered row because we might want to insert more
        # rows, and the layout manager will collapse everything in between. Also we
        # add these first because of the way the tab order is set up
        self.addBtn = tk.Button(self.master, text = "+", command = self.addNew)
        self.addBtn.grid(row = 100, column = 0, sticky = tk.W)

        # Save button; pretty self-explanatory
        self.saveBtn = tk.Button(self.master, text = "Save", command = self.save)
        self.saveBtn.grid(row = 100, column = 2, sticky = tk.W)

        # Quit button; pretty self-explanatory
        self.quitBtn = tk.Button(self.master, text = "QUIT", fg = "red", command = self.quit)
        self.quitBtn.grid(row = 100, column = 3, sticky = tk.E)

        # If there is original json, work through each key and put the fields on the display
        rowNum = 0
        for leftText in sorted(self.originalJson.keys()):
            self.insertRow(rowNum, leftText);
            rowNum = rowNum + 1

        self.nextEmptyRow = rowNum

        self.redoPadding()

    def redoPadding(self):
        for child in self.master.winfo_children():
            child.grid_configure(padx = 5, pady = 5)

    def focusNextWidget(self, event):
        event.widget.tk_focusNext().focus()
        return("break")

    def insertRow(self, rowNum, initialLeft = None):
        tk.Label(self.master, height = 1, text = "Left: ").grid(row = rowNum, column = 0, sticky = tk.W)
        leftBox = tk.Entry(self.master, width = 20)
        leftBox.grid(row = rowNum, column = 1, sticky = tk.W)
        leftBox.bind("<Tab>", self.focusNextWidget)
        if initialLeft is not None:
            leftBox.insert(tk.END, initialLeft)
        tk.Label(self.master, height = 1, text = "Right: ").grid(row = rowNum, column = 2, sticky = tk.W)
        rightBox = tk.Entry(self.master, width = 20)
        rightBox.grid(row = rowNum, column = 3, sticky = tk.W)
        rightBox.bind("<Tab>", self.focusNextWidget)
        if initialLeft is not None:
            rightBox.insert(tk.END, initialLeft)
        self.leftRightEntries.append((leftBox, rightBox))
        leftBox.focus_set()

    def addNew(self):
        # Add a new row before the button
        self.insertRow(self.nextEmptyRow)
        self.nextEmptyRow = self.nextEmptyRow + 1
        self.redoPadding()

    def getCurrent(self):
        # Work through the rows and check stuff
        current = {}
        for (leftEntry, rightEntry) in self.leftRightEntries:
            leftText = leftEntry.get()
            rightText = rightEntry.get()
            if leftText == "" and rightText == "":
                pass
            elif leftText == "":
                print("No leftText specified for rightText [{}]".format(rightText))
            elif rightText == "":
                print("No rightText specified for leftText [{}]".format(leftText))
            else:
                print("lefText: {}, rightText: {}".format(leftText, rightText))
                current[leftText] = rightText
        return current

    def save(self):
        # Get the current values, and then dump the new json to a file, if it's changed!
        finalResult = self.getCurrent()
        if finalResult != self.originalJson:
            if self.inFileName == "":
                self.inFileName = tk.filedialog.asksaveasfilename(**self.fileDlgOpts)

            if self.inFileName != "":
                with open(self.inFileName, 'w') as json_file:
                    json.dump(finalResult, json_file, indent = 4)
                    self.originalJson = finalResult
                tk.messagebox.showinfo("Save Data", "Data saved to {}".format(self.inFileName))
            else:
                tk.messagebox.showwarning("Save Data", "Data has not been saved; no file name was supplied!")
        else:
            tk.messagebox.showwarning("Save Data", "Data has not been saved; there are no changes")

    def quit(self):
        # Deal with quitting when the file's been modified, check original vs current JSON
        reallyQuit = True
        finalResult = self.getCurrent()
        if finalResult != self.originalJson:
            answer = tk.messagebox.askquestion("Quit", "Data has changed; do you really want to quit?", icon = "warning")
            if answer != "yes":
                reallyQuit = False

        if reallyQuit:
            self.master.destroy()

if __name__ == "__main__":
    root = tk.Tk()
    root.title("Inactive Entry Example")
    app = Application(root)
    root.protocol("WM_DELETE_WINDOW", app.quit)
    app.mainloop()

----------
components: Tkinter
messages: 384655
nosy: cosimo193
priority: normal
severity: normal
status: open
title: Entry Widget not editable on Windows 10, but is on Linux Ubuntu 16.04
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42867>
_______________________________________

From report at bugs.python.org  Fri Jan  8 09:46:35 2021
From: report at bugs.python.org (Jin-oh Kang)
Date: Fri, 08 Jan 2021 14:46:35 +0000
Subject: [New-bugs-announce] [issue42868] SpooledTemporaryFile.__iter__ is
 not transparent to rollover
Message-ID: <1610117195.08.0.928321664408.issue42868@roundup.psfhosted.org>


New submission from Jin-oh Kang <jinoh.kang.kr at gmail.com>:

In tempfile, SpooledTemporaryFile.__iter__ is defined as follows:

    # file protocol
    def __iter__(self):
        return self._file.__iter__()

A rollover would switch the underlying _file object from a BytesIO to a TemporaryFile, thereby leaving the original iterator stale.

This may be fixed by:

    def __iter__(self):
        while True:
            line = self._file.readline()
            if not line:
                break
            yield line

Or perhaps:

    def __iter__(self):
        while True:
            file = self._file
            for line in file:
                yield line
                if file is not self._file:
                    break
            else:
                break

----------
components: Library (Lib)
messages: 384674
nosy: jinoh.kang.kr
priority: normal
severity: normal
status: open
title: SpooledTemporaryFile.__iter__ is not transparent to rollover
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42868>
_______________________________________

From report at bugs.python.org  Fri Jan  8 16:55:55 2021
From: report at bugs.python.org (Julien Palard)
Date: Fri, 08 Jan 2021 21:55:55 +0000
Subject: [New-bugs-announce] [issue42869] pydoc does not append .html to
 documentation
Message-ID: <1610142955.15.0.664460414569.issue42869@roundup.psfhosted.org>


New submission from Julien Palard <julien+python at palard.fr>:

Running `python3 -m pydoc ensurepip` gives me:

https://docs.python.org/3.9/library/ensurepip
but it should be:

https://docs.python.org/3.9/library/ensurepip.html

Issue is in getdocloc function on the line:

    docloc = "%s/%s" % (docloc.rstrip("/"), object.__name__.lower())

But it previously worked as the nginx config for the doc server contains:

    # Emulate Apache's content-negotiation. Was a temporary measure,
    # but now people are using it like a feature.
    location ~ ^/((2|3)(\.[0-8])?|dev)/\w+/[\d\w\.]+(?!\.html)$ {
        if (-f "${request_filename}.html") {
            return 301 https://$host:$request_uri.html;
        }
    }

(So yes "people are using it like a feature" contains pydoc :))

Notice the [0-8], which does not match for my /3.9/.

I propose to fix the issue on both sides:
- On psf-salt to allow 3.9 to get the "temporary" measure.
- pydoc side to simplify the code

----------
assignee: docs at python
components: Documentation
messages: 384693
nosy: docs at python, mdk
priority: normal
severity: normal
status: open
title: pydoc does not append .html to documentation
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42869>
_______________________________________

From report at bugs.python.org  Fri Jan  8 18:50:53 2021
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Fri, 08 Jan 2021 23:50:53 +0000
Subject: [New-bugs-announce] [issue42870] Document changed argparse output
 wrt optional arguments in What's new in Python 3.10
Message-ID: <1610149853.78.0.949568295376.issue42870@roundup.psfhosted.org>


New submission from Miro Hron?ok <miro at hroncok.cz>:

A followup from https://bugs.python.org/issue9694

Could the change in output please be mentioned on https://docs.python.org/3.10/whatsnew/3.10.html ?

In Fedora, at least 3 packages fail tests because of the change:

ipython: https://github.com/ipython/ipython/pull/12759
m2r: https://github.com/miyakogi/m2r/pull/62
sphinxcontrib-autoprogram: https://bugzilla.redhat.com/show_bug.cgi?id=1914341

Thanks.

----------
assignee: docs at python
components: Documentation
messages: 384699
nosy: docs at python, hroncok, paul.j3, rhettinger
priority: normal
severity: normal
status: open
title: Document changed argparse output wrt optional arguments in What's new in Python 3.10
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42870>
_______________________________________

From report at bugs.python.org  Fri Jan  8 20:25:30 2021
From: report at bugs.python.org (Renji)
Date: Sat, 09 Jan 2021 01:25:30 +0000
Subject: [New-bugs-announce] [issue42871] Regex compilation crashed if I
 change order of alternatives under quantifier
Message-ID: <1610155530.98.0.643940594728.issue42871@roundup.psfhosted.org>


New submission from Renji <AsoRenji at gmail.com>:

I can compile "((a)|b\2)*" expression and this expression successfully return captures from first repetition and second repetition in one time. But if I write (b\2|(a))* expression, I get "invalid group reference 2 at position 3" error. Either first or second behavior incorrect.
python3 --version Python 3.7.3

import re
text="aba"
#match=re.search(r"(b\2|(a))*",text) - not worked
match=re.search(r"((a)|b\2)*",text)
if(match):
    #show aba ba a
    print(match.group(0)+" "+match.group(1)+" "+match.group(2))

----------
components: Regular Expressions
messages: 384703
nosy: Renji, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Regex compilation crashed if I change order of alternatives under quantifier
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42871>
_______________________________________

From report at bugs.python.org  Sat Jan  9 03:15:26 2021
From: report at bugs.python.org (Hong Xu)
Date: Sat, 09 Jan 2021 08:15:26 +0000
Subject: [New-bugs-announce] [issue42872] Inconsistent exception thrown by
 mkdir on different OSes
Message-ID: <1610180126.52.0.315552258683.issue42872@roundup.psfhosted.org>


New submission from Hong Xu <hong at topbug.net>:

Consider the following code:

-------------------------

import pathlib

def main():
    pathlib.Path('tmp').touch()
    pathlib.Path('tmp/tmp_sub').mkdir(parents=True)

main()

------------------------

Run the code above in an empty directory.

On Linux, it throws a `NotADirectory` exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "main.py", line 5, in main
    pathlib.Path('tmp/tmp_sub').mkdir(parents=True)
  File "/usr/lib/python3.8/pathlib.py", line 1287, in mkdir
    self._accessor.mkdir(self, mode)
NotADirectoryError: [Errno 20] Not a directory: 'tmp/tmp_sub'

-----------------------------

On Windows, it throws a FileExistsError exception:

Traceback (most recent call last):
  File "C:\Users\hong\anaconda3\lib\pathlib.py", line 1284, in mkdir
    self._accessor.mkdir(self, mode)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'tmp\\tmp_sub'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    main()
  File "main.py", line 5, in main
    pathlib.Path('tmp/tmp_sub').mkdir(parents=True)
  File "C:\Users\hong\anaconda3\lib\pathlib.py", line 1288, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "C:\Users\hong\anaconda3\lib\pathlib.py", line 1284, in mkdir
    self._accessor.mkdir(self, mode)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'tmp

----------
components: Library (Lib), Windows
messages: 384715
nosy: Hong Xu, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Inconsistent exception thrown by mkdir on different OSes
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42872>
_______________________________________

From report at bugs.python.org  Sat Jan  9 07:00:18 2021
From: report at bugs.python.org (Mark Dickinson)
Date: Sat, 09 Jan 2021 12:00:18 +0000
Subject: [New-bugs-announce] [issue42873] Exponential time and space
 requirements for compilation of nested try/finally blocks
Message-ID: <1610193618.87.0.555969864656.issue42873@roundup.psfhosted.org>


New submission from Mark Dickinson <dickinsm at gmail.com>:

tl;dr - contrived (but relatively short) code involving nested try/finally blocks can produce disproportionately large bytecode. I'm not expecting or suggesting any action here, but the situation seemed at least worth noting. Feel free to close this issue as a "well don't do that, then" (a.k.a. "wont fix")

Longer: Python 3.9 changed the way that bytecode was generated for try/finally (see #33387). For a "try" block body that can do any of raise, return or fall-off-the-end-of-the-block, the corresponding finally block gets three separate paths in the bytecode. If such trys are nested <n> times, we get 3^n separate paths in the bytecode.

Example code:

----------------
def f():
    try:
        if something(): return
    finally:
        try:
            if something(): return
        finally:
            try:
                if something(): return
            finally:
                try:
                    if something(): return
                finally:
                    try:
                        if something(): return
                    finally:
                        do_cleanup()


import dis
dis.dis(f)
----------------

On my machine, running this and counting the do_cleanup invocations gives, as expected, a result of 243 = 3**5

% python3.9 nested_finally.py | grep do_cleanup | wc -l 
     243

That's fairly benign, but if I scale up to 10 nested blocks, the dis.dis call takes over 10 minutes to complete (the compilation itself still only takes a fraction of a second). The bytecode object is correspondingly large:

>>> len(f.__code__.co_code)
1741356

With 15 levels of nesting, compilation takes several seconds, and
the generated code is (again as expected) a couple of orders of magnitude larger:

>>> len(f.__code__.co_code)
533859040

I didn't try pushing this further than 15 levels of nesting.

As I said above, it's not clear to me whether this is actually an issue that needs to be addressed in practice. It seems unlikely that "real code" :TM: would run into this, but the effect seemed worth noting.

----------
components: Interpreter Core
messages: 384718
nosy: Mark.Shannon, mark.dickinson
priority: normal
severity: normal
status: open
title: Exponential time and space requirements for compilation of nested try/finally blocks
type: performance
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42873>
_______________________________________

From report at bugs.python.org  Sat Jan  9 07:14:13 2021
From: report at bugs.python.org (Martin Wheatley)
Date: Sat, 09 Jan 2021 12:14:13 +0000
Subject: [New-bugs-announce] [issue42874] configure errors
Message-ID: <1610194453.5.0.66216438534.issue42874@roundup.psfhosted.org>


New submission from Martin Wheatley <martin.wheatley.home at gmail.com>:

I'm installin Python 3.91. on a Solaris 10 system (I known it's 'old' but I have a legacy installation to support).

Running ./configure the following errors are seen...

checking PROFILE_TASK... -m test --pgo
checking for --with-lto... no
checking for llvm-profdata... no
grep: illegal option -- q
Usage: grep -hblcnsviw pattern file . . .
checking for -Wextra... yes
checking whether /usr/sfw/bin/gcc accepts and needs -fno-strict-aliasing... yes
checking if we can turn off /usr/sfw/bin/gcc unused result warning... no
checking if we can turn off /usr/sfw/bin/gcc unused parameter warning... yes


and


checking aligned memory access is required... yes
checking for --with-hash-algorithm... default
checking for --with-tzpath... grep: illegal option -- q
grep: illegal option -- E
Usage: grep -hblcnsviw pattern file . . .
"/usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo"
checking for --with-address-sanitizer... no
checking for --with-memory-sanitizer... no
checking for --with-undefined-behavior-sanitizer... no
checking for t_open in -lnsl... yes
checking for socket in -lsocket... yes

----------
components: Installation
messages: 384720
nosy: martin.wheatley.home
priority: normal
severity: normal
status: open
title: configure errors
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42874>
_______________________________________

From report at bugs.python.org  Sat Jan  9 09:34:59 2021
From: report at bugs.python.org (Pavel Ditenbir)
Date: Sat, 09 Jan 2021 14:34:59 +0000
Subject: [New-bugs-announce] [issue42875] argparse incorrectly shows help
 string on a new line in case of long command string
Message-ID: <1610202899.54.0.332342565241.issue42875@roundup.psfhosted.org>


New submission from Pavel Ditenbir <pavel.ditenbir at gmail.com>:

Steps to reproduce.

Run the attached script:
$ python3 argparse-indent-sample.py --help

The output is:
usage: argparse-indent-sample.py [-h] CMD ...

optional arguments:
  -h, --help           show this help message and exit

service:
  CMD                  command to use
    add                add something
    remove             remove something
    a-very-long-command
                       command that does something

Expected output is:
usage: argparse-indent-sample.py [-h] CMD ...

optional arguments:
  -h, --help           show this help message and exit

service:
  CMD                    command to use
    add                  add something
    remove               remove something
    a-very-long-command  command that does something

----------
components: Library (Lib)
files: argparse-indent-sample.py
messages: 384728
nosy: DiPaolo
priority: normal
severity: normal
status: open
title: argparse incorrectly shows help string on a new line in case of long command string
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49730/argparse-indent-sample.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42875>
_______________________________________

From report at bugs.python.org  Sat Jan  9 13:40:48 2021
From: report at bugs.python.org (Andreas Zeller)
Date: Sat, 09 Jan 2021 18:40:48 +0000
Subject: [New-bugs-announce] [issue42876] '''"""''' != '""""'
Message-ID: <1610217648.49.0.452941613308.issue42876@roundup.psfhosted.org>


New submission from Andreas Zeller <zeller at cispa.saarland>:

The following line of code fails in Python 3.6, which I find surprising.

>>> assert '''"""''' == '""""'  # Fails

Note that the following all pass as expected:

>>> assert """'''""" == "'''"  # Passes
>>> assert '''""''' == '""'  # Passes

----------
components: Interpreter Core
messages: 384736
nosy: andreas-zeller
priority: normal
severity: normal
status: open
title: '''"""''' != '""""'
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42876>
_______________________________________

From report at bugs.python.org  Sat Jan  9 14:11:00 2021
From: report at bugs.python.org (Irit Katriel)
Date: Sat, 09 Jan 2021 19:11:00 +0000
Subject: [New-bugs-announce] [issue42877] TracebackException saves more data
 than it needs
Message-ID: <1610219460.26.0.944321944788.issue42877@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at yahoo.com>:

TracebackException saves both __cause__ and __context__ even though format() ignores the __context__ if there is __cause__.

If we change the constructor to save only what format() needs, it will save space and simplify the code.

----------
components: Library (Lib)
messages: 384738
nosy: iritkatriel
priority: normal
severity: normal
status: open
title: TracebackException saves more data than it needs
type: performance
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42877>
_______________________________________

From report at bugs.python.org  Sat Jan  9 16:23:37 2021
From: report at bugs.python.org (Yair Frid)
Date: Sat, 09 Jan 2021 21:23:37 +0000
Subject: [New-bugs-announce] [issue42878]
 urllib.request.HTTPPasswordMgr.is_suburi does not care about order
Message-ID: <1610227417.16.0.538971054996.issue42878@roundup.psfhosted.org>


New submission from Yair Frid <fridyair at gmail.com>:

In docs, it mentions that the function returns true if the base is a 'suburi' of test, but in reality, if is_suburi(base, test) then is_suburi(test, base) which is wrong in most cases (other than where base ~= test, meaning they are the same location)
I am unsure which versions of python are affected, but i suspect most of them (although we probably shouldn't backport this fix?)

----------
components: Library (Lib)
messages: 384741
nosy: Fongeme
priority: normal
pull_requests: 23009
severity: normal
status: open
title: urllib.request.HTTPPasswordMgr.is_suburi does not care about order

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42878>
_______________________________________

From report at bugs.python.org  Sat Jan  9 17:50:07 2021
From: report at bugs.python.org (Jason Oliver)
Date: Sat, 09 Jan 2021 22:50:07 +0000
Subject: [New-bugs-announce] [issue42879] SystemError from class 'int'
Message-ID: <1610232607.93.0.155495951465.issue42879@roundup.psfhosted.org>


New submission from Jason Oliver <jollyoliver657 at gmail.com>:

This is my first bug report so I hope that this is correctly formatted. I am learning how to use pygame and encountered the following error:
SystemError: <class 'int'> returned a result with an error set

I googled the error message and came across the following answer on stackoverflow:
https://stackoverflow.com/a/53796516

This stated that the problem could exist within the implementation of python and that I should file a bug report here.

I am using the following software installed through pip/pip3:
Python 3.8.4
pygame 2.0.1
OS Edition: Windows 10 Pro
OS Version: 20H2
OS build: 19042.685

The bug can be reproduced deterministically.

----------
files: Movement.py
messages: 384743
nosy: jollyoliver657
priority: normal
severity: normal
status: open
title: SystemError from class 'int'
versions: Python 3.8
Added file: https://bugs.python.org/file49731/Movement.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42879>
_______________________________________

From report at bugs.python.org  Sun Jan 10 02:41:58 2021
From: report at bugs.python.org (Ziqiao Kong)
Date: Sun, 10 Jan 2021 07:41:58 +0000
Subject: [New-bugs-announce] [issue42880] ctypes: variadic function call
 still doesn't work on Apple Silicon
Message-ID: <1610264518.97.0.0227668442163.issue42880@roundup.psfhosted.org>


New submission from Ziqiao Kong <ziqiaokong at gmail.com>:

Hello! Thanks for your contribution on porting python to native Apple Silicon. I have noticed that there is an issue https://bugs.python.org/issue41100 and corresponding Github PR https://github.com/python/cpython/pull/22855 stating that variadic functions ABI has been corrected. However, during our test, it still doesn't work on Apple Silicon with brew-installed python 3.9.1 and python 3..10.0a4 built from sources. The details are as follows:

A x86_64 Mojave Macmini with python 3.8:
```
/tmp $ python3 --version
Python 3.8.6
/tmp $ uname -a
Darwin *** 18.7.0 Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT 2020; root:xnu-4903.278.35~1/RELEASE_X86_64 x86_64
/tmp $ cat main.c
#include <stdarg.h>
#include <stdio.h>
#include <stdint.h>
void test(uint64_t end, ...){
  int a;
  va_list valist;
  va_start(valist, end);
  a = va_arg(valist, int);
  va_end(valist);
  printf("%d\n", a);
  return;
}
/tmp $ cc -shared -g main.c -o ./main.dylib
/tmp $ cat test_main.py
from ctypes import *

d = CDLL("./main.dylib")
d.test.restype = None
d.test(c_uint64(12), c_int(34))
/tmp $ python3 test_main.py
34
/tmp $
```

An M1 Macbook Pro with brew-installed python 3.9.1
```
kabeor at kamino /tmp % python3 --version
Python 3.9.1
kabeor at kamino /tmp % cat main.c
#include <stdarg.h>
#include <stdio.h>
#include <stdint.h>
void test(uint64_t end, ...){
  int a;
  va_list valist;
  va_start(valist, end);
  a = va_arg(valist, int);
  va_end(valist);
  printf("%d\n", a);
  return;
}
kabeor at kamino /tmp % cc -shared -g main.c -o ./main.dylib
kabeor at kamino /tmp % cat test_main.py
from ctypes import *

d = CDLL("./main.dylib")
d.test.restype = None
d.test(c_uint64(12), c_int(34))
kabeor at kamino /tmp % python3 test_main.py
48144104
kabeor at kamino /tmp %
```

An M1 Macbook Pro with python 3.10.0a4 built from Github release tarball
```
kabeor at kamino cpython-3.10.0a4 % ./python.exe --version
Python 3.10.0a4
kabeor at kamino cpython-3.10.0a4 % cp /tmp/main.c ./
kabeor at kamino cpython-3.10.0a4 % cp /tmp/test_main.py ./
kabeor at kamino cpython-3.10.0a4 % ./python.exe --version
Python 3.10.0a4
kabeor at kamino cpython-3.10.0a4 % cat ./main.c
#include <stdarg.h>
#include <stdio.h>
#include <stdint.h>
void test(uint64_t end, ...){
  int a;
  va_list valist;
  va_start(valist, end);
  a = va_arg(valist, int);
  va_end(valist);
  printf("%d\n", a);
  return;
}
kabeor at kamino cpython-3.10.0a4 % cc -shared -g main.c -o ./main.dylib
kabeor at kamino cpython-3.10.0a4 % cat test_main.py
from ctypes import *

d = CDLL("./main.dylib")
d.test.restype = None
d.test(c_uint64(12), c_int(34))
kabeor at kamino cpython-3.10.0a4 % ./python.exe test_main.py
3
kabeor at kamino cpython-3.10.0a4 %
```

Thanks in advance!

----------
components: ctypes
messages: 384756
nosy: lawrence-danna-apple, lazymio
priority: normal
severity: normal
status: open
title: ctypes: variadic function call still doesn't work on Apple Silicon
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42880>
_______________________________________

From report at bugs.python.org  Sun Jan 10 06:29:52 2021
From: report at bugs.python.org (Larry Hastings)
Date: Sun, 10 Jan 2021 11:29:52 +0000
Subject: [New-bugs-announce] [issue42881] Should typing.get_type_hints
 change None annotations?
Message-ID: <1610278192.15.0.512262305909.issue42881@roundup.psfhosted.org>


New submission from Larry Hastings <larry at hastings.org>:

PEP 484 says:

    (Note that the return type of __init__ ought to be annotated
     with -> None. The reason for this is subtle. [...]

    https://www.python.org/dev/peps/pep-0484/#the-meaning-of-annotations

If you follow this advice, then call typing.get_type_hints() on your __init__ function, you'll find it has turned "None" into "type(None)".

git blame suggests get_type_hints' behavior was in the initial checkin of typing.py (46dbb7d1032c19163f37785509b8f5b3004416e8).  So it's always behaved this way.

Is "None" still considered the correct return annotation of an __init__?  If so, should typing.get_type_hints() really be changing it to type(None)?

----------
components: Library (Lib)
messages: 384760
nosy: eric.smith, gvanrossum, larry
priority: low
severity: normal
stage: test needed
status: open
title: Should typing.get_type_hints change None annotations?
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42881>
_______________________________________

From report at bugs.python.org  Sun Jan 10 08:00:18 2021
From: report at bugs.python.org (Yannick Jadoul)
Date: Sun, 10 Jan 2021 13:00:18 +0000
Subject: [New-bugs-announce] [issue42882] Restarting the interpreter causes
 UB on 3.10.0a4
Message-ID: <1610283618.64.0.475556781159.issue42882@roundup.psfhosted.org>


New submission from Yannick Jadoul <yannick.jadoul at gmail.com>:

Issue detected in the embedding tests of pybind11, running on the latest alpha of 3.10: https://github.com/pybind/pybind11/issues/2774

I have reduced the weird issue/crash to a minimal reproducer, which consistently reproduces the crash on my Linux machine:

```
#include <Python.h>

int main() {
    Py_InitializeEx(1);

    Py_Finalize();
    Py_InitializeEx(1);

    PyRun_SimpleString("class Widget: pass\n"
                       "class DerivedWidget(Widget):\n"
                       "    def __init__(self):\n"
                       "        super().__init__()\n");

    Py_Finalize();

    printf("Works\n");

    return 0;
}
```

Removing the two lines in the middle that restart the interpreter makes the example work.

I've also bisected CPython to find the issue (3.10.0a3 is fine, 3.10.0a4 is not), and arrived at https://github.com/python/cpython/pull/20058 (ba3d67c2fb04a7842741b1b6da5d67f22c579f33 being the first commit that breaks the example above). But I am not entirely sure where to start debugging.

The reproducing example above consistently crashes on my local machine (SIGABRT, exit code 134):

```
Fatal Python error: compiler_make_closure: lookup '__class__' in DerivedWidget 5 -1
freevars of __init__: ('__class__',)

Python runtime state: initialized

Current thread 0x00007f036485b680 (most recent call first):
<no Python frame>
Aborted (core dumped)
```

But note that in the pybind11 tests, the underlying issue causes a different error (Python throwing a weird, seemingly unrelated exception). So something seems to be messed up in the interpreter internals, and the above example just triggers it.

----------
components: Interpreter Core
messages: 384761
nosy: YannickJadoul
priority: normal
severity: normal
status: open
title: Restarting the interpreter causes UB on 3.10.0a4
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42882>
_______________________________________

From report at bugs.python.org  Sun Jan 10 09:43:44 2021
From: report at bugs.python.org (Haoyang)
Date: Sun, 10 Jan 2021 14:43:44 +0000
Subject: [New-bugs-announce] [issue42883] __init__ function may incur an
 incorrect behavior if passing a list as a parameter and set its default
 value as empty
Message-ID: <1610289824.57.0.574510344597.issue42883@roundup.psfhosted.org>


New submission from Haoyang <mahaoyang7777 at 163.com>:

Here is the bug-triggered code snippet in the file uploaded

class A:
    def __init__(self, b=[]):
        print('b = ', b)
        self.a = b 

for i in range(3):
    a = A()
    a.a.append(1)

    print(a.a)

It seems that when I pass a list "b" to __init__ function with default value empty list. Every time I create a new instance of this class and append one new variable to "self.a", the default value of "b" changed at the next time I create another instance of class A.

The outcome of this code snippet is 
a =  []
[1]
a =  [1]
[1, 1]
a =  [1, 1]
[1, 1, 1]

I am new to python. Is it a legal behavior in python? If yes, what is the principle beneath it? Thanks in advance!

----------
files: test.py
messages: 384765
nosy: haoyang9804
priority: normal
severity: normal
status: open
title: __init__  function may incur an incorrect behavior if passing a list as a parameter and set its default value as empty
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49732/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42883>
_______________________________________

From report at bugs.python.org  Sun Jan 10 14:51:50 2021
From: report at bugs.python.org (Daniel Nicorici)
Date: Sun, 10 Jan 2021 19:51:50 +0000
Subject: [New-bugs-announce] [issue42884] array.index() missing start and end
Message-ID: <1610308310.47.0.185082010282.issue42884@roundup.psfhosted.org>


New submission from Daniel Nicorici <daniel.nicorici at gmail.com>:

Add support for array.index(x [,start [,end]]). Adding start and end would allow to loop thru the entire array (and not start all the time from 0).


Currently, is array.index(x) (and it returns the smallest i such that i is the index of the first occurrence of x in the array). 


For example, bytearray.index() has start and end and list.index() has also start and end, but for some reason start and end are missing from array.index.

----------
messages: 384775
nosy: daniel.nicorici
priority: normal
severity: normal
status: open
title: array.index() missing start and end
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42884>
_______________________________________

From report at bugs.python.org  Sun Jan 10 16:58:21 2021
From: report at bugs.python.org (Arnim Rupp)
Date: Sun, 10 Jan 2021 21:58:21 +0000
Subject: [New-bugs-announce] [issue42885] Regex performance problem with ^
 aka AT_BEGINNING
Message-ID: <1610315901.41.0.467379667553.issue42885@roundup.psfhosted.org>


New submission from Arnim Rupp <erich at rupp.de>:

The re lib needs 7 seconds to check if a billion As start with an x. So e.g. this statement takes this long:

re.search(r'^x', 'A' * 1000000000)

It takes longer, the longer the string is. The string handling is not the problem, checking if it starts which an A takes just 0.00014 seconds. See output and code below:

3.10.0a4+ (heads/master:d16f617, Jan  9 2021, 13:24:45) 
[GCC 7.5.0]
testing string len:  100000
re_test_false:  0.0008246829966083169
testing string len:  1000000000
re_test_false:  7.317708015005337
testing string len:  1000000000
re_test_true:   0.00014710200048284605


import re, timeit, functools, sys

def re_test_true(string):
    print("testing string len: ", len(string))
    re.search(r'^A', string)

def re_test_false(string):
    print("testing string len: ", len(string))
    re.search(r'^x', string)

print(sys.version)

huge_string = 'A' * 100000
print('re_test_false: ', timeit.timeit(functools.partial(re_test_false, huge_string), number=1))

huge_string = 'A' * 1000000000
print('re_test_false: ', timeit.timeit(functools.partial(re_test_false, huge_string), number=1))

print('re_test_true:  ', timeit.timeit(functools.partial(re_test_true, huge_string), number=1))

----------
components: Library (Lib)
files: regex_timeit.py
messages: 384782
nosy: another_try
priority: normal
severity: normal
status: open
title: Regex performance problem with ^ aka AT_BEGINNING
type: performance
versions: Python 3.10
Added file: https://bugs.python.org/file49733/regex_timeit.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42885>
_______________________________________

From report at bugs.python.org  Sun Jan 10 17:44:43 2021
From: report at bugs.python.org (Camion)
Date: Sun, 10 Jan 2021 22:44:43 +0000
Subject: [New-bugs-announce] [issue42886] math.log and math.log10 domain
 error on very large Fractions
Message-ID: <1610318683.55.0.0487505722004.issue42886@roundup.psfhosted.org>


New submission from Camion <camion_spam-pybugs at yahoo.com>:

Python is able to computer the logarithms of very large integers and fractions (by using log(num)-log(denom)), However the fractions.Fraction class fails to do it and raises a math domain error exception.

>>> import math, fractions
>>> f=fractions.Fraction(math.factorial(10000), math.factorial(20000)+1)
>>> math.log(f.numerator)-math.log(f.denominator)
-95966.69390038431
>>> math.log(f)
Traceback (most recent call last):
  File "<pyshell#30>", line 1, in <module>
    math.log(f)
ValueError: math domain error
>>> math.log10(f.numerator)-math.log10(f.denominator)
-41677.80560743537
>>> math.log10(f)
Traceback (most recent call last):
  File "<pyshell#32>", line 1, in <module>
    math.log10(f)
ValueError: math domain error

----------
components: Library (Lib)
messages: 384783
nosy: Camion
priority: normal
severity: normal
status: open
title: math.log and math.log10 domain error on very large Fractions
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42886>
_______________________________________

From report at bugs.python.org  Mon Jan 11 02:38:04 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Mon, 11 Jan 2021 07:38:04 +0000
Subject: [New-bugs-announce] [issue42887] Multiple assignments of attribute
 "__sizeof__" will cause a segfault
Message-ID: <1610350684.08.0.491719857036.issue42887@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

In the following program 1? method  "__sizeof__()" is called and assigned multiple times. The program can work well on Python 3.10. However if I change "__sizeof__()" to  "__sizeof__".  Then a segmentation fault is reported. I think something wrong for the parser when dealing build-in attribute assignment.



program 1: 
=========================
mystr  = "hello123"
for x in range(1000000):
    mystr = mystr.__sizeof__()
    print(mystr)
=========================
56
28
28
.......
28
28

Output: work well as expected.


program 2: 
==========================
mystr = "hello123"
for x in range(1000000):
        mystr = mystr.__sizeof__
        print(mystr)
==========================
<built-in method __sizeof__ of builtin_function_or_method object at 0x7f04d3e0c220>
......
<built-in method __sizeof__ of builtin_function_or_method object at 0x7f04d3e0c4f0>
<built-in method __sizeof__ of builtin_function_or_method object at 0x7f04d3e0c540>
Segmentation fault (core dumped)

Expected output: no segfault.

----------
components: Interpreter Core
messages: 384797
nosy: xxm
priority: normal
severity: normal
status: open
title: Multiple assignments of attribute "__sizeof__" will cause a segfault
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42887>
_______________________________________

From report at bugs.python.org  Mon Jan 11 02:40:22 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Mon, 11 Jan 2021 07:40:22 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue42888=5D_Not_installed_?=
 =?utf-8?b?4oCcbGliZ2NjX3Muc28uMeKAnSBjYXVzZXMgcGFyc2VyIGNyYXNoLg==?=
Message-ID: <1610350822.34.0.431759793604.issue42888@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

The following thread program will cause Python3.10 parser "core dump" due to missing ?libgcc_s.so.1?. "pthread_cancel" cannot work correctly. I am wondering is there any possible to install or link to libgcc_s.so.1 during Python installation?

===================================
import socket
from threading import Thread
class thread(Thread):
      def run(self):
          for res in socket.getaddrinfo('www.google.com',8080):
              sock = socket.socket()
              sock.connect(res[4])

for i in range(2000):
    thread().start()
====================================
Error message:
---------------------------------------------------------------
........
Exception in thread Thread-1346:
Traceback (most recent call last):
  File "/usr/local/python310/lib/python3.10/threading.py", line 960, in _bootstrap_inner
  File "/usr/local/python310/lib/python3.10/threading.py", line 960, in _bootstrap_inner
Exception in thread Thread-1172:
Traceback (most recent call last):
.......
Exception in thread Thread-1509:
Exception in thread Thread-1510:
libgcc_s.so.1 must be installed for pthread_cancel to work
Exception in thread Thread-1511:
Traceback (most recent call last):
Aborted (core dumped)
---------------------------------------------------------------

----------
components: Interpreter Core
messages: 384798
nosy: xxm
priority: normal
severity: normal
status: open
title: Not installed ?libgcc_s.so.1? causes parser crash.
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42888>
_______________________________________

From report at bugs.python.org  Mon Jan 11 02:45:39 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Mon, 11 Jan 2021 07:45:39 +0000
Subject: [New-bugs-announce] [issue42889] Incorrect behavior of Python
 parser after ast node of test program being modified
Message-ID: <1610351139.91.0.540037450972.issue42889@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

The following program will lead to a incorrect behavior of Python parser.  We change all variable to integer( forcely transformed to string) via ast.NodeTransformer. Then we compile the new code and execute it. It's surprising that code like "1=2; print(1)" can be compiled without error message. And more surprisingly, the execution result of "print(1)" is 2 ! !

====================================
import ast
class RewriteName(ast.NodeTransformer):
    def visit_Name(self, node):
        if node.id != "print":
            node.id = str(node.lineno)
        return node


code = "a = 2;print(a)"


myast = ast.parse(code)
transformer = RewriteName()
newast = transformer.visit(myast)

# print(ast.dump(newast))

print("new code:","------------")
print(ast.unparse(newast))
print("------------")

c = compile(newast,'','exec')
exec(c)
=================================

output result:
new code: ------------
1 = 2
print(1)
------------
2

Expected result: (1). Syntax error during compilation. "1" should not be the correct identifier of program, even if it's forcely transformed to string. (2). The output of execution should be "1"  , since the parameter of "print()" in the new code is string "1".  

This incorrect behaviors exists on all version of Python(from Python2 to Python 3).

----------
components: Interpreter Core
messages: 384799
nosy: xxm
priority: normal
severity: normal
status: open
title: Incorrect behavior of Python parser after ast node of test program  being modified
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42889>
_______________________________________

From report at bugs.python.org  Mon Jan 11 05:26:19 2021
From: report at bugs.python.org (Wlodek)
Date: Mon, 11 Jan 2021 10:26:19 +0000
Subject: [New-bugs-announce] [issue42890] bug float to int
Message-ID: <1610360779.86.0.614039765509.issue42890@roundup.psfhosted.org>


New submission from Wlodek <wlodek at freenet.de>:

Python 3.8.5 (default, Jul 28 2020, 12:59:40) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f=0.1234/100
>>> f2= f*10**8
>>> i2= int(f2)
>>> print("float=%f, float_2=%f, int(float_2)=%d" % (f, f2, int(f2)))
float=0.001234, float_2=123400.000000, int(float_2)=123399

----------
components: Library (Lib)
messages: 384807
nosy: wlodek
priority: normal
severity: normal
status: open
title: bug float to int
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42890>
_______________________________________

From report at bugs.python.org  Mon Jan 11 09:48:27 2021
From: report at bugs.python.org (Amirouche Boubekki)
Date: Mon, 11 Jan 2021 14:48:27 +0000
Subject: [New-bugs-announce] [issue42891] segfault with gunicorn and a
 library made with cython bindings
Message-ID: <1610376507.28.0.240401298439.issue42891@roundup.psfhosted.org>


New submission from Amirouche Boubekki <amirouche.boubekki at gmail.com>:

Here is a simple way to reproduce:


from lsm import LSM


db = LSM('db.sqlite')

def app(environ, start_response):
    """Simplest possible application object"""

    for (index, (key, value)) in enumerate(db[b'\x00':b'\xFF']):
        pass

    start_response(b'200', {})
    return b''

db.close()


In my real program, if I add 'global db' in the function `app`, it does not segfault.


program: https://git.sr.ht/~amirouche/segfault
trace: https://wyz.fr/0I-MO

----------
messages: 384836
nosy: amirouche
priority: normal
severity: normal
status: open
title: segfault with gunicorn and a library made with cython bindings
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42891>
_______________________________________

From report at bugs.python.org  Mon Jan 11 11:45:57 2021
From: report at bugs.python.org (Xavier Hausherr)
Date: Mon, 11 Jan 2021 16:45:57 +0000
Subject: [New-bugs-announce] [issue42892] AttributeError in
 email.message.get_body()
Message-ID: <1610383557.89.0.290771399006.issue42892@roundup.psfhosted.org>


New submission from Xavier Hausherr <xavier at shipfix.com>:

Following this issue: https://bugs.python.org/issue33972

Same bug apply to email.message.get_body() with attached email example and the following code: 

        from email.policy import default
        import email

        with open('email_bad_formatted.eml', 'rb') as fp:
            msg = email.message_from_binary_file(fp, policy=default)
        body = msg.get_body()

> Result:
E       AttributeError: 'str' object has no attribute 'is_attachment'

/usr/local/lib/python3.9/email/message.py:978: AttributeError

----------
components: email
files: email_bad_formatted.eml
messages: 384847
nosy: barry, iritkatriel, r.david.murray, xavier2
priority: normal
severity: normal
status: open
title: AttributeError in email.message.get_body()
versions: Python 3.9
Added file: https://bugs.python.org/file49734/email_bad_formatted.eml

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42892>
_______________________________________

From report at bugs.python.org  Mon Jan 11 12:12:45 2021
From: report at bugs.python.org (robpats)
Date: Mon, 11 Jan 2021 17:12:45 +0000
Subject: [New-bugs-announce] [issue42893] Strange XPath search behavior of
 xml.etree.ElementTree.Element.find
Message-ID: <1610385165.69.0.550442509332.issue42893@roundup.psfhosted.org>


New submission from robpats <robotpats at gmail.com>:

Python 3.6.8 / 3.7.9 / 3.8.7

>>> import xml.etree.ElementTree
>>> e = xml.etree.ElementTree.fromstring('<html><div class="row"/><hr/><div/><hr/><div class="row"/><button/></html>')
>>> list(e)
[<Element 'div' at 0x00000000024CD220>, <Element 'hr' at 0x00000000024CD2C0>, <Element 'div' at 0x00000000024F90E0>, <Element 'hr' at 0x00000000024F9130>, <Element 'div' at 0x00000000024F9180>, <Element 'button' at 0x00000000024F91D0>]
>>> e.find("./div[1]")
<Element 'div' at 0x00000000024CD220>
>>> e.find("./div[2]")
<Element 'div' at 0x00000000024F90E0>
>>> e.find("./div[3]")
<Element 'div' at 0x00000000024F9180>
>>> e.find("./hr[1]")
<Element 'hr' at 0x00000000024CD2C0>
>>> e.find("./hr[2]")
<Element 'hr' at 0x00000000024F9130>



# The following different from XPath implementation in Firefox
# https://developer.mozilla.org/en-US/docs/Web/XPath/Snippets

>>> list(e.iterfind("./*"))
[<Element 'div' at 0x00000000024CD220>, <Element 'hr' at 0x00000000024CD2C0>, <Element 'div' at 0x00000000024F90E0>, <Element 'hr' at 0x00000000024F9130>, <Element 'div' at 0x00000000024F9180>, <Element 'button' at 0x00000000024F91D0>]
>>> e.find("./*[1]")
<Element 'div' at 0x00000000024CD220>
>>> e.find("./*[2]")
<Element 'div' at 0x00000000024F90E0>   <-- should be 'hr', same as e.find("./div[2]") instead of e[2]
>>> e.find("./*[3]")
<Element 'div' at 0x00000000024F9180>   <-- same as e.find("./div[3]") instead of e[3]
>>> e.find("./*[4]")


>>> list(e.iterfind("./*[@class='row']"))
[<Element 'div' at 0x00000000024CD220>, <Element 'div' at 0x00000000024F9180>]
>>> e.find("./*[@class='row'][1]")
<Element 'div' at 0x00000000024CD220>
>>> e.find("./*[@class='row'][2]")
>>> e.find("./*[@class='row'][3]")
<Element 'div' at 0x00000000024F9180>   <--- cannot find element at [2] but found at [3]

----------
components: Library (Lib)
messages: 384851
nosy: robpats
priority: normal
severity: normal
status: open
title: Strange XPath search behavior of xml.etree.ElementTree.Element.find
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42893>
_______________________________________

From report at bugs.python.org  Mon Jan 11 13:34:18 2021
From: report at bugs.python.org (Jeff Moguillansky)
Date: Mon, 11 Jan 2021 18:34:18 +0000
Subject: [New-bugs-announce] [issue42894] Debugging native Python modules on
 Windows with Visual Studio Toolchain
Message-ID: <1610390058.13.0.89065377446.issue42894@roundup.psfhosted.org>


New submission from Jeff Moguillansky <jmoguill2 at gmail.com>:

I have a question regarding debugging native Python modules on Windows, with Visual Studio toolchain:

Currently I have a native module (native C code), along with Python API bindings (via Cython), and finally Python code that invokes the native module.  I also use various third-party python modules like Pillow, etc.  

In order to debug on Windows, I have to use the following tricks:
1) Build the native module in Release Mode
2) Disable Compiler Optimization
3) Enable Debug symbols

I can't just use Python distutils out of the box, I have to manually modify the build commands to enable Debugging.

If I just try to build the native module in Debug mode, I get Visual Studio compile errors related to: not being able to mix code built with different C++ runtime libraries.  
Some of the 3rd-party Python modules are only available as Release builds (not Debug builds).

I'm wondering if anyone has encountered a similar issue, and what's your advice?  

On Linux, GNU toolchain, this isn't an issue.  The toolchain lets you mix release and debug libraries, no problem.

----------
components: C API
messages: 384853
nosy: jmoguill2
priority: normal
severity: normal
status: open
title: Debugging native Python modules on Windows with Visual Studio Toolchain
type: compile error
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42894>
_______________________________________

From report at bugs.python.org  Mon Jan 11 13:38:58 2021
From: report at bugs.python.org (JD-Veiga)
Date: Mon, 11 Jan 2021 18:38:58 +0000
Subject: [New-bugs-announce] [issue42895] Return multi-line list
 concatenation without parentheses returns only first operand
Message-ID: <1610390338.44.0.382360444411.issue42895@roundup.psfhosted.org>


New submission from JD-Veiga <jdveiga at gmail.com>:

I was trying to return the concatenation of several lists from a function but, inadvertently, forgot to surround the multi-line concatenation expression with parentheses. 

As a result, the function returns just the first operand and does not perform the concatenation at all. 


Basically, I am doing something like this:

```
def non_parens():
    return [
        1, 2, 3
    ]
    + [
        4, 5
    ]

print(non_parens())

```

which outputs: `[1, 2, 3]`


On the contrary, parenthesised version such as:

```
def with_parens():
    return (
        [
            1, 2, 3
        ]
        + [
            4, 5
        ]
    )

print(with_parens())

```

will output the "expected" result: `[1, 2, 3, 4, 5]`.


Even not breaking the line before the '+' operator:

```
def no_parens_without_newline_before_operator():
    return [
        1, 2, 3
    ] + [
        4, 5
    ]

print(no_parens_without_newline_before_operator())

```

prints `[1, 2, 3, 4, 5]`.



My hypothesis is that first function `non_parens()` does not work because the expression has some kind of error after

```
    return [
        1, 2, 3
    ]
```

and 

```
    + [
        4, 5
    ]
```

is never executed.


However, why does not the parser or the interpreter raise any error or warning about this malformed expression?


"Similar" expressions cause an error:

```
assert [
        1, 2, 3
    ]
    + [
        4, 5
    ]
```

raises: `IndentationError: unexpected indent`

and

```
[
    1, 2, 3
]
+ [
    4, 5
]
```

raises: `TypeError: bad operand type for unary +: 'list'`


What perplexes me the most about breaking lines in this case is that `] + [` works and `]<newline>+ [` does not (against PEP8 --of course, PEP8 is not the parser).


I am sure that I am missing something about expressions, line breaks, or lexical parsing.


I am using Python 3.8.7 and Python 3.9.1



Thank you a lot.

----------
components: Interpreter Core
messages: 384854
nosy: JD-Veiga
priority: normal
severity: normal
status: open
title: Return multi-line list concatenation without parentheses returns only first operand
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42895>
_______________________________________

From report at bugs.python.org  Mon Jan 11 13:55:17 2021
From: report at bugs.python.org (David Murphy)
Date: Mon, 11 Jan 2021 18:55:17 +0000
Subject: [New-bugs-announce] [issue42896] Solaris 11.4 crle output not
 handled correctly
Message-ID: <1610391317.65.0.0245738213738.issue42896@roundup.psfhosted.org>


New submission from David Murphy <damurphy at vmware.com>:

>From porting Python 3.7.8 to Solaris 11.4 (base open version) found that the handling of crle output has changed between Solaris 11.3 and 11.4 and accommodation has not been made for the change.

For example:
Solaris 11.3
root at sol11:~/sol_build/proto_salt# uname -a
SunOS sol11 5.11 11.3 i86pc i386 i86pc
root at sol11:~/sol_build/proto_salt# crle

Default configuration file (/var/ld/ld.config) not found
  Platform:	32-bit LSB 80386
  Default Library Path (ELF):	/lib:/usr/lib  (system default)
  Trusted Directories (ELF):	/lib/secure:/usr/lib/secure  (system default)
root at sol11:~/sol_build/proto_salt# crle -64

Default configuration file (/var/ld/64/ld.config) not found
  Platform:	64-bit LSB AMD64
  Default Library Path (ELF):	/lib/64:/usr/lib/64  (system default)
  Trusted Directories (ELF):	/lib/secure/64:/usr/lib/secure/64  (system default)
root at sol11:~/sol_build/proto_salt#


Solaris 11.4
root at sol114:/export/home/david/dev/dist/run# uname -a
SunOS sol114 5.11 11.4.0.15.0 i86pc i386 i86pc Solaris
root at sol114:/export/home/david/dev/dist/run# crle

Configuration file [version 5]: /var/ld/ld.config
    Platform:	32-bit LSB 80386
    Default Library Path:	/usr/local/openssl/lib:/usr/local/lib:/lib:/usr/lib
    Trusted Directories:	/lib/secure:/usr/lib/secure  (system default)

Command line:
    crle -c /var/ld/ld.config -l /usr/local/openssl/lib:/usr/local/lib:/lib:/usr/lib

root at sol114:/export/home/david/dev/dist/run# crle -64

Configuration file [version 5]: /var/ld/64/ld.config
    Platform:	64-bit LSB AMD64
    Default Library Path:	/usr/local/openssl/lib:/usr/local/lib:/lib/64:/usr/lib/64
    Trusted Directories:	/lib/secure/64:/usr/lib/secure/64  (system default)

Command line:
    crle -64 -c /var/ld/64/ld.config -l /usr/local/openssl/lib:/usr/local/lib:/lib/64:/usr/lib/64

root at sol114:/export/home/david/dev/dist/run#

Note: the missing '(ELF)' from the 'Default Library Path'


Simple fix is the following patch:
david at sol114:~/dev$ cat solaris11_crle.patch
--- util.py         2021-01-08 17:01:58.417014094 +0000
+++ util.py.new 2021-01-08 17:03:21.843483945 +0000
@@ -238,6 +238,10 @@
                     line = line.strip()
                     if line.startswith(b'Default Library Path (ELF):'):
                         paths = os.fsdecode(line).split()[4]
+                    elif line.startswith(b'Default Library Path:'):
+                        ## allow for Solaris 11.4 output
+                        paths = os.fsdecode(line).split()[3]
+            
 
             if not paths:
                 return None
david at sol114:~/dev$

----------
components: ctypes
files: solaris11_crle.patch
keywords: patch
messages: 384856
nosy: dmurphy18
priority: normal
severity: normal
status: open
title: Solaris 11.4 crle output not handled correctly
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49735/solaris11_crle.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42896>
_______________________________________

From report at bugs.python.org  Mon Jan 11 14:55:45 2021
From: report at bugs.python.org (Peter Van Sickel)
Date: Mon, 11 Jan 2021 19:55:45 +0000
Subject: [New-bugs-announce] [issue42897] Expose a way to determine if a
 process has been closed or not
Message-ID: <1610394945.99.0.114538503727.issue42897@roundup.psfhosted.org>


New submission from Peter Van Sickel <pvs at us.ibm.com>:

I have been using the multiprocessing Process class a good bit lately. I have a class that is managing the a given list of processes from launch to completion. 
Recently I started using Process close().  I found myself wanting to determine if a given process instance was closed or not before I did anything like check its exitcode or invoke any of the other methods that raise a ValueError if done so on a closed process.
As far as I can tell there is no exposed way to check if the process is closed.  The Process class has a _closed instance variable and a _check_closed() method, but those are not intended for direct use.
I created a simple wrapper class that has its own closed instance variable and wraps the close() method of Process so that the closed instance variable can be set to True when close() is called and then call super().close() to allow the normal close operation to complete.
It would be convenient if the Process class itself supported an is_closed() method or exposed a Boolean closed attribute to easily determine if a process instance has been closed.

----------
messages: 384860
nosy: petervansickel
priority: normal
severity: normal
status: open
title: Expose a way to determine if a process has been closed or not
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42897>
_______________________________________

From report at bugs.python.org  Mon Jan 11 16:12:27 2021
From: report at bugs.python.org (Kale Kundert)
Date: Mon, 11 Jan 2021 21:12:27 +0000
Subject: [New-bugs-announce] [issue42898] pickle.loads() crashes interpreter
 on invalid input
Message-ID: <1610399547.98.0.314580970043.issue42898@roundup.psfhosted.org>


New submission from Kale Kundert <kale at thekunderts.net>:

I expect `pickle.loads()` to raise `_pickle.UnpicklingError` for any invalid input, but for the specific example shown below, the interpreter crashes after attempting to allocate >16GB of memory.  Note that this input does not have the pickle header (b'0x80'), so it should be easy to distinguish from valid input.

$ python
Python 3.8.2 (default, Apr 13 2020, 11:02:04) 
[Clang 9.0.1 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> pickle.loads(b'January 11')
[1]    624227 killed     python3

----------
messages: 384865
nosy: kalekundert
priority: normal
severity: normal
status: open
title: pickle.loads() crashes interpreter on invalid input

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42898>
_______________________________________

From report at bugs.python.org  Mon Jan 11 17:26:53 2021
From: report at bugs.python.org (Steve Stagg)
Date: Mon, 11 Jan 2021 22:26:53 +0000
Subject: [New-bugs-announce] [issue42899] Possible regression introduced by
 bpo-42615
Message-ID: <1610404013.09.0.265713602389.issue42899@roundup.psfhosted.org>


New submission from Steve Stagg <stestagg at gmail.com>:

This was raised by Mats Wichmann <mats at python.org>  on the python-dev list.

Commit : c71581c7a4192e6ba9a79eccc583aaadab300efa
bpo-42615: Delete redundant jump instructions that only bypass empty blocks (GH-23733)

appears to have changed the behaviour of the following code:

class B:
     def __bool__(self):
         raise AttributeError("don't do that!")

b = B()
try:
   if b:
        pass
except AttributeError:
   print("HI")


Before the change, the output is:

bool(B)
GOT ERROR
Traceback (most recent call last):
  File "../test.py", line 8, in <module>
    if b:
  File "../test.py", line 4, in __bool__
    raise AttributeError("don't do that!")
AttributeError: don't do that!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../test.py", line 12, in <module>
    raise IndexError("Should GET THIS")
IndexError: Should GET THIS


After the change, just:

SHOULDN'T GET THIS

It seems like the entire branch is being eliminated prematurely, but maybe only when the statement is wrapped in a try-except?

----------
messages: 384867
nosy: Mark.Shannon, m, stestagg
priority: normal
severity: normal
status: open
title: Possible regression introduced by bpo-42615
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42899>
_______________________________________

From report at bugs.python.org  Mon Jan 11 20:20:28 2021
From: report at bugs.python.org (Peter)
Date: Tue, 12 Jan 2021 01:20:28 +0000
Subject: [New-bugs-announce] [issue42900] Ternary operator precedence
 relative to bitwise or
Message-ID: <1610414428.66.0.949606114531.issue42900@roundup.psfhosted.org>


New submission from Peter <peter_dulimov at hotmail.com>:

Hello,

I expect the following code to run fine, but the assertion fails. dbg1 is 1, while dbg2 is 3.  I thought they would both be 3.
Note that the only difference between the expressions for dbg1 and dbg2 are the parentheses.
Please accept my apologies if this is expected behavior, but it came as a big surprise to me.


import json
if __name__ == "__main__":
    msg = '''
        {
        "Sync Setup Flags": {
            "Setup Sync": "Enable",
            "Generate Primary Sync": "Enable",
            "Backup Primary Sync": "Disable",
            "Follow Only": "Disable",
            "Use Local Clock": "Disable",
            "Set Active": "Disable"
        }
        }
        '''
    obj = json.loads(msg)
    dbg1 = \
        1 if obj["Sync Setup Flags"]["Setup Sync"] == "Enable" else 0 | \
        2 if obj["Sync Setup Flags"]["Generate Primary Sync"] == "Enable" else 0 | \
        4 if obj["Sync Setup Flags"]["Backup Primary Sync"] == "Enable" else 0 | \
        8 if obj["Sync Setup Flags"]["Follow Only"] == "Enable" else 0 | \
        16 if obj["Sync Setup Flags"]["Use Local Clock"] == "Enable" else 0 | \
        128 if obj["Sync Setup Flags"]["Set Active"] == "Enable" else 0
    dbg2 = \
        (1 if obj["Sync Setup Flags"]["Setup Sync"] == "Enable" else 0) | \
        (2 if obj["Sync Setup Flags"]["Generate Primary Sync"] == "Enable" else 0) | \
        (4 if obj["Sync Setup Flags"]["Backup Primary Sync"] == "Enable" else 0) | \
        (8 if obj["Sync Setup Flags"]["Follow Only"] == "Enable" else 0) | \
        (16 if obj["Sync Setup Flags"]["Use Local Clock"] == "Enable" else 0) | \
        (128 if obj["Sync Setup Flags"]["Set Active"] == "Enable" else 0)

    assert(dbg1 == dbg2)

----------
messages: 384874
nosy: amazingmo
priority: normal
severity: normal
status: open
title: Ternary operator precedence relative to bitwise or
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42900>
_______________________________________

From report at bugs.python.org  Mon Jan 11 20:27:28 2021
From: report at bugs.python.org (Ethan Furman)
Date: Tue, 12 Jan 2021 01:27:28 +0000
Subject: [New-bugs-announce] [issue42901] [Enum] move member creation to
 __set_name__ in order to support __init_subclass__
Message-ID: <1610414848.42.0.389708219553.issue42901@roundup.psfhosted.org>


New submission from Ethan Furman <ethan at stoneleaf.us>:

In discussions about moving the calls to `__set_name__` and `__init_subclass__`, Nick Coughlan made an observation:

Nick Coghlan:
> Both EnumMeta and ABCMeta should probably be relying on `__set_name__`
> for their per-member set up work these days, rather than deferring that
> work until after `__new__` returns.

By having `__set_name__` create the final members, they will be in place for the call to `__init_subclass__`.

----------
assignee: ethan.furman
components: Library (Lib)
messages: 384875
nosy: barry, eli.bendersky, ethan.furman
priority: high
severity: normal
stage: needs patch
status: open
title: [Enum] move member creation to __set_name__ in order to support __init_subclass__
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42901>
_______________________________________

From report at bugs.python.org  Mon Jan 11 22:58:43 2021
From: report at bugs.python.org (houjingyi)
Date: Tue, 12 Jan 2021 03:58:43 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue42902=5D_a_python_embedded?=
 =?utf-8?q?_program=C2=A0may_load_=22C=3A=5CLib=5Cos=2Epy=22_on_windows_sy?=
 =?utf-8?q?stem?=
Message-ID: <1610423923.25.0.321902126582.issue42902@roundup.psfhosted.org>


New submission from houjingyi <houjingyi647 at gmail.com>:

environment: windows 10, python3.8.7 installed to "C:\Program Files\Python38".

datail info: According to https://docs.python.org/3/c-api/init.html: "Py_SetPath() set the default module search path. If this function is called before Py_Initialize(), then Py_GetPath() won?t attempt to compute a default search path but uses the one provided instead."
Write following code that only call Py_Initialize():

#include <iostream>
#include <Python.h>
#include <Windows.h>
using namespace std;
int main()
{
Py_Initialize();
} 

In visual studio add "C:\Program Files\Python38\include" to AdditionalIncludeDirectories, add "C:\Program Files\Python38\libs\python38.lib" to AdditionalDependencies to compile it to poc.exe. Copy "C:\Program Files\Python38\Lib" to "C:\Lib" and modify "C:\Lib\os.py" to execute any code we like. For example we can add "import os" and add "os.system(notepad)" in function "def _exists(name)". Now run poc.exe it will create notepad. 

impact: In my report I showed that a python embedded program?may load "C:\Lib\os.py" which?lower privileged?user can control. If this program runs as administrator then this may cause vertical privilege escalation,?low privileged user gets higher privilege;?If this program do not run as administrator then this may cause vertical privilege escalation,?low privileged user can execute code as others(https://en.wikipedia.org/wiki/Privilege_escalation). In either case, the access control of the windows system is broken.

notice: The report was sent to security at python.org before and they suggested it can be reported publicly.

----------
components: C API
messages: 384882
nosy: houjingyi233
priority: normal
severity: normal
status: open
title: a python embedded program?may load "C:\Lib\os.py" on windows system
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42902>
_______________________________________

From report at bugs.python.org  Mon Jan 11 23:31:59 2021
From: report at bugs.python.org (Eugene Toder)
Date: Tue, 12 Jan 2021 04:31:59 +0000
Subject: [New-bugs-announce] [issue42903] optimize lru_cache for functions
 with no arguments
Message-ID: <1610425919.12.0.56212738564.issue42903@roundup.psfhosted.org>


New submission from Eugene Toder <eltoder at gmail.com>:

It's convenient to use @lru_cache on functions with no arguments to delay doing some work until the first time it is needed. Since @lru_cache is implemented in C, it is already faster than manually caching in a closure variable. However, it can be made even faster and more memory efficient by not using the dict at all and caching just the one result that the function returns.

Here are my timing results. Before my changes:

$ ./python -m timeit -s "import functools; f = functools.lru_cache()(lambda: 1)" "f()"
5000000 loops, best of 5: 42.2 nsec per loop
$ ./python -m timeit -s "import functools; f = functools.lru_cache(None)(lambda: 1)" "f()"
5000000 loops, best of 5: 38.9 nsec per loop

After my changes:

$ ./python -m timeit -s "import functools; f = functools.lru_cache()(lambda: 1)" "f()"
10000000 loops, best of 5: 22.6 nsec per loop

So we get improvement of about 80% compared to the default maxsize and about 70% compared to maxsize=None.

----------
components: Library (Lib)
messages: 384883
nosy: eltoder, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: optimize lru_cache for functions with no arguments
type: performance
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42903>
_______________________________________

From report at bugs.python.org  Tue Jan 12 00:00:15 2021
From: report at bugs.python.org (Paul Bryan)
Date: Tue, 12 Jan 2021 05:00:15 +0000
Subject: [New-bugs-announce] [issue42904] get_type_hints does not provide
 localns for classes
Message-ID: <1610427615.24.0.000795612365541.issue42904@roundup.psfhosted.org>


New submission from Paul Bryan <pbryan at anode.ca>:

According to PEP 563:

> The get_type_hints() function automatically resolves the correct value of globalns for functions and classes. It also automatically provides the correct localns for classes.

This statement about providing correct localns for classes does not appear to be true.

Guido suggested this should be treated as a bug.

----------
components: Library (Lib)
messages: 384885
nosy: gvanrossum, pbryan
priority: normal
severity: normal
status: open
title: get_type_hints does not provide localns for classes
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42904>
_______________________________________

From report at bugs.python.org  Tue Jan 12 02:37:11 2021
From: report at bugs.python.org (Sachit Murarka)
Date: Tue, 12 Jan 2021 07:37:11 +0000
Subject: [New-bugs-announce] [issue42905] Issue while installing numba
Message-ID: <1610437031.96.0.463158276595.issue42905@roundup.psfhosted.org>


New submission from Sachit Murarka <connectsachit at gmail.com>:

I am having Python 3.8
I am installing fastparquet.

It is giving following error. 

I am doing it in Centos

running build_ext
  building 'numba._dynfunc' extension
  Warning: Can't read registry to find the necessary compiler setting
  Make sure that Python modules winreg, win32api or win32con are installed.
  C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC

  creating build/temp.linux-x86_64-3.8
  creating build/temp.linux-x86_64-3.8/numba
  compile options: '-I/usr/local/include/python3.8 -c'
  gcc: numba/_dynfuncmod.c
  gcc -pthread -shared build/temp.linux-x86_64-3.8/numba/_dynfuncmod.o -o build/lib.linux-x86_64-3.8/numba/_dynfunc.cpython-38-x86_64-linux-gnu.so
  building 'numba._dispatcher' extension
  C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPI

----------
components: Library (Lib)
messages: 384898
nosy: connectsachit
priority: normal
severity: normal
status: open
title: Issue while installing numba
type: compile error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42905>
_______________________________________

From report at bugs.python.org  Tue Jan 12 05:39:42 2021
From: report at bugs.python.org (Stijn van Drongelen)
Date: Tue, 12 Jan 2021 10:39:42 +0000
Subject: [New-bugs-announce] [issue42906] python3 -m doctest test.py tests
 the stdlib "time" module instead
Message-ID: <1610447982.97.0.137318779169.issue42906@roundup.psfhosted.org>


New submission from Stijn van Drongelen <stijnvandrongelen at gmail.com>:

I have two files, named time.py and time_.py, with the same contents:

    def foo():
        """Returns 1.

        >>> foo()
        1
        """
        return 1

When I run

    python3 -m doctest -v time_.py

I see the expected behaviour:

    Trying:
        foo()
    Expecting:
        1
    ok
    1 items had no tests:
        time_
    1 items passed all tests:
       1 tests in time_.foo
    1 tests in 2 items.
    1 passed and 0 failed.
    Test passed.

However, when I run

    python3 -m doctest -v time_.py

it seems like doctest is testing the standard library 'time' module, while I expected it to have similar output as seen above:

        30 items had no tests:
        time
        time.asctime
      (... skipped 24 lines for the bug report ...)
        time.time_ns
        time.tzset
    0 tests in 30 items.
    0 passed and 0 failed.
    Test passed.

The same happens when I provide a path with slashes in it.

----------
components: Library (Lib)
messages: 384913
nosy: stijnvandrongelen
priority: normal
severity: normal
status: open
title: python3 -m doctest test.py tests the stdlib "time" module instead
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42906>
_______________________________________

From report at bugs.python.org  Tue Jan 12 06:12:25 2021
From: report at bugs.python.org (Dimo Yordanov)
Date: Tue, 12 Jan 2021 11:12:25 +0000
Subject: [New-bugs-announce] [issue42907] Division betwen float and 100 not
 giving corect result
Message-ID: <1610449945.05.0.326021023645.issue42907@roundup.psfhosted.org>


New submission from Dimo Yordanov <dimo.yordanov669 at gmail.com>:

To replicate what i think is a bug you need a python interpreter in any version. I think. When i type 2.7/100 it returns 0.027000000000000003, but it actually is 0.027 and 2.8/100 returns 0.027999999999999997. And i don't know why

----------
components: Interpreter Core
messages: 384915
nosy: DimoY
priority: normal
severity: normal
status: open
title: Division betwen float and 100 not giving corect result
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42907>
_______________________________________

From report at bugs.python.org  Tue Jan 12 06:25:20 2021
From: report at bugs.python.org (Mark Shannon)
Date: Tue, 12 Jan 2021 11:25:20 +0000
Subject: [New-bugs-announce] [issue42908] Incorrect line numbers at end of
 try-except and with statements containing if False: pass
Message-ID: <1610450720.62.0.32697426383.issue42908@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

The following examples produce incorrect line numbers, due to cleanup code not being marked as artificial

def f():
    try:
        if False:
            pass
    except:
        X
        
def g(a):
    with a:
         if False:
             pass

----------
assignee: Mark.Shannon
messages: 384920
nosy: Mark.Shannon
priority: high
severity: normal
status: open
title: Incorrect line numbers at end of try-except and with statements containing if False: pass

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42908>
_______________________________________

From report at bugs.python.org  Tue Jan 12 08:13:10 2021
From: report at bugs.python.org (robin)
Date: Tue, 12 Jan 2021 13:13:10 +0000
Subject: [New-bugs-announce] [issue42909] Email header with ; ; ; ;
 stuffing takes very long to parse
Message-ID: <1610457190.97.0.487967910026.issue42909@roundup.psfhosted.org>


New submission from robin <robin.eriksson at f-secure.com>:

Attached please find a script which takes on the order of 1 minute to parse even though the embedded message is reasonably trivial.  The main flaw is that the Content-Type: header has a long string of redundant ;;;; which is something some spammers apparently use to bypass some content filters or analyzers.

The script is short enough to inline here for what I hope is convenience for most visitors.


from email import message_from_bytes
#from email.parser import BytesParser
from email.policy import default
##from cProfile import run as cprun

b = (b'''From: me <self at example.com>
To: you <recipient at example.net>
Subject: sample with ;;;;;;;;; stuffing in MIME Content-Type: header
Content-type: text/plain''' + b';' * 54 + b'\n' +
               36 * (b' ' + b';' * 990 + b'\n') + b'''\
Content-transfer-encoding: 7bit
MIME-Version: 1.0

Hello.
''')
## cprun('message_from_bytes(b, policy=default)', sort=1)

m = message_from_bytes(b, policy=default)
#m = BytesParser(policy=default).parsebytes(b)

print(m.as_string())


I have commented out two sets of changes; the ones with a single # demonstrate that the same error happens with BytesParser, and the ones with ## are for profiling the code.

For what it's worth, profiling consistently indicates that it gets stuck in builtins.sum while attempting to parse the message.  Here is a partial cProfile result from Python 3.7.2:

 
         2148205 function calls (2004560 primitive calls) in 34.533 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    160/4   32.356    0.202   32.728    8.182 {built-in method builtins.sum}
        4    0.560    0.140    1.661    0.415 _header_value_parser.py:2380(parse_mime_parameters)
   142772    0.230    0.000    0.230    0.000 {method 'format' of 'str' objects}
   142932    0.203    0.000    0.203    0.000 _header_value_parser.py:866(all_defects)
   142772    0.197    0.000    0.288    0.000 errors.py:84(__init__)
143248/24    0.167    0.000   32.727    1.364 _header_value_parser.py:126(<genexpr>)
   142772    0.151    0.000    0.456    0.000 _header_value_parser.py:2126(get_attribute)
   285848    0.146    0.000    0.146    0.000 _header_value_parser.py:109(__init__)
   142772    0.102    0.000    0.631    0.000 _header_value_parser.py:2241(get_parameter)
   142772    0.091    0.000    0.091    0.000 errors.py:36(__init__)
   143076    0.088    0.000    0.153    0.000 _header_value_parser.py:854(__new__)
   143080    0.065    0.000    0.065    0.000 {built-in method __new__ of type object at 0x101c20640}
       12    0.044    0.004    0.086    0.007 _header_value_parser.py:716(params)
   285839    0.042    0.000    0.042    0.000 {method 'endswith' of 'str' objects}
        3    0.030    0.010   27.512    9.171 message.py:588(get_content_maintype)
   286482    0.029    0.000    0.029    0.000 {method 'append' of 'list' objects}
        2    0.014    0.007    0.014    0.007 <frozen importlib._bootstrap_external>:914(get_data)
        6    0.008    0.001   25.369    4.228 feedparser.py:218(_parsegen)
        4    0.001    0.000    0.001    0.000 {method 'split' of 're.Pattern' objects}
    160/4    0.001    0.000   32.728    8.182 _header_value_parser.py:124(all_defects)
      288    0.001    0.000    0.002    0.000 _header_value_parser.py:1000(get_fws)
      288    0.001    0.000    0.003    0.000 _header_value_parser.py:1217(get_cfws)



Starting the code and doing a KeyboardInterrupt after a few, or many, seconds tends to get a traceback like this, which also points to roughly the same culprit:


^CTraceback (most recent call last):
  File "repro.py", line 18, in <module>
    m = message_from_bytes(b, policy=default)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/__init__.py", line 46, in message_from_bytes
    return BytesParser(*args, **kws).parsebytes(s)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/parser.py", line 124, in parsebytes
    return self.parser.parsestr(text, headersonly)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/parser.py", line 68, in parsestr
    return self.parse(StringIO(text), headersonly=headersonly)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/parser.py", line 57, in parse
    feedparser.feed(data)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/feedparser.py", line 176, in feed
    self._call_parse()
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/feedparser.py", line 180, in _call_parse
    self._parse()
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/feedparser.py", line 256, in _parsegen
    if self._cur.get_content_type() == 'message/delivery-status':
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/message.py", line 578, in get_content_type
    value = self.get('content-type', missing)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/message.py", line 471, in get
    return self.policy.header_fetch_parse(k, v)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/policy.py", line 162, in header_fetch_parse
    return self.header_factory(name, value)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/headerregistry.py", line 589, in __call__
    return self[name](name, value)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/headerregistry.py", line 197, in __new__
    cls.parse(value, kwds)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/headerregistry.py", line 448, in parse
    kwds['defects'].extend(parse_tree.all_defects)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/_header_value_parser.py", line 126, in all_defects
    return sum((x.all_defects for x in self), self.defects)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/_header_value_parser.py", line 126, in <genexpr>
    return sum((x.all_defects for x in self), self.defects)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/_header_value_parser.py", line 126, in all_defects
    return sum((x.all_defects for x in self), self.defects)
  File "/Users/eriker/.pyenv/versions/3.7.2/lib/python3.7/email/_header_value_parser.py", line 126, in <genexpr>
    return sum((x.all_defects for x in self), self.defects)
KeyboardInterrupt


I have been able to repro on Python 3.9 and 3.10rc as well by way of the official Docker images rc-buster and 3.9-slim.

----------
components: email
files: repro.py
messages: 384931
nosy: barry, eriker, r.david.murray
priority: normal
severity: normal
status: open
title: Email header with ;;;; stuffing takes very long to parse
type: resource usage
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49736/repro.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42909>
_______________________________________

From report at bugs.python.org  Tue Jan 12 09:43:59 2021
From: report at bugs.python.org (Casuall)
Date: Tue, 12 Jan 2021 14:43:59 +0000
Subject: [New-bugs-announce] [issue42910] Wrong code appears in the type
 alias introduction
Message-ID: <1610462639.95.0.034878546746.issue42910@roundup.psfhosted.org>


New submission from Casuall <Smile_zjk at qq.com>:

I learned about type aliases on the page?https://docs.python.org/3/library/typing.html?, and I found that following the code (Vector = list[float])in the tutorial, an error occurred (NameError: name 'typing' is not defined) when running.

The following code has the same problem?
  ConnectionOptions = dict[str, str]
  Address = tuple[str, int]
  Server = tuple[Address, ConnectionOptions]

I searched through google and found the correct code?
  from typing import List
  Vector = List[float]

  from typing import Dict, Tuple, Sequence
 
  ConnectionOptions = Dict[str, str]
  Address = Tuple[str, int]
  Server = Tuple[Address, ConnectionOptions]

----------
assignee: docs at python
components: Documentation
messages: 384944
nosy: Smile-zjk, docs at python
priority: normal
severity: normal
status: open
title: Wrong code appears in the type alias introduction

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42910>
_______________________________________

From report at bugs.python.org  Tue Jan 12 09:52:20 2021
From: report at bugs.python.org (Jurjen N.E. Bos)
Date: Tue, 12 Jan 2021 14:52:20 +0000
Subject: [New-bugs-announce] [issue42911] Addition chains for pow saves 10 %
 time!
Message-ID: <1610463140.75.0.177431925065.issue42911@roundup.psfhosted.org>


New submission from Jurjen N.E. Bos <jneb at users.sourceforge.net>:

When looking at the code of pow() with integer exponent, I noticed there is a hard boundary between the binary and "fiveary" (actually 32-ary) computations. Also, the fiveary wasn't really optimal.

So I wrote a proof of concept version of long_pow that dynamically uses addition chains!
It does save over 10 % of multiplications for exponents from 20 to a few hundred bits, and then the saving go down to a few percent for very long numbers. It does not take much more memory nor time for any argument combination I checked.
I tested it on 3.8rc1, but I am planning to port it to 3.10. This is a bit difficult, since *lots* of code around it changed, and I only have Windows 7. However, I'll keep you posted.
See https://github.com/jneb/cpython/tree/38_fast_pow

----------
components: Interpreter Core
files: longobject.c
messages: 384949
nosy: jneb
priority: normal
severity: normal
status: open
title: Addition chains for pow saves 10 % time!
type: performance
versions: Python 3.10
Added file: https://bugs.python.org/file49737/longobject.c

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42911>
_______________________________________

From report at bugs.python.org  Tue Jan 12 16:14:29 2021
From: report at bugs.python.org (Kevin Chen)
Date: Tue, 12 Jan 2021 21:14:29 +0000
Subject: [New-bugs-announce] [issue42912] enum.Flag ~ bitwise negation is
 very slow
Message-ID: <1610486069.95.0.26867920726.issue42912@roundup.psfhosted.org>


Change by Kevin Chen <kevinjchen94 at gmail.com>:


----------
nosy: aspin2
priority: normal
severity: normal
status: open
title: enum.Flag ~ bitwise negation is very slow
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42912>
_______________________________________

From report at bugs.python.org  Tue Jan 12 16:43:53 2021
From: report at bugs.python.org (Michel Hidalgo)
Date: Tue, 12 Jan 2021 21:43:53 +0000
Subject: [New-bugs-announce] [issue42913] asyncio.ProactorEventLoop
 mishandles signal wakeup file descriptor
Message-ID: <1610487833.61.0.883604582572.issue42913@roundup.psfhosted.org>


New submission from Michel Hidalgo <hid.michel at gmail.com>:

asyncio.ProactorEventLoop uses a socket.socketpair and signal.set_wakeup_fd to wake up a loop that's polling I/O. However it does so with no consideration for file descriptors previously set (i.e. no signal number forwarding). Either by user code or by another instance of asyncio.ProactorEventLoop.

The following snippet is enough for the above to cause the loop to hang forever:

import asyncio
import gc

asyncio.set_event_loop(asyncio.ProactorEventLoop())
asyncio.set_event_loop(asyncio.ProactorEventLoop())
gc.collect()
asyncio.get_event_loop().run_forever()


The first asyncio.ProactorEventLoop instance sets a signal wakeup file descriptor on construction (see https://github.com/python/cpython/blob/187f76def8a5bd0af7ab512575cad30cfe624b05/Lib/asyncio/proactor_events.py#L632). The second instances does the same, dropping the file descriptor set by the first one (not good, not necessarily bad). When the garbage collector purges the first instance, signal wakeups are disabled completely (see https://github.com/python/cpython/blob/187f76def8a5bd0af7ab512575cad30cfe624b05/Lib/asyncio/proactor_events.py#L679). The loop cannot be interrupted with Ctrl+C anymore (bad).

----------
components: Windows, asyncio
messages: 384979
nosy: asvetlov, hidmic, paul.moore, steve.dower, tim.golden, yselivanov, zach.ware
priority: normal
severity: normal
status: open
title: asyncio.ProactorEventLoop mishandles signal wakeup file descriptor
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42913>
_______________________________________

From report at bugs.python.org  Tue Jan 12 18:21:21 2021
From: report at bugs.python.org (Felipe)
Date: Tue, 12 Jan 2021 23:21:21 +0000
Subject: [New-bugs-announce] [issue42914] pprint numbers with underscore
Message-ID: <1610493681.29.0.683565759752.issue42914@roundup.psfhosted.org>


New submission from Felipe <felipe.nospam.ochoa at gmail.com>:

It would be nice if pprint learned to insert underscores in long numbers

Current behavior:

>>> pprint.pprint(int(1e9))
1000000000

Desired behavior

>>> pprint.pprint(int(1e9))
1_000_000_000

Wikipedia tells me that "groups of 3" is the international standard to be followed here [1][2]

[1] https://en.wikipedia.org/wiki/ISO_31-0#Numbers
[2] https://en.wikipedia.org/wiki/Decimal_separator#Digit_grouping

----------
components: Library (Lib)
messages: 384982
nosy: fov
priority: normal
severity: normal
status: open
title: pprint numbers with underscore
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42914>
_______________________________________

From report at bugs.python.org  Tue Jan 12 18:31:13 2021
From: report at bugs.python.org (Kevin Chen)
Date: Tue, 12 Jan 2021 23:31:13 +0000
Subject: [New-bugs-announce] [issue42915] enum.Flag ~ bitwise negation is
 very slow and can't be defined as a Flag value
Message-ID: <1610494273.49.0.733516574302.issue42915@roundup.psfhosted.org>


New submission from Kevin Chen <kevinjchen94 at gmail.com>:

Here's a code sample:

```
import time
from enum import Flag, auto


class MyFlag(Flag):
    NONE = 0
    FLAG_1 = auto()
    FLAG_2 = auto()
    FLAG_3 = auto()
    FLAG_4 = auto()
    FLAG_5 = auto()
    FLAG_6 = auto()
    #
    # NOT_FLAG_1_OR_2 = ~FLAG_1 & ~FLAG_2


def test_flag():
    f = MyFlag.NONE
    inverted = (
        ~MyFlag.FLAG_1
        & ~MyFlag.FLAG_2
        & ~MyFlag.FLAG_3
        & ~MyFlag.FLAG_4
        & ~MyFlag.FLAG_5
        & ~MyFlag.FLAG_6
    )
    return f & inverted


INVERTED = (
    ~MyFlag.FLAG_1
    & ~MyFlag.FLAG_2
    & ~MyFlag.FLAG_3
    & ~MyFlag.FLAG_4
    & ~MyFlag.FLAG_5
    & ~MyFlag.FLAG_6
)


def test_flag_cached():
    f = MyFlag.NONE
    return f & INVERTED


if __name__ == "__main__":
    start_time = time.time()
    for _ in range(10_000):
        test_flag()

    elapsed = time.time() - start_time
    print(f"Took normal {elapsed:2f} seconds.")

    start_time = time.time()
    for _ in range(10_000):
        test_flag_cached()

    elapsed = time.time() - start_time
    print(f"Took cached {elapsed:2f} seconds.")
```

And its outputs:
```
Took normal 1.799731 seconds.
Took cached 0.009488 seconds.
```

Basically, bitwise negation is very very slow. From what I can tell, it seems that a lot of time is spent here computing powers of two. I've read elsewhere that flag values are cached, and it looks like negated Flag values can't be cached? This seems related to the second issue, which is that any negated Flag value being defined results in `RecursionError: maximum recursion depth exceeded` as it searches for a good name for Flag.

Obviously, the simple workaround is just to define a constant variable elsewhere with the negated value, but it isn't very obvious anywhere that this is necessary, and I wanted to raise this to see if anyone has knowledge of the implementation details of Flag for possibly resolving this in the class itself.

----------
messages: 384983
nosy: aspin2
priority: normal
severity: normal
status: open
title: enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42915>
_______________________________________

From report at bugs.python.org  Tue Jan 12 19:28:46 2021
From: report at bugs.python.org (Charles Law)
Date: Wed, 13 Jan 2021 00:28:46 +0000
Subject: [New-bugs-announce] [issue42916] Support for DICOM image file
 format in imghdr module
Message-ID: <1610497726.98.0.199039272928.issue42916@roundup.psfhosted.org>


New submission from Charles Law <charles.law at gmail.com>:

DICOM is a file format used frequently in medical imaging (it is also a communications protocol). It has been used since the 80's, and is still widely used by modern medical equipment.

It has a well defined format: http://dicom.nema.org/dicom/2013/output/chtml/part10/chapter_7.html

This proposal is for the addiction of a check to imghdr module to detect DICOM files, with imghdr.what() and return 'dicom' if a dicom file is found.

----------
components: Library (Lib)
messages: 384989
nosy: claw
priority: normal
severity: normal
status: open
title: Support for DICOM image file format in imghdr module
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42916>
_______________________________________

From report at bugs.python.org  Tue Jan 12 20:54:09 2021
From: report at bugs.python.org (Thomas Anderson)
Date: Wed, 13 Jan 2021 01:54:09 +0000
Subject: [New-bugs-announce] [issue42917] Block stack size for frame objects
 should be dynamically sizable
Message-ID: <1610502849.77.0.30833881241.issue42917@roundup.psfhosted.org>


New submission from Thomas Anderson <tomkpz at gmail.com>:

Currently the block stack size is hardcoded to 20.  Similar to how the value stack is dynamically sizable, we should make the block stack dynamically sizable.  This will reduce space on average (since the typical number of blocks for a function is well below 20) and allow code generators to generate code with more deep nesting.  Note: the motivation is not necessarily to reduce memory usage, but to make L1 cache misses less likely for stack objects.

----------
components: Interpreter Core
messages: 384991
nosy: tomkpz
priority: normal
severity: normal
status: open
title: Block stack size for frame objects should be dynamically sizable
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42917>
_______________________________________

From report at bugs.python.org  Wed Jan 13 01:01:57 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Wed, 13 Jan 2021 06:01:57 +0000
Subject: [New-bugs-announce] [issue42918] Nested multi-line expression will
 lead to "compile()" fails
Message-ID: <1610517717.73.0.740461753555.issue42918@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

For build-in function compile() in mode 'single',  only single statement can be well compiled. If we compile multiple statements,  the parser will raise a syntaxError. Seeing the following two programs,  In program 1, 2 statement are compiled. So the parser raises a Syntax error. It's the expected output. However, if we insert a nested multi-line assignment in this code(see program 2), 3 statements are compiled. We expect the parser also raise a Sytax error. But it' not.

Program 1 ( with expected results)
===================================
code1 = """
a = 1
b = 2
"""
c = compile(code1, "", "single")
===================================
Traceback (most recent call last):
  File "/home/xxm/Desktop/nameChanging/report/test1.py", line 641, in <module>
    c = compile(code1, "", "single")
  File "", line 2
    a = 1
         ^
SyntaxError: multiple statements found while compiling a single statement


Program 2 (with unexpected results)
=================================
code2 = """
c ='''
d=1
'''
a = 1
b = 2
"""
c = compile(code2, "", "single")
=================================


Expected out for program 2: Raise a syntaxError too? But it's not.

>> python -V
Python 3.10.0a2
>>uname -a
Linux xxm-System-Product-Name 4.15.0-64-generic #73~16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

----------
messages: 384995
nosy: xxm
priority: normal
severity: normal
status: open
title: Nested multi-line expression will lead to "compile()" fails
type: compile error
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42918>
_______________________________________

From report at bugs.python.org  Wed Jan 13 01:05:45 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Wed, 13 Jan 2021 06:05:45 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue42919=5D_Blank_in_multilin?=
 =?utf-8?q?e_=E2=80=9Cif_expressions=E2=80=9D_will_lead_to_EOF_errors?=
Message-ID: <1610517945.87.0.376741027563.issue42919@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

In build-in function compile() of mode 'single',  single statement can be well compiled. (see program 1 and program 2). However,  if we leave 4 blank spaces before the end of triple-quotation mark of "if expression",(see program 3), The parser will raise syntax error. This is not the expected output.  Spaces in the end of multi-line expression should not affect the compilation results. No error should be reported in this case.


program 1 (expected)
===============================
snippet ="""
a = 1
     """
compile(snippet, "", "single")
===============================

program 2 (expected)
===============================
snippet ="""
if True:
     a = 1
"""
compile(snippet, "", "single")
===============================

program 3 (unexpected)
===========================
snippet ="""
if True:
     a = 1
     """
compile(snippet, "", "single")
===========================
Traceback (most recent call last):
     File "/home/xxm/Desktop/nameChanging/report/test1.py", line 1, in <module>
         compile(snippet, "", "single")
     File "", line 4
SyntaxError: unexpected EOF while parsing

Expected output: No error reported (like program1 and program2)


>> python -V
Python 3.10.0a2
>>uname -a
Linux xxm-System-Product-Name 4.15.0-64-generic #73~16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

----------
components: Interpreter Core
messages: 384996
nosy: xxm
priority: normal
severity: normal
status: open
title: Blank in multiline ?if expressions? will lead to EOF errors
type: compile error
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42919>
_______________________________________

From report at bugs.python.org  Wed Jan 13 02:18:44 2021
From: report at bugs.python.org (Aviral Srivastava)
Date: Wed, 13 Jan 2021 07:18:44 +0000
Subject: [New-bugs-announce] [issue42920] How to add end_lineno in pyclbr?
Message-ID: <1610522324.61.0.549863868319.issue42920@roundup.psfhosted.org>


New submission from Aviral Srivastava <avi.srivastava254084 at gmail.com>:

I created a PR a year ago, https://github.com/python/cpython/pull/16466/files, and now the source code is completely changed. How do I create another PR or if there is another way through which I can get an end line for each object (class and function) as stated here(https://stackoverflow.com/questions/58142456/how-to-get-the-scope-of-a-class-in-terms-of-starting-line-and-end-line-in-python).

----------
components: Library (Lib)
messages: 385002
nosy: kebab-mai-haddi
priority: normal
severity: normal
status: open
title: How to add end_lineno in pyclbr?
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42920>
_______________________________________

From report at bugs.python.org  Wed Jan 13 02:48:07 2021
From: report at bugs.python.org (Joseph Perez)
Date: Wed, 13 Jan 2021 07:48:07 +0000
Subject: [New-bugs-announce] [issue42921] Inferred Optional type of wrapper
 function arguments
Message-ID: <1610524087.97.0.666044800779.issue42921@roundup.psfhosted.org>


New submission from Joseph Perez <joperez at hotmail.fr>:

`typing.get_type_hints` gives a different result for a wrapper created with `functools.wraps` in case of inferred `Optional` arguments (when the default value of the argument is None)

```python
from functools import wraps
from typing import get_type_hints

def foo(bar: int = None): ...

@wraps(foo)
def foo2(*args, **kwargs): ...

print(get_type_hints(foo))  # {'bar': typing.Optional[int]}
print(get_type_hints(foo2))  # {'bar': <class 'int'>}
```

This is because `get_type_hints` use the defauts of the wrapper (`foo2`) and not those of the wrapped function (`foo`).
This is not consistent with some other tools like `inspect.signature` which gives the same signature (and thus same default argument) for the wrapped function and its wrapper.

I think this case has simply been forgotten in the resolution of https://bugs.python.org/issue37838 (fixing `get_type_hints` not taking `wraps` in account at all), because inferred `Optional` is a kind deprecated feature (https://github.com/python/typing/issues/275).

----------
messages: 385005
nosy: joperez
priority: normal
severity: normal
status: open
title: Inferred Optional type of wrapper function arguments
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42921>
_______________________________________

From report at bugs.python.org  Wed Jan 13 03:11:22 2021
From: report at bugs.python.org (Jose MONTES PEREZ)
Date: Wed, 13 Jan 2021 08:11:22 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue42922=5D_Enlace_a_descripc?=
 =?utf-8?q?i=C3=B3n_de_funci=C3=B3n_=27dir=27_faltante?=
Message-ID: <1610525482.8.0.875772925154.issue42922@roundup.psfhosted.org>


New submission from Jose MONTES PEREZ <jmontesp.informatica at gmail.com>:

En la tabla de enlaces a funciones de la p?gina:
https://docs.python.org/es/3/library/functions.html 
no existe un link a la funci?n 'dir' tal como:
https://docs.python.org/es/3/library/functions.html#dir
en su lugar est? duplicado el enlace a la funci?n 'any'

----------
assignee: docs at python
components: Documentation
messages: 385007
nosy: docs at python, jmontesp.informatica
priority: normal
severity: normal
status: open
title: Enlace a descripci?n de funci?n 'dir' faltante
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42922>
_______________________________________

From report at bugs.python.org  Wed Jan 13 05:50:13 2021
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 13 Jan 2021 10:50:13 +0000
Subject: [New-bugs-announce] [issue42923] Py_FatalError(): dump the list of
 extension modules
Message-ID: <1610535013.1.0.942204365805.issue42923@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

When Python cannot report an error and its is not possible to recover from the error, Py_FatalError() displays an error message and exit immediately Python.

It's common that Python crashes are coming from third party extension modules, rather than Python itself. Recent example: bpo-42891 crash reported in Python, but it was a bug in lsm-db third party extension module.

I propose to enhance Py_FatalError() to attempt to dump the list of extension modules, to ease debugging.

I'm working on a PR.

----------
components: Interpreter Core
messages: 385013
nosy: vstinner
priority: normal
severity: normal
status: open
title: Py_FatalError(): dump the list of extension modules
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42923>
_______________________________________

From report at bugs.python.org  Wed Jan 13 06:26:55 2021
From: report at bugs.python.org (tholl)
Date: Wed, 13 Jan 2021 11:26:55 +0000
Subject: [New-bugs-announce] [issue42924] bytearray_repeat copies from
 ob_bytes instead of ob_start
Message-ID: <1610537215.58.0.240765902048.issue42924@roundup.psfhosted.org>


New submission from tholl <py at any.tholl.xyz>:

`bytearray_repeat` uses `ob_bytes` rather than `ob_start` as its source data for copying, leading to incorrect results in some edge cases where the two are distinct. It should probably use `PyByteArray_AS_STRING(self)` just like `bytearray_irepeat` and basically all the other functions.

As far as I can see, these edge cases occur pretty much only after `bytearray_setslice_linear`, where `ob_start` can be adjusted without changing `ob_bytes`. 

A simple example (also attached as poc.py):

>>> ba = bytearray(b'0123456789abcdef')
>>> ba[:10] = b'test'
>>> print(ba)
bytearray(b'testabcdef')
>>> print(ba * 1)
bytearray(b'012345test')

I'll try to submit a PR for this later today.

----------
components: Interpreter Core
files: poc.py
messages: 385017
nosy: tholl
priority: normal
severity: normal
status: open
title: bytearray_repeat copies from ob_bytes instead of ob_start
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49739/poc.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42924>
_______________________________________

From report at bugs.python.org  Wed Jan 13 07:57:56 2021
From: report at bugs.python.org (ChenXuan)
Date: Wed, 13 Jan 2021 12:57:56 +0000
Subject: [New-bugs-announce] [issue42925] Error trace of else inside class
Message-ID: <1610542676.59.0.693705484765.issue42925@roundup.psfhosted.org>


New submission from ChenXuan <522169030 at qq.com>:

Hi, python3.10 seems to give wrong trace here:
class A:
   if 3 < 9:
      a = 1
   else:
      a = 2

result:
    2: class A:
    1:    if 3 < 9:
    1:       a = 1
          else:
    1:       a = 2
       

command: python3.10 -m trace --count -C . a1.py
environment: Linux 5.8.0-36-generic #40~20.04.1-Ubuntu SMP
version: python3.10.0a4

----------
files: a1.py
messages: 385023
nosy: ChenXuan
priority: normal
severity: normal
status: open
title: Error trace of else inside class
versions: Python 3.10
Added file: https://bugs.python.org/file49740/a1.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42925>
_______________________________________

From report at bugs.python.org  Wed Jan 13 10:03:24 2021
From: report at bugs.python.org (Mark Shannon)
Date: Wed, 13 Jan 2021 15:03:24 +0000
Subject: [New-bugs-announce] [issue42926] Split compiler into code-gen,
 optimizer and assembler.
Message-ID: <1610550204.0.0.923150494092.issue42926@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

Currently the compiler operates in three main passes:

Code-gen
Optimize
Assemble

The problem is that these passes use the same basic-block based CFG, leading to unnecessary coupling and inefficiencies.
A basic block CFG is awkward and error-prone for the code-gen, but not very efficient for the optimizer and assembler.

A better design would be for the code-gen to create a single linear sequence of instructions. The optimizer would take this and produce a list of extended-blocks for the assembler to consume.

code-gen -> (list of instructions) -> optimizer
optimizer -> (list of extended blocks) -> assembler

(Extended blocks have a single entry and multiple exits, unlike basic blocks which have a single entry and single exit)

This would:
1. Reduce memory use considerably (the size of instruction and block data structures would be about 60% of current)
2. Be faster (Less CFG management).
3. Produce better code (extended blocks are a better unit for optimization that basic blocks).
4. Be easier to maintain:
  a) Code-gen wouldn't have to worry about creating a correct CFG.
  b) The optimizer wouldn't need to handle empty blocks and track which basic blocks form an extended block.


Apart from the changes to the compiler, it would help if we made all branch instructions absolute (or have a backward dual) to accommodate free reordering of blocks in the optimizer.

----------
messages: 385033
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Split compiler into code-gen, optimizer and assembler.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42926>
_______________________________________

From report at bugs.python.org  Wed Jan 13 23:45:09 2021
From: report at bugs.python.org (Guido van Rossum)
Date: Thu, 14 Jan 2021 04:45:09 +0000
Subject: [New-bugs-announce] [issue42927] Inline cache for slots
Message-ID: <1610599509.18.0.893371992468.issue42927@roundup.psfhosted.org>


New submission from Guido van Rossum <guido at python.org>:

I've been thinking about Python performance improvements, and I played around with an inline cache enhancement that supports slots. The results on a very simple benchmark look promising (30% speedup) but I'm terrible with our benchmarking tools, and this should be considered *very* carefully before we go ahead with it, since it could potentially pessimize the inline cache for standard attributes.

I'll attach a PR in a minute.

----------
messages: 385063
nosy: gvanrossum, pablogsal, yselivanov
priority: normal
severity: normal
stage: test needed
status: open
title: Inline cache for slots
type: performance
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42927>
_______________________________________

From report at bugs.python.org  Thu Jan 14 10:53:34 2021
From: report at bugs.python.org (John Brearley)
Date: Thu, 14 Jan 2021 15:53:34 +0000
Subject: [New-bugs-announce] [issue42928] adafruit_ads1x15.ads1115 not
 comaptible with globals()[pkg_trunc] = importlib.import_module(pkg_trunc)
Message-ID: <1610639614.07.0.761554562908.issue42928@roundup.psfhosted.org>


New submission from John Brearley <brearley at bell.net>:

The first attached script ada_dbg1.py.txt which uses simple hardcoded import statements, the creation of ads1115 objects work fine. The second attached script ada_dbg2.py.txt uses a loop to import a variable list of packages, which allows for more graceful error handling & messaging.

globals()[pkg_trunc] = importlib.import_module(pkg_trunc)

The high level observation is that board, busio & adafruit_extended_bus objects continue to be correctly created after the import loop is done. Its just adafruit_ads1x15.ads1115 that does not like this import method.

The same line of code creating ads10 gets error:
Traceback (most recent call last):
File "ada_dbg2.py", line 33, in
ads10 = adafruit_ads1x15.ads1115.ADS1115(i2c_1, address=0x48)
AttributeError: module 'adafruit_ads1x15' has no attribute 'ads1115'

When I look at the global variables, from ads_dbg1.py, I see:
'adafruit_ads1x15': <module 'adafruit_ads1x15' from '/home/pi/.local/lib/python3.7/site-packages/adafruit_ads1x15/init.py'>,

When I look at the global variables, from ads_dbg2.py, I see:
'adafruit_ads1x15.ads1115': <module 'adafruit_ads1x15.ads1115' from '/home/pi/.local/lib/python3.7/site-packages/adafruit_ads1x15/ads1115.py'>,

So I tried doing the import without the .ads1115 appended, & got:
'adafruit_ads1x15': <module 'adafruit_ads1x15' from '/home/pi/.local/lib/python3.7/site-packages/adafruit_ads1x15/init.py'>,

But I still get an error.
Traceback (most recent call last):
File "ada_dbg2.py", line 33, in
ads10 = adafruit_ads1x15.ads1115.ADS1115(i2c_1, address=0x48)
AttributeError: module 'adafruit_ads1x15' has no attribute 'ads1115'

I went through all 8 permutations of with/without the .ads1115 in the importlib.import_module statement, no luck.

Comments?

----------
components: Interpreter Core
files: ada_dbg1.py
messages: 385069
nosy: jbrearley
priority: normal
severity: normal
status: open
title: adafruit_ads1x15.ads1115 not comaptible with globals()[pkg_trunc] = importlib.import_module(pkg_trunc)
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49741/ada_dbg1.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42928>
_______________________________________

From report at bugs.python.org  Thu Jan 14 16:29:10 2021
From: report at bugs.python.org (fireattack)
Date: Thu, 14 Jan 2021 21:29:10 +0000
Subject: [New-bugs-announce] [issue42929] On Windows,
 shutil.move doesn't raise FileExistsError if dst exists like
 os.rename
Message-ID: <1610659750.09.0.0459892919552.issue42929@roundup.psfhosted.org>


New submission from fireattack <human.peng at gmail.com>:

According to https://docs.python.org/3.8/library/shutil.html#shutil.move

"If the destination already exists but is not a directory, it may be overwritten depending on os.rename() semantics."

I interpret "depending on os.rename() semantics" to mean it will follow os.rename()'s behavior.

According to https://docs.python.org/3/library/os.html#os.rename

"On Windows, if dst exists a FileExistsError is always raised."

However, their behaviors are not the same.

For os.rename, it does raise FileExistsError if dst exists.
For shutil.move, it silently overwrites dst.

It's either a bug in behavior of shutil.move, or the documentation need to be updated.

----------
components: Library (Lib)
messages: 385083
nosy: fireattack
priority: normal
severity: normal
status: open
title: On Windows, shutil.move doesn't raise FileExistsError if dst exists like os.rename
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42929>
_______________________________________

From report at bugs.python.org  Thu Jan 14 17:04:58 2021
From: report at bugs.python.org (Michael XU)
Date: Thu, 14 Jan 2021 22:04:58 +0000
Subject: [New-bugs-announce] [issue42930] xml.parsers.expat results differ
 buffer_text and / or buffer_size
Message-ID: <1610661898.66.0.880926471495.issue42930@roundup.psfhosted.org>


New submission from Michael XU <mfxecon at gmail.com>:

More details available here:
https://stackoverflow.com/questions/65676934/python-xml-parsers-expat-results-differ-based-on-buffer-text-true-or-false-and

Raw data to replicate the issue is available upon request.

----------
components: XML
messages: 385086
nosy: mfxuus
priority: normal
severity: normal
status: open
title: xml.parsers.expat results differ buffer_text and / or buffer_size
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42930>
_______________________________________

From report at bugs.python.org  Thu Jan 14 18:24:01 2021
From: report at bugs.python.org (Setrak Balian)
Date: Thu, 14 Jan 2021 23:24:01 +0000
Subject: [New-bugs-announce] [issue42931] Include randbytes in random.__all__
Message-ID: <1610666641.18.0.781615576275.issue42931@roundup.psfhosted.org>


Change by Setrak Balian <setrak at faculty.ai>:


----------
components: Library (Lib)
nosy: sbalian
priority: normal
severity: normal
status: open
title: Include randbytes in random.__all__
type: behavior
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42931>
_______________________________________

From report at bugs.python.org  Thu Jan 14 23:53:43 2021
From: report at bugs.python.org (lizhu)
Date: Fri, 15 Jan 2021 04:53:43 +0000
Subject: [New-bugs-announce] [issue42932] How to handle import tripartite
 pacakge issue in unittest scripts for python expect mock.Mock
Message-ID: <1610686423.78.0.418946884784.issue42932@roundup.psfhosted.org>


New submission from lizhu <2541058309 at qq.com>:

Hi, all
When i use unittest to test my package which import other pacakge, however other pacakge use more and more tripartite pacakges, and it has an impact on test case, example:

occur an err: moduleNotFindErro

how to fix this issue?

----------
components: Tests
messages: 385098
nosy: lizhu
priority: normal
severity: normal
status: open
title: How to handle import tripartite pacakge  issue in unittest scripts for python expect mock.Mock
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42932>
_______________________________________

From report at bugs.python.org  Fri Jan 15 03:32:18 2021
From: report at bugs.python.org (Kkshitish)
Date: Fri, 15 Jan 2021 08:32:18 +0000
Subject: [New-bugs-announce] [issue42933] OUTPUT WRONG
Message-ID: <1610699538.98.0.545500481144.issue42933@roundup.psfhosted.org>


New submission from Kkshitish <kkshitish at yahoo.com>:

try:
  
  a = 10
  
  if a == 11:
    
    print ("Yes")
    
except:
  
  print ("Out")
  
else:
  
  print ("No")


Output: No

The output should be "Out" but it print "No", The else statement should be print when try condition is execute. But the try condition is not execute on program. But still it print else statement. 

This is not ideal output. If you are disagree with me. So, kindly provide very strong argument over this.

----------
files: main.py
messages: 385100
nosy: Kkshitish
priority: normal
severity: normal
status: open
title: OUTPUT WRONG
type: security
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49745/main.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42933>
_______________________________________

From report at bugs.python.org  Fri Jan 15 06:06:03 2021
From: report at bugs.python.org (Irit Katriel)
Date: Fri, 15 Jan 2021 11:06:03 +0000
Subject: [New-bugs-announce] [issue42934] use TracebackException's new
 compact param in unittest.TestResult
Message-ID: <1610708763.72.0.663493099071.issue42934@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at yahoo.com>:

The TracebackException instance created here
https://github.com/python/cpython/blob/63298930fb531ba2bb4f23bc3b915dbf1e17e9e1/Lib/unittest/result.py#L185
is only used for format(), so it can benefit from the compact param added in issue42877.

----------
components: Library (Lib)
messages: 385105
nosy: iritkatriel
priority: normal
severity: normal
status: open
title: use TracebackException's new compact param in unittest.TestResult
type: performance
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42934>
_______________________________________

From report at bugs.python.org  Fri Jan 15 07:05:38 2021
From: report at bugs.python.org (Pete Wicken)
Date: Fri, 15 Jan 2021 12:05:38 +0000
Subject: [New-bugs-announce] [issue42935] Pickle can't import builtins at
 exit
Message-ID: <1610712338.22.0.647900497597.issue42935@roundup.psfhosted.org>


New submission from Pete Wicken <petewicken at gmail.com>:

Originally found as an issue in Lib/shelve.py; if we attempt to pickle a builtin as the program is exiting then Modules/_pickle.c will fail at the point of the PyImport_Import in save_global. In CPython3.8 this causes a segfault, in CPython3.9 a PicklingError is raised.

This is especially problematic in shelve.py as object pickling is attempted by the __del__ method's call stack when writeback=True. Therefore if the program exits before an explicit sync is called; in 3.8 the data will not be written to disk and a segfault occurs; in 3.9 the data is written to disk, but with an uncaught exception being raised.


Exception demonstrated via shelve on 3.9.1 on MacOS with Clang:

Python 3.9.1 (default, Dec 10 2020, 11:11:14)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open('testing', writeback=True)
>>> s['a'] = Exception
>>> exit()
Exception ignored in: <function Shelf.__del__ at 0x10ed67790>
Traceback (most recent call last):
  File ".../3.9/lib/python3.9/shelve.py", line 162, in __del__
  File ".../3.9/lib/python3.9/shelve.py", line 144, in close
  File ".../3.9/lib/python3.9/shelve.py", line 168, in sync
  File ".../3.9/lib/python3.9/shelve.py", line 124, in __setitem__
_pickle.PicklingError: Can't pickle <class 'Exception'>: import of module 'builtins' failed


Segfault demonstrated via shelve on 3.8.5 on MacOS with Clang (different system from above):

Python 3.8.5 (v3.8.5:580fbb018f, Jul 20 2020, 12:11:27)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open('testing', writeback=True)
>>> s['a'] = Exception
>>> exit()
[1]    10040 segmentation fault  python3.8


Exception demonstrated via shelve on 3.9.1 on RHEL with GCC:

Python 3.9.1 (default, Dec  8 2020, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open("thing", writeback=True)
>>> s['a'] = Exception
>>> 
Exception ignored in: <function Shelf.__del__ at 0x7f3f30b7c820>
Traceback (most recent call last):
  File "/usr/lib64/python3.9/shelve.py", line 162, in __del__
  File "/usr/lib64/python3.9/shelve.py", line 144, in close
  File "/usr/lib64/python3.9/shelve.py", line 168, in sync
  File "/usr/lib64/python3.9/shelve.py", line 124, in __setitem__
_pickle.PicklingError: Can't pickle <class 'Exception'>: import of module 'builtins' failed


Code example to reproduce using Pickle in the class __del__, demonstrated on a RHEL system:

Python 3.9.1 (default, Dec  8 2020, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pickle import DEFAULT_PROTOCOL, Pickler
>>> from io import BytesIO
>>> class T:
...   def __del__(self):
...     f = BytesIO()
...     p = Pickler(f, DEFAULT_PROTOCOL)
...     p.dump(sum)
... 
>>> t = T()
>>> exit()
Exception ignored in: <function T.__del__ at 0x7f5f04d9ef70>
Traceback (most recent call last):
  File "<stdin>", line 5, in __del__
_pickle.PicklingError: Can't pickle <built-in function sum>: import of module 'builtins' failed


Have not tested on 3.6, 3.7 or 3.10.

----------
components: Library (Lib)
messages: 385110
nosy: Wicken
priority: normal
severity: normal
status: open
title: Pickle can't import builtins at exit
type: crash
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42935>
_______________________________________

From report at bugs.python.org  Fri Jan 15 10:33:47 2021
From: report at bugs.python.org (Jens)
Date: Fri, 15 Jan 2021 15:33:47 +0000
Subject: [New-bugs-announce] [issue42936] Decimal module performs wrong
 floor division with negative numbers
Message-ID: <1610724827.91.0.138727691523.issue42936@roundup.psfhosted.org>


New submission from Jens <multiks2200 at gmail.com>:

from decimal import Decimal

print(-0.9//0.123)
# prints -8.0
print(Decimal('-0.9')//Decimal('0.123'))
# prints -7
print(-10//4.2)
# prints -3.0
print(Decimal('-10')//Decimal('4.2'))
# prints -2

----------
messages: 385113
nosy: multiks2200
priority: normal
severity: normal
status: open
title: Decimal module performs wrong floor division with negative numbers
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42936>
_______________________________________

From report at bugs.python.org  Fri Jan 15 18:00:38 2021
From: report at bugs.python.org (CDirkx)
Date: Fri, 15 Jan 2021 23:00:38 +0000
Subject: [New-bugs-announce] [issue42937] 192.0.0.8 (IPv4 dummy address)
 considered globally reachable
Message-ID: <1610751638.64.0.309759393541.issue42937@roundup.psfhosted.org>


New submission from CDirkx <christiaan at dirkx.email>:

Currently the method `ipaddress.is_global` returns true for the IPv4 address 192.0.0.8. This was correct when `is_global` was initially implemented, but in 2015 192.0.0.8 got designated as the "IPv4 dummy address" and not globally reachable, see the IANA special purpose address registry: https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml.

The correct behaviour now, according to the documentation of `is_global` which refers to the IANA registry, would be to return false for 192.0.0.8.

----------
messages: 385127
nosy: cdirkx
priority: normal
severity: normal
status: open
title: 192.0.0.8 (IPv4 dummy address) considered globally reachable
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42937>
_______________________________________

From report at bugs.python.org  Sat Jan 16 03:03:27 2021
From: report at bugs.python.org (Jordy Zomer)
Date: Sat, 16 Jan 2021 08:03:27 +0000
Subject: [New-bugs-announce] [issue42938] ctypes double representation BoF
Message-ID: <1610784207.02.0.722384660803.issue42938@roundup.psfhosted.org>


New submission from Jordy Zomer <jordy at simplyhacker.com>:

Hi, 

There's a buffer overflow in the PyCArg_repr() function in _ctypes/callproc.c.

The buffer overflow happens due to not checking the length of th sprintf() function on line: 

    case 'd':
        sprintf(buffer, "<cparam '%c' (%f)>",
            self->tag, self->value.d);
        break;

Because we control self->value.d we could make it copy _extreme_ values. For example we could make it copy 1e300 which would be a 1 with 300 zero's  to overflow the buffer.

This could potentially cause RCE when a user allows untrusted input in these functions.

A minimal PoC:

>>> from ctypes import *
>>> c_double.from_param(1e300)
*** buffer overflow detected ***: terminated
Aborted


I recommend __always__ controlling how much you copy so I'd use snprintf with a size argument instead.

Best Regards,

Jordy Zomer

----------
components: ctypes
messages: 385136
nosy: JordyZomer
priority: normal
severity: normal
status: open
title: ctypes double representation BoF
type: security
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42938>
_______________________________________

From report at bugs.python.org  Sat Jan 16 09:25:34 2021
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Sat, 16 Jan 2021 14:25:34 +0000
Subject: [New-bugs-announce] [issue42939] Linux's chattr i.e.
 ioctl(FS_IOC_SETFLAGS) is not supported in os.chflags()
Message-ID: <1610807134.64.0.859060580807.issue42939@roundup.psfhosted.org>


New submission from ???? ????????? <socketpair at gmail.com>:

https://man7.org/linux/man-pages/man2/ioctl_iflags.2.html

Seems the ioctl is only way to implement chattr(). I propose to add all these flags to os.chflags() and to gain its support in Linux.

----------
components: Library (Lib)
messages: 385141
nosy: socketpair
priority: normal
severity: normal
status: open
title: Linux's chattr i.e. ioctl(FS_IOC_SETFLAGS) is not supported in os.chflags()
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42939>
_______________________________________

From report at bugs.python.org  Sat Jan 16 11:30:52 2021
From: report at bugs.python.org (Slava Kostrov)
Date: Sat, 16 Jan 2021 16:30:52 +0000
Subject: [New-bugs-announce] [issue42940] Incorrect behavior of
 inspect.signature(f).bind
Message-ID: <1610814652.94.0.812646879341.issue42940@roundup.psfhosted.org>


New submission from Slava Kostrov <slavkostrov at gmail.com>:

>>> def foo(a, /, b=1, **kwargs): pass
...
>>> foo(a=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() missing 1 required positional argument: 'a'

>>> inspect.signature(foo).bind(a=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "\Python\Python39\lib\inspect.py", line 3062, in bind
    return self._bind(args, kwargs)
  File "\Python\Python39\lib\inspect.py", line 2958, in _bind
    raise TypeError(msg) from None
TypeError: 'a' parameter is positional only, but was passed as a keyword

There is a different behaviour between a simple function call and inspect.signature(foo).bind - different TypeError messages.

I think inspect should check for VARKEYWORDS before throwing an error and throw only if there are not.

----------
messages: 385143
nosy: slavkostrov
priority: normal
severity: normal
status: open
title: Incorrect behavior of inspect.signature(f).bind
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42940>
_______________________________________

From report at bugs.python.org  Sat Jan 16 12:10:09 2021
From: report at bugs.python.org (Hingyuen Lam)
Date: Sat, 16 Jan 2021 17:10:09 +0000
Subject: [New-bugs-announce] [issue42941] Infinite loop in asyncio sslproto
Message-ID: <1610817009.05.0.00842749014089.issue42941@roundup.psfhosted.org>


New submission from Hingyuen Lam <linxy95 at gmail.com>:

There is an infinite loop in feed_appdata() of sslproto.py.
The OpenSSL documents said when some fatal I/O error occurred, SSL_ERROR_SYSCALL is returned and no further I/O operations should be performed. This error is ignored in the exception handling of feed_appdata(), and the while loop will never be break.
This error should be raised, the same for feed_ssldata().

----------
components: asyncio
messages: 385144
nosy: asvetlov, linxy95, yselivanov
priority: normal
severity: normal
status: open
title: Infinite loop in asyncio sslproto
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42941>
_______________________________________

From report at bugs.python.org  Sat Jan 16 16:33:20 2021
From: report at bugs.python.org (Arnim Rupp)
Date: Sat, 16 Jan 2021 21:33:20 +0000
Subject: [New-bugs-announce] [issue42942] Feature request: Add decdigest()
 to hashlib
Message-ID: <1610832800.98.0.200773545993.issue42942@roundup.psfhosted.org>


New submission from Arnim Rupp <python at rupp.de>:

Problem: hashlib only offers digest() and hexdigest() but the fastest way to work with hashes is as integer.

The first thing loki does after getting the hashes is to convert them to int:
md5, sha1, sha256 = generateHashes(fileData)
                        md5_num=int(md5, 16)
                        sha1_num=int(sha1, 16)
                        sha256_num=int(sha256, 16)
https://github.com/Neo23x0/Loki/blob/master/loki.py

All the ~50000 hashes to compare are also converted to int after reading them from a file. The comparison is about twice as fast compared to hexdigest in strings because it uses just half the memory. 

(The use case here is to compare these 50,000 hashes to the hashes of all the 200,000 files on a system that gets scanned for malicious files.)

Solution: Add decdigest() to hashlib which returns the int version of the hash. This has 2 advantages: 
1. It saves the time for converting the hash to hex and back
2. Having decdigest() in the documentation inspires more programmers to work with hashes as int opposed to slow strings (where it's performance relevant.)

Should be just few lines of code for each algorithm, I could do the PR.

static PyObject *
_sha3_shake_128_hexdigest(SHA3object *self, PyObject *arg)
{
    PyObject *return_value = NULL;
    unsigned long length;

    if (!_PyLong_UnsignedLong_Converter(arg, &length)) {
        goto exit;
    }
    return_value = _sha3_shake_128_hexdigest_impl(self, length);

https://github.com/python/cpython/blob/63298930fb531ba2bb4f23bc3b915dbf1e17e9e1/Modules/_sha3/clinic/sha3module.c.h

----------
components: Library (Lib)
messages: 385150
nosy: 2d4d
priority: normal
severity: normal
status: open
title: Feature request: Add decdigest() to hashlib
type: performance
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42942>
_______________________________________

From report at bugs.python.org  Sat Jan 16 16:46:19 2021
From: report at bugs.python.org (Ilya Kulakov)
Date: Sat, 16 Jan 2021 21:46:19 +0000
Subject: [New-bugs-announce] [issue42943] singledispatchmethod should expose
 registry of all known overloads
Message-ID: <1610833579.12.0.993134178442.issue42943@roundup.psfhosted.org>


New submission from Ilya Kulakov <kulakov.ilya at gmail.com>:

The wrapper created by singledispatchmethod does not (trivially) expose registry of all known overloads.
Consider the following example:


    @singledispatchmethod
    def on_message(message):
        raise NotImplementedError

    @on_message.register
    def _(message: MessageA): ...

    @on_message.register
    def _(message: MessageB): ...

    loop = ...
    condition = ...
    messages = []

    def receive_message_on_thread(message):
        if ...:  # only signal to asyncio if message can be handled
            messages.append(message)
            loop.call_soon_threadsafe(condition.notify_all)
        else:
            ...

    async def process_messages_on_asyncio():
        while True:
            await condition.wait_for(lambda: len(messages) > 0)

            while len(messages):
                m = messages.pop(0)
                ...


Here messages are delivered outside of asyncio in a separate thread and thus they need to be forwarded into asyncio for further processing in the app. If the flow of messages is high it is desirable to filter inside the thread handler before signaling asyncio.

There are multiple approaches to describe which messages can be handled by the asyncio processor, but singledispatchmethod is the most elegant as it allows to keep everything in one place without if/else or code duplication.

Having a registry (trivially) exposed would allow to write the condition as `isinstance(message, tuple(on_message.registry.keys()))`.

----------
components: Library (Lib)
messages: 385151
nosy: Ilya.Kulakov, lukasz.langa, methane, ncoghlan
priority: normal
severity: normal
status: open
title: singledispatchmethod should expose registry of all known overloads
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42943>
_______________________________________

From report at bugs.python.org  Sat Jan 16 17:58:54 2021
From: report at bugs.python.org (Jon FRANCO)
Date: Sat, 16 Jan 2021 22:58:54 +0000
Subject: [New-bugs-announce] [issue42944] random.Random.sample bug when
 counts is not None
Message-ID: <1610837934.67.0.783808477389.issue42944@roundup.psfhosted.org>


New submission from Jon FRANCO <jonani.franco at hotmail.fr>:

Hello,

If I am reading right, random.Random.sample method has a bug if counts is not None.

Line 482 (of master):
"""
            selections = sample(range(total), k=k)
"""
this is calling the module function 'sample' and not the instance method.

IMO this line should be:
"""
            selections = self.sample(range(total), k=k)
"""

Python 3.9.1 (default, Dec 11 2020, 14:32:07) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from random import Random
>>> r = Random()
>>> r.seed('bug')
>>> r.sample(range(50), 5, counts=range(1,51))
[34, 39, 31, 42, 38]
>>> r.seed('bug')
>>> r.sample(range(50), 5, counts=range(1,51))
[39, 11, 3, 12, 29]  <====== this should be [34, 39, 31, 42, 38]
>>> r.seed('nobug')
>>> r.sample(range(50), 5)
[0, 30, 23, 17, 49]
>>> r.seed('nobug')
>>> r.sample(range(50), 5)
[0, 30, 23, 17, 49]

Regards,
Jon FRANCO

----------
components: Library (Lib)
messages: 385152
nosy: jonfranco, rhettinger
priority: normal
severity: normal
status: open
title: random.Random.sample bug when counts is not None
type: behavior
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42944>
_______________________________________

From report at bugs.python.org  Sat Jan 16 23:12:18 2021
From: report at bugs.python.org (Ryan Heisler)
Date: Sun, 17 Jan 2021 04:12:18 +0000
Subject: [New-bugs-announce] [issue42945] weakref.finalize documentation
 contradicts itself RE: finalizer callback or args referencing object
Message-ID: <1610856738.09.0.530478209911.issue42945@roundup.psfhosted.org>


New submission from Ryan Heisler <ryan.a.heisler at gmail.com>:

In the documentation for `weakref.finalize` (https://docs.python.org/3.9/library/weakref.html#weakref.finalize), it says:

"Note It is important to ensure that func, args and kwargs do not own any references to obj, either directly or indirectly, since otherwise obj will never be garbage collected. In particular, func should not be a bound method of obj."

However, at the bottom of the document, in the section called "Comparing finalizers with __del__() methods" (https://docs.python.org/3.8/library/weakref.html#comparing-finalizers-with-del-methods), the following code is part of an example of how to use `weakref.finalize`:

class TempDir:
    def __init__(self):
        self.name = tempfile.mkdtemp()
        self._finalizer = weakref.finalize(self, shutil.rmtree, self.name)

I believe this code violates the rule that func, args, and kwargs should not have a reference to obj. In the example, obj is the instance of TempDir, and one of the arguments to finalize's callback is an attribute of the same instance of TempDir.

I do not know how to fix this example code. I found it while trying to figure out how to use `weakref.finalize`.

----------
assignee: docs at python
components: Documentation
messages: 385155
nosy: docs at python, ryan.a.heisler
priority: normal
severity: normal
status: open
title: weakref.finalize documentation contradicts itself RE: finalizer callback or args referencing object
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42945>
_______________________________________

From report at bugs.python.org  Sun Jan 17 12:14:51 2021
From: report at bugs.python.org (Florian Bruhin)
Date: Sun, 17 Jan 2021 17:14:51 +0000
Subject: [New-bugs-announce] [issue42946] email: ValueError in get_section
 when parsing header with non-ASCII digit
Message-ID: <1610903691.7.0.354788620236.issue42946@roundup.psfhosted.org>


New submission from Florian Bruhin <python.org at the-compiler.org>:

Found mostly by accident:

>>> import email.headerregistry
>>> reg = email.headerregistry.HeaderRegistry()
>>> h = reg('Content-Disposition', 'inline; 0*?')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/email/headerregistry.py", line 608, in __call__
    return self[name](name, value)
  File "/usr/lib/python3.10/email/headerregistry.py", line 196, in __new__
    cls.parse(value, kwds)
  File "/usr/lib/python3.10/email/headerregistry.py", line 452, in parse
    kwds['parse_tree'] = parse_tree = cls.value_parser(value)
  File "/usr/lib/python3.10/email/_header_value_parser.py", line 2705, in parse_content_disposition_header
    disp_header.append(parse_mime_parameters(value[1:]))
  File "/usr/lib/python3.10/email/_header_value_parser.py", line 2569, in parse_mime_parameters
    token, value = get_parameter(value)
  File "/usr/lib/python3.10/email/_header_value_parser.py", line 2431, in get_parameter
    token, value = get_section(value)
  File "/usr/lib/python3.10/email/_header_value_parser.py", line 2384, in get_section
    section.number = int(digits)
ValueError: invalid literal for int() with base 10: '?'

This probably happens because:

>>> '?'.isdigit()
True
>>> int('?')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '?'

----------
components: Library (Lib)
messages: 385162
nosy: The Compiler, barry, maxking, r.david.murray
priority: normal
severity: normal
status: open
title: email: ValueError in get_section when parsing header with non-ASCII digit
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42946>
_______________________________________

From report at bugs.python.org  Sun Jan 17 12:28:07 2021
From: report at bugs.python.org (Florian Bruhin)
Date: Sun, 17 Jan 2021 17:28:07 +0000
Subject: [New-bugs-announce] [issue42947] email: Handling when both
 extended/ascii parameter (filename*/filename) are present?
Message-ID: <1610904487.73.0.582828355991.issue42947@roundup.psfhosted.org>


New submission from Florian Bruhin <python.org at the-compiler.org>:

Consider this reproducer:

    >>> import email.headerregistry
    >>> reg = email.headerregistry.HeaderRegistry()
    >>> parsed = reg('Content-Disposition', """attachment; filename="foo-ae.html"; filename*=UTF-8''foo-%c3%a4.html""")
    >>> parsed.defects
    (InvalidHeaderDefect('duplicate parameter name; duplicate(s) ignored'),)
    >>> parsed.params['filename']
    'foo-ae.html'

However, the relevant section of RFC 5987 says:

https://greenbytes.de/tech/webdav/rfc5987.html#rfc.section.4.2

> This specification suggests that a parameter using the extended syntax takes precedence. This would allow producers to use both formats without breaking recipients that do not understand the extended syntax yet.

And RFC 6266 says:

https://greenbytes.de/tech/webdav/rfc6266.html#rfc.section.4.3

> Many user agent implementations predating this specification do not understand the "filename*" parameter. Therefore, when both "filename" and "filename*" are present in a single header field value, recipients should pick "filename*" and ignore "filename". This way, senders can avoid special-casing specific user agents by sending both the more expressive "filename*" parameter, and the "filename" parameter as fallback for legacy recipients (see Section 5 for an example).

Also see the related attfnboth and attfnboth2 test cases here:
http://test.greenbytes.de/tech/tc2231/#attfnboth

I'm aware those two RFCs are specific to HTTP - but given that there's a "HTTP" policy and "utils.py" has some HTTP-specific date/time handling, I suppose correct handling of this might be in scope as well?

----------
components: Library (Lib)
messages: 385163
nosy: The Compiler
priority: normal
severity: normal
status: open
title: email: Handling when both extended/ascii parameter (filename*/filename) are present?
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42947>
_______________________________________

From report at bugs.python.org  Sun Jan 17 13:41:52 2021
From: report at bugs.python.org (wim glenn)
Date: Sun, 17 Jan 2021 18:41:52 +0000
Subject: [New-bugs-announce] [issue42948] bytearray.copy is undocumented
Message-ID: <1610908912.39.0.795030432727.issue42948@roundup.psfhosted.org>


New submission from wim glenn <wim.glenn at gmail.com>:

bytearray type has a copy method which seems to be undocumented

https://docs.python.org/3/library/stdtypes.html#bytes-and-bytearray-operations

----------
assignee: docs at python
components: Documentation
messages: 385164
nosy: docs at python, wim.glenn
priority: normal
severity: normal
status: open
title: bytearray.copy is undocumented
type: enhancement
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42948>
_______________________________________

From report at bugs.python.org  Sun Jan 17 17:10:12 2021
From: report at bugs.python.org (Freek Dijkstra)
Date: Sun, 17 Jan 2021 22:10:12 +0000
Subject: [New-bugs-announce] [issue42949] pdb & multiprocessing.Pool:
 AttributeError: module '__main__' has no attribute '__spec__'
Message-ID: <1610921412.78.0.54587473242.issue42949@roundup.psfhosted.org>


New submission from Freek Dijkstra <freek at macfreek.nl>:

Summary:

multiprocessing.Pool contains a bug when the script is invoked with pdb.



Steps to reproduce:

Consider the following script:

```
from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(f, [1, 2, 3]))
```

When called as `python3.10 test.py`, this works fine, and returns `[1, 4, 9]`

When called as `python3.10 -m pdb test.py`, it gives the following exception:

```
freek at minnie ~ ? python3.9 -m pdb test.py
> /Users/freek/test.py(3)<module>()
-> from multiprocessing import Pool
(Pdb) c
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pdb.py", line 1704, in main
    pdb._runscript(mainpyfile)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pdb.py", line 1573, in _runscript
    self.run(statement)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/bdb.py", line 580, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "/Users/freek/test.py", line 3, in <module>
    from multiprocessing import Pool
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 183, in get_preparation_data
    main_mod_name = getattr(main_module.__spec__, "name", None)
AttributeError: module '__main__' has no attribute '__spec__'
```

I can reproduce this in Python 3.8.7, 3.9.1, 3.10.0a3. It works fine in Python 3.7.9.

A workaround is to define `__spec__` in __main__:

```
from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    __spec__ = None
    with Pool(5) as p:
        print(p.map(f, [1, 2, 3]))
```

I'd say that multiprocessing/spawn.py", line 183, in get_preparation_data is flawed
    main_mod_name = getattr(main_module.__spec__, "name", None)

----------
components: Library (Lib)
messages: 385168
nosy: macfreek
priority: normal
severity: normal
status: open
title: pdb & multiprocessing.Pool: AttributeError: module '__main__' has no attribute '__spec__'
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42949>
_______________________________________

From report at bugs.python.org  Sun Jan 17 23:14:44 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Mon, 18 Jan 2021 04:14:44 +0000
Subject: [New-bugs-announce] [issue42950] Incorrect exception behavior in
 handling recursive call.
Message-ID: <1610943284.97.0.842137525249.issue42950@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

Seeing the following programs, we try to catch a recursive call error in exception handling.  The behaviors between Python 3.10.0a4 and older version are inconsistent. The outputs are attached in the end. The output on Python 3.10.0a4 is very weird. Two "print statements" lie in same "except" block to print "exception info" and a string "kk". "kk" is printed once while "exception info" is printed twice! I think a bug probably exists in Python 3.10.0a4 parser on handling stacks. 


=========================
def foo(c):
         try:
             c = c + 1
             print("ss"+str(c))
             foo(c)
         except Exception as e:
             print(str(e))
             print("kk")
         print(c)
c = 0
foo(c)
=========================

Output in Python 3.10.0a2 and older version(expected)
------------------------------------------------------------------------
ss1
ss2
....
ss996
maximum recursion depth exceeded while calling a Python object
kk
996
995
......
3
2
1
------------------------------------------------------------------------

Output in Python 3.10.0a4 (unexpected)
------------------------------------------------------------------------
ss1
ss2
....
ss996
maximum recursion depth exceeded while calling a Python object
maximum recursion depth exceeded while calling a Python object
kk
995
......
3
2
1
------------------------------------------------------------------------

----------
components: Interpreter Core
messages: 385170
nosy: xxm
priority: normal
severity: normal
status: open
title: Incorrect exception behavior in handling recursive call.
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42950>
_______________________________________

From report at bugs.python.org  Sun Jan 17 23:17:00 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Mon, 18 Jan 2021 04:17:00 +0000
Subject: [New-bugs-announce] [issue42951] Random and infinite loop in
 dealing with recursion error for "try-except "
Message-ID: <1610943420.56.0.379356825462.issue42951@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

In issue 42500, recursive calls in "Try-except" are resolved. This PR has fixed the crashes of some programs, such as program 1. And the core dump error is replaced with RecursiveError.  
However, program 2 will not report a RecursiveError. The program will fall into an infinite loop. Even "Ctrl C" cannot stop the infinite loop. I try to track the execution of this program and insert "print" information(see program 3). The output seems random in execution between try branch and except branch!  I think this is a new bug after fixing 42500. I believe the program should also return RecursiveError.


Program 1
=========================== 
def foo():
    try:
        1/0
    except:
        foo()
foo()
================================

Program 2
================================
def foo():
    try:
        foo()
    except:
        foo()
foo()
================================


Program 3
================================
def foo():
    try:
        print("a")
        foo()
    except:
        print("b")
        foo()

foo()
================================
Output for program3( unexpected infinite random loop. ):
......bbaaaabbabbaabbabbaaabbabbaabbabbaaaaaaaabbabbaabbabbaaabbabbaabbabbaaaabbabbaabbabbaaabbabbaabbabbaaaaabbabbaabbabbaaabbabbaabbabbaaaabbabbaabbabbaaabbabbaabbabbaaaaaabbabbaabbabbaaabbabbaabbabbaaaabb......

>>python -V
Python 3.10.0a4

----------
components: Interpreter Core
messages: 385171
nosy: xxm
priority: normal
severity: normal
status: open
title: Random and infinite loop in dealing with recursion error for "try-except "
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42951>
_______________________________________

From report at bugs.python.org  Mon Jan 18 01:26:19 2021
From: report at bugs.python.org (Zackery Spytz)
Date: Mon, 18 Jan 2021 06:26:19 +0000
Subject: [New-bugs-announce] [issue42952] Incorrect handling of
 EC_KEY_new_by_curve_name() in the _ssl module
Message-ID: <1610951179.15.0.310007065401.issue42952@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

A EC_KEY_new_by_curve_name() call in the _ssl__SSLContext_impl() function is not checked for failure.

----------
assignee: christian.heimes
components: Extension Modules, SSL
messages: 385173
nosy: ZackerySpytz, christian.heimes
priority: normal
severity: normal
status: open
title: Incorrect handling of EC_KEY_new_by_curve_name() in the _ssl module
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42952>
_______________________________________

From report at bugs.python.org  Mon Jan 18 02:31:23 2021
From: report at bugs.python.org (Chenxi Liu)
Date: Mon, 18 Jan 2021 07:31:23 +0000
Subject: [New-bugs-announce] [issue42953] After the subclass of enum.Enum is
 imported in some diffenent ways,
 the same enumeration value is judged as False
Message-ID: <1610955083.36.0.549548278035.issue42953@roundup.psfhosted.org>


New submission from Chenxi Liu <liuchenxirs at 163.com>:

When you run the test.py in attachment, it will print a False but the two enumeration value which are acquired in different ways is same

----------
components: Library (Lib)
files: test.zip
messages: 385174
nosy: Lcx994611818
priority: normal
severity: normal
status: open
title: After the subclass of enum.Enum is imported in some diffenent ways, the same enumeration value is judged as False
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49747/test.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42953>
_______________________________________

From report at bugs.python.org  Mon Jan 18 04:59:35 2021
From: report at bugs.python.org (=?utf-8?b?0JXQstCz0LXQvdC40LkgSmVu?=)
Date: Mon, 18 Jan 2021 09:59:35 +0000
Subject: [New-bugs-announce] [issue42954] new "if-as" syntax
Message-ID: <1610963975.6.0.718455756839.issue42954@roundup.psfhosted.org>


New submission from ??????? Jen <jen.soft.master at gmail.com>:

# use
if users.get(uid, None) as profile != None:
    ...

# instead
profile = users.get(uid, None)
if profile != None:
    ...


# full-sample:

>>> users = {1: {"name": "Jen"}}
... uid = 1
... 
... # current syntax:
... profile = users.get(uid, None)
... if profile != None:
...     print(f"user: {profile['name']}")
... #
... 
... # new:
... if users.get(uid, None) as profile != None:
...     print(f"user: {profile['name']}")
... #
...

----------
components: Interpreter Core
messages: 385178
nosy: jen.soft.master
priority: normal
severity: normal
status: open
title: new "if-as" syntax
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42954>
_______________________________________

From report at bugs.python.org  Mon Jan 18 05:08:34 2021
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 18 Jan 2021 10:08:34 +0000
Subject: [New-bugs-announce] [issue42955] Add sys.module_names: list of
 stdlib module names (Python and extension modules)
Message-ID: <1610964514.06.0.696047252341.issue42955@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

Some use cases require to know if a module comes from the stdlib or not. For example, I would like to only dump extension modules which don't come from the stdlib in bpo-42923: "Py_FatalError(): dump the list of extension modules".

Stdlib modules are special. For example, the maintenance and updates are connected to the Python lifecycle. Stdlib modules cannot be updated with "pip install --upgrade". They are shipped with the system ("system" Python). They are usually "read only": on Unix, only the root user can write into /usr directory where the stdlib is installed, whereas modules installed with "pip install --user" can be modified by the current user.

There is a third party project on PyPI which contains the list of stdlib modules:
https://pypi.org/project/stdlib-list/

There is already sys.builtin_module_names:
"A tuple of strings giving the names of all modules that are compiled into this Python interpreter."
https://docs.python.org/dev/library/sys.html#sys.builtin_module_names

I propose to add a similar sys.module_names tuple of strings (module names).

There are different constraints:

* If we add a public sys attribute, users will likely expect the list to be (1) exhaustive (2) correct
* Some extensions are not built if there are missing dependencies. Dependencies are checked after Python "core" (the sys module) is built.
* Some extensions are not available on some platforms.
* This list should be maintained.

Should we only list top level packages, or also submodules? For example, only list "asyncio", or list the 31 submodules (asyncio.base_events, asyncio.futures, ...)? Maybe it can be decided on a case by case basis. For example, I consider that "os.path" is a stdlib module, even it's just an alias to "posixpath" or "ntpath" depending on the platform.

I propose to include all extensions in the list, even if they are not built/available on some platforms. For example, "winsound" would also be listed on Linux, even if the extension is specific to Windows.

I also propose to include stdlib module names even if they are overridden at runtime using PYTHONPATH with a different implementation. For example, "asyncio" would be in the list, even if an user creates "asyncio.py" file. The list would not depend on sys.path.

--

Another option is to add an attribute to modules to mark them as coming from the stdlib. The API would be an attribute: module.__stdlib__ (bool).

The attribute could be set directly in the module code. For example, add "__stdlib__ = True" in Python modules. Similar idea for C extension modules.

Or the attribute could be set after importing the module, in the import site. But we don't control how stdlib modules are imported.

--

For the specific case of bpo-42923, another option is to use a list of stdlib paths, and check module.__file__ to check if a module is a stdlib module, and also use sys.builtin_module_names. And so don't add any API.

----------
components: Library (Lib)
messages: 385180
nosy: vstinner
priority: normal
severity: normal
status: open
title: Add sys.module_names: list of stdlib module names (Python and extension modules)
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42955>
_______________________________________

From report at bugs.python.org  Mon Jan 18 05:31:17 2021
From: report at bugs.python.org (=?utf-8?q?D=C3=A1vid_Nemeskey?=)
Date: Mon, 18 Jan 2021 10:31:17 +0000
Subject: [New-bugs-announce] [issue42956] Easy conversion between range and
 slice
Message-ID: <1610965877.9.0.924226649693.issue42956@roundup.psfhosted.org>


New submission from D?vid Nemeskey <nemeskeyd at gmail.com>:

It would be nice if there was an easy way to convert a range to a slice and vice versa. At least superficially the two are almost the same(*), yet if one wants to convert a slice to a range, he has to go through hoops to avoid potential errors, ending up with code such as:

s = slice(1, 10, 2)
r = range(s.start or 0, s.stop, s.step or 1)

It would be much nicer if the range and slice functions had a signature that accepts the other, which would simplify the above to just r(s).

(*) I don't know the implementation details, but is it even important to have to different objects for effectively the same concept? Cannot these two just be merged?

----------
messages: 385184
nosy: nemeskeyd
priority: normal
severity: normal
status: open
title: Easy conversion between range and slice
type: enhancement
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42956>
_______________________________________

From report at bugs.python.org  Mon Jan 18 14:13:09 2021
From: report at bugs.python.org (simon mackenzie)
Date: Mon, 18 Jan 2021 19:13:09 +0000
Subject: [New-bugs-announce] [issue42957] os.readlink produces wrong result
 on windows
Message-ID: <1610997189.12.0.447653130689.issue42957@roundup.psfhosted.org>


New submission from simon mackenzie <simonm3 at gmail.com>:

os.readlink gives wrong result on python 3.8 onwards for windows

os.readlink("c:/users/simon/v1")
'\\\\?\\d:\\v1'

Should read d:\\v1

----------
components: Windows
messages: 385218
nosy: paul.moore, simon mackenzie, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: os.readlink produces wrong result on windows
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42957>
_______________________________________

From report at bugs.python.org  Mon Jan 18 15:24:48 2021
From: report at bugs.python.org (Alexander Vandenbulcke)
Date: Mon, 18 Jan 2021 20:24:48 +0000
Subject: [New-bugs-announce] [issue42958] filecmp.cmp(shallow=True) isn't
 actually shallow when only mtime differs
Message-ID: <1611001488.57.0.420421194337.issue42958@roundup.psfhosted.org>


New submission from Alexander Vandenbulcke <alexander.vandenbulcke95 at gmail.com>:

Consider the case where 2 files are shallow compared where only the mtime differs (i.e. the mode and size is identical).

With filecmp.cmp(f1, f2, shallow=True) a deep compare would be performed behind the scenes since the guard clauses fell through. This discrepancy is either a problem in the docstring or a problem in the comparison itself.

Two options remain:
- the documentation is altered and describes that, in case only the mtime differs (i.e. mode and size are equal) a deep compare is performed
- the behaviour is restricted in that effectively only a shallow compare is performed
- a shallow compare becomes more restrictive (do not consider the mtime during the compare)

My preference is to adjust the function to safeguard the meaning of 'shallow' in this context.

----------
components: Library (Lib)
messages: 385225
nosy: AlexVndnblcke
priority: normal
severity: normal
status: open
title: filecmp.cmp(shallow=True) isn't actually shallow when only mtime differs
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42958>
_______________________________________

From report at bugs.python.org  Mon Jan 18 15:48:11 2021
From: report at bugs.python.org (Joel Ewaldo)
Date: Mon, 18 Jan 2021 20:48:11 +0000
Subject: [New-bugs-announce] [issue42959] AttributeError: module 'signal'
 has no attribute 'SIGHUP'
Message-ID: <1611002891.36.0.245706132459.issue42959@roundup.psfhosted.org>


New submission from Joel Ewaldo <ametatheton2 at gmail.com>:

I have already tried uninstalling and reinstalling python 3.7.9 via the 64-bit installer on the official python website. Also, I am on the latest version of windows. I have also made sure that signal.py was not a file in my directory and when I import _signal followed with print(dir(_signal)), it returns ['CTRL_BREAK_EVENT', 'CTRL_C_EVENT', 'NSIG', 'SIGABRT', 'SIGBREAK', 'SIGFPE', 'SIGILL', 'SIGINT', 'SIGSEGV', 'SIGTERM', 'SIG_DFL', 'SIG_IGN', '__doc__', '__loader__', '__name__', '__package__', 
'__spec__', 'default_int_handler', 'getsignal', 'set_wakeup_fd', 'signal'].

----------
components: Windows
files: unknown (2).png
messages: 385227
nosy: ametatheton2, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: AttributeError: module 'signal' has no attribute 'SIGHUP'
type: compile error
versions: Python 3.9
Added file: https://bugs.python.org/file49748/unknown (2).png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42959>
_______________________________________

From report at bugs.python.org  Mon Jan 18 16:05:03 2021
From: report at bugs.python.org (David CARLIER)
Date: Mon, 18 Jan 2021 21:05:03 +0000
Subject: [New-bugs-announce] [issue42960] resources module,
 FreeBSD update adding RLIMIT_KQUEUES constant
Message-ID: <1611003903.78.0.469721357758.issue42960@roundup.psfhosted.org>


Change by David CARLIER <devnexen at gmail.com>:


----------
components: FreeBSD
nosy: devnexen, koobs
priority: normal
pull_requests: 23073
severity: normal
status: open
title: resources module, FreeBSD update adding RLIMIT_KQUEUES constant
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42960>
_______________________________________

From report at bugs.python.org  Mon Jan 18 16:16:00 2021
From: report at bugs.python.org (Boris Staletic)
Date: Mon, 18 Jan 2021 21:16:00 +0000
Subject: [New-bugs-announce] [issue42961] Use-after-free (of a heap type)
 during finalization
Message-ID: <1611004560.18.0.844080226766.issue42961@roundup.psfhosted.org>


New submission from Boris Staletic <boris.staletic at gmail.com>:

When a C extension module exposes a heap-type that is used as a base class for a "pure" python class, that gets deallocated during `Py_Finalize()`, `subtype_dealloc()` runs into a use-after-free error.

That is a hell of a run-on sentence. The thing is that every part of the sentence needs to happen, to trigger the error, at least according to valgrind. The whole thing took a very long time to debug and I finally have a concise and self-contained repro:

///////////////////////////////////////////////////////////
#include <Python.h>

static void pybind11_object_dealloc(PyObject *self) {
        auto type = Py_TYPE(self);
        type->tp_free(self);
        Py_DECREF(type);
}

static PyModuleDef pybind11_module_def_m{
        .m_base = PyModuleDef_HEAD_INIT,
        .m_name = "m",
        .m_size = -1,
};

static PyObject *pybind11_init_impl_m() {
        auto m = PyModule_Create(&pybind11_module_def_m);
        PyType_Slot slots[] = {
                {Py_tp_dealloc, (void*)pybind11_object_dealloc},
                {0, nullptr}
        };
        PyType_Spec spec{"m.B", sizeof(PyObject), 0, Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HEAPTYPE, slots};
        auto type = PyType_FromSpec(&spec);
        PyObject_SetAttrString(m, "B", (PyObject*)type);
        return m;
}

int main() {
        PyImport_AppendInittab("m", pybind11_init_impl_m);
        Py_InitializeEx(1);
        PyRun_SimpleString("import m\n"
                           "def t():\n"
                           "  class A:\n"
                           "    class B(m.B):pass\n"
                           "    B()\n"
                           "t()\n");
        Py_Finalize();
}
/////////////////////////////////////////////////////////////////////////

Below is the valgrind-produced traceback of the error:

==3768== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info

==3768== Invalid read of size 1                                                                                                                          [4/36]
==3768==    at 0x4952AD6: subtype_dealloc (typeobject.c:1368)
==3768==    by 0x4928B6C: _Py_DECREF (object.h:448)
==3768==    by 0x4928B6C: _Py_XDECREF (object.h:514)
==3768==    by 0x4928B6C: free_keys_object (dictobject.c:628)
==3768==    by 0x492DAC8: dict_tp_clear (dictobject.c:3289)
==3768==    by 0x4A365EF: delete_garbage (gcmodule.c:1018)
==3768==    by 0x4A365EF: gc_collect_main (gcmodule.c:1301)
==3768==    by 0x4A370A4: gc_collect_with_callback (gcmodule.c:1414)
==3768==    by 0x4A370A4: PyGC_Collect (gcmodule.c:2066)
==3768==    by 0x4A106A6: Py_FinalizeEx (pylifecycle.c:1716)
==3768==    by 0x4A106A6: Py_FinalizeEx (pylifecycle.c:1638)
==3768==    by 0x109337: main (newtype.cpp:36)
==3768==  Address 0x59cc609 is 185 bytes inside a block of size 944 free'd
==3768==    at 0x483B9AB: free (vg_replace_malloc.c:538)
==3768==    by 0x4951830: type_dealloc (typeobject.c:3593)
==3768==    by 0x1091F3: _Py_DECREF (object.h:448)
==3768==    by 0x10922E: pybind11_object_dealloc(_object*) (newtype.cpp:6)
==3768==    by 0x4952AD5: subtype_dealloc (typeobject.c:1362)
==3768==    by 0x4928B6C: _Py_DECREF (object.h:448)
==3768==    by 0x4928B6C: _Py_XDECREF (object.h:514)
==3768==    by 0x4928B6C: free_keys_object (dictobject.c:628)
==3768==    by 0x492DAC8: dict_tp_clear (dictobject.c:3289)
==3768==    by 0x4A365EF: delete_garbage (gcmodule.c:1018)
==3768==    by 0x4A365EF: gc_collect_main (gcmodule.c:1301)
==3768==    by 0x4A370A4: gc_collect_with_callback (gcmodule.c:1414)
==3768==    by 0x4A370A4: PyGC_Collect (gcmodule.c:2066)
==3768==    by 0x4A106A6: Py_FinalizeEx (pylifecycle.c:1716)
==3768==    by 0x4A106A6: Py_FinalizeEx (pylifecycle.c:1638)
==3768==    by 0x109337: main (newtype.cpp:36)
==3768==  Block was alloc'd at
==3768==    at 0x483A77F: malloc (vg_replace_malloc.c:307)
==3768==    by 0x4A37402: _PyObject_GC_Alloc (gcmodule.c:2217)
==3768==    by 0x4A37402: _PyObject_GC_Malloc (gcmodule.c:2244)
==3768==    by 0x49523B5: PyType_GenericAlloc (typeobject.c:1072)
==3768==    by 0x495F2B4: type_new (typeobject.c:2622)
==3768==    by 0x49550F5: type_call (typeobject.c:1039)
==3768==    by 0x48F0EC7: _PyObject_MakeTpCall (call.c:191)
==3768==    by 0x49CD6CF: builtin___build_class__ (bltinmodule.c:232)
==3768==    by 0x493B674: cfunction_vectorcall_FASTCALL_KEYWORDS (methodobject.c:442)
==3768==    by 0x48B0040: _PyObject_VectorcallTstate (abstract.h:114)
==3768==    by 0x48B0040: PyObject_Vectorcall (abstract.h:123)
==3768==    by 0x48B0040: call_function (ceval.c:5379)
==3768==    by 0x48B0040: _PyEval_EvalFrameDefault (ceval.c:3803)
==3768==    by 0x49D383A: _PyEval_EvalFrame (pycore_ceval.h:40)
==3768==    by 0x49D383A: _PyEval_EvalCode (ceval.c:4625)
==3768==    by 0x49D3B9D: _PyEval_EvalCodeWithName (ceval.c:4657)
==3768==    by 0x49D3BED: PyEval_EvalCodeEx (ceval.c:4673)

FWIW, the valgrind command doesn't need any extra flags/switches. Looking at the output, it seems to me that the following happens:

1. [1] Objects/typeobject.c#L1362 calls the user provided (see the attached file) `tp_dealloc(p)`.
2. [2] Immediately after, Objects/typeobject.c#L1368 dereferences the, now dangling, pointer.

The provided `tp_dealloc` looks like this:

static void pybind11_object_dealloc(PyObject *self) {
        auto type = Py_TYPE(self);
        type->tp_free(self);
        Py_DECREF(type);
}


Which seems correct to me, given the "whatsnew" page for python 3.8:

https://docs.python.org/3.8/whatsnew/3.8.html#changes-in-the-c-api


Finally, this found while running an innocent-looking pybind11 test inside valgrind:

https://github.com/pybind/pybind11/pull/2797

[1]: https://github.com/python/cpython/blob/master/Objects/typeobject.c#L1362
[2]: https://github.com/python/cpython/blob/master/Objects/typeobject.c#L1368

----------
components: C API
files: heap_type_base_use_after_free.cpp
messages: 385230
nosy: bstaletic
priority: normal
severity: normal
status: open
title: Use-after-free (of a heap type) during finalization
type: behavior
versions: Python 3.10, Python 3.9
Added file: https://bugs.python.org/file49749/heap_type_base_use_after_free.cpp

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42961>
_______________________________________

From report at bugs.python.org  Mon Jan 18 17:12:34 2021
From: report at bugs.python.org (William Schwartz)
Date: Mon, 18 Jan 2021 22:12:34 +0000
Subject: [New-bugs-announce] [issue42962] Windows: SystemError during
 os.kill(..., signal.CTRL_C_EVENT)
Message-ID: <1611007954.59.0.488720921329.issue42962@roundup.psfhosted.org>


New submission from William Schwartz <wkschwartz at gmail.com>:

I don't have an automated test at this time, but here's how to get os.kill to raise SystemError. I did this on Windows 10 version 20H2 (build 19042.746) with Pythons 3.7.7, 3.8.5, and 3.9.1. os_kill_impl at Modules/posixmodule.c:7833 does not appear to have been modified recently (but I haven't checked its transitive callees), so I imagine you can get the same behavior from os.kill on Python 3.10.

1. Open two consoles, A and B. (I'm using tabs in Windows Terminal, if that matters.)
2. In console A, type but DO NOT EXECUTE the following command:

python -c"import os, signal; os.kill(, signal.CTRL_C_EVENT)"

Move your cursor back before the comma.

3. In console B, create a process that does nothing but print its process identifier and wait long enough for you to type it in console A:

python -c"import os, time; print(os.getpid()); time.sleep(60); print('exiting cleanly')"

Copy or remember the printed PID. Hurry to step 4 before your 60 seconds expires!

4. In console A, type the PID from console B and execute the command.
5. In console B, confirm that the Python exited without printing "exiting cleanly". Oddly, `echo %errorlevel%` will print `0` rather than `-1073741510` (which is `2**32 - 0xc000013a`, the CTRL_C_EVENT exit code), which is what it prints after `python -c"raise KeyboardInturrupt"`.
6. In console A, you should see the following traceback.

OSError: [WinError 87] The parameter is incorrect

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
SystemError: <built-in function kill> returned a result with an error set

----------
components: Extension Modules, Windows
messages: 385235
nosy: William.Schwartz, ncoghlan, paul.moore, petr.viktorin, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows: SystemError during os.kill(..., signal.CTRL_C_EVENT)
type: behavior
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42962>
_______________________________________

From report at bugs.python.org  Mon Jan 18 20:08:51 2021
From: report at bugs.python.org (Sammy Jelin)
Date: Tue, 19 Jan 2021 01:08:51 +0000
Subject: [New-bugs-announce] [issue42963] [multiprocessing] Calling
 pool.terminate() from an error_callback causes deadlock
Message-ID: <1611018531.21.0.537648626366.issue42963@roundup.psfhosted.org>


New submission from Sammy Jelin <sjelin at gmail.com>:

As the title says, calling `pool.terminate()` inside an `error_callback` handler causes a deadlock.  The deadlock always happens, so it's not a race condition or thread-safety issue.

Simple repro:

```
from multiprocessing import Pool
p = Pool()

def error_callback(x):
    print(f'error: {x!r}')
    p.terminate()
    print('this message is never seen, because p.termiante() deadlocks')

p.apply_async(lambda: None, error_callback=error_callback)

# The following lines are technically aren't threadsafe,
# but I manually verified that that wasn't the problem.
p.close()
p.join()
print('this is also never seen, because the task handler is stuck in the deadlock')
```

This will print the following line and then hang:
```
error: PicklingError("Can't pickle <function <lambda> at 0x112c55e18>: attribute lookup <lambda> on __main__ failed")
```

The hanging happens inside `_help_stuff_finish`, when we call `inqueue._rlock.acquire()`.  As far as I can tell, `_handle_tasks` is already holding the lock when it calls the error_callback, so when `_terminate_pool` calls `_help_stuff_finish` a deadlock occurs.

Calling `p.terminate()` from a success callback doesn't appear to be an issue, likely because success callbacks are called via `_handle_results` instead of via `_handle_tasks`.

If calling `p.terminate()` from an error_callback isn't supported, that should be one of those big red warnings in the documentation.

I verified that this bug happens on 3.6 and 3.7, a skimmed the code on the github project to verify that it was likely still an issue.

----------
components: Library (Lib)
messages: 385240
nosy: sjelin
priority: normal
severity: normal
status: open
title: [multiprocessing] Calling pool.terminate() from an error_callback causes deadlock
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42963>
_______________________________________

From report at bugs.python.org  Mon Jan 18 20:58:08 2021
From: report at bugs.python.org (Larry Hastings)
Date: Tue, 19 Jan 2021 01:58:08 +0000
Subject: [New-bugs-announce] [issue42964] Draft PEP blob etc
Message-ID: <b6c8112f-edb0-a860-5a52-5be7590d38f8@hastings.org>


New submission from Larry Hastings <larry at hastings.org>:

You interested in me doing an editing pass / critique of your PEP?? I 
just poked around on Github, and I don't think I can do one of those 
inline code review things until it's a PR.

Also, I'm a PEP editor, I can get it assigned a PEP number when you're 
ready for that.

Cheers,

//arry/

----------
messages: 385243
nosy: larry
priority: normal
severity: normal
status: open
title: Draft PEP blob etc

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42964>
_______________________________________

From report at bugs.python.org  Tue Jan 19 03:11:26 2021
From: report at bugs.python.org (Ofek Lev)
Date: Tue, 19 Jan 2021 08:11:26 +0000
Subject: [New-bugs-announce] [issue42965] Optional callable raises TypeError
Message-ID: <1611043886.41.0.0460380648291.issue42965@roundup.psfhosted.org>


New submission from Ofek Lev <ofekmeister at gmail.com>:

https://docs.python.org/3.9/library/typing.html#callable

```
Python 3.9.1 (default, Jan 12 2021, 16:45:25)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import Optional
>>> from collections.abc import Callable
>>>
>>> Hasher = Optional[Callable[[bytes], bytes]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/typing.py", line 262, in inner
    return func(*args, **kwds)
  File "/usr/local/lib/python3.9/typing.py", line 339, in __getitem__
    return self._getitem(self, parameters)
  File "/usr/local/lib/python3.9/typing.py", line 463, in Optional
    return Union[arg, type(None)]
  File "/usr/local/lib/python3.9/typing.py", line 262, in inner
    return func(*args, **kwds)
  File "/usr/local/lib/python3.9/typing.py", line 339, in __getitem__
    return self._getitem(self, parameters)
  File "/usr/local/lib/python3.9/typing.py", line 451, in Union
    parameters = _remove_dups_flatten(parameters)
  File "/usr/local/lib/python3.9/typing.py", line 231, in _remove_dups_flatten
    return tuple(_deduplicate(params))
  File "/usr/local/lib/python3.9/typing.py", line 205, in _deduplicate
    all_params = set(params)
TypeError: unhashable type: 'list'
>>>
>>> from typing import Tuple
>>> Hasher = Optional[Callable[Tuple[bytes], bytes]]
>>>
```

Tuple type for arguments makes it work

----------
components: Library (Lib)
messages: 385246
nosy: Ofekmeister
priority: normal
severity: normal
status: open
title: Optional callable raises TypeError
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42965>
_______________________________________

From report at bugs.python.org  Tue Jan 19 09:27:00 2021
From: report at bugs.python.org (Tadek Kijkowski)
Date: Tue, 19 Jan 2021 14:27:00 +0000
Subject: [New-bugs-announce] [issue42966] argpare: customizable help
 formatter
Message-ID: <1611066420.6.0.700026495964.issue42966@roundup.psfhosted.org>


New submission from Tadek Kijkowski <tkijkowski at gmail.com>:

Current implementation of the argparse module doesn't make it easy for developers to customize the help text message format. On one hand the module doesn't provide any builtin ways to do it, and the only way is to provide developer's own formatter class, on the other hand developers are threatened, that all internal structures of the module, including API of the HelpFormatter class is subject to change. Trying to customize anything but the most basic things, developer risks that his or hers module will suddenly stop working with a new version of argparse.

For most basic customization there is a set of four alternative formatting classes, but their usage as well as development of additional formatting classes is hindered by following reasons:
- Only one formatting class can be used at a time. Although arguably devloper can safely derive his own formatter class from multiple argparse's formatters - if he or she will come up with this idea.
- At this moment there is no way to pass any configuration data to the formatter object, so each formatter class can only change one aspect of the output format.
- Any new functionality added to argparse help formatting has to take form of new formatter class. If it is significant change, it means duplicating parts of HelpFormatter code and in the future maintaining additional classes.
- Even if additional formatter classes are added to argparse, developers have make sure that the new class is present in the argparse used by their module - either add dependency to specific argparse version, or dynamically check for presence of the new class in argparse module.

The most important part of this enhancement request is change in argparse public API - addition of class CustomHelpFormat. Object of this class would serve as factory for CustomizableHelpFormatter objects and could be passed to ArgumentParser constructor in place of format class type:

custom_format = argparse.CustomHelpFormat()
custom_format.indent_increment = 4
custom_format.raw_description = True
parser = argparse.ArgumentParser(formatter_class=custom_format)

The idea of passing callable instead of class type as formatter_class argument is not new, but this time it would be part of official API with access to internal details and maintained along with the module.

In conrtast to the list of drawbacks above, the advantages of this solution are:
- It is much easier to do basic customization. Developers won't have to search the internet for information how to customize argparse output, when they just want to focus on their own application functionality.
- There is no need to check for presence of future additional help formatter in the argparse module. One just has to set appropriate attribute in the 'custom_format' object. If required functionality is not present in the module on the system, it would be simply ignored, developer could even safely set attribute for future feature not present in released version.
- Detection of implemented feature would is much easier - developer would just call `hasattr(custom_format, "hybrid_text_formatter")` and modify displayed text accordingly in the absence of the requested feature.
- Development of new features should be easier - instead of creating new class for each possible aspect of customization, new features could be combined in one new class, or even added to the base HelpFormatter and just enabled by an attribute.

Altough this solution will not immediately help developers wanting to customize their argparse, the main idea is that it will unclog development of new formatters and incorporating existing propositions and possibly lessen or remove the need for client side customizations.

Initially, the new class would contain attributes corresponding to funcionality of existing alternative formatting classes as well as arguments to default HelpFormatter - indent, max help position and width.

Attached implementation of the CustomizableHelpFormatter class may seem rudimentary or hackish (or maybe not). Perhaps it should implement all appropriate methods instead of patching self.

----------
components: Library (Lib)
files: CustomizableHelpFormatter.txt
messages: 385263
nosy: monkeyman79
priority: normal
severity: normal
status: open
title: argpare: customizable help formatter
type: enhancement
versions: Python 3.10
Added file: https://bugs.python.org/file49750/CustomizableHelpFormatter.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42966>
_______________________________________

From report at bugs.python.org  Tue Jan 19 10:06:49 2021
From: report at bugs.python.org (Adam Goldschmidt)
Date: Tue, 19 Jan 2021 15:06:49 +0000
Subject: [New-bugs-announce] [issue42967] Web cache poisoning - `;
 ` as a query args separator
Message-ID: <1611068809.96.0.58823274544.issue42967@roundup.psfhosted.org>


New submission from Adam Goldschmidt <adamgold7 at gmail.com>:

The urlparse module treats semicolon as a separator (https://github.com/python/cpython/blob/master/Lib/urllib/parse.py#L739) - whereas most proxies today only take ampersands as separators. Link to a blog post explaining this vulnerability: https://snyk.io/blog/cache-poisoning-in-popular-open-source-packages/

When the attacker can separate query parameters using a semicolon (;), they can cause a difference in the interpretation of the request between the proxy (running with default configuration) and the server. This can result in malicious requests being cached as completely safe ones, as the proxy would usually not see the semicolon as a separator, and therefore would not include it in a cache key of an unkeyed parameter - such as `utm_*` parameters, which are usually unkeyed. Let?s take the following example of a malicious request:				

```
GET /?link=http://google.com&utm_content=1;link='><t>alert(1)</script> HTTP/1.1

Host: somesite.com

Upgrade-Insecure-Requests: 1		

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,imag e/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9 Accept-Encoding: gzip, deflate			

Accept-Language: en-US,en;q=0.9 Connection: close			
```

urlparse sees 3 parameters here: `link`, `utm_content` and then `link` again. On the other hand, the proxy considers this full string: `1;link='><t>alert(1)</script>` as the value of `utm_content`, which is why the cache key would only contain `somesite.com/?link=http://google.com`. 

A possible solution could be to allow developers to specify a separator, like werkzeug does:

https://github.com/pallets/werkzeug/blob/6784c44673d25c91613c6bf2e614c84465ad135b/src/werkzeug/urls.py#L833

----------
components: C API
messages: 385266
nosy: AdamGold
priority: normal
severity: normal
status: open
title: Web cache poisoning - `;` as a query args separator
type: security
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42967>
_______________________________________

From report at bugs.python.org  Tue Jan 19 13:24:16 2021
From: report at bugs.python.org (Daniel Grunwald)
Date: Tue, 19 Jan 2021 18:24:16 +0000
Subject: [New-bugs-announce] [issue42968] multiprocessing handle leak on
 Windows when child process is killed during startup/unpickling
Message-ID: <1611080656.89.0.981627513492.issue42968@roundup.psfhosted.org>


New submission from Daniel Grunwald <daniel at danielgrunwald.de>:

Running the attached script deadlocks.
Uncommenting the `time.sleep(1)` in the script makes the deadlock disappear.

For context: our application uses multiple child processes (multiprocessing.Process) and uses pipes (multiprocessing.Pipe) to communicate with them.
If one process fails with an error, the main process will kill all other child processes running concurrently.
We noticed that sometimes (non-deterministically), when an error occurs soon after startup, the main process ends up hanging.

We expect that when we pass the writing half of a connection to a child process and close the connection in the main process, that we will receive EOFError if the child process terminates unexpectedly.
But sometimes the EOFError never comes and our application hangs.

I've reduced the problem to the script attached. With the reduced script, the deadlock happens reliably for me.

I've debugged this a bit, and I think this is happening because passing a connection to the process being started involves reduce_pipe_connection() which creates a copy of the handle within the main process.
When the pickled data is unpickled in the child process, it uses DUPLICATE_CLOSE_SOURCE to close the copy in the main process.
But if the pickled data is never unpickled by the child process, the handle ends up being leaked.
Thus the main process itself holds the writing half of the connection open, causing the recv() call on the reading half to block forever.

----------
components: Windows
files: deadlock.py
messages: 385283
nosy: dgrunwald, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: multiprocessing handle leak on Windows when child process is killed during startup/unpickling
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49751/deadlock.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42968>
_______________________________________

From report at bugs.python.org  Tue Jan 19 14:35:46 2021
From: report at bugs.python.org (Gregory P. Smith)
Date: Tue, 19 Jan 2021 19:35:46 +0000
Subject: [New-bugs-announce] [issue42969] pthread_exit &
 PyThread_exit_thread are harmful
Message-ID: <1611084946.55.0.234619663171.issue42969@roundup.psfhosted.org>


New submission from Gregory P. Smith <greg at krypto.org>:

## BACKGROUND

`PyThread_exit_thread()` calls `pthread_exit(`) and is in turn called from a variety of APIs as documented in the C-API doc update from https://bugs.python.org/issue36427.

The `pthread_exit()` call was originally introduced as a way "resolve" a crashes from daemon threads during shutdown https://bugs.python.org/issue1856.  It did that.  That fix to that even landed in 2.7.8 but was rolled back before 2.7.9 due to a bug in an existing application it exposed at the time (did we miss the implications of that? maybe).  It remained in 3.x.

## PROBLEM

`pthread_exit()` cannot be used blindly by any application.  All code in the threaded application needs to be on board with it and always prepared for any API call they make to potentially lead to thread termination.  Quoting a colleague: "pthread_exit() does not result in stack unwind or local variable destruction".  This means that any code up the stack from the ultimate `pthread_exit()` call that has process-wide state implications that did not go out of its way to register cleanup with `pthread_cleanup_push()` could lead to deadlocks or lost resources. Something implausible to assume that code does.

We're seeing this happen with C/C++ code.  Our C++ builds with `-fno-exceptions` so uncatchable exception based stack unwinding as some pthread_exit implementations may trigger does not happen (and cannot be guaranteed anyways, see bpo-42888).  Said C/C++ code is calling back into Python from a thread and thus must use `PyEval_RestoreThread()` or similar APIs before performing Python C API calls.  If the interpreter is being finalized from another thread... these enter a codepath that ultimately calls `pthread_exit()` leaving corrupt state in the process.  In this case that unexpected thread simply disappearing can lead to a deadlock in our process.

Fundamentally I do not believe the CPython VM should ever call `pthread_exit()` when non-CPython frames are anywhere in the C stack.  This may mean we should never call `pthread_exit()` at all (unsure; but it'd be ideal).

The documentation suggests that all callers in user code of the four C-APIs with the documented `pthread_exit()` caveats need auditing and pre-call `_Py_IsFinalizing()` API checks.  But... I do not believe that would fix anything even if it were done.  `_Py_IsFinalizing()` called without the GIL held means that it could change by the time the `PyEval_RestoreThreads()` API calls it internally do determine if it should exit the thread.  Thus the race condition window would merely be narrowed, not eliminated.  Not good enough.

## CURRENT WORKAROUND (Big Hammer)

Change CPython to call abort() instead of pthread_exit() as that situation is unresolvable and the process dying is better than hanging, partially alive.  That solution isn't friendly, but is better than being silent and allowing deadlock.  A failing process is always better than a hung process, especially a partially hung process.

## SEMI RELATED WORK

https://bugs.python.org/issue42888 - appears to be avoiding some `PyThread_exit_thread()` calls to stop some crashes due to `libgcc_s` being loaded on demand upon thread exit.

----------
components: Interpreter Core
keywords: 3.2regression
messages: 385288
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: pthread_exit & PyThread_exit_thread are harmful
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42969>
_______________________________________

From report at bugs.python.org  Tue Jan 19 15:26:21 2021
From: report at bugs.python.org (Doug Day)
Date: Tue, 19 Jan 2021 20:26:21 +0000
Subject: [New-bugs-announce] [issue42970] File path with blank causes open
 error in 3.8, not in 3.7
Message-ID: <1611087981.59.0.713366355851.issue42970@roundup.psfhosted.org>


New submission from Doug Day <dgday6 at gmail.com>:

The following code generates a path that works in Pythons 3.7.6 on macOS Big Sur but not in Catalina  with 3.8.2.. 
mySrcFldr="~/Library/Mobile Documents/com~apple~CloudDocs/Utilities/"
srcFldr=os.path.expanduser(mySrcFldr)
f=os.path.join(srcFldr,"mortgage.py")

When python tries to run the code file(montage.py) on 3.8.2, an open error results.
/Library/Developer/CommandLineTools/usr/bin/python3: can't open file '/Users/Doug/Library/Mobile': [Errno 2] No such file or directory
 I tried putting a backslash after 'Mobile'. Does not help.
The complete path is:
'/Users/Doug/Library/Mobile Documents/com~apple~CloudDocs/Utilities/mortgage.py'

While the doc changes show some changes in the os.path module, I see no mention of this issue.

----------
components: Interpreter Core
messages: 385290
nosy: dday52
priority: normal
severity: normal
status: open
title: File path with blank causes open error in 3.8, not in 3.7
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42970>
_______________________________________

From report at bugs.python.org  Tue Jan 19 15:56:36 2021
From: report at bugs.python.org (Enji Cooper)
Date: Tue, 19 Jan 2021 20:56:36 +0000
Subject: [New-bugs-announce] [issue42971] Some errnos for BSD/OSX are
 missing from errno module
Message-ID: <1611089796.05.0.463787343348.issue42971@roundup.psfhosted.org>


New submission from Enji Cooper <yaneurabeya at gmail.com>:

Some errnos for BSD/OSX are currently not supported by the errno module. It would be helpful for these to be exposed to the end-user so they could programmatically use them instead of reinventing the wheel in a separate module/C extension, or hardcoding the information in consuming pieces of code.

----------
components: Extension Modules
messages: 385293
nosy: ngie
priority: normal
severity: normal
status: open
title: Some errnos for BSD/OSX are missing from errno module
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42971>
_______________________________________

From report at bugs.python.org  Tue Jan 19 16:54:26 2021
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 19 Jan 2021 21:54:26 +0000
Subject: [New-bugs-announce] [issue42972] [C API] Heap types
 (PyType_FromSpec) must fully implement the GC protocol
Message-ID: <1611093266.11.0.721387186389.issue42972@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

Copy of my email sent to python-dev:
https://mail.python.org/archives/list/python-dev at python.org/thread/C4ILXGPKBJQYUN5YDMTJOEOX7RHOD4S3/

Hi,

In the Python stdlib, many heap types currently don't "properly"
(fully?) implement the GC protocol which can prevent to destroy these
types at Python exit. As a side effect, some other Python objects can
also remain alive, and so are not destroyed neither.

There is an on-going effect to destroy all Python objects at exit
(bpo-1635741). This problem is getting worse when subinterpreters are
involved: Refleaks buildbots failures which prevent to spot other
regressions, and so these "leaks" / "GC bugs" must be fixed as soon as
possible. In my experience, many leaks spotted by tests using
subinterpreters were quite old, it's just that they were ignored
previously.

It's an hard problem and I don't see any simple/obvious solution right
now, except of workarounds that I dislike. Maybe the only good
solution is to fix all heap types, one by one.

== Only the Python stdlib should be affected ==

PyType_FromSpec() was added to Python 3.2 by the PEP 384 to define
"heap types" in C, but I'm not sure if it's popular in practice (ex:
Cython doesn't use it, but defines static types). I expect that most
types to still be defined the old style (static types) in a vas
majority of third party extension modules.

To be clear, static types are not affected by this email.

Third party extension modules using the limited C API (to use the
stable ABI) and PyType_FromSpec() can be affected (if they don't fully
implement the GC protocol).

== Heap type instances now stores a strong reference to their type ==

In March 2019, the PyObject_Init() function was modified in bpo-35810
to keep a strong reference (INCREF) to the type if the type is a heap
type. The fixed problem was that heap types could be destroyed before
the last instance is destroyed.

== GC and heap types ==

The new problem is that most heap types don't collaborate well with
the garbage collector. The garbage collector doesn't know anything
about Python objects, types, reference counting or anything. It only
uses the PyGC_Head header and the traverse functions. If an object
holds a strong reference to an object but its type does not define a
traverse function, the GC cannot guess/infer this reference.

A heap type must respect the following 3 conditions to collaborate with the GC:

    Have the Py_TPFLAGS_HAVE_GC flag;
    Define a traverse function (tp_traverse) which visits the type: Py_VISIT(Py_TYPE(self));
    Instances must be tracked by the GC.

If one of these conditions is not met, the GC can fail to destroy a
type during a GC collection. If an instance is kept alive late while a
Python interpreter is being deleted, it's possible that the type is
never deleted, which can keep indirectly many objects alive and so
don't delete them neither.

In practice, when a type is not deleted, a test using subinterpreter
starts to fail on Refleaks buildbot since it leaks references. Without
subinterpreters, such leak is simply ignored, whereas this is an
on-going effect to delete Python objects at exit (bpo-1635741).

== Boring traverse functions ==

Currently, there is no default traverse implementation which visits the type.

For example, I had the implement the following function for _thread.LockType:

static int
lock_traverse(lockobject self, visitproc visit, void arg)
{
    Py_VISIT(Py_TYPE(self));
    return 0;
}

It's a little bit annoying to have to implement the GC protocol
whereas a lock cannot contain other Python objects, it's not a
container. It's just a thin wrapper to a C lock.

There is exactly one strong reference: to the type.

== Workaround: loop on gc.collect() ==

A workaround is to run gc.collect() in a loop until it returns 0 (no
object was collected).

== Traverse automatically? Nope. ==

Pablo Galindo attempts to automatically visit the type in the traverse function:

https://bugs.python.org/issue40217
https://github.com/python/cpython/commit/0169d3003be3d072751dd14a5c84748ab63...

Moreover, What's New in Python 3.9 contains a long section suggesting
to implement a traverse function for this problem, but it doesn't
suggest to track instances:
https://docs.python.org/dev/whatsnew/3.9.html#changes-in-the-c-api

This solution causes too many troubles, and so instead, traverse
functions were defined on heap types to visit the type.

Currently in the master branch, 89 types are defined as heap types on
a total of 206 types (117 types are defined statically). I don't think
that these 89 heap types respect the 3 conditions to collaborate with
the GC.

== How should we address this issue? ==

I'm not sure what should be done. Working around the issue by
triggering multiple GC collections? Emit a warning in development mode
if a heap type doesn't collaborate well with the GC?

If core developers miss these bugs and have troubles to debug them, I
expect that extension module authors would suffer even more.

== GC+heap type bugs became common  ==

I'm fixing such GC issue for 1 year as part as the work on cleaning
Python objects at exit, and also indirectly related to
subinterpreters. The behavior is surprising, it's really hard to dig
into GC internals and understand what's going on. I wrote an article
on this kind of "GC bugs":
https://vstinner.github.io/subinterpreter-leaks.html

Today, I learnt the hard way that defining a traverse is not enough.
The type constructor (tp_new) must also track instances! See my fix
for _multibytecodec related to CJK codecs:

https://github.com/python/cpython/commit/11ef53aefbecfac18b63cee518a7184f771...
https://bugs.python.org/issue42866

== Reference cycles are common ==

The GC only serves to break reference cycles. But reference cycles are
rare, right? Well...

First of all, most types create reference cycles involing themselves.
For example, a type __mro__ tuple contains the type which already
creates a ref cycle. Type methods can also contain a reference to the
type.

=> The GC must break the cycle, otherwise the type cannot be destroyed

When a function is defined in a Python module, the function
__globals__ is the module namespace (module.__dict__) which...
contains the function. Defining a function in a Python module also
creates a reference cycle which prevents to delete the module
namespace.

If a function is used as a callback somewhere, the whole module
remains "alive" until the reference to the callback is cleared.
Example. os.register_at_fork() and codecs.register() callbacks are
cleared really late during Python finalization. Currently, it's
basically the last objects which are cleared at Python exit. After
that, there is exactly one final GC collection.

=> The GC

== Debug GC issues ==

    gc.get_referents() and gc.get_referrers() can be used to check traverse functions.
    gc.is_tracked() can be used to check if the GC tracks an object.
    Using the gdb debugger on gc_collect_main() helps to see which objects are collected. See for example the finalize_garbage() functions which calls finalizers on unreachable objects.
    The solution is usually a missing traverse functions or a missing Py_VISIT() in an existing traverse function.

== __del__ hack for debugging ==

If you want to play with the issue or if you have to debug a GC issue,
you can use an object which logs a message when it's being deleted:

class VerboseDel:
    def __del__(self):
        print("DELETE OBJECT")
obj = VerboseDel()

Warning: creating such object in a module also prevents to destroy the
module namespace when the last reference to the module is deleted!
__del__.__globals__ contains a reference to the module namespace, and
obj.__class__ contains a reference to the type... Yeah, ref cycle and
GC issues are fun!

== Long email ==

Yeah, I like to put titles in my long emails. Enjoy. Happy hacking!
Victor

--
Night gathers, and now my watch begins. It shall not end until my death

----------
components: C API
messages: 385297
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Heap types (PyType_FromSpec) must fully implement the GC protocol
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42972>
_______________________________________

From report at bugs.python.org  Tue Jan 19 17:21:34 2021
From: report at bugs.python.org (Tadek Kijkowski)
Date: Tue, 19 Jan 2021 22:21:34 +0000
Subject: [New-bugs-announce] [issue42973] argparse: mixing optional and
 positional arguments... not again
Message-ID: <1611094894.69.0.476759327253.issue42973@roundup.psfhosted.org>


New submission from Tadek Kijkowski <tkijkowski at gmail.com>:

I have following use case for argparse:

./prog --tracks=80 image1 --tracks=40 image2 --tracks=80 image3 ...

I expected that the following parser would be able process that command line:

parser = argparse.ArgumentParser()
add = parser.add_argument
add('--tracks', choices=['40', '80'])
add('image', nargs='*', action=_CustomAction)
parser.parse_args()

But it stops with 'unrecognized arguments: image2' error. It turns out that there is no way to prepare parser which could handle this command line.

There was a long discussion about this issue in #14191. There were two approaches proposed:
- Allow 'image' action to trigger multiple times with additional encountered parameters.
- Parse all optionals first and all positionals in second pass.

The first approach was represented by a patch by user guilherme-pg. This patch was slightly criticized (undeservedly IMO) and pretty much ignored.

The discussion then focused on the second approach which ended up with introduction of `parse_intermixed_args` into codebase.

The problem with `parse_intermixed_args` is that it defeats the purpose of having intermixed arguments. When the arguments are effectively reordered, there is no way to match "optionals" with "positionals" with they relate to.

In my option the Guilherme's approach was the correct way to go, I would say it was elegant. The only complaint against that patch is that it unnecessarily modified _StoreAction class - instead of that, the way to go is to use "extend" or custom action.

I allowed myself to simplify the changes a bit for readability. The patch doesn't change default functionality, for the cost of changing public API by adding new argument to ArgumentParser constructor.

With the changes applied, I can parse my command line with this code:

parser = argparse.ArgumentParser(greedy_star=True)
add = parser.add_argument
add('--tracks', choices=['40', '80'])
add('image', nargs='*', action=_CustomAction)
parser.parse_args()

----------
components: Library (Lib)
messages: 385301
nosy: monkeyman79
priority: normal
severity: normal
status: open
title: argparse: mixing optional and positional arguments... not again
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42973>
_______________________________________

From report at bugs.python.org  Tue Jan 19 22:13:57 2021
From: report at bugs.python.org (Brian Romanowski)
Date: Wed, 20 Jan 2021 03:13:57 +0000
Subject: [New-bugs-announce] [issue42974] tokenize reports incorrect end col
 offset and line string when input ends without explicit newline
Message-ID: <1611112437.84.0.130779999528.issue42974@roundup.psfhosted.org>


New submission from Brian Romanowski <romanows at gmail.com>:

The tokenize module's tokenizer functions output incorrect (or at least misleading) information when the content being tokenized does not end in a line ending character.  This is related to the fix for issue<33899>  which added the NEWLINE tokens for this case but did not fill out the whole token tuple correctly.

The bug can be seen by running a version of the test in Lib/test/test_tokenize.py:

import io, tokenize

newline_token_1 = list(tokenize.tokenize(io.BytesIO("x\n".encode('utf-8')).readline))[-2]
newline_token_2 = list(tokenize.tokenize(io.BytesIO("x".encode('utf-8')).readline))[-2]

print(newline_token_1)
print(newline_token_2)

# Prints:
# TokenInfo(type=4 (NEWLINE), string='\n', start=(1, 1), end=(1, 2), line='x\n')
# TokenInfo(type=4 (NEWLINE), string='', start=(1, 1), end=(1, 2), line='')  # bad "end" and "line"!

Notice that "len(newline_token_2.string) == 0" but "newline_token_2.end[1] - newline_token_2.start[1] == 1".  Seems more consistent if the newline_token_2.end == (1, 1).

Also, newline_token_2.line should hold the physical line rather than the empty string.  This would make it consistent with newline_token_1.line.

I'll add a PR shortly with a change so the output from the two cases is:

TokenInfo(type=4 (NEWLINE), string='\n', start=(1, 1), end=(1, 2), line='x\n')
TokenInfo(type=4 (NEWLINE), string='', start=(1, 1), end=(1, 1), line='x')

If this looks reasonable, I can backport it for the other branches.  Thanks!

----------
components: Library (Lib)
messages: 385313
nosy: romanows
priority: normal
severity: normal
status: open
title: tokenize reports incorrect end col offset and line string when input ends without explicit newline
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42974>
_______________________________________

From report at bugs.python.org  Wed Jan 20 05:52:29 2021
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 20 Jan 2021 10:52:29 +0000
Subject: [New-bugs-announce] [issue42975] urllib.parse.parse_qsl(): add an
 option to not consider semicolon (; ) as separator
Message-ID: <1611139949.1.0.449801455781.issue42975@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

urllib.parse.parse_qsl() uses "&" *and* ";" as separators:

>>> urllib.parse.parse_qsl("a=1&b=2&c=3")
[('a', '1'), ('b', '2'), ('c', '3')]
>>> urllib.parse.parse_qsl("a=1&b=2;c=3")
[('a', '1'), ('b', '2'), ('c', '3')]

But the W3C standards evolved and now suggest against considering semicolon (";") as a separator:

https://www.w3.org/TR/2014/REC-html5-20141028/forms.html#url-encoded-form-data

"This form data set encoding is in many ways an aberrant monstrosity, the result of many years of implementation accidents and compromises leading to a set of requirements necessary for interoperability, but in no way representing good design practices. In particular, readers are cautioned to pay close attention to the twisted details involving repeated (and in some cases nested) conversions between character encodings and byte sequences."

"To decode application/x-www-form-urlencoded payloads (...) Let strings be the result of strictly splitting the string payload on U+0026 AMPERSAND characters (&)."

Maybe we should even go further in Python 3.10 and only split at "&" by default, but let the caller to opt-in for ";" separator as well.

----------
components: Library (Lib)
messages: 385327
nosy: vstinner
priority: normal
severity: normal
status: open
title: urllib.parse.parse_qsl(): add an option to not consider semicolon (;) as separator
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42975>
_______________________________________

From report at bugs.python.org  Wed Jan 20 06:21:25 2021
From: report at bugs.python.org (Antony Lee)
Date: Wed, 20 Jan 2021 11:21:25 +0000
Subject: [New-bugs-announce] [issue42976] __text_signature__ parser silently
 drops arguments with certain unsupported default forms
Message-ID: <1611141685.32.0.342483333341.issue42976@roundup.psfhosted.org>


New submission from Antony Lee <anntzer.lee at gmail.com>:

Starting from the keyword-arguments example at https://docs.python.org/3/extending/extending.html#keyword-parameters-for-extension-functions, change the docstring of `parrot` to "parrot(voltage, state, action, type=1<<5)\n--\n\n" (yes, the documented default value for type does not correspond to the actual implementation, but that's irrelevant here).  Compiling the extension module and running pydoc on it yields the following parsed signature for `parrot`: `parrot(voltage, state, action)` i.e. the `type` parameter got silently dropped.  (Note that `1<<5` can legitimately occur, e.g. as a bitmask flag, especially given that one currently cannot refer to globals in __text_signature__ (https://bugs.python.org/issue37881).)

----------
components: Extension Modules
messages: 385335
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: __text_signature__ parser silently drops arguments with certain unsupported default forms

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42976>
_______________________________________

From report at bugs.python.org  Wed Jan 20 09:48:54 2021
From: report at bugs.python.org (DDavid)
Date: Wed, 20 Jan 2021 14:48:54 +0000
Subject: [New-bugs-announce] [issue42977] Tkinter Optionmenu Too Narrow on
 Mac
Message-ID: <1611154134.32.0.566162791222.issue42977@roundup.psfhosted.org>


New submission from DDavid <zjdavid.2003 at gmail.com>:

Hello,

I am building a program with Tkinter. However, I found the Optionmenu widget has inconsistent behaviors on default lengths on a Mac and a Windows PC. 

So I set the width of an optionmenu through menu.configure(width=70). On Windows, this correctly sets the menu width to 70 when the program runs. However, on a Mac, the menu is still very narrow at the start. It only expands to 70 after I select an option. 

I have attached a screenshot to help explain the issue. It does not occur on Windows.

----------
components: Tkinter
files: TKinter Menu issue.png
messages: 385345
nosy: zjdavid
priority: normal
severity: normal
status: open
title: Tkinter Optionmenu Too Narrow on Mac
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49752/TKinter Menu issue.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42977>
_______________________________________

From report at bugs.python.org  Wed Jan 20 10:40:27 2021
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Wed, 20 Jan 2021 15:40:27 +0000
Subject: [New-bugs-announce] [issue42978] Improve error message when "self"
 is missing from the method definition
Message-ID: <1611157227.52.0.513625976683.issue42978@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

We currently say:

Traceback (most recent call last):
  File "/Users/pgalindo3/github/python/master/lel.py", line 5, in <module>
    A().foo(1,2)
TypeError: foo() takes 2 positional arguments but 3 were given

but we should say:
Traceback (most recent call last):
  File "/Users/pgalindo3/github/python/master/lel.py", line 5, in <module>
    A().foo(1,2)
TypeError: A.foo() takes 2 positional arguments but 3 were given. Did you forget 'self' in the method definition?

----------
messages: 385349
nosy: pablogsal
priority: normal
severity: normal
status: open
title: Improve error message when "self" is missing from the method definition

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42978>
_______________________________________

From report at bugs.python.org  Wed Jan 20 12:26:06 2021
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 20 Jan 2021 17:26:06 +0000
Subject: [New-bugs-announce] [issue42979] _zoneinfo: zoneinfomodule_exec()
 doesn't check for PyDateTime_IMPORT failure
Message-ID: <1611163566.26.0.253539458839.issue42979@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

_zoneinfo starts with:

static int
zoneinfomodule_exec(PyObject *m)
{
    PyDateTime_IMPORT;

with:

#define PyDateTime_CAPSULE_NAME "datetime.datetime_CAPI"
#define PyDateTime_IMPORT \
    PyDateTimeAPI = (PyDateTime_CAPI *)PyCapsule_Import(PyDateTime_CAPSULE_NAME, 0)

If PyCapsule_Import() fails, zoneinfomodule_exec() returns 0 (success) with an exception raised.

It should check if the import succeeded or not. Concrete example on AIX where datetime cannot be imported:
https://bugs.python.org/issue42604#msg385347

"./python setup.py build" fails with:

Assertion failed: (item != NULL) ^ (PyErr_Occurred() != NULL), file  Objects/abstract.c, line 163

--

By the way, the import machinery should raise a SystemError if a module exec function raises an exception *and* reports a success: see _Py_CheckFunctionResult().

----------
components: Extension Modules
messages: 385355
nosy: vstinner
priority: normal
severity: normal
status: open
title: _zoneinfo: zoneinfomodule_exec() doesn't check for PyDateTime_IMPORT failure
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42979>
_______________________________________

From report at bugs.python.org  Wed Jan 20 13:44:19 2021
From: report at bugs.python.org (Will Noble)
Date: Wed, 20 Jan 2021 18:44:19 +0000
Subject: [New-bugs-announce] [issue42980] argparse: GNU-style help formatter
Message-ID: <1611168259.85.0.840894983384.issue42980@roundup.psfhosted.org>


New submission from Will Noble <author at will.party>:

argparse allows GNU-style long options (with '=' in between the option and the value as opposed to a space): https://docs.python.org/3/library/argparse.html#option-value-syntax

Call it pickiness, but I'd like to be able to print the help message in that format. Looking at the code, a simple refactor would be much cleaner than implementing my own custom help formatter and having to copy-paste a bunch of implementation details that are potentially subject to change in future releases. I'll submit a patch shortly.

----------
components: Library (Lib)
messages: 385359
nosy: will
priority: normal
severity: normal
status: open
title: argparse: GNU-style help formatter
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42980>
_______________________________________

From report at bugs.python.org  Wed Jan 20 14:02:38 2021
From: report at bugs.python.org (Illia Volochii)
Date: Wed, 20 Jan 2021 19:02:38 +0000
Subject: [New-bugs-announce] [issue42981] Error messages raised by
 _hashlib_scrypt_impl() are slightly misleading
Message-ID: <1611169358.29.0.992442760668.issue42981@roundup.psfhosted.org>


New submission from Illia Volochii <illia.volochii at gmail.com>:

Error messages state that maxmem and dklen must be smaller than INT_MAX, but values equal to INT_MAX are allowed too.

Also, another error message states that n must be a power of 2, but it does not mention that 1 or 2? is not allowed.

----------
components: Extension Modules
messages: 385360
nosy: illia-v
priority: normal
severity: normal
status: open
title: Error messages raised by _hashlib_scrypt_impl() are slightly misleading

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42981>
_______________________________________

From report at bugs.python.org  Wed Jan 20 15:06:39 2021
From: report at bugs.python.org (Illia Volochii)
Date: Wed, 20 Jan 2021 20:06:39 +0000
Subject: [New-bugs-announce] [issue42982] Update suggested number of
 iterations for pbkdf2_hmac()
Message-ID: <1611173199.26.0.961948207202.issue42982@roundup.psfhosted.org>


New submission from Illia Volochii <illia.volochii at gmail.com>:

Documentation [1] suggests using at least 100,000 iterations of SHA-256 as of 2013.

Currently, it is 2021, and it is common to use much more iterations.
For example, Django will use 260,000 by default in the next 3.2 LTS release and 320,000 in 4.0 [2][3].

I suggest suggesting at least 250,000 iterations that is a somewhat round number close to the one used by modern libraries.

[1] https://docs.python.org/3/library/hashlib.html#hashlib.pbkdf2_hmac
[2] https://github.com/django/django/commit/f2187a227f7a3c80282658e699ae9b04023724e5
[3] https://github.com/django/django/commit/a948d9df394aafded78d72b1daa785a0abfeab48

----------
assignee: docs at python
components: Documentation
messages: 385365
nosy: docs at python, illia-v
priority: normal
severity: normal
status: open
title: Update suggested number of iterations for pbkdf2_hmac()

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42982>
_______________________________________

From report at bugs.python.org  Wed Jan 20 15:07:16 2021
From: report at bugs.python.org (Jason Williams)
Date: Wed, 20 Jan 2021 20:07:16 +0000
Subject: [New-bugs-announce] [issue42983] sys.argv incorrectly parses
 command lines with args in environment variables.
Message-ID: <1611173236.13.0.461370515171.issue42983@roundup.psfhosted.org>


New submission from Jason Williams <jasonjwwilliams at gmail.com>:

Passing arguments to a Python script using an environment variable like:

export en_auth="--arg1 test --arg2 \"$(<private_key.pem)\""
./test.py $en_auth

Incorrectly treats the value of arg2 as another argument rather than a value because the file value being inserted starts with "--":
-----BEGIN EC PARAMETERS----- BggqhkjOPQMBBw== -----END EC PARAMETERS-----

sys.argv parses it as:
['--arg1', 'test', '--arg2', '"-----BEGIN', 'EC', 'PARAMETERS-----', 'BggqhkjOPQMBBw==', '-----END', 'EC', 'PARAMETERS-----"']

If the arguments are provided directly to the script it parses correctly:
./test.py --arg1 test --arg2 "$(<private_key.pem)"

['--base-api-url', 'test', '--provisioning-device-private-key', '-----BEGIN EC PARAMETERS-----']

This looks like two bugs...sys.argv is splitting the value incorrectly because its ignoring the quoting, and argparse is treating values that start with -- as arguments.

----------
files: test.py
messages: 385366
nosy: Jason Williams
priority: normal
severity: normal
status: open
title: sys.argv incorrectly parses command lines with args in environment variables.
versions: Python 3.8
Added file: https://bugs.python.org/file49753/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42983>
_______________________________________

From report at bugs.python.org  Wed Jan 20 15:39:50 2021
From: report at bugs.python.org (Rob Simsiman)
Date: Wed, 20 Jan 2021 20:39:50 +0000
Subject: [New-bugs-announce] [issue42984] Python 3.7 and qt Libraries
Message-ID: <1611175190.18.0.861971223122.issue42984@roundup.psfhosted.org>


New submission from Rob Simsiman <rsimsiman at mitre.org>:

Hello,

I am trying to run an application on CentOS-8 that is unable to find qt4 libraries, namely libQtCore.so.4. My PC has qt5 libraries, namely libQt5Core.so.5. How can I configure my Python 3.7 to look for the qt5 libraries instead of the qt4 libraries. Or, do I simply need to grab a newer version of Python?

Thank you,

Rob Simsiman

----------
components: ctypes
messages: 385367
nosy: rsimsiman
priority: normal
severity: normal
status: open
title: Python 3.7 and qt Libraries
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42984>
_______________________________________

From report at bugs.python.org  Wed Jan 20 16:18:02 2021
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 20 Jan 2021 21:18:02 +0000
Subject: [New-bugs-announce] [issue42985] AMD64 Arch Linux Asan 3.x fails:
 command timed out: 1200 seconds without output
Message-ID: <1611177482.17.0.749354598383.issue42985@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

The AMD64 Arch Linux Asan 3.x buildbot worker started to fail at build 262:
https://buildbot.python.org/all/#/builders/582/builds/262
-----------
...
0:39:09 load avg: 1.15 running: test_multiprocessing_forkserver (30.0 sec)
0:39:39 load avg: 1.64 running: test_multiprocessing_forkserver (1 min)
0:39:53 load avg: 1.50 [419/420] test_multiprocessing_forkserver passed (1 min 12 sec)
0:39:53 load avg: 1.50 [420/420] test_dynamicclassattribute passed

command timed out: 1200 seconds without output running (...)
process killed by signal 9
program finished with exit code -1
elapsedTime=3595.378040
-----------

----------
components: Tests
messages: 385373
nosy: vstinner
priority: normal
severity: normal
status: open
title: AMD64 Arch Linux Asan 3.x fails: command timed out: 1200 seconds without output
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42985>
_______________________________________

From report at bugs.python.org  Wed Jan 20 17:51:26 2021
From: report at bugs.python.org (neonene)
Date: Wed, 20 Jan 2021 22:51:26 +0000
Subject: [New-bugs-announce] [issue42986] pegen parser: Crash on SyntaxError
 with f-string on Windows
Message-ID: <1611183086.81.0.804124765933.issue42986@roundup.psfhosted.org>


New submission from neonene <nicesalmon at gmail.com>:

On Windows, Python master crashes using f-string (which has an invalid char with braces) on line 3 and after.
It seems the issue is from commit (e5fe509054183bed9aef42c92da8407d339e8af8).

I tried

1) exec("f'{.}'")
2) exec("\nf'{.}'")
3) exec("\n\nf'{.}'")

commands and results are

1) expected
    >>> exec("f'{.}'")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 1
        (.)
         ^
    SyntaxError: f-string: invalid syntax

2) unexpected (caret indicates nothing)
    >>> exec("\nf'{.}'")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 2

        ^
    SyntaxError: f-string: invalid syntax

3) python crashes
    >>> exec("\n\nf'{.}'")

----------
components: Interpreter Core
messages: 385377
nosy: lys.nikolaou, neonene, pablogsal
priority: normal
severity: normal
status: open
title: pegen parser: Crash on SyntaxError with f-string on Windows
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42986>
_______________________________________

From report at bugs.python.org  Wed Jan 20 23:02:35 2021
From: report at bugs.python.org (bfpiaoran)
Date: Thu, 21 Jan 2021 04:02:35 +0000
Subject: [New-bugs-announce] [issue42987] HTTP header injection in urllib on
 windows
Message-ID: <1611201755.22.0.579642102837.issue42987@roundup.psfhosted.org>


New submission from bfpiaoran <bfpiaoran at qq.com>:

Recently,(on windows) I discovered a security issue during a security review due to urllib.I checked the document and found that it has been fixed at https://bugs.python.org/issue22928.but My python version is 3.7.2 over the fix bug version .Then tried to find the reason
The code is probably as follows under the django framework

```
from urllib.request import urlopen



remote_urls = request.POST.getlist("source[]", [])
for remote_url in remote_urls:
    remote_image = urlopen(remote_url)

```
source[]=http://127.0.0.1:6379/%0d%0aset%20ce%20test%0d%0a/1.jpg

Changing the address will make urlopen connect to the local machine 6379 and set the key value

python3 



Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 22:20:52) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from django.utils.six.moves.urllib.request import urlopen
>>> remote_image = urlopen('http://127.0.0.1:6379/\r\nset ce test\r\n/1.jpg')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 222, in urlopen        return opener.open(url, data, timeout)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 525, in open           response = self._open(req, data)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 543, in _open          '_open', req)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 503, in _call_chain    result = func(*args)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 1345, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 1320, in do_open
    r = h.getresponse()
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\http\client.py", line 1321, in getresponse
    response.begin()
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\http\client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "C:\Users\cuijianxiong\AppData\Local\Programs\Python\Python37-32\lib\http\client.py", line 278, in _read_status
    raise BadStatusLine(line)
http.client.BadStatusLine: $-1



Django will encode and decode urlopen because of the url specification, and accept requests with "\r\n" in the url


It is the responsibility of the developer leveraging Python and its HTTP client libraries to ensure that their (web) application acts in accordance to official HTTP specifications and that no threats to security will arise from their code.
However, newlines inside headers are arguably a special case of breaking the conformity with RFC's in regard to the allowed character set. No illegal character used inside a HTTP header is likely to have a compromising side effect on back-end clients and servers and the integrity of their communication, as a result of the leniency of most web servers. However, a newline character (0x0A) embedded in a HTTP header invariably has the semantic consequence of denoting the start of an additional header line. To put it differently, not sanitizing headers in complete accordance to RFC's could be seen as as virtue in that it gives the programmer a maximum amount of freedom, without having to trade it for any likely or severe security ramifications, so that they may use illegal characters in testing environments and environments that are outlined by an expliticly less strict interpretation of the HTTP protocol. Newlines are special in that they enable anyone who is able to influence the header content, to, in effect, perform additional invocations to add_header().


urlopen('http://106.53.251.216:8888/\r\Auth: test\r\n/1.jpg')


Ncat: Version 7.70 ( https://nmap.org/ncat )
Ncat: Listening on :::8888
Ncat: Listening on 0.0.0.0:8888
Ncat: Connection from 111.202.227.139.
Ncat: Connection from 111.202.227.139:14411.
\Auth: test
/1.jpg HTTP/1.1
Accept-Encoding: identity
Host: 106.53.251.216:8888
User-Agent: Python-urllib/3.7
Connection: close



I think https://bugs.python.org/issue22928 is not very thorough and does not fully detect line breaks

----------
messages: 385389
nosy: bfpiaoran, lemburg, orsenthil
priority: normal
severity: normal
status: open
title: HTTP header injection in urllib on windows
type: security
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42987>
_______________________________________

From report at bugs.python.org  Thu Jan 21 07:18:37 2021
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Thu, 21 Jan 2021 12:18:37 +0000
Subject: [New-bugs-announce] [issue42988] Information disclosure via pydoc -p
Message-ID: <1611231517.86.0.296188996897.issue42988@roundup.psfhosted.org>


New submission from Miro Hron?ok <miro at hroncok.cz>:

Hello Python security,
a Fedora user has reported the following security vulnerability to us (I was able to verify it):

Running `pydoc -p` allows other local users to extract arbitrary files.

Steps to Reproduce:
1. start pydoc on a port
2. as a different user guess or extract the port
3. call getfile on the server to extract arbitrary files, e.g. http://localhost:8888/getfile?key=/home/dave/.ssh/id_rsa

Actual results:
any local user on the multi-user system can read all my keys and secrets

Expected results:
Access is prevented.

Additional info:
At least a warning should be printed, that this is insecure on multi-user systems.

Python notebook works around this by providing a token that is required to access the notepad. Depending on the system being able to read arbitrary files can allow to impersonate my, by  e.g. stealing my ssh-key (if it is non-encrypted) 



I've originally reported this to security at python.org but I was asked to open a public issue here.

----------
components: Library (Lib)
messages: 385412
nosy: hroncok
priority: normal
severity: normal
status: open
title: Information disclosure via pydoc -p
type: security
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42988>
_______________________________________

From report at bugs.python.org  Thu Jan 21 10:00:13 2021
From: report at bugs.python.org (Andrew C)
Date: Thu, 21 Jan 2021 15:00:13 +0000
Subject: [New-bugs-announce] [issue42989] Bug
Message-ID: <1611241213.49.0.0569603079776.issue42989@roundup.psfhosted.org>


New submission from Andrew C <ASCRong at hotmail.com>:

if playerChoice == "2": reports the ":" as a "syntax error"

----------
components: Windows
messages: 385427
nosy: ASCRong, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Bug
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42989>
_______________________________________

From report at bugs.python.org  Thu Jan 21 11:05:52 2021
From: report at bugs.python.org (Mark Shannon)
Date: Thu, 21 Jan 2021 16:05:52 +0000
Subject: [New-bugs-announce] [issue42990] Improve the C code for calling
 Python code
Message-ID: <1611245152.12.0.0348161555926.issue42990@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

Currently, to make a call to Python (modules, classes, etc, not just functions) from C has to use the monster that is `_PyEval_EvalCode`.
As Python has adding features over the years, _PyEval_EvalCode has grown and grown. It is time for a refactor.

`_PyEval_EvalCode` takes 16, yes sixteen parameters!
Those 16 parameters fall into two main groups, one describing the function being called, and the other describing the arguments to the call.
Due to the jumbo size and complexity of _PyEval_EvalCode, we also have specialised forms of it, e.g. _PyFunction_Vectorcall that handle common cases and then bail to _PyEval_EvalCode in other cases.

In outline _PyEval_EvalCode performs the following actions:
1. Make a new frame.
2. Fill in that frame using the arguments and defaults provided.
3. If the code object has flags set for a generator, or coroutine, return a new generator or coroutine.
4. Otherwise call the interpreter with the newly created frame.

As _PyEval_EvalCode or its earlier equivalents have gained arguments, they have a left of list of legacy functions. It is not clear what is the "correct" function to use in C extensions. Hopefully extension code uses the `PyObject_Call` API, but `PyEval_EvalCodeEx` is part of the C-API.


To improve this situation, I propose:

A new C struct, the "frame descriptor" which contains the code object, defaults, globals, and names that describe the code to be executed. `PyFunctionObject` would wrap this, to simplify the common case of calling a Python function. 

Split the Eval functions into vector and tuple/dict forms, both forms taking a "frame descriptor", as well as the arguments.

Mark the older forms of the Eval functions as "legacy", creating a local "frame descriptor" and transforming the arguments in vector or tuple/dict forms, in the legacy functions.

Factor out the frame creation part of the Eval functions.


The above changes will be necessary for PEP 651, but I think would be a worthwhile improvement even if PEP 651 is not accepted.

----------
assignee: Mark.Shannon
components: Interpreter Core
messages: 385432
nosy: Mark.Shannon
priority: normal
severity: normal
stage: needs patch
status: open
title: Improve the C code for calling Python code

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42990>
_______________________________________

From report at bugs.python.org  Thu Jan 21 14:43:34 2021
From: report at bugs.python.org (Ramon)
Date: Thu, 21 Jan 2021 19:43:34 +0000
Subject: [New-bugs-announce] [issue42991] support for splitting multichannel
 audio fragments in audioop module
Message-ID: <1611258214.16.0.1675342268.issue42991@roundup.psfhosted.org>


New submission from Ramon <ramonfraterman at gmail.com>:

All functions from the audioop module that work on stereo fragments, provide the same behaviour on multichannel (i.e. 5.1 channel LPCM) fragments. This is, however, not true for the tomono() function, that only makes sense when supplied with a 2-channel fragment.

Therefore, I suggest adding an extra function to the module that demultiplexes any N-channel fragment and returns a given channel as mono fragment:
audioop.demux(fragment, width, nChannels, channel)

When, for example, applied to a 16 bit stereo fragment, audioop.demux(fragment, 2, 2, 0) would give the same result as audioop.tomono(fragment, 2, 1.0, 0).

----------
components: Extension Modules
messages: 385443
nosy: Th4R4
priority: normal
severity: normal
status: open
title: support for splitting multichannel audio fragments in audioop module
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42991>
_______________________________________

From report at bugs.python.org  Thu Jan 21 15:56:49 2021
From: report at bugs.python.org (Ron Hoffmann)
Date: Thu, 21 Jan 2021 20:56:49 +0000
Subject: [New-bugs-announce] [issue42992] Tkinter bbox coordinates
 incorrectly drawn
Message-ID: <1611262609.83.0.00898948617335.issue42992@roundup.psfhosted.org>


New submission from Ron Hoffmann <rhoffmann at muskokawoods.com>:

position coordinates retrieved from any object on a canvas with
pos = canvas.bbox(object) are returned correctly

but when drawn on the canvas (x0,y0) are correct, but
(x1, y1) are not drawn in the proper positions. x1 has been divided by 2 somewhere and y1 as been multiplied by 2 somewhere.
in order for the bounding box to be drawn correctly
x1 and y1 need to be recalculated as follows

x1 = pos[0] + ( ( pos[2] - pos[0] ) * 2 )
y1 = pos[1] + ( ( pos[3] - pos[1] ) / 2 )

----------
components: Tkinter
messages: 385446
nosy: rhoffmann
priority: normal
severity: normal
status: open
title: Tkinter bbox coordinates incorrectly drawn
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42992>
_______________________________________

From report at bugs.python.org  Thu Jan 21 16:13:02 2021
From: report at bugs.python.org (Christian Buhtz)
Date: Thu, 21 Jan 2021 21:13:02 +0000
Subject: [New-bugs-announce] [issue42993] doc
 xml.etree.ElementTree.ElementTree.write does not mention attribute order
Message-ID: <1611263582.03.0.301072283614.issue42993@roundup.psfhosted.org>


New submission from Christian Buhtz <c.buhtz at posteo.jp>:

The docs for 'xml.etree.ElementTree.ElementTree.write' in Python 3.7 (and possible earlier) does not say a word about the ordering of the attributes.

This makes unittesting hard.

But looking in the code tells me that the attributes are ordered lexically. Important to know. So please upgrade the docu about that.

In Python 3.8 the ordering behavior changed again but is mentioned in the docu. So no need to change in 3.8 or later.

----------
assignee: docs at python
components: Documentation
messages: 385448
nosy: buhtz, docs at python
priority: normal
severity: normal
status: open
title: doc xml.etree.ElementTree.ElementTree.write does not mention attribute order
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42993>
_______________________________________

From report at bugs.python.org  Thu Jan 21 16:41:52 2021
From: report at bugs.python.org (Nathan Beals)
Date: Thu, 21 Jan 2021 21:41:52 +0000
Subject: [New-bugs-announce] [issue42994] Missing MIME types for opus,
 AAC and 3gpp(2)
Message-ID: <1611265312.85.0.212937423943.issue42994@roundup.psfhosted.org>


New submission from Nathan Beals <nathan.beals1 at gmail.com>:

These are officially recognized MIME types by IANA: https://www.iana.org/assignments/media-types/media-types.xhtml#audio

- .opus: audio/opus (https://www.iana.org/assignments/media-types/audio/opus and https://tools.ietf.org/html/rfc7845 for recommended file extension)
- .aac: audip/aac (https://www.iana.org/assignments/media-types/audio/aac)
- .3gp: audio/3gpp (https://www.iana.org/assignments/media-types/audio/3gpp)
- .3g2: audio/3gpp2 (https://www.iana.org/assignments/media-types/audio/3gpp2)

----------
components: Library (Lib)
messages: 385449
nosy: nbeals
priority: normal
severity: normal
status: open
title: Missing MIME types for opus, AAC and 3gpp(2)
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42994>
_______________________________________

From report at bugs.python.org  Thu Jan 21 17:04:11 2021
From: report at bugs.python.org (Brendan Gerrity)
Date: Thu, 21 Jan 2021 22:04:11 +0000
Subject: [New-bugs-announce] [issue42995] Add PurePath.with_suffix_appended()
Message-ID: <1611266651.15.0.998031464768.issue42995@roundup.psfhosted.org>


New submission from Brendan Gerrity <bgerrity at ucdavis.edu>:

Appending a new suffix to a file is common operation.

The operations don't come across as elegant:

e.g. `foo_path.with_suffix(foo_path.suffix + ".old")`

----------
components: Library (Lib)
messages: 385451
nosy: bgerrity
priority: normal
severity: normal
status: open
title: Add PurePath.with_suffix_appended()
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42995>
_______________________________________

From report at bugs.python.org  Thu Jan 21 18:24:06 2021
From: report at bugs.python.org (Illia Volochii)
Date: Thu, 21 Jan 2021 23:24:06 +0000
Subject: [New-bugs-announce] [issue42996] hashlib documentation references
 an obsolete RFC 2898
Message-ID: <1611271446.83.0.500119366754.issue42996@roundup.psfhosted.org>


New submission from Illia Volochii <illia.volochii at gmail.com>:

RFC 2898 mentioned in the "See also" section of hashlib documentation was superseded by RFC 8018.

https://docs.python.org/3/library/hashlib.html
https://www.ietf.org/rfc/rfc8018.txt

----------
assignee: docs at python
components: Documentation
messages: 385457
nosy: docs at python, illia-v
priority: normal
severity: normal
status: open
title: hashlib documentation references an obsolete RFC 2898

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42996>
_______________________________________

From report at bugs.python.org  Thu Jan 21 20:20:05 2021
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Fri, 22 Jan 2021 01:20:05 +0000
Subject: [New-bugs-announce] [issue42997] Improve error message for missing
 : before suites
Message-ID: <1611278405.38.0.0749709348669.issue42997@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

Instead of displaying a generic syntax error:

Python 3.8.6 (default, Oct 10 2020, 18:31:21)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> for x in range
  File "<stdin>", line 1
    for x in range
                 ^
SyntaxError: invalid syntax
>>>

we could display:

>>> for x in range
  File "<stdin>", line 1
    for x in range
                  ^
SyntaxError: expected ':'

The same idea applies for every suite that has a missing ':'

----------
messages: 385463
nosy: pablogsal
priority: normal
severity: normal
status: open
title: Improve error message for missing : before suites
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42997>
_______________________________________

From report at bugs.python.org  Thu Jan 21 22:57:15 2021
From: report at bugs.python.org (Barney Gale)
Date: Fri, 22 Jan 2021 03:57:15 +0000
Subject: [New-bugs-announce] [issue42998] pathlib.Path: add `username`
 argument to `home()`
Message-ID: <1611287835.84.0.363729514225.issue42998@roundup.psfhosted.org>


New submission from Barney Gale <barney.gale at gmail.com>:

The `pathlib.Path.home()` function looks like:

    @classmethod
    def home(cls):
        """Return a new path pointing to the user's home directory (as
        returned by os.path.expanduser('~')).
        """
        return cls(cls()._flavour.gethomedir(None))

If we add a `username=None` parameter and pass this to `gethomedir()` we can easily add a lookup of another user's home directory, so:

    import pathlib
    username = 'phil'
    pathlib.Path.home(username) == pathlib.Path('~' + username).expanduser()

----------
components: Library (Lib)
messages: 385472
nosy: barneygale
priority: normal
severity: normal
status: open
title: pathlib.Path: add `username` argument to `home()`
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42998>
_______________________________________

From report at bugs.python.org  Thu Jan 21 23:24:20 2021
From: report at bugs.python.org (Barney Gale)
Date: Fri, 22 Jan 2021 04:24:20 +0000
Subject: [New-bugs-announce] [issue42999] `pathlib.Path.link_to()`
 documentation is misleading
Message-ID: <1611289460.25.0.942447627462.issue42999@roundup.psfhosted.org>


New submission from Barney Gale <barney.gale at gmail.com>:

Compare the documentation of `symlink_to()` and `link_to()`:

.. method:: Path.symlink_to(target, target_is_directory=False)
    Make this path a symbolic link to *target*.

.. method:: Path.link_to(target)
    Create a hard link pointing to a path named *target*.

In fact, `link_to()` does something like:

    Make *target* a hard link to this path.

Which is unexpected given the naming and inconsistency with `symlink_to()`, but it's the way the current implementation works, and so the documentation should reflect that.

Adding a replacement `hardlink_to()` function is covered here in bpo-39950.

----------
assignee: docs at python
components: Documentation
messages: 385474
nosy: barneygale, docs at python
priority: normal
severity: normal
status: open
title: `pathlib.Path.link_to()` documentation is misleading
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42999>
_______________________________________

From report at bugs.python.org  Thu Jan 21 23:46:13 2021
From: report at bugs.python.org (Darren Skidmore)
Date: Fri, 22 Jan 2021 04:46:13 +0000
Subject: [New-bugs-announce] [issue43000] All SSL requests fail with
 WRONG_VERSION_NUMBER when a packet sniffer is open
Message-ID: <1611290773.31.0.124833320524.issue43000@roundup.psfhosted.org>


New submission from Darren Skidmore <darrenrs0515 at gmail.com>:

As of Python 3.9.1, when attempting to perform any SSL requests when a packet sniffer tool (e.g. Telerik Fiddler) is intercepting SSL traffic, the program will hang for about a minute and then crash with a WRONG_VERSION_NUMBER error. This has been tested to occur with urllib2 and requests modules. Alternatives such as verify=False and adding the Fiddler Root Certificate to Python's cacert.pem file do not rectify this error.

This traceback was generated when attempting to access https://example.com:443 while Fiddler 5.0.20204.45441 was open:

Traceback (most recent call last):
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 696, in urlopen
    self._prepare_proxy(conn)
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 964, in _prepare_proxy
    conn.connect()
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 359, in connect
    conn = self._connect_tls_proxy(hostname, conn)
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 496, in _connect_tls_proxy
    return ssl_wrap_socket(
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 432, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 474, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock)
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Users\Darren\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)

This exact question has recently been asked on StackOverflow: https://stackoverflow.com/questions/65516325/ssl-wrong-version-number-on-python-request

----------
assignee: christian.heimes
components: SSL
messages: 385476
nosy: christian.heimes, darrenrs
priority: normal
severity: normal
status: open
title: All SSL requests fail with WRONG_VERSION_NUMBER when a packet sniffer is open
type: crash
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43000>
_______________________________________

From report at bugs.python.org  Fri Jan 22 06:35:38 2021
From: report at bugs.python.org (Alexei S)
Date: Fri, 22 Jan 2021 11:35:38 +0000
Subject: [New-bugs-announce] [issue43001] python3.9.1 test_embed
 test_tabnanny failed
Message-ID: <1611315338.64.0.350030617212.issue43001@roundup.psfhosted.org>


New submission from Alexei S <asholomitskiy84 at gmail.com>:

== Tests result: FAILURE then FAILURE ==

396 tests OK.

2 tests failed:
    test_embed test_tabnanny

27 tests skipped:
    test_bz2 test_curses test_dbm_gnu test_dbm_ndbm test_devpoll
    test_gdb test_idle test_ioctl test_kqueue test_lzma test_msilib
    test_ossaudiodev test_readline test_smtpnet test_ssl
    test_startfile test_tcl test_tix test_tk test_ttk_guionly
    test_ttk_textonly test_turtle test_winconsoleio test_winreg
    test_winsound test_zipfile64 test_zoneinfo

2 re-run tests:
    test_embed test_tabnanny

Total duration: 10 min 58 sec
Tests result: FAILURE then FAILURE
make: *** [Makefile:1199: test] Error 2

----------
components: Tests
messages: 385490
nosy: asholomitskiy84
priority: normal
severity: normal
status: open
title: python3.9.1 test_embed test_tabnanny failed
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43001>
_______________________________________

From report at bugs.python.org  Fri Jan 22 08:52:41 2021
From: report at bugs.python.org (Ram Rachum)
Date: Fri, 22 Jan 2021 13:52:41 +0000
Subject: [New-bugs-announce] [issue43002] Exception chaining accepts
 exception classes
Message-ID: <1611323561.1.0.365445213907.issue43002@roundup.psfhosted.org>


New submission from Ram Rachum <ram at rachum.com>:

I saw this line of code today: https://github.com/hyperledger/sawtooth-sdk-python/commit/c27b962541c9ae68fd1e6dc691ddee883234f112#diff-eb008203eae2160c5e14c42e5fd2eee164709a93bf5136fa79cc256d4e46eaffR92

I was about to tell this guy that his code is bad since the part after the `from` should be a specific exception, not just an exception class. I thought I should double-check myself first. Lo and behold: 

```
$ cat x.py
def f():
    raise TypeError

try:
    f()
except TypeError:
    raise ValueError from TypeError

$ python x.py
TypeError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "x.py", line 7, in <module>
    raise ValueError from TypeError
ValueError
```

The line doesn't fail, but it silently trims away the stacktrace of the previous exception. (Normally the stacktrace would include what's inside the `f` function.)

I'm not sure whether this is intended behavior or not. The documentation does say "the second expression must be another exception class or instance" but (a) it's not clear what the use case is when it's a class and (b) I doubt that the person who wrote the code above, and any other person who writes such code, would be aware that their traceback got snipped until it's too late and they're stuck with a bug report with incomplete information.

----------
components: Interpreter Core
messages: 385499
nosy: cool-RR
priority: normal
severity: normal
status: open
title: Exception chaining accepts exception classes
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43002>
_______________________________________

From report at bugs.python.org  Fri Jan 22 10:01:43 2021
From: report at bugs.python.org (Michael Clerx)
Date: Fri, 22 Jan 2021 15:01:43 +0000
Subject: [New-bugs-announce] [issue43003] Parts of the API will be removed
 in Python 4.0 ?
Message-ID: <1611327703.19.0.312122851749.issue43003@roundup.psfhosted.org>


New submission from Michael Clerx <cell at michaelclerx.com>:

The unicode C-API documentation has a deprecation warning on this page:

  https://docs.python.org/3.9/c-api/unicode.html#deprecated-py-unicode-apis

stating some functions are

  Deprecated since version 3.3, will be removed in version 4.0.

But as far as I understand there are no current plans for a 4.0.
Should the text read "removed in version 3.10" ?

----------
assignee: docs at python
components: Documentation
messages: 385501
nosy: MichaelClerx, docs at python
priority: normal
severity: normal
status: open
title: Parts of the API will be removed in Python 4.0 ?

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43003>
_______________________________________

From report at bugs.python.org  Fri Jan 22 15:13:42 2021
From: report at bugs.python.org (Tyler Yep)
Date: Fri, 22 Jan 2021 20:13:42 +0000
Subject: [New-bugs-announce] [issue43004] No type variables left in
 collections.abc.Callable
Message-ID: <1611346422.35.0.0652795827703.issue43004@roundup.psfhosted.org>


New submission from Tyler Yep <tyler.yep at robinhood.com>:

The following code crashes when I try to run it, even though it passes when I use `from typing import Callable` instead.

```
from collections.abc import Callable
from typing import Any, TypeVar

V = TypeVar("V")
Function = Callable[[list[V], V, V], float]


def random_fn(fn: Function[Any]) -> Function[Any]:
    return fn
```

----------
messages: 385511
nosy: tyler.yep
priority: normal
severity: normal
status: open
title: No type variables left in collections.abc.Callable
type: crash
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43004>
_______________________________________

From report at bugs.python.org  Fri Jan 22 18:22:37 2021
From: report at bugs.python.org (Ixio)
Date: Fri, 22 Jan 2021 23:22:37 +0000
Subject: [New-bugs-announce] [issue43005] Package Relative Imports - double
 dot doc example not working
Message-ID: <1611357757.62.0.234693814777.issue43005@roundup.psfhosted.org>


New submission from Ixio <ektech at gmx.com>:

I've been trying to do a double dot import without success, I've tried going back to the official documentation in order to figure it out however even the example shown does not work.

Official example : https://docs.python.org/3/reference/import.html#package-relative-imports

The following instructions mimic pretty well the doc example :

```bash
mkdir package
cd package
touch __init__.py
mkdir subpackage1
touch subpackage1/__init__.py
mkdir subpackage2
touch subpackage2/__init__.py
echo "eggs = 42" > subpackage2/moduleZ.py
echo "from ..subpackage2.moduleZ import eggs" > subpackage1/moduleX.py
python subpackage1/moduleX.py
cd subpackage1
python moduleX.py

cd ..
echo "from subpackage1 import moduleX" > moduleA.py
python moduleA.py
```

However I get 3 times the following error :
    from ..subpackage2.moduleZ import eggs
ImportError: attempted relative import beyond top-level package

Even though the docs say "In [...] subpackage1/moduleX.py [...] the following are valid relative imports: [...] from ..subpackage2.moduleZ import eggs".

I have no idea how to fix the doc as I would love to do "from ..subpackage2.moduleZ import eggs" from "subpackage1/moduleX.py" since that's the structure I'm trying to set up for a Python project of mine but obviously it does not seem to work.

My system uses Python 3.9 and I've used Docker to try this scenario on Python 3.8, 3.5 and even 3.2: it does not work either. I've also tried on another machine and asked a friend to try, no luck.

I'm hoping someone here can explain how double dot imports work and fix the documentation accordingly.

----------
assignee: docs at python
components: Documentation
messages: 385515
nosy: Ixio, brett.cannon, docs at python, eric.snow, ncoghlan
priority: normal
severity: normal
status: open
title: Package Relative Imports - double dot doc example not working
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43005>
_______________________________________

From report at bugs.python.org  Sat Jan 23 01:24:23 2021
From: report at bugs.python.org (Zac Hatfield-Dodds)
Date: Sat, 23 Jan 2021 06:24:23 +0000
Subject: [New-bugs-announce] [issue43006] Changed behaviour of
 inspect.signature() in Python 3.10
Message-ID: <1611383063.08.0.672499153283.issue43006@roundup.psfhosted.org>


New submission from Zac Hatfield-Dodds <zac.hatfield.dodds at gmail.com>:

Consider the following snippet, which passes on Python 3.9 and earlier:


    import inspect

    def f(x: int = None):
        pass

    print(inspect.signature(constructor))
    assert inspect.signature(constructor).parameters["a"].annotation == int


But under Python 3.10 (alpha 4), the annotation is instead read as Optional[int].  This is correct for typing.get_type_hints(), but *not* for inspect.signature().

I therefore suspect that this is an accidental side-effect from support for PEP-563 deferred evaluation of annotations.

----------
components: Library (Lib)
messages: 385524
nosy: Zac Hatfield-Dodds
priority: normal
severity: normal
status: open
title: Changed behaviour of inspect.signature() in Python 3.10
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43006>
_______________________________________

From report at bugs.python.org  Sat Jan 23 05:40:30 2021
From: report at bugs.python.org (MeneMeneTekel)
Date: Sat, 23 Jan 2021 10:40:30 +0000
Subject: [New-bugs-announce] [issue43007] set_wakeup_fd() only works in main
 thread
Message-ID: <1611398430.41.0.0624041377425.issue43007@roundup.psfhosted.org>


New submission from MeneMeneTekel <mail at evation.eu>:

This problem is marked as solved in issue38563, BUT the solution refers to some signal issues being solved - I'm using Python 3.8.7 on windows 10, where a signal solution may not work. I.e. my django (vers. 3.1.5) website crashes every second call with the message:

  File "C:\Winpython\WPy64-3870\python-3.8.7.amd64\lib\asyncio\proactor_events.py", line 632, in __init__
    signal.set_wakeup_fd(self._csock.fileno())
ValueError: set_wakeup_fd only works in main thread

----------
components: asyncio
messages: 385528
nosy: MeneMeneTekel, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: set_wakeup_fd() only works in main thread
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43007>
_______________________________________

From report at bugs.python.org  Sat Jan 23 06:30:46 2021
From: report at bugs.python.org (Ken Hilton)
Date: Sat, 23 Jan 2021 11:30:46 +0000
Subject: [New-bugs-announce] [issue43008] IDLE ignores sys.excepthook
Message-ID: <1611401446.12.0.0532585832167.issue43008@roundup.psfhosted.org>


New submission from Ken Hilton <kenny2minecraft at gmail.com>:

Similar to https://bugs.python.org/issue12643 - IDLE ignores sys.excepthook and always prints exceptions.

----------
assignee: terry.reedy
components: IDLE
files: bpo-43008.patch
keywords: patch
messages: 385529
nosy: kenny2minecraft, terry.reedy
priority: normal
pull_requests: 23123
severity: normal
status: open
title: IDLE ignores sys.excepthook
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49757/bpo-43008.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43008>
_______________________________________

From report at bugs.python.org  Sat Jan 23 07:33:41 2021
From: report at bugs.python.org (hai shi)
Date: Sat, 23 Jan 2021 12:33:41 +0000
Subject: [New-bugs-announce] [issue43009] Port curses capi pointer array to
 a struct
Message-ID: <1611405221.76.0.555112594998.issue43009@roundup.psfhosted.org>


New submission from hai shi <shihai1991 at 126.com>:

As the title mentioned. A entire py_curses capi struct would be better than pointer array.

Related PR: PR 24186

----------
assignee: shihai1991
messages: 385532
nosy: shihai1991, vstinner
priority: normal
severity: normal
status: open
title: Port curses capi pointer array to a struct
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43009>
_______________________________________

From report at bugs.python.org  Sat Jan 23 09:26:11 2021
From: report at bugs.python.org (Erez Zinman)
Date: Sat, 23 Jan 2021 14:26:11 +0000
Subject: [New-bugs-announce] [issue43010] @functools.wraps and
 abc.abstractmethod interoperability
Message-ID: <1611411971.69.0.738980453065.issue43010@roundup.psfhosted.org>


New submission from Erez Zinman <erezinman.programmer at gmail.com>:

Consider the following code:

```
from abc import ABC, abstractmethod
from functools import wraps

class A(ABC):
    @abstractmethod
    def f(self):
        pass
    
    @wraps(f)
    def wrapper(self):
        print('f is called!')
        f()
        
class B(A):
    def f(self):
        print('f!')

B()
```

The last line of code results in the following error:
>>> TypeError: Can't instantiate abstract class B with abstract methods wrapper

That happens because `wraps` copies the `__dict__` of the original function. The problem is that at the point of declaration, the `__dict__` also contained `__isabstractmethod__=True` so it was copied as well, and it caused an error on the class' instantiation even though it contained no abstract methods. 
Moreover, this behavior is misleading because the the wrapper function is not abstract. 

Thanks.

----------
components: Extension Modules, Library (Lib)
messages: 385538
nosy: erezinman
priority: normal
severity: normal
status: open
title: @functools.wraps and abc.abstractmethod interoperability
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43010>
_______________________________________

From report at bugs.python.org  Sat Jan 23 09:28:06 2021
From: report at bugs.python.org (Zackery Spytz)
Date: Sat, 23 Jan 2021 14:28:06 +0000
Subject: [New-bugs-announce] [issue43011] DeprecationWarnings in test_ctypes
Message-ID: <1611412086.36.0.506119389141.issue43011@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

./python -m test test_ctypes
0:00:00 load avg: 1.65 Run tests sequentially
0:00:00 load avg: 1.65 [1/1] test_ctypes
/home/lubuntu2/cpython/Lib/ctypes/test/test_parameters.py:243: DeprecationWarning: invalid escape sequence \(
  self.assertRegex(repr(c_char_p.from_param(b'hihi')), "^<cparam 'z' \(0x[A-Fa-f0-9]+\)>$")
/home/lubuntu2/cpython/Lib/ctypes/test/test_parameters.py:244: DeprecationWarning: invalid escape sequence \(
  self.assertRegex(repr(c_wchar_p.from_param('hihi')), "^<cparam 'Z' \(0x[A-Fa-f0-9]+\)>$")

== Tests result: SUCCESS ==

1 test OK.

Total duration: 2.4 sec
Tests result: SUCCESS


These warnings were introduced in bpo-42938 / 916610ef90a0d0761f08747f7b0905541f0977c7.

----------
components: Tests, ctypes
messages: 385539
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: DeprecationWarnings in test_ctypes
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43011>
_______________________________________

From report at bugs.python.org  Sat Jan 23 13:15:55 2021
From: report at bugs.python.org (Barney Gale)
Date: Sat, 23 Jan 2021 18:15:55 +0000
Subject: [New-bugs-announce] [issue43012] Remove pathlib accessors
Message-ID: <1611425755.47.0.494853383169.issue43012@roundup.psfhosted.org>


New submission from Barney Gale <barney.gale at gmail.com>:

Per Pitrou:

> The original intent for the ?accessor? thing was to have a variant that did all accesses under a filesystem tree in a race condition-free way using openat and friends. It turned out to be much too hairy to actually implement, so was entirely abandoned, but the accessor abstraction was left there.

https://discuss.python.org/t/make-pathlib-extensible/3428/2

Accessors are:

- Lacking any internal purpose - '_NormalAccessor' is the only implementation
- Lacking any firm conceptual difference to `Path` objects themselves (inc. subclasses)
- Non-public, i.e. underscore prefixed - '_Accessor' and '_NormalAccessor' 
- Unofficially used to implement customized `Path` objects, but once once bpo-24132 lands there will be a supported route for that.

When bpo-24132 lands, I think we're free to remove accessors and simplify some code

----------
components: Library (Lib)
messages: 385556
nosy: barneygale
priority: normal
severity: normal
status: open
title: Remove pathlib accessors

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43012>
_______________________________________

From report at bugs.python.org  Sat Jan 23 16:20:31 2021
From: report at bugs.python.org (Terry J. Reedy)
Date: Sat, 23 Jan 2021 21:20:31 +0000
Subject: [New-bugs-announce] [issue43013] IDLE:  update code,
 mostly by cleanups of 2.x or 2to3 artifacts
Message-ID: <1611436831.18.0.743701112328.issue43013@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

Tal, just so you know in case you see a resulting merge conflict.  I've been waiting for years to do these until most PRs were closed.  Has not happened, and annoyances continue.

Since there is no search option I can find to search for PRs with a merge conflict, I assume that they are not discovered until a PR is accessed.  So when some of these are done, I will search with 'idle in:title', open, scroll down, and fix.

1. 9 leftover 'object's from 2.x "class C(object):" headers.

2. 6 unneeded 'list's added by 2to3 to make 'for item in list(iterable):'.

3. import 3.x-tkinter name as 2.x-tkinter; needs some care but fixes will be easy to verify.

4. builtin names like 'object', 'dict', and 'type' used as parameter names; I am open to opinion of adding '_' versus abbreviating versus alternative.  

5. ??

----------
assignee: terry.reedy
components: IDLE
messages: 385564
nosy: taleinat, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE:  update code, mostly by cleanups of 2.x or 2to3 artifacts
type: enhancement
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43013>
_______________________________________

From report at bugs.python.org  Sun Jan 24 03:34:14 2021
From: report at bugs.python.org (Anthony Sottile)
Date: Sun, 24 Jan 2021 08:34:14 +0000
Subject: [New-bugs-announce] [issue43014] tokenize spends a lot of time in
 `re.compile(...)`
Message-ID: <1611477254.42.0.0381684920705.issue43014@roundup.psfhosted.org>


New submission from Anthony Sottile <asottile at umich.edu>:

I did some profiling (attached a few files here with svgs) of running this script:

```python
import io
import tokenize

# picked as the second longest file in cpython
with open('Lib/test/test_socket.py', 'rb') as f:
    bio = io.BytesIO(f.read())


def main():
    for _ in range(10):
        bio.seek(0)
        for _ in tokenize.tokenize(bio.readline):
            pass

if __name__ == '__main__':
    exit(main())
```


the first profile is before the optimization, the second is after the optimization

The optimization takes the execution from ~6300ms to ~4500ms on my machine (representing a 28% - 39% improvement depending on how you calculate it)

(I'll attach the pstats and svgs after creation, seems I can only attach one file at once)

----------
components: Library (Lib)
files: out.pstats
messages: 385572
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: tokenize spends a lot of time in `re.compile(...)`
type: performance
versions: Python 3.10, Python 3.9
Added file: https://bugs.python.org/file49759/out.pstats

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43014>
_______________________________________

From report at bugs.python.org  Sun Jan 24 09:20:03 2021
From: report at bugs.python.org (Nathaniel Manista)
Date: Sun, 24 Jan 2021 14:20:03 +0000
Subject: [New-bugs-announce] [issue43015] Add str.replaceall?
Message-ID: <1611498003.08.0.308752741743.issue43015@roundup.psfhosted.org>


New submission from Nathaniel Manista <nathaniel at google.com>:

Chained str.replace calls can sometimes be pretty unattractive; what are the chances that we could have an str.replaceall method? Of type Callable[[Mapping[str, str]], str]?

Check out absl::StrReplaceAll (https://github.com/abseil/abseil-cpp/blob/22771d471930ce88e1e75d0ca9dd8c65a7b0f895/absl/strings/str_replace.h#L73-L98) for a similarly useful function in C++.

----------
components: Unicode
messages: 385581
nosy: Nathaniel Manista, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Add str.replaceall?
type: enhancement
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43015>
_______________________________________

From report at bugs.python.org  Sun Jan 24 11:02:25 2021
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 24 Jan 2021 16:02:25 +0000
Subject: [New-bugs-announce] [issue43016] Improve tests for curses
Message-ID: <1611504145.81.0.962541791137.issue43016@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Currently tests for curses mainly call different functions and methods in random order. Not all argument combinations are covered by tests and results are not checked. I also ran tests with 1750 different values of TERM supported by ncurses, and some tests were failed because not all features are supported on all terminals.

The proposed PR rewrites tests for curses.

* Added tests for every function or group of related functions.

* Added separate tests for optional functions, so that skipped tests are shown it output.

* Optionally skipped tests for functions like beep() and flush() which do not supported an all terminals (surprisingly!).

* Every function which takes variable number of arguments is now tested with all combination of arguments.

* Functions which support different types of argument (str/bytes/int) are now tested with all supported types.

* Function which change the global state are now called in order which restores presumably default value.

* Checked values and types of results of functions.

* Some tests serve role of demos. They call some function in some order and check how it affects results of other functions.

* Tests now work on terminals with small size (15 lines or less). Some terminals have very small default size.

It could still be improved (for example not all functions are tested with non-ASCII strings), but I spent several weekends (including a New Year and both Christmases) on this work and want to take a break.

----------
components: Tests
messages: 385583
nosy: serhiy.storchaka, twouters
priority: normal
severity: normal
status: open
title: Improve tests for curses
type: enhancement
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43016>
_______________________________________

From report at bugs.python.org  Sun Jan 24 13:00:46 2021
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Sun, 24 Jan 2021 18:00:46 +0000
Subject: [New-bugs-announce] [issue43017] Improve error message in the
 parser when using un-parenthesised tuples in comprehensions
Message-ID: <1611511246.56.0.166169475648.issue43017@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

I have seen this confusion many users over the years. Instead of showing:

? python -c '[x,z for x,y in something]'
  File "<string>", line 1
    [x,z for x,y in something]
         ^
SyntaxError: invalid syntax

We can show:

? ./python -c '[x,z for x,y in something]'
  File "<string>", line 1
    [x,z for x,y in something]
     ^
SyntaxError: did you forgot parentheses around the comprehension target?

----------
messages: 385586
nosy: pablogsal
priority: normal
severity: normal
status: open
title: Improve error message in the parser when using un-parenthesised tuples in comprehensions

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43017>
_______________________________________

From report at bugs.python.org  Sun Jan 24 15:20:49 2021
From: report at bugs.python.org (Akuli)
Date: Sun, 24 Jan 2021 20:20:49 +0000
Subject: [New-bugs-announce] [issue43018] unwanted label showing up in
 ttk.LabeledScale
Message-ID: <1611519649.12.0.740627257832.issue43018@roundup.psfhosted.org>


New submission from Akuli <akuviljanen17 at gmail.com>:

import tkinter
from tkinter import ttk
root = tkinter.Tk()
ttk.LabeledScale(root).pack()
root.mainloop()


Run this code and move the slider to center. You see something in front of the number. The problem is this line in ttk.py:

        tmp = Label(self).pack(side=label_side) # place holder

I tried deleting it, but then the label doesn't show up at all, and what remains is just a scale. Packing with a different side works, but that feels like a hack.

----------
components: Tkinter
files: ssss.png
messages: 385591
nosy: Akuli
priority: normal
severity: normal
status: open
title: unwanted label showing up in ttk.LabeledScale
versions: Python 3.9
Added file: https://bugs.python.org/file49765/ssss.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43018>
_______________________________________

From report at bugs.python.org  Mon Jan 25 00:32:51 2021
From: report at bugs.python.org (Felipe Faria)
Date: Mon, 25 Jan 2021 05:32:51 +0000
Subject: [New-bugs-announce] [issue43019] wait_for(to_thread)) does not work
 as expected. Extra documentation or fix needed.
Message-ID: <1611552771.11.0.601335924509.issue43019@roundup.psfhosted.org>


New submission from Felipe Faria <felipefaria at me.com>:

Consider the following:

```
import asyncio
import time

async def async_test():
    while True:
        await asyncio.sleep(1)
        print("in here - async")

def sync_test():
    while True:
        time.sleep(1)
        print("in here - sync")

async def main():
    async def tryexcept(func):
        try:
            await func
        except asyncio.exceptions.TimeoutError:
            print("error thrown")

    await tryexcept(asyncio.wait_for(async_test(), timeout=5))
    await tryexcept(asyncio.wait_for(asyncio.to_thread(sync_test), timeout=5))
    print("back in the main thread")

asyncio.run(main())
```

The above will lead to:

```
in here - async
error thrown
in here - sync
error thrown
back in the main thread
in here - sync
in here - sync
in here - sync
[... continues on forever ...]
```

It seems that the new thread created by `to_thread` is never cancelled and continues running forever despite the timeout error thrown. This might lead to some unwarranted (and hidden) behavior depending on the application.

Frankly, I'm unsure if a possible bug fix is even viable as from my knowledge cancelling threads in Python is not recommended. However, if not, I think a documentation update would be helpful. 

On my end messing with an existing sync library + `to_thread` + `wait_for` led me to believe that the execution had been cancelled, when instead it kept on shooting unexpected web requests. 

Thank you.

----------
components: asyncio
messages: 385602
nosy: asvetlov, felipefaria, yselivanov
priority: normal
severity: normal
status: open
title: wait_for(to_thread)) does not work as expected. Extra documentation or fix needed.
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43019>
_______________________________________

From report at bugs.python.org  Mon Jan 25 01:59:06 2021
From: report at bugs.python.org (Emir)
Date: Mon, 25 Jan 2021 06:59:06 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue43020=5D_str=2Elower_metho?=
 =?utf-8?b?ZCB3aXRoICLEsCIgY2hhcmFjdGVy?=
Message-ID: <1611557946.62.0.291810743958.issue43020@roundup.psfhosted.org>


New submission from Emir <emiryegnidemir7 at gmail.com>:

In Turkish there is a character when you capitalize "i" and it's written as "?". When I use str.lower method, it seems like it works just fine because it prints the character I expected(i). But the problem is when I compare it to normal "i" character (without lowering), the character which is "?" then became "i" has 2 digits(or length) so same words are no more the same. (ex. the word "issue" and the word "?ssue"-->"issue" are not the same when compared) This is a big issue in terms of word counter softwares.

----------
components: Build
messages: 385603
nosy: emiryegnidemir7
priority: normal
severity: normal
status: open
title: str.lower method with "?" character
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43020>
_______________________________________

From report at bugs.python.org  Mon Jan 25 03:42:55 2021
From: report at bugs.python.org (Wietse Jacobs)
Date: Mon, 25 Jan 2021 08:42:55 +0000
Subject: [New-bugs-announce] [issue43021] Unpacking tuple argument in
 combination with inline if statement
Message-ID: <1611564175.96.0.494312223598.issue43021@roundup.psfhosted.org>


New submission from Wietse Jacobs <wietse.j at gmail.com>:

I suspect that I found a bug. If I run the following script:

```
def f(t=None):
    t0, t1 = t if t is not None else [], []
    return t0, t1


def g(t=None):
    if t is None:
        t = [], []
    t0, t1 = t
    return t0, t1


def test():
    res_f = f(t=([1, 1], [2, 2]))
    res_g = g(t=([1, 1], [2, 2]))
    assert res_f == res_g, f'{res_f=} != {res_g=}'


test()
```

I get an assertion error, with:

```
res_f=(([1, 1], [2, 2]), []) != res_g=([1, 1], [2, 2])
```

I expected them to be equal.

----------
messages: 385607
nosy: wietse.j
priority: normal
severity: normal
status: open
title: Unpacking tuple argument in combination with inline if statement
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43021>
_______________________________________

From report at bugs.python.org  Mon Jan 25 07:14:42 2021
From: report at bugs.python.org (Paul Moore)
Date: Mon, 25 Jan 2021 12:14:42 +0000
Subject: [New-bugs-announce] [issue43022] Unable to dynamically load
 functions from python3.dll
Message-ID: <1611576882.72.0.213255422964.issue43022@roundup.psfhosted.org>


New submission from Paul Moore <p.f.moore at gmail.com>:

I am writing a small application using the embedded distribution to run a script supplied by the user. The requirements are very simple, so all I need to do is set up argv and call Py_Main.

I'm trying to load the Py_Main function dynamically (for flexibility - see below) but GetProcAddress returns 0 when loading the function from python3.dll (the stable ABI). This seems to be because the symbols in python3.dll are special "fowarding" symbols, that GetProcAddress can't handle.

Is there a way to dynamically load the Python API from the stable ABI? If there isn't currently, could one be added?

To explain my requirements in a bit more detail, I don't want to statically link, because I want to put the Python distribution in a subdirectory, so that it isn't visible on PATH along with my executable, but I'd rather avoid the complexities involved in adding a dedicated SxS manifest to point the loader to the correct subdirectory for the python3.dll.

Furthermore, I want to provide a graceful fallback if the Python distribution is missing (initially, just a friendly error message, but in future maybe locating an alternative Python installation to use) and static linking won't allow that as I can't recover control if the expected Python DLL isn't present.

The reason I want to use the stable ABI is so that I can upgrade the embedded distribution without needing to rebuild the C code. I could search for python3X.dll, and just ignore the stable ABI. But that's more filesystem code than I really want to write in C... And honestly, I feel that dynamically linking is a perfect use case for the stable ABI, so it really should be supported.

----------
components: Windows
messages: 385621
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Unable to dynamically load functions from python3.dll
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43022>
_______________________________________

From report at bugs.python.org  Mon Jan 25 08:14:36 2021
From: report at bugs.python.org (Ma Lin)
Date: Mon, 25 Jan 2021 13:14:36 +0000
Subject: [New-bugs-announce] [issue43023] Remove a redundant check in
 _PyBytes_Resize()
Message-ID: <1611580476.8.0.756071230609.issue43023@roundup.psfhosted.org>


New submission from Ma Lin <malincns at 163.com>:

Above code already cover this check:

    if (Py_SIZE(v) == newsize) {
        /* return early if newsize equals to v->ob_size */
        return 0;
    }
    if (Py_SIZE(v) == 0) {
-       if (newsize == 0) {
-           return 0;
-       }
        *pv = _PyBytes_FromSize(newsize, 0);
        Py_DECREF(v);
        return (*pv == NULL) ? -1 : 0;
    }

----------
messages: 385626
nosy: malin
priority: normal
severity: normal
status: open
title: Remove a redundant check in _PyBytes_Resize()
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43023>
_______________________________________

From report at bugs.python.org  Mon Jan 25 10:03:19 2021
From: report at bugs.python.org (Irit Katriel)
Date: Mon, 25 Jan 2021 15:03:19 +0000
Subject: [New-bugs-announce] [issue43024] improve signature (in help,
 etc) for functions taking sentinel defaults
Message-ID: <1611586999.49.0.965290563696.issue43024@roundup.psfhosted.org>


New submission from Irit Katriel <iritkatriel at yahoo.com>:

The "sentinel" default value don't render nicely in the signature:

>>> help(traceback.print_exception)
Help on function print_exception in module traceback:

print_exception(exc, /, value=<traceback._Sentinel object at 0x000002402B6EA6A0>, tb=<traceback._Sentinel object at 0x000002402B6EA6A0>, limit=None, file=None, chain=True)

----------
components: Library (Lib)
messages: 385634
nosy: iritkatriel
priority: normal
severity: normal
status: open
title: improve signature (in help, etc) for functions taking sentinel defaults
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43024>
_______________________________________

From report at bugs.python.org  Mon Jan 25 13:45:50 2021
From: report at bugs.python.org (Bhuvanesh Bhatt)
Date: Mon, 25 Jan 2021 18:45:50 +0000
Subject: [New-bugs-announce] [issue43025] Use normal 'i
Message-ID: <1611600350.32.0.540433161176.issue43025@roundup.psfhosted.org>


New submission from Bhuvanesh Bhatt <bhuvaneshbhatt at gmail.com>:

In Python, the letter 'j' denotes the imaginary unit. It would be great if we would follow mathematics in this regard and let the imaginary unit be denoted with an 'i'.

----------
messages: 385641
nosy: bhuvaneshbhatt
priority: normal
severity: normal
status: open
title: Use normal 'i

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43025>
_______________________________________

From report at bugs.python.org  Mon Jan 25 18:46:46 2021
From: report at bugs.python.org (Ezra)
Date: Mon, 25 Jan 2021 23:46:46 +0000
Subject: [New-bugs-announce] [issue43026] Missing words renders meaning
 unclear in fcntl.html
Message-ID: <1611618406.29.0.452944472841.issue43026@roundup.psfhosted.org>


New submission from Ezra <ezrabc at gmail.com>:

At https://docs.python.org/3/library/fcntl.html the docs read:

the fcntl module exposes the F_OFD_GETLK, F_OFD_SETLK and F_OFD_SETLKW constants, which working with open file description locks.

The exact intended meaning is unclear, perhaps:

the fcntl module exposes the F_OFD_GETLK, F_OFD_SETLK and F_OFD_SETLKW constants, which are used for working with open file description locks.

----------
assignee: docs at python
components: Documentation
files: meaning_unclear.png
messages: 385680
nosy: EzraBC, docs at python
priority: normal
severity: normal
status: open
title: Missing words renders meaning unclear in fcntl.html
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49766/meaning_unclear.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43026>
_______________________________________

From report at bugs.python.org  Mon Jan 25 22:52:05 2021
From: report at bugs.python.org (Ma Lin)
Date: Tue, 26 Jan 2021 03:52:05 +0000
Subject: [New-bugs-announce] [issue43027] Calling _PyBytes_Resize() on
 1-byte bytes may raise error
Message-ID: <1611633125.01.0.24786462439.issue43027@roundup.psfhosted.org>


New submission from Ma Lin <malincns at 163.com>:

PyBytes_FromStringAndSize() uses a global cache for 1-byte bytes:
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L147

    if (size == 1 && str != NULL) {
        struct _Py_bytes_state *state = get_bytes_state();
        op = state->characters[*str & UCHAR_MAX];
        if (op != NULL) {
            Py_INCREF(op);
            return (PyObject *)op;
        }
    }

_PyBytes_Resize() will raise an error when (refcount != 1):
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L3029

Then this code will raise an exception:

    obj1 = PyBytes_FromStringAndSize("a", 1);
    obj2 = PyBytes_FromStringAndSize("a", 1);
    ret = _PyBytes_Resize(&obj2, 2);  // ret is -1

BTW, 0-byte bytes comes from a global singleton, but _PyBytes_Resize() processes it before checking refcount:
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L3021

    if (Py_SIZE(v) == 0) {
        if (newsize == 0) {
            return 0;
        }
        *pv = _PyBytes_FromSize(newsize, 0);
        Py_DECREF(v);
        return (*pv == NULL) ? -1 : 0;
    }
    if (Py_REFCNT(v) != 1) {
        goto error;
    }

_PyBytes_Resize() doc:
    
    Only use this to build up a brand new bytes object; don?t use this if the
    bytes may already be known in other parts of the code. It is an error to
    call this function if the refcount on the input bytes object is not one.

----------
components: Interpreter Core
messages: 385689
nosy: malin
priority: normal
severity: normal
status: open
title: Calling _PyBytes_Resize() on 1-byte bytes may raise error
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43027>
_______________________________________

From report at bugs.python.org  Tue Jan 26 00:29:12 2021
From: report at bugs.python.org (Cotton Seed)
Date: Tue, 26 Jan 2021 05:29:12 +0000
Subject: [New-bugs-announce] [issue43028] seeking past the end of a file
 unexpected behavior
Message-ID: <1611638952.77.0.560549779185.issue43028@roundup.psfhosted.org>


New submission from Cotton Seed <cotton.seed at gmail.com>:

Seeking past the end of a file with file objects does not match the same code implemented in terms of file descriptors.  Is this the intended behavior?

Smallest example I could find:

f = open('new_file', 'ab')
print(f.seek(1))
print(f.write(b'foo'))
print(f.tell())
f.close()

This program outputs: 1, 3, 4 as expected, but only creates a 3-byte file:

and creates a 3-byte file:
$ hexdump -C new_file
00000000  66 6f 6f                                          |foo|
00000003

If I use open(..., buffering=0), or flush before the tell, it outputs: 1, 3, 3.

The obvious code with file descriptors:

fd = os.open('log', os.O_WRONLY | os.O_CREAT)
print(os.lseek(fd, 1, os.SEEK_SET))
os.write(fd, b'foo')
os.close(fd)

works as expected, creating a 4-byte file.

Could this be related this issue:

https://bugs.python.org/issue36411

?

----------
components: IO
messages: 385692
nosy: cotton.seed
priority: normal
severity: normal
status: open
title: seeking past the end of a file unexpected behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43028>
_______________________________________

From report at bugs.python.org  Tue Jan 26 04:30:56 2021
From: report at bugs.python.org (Denis Roussel)
Date: Tue, 26 Jan 2021 09:30:56 +0000
Subject: [New-bugs-announce] [issue43029] unittest: Add assertUniqeIn
Message-ID: <1611653456.39.0.201138075595.issue43029@roundup.psfhosted.org>


New submission from Denis Roussel <roussel2nis at gmail.com>:

Allows to test uniqueness of members of a list

----------
components: Tests
messages: 385700
nosy: rousseldenis
priority: normal
severity: normal
status: open
title: unittest: Add assertUniqeIn

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43029>
_______________________________________

From report at bugs.python.org  Tue Jan 26 07:40:22 2021
From: report at bugs.python.org (Matthias Klose)
Date: Tue, 26 Jan 2021 12:40:22 +0000
Subject: [New-bugs-announce] [issue43030] signed/unsigned mismatch in
 Py_UNICODE_ISSPACE macro
Message-ID: <1611664822.48.0.67078426677.issue43030@roundup.psfhosted.org>


New submission from Matthias Klose <doko at debian.org>:

[forwarded from https://bugs.debian.org/961396]

$ cat > foo.c
#include <Python.h>

int main(int argc, char *argv[])
{
        Py_UNICODE x = 0;

        return Py_UNICODE_ISSPACE(x);
}

$ gcc -Wsign-compare -Werror $(pkg-config --cflags python3) foo.c
In file included from /usr/include/python3.9/unicodeobject.h:1026,
                 from /usr/include/python3.9/Python.h:97,
                 from foo.c:1:
foo.c: In function ?main?:
/usr/include/python3.9/cpython/unicodeobject.h:25:11: error: comparison of integer expressions of different signedness: ?Py_UNICODE? {aka ?int?} and ?unsigned int? [-Werror=sign-compare]
   25 |     ((ch) < 128U ? _Py_ascii_whitespace[(ch)] : _PyUnicode_IsWhitespace(ch))
      |           ^
foo.c:7:16: note: in expansion of macro ?Py_UNICODE_ISSPACE?
    7 |         return Py_UNICODE_ISSPACE(x);
      |                ^~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

----------
components: Extension Modules
messages: 385707
nosy: doko
priority: normal
severity: normal
status: open
title: signed/unsigned mismatch in Py_UNICODE_ISSPACE macro
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43030>
_______________________________________

From report at bugs.python.org  Tue Jan 26 08:46:19 2021
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 26 Jan 2021 13:46:19 +0000
Subject: [New-bugs-announce] [issue43031] aarch64 RHEL7 LTO + PGO 3.7:
 "make" hangs when running test_asyncio
Message-ID: <1611668779.87.0.585376079425.issue43031@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

test_asyncio hangs randomly on Python 3.7 on aarch64 RHEL7 LTO + PGO 3.7. The symptom is a failed build failing with:

   retry lost connection compile (retry)

Full error:

   remoteFailed: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.

Example of failed build:

   https://buildbot.python.org/all/#/builders/42/builds/4703

Configure command:

   ./configure --prefix '$(PWD)/target' --with-lto --enable-optimizations

Compile command:

   make -j10 all

End of the make output:
--------
make run_profile_task
make[1]: Entering directory `/home/buildbot/buildarea/3.7.cstratak-RHEL7-aarch64.lto-pgo/build'
./python -m test.regrtest --pgo || true
0:00:00 load avg: 1.07 Run tests sequentially
0:00:00 load avg: 1.07 [  1/416] test_grammar
(...)
0:01:40 load avg: 1.07 [ 24/416] test_asynchat
0:01:42 load avg: 1.07 [ 25/416] test_asyncio

remoteFailed: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.
]
--------

I ran manually "./configure --prefix '$(PWD)/target' --with-lto --enable-optimizations && make -j10 all" twice on the worker directly, but test_asyncio passed successfully (I tried it twice).

----------
components: Tests
messages: 385712
nosy: vstinner
priority: normal
severity: normal
status: open
title: aarch64 RHEL7 LTO + PGO 3.7: "make" hangs when running test_asyncio
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43031>
_______________________________________

From report at bugs.python.org  Tue Jan 26 08:46:44 2021
From: report at bugs.python.org (Robert)
Date: Tue, 26 Jan 2021 13:46:44 +0000
Subject: [New-bugs-announce] [issue43032] Size of pie chart in matplotlib
 (frame affects it)
Message-ID: <1611668804.53.0.845816677703.issue43032@roundup.psfhosted.org>


New submission from Robert <r.rauterkus at posteo.net>:

Hi all.

Is it an issue or on purpose that enabling and disabling the frame in plt.pie() results in different sized pie charts? In my opinion the code below should provide identical sized charts. If it is on purpose, can you give me a reference? I am using the conda version of pyhton but colleagues experience the same behavior.

import matplotlib.pyplot as plt
f = plt.figure(figsize=(10,5),dpi=300)
ax1 = f.add_axes([0,0,0.5,1])
ax2 = f.add_axes([0.5,0,0.5,1])
ax1.pie([180,180],center=(0.5,0.5),radius=0.5,frame=False)
ax2.pie([180,180],center=(0.5,0.5),radius=0.5,frame=True)
plt.show()

----------
messages: 385713
nosy: twisterior
priority: normal
severity: normal
status: open
title: Size of pie chart in matplotlib (frame affects it)
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43032>
_______________________________________

From report at bugs.python.org  Tue Jan 26 18:39:25 2021
From: report at bugs.python.org (Zackery Spytz)
Date: Tue, 26 Jan 2021 23:39:25 +0000
Subject: [New-bugs-announce] [issue43033] Incorrect handling of
 PyObject_SetAttrString() in the _zoneinfo module
Message-ID: <1611704365.65.0.160193977721.issue43033@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

There is a PyObject_SetAttrString() call in zoneinfo_init_subclass() which
is not checked for failure.  This could lead to serious bugs if an
exception occurs in PyObject_SetAttrString().

----------
components: Extension Modules
messages: 385739
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: Incorrect handling of PyObject_SetAttrString() in the _zoneinfo module
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43033>
_______________________________________

From report at bugs.python.org  Tue Jan 26 20:15:02 2021
From: report at bugs.python.org (Jesse Silverman)
Date: Wed, 27 Jan 2021 01:15:02 +0000
Subject: [New-bugs-announce] [issue43034] Python tutorial misleads users
 about floor division behavior
Message-ID: <1611710102.87.0.478016588876.issue43034@roundup.psfhosted.org>


New submission from Jesse Silverman <jessevsilverman at gmail.com>:

I had never worked thru the Python tutorial, it's kind of awesome.

I noticed multiple independent presenters incorrectly describe the behavior of floor division when the signs of the operands don't match.  Not just sloppy ones, some who are usually pretty careful -- people forget.  I found it odd.

Today I read:
https://docs.python.org/3.9/tutorial/introduction.html#using-python-as-a-calculator
'Division (/) always returns a float. To do floor division and get an integer result (discarding any fractional result) you can use the // operator; to calculate the remainder you can use %:'

I know what they mean (there will never be a non-zero fractional component in the result) however, stating that it "discards any fraction result" explicitly suggests that it will round towards zero for all results.  The name of the operator tells us the true story, as does a test, it rounds towards negative infinity for all results.

Hopefully all the people using Python to treat cancer, fly to Mars and run power plants know what the behavior is.  However, anecdotally, I have seen evidence that generally cautious, fairly-detail oriented programmers seem to forget the details of the behavior, this is one place in the docs from which many people first learn about the floor division operator and this page would lead me to believe it operates differently than it does in reality.

If the words right there don't get changed, or the example of -10/3 doesn't get added, would this be a good excuse for a third footnote?  I would actually wish for both the wording to be changed and to show an example like -10 / 3, tho admittedly I came from an Engineering background so I am always worried someone's going to blow up somewhere...

Deepest regards for everyone who gets to regularly close the complaints about floor division working as designed and documented -- but not quite documented on this page which more people may read than others covering the operator.

----------
assignee: docs at python
components: Documentation
messages: 385746
nosy: docs at python, jessevsilverman
priority: normal
severity: normal
status: open
title: Python tutorial misleads users about floor division behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43034>
_______________________________________

From report at bugs.python.org  Wed Jan 27 06:20:36 2021
From: report at bugs.python.org (Karthik Nishanth)
Date: Wed, 27 Jan 2021 11:20:36 +0000
Subject: [New-bugs-announce] [issue43035] FileNotFoundError in
 distutils\file_util.py copy_tree
Message-ID: <1611746436.35.0.546212642615.issue43035@roundup.psfhosted.org>


New submission from Karthik Nishanth <karthik.nishanth at zivid.com>:

I am making a bdist_wheel for a private python binary module.

`python setup.py bdist_wheel` fails on a very specific windows VM repeatedly. This works fine on both a regular windows machine & linux machines.

Attached the complete distutils verbose logs.

Stacktrace for quick search:
---------------------------------------------

     File "C:\Program Files\Python39\lib\site-packages\setuptools\__init__.py", line 165, in setup
       return distutils.core.setup(**attrs)
     File "C:\Program Files\Python39\lib\distutils\core.py", line 148, in setup
       dist.run_commands()
     File "C:\Program Files\Python39\lib\distutils\dist.py", line 966, in run_commands
       self.run_command(cmd)
     File "C:\Program Files\Python39\lib\distutils\dist.py", line 985, in run_command
       cmd_obj.run()
     File "C:\Program Files\Python39\lib\site-packages\wheel\bdist_wheel.py", line 335, in run
       self.run_command('install')
     File "C:\Program Files\Python39\lib\distutils\cmd.py", line 313, in run_command
       self.distribution.run_command(command)
     File "C:\Program Files\Python39\lib\distutils\dist.py", line 985, in run_command
       cmd_obj.run()
     File "C:\Program Files\Python39\lib\site-packages\setuptools\command\install.py", line 61, in run
       return orig.install.run(self)
     File "C:\Program Files\Python39\lib\distutils\command\install.py", line 558, in run
       self.run_command(cmd_name)
     File "C:\Program Files\Python39\lib\distutils\cmd.py", line 313, in run_command
       self.distribution.run_command(command)
     File "C:\Program Files\Python39\lib\distutils\dist.py", line 985, in run_command
       cmd_obj.run()
     File "C:\Program Files\Python39\lib\site-packages\setuptools\command\install_lib.py", line 12, in run
       outfiles = self.install()
     File "C:\Program Files\Python39\lib\distutils\command\install_lib.py", line 111, in install
       outfiles = self.copy_tree(self.build_dir, self.install_dir)
     File "C:\Program Files\Python39\lib\site-packages\setuptools\command\install_lib.py", line 95, in copy_tree
       return orig.install_lib.copy_tree(self, infile, outfile)
     File "C:\Program Files\Python39\lib\distutils\cmd.py", line 354, in copy_tree
       return dir_util.copy_tree(infile, outfile, preserve_mode,
     File "C:\Program Files\Python39\lib\distutils\dir_util.py", line 157, in copy_tree
       copy_tree(src_name, dst_name, preserve_mode,
     File "C:\Program Files\Python39\lib\distutils\dir_util.py", line 161, in copy_tree
       copy_file(src_name, dst_name, preserve_mode,
     File "C:\Program Files\Python39\lib\distutils\file_util.py", line 151, in copy_file
       _copy_file_contents(src, dst)
     File "C:\Program Files\Python39\lib\distutils\file_util.py", line 43, in _copy_file_contents
       raise DistutilsFileError(
   distutils.errors.DistutilsFileError: could not create 'build\bdist.win-amd64\wheel\internal_pkgname-2.3.0_pre_alpha_1_c8273a4c_1_preview.data\purelib\mydeviceapi\mydeviceapi.cp39-win_amd64.pyd': No such file or directory

---------------------------------------------

I may get an [Errno 2] on open(dst, 'wb') if dst has a directory component that does not exist. But distutils logs already say 

> creating dir1
> creating dir1/dir2
> creating dir1/dir2/dir3

so I assume the directories must have been created before calling open().

----------
components: Distutils
files: scratch_4.txt
messages: 385758
nosy: dstufft, eric.araujo, knzivid
priority: normal
severity: normal
status: open
title: FileNotFoundError in distutils\file_util.py copy_tree
type: crash
versions: Python 3.9
Added file: https://bugs.python.org/file49767/scratch_4.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43035>
_______________________________________

From report at bugs.python.org  Wed Jan 27 06:39:48 2021
From: report at bugs.python.org (Xavier Morel)
Date: Wed, 27 Jan 2021 11:39:48 +0000
Subject: [New-bugs-announce] [issue43036] TOS-behaviour documentation is
Message-ID: <1611747588.28.0.967739305084.issue43036@roundup.psfhosted.org>


New submission from Xavier Morel <xavier.morel at masklinn.net>:

I was looking at the disassembly of a fairly straightforward listcomp:

    [e for e in s if e[0]]

  1           0 BUILD_LIST               0
              2 LOAD_FAST                0 (.0)
        >>    4 FOR_ITER                16 (to 22)
              6 STORE_FAST               1 (e)
              8 LOAD_FAST                1 (e)
             10 LOAD_CONST               0 (0)
             12 BINARY_SUBSCR
             14 POP_JUMP_IF_FALSE        4
             16 LOAD_FAST                1 (e)
             18 LIST_APPEND              2
             20 JUMP_ABSOLUTE            4
        >>   22 RETURN_VALUE

6, 8 bothered me because STORE_FAST is documented as

> Stores TOS into the local co_varnames[var_num].

So it seems like it leaves TOS and thus the LOAD is unnecessary, However looking at ceval.c:

        case TARGET(STORE_FAST): {
            PREDICTED(STORE_FAST);
            PyObject *value = POP();
            SETLOCAL(oparg, value);
            FAST_DISPATCH();
        }

so STORE_FAST does pop the TOS and the LOAD_FAST is necessary. This is confusing because there are instructions which literally have POP in their name whose stack behaviour is documented explicitly.

Should all bytecode instructions have their stack behaviour explicitly documented, or only those with an *odd* stack behaviour (e.g. JUMP_IF_FALSE_OR_POP) and the rest maybe covered by a note saying that they will pop their parameters and push back their result or somesuch?

--

Furthermore, maybe optimising `STORE_LOCAL a; LOAD_LOCAL a` to `DUP_TOP; STORE_LOCAL a` would be useful? It obviously would have no effect on bytecode size since wordcode, and `fastlocals[i]` would be in cache and the conditional check likely predicted, but it seems like skipping them entirely would still be more reliable? This idea is somewhat supported by expression assignments already generating the latter:

    >>> @dis.dis
    ... def foo():
    ...     if a := thing():
    ...         do(a)
    ... 
  3           0 LOAD_GLOBAL              0 (thing)
              2 CALL_FUNCTION            0
              4 DUP_TOP
              6 STORE_FAST               0 (a)
              8 POP_JUMP_IF_FALSE       18

  4          10 LOAD_GLOBAL              1 (do)
             12 LOAD_FAST                0 (a)
             14 CALL_FUNCTION            1
             16 POP_TOP
        >>   18 LOAD_CONST               0 (None)
             20 RETURN_VALUE


This optimisation would also trigger for e.g.

    [x[i] for x in xs]

or

    a = foo()
    if a:
        # do thing

making the latter generate bytecode identical to walrus assignments at least for the trivial case: currently the only difference (aside from line numbers) is that the normal assignment generates STORE_FAST;LOAD_FAST while expression assignments generate DUP_TOP;STORE_FAST.

----------
assignee: docs at python
components: Documentation
messages: 385762
nosy: docs at python, xmorel
priority: normal
severity: normal
status: open
title: TOS-behaviour documentation is
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43036>
_______________________________________

From report at bugs.python.org  Wed Jan 27 07:14:36 2021
From: report at bugs.python.org (Stefan Mosoi)
Date: Wed, 27 Jan 2021 12:14:36 +0000
Subject: [New-bugs-announce] [issue43037] Importlib reload by module name
 (String)
Message-ID: <1611749676.48.0.692377110291.issue43037@roundup.psfhosted.org>


New submission from Stefan Mosoi <st3f4n2006 at gmail.com>:

Weird behaviour (maybe it's my opinion) in reload from importlib

if i do:
import importlib
import sys
import datetime
importlib.reload(datetime.timedelta.__module__)

I get
Typeerror: reload() argument must be a module

but if i do

import importlib
import sys
import datetime
importlib.reload(sys.modules.get(datetime.timedelta.__module__))

it works.

The sys.modules.get i got from reload() code:
def reload(module)
    if not module or not isinstance(module, types.ModuleType):
        raise TypeError("reload() argument must be a module")
    try:
        name = module.__spec__.name
    except AttributeError:
        name = module.__name__

    if sys.modules.get(name) is not module:

It wouldn't be easier to check if string do sys.modules.get ?

And as a bonus a reload_module function that gets a class (or anything with __module__ and reloads that module, no question asked?) 

And i know it's easy to implement, but it would be nicer if was better handled in the lib :)

Notes (if it matters):
Python 3.9.1 x64 Windows
I didn't test on other versions.

----------
components: Library (Lib)
messages: 385765
nosy: st3f4n2006
priority: normal
severity: normal
status: open
title: Importlib reload by module name (String)
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43037>
_______________________________________

From report at bugs.python.org  Wed Jan 27 08:04:57 2021
From: report at bugs.python.org (Jan Christoph)
Date: Wed, 27 Jan 2021 13:04:57 +0000
Subject: [New-bugs-announce] [issue43038] ensurepip: tries to use
 setup.py/setup.cfg
Message-ID: <1611752697.42.0.4760269276.issue43038@roundup.psfhosted.org>


New submission from Jan Christoph <con-f-use at gmx.net>:

Running

python3 -Im ensurepip --upgrade --default-pip

in a directory that contains a setup.cfg / setup.py combination, caused ensurepip to try and use these files, leading to

distutils.errors.DistutilsOptionError: error in setup.cfg: command 'build' has no such option 'icons'

A full shell session is attached that contains details on the exact files and python version involved involved.

I was under the impression ensurepip simply ensured that a suitable version of pip is installed with my Python distribution in use.

----------
components: Library (Lib)
files: shell-session.txt
messages: 385767
nosy: con-f-use
priority: normal
severity: normal
status: open
title: ensurepip: tries to use setup.py/setup.cfg
type: behavior
versions: Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49768/shell-session.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43038>
_______________________________________

From report at bugs.python.org  Wed Jan 27 08:06:25 2021
From: report at bugs.python.org (Eric Osborne)
Date: Wed, 27 Jan 2021 13:06:25 +0000
Subject: [New-bugs-announce] [issue43039] tempfile.TemporaryDirectory() name
 string is incorrect
Message-ID: <1611752785.31.0.169033052217.issue43039@roundup.psfhosted.org>


New submission from Eric Osborne <eosborne at vapor.io>:

When I create a temporary directory, the name doesn't come back in a way the application expects it.  I ran into this using gitPython and I'm not entirely clear whether it's a tmpfile issue or a gitpython issue, but I think it's tempfile.  

Repro:

In [12]: td = tempfile.TemporaryDirectory()

In [13]: td.name
Out[13]: '/var/folders/kd/bjhw84ss2x734frx98qh1mw40000gn/T/tmpgralfh9m'

In [14]: str(td)
Out[14]: "<TemporaryDirectory '/var/folders/kd/bjhw84ss2x734frx98qh1mw40000gn/T/tmpgralfh9m'>"

The problem with this is that gitPython takes a temporary directory as an argument and effectively does a mkdir(str(td)), which creates a weird directory in /var/tmp:

eosborne at eosborne-salt-master:/tmp$ ls -alhd *Temp*
drwxrwxr-x 3 eosborne eosborne 4.0K Jan 27 12:42 '<TemporaryDirectory '\'''



I believe that str(td) should return just the directory name and that repr(td) can return the extra bit of text with "<TemporaryDirectory ..>" but that's just my two cents.

I have thrown this issue up on the gitPython list too: https://github.com/gitpython-developers/GitPython/issues/1115

----------
components: Library (Lib)
messages: 385768
nosy: eosborne
priority: normal
severity: normal
status: open
title: tempfile.TemporaryDirectory() name string is incorrect
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43039>
_______________________________________

From report at bugs.python.org  Wed Jan 27 08:07:13 2021
From: report at bugs.python.org (Donovan Baarda)
Date: Wed, 27 Jan 2021 13:07:13 +0000
Subject: [New-bugs-announce] [issue43040] random.py randrange() is very slow
 if the range is a power of 2.
Message-ID: <1611752833.52.0.275617605532.issue43040@roundup.psfhosted.org>


New submission from Donovan Baarda <abo at minkirri.apana.org.au>:

I encountered a very significant slowdown migrating some code from python2.7 to python3.9 that I tracked down to randrange() being slow. After digging I noticed that _randbelow_with_getrandbits() calls getrandbits() 2x as many times, and asks for one more bit each time, than is actually required if the range requested is a power of 2.

I have a GitHub PR on the way to fix this...

----------
components: Library (Lib)
messages: 385769
nosy: abo
priority: normal
severity: normal
status: open
title: random.py randrange() is very slow if the range is a power of 2.
type: performance
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43040>
_______________________________________

From report at bugs.python.org  Wed Jan 27 09:42:22 2021
From: report at bugs.python.org (Daniel Romberg)
Date: Wed, 27 Jan 2021 14:42:22 +0000
Subject: [New-bugs-announce] [issue43041] copying WeakValueDictionary is not
 iteration safe
Message-ID: <1611758542.35.0.93123255334.issue43041@roundup.psfhosted.org>


New submission from Daniel Romberg <daniel at romberg.io>:

The copy operation is not safe to use during iteration. The following test case raises a "RuntimeError: dictionary changed size during iteration":


import weakref

class Class:
  pass

def TEST_weakValue():
  d = weakref.WeakValueDictionary()
  a = Class()
  b = Class()
  d["a"] = a
  d["b"] = b
  e = d.copy()
  
  for key in e:
    a = None
    c = e.copy()

TEST_weakValue()


This is related to https://bugs.python.org/issue35615 where I commented as well, but I couldn't find a way to reopen this issue, which is why I open this one.

We experience a lot fewer crashes in weakref than before https://bugs.python.org/issue35615 had been fixed, however, there are recursive situations in which copy() is invoked while iterating the WeakValueDictionary (e.g., in our case it is a signal/slot implementation where the slots are stored in a WeakValueDictionary). _commit_removals(), which is called at the beginning of the copy operation, might change the dictionary if there are items that are to be removed. If there is an ongoing iteration, the corresponding RuntimeError is raised.

I haven't thought that through entirely, but I wonder whether the copy (and also deepcopy) operation could just blindly copy everything without "committing removals". After the copy, both instances would do their _commit_removals on their own upon access.

----------
components: Library (Lib)
messages: 385779
nosy: djromberg
priority: normal
severity: normal
status: open
title: copying WeakValueDictionary is not iteration safe
type: crash
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43041>
_______________________________________

From report at bugs.python.org  Wed Jan 27 13:42:22 2021
From: report at bugs.python.org (Jesse Silverman)
Date: Wed, 27 Jan 2021 18:42:22 +0000
Subject: [New-bugs-announce] [issue43042] tutorial ambiguous about creation
 of local symbol table for recursive calls
Message-ID: <1611772942.43.0.309900316345.issue43042@roundup.psfhosted.org>


New submission from Jesse Silverman <jessevsilverman at gmail.com>:

In section 4.6 of the tutorial, we find:

"When a function calls another function, a new local symbol table is created for that call."

Now, perhaps because we were just looking at a function that people will often ask you to write both recursively and non-recursively and then ask which one you would use and why...I was thinking "Wait -- when a function calls itself recursively, obviously they need a new local symbol table or local variables won't work??"

I may be confused, in which case the doc is fine and I personally need to better understand the moral equivalent of activation record / stack frame in Python.

If I am not, given that recursion isn't much more computer science oriented or obscure than Fibonacci sequences, could it not be worth the investment of a few extra words as:

"When a function calls another function, or calls itself recursively, a new local symbol table is created for that call."

----------
assignee: docs at python
components: Documentation
messages: 385800
nosy: docs at python, jessevsilverman
priority: normal
severity: normal
status: open
title: tutorial ambiguous about creation of local symbol table for recursive calls
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43042>
_______________________________________

From report at bugs.python.org  Wed Jan 27 17:01:00 2021
From: report at bugs.python.org (Jesse Silverman)
Date: Wed, 27 Jan 2021 22:01:00 +0000
Subject: [New-bugs-announce] [issue43043] Python tutorial could make rules
 for default argument definition clearer
Message-ID: <1611784860.27.0.764773083907.issue43043@roundup.psfhosted.org>


New submission from Jesse Silverman <jessevsilverman at gmail.com>:

I love the tutorial, but have been pointing out things I only knew from brutal experience, other tutorials, reasoning from other languages I know better, or looking things up.  I only mention them if they seem in scope and at the same level of the material which is presented.

Regarding 4.7.1. Default Argument Values of the excellent Python tutorial:

I guessed that I would see the behavior shown below from my understanding of default arguments in several other languages.  I did not read anything in this tutorial section that would lead me to conclude this, and other things mentioned in this section are neither much more or less subtle, more or less obvious, or more or less important.

PS F:\PythonCode> py VarArgs.py
  File "F:\PythonCode\VarArgs.py", line 1
    def varArgs(quote, footnote='', times):
                                         ^
SyntaxError: non-default argument follows default argument

Most introductions to default argument definition in other languages go out of their way to highlight that the default parameters can be only the last N of them, and that all required ones must come first.
Python is intended to be accessible to people no more clever than the other languages intend, so tho it seems obvious to me that they must only come after all required arguments, I remember when it was not obvious (and fortunately the tutorials I read in the past for other languages belabored the point).

The explanatory text of:
giving one of the optional arguments: ask_ok('OK to overwrite the file?', 2)

as it merely says "one of", rather than "the first of" led me to wonder if Python was somehow more flexible/better at mind-reading than the languages I already know well that require all default args to be trailing, and to only let you default the last one, two, or ... N.  The results of the experiment shown above demonstrate it is similar to the behavior in other languages in this regard.

I deeply appreciate the information about when the default arguments get evaluated, which is highly relevant and is not something I would have picked up on from other languages I've used.

A simple line stating that "any default arguments must follow any required arguments in the definition" and something like "on calls, optional arguments may only be omitted from the end of the parameter list, they can't simply be skipped" would be awesome.

I am not sure whether or not this is made perfectly clear in the next section, which has a lot of other new and interesting ideas flying around in it as well, besides this one.

----------
assignee: docs at python
components: Documentation
messages: 385813
nosy: docs at python, jessevsilverman
priority: normal
severity: normal
status: open
title: Python tutorial could make rules for default argument definition clearer
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43043>
_______________________________________

From report at bugs.python.org  Wed Jan 27 18:26:36 2021
From: report at bugs.python.org (=?utf-8?b?0JHQvtGA0LjRgSDQktC10YDRhdC+0LLRgdC60LjQuQ==?=)
Date: Wed, 27 Jan 2021 23:26:36 +0000
Subject: [New-bugs-announce] [issue43044] Python 2.7 documentation links to
 404 pages when the library was moved or renamed
Message-ID: <1611789996.46.0.13568536358.issue43044@roundup.psfhosted.org>


New submission from ????? ?????????? <boris.verk at gmail.com>:

1. go to https://docs.python.org/2/library/stringio.html (note the "/2/")
2. click on "You should upgrade and read the Python documentation for the current stable release." at the top

I expect to be sent to the Python 3 documentation for StringIO (or at least the index.html page of the Python 3 documentaion) but because StringIO was moved from its own library into `io`, the Python 2 docs are linking to a 404 error page, because they just rewrite "/2/" to "/3/" in the URL.

See attached PR for a list of all these broken pages and where they could redirect to.

----------
assignee: docs at python
components: Documentation
messages: 385821
nosy: boris, docs at python
priority: normal
pull_requests: 23179
severity: normal
status: open
title: Python 2.7 documentation links to 404 pages when the library was moved or renamed
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43044>
_______________________________________

From report at bugs.python.org  Wed Jan 27 18:43:06 2021
From: report at bugs.python.org (Lennart Borgman)
Date: Wed, 27 Jan 2021 23:43:06 +0000
Subject: [New-bugs-announce] [issue43045] Instructions for installing pip on
 Ubuntu/WSL2 is incomplete
Message-ID: <1611790986.57.0.0815546947787.issue43045@roundup.psfhosted.org>


New submission from Lennart Borgman <lennart.borgman at gmail.com>:

Please see here for details:

https://stackoverflow.com/questions/65928875/no-module-named-pip-python-3-9/65929009#65929009

----------
messages: 385823
nosy: lennart.borgman
priority: normal
severity: normal
status: open
title: Instructions for installing pip on Ubuntu/WSL2 is incomplete
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43045>
_______________________________________

From report at bugs.python.org  Wed Jan 27 18:52:12 2021
From: report at bugs.python.org (Tadek Kijkowski)
Date: Wed, 27 Jan 2021 23:52:12 +0000
Subject: [New-bugs-announce] [issue43046] argparse: capturing actions
Message-ID: <1611791532.72.0.119102600348.issue43046@roundup.psfhosted.org>


New submission from Tadek Kijkowski <tkijkowski at gmail.com>:

This is spinoff from issue42973.

The purpose of this issue is to provide set of additional action classes for argparse, that could capture mutual relationship between command line arguments for more advanced parsers, where order of command line arguments is meaningful. It is hoped that this will work together with enhancement introduced in issue42973, providing also ability to capture relationship between options and positional parameters.

There will be four new action classes: 'extend_capture', 'append_capture', 'store_capture' and 'capture'.

Here are excerpts from documentation being prepared for the new action classes and an example of use:

* 'extend_capture' - Captures preceding argument values listed in the
  'capture' and 'capture_reset' keyword arguments and creates a dictionary object
  from those values, then adds actual command-line value to the dictionary using
  the 'key' keyword argument, finally the created dictionary is appended to the
  'dest' list. After capturing, all arguments listed in 'capture_reset' are reset
  to their default values. If there are more than one command-line values, a new
  dictionary object is created, and appended to the list for each value, with
  'capture_reset' arguments reset to their default values after first value is
  added.
  Example usage:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument("--load-addr", type=lambda s: int(s, 16))
    >>> parser.add_argument("--exec-addr", type=lambda s: int(s, 16))
    >>> parser.add_argument("--replace", action="store_true")
    >>> parser.add_argument("--file", nargs="*", action="extend_capture",
    ...     capture="*", capture_reset=["load_addr", "exec_addr"])
    >>> parser.parse_args("--replace --load-addr 1900 --exec-addr 8023 "
    ...     "--file CALC !BOOT".split())
    Namespace(load_addr=None, exec_addr=None, replace=True, file=[
        {'replace': True, 'exec_addr': 32803, 'load_addr': 6400, 'file': 'CALC'},
        {'replace': True, 'exec_addr': None, 'load_addr': None, 'file': '!BOOT'}])

capture

The list of attributes to capture. This can be either a single attribute name,
a list (or other iterable type) of names or special value '*'. Name of
attribute associated with each argument is determined by the dest keyword
passed to add_argument method when the argument was
created. If capture_ is '*', all attributes are captured, except for this
argument's own value.

This keyword argument is valid only for 'extend_capture',
'append_capture', 'store_capture' and 'capture' actions.


capture_reset

The list of attributes to capture and reset to default value. As with capture,
this can be '*' to capture and reset all attributes except for this
argument's own value.

This keyword argument is valid only for 'extend_capture',
'append_capture', 'store_capture' and 'capture' actions.


key

The key to use for adding this argument's own command-line value to dictionary
of captured values. If this keyword argument is not specified, the dest is
used.

This keyword argument is valid only for 'extend_capture',
'append_capture' and 'store_capture' actions.

----------
components: Library (Lib)
messages: 385824
nosy: monkeyman79
priority: normal
severity: normal
status: open
title: argparse: capturing actions
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43046>
_______________________________________

From report at bugs.python.org  Wed Jan 27 19:19:29 2021
From: report at bugs.python.org (Ian Wienand)
Date: Thu, 28 Jan 2021 00:19:29 +0000
Subject: [New-bugs-announce] [issue43047] logging.config formatters
 documentation is out of sync with code
Message-ID: <1611793169.86.0.0215519819351.issue43047@roundup.psfhosted.org>


New submission from Ian Wienand <ian at wienand.org>:

The dict based configuration does not mention the "class" option, and neither the ini-file or dict sections mention the style tag.

----------
components: Library (Lib)
messages: 385825
nosy: iwienand
priority: normal
severity: normal
status: open
title: logging.config formatters documentation is out of sync with code
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43047>
_______________________________________

From report at bugs.python.org  Wed Jan 27 20:45:37 2021
From: report at bugs.python.org (Vladimir Feinberg)
Date: Thu, 28 Jan 2021 01:45:37 +0000
Subject: [New-bugs-announce] [issue43048] Printing RecursionError results in
 RecursionError
Message-ID: <1611798337.38.0.626995188313.issue43048@roundup.psfhosted.org>


New submission from Vladimir Feinberg <vlad at sisudata.com>:

Python's standard logger provides an exception() method, which is to be called from except blocks to log out exception traces by pulling from sys.exc_info().

Per https://github.com/python/cpython/blob/3.9/Lib/logging/__init__.py#L617 , logger.exception('my message') eventually boils down to something like traceback.print_exception(ei[0], ei[1], ei[2], None, some_buffer) where ei is the sys.exc_info().

However, the traceback code generates that printout by constructing a TracebackException recursively for every `except` context.

In turn, if a RecursionError was generated from an exception thrown across many except blocks, the TracebackException construction itself will have a RecursionError. This is particularly bad in cases where you'd want to capture this failure information, such as when you're printing the exception, since you'll never find out about the originating error.

Certain (well-used) libraries rely on try/except for control flow, and occasionally do recurse in their except blocks, so such exceptions are not hypothetical.

A solution to this might be to avoid constructing a TracebackException, and instead unwind traceback context using a while loop.

----------
components: Library (Lib)
files: exc.py
messages: 385828
nosy: vlad2
priority: normal
severity: normal
status: open
title: Printing RecursionError results in RecursionError
versions: Python 3.6, Python 3.7, Python 3.9
Added file: https://bugs.python.org/file49769/exc.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43048>
_______________________________________

From report at bugs.python.org  Wed Jan 27 21:10:04 2021
From: report at bugs.python.org (Peter Donis)
Date: Thu, 28 Jan 2021 02:10:04 +0000
Subject: [New-bugs-announce] [issue43049] Use io.IncrementalNewlineDecoder
 for doctest newline conversion
Message-ID: <1611799804.84.0.215071854075.issue43049@roundup.psfhosted.org>


New submission from Peter Donis <peterdonis at alum.mit.edu>:

This is a follow-up to issue 1812:

https://bugs.python.org/issue1812

It was suggested in the discussion on that issue that the newline conversion in doctest that was corrected in that issue could be done using already defined resources in the io module instead of by a custom function. This issue is to provide an issue number for my pull request on github to do that: the io module provides the IncrementalNewlineDecoder object that does exactly what is needed.

----------
components: Tests
messages: 385830
nosy: pdonis
priority: normal
severity: normal
status: open
title: Use io.IncrementalNewlineDecoder for doctest newline conversion
type: enhancement
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43049>
_______________________________________

From report at bugs.python.org  Wed Jan 27 21:24:03 2021
From: report at bugs.python.org (fengjiang)
Date: Thu, 28 Jan 2021 02:24:03 +0000
Subject: [New-bugs-announce] [issue43050] threading timer memory leak
Message-ID: <1611800643.13.0.196440440542.issue43050@roundup.psfhosted.org>


New submission from fengjiang <fengjiang8 at huawei.com>:

Hi,we are transfering code from python2.7 to 3.7 and find that using threading.timer will cause memory leak. It works fine in python2.7 but not 3.7. To repreduce the problem, you can simply run the code below.

While True:
    timer = threading.Timer(5, None)
    timer.start()
    timer.cancel()

you will find the memory of progress increases rapidly

----------
components: Library (Lib)
messages: 385831
nosy: fengjiang
priority: normal
severity: normal
status: open
title: threading timer memory leak
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43050>
_______________________________________

From report at bugs.python.org  Thu Jan 28 00:26:33 2021
From: report at bugs.python.org (Charley Riley)
Date: Thu, 28 Jan 2021 05:26:33 +0000
Subject: [New-bugs-announce] [issue43051] Pip Permissions Error overwriting
 pip directory
Message-ID: <1611811593.92.0.540989910161.issue43051@roundup.psfhosted.org>


New submission from Charley Riley <mcriley821 at gmail.com>:

After installing Python 3.9.1 on Windows 10 system-wide to "C:\Program Files\Python39", I tried to pip install the package 'dirsearch'.

pip refused with the error: "Defaulting to user installation because normal site-packages is not writeable", and proceeded to write to Local AppData (unwanted, so I deleted from AppData).

I changed the permissions of the site-packages folder to allow python access (allowed Users and Application Packages full control). 

Then I retried installing 'dirsearch', which now completed successfully. However, I was warned that my pip version was out of date. So I did:
>>> py -m pip install --upgrade pip
I ensured py --version = 3.9.1

This failed with "ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\program files\\python39\\scripts\\pip.exe'

I then allowed same permissions as site-packages to the scripts folder. 

Retried 
>>> py -m pip install --upgrade pip
and I get an error:
"C:\Program Files\Python39\python.exe: No module named pip"

I navigate to site-packages, where pip directories have been renamed as:
~ip
~ip-21.0.dist-info

----------
components: Installation, Windows
messages: 385838
nosy: mcriley821, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Pip Permissions Error overwriting pip directory
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43051>
_______________________________________

From report at bugs.python.org  Thu Jan 28 01:39:56 2021
From: report at bugs.python.org (Isuru Fernando)
Date: Thu, 28 Jan 2021 06:39:56 +0000
Subject: [New-bugs-announce] [issue43052] _dyld_shared_cache_contains_path
 needs SYSTEM_VERSION_COMPAT=0
Message-ID: <1611815996.85.0.805200252907.issue43052@roundup.psfhosted.org>


New submission from Isuru Fernando <isuruf at gmail.com>:

In macOS Big Sur, if the executable was compiled with MACOSX_DEPLOYMENT_TARGET=10.15
or below, then SYSTEM_VERSION_COMPAT=1 is the default which means that Big Sur
reports itself as 10.16 which in turn means that __builtin_available(macOS 11.0)
will not be triggered.

This can be observed by using the python 3.9.1 universal2 installer and using it on
x86_64 Big Sur or with Rossetta 2 on arm64 Big Sur. (Not an issue with native arm64
as that part is compiled with MACOSX_DEPLOYMENT_TARGET=11.0)

Original issue is that the following returned None.
SYSTEM_VERSION_COMPAT=1 arch -x86_64 /usr/local/bin/python3 -c "from ctypes.util import find_library; print(find_library('AppKit'))"

----------
messages: 385845
nosy: isuruf, ned.deily, ronaldoussoren
priority: normal
pull_requests: 23185
severity: normal
status: open
title: _dyld_shared_cache_contains_path needs SYSTEM_VERSION_COMPAT=0

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43052>
_______________________________________

From report at bugs.python.org  Thu Jan 28 03:03:26 2021
From: report at bugs.python.org (Juraj Sukop)
Date: Thu, 28 Jan 2021 08:03:26 +0000
Subject: [New-bugs-announce] [issue43053] Speed up math.isqrt, again
Message-ID: <1611821006.77.0.580238316666.issue43053@roundup.psfhosted.org>


New submission from Juraj Sukop <juraj.sukop at gmail.com>:

This is a follow up to https://bugs.python.org/issue36887 and https://bugs.python.org/issue36957 .

The new `isqrt` is remarkably simple but it does not split the number at hand optimally. Ideally one would want to have 2n/n division everywhere but since the last iteration takes as much effort as all of the iterations before it this is what the attached code focuses on.

At least in my testing the `isqrt_2` code below improved the performance by 50% (3s down to 2s, for example) and, if used, perhaps the original `isqrt` could do without the final correction `a - (a*a > n)`.

----------
components: Extension Modules
files: isqrt_2.py
messages: 385848
nosy: juraj.sukop
priority: normal
severity: normal
status: open
title: Speed up math.isqrt, again
type: performance
versions: Python 3.8
Added file: https://bugs.python.org/file49770/isqrt_2.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43053>
_______________________________________

From report at bugs.python.org  Thu Jan 28 06:12:05 2021
From: report at bugs.python.org (Mark Shannon)
Date: Thu, 28 Jan 2021 11:12:05 +0000
Subject: [New-bugs-announce] [issue43054] What does the existence of a
 struct in a header file imply about the C-API
Message-ID: <1611832325.97.0.674198865532.issue43054@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

Given the lack of explicit documentation on this subject, and differing opinions among core developers, I though it would be good to discuss how the existence of a struct in a header file constrains the C-API.

Original PR provoking this discussion: https://github.com/python/cpython/pull/24298

Suppose a header file, e.g. funcobject.h, contains a struct, e.g. PyFunctionObject, to what extent is that struct part of the API?


If a struct is present in a header file, there are three options for what means in terms of the API (that make sense to me).

1. That the struct is not part of the API and may be freely changed or deleted.
2. That the struct is produced, or initialized, by an API function, which implies that existing fields will continue to exist, but they can be reorder or added to.
3. That the struct is consumed by an API function, which implies that the struct must keep its exact shape, only adding fields if flags are present in the pre-existing fields to indicate the use of the extension.

PyTypeObject is an example of (3).

We should be able to infer which of the above cases applies, if not explicitly documented, for any struct.

Using PyFunctionObject in funcobject.h as an example:

There is no API function or macro that directly produces or consumes the struct, which would imply case 1. But, the struct is documented as the struct for Python functions, and `PyFunction_Check()` exists, which strongly implies that the following code is OK:

if (PyFunction_Check(obj)) {  
    PyFunctionObject *func = (PyFunctionObject *)obj;
    ...

which therefore implies that (2) applies.
(3) does not apply as there is no API that takes a PyFunctionObject struct as a parameter.

Similar logic can be applied to other parts of the API.


Rather than go through this tortuous analysis for all headers, it might be better to document which structs are part of the API.

----------
messages: 385851
nosy: Mark.Shannon, gvanrossum, petr.viktorin, vstinner
priority: normal
severity: normal
status: open
title: What does the existence of a struct in a header file imply about the C-API

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43054>
_______________________________________

From report at bugs.python.org  Thu Jan 28 06:58:35 2021
From: report at bugs.python.org (Aman Anifer)
Date: Thu, 28 Jan 2021 11:58:35 +0000
Subject: [New-bugs-announce] [issue43055] Inconsistent behaviour when using
 walrus operator with 'and'/'or'
Message-ID: <1611835115.72.0.951775533132.issue43055@roundup.psfhosted.org>


New submission from Aman Anifer <fgoo.edu at hash.fyi>:

Using the walrus operator (:=) alongside 'and'/'or' shows inconsistent behaviour which changes with the order of given conditions. 

For example:
if(False and (x := 0)<1):
    print("Yes")
else:
    print(x)

Gives the error that 'NameError: name 'x' is not defined'
Whereas if the walrus operator is used first, like:
if((x := 0)>1 and False):
    print("Yes")
else:
    print(x)

Prints the value 0 without any error. This behaviour is the same when using 'or'. For example:

if(True or (y:=0)<1):
    print(y)
else:
    print("No")

Gives the same error but this:
if((y:=0)<1 or True):
    print(y)
else:
    print("No")

Prints the value 0.
I am guessing that this is because 'and' doesn't check the second operand if the first operand is False, also 'if' doesn't check the second operand if the first operand is True. I don't know if this is an intended behaviour.

Thanks

----------
components: Interpreter Core
messages: 385853
nosy: FuturisticGoo
priority: normal
severity: normal
status: open
title: Inconsistent behaviour when using walrus operator with 'and'/'or'
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43055>
_______________________________________

From report at bugs.python.org  Thu Jan 28 10:56:59 2021
From: report at bugs.python.org (Jesse Silverman)
Date: Thu, 28 Jan 2021 15:56:59 +0000
Subject: [New-bugs-announce] [issue43056] Use of dicts in sections 4.2 and
 4.7 of Python tutorial a bit confusing
Message-ID: <1611849419.6.0.947675106627.issue43056@roundup.psfhosted.org>


New submission from Jesse Silverman <jessevsilverman at gmail.com>:

The Python tutorial was not my first introduction to Python, but I see things in sections 4.2 and 4.7 that likely would have confused me a lot if it was.

In 4.2 near the end, we have great advice about how to avoid problems with modifying a collection while iterating thru it.

Unfortunately, it both makes use of a dict despite not having introduced it before at all within the tutorial (I wasn't shocked because it was NOT my first introduction to Python) and in fact, the .items() method which is beautifully explained in section 5.6, which hasn't been read yet.

I got thru it a bit shakily, and in a reference, rather than an introductory tutorial, it is quite literally impossible to present everything in a strict sequence, but I would definitely have been thrown for a loop if it was actually my first introduction to Python.

All was good again until I got to section 4.7.2, where I was again glad I had previously consumed tutorial materials concerning dict.  There is a forward reference to:
https://docs.python.org/3.9/library/stdtypes.html#typesmapping

Which every Python programmer should undoubtedly be familiar with eventually, but "why not?" to the more novice-friendly:
https://docs.python.org/3.9/tutorial/datastructures.html#dictionaries

The forward reference from this section to tuples in section 5 of the tutorial suggests that this might have been appropriate as well.

The dict, the tuple and parameter passing are all very basic topics indeed, but I would have found 4.7 very rough going if I had no prior exposure to dict and tuple.

This isn't a complaint demanding a refund for the cost of the tutorial, so much as an indication of where I think true novices attempting to follow the tutorial in sequence would likely be thrown for a loop.  I understand there is a healthy ecosystem of free and paid introductory tutorial material for Python, as evidenced by the fact that I was only working thru the tutorial just now, having been referred to it as a pre-req for other material I will be moving on to soon.

----------
assignee: docs at python
components: Documentation
messages: 385870
nosy: docs at python, jessevsilverman
priority: normal
severity: normal
status: open
title: Use of dicts in sections 4.2 and 4.7 of Python tutorial a bit confusing
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43056>
_______________________________________

From report at bugs.python.org  Thu Jan 28 12:26:53 2021
From: report at bugs.python.org (killv5)
Date: Thu, 28 Jan 2021 17:26:53 +0000
Subject: [New-bugs-announce] [issue43057] Timezone 'Etc/GMT-5' functions as
 Timezone 'Etc/GMT+5'
Message-ID: <1611854813.81.0.604991551457.issue43057@roundup.psfhosted.org>


New submission from killv5 <killianvetter at gmail.com>:

When I run the code attached I get the results below. This is not the expected behavior because I was expecting the GMT-5 timezone instead of GMT+5. When I tried to use GMT-5 I had the result of GMT+5. Luckily the timezone names still work ('EST','CST', etc.)

2021-01-28 19:10:47+05:00
+05

----------
components: Library (Lib)
files: timezone_example.py
messages: 385871
nosy: killv5
priority: normal
severity: normal
status: open
title: Timezone 'Etc/GMT-5' functions as Timezone 'Etc/GMT+5'
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file49771/timezone_example.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43057>
_______________________________________

From report at bugs.python.org  Thu Jan 28 15:17:02 2021
From: report at bugs.python.org (Brett Cohen)
Date: Thu, 28 Jan 2021 20:17:02 +0000
Subject: [New-bugs-announce] [issue43058] setting a logging Handler name
Message-ID: <1611865022.25.0.122108787235.issue43058@roundup.psfhosted.org>


New submission from Brett Cohen <bcohen at livedata.com>:

When using the logging module to add a new handler, the documentation doesn't mention the ability to set the name of the handler. It turns out that it's actually a supported feature by using:

    handler_instance.set_name("YOUR_NAME_HERE")

I'd suggest updating the documentation to indicate the ability to set the name of the handler. It'd be very nice if the name was settable in the constructor of the Handler class. This could be done by including an optional `name` argument that defaults to None:

    class Handler(Filterer):
        def __init__(self, level=NOTSET, name=None):
            Filterer.__init__(self)
            self._name = name
            ...
            ...

I noticed the problem when trying to call logging.config.dictConfig with a config which specified a handler (it crashes: `ValueError: No handler found with name 'my_handler'`).

I initially thought to manually alter the _handlers directly since the docs didn't indicate how to set the name. After digging into the source code I found the undocumented `set_name` function.

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 385880
nosy: bcohen, docs at python, vinay.sajip
priority: normal
severity: normal
status: open
title: setting a logging Handler name
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43058>
_______________________________________

From report at bugs.python.org  Thu Jan 28 16:07:11 2021
From: report at bugs.python.org (user1347091)
Date: Thu, 28 Jan 2021 21:07:11 +0000
Subject: [New-bugs-announce] [issue43059] sqlite3: Externally developed?
Message-ID: <1611868031.72.0.138640197563.issue43059@roundup.psfhosted.org>


New submission from user1347091 <accounts at lrose.de>:

In the documentation of the sqlite3 module it says:

See also:
    https://github.com/ghaering/pysqlite
    The pysqlite web page ? sqlite3 is developed externally under the name ?pysqlite?.

However, the README.md of the linked repository says:

    This project is not actively maintained any more: You are better off using the "sqlite3" module in the Python standard library, which originated from the pysqlite codebase.

Which information is correct? They seem contraticting to me. It looks like this is old information from times where sqlite3 wasn't integrated in the Python standard library yet - or am I missing something?

----------
assignee: docs at python
components: Documentation
messages: 385884
nosy: docs at python, user1347091
priority: normal
severity: normal
status: open
title: sqlite3: Externally developed?
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43059>
_______________________________________

From report at bugs.python.org  Fri Jan 29 04:46:53 2021
From: report at bugs.python.org (Erlend Egeberg Aasland)
Date: Fri, 29 Jan 2021 09:46:53 +0000
Subject: [New-bugs-announce] [issue43060] Convert _decimal C API from
 pointer array to struct
Message-ID: <1611913613.83.0.451385849535.issue43060@roundup.psfhosted.org>


New submission from Erlend Egeberg Aasland <erlend.aasland at innova.no>:

Ref. discussions on bpo-43009 and bpo-41798.

----------
components: Library (Lib)
messages: 385901
nosy: erlendaasland, shihai1991, vstinner
priority: normal
severity: normal
status: open
title: Convert _decimal C API from pointer array to struct
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43060>
_______________________________________

From report at bugs.python.org  Fri Jan 29 09:07:35 2021
From: report at bugs.python.org (Riccardo La Marca)
Date: Fri, 29 Jan 2021 14:07:35 +0000
Subject: [New-bugs-announce] [issue43061] subprocess: feature request: Get
 only the stdout of the last shell command
Message-ID: <1611929255.77.0.806477721139.issue43061@roundup.psfhosted.org>


New submission from Riccardo La Marca <riccardolamarca1 at gmail.com>:

This is a link with some examples of the required functionality:
https://stackoverflow.com/questions/65952314

----------
messages: 385909
nosy: Riccardo La Marca
priority: normal
severity: normal
status: open
title: subprocess: feature request: Get only the stdout of the last shell command
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43061>
_______________________________________

From report at bugs.python.org  Fri Jan 29 09:50:25 2021
From: report at bugs.python.org (HuangFuSL)
Date: Fri, 29 Jan 2021 14:50:25 +0000
Subject: [New-bugs-announce] [issue43062] Non-integer values in
 collections.Counter
Message-ID: <1611931825.03.0.205807042614.issue43062@roundup.psfhosted.org>


New submission from HuangFuSL <huangfusl at outlook.com>:

When I am creating a counter object provided by `collections.Counter` using a mapping object like a dictionary, it seems that Python will not check the validity of the values in the mapping object.

I've checked the following Python script could be successfully executed using Python 3.9.0 on Windows.

```python
>>> from collections import Counter
>>> a = Counter({'0': '0'})
>>> a.elements()
<itertools.chain object at 0x00000252DDB5A4F0>
```

`a.elements()` returns a iterator, iterating through it will normally get the records counted by the `Counter`, but with a `str` object inside, iterating through it will make a `TypeError` raised.

```python
>>> for i in a.elements():
...     pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'str' object cannot be interpreted as an integer
```

Meanwhile, if the counter contains values that cannot be compared such as `False` and `''`, `most_common()` method will fail.

```python
>>> b = Counter({'0': False, '1': ''})
>>> b.most_common()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python39\lib\collections\__init__.py", line 610, in most_common
    return sorted(self.items(), key=_itemgetter(1), reverse=True)
TypeError: '<' not supported between instances of 'bool' and 'str'
```

The `sys.version` variable of my Python interpreter is as follows:

```
>>> import sys
>>> sys.version
'3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]'
>>>
```

I'm not sure whether the result is intentionally designed, but I think such execution results may lead to confusion.

----------
components: Extension Modules
messages: 385913
nosy: HuangFuSL
priority: normal
severity: normal
status: open
title: Non-integer values in collections.Counter
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43062>
_______________________________________

From report at bugs.python.org  Fri Jan 29 11:56:11 2021
From: report at bugs.python.org (Florian Bruhin)
Date: Fri, 29 Jan 2021 16:56:11 +0000
Subject: [New-bugs-announce] [issue43063] zipfile.Path / importlib.resources
 raises KeyError if a file wasn't found
Message-ID: <1611939371.43.0.787270932991.issue43063@roundup.psfhosted.org>


New submission from Florian Bruhin <python.org at the-compiler.org>:

When a package is installed as an egg, importlib.resources.files returns a zipfile.Path rather than a pathlib.Path (maybe it returns other things too, seeing that it's documented to return a importlib.abc.Traversable - I didn't check).

However, those two have a rather odd inconsistency when it comes to reading files which don't actually exist. In that case, zipfile.Path raises KeyError rather than FileNotFoundError. After a "zip tmp/test.zip somefile":

    >>> import zipfile
    >>> p = zipfile.Path('tmp/test.zip')
    >>> (p / 'helloworld').read_text()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/zipfile.py", line 2318, in read_text
        with self.open('r', *args, **kwargs) as strm:
      File "/usr/lib/python3.9/zipfile.py", line 2306, in open
        stream = self.root.open(self.at, zip_mode, pwd=pwd)
      File "/usr/lib/python3.9/zipfile.py", line 1502, in open
        zinfo = self.getinfo(name)
      File "/usr/lib/python3.9/zipfile.py", line 1429, in getinfo
        raise KeyError(
    KeyError: "There is no item named 'helloworld' in the archive"

Note that the "zipp" backport (used by the "importlib_resources" backport) does raise FileNotFoundError instead:

    >>> import zipp
    >>> p2 = zipp.Path('tmp/test.zip')
    >>> (p2 / 'helloworld').read_text()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/site-packages/zipp.py", line 267, in read_text
        with self.open('r', *args, **kwargs) as strm:
      File "/usr/lib/python3.9/site-packages/zipp.py", line 250, in open
        raise FileNotFoundError(self)
    FileNotFoundError: tmp/test.zip/helloworld

And of course, so does pathlib.Path:

    >>> import pathlib
    >>> p3 = pathlib.Path('tmp')
    >>> (p3 / 'helloworld').read_text()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/pathlib.py", line 1255, in read_text
        with self.open(mode='r', encoding=encoding, errors=errors) as f:
      File "/usr/lib/python3.9/pathlib.py", line 1241, in open
        return io.open(self, mode, buffering, encoding, errors, newline,
      File "/usr/lib/python3.9/pathlib.py", line 1109, in _opener
        return self._accessor.open(self, flags, mode)
    FileNotFoundError: [Errno 2] No such file or directory: 'tmp/helloworld'

When using `importlib.resources.files`, this can be very surprising - especially because during testing, the package might not be installed as an egg, so this never turns up until users complain about it (which is what happened in my case - no bad feelings though!).

This seems to have been fixed by jaraco in ebbe8033b1c61854c4b623aaf9c3e170d179f875, by introducing an explicit:

    if not self.exists() and zip_mode == 'r':
        raise FileNotFoundError(self)

in open(), as part of what seems like an unrelated change (bpo-40564 / GH-22371).

While this is arguably a backwards-compatible change between 3.9 and 3.10, it might be a good idea to either adjust Python 3.9 to have the same behavior (perhaps with a new exception which inherits from both KeyError and FileNotFoundError?). At the very least, I feel like this should be documented prominently in the importlib.resources.files, importlib.abc.Traversable and zipfile.Path documentation for 3.9.

As an aside, the error message when using `.iterdir()` is similarly confusing, though that's at least consistent between the stdlib and the zipp backport:

    >>> (p / 'helloworld').iterdir()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/zipfile.py", line 2342, in iterdir
        raise ValueError("Can't listdir a file")
    ValueError: Can't listdir a file

    >>> (p2 / 'helloworld').iterdir()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/site-packages/zipp.py", line 291, in iterdir
        raise ValueError("Can't listdir a file")
    ValueError: Can't listdir a file

    >>> list((p3 / 'helloworld').iterdir())
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/pathlib.py", line 1149, in iterdir
        for name in self._accessor.listdir(self):
    FileNotFoundError: [Errno 2] No such file or directory: 'tmp/helloworld'

----------
components: Library (Lib)
messages: 385919
nosy: The Compiler, alanmcintyre, brett.cannon, jaraco, serhiy.storchaka, twouters
priority: normal
severity: normal
status: open
title: zipfile.Path / importlib.resources raises KeyError if a file wasn't found
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43063>
_______________________________________

From report at bugs.python.org  Fri Jan 29 14:51:37 2021
From: report at bugs.python.org (Selim Belhaouane)
Date: Fri, 29 Jan 2021 19:51:37 +0000
Subject: [New-bugs-announce] [issue43064] Impossible to override signal
 handler set with add_signal_handler in forked process
Message-ID: <1611949897.69.0.881699073289.issue43064@roundup.psfhosted.org>


New submission from Selim Belhaouane <selim.belhaouane at gmail.com>:

Cannot clear signal handler set with loop.add_signal_handler in forked process with signal.signal

# Context

I'm running an async web server with uvicorn[1] and have background processes (using multiprocessing) doing CPU-bound work. uvicorn install signal handlers with loop.add*signal_handler, if available[2]. It also implements the common "attempt graceful shutdown on the first signal, and forcefully shutdown on the second (or more) signal" pattern. The problem I noticed was: when \_forking* at least one process, the server never gracefully shuts down, **even if I install new signal handlers in the subprocesses**.

# Problem

Signal handlers installed with loop.add*signal_handler cannot be cleared in forked processes with `signal.signal` (see Experiment 1), \_unless* using `signal.SIG_DFL` or `signal.SIG_IGN` in the forked processes (see Experiment 3).

For the record:

- When installing signal handlers in the parent process with signal.signal, this is not a problem (see Experiment 2).
- When using multiprocessing with the "spawn" method, this is not a problem. Unsurprising, since signal handlers are not inherited from the parent process when using "spawn" (see Experiment 4).
- When install signal handlers in the child proceess with loop.add_signal_handler, this is not a problem (see Experiment 5).

# Experiments

You'll find a minimal `exp.py` file in the attached archive. There's a few tweakable parameters at the top. I just run this with `python3.X exp.py`, wait a second or two, and hit Ctrl+C.
`results.txt` shows the results of a few experiments with different set of parameters.
Note that the point at which I hit Ctrl+C is indicated by the ? (heart) symbol, due to some terminal weirdness, although this is quite useful in this case!
The subsections below detail what happens when Ctrl+C is hit.
You guys probably know this already, but Ctrl+C basically sends SIGINT to all processes in the process tree.

## Experiment 1

- Installs a signal handler in the parent process with loop.add_signal_handler
- Starts 3 subprocesses with "fork"
- The subprocesses install a new signal handler with signal.signal

Outcome: both the parent handler and child handlers get called (3 calls to handle_sig_worker, 4 calls to handle_sig_main)

**Expected**: I would expect a single call to handle_sig_main, and 3 calls to handle_sig_worker, as in Experiment 2 (which uses signal.signal instead of loop.add_signal_handler) or Experiment 4 (which uses "spawn" instead of "fork") or Experiment 5 ()

## Experiment 2

As Experiment 1, but signal handlers are installed in the parent process with signal.signal

Outcome: the parent handler gets called once, and the child handlers get called 3 times

## Experiment 3

Same as Experiment 1, but this time using `signal.SIG_DFL` as the callback in the child processes.

Outcome: the child processes immediately terminate, thanks to SIG_DFL, and the parent handler gets called only once!

## Experiment 4

Same as Experiment 1, but this time using "spawn" as the multiprocessing start method.

Outcome: same as Experiment 2: parent handlers gets called once, child handlers get called 3 times

## Experiment 5

Same as Experiment 1, but this time installing signal handlers in the child processes with loop.add_signal_handler.

Outcome: Same as Experiment 2.

# Environment

I was able to replicate the issue with python 3.7, 3.8, 3.9 and 3.10 (didn't even try 3.6) on Ubuntu 20.04. I obtained python via `apt`.

[1]: https://www.uvicorn.org/
[2]: https://github.com/encode/uvicorn/blob/61a6cabb4580e1c923df396eac264803f599412c/uvicorn/server.py#L281

----------
components: asyncio
files: python-async-signal-bug.zip
messages: 385928
nosy: asvetlov, selimb, yselivanov
priority: normal
severity: normal
status: open
title: Impossible to override signal handler set with add_signal_handler in forked process
type: behavior
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49774/python-async-signal-bug.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43064>
_______________________________________

From report at bugs.python.org  Fri Jan 29 15:03:18 2021
From: report at bugs.python.org (doublex)
Date: Fri, 29 Jan 2021 20:03:18 +0000
Subject: [New-bugs-announce] [issue43065] 'concurrent.futures' stucks in
 3.9.1
Message-ID: <1611950599.0.0.548007285224.issue43065@roundup.psfhosted.org>


New submission from doublex <doppelbauer at gmx.net>:

Works with python 3.8
Fails with 3.9.1 (tested on CentOS 7 and Ubuntu 2004)
Yes, sometimes is does not stuck...

Command-line: 
python3.9 stuck.py

----------
components: Interpreter Core
files: stuck.py
messages: 385931
nosy: doublex
priority: normal
severity: normal
status: open
title: 'concurrent.futures' stucks in 3.9.1
type: resource usage
versions: Python 3.9
Added file: https://bugs.python.org/file49775/stuck.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43065>
_______________________________________

From report at bugs.python.org  Fri Jan 29 15:28:48 2021
From: report at bugs.python.org (Garrison Taylor)
Date: Fri, 29 Jan 2021 20:28:48 +0000
Subject: [New-bugs-announce] [issue43066] Zipfile with leading slashes
Message-ID: <1611952128.43.0.663068649879.issue43066@roundup.psfhosted.org>


New submission from Garrison Taylor <garrison.taylor at gridunity.com>:

Currently the zipfile library allows you to create invalid zip files. The following code is an example:

from zipfile import ZipFile
import tempfile
temporary_file = tempfile.NamedTemporaryFile()
my_zip = ZipFile(temporary_file.name, 'w')
my_zip.writestr('/some_folder/some_file.txt', 'Some content')
my_zip.close()

The generated zipfile contains "/some_folder/some_file.txt". However, according to the specification for zip files, this is invalid. See below, from the .ZIP File Format Specification version 6.3.9:

       4.4.17.1 The name of the file, with optional relative path.
       The path stored MUST NOT contain a drive or
       device letter, or a leading slash.  All slashes
       MUST be forward slashes '/' as opposed to
       backwards slashes '\' for compatibility with Amiga
       and UNIX file systems etc.  If input came from standard
       input, there is no file name field.  

This is significant because the default Windows Explorer zip file extractor cannot handle zip files that contain a leading slash, producing an error that "The compressed (zipped) folder is invalid."

----------
components: Library (Lib)
messages: 385934
nosy: garrison.taylor
priority: normal
severity: normal
status: open
title: Zipfile with leading slashes
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43066>
_______________________________________

From report at bugs.python.org  Fri Jan 29 16:00:52 2021
From: report at bugs.python.org (=?utf-8?b?0JHQvtGA0LjRgSDQktC10YDRhdC+0LLRgdC60LjQuQ==?=)
Date: Fri, 29 Jan 2021 21:00:52 +0000
Subject: [New-bugs-announce] [issue43067] inspect.signature(bytes.hex)
 raises ValueError "builtin has invalid signature"
Message-ID: <1611954052.56.0.164683452654.issue43067@roundup.psfhosted.org>


New submission from ????? ?????????? <boris.verk at gmail.com>:

On Python 3.9

>>> import inspect
>>> inspect.signature(bytes.hex)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/inspect.py", line 3118, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/usr/lib/python3.9/inspect.py", line 2867, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
  File "/usr/lib/python3.9/inspect.py", line 2322, in _signature_from_callable
    return _signature_from_builtin(sigcls, obj,
  File "/usr/lib/python3.9/inspect.py", line 2135, in _signature_from_builtin
    return _signature_fromstr(cls, func, s, skip_bound_arg)
  File "/usr/lib/python3.9/inspect.py", line 2005, in _signature_fromstr
    raise ValueError("{!r} builtin has invalid signature".format(obj))
ValueError: <method 'hex' of 'bytes' objects> builtin has invalid signature


This is also true for memoryview.hex and bytearray.hex . This happens because

>>> bytes.hex.__text_signature__
'($self, /, sep=<unrepresentable>, bytes_per_sep=1)'

when it would make more sense for it to return

'($self, /, sep='', bytes_per_sep=1)'

or at least sep=None

----------
components: Library (Lib)
messages: 385940
nosy: boris
priority: normal
severity: normal
status: open
title: inspect.signature(bytes.hex) raises ValueError "builtin has invalid signature"
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43067>
_______________________________________

From report at bugs.python.org  Fri Jan 29 16:49:00 2021
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 29 Jan 2021 21:49:00 +0000
Subject: [New-bugs-announce] [issue43068] test_subprocess:
 test_specific_shell() fails on AMD64 FreeBSD Shared 3.x
Message-ID: <1611956940.56.0.594651223595.issue43068@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

https://buildbot.python.org/all/#/builders/483/builds/758

FAIL: test_specific_shell (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_subprocess.py", line 2286, in test_specific_shell
    self.assertEqual(p.stdout.read().strip(), bytes(sh, 'ascii'))
AssertionError: b'' != b'/usr/local/bin/bash'

AMD64 FreeBSD Shared 3.9 is also affected:
https://buildbot.python.org/all/#/builders/151/builds/283

And AMD64 FreeBSD Shared 3.8:
https://buildbot.python.org/all/#/builders/555/builds/197

----------
components: Tests
messages: 385945
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_subprocess: test_specific_shell() fails on AMD64 FreeBSD Shared 3.x
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43068>
_______________________________________

From report at bugs.python.org  Fri Jan 29 17:32:21 2021
From: report at bugs.python.org (Ciprian Dorin Craciun)
Date: Fri, 29 Jan 2021 22:32:21 +0000
Subject: [New-bugs-announce] [issue43069] Python fails to read a script
 whose path is `/dev/fd/X`
Message-ID: <1611959541.65.0.0210578215871.issue43069@roundup.psfhosted.org>


New submission from Ciprian Dorin Craciun <ciprian.craciun at gmail.com>:

Sometimes (especially from wrapper scripts) one would like to call Python with a script that should be read from a file-descriptor, like `python3 /dev/fd/9`;  most likely the backing is a file in `TMPDIR` (perhaps unlinked, like `bash`'s "here documents").

However on OSX (I've tried it on `10.15.7` and `10.13.6`), for some reason if that script is too "large" (more than a couple of characters) it just bails out, even if that file-descriptor is in fact backed by a file.

For example:
~~~~
echo 'print("1234567890")' >/tmp/x.py && python3 /dev/fd/9 9</tmp/x.py
# correctly prints 1234567890

echo 'print("12345678901234567890")' >/tmp/x.py && python3 /dev/fd/9 9</tmp/x.py
# prints nothing, no error, no exit code, etc.

# alternative variant 1, with `bash` "here-documents"
python3 /dev/fd/9 9<<<'print("12345678901234567890")'
# still prints nothing.

# alternative variant 2, that uses `/dev/stdin` instead of `/dev/fd/N`
python3 /dev/stdin <<<'print("12345678901234567890")'
# still prints nothing.

# alternative variant 3, that uses `open` and `exec`
python3 -c 'exec(open("/dev/fd/9").read())' 9<<<'print("12345678901234567890")'
# correctly prints 12345678901234567890
~~~~

The file `/tmp/x.py` is just a simple script that prints that token.

I've tried both Python 3.9.1, 3.8.2 and even 2.7.18 and 2.7.16, all with the same results.  (This makes me think it's actually an OSX issue?)

On Linux this works flawlesly.  Furthermore if one uses something else like `cat`, `bash` or anything else it works.  Thus it is something related with Python on OSX.

Also as seen from the examples, this is not a "shell issue" or something similar;  it just seems to hit a corner case when the script path is `/dev/fd/...` or `/dev/stdin`

----------
components: Interpreter Core, macOS
messages: 385955
nosy: ciprian.craciun, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Python fails to read a script whose path is `/dev/fd/X`
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43069>
_______________________________________

From report at bugs.python.org  Fri Jan 29 18:35:54 2021
From: report at bugs.python.org (Trey Hunner)
Date: Fri, 29 Jan 2021 23:35:54 +0000
Subject: [New-bugs-announce] [issue43070] Control keys stop working after
 pressing Ctrl-C Escape Enter
Message-ID: <1611963354.38.0.774052304558.issue43070@roundup.psfhosted.org>


New submission from Trey Hunner <trey at truthful.technology>:

On Ubuntu Linux 18.04 when I run Python from within a terminal, control keys such as Ctrl-L, Ctrl-A, Ctrl-E, and Ctrl-K work as expected.

However, if I type Ctrl-C Escape Enter (^C followed by Esc followed by Return), control keys stop working as expected. Ctrl-C still causes KeyboardInterrupt to be printed, but pressing Ctrl-L causes "^L" to be shown and Ctrl-A shows "^A".

This happens in Gnome Terminal as well as XTerm both on Ubuntu Linux 18.04. It happens in zsh, bash, tcsh, and sh.

I've reproduced this issue in Python 2.7, 3.6, 3.7, 3.8, 3.9, and 3.10. Some of these Python versions I installed via pyenv, some came from the deadsnakes PPA, and some were preinstalled with Ubuntu.

----------
messages: 385957
nosy: trey
priority: normal
severity: normal
status: open
title: Control keys stop working after pressing Ctrl-C Escape Enter
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43070>
_______________________________________

From report at bugs.python.org  Fri Jan 29 22:26:09 2021
From: report at bugs.python.org (Greg)
Date: Sat, 30 Jan 2021 03:26:09 +0000
Subject: [New-bugs-announce] [issue43071] IDLE: Windows 7 - Trackpad
 two-finger vertical scrolling is not recognized
Message-ID: <1611977169.85.0.900313540733.issue43071@roundup.psfhosted.org>


New submission from Greg <gregsaywhaat at gmail.com>:

Up/down scrolling is not possible with a two-finger swipe on a trackpad. I'm using Lenovo's notably bad UltraNav drivers on Windows 7. Horizontal scrolling works just fine. PgUp and PgDn both behave as normal, as does ctrl + arrow keys.

I'm having this issue with IDLE for 3.8.7, and 3.7.9, but not 2.7.3 (which just happens to be the last version I had installed).

----------
assignee: terry.reedy
components: IDLE
messages: 385968
nosy: Kritzy, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Windows 7 - Trackpad two-finger vertical scrolling is not recognized
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43071>
_______________________________________

From report at bugs.python.org  Fri Jan 29 23:03:55 2021
From: report at bugs.python.org (akshat goyal)
Date: Sat, 30 Jan 2021 04:03:55 +0000
Subject: [New-bugs-announce] [issue43072] Memory Leak when using Thread
Message-ID: <1611979435.69.0.821660649621.issue43072@roundup.psfhosted.org>


New submission from akshat goyal <goyalakshat14 at gmail.com>:

when using Thread and allocating memory resource usage is not down as opposed to when using Process or directly calling function as can be seen in the attached script.

To replicate:
start the script with the three functions one by one and see the memory usage of the script.
Thread has the highest of all three.

----------
components: Library (Lib)
files: simple.py
messages: 385969
nosy: goyalakshat14
priority: normal
severity: normal
status: open
title: Memory Leak when using Thread
type: resource usage
versions: Python 3.6, Python 3.8
Added file: https://bugs.python.org/file49777/simple.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43072>
_______________________________________

From report at bugs.python.org  Fri Jan 29 23:38:58 2021
From: report at bugs.python.org (Eric Poulsen)
Date: Sat, 30 Jan 2021 04:38:58 +0000
Subject: [New-bugs-announce] [issue43073] Adding a ctypes.Union to a
 ctypes.BigEndianStructure results in an error
Message-ID: <1611981538.61.0.504435630016.issue43073@roundup.psfhosted.org>


New submission from Eric Poulsen <eric at zyxod.com>:

Placing a ctypes.Union inside of a ctypes.BigEndianStructure results in "TypeError: This type does not support other endian".  I believe this is a similar problem to issue #4376 (https://bugs.python.org/issue4376)

Minimum repro test case:

import ctypes as ct

class U(ct.Union):
    _pack_=True
    _fields_=[
        ('a', ct.c_int),
        ('b', ct.c_int),
    ]

class S(ct.BigEndianStructure):
    _pack_=True
    _fields_=[
        ('x', ct.c_int),
        ('y', U),
    ]

I believe the fix is similar to that issue, though I admit I don't know enough about this code to be sure.

diff --git a/Lib/ctypes/_endian.py b/Lib/ctypes/_endian.py
index 37444bd6a7..525c5e58c9 100644
--- a/Lib/ctypes/_endian.py
+++ b/Lib/ctypes/_endian.py
@@ -18,6 +18,9 @@ def _other_endian(typ):
     # if typ is structure
     if issubclass(typ, Structure):
         return typ
+    # if typ is union:
+    if issubclass(typ, Union):
+        return typ
     raise TypeError("This type does not support other endian: %s" % typ)
 
 class _swapped_meta(type(Structure)):

----------
components: ctypes
messages: 385970
nosy: MrSurly
priority: normal
severity: normal
status: open
title: Adding a ctypes.Union to a ctypes.BigEndianStructure results in an error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43073>
_______________________________________

From report at bugs.python.org  Sat Jan 30 02:58:09 2021
From: report at bugs.python.org (=?utf-8?b?5Y+M6I2J6YW46YWv?=)
Date: Sat, 30 Jan 2021 07:58:09 +0000
Subject: [New-bugs-announce] [issue43074] Unable to use system proxy with
 HTTPS connection on Windows
Message-ID: <1611993489.22.0.338889763746.issue43074@roundup.psfhosted.org>


New submission from ???? <admin at 0x.mk>:

On Windows you can only set a "host:port" http proxy as system proxy.
```
PS > Get-ItemProperty -Path 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings' | findstr ProxyServer
ProxyServer              : 127.0.0.1:7890
```

But in Python, it assumes that this proxy supports both http and https on the same port:

```
>>> import urllib
>>> urllib.request.getproxies()
{'http': 'http://127.0.0.1:7890', 'https': 'https://127.0.0.1:7890', 'ftp': 'ftp://127.0.0.1:7890'}
```

On Python 3.8.x it works but on 3.9.x it keeps failing to handshake on https traffic.

3.8.x:

```
> .\python38.exe
Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib, requests
>>> urllib.request.getproxies()
{'http': 'http://127.0.0.1:7890', 'https': 'https://127.0.0.1:7890', 'ftp': 'ftp://127.0.0.1:7890'}
>>> requests.get("http://www.google.com")
<Response [200]>
>>> requests.get("https://www.google.com")
<Response [200]>
```

```
> python3
Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib, requests
>>> urllib.request.getproxies()
{'http': 'http://127.0.0.1:7890', 'https': 'https://127.0.0.1:7890', 'ftp': 'ftp://127.0.0.1:7890'}
>>> requests.get("http://www.google.com")
<Response [200]>
>>> requests.get("https://www.google.com")
Traceback (most recent call last):
  File "C:\Python39\lib\site-packages\urllib3\connectionpool.py", line 696, in urlopen
    self._prepare_proxy(conn)
  File "C:\Python39\lib\site-packages\urllib3\connectionpool.py", line 964, in _prepare_proxy
    conn.connect()
  File "C:\Python39\lib\site-packages\urllib3\connection.py", line 359, in connect
    conn = self._connect_tls_proxy(hostname, conn)
  File "C:\Python39\lib\site-packages\urllib3\connection.py", line 496, in _connect_tls_proxy
    return ssl_wrap_socket(
  File "C:\Python39\lib\site-packages\urllib3\util\ssl_.py", line 432, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
  File "C:\Python39\lib\site-packages\urllib3\util\ssl_.py", line 474, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock)
  File "C:\Python39\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Python39\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Python39\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1123)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python39\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Python39\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "C:\Python39\lib\site-packages\urllib3\util\retry.py", line 573, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1123)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python39\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Python39\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python39\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python39\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python39\lib\site-packages\requests\adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1123)')))
```

----------
components: Windows
messages: 385973
nosy: admin3, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Unable to use system proxy with HTTPS connection on Windows
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43074>
_______________________________________

From report at bugs.python.org  Sat Jan 30 03:11:46 2021
From: report at bugs.python.org (yeting li)
Date: Sat, 30 Jan 2021 08:11:46 +0000
Subject: [New-bugs-announce] [issue43075] ReDoS in request
Message-ID: <1611994306.46.0.333529770265.issue43075@roundup.psfhosted.org>


New submission from yeting li <liyt at ios.ac.cn>:

Hi,

I find this regex '(?:^|,)[ \t]*([^ \t]+)[ \t]+' may be stucked by input.

The vulnerable regex is located in https://github.com/python/cpython/blob/5c5a938573ce665f00e362c7766912d9b3f3b44e/Lib/urllib/request.py#L946

The ReDOS vulnerability of the regex is mainly due to the sub-pattern ',([^ \t]+)' and can be exploited with the following string
attack_str = "," * 10000

You can execute redos_python.py to reproduce the ReDos vulnerability.


I am willing to suggest that you replace '(?:^|,)[ \t]*([^ \t]+)[ \t]+' with '(?:^|,)[ \t]*([^ \t,]+)[ \t]+'

Looking forward for your response?!

Best,
Yeting Li

----------
components: Library (Lib)
files: redos_python.py
messages: 385974
nosy: yetingli
priority: normal
severity: normal
status: open
title: ReDoS in request
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49778/redos_python.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43075>
_______________________________________

From report at bugs.python.org  Sat Jan 30 03:13:38 2021
From: report at bugs.python.org (Aleksandr Sigalov)
Date: Sat, 30 Jan 2021 08:13:38 +0000
Subject: [New-bugs-announce] [issue43076] str.split() indexing issue
Message-ID: <1611994418.49.0.767343252792.issue43076@roundup.psfhosted.org>


New submission from Aleksandr Sigalov <tegridycode at gmail.com>:

The following code ran in Google Colab...

=====================================

import sys

print (sys.version)

string = 'WORD=BIRD\nBIRD\nBIRD'

print(string.split())

print('========')

print(string.split('=')[0][0:3])
print(string.split('=')[1][0:3])

=================================

produces this...

=================================

3.6.9 (default, Oct  8 2020, 12:12:24) 
[GCC 8.4.0]
['WORD=BIRD', 'BIRD', 'BIRD']
========
WOR
BIR

=================================

Shouldn't index [0:3] give 4 chars? I looked in the docs and I could not find anything explaining this behavior.

=================================

Thanks.

----------
components: Build
messages: 385975
nosy: tegridycode
priority: normal
severity: normal
status: open
title: str.split() indexing issue
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43076>
_______________________________________

From report at bugs.python.org  Sat Jan 30 10:18:41 2021
From: report at bugs.python.org (Paul Moore)
Date: Sat, 30 Jan 2021 15:18:41 +0000
Subject: [New-bugs-announce] [issue43077] Update bundled pip to 21.0.1 and
 setuptools to 52.0.0
Message-ID: <1612019921.62.0.653210880281.issue43077@roundup.psfhosted.org>


New submission from Paul Moore <p.f.moore at gmail.com>:

I've just released pip 21.0.1, so this updates the bundled copy (as well as updating setuptools).

I assume this should be backported to 3.8 and 3.9.

----------
components: Library (Lib)
messages: 385986
nosy: paul.moore
priority: normal
pull_requests: 23201
severity: normal
status: open
title: Update bundled pip to 21.0.1 and setuptools to 52.0.0
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43077>
_______________________________________

From report at bugs.python.org  Sat Jan 30 12:36:12 2021
From: report at bugs.python.org (Rohan Prasad)
Date: Sat, 30 Jan 2021 17:36:12 +0000
Subject: [New-bugs-announce] [issue43078] Equality Errors when Unpickling
 and Pickling a Dictionary with a nan
Message-ID: <1612028172.34.0.735632313742.issue43078@roundup.psfhosted.org>


New submission from Rohan Prasad <rohandev.1200 at gmail.com>:

Hi,

I'm having a weird issue with floats in dictionaries and equality comparison (see attached screenshot for an example). When I create two dictionaries with a nan value they pass an equivalence test. However, if I pickle and unpickle one of the dictionaries they become no longer equivalent. I would expect that the behavior would be the same irrespective of whether they are pickled/unpickled. Is there something else I should be doing here or is there a way to easily resolve this (short of writing custom ways to compare equality)?

----------
files: numpy-nan-dict-error.png
messages: 385990
nosy: rp4fx12
priority: normal
severity: normal
status: open
title: Equality Errors when Unpickling and Pickling a Dictionary with a nan
type: behavior
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file49781/numpy-nan-dict-error.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43078>
_______________________________________

From report at bugs.python.org  Sat Jan 30 17:17:20 2021
From: report at bugs.python.org (Al)
Date: Sat, 30 Jan 2021 22:17:20 +0000
Subject: [New-bugs-announce] [issue43079] wrong IP address range given for
 ipv4_mapped
Message-ID: <1612045040.37.0.190941589781.issue43079@roundup.psfhosted.org>


New submission from Al <awpythonbugs at sunnyside.com>:

https://docs.python.org/3.9/library/ipaddress.html

This documentation for the 'ipaddress' IPv4/IPv6 Manipulation Library incorrectly states the address range for 'ipv4_mapped' as ::FFFF/96 (which means 0:0:0:0:0:0:0:FFFF/96).  This is just flat out wrong.  There are some websites out there, e.g. APNIC, that also have this error.

The correct address range can be expressed by any of the following methods:
1) 0:0:0:0:0:FFFF:0:0/96
2) ::FFFF:0:0/96
3) 0:0:0:0:0:FFFF::/96

----------
assignee: docs at python
components: Documentation
messages: 385999
nosy: alwhaley, docs at python
priority: normal
severity: normal
status: open
title: wrong IP address range given for ipv4_mapped
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43079>
_______________________________________

From report at bugs.python.org  Sat Jan 30 19:14:38 2021
From: report at bugs.python.org (Lewis Gaul)
Date: Sun, 31 Jan 2021 00:14:38 +0000
Subject: [New-bugs-announce] [issue43080] pprint for dataclass instances
Message-ID: <1612052078.12.0.741479641556.issue43080@roundup.psfhosted.org>


New submission from Lewis Gaul <lewis.gaul at gmail.com>:

Currently the pprint module does not have support for dataclasses. I have implemented support for this and will link the PR once I have the issue number!

See also issue37376 for SimpleNamespace support.

----------
components: Library (Lib)
messages: 386002
nosy: LewisGaul, eric.snow, rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: pprint for dataclass instances
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43080>
_______________________________________

From report at bugs.python.org  Sun Jan 31 00:26:56 2021
From: report at bugs.python.org (Xinmeng Xia)
Date: Sun, 31 Jan 2021 05:26:56 +0000
Subject: [New-bugs-announce] [issue43081] Recursive call crash module
 multiprocessing
Message-ID: <1612070816.14.0.20481029059.issue43081@roundup.psfhosted.org>


New submission from Xinmeng Xia <xiaxm at smail.nju.edu.cn>:

The following code recursively calls function test_forkserver(). However, the output is not expected RecursionError(recursively calling) or OSError(too many open files). An unexpected EOFError ( inside module multiprocessing, not this code itself) is reported. According to the error message, Python interpreter seems to behavior abnormal on line 1, "import multiprocessing". I try other cases in my machine, multiprocessing module can work well. Only this case will crash module multiprocessing.

test.py
======================================
import multiprocessing
import os

def do(i):
      print(test_forkserver(), os.getpid())

def test_forkserver():
      mp = multiprocessing.get_context('forkserver')
      mp.Pool(2).map(do(mp), range(3))
if __name__ == '__main__':
      test_forkserver()
======================================

Attached output:
------------------------------------------------------------------
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/python3.10/lib/python3.10/multiprocessing/forkserver.py", line 258, in main
    fds = reduction.recvfds(s, MAXFDS_TO_SEND + 1)
  File "/usr/local/python3.10/lib/python3.10/multiprocessing/reduction.py", line 159, in recvfds
    raise EOFError
EOFError
...
------------------------------------------------------------------

System Info:
>>uname -a
Linux xxm 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

>>python -V
Python 3.10.0a4

----------
components: Library (Lib)
messages: 386008
nosy: xxm
priority: normal
severity: normal
status: open
title: Recursive call crash module multiprocessing
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43081>
_______________________________________

From report at bugs.python.org  Sun Jan 31 02:56:59 2021
From: report at bugs.python.org (diegoe)
Date: Sun, 31 Jan 2021 07:56:59 +0000
Subject: [New-bugs-announce] [issue43082] descriptor howto: typo in
 "Definition and introduction"
Message-ID: <1612079819.98.0.888102810331.issue43082@roundup.psfhosted.org>


New submission from diegoe <diegoe at gnome.org>:

In
  https://docs.python.org/3.10/howto/descriptor.html#definition-and-introduction

The paragraph reads:
"""
Definition and introduction

In general, a descriptor is an attribute value that has one of the methods in the descriptor protocol. Those methods are __get__(), __set__(), and __delete__(). If any of those methods are defined for an the attribute, it is said to be a descriptor.
"""

I believe it should be either "an attribute" or "the attribute" in the last sentence. Probably missed it because both options would read fine.

----------
assignee: docs at python
components: Documentation
messages: 386011
nosy: diegoe, docs at python
priority: normal
severity: normal
status: open
title: descriptor howto: typo in "Definition and introduction"
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43082>
_______________________________________

From report at bugs.python.org  Sun Jan 31 08:47:29 2021
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 31 Jan 2021 13:47:29 +0000
Subject: [New-bugs-announce] [issue43083] Minor errors related to error
 handling in _sqlite3 module
Message-ID: <1612100849.9.0.66215198535.issue43083@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

There are few minor errors in the _sqlite3:

* In two places the result of PyList_Append() is not checked.
* The lastrowid field of pysqlite_Cursor can contain a reference to destroyed object when GIL is released for calling sqlite3_last_insert_rowid().
* Exception after PyDict_DelItem() in pysqlite_connection_create_collation_impl() is not cleared before calling _pysqlite_seterror() (and there is always an exception set if callable is None). Unlikely it can cause problems, but it is better to be explicit.

----------
components: Extension Modules
messages: 386015
nosy: berker.peksag, ghaering, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Minor errors related to error handling in _sqlite3 module
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43083>
_______________________________________

From report at bugs.python.org  Sun Jan 31 09:44:59 2021
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 31 Jan 2021 14:44:59 +0000
Subject: [New-bugs-announce] [issue43084] curses.window.enclose() should
 return bool
Message-ID: <1612104299.75.0.0836135566859.issue43084@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

curses.window.enclose() is the only curses wrapper which returns 0/1 while the original function is declared returning bool. https://linux.die.net/man/3/wenclose

----------
components: Extension Modules
messages: 386021
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: curses.window.enclose() should return bool
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43084>
_______________________________________

From report at bugs.python.org  Sun Jan 31 11:14:39 2021
From: report at bugs.python.org (Siyavash)
Date: Sun, 31 Jan 2021 16:14:39 +0000
Subject: [New-bugs-announce] [issue43085] Loosening | and |= operator type
 checking restriction
Message-ID: <1612109679.82.0.693593534177.issue43085@roundup.psfhosted.org>


New submission from Siyavash <Varzeghani.Siyavash at gmail.com>:

| and |= operators work only if the operands are of 'dict' type. This means that they can't be used with some of the user defined mapping objects.

The case that especially caught my attention was 'SignalDict' type in decimal standard library. While 'SignalDict' has got an update method like 'dict', it can not be updated with |= operator. The following code results in a TypeError:

decimal.getcontext().traps |= {decimal.DivisionByZero:False}

----------
components: Regular Expressions
messages: 386028
nosy: Siy_V, brandtbucher, ezio.melotti, mrabarnett, steven.daprano
priority: normal
severity: normal
status: open
title: Loosening | and |= operator type checking restriction
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43085>
_______________________________________

From report at bugs.python.org  Sun Jan 31 13:11:52 2021
From: report at bugs.python.org (Idan Moral)
Date: Sun, 31 Jan 2021 18:11:52 +0000
Subject: [New-bugs-announce] [issue43086] Excess data in not handled
 properly in binascii.a2b_base64()
Message-ID: <1612116712.99.0.535653486799.issue43086@roundup.psfhosted.org>


New submission from Idan Moral <idan22moral at gmail.com>:

Currently, when providing binascii.a2b_base64() base-64 input with excess data after the padding ('='/'=='), the excess data is ignored.

Example:

import binascii
binascii.a2b_base64(b'aGVsbG8=')       # b'hello' (valid)
binascii.a2b_base64(b'aGVsbG8==')      # b'hello' (ignoring data)
binascii.a2b_base64(b'aGVsbG8=python') # b'hello' (ignoring data)


Note: MANY libraries (such as the all-time favorite `base64`) use this function as their decoder.


Why is it problematic:
* User input can contain additional data after base64 data, which can lead to unintended behavior in products.
* Well-crafted user input can be used to bypass conditions in code (example in the referenced tweet).
* Can be used to target vulnerable libraries and bypass authentication mechanism such as JWT (potentially).


The logic behind my fix PR on GitHub:
* Before deciding to finish the function (after knowing the fact that we passed the data padding),
  we should check if there's no more data after the padding.
* If excess data exists, we should raise an error, free the allocated writer, and return null.
* Else, everything's fine, and we can proceed to the function's end as previously.


Though not publicly disclosed, this behavior can lead to security issues in heavily-used projects.
Preventing this behavior sounds more beneficial than harmful, since there's no known good usage for this behavior.

>From what I read, the python implementation in not so close (when speaking about this case of course) to the base64 RFC.
(link: https://tools.ietf.org/html/rfc4648#section-3.3)


Thanks to Ori Damari (twitter: https://twitter.com/0xrepnz) for bringing this behavior up,
and thanks to Ryan Mast (twitter: https://twitter.com/rmast), and many of the other great guys for discussing the problem in the comments.

Link to the tweet: https://twitter.com/0xrepnz/status/1355295649915404291

--------------------------

Idan Moral
Twitter: https://twitter.com/idan_moral
GitHub: https://github.com/idan22moral

----------
components: Library (Lib)
messages: 386032
nosy: idan22moral
priority: normal
severity: normal
status: open
title: Excess data in not handled properly in binascii.a2b_base64()
type: security
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43086>
_______________________________________

From report at bugs.python.org  Sun Jan 31 14:07:19 2021
From: report at bugs.python.org (Mark Tolonen)
Date: Sun, 31 Jan 2021 19:07:19 +0000
Subject: [New-bugs-announce] [issue43087] ctypes documentation error
Message-ID: <1612120039.68.0.861661788705.issue43087@roundup.psfhosted.org>


New submission from Mark Tolonen <metolone at hotmail.com>:

At the following documentation link:

https://docs.python.org/3/library/ctypes.html#incomplete-types

The last example should have byte strings:

>>> c1 = cell()
>>> c1.name = b"foo"
>>> c2 = cell()
>>> c2.name = b"bar"

----------
messages: 386037
nosy: metolone
priority: normal
severity: normal
status: open
title: ctypes documentation error

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43087>
_______________________________________

From report at bugs.python.org  Sun Jan 31 17:33:45 2021
From: report at bugs.python.org (Charles Karney)
Date: Sun, 31 Jan 2021 22:33:45 +0000
Subject: [New-bugs-announce] [issue43088] Regression in math.hypot
Message-ID: <1612132425.22.0.477732752771.issue43088@roundup.psfhosted.org>


New submission from Charles Karney <charles at karney.com>:

It would be nice if math.hypot had the property

If
  0 < y1 < y2
then
  math.hypot(x, y1) <= math.hypot(x, y2)

This is not the case in python3 with

  x  = 0.6102683302836215
  y1 = 0.7906090004346522
  y2 = y1 + 1e-16

python2's implementation of math.hypot is OK with these inputs.

----------
components: Library (Lib)
files: hypot_bug.py
messages: 386043
nosy: cffk
priority: normal
severity: normal
status: open
title: Regression in math.hypot
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49783/hypot_bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43088>
_______________________________________

From report at bugs.python.org  Sun Jan 31 22:01:44 2021
From: report at bugs.python.org (=?utf-8?b?0JHQvtGA0LjRgSDQktC10YDRhdC+0LLRgdC60LjQuQ==?=)
Date: Mon, 01 Feb 2021 03:01:44 +0000
Subject: [New-bugs-announce] [issue43089] The optional argument to
 random.Random() is called "x" not "seed"
Message-ID: <1612148504.79.0.846022332701.issue43089@roundup.psfhosted.org>


New submission from ????? ?????????? <boris.verk at gmail.com>:

The documentation https://docs.python.org/3/library/random.html#random.Random says:

> class random.Random([seed])?

But in reality the parameter is called "x" not "seed" (CPython 3.9):

>>> import random
>>> random.Random(seed=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() got an unexpected keyword argument 'seed'
>>> random.Random(x=0)
<random.Random object at 0x20ddc10>

Same goes for random.SystemRandom

It would be better to rename "x" to "seed" but that's not backwards compatible.

----------
components: Library (Lib)
messages: 386052
nosy: boris
priority: normal
pull_requests: 23224
severity: normal
status: open
title: The optional argument to random.Random() is called "x" not "seed"
type: enhancement
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43089>
_______________________________________