From report at bugs.python.org  Sun Sep  1 06:55:51 2019
From: report at bugs.python.org (Guruprasad Venkataramaiah)
Date: Sun, 01 Sep 2019 10:55:51 +0000
Subject: [New-bugs-announce] [issue37998] re.sub(r'\\', r'\\\\',
 s) reporting MemoryError
Message-ID: <1567335351.74.0.503338420306.issue37998@roundup.psfhosted.org>


New submission from Guruprasad Venkataramaiah <guruprasad.venkataramaiah at broadcom.com>:

On Python 3.5:
s = re.sub(r'\\', r'\\\\', s)
  File "/pyuniti/venvs/guruv/lib/python3.5/re.py", line 182, in sub
    return _compile(pattern, flags).sub(repl, string, count)
MemoryError

On Python 3.7
    s = re.sub(r'\\', r'\\\\', s)  
  File "C:\Python\Python37-32\lib\re.py", line 192, in sub
    return _compile(pattern, flags).sub(repl, string, count)
MemoryError

----------
components: Regular Expressions
messages: 350943
nosy: Guruprasad Venkataramaiah, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: re.sub(r'\\', r'\\\\', s) reporting MemoryError
type: resource usage
versions: Python 3.5, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37998>
_______________________________________

From report at bugs.python.org  Sun Sep  1 07:24:04 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 01 Sep 2019 11:24:04 +0000
Subject: [New-bugs-announce] [issue37999] No longer use implicit convertion
 to int with loss
Message-ID: <1567337044.72.0.119982097156.issue37999@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Use only the __index__() method for implicit conversion to iteger, and not __int__().

This converts deprecation warnings added in issue36048 into TypeError.

----------
components: Interpreter Core
messages: 350947
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: No longer use implicit convertion to int with loss
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37999>
_______________________________________

From report at bugs.python.org  Sun Sep  1 08:27:54 2019
From: report at bugs.python.org (Andrey)
Date: Sun, 01 Sep 2019 12:27:54 +0000
Subject: [New-bugs-announce] [issue38000] importlib can not handle module
 file names with periods
Message-ID: <1567340874.27.0.741402287818.issue38000@roundup.psfhosted.org>


New submission from Andrey <andry at inbox.ru>:

```
import os, sys

def import_module(dir_path, module_name, ref_module_name = None):
  module_file_path = os.path.join(dir_path, module_name).replace('\\', '/')
  if sys.version_info[0] > 3 or sys.version_info[0] == 3 and sys.version_info[1] >= 4:
    import importlib
    import_spec = importlib.util.spec_from_file_location(os.path.splitext(module_name)[0] if ref_module_name is None else ref_module_name, os.path.join(dir_path, module_name).replace('\\', '/'))
    import_module = importlib.util.module_from_spec(import_spec)
    import_spec.loader.exec_module(import_module)
    #globals()[module_name if ref_module_name is None else ref_module_name] = import_module # still does not accept modules with periods without this
  else:
    # back compatability
    import imp
    module_file, module_file_name, module_desc = imp.find_module(os.path.splitext(module_name)[0], [os.path.dirname(module_file_path)])
    globals()[module_name if ref_module_name is None else ref_module_name] = imp.load_module(module_file_path, module_file, module_file_name, module_desc)
```

I am trying to import my modules targeted by file path:

```
import_module(MYTOOLS_ROOT, 'cmdoplib.std.py', 'cmdoplib')
import_module(MYTOOLS_ROOT, 'cmdoplib.yaml.py', 'cmdoplib_yaml')
```

And got error:

```
Traceback (most recent call last):
  File "c:\python\x64\37\lib\site-packages\xonsh\proc.py", line 1411, in run
    r = self.f(self.args, sp_stdin, sp_stdout, sp_stderr, spec, spec.stack)
  File "c:\python\x64\37\lib\site-packages\xonsh\proc.py", line 1204, in proxy_two
    return f(args, stdin)
  File "c:\python\x64\37\lib\site-packages\xonsh\aliases.py", line 575, in source_alias
    builtins.execx(src, "exec", ctx, filename=fpath)
  File "c:\python\x64\37\lib\site-packages\xonsh\built_ins.py", line 1497, in __call__
    return self.obj.__call__(*args, **kwargs)
  File "c:\python\x64\37\lib\site-packages\xonsh\execer.py", line 190, in exec
    return exec(code, glbs, locs)
  File "w:/Work/MyProjects/__scm_solutions/all-in-one/_common/tools/cmdoplib.yaml.xsh", line 11, in <module>
    g_yaml_env = cmdoplib_yaml.YamlEnv()
NameError: name 'cmdoplib_yaml' is not defined
```

If try to uncomment this:

```
globals()[module_name if ref_module_name is None else ref_module_name] = import_module
```

All works fine.
Seems the latest version of the python still can not handle modules with the periods in a file name without constructions from the old and well known `imp` module.

It seems for me as a bug or at least as an incomplete (re)implementation.

----------
components: Interpreter Core
messages: 350950
nosy: andry
priority: normal
severity: normal
status: open
title: importlib can not handle module file names with periods
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38000>
_______________________________________

From report at bugs.python.org  Sun Sep  1 10:25:17 2019
From: report at bugs.python.org (Digin Antony)
Date: Sun, 01 Sep 2019 14:25:17 +0000
Subject: [New-bugs-announce] [issue38001] Unexpected behaviour of 'is'
 operator
Message-ID: <1567347917.93.0.344087123361.issue38001@roundup.psfhosted.org>


New submission from Digin Antony <diginantony777 at gmail.com>:

The 'is' operator behave differently on two sets of values 
please find the attachment below

tested environment 
windows 7
python 3.7.3


Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more information.
>>> a=256
>>> b=256
>>> a is b
True
>>> a=257
>>> b=257
>>> a is b
False

----------
components: Interpreter Core
files: bug.png
messages: 350952
nosy: Digin Antony
priority: normal
severity: normal
status: open
title: Unexpected behaviour of  'is'  operator
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48578/bug.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38001>
_______________________________________

From report at bugs.python.org  Sun Sep  1 16:08:53 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 01 Sep 2019 20:08:53 +0000
Subject: [New-bugs-announce] [issue38002] 'ModifiedInterpreter' object has
 no attribute 'interp'
Message-ID: <1567368533.85.0.101387822612.issue38002@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Here's a new traceback I haven't seen before.  I only see these at the end of a session, so I don't know which steps triggered it.

$ python3.8 -m idlelib.idle tmp_pretty_fact_ast.py
Exception in Tkinter callback
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/tkinter/__init__.py", line 1883, in __call__
    return self.func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/runscript.py", line 173, in _run_module_event
    interp.runcode(code)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/pyshell.py", line 756, in runcode
    self.interp.restart_subprocess()
AttributeError: 'ModifiedInterpreter' object has no attribute 'interp'

----------
assignee: terry.reedy
components: IDLE
messages: 350961
nosy: rhettinger, terry.reedy
priority: normal
severity: normal
status: open
title: 'ModifiedInterpreter' object has no attribute 'interp'
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38002>
_______________________________________

From report at bugs.python.org  Sun Sep  1 17:40:09 2019
From: report at bugs.python.org (Bob Kline)
Date: Sun, 01 Sep 2019 21:40:09 +0000
Subject: [New-bugs-announce] [issue38003] Incorrect "fixing" of isinstance
 tests for basestring
Message-ID: <1567374009.32.0.247747586449.issue38003@roundup.psfhosted.org>


New submission from Bob Kline <bkline at rksystems.com>:

We are attempting to convert a large Python 2 code base. Following the guidance of the official documentation (https://docs.python.org/2/library/functions.html#basestring) we created tests in many, many places that look like this:

if isinstance(value, basestring):
    if not isinstance(value, unicode):
        value = value.decode(encoding)
else:
    some other code

It seems that the 2to3 tool is unaware that replacing basestring with str in such cases will break the software.

Here's an example.

$ 2to3 repro.py
...
--- repro.py	(original)
+++ repro.py	(refactored)
@@ -1,8 +1,8 @@
 from frobnitz import transform

 def foo(value, encoding=None):
-    if isinstance(value, basestring):
-        if not isinstance(value, unicode):
+    if isinstance(value, str):
+        if not isinstance(value, str):
             value = value.decode(encoding or "utf-8")
         return value
     else:

Help me understand how this "fix" results in the correct behavior.

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 350964
nosy: bkline
priority: normal
severity: normal
status: open
title: Incorrect "fixing" of isinstance tests for basestring
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38003>
_______________________________________

From report at bugs.python.org  Mon Sep  2 03:57:04 2019
From: report at bugs.python.org (Inada Naoki)
Date: Mon, 02 Sep 2019 07:57:04 +0000
Subject: [New-bugs-announce] [issue38004] Duplicated sections in changelog
Message-ID: <1567411024.45.0.536092660752.issue38004@roundup.psfhosted.org>


New submission from Inada Naoki <songofacandy at gmail.com>:

See TOC in https://docs.python.org/3.7/whatsnew/changelog.html

Some releases (*) have multiple "Library" and "Security" sections.

(*) e.g. 3.6.0a3, 3.6.0a2, 3.6.0a1, 3.5.3rc1, 3.5.2rc1

----------
assignee: docs at python
components: Documentation
messages: 350971
nosy: docs at python, inada.naoki
priority: normal
severity: normal
status: open
title: Duplicated sections in changelog
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38004>
_______________________________________

From report at bugs.python.org  Mon Sep  2 05:11:14 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 02 Sep 2019 09:11:14 +0000
Subject: [New-bugs-announce] [issue38005] Coercing strings and non-integer
 numbers to interpreter ID
Message-ID: <1567415474.91.0.412541266394.issue38005@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Is it correct that strings, bytes objects, bytearrays, floats, fractions, decimals can be coerced to an interpreter ID?

>>> import _xxsubinterpreters as interpreters
>>> id = interpreters.InterpreterID('10', force=True)
>>> id == 10
True
>>> id == 10.1
True
>>> id == '1_0'
True
>>> id == bytearray(b'10')
True

Should not the constructor accept only ints and int-like objects and the comparison return True only for integer numbers?

There are other bugs here: raising OverflowError on comparison with large numbers and silencing all errors in some circumstances that can lead to returning a wrong result when errors like KeyboardInterrupt, RecursionError, MemoryError are occurred.

----------
messages: 350972
nosy: eric.snow, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Coercing strings and non-integer numbers to interpreter ID

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38005>
_______________________________________

From report at bugs.python.org  Mon Sep  2 05:48:27 2019
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 02 Sep 2019 09:48:27 +0000
Subject: [New-bugs-announce] [issue38006] _PyFunction_Vectorcall() can
 segfault on process exit
Message-ID: <1567417707.23.0.718072799916.issue38006@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

I have a case of a segfault on Fedora 32 (currently rawhide) with Python 3.8b4 and FreeIPA. The ipactl Python helper crashes with a segfault when the Python process exits. The segfault occurs in _PyFunction_Vectorcall() with a weakref related local scope function that has a NULL func_code pointer. _PyFunction_Vectorcall() does not check the code object for NULL and access the variable without check:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7be354f in _PyFunction_Vectorcall (func=<function at remote 0x7fffe6ebfd30>, stack=0x7fffffffdb50, nargsf=1, kwnames=0x0) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/call.c:395
395         Py_ssize_t nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames);
(gdb) bt
#0  0x00007ffff7be354f in _PyFunction_Vectorcall (func=<function at remote 0x7fffe6ebfd30>, stack=0x7fffffffdb50, nargsf=1, kwnames=0x0) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/call.c:395
#1  0x00007ffff7bb16b0 in _PyObject_Vectorcall (kwnames=0x0, nargsf=1, args=0x7fffffffdb50, callable=<function at remote 0x7fffe6ebfd30>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/cpython/abstract.h:127
#2  _PyObject_FastCall (nargs=1, args=0x7fffffffdb50, func=<function at remote 0x7fffe6ebfd30>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/cpython/abstract.h:147
#3  object_vacall (base=<optimized out>, callable=<function at remote 0x7fffe6ebfd30>, vargs=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/call.c:1186
#4  0x00007ffff7bb19e1 in PyObject_CallFunctionObjArgs (callable=callable at entry=<function at remote 0x7fffe6ebfd30>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/call.c:1259
#5  0x00007ffff7c3f7c3 in handle_callback (ref=<optimized out>, callback=<function at remote 0x7fffe6ebfd30>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/weakrefobject.c:877
#6  0x00007ffff7c01bf7 in PyObject_ClearWeakRefs (object=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/weakrefobject.c:922
#7  0x00007fffe905babc in ctypedescr_dealloc () from /usr/lib64/python3.8/site-packages/_cffi_backend.cpython-38-x86_64-linux-gnu.so
#8  0x00007fffe9058825 in cfield_dealloc () from /usr/lib64/python3.8/site-packages/_cffi_backend.cpython-38-x86_64-linux-gnu.so
#9  0x00007ffff7ba4c65 in _Py_DECREF (filename=<synthetic pointer>, lineno=541, op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/dictobject.c:2000
#10 _Py_XDECREF (op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:541
#11 free_keys_object (keys=0x55555609f590) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/dictobject.c:580
#12 dictkeys_decref (dk=0x55555609f590) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/dictobject.c:324
#13 dict_dealloc (mp=0x7fffe6dd5340) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/dictobject.c:1994
#14 0x00007fffe905bb25 in ctypedescr_dealloc () from /usr/lib64/python3.8/site-packages/_cffi_backend.cpython-38-x86_64-linux-gnu.so
#15 0x00007ffff7ba5058 in _Py_DECREF (filename=<optimized out>, lineno=541, op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:478
#16 _Py_XDECREF (op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:541
#17 tupledealloc (op=0x7fffe6e3bbb0) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/tupleobject.c:247
#18 0x00007ffff7ba5058 in _Py_DECREF (filename=<optimized out>, lineno=541, op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:478
#19 _Py_XDECREF (op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:541
#20 tupledealloc (op=0x7fffe6e55b80) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/tupleobject.c:247
#21 0x00007ffff7c16b76 in _Py_DECREF (filename=0x7ffff7d2e6a8 "/builddir/build/BUILD/Python-3.8.0b4/Objects/typeobject.c", lineno=1110, op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:478
#22 clear_slots (self=<optimized out>, type=0x55555559d0d0) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/typeobject.c:1110
#23 subtype_dealloc (self=<KeyedRef at remote 0x7fffe6deb220>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/typeobject.c:1262
#24 0x00007ffff7ba4c65 in _Py_DECREF (filename=<synthetic pointer>, lineno=541, op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/dictobject.c:2000
#25 _Py_XDECREF (op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:541
#26 free_keys_object (keys=0x555556078d30) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/dictobject.c:580
#27 dictkeys_decref (dk=0x555556078d30) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/dictobject.c:324
#28 dict_dealloc (mp=0x7fffe6eba6c0) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/dictobject.c:1994
#29 0x00007ffff7b98a4b in _Py_DECREF (filename=<synthetic pointer>, lineno=541, op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:478
#30 _Py_XDECREF (op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:541
#31 cell_dealloc (op=0x7fffe6ebe2e0) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/cellobject.c:84
#32 0x00007ffff7ba5058 in _Py_DECREF (filename=<optimized out>, lineno=541, op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:478
#33 _Py_XDECREF (op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:541
#34 tupledealloc (op=0x7fffe6ebe310) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/tupleobject.c:247
#35 0x00007ffff7b991c2 in _Py_DECREF (filename=0x7ffff7d2e460 "/builddir/build/BUILD/Python-3.8.0b4/Objects/funcobject.c", lineno=584, op=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Include/object.h:478
#36 func_clear (op=0x7fffe6ebfd30) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Objects/funcobject.c:584
#37 0x00007ffff7ba5f84 in delete_garbage (state=<optimized out>, state=<optimized out>, old=<optimized out>, collectable=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Modules/gcmodule.c:929
#38 collect (generation=2, n_collected=0x0, n_uncollectable=0x0, nofail=1, state=0x7ffff7dd9598 <_PyRuntime+344>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Modules/gcmodule.c:1106
#39 0x00007ffff7cac8c2 in _PyGC_CollectNoFail () at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Modules/gcmodule.c:1848
#40 0x00007ffff7cacb74 in PyImport_Cleanup () at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Python/import.c:541
#41 0x00007ffff7cacf56 in Py_FinalizeEx () at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Python/pylifecycle.c:1226
#42 0x00007ffff7cad088 in Py_Exit (sts=0) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Python/pylifecycle.c:2248
#43 0x00007ffff7cad0cf in handle_system_exit () at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Python/pythonrun.c:658
#44 0x00007ffff7cad219 in _PyErr_PrintEx (set_sys_last_vars=1, tstate=0x55555555bfb0) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Python/pythonrun.c:755
#45 PyErr_PrintEx (set_sys_last_vars=set_sys_last_vars at entry=1) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Python/pythonrun.c:755
#46 0x00007ffff7cad23a in PyErr_Print () at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Python/pythonrun.c:761
#47 0x00007ffff7b91b42 in PyRun_SimpleFileExFlags (fp=<optimized out>, filename=<optimized out>, closeit=<optimized out>, flags=0x7fffffffe178) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Python/pythonrun.c:434
#48 0x00007ffff7cae36f in pymain_run_file (cf=0x7fffffffe178, config=0x55555555b410) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Modules/main.c:383
#49 pymain_run_python (exitcode=0x7fffffffe170) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Modules/main.c:567
#50 Py_RunMain () at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Modules/main.c:646
#51 0x00007ffff7cae559 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/python3-3.8.0~b4-1.fc32.x86_64/Modules/main.c:700
#52 0x00007ffff7e21193 in __libc_start_main () from /lib64/libc.so.6
#53 0x000055555555508e in _start ()

It looks like the func object is the local remove function of weakref.WeakValueDictionary() implementation, https://github.com/python/cpython/blob/353053d9ad08fea0e205e6c008b8a4350c0188e6/Lib/weakref.py#L90-L112

All function object fields except func_qualname and vectorcall are already NULL.

(gdb) print ((PyFunctionObject*)func).func_annotations 
$19 = 0x0
(gdb) print ((PyFunctionObject*)func).func_closure 
$20 = 0x0
(gdb) print ((PyFunctionObject*)func).func_code
$21 = 0x0
(gdb) print ((PyFunctionObject*)func).func_defaults 
$22 = 0x0
(gdb) print ((PyFunctionObject*)func).func_dict
$23 = 0x0
(gdb) print ((PyFunctionObject*)func).func_doc
$24 = 0x0
(gdb) print ((PyFunctionObject*)func).func_globals 
$25 = 0x0
(gdb) print ((PyFunctionObject*)func).func_kwdefaults 
$26 = 0x0
(gdb) print ((PyFunctionObject*)func).func_module 
$27 = 0x0
(gdb) print ((PyFunctionObject*)func).func_name
$28 = 0x0
(gdb) print ((PyFunctionObject*)func).func_qualname 
$29 = 'WeakValueDictionary.__init__.<locals>.remove'
(gdb) print ((PyFunctionObject*)func).func_weakreflist 
$30 = 0x0
(gdb) print ((PyFunctionObject*)func).vectorcall 
$31 = (vectorcallfunc) 0x7ffff7be3530 <_PyFunction_Vectorcall>
(gdb) print func.ob_refcnt 
$32 = 135
(gdb) print func.ob_type 
$33 = (struct _typeobject *) 0x7ffff7dd4e80 <PyFunction_Type>

The Fedora downstream bug is https://bugzilla.redhat.com/show_bug.cgi?id=1747901

----------
components: Interpreter Core
keywords: 3.8regression
messages: 350974
nosy: Mark.Shannon, christian.heimes, lukasz.langa, petr.viktorin, vstinner
priority: release blocker
severity: normal
stage: needs patch
status: open
title: _PyFunction_Vectorcall() can segfault on process exit
type: crash
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38006>
_______________________________________

From report at bugs.python.org  Mon Sep  2 06:23:47 2019
From: report at bugs.python.org (Leslie)
Date: Mon, 02 Sep 2019 10:23:47 +0000
Subject: [New-bugs-announce] [issue38007] Regression: name of
 PyType_Spec::slots conflicts with Qt macro
Message-ID: <1567419827.68.0.665093966998.issue38007@roundup.psfhosted.org>


New submission from Leslie <laszlo.nemeth.pub at gmail.com>:

In Feb 20, 2006, in commit https://github.com/python/cpython/commit/c255c7bef7621596869f56d887ac3ccd5b536708 the bug https://bugs.python.org/issue1086854 had been fixed.

In Dec 3, 2010, in commit https://github.com/python/cpython/commit/4d0d471a8031de90a2b1ce99c4ac4780e60b3bc9 basically the same issue has been introduced again, because it added a "slots" member to the added PyType_Spec struct.

The proposed solution is the same like in the earlier commit, which is explained in https://bugs.python.org/msg23759 :
protect PyType_Spec struct members with prefix:
ts_name, ts_doc, ts_basicsize, ts_itemsize, ts_flags, ts_slots

----------
messages: 350977
nosy: Leslie
priority: normal
severity: normal
status: open
title: Regression: name of PyType_Spec::slots conflicts with Qt macro
type: compile error
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38007>
_______________________________________

From report at bugs.python.org  Mon Sep  2 07:44:10 2019
From: report at bugs.python.org (Divij Rajkumar)
Date: Mon, 02 Sep 2019 11:44:10 +0000
Subject: [New-bugs-announce] [issue38008] ContextManager and
 AsyncContextManager protocols can't be subclassed
Message-ID: <1567424650.87.0.55127247234.issue38008@roundup.psfhosted.org>


Change by Divij Rajkumar <drajkuma1 at gmail.com>:


----------
nosy: divijrajkumar
priority: normal
severity: normal
status: open
title: ContextManager and AsyncContextManager protocols can't be subclassed
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38008>
_______________________________________

From report at bugs.python.org  Mon Sep  2 09:56:33 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Mon, 02 Sep 2019 13:56:33 +0000
Subject: [New-bugs-announce] [issue38009] Handle weakreference callbacks
 invoked indirectly in the middle of a gc collection
Message-ID: <1567432593.55.0.481148483159.issue38009@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

A weak reference may try to invoke a callback object that is being
cleaned (tp_clear) by the garbage collector and it may be in an
inconsistent state. As the garbage collector explicitly does not
invoke callbacks that are part of the same cycle isolate as the
weak reference (pretending that the weak reference was destroyed first),
we should act in the same way when invoking callbacks using Objects/weakrefobject.c:handle_callback.

For example, consider the following scenario:

- F is a function.
- An object O is in a cycle with F.
- O has a weak reference with F as a callback.

When running the garbage collector, is possible to end in Objects/weakrefobject.c:handle_callback if the tp_clear of F decrements the references of O, invoking the weak
reference callback that will try to call F, which is in an incosistent
state as is in the middle of its tp_clear and some internal fields may
be NULL.

----------
components: Interpreter Core
messages: 351001
nosy: pablogsal, pitrou, vstinner
priority: normal
severity: normal
status: open
title: Handle weakreference callbacks invoked indirectly in the middle of a gc collection
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38009>
_______________________________________

From report at bugs.python.org  Mon Sep  2 10:19:46 2019
From: report at bugs.python.org (Jason R. Coombs)
Date: Mon, 02 Sep 2019 14:19:46 +0000
Subject: [New-bugs-announce] [issue38010] Synchronize importlib.metadata
 with importlib_metadata 0.20
Message-ID: <1567433986.41.0.642198993364.issue38010@roundup.psfhosted.org>


New submission from Jason R. Coombs <jaraco at jaraco.com>:

Importlib_metadata 0.20 addresses two issues (https://importlib-metadata.readthedocs.io/en/latest/changelog%20(links).html#id1). Sync those changes here.

----------
components: Library (Lib)
messages: 351004
nosy: jaraco
priority: normal
severity: normal
status: open
title: Synchronize importlib.metadata with importlib_metadata 0.20
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38010>
_______________________________________

From report at bugs.python.org  Mon Sep  2 12:25:02 2019
From: report at bugs.python.org (Noam Sturmwind)
Date: Mon, 02 Sep 2019 16:25:02 +0000
Subject: [New-bugs-announce] [issue38011] xml.dom.pulldom splits text data
 at buffer size when parsing from file
Message-ID: <1567441502.07.0.479687799441.issue38011@roundup.psfhosted.org>


New submission from Noam Sturmwind <noam at sturmwind.ca>:

Python 3.7.4

When parsing a file using xml.dom.pulldom.parse(), if the parser is in the middle of text data when default_bufsize is reached it will split the text into multiple DOM Text nodes.

This breaks code expecting that reads the text data using node.firstChild.data.

----------
components: Library (Lib)
files: test.py
messages: 351021
nosy: nsturmwind
priority: normal
severity: normal
status: open
title: xml.dom.pulldom splits text data at buffer size when parsing from file
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48581/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38011>
_______________________________________

From report at bugs.python.org  Mon Sep  2 13:51:03 2019
From: report at bugs.python.org (Pavan Kumar Uppalanchu)
Date: Mon, 02 Sep 2019 17:51:03 +0000
Subject: [New-bugs-announce] [issue38012] Python Fuction min is case sentive
 ?
Message-ID: <1567446663.85.0.249805930926.issue38012@roundup.psfhosted.org>


New submission from Pavan Kumar Uppalanchu <uppalanchupavankumar45 at gmail.com>:

I have printed min('Infinity') and min('infinity') . 
Both are printing different characters. 
In first case , It's printing 'I' and in second case, it's printing 
'F'

----------
messages: 351027
nosy: uppalanchupavankumar45 at gmail.com
priority: normal
severity: normal
status: open
title: Python Fuction min is case sentive ?
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38012>
_______________________________________

From report at bugs.python.org  Mon Sep  2 20:13:43 2019
From: report at bugs.python.org (Michael Yoo)
Date: Tue, 03 Sep 2019 00:13:43 +0000
Subject: [New-bugs-announce] [issue38013] AsyncGenerator breaks when not
 iterated fully with RuntimeError("can't send non-None value to a
 just-started coroutine")
Message-ID: <1567469623.0.0.944759957345.issue38013@roundup.psfhosted.org>


New submission from Michael Yoo <michael at yoo.id.au>:

Version: Python 3.7.3

When I run this code:

import asyncio


class TestAsyncGenerator:
    def __init__(self):
        pass

    async def aiter(self):
        yield 1
        yield 2


async def main():
    gen = TestAsyncGenerator()
    async for number in gen.aiter():
        break


asyncio.run(main())

# unhandled exception during asyncio.run() shutdown
# task: <Task finished coro=<<async_generator_athrow without __name__>()> exception=RuntimeError("can't send non-None value to a just-started coroutine")>
# RuntimeError: can't send non-None value to a just-started coroutine


There is a warning message that I don't expect to see. I would expect breaking from an async iteration to behave as if breaking from a normal iteration - that is, no problems.

However, I see the warning message shown above. Am I missing something? Otherwise, I believe this is a bug. Thanks!

----------
components: asyncio
messages: 351044
nosy: Michael Yoo, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: AsyncGenerator breaks when not iterated fully with RuntimeError("can't send non-None value to a just-started coroutine")
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38013>
_______________________________________

From report at bugs.python.org  Mon Sep  2 22:33:07 2019
From: report at bugs.python.org (Daniel)
Date: Tue, 03 Sep 2019 02:33:07 +0000
Subject: [New-bugs-announce] [issue38014] Python 3.7 does not compile
Message-ID: <1567477987.66.0.578307493025.issue38014@roundup.psfhosted.org>


New submission from Daniel <dmalendez at googlemail.com>:

Hi, I use macOS Catalina 10.15. I use Xcode 10 and Xcode 11, but with that configuration, compiling Python 3.7.14 does not work for me. Any idea whats causing this issue?

user at Users-MacBook-Pro Python-3.7.4 % ./configure --with-framework-name=python36 --with-universal-archs=64-bit --enable-universalsdk=/ --enable-optimizations --enable-framework=/Users/user/Desktop/Python-3.7 
checking build system type... x86_64-apple-darwin19.0.0
checking host system type... x86_64-apple-darwin19.0.0
checking for python3.7... python3.7
checking for --enable-universalsdk... /
checking for --with-universal-archs... 64-bit
checking MACHDEP... checking for --without-gcc... no
checking for --with-icc... no
Configured with: --prefix=/Applications/Xcode-beta.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for a sed that does not truncate output... /usr/bin/sed
checking for --with-cxx-main=<compiler>... no
checking for g++... no
configure:

  By default, distutils will build C++ extension modules with "g++".
  If this is not intended, then set CXX on the configure command line.
  
checking for the platform triplet based on compiler characteristics... darwin
checking for -Wl,--no-as-needed... no
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for the Android API level... not Android
checking for --with-suffix... 
checking for case-insensitive build directory... yes
checking LIBRARY... libpython$(VERSION)$(ABIFLAGS).a
checking LINKCC... $(PURIFY) $(MAINCC)
checking for GNU ld... no
checking for --enable-shared... no
checking for --enable-profiling... no
checking LDLIBRARY... $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)
checking for ar... ar
checking for readelf... no
checking for a BSD-compatible install... /usr/bin/install -c
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for --with-pydebug... no
checking for --with-assertions... no
checking for --enable-optimizations... yes
checking for --with-lto... no
checking for -llvm-profdata... no
configure: llvm-profdata found via xcrun: /usr/bin/xcrun llvm-profdata
checking for -Wextra... yes
checking whether gcc accepts and needs -fno-strict-aliasing... no
checking if we can turn off gcc unused result warning... yes
checking if we can turn off gcc unused parameter warning... yes
checking if we can turn off gcc missing field initializers warning... yes
checking if we can turn off gcc invalid function cast warning... no
checking if we can turn on gcc mixed sign comparison warning... yes
checking if we can turn on gcc unreachable code warning... yes
checking if we can turn on gcc strict-prototypes warning... yes
checking if we can make implicit function declaration an error in gcc... yes
checking which compiler should be used... gcc
checking which MACOSX_DEPLOYMENT_TARGET to use... 10.15
checking whether pthreads are available without options... no
checking whether gcc accepts -Kpthread... no
checking whether gcc accepts -Kthread... no
checking whether gcc accepts -pthread... no
checking whether g++ also accepts flags for thread support... no
checking for ANSI C header files... (cached) yes
checking asm/types.h usability... no
checking asm/types.h presence... no
checking for asm/types.h... no
checking crypt.h usability... no
checking crypt.h presence... no
checking for crypt.h... no
checking conio.h usability... no
checking conio.h presence... no
checking for conio.h... no
checking direct.h usability... no
checking direct.h presence... no
checking for direct.h... no
checking dlfcn.h usability... no
checking dlfcn.h presence... yes
configure: WARNING: dlfcn.h: present but cannot be compiled
configure: WARNING: dlfcn.h:     check for missing prerequisite headers?
configure: WARNING: dlfcn.h: see the Autoconf documentation
configure: WARNING: dlfcn.h:     section "Present But Cannot Be Compiled"
configure: WARNING: dlfcn.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for dlfcn.h... no
checking errno.h usability... no
checking errno.h presence... yes
configure: WARNING: errno.h: present but cannot be compiled
configure: WARNING: errno.h:     check for missing prerequisite headers?
configure: WARNING: errno.h: see the Autoconf documentation
configure: WARNING: errno.h:     section "Present But Cannot Be Compiled"
configure: WARNING: errno.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for errno.h... no
checking fcntl.h usability... no
checking fcntl.h presence... yes
configure: WARNING: fcntl.h: present but cannot be compiled
configure: WARNING: fcntl.h:     check for missing prerequisite headers?
configure: WARNING: fcntl.h: see the Autoconf documentation
configure: WARNING: fcntl.h:     section "Present But Cannot Be Compiled"
configure: WARNING: fcntl.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for fcntl.h... no
checking grp.h usability... no
checking grp.h presence... yes
configure: WARNING: grp.h: present but cannot be compiled
configure: WARNING: grp.h:     check for missing prerequisite headers?
configure: WARNING: grp.h: see the Autoconf documentation
configure: WARNING: grp.h:     section "Present But Cannot Be Compiled"
configure: WARNING: grp.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for grp.h... no
checking ieeefp.h usability... no
checking ieeefp.h presence... no
checking for ieeefp.h... no
checking io.h usability... no
checking io.h presence... no
checking for io.h... no
checking langinfo.h usability... no
checking langinfo.h presence... yes
configure: WARNING: langinfo.h: present but cannot be compiled
configure: WARNING: langinfo.h:     check for missing prerequisite headers?
configure: WARNING: langinfo.h: see the Autoconf documentation
configure: WARNING: langinfo.h:     section "Present But Cannot Be Compiled"
configure: WARNING: langinfo.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for langinfo.h... no
checking libintl.h usability... no
checking libintl.h presence... no
checking for libintl.h... no
checking process.h usability... no
checking process.h presence... no
checking for process.h... no
checking pthread.h usability... no
checking pthread.h presence... yes
configure: WARNING: pthread.h: present but cannot be compiled
configure: WARNING: pthread.h:     check for missing prerequisite headers?
configure: WARNING: pthread.h: see the Autoconf documentation
configure: WARNING: pthread.h:     section "Present But Cannot Be Compiled"
configure: WARNING: pthread.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for pthread.h... no
checking sched.h usability... no
checking sched.h presence... yes
configure: WARNING: sched.h: present but cannot be compiled
configure: WARNING: sched.h:     check for missing prerequisite headers?
configure: WARNING: sched.h: see the Autoconf documentation
configure: WARNING: sched.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sched.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sched.h... no
checking shadow.h usability... no
checking shadow.h presence... no
checking for shadow.h... no
checking signal.h usability... no
checking signal.h presence... yes
configure: WARNING: signal.h: present but cannot be compiled
configure: WARNING: signal.h:     check for missing prerequisite headers?
configure: WARNING: signal.h: see the Autoconf documentation
configure: WARNING: signal.h:     section "Present But Cannot Be Compiled"
configure: WARNING: signal.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for signal.h... no
checking stropts.h usability... no
checking stropts.h presence... no
checking for stropts.h... no
checking termios.h usability... no
checking termios.h presence... yes
configure: WARNING: termios.h: present but cannot be compiled
configure: WARNING: termios.h:     check for missing prerequisite headers?
configure: WARNING: termios.h: see the Autoconf documentation
configure: WARNING: termios.h:     section "Present But Cannot Be Compiled"
configure: WARNING: termios.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for termios.h... no
checking for unistd.h... (cached) yes
checking utime.h usability... no
checking utime.h presence... yes
configure: WARNING: utime.h: present but cannot be compiled
configure: WARNING: utime.h:     check for missing prerequisite headers?
configure: WARNING: utime.h: see the Autoconf documentation
configure: WARNING: utime.h:     section "Present But Cannot Be Compiled"
configure: WARNING: utime.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for utime.h... no
checking poll.h usability... no
checking poll.h presence... yes
configure: WARNING: poll.h: present but cannot be compiled
configure: WARNING: poll.h:     check for missing prerequisite headers?
configure: WARNING: poll.h: see the Autoconf documentation
configure: WARNING: poll.h:     section "Present But Cannot Be Compiled"
configure: WARNING: poll.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for poll.h... no
checking sys/devpoll.h usability... no
checking sys/devpoll.h presence... no
checking for sys/devpoll.h... no
checking sys/epoll.h usability... no
checking sys/epoll.h presence... no
checking for sys/epoll.h... no
checking sys/poll.h usability... no
checking sys/poll.h presence... yes
configure: WARNING: sys/poll.h: present but cannot be compiled
configure: WARNING: sys/poll.h:     check for missing prerequisite headers?
configure: WARNING: sys/poll.h: see the Autoconf documentation
configure: WARNING: sys/poll.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/poll.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/poll.h... no
checking sys/audioio.h usability... no
checking sys/audioio.h presence... no
checking for sys/audioio.h... no
checking sys/xattr.h usability... no
checking sys/xattr.h presence... yes
configure: WARNING: sys/xattr.h: present but cannot be compiled
configure: WARNING: sys/xattr.h:     check for missing prerequisite headers?
configure: WARNING: sys/xattr.h: see the Autoconf documentation
configure: WARNING: sys/xattr.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/xattr.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/xattr.h... no
checking sys/bsdtty.h usability... no
checking sys/bsdtty.h presence... no
checking for sys/bsdtty.h... no
checking sys/event.h usability... no
checking sys/event.h presence... yes
configure: WARNING: sys/event.h: present but cannot be compiled
configure: WARNING: sys/event.h:     check for missing prerequisite headers?
configure: WARNING: sys/event.h: see the Autoconf documentation
configure: WARNING: sys/event.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/event.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/event.h... no
checking sys/file.h usability... no
checking sys/file.h presence... yes
configure: WARNING: sys/file.h: present but cannot be compiled
configure: WARNING: sys/file.h:     check for missing prerequisite headers?
configure: WARNING: sys/file.h: see the Autoconf documentation
configure: WARNING: sys/file.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/file.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/file.h... no
checking sys/ioctl.h usability... no
checking sys/ioctl.h presence... yes
configure: WARNING: sys/ioctl.h: present but cannot be compiled
configure: WARNING: sys/ioctl.h:     check for missing prerequisite headers?
configure: WARNING: sys/ioctl.h: see the Autoconf documentation
configure: WARNING: sys/ioctl.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/ioctl.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/ioctl.h... no
checking sys/kern_control.h usability... no
checking sys/kern_control.h presence... yes
configure: WARNING: sys/kern_control.h: present but cannot be compiled
configure: WARNING: sys/kern_control.h:     check for missing prerequisite headers?
configure: WARNING: sys/kern_control.h: see the Autoconf documentation
configure: WARNING: sys/kern_control.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/kern_control.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/kern_control.h... no
checking sys/loadavg.h usability... no
checking sys/loadavg.h presence... no
checking for sys/loadavg.h... no
checking sys/lock.h usability... no
checking sys/lock.h presence... yes
configure: WARNING: sys/lock.h: present but cannot be compiled
configure: WARNING: sys/lock.h:     check for missing prerequisite headers?
configure: WARNING: sys/lock.h: see the Autoconf documentation
configure: WARNING: sys/lock.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/lock.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/lock.h... no
checking sys/mkdev.h usability... no
checking sys/mkdev.h presence... no
checking for sys/mkdev.h... no
checking sys/modem.h usability... no
checking sys/modem.h presence... no
checking for sys/modem.h... no
checking sys/param.h usability... no
checking sys/param.h presence... yes
configure: WARNING: sys/param.h: present but cannot be compiled
configure: WARNING: sys/param.h:     check for missing prerequisite headers?
configure: WARNING: sys/param.h: see the Autoconf documentation
configure: WARNING: sys/param.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/param.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/param.h... no
checking sys/random.h usability... no
checking sys/random.h presence... yes
configure: WARNING: sys/random.h: present but cannot be compiled
configure: WARNING: sys/random.h:     check for missing prerequisite headers?
configure: WARNING: sys/random.h: see the Autoconf documentation
configure: WARNING: sys/random.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/random.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/random.h... no
checking sys/select.h usability... no
checking sys/select.h presence... yes
configure: WARNING: sys/select.h: present but cannot be compiled
configure: WARNING: sys/select.h:     check for missing prerequisite headers?
configure: WARNING: sys/select.h: see the Autoconf documentation
configure: WARNING: sys/select.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/select.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/select.h... no
checking sys/sendfile.h usability... no
checking sys/sendfile.h presence... no
checking for sys/sendfile.h... no
checking sys/socket.h usability... no
checking sys/socket.h presence... yes
configure: WARNING: sys/socket.h: present but cannot be compiled
configure: WARNING: sys/socket.h:     check for missing prerequisite headers?
configure: WARNING: sys/socket.h: see the Autoconf documentation
configure: WARNING: sys/socket.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/socket.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/socket.h... no
checking sys/statvfs.h usability... no
checking sys/statvfs.h presence... yes
configure: WARNING: sys/statvfs.h: present but cannot be compiled
configure: WARNING: sys/statvfs.h:     check for missing prerequisite headers?
configure: WARNING: sys/statvfs.h: see the Autoconf documentation
configure: WARNING: sys/statvfs.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/statvfs.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/statvfs.h... no
checking for sys/stat.h... (cached) yes
checking sys/syscall.h usability... no
checking sys/syscall.h presence... yes
configure: WARNING: sys/syscall.h: present but cannot be compiled
configure: WARNING: sys/syscall.h:     check for missing prerequisite headers?
configure: WARNING: sys/syscall.h: see the Autoconf documentation
configure: WARNING: sys/syscall.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/syscall.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/syscall.h... no
checking sys/sys_domain.h usability... no
checking sys/sys_domain.h presence... yes
configure: WARNING: sys/sys_domain.h: present but cannot be compiled
configure: WARNING: sys/sys_domain.h:     check for missing prerequisite headers?
configure: WARNING: sys/sys_domain.h: see the Autoconf documentation
configure: WARNING: sys/sys_domain.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/sys_domain.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/sys_domain.h... no
checking sys/termio.h usability... no
checking sys/termio.h presence... no
checking for sys/termio.h... no
checking sys/time.h usability... no
checking sys/time.h presence... yes
configure: WARNING: sys/time.h: present but cannot be compiled
configure: WARNING: sys/time.h:     check for missing prerequisite headers?
configure: WARNING: sys/time.h: see the Autoconf documentation
configure: WARNING: sys/time.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/time.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/time.h... no
checking sys/times.h usability... no
checking sys/times.h presence... yes
configure: WARNING: sys/times.h: present but cannot be compiled
configure: WARNING: sys/times.h:     check for missing prerequisite headers?
configure: WARNING: sys/times.h: see the Autoconf documentation
configure: WARNING: sys/times.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/times.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/times.h... no
checking for sys/types.h... (cached) yes
checking sys/uio.h usability... no
checking sys/uio.h presence... yes
configure: WARNING: sys/uio.h: present but cannot be compiled
configure: WARNING: sys/uio.h:     check for missing prerequisite headers?
configure: WARNING: sys/uio.h: see the Autoconf documentation
configure: WARNING: sys/uio.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/uio.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/uio.h... no
checking sys/un.h usability... no
checking sys/un.h presence... yes
configure: WARNING: sys/un.h: present but cannot be compiled
configure: WARNING: sys/un.h:     check for missing prerequisite headers?
configure: WARNING: sys/un.h: see the Autoconf documentation
configure: WARNING: sys/un.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/un.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/un.h... no
checking sys/utsname.h usability... no
checking sys/utsname.h presence... yes
configure: WARNING: sys/utsname.h: present but cannot be compiled
configure: WARNING: sys/utsname.h:     check for missing prerequisite headers?
configure: WARNING: sys/utsname.h: see the Autoconf documentation
configure: WARNING: sys/utsname.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/utsname.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/utsname.h... no
checking sys/wait.h usability... no
checking sys/wait.h presence... yes
configure: WARNING: sys/wait.h: present but cannot be compiled
configure: WARNING: sys/wait.h:     check for missing prerequisite headers?
configure: WARNING: sys/wait.h: see the Autoconf documentation
configure: WARNING: sys/wait.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/wait.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/wait.h... no
checking pty.h usability... no
checking pty.h presence... no
checking for pty.h... no
checking libutil.h usability... no
checking libutil.h presence... no
checking for libutil.h... no
checking sys/resource.h usability... no
checking sys/resource.h presence... yes
configure: WARNING: sys/resource.h: present but cannot be compiled
configure: WARNING: sys/resource.h:     check for missing prerequisite headers?
configure: WARNING: sys/resource.h: see the Autoconf documentation
configure: WARNING: sys/resource.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/resource.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sys/resource.h... no
checking netpacket/packet.h usability... no
checking netpacket/packet.h presence... no
checking for netpacket/packet.h... no
checking sysexits.h usability... no
checking sysexits.h presence... yes
configure: WARNING: sysexits.h: present but cannot be compiled
configure: WARNING: sysexits.h:     check for missing prerequisite headers?
configure: WARNING: sysexits.h: see the Autoconf documentation
configure: WARNING: sysexits.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sysexits.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for sysexits.h... no
checking bluetooth.h usability... no
checking bluetooth.h presence... no
checking for bluetooth.h... no
checking linux/tipc.h usability... no
checking linux/tipc.h presence... no
checking for linux/tipc.h... no
checking linux/random.h usability... no
checking linux/random.h presence... no
checking for linux/random.h... no
checking spawn.h usability... no
checking spawn.h presence... yes
configure: WARNING: spawn.h: present but cannot be compiled
configure: WARNING: spawn.h:     check for missing prerequisite headers?
configure: WARNING: spawn.h: see the Autoconf documentation
configure: WARNING: spawn.h:     section "Present But Cannot Be Compiled"
configure: WARNING: spawn.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for spawn.h... no
checking util.h usability... no
checking util.h presence... yes
configure: WARNING: util.h: present but cannot be compiled
configure: WARNING: util.h:     check for missing prerequisite headers?
configure: WARNING: util.h: see the Autoconf documentation
configure: WARNING: util.h:     section "Present But Cannot Be Compiled"
configure: WARNING: util.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for util.h... no
checking alloca.h usability... no
checking alloca.h presence... yes
configure: WARNING: alloca.h: present but cannot be compiled
configure: WARNING: alloca.h:     check for missing prerequisite headers?
configure: WARNING: alloca.h: see the Autoconf documentation
configure: WARNING: alloca.h:     section "Present But Cannot Be Compiled"
configure: WARNING: alloca.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for alloca.h... no
checking endian.h usability... no
checking endian.h presence... no
checking for endian.h... no
checking sys/endian.h usability... no
checking sys/endian.h presence... no
checking for sys/endian.h... no
checking sys/sysmacros.h usability... no
checking sys/sysmacros.h presence... no
checking for sys/sysmacros.h... no
checking for dirent.h that defines DIR... no
checking for sys/ndir.h that defines DIR... no
checking for sys/dir.h that defines DIR... no
checking for ndir.h that defines DIR... no
checking for library containing opendir... no
checking whether sys/types.h defines makedev... no
checking for sys/mkdev.h... (cached) no
checking for sys/sysmacros.h... (cached) no
checking bluetooth/bluetooth.h usability... no
checking bluetooth/bluetooth.h presence... no
checking for bluetooth/bluetooth.h... no
checking for net/if.h... no
checking for linux/netlink.h... no
checking for linux/vm_sockets.h... no
checking for linux/can.h... no
checking for linux/can/raw.h... no
checking for linux/can/bcm.h... no
checking for clock_t in time.h... yes
checking for makedev... no
checking for le64toh... no
checking for mode_t... no
checking for off_t... no
checking for pid_t... no
checking for size_t... no
checking for uid_t in sys/types.h... yes
checking for ssize_t... no
checking for __uint128_t... no
checking size of int... 0
checking size of long... 0
checking size of long long... 0
checking size of void *... 0
checking size of short... 0
checking size of float... 0
checking size of double... 0
checking size of fpos_t... 0
checking size of size_t... 0
checking size of pid_t... 0
checking size of uintptr_t... 0
checking for long double support... no
checking size of _Bool... 0
checking size of off_t... 0
checking whether to enable large file support... no
checking size of time_t... 0
checking for pthread_t... no
checking size of pthread_key_t... 0
checking whether pthread_key_t is compatible with int... no
checking for --enable-framework... yes
checking for dyld... always on for Darwin
checking the extension of shared libraries... .so
checking LDSHARED... $(CC) -bundle -undefined dynamic_lookup
checking CCSHARED... 
checking LINKFORSHARED... -Wl,-stack_size,1000000  -framework CoreFoundation $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)
checking CFLAGSFORSHARED... $(CCSHARED)
checking SHLIBS... $(LIBS)
checking for sendfile in -lsendfile... no
checking for dlopen in -ldl... no
checking for shl_load in -ldld... no
checking uuid/uuid.h usability... no
checking uuid/uuid.h presence... yes
configure: WARNING: uuid/uuid.h: present but cannot be compiled
configure: WARNING: uuid/uuid.h:     check for missing prerequisite headers?
configure: WARNING: uuid/uuid.h: see the Autoconf documentation
configure: WARNING: uuid/uuid.h:     section "Present But Cannot Be Compiled"
configure: WARNING: uuid/uuid.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##
checking for uuid/uuid.h... no
checking uuid.h usability... no
checking uuid.h presence... no
checking for uuid.h... no
checking for uuid_generate_time_safe... no
checking for uuid_create... no
checking for uuid_enc_be... no
checking for library containing sem_init... no
checking for textdomain in -lintl... no
checking aligned memory access is required... yes
checking for --with-hash-algorithm... default
checking for --with-address-sanitizer... no
checking for --with-memory-sanitizer... no
checking for --with-undefined-behavior-sanitizer... no
checking for t_open in -lnsl... no
checking for socket in -lsocket... no
checking for --with-libs... no
checking for pkg-config... no
checking for --with-system-expat... no
checking for --with-system-ffi... no
checking for --with-system-libmpdec... no
checking for --enable-loadable-sqlite-extensions... no
checking for --with-tcltk-includes... default
checking for --with-tcltk-libs... default
checking for --with-dbmliborder... 
checking for _POSIX_THREADS in unistd.h... yes
checking for pthread_create in -lpthread... checking for pthread_detach... no
checking for pthread_create in -lpthreads... no
checking for pthread_create in -lc_r... no
checking for __pthread_create_system in -lpthread... no
checking for pthread_create in -lcma... no
configure: error: could not find pthreads on your system
user at Users-MacBook-Pro Python-3.7.4 %

----------
components: macOS
messages: 351049
nosy: m_python, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Python 3.7 does not compile
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38014>
_______________________________________

From report at bugs.python.org  Tue Sep  3 00:02:44 2019
From: report at bugs.python.org (Ma Lin)
Date: Tue, 03 Sep 2019 04:02:44 +0000
Subject: [New-bugs-announce] [issue38015] inline function generates slightly
 inefficient machine code
Message-ID: <1567483364.75.0.314519224234.issue38015@roundup.psfhosted.org>


New submission from Ma Lin <malincns at 163.com>:

Commit 5e63ab0 replaces macro with this inline function:

    static inline int
    is_small_int(long long ival)
    {
        return -NSMALLNEGINTS <= ival && ival < NSMALLPOSINTS;
    }

(by default, NSMALLNEGINTS is 5, NSMALLPOSINTS is 257)


However, when invoking this function, and `sizeof(value) < sizeof(long long)`, there is an unnecessary type casting.

For example, on 32-bit platform, if `value` is `Py_ssize_t`, it needs to be converted to 8-byte `long long` type.

The following assembly code is the beginning part of `PyLong_FromSsize_t(Py_ssize_t v)` function.
(32-bit x86 build generated by GCC 9.2, with `-m32 -O2` option)

Use macro before commit 5e63ab0:
        mov     eax, DWORD PTR [esp+4]
        add     eax, 5
        cmp     eax, 261
        ja      .L2
        sal     eax, 4
        add     eax, OFFSET FLAT:small_ints
        add     DWORD PTR [eax], 1
        ret
.L2:    jmp     PyLong_FromSsize_t_rest(int)

Use inlined function:
        push    ebx
        mov     eax, DWORD PTR [esp+8]
        mov     edx, 261
        mov     ecx, eax
        mov     ebx, eax
        sar     ebx, 31
        add     ecx, 5
        adc     ebx, 0
        cmp     edx, ecx
        mov     edx, 0
        sbb     edx, ebx
        jc      .L7
        cwde
        sal     eax, 4
        add     eax, OFFSET FLAT:small_ints+80
        add     DWORD PTR [eax], 1
        pop     ebx
        ret
.L7:    pop     ebx
        jmp     PyLong_FromSsize_t_rest(int)

On 32-bit x86 platform, 8-byte `long long` is implemented in using two registers, so the machine code is much longer than macro version.

At least these hot functions are suffered from this:
  PyObject* PyLong_FromSsize_t(Py_ssize_t v)
  PyObject* PyLong_FromLong(long v)

Replacing the inline function with a macro version will fix this:
#define IS_SMALL_INT(ival) (-NSMALLNEGINTS <= (ival) && (ival) < NSMALLPOSINTS)

If you want to see assembly code generated by major compilers, you can paste attached file demo.c to https://godbolt.org/
- demo.c was original written by Greg Price.
- use `-m32 -O2` to generate 32-bit build.

----------
components: Interpreter Core
files: demo.c
messages: 351052
nosy: Greg Price, Ma Lin, aeros167, mark.dickinson, rhettinger, sir-sigurd
priority: normal
severity: normal
status: open
title: inline function generates slightly inefficient machine code
versions: Python 3.9
Added file: https://bugs.python.org/file48583/demo.c

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38015>
_______________________________________

From report at bugs.python.org  Tue Sep  3 02:37:52 2019
From: report at bugs.python.org (Mallinath Akkalkot)
Date: Tue, 03 Sep 2019 06:37:52 +0000
Subject: [New-bugs-announce] [issue38016] Results with re.sub and re.replace
 is giving different results
Message-ID: <1567492672.11.0.939939262697.issue38016@roundup.psfhosted.org>


New submission from Mallinath Akkalkot <mallinathva at gmail.com>:

Issue is explained in the attached presentation for the below python script.

# PYTHON script
import re

stringPassed = "START, 0.272342208623734, 0.122712611838329\nCIRCL, 0.2739, 0.1175, 0.2644, 0.1175\nCIRCL, 0.2644, 0.108, 0.2644, 0.1175\nLINE, 0.26237716818855, 0.108\nCIRCL, 0.2564, 0.102522934456486, 0.26237716818855, 0.102\nLINE, 0.255041919210401, 0.087\nLINE, 0.255041919210401, 0.072\nCIRCL, 0.239541747114243, -0.0106446715786942, 0.0269671265080016, 0.0720000000000296\nCIRCL, 0.2391, -0.0130000000000008, 0.2456, -0.0130000000000008\nLINE, 0.2391, -0.0350000000000008\nCIRCL, 0.243376868486329, -0.0411080018232587, 0.2456, -0.0350000000000008\nLINE, 0.24865951460066, -0.0430307277670375\nCIRCL, 0.255041919210401, -0.0521457461886608, 0.245341919210401, -0.0521457461886608\nLINE, 0.255041919210401, -0.087\nLINE, 0.2564, -0.102522934456486\nCIRCL, 0.26237716818855, -0.108, 0.26237716818855, -0.102\nLINE, 0.2644, -0.108\nCIRCL, 0.2739, -0.1175, 0.2644, -0.1175\nCIRCL, 0.272342208623734, -0.122712611838329, 0.2644, -0.1175\n"

retVal = re.findall(r"(\w+,\s+)(-\d+\.\d+)|(\d+\.\d+)", stringPassed, re.MULTILINE)

floatNumbersList = []		# List conatins all Folat numbers
for each in retVal:
	for each in each:
		if each != "":floatNumbersList.append(each)

newStringUpdated = stringPassed
for eachVal in floatNumbersList:
	newVal =  str(float(eachVal)*1000)
	retVals = re.sub(eachVal, newVal, newStringUpdated)
#	retVals = newStringUpdated.replace(eachVal, newVal)
	newStringUpdated = retVals

print(newStringUpdated)

----------
components: Regular Expressions
files: Bug_Description_Regular Expression.pptx
messages: 351066
nosy: Mallinath Akkalkot, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Results with re.sub and re.replace is giving different results
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48584/Bug_Description_Regular Expression.pptx

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38016>
_______________________________________

From report at bugs.python.org  Tue Sep  3 08:48:26 2019
From: report at bugs.python.org (DannyNiu)
Date: Tue, 03 Sep 2019 12:48:26 +0000
Subject: [New-bugs-announce] [issue38017] IO-like interface for SHAKE-* and
 other XOFs.
Message-ID: <1567514906.04.0.888108599586.issue38017@roundup.psfhosted.org>


New submission from DannyNiu <dannyniu at hotmail.com>:

The SHAKE family functions are the first formally defined XOFs (eXtensible Output Functions), previous implementations such as MGF from PKCS#1 relies on ad-hoc construction. 

In the current library, SHAKE-* are given digest-like interface, where their output has to be always obtained from the start; where as in reality, there exists applications that consumes such output progressively, such as deterministic key generation from fixed seed, and more specifically, as the random oracle used in Crystals-Dilithium post-quantum digital signature scheme. 

Therefore, I'd like developers consider adding ``shake.read(len)'' function to support IO-like consumption in hashlib.

----------
components: Library (Lib)
messages: 351084
nosy: dannyniu
priority: normal
severity: normal
status: open
title: IO-like interface for SHAKE-* and other XOFs.
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38017>
_______________________________________

From report at bugs.python.org  Tue Sep  3 09:58:05 2019
From: report at bugs.python.org (Vinay Sharma)
Date: Tue, 03 Sep 2019 13:58:05 +0000
Subject: [New-bugs-announce] [issue38018] Increase Code Coverage for
 multiprocessing.shared_memory
Message-ID: <1567519085.54.0.887939969587.issue38018@roundup.psfhosted.org>


New submission from Vinay Sharma <vinay0410sharma at gmail.com>:

Increase Code coverage for multiprocessing.shared_memory.SharedMemory Class

----------
components: Tests
messages: 351085
nosy: vinay0410
priority: normal
severity: normal
status: open
title: Increase Code Coverage for multiprocessing.shared_memory
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38018>
_______________________________________

From report at bugs.python.org  Tue Sep  3 12:55:09 2019
From: report at bugs.python.org (Marat Sharafutdinov)
Date: Tue, 03 Sep 2019 16:55:09 +0000
Subject: [New-bugs-announce] [issue38019] AttributeError: 'NoneType' object
 has no attribute '_add_reader' / '_remove_reader'
Message-ID: <1567529709.4.0.805713285268.issue38019@roundup.psfhosted.org>


New submission from Marat Sharafutdinov <decaz89 at gmail.com>:

I'm trying to use dwdiff (https://os.ghalkes.nl/dwdiff.html) to compare two text files. While running subprocess I'm getting the following errors:

```
Traceback (most recent call last):
  File "test_dwdiff.py", line 17, in <module>
    asyncio.run(main())
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
    return future.result()
  File "test_dwdiff.py", line 14, in main
    await asyncio.gather(*(dwdiff() for __ in range(25)))
  File "test_dwdiff.py", line 10, in dwdiff
    await process.communicate()
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/subprocess.py", line 187, in communicate
    loop=self._loop)
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/subprocess.py", line 166, in _read_stream
    output = await stream.read()
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/streams.py", line 633, in read
    block = await self.read(self._limit)
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/streams.py", line 646, in read
    self._maybe_resume_transport()
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/streams.py", line 417, in _maybe_resume_transport
    self._transport.resume_reading()
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/unix_events.py", line 498, in resume_reading
    self._loop._add_reader(self._fileno, self._read_ready)
AttributeError: 'NoneType' object has no attribute '_add_reader'
```

and

```
Exception in callback SubprocessStreamProtocol.pipe_data_received(1, b'\xd0\x9f\xd...b8\xd1\x8f.\n')
handle: <Handle SubprocessStreamProtocol.pipe_data_received(1, b'\xd0\x9f\xd...b8\xd1\x8f.\n')>
Traceback (most recent call last):
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/subprocess.py", line 71, in pipe_data_received
    reader.feed_data(data)
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/streams.py", line 440, in feed_data
    self._transport.pause_reading()
  File "/home/decaz/.pyenv/versions/3.7.4/lib/python3.7/asyncio/unix_events.py", line 495, in pause_reading
    self._loop._remove_reader(self._fileno)
AttributeError: 'NoneType' object has no attribute '_remove_reader'
```

The code is the following:
```
import asyncio

async def dwdiff():
    process = await asyncio.create_subprocess_exec(
        '/usr/local/bin/dwdiff', 'f1.txt', 'f2.txt',
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    await process.communicate()

async def main():
    await asyncio.gather(*(dwdiff() for __ in range(25)))

asyncio.run(main())
```

Two text files f1.txt and f2.txt are attached as archive.

Perhaps it relates to https://bugs.python.org/issue29704

----------
components: asyncio
files: f1_and_f2.zip
messages: 351090
nosy: asvetlov, decaz, yselivanov
priority: normal
severity: normal
status: open
title: AttributeError: 'NoneType' object has no attribute '_add_reader' / '_remove_reader'
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48589/f1_and_f2.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38019>
_______________________________________

From report at bugs.python.org  Tue Sep  3 14:28:55 2019
From: report at bugs.python.org (Steve Dower)
Date: Tue, 03 Sep 2019 18:28:55 +0000
Subject: [New-bugs-announce] [issue38020] os.path.realpath crashes in
 Windows Store package
Message-ID: <1567535335.44.0.142880989456.issue38020@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

When installed via the Store, "python3.8 -c 'import sysconfig'" crashes hard.

So far, I've discovered it's inside an os.path.realpath() call, and appears to be a refcounting issue (double DECREF I think), but haven't gotten any further yet.

I'm looking into it.

----------
assignee: steve.dower
components: Windows
keywords: 3.8regression
messages: 351094
nosy: lukasz.langa, paul.moore, steve.dower, tim.golden, zach.ware
priority: release blocker
severity: normal
stage: test needed
status: open
title: os.path.realpath crashes in Windows Store package
type: crash
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38020>
_______________________________________

From report at bugs.python.org  Tue Sep  3 15:02:48 2019
From: report at bugs.python.org (Michael Felt)
Date: Tue, 03 Sep 2019 19:02:48 +0000
Subject: [New-bugs-announce] [issue38021] pep425 tag for AIX is inadequate
Message-ID: <1567537368.97.0.869605154196.issue38021@roundup.psfhosted.org>


New submission from Michael Felt <aixtools at felt.demon.nl>:

PEP425 stats the platform tag is what distutils.util.get_platform() (and sysconfig.get_platform()) returns.

By that definition - anything is okay, as long as something is returned.

However, in practice, it is insufficient. Simplest case - there is no indication of the fitness of the running Python. This, by itself, may be a reason that VARs and others have never attempted to develop binary (compatible) eggs or wheels.

Looking further at missed potential - AIX has an algorithm that can be used to permit a forward (binary) compatible ABI for packaging. The current state of packaging for AIX is to create an RPM or INSTALLP formatted package - and install that. Even pure-python modules must be packaged in this way - to satisfy the "binary" dependencies.

For a binary compatible algorithm to be possible changes will be needed in pypa tools. However, rather than "patch" three sets of tools to correct the inadequacy of the current PEP425 tag for AIX I propose to tag AIX as:
AIX-VRTL-YYWW-BT or
AIX.{}{}{:02d}.{}.{}".format(v,r,int(tl),bd,sz)

where v=version, r=release, tl=technology_level, bd=builddate, and sz=fitness (32|64).

Further, rather than place this code "inline" in distutils or sysconfig create a new support module for AIX - similar to the _osx_support module (I.e. _aix_support).

----------
components: Distutils, Library (Lib)
messages: 351096
nosy: Michael.Felt, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: pep425 tag for AIX is inadequate
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38021>
_______________________________________

From report at bugs.python.org  Tue Sep  3 15:03:42 2019
From: report at bugs.python.org (Tal Einat)
Date: Tue, 03 Sep 2019 19:03:42 +0000
Subject: [New-bugs-announce] [issue38022] IDLE: Upgrade help.html to sphinx 2
Message-ID: <1567537422.32.0.254523116379.issue38022@roundup.psfhosted.org>


New submission from Tal Einat <taleinat at gmail.com>:

The rest of our docs use version 2.x of Sphinx with its new HTML5 output writer.  However, IDLE's help.html is a version of idle.rst rendered with the old writer, via the `-D html4_writer=1` command line parameter.

The reason for this is that the new output is slightly different, and causes some regression in how IDLE's HTML help renderer displays the help information.

I propose to upgrade the IDLE HTML help renderer to properly handle the new output.

----------
assignee: terry.reedy
components: IDLE
messages: 351097
nosy: taleinat, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE: Upgrade help.html to sphinx 2
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38022>
_______________________________________

From report at bugs.python.org  Tue Sep  3 20:18:43 2019
From: report at bugs.python.org (almenon anon)
Date: Wed, 04 Sep 2019 00:18:43 +0000
Subject: [New-bugs-announce] [issue38023] Deal better with StackOverflow
 exception when debugging
Message-ID: <1567556323.3.0.435761006518.issue38023@roundup.psfhosted.org>


New submission from almenon anon <almenon214 at gmail.com>:

Code that runs fine in the command line can crash in the debugger

Note that because https://bugs.python.org/issue10933 is not fixed yet I'm assuming this applies to all python 3 versions but it was confirmed in python 3.6

See https://github.com/microsoft/ptvsd/issues/1379:

@fabioz says it better than I could:

> This is really a CPython bug, not ptvsd (CPython is crashing because of a stack overflow when the stack overflow is thrown inside a tracing function -- because the user code already recursed too much).

> As a note, the debugger is even disabled on the face of a stack overflow (see: https://bugs.python.org/issue10933), so, even if it didn't crash, things wouldn't work very well from that point onwards...

> Maybe we could try to detect that a stack overflow is about to happen and notify about it right before it happens in the debugger so that users could know that the debugger would stop (note that this is actually pretty hard to do from the debugger without killing performance because we don't have a stack in the debugger as we try to run with frames untraced whenever possible and I don't think there's a python API that provides the size of the stack in a fast way -- but maybe I'm wrong, I haven't investigated much).

> As a note, the ideal place for that would probably be in CPython itself (when it got to that condition it could raise the recursion limit a bit and call a handler or at least give some warning if configured to do so as it can be really puzzling to know that the tracing got disabled because of a swallowed StackOverflow).

> So, I'm going to rename this issue to Deal better with StackOverflow in CPython so that it reflects better the issue at hand, but it's not currently high-priority -- that's really a bug in CPython itself and is probably better fixed there (the workaround in user code -- in this case foxdot -- is not throwing a StackOverflow).

----------
components: Interpreter Core
messages: 351107
nosy: almenon anon
priority: normal
severity: normal
status: open
title: Deal better with StackOverflow exception when debugging
type: crash
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38023>
_______________________________________

From report at bugs.python.org  Wed Sep  4 03:41:52 2019
From: report at bugs.python.org (QA.Nathaniel Logarta)
Date: Wed, 04 Sep 2019 07:41:52 +0000
Subject: [New-bugs-announce] [issue38024] adding or subtracting decimals
Message-ID: <1567582912.8.0.129413108747.issue38024@roundup.psfhosted.org>


New submission from QA.Nathaniel Logarta <nathaniel.logarta at neworientalclub88.com>:

Just try to add decimals
It doesn't have any trigger on my end
As long as it is a complex decimal or adding decimal zeroes to non-zero

----------
components: Windows
files: adding or subtracting decimals.PNG
messages: 351117
nosy: QA.Nathaniel Logarta, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: adding or subtracting decimals
versions: Python 3.7
Added file: https://bugs.python.org/file48590/adding or subtracting decimals.PNG

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38024>
_______________________________________

From report at bugs.python.org  Wed Sep  4 07:05:51 2019
From: report at bugs.python.org (Daniel Futterweit)
Date: Wed, 04 Sep 2019 11:05:51 +0000
Subject: [New-bugs-announce] [issue38025] format
Message-ID: <1567595151.82.0.58620854901.issue38025@roundup.psfhosted.org>


Change by Daniel Futterweit <dfutterweit at gmail.com>:


----------
nosy: Daniel_Python
priority: normal
severity: normal
status: open
title: format
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38025>
_______________________________________

From report at bugs.python.org  Wed Sep  4 07:27:21 2019
From: report at bugs.python.org (Inada Naoki)
Date: Wed, 04 Sep 2019 11:27:21 +0000
Subject: [New-bugs-announce] [issue38026] inspect.getattr_static should
 avoid dynamic lookup
Message-ID: <1567596441.56.0.650171643141.issue38026@roundup.psfhosted.org>


New submission from Inada Naoki <songofacandy at gmail.com>:

Reported here: https://github.com/python/cpython/commit/feaefc7f60cd3be7bf4ecc2b73e77d2bfe048403

I merged GH-5351 but it broke inspect.getattr_static behavior.  It should be reverted.

----------
components: Library (Lib)
keywords: 3.8regression
messages: 351121
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: inspect.getattr_static should avoid dynamic lookup
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38026>
_______________________________________

From report at bugs.python.org  Wed Sep  4 08:45:53 2019
From: report at bugs.python.org (Ana)
Date: Wed, 04 Sep 2019 12:45:53 +0000
Subject: [New-bugs-announce] [issue38027] Can't use Numpy, SciPy,
 pandas in Python3.8.0b4 without conda
Message-ID: <1567601153.07.0.326922242017.issue38027@roundup.psfhosted.org>


New submission from Ana <annalischensnow at gmail.com>:

So, all in all, if I try to install Numpy, SciPy, pandas or any related libraries via pip I see several issues, no AIX version written (somewhere?) for Python 3.8, as well as pip addresses to mkl_rt, which is not found in the path.

I am trying to use python3.8 as a system interpreter and pip version: pip 19.2.3 I don't want to use conda. I am trying to test it using my system. What can I do?

----------
components: macOS
messages: 351123
nosy: annelischen, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Can't use Numpy, SciPy, pandas in Python3.8.0b4 without conda
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38027>
_______________________________________

From report at bugs.python.org  Wed Sep  4 12:06:08 2019
From: report at bugs.python.org (Ana)
Date: Wed, 04 Sep 2019 16:06:08 +0000
Subject: [New-bugs-announce] [issue38028] Assertion error in Python 3.8
Message-ID: <1567613168.85.0.729930445227.issue38028@roundup.psfhosted.org>


New submission from Ana <annalischensnow at gmail.com>:

After all, I am trying to install the dev version of Numpy and the other libs. Unfortunately, I am still unable to install them. I have some assertion errors:

ERROR: Exception:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 188, in main
    status = self.run(options, args)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 398, in run
    installed = install_given_reqs(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 54, in install_given_reqs
    requirement.install(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 925, in install
    self.move_wheel_files(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 453, in move_wheel_files
    wheel.move_wheel_files(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/wheel.py", line 324, in move_wheel_files
    scheme = distutils_scheme(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/locations.py", line 104, in distutils_scheme
    assert not (home and prefix), "home={} prefix={}".format(home, prefix)
AssertionError: home=/private/tmp/pip-target-4p5evqc_ prefix=/private/tmp/pip-build-env-b27ru_1v/overlay

That's the first time I see such an issue and googling doesn't resolve it at all

----------
components: macOS
messages: 351138
nosy: annelischen, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Assertion error in Python 3.8
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38028>
_______________________________________

From report at bugs.python.org  Wed Sep  4 12:09:55 2019
From: report at bugs.python.org (Brian Skinn)
Date: Wed, 04 Sep 2019 16:09:55 +0000
Subject: [New-bugs-announce] [issue38029] Should io.TextIOWrapper raise an
 error at instantiation if a StringIO is passed as 'buffer'?
Message-ID: <1567613395.14.0.527188435842.issue38029@roundup.psfhosted.org>


New submission from Brian Skinn <bskinn at alum.mit.edu>:

If I read the docs correctly, io.TextIOWrapper is meant to provide a str-typed interface to an underlying bytes stream.

If a TextIOWrapper is instantiated with the underlying buffer=io.StringIO(), it breaks:

>>> import io
>>> tw = io.TextIOWrapper(io.StringIO())
>>> tw.write(b'abcd\n')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: write() argument must be str, not bytes
>>> tw.write('abcd\n')
5
>>> tw.read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: string argument expected, got 'bytes'
>>> tw.read(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: underlying read() should have returned a bytes-like object, not 'str'



Would it be better for TextIOWrapper to fail earlier, at instantiation-time, for this kind of (unrecoverably?) broken type mismatch?

----------
components: Library (Lib)
messages: 351139
nosy: bskinn
priority: normal
severity: normal
status: open
title: Should io.TextIOWrapper raise an error at instantiation if a StringIO is passed as 'buffer'?
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38029>
_______________________________________

From report at bugs.python.org  Wed Sep  4 16:26:32 2019
From: report at bugs.python.org (Eryk Sun)
Date: Wed, 04 Sep 2019 20:26:32 +0000
Subject: [New-bugs-announce] [issue38030] os.stat fails for block devices
 such as //./PhysicalDrive0
Message-ID: <1567628792.17.0.732081344869.issue38030@roundup.psfhosted.org>


New submission from Eryk Sun <eryksun at gmail.com>:

In issue 37834, I posted a suggest implementation of win32_xstat_impl that included the following snippet of code:

        if (!GetFileInformationByHandle(hFile, &fileInfo)) {
            error = GetLastError();
            if (error != ERROR_INVALID_PARAMETER &&
                error != ERROR_INVALID_FUNCTION &&
                error != ERROR_NOT_SUPPORTED) {
                retval = -1;
                goto cleanup;
            }
            /* Volumes and physical disks are block devices, e.g.
               \\.\C: and \\.\PhysicalDrive0. */
            memset(result, 0, sizeof(*result));
            result->st_mode = 0x6000; /* S_IFBLK */
            goto cleanup;
        }

This is meant to handle the above errors. We know we have FILE_TYPE_DISK. If GetFileInformationByHandle fails with one of the above error codes, then we must have a disk or volume device without a mounted file-system device. This should be reported as a block device.

However it was changed in the final version as follows:

        if (!GetFileInformationByHandle(hFile, &fileInfo)) {
            switch (GetLastError()) {
            case ERROR_INVALID_PARAMETER:
            case ERROR_INVALID_FUNCTION:
            case ERROR_NOT_SUPPORTED:
                retval = -1;
                goto cleanup;
            }
            /* Volumes and physical disks are block devices, e.g.
               \\.\C: and \\.\PhysicalDrive0. */
            memset(result, 0, sizeof(*result));
            result->st_mode = 0x6000; /* S_IFBLK */
            goto cleanup;
        }

Now it's failing on the errors that should be handled, and succeeding on all other errors that should fail. If we want to use a switch statement here, I suggest the following:

        if (!GetFileInformationByHandle(hFile, &fileInfo)) {
            switch (GetLastError()) {
            case ERROR_INVALID_PARAMETER:
            case ERROR_INVALID_FUNCTION:
            case ERROR_NOT_SUPPORTED:
                /* Volumes and physical disks are block devices, e.g.
                   \\.\C: and \\.\PhysicalDrive0. */
                memset(result, 0, sizeof(*result));
                result->st_mode = 0x6000; /* S_IFBLK */
                goto cleanup;
            }
            retval = -1;
            goto cleanup;
        }

----------
components: IO, Windows
messages: 351149
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: high
severity: normal
stage: needs patch
status: open
title: os.stat fails for block devices such as //./PhysicalDrive0
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38030>
_______________________________________

From report at bugs.python.org  Wed Sep  4 17:19:41 2019
From: report at bugs.python.org (Maxwell Bernstein)
Date: Wed, 04 Sep 2019 21:19:41 +0000
Subject: [New-bugs-announce] [issue38031] FileIO.__init__ aborts when opener
 returns bad fd
Message-ID: <1567631981.81.0.541424946013.issue38031@roundup.psfhosted.org>


New submission from Maxwell Bernstein <tekk.nolagi at gmail.com>:

On a debug build, the following causes an abort:

import _io
_io.FileIO("foobar", opener=lambda name, flags: 1000000)

1000000 is not a valid fd. FileIO attempts to raise an IOError from errno, but there is already an exception set when PyErr_SetFromErrno uses PyObject_Call to create the exception.

----------
components: IO
messages: 351150
nosy: tekknolagi
priority: normal
severity: normal
status: open
title: FileIO.__init__ aborts when opener returns bad fd
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38031>
_______________________________________

From report at bugs.python.org  Wed Sep  4 19:21:42 2019
From: report at bugs.python.org (Justin ARthur)
Date: Wed, 04 Sep 2019 23:21:42 +0000
Subject: [New-bugs-announce] [issue38032] lib2to3 doesn't parse Python 3
 identifiers containing non-spacing marks
Message-ID: <1567639302.72.0.55214965787.issue38032@roundup.psfhosted.org>


New submission from Justin ARthur <justinarthur at gmail.com>:

Python 3 code with an identifier that has a non-spacing mark in it does not get tokenized by lib2to3 and will result in an exception thrown in the parsing process.

Parsing the attached file (badvar.py), results in `ParseError: bad token: type=58, value='?', context=('', (1, 1))`

This happens because the Name pattern regular expression in lib2to3 is `r'\w+'` and the word character class doesn't contain non-spacing marks (and possible other [continuation characters allowed in Python 3 identifiers](https://docs.python.org/3/reference/lexical_analysis.html#identifiers)).

(reported by energizer in the Python IRC channel)

----------
components: 2to3 (2.x to 3.x conversion tool), Library (Lib)
files: badvar.py
messages: 351153
nosy: JustinTArthur
priority: normal
severity: normal
status: open
title: lib2to3 doesn't parse Python 3 identifiers containing non-spacing marks
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48592/badvar.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38032>
_______________________________________

From report at bugs.python.org  Wed Sep  4 19:27:04 2019
From: report at bugs.python.org (Chiaki Ishikawa)
Date: Wed, 04 Sep 2019 23:27:04 +0000
Subject: [New-bugs-announce] [issue38033] Use After Free: PyObject_Free
Message-ID: <1567639624.78.0.948902616905.issue38033@roundup.psfhosted.org>


New submission from Chiaki Ishikawa <ishikawa at yk.rim.or.jp>:

Hi, 
I am new to Python bug tracker, so my setting of the fields may be inadequate. If so, apologies in advance.
I think Use After Free bug is a potential security issue and so wanted to report ASAP.

First my environment: I am using Debian GNU/Linux, and its kernel version is: uname -a output, 
SMP Debian 4.19.37-6 (2019-07-18) x86_64 GNU/Linux

Python version is:
# python3 --version
Python 3.7.4

mozilla thunderbird mail client testing framework uses
python as a test driver, and when I was checking the local build of thunderbird under a test suite invoked from |make mozmill| under valgrind (a memory usage checker), the first thing I noticed is the following message from valgrind.
PyObject_Free seems to access a memory location (4 octets) in an already freed block. This happened many times during the test.

I have not bothered to look into the source code of python, but a seasoned developer should be able to figure out where such reference is made.

>From valgrind log:

==30354== Invalid read of size 4
==30354==    at 0x5A29FE: PyObject_Free (in /usr/bin/python3.7)
==30354==    by 0x5B7337: ??? (in /usr/bin/python3.7)
==30354==    by 0x5BBBFF: PyDict_SetItem (in /usr/bin/python3.7)
==30354==    by 0x58DE19: PyType_Ready (in /usr/bin/python3.7)
==30354==    by 0x6482A0: _Py_ReadyTypes (in /usr/bin/python3.7)
==30354==    by 0x63551A: _Py_InitializeCore_impl (in /usr/bin/python3.7)
==30354==    by 0x6357AA: _Py_InitializeCore (in /usr/bin/python3.7)
==30354==    by 0x5E17EC: ??? (in /usr/bin/python3.7)
==30354==    by 0x653D88: ??? (in /usr/bin/python3.7)
==30354==    by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7)
==30354==    by 0x4ACB09A: (below main) (libc-start.c:308)
==30354==  Address 0x4c8b020 is 16 bytes after a block of size 576 free'd
==30354==    at 0x4833FC0: free (vg_replace_malloc.c:538)
==30354==    by 0x5B7337: ??? (in /usr/bin/python3.7)
==30354==    by 0x5BBBFF: PyDict_SetItem (in /usr/bin/python3.7)
==30354==    by 0x58DE19: PyType_Ready (in /usr/bin/python3.7)
==30354==    by 0x6482A0: _Py_ReadyTypes (in /usr/bin/python3.7)
==30354==    by 0x63551A: _Py_InitializeCore_impl (in /usr/bin/python3.7)
==30354==    by 0x6357AA: _Py_InitializeCore (in /usr/bin/python3.7)
==30354==    by 0x5E17EC: ??? (in /usr/bin/python3.7)
==30354==    by 0x653D88: ??? (in /usr/bin/python3.7)
==30354==    by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7)
==30354==    by 0x4ACB09A: (below main) (libc-start.c:308)
==30354==  Block was alloc'd at
==30354==    at 0x4832E13: malloc (vg_replace_malloc.c:307)
==30354==    by 0x5A4B16: PyObject_Malloc (in /usr/bin/python3.7)
==30354==    by 0x5B72BD: ??? (in /usr/bin/python3.7)
==30354==    by 0x5BBBFF: PyDict_SetItem (in /usr/bin/python3.7)
==30354==    by 0x58DE19: PyType_Ready (in /usr/bin/python3.7)
==30354==    by 0x6482A0: _Py_ReadyTypes (in /usr/bin/python3.7)
==30354==    by 0x63551A: _Py_InitializeCore_impl (in /usr/bin/python3.7)
==30354==    by 0x6357AA: _Py_InitializeCore (in /usr/bin/python3.7)
==30354==    by 0x5E17EC: ??? (in /usr/bin/python3.7)
==30354==    by 0x653D88: ??? (in /usr/bin/python3.7)
==30354==    by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7)
==30354==    by 0x4ACB09A: (below main) (libc-start.c:308)
==30354==


TIA

----------
components: Library (Lib)
messages: 351154
nosy: zephyrus00jp
priority: normal
severity: normal
status: open
title: Use After Free: PyObject_Free
type: security
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38033>
_______________________________________

From report at bugs.python.org  Wed Sep  4 21:23:02 2019
From: report at bugs.python.org (wuck)
Date: Thu, 05 Sep 2019 01:23:02 +0000
Subject: [New-bugs-announce] [issue38034] Typo on
 logging.handlers.QueueListener documentation
Message-ID: <1567646582.17.0.242396034373.issue38034@roundup.psfhosted.org>


New submission from wuck <wirelessduck at gmail.com>:

There appears to be a typo on the documentation page for logging.handlers.QueueListener at https://docs.python.org/3/library/logging.handlers.html#logging.handlers.QueueListener

The line "Changed in version 3.5: The respect_handler_levels argument was added." should instead refer to "Changed in version 3.5: The respect_handler_level argument was added.", removing the 's' at the end of respect_handler_level.

----------
assignee: docs at python
components: Documentation
messages: 351161
nosy: docs at python, wuck
priority: normal
severity: normal
status: open
title: Typo on logging.handlers.QueueListener documentation
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38034>
_______________________________________

From report at bugs.python.org  Thu Sep  5 01:01:08 2019
From: report at bugs.python.org (Vinay Sharma)
Date: Thu, 05 Sep 2019 05:01:08 +0000
Subject: [New-bugs-announce] [issue38035] shared_semaphores cannot be shared
 across unrelated processes
Message-ID: <1567659668.43.0.186142682202.issue38035@roundup.psfhosted.org>


New submission from Vinay Sharma <vinay0410sharma at gmail.com>:

Currently, shared semaphores can only be created, and existing semaphores can't be opened. Shared semaphores are opened using the following command.
```
sem_open(name, O_CREAT | O_EXCL, 0600, val)
```
This will raise error if a semaphore which already exists.
This behaviour works well when the file descriptors of these semaphores can be shared with children processes.

But, it doesn't work when an unrelated process which needs access to shared semaphore tries to open it.

----------
components: Library (Lib)
messages: 351176
nosy: vinay0410
priority: normal
severity: normal
status: open
title: shared_semaphores cannot be shared across unrelated processes
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38035>
_______________________________________

From report at bugs.python.org  Thu Sep  5 03:31:45 2019
From: report at bugs.python.org (Nathaniel Smith)
Date: Thu, 05 Sep 2019 07:31:45 +0000
Subject: [New-bugs-announce] [issue38036] ssl docs say that ssl.SSLContext()
 is secure-by-default since 3.6, but it isn't
Message-ID: <1567668705.95.0.408076289223.issue38036@roundup.psfhosted.org>


New submission from Nathaniel Smith <njs at pobox.com>:

Quoting from the docs for ssl.SSLContext:

"Changed in version 3.6: The context is created with secure default values."
- https://docs.python.org/3/library/ssl.html#ssl.SSLContext

This is not true. If you call ssl.SSLContext(), you get a context with cert validation entirely disabled. And this has led to serious security bugs in practice: https://github.com/theelous3/asks/issues/134

Changing the defaults to make them actually secure would of course be nice, but is a complicated question that would need discussion. In the mean time, the docs shouldn't claim that it's secure. There should be a big bold note saying "UNLESS YOU HAVE A VERY GOOD REASON, DON'T USE THIS, USE ssl.create_default_connection()".

----------
messages: 351186
nosy: alex, christian.heimes, dstufft, janssen, njs
priority: normal
severity: normal
status: open
title: ssl docs say that ssl.SSLContext() is secure-by-default since 3.6, but it isn't

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38036>
_______________________________________

From report at bugs.python.org  Thu Sep  5 07:14:08 2019
From: report at bugs.python.org (Ma Lin)
Date: Thu, 05 Sep 2019 11:14:08 +0000
Subject: [New-bugs-announce] [issue38037] Assertion failed: object has
 negative ref count
Message-ID: <1567682048.52.0.677025547591.issue38037@roundup.psfhosted.org>


New submission from Ma Lin <malincns at 163.com>:

Adding these two lines to /Objects/longobject.c will disable the "preallocated small integer pool":

    #define NSMALLPOSINTS  0
    #define NSMALLNEGINTS  0

Then run this reproduce code (attached):

    from enum import IntEnum
    import _signal

    class Handlers(IntEnum):
        A = _signal.SIG_DFL
        B = _signal.SIG_IGN

When the interpreter exits, will get this error:

    d:\dev\cpython\PCbuild\win32>python_d.exe d:\a.py
    d:\dev\cpython\include\object.h:541: _Py_NegativeRefcount: Assertion failed: object has negative ref count
    <object: freed>
    Fatal Python error: _PyObject_AssertFailed

    Current thread 0x0000200c (most recent call first):

3.8 and 3.9 branches are affected.
I'm sorry, this issue is beyond my ability.

----------
files: reproduce.py
messages: 351196
nosy: Ma Lin
priority: normal
severity: normal
status: open
title: Assertion failed: object has negative ref count
versions: Python 3.8, Python 3.9
Added file: https://bugs.python.org/file48594/reproduce.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38037>
_______________________________________

From report at bugs.python.org  Thu Sep  5 10:19:51 2019
From: report at bugs.python.org (Dong-hee Na)
Date: Thu, 05 Sep 2019 14:19:51 +0000
Subject: [New-bugs-announce] [issue38038] Remove urllib.parse._ from
 xmlrpc.client
Message-ID: <1567693191.9.0.654396049137.issue38038@roundup.psfhosted.org>


New submission from Dong-hee Na <donghee.na92 at gmail.com>:

This issue is the same as bpo-22347.
Let's remove the deprecated urllib.parse._splituser and urllib.parse._splittype from xmlrpc.client

----------
components: Library (Lib)
messages: 351199
nosy: corona10, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Remove urllib.parse._ from xmlrpc.client
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38038>
_______________________________________

From report at bugs.python.org  Thu Sep  5 12:48:36 2019
From: report at bugs.python.org (Ilya Valmianski)
Date: Thu, 05 Sep 2019 16:48:36 +0000
Subject: [New-bugs-announce] [issue38039] Segfault when pickling dictionary
 with large pandas dataframes
Message-ID: <1567702116.93.0.155927302531.issue38039@roundup.psfhosted.org>


New submission from Ilya Valmianski <ivalmian at gmail.com>:

Tried pickling a dictionary with multiple pandas tables and python primitive types. Pandas tables are large so full object size is ~200GB but system should not be OOM (crashed with ~300 GB system memory available). Reproduced on two machines running RHEL 7.5. Tried using Python 3.6 and 3.7. Tried pickle and dill. All python versions were installed as environments via Anaconda. All segfault in pickle. Here is a dump:

Fatal Python error: Segmentation fault

Current thread 0x00007f724af23740 (most recent call first):
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 496 in save
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 805 in _batch_appends
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 781 in save_list
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 476 in save
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 751 in save_tuple
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 476 in save
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 634 in save_reduce
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 521 in save
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 847 in _batch_setitems
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 821 in save_dict
  File "/apps/anaconda3/lib/python3.6/site-packages/dill/_dill.py", line 893 in save_module_dict
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 476 in save
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 634 in save_reduce
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 521 in save
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 847 in _batch_setitems
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 821 in save_dict
  File "/apps/anaconda3/lib/python3.6/site-packages/dill/_dill.py", line 893 in save_module_dict
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 476 in save
  File "/apps/anaconda3/lib/python3.6/pickle.py", line 409 in dump
  File "/apps/anaconda3/lib/python3.6/site-packages/dill/_dill.py", line 286 in dump
  File "precompute_control_patients.py", line 220 in main
  File "/apps/anaconda3/lib/python3.6/site-packages/absl/app.py", line 251 in _run_main
  File "/apps/anaconda3/lib/python3.6/site-packages/absl/app.py", line 300 in run
  File "precompute_control_patients.py", line 227 in <module>

----------
messages: 351213
nosy: Ilya Valmianski
priority: normal
severity: normal
status: open
title: Segfault when pickling dictionary with large pandas dataframes
type: crash
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38039>
_______________________________________

From report at bugs.python.org  Thu Sep  5 14:22:56 2019
From: report at bugs.python.org (Sean Happenny)
Date: Thu, 05 Sep 2019 18:22:56 +0000
Subject: [New-bugs-announce] [issue38040] Typo: "Writeable" Should be
 "Writable" in IO Library Documentation
Message-ID: <1567707776.67.0.386340687421.issue38040@roundup.psfhosted.org>


New submission from Sean Happenny <sean at xnor.ai>:

Problem: There are 4 instances of the typo "writeable" in the documentation for the IO library affecting, at least, versions 3.7, 3.8, 3.9, and the latest master of the documentation (https://docs.python.org/[3.7,3.8,3.9]/library/io.html and https://github.com/python/cpython/blob/master/Doc/library/io.rst).  This can cause confusion to the reader.  The instances are under the "BufferedWriter" section (https://docs.python.org/3/library/io.html#io.BufferedWriter) and "BufferedRWPair" section (https://docs.python.org/3.7/library/io.html#io.BufferedRWPair).

Fix: Change all instances of "writeable" to "writable" in the IO library documentation.

----------
assignee: docs at python
components: Documentation, IO
messages: 351216
nosy: Sean Happenny, docs at python
priority: normal
severity: normal
status: open
title: Typo: "Writeable" Should be "Writable" in IO Library Documentation
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38040>
_______________________________________

From report at bugs.python.org  Thu Sep  5 14:52:14 2019
From: report at bugs.python.org (Terry J. Reedy)
Date: Thu, 05 Sep 2019 18:52:14 +0000
Subject: [New-bugs-announce] [issue38041] IDLE Shell: Redesign minimum
 restart line
Message-ID: <1567709534.52.0.152965325603.issue38041@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

Currently, Shell prints restart lines as 
=...= RESTART filename =...=
where filename is 'Shell' or the full pathname.  The code is 

        tag = 'RESTART: ' + (filename if filename else 'Shell')
        halfbar = ((int(console.width) -len(tag) - 4) // 2) * '='
        console.write("\n{0} {1} {0}".format(halfbar, tag))

The initial '\n' is needed because typically the cursor is sitting at a prompt.  The -4 is from #21192 when I added 'Shell' or filename.  It is -2 for the spaces plus a couple of extra spaces that were there before.  I believe -2 is sufficient but will have to test to be sure.

The number of '='s is the currently the same on both ends.  (Because this is easiest.)  It does not have to be, and would sometimes not be if we exactly filled the console width.  I think we should because it would show the actual non-printable margin and would allow 

The minimum number of '='s is 0.  The problem revealed in
https://stackoverflow.com/questions/57795679/why-is-this-code-printing-an-empty-line-when-run-under-idle
is that if console.width is exactly 10 + len(filename), the remaining trailing space wraps to the next line, making it look like a spurious print(' ') had occurred.

Possible solutions:

0. Do nothing.  I rejected this by opening this issue.  The SO beginner was apparently genuinely confused and might not be the only such person.  With no spec for the restart line, this is not an implementation bug but is a design glitch.  As I hinted above, some current details are the result of adding filenames with minimal change.

1. Delete the ' 's surrounding the tag when there are no '='s.  This looks strange because the '='s visually mark a restart line as much as 'RESTART'.

2. Print a minimum of 1 '=' on each end (by adding them to the format). This could result in a wrap of exactly ' =' or '=', either of which are bad.  If there is any wrap, it should be the filename, so the user can recognize it as such.

3. A hybrid solution with a minimum of 1 '=' on the front and 0 on the end, with no space if no '='.  I intend to try this after extracting the restart line into a testable function and writing tests that now fail.

4. Add a horizontal scrollbar to Shell.  This not happening now, might be made optional, and would still require specification decisions.

----------
assignee: terry.reedy
components: IDLE
messages: 351218
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE Shell: Redesign minimum restart line
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38041>
_______________________________________

From report at bugs.python.org  Thu Sep  5 16:50:01 2019
From: report at bugs.python.org (Alexander Kurakin)
Date: Thu, 05 Sep 2019 20:50:01 +0000
Subject: [New-bugs-announce] [issue38042] Crash on double
 list(lxml.etree.iterparse(f)) Edit
Message-ID: <1567716601.15.0.507352834054.issue38042@roundup.psfhosted.org>


New submission from Alexander Kurakin <kuraga333 at mail.ru>:

I have a crash in Python 3.x environments and don't have in 2.7.

===

Crashes in:

Python 3.7.4, Anaconda, Linux 64bit, lxml 4.3.3
OR
Python 3.7.4, Anaconda, Linux 64bit, lxml 4.4.1
OR
Python 3.6.5, Gentoo Linux 64bit, lxml 4.3.3

test.py:

import lxml
import lxml.etree

with open('test.xml', 'rb') as f:
    list(
        lxml.etree.iterparse(f)
    )

    # Traceback (most recent call last):
    # File "test.py", line 18, in <module>
    # lxml.etree.iterparse(f)
    # File "src/lxml/iterparse.pxi", line 209, in lxml.etree.iterparse.__next__
    # File "src/lxml/iterparse.pxi", line 194, in lxml.etree.iterparse.__next__
    # File "src/lxml/iterparse.pxi", line 225, in lxml.etree.iterparse._read_more_events
    # File "src/lxml/parser.pxi", line 1380, in lxml.etree._FeedParser.close
    # Segmentation fault
    list(
        lxml.etree.iterparse(f)
    )

test.xml:

<?xml version="1.0" encoding="UTF-8"?>
<root></root>

(or any else)

GDB's backtrace see in attachment.

===

Doesn't crash in:

Python 2.7.15, Gentoo Linux 64bit, lxml 4.3.3

# Traceback (most recent call last):
#   File "test.py", line 19, in <module>
#     lxml.etree.iterparse(f)
#   File "src/lxml/iterparse.pxi", line 209, in lxml.etree.iterparse.__next__
#    File "/home/sasha/_lxml-bug/test.xml", line 0
lxml.etree.XMLSyntaxError: no element found

===

See also: https://bugs.launchpad.net/lxml/+bug/1833050

----------
files: bt.txt
messages: 351222
nosy: kuraga
priority: normal
severity: normal
status: open
title: Crash on double list(lxml.etree.iterparse(f)) Edit
type: crash
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file48595/bt.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38042>
_______________________________________

From report at bugs.python.org  Fri Sep  6 00:46:54 2019
From: report at bugs.python.org (Greg Price)
Date: Fri, 06 Sep 2019 04:46:54 +0000
Subject: [New-bugs-announce] [issue38043] small cleanups in Unicode
 normalization code
Message-ID: <1567745214.3.0.712701659381.issue38043@roundup.psfhosted.org>


New submission from Greg Price <gnprice at gmail.com>:

Benjamin noticed in reviewing GH-15558 (for #37966) several points where the existing code around Unicode normalization can be improved:

* on the `QuickcheckResult` enum:
  > Maybe `makeunicodedata.py` should output this enum (with better name namespacing)

* > merging `test_normalization` into this file [i.e. `test_unicodedata.py`] for clarity

* > These "boolean int" parameters could be actual `bool`s. [sc. the `nfc` and `k` parameters to `is_normalized_quickcheck`]

None of these are super hard, so good to knock them out while we're thinking of them.

----------
components: Unicode
messages: 351229
nosy: Greg Price, benjamin.peterson, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: small cleanups in Unicode normalization code
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38043>
_______________________________________

From report at bugs.python.org  Fri Sep  6 06:11:18 2019
From: report at bugs.python.org (Jeremy)
Date: Fri, 06 Sep 2019 10:11:18 +0000
Subject: [New-bugs-announce] [issue38044] unittest causing segfaults with
 string malloc in c++ module
Message-ID: <1567764678.03.0.448906028899.issue38044@roundup.psfhosted.org>


New submission from Jeremy <jeremy.westhead456 at gmail.com>:

If a unittest is written which accesses a module written in C++ (I used Pybind11 to allow Python access) which uses malloc for a string, a segmentation fault is caused. This is not replicated when malloc is used for some other data types such as int, char or char*

C++ code:

#include <string>
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
using namespace std;
void t(){
    string * data = (string*)malloc(100*sizeof(string));
    data[0] = "this is a test";
    cout<<data[0]<<endl;
}
PYBIND11_MODULE(TestModule, m){
    m.def("editPointerString", &t);
}

Once compiled and imported this can be run fine with editPointerString() as expected. No errors occur when running this function from any Python functions or from within a class.


However, if this is run through a unittest such as:


class TestUnit(unittest.TestCase):
    def testStringPointer(self):
        editPointerString()


A segmentation fault occurs before the cout<<data[0]<<endl; is executed. Not having an assert statement does not affect this outcome.


However, running 3 or more tests which call this function allows the tests to run and pass - instead displaying there has been a segmentation fault after the OK message from unittest

This can be fixed by using calloc instead of malloc, so may be caused by processing of data which is not cleared when malloc is used?

Also, malloc can be used in functions in tests after a test running calloc has been run.


System info:
Python 3.7.3
Ubuntu 19.04
pybind11 2.3.0
g++ 8.3.0
C++ 8.3.0

----------
components: Extension Modules, Library (Lib)
messages: 351243
nosy: ymerej
priority: normal
severity: normal
status: open
title: unittest causing segfaults with string malloc in c++ module
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38044>
_______________________________________

From report at bugs.python.org  Fri Sep  6 08:18:00 2019
From: report at bugs.python.org (Antony Lee)
Date: Fri, 06 Sep 2019 12:18:00 +0000
Subject: [New-bugs-announce] [issue38045] Flag instance creation is slow
Message-ID: <1567772280.73.0.782923595007.issue38045@roundup.psfhosted.org>


New submission from Antony Lee <anntzer.lee at gmail.com>:

Consider the following example

    from enum import Flag
    F = Flag("F", list("abcdefghijklm"))
    for idx in range(2**len(F) - 1):
        F(idx)

creating all possible combos of 13 flags, so 8192 instances (yes, I know the instances are cached later, but the initial cost is still significant).  Locally, this takes over 4.5s to execute; profiling shows that ~30% of that time is spent executing enum._is_power_of_two(!):

    def _power_of_two(value):
        if value < 1:
            return False
        return value == 2 ** _high_bit(value)

Indeed, replacing the implementation of _power_of_two by

    import math
    _powers_of_two = {
        2 ** i for i in range(math.ceil(math.log2(sys.maxsize)) + 1)}
    def _power_of_two(value):
        return (value in _powers_of_two if value <= sys.maxsize
                else value == 2 ** _high_bit(value))

shaves off ~30% of the runtime (obviously this won't help for huge (>sys.maxsize) flag values, but these are rarer I would think).

An even better fix, I think, would be for Flag to cache `flags_to_check` (updating it at the same time as `_value2member_map_`) instead of recomputing it each time in _decompose

    flags_to_check = [
            (m, v)
            for v, m in list(flag._value2member_map_.items())
            if m.name is not None or _power_of_two(v)
            ]

as this actually needlessly introduces quadratic complexity when iterating over all possible combos (because the size of _value2member_map_ is growing).  (Also, it's not actually clear to me how one can end up with unnamed power-of-two instances in _value2member_map?)

----------
messages: 351244
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: Flag instance creation is slow
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38045>
_______________________________________

From report at bugs.python.org  Fri Sep  6 14:56:05 2019
From: report at bugs.python.org (guillard)
Date: Fri, 06 Sep 2019 18:56:05 +0000
Subject: [New-bugs-announce] [issue38046] JSON sorting type error
Message-ID: <1567796165.48.0.695777601425.issue38046@roundup.psfhosted.org>


New submission from guillard <yoann.reservations at gmail.com>:

If the dict included both str and int keys, sorted returns this error :
TypeError: '<' not supported between instances of 'int' and 'str'

It is a little bit problematic when used with default library behavior, as jsonify in Flask.

possible solution :
try:
    items = sorted(dct.items())
except(TypeError):
    items = sorted(dct.items(), key=lambda kv: str(kv[0]))

----------
files: json_bug_exemple.py
messages: 351268
nosy: yoann934
priority: normal
severity: normal
status: open
title: JSON sorting type error
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48596/json_bug_exemple.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38046>
_______________________________________

From report at bugs.python.org  Fri Sep  6 22:05:23 2019
From: report at bugs.python.org (Haibo Huang)
Date: Sat, 07 Sep 2019 02:05:23 +0000
Subject: [New-bugs-announce] [issue38047] multiarch headers are added when
 cross compiling
Message-ID: <1567821923.97.0.672604626109.issue38047@roundup.psfhosted.org>


Change by Haibo Huang <haibohuang2010 at gmail.com>:


----------
components: Cross-Build
nosy: Alex.Willmer, hhb
priority: normal
pull_requests: 15375
severity: normal
status: open
title: multiarch headers are added when cross compiling
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38047>
_______________________________________

From report at bugs.python.org  Sat Sep  7 03:03:08 2019
From: report at bugs.python.org (Gaurav Kumar Pandit)
Date: Sat, 07 Sep 2019 07:03:08 +0000
Subject: [New-bugs-announce] [issue38048] unususal behavior
Message-ID: <1567839788.75.0.725833597762.issue38048@roundup.psfhosted.org>


New submission from Gaurav Kumar Pandit <grvkmrpandit at gmail.com>:

when I run the programme the cursur is blinking and is taking many number of inputs but in the source code I have not written any syntax for taking input

----------
components: Windows
files: test.py
messages: 351286
nosy: Gaurav Kumar Pandit, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: unususal behavior
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48597/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38048>
_______________________________________

From report at bugs.python.org  Sat Sep  7 05:31:33 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Sep 2019 09:31:33 +0000
Subject: [New-bugs-announce] [issue38049] Add command-line interface for the
 ast module
Message-ID: <1567848693.22.0.0359513964627.issue38049@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The proposed PR adds a simple command-line interface to the ast module, similar to CLI of the tokenize and dis modules. It reads the content of a specified file (or stdin), parses it with ast.parse() and outputs the tree with ast.dump().

It can be used for learning internals of the Python compiler, in addition to tokenize and dis.

----------
components: Library (Lib)
messages: 351297
nosy: benjamin.peterson, brett.cannon, rhettinger, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Add command-line interface for the ast module
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38049>
_______________________________________

From report at bugs.python.org  Sat Sep  7 06:45:58 2019
From: report at bugs.python.org (Sean Frazier)
Date: Sat, 07 Sep 2019 10:45:58 +0000
Subject: [New-bugs-announce] [issue38050] open('file.txt') path not found
Message-ID: <1567853158.61.0.971448699802.issue38050@roundup.psfhosted.org>


New submission from Sean Frazier <sfrazier8689 at gmail.com>:

I am running a problem in 'Think Python' and was having no issues with:

fin = open('words.txt') 

Then when I was working with the reference file, running a function, my IDLE crashed and is no longer able to locate files using [var = open('file.txt')]

I have tried opening as 'r' and 'w', with no luck.

Also, sometimes I do not get an error, but when I go to read the list, it is empty ''.

----------
components: Windows
messages: 351300
nosy: Sean Frazier, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: open('file.txt') path not found
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38050>
_______________________________________

From report at bugs.python.org  Sat Sep  7 13:30:09 2019
From: report at bugs.python.org (John Daily)
Date: Sat, 07 Sep 2019 17:30:09 +0000
Subject: [New-bugs-announce] [issue38051] time.strftime handling %z/%Z badly
Message-ID: <1567877409.54.0.490954987769.issue38051@roundup.psfhosted.org>


New submission from John Daily <pybugs at yoggi.33mail.com>:

My apologies if this is a duplicate; there are more than a few bug reports on time and strftime, but I wasn't able to locate any reporting this specifically.

I'd be happy to do more in-depth tests with some guidance. I see that this is largely the responsibility of system libraries, but it seems likely that Python is doing something incorrectly when invoking them since the behavior is broken in both Mac and Windows.

On both my personal Mac laptop and work Windows laptop, time.strftime('%z') gives the wrong output. On Windows, '%Z' also fails.

I'm currently at -0400 (EDT) and my best guess is that because the tm_isdst flag is set to false, it's interpreting my Eastern time zone as it would be when we're no longer on DST.

Technically displaying "UTC" is also a bug, since the actual time zone is "GMT". Python 3.6 on my Amazon Linux instance handles everything correctly.


Mac, 3.7.3:

>>> epoch = int(datetime.datetime.now().timestamp())
>>> epoch
1567872682
>>> gmt = time.gmtime(epoch)
>>> gmt.tm_zone
'UTC'
>>> time.strftime('%Z', gmt)
'UTC'
>>> time.strftime('%z', gmt)
'-0500'

Python 3.7.1, Windows 7

>>> gmt.tm_zone
'UTC'
>>> time.strftime('%Z', gmt)
'Eastern Standard Time'
>>> time.strftime('%z', gmt)
'-0500'

Python 3.6.8, Amazon Linux

>>> epoch = int(datetime.datetime.now().timestamp())
>>> gmt = time.gmtime(epoch)
>>> gmt.tm_zone
'GMT'
>>> time.strftime('%Z', gmt)
'GMT'
>>> time.strftime('%z', gmt)
'+0000'

----------
messages: 351306
nosy: macintux
priority: normal
severity: normal
status: open
title: time.strftime handling %z/%Z badly
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38051>
_______________________________________

From report at bugs.python.org  Sat Sep  7 13:50:19 2019
From: report at bugs.python.org (Juan Telleria)
Date: Sat, 07 Sep 2019 17:50:19 +0000
Subject: [New-bugs-announce] [issue38052] Include sspipe Module with Core
 Python
Message-ID: <1567878619.06.0.744538231736.issue38052@roundup.psfhosted.org>


New submission from Juan Telleria <jtelleriar at gmail.com>:

Could sspipe be included as a Core Python module?

https://sspipe.github.io/

https://github.com/sspipe/sspipe

https://pypi.org/project/sspipe/

sspipe allows to use syntax such as:

from sspipe import p, px
import numpy as np
import pandas as pd

(
  np.linspace(0, pi, 100) 
  | p({'x': px, 'y': np.sin(px)}) 
  | p(pd.DataFrame)
  | px[px.x > px.y].head()
  | p(print, "Example 6: pandas and numpy support:\n", px)
)

----------
components: Extension Modules
messages: 351307
nosy: Juan Telleria2
priority: normal
severity: normal
status: open
title: Include sspipe Module with Core Python
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38052>
_______________________________________

From report at bugs.python.org  Sat Sep  7 15:08:18 2019
From: report at bugs.python.org (Jon Janzen)
Date: Sat, 07 Sep 2019 19:08:18 +0000
Subject: [New-bugs-announce] [issue38053] Update plistlib documentation
Message-ID: <1567883298.21.0.0945741912998.issue38053@roundup.psfhosted.org>


New submission from Jon Janzen <jjjonjanzen at gmail.com>:

* Update "Mac OS X" to "Apple" since plists are used more widely than just macOS
* Remove notes about the new API being added in 3.4 since 3.4 is no longer supported
* Re-add the UID class documentation (oops, removed in issue36409)

----------
assignee: docs at python
components: Documentation
messages: 351309
nosy: bigfootjon, docs at python
priority: normal
pull_requests: 15381
severity: normal
status: open
title: Update plistlib documentation
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38053>
_______________________________________

From report at bugs.python.org  Sun Sep  8 06:44:15 2019
From: report at bugs.python.org (Alexander Ulyanov)
Date: Sun, 08 Sep 2019 10:44:15 +0000
Subject: [New-bugs-announce] [issue38054] enhance pdb.set_trace() to run
 when the specified condition is true
Message-ID: <1567939455.37.0.149072874331.issue38054@roundup.psfhosted.org>


New submission from Alexander Ulyanov <alex.ulnv at gmail.com>:

Through out my experience with python debugger, I often found myself creating if-statements around set_trace() instructions. This patch introduces a key-word argument that allows to pass in a boolean or a condition that evaluates to boolean to control whether pdb.set_trace() command needs to be executed.

----------
messages: 351324
nosy: alex-ulnv
priority: normal
severity: normal
status: open
title: enhance pdb.set_trace() to run when the specified condition is true
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38054>
_______________________________________

From report at bugs.python.org  Sun Sep  8 10:06:55 2019
From: report at bugs.python.org (Donny Brown)
Date: Sun, 08 Sep 2019 14:06:55 +0000
Subject: [New-bugs-announce] [issue38055] Starting multiprocessing.Process
 causes FileNotFoundError
Message-ID: <1567951615.68.0.110531280911.issue38055@roundup.psfhosted.org>


New submission from Donny Brown <donny.hikari at gmail.com>:

Starting multiprocessing.Process causes FileNotFoundError in certain case:

```
import multiprocessing as mp
import time

def demo(f):
    print(f)

def main():
    cxt=mp.get_context('spawn')
    f=cxt.Value('i', 0)
    p=cxt.Process(target=demo, args=[f])
    p.start()

    return p

if __name__ == "__main__":
    p=main()
    p.join()
```

In this example, 'f' is supposed to stay in memory since it is referred by p._args. However, starting the process raises FileNotFoundError once there is not explicit reference to the arguments passed to the new process in the original process.

----------
components: Interpreter Core
files: test.py
messages: 351328
nosy: Donny Brown, davin, pitrou
priority: normal
severity: normal
status: open
title: Starting multiprocessing.Process causes FileNotFoundError
versions: Python 3.7
Added file: https://bugs.python.org/file48598/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38055>
_______________________________________

From report at bugs.python.org  Sun Sep  8 10:14:30 2019
From: report at bugs.python.org (Ma Lin)
Date: Sun, 08 Sep 2019 14:14:30 +0000
Subject: [New-bugs-announce] [issue38056] Add examples for common text
 encoding Error Handlers
Message-ID: <1567952070.11.0.225789110598.issue38056@roundup.psfhosted.org>


New submission from Ma Lin <malincns at 163.com>:

Text descriptions about `Error Handlers` are not very friendly to novices.
https://docs.python.org/3/library/codecs.html#error-handlers

For example:

    'xmlcharrefreplace'
    Replace with the appropriate XML character reference (only for encoding).  Implemented in :func:`xmlcharrefreplace_errors`. 

    'backslashreplace'
    Replace with backslashed escape sequences. Implemented in :func:`backslashreplace_errors`.

    'namereplace'
    Replace with ``\N{...}`` escape sequences (only for encoding).  Implemented in :func:`namereplace_errors`.

Novices may not know what these are.
Giving some examples may help the reader to understand more intuitively.
The effect picture is attached.

I picked two characters:
?  https://www.compart.com/en/unicode/U+00DF
? https://www.compart.com/en/unicode/U+266C

----------
assignee: docs at python
components: Documentation
files: effect.png
messages: 351329
nosy: Ma Lin, docs at python
priority: normal
severity: normal
status: open
title: Add examples for common text encoding Error Handlers
versions: Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file48599/effect.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38056>
_______________________________________

From report at bugs.python.org  Sun Sep  8 10:58:23 2019
From: report at bugs.python.org (Adorilson Bezerra)
Date: Sun, 08 Sep 2019 14:58:23 +0000
Subject: [New-bugs-announce] [issue38057] Docs: source code don't can be
 translate
Message-ID: <1567954703.31.0.993707976717.issue38057@roundup.psfhosted.org>


New submission from Adorilson Bezerra <adorilson at gmail.com>:

The source code examples in documentation doesn't can be translated because it doesn't included in po files (and after in transifex).

It would be wonderful if we can to translate the code:

class MyClass:
    """A simple example class"""
    i = 12345

    def f(self):
        return 'hello world'

to:

class MinhaClasse:
    """Um exemplo simples de classe"""
    i = 12345
    def f(self):
        return 'ol?, mundo'

----------
assignee: docs at python
components: Documentation
messages: 351330
nosy: adorilson, docs at python
priority: normal
severity: normal
status: open
title: Docs: source code don't can be translate
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38057>
_______________________________________

From report at bugs.python.org  Sun Sep  8 12:13:34 2019
From: report at bugs.python.org (Kevin)
Date: Sun, 08 Sep 2019 16:13:34 +0000
Subject: [New-bugs-announce] [issue38058] Tutorial: 4.2. for Statements
Message-ID: <1567959214.96.0.389227588448.issue38058@roundup.psfhosted.org>


New submission from Kevin <Kfbritton at gmail.com>:

>>> # Measure some strings:
... words = ['cat', 'window', 'defenestrate']
>>> for w in words:
...     print(w, len(w))
...
cat 3
window 6
defenestrate 12


If you need to modify the sequence you are iterating over while inside the loop (for example to duplicate selected items), it is recommended that you first make a copy. Iterating over a sequence does not implicitly make a copy. The slice notation makes this especially convenient:


>>>>>> for w in words[:]:  # Loop over a slice copy of the entire list.
...     if len(w) > 6:
...         words.insert(0, w)
...
>>> words
['defenestrate', 'cat', 'window', 'defenestrate']

words is a tuple and is immutable

----------
assignee: docs at python
components: Documentation
messages: 351331
nosy: Derangedn00b, docs at python
priority: normal
severity: normal
status: open
title: Tutorial: 4.2. for Statements
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38058>
_______________________________________

From report at bugs.python.org  Sun Sep  8 14:22:30 2019
From: report at bugs.python.org (signing_agreement)
Date: Sun, 08 Sep 2019 18:22:30 +0000
Subject: [New-bugs-announce] [issue38059] Using sys.exit() over exit() in
 inspect.py
Message-ID: <1567966950.72.0.597820636326.issue38059@roundup.psfhosted.org>


New submission from signing_agreement <kuglasses at gmail.com>:

The change is arguably trivial, but given that this is a code change in a commonly-used module, I was asked to create an issue and a NEWS entry for it.

----------
components: Library (Lib)
messages: 351336
nosy: signing_agreement
priority: normal
pull_requests: 15387
severity: normal
status: open
title: Using sys.exit() over exit() in inspect.py
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38059>
_______________________________________

From report at bugs.python.org  Sun Sep  8 23:35:45 2019
From: report at bugs.python.org (Sangeeta M Chauhan)
Date: Mon, 09 Sep 2019 03:35:45 +0000
Subject: [New-bugs-announce] [issue38060] precedence (relational,
 logical operator)not working with single value
Message-ID: <1568000145.4.0.491367390087.issue38060@roundup.psfhosted.org>


New submission from Sangeeta M Chauhan <sangeetamchauhan at gmail.com>:

precendence betweeen relational and logical operators not working properly if expression contains single values instead of sub expressions. . Please see attached file

----------
components: Interpreter Core
files: pythonBug.py
messages: 351344
nosy: sangeetamchauhan
priority: normal
severity: normal
status: open
title: precedence (relational,logical operator)not working with single value
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file48600/pythonBug.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38060>
_______________________________________

From report at bugs.python.org  Mon Sep  9 03:25:36 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Sep 2019 07:25:36 +0000
Subject: [New-bugs-announce] [issue38061] FreeBSD: Optimize
 subprocess.Popen(close_fds=True) using closefrom()
Message-ID: <1568013936.15.0.668992536534.issue38061@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

The default value of subprocess.Popen "close_fds" parameter changed to True in Python 3. Compared to Python 2, close_fds=True can make Popen 10x slower: see bpo-37790.

A single close(fd) syscall is cheap. But when MAXFDS (maximum file descriptor number) is high, the loop calling close(fd) on each file descriptor can take several milliseconds.

On FreeBSD, the _posixsubprocess could use closefrom() to reduce the number of syscalls.

On Linux, _posixsubprocess lists the content of /proc/self/fd/ to only close open file descriptor, after fork() and before exec().

----------
components: Library (Lib)
messages: 351351
nosy: vstinner
priority: normal
severity: normal
status: open
title: FreeBSD: Optimize subprocess.Popen(close_fds=True) using closefrom()
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38061>
_______________________________________

From report at bugs.python.org  Mon Sep  9 04:51:28 2019
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 09 Sep 2019 08:51:28 +0000
Subject: [New-bugs-announce] [issue38062] Clarify that atexit.unregister
 matches by equality, not identity
Message-ID: <1568019088.84.0.443323623922.issue38062@roundup.psfhosted.org>


New submission from Mark Dickinson <dickinsm at gmail.com>:

Suppose I have a class that looks like this:

    class A:
        def cleanup(self):
            print("Doing essential cleanup")

and on an instance `a = A()`, I do: `atexit.register(a.cleanup)`.

Then it's not obvious from the documentation that an `atexit.unregister(a.cleanup)` will successfully undo the effect of the reigster call: the second `a.cleanup` is a different object from the first:

    >>> a = A()
    >>> clean1 = a.cleanup
    >>> clean2 = a.cleanup
    >>> clean1 is clean2
    False

Luckily, though the two bound methods are different objects, they're equal:

    >>> clean1 == clean2
    True

and from looking at the source, it's apparent that `atexit.unregister` compares by equality rather than identity, so everything works.

It would be good to add a sentence to the documentation for `atexit.unregister` to clarify that this can be expected to work.

----------
messages: 351363
nosy: mark.dickinson
priority: normal
severity: normal
status: open
title: Clarify that atexit.unregister matches by equality, not identity

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38062>
_______________________________________

From report at bugs.python.org  Mon Sep  9 06:26:30 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Sep 2019 10:26:30 +0000
Subject: [New-bugs-announce] [issue38063] Modify test_socket.py to use
 unittest test discovery
Message-ID: <1568024790.32.0.574936667066.issue38063@roundup.psfhosted.org>


Change by STINNER Victor <vstinner at python.org>:


----------
components: Tests
nosy: vstinner
priority: normal
severity: normal
status: open
title: Modify test_socket.py to use unittest test discovery
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38063>
_______________________________________

From report at bugs.python.org  Mon Sep  9 06:30:04 2019
From: report at bugs.python.org (Poker Online)
Date: Mon, 09 Sep 2019 10:30:04 +0000
Subject: [New-bugs-announce] [issue38064] Agen Poker Online Terpercaya
Message-ID: <1568025004.13.0.211579167516.issue38064@roundup.psfhosted.org>


New submission from Poker Online <aylarlie696 at gmail.com>:

Situs agen poker online terpercayahttps://www.jakartapoker.net/

----------
components: Cross-Build
messages: 351408
nosy: Alex.Willmer, pokeronline63
priority: normal
severity: normal
status: open
title: Agen Poker Online Terpercaya
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38064>
_______________________________________

From report at bugs.python.org  Mon Sep  9 06:56:50 2019
From: report at bugs.python.org (Paul Ganssle)
Date: Mon, 09 Sep 2019 10:56:50 +0000
Subject: [New-bugs-announce] [issue38065] Document the datetime capsule API
Message-ID: <1568026610.33.0.270212554664.issue38065@roundup.psfhosted.org>


New submission from Paul Ganssle <p.ganssle at gmail.com>:

The datetime module has a capsule API, which is very useful for other languages' bindings, but the C API documentation for datetime only covers the C macros: https://docs.python.org/3/c-api/datetime.html

The current extent of the documentation is that everyone who wants to bind to the C API (PyO3, Cython, pypy, etc), you need to just read the struct definition ( https://github.com/python/cpython/blob/master/Include/datetime.h#L150 ) and reads `_datetimemodule.c`. There's even some question as to whether the capsule is public (see, e.g. https://github.com/PyO3/pyo3/pull/393#issuecomment-476664650 ), when in fact I'm fairly certain that it's actually *preferred* to use the capsule API.

Most or many of the macros are thin wrappers around the capsule API, so we may need to figure out whether we want to try and use the same "documentation" for both versions, e.g.:

  .. c:function:: PyObject* PyDateTime_CAPI.Date_FromDate(int year, int month, int day, PyTypeObject* cls)
  .. c:function:: PyObject* PyDate_FromDate(int year, int month, int day)

     Return a :class:`datetime.date` object with the specified year, month and day.

     The version of this function in the capsule module takes an additional argument
     representing the specific subclass to construct.

Could replace:

  .. c:function:: PyObject* PyDate_FromDate(int year, int month, int day)

     Return a :class:`datetime.date` object with the specified year, month and day.

I would say that we also need a paragraph or two at the beginning of the C API document explaining why there are two ways to access most of these things?

A more minor bikeshedding-y issue is how we should stylize these: PyDatetime_CAPI.x? PyDatetime_CAPI->x? A dedicated RST directive? Something else?

----------
assignee: docs at python
components: Documentation
messages: 351427
nosy: belopolsky, docs at python, eric.araujo, ezio.melotti, mdk, p-ganssle, willingc
priority: normal
severity: normal
status: open
title: Document the datetime capsule API
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38065>
_______________________________________

From report at bugs.python.org  Mon Sep  9 07:38:35 2019
From: report at bugs.python.org (Andrew Svetlov)
Date: Mon, 09 Sep 2019 11:38:35 +0000
Subject: [New-bugs-announce] [issue38066] Hide internal asyncio.Stream
 methods
Message-ID: <1568029115.61.0.532730850424.issue38066@roundup.psfhosted.org>


New submission from Andrew Svetlov <andrew.svetlov at gmail.com>:

As discussed in discussion https://github.com/python/cpython/pull/14488#discussion_r322176319

Hide feed_eof(), feed_data(), set_exception() and set_transport() methods.

----------
components: asyncio
messages: 351433
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Hide internal asyncio.Stream methods
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38066>
_______________________________________

From report at bugs.python.org  Mon Sep  9 08:28:17 2019
From: report at bugs.python.org (Emmanuel Arias)
Date: Mon, 09 Sep 2019 12:28:17 +0000
Subject: [New-bugs-announce] [issue38067] Add headers parameter on
 RobotFileParser
Message-ID: <1568032097.51.0.453299797445.issue38067@roundup.psfhosted.org>


New submission from Emmanuel Arias <emmanuelarias30 at gmail.com>:

Hello I open an issue from a PR that I think need discussion:

I write here the @fuwaraido's PR description:

> I found some website like "sony.co.jp" requests User-Agent field in headers when I try to fetch their robots.txt. Unless they returns 403 Forbidden status code and RobotTextParser fails to read robots.txt. Currently there is no way to specify headers so I propose add headers arguments in constructor.

----------
components: Library (Lib)
messages: 351440
nosy: eamanu
priority: normal
pull_requests: 15423
severity: normal
status: open
title: Add headers parameter on RobotFileParser
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38067>
_______________________________________

From report at bugs.python.org  Mon Sep  9 08:53:24 2019
From: report at bugs.python.org (Benjamin Peterson)
Date: Mon, 09 Sep 2019 12:53:24 +0000
Subject: [New-bugs-announce] [issue38068] clean up configure logic for
 gettimeofday
Message-ID: <1568033604.94.0.557427388401.issue38068@roundup.psfhosted.org>


New submission from Benjamin Peterson <benjamin at python.org>:

We assume gettimeofday exists, so we don't need a configure test for it. We should also assume it always takes a second argument per POSIX.

----------
messages: 351446
nosy: benjamin.peterson
priority: normal
severity: normal
status: open
title: clean up configure logic for gettimeofday
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38068>
_______________________________________

From report at bugs.python.org  Mon Sep  9 09:50:57 2019
From: report at bugs.python.org (Dino Viehland)
Date: Mon, 09 Sep 2019 13:50:57 +0000
Subject: [New-bugs-announce] [issue38069] Make _posixsubprocess PEP-384
 compatible
Message-ID: <1568037057.64.0.843985719026.issue38069@roundup.psfhosted.org>


Change by Dino Viehland <dinoviehland at gmail.com>:


----------
assignee: dino.viehland
components: Extension Modules
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Make _posixsubprocess PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38069>
_______________________________________

From report at bugs.python.org  Mon Sep  9 10:05:36 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Sep 2019 14:05:36 +0000
Subject: [New-bugs-announce] [issue38070] visit_decref(): add an assertion
 to check that the object is not freed
Message-ID: <1568037936.88.0.10373402622.issue38070@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

This issue is related to bpo-36389 "Add gc.enable_object_debugger(): detect corrupted Python objects in the GC.

I propose to call _PyObject_IsFreed() in visit_decref() in debug mode, to get a better error message if something goes wrong.

visit_decref() is commonly found in C traceback (ex: gdb backtrace) of bugs related to the garbage collector.

----------
components: Interpreter Core
messages: 351470
nosy: vstinner
priority: normal
severity: normal
status: open
title: visit_decref(): add an assertion to check that the object is not freed
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38070>
_______________________________________

From report at bugs.python.org  Mon Sep  9 10:30:11 2019
From: report at bugs.python.org (Dino Viehland)
Date: Mon, 09 Sep 2019 14:30:11 +0000
Subject: [New-bugs-announce] [issue38071] Make termios PEP-384 compatible
Message-ID: <1568039411.64.0.805971266622.issue38071@roundup.psfhosted.org>


Change by Dino Viehland <dinoviehland at gmail.com>:


----------
components: Extension Modules
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Make termios PEP-384 compatible
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38071>
_______________________________________

From report at bugs.python.org  Mon Sep  9 10:58:35 2019
From: report at bugs.python.org (Dino Viehland)
Date: Mon, 09 Sep 2019 14:58:35 +0000
Subject: [New-bugs-announce] [issue38072] Make grp module PEP-384 compatible
Message-ID: <1568041115.77.0.778733211879.issue38072@roundup.psfhosted.org>


Change by Dino Viehland <dinoviehland at gmail.com>:


----------
assignee: dino.viehland
components: Extension Modules
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Make grp module PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38072>
_______________________________________

From report at bugs.python.org  Mon Sep  9 11:15:55 2019
From: report at bugs.python.org (Dino Viehland)
Date: Mon, 09 Sep 2019 15:15:55 +0000
Subject: [New-bugs-announce] [issue38073] Make pwd module PEP-384 compatible
Message-ID: <1568042155.96.0.920833676792.issue38073@roundup.psfhosted.org>


New submission from Dino Viehland <dinoviehland at gmail.com>:

Make the pwd module PEP-384 compatible

----------
assignee: dino.viehland
components: Extension Modules
messages: 351495
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Make pwd module PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38073>
_______________________________________

From report at bugs.python.org  Mon Sep  9 11:38:51 2019
From: report at bugs.python.org (Dino Viehland)
Date: Mon, 09 Sep 2019 15:38:51 +0000
Subject: [New-bugs-announce] [issue38074] Make zlib PEP-384 compatible
Message-ID: <1568043531.47.0.907223270637.issue38074@roundup.psfhosted.org>


New submission from Dino Viehland <dinoviehland at gmail.com>:

Make zlib PEP-384 compatible

----------
components: Extension Modules
messages: 351501
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Make zlib PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38074>
_______________________________________

From report at bugs.python.org  Mon Sep  9 11:57:58 2019
From: report at bugs.python.org (Dino Viehland)
Date: Mon, 09 Sep 2019 15:57:58 +0000
Subject: [New-bugs-announce] [issue38075] Make random module PEP-384
 compatible
Message-ID: <1568044678.81.0.091085634764.issue38075@roundup.psfhosted.org>


New submission from Dino Viehland <dinoviehland at gmail.com>:

Make random module PEP-384 compatible

----------
assignee: dino.viehland
components: Extension Modules
messages: 351509
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Make random module PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38075>
_______________________________________

From report at bugs.python.org  Mon Sep  9 12:35:08 2019
From: report at bugs.python.org (Dino Viehland)
Date: Mon, 09 Sep 2019 16:35:08 +0000
Subject: [New-bugs-announce] [issue38076] Make struct module PEP-384
 compatible
Message-ID: <1568046908.42.0.641730550956.issue38076@roundup.psfhosted.org>


New submission from Dino Viehland <dinoviehland at gmail.com>:

Make struct module PEP-384 compatible

----------
assignee: dino.viehland
components: Extension Modules
messages: 351524
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Make struct module PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38076>
_______________________________________

From report at bugs.python.org  Mon Sep  9 17:13:04 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 09 Sep 2019 21:13:04 +0000
Subject: [New-bugs-announce] [issue38077] IDLE leaking ARGV into globals()
 namespace
Message-ID: <1568063584.53.0.428247792791.issue38077@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Reproducer:

1) Turn-on IDLE
2) Create an empty file called: tmp.py
3) Press F5 to run the empty file
4) In the output shell window, type dir() which gives

>>> dir()
['__annotations__', '__builtins__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'argv']

----------
assignee: terry.reedy
components: IDLE
messages: 351552
nosy: rhettinger, terry.reedy
priority: high
severity: normal
status: open
title: IDLE leaking ARGV into globals() namespace
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38077>
_______________________________________

From report at bugs.python.org  Mon Sep  9 21:37:51 2019
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 10 Sep 2019 01:37:51 +0000
Subject: [New-bugs-announce] [issue38078] IDLE: Don't run internal code in
 user namespace.
Message-ID: <1568079471.24.0.207748043003.issue38078@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

#38077 fixed the bug of internal runcommand code not deleting 'argv' from the user namespace.  This issue is about not running code there.

When a subprocess is running, pyshell.ModifiedInterpreter.runcommand runs python code created by IDLE in locals == __main__.__dict__, the same as code enter by a user.  This requires that the code carefully clean up after itself.  I believe the same effect could by had more safely by exec-ing internal commands in the run module dict or a fresh temporary dict.

Possible solution.  In run.Executive.runcode, add 'user=True' to runcode signature, add 'if user else {}' to 'self.locals' arg, and add
    def runcommand(self, code):
        return self.runcode(code, user=False).  Then replace 'runcode' with 'runcommand' in pyshell runcommand body.

----------
assignee: terry.reedy
components: IDLE
messages: 351562
nosy: terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE: Don't run internal code in user namespace.
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38078>
_______________________________________

From report at bugs.python.org  Tue Sep 10 02:03:54 2019
From: report at bugs.python.org (Greg Price)
Date: Tue, 10 Sep 2019 06:03:54 +0000
Subject: [New-bugs-announce] [issue38079] _PyObject_VAR_SIZE should avoid
 arithmetic overflow
Message-ID: <1568095434.38.0.419984745356.issue38079@roundup.psfhosted.org>


New submission from Greg Price <gnprice at gmail.com>:

Currently `_PyObject_VAR_SIZE` effectively has a precondition that it must not be passed arguments which would overflow in its arithmetic. If that's violated, it overflows... which is undefined behavior already, and in fact the likely next thing that happens is we allocate a short buffer for a giant array of items, then overflow the buffer, bam!

(The arithmetic is on `Py_ssize_t`, a signed integer type; that's why overflow is UB.)

It's quite difficult to maintain that precondition at call sites, because it effectively means writing a check that duplicates the content of `_PyObject_VAR_SIZE` modulo some algebraic rearrangement.  Empirically we don't consistently manage that: we just fixed an 11-year-old bug where such a check was wrong (luckily in the safe/over-conservative direction), and some other call paths don't immediately appear to be checking at all.

So, I think it would be helpful to move that check into `_PyObject_VAR_SIZE` itself.

Cc'ing the three folks on the interest list "coverity scan", as that one seems the closest match for who'd likely be interested in commenting on this.

Details below, largely from GH-14838 which got me looking at this.

---

In GH-14838, @sir-sigurd fixed an overflow check on a call to PyObject_GC_NewVar.  The call site looked like this:

        if ((size_t)size > ((size_t)PY_SSIZE_T_MAX - sizeof(PyTupleObject) -
                    sizeof(PyObject *)) / sizeof(PyObject *)) {
            return (PyTupleObject *)PyErr_NoMemory();
        }
        op = PyObject_GC_NewVar(PyTupleObject, &PyTuple_Type, size);

but the `- sizeof(PyObject *)` had the wrong sign.  Happily the bug made the check too conservative, but that's basically a lucky coin-flip we had -- a similar bug could just as easily have gone the other direction, making the code vulnerable to arithmetic overflow, then memory corruption from overflowing a too-small buffer.

The bug was present for 11 years.

Sergey, I am still very curious how you discovered the bug. :-)

I feel like there's an opportunity for a more complete fix as a followup to this; that's this thread.

---

My next question on reading that fix was, OK, how can we write this so it's hard to get wrong?

I think the point is that `PyObject_GC_NewVar` really has a precondition, that the `nitems` it's passed (`size` in this caller) not be so big that the allocation will overflow. Specifically `_PyObject_VAR_SIZE(tp, nitems)` needs to not overflow. All the trickiness in this overflow check is basically an algebraic rearrangement of what `_PyObject_VAR_SIZE(&PyTuple_Type, size)` would say, plus substituting in the values actually found at `&PyTuple_Type`.

So, a sort of minimal fix would be to make something with a name like `PyObject_GC_NewVar_WouldOverflow` that encodes that. Maybe that in turn would just be a macro to delegate to a `_PyObject_VAR_SIZE_WOULD_OVERFLOW`, so that each layer only has to know what it actually knows.

I think that immediately raises a next question, though, which is: are other call sites of `PyObject_GC_NewVar` checking properly for overflow? At a quick grep, I see some that don't appear to be checking.

And: why should they have to? It seems like the best place for this check is immediately before the possibly-overflowing computation; or if not there, then the closer to it the better.

So, the ideal solution: `_PyObject_VAR_SIZE` itself would be augmented to first do this check, and only if it won't overflow then go and actually multiply.

There are only a handful of call sites of `_PyObject_VAR_SIZE`, so it wouldn't even be much of a migration to then take care of each of them.

Then we wouldn't need these prophylactic checks at callers' callers -- which (a) this bug demonstrates are hard to consistently get right, and (b) the grep mentioned above suggests we may not be consistently doing in the first place.

----------
components: Interpreter Core
messages: 351573
nosy: Greg Price, brett.cannon, christian.heimes, sir-sigurd, twouters
priority: normal
severity: normal
status: open
title: _PyObject_VAR_SIZE should avoid arithmetic overflow
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38079>
_______________________________________

From report at bugs.python.org  Tue Sep 10 02:14:06 2019
From: report at bugs.python.org (=?utf-8?b?5b6Q6Z2W?=)
Date: Tue, 10 Sep 2019 06:14:06 +0000
Subject: [New-bugs-announce] [issue38080] 2to3 urllib fixer: missing fix for
 urllib.getproxies
Message-ID: <1568096046.76.0.575371443955.issue38080@roundup.psfhosted.org>


New submission from ?? <shiyuchongf at gmail.com>:

urllib.getproxies can be converted to urllib.request.getproxies, and their documentation looks the same.
but in fix_urllib.py and my real test, it is ignored. I do not know why 2to3 does nothing. Is it a bug?

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 351576
nosy: shiyuchong
priority: normal
severity: normal
status: open
title: 2to3 urllib fixer: missing fix for urllib.getproxies
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38080>
_______________________________________

From report at bugs.python.org  Tue Sep 10 02:30:37 2019
From: report at bugs.python.org (=?utf-8?b?0JDQu9C10LrRgdCw0L3QtNGAINCh0LXQvNC10L3QvtCy?=)
Date: Tue, 10 Sep 2019 06:30:37 +0000
Subject: [New-bugs-announce] [issue38081] Different behavior of in 3.7 and
 3.8 os.path.realpath('nul')
Message-ID: <1568097037.9.0.584511449633.issue38081@roundup.psfhosted.org>


New submission from ????????? ??????? <iamsav at gmail.com>:

Windows 10:
```
C:\Users\User\Downloads>py -3.7 -c "import os.path;os.path.realpath('nul')"

C:\Users\User\Downloads>py -3.8 -c "import os.path;os.path.realpath('nul')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python38\lib\ntpath.py", line 592, in realpath
    path = _getfinalpathname_nonstrict(path)
  File "C:\Python38\lib\ntpath.py", line 566, in _getfinalpathname_nonstrict
    path = _readlink_deep(path, seen)
  File "C:\Python38\lib\ntpath.py", line 536, in _readlink_deep
    path = _nt_readlink(path)
OSError: [WinError 1] ???????? ???????: 'nul'
```
I think it's a bug. pip uses this code, so 'pip install pandas' won't work in 3.8

----------
components: Library (Lib)
messages: 351577
nosy: ????????? ???????
priority: normal
severity: normal
status: open
title: Different behavior of in 3.7 and 3.8 os.path.realpath('nul')
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38081>
_______________________________________

From report at bugs.python.org  Tue Sep 10 03:03:53 2019
From: report at bugs.python.org (Abhisek Maiti)
Date: Tue, 10 Sep 2019 07:03:53 +0000
Subject: [New-bugs-announce] [issue38082] datetime.time object incorrectly
 shows associated date in strftime() output
Message-ID: <1568099033.84.0.293529557755.issue38082@roundup.psfhosted.org>


New submission from Abhisek Maiti <mail2abhisek.maiti at gmail.com>:

`
from datetime import time

t = time(13, 5, 45)
`
While `t.year`, `t.month` and `t.day` results into `AttributeError` the following:
`
t.strftime('%Y : %B : %d - %H : %M : %S')
`
generates:

`
1900 : January : 01 - 13 : 05 : 45
`
This is unexpected behavior since `datetime.time` object does not have those attributes. The documentation at https://docs.python.org/3/library/datetime.html specifically states that 

"
class datetime.time

    An idealized time, independent of any particular day, assuming that every day has exactly 24*60*60 seconds (there is no notion of ?leap seconds? here). Attributes: hour, minute, second, microsecond, and tzinfo.
"

----------
components: Library (Lib)
messages: 351580
nosy: Abhisek Maiti
priority: normal
severity: normal
status: open
title: datetime.time object incorrectly shows associated date in strftime() output
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38082>
_______________________________________

From report at bugs.python.org  Tue Sep 10 03:21:56 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Sep 2019 07:21:56 +0000
Subject: [New-bugs-announce] [issue38083] Minor improvements in asdl_c.py
 and Python-ast.c
Message-ID: <1568100116.88.0.525709025274.issue38083@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

* Use the const qualifier for constant C strings. This will save memory for their copy in a modifiable memory.
* Intern field and attribute names. This will save time, as comparing interned strings is faster.
* Temporary incref a borrowed reference to a list item. Arbitrary Python code can be executed when convert an item to C values. The list containing the item can be modified and the reference can became dead if not incref it.

----------
components: Interpreter Core
messages: 351582
nosy: eric.snow, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Minor improvements in asdl_c.py and Python-ast.c
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38083>
_______________________________________

From report at bugs.python.org  Tue Sep 10 04:58:29 2019
From: report at bugs.python.org (Steve Dower)
Date: Tue, 10 Sep 2019 08:58:29 +0000
Subject: [New-bugs-announce] [issue38084] multiprocessing cannot recover
 from crashed worker
Message-ID: <1568105909.03.0.686404819271.issue38084@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

Imitation repro:

import os
from multiprocessing import Pool

def f(x):
    os._exit(0)
    return "success"

if __name__ == '__main__':
    with Pool(1) as p:
        print(p.map(f, [1]))


Obviously a process may crash for various other reasons besides os._exit().

I believe this is the cause of issue37245.

----------
messages: 351594
nosy: davin, pitrou, steve.dower
priority: normal
severity: normal
status: open
title: multiprocessing cannot recover from crashed worker
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38084>
_______________________________________

From report at bugs.python.org  Tue Sep 10 05:09:41 2019
From: report at bugs.python.org (xitop)
Date: Tue, 10 Sep 2019 09:09:41 +0000
Subject: [New-bugs-announce] [issue38085] Interrupting class creation in
 __init_subclass__ may lead to incorrect isinstance() and issubclass()
 results
Message-ID: <1568106581.16.0.558052842785.issue38085@roundup.psfhosted.org>


New submission from xitop <reg.bugs at poti.sk>:

An exception in __init__subclass__ leads under certain circumstances to wrong isinstance() and issubclass() results. The exception probably leaves Python internal data in inconsistent state.

Here is a demonstration program from Stack Overflow:

--- begin --
from abc import ABCMeta

class Animal(metaclass=ABCMeta):
    pass

class Plant(metaclass=ABCMeta):
    def __init_subclass__(cls):
        assert not issubclass(cls, Animal), "Plants cannot be Animals"

class Dog(Animal):
    pass

try:
    class Triffid(Animal, Plant):
        pass
except Exception:
    pass

print("Dog is Animal?", issubclass(Dog, Animal))
print("Dog is Plant?", issubclass(Dog, Plant))
--- end --

Result is:

Dog is Animal? True
Dog is Plant? True

Changing the order of the print statements will result in:

Dog is Plant? False
Dog is Animal? False

Another ill-behaving program and a partial analysis can be found at SO: https://stackoverflow.com/q/57848663/5378816

----------
components: Interpreter Core
messages: 351599
nosy: xitop
priority: normal
severity: normal
status: open
title: Interrupting class creation in __init_subclass__ may lead to incorrect isinstance() and issubclass() results
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38085>
_______________________________________

From report at bugs.python.org  Tue Sep 10 07:36:34 2019
From: report at bugs.python.org (Jason R. Coombs)
Date: Tue, 10 Sep 2019 11:36:34 +0000
Subject: [New-bugs-announce] [issue38086] Synchronize importlib.metadata
 with importlib_metadata 0.21
Message-ID: <1568115394.78.0.274032945021.issue38086@roundup.psfhosted.org>


New submission from Jason R. Coombs <jaraco at jaraco.com>:

Importlib_metadata 0.21 has been released (technically releasing at this moment). This issue serves as an anchor to incorporate those changes (https://gitlab.com/python-devs/importlib_metadata/blob/4b86813ecf7e40e6d553976e3f5604f90d475fb4/importlib_metadata/docs/changelog.rst).

----------
assignee: jaraco
messages: 351625
nosy: barry, jaraco, lukasz.langa
priority: release blocker
severity: normal
stage: needs patch
status: open
title: Synchronize importlib.metadata with importlib_metadata 0.21
type: enhancement
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38086>
_______________________________________

From report at bugs.python.org  Tue Sep 10 09:02:30 2019
From: report at bugs.python.org (Steve Dower)
Date: Tue, 10 Sep 2019 13:02:30 +0000
Subject: [New-bugs-announce] [issue38087] test_ntpath must account for
 casing changes
Message-ID: <1568120550.64.0.710799778395.issue38087@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

Tests in test_ntpath that go via the actual file system (e.g. via realpath()) need to allow for the case of the path potentially changing.

----------
assignee: steve.dower
components: Windows
messages: 351636
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
pull_requests: 15494
severity: normal
status: open
title: test_ntpath must account for casing changes
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38087>
_______________________________________

From report at bugs.python.org  Tue Sep 10 09:03:38 2019
From: report at bugs.python.org (Steve Dower)
Date: Tue, 10 Sep 2019 13:03:38 +0000
Subject: [New-bugs-announce] [issue38088] distutils cannot locate
 vcruntime140.dll with only VS 2019 installed
Message-ID: <1568120618.81.0.389197586577.issue38088@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

Because we are only looking for the VC141 redistributable, if we only have VC142 available then we won't find it.

We should expand the glob() call to allow VC14?\vcruntime140.dll.

----------
assignee: steve.dower
components: Distutils, Windows
messages: 351638
nosy: dstufft, eric.araujo, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
pull_requests: 15495
severity: normal
status: open
title: distutils cannot locate vcruntime140.dll with only VS 2019 installed
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38088>
_______________________________________

From report at bugs.python.org  Tue Sep 10 09:04:47 2019
From: report at bugs.python.org (Steve Dower)
Date: Tue, 10 Sep 2019 13:04:47 +0000
Subject: [New-bugs-announce] [issue38089] Make macOS tests optional on Azure
 Pipelines
Message-ID: <1568120687.8.0.406819210697.issue38089@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

Avoid blocking PRs because of random macOS failures (such as in issue37245).

Also reduce the timeout.

----------
assignee: steve.dower
components: Build
messages: 351639
nosy: steve.dower
priority: normal
pull_requests: 15496
severity: normal
status: open
title: Make macOS tests optional on Azure Pipelines
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38089>
_______________________________________

From report at bugs.python.org  Tue Sep 10 09:06:05 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Tue, 10 Sep 2019 13:06:05 +0000
Subject: [New-bugs-announce] [issue38090] test_ctypes is leaking references
Message-ID: <1568120765.1.0.320006339983.issue38090@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

test_ctypes leaked [22, 22, 22] references, sum=66
3 tests failed again:
    test_ctypes test_import test_importlib


https://buildbot.python.org/all/#/builders/1/builds/710

----------
components: Tests
messages: 351640
nosy: pablogsal
priority: normal
severity: normal
status: open
title: test_ctypes is leaking references
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38090>
_______________________________________

From report at bugs.python.org  Tue Sep 10 09:26:40 2019
From: report at bugs.python.org (Ronan Lamy)
Date: Tue, 10 Sep 2019 13:26:40 +0000
Subject: [New-bugs-announce] [issue38091] Import deadlock detection causes
 deadlock
Message-ID: <1568122000.74.0.240735069302.issue38091@roundup.psfhosted.org>


New submission from Ronan Lamy <ronan.lamy at gmail.com>:

There seems to be a race condition in importlib._bootstrap._ModuleLock that can cause a deadlock. The sequence of operations is as follows:

* Thread 1 calls lock.acquire()
* Thread 1 sets itself as lock.owner and begins importing the module
* Thread 2 calls lock.acquire() and waits for lock.lock
* Thread 2 gets lock.lock
* Thread 1 calls lock.acquire() again, due to a nested import
* Thread 1 sets itself as blocking on lock: _blocking_on[tid1] = lock
* Thread 2 enters lock.has_deadlock()
* Thread 2 busy-waits forever in has_deadlock() because lock.owner == tid1 and _blocking_on[tid1] == lock
* Thread 1 waits forever for lock.lock since thread 2 owns it

The issue was found in pypy3 but it also affects all the recent CPython versions I tried.
I can reliably reproduce the issue by adding an artificial delay to _ModuleLock.has_deadlock(), e.g. with this patch:

diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py
index f167c84..7f7188e 100644
--- a/Lib/test/test_import/__init__.py
+++ b/Lib/test/test_import/__init__.py
@@ -435,10 +435,15 @@ class ImportTests(unittest.TestCase):
                 os.does_not_exist
 
     def test_concurrency(self):
+        def delay_has_deadlock(frame, event, arg):
+            if event == 'call' and frame.f_code.co_name == 'has_deadlock':
+                time.sleep(0.2)
+
         sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'data'))
         try:
             exc = None
             def run():
+                sys.settrace(delay_has_deadlock)
                 event.wait()
                 try:
                     import package

----------
components: Library (Lib)
messages: 351647
nosy: Ronan.Lamy
priority: normal
severity: normal
status: open
title: Import deadlock detection causes deadlock
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38091>
_______________________________________

From report at bugs.python.org  Tue Sep 10 09:32:39 2019
From: report at bugs.python.org (Pierre Glaser)
Date: Tue, 10 Sep 2019 13:32:39 +0000
Subject: [New-bugs-announce] [issue38092] environment variables not passed
 correctly using new virtualenv launching in windows and python3.7+
Message-ID: <1568122359.92.0.800539114246.issue38092@roundup.psfhosted.org>


New submission from Pierre Glaser <pierreglaser at msn.com>:

If I am not mistaken, when creating a new process on Python3.7 and later on Windows, if using a virtualenv, Python now uses a launcher. The launcher is being notified that it must create a virtual-environment Python (and not a system Python) program using the __PYVENV_LAUNCHER__ environment variable, passed as part of the environment of launcher process created using in _winapi.CreateProcess
(see https://github.com/python/cpython/blob/9008be303a89bfab8c3314c6a42330b5523adc8b/Lib/multiprocessing/popen_spawn_win32.py#L73-L75)

However, if I am not mistaken `env` is not passed at the right position (https://github.com/python/cpython/blob/9008be303a89bfab8c3314c6a42330b5523adc8b/Modules/_winapi.c#L1062-L1068). These lines were part of a bugfix patch (see https://bugs.python.org/issue35797), solving an issue for multiprocessing-based packages. We ended trying to backport to loky (https://github.com/tomMoral/loky, a multiprocessing-based package) but the bug was not fixed. Passing 'env' correctly fixed the bug.

Two things:
- It is hard to provide a reproducer for this issue as it requires patching the CPython source code.
- I don't understand why env being not passed correctly does not manifest itself in the test-suite. What is even more troubling is that even with this bug, the virtualenv launcher seems to get that a virtualenv is used when creating processes in multiprocessing (at least, sys.path includes the path/to/venv/lib/python3.x/site-packages). However, I am not familiar with the virtualenv launcher for python3.7+ and windows.

----------
components: Windows
messages: 351650
nosy: paul.moore, pierreglaser, pitrou, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: environment variables not passed correctly using new virtualenv launching in windows and python3.7+
type: crash
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38092>
_______________________________________

From report at bugs.python.org  Tue Sep 10 10:42:41 2019
From: report at bugs.python.org (Lisa Roach)
Date: Tue, 10 Sep 2019 14:42:41 +0000
Subject: [New-bugs-announce] [issue38093] Update MagicMock __aenter__ and
 __aexit__ to return AsyncMock's
Message-ID: <1568126561.19.0.234964443085.issue38093@roundup.psfhosted.org>


New submission from Lisa Roach <lisaroach14 at gmail.com>:

After a discussion with Michael, xtreak, and Ezio we've decided to try to make the process of mocking an async context manager slightly easier.

Currently if you want to mock a context manager that is used like this:

async with cm():  # note that cm is called here
  blah

You need to mock cm as a MagicMock and set __aenter__ and __aexit__ return values directly because they do not exist on MagicMocks.


Our first step to making this easier is setting the calculated return value of MagicMock __aenter__ and __aexit__ to be AsyncMock, which will make it so you do not need to set them specifically.


A future improvement may be to create a ContextManagerMock directly that can take an async kwarg that would return a context manager with AsyncMocks for __aenter__ and __aexit__ automatically.

----------
assignee: lisroach
messages: 351682
nosy: ezio.melotti, lisroach, michael.foord, xtreak
priority: normal
severity: normal
stage: needs patch
status: open
title: Update MagicMock __aenter__ and __aexit__ to return AsyncMock's
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38093>
_______________________________________

From report at bugs.python.org  Tue Sep 10 11:24:16 2019
From: report at bugs.python.org (Gregory P. Smith)
Date: Tue, 10 Sep 2019 15:24:16 +0000
Subject: [New-bugs-announce] [issue38094] unneeded assignment to wb.len in
 PyBytes_Concat using buffer protocol
Message-ID: <1568129056.61.0.163100885596.issue38094@roundup.psfhosted.org>


New submission from Gregory P. Smith <greg at krypto.org>:

https://github.com/python/cpython/pull/15274

----------
assignee: gregory.p.smith
messages: 351700
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: unneeded assignment to wb.len in PyBytes_Concat using buffer protocol
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38094>
_______________________________________

From report at bugs.python.org  Tue Sep 10 15:10:54 2019
From: report at bugs.python.org (Ronan Lamy)
Date: Tue, 10 Sep 2019 19:10:54 +0000
Subject: [New-bugs-announce] [issue38095] Multi-threaded circular import
 fails with _DeadlockError when using importlib
Message-ID: <1568142654.85.0.849631704923.issue38095@roundup.psfhosted.org>


New submission from Ronan Lamy <ronan.lamy at gmail.com>:

The IMPORT_NAME bytecode has a fast path calling PyImport_ImportModuleLevelObject() that behaves differently from importlib. In particular, test_circular_imports() in test_threaded_import.py fails due to a _DeadlockError if you replace some import statements with importlib.import_module() as in

diff --git a/Lib/test/test_importlib/test_threaded_import.py b/Lib/test/test_importlib/test_threaded_import.py
index d1f64c7..243d2c1 100644
--- a/Lib/test/test_importlib/test_threaded_import.py
+++ b/Lib/test/test_importlib/test_threaded_import.py
@@ -60,8 +60,12 @@ circular_imports_modules = {
         x = 'b'
         import D
         """,
-    'C': """import B""",
-    'D': """import A""",
+    'C': """if 1:
+        import importlib
+        B = importlib.import_module('B')""",
+    'D': """if 1:
+        import importlib
+        A = importlib.import_module('A')""",
 }

----------
components: Library (Lib)
messages: 351738
nosy: Ronan.Lamy
priority: normal
severity: normal
status: open
title: Multi-threaded circular import fails with _DeadlockError when using importlib
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38095>
_______________________________________

From report at bugs.python.org  Tue Sep 10 18:05:24 2019
From: report at bugs.python.org (Tim Peters)
Date: Tue, 10 Sep 2019 22:05:24 +0000
Subject: [New-bugs-announce] [issue38096] Clean up the "struct sequence" /
 "named tuple" docs
Message-ID: <1568153124.61.0.864461609465.issue38096@roundup.psfhosted.org>


New submission from Tim Peters <tim at python.org>:

The Glossary has this entry:

"""
struct sequence
A tuple with named elements. Struct sequences expose an interface similar to named tuple in that elements can be accessed either by index or as an attribute. However, they do not have any of the named tuple methods like _make() or _asdict(). Examples of struct sequences include sys.float_info and the return value of os.stat().
"""

It's trying to document the internal CPython implementation `structseq` detail, a class of which can only be created from the C API (no Python interface is exposed).

But the glossary _also_ has an entry for "named tuple", and the docs just above are confusing that for the related but distinct `collections.namedtuple` facility, which does allow creating a named tuple class from Python, but is NOT the only kind of "named tuple".

This is a mess ;-)

I suggest:

- Throw away the "struct sequence" glossary entry, and remove all references to it from the docs.

- Use "named tuple" everywhere instead.

- Clarify the "named tuple" docs to make clear that it applies to any class that supports tuple methods and also named elements.

- Make clear that `collections.namedtuple` classes are "named tuples" classes, but not the only kind, and support a number of methods beyond what's required for "named tuples".

- Optionally, effectively expose `structseq` to Python code, so that users have one obvious way to create their own bare-bones named tuple types.

- I'd prefer to insist that for any named tuple class C,

  issubclass(C, tuple)
  
  is True.  That's true now of structseqs and collections.namedtuples, and is a sane way to make clear that they must support all tuple methods.
  
I'd mark this as "newcomer friendly", except it's likely to become a bikeshedding nightmare ;-)

----------
assignee: docs at python
components: Documentation
messages: 351742
nosy: docs at python, tim.peters
priority: normal
severity: normal
status: open
title: Clean up the "struct sequence" / "named tuple" docs
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38096>
_______________________________________

From report at bugs.python.org  Tue Sep 10 18:27:54 2019
From: report at bugs.python.org (Mann Jani)
Date: Tue, 10 Sep 2019 22:27:54 +0000
Subject: [New-bugs-announce] [issue38097] cmd.Cmd: Allow other readline
 completion methods
Message-ID: <1568154474.22.0.125293373456.issue38097@roundup.psfhosted.org>


New submission from Mann Jani <mann4jani at gmail.com>:

readline has completion methods other than `complete` such as `menu-complete` or `menu-complete-backward` which behave differently.
Currently Cmd class has hardcoded `complete` method to the completion key which means trying to use other methods is not possible in a derived class without re-implementing cmdloop, or using some other crazy workarounds.

IMO Cmd.__init__ should take another argument named completionmethod with the default value of "complete" so that existing usage is not broken but it is possible to pass other values.

----------
components: Library (Lib)
messages: 351748
nosy: mannjani
priority: normal
severity: normal
status: open
title: cmd.Cmd: Allow other readline completion methods
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38097>
_______________________________________

From report at bugs.python.org  Tue Sep 10 19:30:52 2019
From: report at bugs.python.org (Spandan Garg)
Date: Tue, 10 Sep 2019 23:30:52 +0000
Subject: [New-bugs-announce] [issue38098] Special Characters Don't Work When
 Using Virtual Environment
Message-ID: <1568158252.04.0.0991697281074.issue38098@roundup.psfhosted.org>


New submission from Spandan Garg <spandan.garg1706 at gmail.com>:

Recently, I noticed that whenever I execute python from inside a virtual environment, I can't seem to write special characters into the shell. Although, when I exit the virtual environment, this issue no longer occurs. I've confirmed that the versions of python inside and outside the virtual environment are the same.

For example, when I paste the string 'doesn?t' (with the Unicode apostrophe) into the shell without virtual environment, I get the following:
>>> doesn?t     # without virtual environment

When I paste the same string into the shell with a virtual environment active (and the same version of python), I get the following:
>>> doesnt      # with virtual environment

The version of python I am using is python 3.7.4 (though I have reproduced this on python 3.6.5 as well). I am creating the virtual environments using the venv module. Also, I think that this problem only happens on Windows as I couldn?t reproduce it on Linux.

Does anyone know why this may be happening? Please let me know if I should provide any other details.

----------
components: Unicode
messages: 351757
nosy: ezio.melotti, glGarg, vstinner
priority: normal
severity: normal
status: open
title: Special Characters Don't Work When Using Virtual Environment
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38098>
_______________________________________

From report at bugs.python.org  Wed Sep 11 02:21:11 2019
From: report at bugs.python.org (Reed)
Date: Wed, 11 Sep 2019 06:21:11 +0000
Subject: [New-bugs-announce] [issue38099] __dict__ attribute is incorrectly
 stated to be read-only
Message-ID: <1568182871.44.0.335247784266.issue38099@roundup.psfhosted.org>


New submission from Reed <readuw at gmail.com>:

The documentation in this section (https://docs.python.org/3/library/stdtypes.html#special-attributes) states that the __dict__ attribute, and several others, are read-only. In particular, it states:

"The implementation adds a few special read-only attributes to several object types, where they are relevant."

Then it lists several attributes, including __dict__. However, __dict__ is writable. For example:

    class A: pass 
    A().__dict__ = {'x': 1}

Most other listed attributes, such as __class__ and __name__, are writable as well. They should not be documented as read-only.

(Also, I'm not sure why the documentation lists object.__dict__ and instance.__class__. What is the difference between an object and an instance?)

----------
assignee: docs at python
components: Documentation
messages: 351765
nosy: docs at python, reed
priority: normal
severity: normal
status: open
title: __dict__ attribute is incorrectly stated to be read-only
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38099>
_______________________________________

From report at bugs.python.org  Wed Sep 11 05:36:55 2019
From: report at bugs.python.org (Lisa Roach)
Date: Wed, 11 Sep 2019 09:36:55 +0000
Subject: [New-bugs-announce] [issue38100] Spelling error in unittest.mock
 code
Message-ID: <1568194615.26.0.336032437884.issue38100@roundup.psfhosted.org>


New submission from Lisa Roach <lisaroach14 at gmail.com>:

Good first issue.

There is a spelling error in the unittest.mock library, "return_calulator" should be "return_calculator".

This is in the `_set_return_value` method in `Lib/unittest/mock.py`.

----------
components: Library (Lib)
keywords: easy, newcomer friendly
messages: 351778
nosy: lisroach
priority: low
severity: normal
status: open
title: Spelling error in unittest.mock code

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38100>
_______________________________________

From report at bugs.python.org  Wed Sep 11 05:39:29 2019
From: report at bugs.python.org (Lisa Roach)
Date: Wed, 11 Sep 2019 09:39:29 +0000
Subject: [New-bugs-announce] [issue38101] Update devguide triaging keywords
Message-ID: <1568194769.69.0.909868564617.issue38101@roundup.psfhosted.org>


New submission from Lisa Roach <lisaroach14 at gmail.com>:

Currently the devguide triaging keywords page is out of date: 

https://devguide.python.org/triaging/#keywords

It is missing keywords like "easy (C)", "security_issue", and "pep3121".

----------
assignee: docs at python
components: Documentation
keywords: easy
messages: 351781
nosy: docs at python, lisroach
priority: normal
severity: normal
status: open
title: Update devguide triaging keywords

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38101>
_______________________________________

From report at bugs.python.org  Wed Sep 11 05:48:31 2019
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 11 Sep 2019 09:48:31 +0000
Subject: [New-bugs-announce] [issue38102] Document
 importlib.machinery.WindowsRegistryFinder.DEBUG_BUILD
Message-ID: <1568195311.03.0.46215825189.issue38102@roundup.psfhosted.org>


New submission from Brett Cannon <brett at python.org>:

importlib.machinery.WindowsRegistryFinder.DEBUG_BUILD has existed since Python 3.3 (https://hg.python.org/cpython/file/v3.3.0/Lib/importlib/_bootstrap.py#l1750).

----------
assignee: docs at python
components: Documentation
messages: 351784
nosy: brett.cannon, docs at python
priority: normal
severity: normal
status: open
title: Document importlib.machinery.WindowsRegistryFinder.DEBUG_BUILD
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38102>
_______________________________________

From report at bugs.python.org  Wed Sep 11 05:56:22 2019
From: report at bugs.python.org (Ezio Melotti)
Date: Wed, 11 Sep 2019 09:56:22 +0000
Subject: [New-bugs-announce] [issue38103] Duplicate label "examples" in the
 Docs
Message-ID: <1568195782.91.0.281693262362.issue38103@roundup.psfhosted.org>


New submission from Ezio Melotti <ezio.melotti at gmail.com>:

I just rebuilt the docs and got the following error:
Warning, treated as error:
Doc/c-api/typeobj.rst:2456:duplicate label examples, other instance in Doc/distutils/examples.rst
Makefile:49: recipe for target 'build' failed
make: *** [build] Error 2

----------
assignee: ezio.melotti
components: Documentation
messages: 351789
nosy: ezio.melotti
priority: normal
severity: normal
stage: needs patch
status: open
title: Duplicate label "examples" in the Docs
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38103>
_______________________________________

From report at bugs.python.org  Wed Sep 11 06:12:22 2019
From: report at bugs.python.org (Manoj C)
Date: Wed, 11 Sep 2019 10:12:22 +0000
Subject: [New-bugs-announce] [issue38104] File descriptor error when
 subprocess call is used with event loop enabled in main thread
Message-ID: <1568196742.14.0.919718116174.issue38104@roundup.psfhosted.org>


New submission from Manoj C <manojcam1 at gmail.com>:

I have been using asyncio to run subprocess calls in a separate thread. For this purpose I start an event loop in my main thread as per the recommendation - https://docs.python.org/3/library/asyncio-subprocess.html#subprocess-and-threads .

Now when I use a normal subprocess call in the main thread, I start getting following file descriptor error after few iterations:

```
Exception ignored when trying to write to the signal wakeup fd:
BlockingIOError: [Errno 11] Resource temporarily unavailable
```

I have reproduced the problem in a small script and am seeing that the error goes away if I do not start the even loop in the main thread.

```
import asyncio
import subprocess
import time


def try_error():
    for i in range(1,500):
        print(i)
        try:
            subprocess.run(["ls", "-l"], check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        except subprocess.CalledProcessError as e:
            print(f"Exception raised {e.stderr}\nOutput {e.stdout}")



def definite_error():
    w = asyncio.get_child_watcher()
    l = asyncio.get_event_loop()
    try_error()


if __name__ == "__main__":
    definite_error()
```

This is the smallest subset of the code which can reproduce the error. In the original code, I run a asyncio.create_subprocess_exec in a parallel thread. The normal subprocess call is part of third party code which call from the main thread and hence cannot modify it.

----------
components: asyncio
messages: 351795
nosy: Manoj C, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: File descriptor error when subprocess call is used with event loop enabled in main thread
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38104>
_______________________________________

From report at bugs.python.org  Wed Sep 11 06:29:21 2019
From: report at bugs.python.org (Guido Imperiale)
Date: Wed, 11 Sep 2019 10:29:21 +0000
Subject: [New-bugs-announce] [issue38105] hash collision when hash(x) == -2
 causes many calls to __eq__
Message-ID: <1568197761.75.0.53300036491.issue38105@roundup.psfhosted.org>


New submission from Guido Imperiale <crusaderky at gmail.com>:

Take two objects where hash(a) == hash(b) == -2 exactly, but a != b,
When they are inserted in a dict or set, the __eq__ method is invoked a whopping 13 times.

Does this have something to do with the special case of hash(-1) = -2?

class C:
    def __init__(self, x, h):
        self.x = x
        self.h = h

    def __eq__(self, other):
        print(f"{self}.__eq__({other})")
        return self.x == other.x

    def __hash__(self):
        print(f"{self}.__hash__")
        return self.h

    def __repr__(self):
        return f"C({self.x, self.h})"

>>> {C(1, -2), C(2, -2)}
C((1, -2)).__hash__
C((2, -2)).__hash__
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
C((1, -2)).__eq__(C((2, -2)))
{C((1, -2)), C((2, -2))}

>>> {C(1, -3), C(1, -3)}
C((1, -3)).__hash__
C((1, -3)).__hash__
C((1, -3)).__eq__(C((1, -3)))
{C((1, -3))}

>>> {C(1, -1), C(1, -1)}
C((1, -1)).__hash__
C((1, -1)).__hash__
C((1, -1)).__eq__(C((1, -1)))

>>> {C(1, -2), C(1, -2)}
C((1, -2)).__hash__
C((1, -2)).__hash__
C((1, -2)).__eq__(C((1, -2)))
{C((1, -2))}

----------
messages: 351798
nosy: crusaderky
priority: normal
severity: normal
status: open
title: hash collision when hash(x) == -2 causes many calls to __eq__
type: performance
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38105>
_______________________________________

From report at bugs.python.org  Wed Sep 11 07:17:33 2019
From: report at bugs.python.org (Kirill Smelkov)
Date: Wed, 11 Sep 2019 11:17:33 +0000
Subject: [New-bugs-announce] [issue38106] Race in PyThread_release_lock -
 can lead to MEMORY CORRUPTION and DEADLOCK
Message-ID: <1568200653.3.0.865964137512.issue38106@roundup.psfhosted.org>


New submission from Kirill Smelkov <kirr at nexedi.com>:

Hello up there. I believe I've discovered a race in PyThread_release_lock on
Python2.7 that, on systems where POSIX semaphores are not available and Python
locks are implemented with mutexes and condition variables, can lead to MEMORY
CORRUPTION and DEADLOCK. The particular system I've discovered the bug on is
macOS Mojave 10.14.6.

The bug is already fixed on Python3 and the fix for Python2 is easy:

    git cherry-pick 187aa545165d

Thanks beforehand,
Kirill


Bug description

( Please see attached pylock_bug.pyx for the program that triggers the bug for real. )

On Darwin, even though this is considered as POSIX, Python uses
mutex+condition variable to implement its lock, and, as of 20190828, Py2.7
implementation, even though similar issue was fixed for Py3 in 2012, contains
synchronization bug: the condition is signalled after mutex unlock while the
correct protocol is to signal condition from under mutex:

  https://github.com/python/cpython/blob/v2.7.16-127-g0229b56d8c0/Python/thread_pthread.h#L486-L506
  https://github.com/python/cpython/commit/187aa545165d (py3 fix)

PyPy has the same bug for both pypy2 and pypy3:

  https://bitbucket.org/pypy/pypy/src/578667b3fef9/rpython/translator/c/src/thread_pthread.c#lines-443:465
  https://bitbucket.org/pypy/pypy/src/5b42890d48c3/rpython/translator/c/src/thread_pthread.c#lines-443:465

Signalling condition outside of corresponding mutex is considered OK by
POSIX, but in Python context it can lead to at least memory corruption if we
consider the whole lifetime of python level lock. For example the following
logical scenario:

      T1                                          T2

  sema = Lock()
  sema.acquire()

                                              sema.release()

  sema.acquire()
  free(sema)

  ...


can translate to the next C-level calls:

      T1                                          T2

  # sema = Lock()
  sema = malloc(...)
  sema.locked = 0
  pthread_mutex_init(&sema.mut)
  pthread_cond_init (&sema.lock_released)

  # sema.acquire()
  pthread_mutex_lock(&sema.mut)
  # sees sema.locked == 0
  sema.locked = 1
  pthread_mutex_unlock(&sema.mut)


                                              # sema.release()
                                              pthread_mutex_lock(&sema.mut)
                                              sema.locked = 0
                                              pthread_mutex_unlock(&sema.mut)

                      # OS scheduler gets in and relinquishes control from T2
                      # to another process
                                              ...

  # second sema.acquire()
  pthread_mutex_lock(&sema.mut)
  # sees sema.locked == 0
  sema.locked = 1
  pthread_mutex_unlock(&sema.mut)

  # free(sema)
  pthread_mutex_destroy(&sema.mut)
  pthread_cond_destroy (&sema.lock_released)
  free(sema)


  # ...
  e.g. malloc() which returns memory where sema was

                                              ...
                      # OS scheduler returns control to T2
                      # sema.release() continues
                      #
                      # BUT sema was already freed and writing to anywhere
                      # inside sema block CORRUPTS MEMORY. In particular if
                      # _another_ python-level lock was allocated where sema
                      # block was, writing into the memory can have effect on
                      # further synchronization correctness and in particular
                      # lead to deadlock on lock that was next allocated.
                                              pthread_cond_signal(&sema.lock_released)

Note that T2.pthread_cond_signal(&sema.lock_released) CORRUPTS MEMORY as it
is called when sema memory was already freed and is potentially
reallocated for another object.

The fix is to move pthread_cond_signal to be done under corresponding mutex:

  # sema.release()
  pthread_mutex_lock(&sema.mut)
  sema.locked = 0
  pthread_cond_signal(&sema.lock_released)
  pthread_mutex_unlock(&sema.mut)

by cherry-picking commit 187aa545165d ("Signal condition variables with the
mutex held. Destroy condition variables before their mutexes").


Bug history

The bug was there since 1994 - since at least [1]. It was discussed in 2001
with original code author[2], but the code was still considered to be
race-free. In 2010 the place where pthread_cond_signal should be - before or
after pthread_mutex_unlock - was discussed with the rationale to avoid
threads bouncing[3,4,5], and in 2012 pthread_cond_signal was moved to be
called from under mutex, but only for CPython3[6,7].

In 2019 the bug was (re-)discovered while testing Pygolang[8] on macOS with
CPython2 and PyPy2 and PyPy3.

[1] https://github.com/python/cpython/commit/2c8cb9f3d240
[2] https://bugs.python.org/issue433625
[3] https://bugs.python.org/issue8299#msg103224
[4] https://bugs.python.org/issue8410#msg103313
[5] https://bugs.python.org/issue8411#msg113301
[6] https://bugs.python.org/issue15038#msg163187
[7] https://github.com/python/cpython/commit/187aa545165d
[8] https://pypi.org/project/pygolang

----------
components: Interpreter Core, macOS
files: pylock_bug.pyx
messages: 351826
nosy: arigo, beazley, benjamin.peterson, brian.curtin, dabeaz, flox, gvanrossum, jyasskin, kristjan.jonsson, loewis, navytux, ned.deily, paul.moore, pitrou, python-dev, r.david.murray, rhettinger, ronaldoussoren, sbt, scoder, shihao, techtonik, terry.reedy, tim.golden, tim.peters, torsten, vstinner, ysj.ray
priority: normal
severity: normal
status: open
title: Race in PyThread_release_lock - can lead to MEMORY CORRUPTION and DEADLOCK
type: behavior
versions: Python 2.7
Added file: https://bugs.python.org/file48604/pylock_bug.pyx

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38106>
_______________________________________

From report at bugs.python.org  Wed Sep 11 08:16:03 2019
From: report at bugs.python.org (Andrew Svetlov)
Date: Wed, 11 Sep 2019 12:16:03 +0000
Subject: [New-bugs-announce] [issue38107] Replace direct future and task
 contructor calls with factories in asyncio tests
Message-ID: <1568204163.99.0.31241530598.issue38107@roundup.psfhosted.org>


New submission from Andrew Svetlov <andrew.svetlov at gmail.com>:

1. Factories are recommended way for a task and a future creation.
2. Consider this task as asyncio tests tidy up.

----------
components: asyncio
keywords: 3.3regression
messages: 351837
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Replace direct future and task contructor calls with factories in asyncio tests
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38107>
_______________________________________

From report at bugs.python.org  Wed Sep 11 09:24:30 2019
From: report at bugs.python.org (Lisa Roach)
Date: Wed, 11 Sep 2019 13:24:30 +0000
Subject: [New-bugs-announce] [issue38108] Everything in Mock should inherit
 from Base
Message-ID: <1568208270.51.0.869895956527.issue38108@roundup.psfhosted.org>


New submission from Lisa Roach <lisaroach14 at gmail.com>:

Everything within Mock should be inheriting from the top-level parent class, Base (and properly call super).

For multiple inheritance to work correctly they all should be inheriting from the same parent class ultimately. 

Classes that need update:

MagicMixin
AsyncMagicMixin
NonCallableMock (instance = object.__new__(new) should be switched to a super call here)

----------
assignee: lisroach
messages: 351863
nosy: ezio.melotti, lisroach, michael.foord, xtreak
priority: normal
severity: normal
status: open
title: Everything in Mock should inherit from Base
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38108>
_______________________________________

From report at bugs.python.org  Wed Sep 11 09:35:59 2019
From: report at bugs.python.org (Ronan Lamy)
Date: Wed, 11 Sep 2019 13:35:59 +0000
Subject: [New-bugs-announce] [issue38109] Missing constants in Lib/stat.py
Message-ID: <1568208959.37.0.605416078815.issue38109@roundup.psfhosted.org>


New submission from Ronan Lamy <ronan.lamy at gmail.com>:

According to the docs, stat should include constants S_IFDOOR, S_IFPORT, S_IFWHT as well as the related S_IS... functions. However, these are only defined in _stat. I know that stat is a bit special (see bpo-11016), but that goes against PEP 399.

----------
components: Library (Lib)
messages: 351870
nosy: Ronan.Lamy
priority: normal
severity: normal
status: open
title: Missing constants in Lib/stat.py
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38109>
_______________________________________

From report at bugs.python.org  Wed Sep 11 09:40:01 2019
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 11 Sep 2019 13:40:01 +0000
Subject: [New-bugs-announce] [issue38110] Use fdwalk() within
 os.closerange() impl if available
Message-ID: <1568209201.53.0.0873310068645.issue38110@roundup.psfhosted.org>


New submission from Gregory P. Smith <greg at krypto.org>:

On POSIX platforms that support the fdwalk() API, use it within the os.closerange() implementation.

----------
components: Demos and Tools
messages: 351875
nosy: gregory.p.smith
priority: normal
severity: normal
stage: patch review
status: open
title: Use fdwalk() within os.closerange() impl if available
type: performance
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38110>
_______________________________________

From report at bugs.python.org  Wed Sep 11 09:47:43 2019
From: report at bugs.python.org (Redcxx)
Date: Wed, 11 Sep 2019 13:47:43 +0000
Subject: [New-bugs-announce] [issue38111] Error while building Python from
 source
Message-ID: <1568209663.11.0.274214516349.issue38111@roundup.psfhosted.org>


New submission from Redcxx <weilue222 at gmail.com>:

Python failed `make` when building from source if `--enabled-share` is pass in when running `./configure`.

````
...(some output which seem irrevelent)
/bin/ld: /home/e38160wl/Python3.7.4/local/lib/libssl.a(s3_meth.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/home/e38160wl/Python3.7.4/local/lib/libssl.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
ln: failed to access ?libpython3.7m.so.1.0?: No such file or directory
make: *** [libpython3.7m.so] Error 1

````

I tried adding `CFLAGS="-fPIC"` as parameter of `./configure` but it does not make any changes

----------
components: Build
messages: 351880
nosy: Redcxx
priority: normal
severity: normal
status: open
title: Error while building Python from source
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38111>
_______________________________________

From report at bugs.python.org  Wed Sep 11 10:01:19 2019
From: report at bugs.python.org (Petr Viktorin)
Date: Wed, 11 Sep 2019 14:01:19 +0000
Subject: [New-bugs-announce] [issue38112] Compileall improvements
Message-ID: <1568210479.53.0.901531773072.issue38112@roundup.psfhosted.org>


New submission from Petr Viktorin <encukou at gmail.com>:

Hello,
In Fedora, we've long used a Bash script to compile modules to bytecode, as the compileall module was historically unsuitable.
Recently, Lum?r re-wrote our enhancements to compileall, and published the result as compileall2 on PyPI. We'd like to get the improvements back into the standard library so everyone can benefit.
The improvements are:

- Raise the default recursion limit 
- Control of the path baked into the *.pyc file (`-s` to strip a prefix; `-p` to prepend a new one) to support different build/install locations.
- Compiling for multiple optimization levels at once (by specifying -o multiple times)

A PR should follow shortly.

----------
components: Installation
messages: 351887
nosy: petr.viktorin
priority: normal
severity: normal
status: open
title: Compileall improvements

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38112>
_______________________________________

From report at bugs.python.org  Wed Sep 11 10:05:11 2019
From: report at bugs.python.org (Dino Viehland)
Date: Wed, 11 Sep 2019 14:05:11 +0000
Subject: [New-bugs-announce] [issue38113] Make ast module PEP-384 compatible
Message-ID: <1568210711.23.0.951109613555.issue38113@roundup.psfhosted.org>


New submission from Dino Viehland <dinoviehland at gmail.com>:

Remove various static state from the ast module and make it PEP-384 compatible.  This will help make it be more compatible w/ subinterpreters and make it re-usable by 3rd party implementations.

----------
assignee: dino.viehland
components: Extension Modules
messages: 351888
nosy: dino.viehland
priority: normal
severity: normal
status: open
title: Make ast module PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38113>
_______________________________________

From report at bugs.python.org  Wed Sep 11 10:23:28 2019
From: report at bugs.python.org (Steve Dower)
Date: Wed, 11 Sep 2019 14:23:28 +0000
Subject: [New-bugs-announce] [issue38114] Exclude pip.ini from Nuget package
Message-ID: <1568211808.23.0.234312966832.issue38114@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

The pip.ini intended for store packages is being included in the nuget package. We should not do that.

----------
assignee: steve.dower
components: Windows
keywords: 3.8regression
messages: 351900
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Exclude pip.ini from Nuget package
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38114>
_______________________________________

From report at bugs.python.org  Wed Sep 11 10:27:50 2019
From: report at bugs.python.org (Thomas Wouters)
Date: Wed, 11 Sep 2019 14:27:50 +0000
Subject: [New-bugs-announce] [issue38115] Invalid bytecode offsets in
 co_lnotab
Message-ID: <1568212070.98.0.977590744203.issue38115@roundup.psfhosted.org>


New submission from Thomas Wouters <thomas at python.org>:

The peephole optimizer in Python 2.7 and later (and probably a *lot* earlier) has a bug where if the optimizer entirely optimizes away the last line(s) of a function, the lnotab references invalid bytecode offsets:

>>> def f(cond1, cond2):
...     while 1:
...         return 3
...     while 1:
...         return 5
...     return 6
... 
>>> list(dis.findlinestarts(f.__code__))
[(0, 3), (4, 5), (8, 6)]
>>> len(f.__code__.co_code)
8
>>> f.__code__.co_code[8]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: index out of range

The problem is that the lnotab-readjustment in Python/peephole.c doesn't account for trailing NOPs in a bytecode string. I haven't been able to reproduce this before Python 3.8, probably because the optimizer wasn't capable of optimizing things aggressively enough to end a bytecode string with NOPs.

I have a fix for this bug already.

----------
assignee: twouters
components: Interpreter Core
keywords: patch
messages: 351902
nosy: lukasz.langa, pablogsal, twouters
priority: release blocker
severity: normal
status: open
title: Invalid bytecode offsets in co_lnotab
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38115>
_______________________________________

From report at bugs.python.org  Wed Sep 11 10:48:15 2019
From: report at bugs.python.org (Dino Viehland)
Date: Wed, 11 Sep 2019 14:48:15 +0000
Subject: [New-bugs-announce] [issue38116] Make select module PEP-384
 compatible
Message-ID: <1568213295.38.0.552886185542.issue38116@roundup.psfhosted.org>


New submission from Dino Viehland <dinoviehland at gmail.com>:

Remove static PyTypeObject* definitions and move static data to _selectstate to help improve compatibility w/ subinterpreters and re-use by alternate implementations.

----------
assignee: dino.viehland
components: Extension Modules
messages: 351913
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Make select module PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38116>
_______________________________________

From report at bugs.python.org  Wed Sep 11 11:18:48 2019
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 11 Sep 2019 15:18:48 +0000
Subject: [New-bugs-announce] [issue38117] Update to OpenSSL 1.1.1d, 1.1.0l,
 1.0.2t
Message-ID: <1568215128.5.0.058779071611.issue38117@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

OpenSSL versions 1.1.1d, 1.1.0l, and 1.0.2t were released yesterday. They contain a low priority security fix.

----------
components: Extension Modules
messages: 351934
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Update to OpenSSL 1.1.1d, 1.1.0l, 1.0.2t
type: security
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38117>
_______________________________________

From report at bugs.python.org  Wed Sep 11 11:28:57 2019
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Sep 2019 15:28:57 +0000
Subject: [New-bugs-announce] [issue38118] Valgrind warnings when running
 tokenize.py
Message-ID: <1568215737.14.0.192001407155.issue38118@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

Running tokenize.py in Valgrind emits 2 warnings:

$ ./configure --with-valgrind
$ make
$ echo|PYTHONMALLOC=malloc valgrind ./python Lib/tokenize.py

(A)

==30746== Conditional jump or move depends on uninitialised value(s)
==30746==    at 0x4A01F7: PyUnicode_Decode (unicodeobject.c:3395)
==30746==    by 0x4A01F7: PyUnicode_Decode (unicodeobject.c:3350)
==30746==    by 0x4A062E: PyUnicode_FromEncodedObject (unicodeobject.c:3268)
==30746==    by 0x4A090D: unicode_new (unicodeobject.c:15125)
==30746==    by 0x46F515: type_call (typeobject.c:969)
==30746==    by 0x435F82: _PyObject_MakeTpCall (call.c:167)
==30746==    by 0x422B3A: _PyObject_Vectorcall (abstract.h:104)
==30746==    by 0x422B3A: _PyObject_Vectorcall (abstract.h:96)
==30746==    by 0x422B3A: call_function (ceval.c:4984)
==30746==    by 0x423C26: _PyEval_EvalFrameDefault (ceval.c:3496)
==30746==    by 0x4217D2: function_code_fastcall (call.c:292)
==30746==    by 0x4360FC: _PyObject_FastCallDict (call.c:109)
==30746==    by 0x436309: _PyObject_Call_Prepend (call.c:447)
==30746==    by 0x4731F0: slot_tp_init (typeobject.c:6767)
==30746==    by 0x46F562: type_call (typeobject.c:989)

(B)

==30746== Conditional jump or move depends on uninitialised value(s)
==30746==    at 0x4DC8FF: PyState_AddModule (pystate.c:707)
==30746==    by 0x485A5E1: PyInit_zlib (zlibmodule.c:1472)
==30746==    by 0x4CA49C: _PyImport_LoadDynamicModuleWithSpec (importdl.c:164)
==30746==    by 0x4C8462: _imp_create_dynamic_impl (import.c:2292)
==30746==    by 0x4C8462: _imp_create_dynamic (import.c.h:330)
==30746==    by 0x570783: cfunction_vectorcall_FASTCALL (methodobject.c:382)
==30746==    by 0x435ACD: PyVectorcall_Call (call.c:203)
==30746==    by 0x42424F: do_call_core (ceval.c:5031)
==30746==    by 0x42424F: _PyEval_EvalFrameDefault (ceval.c:3557)
==30746==    by 0x4AF4CF: _PyEval_EvalCodeWithName (ceval.c:4296)
==30746==    by 0x435D3F: _PyFunction_Vectorcall (call.c:355)
==30746==    by 0x422AA8: _PyObject_Vectorcall (abstract.h:106)
==30746==    by 0x422AA8: call_function (ceval.c:4984)
==30746==    by 0x428E51: _PyEval_EvalFrameDefault (ceval.c:3465)
==30746==    by 0x4217D2: function_code_fastcall (call.c:292)

Warning (A) goes away when Python is compiled with gcc -O1.

Warning (B) is still present when Python is compiled with gcc -O0.


It's unclear to me if it's a false alarm or a real issue.


On Fedora, the issue can be seen with:

   echo|valgrind python3 /usr/lib64/python3.7/tokenize.py

On Fedora 30, python3 is Python 3.7.4.


Downstream Fedora issue: https://bugzilla.redhat.com/show_bug.cgi?id=1751208

----------
components: Interpreter Core
messages: 351939
nosy: vstinner
priority: normal
severity: normal
status: open
title: Valgrind warnings when running tokenize.py
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38118>
_______________________________________

From report at bugs.python.org  Wed Sep 11 11:58:26 2019
From: report at bugs.python.org (Davin Potts)
Date: Wed, 11 Sep 2019 15:58:26 +0000
Subject: [New-bugs-announce] [issue38119] resource tracker destroys shared
 memory segments when other processes should still have valid access
Message-ID: <1568217506.85.0.770661201111.issue38119@roundup.psfhosted.org>


New submission from Davin Potts <python at discontinuity.net>:

The resource tracker currently destroys (via _posixshmem.shm_unlink) shared memory segments on posix systems when any independently created Python process with a handle on a shared memory segment exits (gracefully or otherwise).  This breaks the expected cross-platform behavior that a shared memory segment persists at least as long as any running process has a handle on that segment.

As described with an example scenario in issue37754:
Let's say a three processes P1, P2 and P3 are trying to communicate using shared memory.
 --> P1 creates the shared memory block, and waits for P2 and P3 to access it.
 --> P2 starts and attaches this shared memory segment, writes some data to it and exits.
 --> Now in case of Unix, shm_unlink is called as soon as P2 exits. (This is by action of the resource tracker.)
 --> Now, P3 starts and tries to attach the shared memory segment.
 --> P3 will not be able to attach the shared memory segment in Unix, because shm_unlink has been called on that segment.
 --> Whereas, P3 will be able to attach to the shared memory segment in Windows.

Another key scenario we expect to work but does not currently:
1. A multiprocessing.managers.SharedMemoryManager is instantiated and started in process A.
2. A shared memory segment is created using that manager in process A.
3. A serialized representation of that shared memory segment is deserialized in process B.
4. Process B does work with the shared memory segment that is also still visible to process A.
5. Process B exits cleanly.
6. Process A reads data from the shared memory segment after process B is gone.  (This currently fails.)

The SharedMemoryManager provides a flexible means for ensuring cleanup of shared memory segments.  The current resource tracker attempts to treat shared memory segments as equivalent to semaphore references, which is too narrow of an interpretation.  As such, the current resource tracker should not be attempting to enforce cleanup of shared memory segments because it breaks expected behavior and significantly limits functionality.

----------
assignee: davin
components: Library (Lib)
messages: 351960
nosy: davin, pablogsal, pitrou, vinay0410, vstinner
priority: normal
severity: normal
status: open
title: resource tracker destroys shared memory segments when other processes should still have valid access
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38119>
_______________________________________

From report at bugs.python.org  Wed Sep 11 12:06:48 2019
From: report at bugs.python.org (Karthikeyan Singaravelan)
Date: Wed, 11 Sep 2019 16:06:48 +0000
Subject: [New-bugs-announce] [issue38120] DeprecationWarning in test_random
 due to invalid seed arguments
Message-ID: <1568218008.0.0.38131396656.issue38120@roundup.psfhosted.org>


New submission from Karthikeyan Singaravelan <tir.karthi at gmail.com>:

Since d0cdeaab76fef8a6e5a04665df226b6659111e4e (bpo-32554) using random.seed raises DeprecationWarning for some types. This is not handled in test_random causing DeprecationWarnings. I will raise a PR for this.

commit d0cdeaab76fef8a6e5a04665df226b6659111e4e
Author: Raymond Hettinger <rhettinger at users.noreply.github.com>
Date:   Thu Aug 22 09:19:36 2019 -0700

    bpo-32554: Deprecate hashing arbitrary types in random.seed() (GH-15382)

----------
components: Tests
messages: 351966
nosy: pablogsal, rhettinger, vstinner, xtreak
priority: normal
severity: normal
status: open
title: DeprecationWarning in test_random due to invalid seed arguments
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38120>
_______________________________________

From report at bugs.python.org  Wed Sep 11 12:41:22 2019
From: report at bugs.python.org (Jason R. Coombs)
Date: Wed, 11 Sep 2019 16:41:22 +0000
Subject: [New-bugs-announce] [issue38121] Synchronize importlib.metadata
 with importlib_metadata 0.22
Message-ID: <1568220082.72.0.518622909571.issue38121@roundup.psfhosted.org>


New submission from Jason R. Coombs <jaraco at jaraco.com>:

Incorporate the changes from the 0.22 release (https://gitlab.com/python-devs/importlib_metadata/blob/0.22/importlib_metadata/docs/changelog.rst) including the new lightweight shim from PathFinder into importlib.metadata.MetadataPathFinder.

----------
assignee: jaraco
messages: 351971
nosy: jaraco
priority: normal
severity: normal
status: open
title: Synchronize importlib.metadata with importlib_metadata 0.22
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38121>
_______________________________________

From report at bugs.python.org  Wed Sep 11 13:24:54 2019
From: report at bugs.python.org (Lisa Roach)
Date: Wed, 11 Sep 2019 17:24:54 +0000
Subject: [New-bugs-announce] [issue38122] AsyncMock force always creating an
 AsyncMock for child mocks
Message-ID: <1568222694.73.0.217906468671.issue38122@roundup.psfhosted.org>


New submission from Lisa Roach <lisaroach14 at gmail.com>:

This idea has been brought up in person before and also discussed on AsyncTest: https://github.com/Martiusweb/asynctest/issues/100


It could be useful if someone has a lot of attributes that are also async that need to be mocked. 

It could probably be done with a flag that gets passed on to _get_child_mock and overrides the if statement to always return an AsyncMock.


Looking mostly for if people think this is a good idea/bad idea and help see around any corners that may be there.

----------
assignee: lisroach
components: Library (Lib)
messages: 351991
nosy: cjw296, ezio.melotti, lisroach, michael.foord, xtreak
priority: low
severity: normal
status: open
title: AsyncMock force always creating an AsyncMock for child mocks
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38122>
_______________________________________

From report at bugs.python.org  Wed Sep 11 14:31:22 2019
From: report at bugs.python.org (Jonathan Gossage)
Date: Wed, 11 Sep 2019 18:31:22 +0000
Subject: [New-bugs-announce] [issue38123] Unable to find Python3.8.0b4 on
 Ubuntu 19004 desktop
Message-ID: <1568226682.04.0.539752167504.issue38123@roundup.psfhosted.org>


New submission from Jonathan Gossage <jgossage at gmail.com>:

I installed Python 3.8.0b4 manually on Ubuntu 19.04 desktop. After the installation that appeared to run OK, I was unable to find python3.8, even though it had been installed in /usr/local/bin and that directory was on the path. I got the result:

jgossage at jgossage-XPS-8700:~$ python3.8 --version
bash: /usr/bin/python3.8: No such file or directory

There was no sign of Python in /etc/alternatives so, I assume that Linux alternatives were not part of the problem. I had no problem finding other files such as pip3.8.

----------
components: Installation
messages: 352009
nosy: Jonathan.Gossage
priority: normal
severity: normal
status: open
title: Unable to find Python3.8.0b4 on Ubuntu 19004 desktop
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38123>
_______________________________________

From report at bugs.python.org  Wed Sep 11 19:08:38 2019
From: report at bugs.python.org (Benjamin Peterson)
Date: Wed, 11 Sep 2019 23:08:38 +0000
Subject: [New-bugs-announce] [issue38124] off-by-one error in
 PyState_GetModule
Message-ID: <1568243318.53.0.112380405251.issue38124@roundup.psfhosted.org>


New submission from Benjamin Peterson <benjamin at python.org>:

https://github.com/python/cpython/blob/3f4db4a0bab073b768fae958e93288bd5d24eadd/Python/pystate.c#L707 should be using > not >=.

----------
components: Interpreter Core
messages: 352031
nosy: benjamin.peterson
priority: normal
severity: normal
status: open
title: off-by-one error in PyState_GetModule
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38124>
_______________________________________

From report at bugs.python.org  Wed Sep 11 19:58:32 2019
From: report at bugs.python.org (Windson Yang)
Date: Wed, 11 Sep 2019 23:58:32 +0000
Subject: [New-bugs-announce] [issue38125] Can' build document in Sphinx
 v2.2.0
Message-ID: <1568246312.85.0.271143869481.issue38125@roundup.psfhosted.org>


New submission from Windson Yang <wiwindson at outlook.com>:

I got two errors when I try to build python document with

    make venv
    make html

The first one is 

> Since v2.0, Sphinx uses "index" as master_doc by default. Please add "master_doc = 'contents'" to your conf.py.

After fixing this one, the second one is 

> /Users/windson/learn/cpython/Doc/library/email.message.rst:4:duplicate object description of email.message, other instance in library/email.compat32-message, use :noindex: for one of them

----------
assignee: docs at python
components: Documentation
messages: 352034
nosy: Windson Yang, docs at python
priority: normal
severity: normal
status: open
title: Can' build document in Sphinx v2.2.0
type: crash
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38125>
_______________________________________

From report at bugs.python.org  Wed Sep 11 22:18:26 2019
From: report at bugs.python.org (pacien)
Date: Thu, 12 Sep 2019 02:18:26 +0000
Subject: [New-bugs-announce] [issue38126] mailcap top level wildcard support
Message-ID: <1568254706.48.0.577471820986.issue38126@roundup.psfhosted.org>


New submission from pacien <vpmfty.pythonbugs at pacien.net>:

given ~/.mailcap containing:
*/*; xdg-open "%s"

>>> mailcap.findmatch(mailcap.getcaps(), 'application/pdf', filename='thing.pdf')
returns: (None, None)
instead of: ('xdg-open "thing.pdf"', {'view': 'xdg-open "%s"', 'lineno': 0})

---

While top-level wildcards aren't defined in RFC1524, most other tools and programs (including mutt for example) handle them properly.

Entries of the form of "*; someprogram" or "*/*; someprogram" are useful for fallbacks when no other more precise type can be found.

----------
components: email
messages: 352038
nosy: barry, pacien, r.david.murray
priority: normal
severity: normal
status: open
title: mailcap top level wildcard support
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38126>
_______________________________________

From report at bugs.python.org  Wed Sep 11 23:37:56 2019
From: report at bugs.python.org (Zackery Spytz)
Date: Thu, 12 Sep 2019 03:37:56 +0000
Subject: [New-bugs-announce] [issue38127] A fatal error when running
 test_ctypes
Message-ID: <1568259476.22.0.837518009168.issue38127@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

When running test_ctypes, I encountered a fatal error.

./python -m test test_ctypes
Run tests sequentially
0:00:00 load avg: 0.13 [1/1] test_ctypes
Fatal Python error: a function returned a result with an error set
MemoryError

The above exception was the direct cause of the following exception:

SystemError: <built-in method from_param of _ctypes.PyCPointerType object at 0x55ff17de6e60> returned a result with an error set

Current thread 0x00007f80d7417140 (most recent call first):
  File "/home/lubuntu2/cpython/Lib/ctypes/test/test_as_parameter.py", line 41 in test_pointers
  File "/home/lubuntu2/cpython/Lib/unittest/case.py", line 616 in _callTestMethod
  File "/home/lubuntu2/cpython/Lib/unittest/case.py", line 659 in run
  File "/home/lubuntu2/cpython/Lib/unittest/case.py", line 719 in __call__
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/lubuntu2/cpython/Lib/test/support/testresult.py", line 162 in run
  File "/home/lubuntu2/cpython/Lib/test/support/__init__.py", line 1996 in _run_suite
  File "/home/lubuntu2/cpython/Lib/test/support/__init__.py", line 2092 in run_unittest
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 209 in _test_module
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 234 in _runtest_inner2
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 270 in _runtest_inner
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 153 in _runtest
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 193 in runtest
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 409 in run_tests_sequential
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 507 in run_tests
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 674 in _main
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 628 in main
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 695 in main
  File "/home/lubuntu2/cpython/Lib/test/__main__.py", line 2 in <module>
  File "/home/lubuntu2/cpython/Lib/runpy.py", line 85 in _run_code
  File "/home/lubuntu2/cpython/Lib/runpy.py", line 192 in _run_module_as_main
Aborted (core dumped)



It seems that this is because the PyObject_IsSubclass() call in PyCPointerType_from_param() is not checked for failure.

----------
components: Extension Modules, ctypes
messages: 352045
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: A fatal error when running test_ctypes
type: crash
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38127>
_______________________________________

From report at bugs.python.org  Wed Sep 11 23:53:29 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 12 Sep 2019 03:53:29 +0000
Subject: [New-bugs-announce] [issue38128] IDLE undo calls get_saved() when
 set to None
Message-ID: <1568260409.36.0.994615457064.issue38128@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Terry, I don't know if these exception reports are useful to you.  If not, feel free to close this.  I run IDLE in day long sessions and only see these tracebacks when I'm exiting, so I don't know the proximate trigger event.


Exception in Tkinter callback
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/tkinter/__init__.py", line 1883, in __call__
    return self.func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/multicall.py", line 176, in handler
    r = l[i](event)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/filelist.py", line 54, in close_all_callback
    reply = edit.close()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/pyshell.py", line 1008, in close
    return EditorWindow.close(self)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/editor.py", line 1077, in close
    reply = self.maybesave()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/outwin.py", line 94, in maybesave
    return 'yes' if self.get_saved() else 'no'
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/editor.py", line 1010, in get_saved
    return self.undo.get_saved()
AttributeError: 'NoneType' object has no attribute 'get_saved'

----------
assignee: terry.reedy
components: IDLE
messages: 352047
nosy: rhettinger, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE undo calls get_saved() when set to None
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38128>
_______________________________________

From report at bugs.python.org  Thu Sep 12 03:06:41 2019
From: report at bugs.python.org (Situs Judi Online Terpercaya)
Date: Thu, 12 Sep 2019 07:06:41 +0000
Subject: [New-bugs-announce] [issue38129] Situs Judi Online Terpercaya
Message-ID: <1568272001.65.0.873223588746.issue38129@roundup.psfhosted.org>


New submission from Situs Judi Online Terpercaya <koreastyle778 at gmail.com>:

Situs https://sukabet.co/ adalah situs judi online terpercaya Indonesia yang menyediakan permainan judi online dan memberikan jaminan 100% menang tanpa kekalahan

----------
components: Distutils
files: images (3).jpg
messages: 352053
nosy: dstufft, eric.araujo, situsjudionlineterpercaya
priority: normal
severity: normal
status: open
title: Situs Judi Online Terpercaya
type: compile error
versions: Python 3.6
Added file: https://bugs.python.org/file48605/images (3).jpg

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38129>
_______________________________________

From report at bugs.python.org  Thu Sep 12 04:13:05 2019
From: report at bugs.python.org (Mohammad Dehghan)
Date: Thu, 12 Sep 2019 08:13:05 +0000
Subject: [New-bugs-announce] [issue38130] Error in section 8.6 of the
 tutorials (Errors and Exceptions, `finally` bullet points))
Message-ID: <1568275985.44.0.899489125827.issue38130@roundup.psfhosted.org>


Change by Mohammad Dehghan <md.unicorn at gmail.com>:


----------
assignee: docs at python
components: Documentation
nosy: Mohammad Dehghan, docs at python
priority: normal
severity: normal
status: open
title: Error in section 8.6 of the tutorials (Errors and Exceptions, `finally` bullet points))
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38130>
_______________________________________

From report at bugs.python.org  Thu Sep 12 06:21:59 2019
From: report at bugs.python.org (Ilya Kamenshchikov)
Date: Thu, 12 Sep 2019 10:21:59 +0000
Subject: [New-bugs-announce] [issue38131] compile(mode='eval') uninformative
 error message
Message-ID: <1568283719.06.0.851695472479.issue38131@roundup.psfhosted.org>


New submission from Ilya Kamenshchikov <ikamenshchikov at gmail.com>:

While trying to construct a valid ast node programmatically, I have tried following: 

import ast

tree = ast.BinOp(left=ast.Num(n=2), right=ast.Num(n=2), op=ast.Add())
expr = ast.Expression(body=[tree])
ast.fix_missing_locations(expr)
exe = compile(expr, filename="", mode="eval")
print(eval(exe))

Unfortunately this gives unhelpful error message:

>>>    exe = compile(expr, filename="", mode="eval")
TypeError: required field "lineno" missing from expr

Turns out I need to make body of ast.Expression not a list, but a node:
expr = ast.Expression(body=tree)  # works

Confusion also comes from naming the field "body", which takes value of a list for ast.Module and some others.

----------
components: Library (Lib)
messages: 352090
nosy: Ilya Kamenshchikov
priority: normal
severity: normal
status: open
title: compile(mode='eval') uninformative error message
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38131>
_______________________________________

From report at bugs.python.org  Thu Sep 12 06:44:02 2019
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 12 Sep 2019 10:44:02 +0000
Subject: [New-bugs-announce] [issue38132] Simplify hashlib implementation
Message-ID: <1568285042.8.0.241070384019.issue38132@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

The hashlib module uses complicated macros and some pointless optimization. The code can be simplified to make maintenance easier. Cleanup also makes it easier to get rid of static global state and to add "usedforsecurity" feature.

* The EVPobject contains a PyObject* with the name of the hashing algorithm as PyUnicode object. The name is rarely used and can be efficiently calculated from a const char* of the EVP_MD_CTX.

* The module caches pre-generated EVP_MD_CTX objects for commonly used hashes like sha256. Tests with timeit has shown that generating a EVP_MD_CTX from a EVP constructor is as fast as copying and reinitializing a EVP_MD_CTX.

* The pre-calculated constructs can be replaced with argument clinic to make the code more readable.

----------
assignee: christian.heimes
components: Extension Modules
messages: 352099
nosy: christian.heimes, gregory.p.smith
priority: normal
severity: normal
status: open
title: Simplify hashlib implementation
type: enhancement
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38132>
_______________________________________

From report at bugs.python.org  Thu Sep 12 07:05:13 2019
From: report at bugs.python.org (Steve Dower)
Date: Thu, 12 Sep 2019 11:05:13 +0000
Subject: [New-bugs-announce] [issue38133] py.exe cannot locate Store package
Message-ID: <1568286313.97.0.196410705778.issue38133@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

The py.exe launcher cannot locate installations from the Store (unless you have no other installations on your machine, which is unlikely).

----------
assignee: steve.dower
components: Windows
messages: 352107
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: py.exe cannot locate Store package
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38133>
_______________________________________

From report at bugs.python.org  Thu Sep 12 07:13:37 2019
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 12 Sep 2019 11:13:37 +0000
Subject: [New-bugs-announce] [issue38134] Remove copy of fast PBKDF2
 algorithm
Message-ID: <1568286817.02.0.48627400928.issue38134@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

bpo #18582 added a C implementation of PBKDF2 algorithm. Back then Python still supported OpenSSL versions with a slow version of PBKDF2-HMAC. Nowadays all supported OpenSSL versions contain an optimized version of the password derivation algorithm.

----------
assignee: christian.heimes
components: Extension Modules
messages: 352111
nosy: christian.heimes
priority: normal
severity: normal
stage: needs patch
status: open
title: Remove copy of fast PBKDF2 algorithm
type: enhancement
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38134>
_______________________________________

From report at bugs.python.org  Thu Sep 12 07:19:40 2019
From: report at bugs.python.org (Mark Shannon)
Date: Thu, 12 Sep 2019 11:19:40 +0000
Subject: [New-bugs-announce] [issue38135] Depth first search in compile.c
 creates wrong BB order for certain CFG.
Message-ID: <1568287180.55.0.964846639884.issue38135@roundup.psfhosted.org>


New submission from Mark Shannon <mark at hotpy.org>:

Consider a flow graph of four nodes, A, B, C, D
where the A, B, C are "next" successors of each other (in order) and C branches to B and B branches to D. Note that there is no "next" link to the D block.

The correct order is A, B, C, D but the 'dfs' function in compile.c produces the order A, B, D, C.

This is not an issue with the current compiler as it always add the "next" link from C to D, but this will become an issue when we use a more powerful CFG based optimizer than the current "peephole" optimizer.

----------
assignee: Mark.Shannon
components: Interpreter Core
messages: 352114
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Depth first search in compile.c creates wrong BB order for certain CFG.
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38135>
_______________________________________

From report at bugs.python.org  Thu Sep 12 08:40:20 2019
From: report at bugs.python.org (Lisa Roach)
Date: Thu, 12 Sep 2019 12:40:20 +0000
Subject: [New-bugs-announce] [issue38136] Remove AsyncMock.assert_awaited_*
Message-ID: <1568292020.39.0.826451235958.issue38136@roundup.psfhosted.org>


New submission from Lisa Roach <lisaroach14 at gmail.com>:

After some discussion about call_count vs await_count, I believe call_count should be counting when things are *awaited* (await foo()), not when they are *called* (foo()). 

I think people expect "calling" to execute the code and give them a return_value, which for asyncio is what happens when you await, not when you call with (). If people disagree about this I am open to discussion, we can change the current functionality and leave in the assert_awaited_* calls.

Currently the code does count asyncio calls when they are awaited, but this makes the assert_awaited_* calls redundant. 

We should remove these in favor of the call_count_* functions.

----------
assignee: lisroach
components: Library (Lib)
messages: 352144
nosy: cjw296, ezio.melotti, lisroach, michael.foord, xtreak
priority: normal
severity: normal
status: open
title: Remove AsyncMock.assert_awaited_*
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38136>
_______________________________________

From report at bugs.python.org  Thu Sep 12 09:36:09 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Thu, 12 Sep 2019 13:36:09 +0000
Subject: [New-bugs-announce] [issue38137] hashopenssl fails to compile on
 MacOS
Message-ID: <1568295369.63.0.535075391802.issue38137@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

I get this when compiling CPython master

Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:89:19: error: implicit declaration of function 'EVP_MD_CTX_new' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    retval->ctx = EVP_MD_CTX_new();
                  ^
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:89:19: warning: this function declaration is not a prototype [-Wstrict-prototypes]
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:89:17: warning: incompatible integer to pointer conversion assigning to 'EVP_MD_CTX *' (aka 'struct env_md_ctx_st *') from 'int' [-Wint-conversion]
    retval->ctx = EVP_MD_CTX_new();
                ^ ~~~~~~~~~~~~~~~~
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:126:5: error: implicit declaration of function 'EVP_MD_CTX_free' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    EVP_MD_CTX_free(self->ctx);
    ^
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:126:5: note: did you mean 'EVP_MD_CTX_create'?
/usr/local/opt/openssl/include/openssl/evp.h:588:13: note: 'EVP_MD_CTX_create' declared here
EVP_MD_CTX *EVP_MD_CTX_create(void);
            ^
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:126:5: warning: this function declaration is not a prototype [-Wstrict-prototypes]
    EVP_MD_CTX_free(self->ctx);
    ^
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:179:16: error: implicit declaration of function 'EVP_MD_CTX_new' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    temp_ctx = EVP_MD_CTX_new();
               ^
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:179:14: warning: incompatible integer to pointer conversion assigning to 'EVP_MD_CTX *' (aka 'struct env_md_ctx_st *') from 'int' [-Wint-conversion]
    temp_ctx = EVP_MD_CTX_new();
             ^ ~~~~~~~~~~~~~~~~
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:195:5: error: implicit declaration of function 'EVP_MD_CTX_free' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    EVP_MD_CTX_free(temp_ctx);
    ^
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:213:16: error: implicit declaration of function 'EVP_MD_CTX_new' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    temp_ctx = EVP_MD_CTX_new();
               ^
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:213:14: warning: incompatible integer to pointer conversion assigning to 'EVP_MD_CTX *' (aka 'struct env_md_ctx_st *') from 'int' [-Wint-conversion]
    temp_ctx = EVP_MD_CTX_new();
             ^ ~~~~~~~~~~~~~~~~
/Users/pgalindo3/github/cpython/Modules/_hashopenssl.c:229:5: error: implicit declaration of function 'EVP_MD_CTX_free' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    EVP_MD_CTX_free(temp_ctx);


? openssl version
LibreSSL 2.6.5

----------
components: Build
messages: 352161
nosy: christian.heimes, pablogsal
priority: normal
severity: normal
status: open
title: hashopenssl fails to compile on MacOS
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38137>
_______________________________________

From report at bugs.python.org  Thu Sep 12 09:37:39 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Thu, 12 Sep 2019 13:37:39 +0000
Subject: [New-bugs-announce] [issue38138] test_importleak is leaking
 references
Message-ID: <1568295459.95.0.00413440178424.issue38138@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

? ./python.exe -m test test_importlib -R 3:3
Run tests sequentially
0:00:00 load avg: 1.45 [1/1] test_importlib
beginning 6 repetitions
123456
......
test_importlib leaked [20, 20, 20] references, sum=60
test_importlib failed

== Tests result: FAILURE ==

1 test failed:
    test_importlib

Total duration: 23 sec 179 m

----------
components: Tests
messages: 352162
nosy: brett.cannon, pablogsal
priority: normal
severity: normal
status: open
title: test_importleak is leaking references
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38138>
_______________________________________

From report at bugs.python.org  Thu Sep 12 09:48:05 2019
From: report at bugs.python.org (Yixing Jiang)
Date: Thu, 12 Sep 2019 13:48:05 +0000
Subject: [New-bugs-announce] [issue38139] [BUG] datetime.strptime could not
 handle timezone
Message-ID: <1568296085.75.0.40792729081.issue38139@roundup.psfhosted.org>


New submission from Yixing Jiang <johnestar.wag at gmail.com>:

datetime.datetime.strptime throws out an error for me, to reproduce, use the following block:

```
>>> import datetime
>>> import pytz
>>> d2 = datetime.datetime(2019, 9, 11, 18, 35, 17, 334000, tzinfo=pytz.timezone('CET'))
>>> d2.strftime('%Y%m%d%H%M%S%f%Z')
'20190911183517334000CET'
>>> datetime.datetime.strptime('20190911183517334000CET', '%Y%m%d%H%M%S%f%Z')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/envs/lab42/lib/python3.6/_strptime.py", line 565, in _strptime_datetime
    tt, fraction = _strptime(data_string, format)
  File "/opt/conda/envs/lab42/lib/python3.6/_strptime.py", line 362, in _strptime
    (data_string, format))
ValueError: time data '20190911183517334000CET' does not match format '%Y%m%d%H%M%S%f%Z'
>>> datetime.datetime.strptime('20190911183517334000', '%Y%m%d%H%M%S%f')
datetime.datetime(2019, 9, 11, 18, 35, 17, 334000)
```

so strptime could only handle the format without %Z.

----------
components: Library (Lib)
messages: 352163
nosy: Yixing Jiang
priority: normal
severity: normal
status: open
title: [BUG] datetime.strptime could not handle timezone
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38139>
_______________________________________

From report at bugs.python.org  Thu Sep 12 10:04:20 2019
From: report at bugs.python.org (Dino Viehland)
Date: Thu, 12 Sep 2019 14:04:20 +0000
Subject: [New-bugs-announce] [issue38140] Py_tp_dictoffset / Py_tp_finalize
 are unsettable in stable API
Message-ID: <1568297060.68.0.400472303762.issue38140@roundup.psfhosted.org>


New submission from Dino Viehland <dinoviehland at gmail.com>:

This makes it impossible to port certain types to the stable ABI and remove statics from the interpreter.

----------
assignee: dino.viehland
components: Interpreter Core
messages: 352167
nosy: dino.viehland, eric.snow
priority: normal
severity: normal
status: open
title: Py_tp_dictoffset / Py_tp_finalize are unsettable in stable API
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38140>
_______________________________________

From report at bugs.python.org  Thu Sep 12 10:59:42 2019
From: report at bugs.python.org (Eric Snow)
Date: Thu, 12 Sep 2019 14:59:42 +0000
Subject: [New-bugs-announce] [issue38141] Use less statics in Argument
 Clinic.
Message-ID: <1568300382.39.0.517598151629.issue38141@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

(This is a sub-task of bpo-36876, "Global C variables are a problem.".)

Currently Argument Clinic generates "_PyArg_Parser _parser" as a static variable.  Dropping "static" solves the problem of thread safety (e.g. for subinterpreters not sharing the GIL).

----------
assignee: eric.snow
components: Argument Clinic
messages: 352193
nosy: eric.snow, larry
priority: normal
severity: normal
stage: needs patch
status: open
title: Use less statics in Argument Clinic.
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38141>
_______________________________________

From report at bugs.python.org  Thu Sep 12 12:40:02 2019
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 12 Sep 2019 16:40:02 +0000
Subject: [New-bugs-announce] [issue38142] Make _hashlib extension module
 PEP-384 compatible
Message-ID: <1568306402.98.0.457651944962.issue38142@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

Updated _hashopenssl.c to be PEP 384 compliant.

----------
assignee: christian.heimes
components: Extension Modules
messages: 352214
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: Make _hashlib extension module PEP-384 compatible
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38142>
_______________________________________

From report at bugs.python.org  Thu Sep 12 13:40:30 2019
From: report at bugs.python.org (Jon Janzen)
Date: Thu, 12 Sep 2019 17:40:30 +0000
Subject: [New-bugs-announce] [issue38143] plistlib: cleanup PEP8 violations
Message-ID: <1568310030.71.0.660066153188.issue38143@roundup.psfhosted.org>


New submission from Jon Janzen <jjjonjanzen at gmail.com>:

There are a series of PEP8 violations in plistlib.

Most of them are spacing issues, a few naming issues, and also some unused function parameters and unused imports that can be done away with.

----------
components: Library (Lib)
messages: 352221
nosy: bigfootjon, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: plistlib: cleanup PEP8 violations
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38143>
_______________________________________

From report at bugs.python.org  Thu Sep 12 14:31:01 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Sep 2019 18:31:01 +0000
Subject: [New-bugs-announce] [issue38144] Add the root_dir and dir_fd
 parameters in glob.glob()
Message-ID: <1568313061.34.0.998675763501.issue38144@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The pattern for the glob() function can be relative and absolute. If it is relative, paths are searched from the current directory. If you want to search them in other directory, you need either change the current directory (which affects other parts of your program) or pass a concatenation of the escaped root directory and a pattern:

   glob.glob(os.path.join(glob.escape(root_dir), pattern))

Most code (even in the stdlib and tools) forget to escape the root directory. It works only until it does not contain metacharacters ('*?[').

When you need paths relative to the root directory, you need to "remove" the root_dir prefix from results (using os.path.relpath() at best).

The proposed PR adds two new parameters in glob.glob() and glob.iglob(): root_dir and dir_fd.

root_dir specifies the root directory for relative pattern. Its effect is the same as chdir before calling glob(). It is similar to the root_dir parameter of shutil.make_archive. For example, you can add py-files in the specified directory to the ZIP archive by:

    with zipfile.ZipFile(archive, 'w') as zf:
        for filename in glob.glob('**/*.py', recursive=True, root_dir=root_dir):
            zf.write(os.path.join(root_dir, filename), arcname=filename)

Adding to archive and copying are simpler if you have paths relative to the specified directory.

The dir_fd parameter is similar to root_dir, but it specifies the root directory as an open directory descriptor instead of a path (as in many os functions). It adds security (nobody can rename and replace the root directory in process) and performance (because of shorter paths).

root_dir and dir_fd can be combined. root_dir is relative to dir_fd and the pattern is relative to root_dir.

If root_dir is absolute, dir_fd is ignored. If the pattern is absolute, root_dir and dir_fd are ignored.

----------
components: Library (Lib)
messages: 352223
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add the root_dir and dir_fd parameters in glob.glob()
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38144>
_______________________________________

From report at bugs.python.org  Thu Sep 12 14:32:03 2019
From: report at bugs.python.org (sinoroc)
Date: Thu, 12 Sep 2019 18:32:03 +0000
Subject: [New-bugs-announce] [issue38145] Fix short option 'd' in 'scommand
 'bdist_dumb'
Message-ID: <1568313123.54.0.58284352566.issue38145@roundup.psfhosted.org>


New submission from sinoroc <sinoroc.code+python at gmail.com>:

In the 'distutils' command 'bdist_dumb', the short option 'd' is used
for both the 'bdist-dir' and 'dist-dir' options.

The bdist-dir option appeared first on 2000-05-13 in commit:
ba0506b

The dist-dir option appeared then on 2000-07-05 in commit:
c4eb84a

There appears to have been no version released between these two
commits, so most likely the global behaviour of the command has
stayed consistent.

The short option d actually triggers the dist-dir option, not the
bdist-dir option.

It is therefore safe to change the short option for bdist-dir. A
choice consistent with other similar distutils commands is 'b'.

----------
components: Distutils
messages: 352224
nosy: dstufft, eric.araujo, sinoroc
priority: normal
pull_requests: 15695
severity: normal
status: open
title: Fix short option 'd' in 'scommand 'bdist_dumb'
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38145>
_______________________________________

From report at bugs.python.org  Thu Sep 12 15:25:44 2019
From: report at bugs.python.org (John Midgett)
Date: Thu, 12 Sep 2019 19:25:44 +0000
Subject: [New-bugs-announce] [issue38146] QVariant NULL returns anomalous
 values in equality statements
Message-ID: <1568316344.45.0.265967787088.issue38146@roundup.psfhosted.org>


New submission from John Midgett <jamidgett at gmail.com>:

Per PEP 8, equality comparisons to None should use the "is" or "is not" keywords. However, if x is a {QVariant}NULL, the equality statement with the "is" keyword returns FALSE whereas the "==" comparison returns TRUE.  

Hence where cond = {QVariant}NULL:

    if cond is None  # Returns False
    if cond == None  # Returns True

This can cause some unexpected code anomalies (as it did for me) when adjusting code for PEP 8 compliance.

----------
components: Interpreter Core
messages: 352227
nosy: John Midgett
priority: normal
severity: normal
status: open
title: QVariant NULL returns anomalous values in equality statements
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38146>
_______________________________________

From report at bugs.python.org  Thu Sep 12 15:34:23 2019
From: report at bugs.python.org (Sergey Fedoseev)
Date: Thu, 12 Sep 2019 19:34:23 +0000
Subject: [New-bugs-announce] [issue38147] add macro for __builtin_unreachable
Message-ID: <1568316863.5.0.507617055887.issue38147@roundup.psfhosted.org>


New submission from Sergey Fedoseev <fedoseev.sergey at gmail.com>:

GCC (along with Clang and ICC) has __builtin_unreachable() and MSVC has __assume() builtins, that can be used to optimize out unreachable conditions.

So we could add macro like this:

#ifdef Py_DEBUG
#   define Py_ASSUME(cond) (assert(cond))
#else
#   if defined(_MSC_VER)
#       define Py_ASSUME(cond) (__assume(cond))
#   elif defined(__GNUC__)
#       define Py_ASSUME(cond) (cond? (void)0: __builtin_unreachable())
#   else
#       define Py_ASSUME(cond) ((void)0);
#   endif
#endif

Here's a pair of really simple examples showing how it can optimize code: https://godbolt.org/z/g9LYXF.

Real world example. _PyLong_Copy() [1] calls _PyLong_New() [2]. _PyLong_New() checks the size, so that overflow does not occur. This check is redundant when _PyLong_New() is called from _PyLong_Copy(). We could add a function that bypass that check, but in LTO build PyObject_MALLOC() is inlined into _PyLong_New() and it also checks the size. Adding Py_ASSUME((size_t)size <= MAX_LONG_DIGITS) allows to bypass both checks. 

[1] https://github.com/python/cpython/blob/3a4f66707e824ef3a8384827590ebaa6ca463dc0/Objects/longobject.c#L287-L309
[2] https://github.com/python/cpython/blob/3a4f66707e824ef3a8384827590ebaa6ca463dc0/Objects/longobject.c#L264-L283

----------
messages: 352228
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: add macro for __builtin_unreachable

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38147>
_______________________________________

From report at bugs.python.org  Fri Sep 13 01:54:41 2019
From: report at bugs.python.org (Andrew Svetlov)
Date: Fri, 13 Sep 2019 05:54:41 +0000
Subject: [New-bugs-announce] [issue38148] Add slots to asyncio transports
Message-ID: <1568354081.06.0.30849196857.issue38148@roundup.psfhosted.org>


New submission from Andrew Svetlov <andrew.svetlov at gmail.com>:

Transports are abstract interfaces.
Adding slots allows using slots in transport implementations as well.
If the implementation doesn't use slots -- everything keeps working.

Slots can help with saving a memory footprint a little.

----------
components: asyncio
messages: 352239
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Add slots to asyncio transports
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38148>
_______________________________________

From report at bugs.python.org  Fri Sep 13 05:07:51 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Sep 2019 09:07:51 +0000
Subject: [New-bugs-announce] [issue38149] sys.audit() is called multiple
 times for glob.glob()
Message-ID: <1568365671.05.0.0499818085141.issue38149@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

sys.audit() for "glob.glob" is called in a recursive function _iglob(). So in process of executing glob.glob() it can be called multiple times, with the original pattern and with patterns for parent directories, while there are metacharacters in them.

I do not think it was intentional.

----------
components: Library (Lib)
messages: 352248
nosy: serhiy.storchaka, steve.dower
priority: normal
severity: normal
status: open
title: sys.audit() is called multiple times for glob.glob()
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38149>
_______________________________________

From report at bugs.python.org  Fri Sep 13 05:15:22 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Sep 2019 09:15:22 +0000
Subject: [New-bugs-announce] [issue38150] test_capi:
 test_c_subclass_of_heap_ctype_with_del_modifying_dunder_class_only_decrefs_once()
 leaks
Message-ID: <1568366122.45.0.513132018706.issue38150@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

vstinner at apu$ ./python -m test test_capi -R 3:3 -m test.test_capi.CAPITest.test_c_subclass_of_heap_ctype_with_del_modifying_dunder_class_only_decrefs_once
Run tests sequentially
0:00:00 load avg: 0.57 [1/1] test_capi
beginning 6 repetitions
123456
......
test_capi leaked [2, 2, 2] references, sum=6
test_capi failed

== Tests result: FAILURE ==

1 test failed:
    test_capi

Total duration: 153 ms
Tests result: FAILURE


Note: Can't we find a shorter test name please?

----------
components: Tests
messages: 352249
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_capi: test_c_subclass_of_heap_ctype_with_del_modifying_dunder_class_only_decrefs_once() leaks
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38150>
_______________________________________

From report at bugs.python.org  Fri Sep 13 05:22:43 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Sep 2019 09:22:43 +0000
Subject: [New-bugs-announce] [issue38151] test_types: test_duck_gen() leaks
 reference
Message-ID: <1568366563.55.0.281787409342.issue38151@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

https://buildbot.python.org/all/#/builders/1/builds/712

Python 3.8 is not affected, only the master branch.

See also bpo-38150.

vstinner at apu$ ./python -m test test_types -R 3:3 -m test.test_types.CoroutineTests.test_duck_gen
Run tests sequentially
0:00:00 load avg: 0.89 [1/1] test_types
beginning 6 repetitions
123456
......
test_types leaked [7, 7, 7] references, sum=21
test_types failed

== Tests result: FAILURE ==

1 test failed:
    test_types

Total duration: 472 ms
Tests result: FAILURE

----------
components: Tests
messages: 352250
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_types: test_duck_gen() leaks reference
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38151>
_______________________________________

From report at bugs.python.org  Fri Sep 13 05:37:08 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Sep 2019 09:37:08 +0000
Subject: [New-bugs-announce] [issue38152] AST change introduced tons of
 reference leaks
Message-ID: <1568367428.12.0.614565351268.issue38152@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

The following change introduced tons of reference leaks:

commit ac46eb4ad6662cf6d771b20d8963658b2186c48c
Author: Dino Viehland <dinoviehland at fb.com>
Date:   Wed Sep 11 10:16:34 2019 -0700

    bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)
    
    Summary: This mostly migrates Python-ast.c to PEP384 and removes all statics from the whole file. This modifies the generator itself that generates the Python-ast.c. It leaves in the usage of _PyObject_LookupAttr even though it's not fully PEP384 compatible (this could always be shimmed in by anyone who needs it).


Full list of tests which leaked in build:
https://buildbot.python.org/all/#/builders/1/builds/712

test_ast
test_asyncio
test_builtin
test_capi
test_check_c_globals
test_clinic
test_compile
test_dataclasses
test_dbm
test_dbm_dumb
test_decimal
test_fstring
test_getpass
test_idle
test_importlib
test_inspect
test_pydoc
test_source_encoding
test_symtable
test_sys
test_type_comments
test_types
test_unittest


vstinner at apu$ ./python -m test -j0 -R 3:3 --fromfile=leak
(...)
test_dbm_dumb leaked [938, 896, 908] references, sum=2742
test_dbm leaked [144, 144, 144] references, sum=432
test_builtin leaked [128, 128, 128] references, sum=384
test_clinic leaked [1935, 1933, 1935] references, sum=5803
test_getpass leaked [21, 21, 21] references, sum=63
test_dataclasses leaked [8, 8, 8] references, sum=24
test_idle leaked [69, 69, 69] references, sum=207
test_inspect leaked [539, 539, 539] references, sum=1617
test_fstring leaked [104, 104, 104] references, sum=312
test_type_comments leaked [1337, 1335, 1337] references, sum=4009
test_types leaked [7, 7, 7] references, sum=21
test_compile leaked [3690, 3688, 3690] references, sum=11068
test_sys leaked [1, 1, 1] references, sum=3
test_pydoc leaked [1504, 1502, 1504] references, sum=4510
test_ast leaked [357792, 357790, 357792] references, sum=1073374
test_ast leaked [12, 11, 12] memory blocks, sum=35
test_unittest leaked [3803, 3801, 3803] references, sum=11407
test_decimal leaked [841, 841, 841] references, sum=2523
test_capi leaked [2, 2, 2] references, sum=6

I interrupted test_asyncio test: it's too slow.


Reverting the commit ac46eb4ad6662cf6d771b20d8963658b2186c48c fix most reference leaks, except of test_capi: bpo-38150.

----------
components: Interpreter Core
messages: 352258
nosy: vstinner
priority: normal
severity: normal
status: open
title: AST change introduced tons of reference leaks
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38152>
_______________________________________

From report at bugs.python.org  Fri Sep 13 05:50:32 2019
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 13 Sep 2019 09:50:32 +0000
Subject: [New-bugs-announce] [issue38153] Normalize hashing algorithm names
Message-ID: <1568368232.54.0.0384008471773.issue38153@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

The hashlib module / PEP 452 and OpenSSL use slightly different conventions for hashing algorithm names. The old and common algorithms like md5 to sha512 use the same strings (all lower case, no dashes or underscores). But new algorithms like sha3_512, shake, sha512_256, and blake2 use different conventions.

The inconsistency bloats the list of available algorithms. Also the builtin OpenSSL constructor does not support Python's preferred names.

>>> import hashlib, _hashlib
>>> sorted(hashlib.algorithms_available)
['blake2b', 'blake2b512', 'blake2s', 'blake2s256', 'md4', 'md5', 'md5-sha1', 'ripemd160', 'sha1', 'sha224', 'sha256', 'sha3-224', 'sha3-256', 'sha3-384', 'sha3-512', 'sha384', 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512', 'sha512', 'sha512-224', 'sha512-256', 'shake128', 'shake256', 'shake_128', 'shake_256', 'sm3', 'whirlpool']
>>> _hashlib.new("sha3_512")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unsupported hash type
>>> _hashlib.new("sha3-512")
<sha3-512 HASH object @ 0x7f1387890840>

I propose to normalize names to Python standard names for HASH.name, repr, list of available algorithms, and for the new() constructor.

----------
assignee: christian.heimes
components: Extension Modules
messages: 352270
nosy: christian.heimes, gregory.p.smith
priority: normal
severity: normal
status: open
title: Normalize hashing algorithm names
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38153>
_______________________________________

From report at bugs.python.org  Fri Sep 13 05:50:39 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Sep 2019 09:50:39 +0000
Subject: [New-bugs-announce] [issue38154] test__xxsubinterpreters: random
 failures on AMD64 FreeBSD CURRENT Shared 3.x
Message-ID: <1568368239.98.0.217777678278.issue38154@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

It seems like test__xxsubinterpreters still has race conditions.

See also bpo-37224 and bpo-33356.

AMD64 FreeBSD CURRENT Shared 3.x:
https://buildbot.python.org/all/#/builders/168/builds/1428

FAIL: test_subinterpreter (test.test__xxsubinterpreters.IsRunningTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test__xxsubinterpreters.py", line 492, in test_subinterpreter
    self.assertTrue(interpreters.is_running(interp))
AssertionError: False is not true

Later, when test__xxsubinterpreters is re-run in verbose mode:

FAIL: test_still_running (test.test__xxsubinterpreters.DestroyTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test__xxsubinterpreters.py", line 765, in test_still_running
    interpreters.destroy(interp)
AssertionError: RuntimeError not raised

----------
components: Tests
messages: 352271
nosy: eric.snow, vstinner
priority: normal
severity: normal
status: open
title: test__xxsubinterpreters: random failures on AMD64 FreeBSD CURRENT Shared 3.x
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38154>
_______________________________________

From report at bugs.python.org  Fri Sep 13 06:23:10 2019
From: report at bugs.python.org (Paul Ganssle)
Date: Fri, 13 Sep 2019 10:23:10 +0000
Subject: [New-bugs-announce] [issue38155] Add __all__ to the datetime module
Message-ID: <1568370190.96.0.124167415349.issue38155@roundup.psfhosted.org>


New submission from Paul Ganssle <p.ganssle at gmail.com>:

Currently the datetime module has no __all__, which means we only advertise what is public and private based on leading underscores. Additionally, because there are two implementations (Python and C), you actually get different things when you do `from datetime import *` depending on whether you have the C module installed or not.

The "easy" part is to add an __all__ variable to Lib/datetime.py for all the documented attributes:

  __all__ = ["date", "datetime", "time", "timedelta", "timezone", "tzinfo", "MINYEAR", "MAXYEAR"]

A "stretch goal" would be to add a test to ensure that `from datetime import *` imports the same set of symbols from the pure python module that it does from the C module. I haven't quite thought through how this would be achieved, probably something in test_datetime (https://github.com/python/cpython/blob/6a517c674907c195660fa9178a7b561de49cc721/Lib/test/test_datetime.py#L1), where we need to import both modules anyway. I think we can accept an "add __all__" PR without tests, though.

----------
components: Library (Lib)
keywords: newcomer friendly
messages: 352280
nosy: belopolsky, p-ganssle
priority: normal
severity: normal
stage: needs patch
status: open
title: Add __all__ to the datetime module
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38155>
_______________________________________

From report at bugs.python.org  Fri Sep 13 06:31:38 2019
From: report at bugs.python.org (Amiy Kumar)
Date: Fri, 13 Sep 2019 10:31:38 +0000
Subject: [New-bugs-announce] [issue38156] input fucntion raises SystemError
 after specific input.
Message-ID: <1568370698.83.0.595206707223.issue38156@roundup.psfhosted.org>


New submission from Amiy Kumar <amiy90067 at gmail.com>:

input function raises a SystemError in bash(similar shells)
when given inputs follow a certain pattern, doesn't happen in python interactive shell.

It causes crash in python scripts even if KeyboardInterrupt and EOFError are handeled.

No EOFError or KeyboardInterrupt error is raised in such cases.

Tested with python script: (file input-test.py)
and command:
python -c 'input()'
both yield same result.

Inputs are represented as: ^D :- CTRL+d, ^C :- CTRL+c 

eg:
input:
sometext^D^D^C

Traceback:
[182]$ python -c 'input()'
sometext^CKeyboardInterrupt

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
SystemError: <built-in function input> returned a result with an error set

-----

inputs:
sometext^D^C^C [or even] sometext^Dsome-more^C^C

Traceback:
[191]$ python input-test.py
sometext^C^CKeyboardInterrupt

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "input-test.py", line 2, in <module>
    tmp = input()
SystemError: <built-in function input> returned a result with an error set

Python version: 3.7
Bash version: 5.0.11

----------
components: IO
files: input-test.py
messages: 352282
nosy: Amiy Kumar
priority: normal
severity: normal
status: open
title: input fucntion raises SystemError after specific input.
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file48606/input-test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38156>
_______________________________________

From report at bugs.python.org  Fri Sep 13 06:54:43 2019
From: report at bugs.python.org (Karthikeyan Singaravelan)
Date: Fri, 13 Sep 2019 10:54:43 +0000
Subject: [New-bugs-announce] [issue38157] Add a recipe in unittest.mock
 examples about mock_open per file
Message-ID: <1568372083.76.0.737420692229.issue38157@roundup.psfhosted.org>


New submission from Karthikeyan Singaravelan <tir.karthi at gmail.com>:

With issue37669 it was proposed to refactor out the mock_open handler to return different mocks per file and an API change to make sure read_data accepts a dictionary of file and return values it can only land on master if accepter. It's already possible now with using side_effect to return per file content. Adding it would be a good example like below so that users can know this usage. I can prepare a PR for this.


from unittest.mock import mock_open, patch

DEFAULT_MOCK_DATA = "default mock data"
data_dict = {"file1": "data1",
             "file2": "data2"}

def open_side_effect(name):
    return mock_open(read_data=data_dict.get(name, DEFAULT_MOCK_DATA))()

with patch(f"{__name__}.open", side_effect=open_side_effect):
    with open("file1") as file1:
        assert file1.read() == "data1"

        with open("file2") as file2:
            assert file2.read() == "data2"

            with open("file1") as file3:
                assert file3.read(1) == "d"

        assert file1.read() == ""

    with open("defaultfile") as file4:
        assert file4.read() == "default mock data"

----------
assignee: docs at python
components: Documentation
messages: 352285
nosy: cjw296, docs at python, lisroach, mariocj89, michael.foord, xtreak
priority: normal
severity: normal
status: open
title: Add a recipe in unittest.mock examples about mock_open per file
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38157>
_______________________________________

From report at bugs.python.org  Fri Sep 13 07:06:37 2019
From: report at bugs.python.org (Stefan Behnel)
Date: Fri, 13 Sep 2019 11:06:37 +0000
Subject: [New-bugs-announce] [issue38158] PyType_Spec docs list non-existant
 field "doc"
Message-ID: <1568372797.52.0.257422503496.issue38158@roundup.psfhosted.org>


New submission from Stefan Behnel <stefan_ml at behnel.de>:

The current documentation of the PyType_Spec struct lists a "doc" member which does not exist. It should be removed from the docs.
https://docs.python.org/3.8/c-api/type.html#c.PyType_Spec

----------
assignee: docs at python
components: Documentation
keywords: easy, newcomer friendly
messages: 352288
nosy: docs at python, scoder
priority: normal
severity: normal
status: open
title: PyType_Spec docs list non-existant field "doc"
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38158>
_______________________________________

From report at bugs.python.org  Fri Sep 13 09:25:39 2019
From: report at bugs.python.org (Petr Viktorin)
Date: Fri, 13 Sep 2019 13:25:39 +0000
Subject: [New-bugs-announce] [issue38159] PyState_AddModule docs should say
 that it's not necessary to call it.
Message-ID: <1568381139.47.0.152206625825.issue38159@roundup.psfhosted.org>


New submission from Petr Viktorin <encukou at gmail.com>:

I've seen extension modules calling PyState_AddModule on init, despite  Python doing this automatically.
It should be clarified that PyState_AddModule is for people who are writing alternative ways of creating/registering module objects.

----------
messages: 352320
nosy: petr.viktorin
priority: normal
severity: normal
status: open
title: PyState_AddModule docs should say that it's not necessary to call it.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38159>
_______________________________________

From report at bugs.python.org  Fri Sep 13 09:33:29 2019
From: report at bugs.python.org (Eric Snow)
Date: Fri, 13 Sep 2019 13:33:29 +0000
Subject: [New-bugs-announce] [issue38160] Add a "PyInterpreterState *" field
 to PyTypeObject.
Message-ID: <1568381609.99.0.399978002813.issue38160@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

A "PyInterpreterState *interp" field on PyTypeObject would allow us to quickly access the originating interpreter from any object.  This will help us avoid more costly lookups.  The additional pointer used for this should not have a significant impact on the memory usage of CPython.

----------
components: Interpreter Core
messages: 352325
nosy: eric.snow, petr.viktorin, scoder, steve.dower
priority: normal
severity: normal
stage: needs patch
status: open
title: Add a "PyInterpreterState *" field to PyTypeObject.
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38160>
_______________________________________

From report at bugs.python.org  Fri Sep 13 11:49:28 2019
From: report at bugs.python.org (Lisa Roach)
Date: Fri, 13 Sep 2019 15:49:28 +0000
Subject: [New-bugs-announce] [issue38161] AsyncMock add `.awaited` like
 `.called`
Message-ID: <1568389768.59.0.850541174467.issue38161@roundup.psfhosted.org>


New submission from Lisa Roach <lisaroach14 at gmail.com>:

Currently Mock has a `.called` attribute, we could add an equivalent `.awaited` that returns True or False if the object has been awaited on.

----------
messages: 352345
nosy: lisroach
priority: normal
severity: normal
status: open
title: AsyncMock add `.awaited` like `.called`

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38161>
_______________________________________

From report at bugs.python.org  Fri Sep 13 12:53:57 2019
From: report at bugs.python.org (Evgeniy Mischuk)
Date: Fri, 13 Sep 2019 16:53:57 +0000
Subject: [New-bugs-announce] [issue38162] Import Long Load
Message-ID: <1568393637.17.0.162733973896.issue38162@roundup.psfhosted.org>


New submission from Evgeniy Mischuk <evgeniy.documents at gmail.com>:

for in TMP_MAX dont break and always continue

----------
components: Windows
messages: 352365
nosy: Evgeniy Mischuk, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
pull_requests: 15733
severity: normal
status: open
title: Import Long Load
type: performance
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38162>
_______________________________________

From report at bugs.python.org  Fri Sep 13 13:05:04 2019
From: report at bugs.python.org (Lisa Roach)
Date: Fri, 13 Sep 2019 17:05:04 +0000
Subject: [New-bugs-announce] [issue38163] AsyncMock child mocks should
 detect their type
Message-ID: <1568394304.53.0.898247355535.issue38163@roundup.psfhosted.org>


New submission from Lisa Roach <lisaroach14 at gmail.com>:

We want child mocks to match their "type" (async or sync). 

Ex:

class Prod:
   async def foo():
      pass
    def bar():
       pass

a_mock = AsyncMock(Prod)
isinstance(a_mock.foo(), AsyncMock) == True
isinstance(a_mock.bar(), MagicMock) == True

Also this should include magic methods:

int(a_mock) should work and return 1 like MagicMocks do.
a_mock.__enter__ should exist and be a MagicMock, just as a_mock.__aenter__ exists and is an AsyncMock.

----------
messages: 352369
nosy: lisroach
priority: normal
severity: normal
status: open
title: AsyncMock child mocks should detect their type

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38163>
_______________________________________

From report at bugs.python.org  Fri Sep 13 13:11:32 2019
From: report at bugs.python.org (Yury Selivanov)
Date: Fri, 13 Sep 2019 17:11:32 +0000
Subject: [New-bugs-announce] [issue38164] poloshing asyncio Streams API
Message-ID: <1568394692.95.0.46825100333.issue38164@roundup.psfhosted.org>


New submission from Yury Selivanov <yselivanov at gmail.com>:

Summary of discussion at https://github.com/python-trio/trio/issues/1208:

* `asyncio.Stream.write()` will start throwing a `DeprecationWarning` asking people to add an `await` if they didn't;

* `asyncio.Stream.close()` will start throwing a `DeprecationWarning` asking people to add an `await` if they didn't;

* `asyncio.Stream.drain()` & `asyncio.Stream.wait_closed()` will start throwing a `DeprecationWarning` telling about a scheduled removal (in Python 3.12) when used on `Process.std*` streams;

* `asyncio.Stream.drain()` & `asyncio.Stream.wait_closed()` will not work at all on Streams created via new 3.8 APIs: `connect()` & `StreamServer`.

----------
assignee: asvetlov
components: asyncio
messages: 352372
nosy: asvetlov, lukasz.langa, yselivanov
priority: release blocker
severity: normal
stage: needs patch
status: open
title: poloshing asyncio Streams API
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38164>
_______________________________________

From report at bugs.python.org  Fri Sep 13 13:50:05 2019
From: report at bugs.python.org (Evgeniy Mischuk)
Date: Fri, 13 Sep 2019 17:50:05 +0000
Subject: [New-bugs-announce] [issue38165] freezes when loading a program on
 imports
Message-ID: <1568397005.25.0.136695117075.issue38165@roundup.psfhosted.org>


New submission from Evgeniy Mischuk <evgeniy.documents at gmail.com>:

freezes when loading a program on imports because user rights are insufficient
python installed on c:/ and requires administrator rights
when run program from not root user

----------
components: Windows
messages: 352384
nosy: Evgeniy Mischuk, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: freezes when loading a program on imports
type: performance
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38165>
_______________________________________

From report at bugs.python.org  Fri Sep 13 17:48:54 2019
From: report at bugs.python.org (Justin McCann)
Date: Fri, 13 Sep 2019 21:48:54 +0000
Subject: [New-bugs-announce] [issue38166] ast identifies incorrect column
 for compound attribute lookups
Message-ID: <1568411334.98.0.109884794691.issue38166@roundup.psfhosted.org>


New submission from Justin McCann <jneilm at gmail.com>:

This issue is related to https://github.com/microsoft/vscode-python/issues/7327 and https://github.com/PyCQA/pylint/issues/3103

For compound attribute access in variable references like a.b.c.d, the AST reports the column of the first variable/attribute in the sequence instead of the specific attribute location. For example, the location of  c is reported as column 0 (a) and not column 4 (c).

Here's the AST test case provided by a pylint developer in a comment on pylint issue 3103; I confirmed the behavior is the same in python 3.8.0b4.
```
$ python3.8
Python 3.8.0b4 (v3.8.0b4:d93605de72, Aug 29 2019, 21:47:47)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> body = ast.parse('''
... print(x.item.akey > 2)
... ''')
>>> # x.item
... x_item = body.body[0].value.args[0].left.value
>>> print(x_item, x_item.attr, x_item.col_offset)
<_ast.Attribute object at 0x10a7751f0> item 6
>>> # x.item.akey
... print(x_item.value, x_item.value.col_offset) .  ### probably should be 8
<_ast.Name object at 0x10a775280> 6
```

Related issues:
* https://bugs.python.org/issue1440601 Add col information to parse & ast nodes
* https://bugs.python.org/issue10769 ast: provide more useful range information


Here is the resulting confusion when you use this output in pylint (and then VSCode highlights only "x" since it's the variable that starts in column 0):

Original pylint/vscode testcase:
```
class TestMe:
    def __init__(self):
        self.item = {'akey': 42}
        self.again = self

x = TestMe()
### pylint error message here is
###    testme.py:11:6: E1101: Instance of 'dict' has no 'akey' member (no-member)
### The problem is with `x.item`, but pylint shows the column for `x`
print(x.item.akey > 2)

print(x.again.item.doesnotexist)
```

Current behavior
$ pylint testme.py  -rn -sn
************* Module testme
testme.py:10:6: E1101: Instance of 'dict' has no 'akey' member (no-member)
testme.py:12:6: E1101: Instance of 'dict' has no 'doesnotexist' member (no-member)

Expected behavior
$ pylint testme.py  -rn -sn
************* Module testme
testme.py:10:8: E1101: Instance of 'dict' has no 'akey' member (no-member)
testme.py:12:14: E1101: Instance of 'dict' has no 'doesnotexist' member (no-member)

$ pylint --version output
pylint 2.3.1
astroid 2.2.5
Python 3.7.4 (default, Jul 9 2019, 18:13:23)
[Clang 10.0.1 (clang-1001.0.46.4)]

----------
components: Library (Lib)
messages: 352393
nosy: Justin McCann
priority: normal
severity: normal
status: open
title: ast identifies incorrect column for compound attribute lookups
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38166>
_______________________________________

From report at bugs.python.org  Fri Sep 13 19:28:40 2019
From: report at bugs.python.org (Paul)
Date: Fri, 13 Sep 2019 23:28:40 +0000
Subject: [New-bugs-announce] [issue38167] O_DIRECT read fails with 4K mmap
 buffer
Message-ID: <1568417320.49.0.42295605579.issue38167@roundup.psfhosted.org>


New submission from Paul <paul.carlisle at gmail.com>:

The following Python3 script fails.

import mmap
import os

fd = os.open(path_to_file, os.O_DIRECT | os.O_RDWR)
fo = os.fdopen(fd, 'rb+')
m = mmap.mmap(-1, 4096)
fo.readinto(m)

But it worked for Python2.  It also works for any other multiple of 4K. For example:

m = mmap.mmap(-1, 8192)
fo.readinto(m)

Is fine!

----------
components: IO
messages: 352397
nosy: yoyoyopcp
priority: normal
severity: normal
status: open
title: O_DIRECT read fails with 4K mmap buffer
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38167>
_______________________________________

From report at bugs.python.org  Sat Sep 14 02:17:23 2019
From: report at bugs.python.org (hai shi)
Date: Sat, 14 Sep 2019 06:17:23 +0000
Subject: [New-bugs-announce] [issue38168] Reflaks in setint() of mmapmodule.c
Message-ID: <1568441843.93.0.712377390707.issue38168@roundup.psfhosted.org>


New submission from hai shi <shihai1991 at 126.com>:

As the title said, in `setint` function, the refcount of `o` should decreased all the time.

----------
components: Interpreter Core
messages: 352412
nosy: shihai1991
priority: normal
severity: normal
status: open
title: Reflaks in setint() of mmapmodule.c
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38168>
_______________________________________

From report at bugs.python.org  Sat Sep 14 03:44:31 2019
From: report at bugs.python.org (Vinay Sharma)
Date: Sat, 14 Sep 2019 07:44:31 +0000
Subject: [New-bugs-announce] [issue38169] Increase Code Coverage for
 SharedMemory and ShareableListe
Message-ID: <1568447071.93.0.822548908149.issue38169@roundup.psfhosted.org>


New submission from Vinay Sharma <vinay0410sharma at gmail.com>:

Add Tests for SharedMemory and ShareableList. I have also added a check to prevent users from creating shared memory of size 0, because after creating mmap will throw error while memory mapping she of size 0.

----------
components: Tests
messages: 352420
nosy: vinay0410
priority: normal
severity: normal
status: open
title: Increase Code Coverage for SharedMemory and ShareableListe
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38169>
_______________________________________

From report at bugs.python.org  Sat Sep 14 09:19:47 2019
From: report at bugs.python.org (Dong-hee Na)
Date: Sat, 14 Sep 2019 13:19:47 +0000
Subject: [New-bugs-announce] [issue38170] imghdr: Support detecting heif
Message-ID: <1568467187.41.0.178563820912.issue38170@roundup.psfhosted.org>


New submission from Dong-hee Na <donghee.na92 at gmail.com>:

I've found that imghdr does not detect HEIF type yet.
I'd like to propose adding to detect HEIF type.

There's already real-life usage within HEIF
(https://help.dropbox.com/en-es/installs-integrations/mobile/ios-formats)

And also we can find a bunch of HEIF related programming issue such as Stack Overflow.
IMHO, supporting the detecting HEIF type is not a bad idea.

I want to hear core developers' opinion about this issue.


reference: https://github.com/strukturag/libheif/issues/83#issuecomment-421427091

----------
messages: 352428
nosy: corona10, rhettinger
priority: low
severity: normal
status: open
title: imghdr: Support detecting heif
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38170>
_______________________________________

From report at bugs.python.org  Sat Sep 14 09:35:27 2019
From: report at bugs.python.org (Patrick Monnerat)
Date: Sat, 14 Sep 2019 13:35:27 +0000
Subject: [New-bugs-announce] [issue38171] super() is passing wrong
 parameters while handling diamond inheritance
Message-ID: <1568468127.45.0.322825823369.issue38171@roundup.psfhosted.org>


New submission from Patrick Monnerat <patrick at monnerat.net>:

Running the attached program outputs:
--------
top.__init__(<__main__.top object at 0x7fc1dea24048>,) called
i1.__init__(<__main__.top object at 0x7fc1dea24048>, 'arg from top') called
i2.__init__(<__main__.top object at 0x7fc1dea24048>, 'arg from i1') called
base.__init__(<__main__.top object at 0x7fc1dea24048>, 'arg from i2') called
base.__init__ returns None
i2.__init__ returns None
i1.__init__ returns None
top.__init__ returns None
--------

i2.__init__() argument is wrong: since it is is not a parent class of i1, it should be "arg from top".

I can understand i2.__init__() is called after i1.__init__() and before base.__init__() but arguments given to super(i1, self).__init__() should not be substituted for calling i2.__init__().

----------
files: super.py
messages: 352429
nosy: monnerat
priority: normal
severity: normal
status: open
title: super() is passing wrong parameters while handling diamond inheritance
type: behavior
Added file: https://bugs.python.org/file48608/super.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38171>
_______________________________________

From report at bugs.python.org  Sat Sep 14 10:47:40 2019
From: report at bugs.python.org (Cooper Lees)
Date: Sat, 14 Sep 2019 14:47:40 +0000
Subject: [New-bugs-announce] [issue38172] Python 3.8 Segfult with
 Bandersnatch pytest Suite
Message-ID: <1568472460.87.0.817204992487.issue38172@roundup.psfhosted.org>


New submission from Cooper Lees <me at cooperlees.com>:

Haven't done research but bandersnatch's (https://github.com/pypa/bandersnatch) Unit tests pass in 3.7 and cause Python to Segmentation Fault in Python 3.8.0b4+.

```
py38 run-test: commands[0] | coverage run -m pytest
============================= test session starts ==============================
platform linux -- Python 3.8.0b4+, pytest-5.1.2, py-1.8.0, pluggy-0.13.0
cachedir: .tox/py38/.pytest_cache
rootdir: /home/travis/build/pypa/bandersnatch, inifile: pytest.ini
plugins: timeout-1.3.3
Fatal Python error: Segmentation fault
```

Full failure on Travis CI: https://travis-ci.org/pypa/bandersnatch/jobs/584973434

----------
messages: 352432
nosy: cooperlees
priority: normal
severity: normal
status: open
title: Python 3.8 Segfult with Bandersnatch pytest Suite
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38172>
_______________________________________

From report at bugs.python.org  Sat Sep 14 11:34:56 2019
From: report at bugs.python.org (Dong-hee Na)
Date: Sat, 14 Sep 2019 15:34:56 +0000
Subject: [New-bugs-announce] [issue38173] [EASY]
 tkinter.filedialog.askopenfilenames should follow PEP8
Message-ID: <1568475296.23.0.464650205482.issue38173@roundup.psfhosted.org>


New submission from Dong-hee Na <donghee.na92 at gmail.com>:

https://github.com/python/cpython/blob/39d87b54715197ca9dcb6902bb43461c0ed701a2/Lib/tkinter/filedialog.py#L397

This should be
options["multiple"]=1 -> options["multiple"] = 1

----------
components: Tkinter
messages: 352438
nosy: corona10, taleinat
priority: low
severity: normal
status: open
title: [EASY] tkinter.filedialog.askopenfilenames should follow PEP8
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38173>
_______________________________________

From report at bugs.python.org  Sat Sep 14 16:10:07 2019
From: report at bugs.python.org (Uche Ogbuji)
Date: Sat, 14 Sep 2019 20:10:07 +0000
Subject: [New-bugs-announce] [issue38174] Security vulnerability in bundled
 expat CVE-2019-15903 (fix available in expat 2.2.8)
Message-ID: <1568491807.22.0.03411253388.issue38174@roundup.psfhosted.org>


New submission from Uche Ogbuji <uche at ogbuji.net>:

cpython bundles expat in Modules/expat/ and needs to be updated to expat-2.2.8 to security vulnerability CVE-2019-15903.

>From Sebastian Pipping on XML-DEV ML:

Expat 2.2.8 [1] has been released yesterday.  This release fixes a
security issue ? a heap buffer over-read known as CVE-2019-15903 [2]
reported by Joonun Jang resulting in Denial of Service ?, starts using
the rand_s function on Windows and MinGW (ending the previous
LoadLibrary hack), includes non-security bugfixes, many build system
fixes and improvements, improvements to xmlwf usability, and more.

For more details regarding the latest release, please check out the
changelog [3].

If you maintain Expat packaging or a bundled copy of Expat or a pinned
version of Expat somewhere, please update to 2.2.8.  Thank you!

[1] https://github.com/libexpat/libexpat/releases/tag/R_2_2_8
[2] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-15903
[3] https://github.com/libexpat/libexpat/blob/R_2_2_8/expat/Changes

----------
components: XML
messages: 352449
nosy: Uche Ogbuji
priority: normal
severity: normal
status: open
title: Security vulnerability in bundled expat CVE-2019-15903 (fix available in expat 2.2.8)
type: security
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38174>
_______________________________________

From report at bugs.python.org  Sun Sep 15 03:29:07 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 15 Sep 2019 07:29:07 +0000
Subject: [New-bugs-announce] [issue38175] Memory leak in comparison of
 sqlite.Row objects
Message-ID: <1568532547.02.0.11116376901.issue38175@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

There is a memory leak in comparison of sqlite.Row objects when row descriptors are different. There were not tests for this case.

Also, the code compares the result of PyObject_RichCompare() with Py_True and Py_False. It is better to avoid such code, because technically PyObject_RichCompare() can return an arbitrary value, although in this particular case description can only be tuple or None (or NULL, but this is other issue).

Also, there is a test for inequality of hash codes. Since hashes depend on hashes of strings, they value is random, and there is a small chance of failure. Hashes should not be tested for inequality.

----------
components: Library (Lib)
messages: 352465
nosy: berker.peksag, ghaering, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Memory leak in comparison of sqlite.Row objects
type: resource usage
versions: Python 2.7, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38175>
_______________________________________

From report at bugs.python.org  Sun Sep 15 05:48:19 2019
From: report at bugs.python.org (STINNER Victor)
Date: Sun, 15 Sep 2019 09:48:19 +0000
Subject: [New-bugs-announce] [issue38176] test_threading leaked [1, 1,
 1] references: test_threads_join
Message-ID: <1568540899.7.0.49327876198.issue38176@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

vstinner at apu$ ./python -m test -R 3:3 test_threading -m test.test_threading.SubinterpThreadingTests.test_threads_join
Run tests sequentially
0:00:00 load avg: 1.11 [1/1] test_threading
beginning 6 repetitions
123456
......
test_threading leaked [1, 1, 1] references, sum=3
test_threading failed

== Tests result: FAILURE ==

1 test failed:
    test_threading

Total duration: 639 ms
Tests result: FAILURE

----------
components: Library (Lib)
messages: 352470
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_threading leaked [1, 1, 1] references: test_threads_join
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38176>
_______________________________________

From report at bugs.python.org  Sun Sep 15 05:56:26 2019
From: report at bugs.python.org (STINNER Victor)
Date: Sun, 15 Sep 2019 09:56:26 +0000
Subject: [New-bugs-announce] [issue38177] test_tools:
 test.test_tools.test_unparse.UnparseTestCase: test_tools leaked [36, 36,
 36] references
Message-ID: <1568541386.19.0.996414824495.issue38177@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

https://buildbot.python.org/all/#/builders/189/builds/153

$ ./python -m test -R 3:3 test_tools -m test.test_tools.test_unparse.UnparseTestCase.test_nonlocal
Run tests sequentially
0:00:00 load avg: 1.48 [1/1] test_tools
beginning 6 repetitions
123456
......
test_tools leaked [36, 36, 36] references, sum=108
test_tools failed

== Tests result: FAILURE ==

1 test failed:
    test_tools

Total duration: 591 ms
Tests result: FAILURE

----------
components: Tests
messages: 352473
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_tools: test.test_tools.test_unparse.UnparseTestCase: test_tools leaked [36, 36, 36] references
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38177>
_______________________________________

From report at bugs.python.org  Sun Sep 15 11:17:34 2019
From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=)
Date: Sun, 15 Sep 2019 15:17:34 +0000
Subject: [New-bugs-announce] [issue38178] Remove explicit "loop" argument
 from EchoClientProtocol example
Message-ID: <1568560654.11.0.446224617149.issue38178@roundup.psfhosted.org>


New submission from Hrvoje Nik?i? <hniksic at gmail.com>:

The EchoClientProtocol example receives a "loop" argument, which is not used at all in the TCP example, and is used to create a future in the UDP example. In modern asyncio code the explicit loop arguments are no longer used since the loop can always be obtained with get_running_loop().

The proposed patch makes the UDP example consistent with the TCP one (by having the constructor accept the on_con_lost future) and removes the loop argument from both.

----------
assignee: docs at python
components: Documentation, asyncio
messages: 352478
nosy: asvetlov, docs at python, hniksic, yselivanov
priority: normal
severity: normal
status: open
title: Remove explicit "loop" argument from EchoClientProtocol example
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38178>
_______________________________________

From report at bugs.python.org  Sun Sep 15 17:37:34 2019
From: report at bugs.python.org (Diego Barriga)
Date: Sun, 15 Sep 2019 21:37:34 +0000
Subject: [New-bugs-announce] [issue38179] Test subprocess fails on Fedora
 30: test_group and test_extra_groups
Message-ID: <1568583454.85.0.891057048801.issue38179@roundup.psfhosted.org>


New submission from Diego Barriga <umoqnier at gmail.com>:

======================================================================
ERROR: test_extra_groups (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/umoqnier/develop/python-dev/cpython/Lib/test/test_subprocess.py", line 1840, in test_extra_groups
    output = subprocess.check_output(
  File "/home/umoqnier/develop/python-dev/cpython/Lib/subprocess.py", line 419, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/home/umoqnier/develop/python-dev/cpython/Lib/subprocess.py", line 497, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/umoqnier/develop/python-dev/cpython/Lib/subprocess.py", line 889, in __init__
    gids.append(grp.getgrnam(extra_group).gr_gid)
KeyError: "getgrnam(): name not found: 'nogroup'"

======================================================================
ERROR: test_group (test.test_subprocess.POSIXProcessTestCase) (group='nogroup')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/umoqnier/develop/python-dev/cpython/Lib/test/test_subprocess.py", line 1800, in test_group
    output = subprocess.check_output(
  File "/home/umoqnier/develop/python-dev/cpython/Lib/subprocess.py", line 419, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/home/umoqnier/develop/python-dev/cpython/Lib/subprocess.py", line 497, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/umoqnier/develop/python-dev/cpython/Lib/subprocess.py", line 862, in __init__
    gid = grp.getgrnam(group).gr_gid
KeyError: "getgrnam(): name not found: 'nogroup'"

----------------------------------------------------------------------

Ran 306 tests in 26.423s

FAILED (errors=2, skipped=31)
test test_subprocess failed

== Tests result: FAILURE ==

1 test failed:
    test_subprocess

Total duration: 26 sec 676 ms
Tests result: FAILURE

----------
components: Tests
messages: 352495
nosy: umoqnier
priority: normal
severity: normal
status: open
title: Test subprocess fails on Fedora 30: test_group and test_extra_groups
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38179>
_______________________________________

From report at bugs.python.org  Sun Sep 15 17:41:09 2019
From: report at bugs.python.org (Diego Barriga)
Date: Sun, 15 Sep 2019 21:41:09 +0000
Subject: [New-bugs-announce] [issue38180] Test pyexpat fails on Fedora 30
Message-ID: <1568583669.12.0.757725102986.issue38180@roundup.psfhosted.org>


New submission from Diego Barriga <umoqnier at gmail.com>:

$ ./python -m test -j8 test_pyexpat
Run tests in parallel using 8 child processes
0:00:00 load avg: 0.66 [1/1/1] test_pyexpat failed
test test_pyexpat failed -- Traceback (most recent call last):
  File "/home/umoqnier/develop/python-dev/cpython/Lib/test/test_pyexpat.py", line 454, in test_exception
    parser.Parse(b"<a><b><c/></b></a>", True)
RuntimeError: a

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/umoqnier/develop/python-dev/cpython/Lib/test/test_pyexpat.py", line 470, in test_exception
    self.assertIn('call_with_frame("StartElement"', entries[1][3])
AssertionError: 'call_with_frame("StartElement"' not found in ''

== Tests result: FAILURE ==

1 test failed:
    test_pyexpat

Total duration: 211 ms
Tests result: FAILURE

----------
components: Tests
messages: 352497
nosy: umoqnier
priority: normal
severity: normal
status: open
title: Test pyexpat fails on Fedora 30
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38180>
_______________________________________

From report at bugs.python.org  Mon Sep 16 02:36:24 2019
From: report at bugs.python.org (=?utf-8?b?6Jab5a6a6LCU55qE5Za1?=)
Date: Mon, 16 Sep 2019 06:36:24 +0000
Subject: [New-bugs-announce] [issue38181] some trable with max when np.nan
 is in the first of a list
Message-ID: <1568615784.18.0.275638737164.issue38181@roundup.psfhosted.org>


New submission from ????? <1147945735 at QQ.COM>:

max([5.0,np.nan])  >>>  5.0
max([np.nan,5.0])  >>>  nan

----------
components: Tests
messages: 352513
nosy: 1147945735 at qq.com
priority: normal
severity: normal
status: open
title: some trable with max when np.nan is in the first of a list
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38181>
_______________________________________

From report at bugs.python.org  Mon Sep 16 03:28:52 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Sep 2019 07:28:52 +0000
Subject: [New-bugs-announce] [issue38182] test_asyncio.test_subprocess:
 test_stdin_stdout() failed on AMD64 FreeBSD 10-STABLE Non-Debug 3.x
Message-ID: <1568618932.29.0.888642013257.issue38182@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

AMD64 FreeBSD 10-STABLE Non-Debug 3.x:
https://buildbot.python.org/all/#/builders/167/builds/1535

======================================================================
ERROR: test_stdin_stdout (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_asyncio/test_subprocess.py", line 129, in test_stdin_stdout
    exitcode, stdout = self.loop.run_until_complete(task)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/asyncio/base_events.py", line 608, in run_until_complete
    return future.result()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/asyncio/tasks.py", line 490, in wait_for
    raise exceptions.TimeoutError()
asyncio.exceptions.TimeoutError

and then when asyncio is re-run in verbose mode:

ERROR: test_stdin_stdout (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_asyncio/test_subprocess.py", line 129, in test_stdin_stdout
    exitcode, stdout = self.loop.run_until_complete(task)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/asyncio/base_events.py", line 608, in run_until_complete
    return future.result()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/asyncio/tasks.py", line 490, in wait_for
    raise exceptions.TimeoutError()
asyncio.exceptions.TimeoutError

----------
components: Tests, asyncio
messages: 352521
nosy: asvetlov, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: test_asyncio.test_subprocess: test_stdin_stdout() failed on AMD64 FreeBSD 10-STABLE Non-Debug 3.x
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38182>
_______________________________________

From report at bugs.python.org  Mon Sep 16 03:39:09 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Sep 2019 07:39:09 +0000
Subject: [New-bugs-announce] [issue38183] test_idle fails on AMD64 FreeBSD
 CURRENT Shared 3.x: GetUserCfgDir() fails with PermissionError
Message-ID: <1568619549.13.0.00779387021845.issue38183@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

I see two problems:

* (1) test_idle fails very badly when the user configuration directory cannot be created: GetUserCfgDir() raises SystemExit when mkdir() fails with "PermissionError: [Errno 13] Permission denied: '/root/.idlerc'".

* (2) HOME environment variable of the AMD64 FreeBSD CURRENT Shared 3.x buildbot worker is set to /root, whereas the "koobs" user directory is: /home/buildbot

The best would be to fix both issues, not only the buildbot.

test_idle should not write into $HOME: it should mock this part, for example use a temporary user directory.

--

https://buildbot.python.org/all/#/builders/168/builds/1457

pythoninfo:

os.getgid: 1002
os.getgrouplist: 1002
os.getgroups: 1002
os.getuid: 1002
os.login: koobs

os.environ[HOME]: /root

pwd.getpwuid(1002): pwd.struct_passwd(pw_name='buildbot', pw_passwd='*', pw_uid=1002, pw_gid=1002, pw_gecos='FreeBSD BuildBot', pw_dir='/home/buildbot', pw_shell='/bin/sh')

The HOME environment variable override password pw_dir='/home/buildbot' directory.



0:10:59 load avg: 4.72 [226/419/1] test_idle failed -- running: test_multiprocessing_spawn (4 min 32 sec), test_tools (3 min 57 sec)
Failed to import test module: idlelib.idle_test.test_autocomplete
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_autocomplete.py", line 10, in <module>
    import idlelib.autocomplete as ac
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/autocomplete.py", line 14, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_browser
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_browser.py", line 3, in <module>
    from idlelib import browser
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/browser.py", line 16, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_codecontext
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_codecontext.py", line 3, in <module>
    from idlelib import codecontext
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/codecontext.py", line 18, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_colorizer
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_colorizer.py", line 3, in <module>
    from idlelib import colorizer
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/colorizer.py", line 6, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_config
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_config.py", line 6, in <module>
    from idlelib import config
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_configdialog
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_configdialog.py", line 5, in <module>
    from idlelib import configdialog
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/configdialog.py", line 25, in <module>
    from idlelib.config import idleConf, ConfigChanges
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_debugobj
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_debugobj.py", line 3, in <module>
    from idlelib import debugobj
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/debugobj.py", line 13, in <module>
    from idlelib.tree import TreeItem, TreeNode, ScrolledCanvas
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/tree.py", line 22, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_editor
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_editor.py", line 3, in <module>
    from idlelib import editor
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/editor.py", line 18, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_filelist
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_filelist.py", line 3, in <module>
    from idlelib import filelist
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/filelist.py", line 7, in <module>
    class FileList:
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/filelist.py", line 10, in FileList
    from idlelib.editor import EditorWindow
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/editor.py", line 18, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_format
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_format.py", line 3, in <module>
    from idlelib import format as ft
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/format.py", line 11, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_help
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_help.py", line 3, in <module>
    from idlelib import help
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/help.py", line 35, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_help_about
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_help_about.py", line 6, in <module>
    from idlelib import help_about
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/help_about.py", line 11, in <module>
    from idlelib import textview
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/textview.py", line 10, in <module>
    from idlelib.colorizer import color_config
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/colorizer.py", line 6, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_history
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_history.py", line 3, in <module>
    from idlelib.history import History
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/history.py", line 3, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_hyperparser
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_hyperparser.py", line 7, in <module>
    from idlelib.editor import EditorWindow
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/editor.py", line 18, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_iomenu
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_iomenu.py", line 3, in <module>
    from idlelib import iomenu
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/iomenu.py", line 14, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_macosx
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_macosx.py", line 8, in <module>
    from idlelib.filelist import FileList
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/filelist.py", line 7, in <module>
    class FileList:
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/filelist.py", line 10, in FileList
    from idlelib.editor import EditorWindow
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/editor.py", line 18, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_mainmenu
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_mainmenu.py", line 4, in <module>
    from idlelib import mainmenu
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/mainmenu.py", line 13, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_outwin
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_outwin.py", line 3, in <module>
    from idlelib import outwin
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/outwin.py", line 8, in <module>
    from idlelib.editor import EditorWindow
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/editor.py", line 18, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_parenmatch
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_parenmatch.py", line 6, in <module>
    from idlelib.parenmatch import ParenMatch
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/parenmatch.py", line 8, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_pathbrowser
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_pathbrowser.py", line 3, in <module>
    from idlelib import pathbrowser
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/pathbrowser.py", line 5, in <module>
    from idlelib.browser import ModuleBrowser, ModuleBrowserTreeItem
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/browser.py", line 16, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_pyshell
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_pyshell.py", line 4, in <module>
    from idlelib import pyshell
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/pyshell.py", line 49, in <module>
    from idlelib.colorizer import ColorDelegator
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/colorizer.py", line 6, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_run
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_run.py", line 3, in <module>
    from idlelib import run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/run.py", line 19, in <module>
    from idlelib import autocomplete  # AutoComplete, fetch_encodings
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/autocomplete.py", line 14, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_runscript
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_runscript.py", line 3, in <module>
    from idlelib import runscript
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/runscript.py", line 18, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_sidebar
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_sidebar.py", line 2, in <module>
    import idlelib.sidebar
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/sidebar.py", line 8, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_squeezer
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_squeezer.py", line 10, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_stackviewer
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_stackviewer.py", line 3, in <module>
    from idlelib import stackviewer
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/stackviewer.py", line 7, in <module>
    from idlelib.debugobj import ObjectTreeItem, make_objecttreeitem
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/debugobj.py", line 13, in <module>
    from idlelib.tree import TreeItem, TreeNode, ScrolledCanvas
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/tree.py", line 22, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_textview
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_textview.py", line 8, in <module>
    from idlelib import textview as tv
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/textview.py", line 10, in <module>
    from idlelib.colorizer import color_config
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/colorizer.py", line 6, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_tree
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_tree.py", line 3, in <module>
    from idlelib import tree
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/tree.py", line 22, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_warning
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_warning.py", line 8, in <module>
    from idlelib import run
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/run.py", line 19, in <module>
    from idlelib import autocomplete  # AutoComplete, fetch_encodings
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/autocomplete.py", line 14, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

Failed to import test module: idlelib.idle_test.test_zoomheight
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 198, in GetUserCfgDir
    os.mkdir(userDir)
PermissionError: [Errno 13] Permission denied: '/root/.idlerc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/idle_test/test_zoomheight.py", line 8, in <module>
    from idlelib.editor import EditorWindow
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/editor.py", line 18, in <module>
    from idlelib.config import idleConf
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 760, in <module>
    idleConf = IdleConf()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 163, in __init__
    self.CreateConfigHandlers()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 169, in CreateConfigHandlers
    self.userdir = userdir = self.GetUserCfgDir()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/idlelib/config.py", line 204, in GetUserCfgDir
    raise SystemExit
SystemExit

test test_idle crashed -- Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 270, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 234, in _runtest_inner2
    test_runner()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/libregrtest/runtest.py", line 208, in _test_module
    raise Exception("errors while loading tests")
Exception: errors while loading tests

----------
assignee: terry.reedy
components: IDLE, Tests
messages: 352523
nosy: terry.reedy, vstinner
priority: normal
severity: normal
status: open
title: test_idle fails on AMD64 FreeBSD CURRENT Shared 3.x: GetUserCfgDir() fails with PermissionError
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38183>
_______________________________________

From report at bugs.python.org  Mon Sep 16 03:47:49 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Sep 2019 07:47:49 +0000
Subject: [New-bugs-announce] [issue38184] [2.7] test_site: test_s_option()
 failed on AMD64 Fedora Rawhide Refleaks 2.7
Message-ID: <1568620069.59.0.0713376910256.issue38184@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

https://buildbot.python.org/all/#/builders/190/builds/151

Re-running test 'test_site' in verbose mode
beginning 6 repetitions
123456

(...)

test_aliasing_mbcs (test.test_site.ImportSideEffectTests) ... ok
test_no_duplicate_paths (test.test_site.ImportSideEffectTests) ... ok
test_setdefaultencoding_removed (test.test_site.ImportSideEffectTests) ... ok
test_setting_copyright (test.test_site.ImportSideEffectTests) ... ok
test_setting_help (test.test_site.ImportSideEffectTests) ... ok
test_setting_quit (test.test_site.ImportSideEffectTests) ... ok
test test_site failed -- Traceback (most recent call last):
  File "/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib/test/test_site.py", line 191, in test_s_option
    self.assertIn(usersite, sys.path)
AssertionError: '/root/.local/lib/python2.7/site-packages' not found in ['/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/target/lib/python27.zip', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib/plat-linux2', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib/lib-tk', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib/lib-old', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/build/lib.linux-x86_64-2.7-pydebug']

test_sitecustomize_executed (test.test_site.ImportSideEffectTests) ... ok

======================================================================
FAIL: test_s_option (test.test_site.HelperFunctionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib/test/test_site.py", line 191, in test_s_option
    self.assertIn(usersite, sys.path)
AssertionError: '/root/.local/lib/python2.7/site-packages' not found in ['/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/target/lib/python27.zip', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib/plat-linux2', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib/lib-tk', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/Lib/lib-old', '/home/buildbot/buildarea/2.7.cstratak-fedora-rawhide-x86_64.refleak/build/build/lib.linux-x86_64-2.7-pydebug']

----------------------------------------------------------------------
Ran 21 tests in 0.017s

FAILED (failures=1, skipped=1)
1 test failed again:
    test_site

----------
components: Tests
messages: 352525
nosy: vstinner
priority: normal
severity: normal
status: open
title: [2.7] test_site: test_s_option() failed on AMD64 Fedora Rawhide Refleaks 2.7
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38184>
_______________________________________

From report at bugs.python.org  Mon Sep 16 06:00:11 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 16 Sep 2019 10:00:11 +0000
Subject: [New-bugs-announce] [issue38185] Weird way of case-insensitive
 indexing of sqlite3.Row
Message-ID: <1568628011.92.0.862406558941.issue38185@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

sqlite3.Row can be indexed by integers and by strings. In the latter case string matching is case insensitive. But the code that implements this is too simple-minded. It compares UTF-8 representation of two strings ignoring some bit. It works for ASCII letters, but has weird behavior for digits, '_' and non-ASCII characters.

For example:

>>> import sqlite3
>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = sqlite3.Row
>>> row = con.execute("select 1 as a_1").fetchone()
>>> row['a_1']
1
>>> row['A_1']
1
>>> row['A_\x11']
1
>>> row['A\x7f1']
1
>>> row = con.execute("select 1 as ?").fetchone()
>>> row["?"]
1
>>> row["?"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: No item with that key
>>> row["?"]
1

----------
components: Library (Lib)
messages: 352533
nosy: berker.peksag, ghaering, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Weird way of case-insensitive indexing of sqlite3.Row
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38185>
_______________________________________

From report at bugs.python.org  Mon Sep 16 08:53:57 2019
From: report at bugs.python.org (Steve Dower)
Date: Mon, 16 Sep 2019 12:53:57 +0000
Subject: [New-bugs-announce] [issue38186] Use FindFirstFile in
 ntpath.realpath() when access is denied
Message-ID: <1568638437.12.0.76713297277.issue38186@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

There remains a case in ntpath.realpath() where a (handled) ERROR_ACCESS_DENIED or ERROR_SHARING_VIOLATION will break out of traversing a series of paths.

However, in this case (in os.stat(), for example), we could request the real name of the file from the directory using FindFirstFileW. This would correct the case and remove any path shortening (e.g. LONGFI~1 to LongFilename) without needing to directly access the file.

----------
components: Windows
messages: 352543
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Use FindFirstFile in ntpath.realpath() when access is denied
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38186>
_______________________________________

From report at bugs.python.org  Mon Sep 16 11:07:46 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Mon, 16 Sep 2019 15:07:46 +0000
Subject: [New-bugs-announce] [issue38187] test.test_tools.test_c_analyzer
 fails in refleak mode
Message-ID: <1568646466.84.0.78149770012.issue38187@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

Running the test suite to check refleaks over test.test_tools.test_c_analyzer will always fail, make it impossible for the refleak buildbots to run correctly. 

Steps to reproduce:

./python -m test test_tools -v -R 3:3


======================================================================
FAIL: test_typical (test.test_tools.test_c_analyzer.test_c_globals.test_find.StaticsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pablogsal/github/cpython/Lib/test/test_tools/test_c_analyzer/test_c_globals/test_find.py", line 319, in test_typical
    self.assertEqual(found, [
AssertionError: Lists differ: [Vari[413 chars]rc1/sub/ham.c', funcname=None, name='var1'), v[318 chars]nt')] != [Vari[413 chars]rc1/spam.c', funcname=None, name='freelist'), [431 chars]nt')]

First differing element 4:
Varia[21 chars]rc1/sub/ham.c', funcname=None, name='var1'), v[30 chars]t *')
Varia[21 chars]rc1/spam.c', funcname=None, name='freelist'), [34 chars]10]')

Second list contains 1 additional elements.
First extra element 8:
Variable(id=ID(filename='Include/spam.h', funcname=None, name='data'), vartype='static const int')

  [Variable(id=ID(filename='src1/spam.c', funcname=None, name='var1'), vartype='static const char *'),
   Variable(id=ID(filename='src1/spam.c', funcname='ham', name='initialized'), vartype='static int'),
   Variable(id=ID(filename='src1/spam.c', funcname=None, name='var2'), vartype='static PyObject *'),
   Variable(id=ID(filename='src1/eggs.c', funcname='tofu', name='ready'), vartype='static int'),
+  Variable(id=ID(filename='src1/spam.c', funcname=None, name='freelist'), vartype='static (PyTupleObject *)[10]'),
   Variable(id=ID(filename='src1/sub/ham.c', funcname=None, name='var1'), vartype='static const char const *'),
   Variable(id=ID(filename='src2/jam.c', funcname=None, name='var1'), vartype='static int'),
   Variable(id=ID(filename='src2/jam.c', funcname=None, name='var2'), vartype='static MyObject *'),
   Variable(id=ID(filename='Include/spam.h', funcname=None, name='data'), vartype='static const int')]

----------------------------------------------------------------------

Ran 219 tests in 3.769s

FAILED (failures=1, skipped=2, expected failures=14)
test test_tools failed
test_tools failed

== Tests result: FAILURE ==

1 test failed:
    test_tools

Total duration: 19 sec 101 ms
Tests result: FAILURE

----------
components: Tests
messages: 352552
nosy: eric.snow, pablogsal
priority: normal
severity: normal
status: open
title: test.test_tools.test_c_analyzer fails in refleak mode
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38187>
_______________________________________

From report at bugs.python.org  Mon Sep 16 13:51:37 2019
From: report at bugs.python.org (Cameron Kennedy)
Date: Mon, 16 Sep 2019 17:51:37 +0000
Subject: [New-bugs-announce] [issue38188] Incorrect Argument Order for Calls
 to _winapi.DuplicateHandle() in multiprocessing.reduction.DupHandle
Message-ID: <1568656297.2.0.121307750927.issue38188@roundup.psfhosted.org>


New submission from Cameron Kennedy <kennedy.c at husky.neu.edu>:

The DuplicateHandle function is utilized by the DupHandle object to duplicate handles for the purpose of sending and receiving between processes on Windows systems. At least on Python 3.7.3, this function is invoked with an incorrect argument order. In multiprocessing.reduction, send_handle passes _winapi.DUPLICATE_SAME_ACCESS as the access argument to the DupHandle constructor, which in-turn passes it to the access argument for _winapi.DuplicateHandle(). Instead, per https://docs.microsoft.com/en-us/windows/win32/api/handleapi/nf-handleapi-duplicatehandle this constant should be passed into the options argument. This bug results in any handles communicated via this method to have meaningless permissions set, which makes them unusable. 

I've monkeypatched the issue with the following code:

try:
    import _winapi
    log = logging.getLogger('')
    log.warning('Patching multiprocessing.reduction to deal with the _winapi.DuplicateHandle() PROCESS_DUP_HANDLE argument order bug.')
    class _PatchedDupHandle(object):
        '''Picklable wrapper for a handle.'''
        def __init__(self, handle, access, pid=None, options=0):
            if pid is None:
                # We just duplicate the handle in the current process and
                # let the receiving process steal the handle.
                pid = os.getpid()
            proc = _winapi.OpenProcess(_winapi.PROCESS_DUP_HANDLE, False, pid)
            try:
                self._handle = _winapi.DuplicateHandle(
                    _winapi.GetCurrentProcess(),
                    handle, proc, access, False, options)
            finally:
                _winapi.CloseHandle(proc)
            self._options = options
            self._access = access
            self._pid = pid

        def detach(self):
            '''Get the handle.  This should only be called once.'''
            # retrieve handle from process which currently owns it
            if self._pid == os.getpid():
                # The handle has already been duplicated for this process.
                return self._handle
            # We must steal the handle from the process whose pid is self._pid.
            proc = _winapi.OpenProcess(_winapi.PROCESS_DUP_HANDLE, False,
                                       self._pid)
            try:
                return _winapi.DuplicateHandle(
                    proc, self._handle, _winapi.GetCurrentProcess(),
                    self._access, False, self._options|_winapi.DUPLICATE_CLOSE_SOURCE)
            finally:
                _winapi.CloseHandle(proc)
    DupHandle = _PatchedDupHandle
    def _patched_send_handle(conn, handle, destination_pid):
        '''Send a handle over a local connection.'''
        dh = DupHandle(handle, 0, destination_pid, _winapi.DUPLICATE_SAME_ACCESS)
        conn.send(dh)
    send_handle=_patched_send_handle
except ImportError:
    pass

The above seems to fix the problem on my machine by adding an additional options property to the DupHandle object and an options argument to send_handle function.

----------
components: Windows
messages: 352560
nosy: m3rc1fulcameron, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Incorrect Argument Order for Calls to _winapi.DuplicateHandle() in multiprocessing.reduction.DupHandle
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38188>
_______________________________________

From report at bugs.python.org  Mon Sep 16 14:19:55 2019
From: report at bugs.python.org (Jonathan Gossage)
Date: Mon, 16 Sep 2019 18:19:55 +0000
Subject: [New-bugs-announce] [issue38189] pip does not run in virtual
 environment in 3.8
Message-ID: <1568657995.84.0.793323146645.issue38189@roundup.psfhosted.org>


New submission from Jonathan Gossage <jgossage at gmail.com>:

Python 3.8 was installed from source on Ubuntu 19.04 desktop and a virtual environment was created with python3.8 -m venv venvrh. When attempting to use pip to install a package, the following error was encountered:

(venvrh) jgossage at jgossage-XPS-8700:~/Projects/Maintenance$ pip install sphinx
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting sphinx
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/sphinx/
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/sphinx/
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/sphinx/
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/sphinx/
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/sphinx/
  Could not fetch URL https://pypi.org/simple/sphinx/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/sphinx/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
  ERROR: Could not find a version that satisfies the requirement sphinx (from versions: none)
ERROR: No matching distribution found for sphinx
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping

----------
assignee: christian.heimes
components: SSL
messages: 352564
nosy: Jonathan.Gossage, christian.heimes
priority: normal
severity: normal
status: open
title: pip does not run in virtual environment in 3.8
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38189>
_______________________________________

From report at bugs.python.org  Mon Sep 16 15:31:55 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Sep 2019 19:31:55 +0000
Subject: [New-bugs-announce] [issue38190] regrtest: test suite completes but
 Tests Result is not displayed and the process hangs
Message-ID: <1568662315.14.0.520273367874.issue38190@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

https://buildbot.python.org/all/#/builders/225/builds/239

...
0:21:16 load avg: 2.54 [422/423] test_venv passed (1 min 31 sec) -- running: test_tools (1 min 27 sec)
running: test_tools (1 min 57 sec)
running: test_tools (2 min 27 sec)
running: test_tools (2 min 57 sec)
running: test_tools (3 min 27 sec)
0:23:42 load avg: 1.48 [423/423] test_tools passed (3 min 52 sec)
command timed out: 1200 seconds without output running ['make', 'buildbottest', 'TESTOPTS=-j2 -j4 ${BUILDBOT_TESTOPTS}', 'TESTPYTHONOPTS=', 'TESTTIMEOUT=900'], attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=2628.905564


It may be a regression caused by bpo-37531.

----------
components: Tests
messages: 352574
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: regrtest: test suite completes but Tests Result is not displayed and the process hangs
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38190>
_______________________________________

From report at bugs.python.org  Mon Sep 16 15:59:55 2019
From: report at bugs.python.org (Glenn Gribble)
Date: Mon, 16 Sep 2019 19:59:55 +0000
Subject: [New-bugs-announce] [issue38191] typing.NamedTuple() should prefix
 parameters with '_'
Message-ID: <1568663995.81.0.542051193907.issue38191@roundup.psfhosted.org>


New submission from Glenn Gribble <dev at gribbelus.com>:

At present, it is not possible to use the shorthand notation to define a NamedTuple with typename or fields.  I.e., NamedTuple('MyTuple', typename=str, fields=int) does not work.  Changing the parameter names to _typename and _fields would allow any non-private, legal identifier to be used in the shorthand notation.
  
>>> import typing
>>> typing.NamedTuple('Example', fieldz=int)
<class '__main__.Example'>
>>> typing.NamedTuple('Example2', fields=int)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Program Files\Python37\lib\typing.py", line 1411, in __new__
    return _make_nmtuple(typename, fields)
  File "C:\Program Files\Python37\lib\typing.py", line 1326, in _make_nmtuple
    types = [(n, _type_check(t, msg)) for n, t in types]
TypeError: 'type' object is not iterable


Of course, it is fairly easy to work around the issue by using fields parameter:

>>> typing.NamedTuple('Example3', [('fields', int)])
<class '__main__.Example3'>


There would be backwards compatibility issues with any code using named arguments for fields or typename.

----------
components: Library (Lib)
messages: 352579
nosy: gribbg
priority: normal
severity: normal
status: open
title: typing.NamedTuple() should prefix parameters with '_'
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38191>
_______________________________________

From report at bugs.python.org  Mon Sep 16 16:34:18 2019
From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=)
Date: Mon, 16 Sep 2019 20:34:18 +0000
Subject: [New-bugs-announce] [issue38192] Fix invocation of
 EchoClientProtocol
Message-ID: <1568666058.33.0.112875968613.issue38192@roundup.psfhosted.org>


New submission from Hrvoje Nik?i? <hniksic at gmail.com>:

This is a followup on issue38178. While testing the new code, I noticed that my change introduced a bug, where the code still attempts to pass "loop" when constructing an EchoClientProtocol. A pull request is attached.

Also, I've noticed that the MyProtocol example under "Connecting Existing Sockets" is still passing an explicit loop, so I created a commit that converts it to the on_con_lost idiom, and included it in the above pull request.

----------
assignee: docs at python
components: Documentation, asyncio
messages: 352584
nosy: asvetlov, docs at python, hniksic, yselivanov
priority: normal
severity: normal
status: open
title: Fix invocation of EchoClientProtocol
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38192>
_______________________________________

From report at bugs.python.org  Mon Sep 16 18:02:08 2019
From: report at bugs.python.org (Will Boyce)
Date: Mon, 16 Sep 2019 22:02:08 +0000
Subject: [New-bugs-announce] [issue38193] http.client should be "runnable"
 like http.server
Message-ID: <1568671328.34.0.971414028957.issue38193@roundup.psfhosted.org>


New submission from Will Boyce <me at willboyce.com>:

I think it would be both sensible and useful if the `http.client` library exposed a "runnable" interface (similar to `http.server`). I would suggest keeping it simple, for example:

python -m http.client <url>
python -m http.client -o <file> <url>

----------
components: Library (Lib)
messages: 352586
nosy: Will Boyce
priority: normal
severity: normal
status: open
title: http.client should be "runnable" like http.server
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38193>
_______________________________________

From report at bugs.python.org  Mon Sep 16 18:45:09 2019
From: report at bugs.python.org (Tim Hoffmann)
Date: Mon, 16 Sep 2019 22:45:09 +0000
Subject: [New-bugs-announce] [issue38194] Consistently add exist_ok /
 missing_ok parameters to directory creation/deletion functions
Message-ID: <1568673909.26.0.25333771691.issue38194@roundup.psfhosted.org>


New submission from Tim Hoffmann <tim.hoffmann at posteo.de>:

The following functions accept exist_ok/missing_ok parameters:

- Path.mkdir(exist_ok)
- os.makedirs(exist_ok)
- shutil.copytree(dirs_exist_ok) - (https://bugs.python.org/issue20849)
- Path.unlink(missing_ok) - (https://bugs.python.org/issue33123)

For consistency, these functions should support them as well (but currently don't):

- os.makedir(exist_ok)
- os.rmdir(missing_ok)
- Path.rmdir(missing_ok)
- os.removedirs(missing_ok)
- shutil.rmtree(missing_ok)

----------
components: Library (Lib)
messages: 352590
nosy: timhoffm
priority: normal
severity: normal
status: open
title: Consistently add exist_ok / missing_ok parameters to directory creation/deletion functions
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38194>
_______________________________________

From report at bugs.python.org  Tue Sep 17 02:51:09 2019
From: report at bugs.python.org (eshkrig)
Date: Tue, 17 Sep 2019 06:51:09 +0000
Subject: [New-bugs-announce] [issue38195] A bug in the multiprocessing module
Message-ID: <1568703069.98.0.135749134145.issue38195@roundup.psfhosted.org>


New submission from eshkrig <eshkrig at gmail.com>:

Hi!

I found a bug in the multiprocessing module.

The documentation (https://docs.python.org/3.6/library/multiprocessing.html) for the Queue.get method says:
"If timeout is a positive number, it blocks at most timeout seconds ..."

But the Queue.get method freezes if the process filling the queue terminates while reading data.

In the attachment is an example of a program (testqueue.py) that freezes after ~ 5-30 seconds.
After pressing <control>-<c>, the output is as follows:
Traceback (most recent call last):
  File "./testqueue.py", line 45, in <module>
    result = ps_queues[cpu].get(timeout=2)
  File "/usr/lib64/python3.6/multiprocessing/queues.py", line 108, in get
    res = self._recv_bytes()
  File "/usr/lib64/python3.6/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/usr/lib64/python3.6/multiprocessing/connection.py", line 411, in _recv_bytes
    return self._recv(size)
  File "/usr/lib64/python3.6/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt

Please fix it.

----------
components: Extension Modules
files: testqueue.py
messages: 352613
nosy: eshkrig
priority: normal
severity: normal
status: open
title: A bug in the multiprocessing module
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file48609/testqueue.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38195>
_______________________________________

From report at bugs.python.org  Tue Sep 17 06:05:42 2019
From: report at bugs.python.org (Arshdeep)
Date: Tue, 17 Sep 2019 10:05:42 +0000
Subject: [New-bugs-announce] [issue38196] Bug fix in folderstats module
Message-ID: <1568714742.05.0.0708040865921.issue38196@roundup.psfhosted.org>


New submission from Arshdeep <arshdeepbal1996 at gmail.com>:

Issue :
While using folderstats module in python3, the records are cascaded from all previous runs of folderstats for that specific session i.e. once the folderstats is run, it will provide the dataframe with all the files/directories in that directory, but if we run it another time, the new data is appended to the old data.

To overcome this issue you can make a minor change in the folderstats module and import it :
init.py : [attached]
In method folderstats() edit the below line to make an addition i.e initialise items=list() so as the data is not appended to the previous list :

idx, items, foldersize, num_files = _recursive_folderstats(
folderpath,
hash_name=hash_name,items=list()
ignore_hidden=ignore_hidden,
verbose=verbose)

----------
components: Library (Lib)
messages: 352626
nosy: arshdeepbal
priority: normal
severity: normal
status: open
title: Bug fix in folderstats module
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38196>
_______________________________________

From report at bugs.python.org  Tue Sep 17 06:34:55 2019
From: report at bugs.python.org (Carl Friedrich Bolz-Tereick)
Date: Tue, 17 Sep 2019 10:34:55 +0000
Subject: [New-bugs-announce] [issue38197] Meaning of tracebacklimit differs
 between sys.tracebacklimit and traceback module
Message-ID: <1568716495.17.0.305052685385.issue38197@roundup.psfhosted.org>


New submission from Carl Friedrich Bolz-Tereick <cfbolz at gmx.de>:

The meaning of sys.tracebacklimit seems to be different than the meaning of the various limit parameters in the traceback module. One shows the top n stack frames, the other the bottom n.

Is this intentional, and if yes, is that difference documented somewhere? (it came up because PyPy just uses the traceback module and has no equivalent of PyTraceBack_Print).

See the attached script to understand the problem. The script formats the same exception twice, once with the traceback module, once by the interpreter. I would have expected them to look the same for all limits, but instead:

$ ./python /tmp/x.py 3
limit 3
from traceback module:
Traceback (most recent call last):
  File "/tmp/x.py", line 19, in <module>
    main()
  File "/tmp/x.py", line 16, in main
    x3()
  File "/tmp/x.py", line 14, in x3
    x2()
ZeroDivisionError: division by zero

from interpreter:
Traceback (most recent call last):
  File "/tmp/x.py", line 14, in x3
    x2()
  File "/tmp/x.py", line 12, in x2
    x1()
  File "/tmp/x.py", line 10, in x1
    1 / 0
ZeroDivisionError: division by zero

----------
files: x.py
messages: 352628
nosy: Carl.Friedrich.Bolz
priority: normal
severity: normal
status: open
title: Meaning of tracebacklimit differs between sys.tracebacklimit and traceback module
versions: Python 3.7
Added file: https://bugs.python.org/file48610/x.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38197>
_______________________________________

From report at bugs.python.org  Tue Sep 17 07:34:37 2019
From: report at bugs.python.org (Francesco Ricciardi)
Date: Tue, 17 Sep 2019 11:34:37 +0000
Subject: [New-bugs-announce] [issue38198] Attributes of pathlib classes are
 not indexed in Windows help file
Message-ID: <1568720077.25.0.0826430559794.issue38198@roundup.psfhosted.org>


New submission from Francesco Ricciardi <francesco.ricciardi at hpe.com>:

Attributes of pathlib classes (e.g. 'stem' or 'root') cannot be searched for in Python Windows help file index.

----------
assignee: docs at python
components: Documentation
messages: 352630
nosy: docs at python, francescor
priority: normal
severity: normal
status: open
title: Attributes of pathlib classes are not indexed in Windows help file
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38198>
_______________________________________

From report at bugs.python.org  Tue Sep 17 09:45:37 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Sep 2019 13:45:37 +0000
Subject: [New-bugs-announce] [issue38199] python3.8-config --ldflags must
 not generate -L/usr/lib64
Message-ID: <1568727937.94.0.575779306656.issue38199@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

Example on Fedora 30:

$ python3.8-config --ldflags
 -L/usr/lib64  -lcrypt -lpthread -ldl  -lutil -lm -lm 

This command output must not generate "-L/usr/lib64".

This bug report is based on this Fedora bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1724190#c3


== Using "-L /usr/lib64" is wrong ==

If for example you are building something against a non-system version of a library, while you have a system version of that library installed. You would then have -L/<path to non-system version> in your LDFLAGS. If some configure script then adds -L/usr/lib64 to your LDFLAGS from python-config you might link to the system version instead, depending on in which order the Makefiles puts the options together.

If your are building a new version of a software of which you already have an older version installed. If the software provides both libraries and executables, the executables should be linked to the new version of the library you just built and not the system version in /usr/lib64. Depending on how the Makefiles are written you might get the wrong version if -L/usr/lib64 is added to LDFLAGS.


== pkg-config default is correct ==

pkg-config --libs does not generate any -L flags if the .pc file contains "libdir=/usr/lib64", "Libs: -L${libdir}". Example on Fedora 30:

$ pkg-config python-3.8 --libs # empty output

$ pkg-config python-3.8-embed --libs
-lpython3.8 

pkg-config has a --keep-system-libs flag which includes -L/usr/lib64 as expected:

$ pkg-config python-3.8-embed --libs --keep-system-libs
-L/usr/lib64 -lpython3.8


If PKG_CONFIG_SYSTEM_LIBRARY_PATH env var is set to a directory different than /usr/lib64, -L/usr/lib64 is generated:

$ PKG_CONFIG_SYSTEM_LIBRARY_PATH=/xxx pkg-config python-3.8-embed --libs  
-L/usr/lib64 -lpython3.8 


== pkg-config internals ==

Look at pkgconfig internals, so we can mimick the behavior.

On my Fedora 30, /usr/lib64/pkgconfig/python3.pc contains "Libs: -L${libdir} -lpython3.7m" with "libdir=/usr/lib64", but "pkg-config python3 --libs" only returns "-lpython3.7m": no -L flag.

pkg-config:

* https://github.com/pkgconf/pkgconf/tree/master/libpkgconf
* https://src.fedoraproject.org/rpms/pkgconf/blob/master/f/pkgconf.spec

pkg-config uses PKG_CONFIG_SYSTEM_LIBRARY_PATH environment variable if set, otherwise it uses &personality->filter_libdirs. "SystemLibraryPaths" value (&personality->filter_libdirs) can be read using the command:
---
$ pkg-config --dump-personality
Triplet: default
DefaultSearchPaths: /usr/lib64/pkgconfig /usr/share/pkgconfig
SystemIncludePaths: /usr/include
SystemLibraryPaths: /usr/lib64
---

The default personality uses SYSTEM_LIBDIR, macro defines by CMakeLists.txt, extract:
---
SET(libdir ${prefix}/lib)
SET(system_libdir "${libdir}" CACHE STRING "specify the system library directory (default LIBDIR)")
SET(SYSTEM_LIBDIR "${system_libdir}")
ADD_DEFINITIONS(-DSYSTEM_LIBDIR=\"${system_libdir}\")
---

Fedora pkgconf specfile overrides the system libdir:
---	
%configure --disable-static \
           --with-pkg-config-dir=%{pkgconf_libdirs} \
           --with-system-includedir=%{_includedir} \
           --with-system-libdir=%{_libdir}
---
https://src.fedoraproject.org/rpms/pkgconf/blob/master/f/pkgconf.spec#_110


== How to fix python3.8-config? ==

The logic to detect the "system library path(s)" looks highly platform dependent. Maybe Python configure script should get an optional --with-system-libdir=PATH which would strip -L option in python-config if it's value is equal to PATH.

pkg-config also supports PKG_CONFIG_SYSTEM_LIBRARY_PATH env var and --keep-system-libs command line option.


== Options ==

* Deprecate python3.8-config: promote pkg-config. Is pkg-config available on all platforms? (Linux, macOS, FreeBSD, AIX, etc.) => easy option, least work to do!

* Document python3.8-config current behavior (document that -L/usr/lib64), modify python3.8-config & update the doc. Need to decide which features do we want (env var and/or cmdline option?).

----------
components: Build
messages: 352639
nosy: vstinner
priority: normal
severity: normal
status: open
title: python3.8-config --ldflags must not generate -L/usr/lib64
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38199>
_______________________________________

From report at bugs.python.org  Tue Sep 17 12:00:08 2019
From: report at bugs.python.org (Matteo Dell'Amico)
Date: Tue, 17 Sep 2019 16:00:08 +0000
Subject: [New-bugs-announce] [issue38200] Adding itertools.pairwise to the
 standard library?
Message-ID: <1568736008.51.0.85414927582.issue38200@roundup.psfhosted.org>


New submission from Matteo Dell'Amico <della at linux.it>:

I use itertools.pairwise all the time and I wonder if the same happens to others. I'm thinking that others may be in the same situation, and having this simple recipe already included in the library would be definitely more convenient than copy/pasting the recipe. Also, it may improve its visibility...

----------
components: Library (Lib)
messages: 352642
nosy: della
priority: normal
severity: normal
status: open
title: Adding itertools.pairwise to the standard library?

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38200>
_______________________________________

From report at bugs.python.org  Tue Sep 17 12:26:00 2019
From: report at bugs.python.org (Serkan)
Date: Tue, 17 Sep 2019 16:26:00 +0000
Subject: [New-bugs-announce] [issue38201] Anotation problem at
 flask_httpauth package
Message-ID: <1568737560.92.0.707944179779.issue38201@roundup.psfhosted.org>


New submission from Serkan <geciciserkan at gmail.com>:

if I use @auth.login_required anotation before @app.route, it is not working. It must be used after @app.route anotation. (flask_httpauth )



Sample:#
from flask_httpauth import HTTPBasicAuth
auth = HTTPBasicAuth()

@auth.login_required
@app.route('/api/v1.0/Scoring', methods=['POST'])
def Scoring():

----------
components: Extension Modules
messages: 352649
nosy: Serkan Gecici
priority: normal
severity: normal
status: open
title: Anotation problem at flask_httpauth package
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38201>
_______________________________________

From report at bugs.python.org  Tue Sep 17 15:27:01 2019
From: report at bugs.python.org (Zackery Spytz)
Date: Tue, 17 Sep 2019 19:27:01 +0000
Subject: [New-bugs-announce] [issue38202] A fatal error in test_dictviews
Message-ID: <1568748421.29.0.795591630213.issue38202@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

When running test_dictviews, I sometimes encounter a fatal error.

./python -m test test_dictviews
Run tests sequentially
0:00:00 load avg: 0.36 [1/1] test_dictviews
python: Objects/typeobject.c:3125: _PyType_Lookup: Assertion `!PyErr_Occurred()' failed.
Fatal Python error: Aborted

Current thread 0x00007fd5506de140 (most recent call first):
  File "/home/lubuntu2/cpython/Lib/test/test_dictviews.py", line 166 in test_items_set_operations
  File "/home/lubuntu2/cpython/Lib/unittest/case.py", line 617 in _callTestMethod
  File "/home/lubuntu2/cpython/Lib/unittest/case.py", line 663 in run
  File "/home/lubuntu2/cpython/Lib/unittest/case.py", line 725 in __call__
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/lubuntu2/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/lubuntu2/cpython/Lib/test/support/testresult.py", line 162 in run
  File "/home/lubuntu2/cpython/Lib/test/support/__init__.py", line 1996 in _run_suite
  File "/home/lubuntu2/cpython/Lib/test/support/__init__.py", line 2092 in run_unittest
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 209 in _test_module
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 234 in _runtest_inner2
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 270 in _runtest_inner
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 153 in _runtest
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/runtest.py", line 193 in runtest
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 409 in run_tests_sequential
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 507 in run_tests
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 674 in _main
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 628 in main
  File "/home/lubuntu2/cpython/Lib/test/libregrtest/main.py", line 695 in main
  File "/home/lubuntu2/cpython/Lib/test/__main__.py", line 2 in <module>
  File "/home/lubuntu2/cpython/Lib/runpy.py", line 85 in _run_code
  File "/home/lubuntu2/cpython/Lib/runpy.py", line 192 in _run_module_as_main
Aborted (core dumped)


It seems that this is because the PyObject_GetIter() call in _PyDictView_Intersect() is not checked for failure.

----------
components: Interpreter Core
messages: 352658
nosy: ZackerySpytz, rhettinger
priority: normal
severity: normal
status: open
title: A fatal error in test_dictviews
type: crash
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38202>
_______________________________________

From report at bugs.python.org  Tue Sep 17 16:58:38 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Sep 2019 20:58:38 +0000
Subject: [New-bugs-announce] [issue38203] regrtest fails to stop
 test_multiprocessing_spawn worker process
Message-ID: <1568753918.39.0.636404406219.issue38203@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

I ran tests locally on my laptop. Tests were running for 2 minutes. I interrupted the process by pressing CTRL+c: regrtest failed to stop immediately the worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037>. Then, regrtest main process hangs in threading._shutdown(). I had to press again CTRL+c to unblock it :-(


This bug may be related to bpo-37531 "Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS".

See also bpo-38190 "regrtest: test suite completes but Tests Result is not displayed and the process hangs" which might be related?


vstinner at apu$ make && ./python -m test -j0 -r
...
0:02:03 load avg: 8.15 [240/419] test_dynamicclassattribute passed -- running: test_multiprocessing_spawn (1 min 17 sec), test_multiprocessing_forkserver (1 min 17 sec), test_gdb (1 min 32 sec), test_multiprocessing_fork (50 sec 894 ms)
0:02:03 load avg: 8.15 [241/419] test_importlib passed -- running: test_multiprocessing_spawn (1 min 17 sec), test_multiprocessing_forkserver (1 min 17 sec), test_gdb (1 min 32 sec), test_multiprocessing_fork (50 sec 998 ms)
0:02:04 load avg: 8.15 [242/419] test_ensurepip passed -- running: test_multiprocessing_spawn (1 min 18 sec), test_multiprocessing_forkserver (1 min 18 sec), test_gdb (1 min 33 sec), test_multiprocessing_fork (51 sec 937 ms)
^C
Kill worker process 6290 running for 2.7 sec
Kill worker process 6290 running for 2.7 sec
Kill worker process 28037 running for 78.3 sec
Kill worker process 6538 running for 1.2 sec
Kill worker process 6538 running for 1.2 sec
Kill worker process 5567 running for 4.6 sec
Kill worker process 5567 running for 4.6 sec
Kill worker process 28020 running for 79.0 sec
Kill worker process 3314 running for 22.5 sec
Kill worker process 3314 running for 22.5 sec
Kill worker process 28020 running for 79.0 sec
Kill worker process 6533 running for 1.4 sec
Kill worker process 6533 running for 1.4 sec
Kill worker process 27563 running for 93.8 sec
Kill worker process 27563 running for 93.8 sec
Kill worker process 31118 running for 52.1 sec
Kill worker process 31118 running for 52.2 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 1.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 2.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 3.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 4.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 5.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 6.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 7.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 8.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 9.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 10.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 11.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 12.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 13.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 14.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 15.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 16.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 17.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 18.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 19.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 20.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 21.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 22.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 23.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 24.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 25.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 26.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 27.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 28.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 29.1 sec
Wait for regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037> for 30.1 sec
Warning -- failed to join a regrtest worker <MultiprocessThread alive test=test_multiprocessing_spawn pid=28037>

== Tests result: INTERRUPTED ==
Test suite interrupted by signal SIGINT.

177 tests omitted:
    test__osx_support test_abstract_numbers test_aifc test_asyncgen
    test_atexit test_audit test_base64 test_binascii test_bisect
    test_buffer test_bz2 test_c_locale_coercion test_calendar
    test_call test_charmapcodec test_clinic test_cmd_line
    test_cmd_line_script test_code test_codecencodings_hk
    test_codecencodings_iso2022 test_codecencodings_jp
    test_codecencodings_kr test_codecmaps_tw test_codecs test_compile
    test_complex test_concurrent_futures test_configparser
    test_contains test_contextlib test_contextlib_async test_copy
    test_copyreg test_coroutines test_crashers test_csv test_ctypes
    test_curses test_decorators test_deque test_dictviews test_dis
    test_doctest test_dtrace test_dynamic test_email test_embed
    test_exception_variations test_exceptions test_faulthandler
    test_file test_finalization test_flufl test_fork1 test_format
    test_future3 test_future5 test_gdb test_genericclass
    test_genericpath test_grp test_heapq test_htmlparser test_httplib
    test_imghdr test_imp test_int test_io test_ioctl test_ipaddress
    test_isinstance test_itertools test_json test_keyword
    test_largefile test_lib2to3 test_linecache test_lltrace
    test_logging test_mailcap test_marshal test_math test_memoryio
    test_multibytecodec test_multiprocessing_fork
    test_multiprocessing_forkserver test_multiprocessing_main_handling
    test_multiprocessing_spawn test_named_expressions test_netrc
    test_numeric_tower test_openpty test_operator test_ordered_dict
    test_os test_ossaudiodev test_parser test_pathlib test_pdb
    test_peepholer test_pickle test_pkg test_plistlib test_popen
    test_poplib test_posixpath test_pow test_print test_profile
    test_property test_pstats test_pyclbr test_pydoc test_random
    test_range test_readline test_regrtest test_repl test_richcmp
    test_robotparser test_sax test_sched test_scope test_script_helper
    test_setcomps test_shelve test_shlex test_site test_smtpd
    test_smtplib test_sndhdr test_socketserver test_source_encoding
    test_spwd test_sqlite test_ssl test_statistics
    test_string_literals test_strptime test_struct test_structmembers
    test_subprocess test_super test_symtable test_syntax test_tabnanny
    test_telnetlib test_tempfile test_threadedtempfile
    test_threading_local test_timeit test_tokenize test_trace
    test_traceback test_tracemalloc test_typechecks test_unicode_file
    test_unicode_file_functions test_unicode_identifiers
    test_unpack_ex test_urllib test_urllib_response test_userlist
    test_userstring test_utf8source test_venv test_wait4 test_warnings
    test_webbrowser test_xml_etree test_yield_from test_zipapp
    test_zipfile test_zipfile64 test_zipimport_support test_zlib

226 tests OK.

16 tests skipped:
    test_devpoll test_kqueue test_msilib test_nis test_smtpnet
    test_startfile test_timeout test_tix test_tk test_ttk_guionly
    test_urllib2net test_urllibnet test_winconsoleio test_winreg
    test_winsound test_xmlrpc_net

Total duration: 2 min 34 sec
Tests result: INTERRUPTED


^CWarning -- Unraisable exception
Exception ignored in: <module 'threading' from '/home/vstinner/python/master/Lib/threading.py'>
Traceback (most recent call last):
  File "/home/vstinner/python/master/Lib/threading.py", line 1390, in _shutdown
    lock.acquire()
KeyboardInterrupt:

----------
components: Tests
messages: 352663
nosy: vstinner
priority: normal
severity: normal
status: open
title: regrtest fails to stop test_multiprocessing_spawn worker process
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38203>
_______________________________________

From report at bugs.python.org  Tue Sep 17 18:05:08 2019
From: report at bugs.python.org (Thomas Knox)
Date: Tue, 17 Sep 2019 22:05:08 +0000
Subject: [New-bugs-announce] [issue38204] Cannot compile on RPi with
 optimizations
Message-ID: <1568757908.38.0.366967403928.issue38204@roundup.psfhosted.org>


New submission from Thomas Knox <tknox at mac.com>:

Trying to compile 3.8.0b4 on an raspberry pi 3B+ and 4 running:

pi at pi4b:~/Downloads/Python-3.8.0b4 $ uname -a
Linux pi4b 4.19.66-v7l+ #1253 SMP Thu Aug 15 12:02:08 BST 2019 armv7l GNU/Linux

Using arguments:

./configure --disable-shared --enable-optimizations --enable-loadable-sqlite-extensions --enable-ipv6 --enable-big-digits --with-lto --with-doc-strings --with-ensurepip --with-openssl=/usr

fails compilation at the linking stage with the errors:

/usr/bin/ld: /usr/bin/ld: DWARF error: offset (7602768) greater than or equal to .debug_str size (517480)
/usr/bin/ld: DWARF error: offset (2013417729) greater than or equal to .debug_str size (517480)
/usr/bin/ld: DWARF error: offset (1375799296) greater than or equal to .debug_str size (517480)
/usr/bin/ld: DWARF error: offset (3959620946) greater than or equal to .debug_str size (517480)
/usr/bin/ld: DWARF error: offset (150994953) greater than or equal to .debug_str size (517480)
/usr/bin/ld: DWARF error: offset (622700) greater than or equal to .debug_str size (517480)
/usr/bin/ld: DWARF error: offset (990813185) greater than or equal to .debug_str size (517480)
/usr/bin/ld: DWARF error: could not find abbrev number 9630
/tmp/ccX29Gpf.lto.o: in function `copy_base':
<artificial>:(.text+0x8374): undefined reference to `__gcov_one_value_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8388): undefined reference to `__gcov_average_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8398): undefined reference to `__gcov_ior_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x840c): undefined reference to `__gcov_one_value_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8420): undefined reference to `__gcov_average_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8430): undefined reference to `__gcov_ior_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x84c8): undefined reference to `__gcov_one_value_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x84dc): undefined reference to `__gcov_average_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x84ec): undefined reference to `__gcov_ior_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x856c): undefined reference to `__gcov_one_value_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8580): undefined reference to `__gcov_average_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8590): undefined reference to `__gcov_ior_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x866c): undefined reference to `__gcov_one_value_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x867c): undefined reference to `__gcov_average_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x868c): undefined reference to `__gcov_ior_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8718): undefined reference to `__gcov_one_value_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8728): undefined reference to `__gcov_average_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x8738): undefined reference to `__gcov_ior_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x87b0): undefined reference to `__gcov_one_value_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x87c8): undefined reference to `__gcov_average_profiler_atomic'
/usr/bin/ld: <artificial>:(.text+0x87d8): undefined reference to `__gcov_ior_profiler_atomic'
...
For hundreds of lines.

Removing the --enable-optimizations flag allows it compile and link.

----------
components: Build
messages: 352668
nosy: DNSGeek
priority: normal
severity: normal
status: open
title: Cannot compile on RPi with optimizations
type: compile error
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38204>
_______________________________________

From report at bugs.python.org  Tue Sep 17 18:12:35 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Sep 2019 22:12:35 +0000
Subject: [New-bugs-announce] [issue38205] Python no longer compiles without
 small integer singletons
Message-ID: <1568758355.81.0.764131668398.issue38205@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

With the following change, Python no longer compiles. I'm sure that it compiled successfully 2 weeks ago.

diff --git a/Objects/longobject.c b/Objects/longobject.c
index 4cf2b0726e..0ad2609882 100644
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -16,10 +16,10 @@ class int "PyObject *" "&PyLong_Type"
 /*[clinic end generated code: output=da39a3ee5e6b4b0d input=ec0275e3422a36e3]*/
 
 #ifndef NSMALLPOSINTS
-#define NSMALLPOSINTS           257
+#define NSMALLPOSINTS           0
 #endif
 #ifndef NSMALLNEGINTS
-#define NSMALLNEGINTS           5
+#define NSMALLNEGINTS           0
 #endif
 
 _Py_IDENTIFIER(little);


Main GCC error:

Objects/longobject.c: In function '_PyLong_Copy':
./Include/pymacro.h:119:26: error: expected expression before 'do'
  119 | #define Py_UNREACHABLE() do { \
      |                          ^~
Objects/longobject.c:83:30: note: in expansion of macro 'Py_UNREACHABLE'
   83 | #define get_small_int(ival) (Py_UNREACHABLE(), NULL)
      |                              ^~~~~~~~~~~~~~


It seems to be a regression introduced by:

commit 6b519985d23bd0f0bd072b5d5d5f2c60a81a19f2
Author: animalize <animalize at users.noreply.github.com>
Date:   Fri Sep 6 14:00:56 2019 +0800

    replace inline function `is_small_int` with a macro version (GH-15710)

----------
components: Interpreter Core
messages: 352669
nosy: vstinner
priority: normal
severity: normal
status: open
title: Python no longer compiles without small integer singletons
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38205>
_______________________________________

From report at bugs.python.org  Tue Sep 17 19:27:39 2019
From: report at bugs.python.org (Ammar Askar)
Date: Tue, 17 Sep 2019 23:27:39 +0000
Subject: [New-bugs-announce] [issue38206] Clarify that tp_dealloc must
 decref for heap allocated type
Message-ID: <1568762859.98.0.230733966822.issue38206@roundup.psfhosted.org>


New submission from Ammar Askar <ammar at ammaraskar.com>:

When dealing with a heap allocated type (https://docs.python.org/3/c-api/typeobj.html#Py_TPFLAGS_HEAPTYPE / PyType_FromSpec), if the type has a custom tp_dealloc function then it MUST decrement the references to the type object itself due to this code block: https://github.com/python/cpython/blob/4a12a178f4a6b9a59d97fecc727f2b6b28dfc85f/Objects/typeobject.c#L1189-L1192

The only mention of this is within the whatsnew entry for 3.8: https://github.com/python/cpython/commit/364f0b0f19cc3f0d5e63f571ec9163cf41c62958#diff-77c703d9a958f6a4b0dc2f692b3fd5b3

This error was made in https://github.com/python/cpython/pull/16127#pullrequestreview-288312751 and https://github.com/python/cpython/pull/16071#pullrequestreview-287819525

It seems like a common pitfall, let's add a note about this in the doc.

----------
assignee: docs at python
components: Documentation
messages: 352672
nosy: ammar2, docs at python
priority: normal
severity: normal
status: open
title: Clarify that tp_dealloc must decref for heap allocated type
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38206>
_______________________________________

From report at bugs.python.org  Tue Sep 17 19:30:48 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Sep 2019 23:30:48 +0000
Subject: [New-bugs-announce] [issue38207] subprocess: On Windows,
 Popen.kill() + Popen.communicate() is blocking until all processes
 using the pipe close the pipe
Message-ID: <1568763048.2.0.765446302919.issue38207@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

On Windows, the communicate() method of subprocess.Popen is implemented with threads calling:

def _readerthread(self, fh, buffer):
    buffer.append(fh.read())
    fh.close()

where fh is one the Popen pipes: popen.stdout or popen.stderr.

For stdout=PIPE and/or stderr=PIPE, Popen._get_handles() creates a pipe with _winapi.CreatePipe(None, 0).

The fh.read() calls only completes when the write end of the pipe is closed: when all open handles of the write end of the pipe are closed.

When using Popen(stdout=PIPE), Python uses the pipe as stdout:

                startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES
                startupinfo.hStdInput = p2cread
                startupinfo.hStdOutput = c2pwrite
                startupinfo.hStdError = errwrite

So far so good, it works well when Python spawns a single process.

--

Problems arise when the child process itself spawns new processes. In that case, the stdout pipe is inherited by other processes.

Popen.communicate() will block until all processes close the pipe.

This behavior is surprising for the following pattern:
------------------
try:
    stdout, stderr = popen.communicate(timeout=5.0)
except subprocess.TimeoutExpired:
    popen.kill()
    stdout, stderr = popen.communicate()
------------------

I would expect that the second communicate() call completes immediately since we just killed the process: Windows knows that child process is dead, Python knows that Popen object has been kill, but fh.read() continues to block (until all processes closed the pipe).

I would expect communicate() to complete immediately.

The problem is that fh.read() only returns once all data is returned at once. If fh.read() call is cancelled somehow, communicate() would just return empty data: we loose all data.

--

Eryk Sun proposed two solutions:
https://bugs.python.org/issue37531#msg350246

"For Windows, subprocess could have a _read_all(file) method that special cases a pipe. The read loop for a pipe would check whether the child has exited. Then call _winapi.PeekNamedPipe on the handle (from get_osfhandle), and do a raw read of the available bytes. If the child has exited or PeekNamedPipe fails (EPIPE), then break, join the partial reads, decode and translate newlines if it's text mode, and return."

and:
https://bugs.python.org/issue37531#msg350329

"Alternatively, instead of special casing the file type and spinning on PeekNamedPipe, the workaround could be based on a multiple-object wait that includes the child process handle. In this case, communicate() would always call _communicate in Windows, regardless of the timeout or number of pipes -- because simplistically calling either self.stdout.read() or self.stderr.read() could hang. 

The implementation would be moderately complicated. If we stop waiting on the reader threads because the process exited, we can give the threads a short time to finish reading and close the files -- maybe 250 ms is enough. But if they haven't exited at this time, we can't simply raise a TimeoutExpired exception if the call hasn't actually timed out.  To finish the _communicate call, we would have to cancel the pending reads and join() the threads.

We can force a reader thread to exit by canceling the read via WINAPI CancelIoEx. However, _readerthread has to be modified to support this. It could be implemented as a loop that calls _winapi.ReadFile to read the output in chunks that get appended to the buffer list. The read loop would break for either ERROR_BROKEN_PIPE or ERROR_OPERATION_ABORTED (canceled). 

The final step in _communicate would be to concatenate the partial reads. If it's text mode, it would also have to decode the bytes and translate newlines. The POSIX implementation of _communicate has to do this, so we already have a _translate_newlines method for this case.

Note that _winapi.WaitForMultipleObjects is interruptible in the main thread via Ctrl+C, which is a bonus improvement since Thread.join() can't be interrupted in Windows."

--

This issue has been discussed in bpo-37531 about regrtest:

https://bugs.python.org/issue37531#msg350181

It impacts the Python test suite when a test uses multiprocessing (for example) which spawns new processes.

----------
components: Library (Lib)
messages: 352673
nosy: vstinner
priority: normal
severity: normal
status: open
title: subprocess: On Windows, Popen.kill() + Popen.communicate() is blocking until all processes using the pipe close the pipe
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38207>
_______________________________________

From report at bugs.python.org  Wed Sep 18 02:17:21 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 18 Sep 2019 06:17:21 +0000
Subject: [New-bugs-announce] [issue38208] Simplify string.Template by using
 __init_subclass__()
Message-ID: <1568787441.71.0.0135329637022.issue38208@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Currently string.Template uses a metaclass. It is possible to get rid of it and use __init_subclass__() instead.

----------
components: Library (Lib)
messages: 352699
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Simplify string.Template by using __init_subclass__()
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38208>
_______________________________________

From report at bugs.python.org  Wed Sep 18 02:30:37 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 18 Sep 2019 06:30:37 +0000
Subject: [New-bugs-announce] [issue38209] Simplify dataclasses.InitVar by
 using __class_getitem__()
Message-ID: <1568788237.84.0.300342046917.issue38209@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Currently dataclasses.InitVar uses a metaclass. It is possible to get rid of it and use __class_getitem__() instead.

----------
components: Library (Lib)
messages: 352702
nosy: eric.smith, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Simplify dataclasses.InitVar by using __class_getitem__()
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38209>
_______________________________________

From report at bugs.python.org  Wed Sep 18 03:10:06 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 18 Sep 2019 07:10:06 +0000
Subject: [New-bugs-announce] [issue38210] Intersection of dict view with
 iterator returns empty set
Message-ID: <1568790606.98.0.898790756289.issue38210@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

In 3.8:

>>> {1: 2, 3: 4}.keys() & {1, 2}
{1}

In master:

>>> {1: 2, 3: 4}.keys() & iter([1, 2])
set()

The behavior was changed in issue27575.

----------
components: Interpreter Core
messages: 352705
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Intersection of dict view with iterator returns empty set
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38210>
_______________________________________

From report at bugs.python.org  Wed Sep 18 04:11:29 2019
From: report at bugs.python.org (Sergey Fedoseev)
Date: Wed, 18 Sep 2019 08:11:29 +0000
Subject: [New-bugs-announce] [issue38211] clean up type_init()
Message-ID: <1568794289.26.0.422992703771.issue38211@roundup.psfhosted.org>


New submission from Sergey Fedoseev <fedoseev.sergey at gmail.com>:

I wrote patch that cleans up type_init():

1. Removes conditions already checked by assert()
2. Removes object_init() call that effectively creates an empty tuple and checks that this tuple is empty

----------
messages: 352710
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: clean up type_init()

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38211>
_______________________________________

From report at bugs.python.org  Wed Sep 18 05:21:37 2019
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Sep 2019 09:21:37 +0000
Subject: [New-bugs-announce] [issue38212] test_multiprocessing_spawn:
 test_queue_feeder_donot_stop_onexc() failed on x86 Windows7 3.x
Message-ID: <1568798497.72.0.146892893981.issue38212@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

x86 Windows7 3.x:
https://buildbot.python.org/all/#/builders/58/builds/3015

If I recall correctly, this buildbot is known to be slow.

It sounds like a race condition.

ERROR: test_queue_feeder_donot_stop_onexc (test.test_multiprocessing_spawn.WithProcessesTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1132, in test_queue_feeder_donot_stop_onexc
    self.assertTrue(q.get(timeout=1.0))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\queues.py", line 108, in get
    raise Empty
_queue.Empty


Extract of the test:

    def test_queue_feeder_donot_stop_onexc(self):
        # bpo-30414: verify feeder handles exceptions correctly
        if self.TYPE != 'processes':
            self.skipTest('test not appropriate for {}'.format(self.TYPE))

        class NotSerializable(object):
            def __reduce__(self):
                raise AttributeError
        with test.support.captured_stderr():
            q = self.Queue()
            q.put(NotSerializable())
            q.put(True)
            # bpo-30595: use a timeout of 1 second for slow buildbots
            self.assertTrue(q.get(timeout=1.0))   # <===== FAIL HERE =====
            close_queue(q)

        ...


Hum, "bpo-30595: use a timeout of 1 second for slow buildbots" sounds like this test depends on time which is bad for reliable tests. Would it be possible to use a more reliable synchronization primitive?

If no solution is found, the workaround is to use a timeout of 60 seconds instead of just 1 second...

----------
components: Tests
messages: 352712
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: test_multiprocessing_spawn: test_queue_feeder_donot_stop_onexc() failed on x86 Windows7 3.x
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38212>
_______________________________________

From report at bugs.python.org  Wed Sep 18 07:25:21 2019
From: report at bugs.python.org (Akash Shende)
Date: Wed, 18 Sep 2019 11:25:21 +0000
Subject: [New-bugs-announce] [issue38213] sys.maxsize returns incorrect
 docstring.
Message-ID: <1568805921.73.0.762997174876.issue38213@roundup.psfhosted.org>


New submission from Akash Shende <akash0x53s at gmail.com>:

>>> import sys
>>> print(sys.maxsize.__doc__)
int([x]) -> integer
int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments
are given.  If x is a number, return x.__int__().  For floating point
numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string,
bytes, or bytearray instance representing an integer literal in the
given base.  The literal can be preceded by '+' or '-' and be surrounded
by whitespace.  The base defaults to 10.  Valid bases are 0 and 2-36.
Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4

----------
components: Library (Lib)
messages: 352716
nosy: akash0x53
priority: normal
severity: normal
status: open
title: sys.maxsize returns incorrect docstring.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38213>
_______________________________________

From report at bugs.python.org  Wed Sep 18 09:50:59 2019
From: report at bugs.python.org (Guido Imperiale)
Date: Wed, 18 Sep 2019 13:50:59 +0000
Subject: [New-bugs-announce] [issue38214] __reduce__ API specs for objects
 with __slots__
Message-ID: <1568814659.52.0.633360812637.issue38214@roundup.psfhosted.org>


New submission from Guido Imperiale <crusaderky at gmail.com>:

The documentation for the pickle module states, about the 3rd element of the tuple returned by __reduce__:

----

Optionally, the object?s state, which will be passed to the object?s __setstate__() method as previously described. If the object has no such method then, the value must be a dictionary and it will be added to the object?s __dict__ attribute.

----

This doesn't seem correct to me. It should instead read:

----

Optionally, the object?s state, which will be passed to the object?s __setstate__() method as previously described. If the object has no such method, then the value must be:

- for objects with only __dict__, a dictionary which will be used to update the object?s __dict__ attribute.
- for objects with only __slots__, a tuple of (None, {<__slots__ key>: <value>, ...})
- for objects with both __dict__ and __slots__, a tuple of ({<__dict__ key>: <value>, ...}, {<__slots__ key>: <value>, ...})

----

----------
assignee: docs at python
components: Documentation
messages: 352728
nosy: crusaderky, docs at python
priority: normal
severity: normal
status: open
title: __reduce__ API specs for objects with __slots__
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38214>
_______________________________________

From report at bugs.python.org  Wed Sep 18 11:56:53 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 18 Sep 2019 15:56:53 +0000
Subject: [New-bugs-announce] [issue38215] Do not import modules in
 star-import when __all__ is not defined.
Message-ID: <1568822213.45.0.69943350833.issue38215@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

If __all__ is not defined in a module "from module import *" will import all keys from module.__dict__ which does not start with underscore. It can add undesired names to the current namespace. The common case is related to importing modules.

1. Importing modules which are imported in the original module for internal use. Every module usually imports several other modules, and since public module names are not underscored, they pollute the namespace of the module. To prevent leaking they names you need to import them under underscored name, like "import sys as _sys" (if you don't want to use __all__). This is cumbersome and makes the module code less readable.

2. Importing submodules. The problem is that the result depends on the history of imports. If you imported "module.submodule" before or if it was imported implicitly by other module, "from module import *" will import the "submodule" name. But if it was not imported before, "from module import *" will import the "submodule" name.

The proposed PR excludes modules from importing by star-import if __all__ is not defined.

Discussion on Python-ideas: https://mail.python.org/archives/list/python-ideas at python.org/thread/AKWL7CRCCFACSITAH2NNFBL5BGRKLKJD/

----------
components: Interpreter Core
messages: 352736
nosy: gvanrossum, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Do not import modules in star-import when __all__ is not defined.
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38215>
_______________________________________

From report at bugs.python.org  Wed Sep 18 12:14:30 2019
From: report at bugs.python.org (Jason R. Coombs)
Date: Wed, 18 Sep 2019 16:14:30 +0000
Subject: [New-bugs-announce] [issue38216] Fix for issue30458 prevents
 crafting invalid requests
Message-ID: <1568823270.72.0.1365123921.issue38216@roundup.psfhosted.org>


New submission from Jason R. Coombs <jaraco at jaraco.com>:

The fix for issue30458 prevents any request line from transmitting non-ascii characters. In some cases, it's useful to transmit these illegal bytes in order to simulate a maliciously-crafted request (such as to ensure a web server responds correctly to such a request). This limitation is a regression from previous behavior that allowed transmission of such invalid requests.

For example, consider [this comment](https://github.com/cherrypy/cherrypy/issues/1781#issuecomment-507836873), where tests in CherryPy were disabled due to emergent failures against nightly builds.

Originally, I reported this issue in issue36274, but I believe this issue is distinct and has a different timeline from that issue.

In [this comment](https://bugs.python.org/issue36274#msg352711), xtreak suggests that the proper solution might be not to transmit raw invalid characters but to actually rely on URL quoting to "transmit" those bytes. While that does seem to exercise CherryPy's error detection properly, I believe it still fails to exercise the receipt and handling of raw invalid bytes, but it does provide a possible satisfactory mitigation to this issue.

----------
messages: 352738
nosy: jaraco
priority: normal
severity: normal
status: open
title: Fix for issue30458 prevents crafting invalid requests

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38216>
_______________________________________

From report at bugs.python.org  Wed Sep 18 12:20:02 2019
From: report at bugs.python.org (Ryan Govostes)
Date: Wed, 18 Sep 2019 16:20:02 +0000
Subject: [New-bugs-announce] [issue38217] argparse should support multiple
 types when nargs > 1
Message-ID: <1568823602.67.0.456772116494.issue38217@roundup.psfhosted.org>


New submission from Ryan Govostes <rgovostes at gmail.com>:

argparse supports consuming multiple command-line arguments with nargs=2, etc. It converts them to the type given in the argument's type parameter.

argparse does not provide a good solution when the input arguments should be different data types. For an example, you cannot have an argument that expects a str followed by an int, '--set-age Bob 34'.

Ordinarily, the suggestion would be to split it into two arguments, like '--set-person Bob --set-age 34'.

However, this becomes awkward with an action such as 'append', where the command line arguments become tedious, like '--set-person Bob --set-age 34 --set-person Alice --set-age 29', or confusing, as in '--set-person Bob --set-person Alice --set-age 34 --set-age 29'.

My proposal is to allow the 'type' parameter to accept a tuple of types:

    p.add_argument('--set-age', nargs=2, type=(str, int))

Since 'nargs' is redundant, this could even be simplified to just:

    p.add_argument('--set-age', type=(str, int))

The resulting parsed argument would then be a tuple of (str, int), as opposed to a list. If action='append', the result would be a list of such tuples.

This creates no backwards compatibility issue because tuple instances are not callable, so this was never valid code that did something else.

A further enhancement could be that when nargs='+' or '*', and a tuple of types is provided, the types are used round robin: '--set-ages Bob 34 Alice 29'. An exception would be raised if it would create an incomplete tuple.

See here for other discussion and workarounds: https://stackoverflow.com/questions/16959101/python-argparse-how-to-have-nargs-2-with-type-str-and-type-int

----------
components: Library (Lib)
messages: 352741
nosy: rgov
priority: normal
severity: normal
status: open
title: argparse should support multiple types when nargs > 1
type: enhancement
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38217>
_______________________________________

From report at bugs.python.org  Wed Sep 18 13:37:13 2019
From: report at bugs.python.org (Jason Plurad)
Date: Wed, 18 Sep 2019 17:37:13 +0000
Subject: [New-bugs-announce] [issue38218] SyntaxError in Tutorial 8.6
 Defining Clean-up Actions
Message-ID: <1568828233.7.0.343332850468.issue38218@roundup.psfhosted.org>


New submission from Jason Plurad <pluradj at gmail.com>:

The second code example in Tutorial section 8.6 defines the function:

def bool_return(): -> bool:

which throws a SyntaxError: invalid syntax

If the function wants to use a return annotation, there should not be a colon after the function name. I have a pull request ready for this.

----------
assignee: docs at python
components: Documentation
messages: 352746
nosy: Jason Plurad, docs at python
priority: normal
severity: normal
status: open
title: SyntaxError in Tutorial 8.6 Defining Clean-up Actions

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38218>
_______________________________________

From report at bugs.python.org  Wed Sep 18 16:46:08 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 18 Sep 2019 20:46:08 +0000
Subject: [New-bugs-announce] [issue38219] Optimize dict.__init__ and
 dict.update for dict argument
Message-ID: <1568839568.62.0.894949186749.issue38219@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

dict.__init__() and dict.update() with a positional argument look up the "keys" attribute of the argument to distinguish a mapping from a sequence of item-value pairs. It has a non-trivial cost. Merging dicts is optimized for exact dicts in PyDict_Merge(), so it would be worth to optimize also this check.

$ ./python -m pyperf timeit -s "d = {}; D = dict" -- "D(d)"
Unpatched:  Mean +- std dev: 154 ns +- 4 ns
Patched:    Mean +- std dev: 110 ns +- 4 ns

$ ./python -m pyperf timeit -s "d = {}" -- "d.update(d)"
Unpatched:  Mean +- std dev: 112 ns +- 3 ns
Patched:    Mean +- std dev: 70.4 ns +- 1.5 ns

The cost of the check is about 40 ns, and it is a significant part of the total time of creating/updating a small dict.

----------
components: Interpreter Core
messages: 352753
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Optimize dict.__init__ and dict.update for dict argument
type: performance
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38219>
_______________________________________

From report at bugs.python.org  Wed Sep 18 17:03:06 2019
From: report at bugs.python.org (Joan)
Date: Wed, 18 Sep 2019 21:03:06 +0000
Subject: [New-bugs-announce] [issue38220] Wrong values for %b and %B in
 ca_ES and ca_AD locales
Message-ID: <1568840586.34.0.840255264722.issue38220@roundup.psfhosted.org>


New submission from Joan <aseques at gmail.com>:

There's an incorrect behaviour in the output for the strftime values when in catalan.
On the documentation at https://docs.python.org/3.7/library/datetime.html#strftime-strptime-behavior:

- %b Month as locale?s abbreviated name. Jan, Feb, ?, Dec (en_US)
- %B Month as locale?s full name. January, February, ?, December (en_US);

It works also in es_ES an others, while in catalan there's an extra article on the month name (and a point in the abbreviated version) that shouldn't be there de "set." and "de setembre".

Where should I go to fix this and get the documented behaviour? I haven't been able to find a translation section on python site.

----------
components: Library (Lib)
messages: 352755
nosy: aseques
priority: normal
severity: normal
status: open
title: Wrong values for %b and %B in ca_ES and ca_AD locales
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38220>
_______________________________________

From report at bugs.python.org  Wed Sep 18 21:53:04 2019
From: report at bugs.python.org (Lindsay Haisley)
Date: Thu, 19 Sep 2019 01:53:04 +0000
Subject: [New-bugs-announce] [issue38221] Enhancement to pydoc for Python
 3.6 to allow full backgrounding as a server
Message-ID: <1568857984.74.0.551082494657.issue38221@roundup.psfhosted.org>


New submission from Lindsay Haisley <fmouse at fmp.com>:

This patch extends the functionality of pydoc3 to allow it to run purely as a server, as is the case with pydoc for python 2.7. If the -d option is specified, pydoc as a server (-p <port> option) may be backgrounded to a subshell with "&".

----------
assignee: docs at python
components: Documentation
files: pydoc3.diff
keywords: patch
messages: 352766
nosy: docs at python, fmouse
priority: normal
severity: normal
status: open
title: Enhancement to pydoc for Python 3.6 to allow full backgrounding as a server
type: enhancement
versions: Python 3.6
Added file: https://bugs.python.org/file48613/pydoc3.diff

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38221>
_______________________________________

From report at bugs.python.org  Thu Sep 19 02:01:05 2019
From: report at bugs.python.org (Miki Tebeka)
Date: Thu, 19 Sep 2019 06:01:05 +0000
Subject: [New-bugs-announce] [issue38222] pathlib Path objects should
 support __format__
Message-ID: <1568872865.53.0.090540874839.issue38222@roundup.psfhosted.org>


New submission from Miki Tebeka <miki.tebeka at gmail.com>:

Currently pathlib.Path cannot be used with string formatting directives. IMO it should.

>>> from pathlib import Path
>>> path = Path('/path/to/enlightenment')
>>> print(f'path is: {path:>50}')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported format string passed to PosixPath.__format__

----------
components: Library (Lib)
messages: 352771
nosy: tebeka
priority: normal
severity: normal
status: open
title: pathlib Path objects should support __format__
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38222>
_______________________________________

From report at bugs.python.org  Thu Sep 19 05:37:09 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Sep 2019 09:37:09 +0000
Subject: [New-bugs-announce] [issue38223] Reorganize test_shutil
Message-ID: <1568885829.07.0.120848262053.issue38223@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Currently most shutil tests are in a single class. Tests for the same function are partially grouped together and partially intermixed with other tests. This makes hard to find an appropriate place for adding new tests and increase risk of hiding test by other test with the same name.

The proposed PR reorganized shutil tests. Added separate classes for particular functions and groups of related functions. It also cleans up some code: creating temporary dirs and monkey-patching shutil.open. test.support.rmtree() is used instead of shutil.rmtree() for cleaning up temporary dirs (it is more reliable for testing purposes).

I am going to add new tests, so I need to clean up the existing code first.

----------
components: Tests
messages: 352777
nosy: giampaolo.rodola, serhiy.storchaka, tarek
priority: normal
severity: normal
status: open
title: Reorganize test_shutil
type: enhancement
versions: Python 2.7, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38223>
_______________________________________

From report at bugs.python.org  Thu Sep 19 05:42:05 2019
From: report at bugs.python.org (Leblond Emmanuel)
Date: Thu, 19 Sep 2019 09:42:05 +0000
Subject: [New-bugs-announce] [issue38224] Windows embeddable distribution
 doesn't ship headers
Message-ID: <1568886125.02.0.449353693253.issue38224@roundup.psfhosted.org>


New submission from Leblond Emmanuel <emmanuel.leblond+bugs.cpython at gmail.com>:

I don't understand why the headers are not provided as part of the distribution.

It would be really easy to rm them once the distribution extracted for usecases that don't need them.

On the other hand, usecases that need them cannot just download the Include folder from the CPython git repository given they get cooked during the build phase.

It's also really inconvenient to extract the headers from the executable distribution.

>From my personal experience, this prevented me from using the distribution each time it would have been a good fit for my need:
- Embedding CPython into the Godot game engine (see https://github.com/touilleMan/godot-python/blob/284f092d9f7893403f7af32cac5978532c0e7d21/platforms/windows-64/SCsub#L55-L62)
- Providing a frozen distribution of python for a script with simple .exe binary that would initialize the CPython insterpeter and run a simple "import mymodule; mymodule.main()" command

----------
components: Build, Windows
messages: 352778
nosy: paul.moore, steve.dower, tim.golden, touilleMan, zach.ware
priority: normal
severity: normal
status: open
title: Windows embeddable distribution doesn't ship headers
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38224>
_______________________________________

From report at bugs.python.org  Thu Sep 19 14:38:39 2019
From: report at bugs.python.org (David Hilton)
Date: Thu, 19 Sep 2019 18:38:39 +0000
Subject: [New-bugs-announce] [issue38225] iscoroutinefunction broken with
 cython - allow tagging of functions as async?
Message-ID: <1568918319.91.0.348855638851.issue38225@roundup.psfhosted.org>


New submission from David Hilton <dhiltonp at gmail.com>:

If a python piece of code imports cython code with async defs, `asyncio.iscoroutinefunction` cannot determine that the code is async.

https://github.com/cython/cython/issues/2273#issuecomment-531537624

scoder is open to marking async defs so that they can be identified, just like `asyncio.coroutine`:

https://github.com/python/cpython/blob/ae239f6b0626e926613a4a1dbafa323bd41fec32/Lib/asyncio/coroutines.py#L156

However, that is an internal interface and `@coroutine` is deprecated.

--------------

Can we have some official way of marking functions as async that will not be deprecated?

The easiest would be for `asyncio.iscoroutinefunction` to look for `_is_coroutine = True`, and promise to keep looking for that value.

This would also allow for functools.partial to easily mark that it's returning an async function, which some people seem to care about.

----------
components: asyncio
messages: 352812
nosy: asvetlov, dhiltonp, yselivanov
priority: normal
severity: normal
status: open
title: iscoroutinefunction broken with cython - allow tagging of functions as async?
type: enhancement
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38225>
_______________________________________

From report at bugs.python.org  Thu Sep 19 16:31:01 2019
From: report at bugs.python.org (=?utf-8?b?0JHQvtGA0LjRgSDQktC10YDRhdC+0LLRgdC60LjQuQ==?=)
Date: Thu, 19 Sep 2019 20:31:01 +0000
Subject: [New-bugs-announce] [issue38226] pickle.dump and load error message
 when file isn't opened in binary mode are not useful
Message-ID: <1568925061.22.0.28448935108.issue38226@roundup.psfhosted.org>


New submission from ????? ?????????? <boris.verk at gmail.com>:

pickle.load() and pickle.dump() take a file object as an argument. That file object must be opened in binary mode ("rb" or "wb"). If it's not, pickle raises a confusing error message.

When pickle.dump()'ing to a file opened in "w" mode instead of "wb" mode it'll raise a TypeError:


TypeError: write() argument must be str, not bytes


I thought it might be getting this because I was pickling bytes objects. 

For pickle.load()'ing from a file opened in "r" mode instead of "rb" mode, it'll raise a UnicodeDecodeError


UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte


There are a couple pages on the internet (just search "pickle" followed by either error message) with people being confused about this. 

pickle should catch these errors and report something more useful.

----------
components: Library (Lib)
messages: 352813
nosy: boris
priority: normal
severity: normal
status: open
title: pickle.dump and load error message when file isn't opened in binary mode are not useful
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38226>
_______________________________________

From report at bugs.python.org  Thu Sep 19 17:41:35 2019
From: report at bugs.python.org (=?utf-8?q?Ionel_Cristian_M=C4=83rie=C8=99?=)
Date: Thu, 19 Sep 2019 21:41:35 +0000
Subject: [New-bugs-announce] [issue38227] Setting a signal handler gets
 multiprocessing.Pool stuck
Message-ID: <1568929295.79.0.555392640133.issue38227@roundup.psfhosted.org>


New submission from Ionel Cristian M?rie? <contact at ionelmc.ro>:

Running `python3.8 mp-bug-python2.8.py` usually gets stuck after a dozen iterations or so.

It appears that if I stop setting that signal handler it doesn't get stuck. Unfortunately I need it to perform critical cleanup.

This is what I got from gdb:

(gdb) py-bt
Traceback (most recent call first):
  File "/usr/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 370, in worker
    `func` and (a, b) becomes func(a, b).
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.8/multiprocessing/process.py", line 569, in _bootstrap
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch
    code = process_obj._bootstrap(parent_sentinel=child_r)
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.8/multiprocessing/context.py", line 276, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.8/multiprocessing/process.py", line 633, in start
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 838, in _repopulate_pool_static
    self._length = None
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 559, in _repopulate_pool
    outqueue.put(None)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/usr/lib/python3.8/multiprocessing/context.py", line 375, in Pool
  File "mp-bug-python2.8.py", line 21, in <module>



And without the macros:



#0  0x00007f51c79fb6d6 in futex_abstimed_wait_cancelable (private=128, abstime=0x0, expected=0, futex_word=0x7f51c7e29000) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem at entry=0x7f51c7e29000, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f51c79fb7c8 in __new_sem_wait_slow (sem=0x7f51c7e29000, abstime=0x0) at sem_waitcommon.c:181
#3  0x00007f51c79fb839 in __new_sem_wait (sem=<optimized out>) at sem_wait.c:42
#4  0x00007f51c58d51f7 in semlock_acquire (self=0x7f51c58a60b0, args=<optimized out>, kwds=<optimized out>) at ./Modules/_multiprocessing/semaphore.c:319
#5  0x000000000067d929 in method_vectorcall_VARARGS_KEYWORDS (func=<method_descriptor at remote 0x7f51c6349c70>, args=0x7f51c58a4eb8, nargsf=<optimized out>, kwnames=<optimized out>)
    at ../Objects/descrobject.c:332
#6  0x000000000042c5dc in _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ../Include/cpython/abstract.h:127
#7  call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x22bd160) at ../Python/ceval.c:4987
#8  _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3486
#9  0x0000000000425ed7 in function_code_fastcall (co=<optimized out>, args=<optimized out>, nargs=1, globals=<optimized out>) at ../Objects/call.c:283
#10 0x00000000006761e4 in _PyObject_Vectorcall (kwnames=0x0, nargsf=1, args=0x7ffe20bcc908, callable=<function at remote 0x7f51c6356820>) at ../Include/cpython/abstract.h:127
#11 method_vectorcall (method=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=0x0) at ../Objects/classobject.c:67
#12 0x000000000042857e in _PyObject_Vectorcall (kwnames=0x0, nargsf=0, args=0x0, callable=<method at remote 0x7f51c665efc0>) at ../Include/cpython/abstract.h:127
#13 _PyObject_CallNoArg (func=<optimized out>) at ../Include/cpython/abstract.h:153
#14 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3287
#15 0x0000000000425ed7 in function_code_fastcall (co=<optimized out>, args=<optimized out>, nargs=1, globals=<optimized out>) at ../Objects/call.c:283
#16 0x0000000000676245 in _PyObject_Vectorcall (kwnames=0x0, nargsf=1, args=0x23cea18, callable=<function at remote 0x7f51c63563a0>) at ../Include/cpython/abstract.h:127
#17 method_vectorcall (method=<optimized out>, args=0x23cea20, nargsf=<optimized out>, kwnames=0x0) at ../Objects/classobject.c:60
#18 0x000000000042cf2f in _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ../Include/cpython/abstract.h:127
#19 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x22bd160) at ../Python/ceval.c:4987
#20 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3500
#21 0x00000000004f9f44 in PyEval_EvalFrameEx (throwflag=0,
    f=Frame 0x23ce820, for file /usr/lib/python3.8/multiprocessing/pool.py, line 370, in worker (inqueue=<SimpleQueue(_reader=<Connection(_handle=3, _readable=True, _writable=False) at remote 0x7f51c589dcd0>, _writer=<Connection(_handle=None, _readable=False, _writable=True) at remote 0x7f51c589d940>, _rlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a60b0>, acquire=<built-in method acquire of _multiprocessing.SemLock object at remote 0x7f51c58a60b0>, release=<built-in method release of _multiprocessing.SemLock object at remote 0x7f51c58a60b0>) at remote 0x7f51c589d0d0>, _poll=<method at remote 0x7f51c58ae2c0>, _wlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a1030>, acquire=<built-in method acquire of _multiprocessing.SemLock object at remote 0x7f51c58a1030>, release=<built-in method release of _multiprocessing.SemLock object at remote 0x7f51c58a1030>) at remote 0x7f51c589deb0>) at remote 0x7f51c589d3a0>, outqueue=<SimpleQueue(_reader=<Connection(_handle=None, _readable=True, _wr...(truncated))
    at ../Python/ceval.c:741
#22 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>, locals=locals at entry=0x0, args=<optimized out>, argcount=6, kwnames=0x0, kwargs=0x7f51c636e628, kwcount=0, kwstep=1,
    defs=0x7f51c65a4828, defcount=4, kwdefs=0x0, closure=0x0, name='worker', qualname='worker') at ../Python/ceval.c:4298
#23 0x000000000043e7a2 in _PyFunction_Vectorcall (func=func at entry=<function at remote 0x7f51c6352d30>, stack=<optimized out>, nargsf=nargsf at entry=6, kwnames=<optimized out>) at ../Objects/call.c:435
#24 0x000000000044143c in PyVectorcall_Call (callable=<function at remote 0x7f51c6352d30>, tuple=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:199
#25 0x0000000000428aec in do_call_core (kwdict={},
    callargs=(<SimpleQueue(_reader=<Connection(_handle=3, _readable=True, _writable=False) at remote 0x7f51c589dcd0>, _writer=<Connection(_handle=None, _readable=False, _writable=True) at remote 0x7f51c589d940>, _rlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a60b0>, acquire=<built-in method acquire of _multiprocessing.SemLock object at remote 0x7f51c58a60b0>, release=<built-in method release of _multiprocessing.SemLock object at remote 0x7f51c58a60b0>) at remote 0x7f51c589d0d0>, _poll=<method at remote 0x7f51c58ae2c0>, _wlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a1030>, acquire=<built-in method acquire of _multiprocessing.SemLock object at remote 0x7f51c58a1030>, release=<built-in method release of _multiprocessing.SemLock object at remote 0x7f51c58a1030>) at remote 0x7f51c589deb0>) at remote 0x7f51c589d3a0>, <SimpleQueue(_reader=<Connection(_handle=None, _readable=True, _writable=False) at remote 0x7f51c58a7670>, _writer=<Connection(_handle=6, _readable=False, _writable=True) at...(truncated),
    func=<function at remote 0x7f51c6352d30>, tstate=0x22bd160) at ../Python/ceval.c:5034
#26 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3559
#27 0x0000000000425ed7 in function_code_fastcall (co=<optimized out>, args=<optimized out>, nargs=1, globals=<optimized out>) at ../Objects/call.c:283
#28 0x000000000042c5dc in _PyObject_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at ../Include/cpython/abstract.h:127
#29 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x22bd160) at ../Python/ceval.c:4987
#30 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3486
#31 0x00000000004f9f44 in PyEval_EvalFrameEx (throwflag=0,
    f=Frame 0x23e7f40, for file /usr/lib/python3.8/multiprocessing/process.py, line 569, in _bootstrap (self=<ForkProcess(_identity=(98,), _config={'authkey': <AuthenticationString at remote 0x7f51c66754c0>, 'semprefix': '/mp', 'daemon': True}, _parent_pid=191540, _parent_name='MainProcess', _popen=None, _closed=False, _target=<function at remote 0x7f51c6352d30>, _args=(<SimpleQueue(_reader=<Connection(_handle=3, _readable=True, _writable=False) at remote 0x7f51c589dcd0>, _writer=<Connection(_handle=None, _readable=False, _writable=True) at remote 0x7f51c589d940>, _rlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a60b0>, acquire=<built-in method acquire of _multiprocessing.SemLock object at remote 0x7f51c58a60b0>, release=<built-in method release of _multiprocessing.SemLock object at remote 0x7f51c58a60b0>) at remote 0x7f51c589d0d0>, _poll=<method at remote 0x7f51c58ae2c0>, _wlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a1030>, acquire=<built-in method acquire of _multiprocessin...(truncated))
    at ../Python/ceval.c:741
#32 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>, locals=locals at entry=0x0, args=<optimized out>, argcount=1, kwnames=0x7f51c589d5c8, kwargs=0x7f51c66fcde8, kwcount=1, kwstep=1,
    defs=0x7f51c6672808, defcount=1, kwdefs=0x0, closure=0x0, name='_bootstrap', qualname='BaseProcess._bootstrap') at ../Python/ceval.c:4298
#33 0x000000000043e7a2 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at ../Objects/call.c:435
#34 0x0000000000676245 in _PyObject_Vectorcall (kwnames=('parent_sentinel',), nargsf=1, args=0x7f51c66fcde0, callable=<function at remote 0x7f51c6681160>) at ../Include/cpython/abstract.h:127
#35 method_vectorcall (method=<optimized out>, args=0x7f51c66fcde8, nargsf=<optimized out>, kwnames=('parent_sentinel',)) at ../Objects/classobject.c:60


This issue doesn't reproduce on python 3.7 or earlier.

----------
files: mp-bug-python2.8.py
messages: 352815
nosy: ionelmc
priority: normal
severity: normal
status: open
title: Setting a signal handler gets multiprocessing.Pool stuck
versions: Python 3.8
Added file: https://bugs.python.org/file48615/mp-bug-python2.8.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38227>
_______________________________________

From report at bugs.python.org  Thu Sep 19 17:51:22 2019
From: report at bugs.python.org (Joan)
Date: Thu, 19 Sep 2019 21:51:22 +0000
Subject: [New-bugs-announce] [issue38228] Missing documentation on strftime
 modifier O
Message-ID: <1568929882.38.0.223065779216.issue38228@roundup.psfhosted.org>


New submission from Joan <aseques at gmail.com>:

In the documentation at https://docs.python.org/3.9/library/datetime.html#strftime-strptime-behavior there's an omission of one of the modifiers that can be add to the strftime parameters (O and E)


Quoting from  http://man7.org/linux/man-pages/man3/strftime.3.html


    Some conversion specifications can be modified by preceding the
    conversion specifier character by the E or O modifier to indicate
    that an alternative format should be used.  If the alternative format
    or specification does not exist for the current locale, the behavior
    will be as if the unmodified conversion specification were used. (SU)
    The Single UNIX Specification mentions %Ec, %EC, %Ex, %EX, %Ey, %EY,
    %Od, %Oe, %OH, %OI, %Om, %OM, %OS, %Ou, %OU, %OV, %Ow, %OW, %Oy,
    where the effect of the O modifier is to use alternative numeric
    symbols (say, roman numerals), and that of the E modifier is to use a
    locale-dependent alternative representation.


The modifier works as expected for the O modifier returning the values defined in ab_alt_mon and alt_mon from the locale (instead of the ones defined in abmon and mon.
I haven't been able to get any results with the E modifier (might not be yet defined in any locale)

A small snippet of code to see the difference:
    import locale
    from datetime import datetime
    locale.setlocale(locale.LC_ALL, 'ca_AD.utf8')
    locale.setlocale(locale.LC_ALL, 'ca_ES.utf8')
    now = datetime.now() # current date and time
    date_time = now.strftime("|%Ob|%b|||%OB|%B|")
    print("date and time:",date_time)

----------
assignee: docs at python
components: Documentation
messages: 352816
nosy: aseques, docs at python
priority: normal
severity: normal
status: open
title: Missing documentation on strftime modifier O
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38228>
_______________________________________

From report at bugs.python.org  Thu Sep 19 20:00:39 2019
From: report at bugs.python.org (David Parks)
Date: Fri, 20 Sep 2019 00:00:39 +0000
Subject: [New-bugs-announce] [issue38229] ClientConnectionError exception
 doesn't serialize propertly
Message-ID: <1568937639.83.0.112440355372.issue38229@roundup.psfhosted.org>


New submission from David Parks <davidparks21 at gmail.com>:

Original question posted here:

https://stackoverflow.com/questions/58019939/attributeerror-str-object-has-no-attribute-errno?noredirect=1#comment102443264_58019939

The following exception is encountered when placing an `aiohttp.client_exceptions.ClientConnectionError` on a `multiprocessing.Queue`. The problem appears to occur during deserialization.

```
Traceback (most recent call last):
  File "model_neural_simplified.py", line 318, in <module>
    main(**arg_parser())
  File "model_neural_simplified.py", line 314, in main
    globals()[command](**kwargs)
  File "model_neural_simplified.py", line 304, in predict
    next_neural_data, next_sample = reader.get_next_result()
  File "/project_neural_mouse/src/asyncs3/s3reader.py", line 174, in get_next_result
    result = future.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "model_neural_simplified.py", line 245, in read_sample
    f_bytes = s3f.read(read_size)
  File "/project_neural_mouse/src/asyncs3/s3reader.py", line 374, in read
    size, b = self._issue_request(S3Reader.READ, (self.url, size, self.position))
  File "/project_neural_mouse/src/asyncs3/s3reader.py", line 389, in _issue_request
    response = self.communication_channels[uuid].get()
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 113, in get
    return _ForkingPickler.loads(res)
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/client_exceptions.py", line 133, in __init__
    super().__init__(os_error.errno, os_error.strerror)
AttributeError: 'str' object has no attribute 'errno'
```

A similar issue is reproducible using the following code snippet:

```
import multiprocessing
import aiohttp


connection_key = aiohttp.client_reqrep.ConnectionKey
ose = OSError('test')

e = aiohttp.client_exceptions.ClientConnectorError(connection_key, ose)
q = multiprocessing.Queue()

q.put(e)
q.get(e)
```

----------
components: asyncio
messages: 352818
nosy: asvetlov, davidparks21, yselivanov
priority: normal
severity: normal
status: open
title: ClientConnectionError exception doesn't serialize propertly
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38229>
_______________________________________

From report at bugs.python.org  Fri Sep 20 08:47:45 2019
From: report at bugs.python.org (longwenzhang)
Date: Fri, 20 Sep 2019 12:47:45 +0000
Subject: [New-bugs-announce] [issue38230] A Path Traversal vulnerability in
 ssl_servers.py
Message-ID: <1568983665.6.0.164381465993.issue38230@roundup.psfhosted.org>


New submission from longwenzhang <pakecalvs at outlook.com>:

There is a Path Traversal vulnerability in https://github.com/python/cpython/blob/master/Lib/test/ssl_servers.py (on windows platform),  Steps to reproduce:
1.Run the script https://github.com/python/cpython/blob/master/Lib/test/ssl_servers.py
2.If you visit the https://127.0.0.1:4433/ , you will see the files in the current directory, 
But if you visit the https://127.0.0.1:4433/c:../,you will jump to parent directory, and if you visit https://127.0.0.1:4433/d:../ , you will see the files of D:\ 
3.I'm sure it?s a Path Traversal and I think the problem is at https://github.com/python/cpython/blob/master/Lib/test/ssl_servers.py#L71 , there is no check about ?word?.

----------
components: Tests
messages: 352844
nosy: longwenzhang
priority: normal
severity: normal
status: open
title: A Path Traversal vulnerability in ssl_servers.py
type: security
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38230>
_______________________________________

From report at bugs.python.org  Fri Sep 20 09:39:39 2019
From: report at bugs.python.org (Matt)
Date: Fri, 20 Sep 2019 13:39:39 +0000
Subject: [New-bugs-announce] [issue38231] Documentation search results focus
 on tutorials and not language specifications
Message-ID: <1568986779.54.0.790565695991.issue38231@roundup.psfhosted.org>


New submission from Matt <excalamus at tutanota.com>:

* Problem: 
Documentation search results favor tutorials and over language specifications

* How to reproduce:
I often forget simple syntax.  For example, say I want to raise an exception.  Clearly, I need to search for "raise".  

1. Go to https://docs.python.org/3.6/search.html
2. Type in "raise"

* Expected result:
A clearly understandable link to the "raise" documentation or something closely related, such as link to the "Built-in Exceptions" documentation 

* Actual result:
The documentation for the "raise" keyword is given in search result number... at first, second, and third glance, I don't even know if a page describing the "raise" command is returned.

The first result is a 2to3 conversion tutorial.
The second result is for urllib.error.
The third result is a unittest.mock tutorial
The fourth result is an optparse tutorial
The fifth result is a tutorial for "simple statements"
The sixth result is a tutorial for "Errors and exceptions".
The seventh result is a for "exception handling"

After much poking around, it turns out that the documentation for "raise" is in the "Simple statements" tutorial.  Knowing to select the correct link requires knowing that "raise" is a statement, as the only indication that this is the correct link is that "raise_stmt" occurs in its preview snippet.

Searching for related terms, such as "exceptions" or "errors", returns similarly indirect results, such as tutorials for "Extending Python with C or C++" and "API Reference".

The experience is the same, regardless of the Python version searched.

* Proposed solution:
I'm not sure how I can help with this, but I would like to.  For example, is there some kind of tagging I could do while meandering through the morass of Python documentation?

----------
assignee: docs at python
components: Documentation
messages: 352846
nosy: docs at python, excalamus
priority: normal
severity: normal
status: open
title: Documentation search results focus on tutorials and not language specifications
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38231>
_______________________________________

From report at bugs.python.org  Fri Sep 20 10:34:10 2019
From: report at bugs.python.org (Andrei Troie)
Date: Fri, 20 Sep 2019 14:34:10 +0000
Subject: [New-bugs-announce] [issue38232] empty local-part in addr_spec
 displayed incorrectly
Message-ID: <1568990050.48.0.808924350002.issue38232@roundup.psfhosted.org>


New submission from Andrei Troie <andrei.troie at bbc.co.uk>:

Given an (RFC-legal) email address with the local part consisting of a quoted empty string (e.g. 'Nobody <""@example.org>'), when I call the 'addr_spec' property, the result no longer includes the quoted empty string (so, in this case, addr_spec would return '@example.org').

----------
components: email
files: example_parser.py
messages: 352852
nosy: andreitroiebbc, barry, r.david.murray
priority: normal
severity: normal
status: open
title: empty local-part in addr_spec displayed incorrectly
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48617/example_parser.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38232>
_______________________________________

From report at bugs.python.org  Fri Sep 20 10:55:16 2019
From: report at bugs.python.org (Alexandre Ouellet)
Date: Fri, 20 Sep 2019 14:55:16 +0000
Subject: [New-bugs-announce] [issue38233] datetime.datetime.fromtimestamp
 have different behaviour on windows and mac
Message-ID: <1568991316.62.0.929906860007.issue38233@roundup.psfhosted.org>


New submission from Alexandre Ouellet <thecell61 at gmail.com>:

in a python console on a windows machine:

import datetime
d = datetime.datetime.fromtimestamp(0)
d
>datetime.datetime(1969, 12, 31, 19, 0)
d.timestamp()
>OSError: [Errno 22] Invalid argument

on a macOS machine : 

d = datetime.datetime.fromtimestamp(-1)
d
>datetime.datetime(1969, 12, 31, 19, 0)
d.timestamp()
>-1.0

both machines running 3.6.9

I would expect them have the same behaviour.

----------
components: Windows, macOS
messages: 352853
nosy: Alexandre Ouellet, ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: datetime.datetime.fromtimestamp have different behaviour on windows and mac
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38233>
_______________________________________

From report at bugs.python.org  Fri Sep 20 11:19:54 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Fri, 20 Sep 2019 15:19:54 +0000
Subject: [New-bugs-announce] [issue38234] The value of Py_SetPath is not
 used to populate the configuration
Message-ID: <1568992794.36.0.557367208602.issue38234@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

When calling Py_SetPath(), the value that is passed in is ignored.

----------
components: Interpreter Core
keywords: 3.8regression
messages: 352856
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: The value of Py_SetPath is not used to populate the configuration
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38234>
_______________________________________

From report at bugs.python.org  Fri Sep 20 12:26:57 2019
From: report at bugs.python.org (Daniel Andersson)
Date: Fri, 20 Sep 2019 16:26:57 +0000
Subject: [New-bugs-announce] [issue38235] Docs of logging module says
 argument is named "lvl". TypeError.
Message-ID: <1568996817.83.0.32795106699.issue38235@roundup.psfhosted.org>


New submission from Daniel Andersson <daniel.4ndersson at gmail.com>:

How to reproduce:

>>> import logging
>>> logging.disable(lvl=logging.ERROR)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: disable() got an unexpected keyword argument 'lvl'

The correct keyword argument name is `level` as can be seen in the Python code Lib/logging/__init__.py:
---> def disable(level=CRITICAL):

The documentation uses `lvl`, see Doc/library/logging.rst:
---> .. function:: disable(lvl=CRITICAL)

The solution would be to rename the argument from `lvl` to `level` in the documentation.

I also noticed some more cases in the logging module docs where `lvl` is used (and `level` is used in the Python code):

* logging.Logger.isEnabledFor(lvl)
* logging.Logger.log(lvl, msg, *args, **kwargs)
* logging.Logger.makeRecord(name, lvl, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None)
* logging.addLevelName(lvl, levelName)
* logging.getLevelName(lvl)

Maybe there are some historical reasons for this that I'm not aware about.

I also found an inconsistency. In the `Handler` class the docs does use `level`:
* logging.Handler.setLevel(level)

I can understand that the English in the documentation might be clearer when written as:
"Associates level `lvl` with text `levelName`"
instead of,
"Associates level `level` with text `levelName`"
- avoids the double-"level".

But at the same time, I usually trust the documentation blindly and was surprised by this error.

In the five listed examples above, `lvl` is only used as a positional argument. Maybe it is more OK to deviate from the actual name in the code in this case compared to keyword arguments, as in logging.disable.

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 352866
nosy: docs at python, penlect, vinay.sajip
priority: normal
severity: normal
status: open
title: Docs of logging module says argument is named "lvl". TypeError.
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38235>
_______________________________________

From report at bugs.python.org  Fri Sep 20 12:44:55 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 20 Sep 2019 16:44:55 +0000
Subject: [New-bugs-announce] [issue38236] Dump the Python path configuration
 at the first import error
Message-ID: <1568997895.56.0.760570797029.issue38236@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

When Python path configuration is not properly configured, Python fails with an error looking like:
---
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007ff0065b2740 (most recent call first):
<no Python frame>
---

This error is very common and hard to debug. It can happen in various cases for various reasons:

* Python is embedded in an application
* PYTHONPATH and/or PYTHONHOME environment variable is set
* pyenv.cfg configuration for virtual environment
* Local .py files which override the standard library

A few examples:

* bpo-38234: Py_SetPath() regression in Python 3.8 which impacts PyInstaller
* https://bugs.python.org/issue35978#msg335363 virtual env
* https://bugs.python.org/issue35969#msg335249 "encodings.py" file in the current directory
* https://stackoverflow.com/questions/55657593/python-3-8-0a3-failed-to-get-the-python-codec-of-the-filesystem-encoding embedded Python
* https://askubuntu.com/questions/640010/fatal-python-error-py-initialize-unable-to-get-the-locale-encoding-syntaxe Python 3 installed by Anaconda which uses Python 2.7 stdlib because of PYTHONPATH
* https://github.com/GNS3/gns3-gui/issues/2267 Python embedded in "gns3"
* https://github.com/JuliaPy/PyCall.jl/issues/410 venv
* https://techglimpse.com/error-executing-python3-5-command-solution/ another random error

Attached PR dumps the Python path configuration if importing the Python codec of the filesystem encoding fails. In the Python initialization, it's the first import which is done.

----------
components: Interpreter Core
messages: 352869
nosy: inada.naoki, ncoghlan, pablogsal, steve.dower, vstinner
priority: normal
severity: normal
status: open
title: Dump the Python path configuration at the first import error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38236>
_______________________________________

From report at bugs.python.org  Fri Sep 20 13:21:42 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 20 Sep 2019 17:21:42 +0000
Subject: [New-bugs-announce] [issue38237] Expose meaningful keyword
 arguments for pow()
Message-ID: <1569000102.29.0.506926854512.issue38237@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Current signature:

    pow(x, y, z=None, /)


Proposed signature:

    pow(base, exp, mod=None)


Benefits:

* Meaningful and self-explanatory parameters in tooltips

* Optionally clearer calls for the three argument form:

     pow(2, 5, mod=4)

* More usable with partial():

     squared = partial(pow, exp=2)

----------
components: Library (Lib)
keywords: easy (C)
messages: 352876
nosy: mark.dickinson, rhettinger
priority: normal
severity: normal
status: open
title: Expose meaningful keyword arguments for pow()
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38237>
_______________________________________

From report at bugs.python.org  Fri Sep 20 15:34:35 2019
From: report at bugs.python.org (Barry Muldrey)
Date: Fri, 20 Sep 2019 19:34:35 +0000
Subject: [New-bugs-announce] [issue38238] site.py reporting user site info
 even if ENABLE_USER_SITE=False
Message-ID: <1569008075.72.0.976020458298.issue38238@roundup.psfhosted.org>


New submission from Barry Muldrey <python at muxing.io>:

site info is accessed externally via getusersitepackages() and getuserbase()... for example see "pip/_internal/locations.py"

If ENABLE_USER_SITE is set to "False," or otherwise disabled,
accesses to these methods should return '' or None (shouldn't they?).

Currently, I have ENABLE_USER_SITE globally disabled (by various means), but pip's call to getusersitepackages() is still returning "~/.local/lib/...".

This seems wrong.  An alternative would be for external tools to manually import site.ENABLE_USER_SITE, but this seems incorrect.

----------
components: Library (Lib)
messages: 352884
nosy: bjmuld
priority: normal
severity: normal
status: open
title: site.py reporting user site info even if ENABLE_USER_SITE=False
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38238>
_______________________________________

From report at bugs.python.org  Fri Sep 20 17:32:40 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 20 Sep 2019 21:32:40 +0000
Subject: [New-bugs-announce] [issue38239] test_gdb fails on AMD64 RHEL8 LTO
 3.x: Unexpected gdb output
Message-ID: <1569015160.3.0.923927814388.issue38239@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

AMD64 RHEL8 LTO 3.x:
https://buildbot.python.org/all/#/builders/312/builds/4

FAIL: test_NULL_ob_type (test.test_gdb.PrettyPrintTests)
FAIL: test_NULL_ptr (test.test_gdb.PrettyPrintTests)
FAIL: test_builtin_method (test.test_gdb.PrettyPrintTests)
FAIL: test_builtins_help (test.test_gdb.PrettyPrintTests)
FAIL: test_bytes (test.test_gdb.PrettyPrintTests)
FAIL: test_corrupt_ob_type (test.test_gdb.PrettyPrintTests)
FAIL: test_corrupt_tp_flags (test.test_gdb.PrettyPrintTests)
FAIL: test_corrupt_tp_name (test.test_gdb.PrettyPrintTests)
FAIL: test_dicts (test.test_gdb.PrettyPrintTests)
FAIL: test_exceptions (test.test_gdb.PrettyPrintTests)
FAIL: test_frozensets (test.test_gdb.PrettyPrintTests)
FAIL: test_int (test.test_gdb.PrettyPrintTests)
FAIL: test_lists (test.test_gdb.PrettyPrintTests)
FAIL: test_modern_class (test.test_gdb.PrettyPrintTests)
FAIL: test_selfreferential_dict (test.test_gdb.PrettyPrintTests)
FAIL: test_selfreferential_list (test.test_gdb.PrettyPrintTests)
FAIL: test_selfreferential_new_style_instance (test.test_gdb.PrettyPrintTests)
FAIL: test_selfreferential_old_style_instance (test.test_gdb.PrettyPrintTests)
FAIL: test_sets (test.test_gdb.PrettyPrintTests)
FAIL: test_singletons (test.test_gdb.PrettyPrintTests)
FAIL: test_strings (test.test_gdb.PrettyPrintTests)
FAIL: test_subclassing_list (test.test_gdb.PrettyPrintTests)
FAIL: test_subclassing_tuple (test.test_gdb.PrettyPrintTests)
FAIL: test_truncation (test.test_gdb.PrettyPrintTests)
FAIL: test_tuples (test.test_gdb.PrettyPrintTests)

Example of failure:

======================================================================
FAIL: test_NULL_ob_type (test.test_gdb.PrettyPrintTests)
Ensure that a PyObject* with NULL ob_type is handled gracefully
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb.py", line 515, in test_NULL_ob_type
    self.assertSane('id(42)',
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb.py", line 486, in assertSane
    self.get_gdb_repr(source,
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb.py", line 261, in get_gdb_repr
    self.fail('Unexpected gdb output: %r\n%s' % (gdb_output, gdb_output))
AssertionError: Unexpected gdb output: 'Breakpoint 1 at 0x565b60: file Objects/longobject.c, line 1113.\n[Thread debugging using libthread_db enabled]\nUsing host libthread_db library "/lib64/libthread_db.so.1".\n\nBreakpoint 1, builtin_id (self=, v=42) at Objects/longobject.c:1113\n1113\tPyLong_FromVoidPtr(void *p)\n#0  builtin_id (self=, v=<unknown at remote 0x94e000>) at Objects/longobject.c:1113\n#1  cfunction_vectorcall_O (func=<built-in method id of module object at remote 0x7ffff7f6eea0>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:442\n#2  _PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=<optimized out>, callable=<built-in method id of module object at remote 0x7ffff7f6eea0>) at ./Include/cpython/abstract.h:96\n#3  call_function (tstate=, pp_stack=, oparg=<optimized out>, kwnames=0x0) at Python/ceval.c:4984\n#4  _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3496\n#5  _PyEval_EvalCodeWithName (_co=<code at remote 0x7ffff7f13920>, globals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, locals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, args=, argcount=0, kwnames=, kwargs=, kwcount=0, kwstep=2, defs=, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4296\n#6  PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=, kwcount=0, kws=, argcount=0, args=, locals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, globals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, _co=<code at remote 0x7ffff7f13920>) at Python/ceval.c:712\n#7  PyEval_EvalCode (locals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, globals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, co=<code at remote 0x7ffff7f13920>) at Python/ceval.c:714\n#8  run_eval_code_obj (locals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, globals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, co=) at Python/pythonrun.c:1117\n#9  run_mod (mod=<optimized out>, filename=\'<string>\', globals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, locals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, flags=, arena=) at Python/pythonrun.c:1139\n#10 PyRun_StringFlags (flags=, locals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, globals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': <type at remote 0x9669a0>, \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': <module at remote 0x7ffff7f6eea0>}, start=257, str="id(42)\\n") at Python/pythonrun.c:1008\n#11 PyRun_SimpleStringFlags (command="id(42)\\n", flags=) at Python/pythonrun.c:460\n#12 pymain_run_command (cf=, command=<optimized out>) at Modules/main.c:260\n#13 pymain_run_python (exitcode=) at Modules/main.c:558\n#14 Py_RunMain () at Modules/main.c:646\n#15 pymain_main (args=) at Modules/main.c:676\n#16 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:700\n#17 __libc_start_main () from /lib64/libc.so.6\n#18 _start () at Python/ceval.c:5444\n'
Breakpoint 1 at 0x565b60: file Objects/longobject.c, line 1113.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, builtin_id (self=, v=42) at Objects/longobject.c:1113
1113	PyLong_FromVoidPtr(void *p)
#0  builtin_id (self=, v=<unknown at remote 0x94e000>) at Objects/longobject.c:1113
#1  cfunction_vectorcall_O (func=<built-in method id of module object at remote 0x7ffff7f6eea0>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:442
#2  _PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=<optimized out>, callable=<built-in method id of module object at remote 0x7ffff7f6eea0>) at ./Include/cpython/abstract.h:96
#3  call_function (tstate=, pp_stack=, oparg=<optimized out>, kwnames=0x0) at Python/ceval.c:4984
#4  _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3496
#5  _PyEval_EvalCodeWithName (_co=<code at remote 0x7ffff7f13920>, globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, args=, argcount=0, kwnames=, kwargs=, kwcount=0, kwstep=2, defs=, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4296
#6  PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=, kwcount=0, kws=, argcount=0, args=, locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, _co=<code at remote 0x7ffff7f13920>) at Python/ceval.c:712
#7  PyEval_EvalCode (locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, co=<code at remote 0x7ffff7f13920>) at Python/ceval.c:714
#8  run_eval_code_obj (locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, co=) at Python/pythonrun.c:1117
#9  run_mod (mod=<optimized out>, filename='<string>', globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, flags=, arena=) at Python/pythonrun.c:1139
#10 PyRun_StringFlags (flags=, locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x9669a0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f6eea0>}, start=257, str="id(42)\n") at Python/pythonrun.c:1008
#11 PyRun_SimpleStringFlags (command="id(42)\n", flags=) at Python/pythonrun.c:460
#12 pymain_run_command (cf=, command=<optimized out>) at Modules/main.c:260
#13 pymain_run_python (exitcode=) at Modules/main.c:558
#14 Py_RunMain () at Modules/main.c:646
#15 pymain_main (args=) at Modules/main.c:676
#16 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:700
#17 __libc_start_main () from /lib64/libc.so.6
#18 _start () at Python/ceval.c:5444

----------
components: Tests
messages: 352896
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_gdb fails on AMD64 RHEL8 LTO 3.x: Unexpected gdb output
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38239>
_______________________________________

From report at bugs.python.org  Fri Sep 20 17:55:35 2019
From: report at bugs.python.org (Vitaly Kruglikov)
Date: Fri, 20 Sep 2019 21:55:35 +0000
Subject: [New-bugs-announce] [issue38240] assertCountEqual is horribly
 misleading, sounds like only counts are being compared
Message-ID: <1569016535.61.0.833203589698.issue38240@roundup.psfhosted.org>


New submission from Vitaly Kruglikov <vitaly.krug at gmail.com>:

assertCountEqual is a horribly misleading name because it misleads the programmer and reader of the test code into thinking that it only compares the counts of the elements.

It's name misrepresents what it does in a really bad way.

----------
components: Tests
messages: 352901
nosy: vitaly.krug
priority: normal
severity: normal
status: open
title: assertCountEqual is horribly misleading, sounds like only counts are being compared
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38240>
_______________________________________

From report at bugs.python.org  Fri Sep 20 18:53:12 2019
From: report at bugs.python.org (Nicholas Neumann)
Date: Fri, 20 Sep 2019 22:53:12 +0000
Subject: [New-bugs-announce] [issue38241] Pickle with protocol=0 in python 3
 does not produce a 'human-readable' format
Message-ID: <1569019992.12.0.181360535458.issue38241@roundup.psfhosted.org>


New submission from Nicholas Neumann <nick2002 at gmail.com>:

The docs for pickle, in python 2, say that the default pickle protocol, 0, produces ASCII. In the python 3 docs, this has changed to "human-readable". While the pickle output with protocol 0 loads fine in python2, it is definitely not human-readable, as it is not valid ASCII and contains every possible byte.

To see a simple example, run this in both python 2 and 3

import pickle
a = bytearray(range(255)) #bytes containing 0..255
b = bytes(a)
c = pickle.dumps(b,protocol=0)
print(c)#human readable in 2, not in 3
c.decode('ascii')#throws in 3, not in 2

----------
messages: 352907
nosy: aggieNick02
priority: normal
severity: normal
status: open
title: Pickle with protocol=0 in python 3 does not produce a 'human-readable' format
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38241>
_______________________________________

From report at bugs.python.org  Fri Sep 20 18:58:45 2019
From: report at bugs.python.org (Yury Selivanov)
Date: Fri, 20 Sep 2019 22:58:45 +0000
Subject: [New-bugs-announce] [issue38242] Revert the new asyncio Streams API
Message-ID: <1569020325.08.0.390897413838.issue38242@roundup.psfhosted.org>


New submission from Yury Selivanov <yselivanov at gmail.com>:

== Context

1. Andrew and I implemented a new streaming API in asyncio 3.8.  The key idea is that there's a single Stream object, combining old StreamReader & StreamWriter APIs.  On top of that, `Stream.write()` and `Stream.close()` became coroutines.  The new API is significantly easier to use, but it required us to:

(a) Make backwards compatible changes to subprocess APIs;
(b) Add two new APIs: `asyncio.connect() -> Stream` and `asyncio.StreamServer`
(c) Soft-deprecate `asyncio.open_connection()` and `asyncio.start_serving()`.


2. The Trio project considers implementing new Streams API (see [1]).  The key idea is to make the core Stream object very simple and then enable building complex protocol pipelines using composition.  Want SSL?  Add an SSL layer.  Have a framed binary protocol?  Push a reusable framing layer to the stack and parse the protocol.  On top of that, Trio wants us to standardize Streams, so that it's possible to write framework agnostic protocol code using async/await and to even reuse things like SSL implementation.


== Problem

I really like how Trio approaches this.

The asyncio.Stream class we have now is overloaded with functionality.  It's not composable.  It's internal buffer and APIs are designed to parsing text protocols (i.e. parsing a complex binary protocol requires an entirely different buffer implementation).

Long story short, I think we should revert the new streaming API from the 3.8 branch and see if Trio & asyncio can design a better Streaming API.  Otherwise we end up in a weird situation where we added a bunch of new APIs to 3.8 which can be deprecated in 3.9.

Worst case scenario we'll just ship our current versions of Streams in 3.9 (not in 3.8).

Thoughts?

[1] https://github.com/python-trio/trio/issues/1208

----------
components: asyncio
messages: 352908
nosy: asvetlov, gvanrossum, lukasz.langa, njs, yselivanov
priority: release blocker
severity: normal
status: open
title: Revert the new asyncio Streams API
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38242>
_______________________________________

From report at bugs.python.org  Fri Sep 20 22:17:30 2019
From: report at bugs.python.org (longwenzhang)
Date: Sat, 21 Sep 2019 02:17:30 +0000
Subject: [New-bugs-announce] [issue38243] A reflected XSS in
 python/Lib/DocXMLRPCServer.py
Message-ID: <1569032250.18.0.545337961077.issue38243@roundup.psfhosted.org>


New submission from longwenzhang <pakecalvs at outlook.com>:

It's "Lib/DocXMLRPCServer.py" in python2x or "Lib/xmlrpc/server.py" in python3x.

Steps to reproduce:

1.Lib/DocXMLRPCServer.py is ?a documenting XML-RPC Server?,In the Class ServerHTMLDoc, method markup(), will escape the Special symbols to safe(such as <," etc).
2.But it only escape the content from server.set_server_name() and server.set_server_documentation(),the "title" content from the server.set_server_title() will not be escaped, so if I set_server_title('123</title><script>alert(1)</script>'), it will cause XSS because not escaped.
3.I see the alert in Chrome by visiting http://127.0.0.1,the Poc is the poc.py?run in python2.7? in attachments.
4.Problems seems to be at
https://github.com/python/cpython/blob/master/Lib/xmlrpc/server.py#L897 "return documenter.page(self.server_title,documentation)".Before this line,variable "documentation" has been escaped but self.server_title not.This is the main cause.

----------
components: Library (Lib)
files: poc.py
messages: 352921
nosy: longwenzhang
priority: normal
severity: normal
status: open
title: A reflected XSS in python/Lib/DocXMLRPCServer.py
type: security
versions: Python 3.7
Added file: https://bugs.python.org/file48619/poc.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38243>
_______________________________________

From report at bugs.python.org  Sat Sep 21 02:12:10 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Sat, 21 Sep 2019 06:12:10 +0000
Subject: [New-bugs-announce] [issue38244] example in ftplib.rst uses
 ftp.debian.org but it's not reachable
Message-ID: <1569046330.36.0.543009773859.issue38244@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

In the first example of Doc/library/ftplib.rst, we use ftp.debian.org but the connection is refused. There is no automatic redirection to ftp.us.debian.org or another server. If we want to keep a consistent example, we should use ftp.us.debian.org or another server.

----------
messages: 352929
nosy: matrixise
priority: normal
severity: normal
status: open
title: example in ftplib.rst uses ftp.debian.org but it's not reachable
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38244>
_______________________________________

From report at bugs.python.org  Sat Sep 21 13:06:30 2019
From: report at bugs.python.org (Srinivasan Samuel)
Date: Sat, 21 Sep 2019 17:06:30 +0000
Subject: [New-bugs-announce] [issue38245] Why am I getting inconsistent
 results in this simple List assignment?
Message-ID: <1569085590.42.0.704218792201.issue38245@roundup.psfhosted.org>


New submission from Srinivasan Samuel <creatorsss at yahoo.com>:

Here is the my direct cut & paste from my Python 3.7.3 Shell
>>> C = 2*[[]]
>>> C
[[], []]
>>> 
>>> M = [[],[]]
>>> M
[[], []]
>>> 
>>> C == M
True
>>> 
>>> M[0].append(5)
>>> M
[[5], []]
>>> 
>>> C[0].append(5)
>>> C
[[5], [5]]
>>> 
>>> C == M
False
>>>

----------
messages: 352945
nosy: pysolo
priority: normal
severity: normal
status: open
title: Why am I getting inconsistent results in this simple List assignment?
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38245>
_______________________________________

From report at bugs.python.org  Sat Sep 21 18:01:21 2019
From: report at bugs.python.org (Infrasonics)
Date: Sat, 21 Sep 2019 22:01:21 +0000
Subject: [New-bugs-announce] [issue38246] pathlib.resolve and .absolute
 leave trailing tilde in home expansion
Message-ID: <1569103281.1.0.00418386254379.issue38246@roundup.psfhosted.org>


New submission from Infrasonics <infrasonics at gmx.de>:

`Path('~').resolve()` gives `PosixPath('/home/youruser/~')`.
Same problem for `.absolute`, yet `.expanduser` works correctly.

----------
components: Library (Lib)
messages: 352954
nosy: Infrasonics
priority: normal
severity: normal
status: open
title: pathlib.resolve and .absolute leave trailing tilde in home expansion
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38246>
_______________________________________

From report at bugs.python.org  Sat Sep 21 18:19:50 2019
From: report at bugs.python.org (Aman Priyanshu)
Date: Sat, 21 Sep 2019 22:19:50 +0000
Subject: [New-bugs-announce] [issue38247] list(str.split(ch)) Does not print
 blank elements upon repeating character(ch)
Message-ID: <1569104390.96.0.0416491313723.issue38247@roundup.psfhosted.org>


New submission from Aman Priyanshu <amanpriyanshusms2001 at gmail.com>:

I have been annoyed by the fact that at multiple where we print list(str.split(ch)) we get empty elements in place of repeated characters (ch).

Example:

####
>>> print(list("Hello World How Are You?".split(" ")))
['Hello', 'World', 'How', 'Are', 'You?']
>>> print(list("Hello World       How Are      You?".split(" ")))
['Hello', 'World', '', '', '', '', '', '', 'How', 'Are', '', '', '', '', '', 'You?']
####

So can it be fixed so that  it gives:

####
>>> print(list("Hello World How Are You?".split(" ")))
['Hello', 'World', 'How', 'Are', 'You?']
>>> print(list("Hello World       How Are      You?".split(" ")))
['Hello', 'World', 'How', 'Are', 'You?']
####

----------
messages: 352955
nosy: AmanPriyanshu
priority: normal
pull_requests: 15902
severity: normal
status: open
title: list(str.split(ch))  Does not print blank elements upon repeating character(ch)
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38247>
_______________________________________

From report at bugs.python.org  Sun Sep 22 04:45:24 2019
From: report at bugs.python.org (Nathaniel Smith)
Date: Sun, 22 Sep 2019 08:45:24 +0000
Subject: [New-bugs-announce] [issue38248] inconsistency in asyncio.Task
 between cancellation while running vs. cancellation immediately after it
 finishes
Message-ID: <1569141924.07.0.413373545347.issue38248@roundup.psfhosted.org>


New submission from Nathaniel Smith <njs at pobox.com>:

Just noticed this while looking at the code to asyncio.Task.__step

asyncio Futures have two different error states: they can have an arbitrary exception set, or they can be marked as cancelled.

asyncio Tasks handle this by detecting what exception was raised by the task code: if it's a CancelledError, then the mark the Future as cancelled, and if it's anything else, they set that as the Future's exception.

There is also a special case handled inside the 'except StopIteration' clause in Task.__step. If we request that a Task be cancelled, but then the task exits normally before we have a chance to throw in a CancelledError, then we also want mark the Future as cancelled. BUT, this special case is handled incorrectly: it does Future.set_exception(CancelledError()), instead of Future.cancel(). Normally it's impossible for a task to end up with its exception set to CancelledError, but it does happen in this one weird edge case, basically as a race condition.

Here's some sample code to illustrate the problem (tested on 3.7):

------

import asyncio

# This gets cancelled normally
async def cancel_early():
    asyncio.current_task().cancel()
    await asyncio.sleep(1)

async def cancel_late():
    asyncio.current_task().cancel()
    # No sleep, so CancelledError doesn't get delivered until after the
    # task exits

async def main():
    t_early = asyncio.create_task(cancel_early())
    await asyncio.sleep(0.1)
    print(f"t_early.cancelled(): {t_early.cancelled()!r}")

    t_late = asyncio.create_task(cancel_late())
    await asyncio.sleep(0.1)
    print(f"t_late.cancelled(): {t_late.cancelled()!r}")
    print(f"t_late.exception(): {t_late.exception()!r}")

asyncio.run(main())


------

Output:

t_early.cancelled(): True
t_late.cancelled(): False
t_late.exception(): CancelledError()

The obvious fix would be to modify the 'except StopIteration' branch to handle this case by calling super().cancel() instead of super().set_exception(...).

Alternatively, I could see an argument that asyncio.Task should always preserve the CancelledError, so that e.g. you can get a proper traceback. In that case we'd need to change the 'except CancelledError' branch to call super().set_exception(...) instead of super().cancel(). This would also need some more changes, like overriding .cancelled() to check for a stored exception and then return isinstance(stored_exc, CancelledError), and maybe others... I'm not sure of the full consequences.

But handling these two cases differently is definitely wrong, that part I'm sure of :-)

----------
messages: 352964
nosy: asvetlov, njs, yselivanov
priority: normal
severity: normal
status: open
title: inconsistency in asyncio.Task between cancellation while running vs. cancellation immediately after it finishes

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38248>
_______________________________________

From report at bugs.python.org  Sun Sep 22 05:47:06 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Sep 2019 09:47:06 +0000
Subject: [New-bugs-announce] [issue38249] Optimize out Py_UNREACHABLE in the
 release mode
Message-ID: <1569145626.92.0.408422678201.issue38249@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Py_UNREACHABLE is used to indicate that a specific point in the program cannot be reached, even if the compiler might otherwise think it can. This is exact the case for __builtin_unreachable in GCC and Clang. I propose to extend Py_UNREACHABLE() to __builtin_unreachable() in the release mode. This will allow the compiler to generate more efficient code.

If there are circumstances in which Py_UNREACHABLE() is reachable, then it is improper use of Py_UNREACHABLE(). It should be replaced with raising an appropriate exception (like TypeError, ValueError, RuntimeError or SystemError) or, in extreme cases, with explicit Py_FatalError()

----------
components: Interpreter Core
messages: 352965
nosy: barry, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Optimize out Py_UNREACHABLE in the release mode
type: performance
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38249>
_______________________________________

From report at bugs.python.org  Sun Sep 22 07:14:02 2019
From: report at bugs.python.org (John Belmonte)
Date: Sun, 22 Sep 2019 11:14:02 +0000
Subject: [New-bugs-announce] [issue38250] enum.Flag should be more set-like
Message-ID: <1569150842.66.0.799345616323.issue38250@roundup.psfhosted.org>


New submission from John Belmonte <john at neggie.net>:

I would like Flag class instances to have more set-like abilities:
  1. iteration, to walk through each set bit of the value
  2. len corresponding to #1
  3. subset operator

I may be told "implement it yourself as instance methods", or that #3 has an idiom (a & b is b).  Ideally though, every Flag user should be able to rely on these being implemented consistently.

When trying to implement #1 without devolving into bit fiddling, naturally one might try to use the class iterator.  Unfortunately the semantics of that enumeration include 0, aliases, and compound values.  I've used Flag in several situations and projects, and so far there hasn't been a case where that was the desired semantics.  Interesting though, if #1 were implemented in the standard library, then we could enumerate all bits of the Flag via iteration of `~MyFlag(0)`... though that's obscuring things behind another idiom.

Thank you for considering.

----------
components: Library (Lib)
messages: 352967
nosy: John Belmonte
priority: normal
severity: normal
status: open
title: enum.Flag should be more set-like
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38250>
_______________________________________

From report at bugs.python.org  Sun Sep 22 07:29:48 2019
From: report at bugs.python.org (Ofer Sadan)
Date: Sun, 22 Sep 2019 11:29:48 +0000
Subject: [New-bugs-announce] [issue38251] urllib.request memory leak /
 overflow
Message-ID: <1569151788.73.0.0306600169217.issue38251@roundup.psfhosted.org>


New submission from Ofer Sadan <ofersadan85 at gmail.com>:

running `urllib.request.urlopen` multiple times causes the memory usage to increase with each run, even after calling `close()` on the request or using `del` on the result

To recreate this problem, run code:

    import urllib.request
    def ip():
        r = urllib.request.urlopen('https://api.ipify.org')
        b = r.read()
        r.close()
        print(len(b))
        return b

    for i in range(1000):
        result = ip()
        del result

Even though `len(b)` has a maximum value of 15 (for this url at least), the memory increases with each run by 200KB - 1MB

----------
components: Library (Lib)
messages: 352968
nosy: Ofer Sadan
priority: normal
severity: normal
status: open
title: urllib.request memory leak / overflow
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38251>
_______________________________________

From report at bugs.python.org  Sun Sep 22 07:50:02 2019
From: report at bugs.python.org (Ma Lin)
Date: Sun, 22 Sep 2019 11:50:02 +0000
Subject: [New-bugs-announce] [issue38252] micro-optimize
 ucs1lib_find_max_char in Windows 64-bit build
Message-ID: <1569153002.15.0.915285235862.issue38252@roundup.psfhosted.org>


New submission from Ma Lin <malincns at 163.com>:

C type `long` is 4-byte integer in 64-bit Windows build. [1]

But `ucs1lib_find_max_char()` function [2] uses SIZEOF_LONG, so it loses a little performance in 64-bit Windows build.

Below is the benchmark of using SIZEOF_SIZE_T and this change:

    -   unsigned long value = *(unsigned long *) _p;
    +   sizt_t value = *(sizt_t *) _p;

D:\dev\cpython\PCbuild\amd64\python.exe -m pyperf timeit -s "b=b'a'*10_000_000; f=b.decode;" "f('latin1')"

    before: 5.83 ms +- 0.05 ms
    after : 5.58 ms +- 0.06 ms

[1] https://stackoverflow.com/questions/384502

[2] https://github.com/python/cpython/blob/v3.8.0b4/Objects/stringlib/find_max_char.h#L9

Maybe there can be more optimizations, so I didn't prepare a PR for this.

----------
components: Interpreter Core
messages: 352970
nosy: Ma Lin, inada.naoki, serhiy.storchaka, sir-sigurd
priority: normal
severity: normal
status: open
title: micro-optimize ucs1lib_find_max_char in Windows 64-bit build
type: performance
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38252>
_______________________________________

From report at bugs.python.org  Sun Sep 22 12:03:34 2019
From: report at bugs.python.org (hai shi)
Date: Sun, 22 Sep 2019 16:03:34 +0000
Subject: [New-bugs-announce] [issue38253] Fix typo of
 Py_SET_ERANGE_IF_OVERFLOW in pyport.h
Message-ID: <1569168214.54.0.709512149535.issue38253@roundup.psfhosted.org>


New submission from hai shi <shihai1991 at 126.com>:

Py_SET_ERANGE_ON_OVERFLOW should be changed to Py_SET_ERANGE_IF_OVERFLOW in pyport.h.

----------
assignee: docs at python
components: Documentation
messages: 352981
nosy: docs at python, shihai1991
priority: normal
severity: normal
status: open
title: Fix typo of Py_SET_ERANGE_IF_OVERFLOW in pyport.h
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38253>
_______________________________________

From report at bugs.python.org  Sun Sep 22 14:02:44 2019
From: report at bugs.python.org (David Parks)
Date: Sun, 22 Sep 2019 18:02:44 +0000
Subject: [New-bugs-announce] [issue38254] Pickle not deserializing an
 OSError exception as expected
Message-ID: <1569175364.49.0.370257231179.issue38254@roundup.psfhosted.org>


New submission from David Parks <davidparks21 at gmail.com>:

Below is a minimum reproducible test case of what appears to be an issue in pickle. 

Before pickling the exception ClientConnectionError, from aiohttp, the property ClientConnectionError._os_error is a PermissionError object (a subclass of OSError). After deserialization ClientConnectionError._os_error appears to be a string based on the exception that is produced. 

I'm not familiar enough with pickle to make much headway here.

```
import aiohttp
import pickle


connection_key = aiohttp.client_reqrep.ConnectionKey
ose = OSError(1, 'unittest')
cce = aiohttp.client_exceptions.ClientConnectorError(connection_key, ose)
cce_pickled = pickle.dumps(cce)
pickle.loads(cce_pickled)
```

```
Traceback (most recent call last):
  File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1758, in <module>
    main()
  File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1752, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1147, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/opt/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/project_neural_mouse/src/ARCHIVE/scratch/scratch_13.py", line 11, in <module>
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/client_exceptions.py", line 133, in __init__
    super().__init__(os_error.errno, os_error.strerror)
AttributeError: 'str' object has no attribute 'errno'
```

----------
components: Library (Lib)
messages: 352985
nosy: davidparks21
priority: normal
severity: normal
status: open
title: Pickle not deserializing an OSError exception as expected
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38254>
_______________________________________

From report at bugs.python.org  Mon Sep 23 03:04:42 2019
From: report at bugs.python.org (=?utf-8?b?R8Opcnk=?=)
Date: Mon, 23 Sep 2019 07:04:42 +0000
Subject: [New-bugs-announce] [issue38255] Replace "method" with "attribute"
 in the description of super()
Message-ID: <1569222282.28.0.777862000618.issue38255@roundup.psfhosted.org>


New submission from G?ry <gery.ogam at gmail.com>:

The description of `super()` uses the word "method" instead of the more general word "attribute".

> super([type[, object-or-type]])
> Return a proxy object that delegates method calls to a parent or sibling class of *type*.  This is useful for accessing inherited methods that have been overridden in a class.

`super()` is not restricted to method access but can also do data attribute access:

```
>>> class A:
...     x = True
... 
>>> class B(A):
...     x = False
... 
>>> B().x
False
>>> super(B, B()).x
True
```

I have just opened a PR to address this issue.

----------
assignee: docs at python
components: Documentation
messages: 352991
nosy: docs at python, maggyero, rhettinger
priority: normal
pull_requests: 15908
severity: normal
status: open
title: Replace "method" with "attribute" in the description of super()
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38255>
_______________________________________

From report at bugs.python.org  Mon Sep 23 03:48:21 2019
From: report at bugs.python.org (marko kreen)
Date: Mon, 23 Sep 2019 07:48:21 +0000
Subject: [New-bugs-announce] [issue38256] binascii.crc32 is not 64-bit clean
Message-ID: <1569224901.03.0.176047397678.issue38256@roundup.psfhosted.org>


New submission from marko kreen <markokr at gmail.com>:

Following code:
----------------------------
import binascii, zlib
bigdata=memoryview(bytearray((1<<32) + 100))

print(binascii.crc32(bigdata))
crc = binascii.crc32(bigdata[:1000])
crc = binascii.crc32(bigdata[1000:], crc)
print(crc)

print(zlib.crc32(bigdata))
crc = zlib.crc32(bigdata[:1000])
crc = zlib.crc32(bigdata[1000:], crc)
print(crc)
---------------------------
Outputs with python3.7

$ python3.7 x.py 
2575877834
2838121701
2838121701
2838121701

----------
components: Library (Lib)
messages: 352992
nosy: zmk
priority: normal
severity: normal
status: open
title: binascii.crc32 is not 64-bit clean
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38256>
_______________________________________

From report at bugs.python.org  Mon Sep 23 07:48:50 2019
From: report at bugs.python.org (SivaKumar NSK)
Date: Mon, 23 Sep 2019 11:48:50 +0000
Subject: [New-bugs-announce] [issue38257] __pycache__ directory with World
 writable permission
Message-ID: <1569239330.58.0.427836280782.issue38257@roundup.psfhosted.org>


New submission from SivaKumar NSK <kavinsivak at gmail.com>:

Hello All,

I am having some python script, which will be loaded when I run lpinfo -v in Manjaro Linux Distro (18.0.4) which has Python 3.7 , it is creating _pycache__ directory with full permission, not root user can edit those file.

May i know on what basis the permission of the __pycache__ directory has been decided.

Thanks in advance.

----------
messages: 353009
nosy: SivaKumar NSK
priority: normal
severity: normal
status: open
title: __pycache__ directory with World writable permission
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38257>
_______________________________________

From report at bugs.python.org  Mon Sep 23 10:35:01 2019
From: report at bugs.python.org (Sebastian Ernst)
Date: Mon, 23 Sep 2019 14:35:01 +0000
Subject: [New-bugs-announce] [issue38258] ctypes ignores when a DLL function
 is called with too many arguments
Message-ID: <1569249301.19.0.433627246846.issue38258@roundup.psfhosted.org>


New submission from Sebastian Ernst <ernst at pleiszenburg.de>:

A c-function with the following signature ...

```C
int16_t __stdcall __declspec(dllimport) square_int(
	int16_t a
	);
```

... is being called with ctypes:

```python
def test_error_callargs_unconfigured_too_many_args():

	dll = ctypes.windll.LoadLibrary('tests/demo_dll.dll')
	square_int = dll.square_int

	with pytest.raises(ValueError):
		a = square_int(1, 2, 3)
```

Expected result: If the function is called with too many (positional) arguments (in the example 3 instead of 1), a `ValueError` should be raised. This is the case for at least CPython 3.4 to 3.7.

Actual result: "Nothing", i.e. no exception. The described test "fails". The function is called without an error - with CPython 3.8.0b4.

If this behavior is intended, is has not been (as far as I can tell) documented.

----------
components: ctypes
messages: 353021
nosy: smernst
priority: normal
severity: normal
status: open
title: ctypes ignores when a DLL function is called with too many arguments
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38258>
_______________________________________

From report at bugs.python.org  Mon Sep 23 10:43:00 2019
From: report at bugs.python.org (Yair Bonastre)
Date: Mon, 23 Sep 2019 14:43:00 +0000
Subject: [New-bugs-announce] [issue38259] There should be a PriorityQueue in
 multiprocessing module
Message-ID: <1569249780.79.0.316857326171.issue38259@roundup.psfhosted.org>


New submission from Yair Bonastre <bonastreyair at gmail.com>:

In Python 3.7 queue module has Queue, LifoQueue, PriorityQueue, SimpleQueue types and in multiprocessing.queue has Queue, SimpleQueue, JoinableQueue.

Why not adding PriorityQueue to multiprocessing.queue?

Now the only way to do it is by making usage of SyncManager check the solution: https://stackoverflow.com/questions/25324560/strange-queue-priorityqueue-behaviour-with-multiprocessing-in-python-2-7-6?answertab=active#tab-top

----------
messages: 353022
nosy: Yair Bonastre
priority: normal
severity: normal
status: open
title: There should be a PriorityQueue in multiprocessing module
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38259>
_______________________________________

From report at bugs.python.org  Mon Sep 23 11:40:12 2019
From: report at bugs.python.org (Luis E.)
Date: Mon, 23 Sep 2019 15:40:12 +0000
Subject: [New-bugs-announce] [issue38260] asyncio.run documentation does not
 mention its return value
Message-ID: <1569253212.87.0.131241104419.issue38260@roundup.psfhosted.org>


New submission from Luis E. <edd at asustin.net>:

The documentation for asyncio.run (https://docs.python.org/3/library/asyncio-task.html#asyncio.run) does not mention the function's return value or lack of one.

Looking at the source, its clear it returns the passed coroutine's value via loop.run_until_complete, but the documentation or the provided example do not make it clear.

----------
assignee: docs at python
components: Documentation, asyncio
messages: 353033
nosy: asvetlov, docs at python, edd07, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.run documentation does not mention its return value
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38260>
_______________________________________

From report at bugs.python.org  Tue Sep 24 00:35:40 2019
From: report at bugs.python.org (Ryang Sohn)
Date: Tue, 24 Sep 2019 04:35:40 +0000
Subject: [New-bugs-announce] [issue38261] Tkinter CheckButton default label
 white in macOS dark mode
Message-ID: <1569299740.39.0.742587153558.issue38261@roundup.psfhosted.org>


New submission from Ryang Sohn <loop.infinitely at gmail.com>:

In macOS, when "Dark Mode" is turned on, TKinter's checkbutton's default label color is white, not black.

----------
components: Tkinter, macOS
files: tkinter_checkbox.py
messages: 353056
nosy: Ryang Sohn, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Tkinter CheckButton default label white in macOS dark mode
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48620/tkinter_checkbox.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38261>
_______________________________________

From report at bugs.python.org  Tue Sep 24 06:11:33 2019
From: report at bugs.python.org (Arno-Can Uestuensoez)
Date: Tue, 24 Sep 2019 10:11:33 +0000
Subject: [New-bugs-announce] [issue38262] Mixins - super calls in bases of
 multiple-inheritance with different parameters
Message-ID: <1569319893.51.0.228639226849.issue38262@roundup.psfhosted.org>


New submission from Arno-Can Uestuensoez <acue.opensource at gmail.com>:

The attached examples evaluate inheritance by mixins with different
arameters and in different orders.

The 'super()' call propagates the call resolution by definition
to the next level, and cares about multiple inheritance as well
as mixins. Where mixins are the superior case as they represent multiple inheritance in any case due to the common top class 'object'.
So far perfect.

The oddity I came around is the simple case of mixins with different
parameters, where both are derived from 'object'. It is obvious, that such a class has to comply to the parents call interface - here object - when to be used standalone. BUT when I ignore this and set intentionally a parameter - for test-purposes of mixins only - the resolution algorithm seems to normalize both for the final call without parameters -  as soon as one of the mixins is a non-parameter call. This is independent from the inheritance order for the derived class.

I expect for the rightmost of the inherited classes to pass the parameters literally.

When both classes have parameters, the parameter is passed to 'object' and correctly raises an exception.

#----------------------------------------------------------

[acue at lap001 mixin-order-with-different-params]$ ./call.sh 

************************

different parameters for mixin classes at diffeent positions

  0: super() method signature has no param
  1: super() method signature has param

final top-class is 'object()' which does not know the 
parameter - here 'propagate'

************************

#---------------
#
# CALL: mixin_C_is_A1B0.py
#
B:C
A:C
C:C

#---------------
#
# CALL: mixin_C_is_A1B1.py
#
Traceback (most recent call last):
  File "mixin_C_is_A1B1.py", line 24, in <module>
    c=C(True)        
  File "mixin_C_is_A1B1.py", line 20, in __init__
    super(C, self).__init__(propagate)
  File "mixin_C_is_A1B1.py", line 4, in __init__
    super(A, self).__init__(propagate)
  File "mixin_C_is_A1B1.py", line 12, in __init__
    super(B, self).__init__(propagate)
TypeError: object.__init__() takes no parameters

#---------------
#
# CALL: mixin_C_is_B0A1.py
#
A:C
B:C
C:C

#---------------
#
# CALL: mixin_C_is_B1A1.py
#
Traceback (most recent call last):
  File "mixin_C_is_B1A1.py", line 25, in <module>
    c=C(True)        
  File "mixin_C_is_B1A1.py", line 20, in __init__
    super(C, self).__init__(propagate)
  File "mixin_C_is_B1A1.py", line 12, in __init__
    super(B, self).__init__(propagate)
  File "mixin_C_is_B1A1.py", line 4, in __init__
    super(A, self).__init__(propagate)
TypeError: object.__init__() takes no parameters

[acue at lap001 mixin-order-with-different-params]$ 


#----------------------------------------------


Is this intentional?

If so, what is the explanation? - Because it potentially hides bugs for later debugging...

----------
files: tests.tar.gz
messages: 353062
nosy: acue
priority: normal
severity: normal
status: open
title: Mixins - super calls in bases of multiple-inheritance with different parameters
type: compile error
versions: Python 2.7, Python 3.6
Added file: https://bugs.python.org/file48621/tests.tar.gz

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38262>
_______________________________________

From report at bugs.python.org  Tue Sep 24 06:36:39 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Sep 2019 10:36:39 +0000
Subject: [New-bugs-announce] [issue38263] [Windows] multiprocessing:
 DupHandle.detach() race condition on
 DuplicateHandle(DUPLICATE_CLOSE_SOURCE)
Message-ID: <1569321399.19.0.207011593125.issue38263@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

On Windows, the multiprocessing DupHandle.detach() method has race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE).

Error on duplicate():

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 107, in spawn_main
    new_handle = reduction.duplicate(pipe_handle,
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Access is denied

Example: bpo-34714


Error on detach():

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 117, in spawn_main
    exitcode = _main(fd)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 127, in _main
    self = reduction.pickle.load(from_parent)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\connection.py", line 951, in rebuild_pipe_connection
    handle = dh.detach()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py", line 133, in detach
    self._access, False, _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied

Example: bpo-34513

----------
components: Library (Lib)
messages: 353064
nosy: pablogsal, pitrou, vstinner
priority: normal
severity: normal
status: open
title: [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38263>
_______________________________________

From report at bugs.python.org  Tue Sep 24 13:56:57 2019
From: report at bugs.python.org (Vinay Sajip)
Date: Tue, 24 Sep 2019 17:56:57 +0000
Subject: [New-bugs-announce] [issue38264] Makefile appears broken - touching
 header causes no build
Message-ID: <1569347817.5.0.215967707465.issue38264@roundup.psfhosted.org>


New submission from Vinay Sajip <vinay_sajip at yahoo.co.uk>:

There appears to be a failure in the Makefile build logic. Steps to reproduce:

1. Make a fresh build using "make clean && make"
2. run "touch Modules/_io/_iomodule.h"
3. Run "make" again.

Observed result: No related files get built.
Expected result: The directly including compilation units Modules/_io/{_iomodule,iobase,fileio,bufferedio,textio,bytesio,stringio}.o should be built. The compilation units which depend on these (e.g. Modules/textio.o) should also be built.

The problem appears to be related to Modules/makesetup, which adds rules to tbe Makefile. An example of a rule is

Modules/textio.o: $(srcdir)/Modules/_io/textio.c; $(CC) $(PY_BUILTIN_MODULE_CFLAGS)  -DPy_BUILD_CORE_BUILTIN -I$(srcdir)/Include/internal -I$(srcdir)/Modules/_io -c $(srcdir)/Modules/_io/textio.c -o Modules/textio.o

The problem here looks like the lack of a dependency between Modules/_io/textio.o and Modules/_io/textio.o - there is only a dependency on Modules/_io/textio.c, which wouldn't be rebuilt on a touch of Modules/_io/_iomodule.h.

----------
components: Build
messages: 353105
nosy: vinay.sajip
priority: high
severity: normal
status: open
title: Makefile appears broken - touching header causes no build
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38264>
_______________________________________

From report at bugs.python.org  Tue Sep 24 14:14:15 2019
From: report at bugs.python.org (Dong-hee Na)
Date: Tue, 24 Sep 2019 18:14:15 +0000
Subject: [New-bugs-announce] [issue38265] Update os.pread to accept the
 length type as size_t
Message-ID: <1569348855.48.0.735036515408.issue38265@roundup.psfhosted.org>


New submission from Dong-hee Na <donghee.na92 at gmail.com>:

I've found this comments.
// TODO length should be size_t!  but Python doesn't support parsing size_t yet.
(https://github.com/python/cpython/blob/279f44678c8b84a183f9eeb85e0b086228154497/Modules/posixmodule.c#L8830)


I don't know when this comment was created.
But at this point, it seems to provide that functionality.
It can be simply solved through the clinic.

How about updating this argument to be size_t and remove the TODO comment?

----------
components: Library (Lib)
messages: 353107
nosy: corona10, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Update os.pread to accept the length type as size_t
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38265>
_______________________________________

From report at bugs.python.org  Tue Sep 24 15:20:41 2019
From: report at bugs.python.org (Wenzel Jakob)
Date: Tue, 24 Sep 2019 19:20:41 +0000
Subject: [New-bugs-announce] [issue38266] Revert bpo-37878: Make
 PyThreadState_DeleteCurrent() Internal
Message-ID: <1569352841.72.0.766028949986.issue38266@roundup.psfhosted.org>


New submission from Wenzel Jakob <wenzel at inf.ethz.ch>:

A commit from a few days ago and discussed in issue #37878 removed an undocumented function PyThreadState_DeleteCurrent() from Python's public API.

This function was exposed for good reasons and simply removing it because it is undocumented strikes me as a somewhat brutal solution to achieve documentation coverage. This function is useful -- even essential -- to delete the current thread's thread state, which cannot be done using the still-public PyThreadState_Delete(). The documentation could simply be a line stating this explicitly.

I was asked to provide an example of an actual usage: this function is used by pybind11, which is a widely used library for creating Python bindings to C++ code. pybind11 only calls PyThreadState_DeleteCurrent() in a rare set of circumstances, but there is no alternative in this case. Specifically, pybind11 can intercept a Python function call on the main() thread and delete the associated thread state, launching a new thread and continuing Python execution there (with a newly created thread state).

Kind of crazy, so why is this useful? The answer is UI libraries. On some platforms, it is not legal to poll UI events on any thread other than the main thread. This means that it's impossible to implement a proper asynchronous UI event loop because Python is hogging the main thread. With the functionality in pybind11's gil_scoped_acquire, it is possible can launch an event polling loop on the main thread, continue running an interactive Python REPL on another thread, and even swap them back e.g. when the user interface is no longer needed.

Best,
Wenzel

----------
components: Interpreter Core
messages: 353114
nosy: eric.snow, nanjekyejoannah, ncoghlan, tcaswell, vstinner, wenzel
priority: normal
severity: normal
status: open
title: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38266>
_______________________________________

From report at bugs.python.org  Tue Sep 24 15:31:44 2019
From: report at bugs.python.org (Kyle Stanley)
Date: Tue, 24 Sep 2019 19:31:44 +0000
Subject: [New-bugs-announce] [issue38267] Add thread timeout for
 loop.shutdown_default_executor
Message-ID: <1569353504.23.0.896727230476.issue38267@roundup.psfhosted.org>


New submission from Kyle Stanley <aeros167 at gmail.com>:

Currently, for the recently added coroutine `loop.shutdown_default_executor()`, the executor shutdown can wait indefinitely for the threads to join. Under normal circumstances, waiting on the threads is appropriate, but there should be a timeout duration in the situation that the threads unable to finish joining. 

The motivation for this was based on the comments from Andrew Svetlov and Yury Selivanov in GH-16284.

The original idea from Andrew was to add the timeout duration as a default for a new parameter in `asyncio.run()` and `loop.shutdown_default_executor()`. However, Yury would prefer for this to be defined as a constant instead and not as a parameter for `asyncio.run()` to avoid the creation of an excessive number of parameters to tweak for the user.

I will attach a PR that adds the constant and the parameter for `loop.shutdown_default_executor()`, which will passed as an argument to `thread.join()`.

----------
messages: 353115
nosy: aeros167, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Add thread timeout for loop.shutdown_default_executor
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38267>
_______________________________________

From report at bugs.python.org  Wed Sep 25 03:00:11 2019
From: report at bugs.python.org (Sophie)
Date: Wed, 25 Sep 2019 07:00:11 +0000
Subject: [New-bugs-announce] [issue38268] Unclear definition of ceil,
 floor and trunc in documentation mathmodule.c
Message-ID: <1569394811.85.0.726445638356.issue38268@roundup.psfhosted.org>


New submission from Sophie <sophiecrespo09 at gmail.com>:

The functions ceil(), floor() and trunc() define their result as an Integral. But an Integral is the result of an integration calculation. Better say Integer IMHO.

----------
assignee: docs at python
components: Documentation
messages: 353151
nosy: Sophie, docs at python
priority: normal
pull_requests: 15958
severity: normal
status: open
title: Unclear definition of ceil, floor and trunc in documentation mathmodule.c
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38268>
_______________________________________

From report at bugs.python.org  Wed Sep 25 03:40:46 2019
From: report at bugs.python.org (Dong-hee Na)
Date: Wed, 25 Sep 2019 07:40:46 +0000
Subject: [New-bugs-announce] [issue38269] test_venv failed on AMD64 Debian
 PGO 3.x
Message-ID: <1569397246.19.0.5627167067.issue38269@roundup.psfhosted.org>


New submission from Dong-hee Na <donghee.na92 at gmail.com>:

https://buildbot.python.org/all/#/builders/47/builds/3578

This issue was found from GH-16359

The main issue of this failure is the lack of storage space.

OSError: [Errno 28] No space left on device: '/tmp/tmpnmcjxia9/bin/python' -> '/tmp/tmpnmcjxia9/bin/python3'

----------
components: Tests
messages: 353154
nosy: corona10, steve.dower
priority: normal
severity: normal
status: open
title: test_venv failed on AMD64 Debian PGO 3.x
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38269>
_______________________________________

From report at bugs.python.org  Wed Sep 25 06:09:58 2019
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 25 Sep 2019 10:09:58 +0000
Subject: [New-bugs-announce] [issue38270] Tests: Avoid MD5 or check for MD5
 availablity
Message-ID: <1569406198.76.0.145895264341.issue38270@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

MD5 hash digest algorithm has known security issues and is getting blocked on systems with strict security policies.

Python uses MD5 in several unittests. These tests should either avoid MD5 and use SHA256 instead. Or tests that really require MD5 should check for the availability of MD5 and skip these tests. The latter are network-related tests that check for digest auth or CRAM-MD5 auth.

Avoidable use of MD5:
* test_hmac tests for default constructor
* test_tarfile checksum

Tests that require MD5:
* test_urllib2_localnet digest auth
* CRAM-MD5 in test_imaplib, test_poplib, test_smtplib

----------
assignee: christian.heimes
components: Tests
messages: 353162
nosy: christian.heimes
priority: high
severity: normal
status: open
title: Tests: Avoid MD5 or check for MD5 availablity
type: resource usage
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38270>
_______________________________________

From report at bugs.python.org  Wed Sep 25 07:10:15 2019
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 25 Sep 2019 11:10:15 +0000
Subject: [New-bugs-announce] [issue38271] test_ssl: replace 3DES with AES
 encrypted private keys
Message-ID: <1569409815.77.0.296324311407.issue38271@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

test_ssl uses two encrypted private key files to verify the password hook. The private key files are currently encrypted with 3DES (DES-EDE3-CBC). The algorithm is very old and blocked by some crypto policies. Let's use AES256 instead. AES is mandatory in all supported OpenSSL versions.

----------
components: Tests
messages: 353173
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: test_ssl: replace 3DES with AES encrypted private keys
type: resource usage
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38271>
_______________________________________

From report at bugs.python.org  Wed Sep 25 07:20:34 2019
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Sep 2019 11:20:34 +0000
Subject: [New-bugs-announce] [issue38272] test_ctypes:
 test_array_in_struct() failed on ARMv7 Debian buster 3.7
Message-ID: <1569410434.07.0.844620154982.issue38272@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

I bet on a regression caused by bpo-22273, commit/16c0f6df62a39f9f7712b1c0577de4eefcb4c1bf.

ARMv7 Debian buster 3.7:
https://buildbot.python.org/all/#/builders/177/builds/396

======================================================================
FAIL: test_array_in_struct (ctypes.test.test_structures.StructureTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/ctypes/test/test_structures.py", line 478, in test_array_in_struct
    self.assertEqual(result, expected)
AssertionError: 3.923919392286974e+252 != 5.85987

----------
components: Tests
messages: 353175
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_ctypes: test_array_in_struct() failed on ARMv7 Debian buster 3.7
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38272>
_______________________________________

From report at bugs.python.org  Wed Sep 25 07:22:40 2019
From: report at bugs.python.org (hai shi)
Date: Wed, 25 Sep 2019 11:22:40 +0000
Subject: [New-bugs-announce] [issue38273] Got an compile failed info about
 typeobject.c
Message-ID: <1569410560.15.0.382291774313.issue38273@roundup.psfhosted.org>


New submission from hai shi <shihai1991 at 126.com>:

Got an compile error, my OS is gcc version 4.8.5 20150623 (Red Hat 4.8.5-36:
 CC='gcc -pthread' LDSHARED='gcc -pthread -shared    ' OPT='-g -Og -Wall'       _TCLTK_INCLUDES='' _TCLTK_LIBS=''       ./python -E ./setup.py  build
python: Objects/typeobject.c:2867: PyType_FromSpecWithBases: Assertion `memb->type == 19' failed.
/bin/sh: line 9: 32047 Aborted                 CC='gcc -pthread' LDSHARED='gcc -pthread -shared    ' OPT='-g -Og -Wall' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py $quiet build
make: *** [sharedmods] Error 134

the relevant bpo is https://bugs.python.org/issue38140, i have no enough time see the detail, so just report it for now.

----------
components: Interpreter Core
messages: 353178
nosy: shihai1991
priority: normal
severity: normal
status: open
title: Got an compile failed info about typeobject.c
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38273>
_______________________________________

From report at bugs.python.org  Wed Sep 25 07:54:57 2019
From: report at bugs.python.org (Derbe Ejere)
Date: Wed, 25 Sep 2019 11:54:57 +0000
Subject: [New-bugs-announce] [issue38274] sementic errors?
Message-ID: <1569412497.64.0.259873259242.issue38274@roundup.psfhosted.org>


New submission from Derbe Ejere <barataart2 at gmail.com>:

a = "hello"
b = "4.3" 

1.Both variables a  and b are type of object String (can be checked with type()
2.Change both to float using float(). 
variable a cannot be changed to float since it is str object. However,
variable b can be changed to float giving 4.3 eventhough it is also a str object. 

CONTRADICTION?!

Thanks,

Derbe Ejere

----------
messages: 353184
nosy: dejere
priority: normal
severity: normal
status: open
title: sementic errors?

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38274>
_______________________________________

From report at bugs.python.org  Wed Sep 25 08:38:51 2019
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 25 Sep 2019 12:38:51 +0000
Subject: [New-bugs-announce] [issue38275] test_ssl: skip tests for disabled
 TLS/SSL versions
Message-ID: <1569415131.43.0.910355334237.issue38275@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

test_ssl has some checks for TLS/SSL versions that are not available or disabled during compile-time of Python and OpenSSL. It's missing run-time checks and checks for TLS 1.0 and 1.1. Several test scenarios assume that TLS 1.0 and 1.1 are always available and properly working. These tests fail when e.g. TLS 1.0 is disabled.

Further more compile-time checks for ssl.HAS_TLSv1 or ssl.PROTOCOL_TLSv1 are not sufficient. OpenSSL 1.1.0+ got more flexible. TLS/SSL versions can be disabled at runtime with a crypto policy. Recent Debian, Fedora, and RHEL 8 systems have TLS 1.0 and 1.1 disabled by default.

----------
assignee: christian.heimes
components: Tests
messages: 353189
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: test_ssl: skip tests for disabled TLS/SSL versions
type: resource usage
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38275>
_______________________________________

From report at bugs.python.org  Wed Sep 25 09:09:13 2019
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Sep 2019 13:09:13 +0000
Subject: [New-bugs-announce] [issue38276] test_asyncio:
 test_cancel_make_subprocess_transport_exec() failed on RHEL7 LTO + PGO 3.x
Message-ID: <1569416953.01.0.173802643321.issue38276@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

test_asyncio failed once, but then passed when re-run in verbose mode.

RHEL7 LTO + PGO 3.x:
https://buildbot.python.org/all/#/builders/258/builds/31

running: test_asyncio (9 min 21 sec)
running: test_asyncio (9 min 51 sec)
running: test_asyncio (10 min 21 sec)
running: test_asyncio (10 min 51 sec)
running: test_asyncio (11 min 21 sec)
running: test_asyncio (11 min 51 sec)
running: test_asyncio (12 min 21 sec)
running: test_asyncio (12 min 51 sec)
running: test_asyncio (13 min 21 sec)
running: test_asyncio (13 min 51 sec)
running: test_asyncio (14 min 21 sec)
running: test_asyncio (14 min 51 sec)
0:15:57 load avg: 0.01 [419/419/1] test_asyncio crashed (Exit code 1)
Timeout (0:15:00)!
Thread 0x00007f76b16ad740 (most recent call first):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/selectors.py", line 468 in select
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/asyncio/base_events.py", line 1837 in _run_once
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/asyncio/base_events.py", line 589 in run_forever
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/asyncio/base_events.py", line 621 in run_until_complete
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/test_asyncio/test_subprocess.py", line 440 in test_cancel_make_subprocess_transport_exec
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/case.py", line 616 in _callTestMethod
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/case.py", line 659 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/case.py", line 719 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/unittest/runner.py", line 176 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/support/__init__.py", line 1996 in _run_suite
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/support/__init__.py", line 2092 in run_unittest
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/runtest.py", line 209 in _test_module
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/runtest.py", line 234 in _runtest_inner2
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/runtest.py", line 270 in _runtest_inner
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/runtest.py", line 140 in _runtest
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/runtest.py", line 193 in runtest
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/runtest_mp.py", line 67 in run_tests_worker
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/main.py", line 651 in _main
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/main.py", line 631 in main
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/libregrtest/main.py", line 704 in main
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/regrtest.py", line 43 in _main
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/regrtest.py", line 47 in <module>
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/runpy.py", line 85 in _run_code
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/runpy.py", line 192 in _run_module_as_main

----------
messages: 353192
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_asyncio: test_cancel_make_subprocess_transport_exec() failed on RHEL7 LTO + PGO 3.x

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38276>
_______________________________________

From report at bugs.python.org  Wed Sep 25 10:20:15 2019
From: report at bugs.python.org (Arne Recknagel)
Date: Wed, 25 Sep 2019 14:20:15 +0000
Subject: [New-bugs-announce] [issue38277] Allowing conditions with
 assignment expressions in comprehensions without parantheses
Message-ID: <1569421215.61.0.96035385426.issue38277@roundup.psfhosted.org>


New submission from Arne Recknagel <arne.recknagel at hotmail.com>:

All code is run on python build from the current 3.8 branch.

The following doesn't work:

  >>> [x for x in 'foo' if y := True]
    File "<stdin>", line 1
      [x for x in 'foo' if y := True]
                             ^
  SyntaxError: invalid syntax

While this does:

  >>> [x for x in 'foo' if (y := True)]
  ['f', 'o', 'o']

Since assignment expressions work without parentheses in normal if-clauses and there being no mention on it explicitly being disallowed, this seems to either be a bug or an oversight in the documentation (with my own opinion, for what it's worth, being that it's a bug).

----------
messages: 353203
nosy: arne, emilyemorehouse
priority: normal
severity: normal
status: open
title: Allowing conditions with assignment expressions in comprehensions without parantheses
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38277>
_______________________________________

From report at bugs.python.org  Wed Sep 25 10:35:17 2019
From: report at bugs.python.org (Ben Spiller)
Date: Wed, 25 Sep 2019 14:35:17 +0000
Subject: [New-bugs-announce] [issue38278] Need a more efficient way to
 perform dict.get(key, default)
Message-ID: <1569422117.26.0.730288338211.issue38278@roundup.psfhosted.org>


New submission from Ben Spiller <spiller.ben at gmail.com>:

In performance-critical python code, it's quite common to need to get an item from a dictionary, falling back on a default (e.g. None, 0 etc) if it doesn't yet exist. The obvious way to do this based on the documentation is to call the dict.get() method:

> python -m timeit -s "d={'abc':123}" "x=d.get('abc',None)"
5000000 loops, best of 5: 74.6 nsec per loop

... however the performance of natural approach is very poor (2.2 times slower!) compared to the time really needed to look up the value:
>python -m timeit -s "d={'abc':123}" "x=d['abc']"
5000000 loops, best of 5: 33 nsec per loop

There are various ways to do this more efficiently, but they all have significant performance or functional drawbacks:

custom dict subclass with __missing__() override: promising approach, but use of a custom class instead of dict seems to increase [] cost significantly:
>python -m timeit -s "class mydict(dict):" -s "  def __missing__(self,key):return None" -s "d = mydict({'abc':123})" "x=d['abc']"
5000000 loops, best of 5: 60.4 nsec per loop

get() with caching of function lookup - somewhat better but not great:
>python -m timeit -s "d={'abc':123}; G=d.get" "x=G('abc',None)"
5000000 loops, best of 5: 59.8 nsec per loop

[] and "in" (inevitably a bit slow due to needing to do the lookup twice):
>python -m timeit -s "d={'abc':123}" "x=d['abc'] if 'abc' in d else None"
5000000 loops, best of 5: 53.9 nsec per loop

try/except approach: quickest solution if it exists, but clunky syntax, and VERY slow if it doesn't exist
>python -m timeit -s "d={'abc':123}" "try:" "   x=d['abc']" "except KeyError: pass"
5000000 loops, best of 5: 41.8 nsec per loop
>python -m timeit -s "d={'abc':123}" "try:" "   x=d['XXX']" "except KeyError: pass"
1000000 loops, best of 5: 174 nsec per loop

collections.defaultdict: reasonable performance if key exists, but unwanted behaviour of adding the key if missing (which if used with an unbounded universe of keys could produce a memory leak):
>python -m timeit -s "import collections; d=collections.defaultdict(lambda: None); d['abc']=123; " "x=d['XXX']"
5000000 loops, best of 5: 34.3 nsec per loop

I bet we can do better! 

Lots of solutions are possible - maybe some kind of peephole optimization to make dict.get() itself perform similarly to the [] operator, or if that's challenging perhaps providing a class or option that behaves like defaultdict but without the auto-adding behaviour and with comparable [] performance to the "dict" type - for example dict.raiseExceptionOnMissing=False, or perhaps even some kind of new syntax (e.g. dict['key', default=None]). Which option would be easiest/nicest?

----------
components: Interpreter Core
messages: 353206
nosy: benspiller
priority: normal
severity: normal
status: open
title: Need a more efficient way to perform dict.get(key, default)
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38278>
_______________________________________

From report at bugs.python.org  Wed Sep 25 13:37:14 2019
From: report at bugs.python.org (Max Voss)
Date: Wed, 25 Sep 2019 17:37:14 +0000
Subject: [New-bugs-announce] [issue38279] multiprocessing example enhancement
Message-ID: <1569433034.47.0.652580755416.issue38279@roundup.psfhosted.org>


New submission from Max Voss <bruno.m.voss at gmail.com>:

Hello all,

I've been trying to understand multiprocessing for a while, I tried multiple times. The PR is a suggested enhancement to the example that made it "click" for me. Or should I say, produced a working result that made sense to me.

Details for each change in the PR. It's short too.

The concept of multiprocessing is easy enough, but the syntax is so unlike regular python and so much happens "behind the curtain" so to speak, it took me a while. When I looked for multiprocessing advice online, many answers seemed unsure if or how their solution worked.

Generally I'd like to help write documentation. So this is a test to see how good your issue handling process is too. :)

----------
assignee: docs at python
components: Documentation
messages: 353222
nosy: BMV, docs at python
priority: normal
severity: normal
status: open
title: multiprocessing example enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38279>
_______________________________________

From report at bugs.python.org  Wed Sep 25 15:05:18 2019
From: report at bugs.python.org (WHK Yan)
Date: Wed, 25 Sep 2019 19:05:18 +0000
Subject: [New-bugs-announce] [issue38280] Feature request: Add support to
 compile to native binary like as golang
Message-ID: <1569438318.78.0.543732572838.issue38280@roundup.psfhosted.org>


New submission from WHK Yan <yan.uniko.102 at gmail.com>:

It would be very useful to be able to compile at a low level a complete Python functional script and generate a native binary for each type of operating system and architecture just like golang does.

For example:

Run script: $ go run script.go
Compile script: $ GOOS=linux GOARCH=amd64 go build -o test test.go

The generated binary does not require goland installed to be executed and does not require complex dependencies. This allows me to distribute applications in binary are an excellent performance without requiring the installation of the entire interpreter.

Golang's problem is that it is not an intuitive language and the learning curve is very long, they had a very good idea but they carried it out in a horrible way.

On the other hand, python is very flexible and very intuitive, the learning curve is very short, I learned python in less than a week and I have been creating python applications for several years, instead I have been trying to learn golang for months and it is excessively strange.

Why not combine the best of both worlds? Unite non-dependence and cross-platform distribution without relying on the interpreter and the most versatile and intuitive language?

For example:

Run script: $ python script.py
Compile pyc $ python -m compileall
Compile native: $ python -m ncompileall -k linux -a amd_64

When compileall generate the pyc files and ncompileall generate a native binaries without complex dependencies or python dependencies.

This would be a great advantage in equipment that does not require additional installation of components.

For example, on servers of some companies they do not allow you to install something that can affect the entire environment or ecosystem, in addition, in many cases the scripts require a specific version of python and cause problems when different versions are required to other applications, the complete compilation at low level would allow to resolve this conflict.

There are currently few projects that allow you to translate a python script to binary but most do not have support or do not have a good performance, if python already brought it integrated it would increase its use even more since it would not depend on the interpreter to be able to distribute a complete application that already include all its modules and others.

----------
components: Cross-Build
messages: 353224
nosy: Alex.Willmer, WHK Yan
priority: normal
severity: normal
status: open
title: Feature request: Add support to compile to native binary like as golang
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38280>
_______________________________________

From report at bugs.python.org  Wed Sep 25 20:49:11 2019
From: report at bugs.python.org (__starrify__)
Date: Thu, 26 Sep 2019 00:49:11 +0000
Subject: [New-bugs-announce] [issue38281] "+" ignored when determining file
 mode in gzip.GzipFile.__init__
Message-ID: <1569458951.07.0.462687490568.issue38281@roundup.psfhosted.org>


New submission from __starrify__ <pengyu at libstarrify.so>:

Currently (as of df69e75) it checks only the initial letter (via `mode.startswith`) of "mode" in gzip.GzipFile.__init__. See also: https://github.com/python/cpython/blob/df69e75/Lib/gzip.py#L183

I'd personally suggest that it takes also "+" into consideration, which shall be regarded as writable.

One common (hm.. at least I've observed more than once recently) case where people may make mistakes is to use a tempfile.NamedTemporaryFile as the file object without specifying explicitly a mode, while expecting the GzipFile to be writable.

A quick example (in Python 3.7.11):
```
>>> import tempfile
>>> tempfile.NamedTemporaryFile().mode
'rb+'
>>> import gzip
>>> gzip.GzipFile(fileobj=tempfile.NamedTemporaryFile(mode='rb+')).writable()
False
>>> gzip.GzipFile(fileobj=tempfile.NamedTemporaryFile(mode='wb')).writable()
True
```

Unfortunately this change request may be backward-incompatible, since previously modes like "r+b" have been treated as read-only. See also: https://github.com/python/cpython/blob/df69e75/Lib/test/test_gzip.py#L464

----------
components: Library (Lib)
messages: 353244
nosy: __starrify__
priority: normal
severity: normal
status: open
title: "+" ignored when determining file mode in gzip.GzipFile.__init__
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38281>
_______________________________________

From report at bugs.python.org  Wed Sep 25 21:11:45 2019
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Sep 2019 01:11:45 +0000
Subject: [New-bugs-announce] [issue38282] socketmodule.c: _FORTIFY_SOURCE=2
 warning in AF_ALG case of getsockaddrarg()
Message-ID: <1569460305.42.0.728817437513.issue38282@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

To test bpo-32375, I just recompiled the master branch of Python twice using these flags:

* -D_FORTIFY_SOURCE=2 -Og
* -D_FORTIFY_SOURCE=2 -O3

I got a few warnings, the same that I get without FORTIFY SOURCE.

Using -D_FORTIFY_SOURCE=2 -O3, I get one warning, but it's no longer from getpath.c:

In file included from /usr/include/string.h:494,
                 from ./Include/Python.h:30,
                 from /home/vstinner/python/master/Modules/socketmodule.c:103:
In function ?memset?,
    inlined from ?getsockaddrarg? at /home/vstinner/python/master/Modules/socketmodule.c:2331:9,
    inlined from ?sock_bind? at /home/vstinner/python/master/Modules/socketmodule.c:3113:10:
/usr/include/bits/string_fortified.h:71:10: warning: ?__builtin_memset? offset [17, 88] from the object at ?addrbuf? is out of the bounds of referenced subobject ?sa? with type ?struct sockaddr? at offset 0 [-Warray-bounds]
   71 |   return __builtin___memset_chk (__dest, __ch, __len, __bos0 (__dest));
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The warning comes from the code:

    case AF_ALG:
        ...
        struct sockaddr_alg *sa;
        sa = (struct sockaddr_alg *)addr_ret;
        memset(sa, 0, sizeof(*sa));

called from:

static PyObject *
sock_bind(PySocketSockObject *s, PyObject *addro)
{
    sock_addr_t addrbuf;
    int addrlen;
    int res;

    if (!getsockaddrarg(s, addro, SAS2SA(&addrbuf), &addrlen, "bind")) {
        return NULL;
    }

----------
components: Library (Lib)
messages: 353251
nosy: christian.heimes, pitrou, vstinner
priority: normal
severity: normal
status: open
title: socketmodule.c: _FORTIFY_SOURCE=2 warning in AF_ALG case of getsockaddrarg()
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38282>
_______________________________________

From report at bugs.python.org  Thu Sep 26 05:10:54 2019
From: report at bugs.python.org (Thomas Haller)
Date: Thu, 26 Sep 2019 09:10:54 +0000
Subject: [New-bugs-announce] [issue38283] sys._getframe(1).f_lineno changed
 behavior in 3.8
Message-ID: <1569489054.37.0.072331097916.issue38283@roundup.psfhosted.org>


New submission from Thomas Haller <thom311 at gmail.com>:

The line number for the frame object changed behaviour in 3.8.

I am seeing this on Fedora rawhide, which ships "3.8.0~b4-1.fc32" package.

That happens when wrapping lines for calling functions.

Before, the line number sys._getframe(1).f_lineno indicate the last line. Now it is the first line.



Reproducer:

>>>
#!/usr/bin/env python

import sys


def func(a, b, c, d):

    frame = sys._getframe(1)

    print('call: %s' % (a))
    print('frame:                %s' % (frame))
    print('frame.f_code.co_name: %s' % (frame.f_code.co_name))
    print('frame.f_lineno:       %s' % (frame.f_lineno))


func('call 1, 1 line', 0, 1, 2)

func('call 2, 2 lines',
     0, 1, 2)

func('call 3, 4 lines',
     0,
     1,
     2)

<<<<


with python2 and python<=3.7, the line numbers are 16, 19, 24, respectively.

With 3.8, it changes to 16, 18, 21.

----------
messages: 353279
nosy: Thomas Haller
priority: normal
severity: normal
status: open
title: sys._getframe(1).f_lineno changed behavior in 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38283>
_______________________________________

From report at bugs.python.org  Thu Sep 26 07:19:03 2019
From: report at bugs.python.org (Blindfreddy)
Date: Thu, 26 Sep 2019 11:19:03 +0000
Subject: [New-bugs-announce] [issue38284] signal.sigwait* do not intercept
 certain signals
Message-ID: <1569496743.5.0.75731685223.issue38284@roundup.psfhosted.org>


New submission from Blindfreddy <forum at roznowski.net>:

On debian, signal.sigwait/sigwaitinfo/sigtimedwait do not properly handle the below signals (NOK).

Steps to reproduce

1. start python and type:
>>> import signal
>>> signal.sigwait([<any of the below signals with NOK])
2. send corresponding signal to the python process

Signal Name	Signal Number	Result	Description / Output
SIGHUP	1	NOK	Process terminates with 'HANGUP'
SIGINT	2	OK	
SIGQUIT	3	NOK	Process terminates with 'QUIT'
SIGILL	4	OK	
SIGTRAP	5	NOK	Process terminates with 'Trace/breakpoint trap'
SIGABRT	6	OK	
SIGEMT	7	NOK	Process terminates with 'Bus error'
SIGFPE	8	OK	
SIGKILL	9	OK	cannot be caught by design
SIGBUS	10	NOK	Process terminates with 'User defined signal 1'
SIGSEGV	11	OK	
SIGSYS	12	NOK	Process terminates with 'User defined signal 2'
SIGPIPE	13	NOK	signal not handled
SIGALRM	14	NOK	Process terminates with 'Alarm clock'
SIGTERM	15	NOK	Process terminates with 'Terminated'
SIGURG	16	NOK	Process terminates with 'Stack fault'
SIGSTOP	17	NOK	signal not handled
SIGCHLD	20	OK	
SIGUSR1	30	NOK	Process terminates with 'Power failure''
SIGUSR2	31	OK

----------
messages: 353294
nosy: Blindfreddy
priority: normal
severity: normal
status: open
title: signal.sigwait* do not intercept certain signals
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38284>
_______________________________________

From report at bugs.python.org  Thu Sep 26 08:05:11 2019
From: report at bugs.python.org (=?utf-8?q?Malvers=C3=A1n?=)
Date: Thu, 26 Sep 2019 12:05:11 +0000
Subject: [New-bugs-announce] [issue38285] Asyncio BaseEventLoop can support
 socket types other than SOCK_STREAM
Message-ID: <1569499511.98.0.234005447809.issue38285@roundup.psfhosted.org>


New submission from Malvers?n <malverspam at gmail.com>:

Currently the BaseEventLoop class in asyncio has explicit checks to raise ValueError when creating a connection if the socket argument has a type other than SOCK_STREAM:
.create_connection()
.create_server()

This is also applicable for class _UnixSelectorEventLoop:
.create_unix_connection()
.create_unix_server()

But the fact is that it actually supports other socket types, like SOCK_SEQPACKET for example.

Currently you can test this by dirty-hacking the socket class "type" property to momentarily trick the event loop into thinking that any socket is of SOCK_STREAM type.

<code>
# First create an AF_UNIX, SOCK_SEQPACKET socket.
sock = socket.socket(socket.AddressFamily.AF_UNIX, socket.SocketKind.SOCK_SEQ_PACKET)
sock.connect(path)

params = { "sock" : sock, "protocol_factory" : lambda: protocol }

# Now do the trick.
hack = (params["sock"].type != socket.SocketKind.SOCK_STREAM)

if hack:
    # Substitute class property getter with fixed value getter.
    socket_property = socket.socket.type
    socket.socket.type = property(lambda self: socket.SocketKind.SOCK_STREAM, None, None,)

# Use the socket normally to create connection and run the event loop.
loop = asyncio.new_event_loop()
coroutine = loop.create_unix_connection(**params)    # It also works with .create_connection()
transport, protocol = loop.run_until_complete(coroutine)

# Revert the trick.
if hack:
    # Restore class property getter.
    socket.socket.type = socket_property
</code>

As dirty as it looks, this works flawlessy. It just tricks the event loop .create_connection() call to bypass the explicit check of using a SOCK_STREAM socket. This done, THE EVENT LOOP SUPPORTS SOCK_SEQPACKET PERFECTLY.

This is the solution I'm currently using to communicate an application with a local daemon, but I would really prefer to have the SOCK_SEQPACKET support allowed into the event loop itself. Having in mind that it simply works with other socket types, I find that limiting the use of the event loop with an explicit SOCK_STREAM-only check is somehow artificial and unrealistic.

Thanks in advance for your attention.

----------
components: asyncio
messages: 353296
nosy: asvetlov, malversan, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio BaseEventLoop can support socket types other than SOCK_STREAM
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38285>
_______________________________________

From report at bugs.python.org  Thu Sep 26 10:12:19 2019
From: report at bugs.python.org (Charles Coulombe)
Date: Thu, 26 Sep 2019 14:12:19 +0000
Subject: [New-bugs-announce] [issue38286] tarfile forgets set sgid when
 targetpath has it
Message-ID: <1569507139.8.0.49605570029.issue38286@roundup.psfhosted.org>


New submission from Charles Coulombe <charles.coulombe at calculquebec.ca>:

An archive that does not have sgid that is extracted in a directory with sgid set does not end up having its sgid set since the targetpath is chmod with the mode of the tarinfo. (Lib/tarfile.py#L2277) 

For comparison, an archive extracted with tar has the sgid bit set.

I added a patch as attachment to gather comments before making this PR.

----------
components: Library (Lib)
files: tarfile_sgid.patch
keywords: patch
messages: 353305
nosy: Charles Coulombe
priority: normal
severity: normal
status: open
title: tarfile forgets set sgid when targetpath has it
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
Added file: https://bugs.python.org/file48626/tarfile_sgid.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38286>
_______________________________________

From report at bugs.python.org  Thu Sep 26 12:29:50 2019
From: report at bugs.python.org (Pat Gunn)
Date: Thu, 26 Sep 2019 16:29:50 +0000
Subject: [New-bugs-announce] [issue38287] tempfile.TemporaryDirectory()
 should behave the same as a context manager as when used directly
Message-ID: <1569515390.19.0.594047788095.issue38287@roundup.psfhosted.org>


New submission from Pat Gunn <pgunn01 at gmail.com>:

Right now, when tempfile.TemporaryDirectory() is used as a context manager, the context variable is stringy, usable as a string containing the directory name it made. When used directly, it returns an object that does not coerce to a nice string, instead requiring the .name method to be called.

I propose adding a __str__() to the class that causes it to serialise nicely into a string containing just the directory name, so the (in my view surprising) differences in behaviour are minimised.

----------
components: Library (Lib)
messages: 353327
nosy: Pat Gunn
priority: normal
severity: normal
status: open
title: tempfile.TemporaryDirectory() should behave the same as a context manager as when used directly
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38287>
_______________________________________

From report at bugs.python.org  Thu Sep 26 12:43:50 2019
From: report at bugs.python.org (Alori)
Date: Thu, 26 Sep 2019 16:43:50 +0000
Subject: [New-bugs-announce] [issue38288] shutil.make_archive() should allow
 setting zipfile.ZipFile() 'strict_timestamps'
Message-ID: <1569516230.4.0.693896047629.issue38288@roundup.psfhosted.org>


New submission from Alori <valorien at gmail.com>:

https://bugs.python.org/issue34097 addressed a zipfile.ZipFile() issue where zip files with timestamps prior to the year 1980 could not be created. The fix adds the strict_timestamps=True|False keyword argument to the ZipFile class.

shutil.make_archive() is a high-level utility that uses ZipFile() to simplify zip file creation. Unfortunately, it doesn't support the new 'strict_timestamps' keyword which means users encountering the 'ValueError: ZIP does not support timestamps before 1980' exception must revert to using zipfile.ZipFile() instead.

I believe adding the 'strict_timestamps' keyword to shutil.make_archive() and propagating it to zipfileZipFile()

----------
components: Library (Lib)
messages: 353329
nosy: valorien
priority: normal
severity: normal
status: open
title: shutil.make_archive() should allow setting zipfile.ZipFile() 'strict_timestamps'
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38288>
_______________________________________

From report at bugs.python.org  Thu Sep 26 13:59:03 2019
From: report at bugs.python.org (=?utf-8?q?Micha=C5=82_Fronczyk?=)
Date: Thu, 26 Sep 2019 17:59:03 +0000
Subject: [New-bugs-announce] [issue38289] Add an option to point to an
 additional "site" directory
Message-ID: <1569520743.77.0.742193184131.issue38289@roundup.psfhosted.org>


New submission from Micha? Fronczyk <fronczyk at gmail.com>:

An environment variable to point to an additional site-packages directory (without disabling the one in Python itself) location would allow having different sets of extra packages that can be enabled by the users in their or shared Python installations.

It'd be helpful in situations where we'd like other teams to maintain their sets of extra packages that could be easily added by just pointing the interpreter to the location of the additional site-packages dir that would fill the rest of the sys.path using the path configuration files if needed.

Right now site.py doesn't seem to allow that and modifying it would mean that the extra packages can be used only with the modified installations of Python. 

Using PYTHONPATH is not ideal because it doesn't look for the path configuration files (.pth), thus you can't just point to a single location that would expand itself to the additional paths.

----------
components: Library (Lib)
messages: 353332
nosy: Micha? Fronczyk
priority: normal
severity: normal
status: open
title: Add an option to point to an additional "site" directory
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38289>
_______________________________________

From report at bugs.python.org  Fri Sep 27 03:13:20 2019
From: report at bugs.python.org (hongweipeng)
Date: Fri, 27 Sep 2019 07:13:20 +0000
Subject: [New-bugs-announce] [issue38290] cleanup ababstractproperty in
 typing.py
Message-ID: <1569568400.14.0.28730214811.issue38290@roundup.psfhosted.org>


New submission from hongweipeng <hongweichen8888 at sina.com>:

`abc.ababstractproperty` was deprecated since version 3.3. https://docs.python.org/3/library/abc.html#abc.abstractproperty

I checked the code and found that only `typing.py` is still used. I created a PR and replaced it with the one written in the documentation.

----------
components: Library (Lib)
messages: 353348
nosy: hongweipeng
priority: normal
severity: normal
status: open
title: cleanup ababstractproperty in typing.py
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38290>
_______________________________________

From report at bugs.python.org  Fri Sep 27 07:48:37 2019
From: report at bugs.python.org (dgelessus)
Date: Fri, 27 Sep 2019 11:48:37 +0000
Subject: [New-bugs-announce] [issue38291] Unclear status of the typing.io
 and typing.re pseudo-modules in docs and runtime
Message-ID: <1569584917.37.0.450943031305.issue38291@roundup.psfhosted.org>


New submission from dgelessus <dgelessus+bugs.python.org at me.com>:

According to bpo-35089 (and the GitHub issues linked there), the typing.io and typing.re modules should no longer be used. Starting with Python 3.6, the typing documentation no longer mentions the typing.io and typing.re modules, and instead documents their contents as part of the main typing module.

However, the typing module at runtime still supports typing.io and typing.re as before. Since these modules are not meant to be used anymore and are no longer documented, I would expect at least a DeprecationWarning when using them at runtime.

The documentation on this could also be a bit clearer. As long as the modules are still supported at runtime, I would expect the documentation to mention that they still exist, but shouldn't be used anymore. In its current state, the documentation is confusing when coming from Python 3.5 (where typing.io and typing.re were the only documented way to access IO, Pattern, etc., but accessing typing.IO and typing.Pattern directly also works at runtime).

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 353356
nosy: dgelessus, docs at python
priority: normal
severity: normal
status: open
title: Unclear status of the typing.io and typing.re pseudo-modules in docs and runtime
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38291>
_______________________________________

From report at bugs.python.org  Fri Sep 27 09:20:38 2019
From: report at bugs.python.org (Amir Emami)
Date: Fri, 27 Sep 2019 13:20:38 +0000
Subject: [New-bugs-announce] [issue38292] tkinter variable classes don't
 link to widget if matplotlib's set_cmap() function is called before the
 tkinter GUI is instantiated
Message-ID: <1569590438.84.0.268389191233.issue38292@roundup.psfhosted.org>


New submission from Amir Emami <amirali1376 at gmail.com>:

TkInter variable classes don't link to a widget (tested on Checkbutton and Scale) if matplotlib.pyplot.set_cmap() function is called before the tkinter.Tk() call which instantiates the root of a TkInter GUI. There is no problem if it is called after this, though.

Simple example with checkbox attached below:

### Test program start ##############

import matplotlib.pyplot as plt
import tkinter as tk

plt.set_cmap('viridis') # <--- when placed here, breaks the variable
root = tk.Tk()
#plt.set_cmap('viridis') # <--- when placed here, everything is fine

# creation of variable class and widget
var = tk.BooleanVar()
tk.Checkbutton(root, variable=var).pack()

# for printing result
def on_click():
    print(var.get())
tk.Button(root, text="Print State to Console", command=on_click).pack()

root.mainloop()

----------
components: Tkinter
files: tk and plt conflict.py
messages: 353359
nosy: amiraliemami
priority: normal
severity: normal
status: open
title: tkinter variable classes don't link to widget if matplotlib's set_cmap() function is called before the tkinter GUI is instantiated
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file48627/tk and plt conflict.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38292>
_______________________________________

From report at bugs.python.org  Fri Sep 27 12:16:43 2019
From: report at bugs.python.org (=?utf-8?q?Gu=C3=B0ni_Nathan?=)
Date: Fri, 27 Sep 2019 16:16:43 +0000
Subject: [New-bugs-announce] [issue38293] Deepcopying property objects
 results in unexpected TypeError
Message-ID: <1569601003.51.0.0962889637632.issue38293@roundup.psfhosted.org>


New submission from Gu?ni Nathan <gudni.nathan at gmail.com>:

Currently, attempting to deepcopy a property object will result in an unexpected TypeError:

>>> import copy
>>> obj = property()
>>> new_obj = copy.deepcopy(obj)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1264.0_x64__qbz5n2kfra8p0\lib\copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle property objects


What I believe is happening here is that since property objects are not treated by the copy module as atomic, they are passed off to be pickled and so our error is raised.
This can be fixed in a similar manner to how it works for type objects, function objects and more.
Adding a single line of code to Lib/copy.py after line 208:

    d[property] = _deepcopy_atomic

Means that property objects will be treated as atomic, and therefore returned as-is.

----------
components: Library (Lib)
messages: 353375
nosy: Gu?ni Nathan
priority: normal
severity: normal
status: open
title: Deepcopying property objects results in unexpected TypeError
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38293>
_______________________________________

From report at bugs.python.org  Fri Sep 27 13:06:37 2019
From: report at bugs.python.org (=?utf-8?q?Ricardo_B=C3=A1nffy?=)
Date: Fri, 27 Sep 2019 17:06:37 +0000
Subject: [New-bugs-announce] [issue38294] re.escape seems to miss some
 importante regex characters
Message-ID: <1569603997.59.0.0237297108404.issue38294@roundup.psfhosted.org>


New submission from Ricardo B?nffy <rbanffy at gmail.com>:

Under Python 3.6, re.escape escapes "/"

In [1]: import re

In [2]: re.escape('http://workday.com')
Out[2]: 'http\\:\\/\\/workday\\.com'

Under 3.7 and 3.8, "/" is not escaped.

In [1]: import re

In [2]: re.escape('http://workday.com')
Out[2]: 'http://workday\\.com'

Is this change deliberate? For a minor release (3.6 to 3.7) this broke some code that generated `sed` commands to run on a remote server.

----------
components: Regular Expressions
messages: 353380
nosy: Ricardo B?nffy, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: re.escape seems to miss some importante regex characters
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38294>
_______________________________________

From report at bugs.python.org  Fri Sep 27 13:24:03 2019
From: report at bugs.python.org (Barry A. Warsaw)
Date: Fri, 27 Sep 2019 17:24:03 +0000
Subject: [New-bugs-announce] [issue38295] macOS Catalina test failures
Message-ID: <1569605043.98.0.657218273343.issue38295@roundup.psfhosted.org>


New submission from Barry A. Warsaw <barry at python.org>:

test_py_compile fails on macOS Catalina beta (19A573a)

======================================================================
ERROR: test_relative_path (test.test_py_compile.PyCompileTestsWithSourceEpoch)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/bwarsaw/projects/python/3.8/Lib/test/test_py_compile.py", line 30, in wrapper
    return fxn(*args, **kwargs)
  File "/Users/bwarsaw/projects/python/3.8/Lib/test/test_py_compile.py", line 111, in test_relative_path
    py_compile.compile(os.path.relpath(self.source_path),
  File "/Users/bwarsaw/projects/python/3.8/Lib/py_compile.py", line 157, in compile
    os.makedirs(dirname)
  File "/Users/bwarsaw/projects/python/3.8/Lib/os.py", line 221, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '../../../../../../../var/folders/w6/w_2zrjgj7bgdgkhdjm4_r9s4000slb/T/tmpq3p6aoly'

======================================================================
ERROR: test_relative_path (test.test_py_compile.PyCompileTestsWithoutSourceEpoch)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/bwarsaw/projects/python/3.8/Lib/test/test_py_compile.py", line 20, in wrapper
    return fxn(*args, **kwargs)
  File "/Users/bwarsaw/projects/python/3.8/Lib/test/test_py_compile.py", line 111, in test_relative_path
    py_compile.compile(os.path.relpath(self.source_path),
  File "/Users/bwarsaw/projects/python/3.8/Lib/py_compile.py", line 157, in compile
    os.makedirs(dirname)
  File "/Users/bwarsaw/projects/python/3.8/Lib/os.py", line 221, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '../../../../../../../var/folders/w6/w_2zrjgj7bgdgkhdjm4_r9s4000slb/T/tmpquor0q8r'

----------------------------------------------------------------------

----------
components: Tests
messages: 353381
nosy: barry, ned.deily
priority: normal
severity: normal
status: open
title: macOS Catalina test failures
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38295>
_______________________________________

From report at bugs.python.org  Fri Sep 27 13:26:09 2019
From: report at bugs.python.org (Kit Choi)
Date: Fri, 27 Sep 2019 17:26:09 +0000
Subject: [New-bugs-announce] [issue38296] unittest expectedFailure does not
 differentiate errors from failures
Message-ID: <1569605169.32.0.911157248644.issue38296@roundup.psfhosted.org>


New submission from Kit Choi <kit at kychoi.org>:

I expect the following test to fail, because an "error" is not a "failure".
Unexpectedly, the test passes:

```
class TestFailure(unittest.TestCase):

    @unittest.expectedFailure
    def test_expected_failure(self):
        raise TypeError()   # for example, a typo.
```

```
$ python -m unittest test_main
x
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK (expected failures=1)
```

This behaviour exists since Python 2.7, and is still true for the Python 3.8.0b1

----------
components: Tests
messages: 353382
nosy: Kit Choi
priority: normal
severity: normal
status: open
title: unittest expectedFailure does not differentiate errors from failures
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38296>
_______________________________________

From report at bugs.python.org  Fri Sep 27 14:16:34 2019
From: report at bugs.python.org (Dominic Littlewood)
Date: Fri, 27 Sep 2019 18:16:34 +0000
Subject: [New-bugs-announce] [issue38297] Imports at top of module is often
 not used
Message-ID: <1569608194.16.0.811329598218.issue38297@roundup.psfhosted.org>


New submission from Dominic Littlewood <11dlittlewood at gmail.com>:

In PEP 8, it is stated that:
"Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants."

Note the word "always". This advice makes sense because it cuts down on performance costs.

I recently got into an argument about this, and so I created a script to find all the times in the standard library that an import statement was used inside a function or method. I was expecting to get one or two, but it's safe to say that the answer 1576 was a little surprising. (I was on 3.7.4, if anyone wants to check.)

It turns out that more than one in five modules (defined as any .py file) contain at least one violation of this rule. In the standard library, there are more violations of the rule than there are while loops. Do we need to either a) fix large parts of the standard library or b) rewrite PEP 8 to say that this might not actually be all that bad?

----------
components: Library (Lib)
messages: 353385
nosy: plokmijnuhby
priority: normal
severity: normal
status: open
title: Imports at top of module is often not used
type: performance

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38297>
_______________________________________

From report at bugs.python.org  Fri Sep 27 14:26:37 2019
From: report at bugs.python.org (Kefei Lu)
Date: Fri, 27 Sep 2019 18:26:37 +0000
Subject: [New-bugs-announce] [issue38298] Base class of generic type has
 wrong `cls` argument in classmethods
Message-ID: <1569608797.01.0.364223398696.issue38298@roundup.psfhosted.org>


New submission from Kefei Lu <kludev at gmail.com>:

This is a new regression in Python3.7.


Create the following file as `test.py`

```
# test.py

import typing as t


T = t.TypeVar("T")


class BaseOfGeneric:

    @classmethod
    def f(cls):
        # when called from an instantiated generic type, e.g.,
        # `MyList[int]`, expect `cls` to be the GenericAlias with its type
        # argument already insteantiated
        print(f"current class is {cls}")
        print(f"current class's type: {type(cls)}")


class MyList(t.List[T], BaseOfGeneric):
    pass


MyIntList = MyList[int]

MyIntList.f()
```

Run with Python3.6:

    >>> python3.6 ./test.py 
    current class is __main__.MyList[int]
                                     ^^^ as expected

    current class's type: <class 'typing.GenericMeta'>

EXPECTED BEHAVIOR: The outcome is expected: `cls` of `BaseOfGeneric.f` should be `MyList` **with** type argument `int`


However this is wrong in Python3.7:

    >>> python3.7 ./test.py 
    current class is <class '__main__.MyList'>
                             ^^^^^^^^^^^^^^^ type argument is LOST !!!

    current class's type: <class 'type'>

Note that `cls` of `BaseOfGeneric.f` has lost the type argument `int`! It is not expected.

----------
messages: 353386
nosy: kflu
priority: normal
severity: normal
status: open
title: Base class of generic type has wrong `cls` argument in classmethods
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38298>
_______________________________________

From report at bugs.python.org  Fri Sep 27 15:26:42 2019
From: report at bugs.python.org (Geoge R. Goffe)
Date: Fri, 27 Sep 2019 19:26:42 +0000
Subject: [New-bugs-announce] [issue38299] build errors - latest "git pull"
Message-ID: <1569612402.25.0.13974147855.issue38299@roundup.psfhosted.org>


New submission from Geoge R. Goffe <grgoffe at yahoo.com>:

SUMMARY: AddressSanitizer: 240 byte(s) leaked in 3 allocation(s).

----------
components: Build
files: build-python-3.log
messages: 353391
nosy: grgoffe at yahoo.com
priority: normal
severity: normal
status: open
title: build errors - latest "git pull"
type: compile error
versions: Python 3.9
Added file: https://bugs.python.org/file48629/build-python-3.log

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38299>
_______________________________________

From report at bugs.python.org  Fri Sep 27 15:45:13 2019
From: report at bugs.python.org (Ian)
Date: Fri, 27 Sep 2019 19:45:13 +0000
Subject: [New-bugs-announce] [issue38300] Documentation says destuction of
 TemporaryDirectory object will also delete it, but it does not.
Message-ID: <1569613513.83.0.841997670509.issue38300@roundup.psfhosted.org>


New submission from Ian <iarp.comptech at gmail.com>:

The documentation found here https://docs.python.org/3.7/library/tempfile.html#tempfile.TemporaryDirectory states the following

"On completion of the context or destruction of the temporary directory object the newly created temporary directory and all its contents are removed from the filesystem."

However calling del on the object does not call the cleanup method.

t = tempfile.TemporaryDirectory()
del t

I'm not sure if that is incorrect documentation or my own misunderstanding of what you call destruction. I tested adding my own def __del__(): self.cleanup() which worked as I expected.

----------
messages: 353393
nosy: iarp
priority: normal
severity: normal
status: open
title: Documentation says destuction of TemporaryDirectory object will also delete it, but it does not.
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38300>
_______________________________________

From report at bugs.python.org  Fri Sep 27 17:35:16 2019
From: report at bugs.python.org (=?utf-8?b?SmVzw7pzIENlYSBBdmnDs24=?=)
Date: Fri, 27 Sep 2019 21:35:16 +0000
Subject: [New-bugs-announce] [issue38301] If "_REENTARNT" is defined in the
 header file, we should use "-D_REENTRANT" when compiling
Message-ID: <1569620116.14.0.859421629509.issue38301@roundup.psfhosted.org>


New submission from Jes?s Cea Avi?n <jcea at jcea.es>:

When compiling on Solaris family, we define "_REENTRANT" in "Pyconfig.h".  That file is included when compiling C extensions, but if the extension is made of several C files, it could be included only in the file that interfaces with python runtime.

Unfortunately, "_REENTRANT" modifies the way C code is compiled, so compiling a C extension with some source files with "_REENTRANT" and some other without that "define" can produce bizarre errors and crashes.

One example: https://github.com/joyent/pkgsrc/issues/207

In particular, "include" order can make a big difference: https://github.com/jnwatson/py-lmdb/issues/213

I suggest that when "_REENTRANT" is defined in "Pyconfig.h", Python C extensions compilation machinery adds "-D_REENTRANT" to the compilation flags.

The patch is trivial.

----------
assignee: jcea
components: Build
messages: 353416
nosy: jcea
priority: high
severity: normal
status: open
title: If "_REENTARNT" is defined in the header file, we should use "-D_REENTRANT" when compiling
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38301>
_______________________________________

From report at bugs.python.org  Fri Sep 27 17:55:15 2019
From: report at bugs.python.org (Adi)
Date: Fri, 27 Sep 2019 21:55:15 +0000
Subject: [New-bugs-announce] [issue38302] __rpow__ not reached when __ipow__
 returns NotImplemented
Message-ID: <1569621315.69.0.377751123288.issue38302@roundup.psfhosted.org>


New submission from Adi <deepspace2 at gmail.com>:

Due to shared code between the 2 and 3 forms of pow, the following code causes a TypeError:

class A:
    def __init__(self, val):
        self.val = val
    
    def __ipow__(self, other):
        return NotImplemented

class B:
    def __init__(self, val):
        self.val = val
    
    def __rpow__(self, other):
        return A(other.val ** self.val)

a = A(2)
b = B(2)
a **= b


(https://stackoverflow.com/questions/58141475/ipow-raising-typeerror-when-left-hand-side-object-returns-notimplemented)

----------
components: Interpreter Core
messages: 353419
nosy: DeepSpace
priority: normal
severity: normal
status: open
title: __rpow__ not reached when __ipow__ returns NotImplemented
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38302>
_______________________________________

From report at bugs.python.org  Fri Sep 27 19:13:55 2019
From: report at bugs.python.org (Tyler Kieft)
Date: Fri, 27 Sep 2019 23:13:55 +0000
Subject: [New-bugs-announce] [issue38303] Make audioop PEP-384 compatible
Message-ID: <1569626035.23.0.605730038831.issue38303@roundup.psfhosted.org>


Change by Tyler Kieft <tyler.kieft at gmail.com>:


----------
components: Extension Modules
nosy: Tyler Kieft
priority: normal
severity: normal
status: open
title: Make audioop PEP-384 compatible
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38303>
_______________________________________

From report at bugs.python.org  Fri Sep 27 20:32:45 2019
From: report at bugs.python.org (STINNER Victor)
Date: Sat, 28 Sep 2019 00:32:45 +0000
Subject: [New-bugs-announce] [issue38304] PEP 587 implementation is not ABI
 forward compatible
Message-ID: <1569630765.03.0.323163871576.issue38304@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

The current implementation of the PEP 587 is not ready for future evolutions of the PyPreConfig and PyConfig structure. Any change will break the ABI compatibility. Their _config_version field is useless.

The first versions of the PEP 587 used a macro to statically initialize the structure. It was decided to get ride of macros to use functions instead.

Example:

   PyConfig config;
   PyStatus status = PyConfig_InitPythonConfig(&config);

PyConfig_InitPythonConfig() gets unitialized memory and doesn't know the size of the config variable.

I propose to require to store the size of the structure in the structure directly:

* Add PyPreConfig.struct_size and PyConfig.struct_size
* Require to initialize these fields to sizeof(PyPreConfig) and sizeof(PyConfig) respectively

Example:
     
   PyConfig config;
   config.struct_size = sizeof(PyConfig);
   PyStatus status = PyConfig_InitPythonConfig(&config);

Thanks to the struct_size field, PyConfig_InitPythonConfig() can support old PyConfig structures without breaking the ABI.

If an old PyConfig is passed to PyConfig_Read(): newer fields will be ignored.

----------
components: Interpreter Core
messages: 353430
nosy: lukasz.langa, vstinner
priority: release blocker
severity: normal
status: open
title: PEP 587 implementation is not ABI forward compatible
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38304>
_______________________________________

From report at bugs.python.org  Sat Sep 28 09:51:46 2019
From: report at bugs.python.org (Tim Laurence)
Date: Sat, 28 Sep 2019 13:51:46 +0000
Subject: [New-bugs-announce] [issue38305] https://2.python-requests.org/
 missing TLS certicate
Message-ID: <1569678706.53.0.557661175696.issue38305@roundup.psfhosted.org>


New submission from Tim Laurence <timdaman at gmail.com>:

I am unsure how to route this given the recent transition of Requests to PSF so my apologies if this is the wrong spot.

The page where I think most people look for Requests documentation appears to be broken 
  https://2.python-requests.org/

When I look more closely it looks like the server may not have a TLS certificate loaded. 

When I connect via http I get a redirection page to https://requests.kennethreitz.org/en/master/


Below is a dump of what openssl sees when trying to connect.

===========================
$ openssl s_client -connect 2.python-requests.org:443
CONNECTED(00000005)
4608755308:error:14004438:SSL routines:CONNECT_CR_SRVR_HELLO:tlsv1 alert internal error:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.260.1/libressl-2.6/ssl/ssl_pkt.c:1205:SSL alert number 80
4608755308:error:140040E5:SSL routines:CONNECT_CR_SRVR_HELLO:ssl handshake failure:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.260.1/libressl-2.6/ssl/ssl_pkt.c:585:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Start Time: 1569678290
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

----------
components: Library (Lib)
messages: 353452
nosy: timdaman
priority: normal
severity: normal
status: open
title: https://2.python-requests.org/ missing TLS certicate

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38305>
_______________________________________

From report at bugs.python.org  Sat Sep 28 10:43:05 2019
From: report at bugs.python.org (Antoine Pietri)
Date: Sat, 28 Sep 2019 14:43:05 +0000
Subject: [New-bugs-announce] [issue38306] High level API for
 loop.run_in_executor(None, ...)?
Message-ID: <1569681785.01.0.799144749415.issue38306@roundup.psfhosted.org>


New submission from Antoine Pietri <antoine.pietri1 at gmail.com>:

In 3.7 a lot of high level functions have been added to the asyncio API, but nothing to start blocking functions as non-blocking threads. You still have to call get_event_loop() then await loop.run_in_executor(None, callable). I think this pattern is *very* common and could use a higher level interface.

Would an API like this make sense?

async def create_thread(callable, *args, *, kwargs=None, loop=None, timeout=None)

Then it could just be used like this:

await asyncio.create_thread(my_blocking_read, 4096, timeout=10)

This API could wrap the run_in_executor in an asyncio.Task, that way
you wouldn't have to await it to start the thread. There's evidence
that this has confused people in the past:

https://stackoverflow.com/questions/54263558/is-asyncio-run-in-executor-specified-ambiguously

----------
components: asyncio
messages: 353456
nosy: antoine.pietri, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: High level API for loop.run_in_executor(None, ...)?
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38306>
_______________________________________

From report at bugs.python.org  Sat Sep 28 13:26:30 2019
From: report at bugs.python.org (Aviral)
Date: Sat, 28 Sep 2019 17:26:30 +0000
Subject: [New-bugs-announce] [issue38307] Provide Class' end line in
 readmodule module
Message-ID: <1569691590.29.0.185701662636.issue38307@roundup.psfhosted.org>


New submission from Aviral <hi at aviralsrivastava.com>:

Currently, the `readmodule` returns the starting line of the classes but not the end line. This should be provided as well in order to get the scope of the class, mainly to help check what all imports were used in a class.

----------
components: Library (Lib)
messages: 353467
nosy: aviral
priority: normal
severity: normal
status: open
title: Provide Class' end line in readmodule module
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38307>
_______________________________________

From report at bugs.python.org  Sat Sep 28 14:28:15 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 28 Sep 2019 18:28:15 +0000
Subject: [New-bugs-announce] [issue38308] Add optional weighting to
 statistics.harmonic_mean()
Message-ID: <1569695295.83.0.838130885601.issue38308@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Currently, harmonic_mean() is difficult to use in real applications because it assumes equal weighting.  While that is sometimes true, the API precludes a broad class of applications where the weights are uneven.

That is easily remedied with an optional *weights* argument modeled after the API for random.choices():

    harmonic_mean(data, weights=None)


Examples
--------

Suppose a car travels 40 km/hr for 5 km, and when traffic clears, speeds-up to 60 km/hr for the remaining 30 km of the journey. What is the average speed?

    >>> harmonic_mean([40, 60], weights=[5, 30])
    56.0

Suppose an investor owns shares in each of three companies, with P/E (price/earning) ratios of 2.5, 3 and 10, and with market values of 10,000, 7,200, and 12,900 respectively.  What is the weighted average P/E ratio for the investor?s portfolio?

    >>> avg_pe = harmonic_mean([2.5, 3, 10], weights=[10_000, 7_200, 12_900])
    >>> round(avg_pe, 1)
    3.9


Existing workarounds
--------------------

It is possible to use the current API for theses tasks, but it is inconvenient, awkward, slow, and only works with integer ratios:

    >>> harmonic_mean([40]*5 + [60]*30)
    56.0

    >>> harmonic_mean([2.5]*10_000 + [3]*7_200 + [10]*12_900)
    3.9141742522756826


Algorithm
---------

Following the formula at https://en.wikipedia.org/wiki/Harmonic_mean#Weighted_harmonic_mean , the algorithm is straight forward:

    def weighted_harmonic_mean(data, weights):
        num = den = 0
        for x, w in zip(data, weights):
            num += w
            den += w / x
        return num / den


PR
--

If you're open to this suggestion, I'll work-up a PR modeled after the existing code and that uses _sum() and _fail_neg() for exactness and data validity checks.

----------
assignee: steven.daprano
components: Library (Lib)
messages: 353469
nosy: rhettinger, steven.daprano
priority: normal
severity: normal
status: open
title: Add optional weighting to statistics.harmonic_mean()
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38308>
_______________________________________

From report at bugs.python.org  Sat Sep 28 14:40:52 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 28 Sep 2019 18:40:52 +0000
Subject: [New-bugs-announce] [issue38309] Avoid creating duplicate PATH
 entries with the macOS installer
Message-ID: <1569696052.2.0.379092871214.issue38309@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

The macOS installer updates .bash_profile to add the current python version to the PATH.

The problem is that it doesn't check to see if that work has already been done in a previous installation.  So, as a person install successive beta releases, release candidates, a final release, and then maintenance releases, the PATH variable grows and grows.

As it stands now, a manual edit is required every after every update/upgrade.

----------
assignee: ned.deily
components: Build
messages: 353471
nosy: ned.deily, rhettinger
priority: normal
severity: normal
status: open
title: Avoid creating duplicate PATH entries with the macOS installer
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38309>
_______________________________________

From report at bugs.python.org  Sat Sep 28 15:15:08 2019
From: report at bugs.python.org (Brandt Bucher)
Date: Sat, 28 Sep 2019 19:15:08 +0000
Subject: [New-bugs-announce] [issue38310] Opcode predictions for asserts,
 class definitions, and some calls.
Message-ID: <1569698108.52.0.160533425864.issue38310@roundup.psfhosted.org>


New submission from Brandt Bucher <brandtbucher at gmail.com>:

This patch adds four new opcode predictions:

BUILD_MAP_UNPACK_WITH_CALL -> CALL_FUNCTION_EX:
 - Emitted whenever more than one map of **kwargs is unpacked into a call.
 - Pair *always* occurs together.

LOAD_BUILD_CLASS -> LOAD_CONST:
 - Emitted whenever a class is defined *without* the use of cell vars.
 - Occurs ~93% of the time, in my analysis.

LOAD_BUILD_CLASS -> LOAD_CLOSURE:
 - Emitted whenever a class is defined *with* the use of cell vars.
 - Occurs the other ~7% of the time

LOAD_ASSERTION_ERROR -> RAISE_VARARGS:
 - Emitted whenever the one-argument form of "assert" is used.
 - Occurs ~91% of the time.

----------
components: Interpreter Core
messages: 353472
nosy: brandtbucher
priority: normal
severity: normal
status: open
title: Opcode predictions for asserts, class definitions, and some calls.
type: performance
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38310>
_______________________________________

From report at bugs.python.org  Sat Sep 28 17:17:09 2019
From: report at bugs.python.org (David CARLIER)
Date: Sat, 28 Sep 2019 21:17:09 +0000
Subject: [New-bugs-announce] [issue38311] macOS sqlite 3 module build fix
Message-ID: <1569705429.89.0.892890013712.issue38311@roundup.psfhosted.org>


Change by David CARLIER <devnexen at gmail.com>:


----------
components: Build
nosy: devnexen
priority: normal
pull_requests: 16052
severity: normal
status: open
title: macOS sqlite 3 module build fix
type: compile error
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38311>
_______________________________________

From report at bugs.python.org  Sat Sep 28 21:40:08 2019
From: report at bugs.python.org (Anthony Sottile)
Date: Sun, 29 Sep 2019 01:40:08 +0000
Subject: [New-bugs-announce] [issue38312] curses: add `set_tabsize` and
 `set_escdelay`
Message-ID: <1569721208.44.0.258918770799.issue38312@roundup.psfhosted.org>


New submission from Anthony Sottile <asottile at umich.edu>:

https://linux.die.net/man/3/set_escdelay
https://linux.die.net/man/3/set_tabsize

I'd like to help with this, but I don't even know where to start with argumentclinic -- any points would be greatly appreciated

presumably I should also add getters for these as well to retrieve the global variable from the curses side?

----------
components: Extension Modules
messages: 353477
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: curses: add `set_tabsize` and `set_escdelay`
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38312>
_______________________________________

From report at bugs.python.org  Sat Sep 28 23:03:53 2019
From: report at bugs.python.org (Gautam)
Date: Sun, 29 Sep 2019 03:03:53 +0000
Subject: [New-bugs-announce] [issue38313] Crash/No start
Message-ID: <1569726233.14.0.289073858967.issue38313@roundup.psfhosted.org>


New submission from Gautam <mathurgautam6 at gmail.com>:

Hi
Python in my Laptop used to work well but now a days i is not working in fact it wont even start. Even after pressing it for half an hour no screen would appear 
pls check

----------
components: Windows
messages: 353482
nosy: Rick Grimes, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Crash/No start
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38313>
_______________________________________

From report at bugs.python.org  Sun Sep 29 07:22:13 2019
From: report at bugs.python.org (Andrew Svetlov)
Date: Sun, 29 Sep 2019 11:22:13 +0000
Subject: [New-bugs-announce] [issue38314] Implement unix
 read_pipe.is_reading() method
Message-ID: <1569756133.59.0.955288942116.issue38314@roundup.psfhosted.org>


New submission from Andrew Svetlov <andrew.svetlov at gmail.com>:

Sockets support it, there is no reason for missing the method in unix pipe.

----------
components: asyncio
messages: 353494
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Implement unix read_pipe.is_reading() method
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38314>
_______________________________________

From report at bugs.python.org  Sun Sep 29 12:12:39 2019
From: report at bugs.python.org (Mark Amery)
Date: Sun, 29 Sep 2019 16:12:39 +0000
Subject: [New-bugs-announce] [issue38315] Provide defaultdict variant that
 passes key to default_factory
Message-ID: <1569773559.86.0.293665151535.issue38315@roundup.psfhosted.org>


New submission from Mark Amery <markamery at btinternet.com>:

There seems to be at least some demand for the ability to have the key being accessed be passed to the factory function of a defaultdict. See e.g. https://stackoverflow.com/q/2912231/1709587 (13k views) or previous enhancement suggestion https://bugs.python.org/issue30408.

However, as noted on that issue, defaultdict cannot simply be modified to pass the key to the factory function without breaking backwards compatibility.

Perhaps it makes sense, then, to have a separate class in the collections module, perhaps called keydefaultdict, that is like defaultdict but whose factory function receives one argument, which is the key being accessed? Having this built into the standard library would be slightly more convenient than either making an ad-hoc dict subclass every time this functionality is needed or having to roll your own subclass like the one at https://stackoverflow.com/a/2912455/1709587.

----------
components: Library (Lib)
messages: 353502
nosy: ExplodingCabbage
priority: normal
severity: normal
status: open
title: Provide defaultdict variant that passes key to default_factory
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38315>
_______________________________________

From report at bugs.python.org  Sun Sep 29 16:34:54 2019
From: report at bugs.python.org (Paul Sokolovsky)
Date: Sun, 29 Sep 2019 20:34:54 +0000
Subject: [New-bugs-announce] [issue38316] docs: Code object's "co_stacksize"
 field is described with mistake
Message-ID: <1569789294.48.0.284805020981.issue38316@roundup.psfhosted.org>


New submission from Paul Sokolovsky <pfalcon at users.sourceforge.net>:

CPython's Data Model -> Internal types -> Code objects, direct link as of version 3.7 is: https://docs.python.org/3.7/reference/datamodel.html?highlight=co_stacksize#index-55 , states following:

* "co_nlocals is the number of local variables used by the function (including arguments);"
* "co_stacksize is the required stack size (including local variables);"

However, practical checking shows that co_stacksize is the required stack size WITHOUT local variables. One could argue about the meaning of "local variables" in the description of co_stacksize above, saying that what is meant is temporary stack variables of something. That's why I quoted above co_nlocals description too, it clearly shows that local variebles are local function variables (include functions args), and nothing else.

Code to reproduce:

======
def func():
    a = 1
    b = 2
    c = 3

print("co_stacksize", func.__code__.co_stacksize)
print("co_nlocals", func.__code__.co_nlocals)
======

Result of running:
======
$ python3.7 script_under_test.py 
co_stacksize 1
co_nlocals 3
======

Clearly, co_stacksize doesn't include the size of local vars, or it would have been larger than co_nlocals in printout.

----------
assignee: docs at python
components: Documentation
messages: 353508
nosy: docs at python, pfalcon
priority: normal
severity: normal
status: open
title: docs: Code object's "co_stacksize" field is described with mistake
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38316>
_______________________________________

From report at bugs.python.org  Sun Sep 29 18:54:37 2019
From: report at bugs.python.org (STINNER Victor)
Date: Sun, 29 Sep 2019 22:54:37 +0000
Subject: [New-bugs-announce] [issue38317] PyConfig (PEP 587):
 PyConfig.warnoptions should have the highest priority
Message-ID: <1569797677.63.0.958144082702.issue38317@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

The PEP 587 says that PyConfig.warnoptions has the highest priority for warnings options:
https://www.python.org/dev/peps/pep-0587/#priority-and-rules

But in the current implementation, PyConfig.warnoptions has... the lowest priority :-(

Attached PR not only fix the issue but also add tests to ensure that every ways to set warnings options have the expected priority.

Priority of warnings options, lowest to highest:

- any implicit filters added by _warnings.c/warnings.py
- PyConfig.dev_mode: "default" filter
- PYTHONWARNINGS environment variable
- '-W' command line options
- PyConfig.bytes_warning ('-b', '-bb'): "default::BytesWarning"
  or "error::BytesWarning" filter
- early PySys_AddWarnOption() calls
- PyConfig.warnoptions

----------
components: Interpreter Core
messages: 353513
nosy: vstinner
priority: normal
severity: normal
status: open
title: PyConfig (PEP 587): PyConfig.warnoptions should have the highest priority
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38317>
_______________________________________

From report at bugs.python.org  Sun Sep 29 22:05:56 2019
From: report at bugs.python.org (Michael Everitt)
Date: Mon, 30 Sep 2019 02:05:56 +0000
Subject: [New-bugs-announce] [issue38318] Issues linking with ncurses and
 tinfo (cannot resolve symbols)
Message-ID: <1569809156.07.0.948937719189.issue38318@roundup.psfhosted.org>


New submission from Michael Everitt <michael at 2e0cer.net>:

tinfo library detection as part of ncurses is inconsistent, and causes build failure when expected symbols aren't found at link-time.
This breaks compilation of the 'readline' module as the attached log shows.
It seems to work in some configurations and not others, different arches and libc's seem to be affected differently (suspect linker variations).
It would appear that uclibc-ng triggers this failure consistently, as I have verified on both x86_64 and armv7a Arches.

See https://bugs.gentoo.org/692128 for further details.

----------
components: Build
files: file_692128.txt
messages: 353523
nosy: veremitz
priority: normal
severity: normal
status: open
title: Issues linking with ncurses and tinfo (cannot resolve symbols)
type: compile error
versions: Python 2.7, Python 3.6
Added file: https://bugs.python.org/file48631/file_692128.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38318>
_______________________________________

From report at bugs.python.org  Mon Sep 30 03:37:51 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 30 Sep 2019 07:37:51 +0000
Subject: [New-bugs-announce] [issue38319] shutil.copyfile(): os.sendfile()
 fails with OverflowError on 32-bit system
Message-ID: <1569829071.99.0.925566441615.issue38319@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

Error on a 32-bit buildbot worker where ssize_t maximum = 2,147,483,647 (2**31-1) bytes = ~2.0 GiB.

test_largefile uses:

# size of file to create (>2 GiB; 2 GiB == 2,147,483,648 bytes)
size = 2_500_000_000

x86 Gentoo Installed with X 3.x:
https://buildbot.python.org/all/#/builders/103/builds/3162

======================================================================
ERROR: test_it (test.test_largefile.TestCopyfile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.x.ware-gentoo-x86.installed/build/target/lib/python3.9/test/test_largefile.py", line 160, in test_it
    shutil.copyfile(TESTFN, TESTFN2)
  File "/buildbot/buildarea/cpython/3.x.ware-gentoo-x86.installed/build/target/lib/python3.9/shutil.py", line 266, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/buildbot/buildarea/cpython/3.x.ware-gentoo-x86.installed/build/target/lib/python3.9/shutil.py", line 145, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
OverflowError: Python int too large to convert to C ssize_t

On Linux (Fedora 30), man sendfile shows me the prototype:

       ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);

Extract of Lib/shutil.py:

    # Hopefully the whole file will be copied in a single call.
    # sendfile() is called in a loop 'till EOF is reached (0 return)
    # so a bufsize smaller or bigger than the actual file size
    # should not make any difference, also in case the file content
    # changes while being copied.
    try:
        blocksize = max(os.fstat(infd).st_size, 2 ** 23)  # min 8MB
    except Exception:
        blocksize = 2 ** 27  # 128MB

    offset = 0
    while True:
        try:
            sent = os.sendfile(outfd, infd, offset, blocksize)
        except OSError as err:
            ...
        else:
            if sent == 0:
                break  # EOF
            offset += sent

Extract of the Linux implementation of os.sendfile():

    int in, out;
    Py_ssize_t ret;
    off_t offset;
    ...
    Py_ssize_t count;
    PyObject *offobj;
    static char *keywords[] = {"out", "in",
                                "offset", "count", NULL};
    if (!PyArg_ParseTupleAndKeywords(args, kwdict, "iiOn:sendfile",
            keywords, &out, &in, &offobj, &count))
        return NULL;
    ...
    if (!Py_off_t_converter(offobj, &offset))
        return NULL;

    do {
        Py_BEGIN_ALLOW_THREADS
        ret = sendfile(out, in, &offset, count);
        Py_END_ALLOW_THREADS
    } while (ret < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals()));

with:

static int
Py_off_t_converter(PyObject *arg, void *addr)
{
#ifdef HAVE_LARGEFILE_SUPPORT
    *((Py_off_t *)addr) = PyLong_AsLongLong(arg);
#else
    *((Py_off_t *)addr) = PyLong_AsLong(arg);
#endif
    if (PyErr_Occurred())
        return 0;
    return 1;
}


I understand that the error comes from the 4th sendfile() parameter: "count". The C code (of the Linux implementation) uses the "n" format for Py_ssize_t: Python/getargs.c calls PyLong_AsSsize_t().

On a 64-bit system, it's less likely to reach Py_ssize_t maximum value (max = 2**63-1), but it's easy to reach on a 32-bit system (max = 2**31-1).

----------
components: Library (Lib)
messages: 353549
nosy: giampaolo.rodola, nanjekyejoannah, pablogsal, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: shutil.copyfile(): os.sendfile() fails with OverflowError on 32-bit system
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38319>
_______________________________________

From report at bugs.python.org  Mon Sep 30 05:14:01 2019
From: report at bugs.python.org (Kit Choi)
Date: Mon, 30 Sep 2019 09:14:01 +0000
Subject: [New-bugs-announce] [issue38320] Clarify unittest expectedFailure
 behaviour in the documentation
Message-ID: <1569834841.91.0.690142859839.issue38320@roundup.psfhosted.org>


New submission from Kit Choi <kchoi at enthought.com>:

Following discussion in https://bugs.python.org/issue38296

The docstring of unittest.expectedFailure is misleading for people who differentiate "error" and "failure" when they read the sentence. This has a consequence of developers using the decorator without noticing unexpected errors are also silenced, which mean running a risk of their tests becoming unmaintained and invalid in the future.

I suggest updating the documentation to include a mention of the current behaviour of silencing unexpected errors, so that developers are aware of this.

Something like this?:

Mark the test such that unexpected success results in a failure. If an exception (BaseException excluding KeyboardInterrupt) occurs, the test will be considered a success. If the test passes, it will be considered a failure.

----------
assignee: docs at python
components: Documentation, Tests
messages: 353557
nosy: Kit Choi2, docs at python
priority: normal
severity: normal
status: open
title: Clarify unittest expectedFailure behaviour in the documentation
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38320>
_______________________________________

From report at bugs.python.org  Mon Sep 30 06:08:44 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 30 Sep 2019 10:08:44 +0000
Subject: [New-bugs-announce] [issue38321] Windows: compiler warnings when
 building Python 3.8
Message-ID: <1569838124.87.0.490940014864.issue38321@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

c:\program files (x86)\windows kits\10\include\10.0.17763.0\um\winbase.h(103): warning C4005: 'Yield': macro redefinition

c:\vstinner\python\3.8\modules\_asynciomodule.c(2667): warning C4102: 'set_exception': unreferenced label

c:\vstinner\python\3.8\modules\_ctypes\stgdict.c(704): warning C4244: 'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data

----------
components: Interpreter Core
messages: 353560
nosy: vstinner
priority: normal
severity: normal
status: open
title: Windows: compiler warnings when building Python 3.8
type: compile error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38321>
_______________________________________

From report at bugs.python.org  Mon Sep 30 07:27:09 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 30 Sep 2019 11:27:09 +0000
Subject: [New-bugs-announce] [issue38322] Azure Pipelines: appx tests fail:
 init_fs_encoding: failed to get the Python codec of the filesystem encoding
Message-ID: <1569842829.19.0.574952249624.issue38322@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'D:\a\1\b\layout-appx-amd64\python.exe'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = 'D:\\a\\1\\b\\layout-appx-amd64\\.\\.\\.\\python.exe'
  sys.base_prefix = 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o'
  sys.base_exec_prefix = 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o'
  sys.executable = 'D:\\a\\1\\b\\layout-appx-amd64\\.\\.\\.\\python.exe'
  sys.prefix = 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o'
  sys.exec_prefix = 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o'
  sys.path = [
    'D:\\a\\1\\b\\layout-appx-amd64\\python38.zip',
    'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o\\DLLs',
    'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o\\lib',
    'D:\\a\\1\\b\\layout-appx-amd64\\.\\.\\.',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

----------
components: Windows
messages: 353565
nosy: lukasz.langa, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: release blocker
severity: normal
status: open
title: Azure Pipelines: appx tests fail: init_fs_encoding: failed to get the Python codec of the filesystem encoding
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38322>
_______________________________________

From report at bugs.python.org  Mon Sep 30 08:57:35 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 30 Sep 2019 12:57:35 +0000
Subject: [New-bugs-announce] [issue38323] test_ayncio:
 test_close_kill_running() hangs on AMD64 RHEL7 Refleaks 3.x
Message-ID: <1569848255.08.0.155489980043.issue38323@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

test_asyncio fails once on AMD64 RHEL7 Refleaks 3.x, and then test_close_kill_running() was killed after 3h 15 min. I guess that it hangs, but I'm not 100% sure. When test_asyncio was re-run, it seems like test_asyncio was run 3x successful but it hanged at the 4rd run. Refleaks runs each test 6 times.

https://buildbot.python.org/all/#/builders/264/builds/10

0:17:04 load avg: 3.01 [416/419/1] test_asyncio failed (12 min 21 sec) -- running: test_multiprocessing_fork (6 min 34 sec), test_concurrent_futures (10 min 52 sec), test_multiprocessing_spawn (13 min 53 sec)
beginning 6 repetitions
123456
.Unknown child process pid 25032, will report returncode 255
Loop <_UnixSelectorEventLoop running=False closed=True debug=False> that handles pid 25032 is closed
.Unknown child process pid 19349, will report returncode 255
Child watcher got an unexpected pid: 19349
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.refleak/build/Lib/asyncio/unix_events.py", line 1213, in _do_waitpid
    loop, callback, args = self._callbacks.pop(pid)
KeyError: 19349
test test_asyncio failed -- Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.refleak/build/Lib/test/test_asyncio/test_subprocess.py", line 156, in test_shell
    self.assertEqual(exitcode, 7)
AssertionError: 255 != 7

(...)
0:21:59 load avg: 0.52 [419/419/1] test_concurrent_futures passed (15 min 46 sec)
(...)
== Tests result: FAILURE ==

404 tests OK.

10 slowest tests:
- test_concurrent_futures: 15 min 46 sec
- test_multiprocessing_spawn: 14 min 7 sec
- test_asyncio: 12 min 21 sec
- test_multiprocessing_forkserver: 8 min 59 sec
- test_multiprocessing_fork: 7 min 13 sec
- test_io: 3 min 47 sec
- test_subprocess: 3 min 30 sec
- test_pickle: 2 min 51 sec
- test_capi: 2 min 47 sec
- test_zipfile: 2 min 41 sec

1 test failed:
    test_asyncio

14 tests skipped:
    test_devpoll test_gdb test_ioctl test_kqueue test_msilib
    test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly
    test_winconsoleio test_winreg test_winsound test_zipfile64

Re-running failed tests in verbose mode

Re-running test_asyncio in verbose mode
beginning 6 repetitions
123456
(...)

test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessSafeWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests) ... ok
...


test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessSafeWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests) ... ok
...


test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessSafeWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests) ... ok
...


test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessSafeWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests) ... ok
...
test_close_kill_running (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_communicate (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_communicate_ignore_broken_pipe (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_create_subprocess_exec_text_mode_fails (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_create_subprocess_exec_with_path (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_create_subprocess_shell_text_mode_fails (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_devnull_error (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_devnull_input (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_devnull_output (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_empty_input (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_exec_loop_deprecated (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_kill (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_pause_reading (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_popen_error (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_popen_error_with_stdin_pipe (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_process_create_warning (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_read_stdout_after_process_exit (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_send_signal (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_shell (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_shell_loop_deprecated (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_start_new_session (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_stdin_broken_pipe (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_stdin_not_inheritable (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_stdin_stdout (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_subprocess_protocol_create_warning (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_terminate (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... ok
test_cancel_make_subprocess_transport_exec (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests) ... ok
test_cancel_post_init (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests) ... ok
test_cancel_process_wait (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests) ... ok
test_close_dont_kill_finished (test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests) ... ok
...Timeout (3:15:00)!
Thread 0x00007f666a7a1740 (most recent call first):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.refleak/build/Lib/selectors.py", line 468 in select
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.refleak/build/Lib/asyncio/base_events.py", line 1837 in _run_once
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.refleak/build/Lib/asyncio/base_events.py", line 589 in run_forever
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.refleak/build/Lib/asyncio/base_events.py", line 621 in run_until_complete
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.refleak/build/Lib/test/test_asyncio/test_subprocess.py", line 484 in test_close_kill_running

----------
components: Tests, asyncio
messages: 353568
nosy: asvetlov, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: test_ayncio: test_close_kill_running() hangs on AMD64 RHEL7 Refleaks 3.x
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38323>
_______________________________________

From report at bugs.python.org  Mon Sep 30 09:14:37 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 30 Sep 2019 13:14:37 +0000
Subject: [New-bugs-announce] [issue38324] test_locale and test__locale
 failures on Windows
Message-ID: <1569849277.36.0.00648751871696.issue38324@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

On Windows 10 version 1903, 3 locale tests fail:

vstinner at WIN C:\vstinner\python\3.8>python -m test -v test_locale test__locale

======================================================================
ERROR: test_getsetlocale_issue1813 (test.test_locale.TestMiscellaneous)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\vstinner\python\3.8\lib\test\test_locale.py", line 567, in test_getsetlocale_issue1813
    locale.setlocale(locale.LC_CTYPE, loc)
  File "C:\vstinner\python\3.8\lib\locale.py", line 608, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

======================================================================
ERROR: test_float_parsing (test.test__locale._LocaleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\vstinner\python\3.8\lib\test\test__locale.py", line 184, in test_float_parsing
    if localeconv()['decimal_point'] != '.':
UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: decoding error

======================================================================
ERROR: test_lc_numeric_localeconv (test.test__locale._LocaleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\vstinner\python\3.8\lib\test\test__locale.py", line 130, in test_lc_numeric_localeconv
    formatting = localeconv()
UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: decoding error

test_float_parsing() fails for locales:

* "ka_GE"
* "fr_FR.UTF-8"

test_lc_numeric_localeconv() fails for locales:

* 'ka_GE'
* 'fr_FR.UTF-8'
* 'ps_AF'

test_getsetlocale_issue1813() fails with:

   testing with ('tr_TR', 'ISO8859-9')

Example:

vstinner at WIN C:\vstinner\python\3.8>python
Running Release|x64 interpreter...
Python 3.8.0b4+ (heads/pr/16490:8a204fd07c, Sep 30 2019, 14:29:53) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>>> import locale
>>> locale.setlocale(locale.LC_CTYPE, 'tr_TR')
'tr_TR'
>>> loc=locale.getlocale(locale.LC_CTYPE)  
>>> loc
('tr_TR', 'ISO8859-9')
>>> locale.setlocale(locale.LC_CTYPE, loc) 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\vstinner\python\3.8\lib\locale.py", line 608, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

# It works using the low-level _locale module
# which doesn't parse setlocale() output

>>> import _locale                        
>>> _locale.setlocale(_locale.LC_CTYPE, None) 
'tr_TR'
>>> locale.setlocale(locale.LC_CTYPE, "tr_TR") 
'tr_TR'

----------
messages: 353570
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_locale and test__locale failures on Windows

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38324>
_______________________________________

From report at bugs.python.org  Mon Sep 30 09:20:15 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 30 Sep 2019 13:20:15 +0000
Subject: [New-bugs-announce] [issue38325] [Windows] test_winconsoleio
 failures
Message-ID: <1569849615.84.0.245223982483.issue38325@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at python.org>:

On Windows 10 version 1903, test_winconsoleio even when run from cmd.exe console.

C:\vstinner\python\3.8>python -m test -v test_winconsoleio
Running Release|x64 interpreter...
== CPython 3.8.0b4+ (heads/pr/16490:8a204fd07c, Sep 30 2019, 14:29:53) [MSC v.1916 64 bit (AMD64)]
== Windows-10-10.0.18362-SP0 little-endian
== cwd: C:\vstinner\python\3.8\build\test_python_1616
== CPU count: 2
== encodings: locale=cp1252, FS=utf-8
Run tests sequentially
0:00:00 [1/1] test_winconsoleio
test_abc (test.test_winconsoleio.WindowsConsoleIOTests) ... ok
test_conin_conout_names (test.test_winconsoleio.WindowsConsoleIOTests) ... ok
test_conout_path (test.test_winconsoleio.WindowsConsoleIOTests) ... ok
test_ctrl_z (test.test_winconsoleio.WindowsConsoleIOTests) ... ?^Z
ok
test_input (test.test_winconsoleio.WindowsConsoleIOTests) ... abc123
??????
A?B ??AA?
?ERROR
test_open_fd (test.test_winconsoleio.WindowsConsoleIOTests) ... ok
test_open_name (test.test_winconsoleio.WindowsConsoleIOTests) ... ok
test_partial_reads (test.test_winconsoleio.WindowsConsoleIOTests) ... ?ERROR
test_partial_surrogate_reads (test.test_winconsoleio.WindowsConsoleIOTests) ... ?ERROR
test_write_empty_data (test.test_winconsoleio.WindowsConsoleIOTests) ... ok

======================================================================
ERROR: test_input (test.test_winconsoleio.WindowsConsoleIOTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\vstinner\python\3.8\lib\test\test_winconsoleio.py", line 148, in test_input
    self.assertStdinRoundTrip('\U00100000\U0010ffff\U0010fffd')
  File "C:\vstinner\python\3.8\lib\test\test_winconsoleio.py", line 135, in assertStdinRoundTrip
    actual = input()
OSError: [WinError 87] Param?tre incorrect

======================================================================
ERROR: test_partial_reads (test.test_winconsoleio.WindowsConsoleIOTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\vstinner\python\3.8\lib\test\test_winconsoleio.py", line 161, in test_partial_reads
    b = stdin.read(read_count)
OSError: [WinError 87] Param?tre incorrect

======================================================================
ERROR: test_partial_surrogate_reads (test.test_winconsoleio.WindowsConsoleIOTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\vstinner\python\3.8\lib\test\test_winconsoleio.py", line 178, in test_partial_surrogate_reads
    b = stdin.read(read_count)
OSError: [WinError 87] Param?tre incorrect

----------------------------------------------------------------------

Ran 10 tests in 0.013s

FAILED (errors=3)
test test_winconsoleio failed
test_winconsoleio failed

== Tests result: FAILURE ==

1 test failed:
    test_winconsoleio

Total duration: 62 ms
Tests result: FAILURE

----------
components: Windows
messages: 353571
nosy: paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: [Windows] test_winconsoleio failures
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38325>
_______________________________________

From report at bugs.python.org  Mon Sep 30 10:08:11 2019
From: report at bugs.python.org (Nick Coghlan)
Date: Mon, 30 Sep 2019 14:08:11 +0000
Subject: [New-bugs-announce] [issue38326] Concerns with the last minute
 changes to the PEP 587 API
Message-ID: <1569852491.15.0.285571416668.issue38326@roundup.psfhosted.org>


New submission from Nick Coghlan <ncoghlan at gmail.com>:

(Nosy list is RM, PEP 587 BDFL-Delegate, PEP 587 author)

Filing as a release blocker, given that I don't think we should ship rc1 until consensus has been reached on the last minute changes to the PEP 587 configuration API.

Thread at https://mail.python.org/archives/list/python-dev at python.org/thread/C7Z2NA2DTM3DLOZCFQAK5A2WFYO3PHHX/

----------
messages: 353573
nosy: lukasz.langa, ncoghlan, twouters, vstinner
priority: release blocker
severity: normal
stage: commit review
status: open
title: Concerns with the last minute changes to the PEP 587 API
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38326>
_______________________________________

From report at bugs.python.org  Mon Sep 30 11:19:05 2019
From: report at bugs.python.org (Emil Bode)
Date: Mon, 30 Sep 2019 15:19:05 +0000
Subject: [New-bugs-announce] [issue38327] Creating set with empty string
 returns empty set
Message-ID: <1569856745.41.0.582082569134.issue38327@roundup.psfhosted.org>


New submission from Emil Bode <BodeEmil at gmail.com>:

Initializing/creating a new set with an empty string via the set-command returns an empty set instead of set with the empty string.

As in:
While set('somestring') gives me a set of size one, 
set('') gives me an empty set (of size zero), just as set() would do

It is possible to create a set with an empty string, as the command {''} and set(['']) work just as expected

----------
messages: 353585
nosy: EmilBode
priority: normal
severity: normal
status: open
title: Creating set with empty string returns empty set
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38327>
_______________________________________

From report at bugs.python.org  Mon Sep 30 12:57:25 2019
From: report at bugs.python.org (Brandt Bucher)
Date: Mon, 30 Sep 2019 16:57:25 +0000
Subject: [New-bugs-announce] [issue38328] Speed up the creation time of
 constant list literals.
Message-ID: <1569862645.36.0.924344090693.issue38328@roundup.psfhosted.org>


New submission from Brandt Bucher <brandtbucher at gmail.com>:

The attached PR contains a small change to the peephole optimizer that converts sequences of:

LOAD_CONST(a), LOAD_CONST(b), ..., BUILD_LIST(n)

to

LOAD_CONST((a, b, ...)), BUILD_LIST_UNPACK(1)

The improvement quickly becomes significant for lists larger than a few items (elements vs speedup):

 5:  ~5%
10: ~20%
15: ~25%
20: ~30%
25: ~35%
30: ~45%
35: ~50%

This can be tested on any version of Python by comparing the performance of "[0, 1, 2, ...]" vs "[*(0, 1, 2, ...)]". The common cases of empty and single-element lists are not affected by this change.

This is related to bpo-33325, but that was an invasive change for all collection literals that had an unknown affect on performance. I've limited this one to lists and kept it to a few lines in the peephole optimizer.

----------
components: Interpreter Core
messages: 353599
nosy: brandtbucher
priority: normal
severity: normal
status: open
title: Speed up the creation time of constant list literals.
type: performance
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38328>
_______________________________________

From report at bugs.python.org  Mon Sep 30 15:37:04 2019
From: report at bugs.python.org (Kevin Packard)
Date: Mon, 30 Sep 2019 19:37:04 +0000
Subject: [New-bugs-announce] [issue38329] Top level symlinks are broken in
 the Python 3.7.4 framework for macOS.
Message-ID: <1569872224.76.0.739484042885.issue38329@roundup.psfhosted.org>


New submission from Kevin Packard <pythonorg at blackfrog.com>:

The macOS installer for Python 3.7.4 places "Python.framework" into /Library/Frameworks/  However, the top-level symlinks inside the framework are broken for "Headers" and "Resources". Because of this, the framework can not be imported into an Xcode project.

----------
components: macOS
messages: 353612
nosy: KevinPackard, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Top level symlinks are broken in the Python 3.7.4 framework for macOS.
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38329>
_______________________________________

From report at bugs.python.org  Mon Sep 30 16:24:01 2019
From: report at bugs.python.org (apmatthews)
Date: Mon, 30 Sep 2019 20:24:01 +0000
Subject: [New-bugs-announce] [issue38330] httplib specifies content-length
 when transfer-encoding present
Message-ID: <1569875041.02.0.374553203578.issue38330@roundup.psfhosted.org>


New submission from apmatthews <apmatthews at gmail.com>:

RFC 7230 3.3.2 states "A sender MUST NOT send a Content-Length header field in any message that contains a Transfer-Encoding header field."

When trying to perform a chunked request:

import httplib
http = httplib.HTTPSConnection('google.com')
http.request("POST", '/', None, {'Content-Type': 'text/plain',
                                 'Transfer-Encoding': 'chunked'})

Resulting headers include:

Content-Length: 0
Transfer-Encoding: chunked

The receiving server should ignore the Content-Length in this case but some versions of IIS don't and consequently fail to accept the chunks that follow.

----------
components: Library (Lib)
messages: 353617
nosy: apmatthews
priority: normal
severity: normal
status: open
title: httplib specifies content-length when transfer-encoding present
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38330>
_______________________________________

From report at bugs.python.org  Mon Sep 30 16:39:38 2019
From: report at bugs.python.org (Huyston)
Date: Mon, 30 Sep 2019 20:39:38 +0000
Subject: [New-bugs-announce] [issue38331] Exec not recognizing global
 variables inside function
Message-ID: <1569875978.41.0.850227388561.issue38331@roundup.psfhosted.org>


New submission from Huyston <prajogaosu at gmail.com>:

This seems like a bug for me, forgive me if its not.

Consider the following script:

def func():
    print(var)

my_globals = {'func':func,'var':14}
exec("print(var);func()",my_globals,my_globals)

This is the output:

14
Traceback (most recent call last):
  File "bug.py", line 5, in <module>
    exec("print(var);func()",my_globals,my_globals)
  File "<string>", line 1, in <module>
  File "bug.py", line 2, in func
    print(var)
NameError: name 'var' is not defined

The first line gives the expected result (14). However the second line throws the NameError.

If 'func' is defined inside the exec argument string, then it prints 14 and 14, which is the expected result (for me at least).

Ex:

def func():
    print(var)

my_globals = {'func':func,'var':14}
exec("def func():\n    print(var)\nprint(var);func()",my_globals,my_globals)

Result:
14
14

So is this really a bug or is this the expected behavior somehow?

----------
components: Interpreter Core
messages: 353622
nosy: Huyston
priority: normal
severity: normal
status: open
title: Exec not recognizing global variables inside function
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38331>
_______________________________________

From report at bugs.python.org  Mon Sep 30 16:57:33 2019
From: report at bugs.python.org (Andrei Troie)
Date: Mon, 30 Sep 2019 20:57:33 +0000
Subject: [New-bugs-announce] [issue38332] invalid content-transfer-encoding
 in encoded-word causes KeyError
Message-ID: <1569877053.41.0.568400002445.issue38332@roundup.psfhosted.org>


New submission from Andrei Troie <andreitroie90 at gmail.com>:

The following will cause a KeyError on email.message.get()

import email
import email.policy

text = "Subject: =?us-ascii?X?somevalue?="
eml = email.message_from_string(text, policy=email.policy.default)
eml.get('Subject')

This is caused by the fact that the code in _encoded_words.py assumes the content-transfer-encoding of an encoded-word is always 'q' or 'b' (after lowercasing): https://github.com/python/cpython/blob/aca8c406ada3bb547765b262bed3ac0cc6be8dd3/Lib/email/_encoded_words.py#L178

I realise it's probably a silly edge case and I haven't (yet) encountered something like this in the wild, but it does seem contrary to the spirit of the email library to raise an exception like this that can propagate all the way to email.message.get().

----------
components: email
messages: 353624
nosy: aft90, barry, r.david.murray
priority: normal
severity: normal
status: open
title: invalid content-transfer-encoding in encoded-word causes KeyError
type: crash
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38332>
_______________________________________