From report at bugs.python.org Wed May 1 03:22:40 2019 From: report at bugs.python.org (Batuhan) Date: Wed, 01 May 2019 07:22:40 +0000 Subject: [New-bugs-announce] [issue36764] Types module doesn't have a type for _abc_data Message-ID: <1556695360.21.0.794709437035.issue36764@roundup.psfhosted.org> New submission from Batuhan : Types module doesn't have a type for _abc_data ---------- messages: 341180 nosy: isidentical priority: normal severity: normal status: open title: Types module doesn't have a type for _abc_data _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 1 06:47:02 2019 From: report at bugs.python.org (Kasra Vand) Date: Wed, 01 May 2019 10:47:02 +0000 Subject: [New-bugs-announce] [issue36765] Invalid grammar for f_expression Message-ID: <1556707622.28.0.558014565805.issue36765@roundup.psfhosted.org> New submission from Kasra Vand : Due to the discussion in following SO question https://stackoverflow.com/questions/55933956/what-does-a-star-asterisk-do-in-f-string/55934472#55933956 and the inconsistency of the source behaviour with the documentation I think using `"*" or_expr` for f_expression is wrong or at least not what it meant to be and very vague. I was wondering if there's any reason for using `"*" or_expr`. ---------- assignee: docs at python components: Documentation messages: 341186 nosy: Kasra Vand, docs at python priority: normal severity: normal status: open title: Invalid grammar for f_expression type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 1 09:12:27 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Wed, 01 May 2019 13:12:27 +0000 Subject: [New-bugs-announce] [issue36766] Typos in docs and code comments Message-ID: <1556716347.32.0.805084032343.issue36766@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : I ran aspell through the files and found below to be typos while manually filtering the aspell output. I have classified it into sections since I am not sure if code comments are worth fixing and added relevant files where they are present. I am tagging the issue for PyCon US dev sprints. Please don't make a PR for this if you have already contributed a PR. # Docs folder Cotnent -> Content-Transfer-Encoding (Doc/library/email.generator.rst) attibute -> attribute (Doc/library/pyclbr.rst) milleseconds -> milliseconds (Doc/library/idle.rst) # Code comments actuall -> actual (Lib/test/test_dataclasses.py) actully -> actually (Lib/contextlib.py) agains -> against (Lib/pathlib.py) Amerian -> American (Lib/test/test_random.py) Conflic -> Conflict (Lib/idlelib/idle_test/test_config.py) Depedent -> Dependent (Lib/test/datetimetester.py) Diffent -> Different (Lib/test/support/__init__.py) Diplay -> Display (Objects/object.c) Doens't -> Doesn't (Lib/test/test_ssl.py) Dono't -> Don't (Lib/test/test_importlib/test_lazy.py) intermitted -> intermittent (Lib/test/pickletester.py) invokation -> Invocation (Lib/platform.py) occuring -> occurring (Lib/test/test_tools/test_i18n.py) permuations -> permutations (Lib/tokenize.py) resouce -> resource (Lib/multiprocessing/resource_sharer.py) statists -> statistics (Lib/pstats.py) # Docstring Dafault -> Default (Lib/turtle.py) Diplay -> Display (Lib/idlelib/help.py) Intput -> Input (Lib/lib2to3/pgen2/tokenize.py) milleseconds -> milliseconds (Lib/idlelib/configdialog.py, Lib/idlelib/help.html) ---------- assignee: Mariatta components: Documentation messages: 341194 nosy: Mariatta, cheryl.sabella, xtreak priority: normal severity: normal status: open title: Typos in docs and code comments type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 1 09:49:57 2019 From: report at bugs.python.org (Senhui Guo) Date: Wed, 01 May 2019 13:49:57 +0000 Subject: [New-bugs-announce] [issue36767] Segmentation fault when running c extension on macOS Message-ID: <1556718597.1.0.626589085866.issue36767@roundup.psfhosted.org> New submission from Senhui Guo : I was trying to build a c extension with cpython while it keeps crashing when I get it running, the code is quite simple: ```cpp #include "Python.h" int main () { PyObject *p; p = PySet_New(NULL); } ``` I build with command `clang -I/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/include/python3.7m -L/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/config-3.7m-darwin -lpython3.7m test.c -o run` when I ran it `./run`, segmentation fault: 11 kicks in However, if I go with python2, it worked fine: `clang -I /usr/local/Cellar/python\@2/2.7.16/Frameworks/Python.framework/Versions/2.7/include/python2.7 -L /usr/local/opt/python\@2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/ -lpython2.7 test.c -o run` I was wondering if it is the problem with macOS? I tried build python from source, but didn't work, keeps crashing ---------- components: macOS messages: 341196 nosy: Senhui Guo, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: Segmentation fault when running c extension on macOS type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 1 10:01:56 2019 From: report at bugs.python.org (Charles P) Date: Wed, 01 May 2019 14:01:56 +0000 Subject: [New-bugs-announce] [issue36768] distutils.util.convert_path mangles windows paths with forward slashes Message-ID: <1556719316.49.0.543223746824.issue36768@roundup.psfhosted.org> New submission from Charles P : https://github.com/python/cpython/blob/master/Lib/distutils/util.py#L106-L131 Due to the split('/') and os.path.join(), this function converts an absolute path of the form "C:/foobar" into a relative "C:foobar", which is likely to be entirely different Usecase: pip install --prefix=/home/charles/python foobar within an MSYS2 terminal automagically converts the path to C:/Users/charles/python for some reason or another, but it's not exactly uncommon for users to use forward slashes on Windows regardless I'm not entirely sure what the correct fix would be here, or even if it should be fixed at a higher level - in setuptools or pip ---------- components: Distutils messages: 341197 nosy: LordAro, dstufft, eric.araujo priority: normal severity: normal status: open title: distutils.util.convert_path mangles windows paths with forward slashes type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 1 10:12:45 2019 From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=) Date: Wed, 01 May 2019 14:12:45 +0000 Subject: [New-bugs-announce] [issue36769] doc Document that fnmatch.filter supports any kind of iterable not just lists Message-ID: <1556719965.28.0.191162970906.issue36769@roundup.psfhosted.org> New submission from Andr?s Delfino : Documentation on fnmatch.filter says: Return the subset of the list of names that match pattern. It is the same as [n for n in names if fnmatch(n, pattern)], but implemented more efficiently. But the function actual accepts any kind of iterable. I think it should be documented. ---------- assignee: docs at python components: Documentation messages: 341198 nosy: adelfino, docs at python priority: normal severity: normal status: open title: doc Document that fnmatch.filter supports any kind of iterable not just lists type: enhancement versions: Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 1 10:36:08 2019 From: report at bugs.python.org (Owen Chia) Date: Wed, 01 May 2019 14:36:08 +0000 Subject: [New-bugs-announce] [issue36770] stdlib - shutil.make_archive - add support for different ZIP compression method Message-ID: <1556721368.48.0.786007999998.issue36770@roundup.psfhosted.org> New submission from Owen Chia : if you just want to use different zip compression method, no need to rewrite entire _make_zipfile function. e.g. >>> shutil.make_archive('archive', 'zip_lzma', '/path/to/whatever') ---------- components: Library (Lib) files: shutil.make_archive.patch keywords: patch messages: 341201 nosy: giampaolo.rodola, owenchia, tarek priority: normal severity: normal status: open title: stdlib - shutil.make_archive - add support for different ZIP compression method type: enhancement versions: Python 3.8 Added file: https://bugs.python.org/file48293/shutil.make_archive.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 1 11:20:07 2019 From: report at bugs.python.org (CJ Kucera) Date: Wed, 01 May 2019 15:20:07 +0000 Subject: [New-bugs-announce] [issue36771] Feature Request: An option to os.walk() to return os.DirEntry lists instead of just filenames/dirnames Message-ID: <1556724007.57.0.489851634937.issue36771@roundup.psfhosted.org> New submission from CJ Kucera : It'd be nice to have an option to os.walk which would return DirEntry objects in its return tuple, as opposed to just the string filenames/dirnames. (Or failing that, an alternate function which does so.) The function already uses os.scandir() internally, so the DirEntry objects already exist -- I assume it'd be a pretty easy change. At the moment, if I want to be efficient and use os.scandir() myself, I've got to basically reimplement os.walk(), which seems silly since os.walk is already calling scandir itself. ---------- components: Library (Lib) messages: 341210 nosy: apocalyptech priority: normal severity: normal status: open title: Feature Request: An option to os.walk() to return os.DirEntry lists instead of just filenames/dirnames type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 1 20:40:09 2019 From: report at bugs.python.org (Raymond Hettinger) Date: Thu, 02 May 2019 00:40:09 +0000 Subject: [New-bugs-announce] [issue36772] Let lru_cache be used as a decorator with no arguments Message-ID: <1556757609.15.0.729025293802.issue36772@roundup.psfhosted.org> New submission from Raymond Hettinger : Follow the lead of the dataclasses module and allow lru_cache() to be used as a straight decorator rather than as a function that returns a decorator. Both of these would now be supported: @lru_cache def f(x): ... @lru_cache(maxsize=256) def f(x): ... ---------- components: Library (Lib) messages: 341239 nosy: eric.smith, rhettinger, serhiy.storchaka priority: normal severity: normal status: open title: Let lru_cache be used as a decorator with no arguments type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 2 04:24:59 2019 From: report at bugs.python.org (papad) Date: Thu, 02 May 2019 08:24:59 +0000 Subject: [New-bugs-announce] [issue36773] Race condition during pickle.load() Message-ID: <1556785499.96.0.563265579216.issue36773@roundup.psfhosted.org> New submission from papad : There seems to be a race condition when unpickling the same object from different threads (My guess is it's related to imports that are related to the unpickled object). I have used the following files for reproduction (File contents are in the attached archive, also pasted at the bottom of the issue report): import_me.py - Containing the object we will pickle pickle_object.py - the code to pickle the object trigger.py - a piece of code to unpickle the object from different threads simultaneously and trigger the crash I have used the files in the attached archive (containing the 3 files mentioned above) to reproduce run the following commands: pickle_object.py trigger.py Running trigger.py will crash on about 50% of the runs with the following error: Traceback (most recent call last): File "./trigger.py", line 16, in pickle_load_thread pickle.load(h) AttributeError: Can't get attribute 'PickleMe' on I have tested this on the following software stacks: 1. python:3.7.3 docker - linux version: 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 GNU/Linux - python: 3.7.3 - distro: Debian GNU/Linux 9 (stretch) 2. my laptop - linux version: 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux - python: 3.6.7 - distro: Ubuntu 18.04.2 LTS I'm uncertain if this is a bug related to pickle or python module importing. Similar issues I've found: https://bugs.python.org/issue12680 and https://bugs.python.org/issue30891. --------------------------------------------------------------------------------------- filename: import_me.py #! /usr/bin/python3 class PickleMe(object): def __init__(self): self.a = "a" --------------------------------------------------------------------------------------- filename: pickle_object.py #! /usr/bin/python3 import pickle from import_me import PickleMe p = PickleMe() with open('pickled', 'wb') as h: pickle.dump(p, h) --------------------------------------------------------------------------------------- filename: trigger.py #! /usr/bin/python3 import threading import pickle import logging threads = [] def pickle_load_thread(): logging.error("Thread %d loading", threading.get_ident()) try: with open('pickled', 'rb') as h: pickle.load(h) except: logging.exception("Exception in loading") def start_pickle_load_thread(): for x in range(2): load_thread = threading.Thread(target=pickle_load_thread) load_thread.daemon = True threads.append(load_thread) for x in threads: x.start() if __name__ == '__main__': start_pickle_load_thread() for t in threads: t.join() ---------- components: Library (Lib) files: reproduce.tar.gz messages: 341257 nosy: papad priority: normal severity: normal status: open title: Race condition during pickle.load() type: crash versions: Python 3.7 Added file: https://bugs.python.org/file48294/reproduce.tar.gz _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 2 07:40:45 2019 From: report at bugs.python.org (Eric V. Smith) Date: Thu, 02 May 2019 11:40:45 +0000 Subject: [New-bugs-announce] [issue36774] f-strings: Add a !d conversion for ease of debugging Message-ID: <1556797245.95.0.619255948077.issue36774@roundup.psfhosted.org> New submission from Eric V. Smith : I originally propsed this here: https://mail.python.org/pipermail/python-ideas/2018-October/053956.html, and there has also been some discussion on discourse. The proposal is to add a !d "type conversion" to f-strings, to make for simpler "print-based debugging". The idea is that !d would be like !r, etc., except that the resulting string would be: - the text of the expression, followed by - an equal sign, followed by - the value of the expression So this code: s = 'A string' result = f'{s!d}' Would set result to 's="A string"'. Note that the text of the expression is exactly what's between the opening brace and the !. So: result = f'{s !d}' Would set result to 's ="A string"'. Note the space before the equal sign. I can't decide if I'm going to allow a format specifier. If I do, it would apply to the entire resulting string. So: result = f'{s!d:*^20}' Would set result to '****s="A string"****'. But I might reserve format specs for further use, perhaps to apply to the value of the expression instead of the resulting string. Of course this is of most value for more complex expressions. This: v = 3 result = f'{v*9+15!d}' Would set result to 'v*9+15=42'. I'd expect this mostly to be used with print() or logging, but it's a general f-string feature. I have a patch almost ready. ---------- assignee: eric.smith components: Interpreter Core messages: 341261 nosy: eric.smith priority: normal severity: normal stage: needs patch status: open title: f-strings: Add a !d conversion for ease of debugging type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 2 10:51:44 2019 From: report at bugs.python.org (STINNER Victor) Date: Thu, 02 May 2019 14:51:44 +0000 Subject: [New-bugs-announce] [issue36775] Rework filesystem codec implementation Message-ID: <1556808704.71.0.363298231326.issue36775@roundup.psfhosted.org> New submission from STINNER Victor : I would like to add callbacks in PyInterpreterState for the filesystem encoding to avoid parsing the encoding name at each encode/decode call (PyUnicode_EncodeFSDefault, PyUnicode_DecodeFSDefault). ---------- components: Interpreter Core messages: 341267 nosy: vstinner priority: normal severity: normal status: open title: Rework filesystem codec implementation versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 2 11:18:50 2019 From: report at bugs.python.org (STINNER Victor) Date: Thu, 02 May 2019 15:18:50 +0000 Subject: [New-bugs-announce] [issue36776] test_tools: test_lll_multiple_dirs() failed on AMD64 Windows7 SP1 3.x Message-ID: <1556810330.96.0.118002544008.issue36776@roundup.psfhosted.org> New submission from STINNER Victor : AMD64 Windows7 SP1 3.x: https://buildbot.python.org/all/#/builders/40/builds/2106 ERROR: test_lll_multiple_dirs (test.test_tools.test_lll.lllTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_tools\test_lll.py", line 24, in test_lll_multiple_dirs os.symlink(fn, os.path.join(dir, 'symlink')) OSError: [WinError 1314] A required privilege is not held by the client: 'C:\\Users\\Buildbot\\AppData\\Local\\Temp\\tmp1wogs146\\foo1' -> 'C:\\Users\\Buildbot\\AppData\\Local\\Temp\\tmp1wogs146\\symlink' The function failed twice. ---------- components: Tests messages: 341272 nosy: pablogsal, vstinner priority: normal severity: normal status: open title: test_tools: test_lll_multiple_dirs() failed on AMD64 Windows7 SP1 3.x versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 2 12:47:03 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Thu, 02 May 2019 16:47:03 +0000 Subject: [New-bugs-announce] [issue36777] unittest discover throws TypeError on empty packages Message-ID: <1556815623.31.0.225997144606.issue36777@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : In the given folder structure unittest discover fails. I think this is due to issue32303 where __file__ was set as None and I guess empty packages didn't have __file__ set before which has None value now. Hence the_module.__file__ returns None and thus subsequent calls to os.path.dirname raise TypeError. There is one another location in the same file in unittest module causing this issue. See also issue36406 for similar error in doctest with the change. I reverted bbbcf8693b876daae4469765aa62f8924f39a7d2 just to confirm the issue. I am adding devs in issue32303 and unittest. Any thoughts on this would be helpful. ? cpython git:(master) ? tree test1 test1 ??? test2 ??? test_foo.py 1 directory, 1 file ? cpython git:(master) ? ./python.exe -m unittest discover test1.test2 Traceback (most recent call last): File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/__main__.py", line 18, in main(module=None) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/main.py", line 100, in __init__ self.parseArgs(argv) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/main.py", line 124, in parseArgs self._do_discovery(argv[2:]) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/main.py", line 244, in _do_discovery self.createTests(from_discovery=True, Loader=Loader) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/main.py", line 154, in createTests self.test = loader.discover(self.start, self.pattern, self.top) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/loader.py", line 306, in discover os.path.dirname((the_module.__file__))) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/posixpath.py", line 152, in dirname p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType # 3.6.4 works that doesn't have the change ? cpython git:(master) ? python3.6 -m unittest discover test1.test2 . ---------------------------------------------------------------------- Ran 1 test in 0.000s OK A unittest would be as below : def test_empty_package_discovery(self): # bpo-36789: Return zero test cases when using discovery in # empty packages. with support.temp_dir() as path: dirname, basename = os.path.split(path) os.mkdir(os.path.join(path, 'test2')) with support.DirsOnSysPath(dirname): loader = unittest.TestLoader() empty_package = f"{basename}.test2" tests_count = loader.discover(empty_package).countTestCases() self.assertEqual(loader.discover(empty_package).countTestCases(), 0) One possible fix would be to check for __file__ == None and return empty list of test cases but this causes a behavior change like below where my patch returns 0 and python 3.6 returns 1 test. # Patch diff --git a/Lib/unittest/loader.py b/Lib/unittest/loader.py index ba7105e1ad..f465b2419f 100644 --- a/Lib/unittest/loader.py +++ b/Lib/unittest/loader.py @@ -302,6 +302,10 @@ class TestLoader(object): the_module = sys.modules[start_dir] top_part = start_dir.split('.')[0] try: + filepath = the_module.__file__ + # __file__ is None for empty packages. Hence return empty list of tests. + if filepath == None: + return self.suiteClass([]) start_dir = os.path.abspath( os.path.dirname((the_module.__file__))) except AttributeError: # Behavior change ? cpython git:(unittest-empty-package) ? ./python.exe -m unittest discover test1.test2 ---------------------------------------------------------------------- Ran 0 tests in 0.000s OK ? cpython git:(unittest-empty-package) ? python3.6 -m unittest discover test1.test2 . ---------------------------------------------------------------------- Ran 1 test in 0.000s OK My patch also causes reference leak and I am not sure why. So I guess mine is not the best approach to solving this. ? cpython git:(unittest-empty-package) ? ./python.exe -m test --fail-env-changed -R 3:3 test_unittest Run tests sequentially 0:00:00 load avg: 1.46 [1/1] test_unittest beginning 6 repetitions 123456 ...... test_unittest leaked [99, 99, 99] references, sum=297 test_unittest leaked [38, 38, 38] memory blocks, sum=114 test_unittest failed in 43 sec 634 ms == Tests result: FAILURE == 1 test failed: test_unittest Total duration: 43 sec 655 ms Tests result: FAILURE ---------- components: Library (Lib) messages: 341285 nosy: barry, brett.cannon, eric.smith, ezio.melotti, michael.foord, rbcollins, xtreak priority: normal severity: normal status: open title: unittest discover throws TypeError on empty packages type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 2 16:46:25 2019 From: report at bugs.python.org (Paul Monson) Date: Thu, 02 May 2019 20:46:25 +0000 Subject: [New-bugs-announce] [issue36778] test_site.StartupImportTests.test_startup_imports fails if default code page is not cp1252 Message-ID: <1556829985.34.0.069265948139.issue36778@roundup.psfhosted.org> New submission from Paul Monson : Windows desktop skus have a default ANSI codepage (returned by GetACP()) of 1252 (Western European). Windows IoT Core and Windows Nano Server have a default codepage of 65001 (UTF-8). This causes test_site.StartupImportTests.test_startup_imports to fail on Windows IoT Core and Windows Nano Server because cp65001.py is loaded instead of the frozen cp1252.py at startup. I tried changing the default codepage to 65001 on my dev machine and rebuilding Python and it had no effect that I could tell on the generated frozen importlibs. The simplest solutions would be for the test_startup_imports test to be skipped or changed to pass when the locale.getpreferredencoding() returns 'cp65001' ---------- components: Tests, Windows messages: 341316 nosy: Paul Monson, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: test_site.StartupImportTests.test_startup_imports fails if default code page is not cp1252 type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 2 20:45:22 2019 From: report at bugs.python.org (Paul Monson) Date: Fri, 03 May 2019 00:45:22 +0000 Subject: [New-bugs-announce] [issue36779] strptime returns empty string on Windows if default codepage is a Unicode codepage Message-ID: <1556844322.34.0.979846087129.issue36779@roundup.psfhosted.org> New submission from Paul Monson : Need to work around a CRT bug in the use of _tzset() + _tzname[] Calling setlocale(LC_CTYPE, "") on a system where GetACP() returns CP_UTF8 results in empty strings in _tzname[]. This causes time.tzname to be an empty string. I have reported the bug. One possible workaround is to temporarily change the locale by calling setlocale(LC_CTYPE, "C") before calling _tzset and restore the current locale after if the GetACP() == CP_UTF8 or CP_UTF7 ---------- messages: 341318 nosy: Paul Monson priority: normal severity: normal status: open title: strptime returns empty string on Windows if default codepage is a Unicode codepage _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 05:28:03 2019 From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=) Date: Fri, 03 May 2019 09:28:03 +0000 Subject: [New-bugs-announce] [issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors Message-ID: <1556875683.94.0.285155490659.issue36780@roundup.psfhosted.org> New submission from Hrvoje Nik?i? : At interpreter shutdown, Python waits for all pending futures of all executors to finish. There seems to be no way to disable the wait for pools that have been explicitly shut down with pool.shutdown(wait=False). The attached script demonstrates the issue. In our code the futures are running blocking network calls that can be canceled by the user. The cancel action automatically cancels the pending futures and informs the running ones that they should exit. The remaining futures are those whose callables are "stuck" in network calls with long or infinite timeouts, such as reading from a non-responding network filesystem. Since those can't be interrupted, we use pool.shutdown(wait=False) to disown the whole pool. This works nicely, until the application exit, at which point it blocks trying to wait for the pending futures to finish. This can take an arbitrary amount of time, possibly never finishing. A similar question has come up on StackOverflow, with the only answer recommending to unregister concurrent.futures.thread._python_exit: https://stackoverflow.com/q/48350257/1600898 . We are ourselves using a similar hack, but we would like to contribute a proper way to disown an executor. The current behavior is explicitly documented, so presumably it can't be (easily) changed, but we could add a "disown" keyword argument to shutdown(), or a new disown() method, which would serve to explicitly disable the waiting. If this is considered desirable, I will create a pull request. ---------- components: Library (Lib) files: pool-shutdown messages: 341329 nosy: hniksic priority: normal severity: normal status: open title: Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors versions: Python 3.7, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file48297/pool-shutdown _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 05:49:33 2019 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 03 May 2019 09:49:33 +0000 Subject: [New-bugs-announce] [issue36781] Optimize sum() for bools Message-ID: <1556876973.98.0.990157093134.issue36781@roundup.psfhosted.org> New submission from Serhiy Storchaka : To count the number of items that satisfy certain condition you can use either sum(1 for x in data if pred(x)) or sum(pred(x) for x in data) where pred(x) is a boolean expression. The latter case is shorter but slower. There are two causes for this: 1. The generator expression needs to generate more items, not only when pred(x) is true, but also when pred(x) is false. 2. sum() is optimized for integers and floats, but not for bools. The first cause is out of the scope of this issue, but sum() can optimized for bools. $ ./python -m timeit -s "a = [True] * 10**6" -- "sum(a)" Unpatched: 10 loops, best of 5: 22.3 msec per loop Patched: 50 loops, best of 5: 6.26 msec per loop $ ./python -m timeit -s "a = list(range(10**6))" -- "sum(x % 2 == 0 for x in a)" Unpatched: 5 loops, best of 5: 89.8 msec per loop Patched: 5 loops, best of 5: 67.5 msec per loop $ ./python -m timeit -s "a = list(range(10**6))" -- "sum(1 for x in a if x % 2 == 0)" 5 loops, best of 5: 53.9 msec per loop ---------- components: Interpreter Core messages: 341330 nosy: rhettinger, serhiy.storchaka priority: normal severity: normal status: open title: Optimize sum() for bools type: performance versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 09:46:57 2019 From: report at bugs.python.org (Paul Ganssle) Date: Fri, 03 May 2019 13:46:57 +0000 Subject: [New-bugs-announce] [issue36782] Add tests for the datetime C API Message-ID: <1556891217.54.0.801221071749.issue36782@roundup.psfhosted.org> New submission from Paul Ganssle : A decent fraction of the datetime C API has no tests. If we had tests, we could have prevented bpo #36025, which was a regression in the FromTimestamp method. I would like to open this issue to suggest the addition of tests for the full interface, to prevent further regressions here. To write a test, first add a wrapper function in the _testcapimodule module: https://github.com/python/cpython/blob/master/Modules/_testcapimodule.c#L2218 Then test that function here: https://github.com/python/cpython/blob/master/Lib/test/datetimetester.py#L5821 See an example at GH PR 11922: https://github.com/python/cpython/pull/11922 I recommend testing *both* the version of the function accessed by the Macro (e.g. PyDate_FromTimestamp) and from the C API capsule (e.g. PyDateTimeApi->FromTimestamp), since projects access these functions in both ways. C API Documentation is here: https://docs.python.org/3/c-api/datetime.html Currently untested: - PyDate_FromDate / PyDateTimeAPI->Date_FromDate - PyDateTime_FromDateAndTime / PyDateTimeAPI->DateTime_FromDateAndTime - PyDateTime_FromDateAndTimeAndFold / PyDateTimeAPI->DateTime_FromDateAndTimeAndFold - PyTime_FromTime -> PyDateTimeAPI->Time_FromTime - PyTime_FromTimeAndFold -> PyDateTime->Time_FromTimeAndFold - PyDelta_FromDSU / PyDateTime->Delta_FromDelta Untested macros with no corresponding API module: - PyDateTime_DATE_GET_YEAR - PyDateTime_DATE_GET_MONTH - PyDateTime_DATE_GET_DAY - PyDateTime_DATE_GET_HOUR - PyDateTime_DATE_GET_MINUTE - PyDateTime_DATE_GET_SECOND - PyDateTime_DATE_GET_MICROSECOND - PyDateTime_DELTA_GET_DAYS - PyDateTime_DELTA_GET_SECONDS - PyDateTime_DELTA_GET_MICROSECONDS I can spawn smaller issues for this if that's preferred, but I figured we'd start with this big meta-issue. ---------- components: Tests keywords: easy messages: 341336 nosy: p-ganssle priority: low severity: normal status: open title: Add tests for the datetime C API type: enhancement versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 09:51:55 2019 From: report at bugs.python.org (Paul Ganssle) Date: Fri, 03 May 2019 13:51:55 +0000 Subject: [New-bugs-announce] [issue36783] No documentation for _FromXandFold C API functions Message-ID: <1556891515.33.0.269336043459.issue36783@roundup.psfhosted.org> New submission from Paul Ganssle : In Python 3.6, Time_FromTimeAndFold and PyDateTime_FromDateAndTimeAndFold were added as part of the PEP 495 implementation, but no documentation was added for the C API portions of this change: https://docs.python.org/3.7/c-api/datetime.html The functions were added to this portion of the C API capsule: https://github.com/python/cpython/blob/master/Include/datetime.h#L173 Macros are here: https://github.com/python/cpython/blob/master/Include/datetime.h#L222 These functions should be documented, with `..versionadded:: 3.6` ---------- assignee: docs at python components: Documentation keywords: easy messages: 341337 nosy: docs at python, p-ganssle priority: low severity: normal status: open title: No documentation for _FromXandFold C API functions versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 10:02:55 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Fri, 03 May 2019 14:02:55 +0000 Subject: [New-bugs-announce] [issue36784] __import__ with empty folder after importlib.invalidate_caches causes reference leak Message-ID: <1556892175.48.0.910434484078.issue36784@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : I originally hit upon this with issue36777 where I have used support.script_helper.make_script which calls importlib.invalidate_caches and then trying to use __import__ on an empty folder causes reference leak. I tried 3.8 to 3.5 and it exists on each version. A sample script as below. I tried using try finally instead of DirsOnSysPath in doubt and it still causes leak. I couldn't find any issues on search and let me know if I am using something in an incorrect manner. import importlib import unittest import os, sys import os.path from test import support def test_importlib_cache(): with support.temp_dir() as path: dirname, basename = os.path.split(path) os.mkdir(os.path.join(path, 'test2')) importlib.invalidate_caches() with support.DirsOnSysPath(dirname): __import__("{basename}.test2".format(basename=basename)) class Tests(unittest.TestCase): def test_bug(self): for _ in range(10): test_importlib_cache() ? cpython git:(master) ? ./python.exe -m test -R 3:3 test_import_bug Run tests sequentially 0:00:00 load avg: 1.56 [1/1] test_import_bug beginning 6 repetitions 123456 ...... test_import_bug leaked [980, 980, 980] references, sum=2940 test_import_bug leaked [370, 370, 370] memory blocks, sum=1110 test_import_bug failed == Tests result: FAILURE == 1 test failed: test_import_bug Total duration: 1 sec 529 ms Tests result: FAILURE I also tried __import__('test1.test2') instead of __import__("{basename}.test2".format(basename=basename)) and the program doesn't cause reference leak. Moving importlib.invalidate_caches() above support.temp_dir() also causes leak so I guess it's not something to do with temporary directories that are cleaned up after tests. ? cpython git:(master) ? mkdir -p test1/test2 ? cpython git:(master) ? ./python.exe -m test -R 3:3 test_import_bug Run tests sequentially 0:00:00 load avg: 1.97 [1/1] test_import_bug beginning 6 repetitions 123456 ...... test_import_bug passed == Tests result: SUCCESS == 1 test OK. Total duration: 557 ms Tests result: SUCCESS ---------- components: Library (Lib) messages: 341338 nosy: brett.cannon, eric.snow, ncoghlan, xtreak priority: normal severity: normal status: open title: __import__ with empty folder after importlib.invalidate_caches causes reference leak type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 14:28:21 2019 From: report at bugs.python.org (Antoine Pitrou) Date: Fri, 03 May 2019 18:28:21 +0000 Subject: [New-bugs-announce] [issue36785] Implement PEP 574 Message-ID: <1556908101.83.0.107763193732.issue36785@roundup.psfhosted.org> New submission from Antoine Pitrou : Now that PEP 574 (Pickle protocol 5 with out-of-band data) has been accepted, it should be implemented. ---------- components: Extension Modules, Library (Lib) messages: 341349 nosy: ncoghlan, pitrou priority: normal severity: normal stage: needs patch status: open title: Implement PEP 574 type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 14:37:08 2019 From: report at bugs.python.org (Antoine Pitrou) Date: Fri, 03 May 2019 18:37:08 +0000 Subject: [New-bugs-announce] [issue36786] "make install" should run compileall in parallel Message-ID: <1556908628.98.0.810004261153.issue36786@roundup.psfhosted.org> New submission from Antoine Pitrou : Title says it all. Currently, "make install" will bytecode-compile Python sources sequentially even though compileall supports doing it in parallel. ---------- components: Build messages: 341350 nosy: pitrou priority: normal severity: normal stage: needs patch status: open title: "make install" should run compileall in parallel type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 16:30:35 2019 From: report at bugs.python.org (Gawain Bolton) Date: Fri, 03 May 2019 20:30:35 +0000 Subject: [New-bugs-announce] [issue36787] Python3 regresison: String formatting of None object Message-ID: <1556915435.58.0.661540149736.issue36787@roundup.psfhosted.org> New submission from Gawain Bolton : Python 2.7.16 (default, Apr 6 2019, 01:42:57) [GCC 8.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print('{:^10}'.format(None)) None However this does not work with Python3: Python 3.7.3 (default, Apr 3 2019, 05:39:12) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print('{:^10}'.format(None)) Traceback (most recent call last): File "", line 1, in TypeError: unsupported format string passed to NoneType.__format__ Given that the None type is output as a string, it makes sense for string formatting options to be usable with it. It also makes code less fragile and avoids having to write special cases for when values could be None. ---------- components: Library (Lib) messages: 341355 nosy: Gawain Bolton priority: normal severity: normal status: open title: Python3 regresison: String formatting of None object type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 17:50:00 2019 From: report at bugs.python.org (TheComet) Date: Fri, 03 May 2019 21:50:00 +0000 Subject: [New-bugs-announce] [issue36788] Add clamp() function to builtins Message-ID: <1556920200.45.0.0628062537563.issue36788@roundup.psfhosted.org> New submission from TheComet : It would be nice to have a clamp() builtin in addition to min() and max() so one can type e.g. "clamp(value, 2, 5)" instead of having to type "min(max(value, 5), 2)". ---------- components: Library (Lib) messages: 341358 nosy: TheComet priority: normal severity: normal status: open title: Add clamp() function to builtins type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 20:00:17 2019 From: report at bugs.python.org (mbiggs) Date: Sat, 04 May 2019 00:00:17 +0000 Subject: [New-bugs-announce] [issue36789] Unicode HOWTO incorrectly states that UTF-8 contains no zero bytes Message-ID: <1556928017.33.0.648089706151.issue36789@roundup.psfhosted.org> New submission from mbiggs : In the Unicode HOWTO: http://docs.python.org/3.3/howto/unicode.html It says the following: "UTF-8 has several convenient properties: (...) 2. A Unicode string is turned into a sequence of bytes containing no embedded zero bytes. This avoids byte-ordering issues, and means UTF-8 strings can be processed by C functions such as strcpy() and sent through protocols that can?t handle zero bytes." This is not right. UTF-8 uses the zero byte to represent the Unicode codepoint U+0000 (the ASCII NULL character). This is a valid character in UTF-8 and is handled just fine by python's UTF-8 string encoding/decoding. ---------- assignee: docs at python components: Documentation messages: 341363 nosy: docs at python, mbiggs priority: normal severity: normal status: open title: Unicode HOWTO incorrectly states that UTF-8 contains no zero bytes versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 3 20:10:11 2019 From: report at bugs.python.org (Alexander Riccio) Date: Sat, 04 May 2019 00:10:11 +0000 Subject: [New-bugs-announce] [issue36790] test_asyncio fails with application verifier! Message-ID: <1556928611.19.0.664021052346.issue36790@roundup.psfhosted.org> New submission from Alexander Riccio : I compiled PCBuild Debug x64 from an updated clone of upstream, and when running the testsuite under Application Verifier with handle verification, the test triggers an invalid handle access by passing an invalid overlapped handle to CancelIoEx with this code: Py_CancelIoEx(self->handle, &self->overlapped) (where self->handle appears to be the offending variable). I have no idea who's calling _overlapped.cancel, and a quick spelunking through the codebase only confuses me more. ---------- components: Tests files: python_test_invalid_handle_failure.TXT messages: 341365 nosy: Alexander Riccio priority: normal severity: normal status: open title: test_asyncio fails with application verifier! versions: Python 3.8 Added file: https://bugs.python.org/file48298/python_test_invalid_handle_failure.TXT _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 4 01:54:49 2019 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 04 May 2019 05:54:49 +0000 Subject: [New-bugs-announce] [issue36791] sum() relies on C signed overflow behaviour Message-ID: <1556949289.05.0.226850923115.issue36791@roundup.psfhosted.org> New submission from Serhiy Storchaka : sum() assumes that an arithmetic operation on signed longs will wrap modulo 2**(bits_in_long) on overflow. However, signed overflow causes undefined behaviour according to the C standards (e.g., C99 6.5, para. 5), and gcc is known to assume that signed overflow never occurs in correct code, and to make use of this assumption when optimizing. See also issue7406. ---------- components: Interpreter Core messages: 341374 nosy: mark.dickinson, rhettinger, serhiy.storchaka priority: normal severity: normal status: open title: sum() relies on C signed overflow behaviour type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 4 08:33:18 2019 From: report at bugs.python.org (Charlie Clark) Date: Sat, 04 May 2019 12:33:18 +0000 Subject: [New-bugs-announce] [issue36792] zipfile.writestr causes a Python crash on Windows if the locale is set Message-ID: <1556973198.72.0.765243261905.issue36792@roundup.psfhosted.org> New submission from Charlie Clark : Based on a bug report (https://bitbucket.org/openpyxl/openpyxl/issues/1266/locale) from a user of the openpyxl library I've identified a bug in the zipfile module that causes the Python process to crash on Windows. Currently tested with Python 3.7.3 (32-bit on Windows 10). Sample code import faulthandler import locale from zipfile import ZipFile faulthandler.enable() locale.setlocale(locale.LC_ALL, 'de_DE') out = open("out.zip", "wb") archive = ZipFile(out, "w") archive.writestr("properties.xml", b"") faulthandler fingers line 1757 as the culprit but running this line locally does not cause the crash. The issue seems to be limited to Windows. ---------- components: Interpreter Core, Windows messages: 341380 nosy: CharlieClark, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: zipfile.writestr causes a Python crash on Windows if the locale is set versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 4 09:10:23 2019 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 04 May 2019 13:10:23 +0000 Subject: [New-bugs-announce] [issue36793] Do not set tp_str Message-ID: <1556975423.86.0.374987997006.issue36793@roundup.psfhosted.org> New submission from Serhiy Storchaka : object.__str__() calls the __repr__() method. Therefore by default you do not need to define the __str__() method if it is the same as __repr__(): just define the __repr__() method. But there are few builtin classes which define both __repr__() and __str__() methods which are equal. In most cases this is a legacy of Python 2 where they where different. 1. float and complex. In earlier Python 2 versions the repr of floating point number was longer (and more precise) that the str which was limited for readability. This was changed several times and in Python 3 the repr and the str are equal and contain as much digits as needed for the closest decimal approximation. 2. int. In Python 2 the repr of long integer was different from the str: it contained the "L" suffix. 3. bool. Since it is an int subclass with different repr, it needs to define __str__ to override int's __str__. 4. subprocess.Handle and sre_constants._NamedIntConstant. As bool they need to override int's __str__. 5. doctest.DocTestCase, doctest.DocFileCase. They need to override unittest.TestCase's __str__. 6. http.client.IncompleteRead, xmlrpc.client.Error. They need to override BaseException's __str__ (don't know why they want to do this). 7. asyncore.dispatcher, email.charset.Charset, logging.LogRecord, xmlrpc.client.MultiCall, xmlrpc.client.ServerProxy, decimal.Context (C implementation only), _pydecimal._WorkRep. There is no need to define __str__. In most of these cases the __str__ definition is redundant and can be removed. In cases 5 and 6 it is better to reset __str__ to object.__str__. The only failing tests in the Python testsuite is the json module test. The json module calls int.__str__ explicitly. It can be fixed by replacing it with int.__repr__. The user visible effect of these changes is that defining __repr__ in a subclass of these classes will affect the str() result (as for most of other classes). Although it is unlikely that you want to keep the str representation of the parent class when change the repr in the child class. ---------- components: Interpreter Core messages: 341382 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Do not set tp_str versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 4 10:20:06 2019 From: report at bugs.python.org (Mikhail Gerasimov) Date: Sat, 04 May 2019 14:20:06 +0000 Subject: [New-bugs-announce] [issue36794] asyncio.Lock documentation in Py3.8 lacks parts presented in documentation in Py3.6 Message-ID: <1556979606.65.0.142990517106.issue36794@roundup.psfhosted.org> New submission from Mikhail Gerasimov : Compare: https://docs.python.org/3.6/library/asyncio-sync.html#asyncio.Lock https://docs.python.org/3.8/library/asyncio-sync.html#asyncio.Lock First version is much more detailed. It allows to avoid confusions like one with unlocking order: https://stackoverflow.com/q/55951233/1113207 ---------- assignee: docs at python components: Documentation, asyncio messages: 341385 nosy: asvetlov, docs at python, germn, yselivanov priority: normal severity: normal status: open title: asyncio.Lock documentation in Py3.8 lacks parts presented in documentation in Py3.6 type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 4 13:48:38 2019 From: report at bugs.python.org (Antoine Pitrou) Date: Sat, 04 May 2019 17:48:38 +0000 Subject: [New-bugs-announce] [issue36795] "make venv" failed in Docs Message-ID: <1556992118.26.0.759759719454.issue36795@roundup.psfhosted.org> New submission from Antoine Pitrou : $ make venv python3 -m venv ./venv ./venv/bin/python3 -m pip install -U Sphinx blurb python-docs-theme /home/antoine/cpython/default/Doc/venv/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.22) or chardet (2.3.0) doesn't match a supported version! Collecting Sphinx Cache entry deserialization failed, entry ignored Exception: Traceback (most recent call last): File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run wb.build(autobuilding=True) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/wheel.py", line 749, in build self.requirement_set.prepare_files(self.finder) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files ignore_dependencies=self.ignore_dependencies)) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file require_hashes File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link self.link = finder.find_requirement(self, upgrade) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement all_candidates = self.find_all_candidates(req.name) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates for page in self._get_pages(url_locations, project_name): File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages page = self._get_page(location) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page return HTMLPage.get_page(link, session=self.session) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/index.py", line 811, in get_page inst = cls(resp.content, resp.url, resp.headers) File "/home/antoine/cpython/default/Doc/venv/lib/python3.6/site-packages/pip/index.py", line 731, in __init__ namespaceHTMLElements=False, TypeError: parse() got an unexpected keyword argument 'transport_encoding' Makefile:126: recipe for target 'venv' failed make: *** [venv] Error 2 ---------- assignee: docs at python components: Documentation messages: 341395 nosy: brett.cannon, docs at python, ned.deily, pitrou priority: critical severity: normal stage: needs patch status: open title: "make venv" failed in Docs type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 4 15:31:40 2019 From: report at bugs.python.org (Zackery Spytz) Date: Sat, 04 May 2019 19:31:40 +0000 Subject: [New-bugs-announce] [issue36796] Error handling cleanup in _testcapimodule.c Message-ID: <1556998300.08.0.689848500132.issue36796@roundup.psfhosted.org> New submission from Zackery Spytz : Many functions in _testcapimodule.c lack error handling. This can cause spurious errors when stress testing the interpreter. ---------- components: Extension Modules messages: 341399 nosy: ZackerySpytz priority: normal severity: normal status: open title: Error handling cleanup in _testcapimodule.c versions: Python 2.7, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 4 17:05:47 2019 From: report at bugs.python.org (Nick Coghlan) Date: Sat, 04 May 2019 21:05:47 +0000 Subject: [New-bugs-announce] [issue36797] Cull more oudated distutils information Message-ID: <1557003947.84.0.478951846534.issue36797@roundup.psfhosted.org> New submission from Nick Coghlan : Prompted by #33071, I'm going to do a pass through the legacy distutils documentation deleting outdated information that is better maintained elsewhere. ---------- assignee: ncoghlan messages: 341407 nosy: jaraco, ncoghlan priority: normal severity: normal stage: needs patch status: open title: Cull more oudated distutils information type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 4 21:23:34 2019 From: report at bugs.python.org (Eric V. Smith) Date: Sun, 05 May 2019 01:23:34 +0000 Subject: [New-bugs-announce] [issue36798] := breaks f-strings Message-ID: <1557019414.93.0.89677106735.issue36798@roundup.psfhosted.org> New submission from Eric V. Smith : The walrus operator breaks f-strings, because the f-string scanner sees the colon as the end of the expression. >>> x = '10' >>> f'{x:=10}' Traceback (most recent call last): File "", line 1, in ValueError: '=' alignment not allowed in string format specifier This becomes: format(x, '=10'), which is an error if x is a string. ---------- assignee: eric.smith components: Interpreter Core messages: 341413 nosy: eric.smith, larry, lukasz.langa priority: release blocker severity: normal status: open title: := breaks f-strings versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 02:09:02 2019 From: report at bugs.python.org (Reuben Thomas) Date: Sun, 05 May 2019 06:09:02 +0000 Subject: [New-bugs-announce] [issue36799] Typo in ctypes documentation Message-ID: <1557036542.96.0.672219946485.issue36799@roundup.psfhosted.org> New submission from Reuben Thomas : "It is possible to defined" ? "It is possible to define" ---------- assignee: docs at python components: Documentation messages: 341419 nosy: docs at python, rrt priority: normal severity: normal status: open title: Typo in ctypes documentation versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 02:32:07 2019 From: report at bugs.python.org (Yuval Greenfield) Date: Sun, 05 May 2019 06:32:07 +0000 Subject: [New-bugs-announce] [issue36800] Invalid coding error hidden on Windows Message-ID: <1557037927.09.0.679611511238.issue36800@roundup.psfhosted.org> New submission from Yuval Greenfield : These lines fail on Windows in a surprising way: # -*- coding: utf8 -*- import threading print("threading %s" % threading) Normally they would throw this: ```File "C:\Python36\Lib\site-packages\missinglink_kernel\callback\log_monitor.py", line 1 SyntaxError: encoding problem: utf8``` But attached is a file that throws this instead: ``` Traceback (most recent call last): File "C:\Python36\Lib\site-packages\missinglink_kernel\callback\log_monitor.py", line 2, in import threading NameError: name 'threading' is not defined ``` It seems that the amount of lines in the file will cause the exception hiding bug to manifest. This issue did reproduce on my Windows machine here: * Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32 * Python 3.7.1 (default, Dec 10 2018, 22:54:23) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32 * Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)] on win32 It did NOT reproduce on my Mac at all: It does not repro on my mac at all: * Python 3.7.1 (default, Dec 14 2018, 13:28:58) [Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin * Python 3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37) [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin ---------- components: Windows files: log_monitor.py messages: 341420 nosy: paul.moore, steve.dower, tim.golden, ubershmekel, zach.ware priority: normal severity: normal status: open title: Invalid coding error hidden on Windows type: compile error versions: Python 3.7 Added file: https://bugs.python.org/file48300/log_monitor.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 09:44:32 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Sun, 05 May 2019 13:44:32 +0000 Subject: [New-bugs-announce] [issue36801] Wait for connection_lost in StreamWriter.drain Message-ID: <1557063872.25.0.865813796999.issue36801@roundup.psfhosted.org> New submission from Andrew Svetlov : Now `await writer.drain()` performs `await sleep(0)` if underlying transport is closing. It works well only for plain sockets. SSL transport needs more context switches to shut down the SSL connection. Drain wakes up too early to check if the transport is actually closed. ---------- components: asyncio messages: 341445 nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Wait for connection_lost in StreamWriter.drain type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 10:10:52 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Sun, 05 May 2019 14:10:52 +0000 Subject: [New-bugs-announce] [issue36802] Revert back StreamWriter awrite/aclose but provide await writer.write() and await writer.close() Message-ID: <1557065452.04.0.483828473478.issue36802@roundup.psfhosted.org> New submission from Andrew Svetlov : Yuri and I decided that `writer.awrite()` and `writer.aclose()` look ugly. Instead, we return awaitable object from these methods to allow both `writer.write(b'data')` and `await writer.write(b'data')` for the method. The same for `writer.close()` call. ---------- components: asyncio messages: 341450 nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Revert back StreamWriter awrite/aclose but provide await writer.write() and await writer.close() versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 11:33:43 2019 From: report at bugs.python.org (dibya ranjan mishra) Date: Sun, 05 May 2019 15:33:43 +0000 Subject: [New-bugs-announce] [issue36803] Getting a lot of runtime misaligned address error while building python 2.7.6 with UBSAN Message-ID: <1557070423.41.0.167231888065.issue36803@roundup.psfhosted.org> New submission from dibya ranjan mishra : I downloaded python 2.7.6 source and am trying to build with UBSAN flag following the instructions https://devguide.python.org/clang/. But I am getting a lot of misaligned address errors when I run make or make test command. Below is an example: Objects/listobject.c:161:5: runtime error: member access within misaligned address 0x2adff0773228 for type 'struct (anonymous struct at Include/objimpl.h:253:5)', which requires 16 byte alignment I need this ubsan enabled python 2.7 as a part of my project. ---------- components: Build messages: 341453 nosy: dibya ranjan mishra priority: normal severity: normal status: open title: Getting a lot of runtime misaligned address error while building python 2.7.6 with UBSAN type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 14:06:32 2019 From: report at bugs.python.org (Mark Green) Date: Sun, 05 May 2019 18:06:32 +0000 Subject: [New-bugs-announce] [issue36804] Pythonr fo Message-ID: <1557079592.59.0.449925637296.issue36804@roundup.psfhosted.org> Change by Mark Green : ---------- nosy: Mark Green priority: normal severity: normal status: open title: Pythonr fo _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 14:30:29 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Sun, 05 May 2019 18:30:29 +0000 Subject: [New-bugs-announce] [issue36805] Don't close subprocess stream if it's stdin is closed Message-ID: <1557081029.41.0.726970758357.issue36805@roundup.psfhosted.org> New submission from Andrew Svetlov : Closing stdin FD by child stream is a legal operation, there is no reason to close entire subprocess transport immediately. ---------- components: asyncio messages: 341467 nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Don't close subprocess stream if it's stdin is closed type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 15:15:37 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Sun, 05 May 2019 19:15:37 +0000 Subject: [New-bugs-announce] [issue36806] Forbid creating of stream objects outside of asyncio Message-ID: <1557083737.76.0.720112204128.issue36806@roundup.psfhosted.org> New submission from Andrew Svetlov : They were intended to be used by asyncio factories like open_connection from the very beginning but internals was leaked into asyncio top-level namespace. The idea is: 1. provide `_asyncio_internal` keyword-only parameter to leaked classes constructor 2. have it False by default 3. Use `_asyncio_internal=True` when called from asyncio code. 3. Raise DeprecationWarning if `_asyncio_internal` is False ---------- components: asyncio messages: 341472 nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Forbid creating of stream objects outside of asyncio versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 16:22:26 2019 From: report at bugs.python.org (Guido van Rossum) Date: Sun, 05 May 2019 20:22:26 +0000 Subject: [New-bugs-announce] [issue36807] IDLE doesn't call os.fsync() Message-ID: <1557087746.35.0.383866352803.issue36807@roundup.psfhosted.org> New submission from Guido van Rossum : This came up during today's final PyCon keynote -- IDLE was called out as one of the two editors *not* to use when live-coding on Adafruit's Circuit Playground Express (https://www.adafruit.com/product/3333). I *think* that the problem referred to is that IDLE doesn't guarantee that the bits are actually flushed to disk -- they may linger in the OS filesystem cache. ---------- assignee: terry.reedy components: IDLE messages: 341474 nosy: gvanrossum, terry.reedy priority: normal severity: normal stage: patch review status: open title: IDLE doesn't call os.fsync() type: behavior versions: Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 19:35:09 2019 From: report at bugs.python.org (Arthur Goldberg) Date: Sun, 05 May 2019 23:35:09 +0000 Subject: [New-bugs-announce] [issue36808] Understanding "cannot import name" exception Message-ID: <1557099309.88.0.0800567814981.issue36808@roundup.psfhosted.org> New submission from Arthur Goldberg : I'm attempting to better understand an ImportError: cannot import name '' error by reading the cpython source. But I cannot find this error in the source. The closest I find is in cpython/Python/ceval.c, lines 5060 & 5068, but they both say: cannot import name %R from %R (*) and my message doesn't include the 'from ...'. I'm using: Python 3.6.5 (default, Sep 14 2018, 14:56:31) [GCC 7.3.0] on linux Thanks, Arthur ---------- components: Interpreter Core messages: 341478 nosy: ArthurGoldberg priority: normal severity: normal status: open title: Understanding "cannot import name" exception type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 5 23:27:40 2019 From: report at bugs.python.org (Furzoom) Date: Mon, 06 May 2019 03:27:40 +0000 Subject: [New-bugs-announce] [issue36809] Crash for test test_importlib Message-ID: <1557113260.01.0.872364127661.issue36809@roundup.psfhosted.org> New submission from Furzoom : 0:06:04 load avg: 0.73 [185/416] test_importlib Fatal Python error: Segmentation fault Current thread 0x00002b774411ee40 (most recent call first): File "/home/mn/Downloads/Python-3.7.3/Lib/test/support/__init__.py", line 1543 in gc_collect File "/home/mn/Downloads/Python-3.7.3/Lib/test/test_importlib/test_locks.py", line 132 in test_all_locks File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/case.py", line 615 in run File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/case.py", line 663 in __call__ File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 122 in run File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 84 in __call__ File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 122 in run File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 84 in __call__ File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 122 in run File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 84 in __call__ File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 122 in run File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 84 in __call__ File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 122 in run File "/home/mn/Downloads/Python-3.7.3/Lib/unittest/suite.py", line 84 in __call__ File "/home/mn/Downloads/Python-3.7.3/Lib/test/support/testresult.py", line 162 in run File "/home/mn/Downloads/Python-3.7.3/Lib/test/support/__init__.py", line 1899 in _run_suite File "/home/mn/Downloads/Python-3.7.3/Lib/test/support/__init__.py", line 1995 in run_unittest File "/home/mn/Downloads/Python-3.7.3/Lib/test/libregrtest/runtest.py", line 178 in test_runner File "/home/mn/Downloads/Python-3.7.3/Lib/test/libregrtest/runtest.py", line 182 in runtest_inner File "/home/mn/Downloads/Python-3.7.3/Lib/test/libregrtest/runtest.py", line 137 in runtest File "/home/mn/Downloads/Python-3.7.3/Lib/test/libregrtest/main.py", line 408 in run_tests_sequential File "/home/mn/Downloads/Python-3.7.3/Lib/test/libregrtest/main.py", line 515 in run_tests File "/home/mn/Downloads/Python-3.7.3/Lib/test/libregrtest/main.py", line 619 in _main File "/home/mn/Downloads/Python-3.7.3/Lib/test/libregrtest/main.py", line 586 in main File "/home/mn/Downloads/Python-3.7.3/Lib/test/libregrtest/main.py", line 640 in main File "/home/mn/Downloads/Python-3.7.3/Lib/test/regrtest.py", line 46 in _main File "/home/mn/Downloads/Python-3.7.3/Lib/test/regrtest.py", line 50 in File "/home/mn/Downloads/Python-3.7.3/Lib/runpy.py", line 85 in _run_code File "/home/mn/Downloads/Python-3.7.3/Lib/runpy.py", line 193 in _run_module_as_main ---------- components: Tests files: import_test.log messages: 341481 nosy: furzoom priority: normal severity: normal status: open title: Crash for test test_importlib type: crash versions: Python 3.7 Added file: https://bugs.python.org/file48301/import_test.log _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 03:34:51 2019 From: report at bugs.python.org (Lovasoa) Date: Mon, 06 May 2019 07:34:51 +0000 Subject: [New-bugs-announce] [issue36810] Recursive type annotations do not work in documentation tests Message-ID: <1557128091.8.0.92390495342.issue36810@roundup.psfhosted.org> New submission from Lovasoa : When using documentations tests to test a function that uses typing.get_type_hints, an error is thrown. The error is thrown when trying to get the type hints for a class that was defined in a documentation test. I was able to reproduce it both on current and PEP563 type annotations: https://gist.github.com/lovasoa/74ea62a89f5bf073b0e0c2f222008ae3 ---------- components: Interpreter Core files: test_recursive.py messages: 341489 nosy: lovasoa priority: normal severity: normal status: open title: Recursive type annotations do not work in documentation tests type: behavior versions: Python 3.7 Added file: https://bugs.python.org/file48302/test_recursive.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 10:27:22 2019 From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=) Date: Mon, 06 May 2019 14:27:22 +0000 Subject: [New-bugs-announce] [issue36811] Warning when compiling _elementree.c Message-ID: <1557152842.63.0.0923910614914.issue36811@roundup.psfhosted.org> New submission from St?phane Wirtel : Hi Stefan, When I compile the last master, I get this warning. Could do you try? Thanks /home/stephane/src/github.com/python/cpython/Include -I/home/stephane/src/github.com/python/cpython -c /home/stephane/src/github.com/python/cpython/Modules/_elementtree.c -o build/temp.linux-x86_64-3.8-pydebug/home/stephane/src/github.com/python/cpython/Modules/_elementtree.o /home/stephane/src/github.com/python/cpython/Modules/_elementtree.c: In function ?checkpath?: /home/stephane/src/github.com/python/cpython/Modules/_elementtree.c:1174:44: warning: suggest parentheses around ?&&? within ?||? [-Wparentheses] p[1] == '}' || p[1] == '*' && p[2] == '}')) { ~~~~~~~~~~~~^~~~~~~~~~~~~~ ---------- assignee: scoder components: Build messages: 341507 nosy: matrixise, scoder priority: normal severity: normal status: open title: Warning when compiling _elementree.c versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 11:03:31 2019 From: report at bugs.python.org (Matthew Tanous) Date: Mon, 06 May 2019 15:03:31 +0000 Subject: [New-bugs-announce] [issue36812] posix_spawnp returns error when used with file_actions Message-ID: <1557155011.55.0.26778375432.issue36812@roundup.psfhosted.org> New submission from Matthew Tanous : Ran into this on macOS while trying to play around with the new posix_spawn bindings. It appears to me that the file_actions path is not what is being used by file_actions here. It may be that I am misunderstanding something, but I thought I would bring it up. Python 3.8.0a3 (v3.8.0a3:9a448855b5, Mar 25 2019, 17:05:20) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> file_actions = [(os.POSIX_SPAWN_OPEN, 1, '.tmp/temp_file', os.O_CREAT | os.O_RDWR, 777)] >>> os.posix_spawnp('whoami', ['whoami'], file_actions=file_actions) Traceback (most recent call last): File "", line 1, in TypeError: posix_spawnp() takes exactly 3 positional arguments (2 given) >>> os.posix_spawnp('whoami', ['whoami'], os.environ, file_actions=file_actions) Traceback (most recent call last): File "", line 1, in PermissionError: [Errno 13] Permission denied: 'whoami' ---------- components: Library (Lib) messages: 341518 nosy: Matthew Tanous priority: normal severity: normal status: open title: posix_spawnp returns error when used with file_actions type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 11:03:59 2019 From: report at bugs.python.org (Bar Harel) Date: Mon, 06 May 2019 15:03:59 +0000 Subject: [New-bugs-announce] [issue36813] QueueListener not calling task_done upon termination Message-ID: <1557155039.79.0.434860875914.issue36813@roundup.psfhosted.org> New submission from Bar Harel : QueueListener does not call task_done upon termination, causing an unsuspecting thread to deadlock. Steps to reproduce: >>> import queue >>> q = queue.Queue() >>> from logging.handlers import QueueListener >>> h = QueueListener(q) >>> h.start() >>> h.stop() # Goodbye cruel world! >>> q.join() Fixing and uploading a patch as we speak. ---------- components: Library (Lib) messages: 341519 nosy: bar.harel priority: normal severity: normal status: open title: QueueListener not calling task_done upon termination type: behavior versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 11:05:56 2019 From: report at bugs.python.org (Matthew Tanous) Date: Mon, 06 May 2019 15:05:56 +0000 Subject: [New-bugs-announce] [issue36814] posix_spawn explicit file_actions=None throws error Message-ID: <1557155156.27.0.0390065046289.issue36814@roundup.psfhosted.org> New submission from Matthew Tanous : Allowing posix_spawn file_actions to default to None works, but explicitly setting it throws a TypeError: Python 3.8.0a3 (v3.8.0a3:9a448855b5, Mar 25 2019, 17:05:20) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> os.posix_spawnp('whoami', ['whoami'], os.environ, file_actions=None) Traceback (most recent call last): File "", line 1, in TypeError: file_actions must be a sequence or None This required me to default to an empty sequence to complete it: >>> os.posix_spawnp('whoami', ['whoami'], os.environ, file_actions=[]) 6308 ---------- components: Library (Lib) messages: 341522 nosy: Matthew Tanous priority: normal severity: normal status: open title: posix_spawn explicit file_actions=None throws error versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 11:43:06 2019 From: report at bugs.python.org (Pam McA'Nulty) Date: Mon, 06 May 2019 15:43:06 +0000 Subject: [New-bugs-announce] [issue36815] test_socket test_host_resolution_bad_address test failure Message-ID: <1557157386.93.0.718979150708.issue36815@roundup.psfhosted.org> New submission from Pam McA'Nulty : - Cloned a new cpython onto my new MacOS system (Mojave 10.14.4) - built as per https://devguide.python.org/ - test failure: ``` ./python.exe -m test -j3 test_socket Run tests in parallel using 3 child processes 0:00:24 load avg: 0.88 [1/1/1] test_socket failed /Users/pamelamcanulty/cpython/Lib/test/test_socket.py:2424: RuntimeWarning: received malformed or improperly-truncated ancillary data result = sock.recvmsg(bufsize, *args) /Users/pamelamcanulty/cpython/Lib/test/test_socket.py:2515: RuntimeWarning: received malformed or improperly-truncated ancillary data result = sock.recvmsg_into([buf], *args) test test_socket failed -- Traceback (most recent call last): File "/Users/pamelamcanulty/cpython/Lib/test/test_socket.py", line 936, in test_host_resolution_bad_address socket.gethostbyname(addr) AssertionError: OSError not raised : ::1q == Tests result: FAILURE == 1 test failed: test_socket Total duration: 24 sec 153 ms Tests result: FAILURE ``` ---------- components: macOS messages: 341542 nosy: Pam.McANulty, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: test_socket test_host_resolution_bad_address test failure type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 14:05:40 2019 From: report at bugs.python.org (Gregory P. Smith) Date: Mon, 06 May 2019 18:05:40 +0000 Subject: [New-bugs-announce] [issue36816] self-signed.pythontest.net TLS certificate key is too weak Message-ID: <1557165940.63.0.800219956591.issue36816@roundup.psfhosted.org> New submission from Gregory P. Smith : test_httplib uses self-signed.pythontest.net in it's test_networked_good_cert test. On modern Linux distros (current Debian testing sid), the certificate it currently uses is rightfully rejected as being too weak: ERROR: test_networked_good_cert (test.test_httplib.HTTPSTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/greg/oss/cpython/Lib/test/test_httplib.py", line 1628, in test_networked_good_cert h.request('GET', '/') File "/home/greg/oss/cpython/Lib/http/client.py", line 1221, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/greg/oss/cpython/Lib/http/client.py", line 1267, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/greg/oss/cpython/Lib/http/client.py", line 1216, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/greg/oss/cpython/Lib/http/client.py", line 1004, in _send_output self.send(msg) File "/home/greg/oss/cpython/Lib/http/client.py", line 944, in send self.connect() File "/home/greg/oss/cpython/Lib/http/client.py", line 1383, in connect self.sock = self._context.wrap_socket(self.sock, File "/home/greg/oss/cpython/Lib/ssl.py", line 405, in wrap_socket return self.sslsocket_class._create( File "/home/greg/oss/cpython/Lib/ssl.py", line 853, in _create self.do_handshake() File "/home/greg/oss/cpython/Lib/ssl.py", line 1117, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1055) The TLS certificate on the server needs to be updated to something modern. I _believe_ this can be done by someone with infrastructure access via an update to https://github.com/python/pythontestdotnet/tree/master/tls Assigning to EWDurbin for triage and redirection to someone else infrastructury if he's not the right person. How to know if it has been fixed? Monitor the test_networked_good_cert test on any "Debian buster" builtbot(s) such as https://buildbot.python.org/all/#/workers/23 to make sure it is not skipped. (the test _currently_ fails, I am going to have it be _skipped_ on this specific key too small error for the time being to get that stable buildbot green again) ---------- assignee: EWDurbin components: SSL, Tests messages: 341579 nosy: EWDurbin, gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: self-signed.pythontest.net TLS certificate key is too weak type: behavior versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 14:10:56 2019 From: report at bugs.python.org (Eric V. Smith) Date: Mon, 06 May 2019 18:10:56 +0000 Subject: [New-bugs-announce] [issue36817] Add = to f-strings for easier debugging. Message-ID: <1557166256.56.0.583302074071.issue36817@roundup.psfhosted.org> New submission from Eric V. Smith : This is an alternative proposal to issue36774. We (Eric V. Smith and Larry Hastings) propose a minor language change. This level of change doesn't require a PEP, so in this post-BDFL world what we need is "a consensus among core developers". So please vote! Note that "+" is typed using "shift-=", and the "1" key can be found very nearby. Python programmers often use "printf-style" debugging. In the (really) bad old days this was pretty wordy: print "foo=", foo, "bar=", bar f-strings make this slightly nicer to type: print(f"foo={foo} bar={bar}") But you still have to repeat yourself: you have to write out the *string* "foo", and then the *expession* "foo". Wouldn't it be nice if you didn't have to? f-strings are uniquely able to help with this. Their implementation requires them to know the original text of the expression which they then compile. It's not difficult for f-strings to retain the text and prepend it; the tricky part is figuring out how to spell it. The initial idea was to use an f-string "conversion", which we originally spelled "!=": f'{foo!=}' This spelling won't work, because f-strings permit arbitrary Python expressions, and != of course tests for inequality. We considered other spellings: !d (debug) !e (equals) !x (?) !! (easy to type) We'd planned to go with !d. In fact Eric gave a lightning talk about this on Friday night and used this spelling. And then! On Saturday, the best spelling revealed itself! Behold the majesty of: {foo=} This code: foo=5 print(f"{foo=}") would print foo=5 With this spelling change, we've also refined the semantics. By default, f-strings use format() (technically they call __format__ on the value). But the point of this is for debugging. But you want repr() for debugging. When you use this on a string, you want to see the quoted string; when you use this on a datetime object, you want to see the datetime repr, not the default formatted string. Second, this is now composable with conversions. So you can use {foo=!s} to use str() instead of repr() on the value. Relatedly, we've added a new conversion: "!f" means "use format()", which you could never explicitly specify before. For example, to only format pi to two decimal places: f"{math.pi=!f:.2f}" => "3.14" Finally, and this is the best part: what if you want whitespace around the equals sign? Well, to the left is no problem; whitespace is preserved from the original text inside the curly braces: f"{ chr(65) =}" => " chr(65) ='A'" But we also explicitly permit, and preserve, whitespace *after* the equals sign: f"{chr(65) = }" => "chr(65) = 'A'" What's particularly elegant is that we simply preserve all the characters up to the final delimiter. The equals sign sets a flag but doesn't stop flushing through text. So this: vvvvvvvvvv f"{chr(65) = }" is *exactly* the same as this: vvvvvvvvvv "chr(65) = 'A'" Please vote! Eric and /arry ---------- assignee: eric.smith components: Interpreter Core messages: 341582 nosy: eric.smith, larry priority: normal severity: normal stage: patch review status: open title: Add = to f-strings for easier debugging. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 14:45:44 2019 From: report at bugs.python.org (Eric Snow) Date: Mon, 06 May 2019 18:45:44 +0000 Subject: [New-bugs-announce] [issue36818] Add PyInterpreterState.runtime. Message-ID: <1557168344.92.0.643564511932.issue36818@roundup.psfhosted.org> New submission from Eric Snow : Currently we use the _PyRuntime static global to access the runtime state in various places. At the same time, in thread contexts we get access to the thread state from Thread-Local Storage and the interpreter state by indirection from there. We should do the same for the runtime state instead of using the global directly. My plan is to add a PyInterpreterState.runtime field. It can then be used in the same way we use PyThreadState.interp to access the interpreter state. ---------- assignee: eric.snow components: Interpreter Core messages: 341595 nosy: eric.snow, vstinner priority: normal severity: normal stage: patch review status: open title: Add PyInterpreterState.runtime. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 14:51:21 2019 From: report at bugs.python.org (Andrei Talaba) Date: Mon, 06 May 2019 18:51:21 +0000 Subject: [New-bugs-announce] [issue36819] Crash during encoding using UTF-16/32 and custom error handler Message-ID: <1557168681.45.0.232948256283.issue36819@roundup.psfhosted.org> New submission from Andrei Talaba : The CPython interpreter write out-of-bounds of allocated memory in certain edge cases in the utf-16 and utf-32 encoders. The attached script registers two error handlers that either write one ascii character, or two bytes, and tells the encoder to start again from the start of the encoding error. The script then tries to encode an invalid codepoint in either utf-16 or utf-32. Each of the calls to encode independently cause segfaults Since the encoder starts over again and keeps trying to append the result of the error handler, the lack of proper re-allocations leads to a buffer overflow, and corrupts the stack. ---------- components: Interpreter Core files: encode_crash.py messages: 341599 nosy: atalaba, serhiy.storchaka, vstinner priority: normal severity: normal status: open title: Crash during encoding using UTF-16/32 and custom error handler type: crash versions: Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48304/encode_crash.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 15:32:15 2019 From: report at bugs.python.org (Mario Corchero) Date: Mon, 06 May 2019 19:32:15 +0000 Subject: [New-bugs-announce] [issue36820] Captured exceptions are keeping user objects alive unnecessarily in the stdlib Message-ID: <1557171135.51.0.216377454002.issue36820@roundup.psfhosted.org> New submission from Mario Corchero : There are multiple places in the standard library where the code captures an exception and reraises it later (outside of the original except). This is known to cause a cycle as saving the exception has a traceback that eventually points back to the exception, but it is especially problematic as it keeps alive the objects that the user has as well. This can be reproduced with the following example: ``` import weakref class A: pass ref = None def x(): global ref cool_var = A() ref = weakref.ref(cool_var) assert ref() try: 1/0 except Exception as e: ee = e try: x() except Exception: pass print(ref()) assert ref() is None ``` There are places in the standard library that try to get around this. See https://github.com/python/cpython/commit/acb9fa79fa6453c2bbe3ccfc9cad2837feb90093 as an example. This change did not include the exception path though, when the `err` variable is raised, the issue is still present. This issue is to fix the instances found in socket.py, codeop.py and dyld.py. By either setting the variable to None to remove the name or if it needs to be reraised by using a finally to delete it. This is basically the same that a try except would do, which automagically adds a finally to delete the local name used in the `except X as y`. There was a discussion with twouters,pablogsal,barry about potentially adding something extra to the exception handling to try to clean this up but it was suggested the best approach is for the user to handle it and maybe a linter to potentially flag it, as there might also be multiple references to the exception being captured. It was also proposed to just change the situations where this happens in the standard library to remove the name if possible. Note that eventually those objects will just be collected by the gc, it is just an improvement. I'm preparing a PR for those three modules. ---------- components: Library (Lib) messages: 341620 nosy: mariocj89, pablogsal, twouters priority: low severity: normal status: open title: Captured exceptions are keeping user objects alive unnecessarily in the stdlib type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 18:47:32 2019 From: report at bugs.python.org (anthony shaw) Date: Mon, 06 May 2019 22:47:32 +0000 Subject: [New-bugs-announce] [issue36821] Termios module largely untested Message-ID: <1557182852.99.0.825312332022.issue36821@roundup.psfhosted.org> New submission from anthony shaw : I noticed that the termios.c module is largely untested. There is some coverage via test_pty, test_ioctl and test_getpass, but there is nothing to cover regression and the behaviours in the module functions. Tests are required for: - termios.tcgetattr() - termios.tcsetattr() - termios.tcsendbreak() - termios.tcdrain() - termios.tcflush() - termios.tcflow() ---------- assignee: anthonypjshaw components: Tests messages: 341656 nosy: anthonypjshaw priority: normal severity: normal status: open title: Termios module largely untested type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 19:48:23 2019 From: report at bugs.python.org (Sanyam Khurana) Date: Mon, 06 May 2019 23:48:23 +0000 Subject: [New-bugs-announce] [issue36822] Minor grammatical fix in glossary.rst Message-ID: <1557186503.72.0.659127462032.issue36822@roundup.psfhosted.org> New submission from Sanyam Khurana : While working on translations, I saw a minor grammatical error in `Doc/glossary.rst`: ``` * The default Python prompt of the interactive shell when entering code for an indented code block ... ``` should be ``` * The default Python prompt of the interactive shell when entering the code for an indented code block ... ``` Attaching the path with this. ---------- assignee: docs at python components: Documentation messages: 341665 nosy: CuriousLearner, docs at python priority: normal severity: normal stage: needs patch status: open title: Minor grammatical fix in glossary.rst type: enhancement versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 6 23:47:46 2019 From: report at bugs.python.org (Ryan Avery) Date: Tue, 07 May 2019 03:47:46 +0000 Subject: [New-bugs-announce] [issue36823] shutil.copytree copies directories and files but fails with that same directory with '[Errno 1] Operation not permitted') Message-ID: <1557200866.03.0.146601930994.issue36823@roundup.psfhosted.org> New submission from Ryan Avery : I am trying to use shutil.copytree on an Azure VM that has Azure FileStorage mounted with SMB 3.0. When I run the following to copy directories from one location on my Azure File Storage to another location on my File Storage, the whole directory, subdirectory, and files are copied succesfully but then this errors on those very same files: shutil.copytree( os.path.join(wflow.TRAIN, 'tile_512-4608'), os.path.join(wflow.TEST, 'tile_512-4608') ) Error: [('/mnt/point/landsat-1024-cp/train/tile_512-4608/image', '/mnt/point/landsat-1024-cp/test/tile_512-4608/image', '[Errno 1] Operation not permitted'), ('/mnt/point/landsat-1024-cp/train/tile_512-4608/mask', '/mnt/point/landsat-1024-cp/test/tile_512-4608/mask', '[Errno 1] Operation not permitted'), ('/mnt/point/landsat-1024-cp/train/tile_512-4608', '/mnt/point/landsat-1024-cp/test/tile_512-4608', '[Errno 1] Operation not permitted')] After this error, all of the items listed above were actually copied so I'm not sure why this errors ---------- components: Library (Lib) messages: 341677 nosy: rbavery priority: normal severity: normal status: open title: shutil.copytree copies directories and files but fails with that same directory with '[Errno 1] Operation not permitted') type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 02:37:53 2019 From: report at bugs.python.org (Daniel Fortunov) Date: Tue, 07 May 2019 06:37:53 +0000 Subject: [New-bugs-announce] [issue36824] Refactor str tests to reflect that str and unicode are merged in Python 3 Message-ID: <1557211073.23.0.0911127825046.issue36824@roundup.psfhosted.org> New submission from Daniel Fortunov : Unit tests of `str` and related types (e.g. `UserString`) contain nomenclature and structure that dates back to the Python 2 distinction between `str` and `unicode`. Previously it was undesirable to disturb the structure of these tests too much as this would complicate the merging of bug fixes back to Python 2, however with Python 2 drawing near to end-of-life this is perhaps less of a concern. I would propose the following changes as a start: - Rename test_unicode.py to test_str.py, and `UnicodeTest` class to `StrTest` (to reflect the type that is now being tested) - Remove `MixinStrUnicodeUserStringTest` class and move its tests into `CommonTest` (The comment for this class says # additional tests that only work for # stringlike objects, i.e. str, UserString but in the absence of `unicode` the `CommonTest` class is also only used for these two types now.) - Promote tests from the current `UnicodeTest` class to `CommonTest` where it makes sense to do so; remove checks that no longer make sense. (e.g. checks around mixed `str`/`unicode` arguments that are all just `str` now) Maybe the duplicative `checkequalnofix()` method can also go away now? - The `BadSeq1` helper class is not used because the corresponding check was commented out in 2007. Either reinstate the check, or remove this class. I'm happy to submit a PR for this, but just wanted to get some feedback before making the changes. ---------- components: Library (Lib) messages: 341685 nosy: dfortunov, ncoghlan, xtreak priority: normal severity: normal status: open title: Refactor str tests to reflect that str and unicode are merged in Python 3 type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 04:35:49 2019 From: report at bugs.python.org (=?utf-8?q?R=C3=A9mi_Lapeyre?=) Date: Tue, 07 May 2019 08:35:49 +0000 Subject: [New-bugs-announce] [issue36825] Make TestCase aware of the command line arguments given to TestProgram Message-ID: <1557218149.53.0.915842063513.issue36825@roundup.psfhosted.org> New submission from R?mi Lapeyre : Hi, to make unittest more extensible and for issue 18765 (having a way to run pdb when tests fail), I would like to make TestCase aware of the command line arguments given to the TestProgram so they can adapt their behavior based on them. I suggested this change on python-ideas (https://mail.python.org/pipermail/python-ideas/2019-March/055842.html) but it did not get much attention. I'm opening a PR in the hope to get more feedback and will start writing documentation. Please comment if this is not appropriate or regarding anything I might have not thought of. ---------- components: Tests messages: 341697 nosy: remi.lapeyre priority: normal severity: normal status: open title: Make TestCase aware of the command line arguments given to TestProgram type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 04:39:53 2019 From: report at bugs.python.org (Eric V. Smith) Date: Tue, 07 May 2019 08:39:53 +0000 Subject: [New-bugs-announce] [issue36826] ast_unparser.c doesn't handle := expressions Message-ID: <1557218393.13.0.153160088423.issue36826@roundup.psfhosted.org> New submission from Eric V. Smith : If either of these lines are added to test_annotations() in Lib/test/test_future.py, a SystemError will be raised: eq("(x:=10)") eq("f'{(x:=10):=10}'") Traceback (most recent call last): File "/Users/eric/local/python/cpython/Lib/test/test_future.py", line 258, in test_annotations eq("(x:=10)") File "/Users/eric/local/python/cpython/Lib/test/test_future.py", line 134, in assertAnnotationEqual actual = self.getActual(annotation) File "/Users/eric/local/python/cpython/Lib/test/test_future.py", line 121, in getActual exec(self.template.format(ann=annotation), {}, scope) SystemError: unknown expression kind ---------- components: Interpreter Core messages: 341699 nosy: emilyemorehouse, eric.smith, lukasz.langa priority: release blocker severity: normal stage: needs patch status: open title: ast_unparser.c doesn't handle := expressions type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 07:42:08 2019 From: report at bugs.python.org (Alexey Muranov) Date: Tue, 07 May 2019 11:42:08 +0000 Subject: [New-bugs-announce] [issue36827] Overriding __new__ method with itself changes behaviour of the class Message-ID: <1557229328.38.0.306180402683.issue36827@roundup.psfhosted.org> New submission from Alexey Muranov : I expect that overriding methods with themselves like in the following example should not change the behaviour of the class: class C: a = 1 def __init__(self, b): self.b = b def f(self, x): return x*self.a + self.b + 1 @classmethod def g(cls, y): return y*cls.a + 2 @staticmethod def h(z): return z + 3 class D(C): def f(self, *args, **kwargs): return super(__class__, self).f(*args, **kwargs) @classmethod def g(cls, *args, **kwargs): return super(__class__, cls).g(*args, **kwargs) @staticmethod def h(*args, **kwargs): return super(__class__, __class__).h(*args, **kwargs) c = C(7) d = D(7) assert c.f(42) == d.f(42) assert c.g(42) == d.g(42) assert c.h(42) == d.h(42) (Moreover, I expect to be able to extend superclass method this way.) However, this does not work with `__new__`: class C: def __init__(self, x): self.x = x print(x) class D(C): @staticmethod def __new__(*args, **kwargs): return super(__class__, __class__).__new__(*args, **kwargs) C(7) # fine D(7) # TypeError: object.__new__() takes exactly one argument I think this is not a desirable feature. I would call it a bug. By the way, I understand why `object`'s `__init__` can complain about a wrong number of arguments, but I do not see a point in making `object`'s `__new__` complain about it. Here is my current workaround: class T: @staticmethod def __new__(cls, *_args, **_kwargs): return object.__new__(cls) class E(C, T): @staticmethod def __new__(*args, **kwargs): return super(__class__, __class__).__new__(*args, **kwargs) C(42) # fine E(42) # fine A possibly related issue: #32768 (https://bugs.python.org/issue32768) ---------- components: Interpreter Core messages: 341705 nosy: alexey-muranov priority: normal severity: normal status: open title: Overriding __new__ method with itself changes behaviour of the class type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 08:09:11 2019 From: report at bugs.python.org (Hans Ginzel) Date: Tue, 07 May 2019 12:09:11 +0000 Subject: [New-bugs-announce] [issue36828] Cannot install et_xmlfile Message-ID: <1557230951.74.0.443191193294.issue36828@roundup.psfhosted.org> New submission from Hans Ginzel : See attached log file, please. How to reproduce: rem Download https://www.python.org/ftp/python/3.7.3/python-3.7.3-embed-amd64.zip mkdir C:\Tools\Python unzip python-3.7.3-embed-amd64.zip -d C:\Tools\Python\3.7.3 path C:\Tools\Python\3.7.3;C:\Tools\Python\3.7.3\Scripts;%PATH% echo Remove comment in "#import site" on the last line, please. notepad C:\Tools\Python\3.7.3\python37._pth rem Download https://bootstrap.pypa.io/get-pip.py python get-pip.py Collecting pip Downloading https://files.pythonhosted.org/packages/5c/e0/be401c003291b56efc55aeba6a80ab790d3d4cece2778288d65323009420/pip-19.1.1-py2.py3-none-any.whl (1.4MB) |????????????????????????????????| 1.4MB 656kB/s Collecting setuptools Downloading https://files.pythonhosted.org/packages/ec/51/f45cea425fd5cb0b0380f5b0f048ebc1da5b417e48d304838c02d6288a1e/setuptools-41.0.1-py2.py3-none-any.whl (575kB) |????????????????????????????????| 583kB 3.2MB/s Collecting wheel Downloading https://files.pythonhosted.org/packages/96/ba/a4702cbb6a3a485239fbe9525443446203f00771af9ac000fa3ef2788201/wheel-0.33.1-py2.py3-none-any.whl Installing collected packages: pip, setuptools, wheel Successfully installed pip-19.1.1 setuptools-41.0.1 wheel-0.33.1 C:\Users\Hans\Downloads>pip install --log et-xmlfile.log et-xmlfile Collecting et-xmlfile Downloading https://files.pythonhosted.org/packages/22/28/a99c42aea746e18382ad9fb36f64c1c1f04216f41797f2f0fa567da11388/et_xmlfile-1.0.1.tar.gz ERROR: Command "python setup.py egg_info" failed with error code 1 in C:\Users\Hans\AppData\Local\Temp\pip-install-qws9gvg0\et-xmlfile\ ---------- components: Installation files: et-xmlfile.log messages: 341707 nosy: hginzel priority: normal severity: normal status: open title: Cannot install et_xmlfile type: crash versions: Python 3.7 Added file: https://bugs.python.org/file48308/et-xmlfile.log _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 08:10:07 2019 From: report at bugs.python.org (Thomas Grainger) Date: Tue, 07 May 2019 12:10:07 +0000 Subject: [New-bugs-announce] [issue36829] CLI option to make PyErr_WriteUnraisable abortthe current process Message-ID: <1557231007.92.0.23824097772.issue36829@roundup.psfhosted.org> New submission from Thomas Grainger : Currently it's quite easy for these errors to go unnoticed. I'd like a way to easily detect these in CI. nedbat suggested piping the process output to another tool, and looking for 'Exception ignored in:' but this seems a little diff ---------- components: Interpreter Core messages: 341708 nosy: graingert priority: normal severity: normal status: open title: CLI option to make PyErr_WriteUnraisable abortthe current process type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 08:53:12 2019 From: report at bugs.python.org (keroru) Date: Tue, 07 May 2019 12:53:12 +0000 Subject: [New-bugs-announce] [issue36830] Typo in collections.deque Message-ID: <1557233592.36.0.483652950266.issue36830@roundup.psfhosted.org> New submission from keroru : https://docs.python.org/ja/3/library/collections.html#collections.deque has a typo in description of "rotate(n=1)" in Japanese. I suppose that "d.appendleft(d.popleft())" should be "d.append(d.popleft())". Thank you! ---------- assignee: docs at python components: Documentation messages: 341715 nosy: docs at python, keroru priority: normal severity: normal status: open title: Typo in collections.deque versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 09:19:59 2019 From: report at bugs.python.org (Steve Dower) Date: Tue, 07 May 2019 13:19:59 +0000 Subject: [New-bugs-announce] [issue36831] ElementTree.find attribute matching with empty namespace Message-ID: <1557235199.92.0.526138899784.issue36831@roundup.psfhosted.org> New submission from Steve Dower : I suspect this is related to the fix for issue30485, but haven't fully debugged it. In PC/layout/support/appxmanifest.py, I have code that looks up an existing element and adds it if missing: def find_or_add(xml, element, attr=None, always_add=False): if always_add: e = None else: q = element if attr: q += "[@{}='{}']".format(*attr) e = xml.find(q, APPXMANIFEST_NS) if e is None: ... return e For my 3.8.0a4 build, this started failing to match elements with attributes included. As a result, I ended up with N elements in a space where the schema only allows 1, and part of the 3.8.0a4 release is now blocked (okay, it's unblocked because I manually fixed a file and built the rest by hand, but it still held things up by a day). I found that by removing the empty string entry in my namespaces dictionary the issue is fixed: APPXMANIFEST_NS = { #"": "http://schemas.microsoft.com/appx/manifest/foundation/windows10", "m": "http://schemas.microsoft.com/appx/manifest/foundation/windows10", ... } So I'm not exactly sure what's going on, but as code that worked fine in 3.8.0a3 and earlier now no longer works, I think we should see whether it was a deliberate break or accidental. If there's no reason to regress users, I'd prefer to avoid it. ---------- components: XML keywords: 3.7regression messages: 341721 nosy: eli.bendersky, lukasz.langa, scoder, steve.dower priority: normal severity: normal stage: needs patch status: open title: ElementTree.find attribute matching with empty namespace type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 09:47:37 2019 From: report at bugs.python.org (Jason R. Coombs) Date: Tue, 07 May 2019 13:47:37 +0000 Subject: [New-bugs-announce] [issue36832] Port zipp to zipfile Message-ID: <1557236857.3.0.339513388865.issue36832@roundup.psfhosted.org> New submission from Jason R. Coombs : The [zipp package](https://pypi.org/project/zipp) implements a pathlib-compatable wrapper for zipfiles and is used by the importlib_metadata project. The port of importlib_metadata to importlib.metadata (issue34632) currently embeds that functionality, but this functionality is probably better exposed as part of the zipfile or pathlib modules. Yesterday, at PyCon sprints, we considered which stdlib module would be better to host this functionality, and a quick analysis revealed that there's no pathlib dependency but there is a zipfile dependency, suggesting that zipfile should be preferred. ---------- assignee: jaraco messages: 341727 nosy: barry, jaraco priority: normal severity: normal status: open title: Port zipp to zipfile _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 10:17:31 2019 From: report at bugs.python.org (Paul Ganssle) Date: Tue, 07 May 2019 14:17:31 +0000 Subject: [New-bugs-announce] [issue36833] Add tests for Datetime C API Macros Message-ID: <1557238651.7.0.774867491617.issue36833@roundup.psfhosted.org> New submission from Paul Ganssle : This is a child issue for bpo 36782, specifically for testing the macro-only datetime C API functions Untested macros with no corresponding API module: - PyDateTime_GET_YEAR - PyDateTime_GET_MONTH - PyDateTime_GET_DAY - PyDateTime_DATE_GET_HOUR - PyDateTime_DATE_GET_MINUTE - PyDateTime_DATE_GET_SECOND - PyDateTime_DATE_GET_MICROSECOND - PyDateTime_TIME_GET_HOUR - PyDateTime_TIME_GET_MINUTE - PyDateTime_TIME_GET_SECOND - PyDateTime_TIME_GET_MICROSECOND - PyDateTime_DELTA_GET_DAYS - PyDateTime_DELTA_GET_SECONDS - PyDateTime_DELTA_GET_MICROSECONDS For all of these, I think you can write one "wrapper function" that just extracts all fields as a tuple (4 C API wrapper functions - date, datetime, time, timedelta). For each function, test with both the standard class and a subclass. ---------- assignee: p-ganssle components: Tests messages: 341731 nosy: edison.abahurire, p-ganssle priority: normal severity: normal status: open title: Add tests for Datetime C API Macros type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 10:44:21 2019 From: report at bugs.python.org (Sam Park) Date: Tue, 07 May 2019 14:44:21 +0000 Subject: [New-bugs-announce] [issue36834] mock.patch.object does not persist __module__ name for functions Message-ID: <1557240261.05.0.942052494289.issue36834@roundup.psfhosted.org> New submission from Sam Park : The expectation is that the __module__ attribute for a patched function should persist after patching. Minimal test case is attached. Simply run pytest in a venv with the files. Output: def test_zxc(): with mock.patch.object(mymodule, 'asd', side_effect=mymodule.asd, autospec=True) as spy_asd: > assert spy_asd.__module__ == 'mymodule' E AssertionError: assert None == 'mymodule' E + where None = .__module__ test_mymodule.py:8: AssertionError Originally reported at https://github.com/pytest-dev/pytest-mock/issues/146 before it was determined this was a unittest.mock issue. Happens on both Python 2.7 and 3.7. Probably not really tied to a specific Python version and more of mock library issue. My local venv: Python 3.7.2 pytest 4.4.1 ---------- components: Library (Lib), Tests files: minimal-test-case.zip messages: 341738 nosy: spark priority: normal severity: normal status: open title: mock.patch.object does not persist __module__ name for functions versions: Python 2.7, Python 3.7 Added file: https://bugs.python.org/file48309/minimal-test-case.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 12:09:33 2019 From: report at bugs.python.org (Eric Snow) Date: Tue, 07 May 2019 16:09:33 +0000 Subject: [New-bugs-announce] [issue36835] Move the warnings runtime state into per-interpreter state. Message-ID: <1557245373.48.0.875552675923.issue36835@roundup.psfhosted.org> New submission from Eric Snow : Currently the warnings module uses runtime-global state (PyRuntimeState.warnings). That should be moved down to per-interpreter state. There are three possible places: 1. the module's "module state" 2. the module's __dict__ 3. PyInterpreterState.warnings (new) I have a patch for the first option. ---------- assignee: eric.snow components: Interpreter Core messages: 341763 nosy: eric.snow priority: normal severity: normal stage: needs patch status: open title: Move the warnings runtime state into per-interpreter state. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 12:20:03 2019 From: report at bugs.python.org (Sanyam Khurana) Date: Tue, 07 May 2019 16:20:03 +0000 Subject: [New-bugs-announce] [issue36836] Test Suite not working on 3.4 and 3.5 on MacOS Mojave Message-ID: <1557246003.67.0.167148241592.issue36836@roundup.psfhosted.org> New submission from Sanyam Khurana : I was testing this issue: https://bugs.python.org/issue10598 I checked both Python 3.4, 3.5, 3.6 and tagged commit v3.7.0 but I'm not able to run the test suite on any of them. I'm currently working on MacOS Mojave 10.14.4 I landed on issue https://bugs.python.org/issue36432 which was superseeded by https://bugs.python.org/issue34602 while searching for the error I got: ``` $ make test running build running build_ext building '_ssl' extension gcc -Wno-unused-result -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/usr/local/include -I/Users/sanyamkhurana/cpython/Include -I/Users/sanyamkhurana/cpython -c /Users/sanyamkhurana/cpython/Modules/_ssl.c -o build/temp.macosx-10.14-x86_64-3.6-pydebug/Users/sanyamkhurana/cpython/Modules/_ssl.o gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.14-x86_64-3.6-pydebug/Users/sanyamkhurana/cpython/Modules/_ssl.o -L/usr/local/lib -lssl -lcrypto -o build/lib.macosx-10.14-x86_64-3.6-pydebug/_ssl.cpython-36dm-darwin.so ld: library not found for -lssl clang: error: linker command failed with exit code 1 (use -v to see invocation) building '_hashlib' extension gcc -Wno-unused-result -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/usr/local/include -I/Users/sanyamkhurana/cpython/Include -I/Users/sanyamkhurana/cpython -c /Users/sanyamkhurana/cpython/Modules/_hashopenssl.c -o build/temp.macosx-10.14-x86_64-3.6-pydebug/Users/sanyamkhurana/cpython/Modules/_hashopenssl.o gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.14-x86_64-3.6-pydebug/Users/sanyamkhurana/cpython/Modules/_hashopenssl.o -L/usr/local/lib -lssl -lcrypto -o build/lib.macosx-10.14-x86_64-3.6-pydebug/_hashlib.cpython-36dm-darwin.so ld: library not found for -lssl clang: error: linker command failed with exit code 1 (use -v to see invocation) building '_tkinter' extension gcc -Wno-unused-result -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -DWITH_APPINIT=1 -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tcl.framework/Versions/Current/PrivateHeaders -I/System/Library/Frameworks/Tk.framework/Headers -I/System/Library/Frameworks/Tk.framework/Versions/Current/PrivateHeaders -I/usr/X11R6/include -I./Include -I. -I/usr/local/include -I/Users/sanyamkhurana/cpython/Include -I/Users/sanyamkhurana/cpython -c /Users/sanyamkhurana/cpython/Modules/_tkinter.c -o build/temp.macosx-10.14-x86_64-3.6-pydebug/Users/sanyamkhurana/cpython/Modules/_tkinter.o -framework Tk clang: warning: -framework Tk: 'linker' input unused [-Wunused-command-line-argument] In file included from /Users/sanyamkhurana/cpython/Modules/_tkinter.c:50: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/tk.h:78:11: fatal error: 'X11/Xlib.h' file not found # include ^~~~~~~~~~~~ 1 error generated. Python build finished successfully! The necessary bits to build these optional modules were not found: _dbm _sqlite3 nis ossaudiodev spwd zlib To find the necessary bits, look in setup.py in detect_modules() for the module's name. The following modules found by detect_modules() in setup.py, have been built by the Makefile instead, as configured by the Setup files: atexit pwd time Failed to build these modules: _hashlib _ssl _tkinter running build_scripts copying and adjusting /Users/sanyamkhurana/cpython/Tools/scripts/pydoc3 -> build/scripts-3.6 copying and adjusting /Users/sanyamkhurana/cpython/Tools/scripts/idle3 -> build/scripts-3.6 copying and adjusting /Users/sanyamkhurana/cpython/Tools/scripts/2to3 -> build/scripts-3.6 copying and adjusting /Users/sanyamkhurana/cpython/Tools/scripts/pyvenv -> build/scripts-3.6 changing mode of build/scripts-3.6/pydoc3 from 644 to 755 changing mode of build/scripts-3.6/idle3 from 644 to 755 changing mode of build/scripts-3.6/2to3 from 644 to 755 changing mode of build/scripts-3.6/pyvenv from 644 to 755 renaming build/scripts-3.6/pydoc3 to build/scripts-3.6/pydoc3.6 renaming build/scripts-3.6/idle3 to build/scripts-3.6/idle3.6 renaming build/scripts-3.6/2to3 to build/scripts-3.6/2to3-3.6 renaming build/scripts-3.6/pyvenv to build/scripts-3.6/pyvenv-3.6 ./python.exe -E -c 'import sys ; from sysconfig import get_platform ; print("%s-%d.%d" % (get_platform(), *sys.version_info[:2]))' >platform ./python.exe ./Tools/scripts/run_tests.py /Users/sanyamkhurana/cpython/python.exe -W default -bb -E -W error::BytesWarning -m test -r -w -j 0 -u all,-largefile,-audio,-gui Traceback (most recent call last): File "/Users/sanyamkhurana/cpython/Lib/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/Users/sanyamkhurana/cpython/Lib/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/sanyamkhurana/cpython/Lib/test/__main__.py", line 2, in main() File "/Users/sanyamkhurana/cpython/Lib/test/libregrtest/main.py", line 532, in main Regrtest().main(tests=tests, **kwargs) File "/Users/sanyamkhurana/cpython/Lib/test/libregrtest/main.py", line 468, in main self._main(tests, kwargs) File "/Users/sanyamkhurana/cpython/Lib/test/libregrtest/main.py", line 482, in _main setup_tests(self.ns) File "/Users/sanyamkhurana/cpython/Lib/test/libregrtest/setup.py", line 68, in setup_tests resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) ValueError: current limit exceeds maximum limit make: *** [test] Error 1 ``` It's happening in 3.4, 3.5, 3.6 branch. It's however not working on the master branch and the test suite is working completely fine on the master branch. ---------- components: Tests messages: 341770 nosy: CuriousLearner, cheryl.sabella, vstinner priority: normal severity: normal status: open title: Test Suite not working on 3.4 and 3.5 on MacOS Mojave type: behavior versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 12:23:26 2019 From: report at bugs.python.org (Benjamin Kane) Date: Tue, 07 May 2019 16:23:26 +0000 Subject: [New-bugs-announce] [issue36837] Make il8n tools available from `python -m` Message-ID: <1557246206.07.0.412069516419.issue36837@roundup.psfhosted.org> New submission from Benjamin Kane : Localizing a Python application involves using the `gettext` standard library module to read .mo files. There are three scripts to assist with this in https://github.com/bbkane/cpython/tree/master/Tools/i18n : - makelocalealias.py : Convert the X11 locale.alias file into a mapping dictionary suitable for locale.py. - msgfmt.py : Generate binary message catalog from textual translation description - pygettext.py : Generate .pot files identical to what GNU xgettext[2] generates for C and C++ code (these can be translated by msgfmt.py) I recently wrote a tutorial to localize a Python Script ( https://github.com/bbkane/arcade/blob/bbkane/add_localization_example/doc/examples/text_loc_example.rst ) and I had to tell my users (a student audience) to download these scripts from GitHub. I would have been much happier to ask them to use a builtin Python tool available from the `-m` switch (similar to `python -m json.tool`), so this issue is to add that. The docs ( https://docs.python.org/3/library/gettext.html#internationalizing-your-programs-and-modules ) mention these scripts, but do not provide any information on how to get them. Possible solutions: - turn gettext.py into a package and put these scripts into a tool subpackage (similar to json.tool) - Add a separate package (il8n perhaps) and put these scripts into there - Add links to these scripts and instructions to use them in the docs. ---------- messages: 341771 nosy: bbkane priority: normal severity: normal status: open title: Make il8n tools available from `python -m` _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 15:37:17 2019 From: report at bugs.python.org (Gregory P. Smith) Date: Tue, 07 May 2019 19:37:17 +0000 Subject: [New-bugs-announce] [issue36838] running 'make html' from the Doc tree emits an unwelcoming error message Message-ID: <1557257837.73.0.0332753639305.issue36838@roundup.psfhosted.org> New submission from Gregory P. Smith : Running make in the Doc directory does not install all necessary tooling on its own. This is a hurdle to getting beginners up to speed on making documentation changes (often their very first changes in the CPython project). ``` :~/oss/cpython/throwaway:master$ make -C Doc html make: Entering directory '/.../oss/cpython/throwaway/Doc' mkdir -p build Building NEWS from Misc/NEWS.d with blurb /bin/sh: line 5: blurb: command not found Makefile:44: recipe for target 'build' failed make: *** [build] Error 127 make: Leaving directory '/.../oss/cpython/throwaway/Doc' ``` Doc builders need to run `make -C Doc venv` first. Just updating the logic to print that as an error message when the venv or any of the necessary tools within it are missing would be more welcoming. ---------- components: Build messages: 341806 nosy: gregory.p.smith priority: normal severity: normal status: open title: running 'make html' from the Doc tree emits an unwelcoming error message type: compile error versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 15:56:26 2019 From: report at bugs.python.org (Dino Viehland) Date: Tue, 07 May 2019 19:56:26 +0000 Subject: [New-bugs-announce] [issue36839] Support the buffer protocol in code objects Message-ID: <1557258986.94.0.539165158195.issue36839@roundup.psfhosted.org> New submission from Dino Viehland : Many Python deployments involve large code based that are used in scenarios where processes are fork-and-exec'd. When running in these environments code objects can end up occupying a lot of memory. Because the code objects are on random pages and are ref counted the ref counts can cause all of the code to not be shared across processes. If code objects supported the buffer protocol it would be possible to load code from a memory mapped file which is shared across all of the processes. ---------- assignee: dino.viehland components: Interpreter Core messages: 341808 nosy: dino.viehland priority: normal severity: normal stage: needs patch status: open title: Support the buffer protocol in code objects type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 16:10:06 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Tue, 07 May 2019 20:10:06 +0000 Subject: [New-bugs-announce] [issue36840] Add stream.abort() async method Message-ID: <1557259806.05.0.192162125401.issue36840@roundup.psfhosted.org> New submission from Andrew Svetlov : It should call underlying `transport.abort()`, then wait for closing event (`await self.wait_closed()`) ---------- components: asyncio messages: 341811 nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Add stream.abort() async method versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 16:32:33 2019 From: report at bugs.python.org (Mitar) Date: Tue, 07 May 2019 20:32:33 +0000 Subject: [New-bugs-announce] [issue36841] Supporting customization of float encoding in JSON Message-ID: <1557261153.38.0.171143852563.issue36841@roundup.psfhosted.org> New submission from Mitar : Currently, there is only one argument which allows customization how float numbers are encoded in JSON: allow_nan. But this does not allow one to hook into the encoding of floating points really. The JSONEncoder is not called for float numbers. The motivation here is that we would like to encode NaN and Infinity values differently, instead of non-standard approach and instead of raising an exception. The "load" counterpart has "parse_float" which one can use to hook into parsing floats. I would suggest something similar, maybe "encode_float" which if provided would be used instead of the default. ---------- components: Library (Lib) messages: 341817 nosy: mitar priority: normal severity: normal status: open title: Supporting customization of float encoding in JSON _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 16:48:32 2019 From: report at bugs.python.org (Steve Dower) Date: Tue, 07 May 2019 20:48:32 +0000 Subject: [New-bugs-announce] [issue36842] Implement PEP 578 Message-ID: <1557262112.79.0.0300199683807.issue36842@roundup.psfhosted.org> New submission from Steve Dower : Implement PEP 578 ---------- assignee: steve.dower messages: 341819 nosy: christian.heimes, steve.dower priority: normal pull_requests: 13091 severity: normal stage: patch review status: open title: Implement PEP 578 type: enhancement versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 16:59:51 2019 From: report at bugs.python.org (Robert Boehne) Date: Tue, 07 May 2019 20:59:51 +0000 Subject: [New-bugs-announce] [issue36843] AIX build fails with failure to get random numbers Message-ID: <1557262791.73.0.95279707159.issue36843@roundup.psfhosted.org> New submission from Robert Boehne : build fails with: ./python -E ../../Python-3.7.3/setup.py build Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python ---------- components: Build messages: 341821 nosy: Robert Boehne priority: normal severity: normal status: open title: AIX build fails with failure to get random numbers type: compile error versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 17:11:03 2019 From: report at bugs.python.org (Anthony Sottile) Date: Tue, 07 May 2019 21:11:03 +0000 Subject: [New-bugs-announce] [issue36844] abiflag `m` is no longer showing when compiled with --enable-shared Message-ID: <1557263463.69.0.218039156148.issue36844@roundup.psfhosted.org> New submission from Anthony Sottile : This appears to be a regression between 3.8a3 and 3.8a4 -- though it may be intentional and I'm missing something? I noticed this while packaging 3.8 for deadsnakes https://github.com/deadsnakes/python3.8 I've created a minimal reproduction: $ git checkout v3.8.0a4 && git clean -fxfd && ./configure --enable-shared && make -j4 && ls *.so build/lib*/*sysconfigdata* ... build/lib.linux-x86_64-3.8/_sysconfigdata__linux_x86_64-linux-gnu.py libpython3.8.so libpython3.so $ git checkout v3.8.0a3 && git clean -fxfd && ./configure --enable-shared && make -j4 && ls *.so build/lib*/*sysconfigdata* ... build/lib.linux-x86_64-3.8/_sysconfigdata_m_linux_x86_64-linux-gnu.py libpython3.8m.so libpython3.so Notice how the abiflag in sysconfigdata in 3.8a3 is 'm' but in 3.8a4 it is '' My WAG at the relevant patch is 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b ---------- components: Build messages: 341827 nosy: Anthony Sottile priority: normal severity: normal status: open title: abiflag `m` is no longer showing when compiled with --enable-shared versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 18:53:58 2019 From: report at bugs.python.org (Nicolai Moore) Date: Tue, 07 May 2019 22:53:58 +0000 Subject: [New-bugs-announce] [issue36845] ipaddres.IPv4Network and ipaddress.IPv6Network tuple construction will accept out of valid range prefixlen Message-ID: <1557269638.8.0.459235898725.issue36845@roundup.psfhosted.org> New submission from Nicolai Moore : When using the tuple-form of constructing IPv4Network and IPv6Network will accept prefixlen outside of the normal allowed ranges. Example: >>> import ipaddress >>> ipaddress.IPv4Network(('172.21.1.0', 400)) IPv4Network('172.21.1.0/400') If given a negative number, it will error but not with a particularly useful error: >>> x = ipaddress.IPv4Network(('172.21.1.0', -1)) Traceback (most recent call last): File "", line 1, in File "/usr/lib64/python3.7/ipaddress.py", line 1532, in __init__ self.netmask, self._prefixlen = self._make_netmask(mask) File "/usr/lib64/python3.7/ipaddress.py", line 1112, in _make_netmask netmask = IPv4Address(cls._ip_int_from_prefix(prefixlen)) File "/usr/lib64/python3.7/ipaddress.py", line 444, in _ip_int_from_prefix return cls._ALL_ONES ^ (cls._ALL_ONES >> prefixlen) ValueError: negative shift count Looking at the code, I think all that is needed is a range check within the respective _make_netmask methods in _BaseV4 and _BaseV6 classes ---------- components: Library (Lib) messages: 341839 nosy: niconorsk priority: normal severity: normal status: open title: ipaddres.IPv4Network and ipaddress.IPv6Network tuple construction will accept out of valid range prefixlen type: behavior versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 22:23:43 2019 From: report at bugs.python.org (Dan Snider) Date: Wed, 08 May 2019 02:23:43 +0000 Subject: [New-bugs-announce] [issue36846] range_iterator does not use __index__ Message-ID: <1557282223.61.0.192486675978.issue36846@roundup.psfhosted.org> New submission from Dan Snider : I wouldn't even know where to begin to try and find a palatable solution for this that wouldn't be summarily dismissed. Perhaps it isn't unreasonable to suggest PyNumber_Index shouldn't use the less stringent PyLong_Check as the entry to the fast path. That is what happens right now and it can prevent the __index__ method defined for an int subtype being called in certain situation such as this one. Here's a silly but valid demonstration of what happens when there is more than 1 way to skin a... index. Apologies if it is unreadable but I wanted it to be a 'single' repl statement and cmd was being uncooperative without it being squished like this. if not not not not not not True: class Duper(super): def __call__(self, attr, *args): func = super.__getattribute__(self, attr) this = super.__self__.__get__(self) print(f'{this!r}.{func.__name__}(%s)'%', '.join(map(repr, args))) return super.__self_class__.__get__(self)(func(*args)) @classmethod class unbound(classmethod): def __set_name__(self, owner, name): setattr(owner, name, self.__func__(owner)) class Hex(int): __slots__ = () __call__ = __self__ = Duper.unbound() def __neg__(self): return self('__neg__') def __abs__(self): return self('__abs__') def __add__(a, b): return a('__add__', b) def __sub__(a, b): return a('__sub__', b) def __mul__(a, b): return a('__mul__', b) def __radd__(a, b): return a('__radd__', b) def __rsub__(a, b): return a('__rsub__', b) def __rmul__(a, b): return a('__rmul__', b) def __floordiv__(a, b): return a('__floordiv__', b) def __rfloordiv__(a, b): return a('__rfloordiv__', b) def __repr__(self): return f'({self.__self__.__pos__():#02x})' a, b, c, i = (Hex(i) for i in (0, 10, 2, 2)) print(f'creating range({a}, {b}, {c})...') r = range(a, b, c) print('', '-'*78) print(f'accessing the element at r[{i!r}]...') v = r[i] print('', '-'*78) print('iterating over the range...') for i in r: pass print('are we there yet?...\n') ---------- messages: 341845 nosy: bup priority: normal severity: normal status: open title: range_iterator does not use __index__ versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 7 22:52:46 2019 From: report at bugs.python.org (Jayesh) Date: Wed, 08 May 2019 02:52:46 +0000 Subject: [New-bugs-announce] [issue36847] Segmentation fault (core dumped) Found when we import "schedule" and "mysql.connector" togather. Message-ID: <1557283966.35.0.5561345962.issue36847@roundup.psfhosted.org> New submission from Jayesh : When we import following two library, then coredump found. import sys #!/usr/bin/python import schedule import mysql.connector Python 2.7.12 Ubuntu 16.04 x64 Package Version ---------------------- ------- adium-theme-ubuntu 0.3.4 configparser 3.7.4 mysql-connector-python 8.0.16 MySQL-python 1.2.5 pip 19.1.1 protobuf 3.7.1 py-mysql 1.0 schedule 0.6.0 selenium 3.141.0 setuptools 20.7.0 six 1.12.0 unity-lens-photos 1.0 urllib3 1.25.2 wheel 0.29.0 ---------- components: Library (Lib) files: poc.py messages: 341848 nosy: jay.net.in priority: normal severity: normal status: open title: Segmentation fault (core dumped) Found when we import "schedule" and "mysql.connector" togather. versions: Python 2.7 Added file: https://bugs.python.org/file48312/poc.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 02:57:13 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Wed, 08 May 2019 06:57:13 +0000 Subject: [New-bugs-announce] [issue36848] autospec fails with AttributeError when mocked class has __signature__ non-writeable Message-ID: <1557298633.85.0.260410761482.issue36848@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : With issue17185 __signature__ was set so that inspect can use this to return signature for mock objects. But in PySide2 it has __signature__ set as a non-writeable property and hence trying to set __signature__ fails. It's actually setting __signature__ for mocked object but still has this error for PySide. mock backport report : https://github.com/testing-cabal/mock/issues/464 pytest-qt report : https://github.com/pytest-dev/pytest-qt/issues/258 ---------- components: Library (Lib) keywords: 3.7regression messages: 341855 nosy: cjw296, mariocj89, michael.foord, xtreak priority: normal severity: normal status: open title: autospec fails with AttributeError when mocked class has __signature__ non-writeable type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 03:12:15 2019 From: report at bugs.python.org (Michael Osipov) Date: Wed, 08 May 2019 07:12:15 +0000 Subject: [New-bugs-announce] [issue36849] Native libcurses on HP-UX not properly detected Message-ID: <1557299535.01.0.0874998234363.issue36849@roundup.psfhosted.org> New submission from Michael Osipov <1983-01-06 at gmx.net>: The module _curses fails to build because curses cannot be found on HP-UX. In this case, it has an unorthodox location: > $ ll /usr/lib/hpux32/libcurses.so lr-xr-xr-x 1 bin bin 17 2018-09-07 15:29 /usr/lib/hpux32/libcurses.so -> ./libxcurses.so.1* A PR is in preparation ---------- components: Build messages: 341856 nosy: michael-o priority: normal severity: normal status: open title: Native libcurses on HP-UX not properly detected type: compile error versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 03:43:31 2019 From: report at bugs.python.org (hans.meine) Date: Wed, 08 May 2019 07:43:31 +0000 Subject: [New-bugs-announce] [issue36850] shutil.copy2 fails with even with source network filesystem not supporting extended attributes Message-ID: <1557301411.76.0.646767757374.issue36850@roundup.psfhosted.org> New submission from hans.meine : This is a near duplicate of #24564, but has a slightly smaller scope. We're using a CIFS filesystem and noticed that copying files *from* that filesystem to a local path with `shutil.copy2` does not work, but fails with an `OSError: [Errno 22] Invalid argument`. This is because `copy2()` calls `copystat()` which contains try: names = os.listxattr(src, follow_symlinks=follow_symlinks) except OSError as e: if e.errno not in (errno.ENOTSUP, errno.ENODATA): raise In our case with our CIFS mount, and in the NFS mounts mentioned in #24564, `os.listxattr()` raises an IOError with `errno.EINVAL` (22). It was proposed in #24564 already to also ignore EINVAL, but with this issue I wanted to specifically propose to do so for the *reading* part. While I also think that copy2() should work with both source *and* target filesystems not supporting xattr, our use case where the likelihood of data loss is zero (because there have never been xattr in the first place) is particularly annoying. ---------- components: Library (Lib) messages: 341857 nosy: hans-meine priority: normal severity: normal status: open title: shutil.copy2 fails with even with source network filesystem not supporting extended attributes versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 11:40:55 2019 From: report at bugs.python.org (Pablo Galindo Salgado) Date: Wed, 08 May 2019 15:40:55 +0000 Subject: [New-bugs-announce] [issue36851] Frame stack is not cleaned after execution is finished with return Message-ID: <1557330055.15.0.678827851719.issue36851@roundup.psfhosted.org> New submission from Pablo Galindo Salgado : When evaluating a frame object, it is possible to exit the execution with some variables in the stack. The frame deallocator (frame_dealloc) only cleans the stack if f_stacktop is not NULL, but this only happens for generators and when trace functions are set. The eval loop does this cleanup already if an exception is being raised, but not if a RETURN_VALUE is set. In the PyconUS sprints, Dino and I have been working on this and we have decided that the cleanest approach is shared the same goto label with the path for exception handling, effectively cleaning up ---------- components: Interpreter Core messages: 341891 nosy: dino.viehland, pablogsal priority: normal severity: normal status: open title: Frame stack is not cleaned after execution is finished with return versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 12:01:08 2019 From: report at bugs.python.org (Matthias Schoepfer) Date: Wed, 08 May 2019 16:01:08 +0000 Subject: [New-bugs-announce] [issue36852] Python3.7.2 fails to cross-compile (yocto / openembedded) when target is mips softfloat Message-ID: <1557331268.61.0.662119534756.issue36852@roundup.psfhosted.org> New submission from Matthias Schoepfer : Python 3.7.2 fails to cross compile for yocto / openembedded when target is mips softfloat, because the OS triple is not detected correctly by configure.ac. The configure script magic just catches for __mips__hard_float defines, but I guess, __mips__soft_float also indicates a mips architecture. Added patch ---------- components: Cross-Build files: 0001-fixing-finding-of-triplet-on-softfloat-mips.patch keywords: patch messages: 341894 nosy: Alex.Willmer, mschoepf priority: normal severity: normal status: open title: Python3.7.2 fails to cross-compile (yocto / openembedded) when target is mips softfloat versions: Python 3.7 Added file: https://bugs.python.org/file48318/0001-fixing-finding-of-triplet-on-softfloat-mips.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 12:03:28 2019 From: report at bugs.python.org (Jason R. Coombs) Date: Wed, 08 May 2019 16:03:28 +0000 Subject: [New-bugs-announce] [issue36853] inconsistencies in docs builds (Sphinx 2) Message-ID: <1557331408.26.0.606381835406.issue36853@roundup.psfhosted.org> New submission from Jason R. Coombs : In working on some docs contributions, I've run into issues where docs builds are failing in CI differently than they're failing locally. Locally, running `make venv` from `Docs/` results in Sphinx 2, whereas on Azure Pipelines, the docs are built with Sphinx 1.8.2 (https://github.com/python/cpython/blob/1d4b16051f8550fd7dada3670a3e83ae13b99d3b/.azure-pipelines/docs-steps.yml#L15). When running with Sphinx 2, this error emerges when running `make suspicious` when a change is around that triggers suspicious code (such as this commit https://github.com/python/cpython/pull/12547/commits/9bde7faf6f051d4a7306ac8629d915ce069392f7): Exception occurred: File "/Users/jaraco/code/public/cpython/Doc/tools/extensions/suspicious.py", line 154, in report_issue self.warn('[%s:%d] "%s" found in "%-.120s"' % AttributeError: 'CheckSuspiciousMarkupBuilder' object has no attribute 'warn' First, we probably want to make these processes consistent (define the dependencies in exactly one place). Second, we should probably determine why the doc builds are failing on Sphinx 2 and update the `suspicious` code to support Sphinx 2. ---------- assignee: docs at python components: Documentation messages: 341897 nosy: docs at python, jaraco priority: normal severity: normal status: open title: inconsistencies in docs builds (Sphinx 2) type: compile error versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 13:00:25 2019 From: report at bugs.python.org (Eric Snow) Date: Wed, 08 May 2019 17:00:25 +0000 Subject: [New-bugs-announce] [issue36854] GC operates out of global runtime state. Message-ID: <1557334825.47.0.818401533303.issue36854@roundup.psfhosted.org> New submission from Eric Snow : (also see #24554) We need to move GC state from _PyRuntimeState to PyInterpreterState. ---------- assignee: eric.snow components: Interpreter Core messages: 341911 nosy: eric.snow, pablogsal priority: normal severity: normal status: open title: GC operates out of global runtime state. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 13:05:33 2019 From: report at bugs.python.org (Gregory P. Smith) Date: Wed, 08 May 2019 17:05:33 +0000 Subject: [New-bugs-announce] [issue36855] update tests relying on a copy of a TLS certificate (pem) to support two versions Message-ID: <1557335133.11.0.0798454230834.issue36855@roundup.psfhosted.org> New submission from Gregory P. Smith : We're having pain today due to the Lib/test/selfsigned_pythontestdotnet.pem certificate update sychronization. This wouldn't be painful if our tests relying on a specific certificate matching something elsewhere on the network supported multiple possible certificates. Specifically an old and a new cert. When doing that, we could go ahead and commit the new cert, moving the existing one to old, before flipping the update on the test infrastructure. in all branches. that'd prevent massive PR CI and buildbot failure fallout. ---------- components: Tests messages: 341912 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: update tests relying on a copy of a TLS certificate (pem) to support two versions type: enhancement versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 13:23:39 2019 From: report at bugs.python.org (Xi Ruoyao) Date: Wed, 08 May 2019 17:23:39 +0000 Subject: [New-bugs-announce] [issue36856] faulthandler._stack_overflow doesn't work on x86-linux with KPTI enabled Message-ID: <1557336219.34.0.364472599022.issue36856@roundup.psfhosted.org> New submission from Xi Ruoyao : In faulthandler.c STACK_OVERFLOW_MAX_SIZE is defined to 100 * 1024 * 1024 (100MB). But recently KPTI has been applied to mitigate Meltdown (CVE-2017-5754) so the userspace stack pointer may be very close to 0xffffffff on Linux systems with 32-bit x86. For example, on my laptop it's sometimes 0xffffcc20. So the expression sp + STACK_OVERFLOW_MAX_SIZE overflows and becomes a very small number. That causes faulthandler._stack_overflow() to bail out after only one recursive call: Traceback (most recent call last): File "", line 1, in RuntimeError: unable to raise a stack overflow (allocated 4124 bytes on the stack, 1 recursive calls) ---------- components: Extension Modules messages: 341915 nosy: xry111 priority: normal severity: normal status: open title: faulthandler._stack_overflow doesn't work on x86-linux with KPTI enabled type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 14:11:15 2019 From: report at bugs.python.org (John Belmonte) Date: Wed, 08 May 2019 18:11:15 +0000 Subject: [New-bugs-announce] [issue36857] bisect should support descending order Message-ID: <1557339075.35.0.730492365284.issue36857@roundup.psfhosted.org> New submission from John Belmonte : because "list.pop()" use case: maintain large ordered list and efficiently remove min item list.pop() is O(1) but yields the max item. There is no efficient removal of the min item. list is by far the fastest collection to use with insort(). While deque offers O(1) popleft(), insort() performance with deque is not acceptable. Lack of descending support in bisect necessitates workarounds such as using negative-valued items, which hurts code readability and moreover assumes that values are numbers. ---------- components: Library (Lib) messages: 341921 nosy: John Belmonte priority: normal severity: normal status: open title: bisect should support descending order type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 16:36:56 2019 From: report at bugs.python.org (Eric V. Smith) Date: Wed, 08 May 2019 20:36:56 +0000 Subject: [New-bugs-announce] [issue36858] f-string '=' debugging output needs to be documented Message-ID: <1557347816.05.0.714667733239.issue36858@roundup.psfhosted.org> New submission from Eric V. Smith : There are two problems here: what to call this feature, and where to put it. The current f-string documentation is in Doc/reference/lexical_analysis.rst, which I'm not sure is the best place to add discussion about how this feature works. This feature is run-time behavior, and goes beyond what the lexer documentation should show. ---------- assignee: docs at python components: Documentation messages: 341941 nosy: docs at python, eric.smith priority: normal severity: normal status: open title: f-string '=' debugging output needs to be documented type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 19:08:24 2019 From: report at bugs.python.org (Charles) Date: Wed, 08 May 2019 23:08:24 +0000 Subject: [New-bugs-announce] [issue36859] sqlite3 dml statement detection does not account for CTEs Message-ID: <1557356904.4.0.529350025795.issue36859@roundup.psfhosted.org> New submission from Charles : In statement.c, there is some logic which detects whether or not an incoming statement is a DML-type. The logic, as of 2019-05-08, I am referring to is here: https://github.com/python/cpython/blob/fc662ac332443a316a120fa5287c235dc4f8739b/Modules/_sqlite/statement.c#L78-L93 To demonstrate the bug: import sqlite3 conn = sqlite3.connect(':memory:') conn.execute('create table kv ("key" text primary key, "value" integer)') conn.execute('insert into kv (key, value) values (?, ?), (?, ?)', ('k1', 1, 'k2', 2)) assert conn.in_transaction # Yes we are in a transaction. conn.commit() assert not conn.in_transaction # Not anymore, as expected. rc = conn.execute( 'with c(k, v) as (select key, value + 10 from kv) ' 'update kv set value=(select v from c where k=kv.key)') print(rc.rowcount) # Should be 2, prints "-1". #assert conn.in_transaction # !!! Fails. curs = conn.execute('select * from kv order by key;') print(curs.fetchall()) # [('k1', 11), ('k2', 12)] ---------- components: Library (Lib) messages: 341949 nosy: coleifer priority: normal severity: normal status: open title: sqlite3 dml statement detection does not account for CTEs type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 22:17:50 2019 From: report at bugs.python.org (Alex) Date: Thu, 09 May 2019 02:17:50 +0000 Subject: [New-bugs-announce] [issue36860] Inconsistent/Undocumented behavior with pathlib resolve and absolute on Windows Message-ID: <1557368270.63.0.171172492568.issue36860@roundup.psfhosted.org> New submission from Alex : Maybe I am doing something wrong here but: On Windows 10: > python --version Python 3.7.1 > python >>> from pathlib import Path >>> Path("test.py").resolve() WindowsPath('test.py') >>> Path("test.py").absolute() # this is undocumented in https://docs.python.org/3.7/library/pathlib.html WindowsPath('C:/Users/Name/Desktop/test.py') On Ubuntu 18.04 LTS: $ python3.7 --version Python 3.7.3 $ python3.7 >>> from pathlib import Path >>> Path("test.py").resolve() PosixPath('/home/name/test.py') >>> Path("test.py").absolute() # same as above undocumented PosixPath('/home/name/test.py') Why in Windows does this fail, but not Linux? Why is .absolute() undocumented? I've looked around trying to figure out what the intended behavior is but there is conflicting information. Resolve seems to not work correctly on Windows. Bug? ---------- components: Library (Lib), Windows messages: 341953 nosy: alexjacobson95, eryksun, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Inconsistent/Undocumented behavior with pathlib resolve and absolute on Windows type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 8 22:50:38 2019 From: report at bugs.python.org (Benjamin Peterson) Date: Thu, 09 May 2019 02:50:38 +0000 Subject: [New-bugs-announce] [issue36861] Update to Unicode 12.1.0 Message-ID: <1557370238.97.0.687491315858.issue36861@roundup.psfhosted.org> New submission from Benjamin Peterson : A minor release of Unicode 12 has been issued to pick up the Reiwa ligature: http://blog.unicode.org/2019/05/unicode-12-1-en.html We're already using Unicode 12.0.0, so this is will be a very minor upgrade. ---------- assignee: benjamin.peterson messages: 341954 nosy: benjamin.peterson priority: normal severity: normal status: open title: Update to Unicode 12.1.0 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 00:41:59 2019 From: report at bugs.python.org (anthony shaw) Date: Thu, 09 May 2019 04:41:59 +0000 Subject: [New-bugs-announce] [issue36862] Add Visual Studio 2019 support to PCBuild/env.bat Message-ID: <1557376919.34.0.533849563508.issue36862@roundup.psfhosted.org> New submission from anthony shaw : The PCBuild tool should support VS2019 but default to 2017 as per documentation. PR to follow. ---------- assignee: anthonypjshaw components: Build messages: 341963 nosy: anthonypjshaw priority: normal severity: normal status: open title: Add Visual Studio 2019 support to PCBuild/env.bat type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 04:39:22 2019 From: report at bugs.python.org (rhubarbdog x) Date: Thu, 09 May 2019 08:39:22 +0000 Subject: [New-bugs-announce] [issue36863] argparse doesn't like options in the middle of arguments Message-ID: <1557391162.39.0.325223416239.issue36863@roundup.psfhosted.org> New submission from rhubarbdog x : I've found a bug with argparse and reported it upstream. `command --option source destination` works `command soucre destination --option` works but `command source --option destination` doesn't outputting the usage text block associated `do_command` method and outputs error message `command: error: unrecognized arguments: destination` ---------- components: Argument Clinic messages: 341973 nosy: larry, rhubarbdog x priority: normal severity: normal status: open title: argparse doesn't like options in the middle of arguments _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 07:54:49 2019 From: report at bugs.python.org (Jules Lasne) Date: Thu, 09 May 2019 11:54:49 +0000 Subject: [New-bugs-announce] [issue36864] Double Spaces in the documentation Message-ID: <1557402889.33.0.243843870102.issue36864@roundup.psfhosted.org> New submission from Jules Lasne : Hello, I've come to open this issue today because of the apparent split there is in the documentation about double spaces after a period. Here is a link to an article explaining some of the issue https://www.writing-skills.com/one-space-two-full-stop Anyway, the opinion isn't really important here but, when translating the documentation we waste a lot of time removing the double spaces that are ugly and useless in French. I'd like to propose a PR to remove all double spaces after a period from the documentation as they serve no apparent purpose. I'm open for discussion of course ---------- assignee: docs at python components: Documentation messages: 341977 nosy: docs at python, seluj78 priority: normal severity: normal status: open title: Double Spaces in the documentation _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 09:22:23 2019 From: report at bugs.python.org (Nathaniel Gaertner) Date: Thu, 09 May 2019 13:22:23 +0000 Subject: [New-bugs-announce] [issue36865] FileInput does not allow 'rt' mode, but all its existing delegates do Message-ID: <1557408143.65.0.0603138385641.issue36865@roundup.psfhosted.org> New submission from Nathaniel Gaertner : While looking at https://bugs.python.org/issue5758 I noticed that 'rt' support had been added to gzip and bz2 in 3.3, but FileInput still limited its mode options to 'r', 'rb', 'rU', and 'U'. It seems like 'rt' should be just fine here as well, and would help clarify issue 5758, since 'r' defaults to 'rt' for open(...) but defaults to 'rb' for gzip and bz2. I wrote up the code and modified the mode unit test to try every allowed mode. I'll attach a PR to this once I have it ready. ---------- components: Library (Lib) messages: 341978 nosy: natgaertner priority: normal severity: normal status: open title: FileInput does not allow 'rt' mode, but all its existing delegates do type: enhancement versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 11:48:32 2019 From: report at bugs.python.org (Matt Martz) Date: Thu, 09 May 2019 15:48:32 +0000 Subject: [New-bugs-announce] [issue36866] Certificate verification errors in urllib.request become URLError Message-ID: <1557416912.64.0.674817399159.issue36866@roundup.psfhosted.org> New submission from Matt Martz : The behavior of how SSL certificate validation is handled was changed in https://bugs.python.org/issue31399 This introduced a new exception, ssl.SSLCertVerificationError, which is raised for any certificate validation error, instead of the previous exception ssl.CertificateError. The primary difference here comes into play in urllib.request: https://github.com/python/cpython/blob/da0847048aa7f934573fa449cea8643def056aa5/Lib/urllib/request.py#L1314-L1318 Previously ssl.CertificateError was not derived from OSError, it instead was derived from ValueError. As such, as of Python3.7, ssl.SSLCertVerificationError is caught by the exception handling referenced above, as it is derived from OSError, and raised as a URLError, causing exception handling issues. You must now introspect e.reason to determine if the exception was caused due to certificate verification or any other URLError, instead of simply catching separate exception types. ---------- assignee: christian.heimes components: Library (Lib), SSL messages: 341985 nosy: christian.heimes, sivel, vstinner priority: normal severity: normal status: open title: Certificate verification errors in urllib.request become URLError type: behavior versions: Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 13:36:01 2019 From: report at bugs.python.org (Pierre Glaser) Date: Thu, 09 May 2019 17:36:01 +0000 Subject: [New-bugs-announce] [issue36867] Make semaphore_tracker track other system resources Message-ID: <1557423361.05.0.0948606859067.issue36867@roundup.psfhosted.org> New submission from Pierre Glaser : Hi all, Olivier Grisel, Thomas Moreau and myself are currently working on increasing the range of action of the semaphore_tracker in Python. multiprocessing.semaphore_tracker is a little known module, that launches a server process used to track the life cycle of semaphores created in a python session, and potentially cleanup those semaphores after all python processes of the session terminated. Normally, python processes cleanup semaphores they create. This is however not not guaranteed if the processes get violently interrupted (using for example the bash command "killall python") A note on why the semaphore_tracker was introduced: Cleaning up semaphores after termination is important because the system only supports a limited number of named semaphores, and they will not be automatically removed till the next reboot. Now, Python 3.8 introduces shared memory segments creation. Shared memory is another sensitive global system resource. Currently, unexpected termination of processes that created memory segments will result in leaking those memory segments. This can be problematic for large compute clusters with many users and that are rebooted rarely. For this reason, we expanded the semaphore_tracker to also track shared memory segments, and renamed it resource_tracker. Shared memory segments get automatically tracked by the resource tracker when they are created. This is a first, self-contained fix. (1) Additionally, supporting shared memory tracking led to a more generic design for the resource_tracker. The resource_tracker can be now easily extended to track arbitrary resource types. A public API could potentially be exposed for users willing to track other types. One for example may want to add tracking for temporary folders creating during python sessions. Another use case is the one of joblib, which is a widely-used parallel-computing package, and also the backend of scikit-learn. Joblib relies heavily on memmapping. A public API could extend the resource_tracker to track memmap-ed objects with very little code. Therefore, this issue serves two purposes: - referencing the semaphore_tracker enhancement mentioned in (1) - discussing a potentially public resource_tracker API. ---------- components: Library (Lib) messages: 341987 nosy: pablogsal, pierreglaser, pitrou priority: normal severity: normal status: open title: Make semaphore_tracker track other system resources type: resource usage versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 14:15:47 2019 From: report at bugs.python.org (Josh Rosenberg) Date: Thu, 09 May 2019 18:15:47 +0000 Subject: [New-bugs-announce] [issue36868] New behavior of OpenSSL hostname verification not exposed, incorrectly documented Message-ID: <1557425747.49.0.948212766878.issue36868@roundup.psfhosted.org> New submission from Josh Rosenberg : The What's New in 3.7 docs mention the change from #31399 to use OpenSSL's built-in hostname verification ( https://docs.python.org/3/whatsnew/3.7.html#ssl ), but aside from that, information about the change is largely undiscoverable and/or wrong. Specific problems: 1. The What's New docs repeatedly mention SSLContext.host_flags as the means of modifying behavior. The actual property is underscore prefixed though, as SSLContext._host_flags. Since SSLContext supports the creation of arbitrary names via __dict__, assigning to context.host_flags silently "works", it just fails to *do* anything (nothing ever reads it). 2. None of the flags are documented anywhere; the only way to discover them is to import _ssl (they're not exposed on ssl itself, just the internal C extension), then scan through the exposed names (they're all prefixed with HOSTFLAG_ AFAICT) 3. All of the flags are raw numeric values, but it seems like they should be IntEnums, like the other flags exposed by SSL (among other things, it would make it much easier to interpret the default _host_flags (currently it's just 4, when it could display as ) 4. Nothing about this change, _host_flags/host_flags, or the values of the flags themselves is mentioned on the ssl docs at all. 5. This unintentionally made a behavioral change (one that bit me, and may bite other folks using docker swarm, NETBIOS hostnames, etc.). Python's match_hostname implementation was fine with host names containing underscores (e.g. if the cert was wildcarded to *.example.com, it would match a_b_c.example.com just fine); they're not technically legal by the strict reading of the specs for host names (they're apparently legal for domain names, but not host names, which differ in ways I don't fully understand), but stuff like docker swarm names their services that way automatically, most (all?) browsers support visiting them, etc. It looks like OpenSSL (at least the 1.1.0g my Python 3.7.2 was built against) treats underscores as unmatchable, so any attempt to connect to such a host name in Python 3.7.2 dies with a SSLCertVerificationError, claiming a "Hostname mismatch, certificate is not valid for 'name_with_underscores.example.com'." I discovered all this because 3.7 broke some scripts I use to connect to docker swarm services. Before I realized the issue was underscores, I was trying to figure out how to tweak the host name checks (assuming maybe something was broken with wildcard matching), and stumbled across all the other issues with the docs, the lack of flag definition exposure, etc. For the record, I think it's reasonable to require legal host names (it was easy enough to fix for my case; I just updated our docker DNS server to provide aliases using only hyphens and changed the script to use the alias host names), but it would be nice if it was explicitly documented, and ideally, that Python itself recognize that underscores won't work and explicitly raise an exception saying why, rather than letting OpenSSL perform the rejection with a (to someone who doesn't know about the underscore issue) confusing error message. ---------- assignee: docs at python components: Documentation, Extension Modules, Library (Lib), SSL keywords: 3.6regression messages: 341990 nosy: christian.heimes, docs at python, josh.r, vstinner priority: normal severity: normal status: open title: New behavior of OpenSSL hostname verification not exposed, incorrectly documented type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 20:11:13 2019 From: report at bugs.python.org (Emmanuel Arias) Date: Fri, 10 May 2019 00:11:13 +0000 Subject: [New-bugs-announce] [issue36869] Avoid warning of unused variables Message-ID: <1557447073.75.0.208623856666.issue36869@roundup.psfhosted.org> New submission from Emmanuel Arias : When run ./configure && make -j4 there are two warnings on Object/dictobject.c file about ix and hash variable are not used. Zachary Ware make me note that when _PyObject_ASSERT is call and NDEBUG is defined that expand to ((void)0). So this PR add a #ifndef check to avoid the warnings. ---------- components: Build messages: 342016 nosy: eamanu priority: normal pull_requests: 13141 severity: normal status: open title: Avoid warning of unused variables type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 20:57:51 2019 From: report at bugs.python.org (STINNER Victor) Date: Fri, 10 May 2019 00:57:51 +0000 Subject: [New-bugs-announce] [issue36870] test_asyncio: test_drain_raises() fails randomly on Windows Message-ID: <1557449871.52.0.351453555458.issue36870@roundup.psfhosted.org> New submission from STINNER Victor : https://ci.appveyor.com/project/python/cpython/builds/24376676 ERROR: test_drain_raises (test.test_asyncio.test_streams.StreamTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\projects\cpython\lib\test\test_asyncio\test_streams.py", line 820, in test_drain_raises self.loop.run_until_complete(client(*addr)) File "C:\projects\cpython\lib\asyncio\base_events.py", line 584, in run_until_complete return future.result() File "C:\projects\cpython\lib\test\test_asyncio\test_streams.py", line 810, in client await writer.drain() File "C:\projects\cpython\lib\asyncio\streams.py", line 351, in drain await fut File "C:\projects\cpython\lib\asyncio\selector_events.py", line 860, in write n = self._sock.send(data) ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine ---------- components: Tests messages: 342023 nosy: pablogsal, vstinner priority: normal severity: normal status: open title: test_asyncio: test_drain_raises() fails randomly on Windows versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 21:20:28 2019 From: report at bugs.python.org (Gregory P. Smith) Date: Fri, 10 May 2019 01:20:28 +0000 Subject: [New-bugs-announce] [issue36871] Misleading error from unittest.mock's assert_has_calls Message-ID: <1557451228.11.0.694440628534.issue36871@roundup.psfhosted.org> New submission from Gregory P. Smith : Thing: mock_thing.method sig=(a=None, b=0) F ====================================================================== FAIL: test_has_calls_on_thing (__main__.MockCallTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/google/home/gps/mock_call_test.py", line 25, in test_has_calls_on_thing mock_thing.assert_has_calls([ File "/usr/local/google/home/gps/oss/cpython/gpshead/Lib/unittest/mock.py", line 843, in assert_has_calls raise AssertionError( AssertionError: Calls not found. Expected: [call.method(0.5, b=3000), call.method(0.6, b=6000), call.method(0.7, b=9000)] Actual: [call.method(0.5, b=3000), call.method(0.6, b=6000), call.method(0.7, b=9000)]. See the attached mock_call_test.py. ---------- components: Library (Lib) files: mock_call_test.py messages: 342028 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: Misleading error from unittest.mock's assert_has_calls type: behavior versions: Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48323/mock_call_test.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 23:15:16 2019 From: report at bugs.python.org (David Collins) Date: Fri, 10 May 2019 03:15:16 +0000 Subject: [New-bugs-announce] [issue36872] passing negative values through modules Message-ID: <1557458116.02.0.0587666073886.issue36872@roundup.psfhosted.org> New submission from David Collins : program_data.py file contains a list import list_function str_list4 = ['i', 't'] str_list4 = list_function.insert_value(str_list4, 's', -1) print(str_list4) list_function.py file def insert_value(my_list, value, insert_position): counter = 0 print('Question 5') print(my_list, value, insert_position) for index in my_list: counter += 1 when passing the negative number in insert_position between the two modules this causes str_list4 = list_function.insert_value(str_list4, 's', -1) File "D:\part A\list_function.py", line 85, in insert_value for index in my_list: TypeError: 'NoneType' object is not iterable my_list now becomes lost, if the negative value is changed to positive this clears up . if you assign the -1 to a variable or a contstant it does not matter it still causes the same issue. however if you are running this code in the same module it works fine. this only seems to appear when attempting to pass between 2 modules a negative number. ---------- assignee: terry.reedy components: IDLE messages: 342049 nosy: coldy028, terry.reedy priority: normal severity: normal status: open title: passing negative values through modules type: compile error versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 9 23:41:31 2019 From: report at bugs.python.org (STINNER Victor) Date: Fri, 10 May 2019 03:41:31 +0000 Subject: [New-bugs-announce] [issue36873] http.server: Document explicitly that symbolic links are followed Message-ID: <1557459691.12.0.556693975132.issue36873@roundup.psfhosted.org> New submission from STINNER Victor : http.server documentation starts with a red warning: "Warning: http.server is not recommended for production. It only implements basic security checks." https://docs.python.org/dev/library/http.server.html It would help to be even more explicit on what it means. For example, document that symbolic links are followed and SimpleHTTPRequestHandler directory can be "escaped" following symbolic links. ---------- assignee: docs at python components: Documentation messages: 342054 nosy: docs at python, vstinner priority: normal severity: normal status: open title: http.server: Document explicitly that symbolic links are followed type: security versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 10 05:51:15 2019 From: report at bugs.python.org (Pierre van de Laar) Date: Fri, 10 May 2019 09:51:15 +0000 Subject: [New-bugs-announce] [issue36874] Support CDATA by xml.etree.(c)ElementTree Message-ID: <1557481875.81.0.586413890873.issue36874@roundup.psfhosted.org> New submission from Pierre van de Laar : I would like to add information to CDATA in an Xml Tree. Turns out I am not the only one: https://stackoverflow.com/questions/174890/how-to-output-cdata-using-elementtree Can the library be extended to also support CDATA (similar to Comment)? Saves a lot of hacking... ---------- components: XML messages: 342067 nosy: Pierre van de Laar priority: normal severity: normal status: open title: Support CDATA by xml.etree.(c)ElementTree type: enhancement versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 10 08:34:34 2019 From: report at bugs.python.org (Carmen Bianca Bakker) Date: Fri, 10 May 2019 12:34:34 +0000 Subject: [New-bugs-announce] [issue36875] argparse does not ship with translations Message-ID: <1557491674.85.0.800524916833.issue36875@roundup.psfhosted.org> New submission from Carmen Bianca Bakker : Although argparse contains translatable strings, translations for those strings do not ship with Python. This means that any program that uses argparse must separately translate argparse, which is a lot of duplicated work. Moreover, if argparse is translated downstream, it is only translated against a single version of Python. If a user uses a different version of Python, there is no guarantee that the strings (and thus the translations) will be the same. Shipping translations for argparse together with Python would solve this issue. As a caveat, the translations cannot be in `/usr/share/locale` or equivalent. Users may have multiple versions of Python installed, so there would be a namespace conflict. ---------- messages: 342073 nosy: carmenbianca priority: normal severity: normal status: open title: argparse does not ship with translations type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 10 15:01:42 2019 From: report at bugs.python.org (Eric Snow) Date: Fri, 10 May 2019 19:01:42 +0000 Subject: [New-bugs-announce] [issue36876] Global C variables are a problem. Message-ID: <1557514902.13.0.853517754348.issue36876@roundup.psfhosted.org> New submission from Eric Snow : We still have a bunch of "global" C variables (static globals, static locals, maybe thread-local storage) in our code-base that break the isolation between interpreters. I added Tools/c-globals/check-c-globals.py a while back to help identify such variables, however more have crept in. I also did not take static locals or thread-locals into account. To address the above, we should do the following: 1. update check-c-globals.py to identify static locals (and thread-locals) 2. deal with any identified globals * move them to _PyRuntimeState (or thread-locals to PyThreadState, etc.) * ignore them by adding them to Tools/c-globals/ignored-globals.txt 3. add check-c-globals.py to "make check" 4. (if "make check" isn't already there), ensure check-c-globals.py is run at some point in CI Separately, we should move fields out of _PyRuntimeState into PyInterpreterState wherever possible. That can also be done at step 2 if it's not too much work. ---------- assignee: eric.snow components: Interpreter Core messages: 342119 nosy: eric.snow, vstinner priority: normal severity: normal stage: needs patch status: open title: Global C variables are a problem. type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 10 15:11:57 2019 From: report at bugs.python.org (Eric Snow) Date: Fri, 10 May 2019 19:11:57 +0000 Subject: [New-bugs-announce] [issue36877] [meta] Move fields from _PyRuntimeState to PyInterpreterState. Message-ID: <1557515517.79.0.229526383765.issue36877@roundup.psfhosted.org> New submission from Eric Snow : We have quite a bit of global state the runtime that effectively breaks the isolation between interpreters. Some of it exists as "global" C variables (see #36876) and the rest as fields on _PyRuntimeState. The offending state should be moved to PyInterpreterState. See Include/internal/pycore_pystate.h for the _PyRuntimeState and PyInterpreterState structs. ---------- assignee: eric.snow components: Interpreter Core messages: 342120 nosy: eric.snow, vstinner priority: normal severity: normal status: open title: [meta] Move fields from _PyRuntimeState to PyInterpreterState. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 10 16:40:16 2019 From: report at bugs.python.org (Michael Sullivan) Date: Fri, 10 May 2019 20:40:16 +0000 Subject: [New-bugs-announce] [issue36878] ast.parse with type_comments=True should allow extra text after # type: ignore Message-ID: <1557520816.53.0.629625329103.issue36878@roundup.psfhosted.org> New submission from Michael Sullivan : Per discussion during the typing summit at PyCon, it would be a good idea to allow extra information to be included in `# type: ignore` comments, in order to allow behavior such as suppressing individual errors (for example, with syntax like `# type: ignore[E1000]`, to suppress error 1000). My proposal, then, is to generalize the definition of type: ignore comments to be `# type: ignore followed by a non-alphanumeric character. Then `# type: ignore[E1000]` and `# type: ignore E1000` would be valid type ignore comments while `# type: ignoreE1000` would not be. Now that ast.parse can parse type_comments, this needs to make it into 3.8, basically, if we want to do this and be able to use the ast type_comment feature. Ideally, the text of the type ignore would be actually included in the produced AST. As a bare minimum first step, we need to recognize type ignores with extra information and report them (and, critically, not detect them as regular type comments and produce errors when they appear in unexpected places). I'll put up a PR to do the second part shortly. ---------- components: Interpreter Core messages: 342130 nosy: gvanrossum, levkivskyi, msullivan priority: normal severity: normal status: open title: ast.parse with type_comments=True should allow extra text after # type: ignore type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 10 16:51:01 2019 From: report at bugs.python.org (Scaler) Date: Fri, 10 May 2019 20:51:01 +0000 Subject: [New-bugs-announce] [issue36879] bug with round() and "numpy floats" Message-ID: <1557521461.62.0.231118714409.issue36879@roundup.psfhosted.org> New submission from Scaler : round()'s documentation (https://docs.python.org/3/library/functions.html#round) says that "The return value is an integer if ndigits is omitted or None." Works well with "built-in floats", but not with "numpy floats" (didn't know they were different): >>> import numpy as np >>> round(5.5) 6 >>> round(np.round(5.5)-0.5) 6.0 As a side note, is there any reason why round() does not return value as an integer if ndigits is 0 or negative also? ---------- messages: 342132 nosy: Scaler priority: normal severity: normal status: open title: bug with round() and "numpy floats" type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 10 18:39:01 2019 From: report at bugs.python.org (dgelessus) Date: Fri, 10 May 2019 22:39:01 +0000 Subject: [New-bugs-announce] [issue36880] Returning None from a callback with restype py_object decrements None's refcount too much Message-ID: <1557527941.1.0.208797159552.issue36880@roundup.psfhosted.org> New submission from dgelessus : This occurs when writing a ctypes callback in Python whose restype is ctypes.py_object. If the callback returns None, the refcount of None is decremented once too often. This happens every time the callback is called, and if done often enough, Python attempts to deallocate None and crashes. To reproduce: $ bin/python3 Python 3.8.0a4+ (heads/master:09532feeec, May 10 2019, 23:53:49) [Clang 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> import ctypes >>> FUNTYPE = ctypes.CFUNCTYPE(ctypes.py_object) >>> @FUNTYPE ... def fun(): ... return None ... >>> print(fun()) None >>> sys.getrefcount(None) 4329 >>> print(fun()) None >>> sys.getrefcount(None) 4327 >>> print(fun()) None >>> sys.getrefcount(None) 4326 >>> while True: ... fun() ... Fatal Python error: deallocating None Current thread 0x00007fff7bf80000 (most recent call first): File "", line 2 in Abort trap: 6 # exits with code 134 (SIGABRT) I've attached the crash report generated by OS X. (It's a plain text file, despite the extension.) Interestingly, this only happens with None. Other returned objects are refcounted correctly: >>> thing = object() >>> @FUNTYPE ... def otherfun(): ... return thing ... >>> print(otherfun()) >>> sys.getrefcount(thing) 2 >>> print(otherfun()) >>> sys.getrefcount(thing) 2 >>> print(otherfun()) >>> sys.getrefcount(thing) 2 I could reproduce this on the following configurations: * Python 3.8.0a4 (self-compiled from master branch: 09532feeece39d5ba68a0d47115ce1967bfbd58e) on OS X 10.11 (x86_64) * Python 3.7.3 (python.org installer) on OS X 10.11 (x86_64) * Python 3.6.8 (from package manager) on Kubuntu 18.10 (x86_64) * Python 3.5.3 (from package manager) on Raspbian Stretch (armv6) ---------- components: ctypes files: python3.8_2019-05-11-000251.crash messages: 342141 nosy: dgelessus priority: normal severity: normal status: open title: Returning None from a callback with restype py_object decrements None's refcount too much type: crash versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48325/python3.8_2019-05-11-000251.crash _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 10 22:29:04 2019 From: report at bugs.python.org (Randy Eckman) Date: Sat, 11 May 2019 02:29:04 +0000 Subject: [New-bugs-announce] [issue36881] isinstance raises TypeError for metaclass with metaclass=ABCMeta Message-ID: <1557541744.61.0.162387429511.issue36881@roundup.psfhosted.org> New submission from Randy Eckman : >>> from abc import ABCMeta >>> class AbstractMeta(type, metaclass=ABCMeta): pass ... >>> class Meta1(AbstractMeta): pass ... >>> class Meta2(AbstractMeta): pass ... >>> class ClassA(metaclass=Meta1): pass ... >>> isinstance(ClassA, Meta2) Traceback (most recent call last): File "", line 1, in File "C:\miniconda\miniconda3\lib\abc.py", line 139, in __instancecheck__ return _abc_instancecheck(cls, instance) TypeError: __subclasscheck__() takes exactly one argument (0 given) This is driven by the fact that the metaclass of AbstractMeta is ABCMeta; if I change its metaclass to something else that does not contain ABCMeta in the inheritance chain, isinstance returns False as expected. Possibly related to Issue 2325? ---------- components: Interpreter Core messages: 342152 nosy: emanspeaks priority: normal severity: normal status: open title: isinstance raises TypeError for metaclass with metaclass=ABCMeta versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 03:05:13 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Sat, 11 May 2019 07:05:13 +0000 Subject: [New-bugs-announce] [issue36884] DeprecationWarning in test_asyncio.test_pep492.StreamReaderTests.test_readline Message-ID: <1557558313.36.0.153269520968.issue36884@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : It seems there was a deprecation warning added in ad4ed872415d00fcdfaa52a08108ec752b115000 that causes error when test_asyncio.test_pep492.StreamReaderTests.test_readline is ran with -Werror. ? cpython git:(master) ./python.exe -Werror -m unittest -v test.test_asyncio.test_pep492.StreamReaderTests.test_readline test_readline (test.test_asyncio.test_pep492.StreamReaderTests) ... ERROR ====================================================================== ERROR: test_readline (test.test_asyncio.test_pep492.StreamReaderTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_asyncio/test_pep492.py", line 97, in test_readline stream = asyncio.StreamReader(loop=self.loop) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/streams.py", line 446, in __init__ warnings.warn(f"{self.__class__} should be instaniated " DeprecationWarning: should be instaniated by asyncio internals only, please avoid its creation from user code ---------------------------------------------------------------------- Ran 1 test in 0.003s FAILED (errors=1) # Before commit ? cpython git:(master) git checkout ad4ed872415d00fcdfaa52a08108ec752b115000~1 Lib/asyncio ? cpython git:(master) ? ./python.exe -Werror -m unittest -v test.test_asyncio.test_pep492.StreamReaderTests.test_readline test_readline (test.test_asyncio.test_pep492.StreamReaderTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 0.004s OK ---------- components: asyncio messages: 342170 nosy: asvetlov, xtreak, yselivanov priority: normal severity: normal status: open title: DeprecationWarning in test_asyncio.test_pep492.StreamReaderTests.test_readline type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 03:08:11 2019 From: report at bugs.python.org (Stefan Behnel) Date: Sat, 11 May 2019 07:08:11 +0000 Subject: [New-bugs-announce] [issue36885] Clean up makeunicode.py script Message-ID: <1557558491.4.0.613612433422.issue36885@roundup.psfhosted.org> New submission from Stefan Behnel : The code generation in the makeunicode.py script is more difficult to read than necessary due to the many use of "print(file=fp)" everywhere. Moving the "file" argument out of the way makes it easier to read through the actual code that is being generated. ---------- components: Demos and Tools messages: 342171 nosy: benjamin.peterson, scoder priority: low severity: normal stage: patch review status: open title: Clean up makeunicode.py script type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 06:35:09 2019 From: report at bugs.python.org (Big Stone) Date: Sat, 11 May 2019 10:35:09 +0000 Subject: [New-bugs-announce] [issue36886] problem with Types on Python-3.8.0a4 Message-ID: <1557570909.72.0.695431824181.issue36886@roundup.psfhosted.org> New submission from Big Stone : remark: I get a similar error from two packages, when experimenting with Python-3.8.0a4 on Windows ```` import asyncio await asyncio.sleep(0) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) C:\WinP\bd38\bu\WPy64-3800a4\python-3.8.0a4.amd64\lib\site-packages\IPython\core\interactiveshell.py in removed_co_newlocals(function) 139 CO_NEWLOCALS = 0x0002 140 code = function.__code__ --> 141 new_code = CodeType( 142 code.co_argcount, 143 code.co_kwonlyargcount, TypeError: an integer is required (got type bytes) ```` or else: ```` import cloudpickle ```` ```` import cloudpickle 1 0 1 3 19 b'\x87\x00f\x01d\x01d\x02\x84\x08\x01\x00|\x00\x89\x00d\x00S\x00' (None, at 0x000001B36B7D49B0, file "C:\WinP\bd38\bu\WPy64-3800a4\python-3.8.0a4.amd64\lib\site-packages\cloudpickle\cloudpickle.py", line 107>, '_make_cell_set_template_code..inner..') () ('value',) C:\WinP\bd38\bu\WPy64-3800a4\python-3.8.0a4.amd64\lib\site-packages\cloudpickle\cloudpickle.py inner 106 b'\x00\x01\x0c\x01' ('cell',) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in ----> 1 import cloudpickle C:\WinP\bd38\bu\WPy64-3800a4\python-3.8.0a4.amd64\lib\site-packages\cloudpickle\__init__.py in 1 from __future__ import absolute_import 2 ----> 3 from cloudpickle.cloudpickle import * 4 5 __version__ = '1.0.0' C:\WinP\bd38\bu\WPy64-3800a4\python-3.8.0a4.amd64\lib\site-packages\cloudpickle\cloudpickle.py in 165 166 --> 167 _cell_set_template_code = _make_cell_set_template_code() 168 169 C:\WinP\bd38\bu\WPy64-3800a4\python-3.8.0a4.amd64\lib\site-packages\cloudpickle\cloudpickle.py in _make_cell_set_template_code() 146 co.co_lnotab, 147 co.co_cellvars) --> 148 return types.CodeType( 149 co.co_argcount, 150 co.co_kwonlyargcount, TypeError: an integer is required (got type bytes) ---------- components: Windows messages: 342182 nosy: Big Stone, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: problem with Types on Python-3.8.0a4 versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 09:46:56 2019 From: report at bugs.python.org (Mark Dickinson) Date: Sat, 11 May 2019 13:46:56 +0000 Subject: [New-bugs-announce] [issue36887] Add integer square root, math.isqrt Message-ID: <1557582416.09.0.555439074551.issue36887@roundup.psfhosted.org> New submission from Mark Dickinson : The integer square root[1] is a common number-theoretic primitive, useful for example in primality testing. This is something I've had to code up myself multiple times, and is also something that's quite easy to get wrong, or implement in a way that's inefficient for large inputs. I propose adding a math module function `math.isqrt` implementing the integer square root. Like `math.gcd`, it should accept any integer-like object `n` (i.e., `int`s and anything implementing `__index__`), and for nonnegative inputs, should return the largest int `a` satisfying `a * a <= n`. Negative inputs should give a ValueError; non-integer inputs (including `float`s) should give a TypeError. I'll create a PR shortly with a basic implementation; optimizations can happen later. [1] https://en.wikipedia.org/wiki/Integer_square_root ---------- assignee: mark.dickinson components: Extension Modules messages: 342187 nosy: mark.dickinson priority: normal severity: normal stage: needs patch status: open title: Add integer square root, math.isqrt type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 11:36:32 2019 From: report at bugs.python.org (Thomas Moreau) Date: Sat, 11 May 2019 15:36:32 +0000 Subject: [New-bugs-announce] [issue36888] Create a way to check that the parent process is alive for deamonized processes Message-ID: <1557588992.87.0.32954176975.issue36888@roundup.psfhosted.org> New submission from Thomas Moreau : In the std lib, the semaphore_tracker and the Manager rely on daemonized processes that are launched with server like loops. The cleaning of such processes is made complicated by the fact that there is no cannonical way to check that the parent process is alive. I propose to add in context a parent_process function that would give access to a Process object representing the parent process. This way, we could benefit from sentinel to improve the clean up of this process that can be left dangling in case of hard stop from the main interpreter. ---------- components: Library (Lib) messages: 342199 nosy: tomMoral priority: normal severity: normal status: open title: Create a way to check that the parent process is alive for deamonized processes type: enhancement versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 14:15:14 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Sat, 11 May 2019 18:15:14 +0000 Subject: [New-bugs-announce] [issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream Message-ID: <1557598514.53.0.379470651864.issue36889@roundup.psfhosted.org> New submission from Andrew Svetlov : The separation was a bad idea, sorry. Both classes are coupled too much. 1. Writer should know about the reader to make `drain()` work 2. Reader has no .close() method. The idea is: 1. Merge StreamReader and StreamWriter into just a Stream. 2. The Stream should support three modes: readonly, writeonly, readwrite. 3. Keep StreamReader/StreamWriter but deprecate these classes. 4. Replace all asyncio internals to use Stream class. ---------- components: asyncio messages: 342212 nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Merge StreamWriter and StreamReader into just asyncio.Stream versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 14:19:46 2019 From: report at bugs.python.org (S. J. Cunningham) Date: Sat, 11 May 2019 18:19:46 +0000 Subject: [New-bugs-announce] [issue36890] python-3.7.3-macosx10.6.pkg verification error Message-ID: <1557598786.16.0.639636022435.issue36890@roundup.psfhosted.org> New submission from S. J. Cunningham : I can't Install the latest python-3.7.3-macosx10.6.pkg on my 10.6.8 system. Error message says failure to verify. I posted on the forms as suggested (https://python-forum.io/Thread-Cannot-Install-python-3-7-3-macosx10-6-pkg) but have received no response. Would appreciate any help. I cannot instal the latest version of Python (python-3.7.3-macosx10.6.pkg) on Mac OS 10.6.8. The installer throws the following error: The operation couldn?t be completed. (com.apple.installer.pagecontroller error -1.) Couldn't open "python-3.7.3-macosx10.6.pkg". And leaves the following messages in the console log. 5/9/19 4:00:25 PM Installer @(#)PROGRAM:Install PROJECT:Install-596.1 5/9/19 4:00:25 PM Installer @(#)PROGRAM:Installer PROJECT:Installer-430.1 5/9/19 4:00:25 PM Installer Hardware: MacBookPro3,1 @ 2.20 GHz (x 2), 4096 MB RAM 5/9/19 4:00:25 PM Installer Running OS Build: Mac OS X 10.6.8 (10K549) 5/9/19 4:00:25 PM Installer Failed to verify data against certificate. 5/9/19 4:00:25 PM Installer Invalid Distribution File/Package ---------- components: macOS messages: 342214 nosy: ned.deily, ronaldoussoren, steve_314 priority: normal severity: normal status: open title: python-3.7.3-macosx10.6.pkg verification error versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 15:32:48 2019 From: report at bugs.python.org (vx1920) Date: Sat, 11 May 2019 19:32:48 +0000 Subject: [New-bugs-announce] [issue36891] Additional startup plugin for vendors Message-ID: <1557603168.21.0.544720725277.issue36891@roundup.psfhosted.org> New submission from vx1920 : Proposed modification in site.py allows "sitevendor" customization plugin in addition to existed "sitecustomize" and "usercustomuze". More info displayed when "python.exe -m site" typed from command line. Vendors of Python sites (like Anaconda) can place there their own startup code. Mentioned "sitecustomize" and "usercustomuze" works as before. ---------- components: Library (Lib) messages: 342218 nosy: vx1920 priority: normal pull_requests: 13164 severity: normal status: open title: Additional startup plugin for vendors type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 11 19:00:01 2019 From: report at bugs.python.org (musou1500) Date: Sat, 11 May 2019 23:00:01 +0000 Subject: [New-bugs-announce] [issue36892] "Modules" section in Tutorial contains incorrect description about __init__.py Message-ID: <1557615601.14.0.911128255467.issue36892@roundup.psfhosted.org> New submission from musou1500 : "Modules" section in the tutorial says that https://docs.python.org/3/tutorial/modules.html "The __init__.py files are required to make Python treat directories containing the file as packages. " But, thanks to PEP 420, __init__.py is not required after Python 3.3. https://www.python.org/dev/peps/pep-0420/ ---------- assignee: docs at python components: Documentation messages: 342228 nosy: docs at python, musou1500 priority: normal severity: normal status: open title: "Modules" section in Tutorial contains incorrect description about __init__.py type: enhancement versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 12 07:06:18 2019 From: report at bugs.python.org (Matthew) Date: Sun, 12 May 2019 11:06:18 +0000 Subject: [New-bugs-announce] [issue36893] email.headerregistry.Address blocks Unicode local part addr_spec accepted elsewhere Message-ID: <1557659178.71.0.414757697263.issue36893@roundup.psfhosted.org> New submission from Matthew : The parser for passing an addr_spec to email.headerregistry.Address does not allow non-ASCII local parts, but the rest of the email package handles them fine, either straight (with explicit references to RFC6532 and SMTPUTF8), or encoding as expected. Apologies if I've misunderstood something. >>> from email.message import EmailMessage >>> msg = EmailMessage() >>> msg['To'] = 'Matth?w ' >>> msg.as_string() 'To: =?utf-8?q?Matth=C3=A9w?= <=?utf-8?q?a=C3=A9?=@example.com>\n\n' >>> msg['To'].addresses[0] Address(display_name='Matth?w', username='a?', domain='example.com') >>> msg['To'].addresses[0].addr_spec 'a?@example.com' >>> email.headerregistry.Address(addr_spec=msg['To'].addresses[0].addr_spec) Traceback (most recent call last): File "", line 1, in File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/email/headerregistry.py", line 48, in __init__ raise a_s.all_defects[0] email.errors.NonASCIILocalPartDefect: local-part contains non-ASCII characters) >>> ---------- components: email messages: 342254 nosy: barry, dracos, r.david.murray priority: normal severity: normal status: open title: email.headerregistry.Address blocks Unicode local part addr_spec accepted elsewhere versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 12 13:03:41 2019 From: report at bugs.python.org (Antoine Pitrou) Date: Sun, 12 May 2019 17:03:41 +0000 Subject: [New-bugs-announce] [issue36894] test_multiprocessing_spawn regression on Windows Message-ID: <1557680621.73.0.0423240629914.issue36894@roundup.psfhosted.org> New submission from Antoine Pitrou : After bpo-36867, test_multiprocessing_spawn fails running any test on Windows (but it isn't noticed as a fail build). https://ci.appveyor.com/project/python/cpython/builds/24485897#L1264 0:04:05 load avg: 5.39 [389/421] test_multiprocessing_spawn run no tests ERROR ====================================================================== ERROR: setUpModule (test.test_multiprocessing_spawn) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\projects\cpython\lib\test\_test_multiprocessing.py", line 5498, in setUpModule multiprocessing.set_forkserver_preload(PRELOAD) File "C:\projects\cpython\lib\multiprocessing\context.py", line 183, in set_forkserver_preload from .forkserver import set_forkserver_preload File "C:\projects\cpython\lib\multiprocessing\forkserver.py", line 14, in from . import resource_tracker File "C:\projects\cpython\lib\multiprocessing\resource_tracker.py", line 24, in import _posixshmem ModuleNotFoundError: No module named '_posixshmem' ---------------------------------------------------------------------- Ran 0 tests in 0.005s FAILED (errors=1) ---------- components: Library (Lib), Tests messages: 342262 nosy: pierreglaser, pitrou priority: deferred blocker severity: normal stage: needs patch status: open title: test_multiprocessing_spawn regression on Windows type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 12 17:21:34 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Sun, 12 May 2019 21:21:34 +0000 Subject: [New-bugs-announce] =?utf-8?q?=5Bissue36895=5D_time=2Eclock=28?= =?utf-8?b?KSBtYXJrZWQgZm9yIHJlbW92YWwgaW4gMy44IOKAk8Kgc3RpbGwgdGhlcmUu?= Message-ID: <1557696094.54.0.878450659131.issue36895@roundup.psfhosted.org> New submission from Matthias Bussonnier : Deprecation message in `timemodule.c` says: > "time.clock has been deprecated in Python 3.3 and will be removed from Python 3.8: use time.perf_counter or time.process_time instead" Should be bumped to 3.9 ??or time.clock should be removed. Deprecation was added by Victor Stinner. ---------- components: Library (Lib) messages: 342267 nosy: mbussonn priority: normal severity: normal status: open title: time.clock() marked for removal in 3.8 ??still there. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 12 18:03:30 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Sun, 12 May 2019 22:03:30 +0000 Subject: [New-bugs-announce] [issue36896] clarify in types.rst that FunctionTypes & co constructors don't have stable signature Message-ID: <1557698610.19.0.556381141599.issue36896@roundup.psfhosted.org> New submission from Matthias Bussonnier : >From bpo-36886, IT is unclear the FunctionTypes, CodeTypes ... etc are not stable between python versions, and the recent addition of `:=` change some of the signatures. This suggest 2 things: - A CYA sentence in types.rst "These types are not supposed to be instantiated outside of CPython internals and constructor signatures will vary between python versions." or alike - As many people don't read online documentation but on the docstring via calling `help()`, to add something similar to all the docstrings of said-objects constructors. ---------- assignee: docs at python components: Documentation messages: 342271 nosy: docs at python, mbussonn priority: normal severity: normal status: open title: clarify in types.rst that FunctionTypes & co constructors don't have stable signature versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 12 20:59:51 2019 From: report at bugs.python.org (Matthew Gamble) Date: Mon, 13 May 2019 00:59:51 +0000 Subject: [New-bugs-announce] [issue36897] shlex doesn't differentiate escaped characters in output Message-ID: <1557709191.27.0.852663831837.issue36897@roundup.psfhosted.org> New submission from Matthew Gamble : The output of the following invocations are exactly the same: list(shlex.shlex('a ; b', posix=True, punctuation_chars=True)) list(shlex.shlex('a \; b', posix=True, punctuation_chars=True)) They both output the following: ['a', ';', 'b'] This makes it impossible to determine when the user wanted to escape the semi-colon for some reason, such as if they were using find's `-exec` argument. ---------- components: Library (Lib) messages: 342276 nosy: Matthew Gamble priority: normal severity: normal status: open title: shlex doesn't differentiate escaped characters in output type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 03:17:56 2019 From: report at bugs.python.org (Chuang Men) Date: Mon, 13 May 2019 07:17:56 +0000 Subject: [New-bugs-announce] [issue36898] Add parameter @case_sensitive to glob and rglob in pathlib Message-ID: <1557731876.23.0.465761944937.issue36898@roundup.psfhosted.org> New submission from Chuang Men : In pathlib, I add a parameter @case_sensitive to glob and rglob. Sometimes the extension would be in upper case but sometimes it would be lower case, for example: *.tif and *.TIF. So the parameter @case_sensitive may be useful in some cases. Usage example: In [1]: from pathlib import Path In [2]: path = Path('.') In [3]: for each_file in path.glob('*.tif'): ...: print(each_file) ...: a.tif b.tif In [4]: for each_file in path.rglob('*.TIF'): ...: print(each_file) ...: c.TIF TEST/d.TIF In [5]: for each_file in path.glob('*.TIF', case_sensitive=False): ...: print(each_file) ...: a.tif c.TIF b.tif In [6]: for each_file in path.rglob('*.TIF', case_sensitive=False): ...: print(each_file) ...: a.tif c.TIF b.tif TEST/d.TIF TEST/e.tif ---------- components: Library (Lib) files: pathlib.py messages: 342284 nosy: Chuang Men priority: normal severity: normal status: open title: Add parameter @case_sensitive to glob and rglob in pathlib type: enhancement versions: Python 3.7 Added file: https://bugs.python.org/file48328/pathlib.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 05:29:19 2019 From: report at bugs.python.org (Piotr Kamoda) Date: Mon, 13 May 2019 09:29:19 +0000 Subject: [New-bugs-announce] [issue36899] datetime utcfromtimestamp ignores astimezone Message-ID: <1557739759.84.0.347644793793.issue36899@roundup.psfhosted.org> New submission from Piotr Kamoda : See below code: >>> datetime.utcfromtimestamp(1557395250).astimezone(get_localzone()).strftime('%Y-%m-%d %H:%M:%S %z %Z') '2019-05-09 09:47:30 +0200 CEST' >>> datetime.fromtimestamp(1557395250).astimezone(get_localzone()).strftime('%Y-%m-%d %H:%M:%S %z %Z') '2019-05-09 11:47:30 +0200 CEST' As you can see, utcfromtimestamp refuses to be 'timezoned', second one is correct. ---------- messages: 342297 nosy: Piotr Kamoda priority: normal severity: normal status: open title: datetime utcfromtimestamp ignores astimezone versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 07:07:28 2019 From: report at bugs.python.org (STINNER Victor) Date: Mon, 13 May 2019 11:07:28 +0000 Subject: [New-bugs-announce] [issue36900] Use _PyCoreConfig rather than global configuration variables Message-ID: <1557745648.65.0.648187833126.issue36900@roundup.psfhosted.org> New submission from STINNER Victor : PyInterpreterState.config is now preferred over global configuration variables, so two interpreters can have a different configuration, the "state" of an interpreter is now better defined (especially when and how its configuration is set). Attached PRs fix these isssues. ---------- components: Interpreter Core messages: 342310 nosy: vstinner priority: normal severity: normal status: open title: Use _PyCoreConfig rather than global configuration variables versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 07:57:47 2019 From: report at bugs.python.org (Cheryl Sabella) Date: Mon, 13 May 2019 11:57:47 +0000 Subject: [New-bugs-announce] [issue36901] Fix leaks in /PC/bdist_wininst/install.c Message-ID: <1557748667.87.0.123726768405.issue36901@roundup.psfhosted.org> New submission from Cheryl Sabella : /PC/bdist_wininst/install.c contains some memory leaks. PR2142 appears to be abandoned, so a new PR needs to be opened with the changes and also with the additional changes requested by Serhiy. ---------- components: Installation messages: 342311 nosy: cheryl.sabella, serhiy.storchaka, vstinner priority: normal severity: normal stage: needs patch status: open title: Fix leaks in /PC/bdist_wininst/install.c type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 08:34:59 2019 From: report at bugs.python.org (rudragavara) Date: Mon, 13 May 2019 12:34:59 +0000 Subject: [New-bugs-announce] [issue36902] Wireless Network Engineer Message-ID: <1557750899.2.0.375808424646.issue36902@roundup.psfhosted.org> New submission from rudragavara : A Wireless Network Engineer is responsible for installing, configuring and maintaining wireless network equipment, network management and security including 802.11 b/g/n/ac standards and industry best practices for implementing high-density WIFI solutions. Moreover, to put it concisely, this person needs to assess, plan and develop for several operations capabilities for wireless telecommunications. As more companies continue to embrace Wireless LAN, the demand for wireless network engineers has grown manifold in the recent past. The proliferation of mobile applications, which require testing in a wireless environment, is also one of the other reasons why enterprises need their services more, currently. Read More; https://www.fieldengineer.com/engineers/field-engineer-jobs ---------- messages: 342314 nosy: rudragavara priority: normal severity: normal status: open title: Wireless Network Engineer _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 09:55:54 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Mon, 13 May 2019 13:55:54 +0000 Subject: [New-bugs-announce] [issue36903] ResourceWarning in test_logging.HandlerTest.test_post_fork_child_no_deadlock Message-ID: <1557755754.69.0.913565279245.issue36903@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : There was a test added with 64aa6d2000665efb1a2eccae176df9520bf5f5e6. It opens a stream of '/dev/null' in the test and doesn't close it causing ResourceWarning. A fix would be to assign it to a variable and close it as part of test's cleanup process. I will raise a PR for this. ./python.exe -m unittest test.test_logging.HandlerTest.test_post_fork_child_no_deadlock /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_logging.py:142: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='wt' encoding='UTF-8'> loggerDict.clear() ResourceWarning: Enable tracemalloc to get the object allocation traceback . ---------------------------------------------------------------------- Ran 1 test in 0.606s OK ---------- components: Tests messages: 342330 nosy: gregory.p.smith, xtreak priority: normal severity: normal status: open title: ResourceWarning in test_logging.HandlerTest.test_post_fork_child_no_deadlock type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 11:22:54 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Mon, 13 May 2019 15:22:54 +0000 Subject: [New-bugs-announce] [issue36904] Implement _PyStack_UnpackDict() with a single allocation Message-ID: <1557760974.34.0.94387480475.issue36904@roundup.psfhosted.org> New submission from Jeroen Demeyer : _PyStack_UnpackDict() is used to convert from the FastCallDict calling convention to FastCallKeywords. It currently needs two allocations: one for the tuple of keyword names and one for the array of arguments (positional and keyword). This can be optimized by using a single allocation, storing everything in a single tuple. That tuple can then be artificially truncated to the required size when done. ---------- components: Library (Lib) messages: 342339 nosy: Mark.Shannon, jdemeyer, petr.viktorin, scoder priority: normal severity: normal status: open title: Implement _PyStack_UnpackDict() with a single allocation type: performance versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 11:52:02 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Mon, 13 May 2019 15:52:02 +0000 Subject: [New-bugs-announce] [issue36905] test_typing.GetTypeHintTests.test_get_type_hints_modules_forwardref unexpected success while running whole test suite sequentially Message-ID: <1557762722.1.0.805190875774.issue36905@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : This happens only while running the tests sequentially and also with whole test suite. This error is present in optional GCC build consistently as below : https://travis-ci.org/python/cpython/jobs/531845094#L2103 Running whole test suite sequentially in verbose mode brings below message locally : test_get_type_hints_modules_forwardref (test.test_typing.GetTypeHintTests) ... unexpected success I am not sure of the relation in running this sequentially along with the whole test suite. Feel free to close this if this is a false positive. ---------- components: Tests messages: 342348 nosy: levkivskyi, xtreak priority: normal severity: normal status: open title: test_typing.GetTypeHintTests.test_get_type_hints_modules_forwardref unexpected success while running whole test suite sequentially type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 14:40:31 2019 From: report at bugs.python.org (Gregory P. Smith) Date: Mon, 13 May 2019 18:40:31 +0000 Subject: [New-bugs-announce] [issue36906] Compile time textwrap.dedent() equivalent for str or bytes literals Message-ID: <1557772831.22.0.440689390024.issue36906@roundup.psfhosted.org> New submission from Gregory P. Smith : A Python pattern in code is to keep everything indented to look pretty while, yet when the triple quoted multiline string in question needs to not have leading whitespace, calling textwrap.dedent("""long multiline constant""") is a common pattern. rather than doing this computation at runtime, this is something that'd make sense to do at compilation time. A natural suggestion for this would be a new letter prefix for multiline string literals that triggers this. Probably not worth "wasting" a letter on this, so I'll understand if we reject the idea, but it'd be nice to have rather than importing textwrap and calling it all over the place just for this purpose. There are many workarounds but an actual syntax would enable writing code that looked like this: ```python class Castle: def __init__(self, name, lyrics=None): if not lyrics: lyrics = df"""\ We're knights of the round table We dance whene'er we're able We do routines and scenes With footwork impeccable. We dine well here in {name} We eat ham and jam and spam a lot. """ self._name = name self._lyrics = lyrics ``` Without generating a larger temporary always in memory string literal in the code object that gets converted at runtime to the desired dedented form via a textwrap.dedent() call. I chose "d" as the the letter to mean dedent. I don't have a strong preference if we ever do make this a feature. ---------- components: Interpreter Core messages: 342373 nosy: gregory.p.smith priority: low severity: normal status: open title: Compile time textwrap.dedent() equivalent for str or bytes literals type: enhancement versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 15:40:04 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Mon, 13 May 2019 19:40:04 +0000 Subject: [New-bugs-announce] [issue36907] Crash due to borrowed references in _PyStack_UnpackDict() Message-ID: <1557776404.01.0.559706785576.issue36907@roundup.psfhosted.org> New submission from Jeroen Demeyer : class IntWithDict: def __init__(self, **kwargs): self.kwargs = kwargs def __index__(self): self.kwargs.clear() L = [2**i for i in range(10000)] return 0 x = IntWithDict(dont_inherit=float()) compile("", "", "", x, **x.kwargs) The above crashes CPython due to the usage of borrowed references in _PyStack_UnpackDict(): the dict x.kwargs contains the only reference to the float() object stored in x.kwargs When parsing the arguments, x.__int__() is called, which clears the dict, removing the only reference to that float() ---------- components: Interpreter Core messages: 342377 nosy: jdemeyer, vstinner priority: normal severity: normal status: open title: Crash due to borrowed references in _PyStack_UnpackDict() versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 17:15:22 2019 From: report at bugs.python.org (Ned Batchelder) Date: Mon, 13 May 2019 21:15:22 +0000 Subject: [New-bugs-announce] [issue36908] "This module is always available" is confusing Message-ID: <1557782122.73.0.455327263081.issue36908@roundup.psfhosted.org> New submission from Ned Batchelder : The math and cmath modules say this in their docs: "This module is always available." This lead a beginner to believe they didn't need to be imported. Nearly the entire standard library is always available in this sense (they can be imported). This sentence isn't useful. Can we change the first paragraph of the math module to be: "This module provides access to the mathematical functions defined by the C standard." (and similarly for the cmath module.) ---------- assignee: docs at python components: Documentation messages: 342386 nosy: docs at python, nedbat priority: normal severity: normal status: open title: "This module is always available" is confusing _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 18:21:50 2019 From: report at bugs.python.org (wim glenn) Date: Mon, 13 May 2019 22:21:50 +0000 Subject: [New-bugs-announce] [issue36909] LastUpdatedOrderedDict recipe uses super() unnecessarily Message-ID: <1557786110.6.0.397786860144.issue36909@roundup.psfhosted.org> New submission from wim glenn : Section https://docs.python.org/3/library/collections.html#ordereddict-examples-and-recipes class LastUpdatedOrderedDict(OrderedDict): 'Store items in the order the keys were last added' def __setitem__(self, key, value): super().__setitem__(key, value) super().move_to_end(key) Why does it use super().move_to_end(key), isn't self.move_to_end(key) more direct/Pythonic? ---------- assignee: docs at python components: Documentation messages: 342395 nosy: docs at python, rhettinger, wim.glenn priority: normal pull_requests: 13209 severity: normal status: open title: LastUpdatedOrderedDict recipe uses super() unnecessarily _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 21:48:39 2019 From: report at bugs.python.org (Mark Sapiro) Date: Tue, 14 May 2019 01:48:39 +0000 Subject: [New-bugs-announce] [issue36910] Certain Malformed email causes email.parser to throw AttributeError Message-ID: <1557798519.08.0.310106320232.issue36910@roundup.psfhosted.org> New submission from Mark Sapiro : The code in the attached parse_bug.py file when run with Python 3.5, 3.6 or 3.7 throws AttributeError with this traceback: ``` Traceback (most recent call last): File "parse_bug.py", line 9, in """) File "/usr/local/lib/python3.7/email/parser.py", line 124, in parsebytes return self.parser.parsestr(text, headersonly) File "/usr/local/lib/python3.7/email/parser.py", line 68, in parsestr return self.parse(StringIO(text), headersonly=headersonly) File "/usr/local/lib/python3.7/email/parser.py", line 58, in parse return feedparser.close() File "/usr/local/lib/python3.7/email/feedparser.py", line 187, in close self._call_parse() File "/usr/local/lib/python3.7/email/feedparser.py", line 180, in _call_parse self._parse() File "/usr/local/lib/python3.7/email/feedparser.py", line 323, in _parsegen if (self._cur.get('content-transfer-encoding', '8bit').lower() AttributeError: 'Header' object has no attribute 'lower' ``` The triggering condition appears to be the Content-Transfer-Encoding: header with a non-ascii character in the headers of a multipart part. The parser should probably throw email.errors.HeaderParseError with a MalformedHeaderDefect in this case rather than AttributeError. While arguably code should defend against unanticipated exceptions, the fact that such an exception can be thrown while parsing an arbitrary message could be considered a security issue. ---------- components: email files: parse_bug.py messages: 342415 nosy: barry, msapiro, r.david.murray priority: normal severity: normal status: open title: Certain Malformed email causes email.parser to throw AttributeError type: behavior versions: Python 3.5, Python 3.6, Python 3.7 Added file: https://bugs.python.org/file48330/parse_bug.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 22:02:23 2019 From: report at bugs.python.org (Amber Brown) Date: Tue, 14 May 2019 02:02:23 +0000 Subject: [New-bugs-announce] [issue36911] ast.parse outputs ast.Strs which do not differentiate between the ASCII codepoint 12 (literal new line) and the ASCII codepoints 134 and 156 ("\n") Message-ID: <1557799343.57.0.564179832858.issue36911@roundup.psfhosted.org> New submission from Amber Brown : reproducing case: file.py: ``` """ Hello \n blah. """ ``` And then in a REPL (2.7 or 3+): ``` >>> import ast >>> f = ast.parse(open("test.py", 'rb').read()) >>> f <_ast.Module object at 0x7f609d0a4d68> >>> f.body[0] <_ast.Expr object at 0x7f609d0a4e10> >>> f.body[0].value <_ast.Str object at 0x7f609d02b780> >>> f.body[0].value.s '\nHello \n blah.\n' >>> repr(f.body[0].value.s) "'\\nHello \\n blah.\\n'" ``` Expected behaviour: ``` >>> repr(f.body[0].value.s) "'\\nHello \\\\n blah.\\n'" ``` ---------- components: Library (Lib) messages: 342417 nosy: hawkowl priority: normal severity: normal status: open title: ast.parse outputs ast.Strs which do not differentiate between the ASCII codepoint 12 (literal new line) and the ASCII codepoints 134 and 156 ("\n") versions: Python 2.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 13 22:54:04 2019 From: report at bugs.python.org (Guido van Rossum) Date: Tue, 14 May 2019 02:54:04 +0000 Subject: [New-bugs-announce] [issue36912] Can't left-align strings using f-strings or format() Message-ID: <1557802444.81.0.632526308066.issue36912@roundup.psfhosted.org> New submission from Guido van Rossum : I sometimes like to use format specs like "%-10s" % foo, which left-aligns the string foo in a field of 10 spaces: >>> '%10s' % foo ' abc' >>> foo = 'abc' >>> '%-10s' % foo 'abc ' This is documented in "Format Specification Mini-Language": https://docs.python.org/3/library/string.html#format-specification-mini-language However it does not work with format() and f-strings: >>> format(foo, '10s') 'abc ' >>> format(foo, '-10s') Traceback (most recent call last): File "", line 1, in ValueError: Sign not allowed in string format specifier >>> f'{foo:10}' 'abc ' >>> f'{foo:-10}' Traceback (most recent call last): File "", line 1, in ValueError: Sign not allowed in string format specifier In each case, without a sign it works, and right-aligns. What am I missing? ---------- messages: 342421 nosy: gvanrossum priority: normal severity: normal status: open title: Can't left-align strings using f-strings or format() type: behavior versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 14 03:35:28 2019 From: report at bugs.python.org (Tomer Vromen) Date: Tue, 14 May 2019 07:35:28 +0000 Subject: [New-bugs-announce] [issue36913] Missing documentation for decorators Message-ID: <1557819328.64.0.0474936169734.issue36913@roundup.psfhosted.org> New submission from Tomer Vromen : The documentation for decorators (for methods and classes) is pretty lacking. Searching for "decorator" ( https://docs.python.org/3/search.html?q=decorator ) brings up a lot of libraries that use decorators, but no documentation explaining what they are and how they work. The documentation should have a dedicated page for explaining decorators, and this should be the 1st search result. -- In the meantime, then search should give as a 1st result a link to the definition of decorator in the glossary: https://docs.python.org/3.7/glossary.html#glossary Actually, it seems that there is no way to directly link to a paragraph in the glossary - so that should be added as well. In general, it would be nice if a search for some term X would show as a 1st result the definition of X in the glossary (if it exists there). Thanks! ---------- assignee: docs at python components: Documentation messages: 342435 nosy: docs at python, tomerv priority: normal severity: normal status: open title: Missing documentation for decorators type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 14 05:29:07 2019 From: report at bugs.python.org (Vadim Kantorov) Date: Tue, 14 May 2019 09:29:07 +0000 Subject: [New-bugs-announce] [issue36914] zipimport does not load ZIP files produced by GitHub Message-ID: <1557826147.57.0.863332328412.issue36914@roundup.psfhosted.org> New submission from Vadim Kantorov : Python does not like source code zip-files produced by GitHub (zipimporting source from GitHub can be useful in some ad-hoc scripting scenarios e.g. I wanted to use it in Mozilla Iodide context) wget https://github.com/python/black/archive/master.zip python3.7 -m 'import zipimport; zipimport.zipimporter("master.zip")' # zipimport.ZipImportError: not a Zip file: 'master.zip' unzip master.zip zip -r master2.zip black-master python3.7 -m 'import zipimport; zipimport.zipimporter("master2.zip")' # OK ---------- components: Library (Lib) messages: 342439 nosy: Vadim Kantorov priority: normal severity: normal status: open title: zipimport does not load ZIP files produced by GitHub type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 14 08:05:07 2019 From: report at bugs.python.org (STINNER Victor) Date: Tue, 14 May 2019 12:05:07 +0000 Subject: [New-bugs-announce] [issue36915] regrtest: when interrupted, temporary directory is not removed Message-ID: <1557835507.9.0.901051073741.issue36915@roundup.psfhosted.org> New submission from STINNER Victor : I modified regrtest (Python test runner) to kill worker processes using SIGKILL when the main process is interrupted by CTRL+C. Problem: nothing clears temporary directories created by these worker processes. I'm working on a fix. ---------- components: Tests messages: 342451 nosy: vstinner priority: normal severity: normal status: open title: regrtest: when interrupted, temporary directory is not removed versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 14 08:35:05 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Tue, 14 May 2019 12:35:05 +0000 Subject: [New-bugs-announce] [issue36916] Swallow unhandled exception report introduced by 36802 Message-ID: <1557837305.49.0.626105934017.issue36916@roundup.psfhosted.org> New submission from Andrew Svetlov : In #36802 when old-style writer.write() is used without awaiting and an exception is raised from writer.drain() method asyncio reports about the unhandled exception. The proposed fix adds a done callback to task for swallowing the exception in such case. ---------- messages: 342457 nosy: asvetlov priority: normal severity: normal status: open title: Swallow unhandled exception report introduced by 36802 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 14 10:21:22 2019 From: report at bugs.python.org (Anthony Sottile) Date: Tue, 14 May 2019 14:21:22 +0000 Subject: [New-bugs-announce] [issue36917] ast.NodeVisitor no longer calls visit_Str Message-ID: <1557843682.58.0.0807679622364.issue36917@roundup.psfhosted.org> New submission from Anthony Sottile : More fallout from the Constant change in https://bugs.python.org/issue32892 minimal reproduction: import ast class V(ast.NodeVisitor): def visit_Str(self, node): print(node.s) def main(): V().visit(ast.parse('x = "hi"')) if __name__ == '__main__': exit(main()) $ python3.7 t.py hi $ python3.8 t.py $ python3.8 --version --version Python 3.8.0a4 (default, May 8 2019, 01:43:53) [GCC 7.4.0] ---------- components: Library (Lib) messages: 342463 nosy: Anthony Sottile, serhiy.storchaka priority: normal severity: normal status: open title: ast.NodeVisitor no longer calls visit_Str versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 14 13:55:07 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Tue, 14 May 2019 17:55:07 +0000 Subject: [New-bugs-announce] [issue36918] ValueError warning in test_urllib due to io.IOBase destructor Message-ID: <1557856507.99.0.0146189740679.issue36918@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : Issue for https://bugs.python.org/issue18748#msg340059. comment : Is there someone interested to debug remaining "Exception ignored:" logs in test_urllib? test_invalid_redirect (test.test_urllib.urlopen_HttpTests) ... Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. ok test_read_bogus (test.test_urllib.urlopen_HttpTests) ... Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. ok test_redirect_limit_independent (test.test_urllib.urlopen_HttpTests) ... Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. Exception ignored in: Traceback (most recent call last): File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close super().close() # set "closed" flag File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush self.fp.flush() ValueError: I/O operation on closed file. ok It's unclear to be if it's a bug in http.client, a bug in the test... or something else. ---------- components: Tests messages: 342492 nosy: vstinner, xtreak priority: normal severity: normal status: open title: ValueError warning in test_urllib due to io.IOBase destructor type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 14 16:28:37 2019 From: report at bugs.python.org (Pavel Koneski) Date: Tue, 14 May 2019 20:28:37 +0000 Subject: [New-bugs-announce] [issue36919] Exception form 'compile' reports a newline char not present in input Message-ID: <1557865717.35.0.0613603941425.issue36919@roundup.psfhosted.org> New submission from Pavel Koneski : Since Python 3.2, input in 'exec' mode of 'compile' does not have to end in a newline anymore. However, it creates a surprising behavior when a 'SyntaxError' is reported: >>> try: compile('try', '', 'exec') ... except SyntaxError as ex: print(repr(ex)) ... SyntaxError('invalid syntax', ('', 1, 4, 'try\n')) The 'text' field of the exception thrown contains an additional newline character that was not present in the input. Is it: a. Proper Python language behavior? b. CPython implementation artifact? c. A bug? In case of: a. I will submit a patch to IronPython, which does not add an extra newline at the moment. b. I can submit a patch to CPython to make StdLib tests implementation independent. c. This inquiry can serve as a bug report. ---------- components: Interpreter Core messages: 342515 nosy: BCSharp priority: normal severity: normal status: open title: Exception form 'compile' reports a newline char not present in input type: behavior versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 14 22:03:29 2019 From: report at bugs.python.org (Udi Meiri) Date: Wed, 15 May 2019 02:03:29 +0000 Subject: [New-bugs-announce] [issue36920] inspect.getcallargs sees optional arg to builtin as required Message-ID: <1557885809.35.0.0229737900946.issue36920@roundup.psfhosted.org> New submission from Udi Meiri : $ python3.7 Python 3.7.3rc1 (default, Mar 13 2019, 11:01:15) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import inspect >>> inspect.getfullargspec(str.strip) FullArgSpec(args=['self', 'chars'], varargs=None, varkw=None, defaults=None, kwonlyargs=[], kwonlydefaults=None, annotations={}) >>> inspect.signature(str.strip) >>> inspect.signature(str.strip).bind('a') >>> inspect.getcallargs(str.strip, 'a') Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.7/inspect.py", line 1372, in getcallargs _missing_arguments(f_name, req, True, arg2value) File "/usr/lib/python3.7/inspect.py", line 1302, in _missing_arguments "" if missing == 1 else "s", s)) TypeError: strip() missing 1 required positional argument: 'chars' ---------- components: Library (Lib) messages: 342529 nosy: Udi Meiri priority: normal severity: normal status: open title: inspect.getcallargs sees optional arg to builtin as required type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 04:32:31 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Wed, 15 May 2019 08:32:31 +0000 Subject: [New-bugs-announce] [issue36921] Deprecate yield from and @coroutine in asyncio Message-ID: <1557909151.77.0.533525553848.issue36921@roundup.psfhosted.org> Change by Andrew Svetlov : ---------- components: asyncio nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Deprecate yield from and @coroutine in asyncio versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 04:39:45 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Wed, 15 May 2019 08:39:45 +0000 Subject: [New-bugs-announce] [issue36922] Implement Py_TPFLAGS_METHOD_DESCRIPTOR Message-ID: <1557909585.08.0.913937443168.issue36922@roundup.psfhosted.org> New submission from Jeroen Demeyer : The new flag Py_TPFLAGS_METHOD_DESCRIPTOR proposed in PEP 590 is meant for classes whose instances behave like unbound methods. In other words, it's meant for objects supporting the LOAD_METHOD optimization. There are two such classes in CPython: function and method_descriptor. But the goal is to enable more such classes. This is independent from the rest of PEP 590 so let's implement it separately. ---------- components: Interpreter Core messages: 342554 nosy: Mark.Shannon, jdemeyer, petr.viktorin priority: normal severity: normal status: open title: Implement Py_TPFLAGS_METHOD_DESCRIPTOR type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 05:02:38 2019 From: report at bugs.python.org (Augustin PROLONGEAU) Date: Wed, 15 May 2019 09:02:38 +0000 Subject: [New-bugs-announce] [issue36923] Implemented __str__ for zip and map objects Message-ID: <1557910958.96.0.205285122739.issue36923@roundup.psfhosted.org> New submission from Augustin PROLONGEAU : I think this would help development a lot. Below an example of how i would see the function written for zip: def __str__(self): return str((*self,)) # And for __repr__ function def __repr__(self): return f'zip{str(self)}' ---------- messages: 342556 nosy: AugPro priority: normal severity: normal status: open title: Implemented __str__ for zip and map objects type: enhancement versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 07:58:26 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Wed, 15 May 2019 11:58:26 +0000 Subject: [New-bugs-announce] [issue36924] Simplify implementation of classmethod_descriptor.__call__ Message-ID: <1557921506.83.0.125634021475.issue36924@roundup.psfhosted.org> New submission from Jeroen Demeyer : The class classmethod_descriptor implements classmethods for builtin functions. Unlike the plain classmethod class (which is used for Python classmethods), instances of classmethod_descriptor are callable. However, calling them is unlikely to happen in practice: the only way to obtain such an object is to extract from the class __dict__. Therefore, the implementation of __call__ does not need to be optimized: we can just call __get__ and then call the result. Doing that allows a simpler implementation of PEP 590. ---------- components: Interpreter Core messages: 342566 nosy: Mark.Shannon, jdemeyer, petr.viktorin priority: normal severity: normal status: open title: Simplify implementation of classmethod_descriptor.__call__ versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 08:42:34 2019 From: report at bugs.python.org (wuwentao) Date: Wed, 15 May 2019 12:42:34 +0000 Subject: [New-bugs-announce] [issue36925] python3.7.3 can't work after MacOS upgrade from v10.14.4 to v10.14.5 Message-ID: <1557924154.05.0.857477832316.issue36925@roundup.psfhosted.org> New submission from wuwentao : Hi My python3.7.3 works well in MacOS v10.14.4 yesterday, today I have upgrade my MacOS to v10.14.5, after upgrade,I just found python3 and pip3 can't works, all the command will be run as a background program, but not foreground program. but MacOS builtin python 2.7 works well. so I have reported it to Apply Support for this upgrade bug. in addition, I think I also should report it to python bug tracker. for example: wtwu at wtwu-mbp:~$ ps aux |grep python wtwu 27839 0.0 0.0 4277296 844 s003 S+ 8:41PM 0:00.00 grep python wtwu at wtwu-mbp:~$ python Python 2.7.10 (default, Feb 22 2019, 21:55:15) [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> print "hello world" hello world >>> exit() wtwu at wtwu-mbp:~$ python3 [1]+ Stopped python3 wtwu at wtwu-mbp:~$ Python 3.7.3 (default, Mar 27 2019, 09:23:15) [Clang 10.0.1 (clang-1001.0.46.3)] on darwin Type "help", "copyright", "credits" or "license" for more information. [1]+ Stopped python3 wtwu at wtwu-mbp:~$ ps aux |grep python wtwu 27851 0.0 0.0 4268080 820 s003 S+ 8:41PM 0:00.00 grep python wtwu 27847 0.0 0.1 4263232 8576 s003 T 8:41PM 0:00.04 /usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python wtwu at wtwu-mbp:~$ python3 [2]+ Stopped python3 wtwu at wtwu-mbp:~$ Python 3.7.3 (default, Mar 27 2019, 09:23:15) [Clang 10.0.1 (clang-1001.0.46.3)] on darwin Type "help", "copyright", "credits" or "license" for more information. [2]+ Stopped python3 wtwu at wtwu-mbp:~$ ps aux |grep python wtwu 27857 0.0 0.0 4277296 836 s003 S+ 8:42PM 0:00.00 grep python wtwu 27854 0.0 0.1 4274496 8596 s003 T 8:42PM 0:00.04 /usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python wtwu 27847 0.0 0.1 4263232 8576 s003 T 8:41PM 0:00.04 /usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python wtwu at wtwu-mbp:~$ wtwu at wtwu-mbp:~$ ---------- components: macOS messages: 342570 nosy: ned.deily, ronaldoussoren, wuwentao priority: normal severity: normal status: open title: python3.7.3 can't work after MacOS upgrade from v10.14.4 to v10.14.5 type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 11:46:01 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Wed, 15 May 2019 15:46:01 +0000 Subject: [New-bugs-announce] [issue36926] Implement methoddescr_call without _PyMethodDef_RawFastCallDict Message-ID: <1557935161.99.0.62795791689.issue36926@roundup.psfhosted.org> New submission from Jeroen Demeyer : Once PEP 590 is implemented, it makes sense to focus on using the new "vectorcall" calling convention, which is essentially what is currently FastCallKeywords. To simplify things, it would also be good to use FastCallDict in as few places as possible and actually get rid of that completely. One place where using FastCallKeywords instead of FastCallDict is bad for performance is when using METH_VARARGS|METH_KEYWORDS: the dict would be converted to a tuple and then back to a dict. For builtin_function_or_method instances, there is already special code to deal with METH_VARARGS. But for method_descriptor, there is no such code and this issue fixes that. More precisely, a specialized function is used for implementing calls of method_descriptor with the METH_VARARGS|METH_KEYWORDS signature. Other calls use FastCallKeywords instead, in which case there is no performance loss. ---------- components: Interpreter Core messages: 342579 nosy: Mark.Shannon, jdemeyer, petr.viktorin priority: normal severity: normal status: open title: Implement methoddescr_call without _PyMethodDef_RawFastCallDict versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 13:04:45 2019 From: report at bugs.python.org (Aaron Hall) Date: Wed, 15 May 2019 17:04:45 +0000 Subject: [New-bugs-announce] [issue36927] traceback docstrings should explicitly state return values instead of referring to other functions Message-ID: <1557939885.98.0.0482711491834.issue36927@roundup.psfhosted.org> New submission from Aaron Hall : I've written three (or more) answers on Stack Overflow about how to use the functions in the traceback module, and I code Python all day long. Embarrassing confession: I just recommended the wrong traceback function in email to fix the incorrect usage of another of these functions after pulling up the docs because. I corrected myself before anyone else could correct me, but I find these docstrings incredibly frustrating and problematic. May I please give them a little more verbiage about their return values? e.g.: def format_tb(tb, limit=None): """A shorthand for 'format_list(extract_tb(tb, limit))'.""" return extract_tb(tb, limit=limit).format() should be: def format_tb(tb, limit=None): """A shorthand for 'format_list(extract_tb(tb, limit))', which returns a list of strings ready for printing'. """ return extract_tb(tb, limit=limit).format() In fact, perhaps the "shorthand" part is an implementation detail that may not even be correct (it doesn't immediately seem to be) and should be removed. ---------- assignee: docs at python components: Documentation messages: 342588 nosy: Aaron Hall, docs at python priority: normal severity: normal status: open title: traceback docstrings should explicitly state return values instead of referring to other functions versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 14:11:23 2019 From: report at bugs.python.org (=?utf-8?q?Stefan_H=C3=B6lzl?=) Date: Wed, 15 May 2019 18:11:23 +0000 Subject: [New-bugs-announce] [issue36928] linkt threading.settrace to sys.settrace Message-ID: <1557943883.68.0.641257607459.issue36928@roundup.psfhosted.org> New submission from Stefan H?lzl : The documentation of sys.settrace suggest to call it from every created thread to enable tracing within threads. I would suggest to add a link to threading.settrace which automatically sets a trace function for every by threading module created thread. link to docs: https://docs.python.org/3.5/library/sys.html#sys.settrace https://docs.python.org/3.5/library/threading.html#threading.settrace ---------- assignee: docs at python components: Documentation messages: 342591 nosy: docs at python, stefanhoelzl priority: normal severity: normal status: open title: linkt threading.settrace to sys.settrace versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 14:35:30 2019 From: report at bugs.python.org (Maxwell Bernstein) Date: Wed, 15 May 2019 18:35:30 +0000 Subject: [New-bugs-announce] [issue36929] Other Python _io implementations may not expose _io in their type names Message-ID: <1557945330.73.0.750176002779.issue36929@roundup.psfhosted.org> New submission from Maxwell Bernstein : For a vanishingly small number of internal types, CPython sets the tp_name slot to mod_name.type_name, either in the PyTypeObject or the PyType_Spec. There are a few minor places where this surfaces: * Custom repr functions for those types (some of which ignore the tp_name in favor of using a string literal, such as _io.TextIOWrapper) * Pickling error messages The existing test suite only tests the former. This makes it tricky for other Python implementations to pass the test suite if they do not expose the module name (_io, _ssl, _tkinter, etc) in their type names. ---------- assignee: christian.heimes components: IO, SSL, Tests, Tkinter messages: 342593 nosy: christian.heimes, tekknolagi priority: normal severity: normal status: open title: Other Python _io implementations may not expose _io in their type names type: behavior versions: Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 16:59:59 2019 From: report at bugs.python.org (Immo Wetzel) Date: Wed, 15 May 2019 20:59:59 +0000 Subject: [New-bugs-announce] [issue36930] Windows proxy settings automatically used ... partly Message-ID: <1557953999.04.0.185273095926.issue36930@roundup.psfhosted.org> New submission from Immo Wetzel : I do run python 2.7/3.7 on a windows 7 host. This host is AD managed. As part of the group policy the internet settings are set to automatically proxy configuration. The configuration works. cos IE and Chrome can reach external and internal hosts. Some of the internal are excluded from proxy via no proxy rules inside the pac file. But python request and urllib2 are not able to reach the internal addresses which are excluded via the proxy file rules. Python always tries to get the data from the proxy. the only solution was an extra environmental variable NO_PROXY with the ips not to be reached via the proxy. these excluded IPs are common IPs and not private ! (90.0.0.0/8) is part of the pac exclusion ---------- assignee: terry.reedy components: IDLE, IO, Library (Lib), Windows messages: 342601 nosy: iwetzel, paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware priority: normal severity: normal status: open title: Windows proxy settings automatically used ... partly versions: Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 18:40:51 2019 From: report at bugs.python.org (Justin Rose) Date: Wed, 15 May 2019 22:40:51 +0000 Subject: [New-bugs-announce] [issue36931] json lib doesnt want to load from file Message-ID: <1557960051.39.0.911245380444.issue36931@roundup.psfhosted.org> New submission from Justin Rose : when I run the included file i get an error that looks like: Traceback (most recent call last): File "/home/justin/Desktop/pkmn/main.py", line 10, in expansion = json.load(expan_list) File "/usr/lib/python3.6/json/__init__.py", line 296, in load return loads(fp.read(), AttributeError: 'str' object has no attribute 'read' dont know what to make of it ---------- components: Extension Modules files: main.py messages: 342609 nosy: Justin Rose priority: normal severity: normal status: open title: json lib doesnt want to load from file type: resource usage versions: Python 3.6 Added file: https://bugs.python.org/file48331/main.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 18:40:53 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Wed, 15 May 2019 22:40:53 +0000 Subject: [New-bugs-announce] [issue36932] asyncio-task.rst could use proper deprecated-removed directive Message-ID: <1557960053.75.0.0870422245475.issue36932@roundup.psfhosted.org> New submission from Matthias Bussonnier : SOme place in the documentation do not use the Deprected-remove directive (which is nice as it has a consistent pharing and is easy to search for), and deprecation warnings do not always have the version since deprecation which could be improved. ---------- messages: 342610 nosy: mbussonn priority: normal severity: normal status: open title: asyncio-task.rst could use proper deprecated-removed directive _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 19:11:17 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Wed, 15 May 2019 23:11:17 +0000 Subject: [New-bugs-announce] [issue36933] sys.set_coroutine_wrapper documented as to be removed in 3.8 (still there) Message-ID: <1557961877.88.0.551239006015.issue36933@roundup.psfhosted.org> New submission from Matthias Bussonnier : See issue32591 It was deprecated in 3.7, so maybe removed only in 3.9? ---------- components: Interpreter Core messages: 342615 nosy: mbussonn priority: normal severity: normal status: open title: sys.set_coroutine_wrapper documented as to be removed in 3.8 (still there) versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 15 22:43:21 2019 From: report at bugs.python.org (Farhan Sajjad) Date: Thu, 16 May 2019 02:43:21 +0000 Subject: [New-bugs-announce] [issue36934] C API Function PyLong_AsDouble Returning Wrong Value Message-ID: <1557974601.26.0.0658552402668.issue36934@roundup.psfhosted.org> New submission from Farhan Sajjad : Found this rather obscure behavior where certain 64 bit numbers are changing (probably losing precision somewhere down the call chain) if converted from PyLong to double using the PyLong_AsDouble C API function. TO REPRODUCE: #define __SIZEOF_STRS__ 512 static PyObject* test_pylong(PyObject* self, PyObject* args) { char rBuffer[__SIZEOF_STRS__]; char* strValue; if (!PyArg_ParseTuple(args, "s", &strValue)) return NULL; { printf("%s AS INGRESS\n", strValue); double dblValue = PyLong_AsDouble( PyLong_FromString(strValue, NULL, 10)); snprintf(rBuffer, __SIZEOF_STRS__, "%.0f", PyLong_AsDouble(PyLong_FromString(strValue, NULL, 10))); printf("CONVERT 1: %.0f\nCONVERT 2: %s\n", dblValue, rBuffer); } Py_RETURN_NONE; } Test: >>> test_pylong("1639873214337061279") 1639873214337061279 AS INGRESS CONVERT 1: 1639873214337061376 CONVERT 2: 1639873214337061376 ---------- messages: 342619 nosy: sajjadfx priority: normal severity: normal status: open title: C API Function PyLong_AsDouble Returning Wrong Value type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 01:13:20 2019 From: report at bugs.python.org (Zackery Spytz) Date: Thu, 16 May 2019 05:13:20 +0000 Subject: [New-bugs-announce] [issue36935] bpo-35813 introduced usage of the deprecated PyErr_SetFromWindowsErrWithUnicodeFilename() function Message-ID: <1557983600.93.0.891686339403.issue36935@roundup.psfhosted.org> New submission from Zackery Spytz : In e895de3e7f3cc2f7213b87621cfe9812ea4343f0 / bpo-35813, the deprecated function PyErr_SetFromWindowsErrWithUnicodeFilename() was added in two functions in Modules/_winapi.c. This function was deprecated in 3.3 (and all occurrences of it were removed). Also, if bpo-33407 is accepted, usage of this function will cause compiler warnings. See also bpo-19569. ---------- components: Extension Modules messages: 342626 nosy: ZackerySpytz, davin priority: normal severity: normal status: open title: bpo-35813 introduced usage of the deprecated PyErr_SetFromWindowsErrWithUnicodeFilename() function versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 05:21:33 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Thu, 16 May 2019 09:21:33 +0000 Subject: [New-bugs-announce] [issue36936] CALL_FUNCTION_KW opcode: keyword names must be non-empty Message-ID: <1557998493.78.0.0526035265972.issue36936@roundup.psfhosted.org> New submission from Jeroen Demeyer : Document and add an assertion that the "keyword names" tuple of the CALL_FUNCTION_KW opcode must be non-empty. This is already the case with the current compiler: if there are no keyword arguments in a call, then the CALL_FUNCTION_KW opcode is not used. In light of https://github.com/python/peps/pull/1049 it's good to make this an explicit guarantee. ---------- components: Interpreter Core messages: 342631 nosy: Mark.Shannon, jdemeyer, petr.viktorin, vstinner priority: normal severity: normal status: open title: CALL_FUNCTION_KW opcode: keyword names must be non-empty type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 05:59:07 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Thu, 16 May 2019 09:59:07 +0000 Subject: [New-bugs-announce] [issue36937] New _PyObject_MakeTpCall() function Message-ID: <1558000747.27.0.378828568729.issue36937@roundup.psfhosted.org> New submission from Jeroen Demeyer : Add a new private function PyObject *_PyObject_MakeTpCall(PyObject *callable, PyObject *const *args, Py_ssize_t nargs, PyObject *keywords) to call "callable" using tp_call, but with arguments given using the FastCallKeywords or FastCallDict convention (both are allowed, see also https://github.com/python/peps/pull/1038). The code is not new, it's essentially moving the tp_call case out of _PyObject_FastCallKeywords() and _PyObject_FastCallDict(). This was first proposed (as public API under the name PyCall_MakeTpCall) for PEP 590 but it makes sense to do this independently. ---------- components: Interpreter Core messages: 342632 nosy: Mark.Shannon, jdemeyer, petr.viktorin, vstinner priority: normal severity: normal status: open title: New _PyObject_MakeTpCall() function type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 09:20:46 2019 From: report at bugs.python.org (Khalil) Date: Thu, 16 May 2019 13:20:46 +0000 Subject: [New-bugs-announce] [issue36938] Py_XDECREF on PyUnicodeobject raises SIGSEGV signal Message-ID: <1558012846.53.0.774298096399.issue36938@roundup.psfhosted.org> New submission from Khalil : I Have a set of callbacks from a C extension to a Python code and I noticed that when I report a unicode string to the Python code, and use the Py_XDECREF on it then whole application crashes with the SIGSEGV signal.This is a snippet of the codes: /***************************/ .... PyObject *MyString = PyUnicode_FromString("BlaBla"); PyTuple_SetItem(MyTuple, 0, MyString); PyObject_CallObject(callback, PyTuple); Py_XDECREF(MyString); Py_XDECREF(MyTuple); ... /***********************************/ when I create my string within the set item then it works fine, like below: .... PyTuple_SetItem(MyTuple, 0, PyUnicode_FromString("BlaBla")); PyObject_CallObject(callback, PyTuple); Py_XDECREF(MyTuple); ... ---------- components: Extension Modules messages: 342642 nosy: Khalilmtg priority: normal severity: normal status: open title: Py_XDECREF on PyUnicodeobject raises SIGSEGV signal type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 10:19:44 2019 From: report at bugs.python.org (jack1142) Date: Thu, 16 May 2019 14:19:44 +0000 Subject: [New-bugs-announce] [issue36939] Allow to use shutil.copytree for existing destination directory with optional argument Message-ID: <1558016384.86.0.469064114127.issue36939@roundup.psfhosted.org> New submission from jack1142 : Currently shutil.copytree will allow to copy tree only if destination directory doesn't exist. I think there could be added `exist_ok` keyword argument (defaulting to `False`), which when set to `True` would prevent function from raising `FileExistsError`. This is pretty easy to implement as `os.makedirs` command that raises this error already has `exist_ok` kwarg, which prevent function from raising that exception already. ---------- components: Library (Lib) messages: 342645 nosy: jack1142 priority: normal severity: normal status: open title: Allow to use shutil.copytree for existing destination directory with optional argument type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 11:26:17 2019 From: report at bugs.python.org (STINNER Victor) Date: Thu, 16 May 2019 15:26:17 +0000 Subject: [New-bugs-announce] [issue36940] Update Py_FrozenMain() for _PyCoreConfig (PEP 587) Message-ID: <1558020377.53.0.622883687871.issue36940@roundup.psfhosted.org> New submission from STINNER Victor : Python/frozenmain.c should use pre-initialization and be adapted for _PyCoreConfig. Py_FrozenMain() reimplements some features which are now implemented by _Py_InitializeFromConfig(): * disable C streams (stdin, stdout, stderr) buffering * decode argv using Py_DecodeLocale() * set the program name (call Py_SetProgramName()) * set sys.argv * reimplement the REPL It seems like it could use _Py_RunMain(), but I'm not sure. ---------- components: Interpreter Core messages: 342653 nosy: ncoghlan, steve.dower, twouters, vstinner priority: normal severity: normal status: open title: Update Py_FrozenMain() for _PyCoreConfig (PEP 587) versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 17:34:52 2019 From: report at bugs.python.org (Paul Monson) Date: Thu, 16 May 2019 21:34:52 +0000 Subject: [New-bugs-announce] [issue36941] Windows build changes for Windows ARM64 Message-ID: <1558042492.97.0.749926608424.issue36941@roundup.psfhosted.org> New submission from Paul Monson : Add build file changes for Windows ARM64 ---------- components: Build, Windows messages: 342672 nosy: Paul Monson, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Windows build changes for Windows ARM64 type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 17:37:06 2019 From: report at bugs.python.org (Paul Monson) Date: Thu, 16 May 2019 21:37:06 +0000 Subject: [New-bugs-announce] [issue36942] Windows code changes for Windows ARM64 Message-ID: <1558042626.28.0.817176769827.issue36942@roundup.psfhosted.org> New submission from Paul Monson : Add ifdef changes for Windows ARM64 ---------- components: Windows messages: 342673 nosy: Paul Monson, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Windows code changes for Windows ARM64 type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 17:43:11 2019 From: report at bugs.python.org (Paul Monson) Date: Thu, 16 May 2019 21:43:11 +0000 Subject: [New-bugs-announce] [issue36943] Windows test changes for Windows ARM64 Message-ID: <1558042991.15.0.439006116362.issue36943@roundup.psfhosted.org> New submission from Paul Monson : Add Windows test changes for Windows ARM64 ---------- components: Windows messages: 342674 nosy: Paul Monson, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Windows test changes for Windows ARM64 versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 17:50:56 2019 From: report at bugs.python.org (Paul Monson) Date: Thu, 16 May 2019 21:50:56 +0000 Subject: [New-bugs-announce] [issue36944] W Message-ID: <1558043456.14.0.422986746139.issue36944@roundup.psfhosted.org> Change by Paul Monson : ---------- nosy: Paul Monson priority: normal severity: normal status: open title: W _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 16 19:16:46 2019 From: report at bugs.python.org (STINNER Victor) Date: Thu, 16 May 2019 23:16:46 +0000 Subject: [New-bugs-announce] [issue36945] Add _PyPreConfig.configure_locale: allow to leave LC_CTYPE unchanged when embedding Python Message-ID: <1558048606.99.0.609922696574.issue36945@roundup.psfhosted.org> New submission from STINNER Victor : Py_Initiliaze() always call setlocale(LC_CTYPE, "") on all platforms to set the LC_CTYPE locale to the user preferred locale. That's fine when Py_Main() is used: when Python is the only "owner" of a process. I'm not sure that it's fine when Python is embedded into an application. I propose to add _PyPreConfig.configure_locale: when set to 0, it leaves the LC_CTYPE locale unchanged. The caller is responsible to choose its favorite locale. I'm not sure if it's real issue in practice. Maybe users learnt how to workaround this limitation. By the way, I modified Python to always call setlocale(LC_CTYPE, "") on Windows, bpo-34485: commit 177d921c8c03d30daa32994362023f777624b10d Author: Victor Stinner Date: Wed Aug 29 11:25:15 2018 +0200 bpo-34485, Windows: LC_CTYPE set to user preference (GH-8988) On Windows, the LC_CTYPE is now set to the user preferred locale at startup: _Py_SetLocaleFromEnv(LC_CTYPE) is now called during the Python initialization. Previously, the LC_CTYPE locale was "C" at startup, but changed when calling setlocale(LC_CTYPE, "") or setlocale(LC_ALL, ""). pymain_read_conf() now also calls _Py_SetLocaleFromEnv(LC_CTYPE) to behave as _Py_InitializeCore(). Moreover, it doesn't save/restore the LC_ALL anymore. On Windows, standard streams like sys.stdout now always use surrogateescape error handler by default (ignore the locale). This change caused a performance regression: bpo-35195 "[Windows] Python 3.7 initializes LC_CTYPE locale at startup, causing performance issue on msvcrt isdigit()". Attached PR implements proposed change. ---------- components: Interpreter Core messages: 342677 nosy: ncoghlan, steve.dower, twouters, vstinner priority: normal severity: normal status: open title: Add _PyPreConfig.configure_locale: allow to leave LC_CTYPE unchanged when embedding Python versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 01:48:44 2019 From: report at bugs.python.org (Zackery Spytz) Date: Fri, 17 May 2019 05:48:44 +0000 Subject: [New-bugs-announce] [issue36946] Possible signed integer overflow in slice handling Message-ID: <1558072124.41.0.0531416904304.issue36946@roundup.psfhosted.org> New submission from Zackery Spytz : Python 3.8.0a4+ (heads/master:870b035bc6, May 16 2019, 20:53:02) [GCC 9.0.1 20190402 (experimental) [trunk revision 270074]] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> 'hi'[1::sys.maxsize] Objects/unicodeobject.c:14038:55: runtime error: signed integer overflow: 1 + 9223372036854775807 cannot be represented in type 'long int' 'i' >>> This is because unicode_subscript() performs an extra addition (cur += step) at the end of the for loop (which can overflow). The result of that final addition is not actually used. A patch to fix this issue was posted on #1621, but it seems that the patch has been abandoned. The bug is also described in detail in that issue. I have tweaked the patch and will open a PR. ---------- components: Extension Modules, Interpreter Core messages: 342689 nosy: ZackerySpytz, martin.panter priority: normal severity: normal status: open title: Possible signed integer overflow in slice handling type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 10:27:51 2019 From: report at bugs.python.org (Nicholas Matthews) Date: Fri, 17 May 2019 14:27:51 +0000 Subject: [New-bugs-announce] [issue36947] Fix 3.3.3.1 Metaclasses Documentation Message-ID: <1558103271.07.0.643884881507.issue36947@roundup.psfhosted.org> New submission from Nicholas Matthews : Currently the final sentence of the second paragraph reads: "In the following example, both MyClass and MySubclass are instances of Meta:" It should read something like: "In the following example, both MyClass and MySubclass have the metaclass Meta, and new instances will be created using Meta:" Classes are created by their metaclass, but cannot be said to be instances of their metaclass, correct? ---------- components: Library (Lib) messages: 342723 nosy: Nicholas Matthews priority: normal severity: normal status: open title: Fix 3.3.3.1 Metaclasses Documentation type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 12:09:52 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Fri, 17 May 2019 16:09:52 +0000 Subject: [New-bugs-announce] [issue36948] NameError in urllib.request.URLopener.retrieve Message-ID: <1558109392.44.0.978961343501.issue36948@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : In issue27485 the deprecated functions were replaced with underscore prefixed ones due to which imports where modified. Some of the places where not changed causing NameError in using urllib.request.URLopener.retrieve for local files and non-local files which is deprecated. I found these undefined names while running flake8 on Lib folder. I will raise a PR with tests for the same. Sample Error : ./python.exe Python 3.8.0a4+ (heads/master:870b035bc6, May 17 2019, 16:28:23) [Clang 7.0.2 (clang-700.1.81)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from urllib.request import URLopener >>> URLopener().retrieve('file:///tmp/a.txt') sys:1: DeprecationWarning: URLopener style of invoking requests is deprecated. Use newer urlopen functions/methods Traceback (most recent call last): File "", line 1, in File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/urllib/request.py", line 1786, in retrieve return url2pathname(splithost(url1)[1]), hdrs NameError: name 'splithost' is not defined ---------- components: Library (Lib) messages: 342733 nosy: cheryl.sabella, xtreak priority: normal severity: normal status: open title: NameError in urllib.request.URLopener.retrieve type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 12:46:26 2019 From: report at bugs.python.org (Steve Dower) Date: Fri, 17 May 2019 16:46:26 +0000 Subject: [New-bugs-announce] [issue36949] WeakSet.__repr__ and __str__ do not show contents of the set Message-ID: <1558111586.51.0.375151811101.issue36949@roundup.psfhosted.org> New submission from Steve Dower : This spoils the output of our test suite when there are dangling threads, as the basic "print" doesn't show anything useful. ---------- messages: 342734 nosy: steve.dower priority: normal severity: normal stage: needs patch status: open title: WeakSet.__repr__ and __str__ do not show contents of the set type: enhancement versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 14:21:48 2019 From: report at bugs.python.org (STINNER Victor) Date: Fri, 17 May 2019 18:21:48 +0000 Subject: [New-bugs-announce] [issue36950] test.support: add an helper to wait for an event with a timeout Message-ID: <1558117308.95.0.715352595892.issue36950@roundup.psfhosted.org> New submission from STINNER Victor : The commit cbe72d842646ded2454784679231e3d1e6252e72 is a good example: deadline = time.monotonic() + 60 t = 0.1 while time.monotonic() < deadline: time.sleep(t) t = min(t*2, 5) try: smm = shared_memory.SharedMemory(name, create=False) except FileNotFoundError: break else: raise AssertionError("A SharedMemory segment was leaked after" " a process was abruptly terminated.") It would be nice to convert this code pattern into an helper function in test.support. It's common to have to wait for something in tests. ---------- components: Tests messages: 342751 nosy: vstinner priority: normal severity: normal status: open title: test.support: add an helper to wait for an event with a timeout versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 15:50:54 2019 From: report at bugs.python.org (Kevin Shweh) Date: Fri, 17 May 2019 19:50:54 +0000 Subject: [New-bugs-announce] [issue36951] Wrong types for PyMemberDefs in Objects/typeobject.c Message-ID: <1558122654.3.0.508768199462.issue36951@roundup.psfhosted.org> New submission from Kevin Shweh : In Objects/typeobject.c, the PyMemberDefs for __flags__, __weakrefoffset__, and __dictoffset__ all use T_LONG: {"__flags__", T_LONG, offsetof(PyTypeObject, tp_flags), READONLY}, {"__weakrefoffset__", T_LONG, offsetof(PyTypeObject, tp_weaklistoffset), READONLY}, ... {"__dictoffset__", T_LONG, offsetof(PyTypeObject, tp_dictoffset), READONLY}, {"__weakrefoffset__", T_LONG, offsetof(PyTypeObject, tp_weaklistoffset), READONLY}, but in Include/object.h or Include/cpython/object.h, the corresponding struct members have types unsigned long, Py_ssize_t, and Py_ssize_t respectively: /* Flags to define presence of optional/expanded features */ unsigned long tp_flags; ... /* weak reference enabler */ Py_ssize_t tp_weaklistoffset; ... Py_ssize_t tp_dictoffset; These uses of T_LONG should be changed to T_ULONG and T_PYSSIZE_T. This was checked on 3.7.3 and master. ---------- components: Interpreter Core messages: 342759 nosy: Kevin Shweh priority: normal severity: normal status: open title: Wrong types for PyMemberDefs in Objects/typeobject.c type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 17:25:16 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Fri, 17 May 2019 21:25:16 +0000 Subject: [New-bugs-announce] [issue36952] fileinput input's and Fileinput's bufsize=0 marked for removal in 3.8 Message-ID: <1558128316.3.0.00753032861072.issue36952@roundup.psfhosted.org> New submission from Matthias Bussonnier : Can likely thus be removed now. See also https://bugs.python.org/issue33563 that was tracking better deprecation warnings. If not removed documentation should bump removal to 3.9 ---------- components: Library (Lib) messages: 342769 nosy: mbussonn priority: normal severity: normal status: open title: fileinput input's and Fileinput's bufsize=0 marked for removal in 3.8 versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 20:52:44 2019 From: report at bugs.python.org (Paul Ganssle) Date: Sat, 18 May 2019 00:52:44 +0000 Subject: [New-bugs-announce] [issue36953] Remove collections ABCs? Message-ID: <1558140764.46.0.049733618419.issue36953@roundup.psfhosted.org> New submission from Paul Ganssle : In PR 5640 (https://github.com/python/cpython/pull/5460), a warning was added that importing ABCs from collections directly is deprecated and will be removed in Python 3.8, but they have not yet been removed and the warning is still active. If they are going to be removed, presumably it needs to be done before the feature freeze at the end of May, otherwise, in the meantime, I think the warning needs to be updated to say that it will be removed in Python 3.9. ---------- components: Library (Lib) messages: 342779 nosy: p-ganssle, rhettinger, serhiy.storchaka, stutzbach priority: normal severity: normal status: open title: Remove collections ABCs? versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 17 21:23:46 2019 From: report at bugs.python.org (Gordon P. Hemsley) Date: Sat, 18 May 2019 01:23:46 +0000 Subject: [New-bugs-announce] [issue36954] test_recursive_repr breaks tracing in test_xml_etree Message-ID: <1558142626.27.0.825093654504.issue36954@roundup.psfhosted.org> New submission from Gordon P. Hemsley : When running test_xml_etree with tracing, e.g. when running test coverage, tracing breaks after the execution of test_recursive_repr. ---------- components: Tests messages: 342783 nosy: blueyed, gphemsley, serhiy.storchaka priority: normal pull_requests: 13311 severity: normal status: open title: test_recursive_repr breaks tracing in test_xml_etree versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 18 05:53:58 2019 From: report at bugs.python.org (Frode Byrkjeland) Date: Sat, 18 May 2019 09:53:58 +0000 Subject: [New-bugs-announce] [issue36955] Python3 - mulltiprocessing Message-ID: <1558173238.6.0.665065022806.issue36955@roundup.psfhosted.org> New submission from Frode Byrkjeland : multiprocessing-2.6.2.1.tar.gz contains setup.py with python2 print statement in stead of python3 print( ). This results in an error when trying to install. Found on Ubuntu 18.04 x86_64 running python3.6.7 error seems to be ok after doing changes on the print statements in that file. install cmd: "python3 -m pip install multiprocessing" ---------- messages: 342789 nosy: fbyrkjeland priority: normal severity: normal status: open title: Python3 - mulltiprocessing _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 18 09:53:21 2019 From: report at bugs.python.org (Dan Snider) Date: Sat, 18 May 2019 13:53:21 +0000 Subject: [New-bugs-announce] [issue36956] Calling "functions" used to implement generators/comps easily cause crash Message-ID: <1558187601.17.0.878025070628.issue36956@roundup.psfhosted.org> New submission from Dan Snider : As far as I know, generators, set comprehensions, list comprehensions, and dict comprehensions, (along with their asynchronous variants) are implemented by first calling the GET_(A)ITER opcode and then building and calling a function that acepts the resulting iterator as its sole argument. Assigning the code object used to make that function (or using it in the types.FunctionType constructor) and then calling it with a non-iterator argument will obviously cause a crash since the FOR_ITER opcode rightly expects that it will never have to deal with non-iterators and calls tp_iternext without checking if it exists. The 4-liner demonstrates the crash: if 1: fn = lambda: None gi = (i for i in ()) fn.__code__ = gi.gi_code [*fn("abc")] ---------- messages: 342797 nosy: bup priority: normal severity: normal status: open title: Calling "functions" used to implement generators/comps easily cause crash type: crash versions: Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 18 11:43:46 2019 From: report at bugs.python.org (Mark Dickinson) Date: Sat, 18 May 2019 15:43:46 +0000 Subject: [New-bugs-announce] [issue36957] Speed up math.isqrt Message-ID: <1558194226.75.0.887951661142.issue36957@roundup.psfhosted.org> New submission from Mark Dickinson : The `math.isqrt` algorithm introduce in GH-36887 currently works entirely with Python long integers. That's unnecessarily inefficient for small inputs. For n < 2**64, `math.isqrt(n)` can be computed, via exactly the same algorithm, using entirely C integer arithmetic. For larger n, the first 5 iterations of the algorithm can similarly be performed entirely in C integer arithmetic, and we can then switch to long integer arithmetic for subsequent iterations. On my machine, these simple changes make a substantial difference (4x faster) for small inputs, and a significant but less substantial difference (70% speedup) for inputs not much larger than 2**64. The speedup for huge integers is likely to be much smaller, percentage-wise. Some timings: Unpatched --------- lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt" "[isqrt(n) for n in range(2, 1000)]" 1000 loops, best of 5: 327 usec per loop lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt; x = range(2**63-1000, 2**63+1000)" "[isqrt(n) for n in x]" 200 loops, best of 5: 1.44 msec per loop lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt; x = range(2**95-1000, 2**95+1000)" "[isqrt(n) for n in x]" 200 loops, best of 5: 1.64 msec per loop Patched (PR imminent) ------- lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt" "[isqrt(n) for n in range(2, 1000)]" 5000 loops, best of 5: 78.1 usec per loop lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt; x = range(2**63-1000, 2**63+1000)" "[isqrt(n) for n in x]" 1000 loops, best of 5: 355 usec per loop lovelace:cpython mdickinson$ ./python.exe -m timeit -s "from math import isqrt; x = range(2**95-1000, 2**95+1000)" "[isqrt(n) for n in x]" 500 loops, best of 5: 954 usec per loop ---------- assignee: mark.dickinson components: Extension Modules messages: 342799 nosy: mark.dickinson priority: normal severity: normal stage: needs patch status: open title: Speed up math.isqrt type: performance versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 18 15:38:04 2019 From: report at bugs.python.org (Random832) Date: Sat, 18 May 2019 19:38:04 +0000 Subject: [New-bugs-announce] [issue36958] IDLE should print exit message or status if one is provided Message-ID: <1558208284.57.0.662861536384.issue36958@roundup.psfhosted.org> New submission from Random832 : IDLE currently just returns to the interactive prompt when a script exits, even if SystemExit has arguments. This can be confusing to new users if they are using sys.exit to print a message. ---------- assignee: terry.reedy components: IDLE files: run.py.patch keywords: patch messages: 342809 nosy: Random832, terry.reedy priority: normal severity: normal status: open title: IDLE should print exit message or status if one is provided versions: Python 3.7 Added file: https://bugs.python.org/file48335/run.py.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 18 15:46:43 2019 From: report at bugs.python.org (Gordon P. Hemsley) Date: Sat, 18 May 2019 19:46:43 +0000 Subject: [New-bugs-announce] [issue36959] ISO date errors in _strptime are jumbled Message-ID: <1558208803.84.0.442364090777.issue36959@roundup.psfhosted.org> New submission from Gordon P. Hemsley : This has not been apparent because the tests for this code are not testing what they think they're testing. ---------- components: Library (Lib), Tests messages: 342810 nosy: gphemsley priority: normal severity: normal status: open title: ISO date errors in _strptime are jumbled versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 18 16:17:58 2019 From: report at bugs.python.org (Brad Solomon) Date: Sat, 18 May 2019 20:17:58 +0000 Subject: [New-bugs-announce] [issue36960] Make datetime docs more user-friendly Message-ID: <1558210678.84.0.258939294256.issue36960@roundup.psfhosted.org> New submission from Brad Solomon : The datetime docs are chalk full of detail. This is a positive aspect, and represents a huge amount of work by Tim Peters and A.M. Kuchling. However, it also may function as an obstacle for beginner readers and those simply seeking to answer a basic question or see a straightforward usage example. Rather than seeing an example-based explanation of a common use-case, they are bombarded with technical detail and edge cases. I propose some restructuring of the datetime docs with the goal of making them more reader-friendly. The goal is not to eliminate any of the detail, but to restructure things so as to bring the "everyday" parts into more prominent real estate. The changes here all make an effort to reflect what's espoused by "Documenting Python" at https://devguide.python.org/documenting/. I have some additional changes in mind but wanted to put this here now to gauge receptiveness to the existing changes. ---------- assignee: docs at python components: Documentation messages: 342813 nosy: bsolomon1124, docs at python priority: normal pull_requests: 13321 severity: normal status: open title: Make datetime docs more user-friendly type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 18 18:06:09 2019 From: report at bugs.python.org (Pablo Galindo Salgado) Date: Sat, 18 May 2019 22:06:09 +0000 Subject: [New-bugs-announce] [issue36961] ast_unparser.c doesn't handle PEP570 Message-ID: <1558217169.86.0.894660058597.issue36961@roundup.psfhosted.org> New submission from Pablo Galindo Salgado : Any line in test_annotations() containing positional-only markers in Lib/test/test_future.py will fail: eq('lambda x, /: x') Run tests sequentially 0:00:00 load avg: 0.71 [1/1] test_future test test_future failed -- Traceback (most recent call last): File "/home/pablogsal/github/cpython/Lib/test/test_future.py", line 186, in test_annotations eq("lambda a, /, b, c=True, *vararg, d, e='str', **kwargs: a + b") File "/home/pablogsal/github/cpython/Lib/test/test_future.py", line 141, in assertAnnotationEqual self.assertEqual(actual, expected) AssertionError: "lambda b, c=True, *vararg, d, e='str', **kwargs: a + b" != "lambda a, /, b, c=True, *vararg, d, e='str', **kwargs: a + b" - lambda b, c=True, *vararg, d, e='str', **kwargs: a + b + lambda a, /, b, c=True, *vararg, d, e='str', **kwargs: a + b ? ++++++ test_future failed == Tests result: FAILURE == 1 test failed: test_future Total duration: 46 ms Tests result: FAILURE ---------- assignee: pablogsal components: Interpreter Core messages: 342819 nosy: pablogsal priority: normal severity: normal status: open title: ast_unparser.c doesn't handle PEP570 versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 18 23:18:37 2019 From: report at bugs.python.org (Batuhan) Date: Sun, 19 May 2019 03:18:37 +0000 Subject: [New-bugs-announce] [issue36962] Cant sort ModuleInfo instances Message-ID: <1558235917.88.0.227465723782.issue36962@roundup.psfhosted.org> New submission from Batuhan : I can't sort the result of iter_modules; >>> import random, pkgutil >>> modules = list(pkgutil.iter_modules(None)) >>> random.shuffle(modules) >>> sorted(modules) Traceback (most recent call last): File "", line 1, in TypeError: '<' not supported between instances of 'FileFinder' and 'FileFinder' ---------- components: Library (Lib) messages: 342830 nosy: BTaskaya priority: normal severity: normal status: open title: Cant sort ModuleInfo instances versions: Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 19 00:49:20 2019 From: report at bugs.python.org (Apoorv Reddy) Date: Sun, 19 May 2019 04:49:20 +0000 Subject: [New-bugs-announce] [issue36963] PyDict_GetItem SegFaults on simple dictionary lookup when using Ctypes Message-ID: <1558241360.13.0.269765861936.issue36963@roundup.psfhosted.org> New submission from Apoorv Reddy : I'm trying to use ctypes to speed up an internal function in my project. However, I'm getting a segmentation fault on a simple dictionary lookup in my C Code, on PyDict_GetItem(dict, key). I have supplied a minimal version of the code I'm trying in C, which segfaults, with the Makefile and the Python driver code which loads the shared library.I have created a simple conda environment for Python 3.6 for the same. ---------- components: ctypes files: bug.tar.gz messages: 342834 nosy: apoorvreddy priority: normal severity: normal status: open title: PyDict_GetItem SegFaults on simple dictionary lookup when using Ctypes type: crash versions: Python 3.6 Added file: https://bugs.python.org/file48336/bug.tar.gz _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 19 08:07:19 2019 From: report at bugs.python.org (Marco Sulla) Date: Sun, 19 May 2019 12:07:19 +0000 Subject: [New-bugs-announce] [issue36964] `python3 -m venv NAME`: virtualenv is not portable Message-ID: <1558267639.25.0.0700835222258.issue36964@roundup.psfhosted.org> New submission from Marco Sulla : I'm telling about python3 -m venv VIRTUALENV_NAME, not about the virtualenv binary. Some remarks: 1. `VIRTUAL_ENV` variable in `activate` script is the absolute path of the virtualenv folder 2. A symlink to the `python3` bin of the machine is created. This makes the virtualenv difficult to export to another machine. The VIRTUAL_ENV variable must be manually changed. Furthermore I do not understand why the simlink is created. I suppose that `python3` is already on the `PATH`, so what's the purpose of simlink? I propose to makes VIRTUAL_ENV eqauls to the parent folder of the directory where `activate` resides. It makes it possible to move the virtualenv and copy it to another machine with the same OS. ---------- components: Library (Lib) messages: 342842 nosy: Marco Sulla priority: normal severity: normal status: open title: `python3 -m venv NAME`: virtualenv is not portable type: enhancement versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 19 09:02:26 2019 From: report at bugs.python.org (Erik Janssens) Date: Sun, 19 May 2019 13:02:26 +0000 Subject: [New-bugs-announce] [issue36965] use of STATUS_CONTROL_C_EXIT in Modules/main.c breaks compilation with non MSC compilers Message-ID: <1558270946.43.0.622692216105.issue36965@roundup.psfhosted.org> New submission from Erik Janssens : in Modules/main.c STATUS_CONTROL_C_EXIT is included conditionally if compiling when _MSC_VER is defined. Later on STATUS_CONTROL_C_EXIT is used if MS_WINDOWS is defined. This breaks compilation of main.c with any non MSC compiler while compiling for MS Windows. This appears to have been introduced in GH-12123 for bpo-36142 ---------- components: Windows messages: 342845 nosy: erikjanss, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: use of STATUS_CONTROL_C_EXIT in Modules/main.c breaks compilation with non MSC compilers type: compile error versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 19 10:18:39 2019 From: report at bugs.python.org (Lysandros Nikolaou) Date: Sun, 19 May 2019 14:18:39 +0000 Subject: [New-bugs-announce] [issue36966] Export __VENV_PROMPT__ as environment variable Message-ID: <1558275519.03.0.191785670676.issue36966@roundup.psfhosted.org> New submission from Lysandros Nikolaou : For those, who use custom tools for their terminals, prepending __VENV_PROMPT__ to PS1 doesn't really help all that much, because it gets overwritten by those tools. Would it make sense to export __VENV_PROMPT__ as an environment variable upon activation, so that these tools can make use of it, in order to edit PS1 accordingly? ---------- components: Library (Lib) messages: 342855 nosy: lys.nikolaou priority: normal severity: normal status: open title: Export __VENV_PROMPT__ as environment variable type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 19 17:24:06 2019 From: report at bugs.python.org (Gordon P. Hemsley) Date: Sun, 19 May 2019 21:24:06 +0000 Subject: [New-bugs-announce] [issue36967] Eliminate unnecessary check in _strptime when determining AM/PM Message-ID: <1558301046.43.0.242836736511.issue36967@roundup.psfhosted.org> New submission from Gordon P. Hemsley : Since __calc_am_pm() explicitly limits self.am_pm to 2 values, there are only ever 3 possible values of %p: AM, PM, or blank. Since blank is treated the same as AM, there is only the need to check whether %p is PM. This eliminates an unnecessary comparison and doubly ensures that there is no unhandled case. ---------- messages: 342872 nosy: gphemsley, p-ganssle priority: normal severity: normal status: open title: Eliminate unnecessary check in _strptime when determining AM/PM versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 19 17:25:46 2019 From: report at bugs.python.org (Daniel Law) Date: Sun, 19 May 2019 21:25:46 +0000 Subject: [New-bugs-announce] [issue36968] Top level transient modal windows stopping deiconify on windows 10 Message-ID: <1558301146.73.0.73309397079.issue36968@roundup.psfhosted.org> New submission from Daniel Law : python 3.6 and also found it in 3.7.3, on windows 10. when a main application window built with Tkinter has a modal top level window (transient, grab_set) if it is minimised (which get blocked unless you click show desktop or shake another application around) any and all atempts to reopen or switch to the application again are blocked and the application has to be closed and reopened. gif attached of this in practice. ---------- components: Windows files: arrow_rotate_clockwise_48x48.png messages: 342873 nosy: Daniel Law, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Top level transient modal windows stopping deiconify on windows 10 type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file48337/arrow_rotate_clockwise_48x48.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 19 18:55:08 2019 From: report at bugs.python.org (daniel hahler) Date: Sun, 19 May 2019 22:55:08 +0000 Subject: [New-bugs-announce] [issue36969] pdb: do_args: display/handle keyword-only arguments Message-ID: <1558306508.25.0.885142931529.issue36969@roundup.psfhosted.org> New submission from daniel hahler : With a program like the following, `args` will not display the keyword-only argument: ``` def f1(arg=None, *, kwonly=None): __import__('pdb').set_trace() f1() ``` ``` (Pdb) args arg = 'arg' kw = 'kw' ``` Related code: https://github.com/python/cpython/blob/5c08ce9bf712acbb3f05a3a57baf51fcb534cdf0/Lib/pdb.py#L1129-L1144 ---------- components: Library (Lib) messages: 342878 nosy: blueyed priority: normal severity: normal status: open title: pdb: do_args: display/handle keyword-only arguments type: behavior versions: Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 04:40:40 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Mon, 20 May 2019 08:40:40 +0000 Subject: [New-bugs-announce] [issue36970] Rename _PyObject_FastCall functions Message-ID: <1558341640.29.0.256678092475.issue36970@roundup.psfhosted.org> New submission from Jeroen Demeyer : As preparation for PEP 590, rename _PyObject_FastCallKeywords -> _PyObject_Vectorcall _PyObject_FastCallDict -> _PyObject_VectorcallDict ---------- components: Interpreter Core messages: 342892 nosy: Mark.Shannon, jdemeyer, petr.viktorin priority: normal severity: normal status: open title: Rename _PyObject_FastCall functions type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 11:14:09 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Mon, 20 May 2019 15:14:09 +0000 Subject: [New-bugs-announce] [issue36971] Add subsections in C API "Common Object Structures" page Message-ID: <1558365249.02.0.267009483185.issue36971@roundup.psfhosted.org> New submission from Jeroen Demeyer : The page https://docs.python.org/3/c-api/structures.html could be better structured by arranging the content in sub-sections. ---------- assignee: docs at python components: Documentation messages: 342911 nosy: docs at python, jdemeyer priority: normal severity: normal status: open title: Add subsections in C API "Common Object Structures" page versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 12:09:29 2019 From: report at bugs.python.org (Paul Dagnelie) Date: Mon, 20 May 2019 16:09:29 +0000 Subject: [New-bugs-announce] [issue36972] Add SupportsIndex Message-ID: <1558368569.89.0.17531716566.issue36972@roundup.psfhosted.org> New submission from Paul Dagnelie : In order to allow hex() oct() and bin() to be used on user-defined classes, and to check if they can be used on a class at runtime, a SupportsIndex protocol would be useful to integrate. A PR already exists in the backport repo for this issue: https://github.com/python/typing/pull/630 ---------- components: Library (Lib) messages: 342925 nosy: pcd1193182 priority: normal severity: normal status: open title: Add SupportsIndex type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 13:45:36 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Mon, 20 May 2019 17:45:36 +0000 Subject: [New-bugs-announce] [issue36973] test_json.test_recursion.TestPyRecursion.test_endless_recursion stack overflow in AMD64 Windows8.1 Non-Debug 3.x Message-ID: <1558374336.39.0.468050668046.issue36973@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : test_json results in stack overflow after the commit to implement __repr__ for weakset. This is very much similar to the one consistently occurring on my inspect module PR in Windows tests only for the past two weeks. My change was in changing inspect module and more specifically in importing ast at the top of inspect module. The relevant commit for buildbot failure is in implementing __repr__ for weakset. Buildbot error : https://buildbot.python.org/all/#/builders/12/builds/2497 Error in my PR that is consistent and same : https://ci.appveyor.com/project/python/cpython/builds/24631185#L1547 ---------- components: Tests messages: 342937 nosy: steve.dower, vstinner, xtreak priority: normal severity: normal status: open title: test_json.test_recursion.TestPyRecursion.test_endless_recursion stack overflow in AMD64 Windows8.1 Non-Debug 3.x type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 14:01:27 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Mon, 20 May 2019 18:01:27 +0000 Subject: [New-bugs-announce] [issue36974] Implement PEP 590 Message-ID: <1558375287.14.0.518510919121.issue36974@roundup.psfhosted.org> Change by Jeroen Demeyer : ---------- components: Interpreter Core nosy: Mark.Shannon, jdemeyer, petr.viktorin priority: normal severity: normal status: open title: Implement PEP 590 type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 14:13:43 2019 From: report at bugs.python.org (alter-bug-tracer) Date: Mon, 20 May 2019 18:13:43 +0000 Subject: [New-bugs-announce] [issue36975] csv: undocumented UnicodeDecodeError on malformed file Message-ID: <1558376023.37.0.392859277.issue36975@roundup.psfhosted.org> New submission from alter-bug-tracer : UnicodeDecodeError is thrown instead of csv.Error when parsing malformed inputs. Examples: 1. file0 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 0: invalid continuation byte Traceback (most recent call last): File "csv_parser.py", line 6, in for row in reader: File "/usr/local/lib/python3.8/csv.py", line 111, in __next__ row = next(self.reader) File "/usr/local/lib/python3.8/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) 2. file1 UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 51: invalid start byte Traceback (most recent call last): File "csv_parser.py", line 6, in for row in reader: File "/usr/local/lib/python3.8/csv.py", line 110, in __next__ self.fieldnames File "/usr/local/lib/python3.8/csv.py", line 97, in fieldnames self._fieldnames = next(self.reader) File "/usr/local/lib/python3.8/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) (file0, file1 and csv_parser.py attached) ---------- files: csv.zip messages: 342939 nosy: alter-bug-tracer priority: normal severity: normal status: open title: csv: undocumented UnicodeDecodeError on malformed file type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file48339/csv.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 14:19:58 2019 From: report at bugs.python.org (alter-bug-tracer) Date: Mon, 20 May 2019 18:19:58 +0000 Subject: [New-bugs-announce] [issue36976] email: AttributeError Message-ID: <1558376398.04.0.740342940317.issue36976@roundup.psfhosted.org> New submission from alter-bug-tracer : The 'lower' method is called on a Header object when trying to parse the attached file. Code: import email import sys with open(sys.argv[1], "rb") as f: msg = email.message_from_binary_file(f) print (len(msg)) Traceback: msg = email.message_from_binary_file(f) File "/usr/lib/python3.5/email/__init__.py", line 62, in message_from_binary_file return BytesParser(*args, **kws).parse(fp) File "/usr/lib/python3.5/email/parser.py", line 110, in parse return self.parser.parse(fp, headersonly) File "/usr/lib/python3.5/email/parser.py", line 57, in parse feedparser.feed(data) File "/usr/lib/python3.5/email/feedparser.py", line 178, in feed self._call_parse() File "/usr/lib/python3.5/email/feedparser.py", line 182, in _call_parse self._parse() File "/usr/lib/python3.5/email/feedparser.py", line 322, in _parsegen if (self._cur.get('content-transfer-encoding', '8bit').lower() AttributeError: 'Header' object has no attribute 'lower' ---------- files: file0.zip messages: 342940 nosy: alter-bug-tracer priority: normal severity: normal status: open title: email: AttributeError type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file48340/file0.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 16:00:34 2019 From: report at bugs.python.org (Pierre Glaser) Date: Mon, 20 May 2019 20:00:34 +0000 Subject: [New-bugs-announce] [issue36977] SharedMemoryManager should relase its resources when its parent process dies Message-ID: <1558382434.0.0.963288474875.issue36977@roundup.psfhosted.org> New submission from Pierre Glaser : The new multiprocessing.managers.SharedMemoryManager spawns a server that delivers memory segments to a parent Python process. If the parent process terminates unexpectedly, we should now make the manager process notice this termination it using the recent multiprocessing.parent_process object (that comes with a sentinel), and shut it down. ---------- components: Library (Lib) messages: 342948 nosy: pierreglaser, pitrou priority: normal severity: normal status: open title: SharedMemoryManager should relase its resources when its parent process dies _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 17:08:17 2019 From: report at bugs.python.org (Marco Sulla) Date: Mon, 20 May 2019 21:08:17 +0000 Subject: [New-bugs-announce] [issue36978] `python3 -m pip install` has no `--requirement` option on Windows Message-ID: <1558386497.6.0.534415104881.issue36978@roundup.psfhosted.org> New submission from Marco Sulla : It's really useful and easy to have a requirements.txt. It integrates also with Github, that tells you if you're specifying a version of the library with security issues. I don't understand why this flag is missing in Windows builds. It seems to me not too much difficult to implement. Please? ^^ ---------- components: Library (Lib) messages: 342955 nosy: Marco Sulla priority: normal severity: normal status: open title: `python3 -m pip install` has no `--requirement` option on Windows versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 17:48:43 2019 From: report at bugs.python.org (Chris Hargreaves) Date: Mon, 20 May 2019 21:48:43 +0000 Subject: [New-bugs-announce] [issue36979] ncurses extension uses wrong include path Message-ID: <1558388923.16.0.697144008722.issue36979@roundup.psfhosted.org> New submission from Chris Hargreaves : This is similar to: https://bugs.python.org/issue28190 Not cross-compiling, but using a different ncurses version than is provided under /usr/include/ncursesw Specifying CPPFLAGS to have "-I/path/to/ncurses/include" does not override the "/usr/include/ncursesw" in setup.py if curses_library == 'ncursesw': curses_defines.append(('HAVE_NCURSESW', '1')) if not cross_compiling: curses_includes.append('/usr/include/ncursesw') Python 2.7.x does not have this issue, but 3.6.x and 3.7.x do. 2 and 3 have different ways of setting up the include path for curses when building the extension. In my case, removing the curses_include.append from setup.py results in a working extension. It probably makes sense that Extension(include_dirs=) take priority over Python build CPPFLAGS, setup.py may need to be more cautious about adding the ncurses include path. Only tested in 2.7, 3.6, 3.7. ---------- components: Build, Library (Lib) messages: 342957 nosy: chargr priority: normal severity: normal status: open title: ncurses extension uses wrong include path type: compile error versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 20:31:29 2019 From: report at bugs.python.org (stefan) Date: Tue, 21 May 2019 00:31:29 +0000 Subject: [New-bugs-announce] [issue36980] pass-by-reference clues Message-ID: <1558398689.74.0.962126355823.issue36980@roundup.psfhosted.org> New submission from stefan : I often get unexpected results when a called function results in a change in a variable because the function gets a pass by reference. For example, consider this snippet of code that manipulates the first column of a 3x3 matrix that it gets. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ import numpy as np def changeValue(kernel): kernel[0,0]=kernel[0,0]+ 2 kernel[1,0]=kernel[1,0]+ 2 kernel[2,0]=kernel[2,0]+ 2 return kernel myKernel = np.array(( [0, -1, 0], [-1, 5, -1], [0, -1, 0]), dtype="int") CVkernel=myKernel print(CVkernel) a=changeValue(myKernel) print(a) print(CVkernel) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I get the following output [[ 0 -1 0] [-1 5 -1] [ 0 -1 0]] [[ 2 -1 0] [ 1 5 -1] [ 2 -1 0]] [[ 2 -1 0] [ 1 5 -1] [ 2 -1 0]] The value of myKernel clobbers CVkernel. I think there is an unintentional call-by-reference (pass-by-reference?) going on but I am not sure why. If I define the function slightly differently def changeValue2(kernel): kernel=kernel + 2 return kernel Then CVkernel is left untouched [[ 0 -1 0] [-1 5 -1] [ 0 -1 0]] [[2 1 2] [1 7 1] [2 1 2]] [[ 0 -1 0] [-1 5 -1] [ 0 -1 0]] What is going on here? EDIT Even when I use a 'safe' function call that does not clobber CVkernel, like kernel=kernel + 2 , the id of myKernel and CVkernel are the same. id of myKernel 139994865303344 myKernel [[ 0 -1 0] [-1 5 -1] [ 0 -1 0]] id of CVKernel 139994865303344 CVKernel [[ 0 -1 0] [-1 5 -1] [ 0 -1 0]] **call made to changeValue2** id of myKernel 139994865303344 myKernel [[ 0 -1 0] [-1 5 -1] [ 0 -1 0]] id of CVKernel 139994865303344 CVKernel [[ 0 -1 0] [-1 5 -1] [ 0 -1 0]] output a [[2 1 2] [1 7 1] [2 1 2]] Shouldn't the id of each variable be different if they are different instances? Would it possible for the python interpreter/compiler to let me know when a function is going to clobber a variable that is not used in the function or passed to the function or returned by the function ---------- messages: 342967 nosy: skypickle priority: normal severity: normal status: open title: pass-by-reference clues type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 22:23:49 2019 From: report at bugs.python.org (viocal) Date: Tue, 21 May 2019 02:23:49 +0000 Subject: [New-bugs-announce] [issue36981] asyncio transport.write() memory out Message-ID: <1558405429.22.0.902415488536.issue36981@roundup.psfhosted.org> New submission from viocal : in asyncio when filedata than free memory(hardware) will be memory out Or killed by OS for buf in filedata: transport.write(buf) #to client I try it todo: abort transporting to protect application be killed by OS modified selector_events.py def _write_ready(self): assert self._buffer, 'Data should not be empty' if self._conn_lost: return try: n = self._sock.send(self._buffer) except (BlockingIOError, InterruptedError): pass except Exception as exc: self._loop._remove_writer(self._sock_fd) self._buffer.clear() self._fatal_error(exc, 'Fatal write error on socket transport') if self._empty_waiter is not None: self._empty_waiter.set_exception(exc) return else: try: if n: del self._buffer[:n] self._maybe_resume_protocol() # May append to buffer. if not self._buffer: self._loop._remove_writer(self._sock_fd) if self._empty_waiter is not None: self._empty_waiter.set_result(None) if self._closing: self._call_connection_lost(None) elif self._eof: self._sock.shutdown(socket.SHUT_WR) except Exception as exc: #(MemoryError) self._buffer.clear() self._loop._remove_writer(self._sock_fd) self._fatal_error(exc, 'Fatal write error on Selector SocketTransport write ready') if self._empty_waiter is not None: self._empty_waiter.set_exception(exc) return ---------- components: asyncio messages: 342973 nosy: asvetlov, viocal, yselivanov priority: normal severity: normal status: open title: asyncio transport.write() memory out type: resource usage versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 22:43:51 2019 From: report at bugs.python.org (Jeffrey Kintscher) Date: Tue, 21 May 2019 02:43:51 +0000 Subject: [New-bugs-announce] [issue36982] Add support for extended color functions in ncurses 6.1 Message-ID: <1558406631.2.0.0939623901621.issue36982@roundup.psfhosted.org> New submission from Jeffrey Kintscher : ncurses 6.1 adds extended color functions to support terminals with 256 colors (e.g. xterm-256color). The extended functions take color pair values that are signed integers because the existing functions only take signed 16-bit values. My goal with this issue is to transparently use the ncurses extended color functions when compiling with ncurses 6.1 or newer. This should be straightforward and transparent to curses module users because the short int restrictions are in the ncurses library and not in the curses module API. This will fix the problems observed in issue #36630 but is broader, which is why I created a separete issue. I will work on this and post a PR whit it is ready. ---------- components: Library (Lib) messages: 342974 nosy: websurfer5 priority: normal severity: normal status: open title: Add support for extended color functions in ncurses 6.1 versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 20 23:40:16 2019 From: report at bugs.python.org (Anthony Sottile) Date: Tue, 21 May 2019 03:40:16 +0000 Subject: [New-bugs-announce] [issue36983] typing.__all__ has drifted from actual contents Message-ID: <1558410016.34.0.625577465387.issue36983@roundup.psfhosted.org> New submission from Anthony Sottile : notably it is missing ForwardRef, OrderedDict, ChainMap in python3.8 it is missing others in other versions I'm going to attempt to write a test which should classify things that should belong there ---------- components: Library (Lib) messages: 342975 nosy: Anthony Sottile priority: normal severity: normal status: open title: typing.__all__ has drifted from actual contents versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 01:20:35 2019 From: report at bugs.python.org (Anthony Sottile) Date: Tue, 21 May 2019 05:20:35 +0000 Subject: [New-bugs-announce] [issue36984] typing docs "versionadded" is inaccurate for many attributes Message-ID: <1558416035.31.0.248938950481.issue36984@roundup.psfhosted.org> New submission from Anthony Sottile : expanding on https://bugs.python.org/issue36983 the docs are also a bit out of date in places I'm not sure how to document something that appeared in two versions, but I'll leave that part to review. Using data generated / collected in https://github.com/asottile/flake8-typing-imports via this script: https://github.com/asottile/flake8-typing-imports/blob/master/bin/build-generated and then analyzed with this script: from flake8_typing_imports import SYMBOLS ALL = set().union(*(v for _, v in SYMBOLS)) for k in sorted(ALL): state = False for i, (v, ks) in enumerate(SYMBOLS): if state is False and k in ks: print(f'{k}: new in {v}') state = True elif state is True and k not in ks: print(f'=> {k}: removed in {v}') state = False I've found the following: $ python3 t.py | grep -v 'new in 3\.5\.0' AsyncContextManager: new in 3.5.4 => AsyncContextManager: removed in 3.6.0 AsyncContextManager: new in 3.6.2 AsyncGenerator: new in 3.6.1 AsyncIterable: new in 3.5.2 AsyncIterator: new in 3.5.2 Awaitable: new in 3.5.2 ChainMap: new in 3.5.4 => ChainMap: removed in 3.6.0 ChainMap: new in 3.6.1 ClassVar: new in 3.5.3 Collection: new in 3.6.0 ContextManager: new in 3.5.4 Coroutine: new in 3.5.3 Counter: new in 3.5.4 => Counter: removed in 3.6.0 Counter: new in 3.6.1 DefaultDict: new in 3.5.2 Deque: new in 3.5.4 => Deque: removed in 3.6.0 Deque: new in 3.6.1 ForwardRef: new in 3.7.0 GenericMeta: new in 3.5.4 => GenericMeta: removed in 3.6.0 GenericMeta: new in 3.6.1 => GenericMeta: removed in 3.7.0 NewType: new in 3.5.2 NoReturn: new in 3.5.4 => NoReturn: removed in 3.6.0 NoReturn: new in 3.6.2 OrderedDict: new in 3.7.2 TYPE_CHECKING: new in 3.5.2 Text: new in 3.5.2 Type: new in 3.5.2 ---------- assignee: docs at python components: Documentation messages: 342978 nosy: Anthony Sottile, docs at python priority: normal severity: normal status: open title: typing docs "versionadded" is inaccurate for many attributes versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 01:35:02 2019 From: report at bugs.python.org (Anthony Sottile) Date: Tue, 21 May 2019 05:35:02 +0000 Subject: [New-bugs-announce] [issue36985] typing.ForwardRef is undocumented Message-ID: <1558416902.0.0.868432422236.issue36985@roundup.psfhosted.org> New submission from Anthony Sottile : - New since 3.7.0 - is exposed by some public apis: - the `__doc__` attribute of `typing` mentions `ForwardRef` - `get_type_hints` can expose an instance of it: https://bugs.python.org/issue35834 Should this be documented? including in __all__? ---------- assignee: docs at python components: Documentation messages: 342979 nosy: Anthony Sottile, docs at python priority: normal severity: normal status: open title: typing.ForwardRef is undocumented versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 03:13:29 2019 From: report at bugs.python.org (alter-bug-tracer) Date: Tue, 21 May 2019 07:13:29 +0000 Subject: [New-bugs-announce] [issue36986] tarfile: unexpected IsADirectoryError on extraction Message-ID: <1558422809.19.0.381089325981.issue36986@roundup.psfhosted.org> New submission from alter-bug-tracer : The following code creates a new directory 'output' and extracts the tar archive given as argv in it. When a malformed archive (attached) is given as argv, a IsADirectoryError is thrown, as opposed to extracting the file contained in the archive to the 'output' directory. Code: import tarfile import sys import os def extract(archive_path, destination_path): if not tarfile.is_tarfile(archive_path): print ('Not a tar file') return tar = tarfile.open(archive_path, 'r') for info in tar.getmembers(): tar.extractall(destination_path, members=[tar.getmember(info.name)]) destination_path = 'output' os.mkdir(destination_path) extract(sys.argv[1], destination_path) Result: Traceback (most recent call last): File "code.py", line 15, in extract(sys.argv[1], destination_path) File "code.py", line 11, in extract tar.extractall(destination_path, members=[tar.getmember(info.name)]) File "/usr/lib/python3.6/tarfile.py", line 2010, in extractall numeric_owner=numeric_owner) File "/usr/lib/python3.6/tarfile.py", line 2052, in extract numeric_owner=numeric_owner) File "/usr/lib/python3.6/tarfile.py", line 2122, in _extract_member self.makefile(tarinfo, targetpath) File "/usr/lib/python3.6/tarfile.py", line 2163, in makefile with bltn_open(targetpath, "wb") as target: IsADirectoryError: [Errno 21] Is a directory: 'output' ---------- files: input0.tar messages: 342983 nosy: alter-bug-tracer priority: normal severity: normal status: open title: tarfile: unexpected IsADirectoryError on extraction type: behavior versions: Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48341/input0.tar _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 04:26:26 2019 From: report at bugs.python.org (Aprila Hijriyan) Date: Tue, 21 May 2019 08:26:26 +0000 Subject: [New-bugs-announce] [issue36987] Dictionary: why is the value not used up? Message-ID: <1558427186.18.0.280662409984.issue36987@roundup.psfhosted.org> New submission from Aprila Hijriyan : Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15) [GCC 7.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> d = {"a": 1, "b": 2} >>> d ["c"] = d >>> d ["c"] {'a': 1, 'c': {...}, 'b': 2} >>> d ["c"] ["c"] {'a': 1, 'c': {...}, 'b': 2} why does the key value 'c' have a 'c' key in it? ---------- messages: 342984 nosy: Aprila Hijriyan priority: normal severity: normal status: open title: Dictionary: why is the value not used up? versions: Python 2.7, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 06:49:55 2019 From: report at bugs.python.org (alter-bug-tracer) Date: Tue, 21 May 2019 10:49:55 +0000 Subject: [New-bugs-announce] [issue36988] zipfile: string IndexError on extract Message-ID: <1558435795.93.0.150119411648.issue36988@roundup.psfhosted.org> New submission from alter-bug-tracer : The following code throws an IndexError when attempting to extract a malformed archive (attached): import zipfile import sys zf = zipfile.ZipFile(sys.argv[1]) for info in zf.infolist(): zf.extract(info.filename) Result: Traceback (most recent call last): File "code.py", line 6, in zf.extract(info.filename) File "/usr/lib/python3.6/zipfile.py", line 1507, in extract return self._extract_member(member, path, pwd) File "/usr/lib/python3.6/zipfile.py", line 1572, in _extract_member if member.is_dir(): File "/usr/lib/python3.6/zipfile.py", line 531, in is_dir return self.filename[-1] == '/' IndexError: string index out of range ---------- files: file0.zip messages: 343006 nosy: alter-bug-tracer priority: normal severity: normal status: open title: zipfile: string IndexError on extract type: behavior versions: Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48345/file0.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 07:35:53 2019 From: report at bugs.python.org (Michael Felt) Date: Tue, 21 May 2019 11:35:53 +0000 Subject: [New-bugs-announce] [issue36989] test_thread fails because symbol is (no longer) exported Message-ID: <1558438553.77.0.394995800321.issue36989@roundup.psfhosted.org> New submission from Michael Felt : On AIX, with commit 4fb15021890d327023aefd95f5a84ac33b037d19 (HEAD -> master, origin/master, origin/HEAD) test_thread is failing. The three sub_tests that exit as ERROR are: ERROR: test_threads_join_2 (test.test_threading.SubinterpThreadingTests) ERROR: test_frame_tstate_tracing (test.test_threading.ThreadTests) These two have in common: Traceback (most recent call last): ... ImportError: 0509-130 Symbol resolution failed for /home/buildbot/python-master/build/lib.aix-7.1-3.8-pydebug/_testcapi.so because: 0509-136 Symbol _PyMem_GetAllocatorsName (number 191) is not exported from dependent module python. 0509-192 Examine .loader section symbols with the 'dump -Tv' command. FAIL: test_daemon_threads_fatal_error (test.test_threading.SubinterpThreadingTests) I am guessing that: ====================================================================== FAIL: test_daemon_threads_fatal_error (test.test_threading.SubinterpThreadingTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildbot/python-master/Lib/test/test_threading.py", line 942, in test_daemon_threads_fatal_error self.assertIn("Fatal Python error: Py_EndInterpreter: " AssertionError: 'Fatal Python error: Py_EndInterpreter: not the last thread' not found in 'Traceback (most recent call last):\n File "", line 2, in \nImportError: \t0509-130 Symbol resolution failed for /home/buildbot/python-master/build/lib.aix-7.1-3.8-pydebug/_testcapi.so because:\n\t0509-136 Symbol _PyMem_GetAllocatorsName (number 191) is not exported from\n\t\t dependent module python.\n\t0509-192 Examine .loader section symbols with the\n\t\t \'dump -Tv\' command.' ---------------------------------------------------------------------- is caused by the earlier failures. ---------- components: Build, Tests messages: 343016 nosy: Michael.Felt priority: normal severity: normal status: open title: test_thread fails because symbol is (no longer) exported versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 07:56:33 2019 From: report at bugs.python.org (Michael Felt) Date: Tue, 21 May 2019 11:56:33 +0000 Subject: [New-bugs-announce] [issue36990] test_asyncio.test_create_connection_ipv6_scope fails(in mock test?) Message-ID: <1558439793.15.0.664756328285.issue36990@roundup.psfhosted.org> New submission from Michael Felt : ====================================================================== test_create_connection_ipv6_scope (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildbot/python-master/Lib/unittest/mock.py", line 1226, in patched return func(*args, **keywargs) File "/home/buildbot/python-master/Lib/test/test_asyncio/test_base_events.py", line 1316, in test_create_connection_ipv6_scope sock.connect.assert_called_with(('fe80::1', 80, 0, 1)) File "/home/buildbot/python-master/Lib/unittest/mock.py", line 838, in assert_called_with raise AssertionError(_error_message()) from cause AssertionError: expected call not found. Expected: connect(('fe80::1', 80, 0, 1)) Actual: connect(('fe80::1', 80, 0, 0)) More details: buildbot at x064:[/home/buildbot/python-master]nohup ./python -m test -v test_asyncio | egrep -v "ok$" | grep -v " ... skipped " == CPython 3.8.0a4+ (heads/master:4fb1502189, May 21 2019, 11:08:13) [GCC 4.7.4] == AIX-1-00C291F54C00-powerpc-32bit big-endian == cwd: /home/buildbot/python-master/build/test_python_17694732 == CPU count: 4 == encodings: locale=ISO8859-1, FS=iso8859-1 Run tests sequentially 0:00:00 [1/1] test_asyncio test_create_connection_ipv6_scope (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... FAIL test_communicate_ignore_broken_pipe (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) ... /home/buildbot/python-master/Lib/inspect.py:2819: RuntimeWarning: coroutine 'AsyncMockMixin._mock_call' was never awaited params = OrderedDict(((param.name, param) RuntimeWarning: Enable tracemalloc to get the object allocation traceback Future exception was never retrieved future: Traceback (most recent call last): File "/home/buildbot/python-master/Lib/asyncio/subprocess.py", line 162, in _feed_stdin await self.stdin.drain() File "/home/buildbot/python-master/Lib/asyncio/streams.py", line 443, in drain await self._protocol._drain_helper() File "/home/buildbot/python-master/Lib/asyncio/streams.py", line 200, in _drain_helper await waiter File "/home/buildbot/python-master/Lib/asyncio/unix_events.py", line 661, in _write_ready n = os.write(self._fileno, self._buffer) BrokenPipeError: [Errno 32] Broken pipe test_communicate_ignore_broken_pipe (test.test_asyncio.test_subprocess.SubprocessSafeWatcherTests) ... Future exception was never retrieved future: Traceback (most recent call last): File "/home/buildbot/python-master/Lib/asyncio/subprocess.py", line 162, in _feed_stdin await self.stdin.drain() File "/home/buildbot/python-master/Lib/asyncio/streams.py", line 443, in drain await self._protocol._drain_helper() File "/home/buildbot/python-master/Lib/asyncio/streams.py", line 200, in _drain_helper await waiter File "/home/buildbot/python-master/Lib/asyncio/unix_events.py", line 661, in _write_ready n = os.write(self._fileno, self._buffer) BrokenPipeError: [Errno 32] Broken pipe test_cancel_at_end (test.test_asyncio.test_tasks.CTaskSubclass_PyFuture_Tests) test_cancel_gather_1 (test.test_asyncio.test_tasks.CTaskSubclass_PyFuture_Tests) test_return_coroutine_from_coroutine (test.test_asyncio.test_tasks.CTaskSubclass_PyFuture_Tests) test_cancel_at_end (test.test_asyncio.test_tasks.CTask_CFuture_SubclassTests) test_cancel_gather_1 (test.test_asyncio.test_tasks.CTask_CFuture_SubclassTests) test_return_coroutine_from_coroutine (test.test_asyncio.test_tasks.CTask_CFuture_SubclassTests) test_cancel_at_end (test.test_asyncio.test_tasks.CTask_CFuture_Tests) test_cancel_gather_1 (test.test_asyncio.test_tasks.CTask_CFuture_Tests) test_return_coroutine_from_coroutine (test.test_asyncio.test_tasks.CTask_CFuture_Tests) test_cancel_at_end (test.test_asyncio.test_tasks.CTask_PyFuture_Tests) test_cancel_gather_1 (test.test_asyncio.test_tasks.CTask_PyFuture_Tests) test_return_coroutine_from_coroutine (test.test_asyncio.test_tasks.CTask_PyFuture_Tests) test_cancel_at_end (test.test_asyncio.test_tasks.PyTask_CFutureSubclass_Tests) test_cancel_gather_1 (test.test_asyncio.test_tasks.PyTask_CFutureSubclass_Tests) test_return_coroutine_from_coroutine (test.test_asyncio.test_tasks.PyTask_CFutureSubclass_Tests) test_cancel_at_end (test.test_asyncio.test_tasks.PyTask_CFuture_Tests) test_cancel_gather_1 (test.test_asyncio.test_tasks.PyTask_CFuture_Tests) test_return_coroutine_from_coroutine (test.test_asyncio.test_tasks.PyTask_CFuture_Tests) test_cancel_at_end (test.test_asyncio.test_tasks.PyTask_PyFuture_SubclassTests) test_cancel_gather_1 (test.test_asyncio.test_tasks.PyTask_PyFuture_SubclassTests) test_return_coroutine_from_coroutine (test.test_asyncio.test_tasks.PyTask_PyFuture_SubclassTests) test_cancel_at_end (test.test_asyncio.test_tasks.PyTask_PyFuture_Tests) test_cancel_gather_1 (test.test_asyncio.test_tasks.PyTask_PyFuture_Tests) test_return_coroutine_from_coroutine (test.test_asyncio.test_tasks.PyTask_PyFuture_Tests) test_run_coroutine_threadsafe (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) test_run_coroutine_threadsafe_task_cancelled (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) test_run_coroutine_threadsafe_task_factory_exception (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) test_run_coroutine_threadsafe_with_exception (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) test_run_coroutine_threadsafe_with_timeout (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) test.test_asyncio.test_windows_utils (unittest.loader.ModuleSkipped) ... test test_asyncio failed skipped 'Windows only' ====================================================================== FAIL: test_create_connection_ipv6_scope (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildbot/python-master/Lib/unittest/mock.py", line 1226, in patched return func(*args, **keywargs) File "/home/buildbot/python-master/Lib/test/test_asyncio/test_base_events.py", line 1316, in test_create_connection_ipv6_scope sock.connect.assert_called_with(('fe80::1', 80, 0, 1)) File "/home/buildbot/python-master/Lib/unittest/mock.py", line 838, in assert_called_with raise AssertionError(_error_message()) from cause AssertionError: expected call not found. Expected: connect(('fe80::1', 80, 0, 1)) Actual: connect(('fe80::1', 80, 0, 0)) ---------------------------------------------------------------------- Ran 1977 tests in 70.591s FAILED (failures=1, skipped=48) test_asyncio failed in 1 min 10 sec == Tests result: FAILURE == 1 test failed: test_asyncio ---------- components: Tests messages: 343019 nosy: Michael.Felt priority: normal severity: normal status: open title: test_asyncio.test_create_connection_ipv6_scope fails(in mock test?) versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 08:26:42 2019 From: report at bugs.python.org (alter-bug-tracer) Date: Tue, 21 May 2019 12:26:42 +0000 Subject: [New-bugs-announce] [issue36991] zipfile: AttributeError on extract Message-ID: <1558441602.37.0.911226270551.issue36991@roundup.psfhosted.org> New submission from alter-bug-tracer : The following code throws an AttributeError when attempting to extract a malformed archive (attached): import zipfile import sys zf = zipfile.ZipFile(sys.argv[1]) for info in zf.infolist(): zf.extract(info.filename) Result: Traceback (most recent call last): File "code.py", line 6, in zf.extract(info.filename) File "/usr/local/lib/python3.8/zipfile.py", line 1607, in extract return self._extract_member(member, path, pwd) File "/usr/local/lib/python3.8/zipfile.py", line 1677, in _extract_member with self.open(member, pwd=pwd) as source, \ File "/usr/local/lib/python3.8/zipfile.py", line 1548, in open return ZipExtFile(zef_file, mode, zinfo, zd, True) File "/usr/local/lib/python3.8/zipfile.py", line 801, in __init__ self._decompressor = _get_decompressor(self._compress_type) File "/usr/local/lib/python3.8/zipfile.py", line 708, in _get_decompressor return bz2.BZ2Decompressor() AttributeError: 'NoneType' object has no attribute 'BZ2Decompressor' ---------- files: attr0.zip messages: 343035 nosy: alter-bug-tracer priority: normal severity: normal status: open title: zipfile: AttributeError on extract type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file48346/attr0.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 08:29:31 2019 From: report at bugs.python.org (alter-bug-tracer) Date: Tue, 21 May 2019 12:29:31 +0000 Subject: [New-bugs-announce] [issue36992] zipfile: AttributeError on extract (LZMA) Message-ID: <1558441771.7.0.234228276725.issue36992@roundup.psfhosted.org> New submission from alter-bug-tracer : The following code throws an AttributeError when attempting to extract a malformed archive (attached): import zipfile import sys zf = zipfile.ZipFile(sys.argv[1]) for info in zf.infolist(): zf.extract(info.filename) Result: Traceback (most recent call last): File "code.py", line 6, in zf.extract(info.filename) File "/usr/local/lib/python3.8/zipfile.py", line 1607, in extract return self._extract_member(member, path, pwd) File "/usr/local/lib/python3.8/zipfile.py", line 1679, in _extract_member shutil.copyfileobj(source, target) File "/usr/local/lib/python3.8/shutil.py", line 198, in copyfileobj buf = fsrc_read(length) File "/usr/local/lib/python3.8/zipfile.py", line 903, in read data = self._read1(n) File "/usr/local/lib/python3.8/zipfile.py", line 986, in _read1 data = self._decompressor.decompress(data) File "/usr/local/lib/python3.8/zipfile.py", line 635, in decompress self._decomp = lzma.LZMADecompressor(lzma.FORMAT_RAW, filters=[ AttributeError: 'NoneType' object has no attribute 'LZMADecompressor' ---------- files: attr1.zip messages: 343036 nosy: alter-bug-tracer priority: normal severity: normal status: open title: zipfile: AttributeError on extract (LZMA) type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file48347/attr1.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 08:32:49 2019 From: report at bugs.python.org (alter-bug-tracer) Date: Tue, 21 May 2019 12:32:49 +0000 Subject: [New-bugs-announce] [issue36993] zipfile: tuple IndexError on extract Message-ID: <1558441969.28.0.928640082083.issue36993@roundup.psfhosted.org> New submission from alter-bug-tracer : The following code throws an IndexError when attempting to extract a malformed archive (attached): import zipfile import sys zf = zipfile.ZipFile(sys.argv[1]) for info in zf.infolist(): zf.extract(info.filename) Result: Traceback (most recent call last): File "code.py", line 4, in zf = zipfile.ZipFile(sys.argv[1]) File "/usr/local/lib/python3.8/zipfile.py", line 1230, in __init__ self._RealGetContents() File "/usr/local/lib/python3.8/zipfile.py", line 1353, in _RealGetContents x._decodeExtra() File "/usr/local/lib/python3.8/zipfile.py", line 480, in _decodeExtra self.file_size = counts[idx] IndexError: tuple index out of range ---------- files: index_tuple.zip messages: 343038 nosy: alter-bug-tracer priority: normal severity: normal status: open title: zipfile: tuple IndexError on extract versions: Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48348/index_tuple.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 09:05:13 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Tue, 21 May 2019 13:05:13 +0000 Subject: [New-bugs-announce] [issue36994] Missing tests for CALL_FUNCTION_EX opcode profiling method_descriptor Message-ID: <1558443913.24.0.484084553797.issue36994@roundup.psfhosted.org> New submission from Jeroen Demeyer : The branch for profiling a method_descriptor inside do_call_core is not tested by the test suite. ---------- components: Tests messages: 343044 nosy: jdemeyer priority: normal severity: normal status: open title: Missing tests for CALL_FUNCTION_EX opcode profiling method_descriptor versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 10:53:24 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Tue, 21 May 2019 14:53:24 +0000 Subject: [New-bugs-announce] [issue36995] tp_print -> tp_vectorcall_offset Message-ID: <1558450404.41.0.511752439541.issue36995@roundup.psfhosted.org> New submission from Jeroen Demeyer : If PEP 590 is accepted: in code comments, replace tp_print by tp_vectorcall_offset and (while we're at it) tp_reserved by tp_as_async. ---------- components: Interpreter Core messages: 343061 nosy: Mark.Shannon, jdemeyer, petr.viktorin priority: normal severity: normal status: open title: tp_print -> tp_vectorcall_offset versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 11:22:08 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Tue, 21 May 2019 15:22:08 +0000 Subject: [New-bugs-announce] [issue36996] unittest.mock.patch decorator doesn't work with async functions Message-ID: <1558452128.6.0.303675531512.issue36996@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : I came across this while using AsyncMock but it seems to apply to Mock/MagicMock too. When patch decorator is used to wrap an async function to mock sync or async functions it doesn't seem to work. Manually adding patcher or using patch as context manager works. Meanwhile sync_main which is not an async function works fine. Is this an expected behavior with @patch and async def? Does evaluating an async function with asyncio.run has any effect on this? Debugging it tells me the correct object is being replaced with AsyncMock inside patch. import asyncio from unittest.mock import patch, AsyncMock mock = AsyncMock() async def foo(): pass def bar(): pass @patch(f"{__name__}.foo", mock) @patch(f"{__name__}.bar", mock) async def main(): print(f"Inside main {foo=}") patcher1 = patch(f"{__name__}.foo", mock) patcher2 = patch(f"{__name__}.bar", mock) print(f"Inside main before patch start {foo} {bar}") patcher1.start() patcher2.start() print(f"Inside main after patch start {foo} {bar}") await foo() with patch(f"{__name__}.foo", mock): with patch(f"{__name__}.bar", mock): print(f"Inside main with {foo} {bar}") await foo() @patch(f"{__name__}.foo", mock) @patch(f"{__name__}.bar", mock) def sync_main(): print(f"Inside sync_main patch {foo} {bar}") with patch(f"{__name__}.foo", mock): with patch(f"{__name__}.bar", mock): print(f"Inside sync_main with {foo} {bar}") if __name__ == "__main__": asyncio.run(main()) sync_main() ./python.exe foo.py Inside main foo= Inside main before patch start Inside main after patch start Inside main with Inside sync_main patch Inside sync_main with ---------- components: Library (Lib), asyncio messages: 343067 nosy: asvetlov, cjw296, lisroach, michael.foord, xtreak, yselivanov priority: normal severity: normal status: open title: unittest.mock.patch decorator doesn't work with async functions type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 13:00:09 2019 From: report at bugs.python.org (Christian Heimes) Date: Tue, 21 May 2019 17:00:09 +0000 Subject: [New-bugs-announce] [issue36997] Document that spwd is considered harmful Message-ID: <1558458009.82.0.580344483538.issue36997@roundup.psfhosted.org> New submission from Christian Heimes : The spwd module has several flaws. Especially the combination of spwd and crypt for password verification is dangerous and in almost all cases technically wrong. # don't do this! pw1 = spwd.getspnam(username).sp_pwd pw2 = crypt.crypt(password, pw1) if pw1 == pw2: ... On BSD, Linux, and macOS, account and credential verification must go through PAM. Also see: https://mail.python.org/pipermail/python-dev/2019-May/157562.html https://mail.python.org/pipermail/python-dev/2019-May/157564.html ---------- assignee: christian.heimes components: Documentation, Extension Modules messages: 343072 nosy: christian.heimes priority: high severity: normal stage: needs patch status: open title: Document that spwd is considered harmful type: security versions: Python 2.7, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 13:33:06 2019 From: report at bugs.python.org (Toshio Kuratomi) Date: Tue, 21 May 2019 17:33:06 +0000 Subject: [New-bugs-announce] [issue36998] distutils sdist command fails to create MANIFEST if any filenames are undecodable Message-ID: <1558459986.87.0.362886106344.issue36998@roundup.psfhosted.org> New submission from Toshio Kuratomi : An sdist may contain files whose names are undecodable in the current locale. For instance, the sdist might include some files for testing whose filenames are undecodable because that's the format of the input for that application. Currently, trying to create the sdist fails with output similar to this: Traceback (most recent call last): File "setup.py", line 330, in main() File "setup.py", line 325, in main setup(**setup_params) File "/home/badger/.local/lib/python3.5/site-packages/setuptools/__init__.py", line 145, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.5/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command cmd_obj.run() File "setup.py", line 137, in run SDist.run(self) File "/usr/lib/python3.5/distutils/command/sdist.py", line 158, in run self.get_file_list() File "/usr/lib/python3.5/distutils/command/sdist.py", line 214, in get_file_list self.write_manifest() File "/usr/lib/python3.5/distutils/command/sdist.py", line 362, in write_manifest "writing manifest file '%s'" % self.manifest) File "/usr/lib/python3.5/distutils/cmd.py", line 336, in execute util.execute(func, args, msg, dry_run=self.dry_run) File "/usr/lib/python3.5/distutils/util.py", line 301, in execute func(*args) File "/usr/lib/python3.5/distutils/file_util.py", line 236, in write_file f.write(line + "\n") UnicodeEncodeError: 'ascii' codec can't encode characters in position 45-46: ordinal not in range(128) (I replicated the failure case by setting my locale to POSIX and using a standard utf-8 filename but this also applies to having a filename that is not actually text in any locale... as I said, filenames used for testing can run the gamut of odd choices). This traceback is interesting as it occurs during writing of the MANIFEST. That shows that the undecodable file is read in correctly. It's only when writing the file that it fails. Some further debugging showed me that the filename is read in using the surrogateescape error handler. So we can round trip the filename by using the surrogateescase error handler when writing it out. I tested making the following change: - f = open(filename, "w") + f = open(filename, "w", errors="surrogateescape") and sure enough, the sdist is now created correctly. I'll submit a PR to fix this. ---------- components: Distutils messages: 343074 nosy: a.badger, dstufft, eric.araujo priority: normal severity: normal status: open title: distutils sdist command fails to create MANIFEST if any filenames are undecodable versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 16:45:37 2019 From: report at bugs.python.org (=?utf-8?q?Alex_Gr=C3=B6nholm?=) Date: Tue, 21 May 2019 20:45:37 +0000 Subject: [New-bugs-announce] [issue36999] Expose the coroutine object in asyncio task objects Message-ID: <1558471537.52.0.812163346713.issue36999@roundup.psfhosted.org> New submission from Alex Gr?nholm : Both curio and trio expose the coroutine object belonging to a task as the "coro" attribute. Asyncio exposes this as "_coro" (as does uvloop) but it would be nice to have at least a read-only attribute exposing this as part of the public API. In some of my libraries I need this object, sometimes for debugging and other times for implementing "crazy" schemes like running parts of the the coroutine function code in a worker thread. ---------- components: asyncio messages: 343092 nosy: alex.gronholm, asvetlov, yselivanov priority: normal severity: normal status: open title: Expose the coroutine object in asyncio task objects type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 16:50:16 2019 From: report at bugs.python.org (Mathis Hammel) Date: Tue, 21 May 2019 20:50:16 +0000 Subject: [New-bugs-announce] [issue37000] _randbelow_with_getrandbits function inefficient with powers of two Message-ID: <1558471816.52.0.343821776026.issue37000@roundup.psfhosted.org> New submission from Mathis Hammel : In case _randbelow_with_getrandbits is called with a power of two as its argument (say 2^k), the function will consume k+1 random bits instead of k. Instead of never being rejected, the sampled number will be rejected on average once per call, causing a computational overhead in addition to wasting k+2 bits on average. This is due to taking n.bit_size() instead of (n-1).bit_size() for the size of the random candidates. Using n instead of n-1 is apparently deliberate in order to save the case where n = 1, but this could be avoided by directly returning 0 if n == 1. ---------- components: Library (Lib) messages: 343094 nosy: Mathis Hammel, mark.dickinson, rhettinger priority: normal severity: normal status: open title: _randbelow_with_getrandbits function inefficient with powers of two type: performance versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 17:01:35 2019 From: report at bugs.python.org (Dino Viehland) Date: Tue, 21 May 2019 21:01:35 +0000 Subject: [New-bugs-announce] [issue37001] symtable.symtable doesn't accept bytes which leads to a mismatch from compile() Message-ID: <1558472495.26.0.119384648617.issue37001@roundup.psfhosted.org> New submission from Dino Viehland : symtable is useful when combined with compile() to AST to understand what the names bind to. But symtable.symtable() doesn't accept a bytes object, while compile does. Ultimately these feed down to the same API, and could easily lead to subtle mismatches due to encodings. The workaround seems to be to use the tokenize.detect_encoding to discover the encoding and then do the encoding from Python, but this seems wasteful. ---------- assignee: dino.viehland components: Library (Lib) messages: 343096 nosy: dino.viehland priority: normal severity: normal status: open title: symtable.symtable doesn't accept bytes which leads to a mismatch from compile() versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 17:31:10 2019 From: report at bugs.python.org (Dino Viehland) Date: Tue, 21 May 2019 21:31:10 +0000 Subject: [New-bugs-announce] [issue37002] PyType_FromSpec can't create immutable types Message-ID: <1558474270.84.0.958874839775.issue37002@roundup.psfhosted.org> New submission from Dino Viehland : This is another catch of using PyType_FromSpec (similar to https://bugs.python.org/issue26979 but also completely different). Because PyType_FromSpec produces a heap type it gets the Py_TPFLAGS_HEAPTYPE flag set on the newly produced type. To enforce the immutability of built-in types type_setattro checks this flag. Therefore these types end up being mutable, e.g: import _ssl _ssl.SSLError.x = 42 In addition to not providing parity for replacing PyType_Ready in the stable API, it also means that this could presumably bleed across sub-interpreters or just provide surprising and bad results. ---------- components: Interpreter Core messages: 343100 nosy: dino.viehland, eric.snow, vstinner priority: normal severity: normal status: open title: PyType_FromSpec can't create immutable types versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 18:03:08 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Tue, 21 May 2019 22:03:08 +0000 Subject: [New-bugs-announce] [issue37003] ast unparse does not support f-string new debug format. Message-ID: <1558476188.31.0.971318282226.issue37003@roundup.psfhosted.org> New submission from Matthias Bussonnier : All in title, f"{x=}" implemented in https://bugs.python.org/issue36817 are not liked by unparse: https://github.com/python/cpython/pull/13473, ---------- components: Library (Lib) messages: 343105 nosy: mbussonn priority: normal severity: normal status: open title: ast unparse does not support f-string new debug format. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 22:09:15 2019 From: report at bugs.python.org (Dennis Sweeney) Date: Wed, 22 May 2019 02:09:15 +0000 Subject: [New-bugs-announce] [issue37004] SequenceMatcher.ratio() noncommutativity not well-documented Message-ID: <1558490955.4.0.899055526874.issue37004@roundup.psfhosted.org> New submission from Dennis Sweeney : I understand that the SequenceMatcher's ratio method does not guarantee that SequenceMatcher(None, a, b).ratio() == SequenceMatcher(None, b, a).ratio(). Below is a counterexample: # Example from https://mail.python.org/pipermail/python-list/2010-November/593063.html >>> SequenceMatcher(None, 'BRADY', 'BYRD').ratio() 0.6666666666666666 >>> SequenceMatcher(None, 'BYRD', 'BRADY').ratio() 0.4444444444444444 I was recently solving a problem that required a textual similarity ratio function and I wrongly assumed that SequenceMatcher treated both input strings symmetrically, which was an extremely difficult bug to find, especially because for many simple tests, the ratio IS symmetric: >>> SequenceMatcher(None, 'apple', 'banana').ratio() 0.18181818181818182 >>> SequenceMatcher(None, 'banana', 'apple').ratio() 0.18181818181818182 I would like to see a clearer warning of this asymmetry in the documentation for the difflib module. Perhaps something like .. note:: Caution: The result of a :meth:`ratio` call is *NOT* symmetric with respect to the order of the arguments. For instance:: >>> SequenceMatcher(None, 'brady', 'byrd').ratio() 0.6666666666666666 >>> SequenceMatcher(None, 'byrd', 'brady').ratio() 0.4444444444444444 Without such a note near the ratio methods' documentations, it is far too easy to google for a Python stdlib functionality for computing text similarity, skip straight to the ratio method, look at the examples given, try some of your own simple examples, and accidentally convince oneself that this symmetry exists. ---------- messages: 343138 nosy: Dennis Sweeney, docs at python priority: normal severity: normal status: open title: SequenceMatcher.ratio() noncommutativity not well-documented _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 21 23:36:08 2019 From: report at bugs.python.org (Dobatymo) Date: Wed, 22 May 2019 03:36:08 +0000 Subject: [New-bugs-announce] [issue37005] bz2 module doesn't write end-of-stream marker Message-ID: <1558496168.36.0.56962236665.issue37005@roundup.psfhosted.org> New submission from Dobatymo : According to https://en.wikipedia.org/wiki/Bzip2 the reference implementation of bzip2 writes an end-of-stream marker (also called stream footer) with a magic number and a stream checksum to the file. Python does not do so. The files can still be read by all bzip2 compatible software I tried. For completeness and better error detection however, writing this marker (optionally maybe) would be useful. ---------- messages: 343143 nosy: Dobatymo priority: normal severity: normal status: open title: bz2 module doesn't write end-of-stream marker type: behavior versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 05:20:50 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Wed, 22 May 2019 09:20:50 +0000 Subject: [New-bugs-announce] [issue37006] Add top level await statement support for doctest Message-ID: <1558516850.14.0.4225480118.issue37006@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : Since issue34616 is merged that allows using compile flags to support top level await statements I think it would be good to add support for top level await in doctest. This would help in concise examples in docs where await statements need to be wrapped in async def wrapper functions currently. This can be done using a doctest flag like ALLOW_TOP_LEVEL_AWAIT so that places where top level await is needed it can be explicitly marked as such so that when users copy paste code they are aware that it requires top level await statement. I have implemented a simple patch where ALLOW_TOP_LEVEL_AWAIT flag (not to be confused with ast module flag) is added to doctest and if the doctest line has the flag then the ast flag is added th compileflags and then await eval(code_object) is used and then the awaitabe is executed with asyncio.run. Synchronous code has usual exec(code_object). I don't see any doctest failures with this patch against our Doc folder and test_doctest. Few downsides is that it requires ast import for the flag value which could be little heavy but inspect module is already imported and I think it's an okay tradeoff for doctest. I have used asyncio.run and I am not sure if there is an efficient way to run awaitables. Feedback welcome. Patch : https://github.com/python/cpython/compare/master...tirkarthi:asyncio-await-doctest # await_flag_doctest.rst >>> import asyncio >>> await asyncio.sleep(1.0) # doctest: +ALLOW_TOP_LEVEL_AWAIT cpython git:(asyncio-await-doctest) time ./python.exe -m doctest await_flag_doctest.rst ./python.exe -m doctest await_flag_doctest.rst 0.31s user 0.02s system 24% cpu 1.343 total # await_no_flag_doctest.rst that will fail >>> import asyncio >>> await asyncio.sleep(1.0) cpython git:(asyncio-await-doctest) time ./python.exe -m doctest await_no_flag_doctest.rst ********************************************************************** File "await_no_flag_doctest.rst", line 2, in await_no_flag_doctest.rst Failed example: await asyncio.sleep(1.0) Exception raised: Traceback (most recent call last): File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/doctest.py", line 1338, in __run code = compile(example.source, filename, "single", File "", line 1 SyntaxError: 'await' outside function ********************************************************************** 1 items had failures: 1 of 2 in await_no_flag_doctest.rst ***Test Failed*** 1 failures. ./python.exe -m doctest await_no_flag_doctest.rst 0.35s user 0.03s system 94% cpu 0.393 total ---------- components: asyncio messages: 343160 nosy: asvetlov, mbussonn, xtreak, yselivanov priority: normal severity: normal status: open title: Add top level await statement support for doctest type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 07:01:24 2019 From: report at bugs.python.org (Dmitry Tantsur) Date: Wed, 22 May 2019 11:01:24 +0000 Subject: [New-bugs-announce] [issue37007] Implement socket.if_{nametoindex, indextoname} for Windows Message-ID: <1558522884.94.0.451573431034.issue37007@roundup.psfhosted.org> New submission from Dmitry Tantsur : These two calls are currently Unix-only in the socket module. However, Windows supports them starting with Vista: https://docs.microsoft.com/en-us/windows/desktop/api/netioapi/nf-netioapi-if_indextoname https://docs.microsoft.com/en-us/windows/desktop/api/netioapi/nf-netioapi-if_nametoindex I checked with ctypes and they do seem to work. Unfortunately, if_nameindex is not implemented. ---------- components: Library (Lib) messages: 343171 nosy: dtantsur priority: normal severity: normal status: open title: Implement socket.if_{nametoindex,indextoname} for Windows type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 07:19:07 2019 From: report at bugs.python.org (=?utf-8?q?Damien_Nad=C3=A9?=) Date: Wed, 22 May 2019 11:19:07 +0000 Subject: [New-bugs-announce] [issue37008] make unittest.mock.mock_open honor next() Message-ID: <1558523947.08.0.447794454948.issue37008@roundup.psfhosted.org> New submission from Damien Nad? : While we can iterate on a mock_open handle, it is not actually possible to perform a next() on it. My use case is the following: I have a file with a one-line header that I want to skip. So, roughly, my code is like with open(filename, 'r') as my_file: confirm_header(next(my_file)) for line in my_file: do_something(line) And while writing a unit test on this code, the handle returned by mock_open(read_data=my_read_data) returns a mock instead of the first line of my_read_data. ---------- components: Library (Lib) messages: 343177 nosy: Anvil priority: normal severity: normal status: open title: make unittest.mock.mock_open honor next() type: enhancement versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 08:01:09 2019 From: report at bugs.python.org (Michael Felt) Date: Wed, 22 May 2019 12:01:09 +0000 Subject: [New-bugs-announce] [issue37009] Threading and THREAD_SAFE for AIX Message-ID: <1558526469.21.0.552690908728.issue37009@roundup.psfhosted.org> New submission from Michael Felt : For years Python includes the file /usr/include/pthread.h. The AIX documentation states that this needs to be the first include file included OR the define _THREAD_SAFE needs to be defined. As this may have been true, might still be true, or might have never been true - this patch assures that the define is added to BASECFLAGS for AIX - and will not be forgotten during builds. It may be advisable to include this in backports. This "conditional requirement" has been accurate for over 20 years. ---------- components: Build messages: 343184 nosy: Michael.Felt priority: normal severity: normal status: open title: Threading and THREAD_SAFE for AIX versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 08:19:41 2019 From: report at bugs.python.org (Nick Coghlan) Date: Wed, 22 May 2019 12:19:41 +0000 Subject: [New-bugs-announce] [issue37010] Review performance of inspect.getfullargspec Message-ID: <1558527581.74.0.915249229552.issue37010@roundup.psfhosted.org> New submission from Nick Coghlan : (Splitting out a separate performance issue from https://bugs.python.org/issue36751#msg342683) There can be two quite different reasons for inspecting a function signature: * inspecting arbitrary functions to learn as much about them as possible in order to present good coding hints and other guidance to a developer * inspecting function and method implementations passed to a runtime API in order to call them correctly inspect.signature focuses on the former use case, and as a result ended up being markedly slower than the simpler inspect.getfullargspec implementation that it replaced. At the moment, inspect.getfullargspec is being kept around solely as a backwards compatibility API - it calls inspect.signature internally, but then throws away the rich objects that function creates, replacing them with simple Python builtins. It seems plausible that we could reverse that relationship, and instead have inspect.signature use inspect.getfullargspec as a lower level API that produces less self-descriptive output, but also avoids creating a lot of instances of custom Python objects. (Sample performance data can be seen in https://gist.github.com/zzzeek/0eb0636fa3917f36ffd887d9f765c208) ---------- messages: 343186 nosy: ncoghlan priority: normal severity: normal stage: needs patch status: open title: Review performance of inspect.getfullargspec type: performance _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 08:44:44 2019 From: report at bugs.python.org (daniel hahler) Date: Wed, 22 May 2019 12:44:44 +0000 Subject: [New-bugs-announce] [issue37011] pdb: restore original tracing function instead of sys.settrace(None) Message-ID: <1558529084.78.0.178705379805.issue37011@roundup.psfhosted.org> New submission from daniel hahler : bdb/pdb currently uses `sys.settrace(None)` when uninstalling its trace function (trace_dispatch), but should rather store the original trace function in the beginning and use this instead of `None`. While typically pdb is not used in tests, it is just good practice, given that there can only be a single trace function. I've done this via monkeypatching for pdbpp's tests, which resulted in an easy 2% coverage gain (https://github.com/antocuni/pdb/pull/253). ---------- components: Library (Lib) messages: 343188 nosy: blueyed priority: normal severity: normal status: open title: pdb: restore original tracing function instead of sys.settrace(None) type: behavior versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 10:04:10 2019 From: report at bugs.python.org (Petr Viktorin) Date: Wed, 22 May 2019 14:04:10 +0000 Subject: [New-bugs-announce] [issue37012] Possible crash due to PyType_FromSpecWithBases() Message-ID: <1558533850.59.0.459054292807.issue37012@roundup.psfhosted.org> New submission from Petr Viktorin : If the PyObject_MALLOC() call failed in PyType_FromSpecWithBases(), PyObject_Free() would be called on a static string in type_dealloc(). Fixed by Zackery Spytz in pull request 10304. I'm opening the issue retroactively. ---------- messages: 343194 nosy: petr.viktorin priority: normal severity: normal status: open title: Possible crash due to PyType_FromSpecWithBases() versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 12:28:25 2019 From: report at bugs.python.org (Zackery Spytz) Date: Wed, 22 May 2019 16:28:25 +0000 Subject: [New-bugs-announce] [issue37013] Fatal Python error in socket.if_indextoname() Message-ID: <1558542505.9.0.778249415018.issue37013@roundup.psfhosted.org> New submission from Zackery Spytz : Python 3.8.0a4+ (heads/master:ef9d9b6312, May 22 2019, 08:35:25) [GCC 9.0.1 20190402 (experimental) [trunk revision 270074]] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket.if_indextoname(2**64 - 1) Fatal Python error: a function returned NULL without setting an error SystemError: returned NULL without setting an error Current thread 0x00007f29d708d140 (most recent call first): File "", line 1 in Aborted (core dumped) This is because socket.if_indextoname() does not use PyErr_Occurred() when checking PyLong_AsUnsignedLong() for failure. ---------- components: Extension Modules messages: 343215 nosy: ZackerySpytz, serhiy.storchaka, vstinner priority: normal severity: normal status: open title: Fatal Python error in socket.if_indextoname() type: crash versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 13:26:07 2019 From: report at bugs.python.org (Grant Wu) Date: Wed, 22 May 2019 17:26:07 +0000 Subject: [New-bugs-announce] [issue37014] fileinput module should document that openhook and mode are ignored when reading from stdin Message-ID: <1558545967.04.0.906137299955.issue37014@roundup.psfhosted.org> New submission from Grant Wu : https://github.com/python/cpython/blob/master/Lib/fileinput.py#L326 shows that the openhook and mode are ignored when reading from stdin. Since part of fileinput's functionality is to abstract over whether one is reading from stdin or over a file, I think this abstraction leak should be documented. One common use case where this might break is when attempting to set the file encoding using the included fileinput.hook_encoded functionality. ---------- assignee: docs at python components: Documentation messages: 343221 nosy: Grant Wu2, docs at python priority: normal severity: normal status: open title: fileinput module should document that openhook and mode are ignored when reading from stdin versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 13:56:03 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Wed, 22 May 2019 17:56:03 +0000 Subject: [New-bugs-announce] [issue37015] Fix asyncio mock wranings Message-ID: <1558547763.57.0.769583160761.issue37015@roundup.psfhosted.org> New submission from Andrew Svetlov : After merging https://github.com/python/cpython/pull/9296 asyncio test suite prints a lot of warnings like Exception ignored in: Traceback (most recent call last): File "/home/andrew/projects/cpython/Lib/warnings.py", line 510, in _warn_unawaited_coroutine warn(msg, category=RuntimeWarning, stacklevel=2, source=coro) RuntimeWarning: coroutine 'AsyncMockMixin._mock_call' was never awaited I believe this is not a sign of AsyncMock problem but asyncio tests should be fixed. Need more investigations though. ---------- components: Tests messages: 343223 nosy: asvetlov priority: normal severity: normal status: open title: Fix asyncio mock wranings type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 17:45:30 2019 From: report at bugs.python.org (Soumya Mohanty) Date: Wed, 22 May 2019 21:45:30 +0000 Subject: [New-bugs-announce] [issue37016] Python embedded in C++ cannot open a file Message-ID: <1558561530.97.0.560332207187.issue37016@roundup.psfhosted.org> New submission from Soumya Mohanty : Hello, I am trying to open a pickled file and load it in my python file. This python file will be called from a c++ program. Please find the C++ program attached. Py_Initialize and Py_Finalize are being done in a separate file called pyhelper.hpp Python code : def test(): print("In function test of pyemb.py file \n") import pickle with open('filepath', 'rb') as f_in: C = pickle.load(f_in) I am getting the following error and cant find any way to fix it: Exception ignored in: Traceback (most recent call last): File "C:\Anaconda3\envs\Deep_Learning\lib\threading.py", line 1289, in _shutdown assert tlock.locked() SystemError: returned a result with an error set ---------- files: PYTHON_PROJ.cpp messages: 343246 nosy: mohantys priority: normal severity: normal status: open title: Python embedded in C++ cannot open a file versions: Python 3.6 Added file: https://bugs.python.org/file48350/PYTHON_PROJ.cpp _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 22 20:32:24 2019 From: report at bugs.python.org (Michael J. Sullivan) Date: Thu, 23 May 2019 00:32:24 +0000 Subject: [New-bugs-announce] [issue37017] Use LOAD_METHOD optimization in CallMethod C API functions Message-ID: <1558571544.48.0.863461247465.issue37017@roundup.psfhosted.org> New submission from Michael J. Sullivan : The different varieties of PyObject_CallMethod* routines all operate by doing a PyObject_GetAttr to fetch an object to call. It seems likely to be worthwhile to take advantage of the LOAD_METHOD optimization that avoids creating a bound method object when calling a method. ---------- components: Extension Modules messages: 343259 nosy: msullivan priority: normal severity: normal status: open title: Use LOAD_METHOD optimization in CallMethod C API functions versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 01:22:46 2019 From: report at bugs.python.org (Glenn Linderman) Date: Thu, 23 May 2019 05:22:46 +0000 Subject: [New-bugs-announce] [issue37018] put http.server on a diet Message-ID: <1558588966.15.0.498187197609.issue37018@roundup.psfhosted.org> New submission from Glenn Linderman : The idea inspired by the email exchange below is basically in three parts: 1. investigate the various popular web server frameworks, to determine what parts of http.server they depend on. For example, bottle.py depends only on BaseHTTPRequestHandler and HTTPServer, which is less than half the code in http.server. Because of the deficiencies in the remaining parts, it seems unlikely that other frameworks use much more. 2. Trim http.server to those useful parts, removing the the rest from stdlib. Many of the "enhanced features" of http.server are such minimal enhancements that they are feature-poor and out-of-date with respect to current web server standards. The novice user is likely to be enticed into a swamp of missing capability when attempting to use them, as I was. It would take significant work to implement true CGI together with SSL on forking OSes; it took me significant work to implement true CGI together with SSL on Windows (non-forking). I gave up trying to do it on Linux, and switched to bottle. 3. Enhance what is left of http.server to support SSL and threading, so that the web frameworks that use http.server as a test server can at least offer those capabilities as well. It isn't too hard to add those things for bottle.py, but it would be nicer if users didn't have to google for the blog posts that show how, and reimplement it (most of the blog posts are somewhat dated). On 5/22/2019 4:09 AM, Christian Heimes wrote: > On 22/05/2019 01.11, Glenn Linderman wrote: >> On 5/21/2019 2:00 PM, Nathaniel Smith wrote: >>> On Tue, May 21, 2019 at 10:43 AM Glenn Linderman wrote: >>>> After maintaining my own version of http.server to fix or workaround some of its deficiencies for some years, I discovered bottle.py. It has far more capability, is far better documented, and is just as quick to deploy. While I haven't yet converted all past projects to use bottle.py, it will likely happen in time, unless something even simpler to use is discovered, although I can hardly imagine that happening. >>> bottle.py uses http.server for its local development mode (the one you >>> see in their quickstart example at the top of their README). Same with >>> flask, django, and probably a bunch of other frameworks. It's *very* >>> widely used. >>> >>> -n >>> >> The source for bottle.py version 0.13-dev has an import for http.client, but not http.server. I hadn't tracked down every indirect dependency in the bottle.py source code, but it seems that if one uses the "default server" for bottle, that it is "wsgiref", imported from wsgiref.simple_server, and that in turn does import BaseHTTPRequestHandler and HTTPServer from http.server. >> >> It is the higher-level code in http.server that has significant deficiencies that have caused me problems over the years... a "SimpleHTTPRequestHandler" that is so simple it doesn't do POST, PUT or PASTE, a "CGIHTTPRequestHandler" that only implements part of the CGI protocol, only CGI support in POST, no support for PUT or PASTE, and no support for https, and not much bug fix activity in those areas. >> >> Maybe http.server should be split into the "basic parts" (used by bottle.py, and other frameworks), and the "higher-level parts", which could then be discarded by this PEP! At this point, though, I'd have to agree that the whole should not be discarded. Thanks for making me dig deeper. > > The idea has merrit. However I feel its out of scope for the PEP [594]. The http.server module and socketserver module are still widely used for debug and toy examples. > > Could you please open a bug to track your proposal? We may pursue it in a couple of years from now. ---------- messages: 343275 nosy: v+python priority: normal severity: normal status: open title: put http.server on a diet _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 03:09:02 2019 From: report at bugs.python.org (Shannon) Date: Thu, 23 May 2019 07:09:02 +0000 Subject: [New-bugs-announce] [issue37019] using pathlib objects to create symlinks produces broken links Message-ID: <1558595342.46.0.940919584419.issue37019@roundup.psfhosted.org> New submission from Shannon : when using pathlib objects to define src and dst for os.symlink (also relevant for Path(dst).symlink_to(src)), if the src path is not absolute, or relative to the directory the link is being created in, a broken link will be created. example/ src = pathlib.Path('dir1/file') dst = pathlib.Path('dir2/file') os.symlink(src, dst) # or dst.symlink_to(src) this will create a broken link in dir2, attempting to link to dir1/file, relative to dir2. It seems to me, if src given is a pathlib object (relative to cwd), the linking process should be smart enough to point the link the created symlink to the right place. os.symlink(src.absolute(), dst) works, but creates an absolute symlink which may not be desired behaviour. My current workaround is: os.symlink(os.path.relpath(src, dst.parent), dst) which creates a working relative symlink as desired. I would suggest this should be the default behaviour of both os.symlink and pathlib.Path().symlink_to when a non-absolute path object is given as src. Interestingly, src.relative_to(dst.parent) will raise a ValueError while os.path.relpath(src, dst.parent) correctly returns '../dir1/file'. I also think Path().relative_to should be changed to match the behaviour of os.path.relpath here, but perhaps that is a separate issue. ---------- components: Library (Lib) messages: 343277 nosy: skeo priority: normal severity: normal status: open title: using pathlib objects to create symlinks produces broken links type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 07:55:29 2019 From: report at bugs.python.org (Antony) Date: Thu, 23 May 2019 11:55:29 +0000 Subject: [New-bugs-announce] [issue37020] Invalid floating point multiplication result Message-ID: <1558612529.93.0.512106362254.issue37020@roundup.psfhosted.org> New submission from Antony : Incorrect Multiplication result of number 40.95 >>> 40.94 * 100 4094.0 >>> 40.96 * 100 4096.0 but: >>> 40.95 * 100 4095.0000000000005 checked with: Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15) and Python 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] on linux ---------- messages: 343287 nosy: Tonimore priority: normal severity: normal status: open title: Invalid floating point multiplication result type: behavior versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 11:16:57 2019 From: report at bugs.python.org (Raymond Hettinger) Date: Thu, 23 May 2019 15:16:57 +0000 Subject: [New-bugs-announce] [issue37021] Can _random.getrandbits() be converted to METH_FASTCALL? Message-ID: <1558624618.0.0.375945744552.issue37021@roundup.psfhosted.org> Change by Raymond Hettinger : ---------- components: Extension Modules nosy: rhettinger, serhiy.storchaka, vstinner priority: normal severity: normal status: open title: Can _random.getrandbits() be converted to METH_FASTCALL? type: performance versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 12:02:14 2019 From: report at bugs.python.org (daniel hahler) Date: Thu, 23 May 2019 16:02:14 +0000 Subject: [New-bugs-announce] [issue37022] pdb: do_p and do_pp swallow exceptions from __repr__ Message-ID: <1558627334.31.0.41290624509.issue37022@roundup.psfhosted.org> New submission from daniel hahler : Given: ``` class BadRepr: def __repr__(self): raise Exception('repr_exc') obj = BadRepr() __import__('pdb').set_trace() ``` ``` (Pdb) p obj (Pdb) pp obj (Pdb) ``` Possible patch - clumsy due to `self._getval` both printing any error already, and raising the exception: ``` diff --git i/Lib/pdb.py w/Lib/pdb.py index f5d33c27fc..59a419d961 100755 --- i/Lib/pdb.py +++ w/Lib/pdb.py @@ -1177,18 +1177,28 @@ def do_p(self, arg): Print the value of the expression. """ try: - self.message(repr(self._getval(arg))) + val = self._getval(arg) except: - pass + return + try: + self.message(repr(val)) + except: + exc_info = sys.exc_info()[:2] + self.error(traceback.format_exception_only(*exc_info)[-1].strip()) def do_pp(self, arg): """pp expression Pretty-print the value of the expression. """ try: - self.message(pprint.pformat(self._getval(arg))) + val = self._getval(arg) except: - pass + return + try: + self.message(pprint.pformat(val)) + except: + exc_info = sys.exc_info()[:2] + self.error(traceback.format_exception_only(*exc_info)[-1].strip()) complete_print = _complete_expression complete_p = _complete_expression ``` ---------- components: Library (Lib) messages: 343306 nosy: blueyed priority: normal severity: normal status: open title: pdb: do_p and do_pp swallow exceptions from __repr__ type: behavior versions: Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 13:16:38 2019 From: report at bugs.python.org (STINNER Victor) Date: Thu, 23 May 2019 17:16:38 +0000 Subject: [New-bugs-announce] [issue37023] test_gdb failed on AMD64 Debian PGO 3.x Message-ID: <1558631798.29.0.597769613412.issue37023@roundup.psfhosted.org> New submission from STINNER Victor : It's likely a regression caused by bpo-36842. https://buildbot.python.org/all/#/builders/47/builds/2854 Example of failure: ====================================================================== FAIL: test_NULL_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with NULL ob_type is handled gracefully ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Lib/test/test_gdb.py", line 533, in test_NULL_ob_type self.assertSane('id(42)', File "/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Lib/test/test_gdb.py", line 504, in assertSane self.get_gdb_repr(source, File "/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Lib/test/test_gdb.py", line 278, in get_gdb_repr self.fail('Unexpected gdb output: %r\n%s' % (gdb_output, gdb_output)) AssertionError: Unexpected gdb output: 'Breakpoint 1 at 0x201df0: file Python/bltinmodule.c, line 1217.\n[Thread debugging using libthread_db enabled]\nUsing host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".\n\nBreakpoint 1, builtin_id () at Python/bltinmodule.c:1217\n1217\t{\n#0 builtin_id () at Python/bltinmodule.c:1217\n#1 _PyMethodDef_RawFastCallKeywords () at Objects/call.c:650\n#2 _PyCFunction_FastCallKeywords (func=, args=, nargs=, kwnames=) at Objects/call.c:736\n#3 call_function (kwnames=0x0, oparg=1, pp_stack=, tstate=) at Python/ceval.c:4831\n#4 _PyEval_EvalFrameDefault () at Python/ceval.c:3347\n#5 PyEval_EvalFrameEx (throwflag=0, f=Frame 0x7ffff6e94800, for file , line 1, in ()) at Python/ceval.c:685\n#6 _PyEval_EvalCodeWithName () at Python/ceval.c:4173\n#7 PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=, kwcount=0, kws=, argcount=0, args=, locals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': , \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': }, globals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': , \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': }, _co=) at Python/ceval.c:4202\n#8 PyEval_EvalCode (co=, globals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': , \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': }, locals={\'__name__\': \'__main__\', \'__doc__\': None, \'__package__\': None, \'__loader__\': , \'__spec__\': None, \'__annotations__\': {}, \'__builtins__\': }) at Python/ceval.c:662\n#9 run_eval_code_obj () at Python/pythonrun.c:1078\n#10 run_mod () at Python/pythonrun.c:1100\n#11 PyRun_StringFlags () at Python/pythonrun.c:987\n#12 PyRun_SimpleStringFlags () at Python/pythonrun.c:461\n#13 pymain_run_command (cf=, command=) at Modules/main.c:241\n#14 pymain_run_python (exitcode=) at Modules/main.c:522\n#15 _Py_RunMain () at Modules/main.c:610\n#16 pymain_main () at Modules/main.c:640\n#17 _Py_UnixMain (argc=, argv=) at Modules/main.c:664\n#18 __libc_start_main (main=
, argc=7, argv=, init=, fini=, rtld_fini=, stack_end=) at ../csu/libc-start.c:291\n#19 _start ()\n' Breakpoint 1 at 0x201df0: file Python/bltinmodule.c, line 1217. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Breakpoint 1, builtin_id () at Python/bltinmodule.c:1217 1217 { #0 builtin_id () at Python/bltinmodule.c:1217 #1 _PyMethodDef_RawFastCallKeywords () at Objects/call.c:650 #2 _PyCFunction_FastCallKeywords (func=, args=, nargs=, kwnames=) at Objects/call.c:736 #3 call_function (kwnames=0x0, oparg=1, pp_stack=, tstate=) at Python/ceval.c:4831 #4 _PyEval_EvalFrameDefault () at Python/ceval.c:3347 #5 PyEval_EvalFrameEx (throwflag=0, f=Frame 0x7ffff6e94800, for file , line 1, in ()) at Python/ceval.c:685 #6 _PyEval_EvalCodeWithName () at Python/ceval.c:4173 #7 PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=, kwcount=0, kws=, argcount=0, args=, locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': , '__spec__': None, '__annotations__': {}, '__builtins__': }, globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': , '__spec__': None, '__annotations__': {}, '__builtins__': }, _co=) at Python/ceval.c:4202 #8 PyEval_EvalCode (co=, globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': , '__spec__': None, '__annotations__': {}, '__builtins__': }, locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': , '__spec__': None, '__annotations__': {}, '__builtins__': }) at Python/ceval.c:662 #9 run_eval_code_obj () at Python/pythonrun.c:1078 #10 run_mod () at Python/pythonrun.c:1100 #11 PyRun_StringFlags () at Python/pythonrun.c:987 #12 PyRun_SimpleStringFlags () at Python/pythonrun.c:461 #13 pymain_run_command (cf=, command=) at Modules/main.c:241 #14 pymain_run_python (exitcode=) at Modules/main.c:522 #15 _Py_RunMain () at Modules/main.c:610 #16 pymain_main () at Modules/main.c:640 #17 _Py_UnixMain (argc=, argv=) at Modules/main.c:664 #18 __libc_start_main (main=
, argc=7, argv=, init=, fini=, rtld_fini=, stack_end=) at ../csu/libc-start.c:291 #19 _start () ---------- components: Tests messages: 343314 nosy: vstinner priority: normal severity: normal status: open title: test_gdb failed on AMD64 Debian PGO 3.x versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 13:26:21 2019 From: report at bugs.python.org (Lysandros Nikolaou) Date: Thu, 23 May 2019 17:26:21 +0000 Subject: [New-bugs-announce] [issue37024] SQLite flag in configure due to homebrew not linking sqlite Message-ID: <1558632381.76.0.984637995348.issue37024@roundup.psfhosted.org> New submission from Lysandros Nikolaou : Since recently, Homebrew refuses to link sqlite. Upon researching the whole thing, I found out that this is now considered a feature of Homebrew and not a bug. Thus homebrew users on macOS do not get the SQLite module installed by default, because it is not found and have to overwrite CPPFLAGS. Would it make sense to include a --with-sqlite flag, like we do for openssl? ---------- components: Build messages: 343317 nosy: lys.nikolaou priority: normal severity: normal status: open title: SQLite flag in configure due to homebrew not linking sqlite type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 17:13:39 2019 From: report at bugs.python.org (Ralf Habacker) Date: Thu, 23 May 2019 21:13:39 +0000 Subject: [New-bugs-announce] [issue37025] Misleading error message "Python failed to load the default activation context" Message-ID: <1558646019.4.0.18611977167.issue37025@roundup.psfhosted.org> New submission from Ralf Habacker : When I started Kicad under Windows I got the following error message in dbgview: "Python could not load the default activation context". This message is issued by the Python runtime environment when checking the return value of the Windows API function AddRefActCtx() if the return value is zero (see https://github.com/python/cpython/blob/a3488e5902f5c26e5cc289aec2518e7b5058e5d1/PC/dl_nt.c#L107). According to https://docs.microsoft.com/de-de/windows/desktop/api/winbase/nf-winbase-AddRefActCtx, this function does not return a value, which makes the check result dependent on temporary internal register/stack values and does not indicate a real problem. This check should be completely removed so as not to waste the precious time of developers and users searching for the reason for this misleading message. The corresponding definitions https://github.com/python/cpython/blob/2.7/PC/dl_nt.c#L36 (and line 37) should also be adjusted. The issue is also present in version 2.7. ---------- components: Windows messages: 343329 nosy: paul.moore, rhabacker, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Misleading error message "Python failed to load the default activation context" type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 18:06:55 2019 From: report at bugs.python.org (jmberg) Date: Thu, 23 May 2019 22:06:55 +0000 Subject: [New-bugs-announce] [issue37026] socketserver: BaseServer.handle_request() infinite loop Message-ID: <1558649215.56.0.199533115703.issue37026@roundup.psfhosted.org> New submission from jmberg : Hi, Alright - you may very well consider this to be a stupid idea and everything, but I figured I'd report it anyway, just so it's there even if you decide to ignore it. For context, I'm running python in a ARCH=um Linux system that has completely virtual time, using basically this patch: https://patchwork.ozlabs.org/patch/1095055/ Now, as I replied there myself, you would in fact think that this can lead to infinite loops, but not really in a select/poll loop in the socket server, hence this report. We have the following code: [...] if timeout is not None: deadline = time() + timeout # Wait until a request arrives or the timeout expires - the loop is # necessary to accommodate early wakeups due to EINTR. with _ServerSelector() as selector: selector.register(self, selectors.EVENT_READ) while True: ready = selector.select(timeout) if ready: return self._handle_request_noblock() else: if timeout is not None: timeout = deadline - time() if timeout < 0: return self.handle_timeout() Assume that a timeout is given, so deadline is set to time() + timeout. Now, if selector.select(timeout) returns [] because nothing was ready for reading, you'd expect us to treat this as a timeout, right? However, in my case of virtual time with infinite processing power, it doesn't. Let's say that timeout is 1 (second) and that the call to time() returned 10000 (clearly not realistic, but doesn't matter). Now, in a virtual time implementation with infinite processing power, select(timeout) will sleep for *exactly* 1 second, and return nothing is ready. Then, in the else: branch, we set "timeout = deadline - time()" - but now time will return 10001 (remember we slept for exactly 1 second), and timeout will be 0. We will thus not handle a timeout, instead, we'll go into select() again with a timeout of 0. Due to the "infinite processing power" aspect of this system, this will happen over and over again. The trivial fix here is to handle it as a timeout "if timeout <= 0" rather than just "if timeout < 0". Obviously, I can also fix my virtual time system, for example, I can give it less processing power by actually interrupting the process after some real time (which I implemented in https://patchwork.ozlabs.org/patch/1095814/). An alternative is to make every "what time is it now" request actually take some virtual time, thus the time() calls cannot return the same value over and over again. This also fixes the issue I saw. But it stands to reason that if it should happen that the new timeout is actually 0, we should really treat it as a timeout here in this code and make it <=0 rather than just <0. ---------- components: Library (Lib) messages: 343330 nosy: jmberg priority: normal severity: normal status: open title: socketserver: BaseServer.handle_request() infinite loop type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 18:46:13 2019 From: report at bugs.python.org (Yury Selivanov) Date: Thu, 23 May 2019 22:46:13 +0000 Subject: [New-bugs-announce] [issue37027] Return a safe proxy over a socket from get_extra_info('socket') Message-ID: <1558651573.81.0.166270912079.issue37027@roundup.psfhosted.org> New submission from Yury Selivanov : Currently asyncio exposes the underlying transport socket directly, providing a way to modify (in a potentially disruptive way) the underlying transport connection. ---------- messages: 343332 nosy: yselivanov priority: normal severity: normal status: open title: Return a safe proxy over a socket from get_extra_info('socket') _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 23 18:57:13 2019 From: report at bugs.python.org (Yury Selivanov) Date: Thu, 23 May 2019 22:57:13 +0000 Subject: [New-bugs-announce] [issue37028] Implement asyncio repl Message-ID: <1558652233.35.0.225299391791.issue37028@roundup.psfhosted.org> New submission from Yury Selivanov : Having an asyncio enabled repr where a top-level "await" possible would be a huge productivity boost. Using asyncio.run() in a REPL is hard, and besides it spawns a new event loop on every call. The big idea: we want users to be able to do this: $ python -m asyncio >>> await asyncio.sleep(10, return='hello') # after 10 seconds hello ---------- components: asyncio messages: 343334 nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Implement asyncio repl versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 01:33:57 2019 From: report at bugs.python.org (Inada Naoki) Date: Fri, 24 May 2019 05:33:57 +0000 Subject: [New-bugs-announce] [issue37029] PyObject_Free is O(N^2) where N = # of arenas Message-ID: <1558676037.99.0.0721055204531.issue37029@roundup.psfhosted.org> New submission from Inada Naoki : Reported here: * https://stackoverflow.com/questions/56228799/python-hangs-indefinitely-trying-to-delete-deeply-recursive-object * https://mail.python.org/pipermail/python-dev/2019-May/157635.html ---------- components: Interpreter Core messages: 343344 nosy: inada.naoki, tim.peters priority: normal severity: normal status: open title: PyObject_Free is O(N^2) where N = # of arenas type: performance versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 03:25:53 2019 From: report at bugs.python.org (Aldwin Pollefeyt) Date: Fri, 24 May 2019 07:25:53 +0000 Subject: [New-bugs-announce] [issue37030] Lib/cmd.py: Hide undocumented commands in help and completenames Message-ID: <1558682753.85.0.407660629596.issue37030@roundup.psfhosted.org> New submission from Aldwin Pollefeyt : A flag, defaulting to false. If true, :meth:`do_help` and :meth:`completenames` won't include undocumented commands (that is, there are do_*() methods without corresponding help_*() methods). ---------- components: Library (Lib) messages: 343346 nosy: aldwinaldwin priority: normal severity: normal status: open title: Lib/cmd.py: Hide undocumented commands in help and completenames type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 04:08:57 2019 From: report at bugs.python.org (STINNER Victor) Date: Fri, 24 May 2019 08:08:57 +0000 Subject: [New-bugs-announce] [issue37031] signalmodule.c: reuse runtime->main_thread Message-ID: <1558685337.89.0.451873883908.issue37031@roundup.psfhosted.org> New submission from STINNER Victor : Currently, signalmodule.c tracks the "main thread" and the "main interpreter": main_thread = PyThread_get_thread_ident(); main_interp = _PyInterpreterState_Get(); This information is already tracked in the global _PyRuntime variable which is updated at fork by _PyRuntimeState_ReInitThreads() (called by PyOS_AfterFork_Child). Attached PR removes main_thread and main_interp from signalmodule.c, to reuse _PyRuntime. It should help to ensure that Python remains consistent (less risk of inconsistency if one variable is updated, but not the other). By the way, _PyInterpreterState_DeleteExceptMain() is called before _PyRuntimeState_ReInitThreads(). _PyInterpreterState_DeleteExceptMain() acquires runtime->interpreters.mutex, whereas this lock is reset by _PyRuntimeState_ReInitThreads(). Maybe I introduced this bug recently when I refactored code, I don't know. But it sounds like a bug as well. Note: PyOS_AfterFork_Child() doesn't update runtime->interpreters->main, but os.fork() raises a RuntimeError if it's not called from the main interpreter. So it's fine. ---------- components: Interpreter Core messages: 343350 nosy: vstinner priority: normal severity: normal status: open title: signalmodule.c: reuse runtime->main_thread versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 05:25:55 2019 From: report at bugs.python.org (STINNER Victor) Date: Fri, 24 May 2019 09:25:55 +0000 Subject: [New-bugs-announce] [issue37032] Add CodeType.replace() method Message-ID: <1558689955.85.0.932624363868.issue37032@roundup.psfhosted.org> New submission from STINNER Victor : Each type types.CodeType constructor changes, a lot of applications break. Python 3.8 added a new parameter which broke for example the Genshi project, just to name one. I propose to add a new CodeType.replace() method, similar to datetime.datetime.replace and namedtuple._replace() for example: $ python3 Python 3.7.3 (default, Mar 27 2019, 13:41:07) >>> import datetime >>> d=datetime.datetime.now() >>> d2=d.replace(year=2010) >>> d, d2 (datetime.datetime(2019, 5, 24, 11, 25, 3, 839966), datetime.datetime(2010, 5, 24, 11, 25, 3, 839966)) >>> import collections >>> Point = collections.namedtuple("Point", "x y") >>> p=Point(1, 2) >>> p2=p._replace(y=3) >>> p, p2 (Point(x=1, y=2), Point(x=1, y=3)) ---------- components: Interpreter Core messages: 343360 nosy: pablogsal, vstinner priority: normal severity: normal status: open title: Add CodeType.replace() method versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 05:38:42 2019 From: report at bugs.python.org (Vykouk) Date: Fri, 24 May 2019 09:38:42 +0000 Subject: [New-bugs-announce] [issue37033] Dictionary defaults .get(), .pop(), .setdefault() Message-ID: <1558690722.29.0.185013402146.issue37033@roundup.psfhosted.org> New submission from Vykouk : The methods .get(), .pop(), .setdefault() evaluate defaults even though the key already exists in the dictionary: # -*- coding: utf-8 -*- def deflt(x): print('\nKey', x, 'is not in the dictionary') return 'Default' dicti = {1:'one', 2:'two', 3:'three'} key = 2 print('\ndicti \t\t', dicti) print('\t\t key =',key) print('get \t\t', dicti.get(key, deflt(key))) print('setdefault \t',dicti.setdefault(key, deflt(key))) print('dicti \t\t', dicti) print('pop \t\t', dicti.pop(key, deflt(key))) print('dicti \t\t', dicti) print('setdefault \t',dicti.setdefault(key, deflt(key))) print('dicti \t\t', dicti) ''' Printed outputs: Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit (AMD64)] Type "copyright", "credits" or "license" for more information. IPython 7.1.1 -- An enhanced Interactive Python. runfile('C:/Temp/Smazat/Dictionary_Defaults.py', wdir='C:/Temp/Smazat') dicti {1: 'one', 2: 'two', 3: 'three'} key = 2 Key 2 is not in the dictionary # ??? get two Key 2 is not in the dictionary # ??? setdefault two dicti {1: 'one', 2: 'two', 3: 'three'} Key 2 is not in the dictionary # ??? pop two dicti {1: 'one', 3: 'three'} Key 2 is not in the dictionary setdefault Default dicti {1: 'one', 3: 'three', 2: 'Default'} ''' ---------- files: Dictionary_Defaults.py messages: 343362 nosy: vykouk priority: normal severity: normal status: open title: Dictionary defaults .get(), .pop(), .setdefault() type: behavior versions: Python 3.7 Added file: https://bugs.python.org/file48352/Dictionary_Defaults.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 06:29:45 2019 From: report at bugs.python.org (STINNER Victor) Date: Fri, 24 May 2019 10:29:45 +0000 Subject: [New-bugs-announce] [issue37034] Argument Clinic omits name of keyword-only parameter on _PyArg_BadArgument() call Message-ID: <1558693785.23.0.512986851321.issue37034@roundup.psfhosted.org> New submission from STINNER Victor : Example of generated code: if (!PyBytes_Check(args[15])) { _PyArg_BadArgument("replace", 16, "bytes", args[15]); goto exit; } Error message: TypeError: replace() argument 16 must be bytes, not tuple It is the 'lnotab' parameter which is a keywoard-only parameter. I expect the parameter name in the error message. Note: I got this error while implementing bpo-37032. ---------- components: Interpreter Core messages: 343365 nosy: serhiy.storchaka, vstinner priority: normal severity: normal status: open title: Argument Clinic omits name of keyword-only parameter on _PyArg_BadArgument() call versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 10:32:57 2019 From: report at bugs.python.org (Andrew Svetlov) Date: Fri, 24 May 2019 14:32:57 +0000 Subject: [New-bugs-announce] [issue37035] Don't log OSError exceptions in asyncio transports Message-ID: <1558708377.15.0.222546736903.issue37035@roundup.psfhosted.org> New submission from Andrew Svetlov : Currently asyncio uses `loop.call_exception_handler()` for logging *fatal* exceptions from underlying sockets before calling protocol.connection_lost(). There is a list of exceptions that are not logged: BrokenPipeError, ConnectionResetError, ConnectionAbortedError Later ssl.SSLCertVerificationError was added. There is a request to skip TimeoutError as well: #34148 aiohttp has a bug report about logging SSLError: https://github.com/aio-libs/aiohttp/issues/3535 I am pretty sure that the network subsystem can raise other exceptions that users don't want to see in logs. I suggest changing suppression logic to skip logging of *all* OSError exceptions and eliminate _FATAL_ERROR_IGNORE list. OSError is a sign of communication over network to peer, these problems cannot be avoided by fixing a program logic. All other exceptions are indicators for programming errors, the user's source code should be fixed to get rid of such problems (e.g. AttributeError raised by protocol callback etc). Logging non-OSErrors is pretty nice and desired feature. Note, the list now contains OSError derived exceptions only, both TimeoutError and SSLError falls into this category too. Not-logged exceptions are not skipped but reported to the user by protocol.connection_lost(exc) callback. User code can (and should) handle them properly. The fix is very simple, and don't change application logic. It can be backported to 3.7 as well. ---------- components: asyncio messages: 343389 nosy: asvetlov, yselivanov priority: normal severity: normal status: open title: Don't log OSError exceptions in asyncio transports versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 12:08:32 2019 From: report at bugs.python.org (Josh Rosenberg) Date: Fri, 24 May 2019 16:08:32 +0000 Subject: [New-bugs-announce] [issue37036] Iterating a text file by line should not implicitly disable tell Message-ID: <1558714112.82.0.415931508082.issue37036@roundup.psfhosted.org> New submission from Josh Rosenberg : TextIOWrapper explicitly sets the `telling` flag to 0 when .__next__ ( textiowrapper_iternext ) is called ( https://github.com/python/cpython/blob/3.7/Modules/_io/textio.c#L2974 ), e.g. during standard for loops over the file of this form, trying to call tell raises an exception: with open(filename) as myfile: for line in myfile: myfile.tell() which raises: OSError: telling position disabled by next() call while the effectively equivalent: with open(filename) as myfile: for line in iter(myfile.readline, ''): myfile.tell() works fine. The implementation of __next__ and readline is almost identical (__next__ is calling readline and handling the EOF sentinel per the iterator protocol, that's all). Given they're implemented identically, I see no reason aside from nannying (discouraging slow operations like seek/tell during iteration by forbidding them on the most convenient means of iterating) to forbid tell after beginning iteration, but not after readline. Given the general Python philosophy of "we're all adults here", I don't see nannying as a good reason, which leaves the performance benefit of avoiding snapshotting as the only compelling reason to do this. But the performance benefit is trivial; in local tests, the savings from avoiding that work is barely noticeable, per ipython microbenchmarks (on 3.7.2 Linux x86-64): >>> %%timeit -r5 from collections import deque; f = open('American-english.txt'); consume = deque(maxlen=0).extend ... f.seek(0) ... consume(iter(f.readline, '')) ... ... 15.8 ms ? 38.4 ?s per loop (mean ? std. dev. of 5 runs, 100 loops each) >>> %%timeit -r5 from collections import deque; f = open('American-english.txt'); consume = deque(maxlen=0).extend ... f.seek(0) ... next(f) # Triggers optimization for all future read_chunk calls ... consume(iter(f.readline, '')) # Otherwise iterated identically ... ... 15.7 ms ? 98.5 ?s per loop (mean ? std. dev. of 5 runs, 100 loops each) The two blocks are identical except that the second one explicitly advances the file one line at the beginning with next(f) to flip `telling` to 0 so future calls to readline don't involve the snapshotting code in textiowrapper_read_chunk. Calling consume(f) would drop the time to 9.86 ms, but that's saying more about the optimization of the raw iterator protocol over method calls than it is about the `telling` optimization; I used two arg iter in both cases to keep the code paths as similar as possible so the `telling`. For reference, the input file was 931708 bytes (931467 characters thanks to a few scattered non-ASCII UTF-8 characters), 98569 lines long. Presumably, the speed difference of 0.1 ms can be chalked up to the telling optimization, so removing it would increase the cost of normal iteration from 9.86 ms to 9.96 ms. That seems de minimis to my mind, in the context of text oriented I/O. Given that, it seems like triggering this optimization via __next__ should be dropped; it's a microoptimization at best, that's mostly irrelevant compared to the overhead of text-oriented I/O, and it introduces undocumented limitations on the use of TextIOWrapper. The changes would be to remove all use of the `telling` variable, and (if we want to keep the optimization for unseekable files, where at least no functionality is lost by having it), change the two tests in textiowrapper_read_chunk to test `seekable` in its place. Or we drop the optimization entirely and save 50+ lines of code that provide a fairly tiny benefit in any event. ---------- components: IO, Library (Lib) messages: 343402 nosy: josh.r priority: normal severity: normal status: open title: Iterating a text file by line should not implicitly disable tell versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 12:48:42 2019 From: report at bugs.python.org (Steve Dower) Date: Fri, 24 May 2019 16:48:42 +0000 Subject: [New-bugs-announce] [issue37037] Enable rpath remapping in makefile Message-ID: <1558716522.41.0.382663630669.issue37037@roundup.psfhosted.org> New submission from Steve Dower : I'd like to be able to build libpython and have it already have the install path stripped out so I can use rpath instead. This libpython is going to be distributed with other applications. Right now, I'm running `install_name_tool -id "@rpath/libpython3.7m.dylib" libpython3.7m.dylib` as a post-build step, (and will then have to figure out what breaks next ;) ) It's probably a simple makefile/autoconf update to add an option to run the necessary commands as part of build, but I can't figure it out myself. So I'm hoping someone can help or has already done it and wants to share. ---------- components: Build messages: 343411 nosy: ned.deily, ronaldoussoren, steve.dower, twouters priority: normal severity: normal stage: needs patch status: open title: Enable rpath remapping in makefile type: enhancement versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 17:56:19 2019 From: report at bugs.python.org (Terry J. Reedy) Date: Fri, 24 May 2019 21:56:19 +0000 Subject: [New-bugs-announce] [issue37038] Make idlelib/run.py runnable. Message-ID: <1558734979.99.0.70787522165.issue37038@roundup.psfhosted.org> New submission from Terry J. Reedy : Slightly simplified, pyshell.ModifiedInterpreter.built_subprocess runs f'{sys.executable} -c "__import__('idlelib.run').run.main()"'. "__import__('idlelib.run')" creates sys.modules['idlelib'] from idlelib/__init__.py, creates sys.modules['idlelib.run'] from idlelib/run.py, binds 'run' to the idlelib.run module in the idlelib module, and returns the idlelib module. It does not bind any names in the main module of the new process. '.run' gets the idlelib.run module and '.main()' executes its main function. This eventually executes user code in the so-far untouched main module. During the creation of the idlelib.run module, various other idlelib module are imported. Some of these import both tkinter and its submodules, such as tkinter.font, adding names such as 'font' to the tkinter modules. To prevent user code running when it should not, these added names are deleted. #25507. Once the deletion code is run, it will fail with AttributeError if run again, such as from an IDLE editor. The patch solves this by adding a flag that indicates that the file has already be imported into the process. ---------- assignee: terry.reedy components: IDLE messages: 343433 nosy: terry.reedy priority: normal severity: normal stage: needs patch status: open title: Make idlelib/run.py runnable. type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 24 21:15:52 2019 From: report at bugs.python.org (Terry J. Reedy) Date: Sat, 25 May 2019 01:15:52 +0000 Subject: [New-bugs-announce] [issue37039] IDLE: Zoomheight restores to default, not previous size Message-ID: <1558746952.08.0.659874103441.issue37039@roundup.psfhosted.org> New submission from Terry J. Reedy : On both Windows and Mac ---------- messages: 343448 nosy: cheryl.sabella, terry.reedy priority: normal severity: normal status: open title: IDLE: Zoomheight restores to default, not previous size type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 00:22:05 2019 From: report at bugs.python.org (Dan Rose) Date: Sat, 25 May 2019 04:22:05 +0000 Subject: [New-bugs-announce] [issue37040] checking for membership in itertools.count enters infinite loop with no way to exit Message-ID: <1558758125.27.0.730728069526.issue37040@roundup.psfhosted.org> New submission from Dan Rose : Checking membership in `itertools.count()` will either return True in linear time or enter an infinite loop that cannot be terminated with Ctrl-c. This ``` import itertools 1 in itertools.count(0,2) ``` It is expected that the above code will use an efficient (O(1)) membership algorithm like range objects. ---------- components: Library (Lib) messages: 343452 nosy: Dan Rose priority: normal severity: normal status: open title: checking for membership in itertools.count enters infinite loop with no way to exit versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 06:26:20 2019 From: report at bugs.python.org (Andre Roberge) Date: Sat, 25 May 2019 10:26:20 +0000 Subject: [New-bugs-announce] [issue37041] IDLE: path browser unusable on some displays Message-ID: <1558779980.36.0.709549198107.issue37041@roundup.psfhosted.org> New submission from Andre Roberge : On my computer (Windows 10, screen resolution 3000 x 2000, scaling of text and other elements set at 200% as the recommended value), the path browser is essentially unusable as the items overlap each other. See the attached image. I found that changing the following: def draw(self, x, y): # XXX This hard-codes too many geometry constants! dy = 40 # changed from 20 in tree.py solved the problem. ---------- files: path_browser_problem.png messages: 343466 nosy: aroberge, terry.reedy priority: normal severity: normal status: open title: IDLE: path browser unusable on some displays Added file: https://bugs.python.org/file48355/path_browser_problem.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 08:09:54 2019 From: report at bugs.python.org (ixje) Date: Sat, 25 May 2019 12:09:54 +0000 Subject: [New-bugs-announce] [issue37042] wait_for(coro, timeout=0) memleak Message-ID: <1558786194.92.0.150909296511.issue37042@roundup.psfhosted.org> New submission from ixje : I have a networked process that looks somewhat like this in its most basic form ``` import asyncio shutting_down = False async def read_message(reader, timeout=30): async def _read(reader: asyncio.StreamReader): try: d = await reader.readexactly(24) # do something with the data print("I'm never called") except: return None try: return await asyncio.wait_for(_read(reader), timeout) except Exception: return None async def my_service(): reader, writer = await asyncio.open_connection('127.0.0.1', 20333) while not shutting_down: m = await read_message(reader, timeout=0) if m is None: continue # else process message if __name__ == "__main__": loop = asyncio.get_event_loop() loop.create_task(my_service()) loop.run_forever() ``` read_message() has a default timeout of 30, but I thought setting it to 0 (instead of None) would be equal to blocking. This bleeds 16GB of memory in ~3 minutes. A minimal example is provided. I manually applied the patch of https://bugs.python.org/issue36613 to a self compiled build of 3.7.3 (https://github.com/python/cpython/commit/ef4ec6ed12d6c6200a85068f60483723298b6ff4) on Ubuntu 18.04 and that did not solve the problem. ---------- components: asyncio files: test_leak_minimal.py messages: 343470 nosy: asvetlov, ixje, yselivanov priority: normal severity: normal status: open title: wait_for(coro, timeout=0) memleak type: resource usage versions: Python 3.7 Added file: https://bugs.python.org/file48356/test_leak_minimal.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 09:14:10 2019 From: report at bugs.python.org (Jason R. Coombs) Date: Sat, 25 May 2019 13:14:10 +0000 Subject: [New-bugs-announce] [issue37043] Buildbots fail when new files are added Message-ID: <1558790050.59.0.505947951024.issue37043@roundup.psfhosted.org> New submission from Jason R. Coombs : As [reported here](https://bugs.python.org/issue34632#msg343445), I submitted a pull request that passed all tests locally and in CI, but when accepted, build bots started to fail as a result of new files having been added to the project. It seems it's necessary in Makefile.pre.in to duplicate the git manifest to declare those directories... but only for buildbot builds. I don't fully understand why that is the case, but it would be nicer if there were protections from this footgun, especially since this issue may manifest several times in the same PR. I can think of some ways to prevent this undesirable behavior: - Include buildbot builds in the GitHub merge request checks. - Add a GitHub bot that checks merge requests that create directories and alert the review of directories created if Makefile.pre.in doesn't include changes reflecting those directories. - Remove the reliance on a separate directory listing in Makefile.pre.in. ---------- components: Tests messages: 343478 nosy: jaraco priority: normal severity: normal status: open title: Buildbots fail when new files are added versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 09:58:32 2019 From: report at bugs.python.org (Jason R. Coombs) Date: Sat, 25 May 2019 13:58:32 +0000 Subject: [New-bugs-announce] [issue37044] Build/test artifacts not ignored for framework build Message-ID: <1558792712.2.0.796235844945.issue37044@roundup.psfhosted.org> New submission from Jason R. Coombs : When developing on macOS, after some build/test operations (I'm not sure exactly which, but seemingly relating to a framework build), artifacts are generated which aren't ignored. As a result, it's easy for them to leak into a merge request as they did with GH-12547 (issue34632). For example: ``` cpython master $ ./configure --enable-framework=/Users/jaraco/Library/Frameworks && make ... cpython master $ git status On branch master Your branch is up to date with 'origin/master'. Untracked files: (use "git add ..." to include in what will be committed) Mac/Resources/app/Info.plist Mac/Resources/framework/Info.plist Python.framework/Python Python.framework/Versions/ nothing added to commit but untracked files present (use "git add" to track) ``` (also Python.framework/Resources/, except that's in the repo at the moment, the reason for reporting this issue) Is there not a reason to ignore these artifacts so they don't risk being added to the commit? ---------- components: macOS messages: 343479 nosy: jaraco, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: Build/test artifacts not ignored for framework build type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 14:29:03 2019 From: report at bugs.python.org (Ivan Levkivskyi) Date: Sat, 25 May 2019 18:29:03 +0000 Subject: [New-bugs-announce] [issue37045] Implement PEP 591: add Final qualifier and @final decorator to typing Message-ID: <1558808943.82.0.267698974314.issue37045@roundup.psfhosted.org> New submission from Ivan Levkivskyi : The actual implementation is performed by type checkers like mypy. We just need to add the names to the `typing` module. ---------- assignee: levkivskyi components: Library (Lib) messages: 343501 nosy: gvanrossum, levkivskyi, msullivan priority: normal severity: normal stage: needs patch status: open title: Implement PEP 591: add Final qualifier and @final decorator to typing type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 15:08:01 2019 From: report at bugs.python.org (Ivan Levkivskyi) Date: Sat, 25 May 2019 19:08:01 +0000 Subject: [New-bugs-announce] [issue37046] Implement PEP 586: add Literal type constructor to typing Message-ID: <1558811281.19.0.407876730009.issue37046@roundup.psfhosted.org> New submission from Ivan Levkivskyi : The actual implementation is performed by type checkers like mypy. We just need to add `Literal` to the `typing` module. ---------- assignee: levkivskyi components: Library (Lib) messages: 343503 nosy: gvanrossum, levkivskyi, michael0x2a priority: normal severity: normal stage: needs patch status: open title: Implement PEP 586: add Literal type constructor to typing type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 15:38:44 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Sat, 25 May 2019 19:38:44 +0000 Subject: [New-bugs-announce] [issue37047] Refactor AsyncMock setup logic in create_autospec Message-ID: <1558813124.86.0.778430760626.issue37047@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : * In create_autospec there is some logic to detect if the function is an async function this could be refactored out as a private function. * create_autospec has initialization code for async mock. For synchronous functions this is done with _setup_func and is called during setting signature. AsyncMock needs different setup logic code to setup the functions but the code can be refactored out of create_autospec into _setup_async_func. * In create_autospec awaited attribute is not set for AsyncMock during setup [0] and hence this causes AttributeError when the coroutine is awaited. awaited attribute can be also initialized. import asyncio from unittest.mock import create_autospec async def foo(): pass spec = create_autospec(foo) awaitable = spec() async def main(): await awaitable asyncio.run(main()) Traceback (most recent call last): File "/tmp/spam.py", line 13, in asyncio.run(main()) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/runners.py", line 43, in run return loop.run_until_complete(main) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/base_events.py", line 614, in run_until_complete return future.result() File "/tmp/spam.py", line 11, in main await awaitable File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 2045, in _mock_call return await proxy() File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 2043, in proxy await self.awaited._notify() File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 596, in __getattr__ raise AttributeError("Mock object has no attribute %r" % name) AttributeError: Mock object has no attribute 'awaited' * In the setup logic to create attributes like assert_not_awaited uses the pattern setattr(mock, a, f) at [1] . But due to late binding 'a' in the function f has the last value of the loop 'assert_not_awaited' and hence calling other helpers also calls assert_not_awaited. This could be resolved by using a partial function that binds the attribute value early in the loop and respective function would be used in getattr. >>> spec.assert_awaited_once_with(1) # Due to late binding assert_not_awaited is always called TypeError: assert_not_awaited() takes 1 positional argument but 2 were given * assert_not_awaited has the error message indicating it should be awaited once [2] . This can be changed to indicate something like "Expected mock to not have been awaited". >>> spec.assert_not_awaited() AssertionError: Expected mock to have been awaited once. Awaited 1 times. * mock docs have list of magic methods implemented where __aenter__, __aexit__, __aiter__ and __anext__ could be documented with versionadded directive. [3] I have a PR with the above changes that I will post shortly for review. [0] https://github.com/python/cpython/blob/7114c6504a60365b8b0cd718da0ec8a737599fb9/Lib/unittest/mock.py#L2506 [1] https://github.com/python/cpython/blob/7114c6504a60365b8b0cd718da0ec8a737599fb9/Lib/unittest/mock.py#L2518 [2] https://github.com/python/cpython/blob/7114c6504a60365b8b0cd718da0ec8a737599fb9/Lib/unittest/mock.py#L2154 [3] https://docs.python.org/3.8/library/unittest.mock.html#mocking-magic-methods ---------- components: Library (Lib) messages: 343504 nosy: asvetlov, cjw296, lisroach, mariocj89, michael.foord, xtreak, yselivanov priority: normal severity: normal status: open title: Refactor AsyncMock setup logic in create_autospec type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 15:44:14 2019 From: report at bugs.python.org (Christian Heimes) Date: Sat, 25 May 2019 19:44:14 +0000 Subject: [New-bugs-announce] [issue37048] ssl module: QUIC support for HTTP/3 Message-ID: <1558813454.34.0.910569571788.issue37048@roundup.psfhosted.org> New submission from Christian Heimes : This ticket collects information for QUIC [1][2] support and tracks, which APIs have to be added to Python in order to implement a QUIC protocol stack on top of Python's ssl and socket module. QUIC is a "UDP-Based Multiplexed and Secure Transport" protocol. It will replace TCP and TLS record layer as transport channels in the upcoming HTTP/3 [3][4] standard. Although it's UDP, QUIC does *not* use DTLS (Datagram TLS, vulgo TLS over UDP). As far as I understand QUIC at the moment, the ssl module has to gain two additional features: 1. A way to send/receive TLS messages that are not wrapped in the TLS record layer. 2. A key callback that gets called whenever key material is exchanged during handshake or updated later on. OpenSSL does not implement the necessary APIs, yet [5]. Tatsuhiro Tsujikawa's experimental OpenSSL fork [6] implements (1) as a SSL option SSL_MODE_QUIC_HACK and (2) as a callback that acts on five different key types. (Disclaimer: My current understanding of QUIC is very limited.) [1] https://tools.ietf.org/html/draft-ietf-quic-transport-20 [2] https://en.wikipedia.org/wiki/QUIC [2] https://http3-explained.haxx.se/en/ [4] https://en.wikipedia.org/wiki/HTTP/3 [5] https://daniel.haxx.se/blog/2019/01/21/quic-and-missing-apis/ [6] https://github.com/tatsuhiro-t/openssl/commits/quic-draft-17 ---------- assignee: christian.heimes components: SSL messages: 343505 nosy: alex, christian.heimes, dstufft, janssen, njs priority: normal severity: normal status: open title: ssl module: QUIC support for HTTP/3 type: enhancement versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 15:44:49 2019 From: report at bugs.python.org (Ivan Levkivskyi) Date: Sat, 25 May 2019 19:44:49 +0000 Subject: [New-bugs-announce] [issue37049] Implement PEP 589: add TypedDict to typing Message-ID: <1558813489.97.0.456123370037.issue37049@roundup.psfhosted.org> New submission from Ivan Levkivskyi : The actual implementation is performed by type checkers like mypy. We just need to add `TypedDict` to the `typing` module. ---------- assignee: levkivskyi components: Library (Lib) messages: 343506 nosy: gvanrossum, levkivskyi priority: normal severity: normal stage: needs patch status: open title: Implement PEP 589: add TypedDict to typing type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 16:57:14 2019 From: report at bugs.python.org (Eric V. Smith) Date: Sat, 25 May 2019 20:57:14 +0000 Subject: [New-bugs-announce] [issue37050] Remove expr_text from ast node FormattedValue Message-ID: <1558817834.95.0.692781617754.issue37050@roundup.psfhosted.org> New submission from Eric V. Smith : I added the expr_text optional field to the FormattedValue node in order to implement the '=' feature of f-strings (see issue 36817). However, the expr_text field isn't strictly needed. Instead, the same feature could be added with another Constant string node child of the JoinedStr node. I'm going to remove expr_text and use another Constant in order to remove this change to the 3.8 ast nodes. I have a patch mostly worked out, I'll have it ready in the next day or two. I want to get this in to 3.8 beta 1, because otherwise we're stuck with the expr_text implementation. ---------- assignee: eric.smith components: Interpreter Core messages: 343511 nosy: eric.smith, lukasz.langa priority: release blocker severity: normal stage: needs patch status: open title: Remove expr_text from ast node FormattedValue versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat May 25 17:10:53 2019 From: report at bugs.python.org (John Riehl) Date: Sat, 25 May 2019 21:10:53 +0000 Subject: [New-bugs-announce] [issue37051] Glossary item "hashable" incorrect Message-ID: <1558818653.08.0.592312048855.issue37051@roundup.psfhosted.org> New submission from John Riehl : The entry in the glossary for "hashable" (https://docs.python.org/3/glossary.html#term-hashable) states "All of Python?s immutable built-in objects are hashable." Tuples are described as immutable sequence types (https://docs.python.org/3/library/stdtypes.html#immutable-sequence-types), but they are not hashable unless all of their elements are hashable. Suggest updating the glossary to reflect this. ---------- assignee: docs at python components: Documentation messages: 343514 nosy: docs at python, john.riehl priority: normal severity: normal status: open title: Glossary item "hashable" incorrect type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 03:51:03 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Sun, 26 May 2019 07:51:03 +0000 Subject: [New-bugs-announce] [issue37052] Add examples for mocking async for and async context manager in unittest.mock docs Message-ID: <1558857063.08.0.951093658338.issue37052@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : Since issue26467 implemented AsyncMock along with async dunder methods for MagicMock it enables users to mock async for and async with statements. Currently examples of how to use this is present only in tests and would be good to add it to docs. There is a docs page for mock that contains similar cookbook style examples [0] where I hope these can be added. I can raise a PR with these examples if it's okay. Do you think it's worthy enough to add these examples? Any additional examples you find around asyncio and mock that can be documented ? An example of mocking async for statement by setting return value for __aiter__ method : # aiter_example.py import asyncio from unittest.mock import MagicMock mock = MagicMock() mock.__aiter__.return_value = range(3) async def main(): print([i async for i in mock]) asyncio.run(main()) $ ./python.exe aiter_example.py [0, 1, 2] An example of mocking async with statement by implementing __aenter__ and __aexit__ method. In this example __aenter__ and __aexit__ are not called. __aenter__ and __aexit__ implementations are tested to have been called in the test at [1]. These tests work since MagicMock is returned during attribute access (mock_instance.entered) which is always True in boolean context under assertTrue. I will raise a separate PR to discuss this since normally while mocking __enter__ and __exit__ the class's __enter__ and __exit__ are not used as a side_effect for the mock calls unless they are set explicitly. # aenter_example.py import asyncio from unittest.mock import MagicMock class WithAsyncContextManager: async def __aenter__(self, *args, **kwargs): return self async def __aexit__(self, *args, **kwargs): pass instance = WithAsyncContextManager() mock_instance = MagicMock(instance) async def main(): async with mock_instance as result: print("entered") asyncio.run(main()) ./python.exe aenter_example.py entered [0] https://docs.python.org/3/library/unittest.mock-examples.html [1] https://github.com/python/cpython/blob/47dd2f9fd86c32a79e77fef1fbb1ce25dc929de6/Lib/unittest/test/testmock/testasync.py#L306 ---------- assignee: docs at python components: Documentation messages: 343536 nosy: asvetlov, cjw296, docs at python, lisroach, mariocj89, michael.foord, xtreak, yselivanov priority: normal severity: normal status: open title: Add examples for mocking async for and async context manager in unittest.mock docs type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 08:41:33 2019 From: report at bugs.python.org (Chih-Hsuan Yen) Date: Sun, 26 May 2019 12:41:33 +0000 Subject: [New-bugs-announce] [issue37053] Tools/parser/unparse.py does not handle u"bar" correctly Message-ID: <1558874493.33.0.876031748189.issue37053@roundup.psfhosted.org> New submission from Chih-Hsuan Yen : Currently (135c6a56e55d2f4f8718b3b9f03ce3c692b15f0f) the following test fails: ./python -m test.regrtest -v test_tools.test_unparse -u cpu (Note that -u cpu is needed to force test_unparse.DirectoryTestCase to check all files under Lib/ and Lib/test/.) An example error message: ====================================================================== FAIL: test_files (test.test_tools.test_unparse.DirectoryTestCase) (filename='/home/yen/Projects/cpython/Lib/test/test_typing.py' ) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yen/Projects/cpython/Lib/test/test_tools/test_unparse.py", line 309, in test_files self.check_roundtrip(source) File "/home/yen/Projects/cpython/Lib/test/test_tools/test_unparse.py", line 132, in check_roundtrip self.assertASTEqual(ast1, ast2) File "/home/yen/Projects/cpython/Lib/test/test_tools/test_unparse.py", line 124, in assertASTEqual self.assertEqual(ast.dump(ast1), ast.dump(ast2)) AssertionError: 'Modu[88178 chars]kind=\'u\')], ctx=Load())), ctx=Load()))], dec[421987 chars]=[])' != 'Modu[88178 chars]kind=No ne)], ctx=Load())), ctx=Load()))], deco[421986 chars]=[])' ---------------------------------------------------------------------- Apparently that's because Tools/parser/unparse.py does not handle strings like u"bar" correctly. I managed to "fix" it with the following patch: diff --git a/Tools/parser/unparse.py b/Tools/parser/unparse.py index 385902ef4b..a25b5e49df 100644 --- a/Tools/parser/unparse.py +++ b/Tools/parser/unparse.py @@ -399,6 +399,8 @@ class Unparser: elif value is ...: self.write("...") else: + if t.kind == 'u': + self.write("u") self._write_constant(t.value) def _List(self, t): Not sure if this is the correct approach, though. ---------- components: Demos and Tools messages: 343546 nosy: yan12125 priority: normal severity: normal status: open title: Tools/parser/unparse.py does not handle u"bar" correctly type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 10:54:40 2019 From: report at bugs.python.org (Antoine Pitrou) Date: Sun, 26 May 2019 14:54:40 +0000 Subject: [New-bugs-announce] [issue37054] Ignored exceptions in test_memoryio Message-ID: <1558882480.22.0.119118964732.issue37054@roundup.psfhosted.org> New submission from Antoine Pitrou : I see the following kind of exceptions when running test_memoryio: Exception ignored in: Traceback (most recent call last): File "/home/antoine/cpython/default/Lib/_pyio.py", line 409, in __del__ self.close() File "/home/antoine/cpython/default/Lib/_pyio.py", line 2152, in close if self.buffer is not None and not self.closed: File "/home/antoine/cpython/default/Lib/_pyio.py", line 2093, in buffer return self._buffer AttributeError: 'StringIO' object has no attribute '_buffer' Exception ignored in: Traceback (most recent call last): File "/home/antoine/cpython/default/Lib/_pyio.py", line 409, in __del__ self.close() File "/home/antoine/cpython/default/Lib/_pyio.py", line 2152, in close if self.buffer is not None and not self.closed: File "/home/antoine/cpython/default/Lib/_pyio.py", line 2093, in buffer return self._buffer AttributeError: 'StringIO' object has no attribute '_buffer' ok It seems this could be related to issue18748. ---------- components: Library (Lib), Tests messages: 343552 nosy: pitrou, vstinner priority: normal severity: normal status: open title: Ignored exceptions in test_memoryio type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 12:08:52 2019 From: report at bugs.python.org (=?utf-8?q?R=C3=A9mi_Lapeyre?=) Date: Sun, 26 May 2019 16:08:52 +0000 Subject: [New-bugs-announce] [issue37055] Numerous warnings with blake2 module Message-ID: <1558886932.4.0.665756848832.issue37055@roundup.psfhosted.org> New submission from R?mi Lapeyre : Issue 33164 (commit 51aa35e9e17eef60d04add9619fe2a7eb938358c) introduced numerous new warnings on MacOS: /Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:192:9: warning: 'blake2b_init' macro redefined [-Wmacro-redefined] #define blake2b_init BLAKE2_IMPL_NAME(blake2b_init) ^ /Users/remi/src/cpython/Modules/_blake2/blake2ns.h:10:9: note: previous definition is here #define blake2b_init PyBlake2_blake2b_init ^ In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32: /Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:193:9: warning: 'blake2b_init_param' macro redefined [-Wmacro-redefined] #define blake2b_init_param BLAKE2_IMPL_NAME(blake2b_init_param) ^ /Users/remi/src/cpython/Modules/_blake2/blake2ns.h:12:9: note: previous definition is here #define blake2b_init_param PyBlake2_blake2b_init_param ^ In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32: /Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:194:9: warning: 'blake2b_init_key' macro redefined [-Wmacro-redefined] #define blake2b_init_key BLAKE2_IMPL_NAME(blake2b_init_key) ^ /Users/remi/src/cpython/Modules/_blake2/blake2ns.h:11:9: note: previous definition is here #define blake2b_init_key PyBlake2_blake2b_init_key ^ In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32: /Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:195:9: warning: 'blake2b_update' macro redefined [-Wmacro-redefined] #define blake2b_update BLAKE2_IMPL_NAME(blake2b_update) ^ /Users/remi/src/cpython/Modules/_blake2/blake2ns.h:13:9: note: previous definition is here #define blake2b_update PyBlake2_blake2b_update ^ In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32: /Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:196:9: warning: 'blake2b_final' macro redefined [-Wmacro-redefined] #define blake2b_final BLAKE2_IMPL_NAME(blake2b_final) ^ /Users/remi/src/cpython/Modules/_blake2/blake2ns.h:9:9: note: previous definition is here #define blake2b_final PyBlake2_blake2b_final ^ In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32: /Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:197:9: warning: 'blake2b' macro redefined [-Wmacro-redefined] #define blake2b BLAKE2_IMPL_NAME(blake2b) ^ /Users/remi/src/cpython/Modules/_blake2/blake2ns.h:7:9: note: previous definition is here #define blake2b PyBlake2_blake2b I'm not sure what should be the public API exactly and where to fix that. ---------- messages: 343556 nosy: remi.lapeyre priority: normal severity: normal status: open title: Numerous warnings with blake2 module _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 12:36:06 2019 From: report at bugs.python.org (Pablo Galindo Salgado) Date: Sun, 26 May 2019 16:36:06 +0000 Subject: [New-bugs-announce] [issue37056] test_tools is failing on several buildbots Message-ID: <1558888566.71.0.0627400605993.issue37056@roundup.psfhosted.org> New submission from Pablo Galindo Salgado : BUILDBOT FAILURE REPORT ======================= Builder name: AMD64 Debian root 3.x Builder url: https://buildbot.python.org/all/#/builders/27/ Build url: https://buildbot.python.org/all/#/builders/27/builds/3006 Failed tests ------------ - test_files (test.test_tools.test_unparse.DirectoryTestCase) (filename='/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_typing.py') Test leaking resources ---------------------- Build summary ------------- == Tests result: FAILURE then FAILURE == 401 tests OK. 10 slowest tests: - test_concurrent_futures: 2 min 57 sec - test_multiprocessing_spawn: 2 min 22 sec - test_tools: 2 min 3 sec - test_multiprocessing_forkserver: 1 min 31 sec - test_multiprocessing_fork: 1 min 10 sec - test_tokenize: 1 min 2 sec - test_asyncio: 48 sec 814 ms - test_lib2to3: 42 sec 344 ms - test_io: 41 sec 325 ms - test_nntplib: 41 sec 260 ms 1 test failed: test_tools 20 tests skipped: test_devpoll test_gdb test_idle test_ioctl test_kqueue test_msilib test_ossaudiodev test_smtpnet test_ssl test_startfile test_tcl test_tix test_tk test_ttk_guionly test_ttk_textonly test_turtle test_winconsoleio test_winreg test_winsound test_zipfile64 1 re-run test: test_tools Total duration: 15 min 26 sec Tracebacks ---------- ```traceback Traceback (most recent call last): File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_tools/test_unparse.py", line 309, in test_files self.check_roundtrip(source) File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_tools/test_unparse.py", line 132, in check_roundtrip self.assertASTEqual(ast1, ast2) File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_tools/test_unparse.py", line 124, in assertASTEqual self.assertEqual(ast.dump(ast1), ast.dump(ast2)) AssertionError: 'Modu[88178 chars]kind=\'u\')], ctx=Load())), ctx=Load()))], dec[421987 chars]=[])' != 'Modu[88178 chars]kind=None)], ctx=Load())), ctx=Load()))], deco[421986 chars]=[])' ``` Current builder status ---------------------- The builder is failing currently Other builds with similar failures ---------------------------------- - https://buildbot.python.org/all/#/builders/99/builds/2698 - https://buildbot.python.org/all/#/builders/99/builds/2699 - https://buildbot.python.org/all/#/builders/99/builds/2700 - https://buildbot.python.org/all/#/builders/85/builds/2834 - https://buildbot.python.org/all/#/builders/85/builds/2835 - https://buildbot.python.org/all/#/builders/85/builds/2836 - https://buildbot.python.org/all/#/builders/40/builds/2326 - https://buildbot.python.org/all/#/builders/40/builds/2327 - https://buildbot.python.org/all/#/builders/40/builds/2328 - https://buildbot.python.org/all/#/builders/176/builds/541 - https://buildbot.python.org/all/#/builders/176/builds/542 - https://buildbot.python.org/all/#/builders/176/builds/543 - https://buildbot.python.org/all/#/builders/141/builds/1824 - https://buildbot.python.org/all/#/builders/141/builds/1825 - https://buildbot.python.org/all/#/builders/145/builds/1705 - https://buildbot.python.org/all/#/builders/145/builds/1706 - https://buildbot.python.org/all/#/builders/145/builds/1707 - https://buildbot.python.org/all/#/builders/167/builds/1069 - https://buildbot.python.org/all/#/builders/167/builds/1070 - https://buildbot.python.org/all/#/builders/167/builds/1071 - https://buildbot.python.org/all/#/builders/3/builds/2740 - https://buildbot.python.org/all/#/builders/3/builds/2741 - https://buildbot.python.org/all/#/builders/3/builds/2742 - https://buildbot.python.org/all/#/builders/47/builds/2875 - https://buildbot.python.org/all/#/builders/47/builds/2876 - https://buildbot.python.org/all/#/builders/27/builds/3007 - https://buildbot.python.org/all/#/builders/27/builds/3008 - https://buildbot.python.org/all/#/builders/53/builds/3013 - https://buildbot.python.org/all/#/builders/53/builds/3014 - https://buildbot.python.org/all/#/builders/53/builds/3015 - https://buildbot.python.org/all/#/builders/21/builds/3027 - https://buildbot.python.org/all/#/builders/21/builds/3028 - https://buildbot.python.org/all/#/builders/21/builds/3029 - https://buildbot.python.org/all/#/builders/13/builds/3040 - https://buildbot.python.org/all/#/builders/13/builds/3041 - https://buildbot.python.org/all/#/builders/13/builds/3042 - https://buildbot.python.org/all/#/builders/12/builds/2557 - https://buildbot.python.org/all/#/builders/12/builds/2558 - https://buildbot.python.org/all/#/builders/12/builds/2559 ---------- components: Tests messages: 343561 nosy: pablogsal priority: normal severity: normal status: open title: test_tools is failing on several buildbots versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 13:30:05 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Sun, 26 May 2019 17:30:05 +0000 Subject: [New-bugs-announce] [issue37057] suspicious.py sphinx extension access non-existing attribute. Message-ID: <1558891805.96.0.441991162477.issue37057@roundup.psfhosted.org> New submission from Matthias Bussonnier : On a local branch with some modified docs ``` Traceback (most recent call last): File "/Users/bussonniermatthias/dev/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/cmd/build.py", line 284, in build_main app.build(args.force_all, filenames) File "/Users/bussonniermatthias/dev/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/application.py", line 337, in build self.builder.build_update() File "/Users/bussonniermatthias/dev/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 324, in build_update self.build(to_build, File "/Users/bussonniermatthias/dev/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 392, in build self.finish() File "/Users/bussonniermatthias/dev/cpython/Doc/tools/extensions/suspicious.py", line 118, in finish self.warn('Found %s/%s unused rules:' % AttributeError: 'CheckSuspiciousMarkupBuilder' object has no attribute 'warn' ``` My guess is that it should be self.logger.warn, and not self.warn. ---------- messages: 343569 nosy: mbussonn priority: normal severity: normal status: open title: suspicious.py sphinx extension access non-existing attribute. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 14:52:14 2019 From: report at bugs.python.org (Ivan Levkivskyi) Date: Sun, 26 May 2019 18:52:14 +0000 Subject: [New-bugs-announce] [issue37058] Implement PEP 544: add Protocol to typing Message-ID: <1558896734.9.0.411733767258.issue37058@roundup.psfhosted.org> New submission from Ivan Levkivskyi : The implementation would be basically copying some code from `typing_extension` and fixing some important known issues (such as renaming @runtime to @runtime_checkable and allowing sub-protocols of builtin protocols). Also ideally we should get rid of the metaclass `_ProtocolMeta`. ---------- assignee: levkivskyi components: Library (Lib) messages: 343576 nosy: gvanrossum, levkivskyi, lukasz.langa priority: normal severity: normal stage: needs patch status: open title: Implement PEP 544: add Protocol to typing type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 16:28:30 2019 From: report at bugs.python.org (Erika) Date: Sun, 26 May 2019 20:28:30 +0000 Subject: [New-bugs-announce] [issue37059] Pre-compile Directive to Remind Coders that Men are Pigs Message-ID: <1558902510.44.0.685577940512.issue37059@roundup.psfhosted.org> New submission from Erika : Guido Van Rossum recently let the community know that sexism is keeping women away from Python. A simple directive at the top of each py file would help remind us. Throw an error unless #MenArePigs is not in line-1 of the file. ---------- assignee: docs at python components: Documentation messages: 343580 nosy: docs at python, erika1998 priority: normal severity: normal status: open title: Pre-compile Directive to Remind Coders that Men are Pigs versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 20:02:41 2019 From: report at bugs.python.org (Gregory Szorc) Date: Mon, 27 May 2019 00:02:41 +0000 Subject: [New-bugs-announce] [issue37060] import ctypes fails with a statically linked interpreter due to dlopen() failure Message-ID: <1558915361.55.0.999227161151.issue37060@roundup.psfhosted.org> New submission from Gregory Szorc : ctypes/__init__.py calls _ctypes.dlopen(None) on Linux as part of code execution during module import. Unfortunately, dlopen() doesn't work if the current executable isn't a dynamic executable. Using a fully statically linked Python executable: $ ldd python3.7 not a dynamic executable $ python3.7 >>> import _ctypes >>> _ctypes.dlopen(None) Traceback (most recent call last): File "", line 1, in OSError: Dynamic loading not supported >>> import ctypes Traceback (most recent call last): File "", line 1, in File "/home/gps/src/python-build-standalone.git/build/python/install/lib/python3.7/ctypes/__init__.py", line 444, in pythonapi = PyDLL(None) File "/home/gps/src/python-build-standalone.git/build/python/install/lib/python3.7/ctypes/__init__.py", line 356, in __init__ self._handle = _dlopen(self._name, mode) OSError: Dynamic loading not supported I think it is a bug that `import ctypes` raises OSError at import time in this environment. I can make a compelling argument that this error should either be suppressed or converted to an ImportError. Producing a fully statically linked Python executable is a bit of work and isn't easily accomplished with the existing build system. My "python-build-standalone" project automates the process. A fully statically linked Python executable is available in the zstd compressed archive at https://github.com/indygreg/python-build-standalone/releases/download/20190505/cpython-3.7.3-linux64-musl-20190526T0219.tar.zst under the python/install/bin/python3.7 path. Simply extract that archive and run that binary to reproduce. ---------- components: ctypes messages: 343588 nosy: indygreg priority: normal severity: normal status: open title: import ctypes fails with a statically linked interpreter due to dlopen() failure versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun May 26 21:17:32 2019 From: report at bugs.python.org (Bob Vegene) Date: Mon, 27 May 2019 01:17:32 +0000 Subject: [New-bugs-announce] [issue37061] The strangest glitch I have ever seen - incorrect indenterror, even on commented out code. Message-ID: <1558919852.84.0.963565120653.issue37061@roundup.psfhosted.org> New submission from Bob Vegene : So, I'm making a simple program that will allow me to quickly copy programs to System32 for use on the command line. When I tried testing this in a file called test.py in the same directory as sys32move.py, I got a very strange error. IndentError. I am very experienced with python and I know how to fix this and why it is caused. I fixed what I thought I could, but the I realized something was off. I commented out everything in sys32move.py and it still outputs this error. Now, commenting this out is unnecessary, as this is a false error anyway. This is test.py: try: import sys32pack install_folder(testfolder) except Exception as e: print(e) input() (you get same behavior without the catching of the error.) SYS32PACK.py: ---------- components: Build, Cross-Build files: sys32pack.py messages: 343591 nosy: Alex.Willmer, bob.vegena priority: normal severity: normal status: open title: The strangest glitch I have ever seen - incorrect indenterror, even on commented out code. type: behavior versions: Python 3.9 Added file: https://bugs.python.org/file48363/sys32pack.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 02:03:00 2019 From: report at bugs.python.org (Reuben Thomas) Date: Mon, 27 May 2019 06:03:00 +0000 Subject: [New-bugs-announce] [issue37062] `AutoNumber` class in enum documentation: support *args in constructor Message-ID: <1558936980.63.0.0585299344314.issue37062@roundup.psfhosted.org> New submission from Reuben Thomas : By changing one line of AutoNumber: def __new__(cls): to def __new__(cls, *args): Enums derived from AutoNumber can now support constructors that take named arguments; for example: class Color(AutoNumber): def __init__(self, pantone=None): self.pantone = pantone or 'unknown' class Swatch(Color): AUBURN = ('3497') SEA_GREEN = ('1246') BLEACHED_CORAL = () # New color, no Pantone code yet! Without the change, one gets the error: TypeError: __new__() takes 1 positional argument but 2 were given I attach runnable demonstration code. ---------- assignee: docs at python components: Documentation files: foo.py messages: 343607 nosy: docs at python, rrt priority: normal severity: normal status: open title: `AutoNumber` class in enum documentation: support *args in constructor versions: Python 3.8 Added file: https://bugs.python.org/file48365/foo.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 06:57:38 2019 From: report at bugs.python.org (Dan Snider) Date: Mon, 27 May 2019 10:57:38 +0000 Subject: [New-bugs-announce] [issue37063] Incorrect application of func.__defaults__ by inspect's signature APIs Message-ID: <1558954658.26.0.250057323757.issue37063@roundup.psfhosted.org> New submission from Dan Snider : The interpreter matches the values in func.__defaults__ with the calculated positional argument names "right-to-left", while inspect does does so "left-to-right". In every day usage, for "normal" functions, generated by the compiler from "normal", hand-typed source code, it is impossible to encounter this. >>> def f(x): return x >>> x.__defaults__ = (1,2,3) >>> inspect.getfullargspec(f) FullArgSpec(args=['x'], varargs=None, varkw=None, defaults=(1,), kwonlyargs=[], kwonlydefaults=None, annotations={}) >>> f() 3 >>> f.__defaults__ (1, 2, 3) So I'll be honest: I've submitted this particularly inconsequential bug report as a pretext to ask for advice on how to contribute here, because frankly, this is getting out of hand. While I haven't counted, it would not surprise me if I have more than 20 open tickets on here currently. While that may not be uncommon it does bother me slightly. And those are just the ones I had the time and/or remembered to to submit a ticket for. There's another 45-50 whose location I'm certain of, fully documented and fixed. If this little digression is against some sort of rules, then I apologize now and must concur. That being said, despite my lack of a formal education, my seemingly never-ending quest to push the interpreter to its limits, hunting for the most esoteric of optimizations (and flaws) has turned out to be quite the hobby. I've aquired a great deal of knowledge on the subject which, in the grand scheme of things is being wasted, as the only thing of actual significance I could see myself doing with it is using it to contribute here. But when I consider the amount of time I've wasted on the matter already on this, I've totally lost faith in my own ability to judge what might constitute a good resource on the subject of GH. The other venues I'm aware of haven't been helpful to me so I don't even bother with them anymore. I will read the dev guide again, since it has been a while. However to my recollection the problems I had with it wasn't technicality per se, it was that it was it was full of totally foreign terminology to me, with words and phrases such as "pulling on this while the squisher of that squasher pushes on that". ---------- components: Library (Lib) messages: 343613 nosy: bup priority: normal severity: normal status: open title: Incorrect application of func.__defaults__ by inspect's signature APIs type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 09:09:47 2019 From: report at bugs.python.org (Patrik Kopkan) Date: Mon, 27 May 2019 13:09:47 +0000 Subject: [New-bugs-announce] [issue37064] Feature request: option to keep/add flags to pathfix. Message-ID: <1558962587.87.0.563974081614.issue37064@roundup.psfhosted.org> New submission from Patrik Kopkan : Hello, Pathfix is nice simple tool for changing shebang lines. However, it is lacking a way how to keep/add flags making this tool impractical sometimes. ---------- messages: 343622 nosy: pkopkan priority: normal severity: normal status: open title: Feature request: option to keep/add flags to pathfix. _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 11:11:54 2019 From: report at bugs.python.org (Denis S. Otkidach) Date: Mon, 27 May 2019 15:11:54 +0000 Subject: [New-bugs-announce] [issue37065] File and lineno is not reported for syntax error in f-string Message-ID: <1558969914.07.0.850758121851.issue37065@roundup.psfhosted.org> New submission from Denis S. Otkidach : Minimal example to reproduce: -->8-- >>> with open('f_bug.py', 'w') as fp: ... fp.write('f"{a b}"') ... 8 >>> import f_bug Traceback (most recent call last): File "", line 1, in File "", line 1 (a b) ^ SyntaxError: invalid syntax -->8-- Here we see in track trace "" and line number in erroneous expression in f-string, but no "f_bug.py" and line number in it. ---------- components: Interpreter Core messages: 343639 nosy: ods priority: normal severity: normal status: open title: File and lineno is not reported for syntax error in f-string type: behavior versions: Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 12:06:15 2019 From: report at bugs.python.org (Siming Yuan) Date: Mon, 27 May 2019 16:06:15 +0000 Subject: [New-bugs-announce] [issue37066] os.execl opening a new bash shell doesn't work if initfile/rcfile provided Message-ID: <1558973175.26.0.897834780365.issue37066@roundup.psfhosted.org> New submission from Siming Yuan : Using os.execl you can open a new bash shell (eg, using python to process some env before opening a new shell. $ echo $SHLVL 1 $ ~/.pyenv/versions/3.6.4/bin/python3 Python 3.6.4 (default, Feb 5 2018, 16:53:35) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.execl('/bin/bash', '') $ echo $SHLVL 2 Doing the above works with just /bin/bash no arguments. (notice SHLVL incrementing) But providing your own custom --init-file or --rcfile, doesn't. eg - /bin/bashrc --rcfile $ echo $SHLVL 1 $ ~/.pyenv/versions/3.6.4/bin/python3 Python 3.6.4 (default, Feb 5 2018, 16:53:35) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.execl('/bin/bash', '--rcfile','/users/me/venv/bin/activate') $ echo $SHLVL 1 this can be replicated in Python 3.5 to 3.7 can be worked-around if using a wrapper.sh file with: #! /bin/bash exec /bin/bash --rcfile /users/me/venv/bin/activate and running this file in os.execl() instead. ---------- messages: 343654 nosy: siming85 priority: normal severity: normal status: open title: os.execl opening a new bash shell doesn't work if initfile/rcfile provided versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 12:08:47 2019 From: report at bugs.python.org (Siming Yuan) Date: Mon, 27 May 2019 16:08:47 +0000 Subject: [New-bugs-announce] [issue37067] os.execl doesn't allow for empty string in mac Message-ID: <1558973327.05.0.558912294588.issue37067@roundup.psfhosted.org> New submission from Siming Yuan : the following works in Linux import os os.execl('/bin/bash', '') doesn't in mac: >>> import os >>> os.execl('/bin/bash', '') Traceback (most recent call last): File "", line 1, in File "/Users/me/.pyenv/versions/3.6.4/lib/python3.6/os.py", line 527, in execl execv(file, args) ValueError: execv() arg 2 first element cannot be empty works if you add a space >>> os.execl('/bin/bash', ' ') notice the space in 2nd argument. technically it is also possible to run a command without arguments - why not allow for the case where *args is []? >>> os.execl('/bin/bash') Traceback (most recent call last): File "", line 1, in File "/Users/siyuan/.pyenv/versions/3.6.4/lib/python3.6/os.py", line 527, in execl execv(file, args) ValueError: execv() arg 2 must not be empty >>> ---------- components: macOS messages: 343655 nosy: ned.deily, ronaldoussoren, siming85 priority: normal severity: normal status: open title: os.execl doesn't allow for empty string in mac versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 15:11:14 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Mon, 27 May 2019 19:11:14 +0000 Subject: [New-bugs-announce] [issue37068] Emit SyntaxWarning for f-strings without expressions ? Message-ID: <1558984274.64.0.992654624102.issue37068@roundup.psfhosted.org> New submission from Matthias Bussonnier : Or more precisely I suggest to emit a SyntaxWarning for JoinedStr where no f-string members has an expression. There seem to be a couple of case where f-strings without expressions may indicate that the programmer may have made a mistake; though there are also a number of case where f-string w/o expressions are legitimate; I believe we should be able to emit a SyntaxWarning in case where `f` are really unnecessary or might be a mistakes. First case where f-string w/o expressions are legitimate: multiline joined string, to make sure each line is aligned. Example from CPython source: # not a SyntaxWarning. msg = ( f'error while attempting to bind on ' f'address {laddr!r}: ' f'{exc.strerror.lower()}' ) The first line has obviously no expression, but the f is useful for visual consistency, and we likely do not want a warning. Though in the above case we can imagine the following typo : msg = ( f'error while attempting to bind on ', #SyntaxWarning here f'address {laddr!r}: ', f'{exc.strerror.lower()}' ) Easy to make and in this case, the expression-less f-string is likely an error. In this case a syntax warning would help to distinguish that there are trailing commas. Another case from the cpython is the following: fullName = f'%s.%s.%s' % ( #SyntaxWarning here testCaseClass.__module__, testCaseClass.__qualname__, attrname ) Looking at the history; I believe the author was meaning to change to an f-string, but got interrupted half-way and only added the prefix. Pep 498 does not seem to say anything about f-string w/o expressions; but test-suite appear to test that f-string without expression do not raise an error. I do not believe that making it an error is in anyway desirable; I believe a SyntaxWarning would align with the current warning on invalid escape sequence, and help ??mostly during refactoring, if an f-string loses some of its parameters, and the f"" if non-intentionally kept. ---------- components: Interpreter Core messages: 343672 nosy: mbussonn priority: normal severity: normal status: open title: Emit SyntaxWarning for f-strings without expressions ? versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 18:51:24 2019 From: report at bugs.python.org (STINNER Victor) Date: Mon, 27 May 2019 22:51:24 +0000 Subject: [New-bugs-announce] [issue37069] regrtest: log unraisable exceptions and uncaught thread exceptions Message-ID: <1558997484.87.0.618934402283.issue37069@roundup.psfhosted.org> New submission from STINNER Victor : Python 3.8 got 2 new hooks: sys.unraisablehook and threading.excepthook. It would be interesting to catch these exceptions and display them again in the test summary. I modified the io.IOBase destructor to log close() exception. There are still open issues: see bpo-18748. Once these issues will be fixed, maybe it would even be interesting to make a test fail if at least one uncaught thread exception is raised or an unraisable exception is logged? ---------- components: Tests messages: 343693 nosy: vstinner priority: normal severity: normal status: open title: regrtest: log unraisable exceptions and uncaught thread exceptions versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 20:33:18 2019 From: report at bugs.python.org (Eric V. Smith) Date: Tue, 28 May 2019 00:33:18 +0000 Subject: [New-bugs-announce] [issue37070] Clean up f-string debug handling, including potential memory leaks Message-ID: <1559003598.17.0.659743491517.issue37070@roundup.psfhosted.org> New submission from Eric V. Smith : This addresses a few issues Serhiy found in the review for issue 37050 at https://github.com/python/cpython/pull/13597. Patch to follow. ---------- assignee: eric.smith components: Interpreter Core messages: 343722 nosy: eric.smith, serhiy.storchaka priority: normal severity: normal status: open title: Clean up f-string debug handling, including potential memory leaks type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 21:23:00 2019 From: report at bugs.python.org (Hoang Duy Tran) Date: Tue, 28 May 2019 01:23:00 +0000 Subject: [New-bugs-announce] [issue37071] HTMLParser mistakenly inventing new tags while parsing Message-ID: <1559006580.21.0.290263720763.issue37071@roundup.psfhosted.org> New submission from Hoang Duy Tran : I have been working with some 'difficult' HTML files generated by Sphinx's RST. The following block of text is the RST original content: ---------------------------------------------------- Animation Playback Options ========================== ``-a`` ```` ```` Playback ````, only operates this way when not running in background. ``-p`` ```` ```` Open with lower left corner at ````, ````. ``-m`` Read from disk (Do not buffer). ``-f`` ```` ```` Specify FPS to start with. ``-j`` ```` Set frame step to ````. ``-s`` ```` Play from ````. ``-e`` ```` Play until ````. ---------------------------------------------------- This is the HTML block that is generated by Sphinx: ----------------------------------------------------
Animation Playback Options-a Playback , only operates this way when not running in background.-p Open with lower left corner at , .-mRead from disk (Do not buffer).-f Specify FPS to start with.-j Set frame step to .-s Play from .-e Play until .
---------------------------------------------------- I then use the BeautifulSoup, which uses the HTMLParser, to beautify and parse the HTML document and I've noticed that every incident of data that leads with a "<" and ends with ">", for example: .... has been misunderstood by the HTMLParser's library as a TAG, and then it INVENTS a CLOSED TAGS for it ie. and which when reversing, ie. turning from HTML back to normal text, muted out the original data, leading to TRUNCATION/LOST of DATA. Here is the content of the beautify generated data, issue lines are marked with '#**************************' to make it easier for you to identify. ----------------------------------------------------
Animation Playback Options -a #************************** #************************** #************************** #************************** Playback #************************** #************************** , only operates this way when not running in background. -p #************************** #************************** #************************** #************************** Open with lower left corner at #************************** #************************** , #************************** #************************** . -m Read from disk (Do not buffer). -f #************************** #************************** #************************** #************************** Specify FPS to start with. -j #************************** Set frame step to #************************** . -s #************************** Play from #************************** . -e #************************** Play until #************************** .
---------------------------------------------------- I enclosed the HTML file generated by Sphinx to allow you test this issue with the actual data. Here is the URL of the HTML file: https://docs.blender.org/manual/en/dev/advanced/command_line/arguments.html Kind Regards, Hoang Tran ---------- components: Library (Lib) files: arguments.html messages: 343724 nosy: htran priority: normal severity: normal status: open title: HTMLParser mistakenly inventing new tags while parsing type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file48367/arguments.html _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon May 27 21:52:54 2019 From: report at bugs.python.org (Graham Dumpleton) Date: Tue, 28 May 2019 01:52:54 +0000 Subject: [New-bugs-announce] [issue37072] PyNode_Compile() crashes in Python 3.8. Message-ID: <1559008374.77.0.70293248511.issue37072@roundup.psfhosted.org> New submission from Graham Dumpleton : The code: #include int main(int argc, char *argv[]) { FILE *fp = NULL; PyObject *co = NULL; struct _node *n = NULL; const char * filename = "/dev/null"; Py_Initialize(); fprintf(stderr, "START\n"); fp = fopen(filename, "r"); fprintf(stderr, "CALL PyParser_SimpleParseFile()\n"); n = PyParser_SimpleParseFile(fp, filename, Py_file_input); fprintf(stderr, "CALL PyNode_Compile()\n"); co = (PyObject *)PyNode_Compile(n, filename); fprintf(stderr, "DONE\n"); Py_Finalize(); return 0; } has worked fine since Python 2.3 (and maybe earlier) through Python 3.7, but now crashes in Python 3.8. It crashes in PyNode_Compile(). START CALL PyParser_SimpleParseFile() CALL PyNode_Compile() Segmentation fault: 11 Although it is part of the public interface of compile.h, the PyNode_Compile() seems never to actually be called anywhere in Python itself, and perhaps isn't even covered by tests. So if Python 3.8 internal changes mean this function implementation needs to be changed, that fact may have been missed. ---------- messages: 343727 nosy: grahamd priority: normal severity: normal status: open title: PyNode_Compile() crashes in Python 3.8. versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 03:18:04 2019 From: report at bugs.python.org (Windson Yang) Date: Tue, 28 May 2019 07:18:04 +0000 Subject: [New-bugs-announce] [issue37073] clarify functions docs in IO modules and Bytes Objects Message-ID: <1559027884.63.0.0963085588001.issue37073@roundup.psfhosted.org> New submission from Windson Yang : > PyBytes_FromStringAndSize(const char *v, Py_ssize_t len): > Return a new bytes object with a copy of the string v as value and length len on success, and NULL on failure. If v is NULL, the contents of the bytes object are uninitialized. The function actually copies `len` bytes from string v instead of the whole string. (link 1) I suggested we can change to > Return a new bytes object with a copy of the first len bytes of string v as value and length len on success... > readinto(b) > Read bytes into a pre-allocated, writable bytes-like object b and return the number of bytes read. For example, b might be a bytearray. The function will call _bufferediobase_readinto_generic(link 2) which may return NULL. Maybe we can change to > and return the number of bytes that read succeed (it may be smaller than b or NULL if failed)... 1. https://github.com/python/cpython/blob/master/Objects/bytesobject.c#L126 2. https://github.com/python/cpython/blob/master/Modules/_io/bufferedio.c#L962 ---------- assignee: docs at python components: Documentation messages: 343741 nosy: Windson Yang, docs at python priority: normal severity: normal status: open title: clarify functions docs in IO modules and Bytes Objects type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 03:37:41 2019 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 28 May 2019 07:37:41 +0000 Subject: [New-bugs-announce] [issue37074] os.stat() does not work for NUL and CON Message-ID: <1559029061.14.0.328684807212.issue37074@roundup.psfhosted.org> New submission from Serhiy Storchaka : >>> os.stat('nul') Traceback (most recent call last): File "", line 1, in OSError: [WinError 1] Incorrect function: 'nul' >>> os.stat('con') Traceback (most recent call last): File "", line 1, in OSError: [WinError 87] The parameter is incorrect: 'con' But os.open() and os.fstat() work. >>> os.fstat(os.open('nul', os.O_RDONLY)) os.stat_result(st_mode=8192, st_ino=0, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0, st_atime=0, st_mtime=0, st_ctime=0) >>> os.fstat(os.open('con', os.O_RDONLY)) os.stat_result(st_mode=8192, st_ino=0, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0, st_atime=0, st_mtime=0, st_ctime=0) ---------- components: Windows messages: 343745 nosy: paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: os.stat() does not work for NUL and CON type: behavior versions: Python 2.7, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 05:50:10 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Tue, 28 May 2019 09:50:10 +0000 Subject: [New-bugs-announce] [issue37075] Error message improvement for AsyncMock Message-ID: <1559037010.51.0.223468847551.issue37075@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : * assert_has_awaits has a comma in between the error message string used in AssertionError and hence error is raised as a tuple instead of a string. import asyncio from unittest.mock import AsyncMock, call async def main(): a = AsyncMock() await a(1) a.assert_has_awaits([call(2)]) asyncio.run(main()) Traceback (most recent call last): File "/tmp/foo.py", line 9, in asyncio.run(main()) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/runners.py", line 43, in run return loop.run_until_complete(main) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/base_events.py", line 608, in run_until_complete return future.result() File "/tmp/foo.py", line 7, in main a.assert_has_awaits([call(2)]) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 2164, in assert_has_awaits raise AssertionError( AssertionError: ('Awaits not found.\nExpected: [call(2)]\n', 'Actual: [call(1)]') It should be printed as a single string as below : AssertionError: Awaits not found. Expected: [call(2)] Actual: [call(1)] * assert_awaited_with uses _format_mock_failure_message which has "call" hardcoded but most of the error messages in AsyncMock use "await". assert_awaited_with error seems little confusing as it's same as assert_called_with. This is mostly a consistency nit as in below where call to a(2) is made but it's not awaited that should be noted in the message. The fix is simple which would be to make the string 'call' as a parameter that defaults to 'call' and needs to be passed as 'await' from assert_awaited_with error formatting. This would make "expected call not found" as "expected await not found" in below program. import asyncio from unittest.mock import AsyncMock, call async def main(): a = AsyncMock() await a(1) a(2) a.assert_awaited_with(2) asyncio.run(main()) ./python.exe /tmp/foo.py /tmp/foo.py:7: RuntimeWarning: coroutine 'AsyncMockMixin._mock_call' was never awaited a(2) RuntimeWarning: Enable tracemalloc to get the object allocation traceback Traceback (most recent call last): File "/tmp/foo.py", line 10, in asyncio.run(main()) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/runners.py", line 43, in run return loop.run_until_complete(main) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/base_events.py", line 608, in run_until_complete return future.result() File "/tmp/foo.py", line 8, in main a.assert_awaited_with(2) File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 2118, in assert_awaited_with raise AssertionError(_error_message()) from cause AssertionError: expected call not found. Expected: mock(2) Actual: mock(1) I will raise a PR for the above shortly. ---------- components: Library (Lib) messages: 343753 nosy: asvetlov, lisroach, xtreak priority: normal severity: normal status: open title: Error message improvement for AsyncMock type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 06:15:10 2019 From: report at bugs.python.org (STINNER Victor) Date: Tue, 28 May 2019 10:15:10 +0000 Subject: [New-bugs-announce] [issue37076] _thread.start_new_thread(): call sys.unraisablehook() to handle uncaught exceptions Message-ID: <1559038510.38.0.6664392813.issue37076@roundup.psfhosted.org> New submission from STINNER Victor : Python 3.8 now has 3 hooks for uncaught exceptions: * sys.excepthook() * sys.unraisablehook() * threading.excepthook() _thread.start_new_thread() calls none of these hooks, but directly logs the exception. I propose to modify it to reuse sys.unraisablehook(): see attached PR. -- Using threading.excepthook() would be another option, but _thread.start_new_thread() is usually wrapped in threading.Thread.run() which already uses threading.excepthook(). It might be surprising to see two bugs from the same thread using threading.excepthook(). Moreover, right now, _thread is the "low-level" API to access threads: it doesn't acces threading. I'm not comfortable by adding an inter-dependency between _thread (low-level) and threading (high-level). If threading.Thread.run() is correctly written, it should not raise an exception. In the worst case, threading.excepthook should handle the exception. _thread.start_new_thread() should never get an exception raised by threading if threading.excepthook does correctly its job. ---------- components: Library (Lib) messages: 343756 nosy: vstinner priority: normal severity: normal status: open title: _thread.start_new_thread(): call sys.unraisablehook() to handle uncaught exceptions versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 07:28:55 2019 From: report at bugs.python.org (Michael Felt) Date: Tue, 28 May 2019 11:28:55 +0000 Subject: [New-bugs-announce] [issue37077] Threading: add builtin TID attribute to Thread objects for AIX Message-ID: <1559042935.56.0.899309127578.issue37077@roundup.psfhosted.org> New submission from Michael Felt : As issue36084 is already closed - opening a new issue for the PR to add this support for AIX. ---------- messages: 343765 nosy: Michael.Felt priority: normal severity: normal status: open title: Threading: add builtin TID attribute to Thread objects for AIX versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 08:37:32 2019 From: report at bugs.python.org (suraj) Date: Tue, 28 May 2019 12:37:32 +0000 Subject: [New-bugs-announce] [issue37078] norton login Message-ID: <1559047052.73.0.927653434628.issue37078@roundup.psfhosted.org> New submission from suraj : Norton Login - Manage Your Norton Subscription, Setup Norton, Norton Sign in etc at Norton.com/setup and Manage Norton Services using Norton Account Here! norton login ---------- files: Untitled design.png messages: 343768 nosy: norton priority: normal severity: normal status: open title: norton login Added file: https://bugs.python.org/file48372/Untitled design.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 10:25:53 2019 From: report at bugs.python.org (Jizhou Yang) Date: Tue, 28 May 2019 14:25:53 +0000 Subject: [New-bugs-announce] [issue37079] PEM cadata causes ssl.SSLError: nested ans1 error Message-ID: <1559053553.63.0.405634897269.issue37079@roundup.psfhosted.org> Change by Jizhou Yang : ---------- assignee: christian.heimes components: SSL nosy: Jizhou Yang, christian.heimes priority: normal severity: normal status: open title: PEM cadata causes ssl.SSLError: nested ans1 error type: crash versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 11:37:32 2019 From: report at bugs.python.org (Jarl Gullberg) Date: Tue, 28 May 2019 15:37:32 +0000 Subject: [New-bugs-announce] [issue37080] Cannot compile Python3.7.3 on Alt-F (ARM) Message-ID: <1559057852.62.0.663214501823.issue37080@roundup.psfhosted.org> New submission from Jarl Gullberg : I am currently trying to compile the latest stable release of Python (3.7.3 at the time of writing) on a small ARM device (a DNS-323A1 NAS), running Alt-F (a custom linux-based firmware) using GCC 4.3.3. Unfortunately, I've hit a major snag that I just can't seem to get past. In short, the compilation goes well until the [sharedmods] stage, where it encounters an error related to readline and the module not having been compiled with -fPIC. I've included a truncated log (taken from a second make invocation after the first failed) that displays the issue. This is the ./configure invocation I've used: '--prefix=/usr' '--enable-shared' '--with-system-expat' '--with-system-ffi' '--with-ensurepip=yes' '--enable-optimizations' '--enable-ipv6' 'CFLAGS=-fPIC' 'CXXFLAGS=-fPIC' Any assistance would be much appreciated. ---------- components: Build files: make.log messages: 343796 nosy: Jarl Gullberg priority: normal severity: normal status: open title: Cannot compile Python3.7.3 on Alt-F (ARM) type: compile error versions: Python 3.7 Added file: https://bugs.python.org/file48374/make.log _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 11:44:52 2019 From: report at bugs.python.org (Christian Heimes) Date: Tue, 28 May 2019 15:44:52 +0000 Subject: [New-bugs-announce] [issue37081] Test with OpenSSL 1.1.1c Message-ID: <1559058292.64.0.780398663722.issue37081@roundup.psfhosted.org> New submission from Christian Heimes : OpenSSL 1.1.1c is out, https://github.com/openssl/openssl/blob/OpenSSL_1_1_1c/CHANGES ---------- assignee: christian.heimes components: SSL, Tests messages: 343797 nosy: christian.heimes priority: high severity: normal status: open title: Test with OpenSSL 1.1.1c versions: Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 14:26:23 2019 From: report at bugs.python.org (Matthias Bussonnier) Date: Tue, 28 May 2019 18:26:23 +0000 Subject: [New-bugs-announce] [issue37082] Assignment expression operator (walrus) not in built-in help() Message-ID: <1559067983.32.0.880606504508.issue37082@roundup.psfhosted.org> New submission from Matthias Bussonnier : Do the following at a Python prompt: ``` >>> help() Welcome to Python 3.8's help utility! [...]SNIP help> symbols Here is a list of the punctuation symbols which Python assigns special meaning to. Enter any symbol to get more help. != + <= __ " += <> ` """ , == b" % - > b' %= -= >= f" & . >> f' &= ... >>= j ' / @ r" ''' // J r' ( //= [ u" ) /= \ u' * : ] | ** < ^ |= **= << ^= ~ *= <<= _ ``` Oh no ! Our favorite `:=` is missing :-( ---------- assignee: docs at python components: Documentation messages: 343813 nosy: docs at python, mbussonn priority: normal severity: normal status: open title: Assignment expression operator (walrus) not in built-in help() versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 17:37:48 2019 From: report at bugs.python.org (Pablo Galindo Salgado) Date: Tue, 28 May 2019 21:37:48 +0000 Subject: [New-bugs-announce] [issue37083] Document TYPE_COMMENT in documentation reference for compound statements Message-ID: <1559079468.64.0.0790187408059.issue37083@roundup.psfhosted.org> New submission from Pablo Galindo Salgado : The last changes in the grammar regarding TYPE_COMMENTS should be added to this section: https://docs.python.org/3/reference/compound_stmts.html#function-definitions ---------- assignee: docs at python components: Documentation messages: 343820 nosy: docs at python, gvanrossum, levkivskyi, pablogsal priority: normal severity: normal stage: needs patch status: open title: Document TYPE_COMMENT in documentation reference for compound statements type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue May 28 17:53:56 2019 From: report at bugs.python.org (Brian Spratke) Date: Tue, 28 May 2019 21:53:56 +0000 Subject: [New-bugs-announce] [issue37084] _ctypes not failing, can't find reason Message-ID: <1559080436.17.0.198846412068.issue37084@roundup.psfhosted.org> New submission from Brian Spratke : I am trying to cross compile Python 3.7 for Android. I have Python building, but I keep getting an error that _ctypes failed to build, but I see nothing that jumps out as a reason. _ctypes_test builds, before that I see this INFO message INFO: Can't locate Tcl/Tk libs and/or headers grpmodule and crypt module have issues as well, but I do not feel that those are related. Are there any other ideas people can throw out? ---------- components: Cross-Build files: Python3.7Output.zip messages: 343822 nosy: Alex.Willmer, Brian Spratke priority: normal severity: normal status: open title: _ctypes not failing, can't find reason versions: Python 3.7 Added file: https://bugs.python.org/file48375/Python3.7Output.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 02:53:00 2019 From: report at bugs.python.org (Karl Ding) Date: Wed, 29 May 2019 06:53:00 +0000 Subject: [New-bugs-announce] [issue37085] Expose additional socket constants for CAN_BCM flags Message-ID: <1559112780.91.0.584487288185.issue37085@roundup.psfhosted.org> New submission from Karl Ding : When reading through the values exposed via the socket library, I noticed that currently, only the SocketCAN BCM opcode enums are exposed via the socket constants. These correspond to the following from the Linux headers: enum { TX_SETUP = 1, /* create (cyclic) transmission task */ TX_DELETE, /* remove (cyclic) transmission task */ TX_READ, /* read properties of (cyclic) transmission task */ TX_SEND, /* send one CAN frame */ RX_SETUP, /* create RX content filter subscription */ RX_DELETE, /* remove RX content filter subscription */ RX_READ, /* read properties of RX content filter subscription */ TX_STATUS, /* reply to TX_READ request */ TX_EXPIRED, /* notification on performed transmissions (count=0) */ RX_STATUS, /* reply to RX_READ request */ RX_TIMEOUT, /* cyclic message is absent */ RX_CHANGED /* updated CAN frame (detected content change) */ }; It would be nice to expose the BCM flags #defines as well. #define SETTIMER 0x0001 #define STARTTIMER 0x0002 #define TX_COUNTEVT 0x0004 #define TX_ANNOUNCE 0x0008 #define TX_CP_CAN_ID 0x0010 #define RX_FILTER_ID 0x0020 #define RX_CHECK_DLC 0x0040 #define RX_NO_AUTOTIMER 0x0080 #define RX_ANNOUNCE_RESUME 0x0100 #define TX_RESET_MULTI_IDX 0x0200 #define RX_RTR_FRAME 0x0400 #define CAN_FD_FRAME 0x0800 These BCM flags are used as part of the BCM header that has the aforementioned opcodes. The flags are set on the bcm_msg_head struct: struct bcm_msg_head { __u32 opcode; __u32 flags; __u32 count; struct bcm_timeval ival1, ival2; canid_t can_id; __u32 nframes; struct can_frame frames[0]; }; The existing documentation for the BCM constants (https://docs.python.org/3/library/socket.html#socket.CAN_BCM) seems to imply that these constants should already be included, but they are not. See the Linux headers for more details: https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/can/bcm.h ---------- components: Library (Lib) messages: 343865 nosy: karlding priority: normal severity: normal status: open title: Expose additional socket constants for CAN_BCM flags type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 06:37:02 2019 From: report at bugs.python.org (Justin Fay) Date: Wed, 29 May 2019 10:37:02 +0000 Subject: [New-bugs-announce] [issue37086] time.sleep error message misleading Message-ID: <1559126222.88.0.100290912588.issue37086@roundup.psfhosted.org> New submission from Justin Fay : Using python3.6 and calling `time.sleep` with an invalid argument the `TypeError` raised has the error message "TypeError: an integer is required". This is not the case as a float or integer is acceptable. Using python 2.7 the error message given is better "TypeError: a float is required". ---------- messages: 343880 nosy: Justin Fay priority: normal severity: normal status: open title: time.sleep error message misleading versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 12:32:45 2019 From: report at bugs.python.org (David Carlier) Date: Wed, 29 May 2019 16:32:45 +0000 Subject: [New-bugs-announce] [issue37087] Adding native id support for openbsd Message-ID: <1559147565.3.0.574292103957.issue37087@roundup.psfhosted.org> New submission from David Carlier : Following up on bpo-36084 ---------- messages: 343896 nosy: David Carlier priority: normal pull_requests: 13549 severity: normal status: open title: Adding native id support for openbsd versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 12:46:29 2019 From: report at bugs.python.org (Yury Selivanov) Date: Wed, 29 May 2019 16:46:29 +0000 Subject: [New-bugs-announce] [issue37088] Add a way to schedule a function to be called from the main thread Message-ID: <1559148389.14.0.649589094885.issue37088@roundup.psfhosted.org> New submission from Yury Selivanov : When asyncio event loop is created in a non-main thread, it needs to initialize a so called ChildWatcher for it (a helper object to intercept subprocesses exits). Doing that requires to run code in the main thread. I propose to add a 'sys.addpendingcall' function that will simply expose the already existing Py_AddPendingCall to the pure Python land. ---------- components: Interpreter Core, asyncio messages: 343897 nosy: asvetlov, vstinner, yselivanov priority: normal severity: normal status: open title: Add a way to schedule a function to be called from the main thread versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 16:52:39 2019 From: report at bugs.python.org (Anthony Sottile) Date: Wed, 29 May 2019 20:52:39 +0000 Subject: [New-bugs-announce] [issue37089] `import Lib.os` works on windows (but shouldn't) Message-ID: <1559163159.15.0.0136841146516.issue37089@roundup.psfhosted.org> New submission from Anthony Sottile : Additionally, virtualenvs have the root of their directory on `sys.path` -- this is unlike posix behaviour For example: $ python Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD6 4)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import Lib.os >>> Lib.os.__file__ 'C:\\Python37\\Lib\\os.py' >>> Lib.os.listdir('C:\\') ['$Recycle.Bin', 'BGinfo', 'Documents and Settings', 'pagefile.sys', 'PerfLogs', 'Program Files', 'Program Files (x86)', 'ProgramData', 'Python27', 'Python37', 'Recovery', 'swapfile.sys', 'System Volume Information', 'Users', 'Windows'] ---------- components: Library (Lib), Windows messages: 343923 nosy: Anthony Sottile, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: `import Lib.os` works on windows (but shouldn't) versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 17:38:35 2019 From: report at bugs.python.org (Petr Viktorin) Date: Wed, 29 May 2019 21:38:35 +0000 Subject: [New-bugs-announce] [issue37090] test_gdb's test_pycfunction should test all calling conventions Message-ID: <1559165915.61.0.362997663954.issue37090@roundup.psfhosted.org> New submission from Petr Viktorin : test_gdb.StackNavigationTests.test_pycfunction checks that the GDB integration can pick up calls to C-API functions. Currently it includes the comment: > Tested function must not be defined with METH_NOARGS or METH_O, > otherwise call_function() doesn't call PyCFunction_Call() This is (now?) false; furthermore it looks like all builtin_function_or_method are discoverable. As the code paths for various METH_* conventions are diverging due to optimizations, we should check they continue to be covered. ---------- assignee: petr.viktorin messages: 343927 nosy: petr.viktorin priority: normal severity: normal status: open title: test_gdb's test_pycfunction should test all calling conventions versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 18:52:02 2019 From: report at bugs.python.org (hw) Date: Wed, 29 May 2019 22:52:02 +0000 Subject: [New-bugs-announce] [issue37091] subprocess - uncaught PermissionError in send_signal can cause hang Message-ID: <1559170322.53.0.309237543574.issue37091@roundup.psfhosted.org> New submission from hw <13hurdw at gmail.com>: Python 3.7.3 Ubuntu 18.10 Cosmic https://github.com/python/cpython/pull/13669 Encountered a condition where uncaught PermissionError caused a hang running a subprocess command with sudo -u Traceback (most recent call last): File "/usr/lib/python3.7/subprocess.py", line 474, in run stdout, stderr = process.communicate(input, timeout=timeout) File "/usr/lib/python3.7/subprocess.py", line 939, in communicate stdout, stderr = self._communicate(input, endtime, timeout) File "/usr/lib/python3.7/subprocess.py", line 1707, in _communicate self.wait(timeout=self._remaining_time(endtime)) File "/usr/lib/python3.7/subprocess.py", line 990, in wait return self._wait(timeout=timeout) File "/usr/lib/python3.7/subprocess.py", line 1616, in _wait raise TimeoutExpired(self.args, timeout) subprocess.TimeoutExpired: Command '['sudo', '-u', 'chrome', '/snap/bin/chromium', '--headless', '--disable-gpu', '--hide-scrollbars', '--ignore-certificate-errors', '--enable-sandbox', '--incognito', '--mute-audio', '--disable-databases', '--enable-strict-powerful-feature-restrictions', '--no-pings', '--no-referrers', '--timeout=30000', '--window-size=1280,1000', '--screenshot=[REDACTED]_screenshot.png', 'https://[REDACTED]']' timed out after 59.9998986274004 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.7/subprocess.py", line 476, in run process.kill() File "/usr/lib/python3.7/subprocess.py", line 1756, in kill self.send_signal(signal.SIGKILL) File "/usr/lib/python3.7/subprocess.py", line 1746, in send_signal os.kill(self.pid, sig) PermissionError: [Errno 1] Operation not permitted ---------- components: Library (Lib) messages: 343939 nosy: Hw priority: normal pull_requests: 13558 severity: normal status: open title: subprocess - uncaught PermissionError in send_signal can cause hang type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 19:03:42 2019 From: report at bugs.python.org (Dave Johansen) Date: Wed, 29 May 2019 23:03:42 +0000 Subject: [New-bugs-announce] [issue37092] LoggerAdapter doesn't have fatal() like Logger Message-ID: <1559171022.38.0.633277520355.issue37092@roundup.psfhosted.org> New submission from Dave Johansen : Using LoggerAdapter is a convenient way to add extra info to all logs, but it doesn't have the fatal() method like Logger, so it isn't a drop in replacement like it should be. ---------- components: Library (Lib) messages: 343941 nosy: Dave Johansen priority: normal severity: normal status: open title: LoggerAdapter doesn't have fatal() like Logger type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 19:32:09 2019 From: report at bugs.python.org (Tim Burke) Date: Wed, 29 May 2019 23:32:09 +0000 Subject: [New-bugs-announce] [issue37093] http.client aborts header parsing upon encountering non-ASCII header names Message-ID: <1559172729.29.0.908018369806.issue37093@roundup.psfhosted.org> New submission from Tim Burke : First, spin up a fairly trivial http server: import wsgiref.simple_server def app(environ, start_response): start_response('200 OK', [ ('Some-Canonical', 'headers'), ('sOme-CRAzY', 'hEaDERs'), ('Utf-8-Values', '\xe2\x9c\x94'), ('s\xc3\xb6me-UT\xc6\x92-8', 'in the header name'), ('some-other', 'random headers'), ]) return [b'Hello, world!\n'] if __name__ == '__main__': httpd = wsgiref.simple_server.make_server('', 8000, app) while True: httpd.handle_request() Note that this code works equally well on py2 or py3; the interesting bytes on the wire are the same on either. Verify the expected response using an independent tool such as curl: $ curl -v http://localhost:8000 * Trying ::1... * TCP_NODELAY set * connect to ::1 port 8000 failed: Connection refused * Trying 127.0.0.1... * TCP_NODELAY set * Connected to localhost (127.0.0.1) port 8000 (#0) > GET / HTTP/1.1 > Host: localhost:8000 > User-Agent: curl/7.64.0 > Accept: */* > * HTTP 1.0, assume close after body < HTTP/1.0 200 OK < Date: Wed, 29 May 2019 23:02:37 GMT < Server: WSGIServer/0.2 CPython/3.7.3 < Some-Canonical: headers < sOme-CRAzY: hEaDERs < Utf-8-Values: ? < s?me-UT?-8: in the header name < some-other: random headers < Content-Length: 14 < Hello, world! * Closing connection 0 Check that py2 includes all the same headers: $ python2 -c 'import pprint, urllib; resp = urllib.urlopen("http://localhost:8000"); pprint.pprint((dict(resp.info().items()), resp.read()))' ({'content-length': '14', 'date': 'Wed, 29 May 2019 23:03:02 GMT', 'server': 'WSGIServer/0.2 CPython/3.7.3', 'some-canonical': 'headers', 'some-crazy': 'hEaDERs', 'some-other': 'random headers', 's\xc3\xb6me-ut\xc6\x92-8': 'in the header name', 'utf-8-values': '\xe2\x9c\x94'}, 'Hello, world!\n') But py3 *does not*: $ python3 -c 'import pprint, urllib.request; resp = urllib.request.urlopen("http://localhost:8000"); pprint.pprint((dict(resp.info().items()), resp.read()))' ({'Date': 'Wed, 29 May 2019 23:04:09 GMT', 'Server': 'WSGIServer/0.2 CPython/3.7.3', 'Some-Canonical': 'headers', 'Utf-8-Values': '?\x9c\x94', 'sOme-CRAzY': 'hEaDERs'}, b'Hello, world!\n') Instead, it is missing the first header that has a non-ASCII name as well as all subsequent headers (even if they are all-ASCII). Interestingly, the response body is intact. This is eventually traced back to email.feedparser's expectation that all headers conform to rfc822 and its assumption that anything that *doesn't* conform must be part of the body: https://github.com/python/cpython/blob/v3.7.3/Lib/email/feedparser.py#L228-L236 However, http.client has *already* determined the boundary between headers and body in parse_headers, and sent everything that it thinks is headers to the parser: https://github.com/python/cpython/blob/v3.7.3/Lib/http/client.py#L193-L214 ---------- messages: 343942 nosy: tburke priority: normal severity: normal status: open title: http.client aborts header parsing upon encountering non-ASCII header names versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 21:50:03 2019 From: report at bugs.python.org (=?utf-8?b?6bqm5qCL6ZOW?=) Date: Thu, 30 May 2019 01:50:03 +0000 Subject: [New-bugs-announce] [issue37094] Provide an example for TestCase.skipTest in unittest doc Message-ID: <1559181003.42.0.162580428843.issue37094@roundup.psfhosted.org> New submission from ??? : I found that there's an function TestCase.skipTest(reason)[0] supports skipping test in testing, but there's no example for this function in chapter Skipping tests[1] and expected failures I create PR[2] for updating the example for TestCase.skipTest in unittest doc. The example is from Lib/unittest/test, the test for unittest. I am sorry for that i don't know if the issue tracker support markdown link syntax, so i just copy and paste the link below: [0] https://docs.python.org/3.7/library/unittest.html#unittest.TestCase.skipTest [1]https://docs.python.org/3.7/library/unittest.html#skipping-tests-and-expected-failures [2]https://github.com/python/cpython/pull/13645 ---------- assignee: docs at python components: Documentation messages: 343943 nosy: docs at python, ??? priority: normal pull_requests: 13561 severity: normal status: open title: Provide an example for TestCase.skipTest in unittest doc type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed May 29 23:42:48 2019 From: report at bugs.python.org (Evan Greenup) Date: Thu, 30 May 2019 03:42:48 +0000 Subject: [New-bugs-announce] [issue37095] [Feature Request]: Add zstd support in tarfile Message-ID: <1559187768.11.0.7498103877.issue37095@roundup.psfhosted.org> New submission from Evan Greenup : Zstandard is getting more and more popular. It could be awesome if tarfile support this compression format for .tar.zst file. ---------- components: Library (Lib) messages: 343945 nosy: evan0greenup priority: normal severity: normal status: open title: [Feature Request]: Add zstd support in tarfile type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 03:18:19 2019 From: report at bugs.python.org (Giampaolo Rodola') Date: Thu, 30 May 2019 07:18:19 +0000 Subject: [New-bugs-announce] [issue37096] Add large-file tests for modules using sendfile(2) Message-ID: <1559200699.13.0.758657145229.issue37096@roundup.psfhosted.org> New submission from Giampaolo Rodola' : The need for this emerged in: https://bugs.python.org/issue36610#msg343948. We currently use sendfile(2) syscall in high-level shutil.copyfile() on Linux and socket.sendfile() on UNIX. In addition this PR also tests shutil.copyfile() implementation on OSX which takes advantage of fcopyfile(2) syscall. The goal is to make sure "fast-copy" syscalls can handle files > 2G. On my Ubuntu 18.04 with SSD disk this introduces a 5 secs slowdown to the test run. According to: https://github.com/golang/go/issues/13892 ...problems may arise on SunOS, so if a BB will turn red this is expected. ---------- components: Library (Lib) messages: 343951 nosy: giampaolo.rodola, rosslagerwall priority: normal severity: normal status: open title: Add large-file tests for modules using sendfile(2) versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 03:43:48 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Thu, 30 May 2019 07:43:48 +0000 Subject: [New-bugs-announce] [issue37097] python_is_optimized() false negatives Message-ID: <1559202228.98.0.967762980868.issue37097@roundup.psfhosted.org> New submission from Jeroen Demeyer : The function python_is_optimized() in Lib/test/support.py: def python_is_optimized(): """Find if Python was built with optimizations.""" cflags = sysconfig.get_config_var('PY_CFLAGS') or '' final_opt = "" for opt in cflags.split(): if opt.startswith('-O'): final_opt = opt return final_opt not in ('', '-O0', '-Og') However, it seems that the *default* (when no special CFLAGS are configured) is -O3. My Python build is done with this default -O3 but python_is_optimized() returns False. This is the reason why I didn't catch the buildbot failure at https://github.com/python/cpython/pull/13185#issuecomment-497062965 (this test is only run when python_is_optimized() is True) ---------- components: Tests messages: 343952 nosy: benjamin.peterson, jdemeyer, petr.viktorin, pitrou priority: normal severity: normal status: open title: python_is_optimized() false negatives type: behavior versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 03:48:02 2019 From: report at bugs.python.org (Jeroen Demeyer) Date: Thu, 30 May 2019 07:48:02 +0000 Subject: [New-bugs-announce] [issue37098] test_memfd_create() test failure Message-ID: <1559202482.0.0.225844714186.issue37098@roundup.psfhosted.org> New submission from Jeroen Demeyer : On an older Linux system: ====================================================================== ERROR: test_memfd_create (test.test_os.MemfdCreateTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/src/sage-config/local/src/cpython/Lib/test/test_os.py", line 3128, in test_memfd_create fd = os.memfd_create("Hi", os.MFD_CLOEXEC) OSError: [Errno 38] Function not implemented ---------- components: Tests messages: 343953 nosy: ZackerySpytz, christian.heimes, jdemeyer priority: normal severity: normal status: open title: test_memfd_create() test failure type: behavior versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 07:24:20 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Thu, 30 May 2019 11:24:20 +0000 Subject: [New-bugs-announce] [issue37099] test_inspect generates DeprecationWarning Message-ID: <1559215460.17.0.300068245937.issue37099@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : In PR 13245 getfullargspec was undeprecated. But functions like getargspec and formatargspec still have deprecation warnings in code. As part of the PR code to ignore these warnings in test were also removed though the actual warning remains in Lib/inspect.py . This generates below warnings. ./python.exe -Wall -m test test_inspect Run tests sequentially 0:00:00 load avg: 3.05 [1/1] test_inspect /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:759: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:753: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() args, varargs, varkw, defaults = inspect.getargspec(routine) /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, /Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_inspect.py:777: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults, == Tests result: SUCCESS == 1 test OK. Total duration: 1 sec 662 ms Tests result: SUCCESS ---------- components: Tests messages: 343971 nosy: pablogsal, xtreak priority: normal severity: normal status: open title: test_inspect generates DeprecationWarning type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 08:57:05 2019 From: report at bugs.python.org (Karthikeyan Singaravelan) Date: Thu, 30 May 2019 12:57:05 +0000 Subject: [New-bugs-announce] [issue37100] test_coroutine.test_unawaited_warning_when_module_broken fails on -Werror Message-ID: <1559221025.29.0.0441448239991.issue37100@roundup.psfhosted.org> New submission from Karthikeyan Singaravelan : Running test_coroutines under -Werror raises error. It seems ZeroDivisionError is expected and RuntimeWarning is raised and happens only under -Werror. This test was added with e4d300e07c3 . ./python.exe -Werror -m test test_coroutines Run tests sequentially 0:00:00 load avg: 2.02 [1/1] test_coroutines test test_coroutines failed -- Traceback (most recent call last): File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_coroutines.py", line 2263, in test_unawaited_warning_when_module_broken self.assertEqual(cm.unraisable.exc_type, ZeroDivisionError) AssertionError: != test_coroutines failed == Tests result: FAILURE == 1 test failed: test_coroutines Total duration: 436 ms Tests result: FAILURE ---------- components: Tests messages: 343974 nosy: vstinner, xtreak priority: normal severity: normal status: open title: test_coroutine.test_unawaited_warning_when_module_broken fails on -Werror versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 11:49:09 2019 From: report at bugs.python.org (Dale Visser) Date: Thu, 30 May 2019 15:49:09 +0000 Subject: [New-bugs-announce] [issue37101] Filterer.filter can be rewritten using built-ins just as efficient much more readable Message-ID: <1559231349.65.0.379811854452.issue37101@roundup.psfhosted.org> New submission from Dale Visser : Alternative version of Filterer.filter(...) would look like this, which takes advantage of efficient Python 3.x built-ins, and is immediately understandable: def _filter_callable(filter): return filter.filter if hasattr(filter, 'filter') else filter def filter(self, record): filters = map(_filter_callable, self.filters) return all(f(record) for f in filters) I will add a tested pull request on GitHub. ---------- components: Library (Lib) messages: 343983 nosy: dwvisser priority: normal severity: normal status: open title: Filterer.filter can be rewritten using built-ins just as efficient much more readable type: enhancement versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 14:00:00 2019 From: report at bugs.python.org (Gregory P. Smith) Date: Thu, 30 May 2019 18:00:00 +0000 Subject: [New-bugs-announce] [issue37102] Automatically dedent docstring constants by default Message-ID: <1559239200.57.0.0816589870282.issue37102@roundup.psfhosted.org> New submission from Gregory P. Smith : I'm spawning this issue of as a separate feature from https://bugs.python.org/issue36906 (adding string dedent method and an optimization to do it at compile timer on constants). It'd be great if docstrings were given a similar treatment. Right now we carry the whitespace burden within the str constants for __doc__ stored in all code objects in the process. This adds up. This is not _quite_ the same as calling textwrap.dedent() or the upcoming str.dedent method on them at compile time. We need to special case the first line of docstrings. inspect.getdoc() is our library function that does this for us today. Change of breaking things with this change? not impossible, but extremely minor. Something using docstrings as data _and_ depending on leading indentation whitespace preservation would not like this. I am not aware of anything that would ever do that. ---------- components: Interpreter Core messages: 343990 nosy: gregory.p.smith priority: normal severity: normal status: open title: Automatically dedent docstring constants by default type: enhancement versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 15:50:25 2019 From: report at bugs.python.org (Anders Kaseorg) Date: Thu, 30 May 2019 19:50:25 +0000 Subject: [New-bugs-announce] [issue37103] Undo deprecation of optparse Message-ID: <1559245825.26.0.384245211115.issue37103@roundup.psfhosted.org> New submission from Anders Kaseorg : The optparse library is currently marked in the documentation as deprecated in favor of argparse. However, argparse uses a nonstandard reinterpretation of Unix command line grammars that makes certain arguments impossible to express, and causes scripts to break when ported from optparse to argparse. See the bug report I filed nine years ago: https://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) The conclusion of the core developers (e.g. msg309691) seems to have been that although it?s a valid bug, it can?t or won?t be fixed with the current argparse architecture. I was asked by another core developer to file a bug report for the de-deprecation of optparse (https://discuss.python.org/t/pep-594-removing-dead-batteries-from-the-standard-library/1704/20), so here it is. ---------- assignee: docs at python components: Documentation, Library (Lib) messages: 343997 nosy: andersk, docs at python priority: normal severity: normal status: open title: Undo deprecation of optparse versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 18:03:29 2019 From: report at bugs.python.org (Mario Corchero) Date: Thu, 30 May 2019 22:03:29 +0000 Subject: [New-bugs-announce] [issue37104] logging.Logger.disabled is not documented Message-ID: <1559253809.15.0.569168617436.issue37104@roundup.psfhosted.org> New submission from Mario Corchero : Just realised the "disabled" attribute of the Logger class is not documented in https://docs.python.org/3/library/logging.html#logging.Logger. Any reason to not have it documented? I'll send a PR otherwise. This comes as I was trying to point to the docs of the attribute in https://bugs.python.org/issue29143 ---------- assignee: docs at python components: Documentation messages: 344007 nosy: docs at python, mariocj89 priority: low severity: normal status: open title: logging.Logger.disabled is not documented type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu May 30 21:40:39 2019 From: report at bugs.python.org (Emmanuel Arias) Date: Fri, 31 May 2019 01:40:39 +0000 Subject: [New-bugs-announce] [issue37105] Add deprecated-remove information on stream doc Message-ID: <1559266839.05.0.214061892347.issue37105@roundup.psfhosted.org> New submission from Emmanuel Arias : According to the code on streams.py the functions: open_connection(), start_server(), open_unix_connection(), start_unix_server() are deprecated. I inform that on documentation. ---------- assignee: docs at python components: Documentation messages: 344015 nosy: docs at python, eamanu priority: normal severity: normal status: open title: Add deprecated-remove information on stream doc type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 00:37:21 2019 From: report at bugs.python.org (MANI M) Date: Fri, 31 May 2019 04:37:21 +0000 Subject: [New-bugs-announce] [issue37106] python re.escape doesn't escape some special characters. Message-ID: <1559277441.34.0.827558992585.issue37106@roundup.psfhosted.org> New submission from MANI M : Recently I figured out an issue in python3 re which doesn't escape some special characters. Not sure whether this bug has been reported already. Have attached screenshots for your reference. Steps to reproduce: 1. wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz 2. tar -xvzf Python-3.7.3.tar.xz 3. cd Python-3.7.3 4. ./configure 5. make 6. make install. GCC version: gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36) OS: CentOS Linux release 7.6.1810 (Core) ---------- components: Regular Expressions files: python_3.7.3_bug.png messages: 344020 nosy: MANI M, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: python re.escape doesn't escape some special characters. type: behavior versions: Python 3.7 Added file: https://bugs.python.org/file48377/python_3.7.3_bug.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 02:14:55 2019 From: report at bugs.python.org (Chris Withers) Date: Fri, 31 May 2019 06:14:55 +0000 Subject: [New-bugs-announce] [issue37107] ensurepip --upgrade doesn't change the version of pip used by venv Message-ID: <1559283295.29.0.266679199585.issue37107@roundup.psfhosted.org> New submission from Chris Withers : $ python3.7 -m ensurepip --upgrade Looking in links: /var/folders/m6/tsd59qsj7pd_lldh4mhwh6kh0000gn/T/tmpqk_vncev Requirement already up-to-date: setuptools in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (39.0.1) Requirement already up-to-date: pip in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (18.1) But: $ python3.7 -m venv /tmp/test_venv $ /tmp/test_venv/bin/pip --version pip 10.0.1 from /private/tmp/test_venv/lib/python3.7/site-packages/pip (python 3.7) ---------- components: Library (Lib) messages: 344029 nosy: cjw296 priority: normal severity: normal status: open title: ensurepip --upgrade doesn't change the version of pip used by venv versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 04:27:50 2019 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 31 May 2019 08:27:50 +0000 Subject: [New-bugs-announce] [issue37108] Positional-only arguments break super() Message-ID: <1559291270.55.0.00118529711916.issue37108@roundup.psfhosted.org> New submission from Serhiy Storchaka : See the example pep570super.py. super() without arguments does not work for positional-only arguments. Traceback (most recent call last): File "pep570super.py", line 13, in print(C()) File "pep570super.py", line 10, in __new__ return super().__new__(cls) RuntimeError: super(): no arguments ---------- components: Interpreter Core files: pep570super.py messages: 344034 nosy: ncoghlan, pablogsal, serhiy.storchaka priority: normal severity: normal status: open title: Positional-only arguments break super() type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file48379/pep570super.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 04:52:17 2019 From: report at bugs.python.org (Antony Lee) Date: Fri, 31 May 2019 08:52:17 +0000 Subject: [New-bugs-announce] [issue37109] Inacurrate documentation regarding return type of math.factorial Message-ID: <1559292737.06.0.619691147167.issue37109@roundup.psfhosted.org> New submission from Antony Lee : https://docs.python.org/3/library/math.html says The following functions are provided by this module. Except when explicitly noted otherwise, all return values are floats. and math.factorial(x) Return x factorial. Raises ValueError if x is not integral or is negative. but math.factorial returns an int (which is perfectly reasonable and just needs to be documented): In [3]: math.factorial(5) Out[3]: 120 ---------- assignee: docs at python components: Documentation messages: 344039 nosy: Antony.Lee, docs at python priority: normal severity: normal status: open title: Inacurrate documentation regarding return type of math.factorial versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 05:07:25 2019 From: report at bugs.python.org (Christoph Zwerschke) Date: Fri, 31 May 2019 09:07:25 +0000 Subject: [New-bugs-announce] [issue37110] Clarify hashability of custom class instances Message-ID: <1559293645.61.0.954260937021.issue37110@roundup.psfhosted.org> New submission from Christoph Zwerschke : The Python documentation says about hashability in the glossary (https://docs.python.org/3/glossary.html#term-hashable): "Objects which are instances of user-defined classes are hashable by default." This is not quite true. Objects of a user-defined class with an __eq__ method are not hashable. Maybe it would be better to make this more explicit: "Objects which are instances of user_defined classes without custom __eq__ and __hash__ methods are hashable by default." ---------- assignee: docs at python components: Documentation messages: 344042 nosy: cito, docs at python priority: normal severity: normal status: open title: Clarify hashability of custom class instances type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 05:57:34 2019 From: report at bugs.python.org (Jonathan) Date: Fri, 31 May 2019 09:57:34 +0000 Subject: [New-bugs-announce] [issue37111] Logging - Inconsistent behaviour when handling unicode Message-ID: <1559296654.15.0.378802674413.issue37111@roundup.psfhosted.org> New submission from Jonathan : Python is inconsistent in how it handles errors that have some unicode characters. It works to screen but fails to log This works: ``` >>> import logging >>> logging.error('???1') ERROR:root:???1 ``` The following breaks: ``` >>> import logging >>> logging.basicConfig(filename='c:\\my_log.log') >>> logging.error('???1') ``` This raises a unicode error: UnicodeEncodeError: 'charmap' codec can't encode characters in position 11-13: character maps to Python 3.6.3 Given that the file created by the logger is utf-8, it's unclear why it doesn't work. I found a workaround by using a Handler, but surely the loggers should all work the same way so that people don't get unpleasant surprises that even more painful to debug when things only break in certain logging modes? ---------- messages: 344053 nosy: jonathan-lp priority: normal severity: normal status: open title: Logging - Inconsistent behaviour when handling unicode versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 07:31:31 2019 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 31 May 2019 11:31:31 +0000 Subject: [New-bugs-announce] [issue37112] Error in compilig the AST for functions with optional positional arguments Message-ID: <1559302291.91.0.202997088134.issue37112@roundup.psfhosted.org> New submission from Serhiy Storchaka : >>> import ast >>> sample = 'def f(a=1, /): pass' >>> compile(sample, '?', 'exec') at 0x7f7cacdfd5c0, file "?", line 1> >>> compile(ast.parse(sample, '?'), '?', 'exec') Traceback (most recent call last): File "", line 1, in ValueError: more positional defaults than args on arguments ---------- components: Interpreter Core messages: 344064 nosy: benjamin.peterson, brett.cannon, pablogsal, serhiy.storchaka, yselivanov priority: normal severity: normal status: open title: Error in compilig the AST for functions with optional positional arguments type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 07:32:40 2019 From: report at bugs.python.org (Richard Neumann) Date: Fri, 31 May 2019 11:32:40 +0000 Subject: [New-bugs-announce] =?utf-8?b?W2lzc3VlMzcxMTNdICfDnycudXBwZXIo?= =?utf-8?b?KSBzaG91bGQgcmV0dXJuICfhup4n?= Message-ID: <1559302360.96.0.658137467016.issue37113@roundup.psfhosted.org> New submission from Richard Neumann : Currently, calling the method .upper() on a string containing '?' will replace this character by 'SS'. It should, however, be replaced by '?'. ---------- components: Unicode messages: 344065 nosy: Richard Neumann, ezio.melotti, vstinner priority: normal severity: normal status: open title: '?'.upper() should return '?' type: behavior versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 08:40:18 2019 From: report at bugs.python.org (Akilesh K) Date: Fri, 31 May 2019 12:40:18 +0000 Subject: [New-bugs-announce] [issue37114] lstrip remove extra characters in the presence of a matching number Message-ID: <1559306418.29.0.0735578073723.issue37114@roundup.psfhosted.org> New submission from Akilesh K : When the argument to lstrip / strip has a number and it matches the string it begins to act different. >>> text = "apiv1appliance" >>> text.strip("apiv1") 'liance' >>> text.strip("apiv2") '1appliance' >>> text.strip("a") 'piv1appliance' >>> text.strip("ap") 'iv1appliance' >>> text.strip("api") 'v1appliance' >>> text.strip("apiv") '1appliance' >>> text.strip("apiv1") 'liance' >>> text.strip("apiv2") '1appliance' ---------- messages: 344070 nosy: Akilesh K priority: normal severity: normal status: open title: lstrip remove extra characters in the presence of a matching number type: behavior versions: Python 2.7, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 09:23:41 2019 From: report at bugs.python.org (Pablo Galindo Salgado) Date: Fri, 31 May 2019 13:23:41 +0000 Subject: [New-bugs-announce] [issue37115] Support annotations in positional-only arguments Message-ID: <1559309021.21.0.994521369613.issue37115@roundup.psfhosted.org> Change by Pablo Galindo Salgado : ---------- components: Interpreter Core nosy: pablogsal priority: normal severity: normal status: open title: Support annotations in positional-only arguments versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 10:47:41 2019 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 31 May 2019 14:47:41 +0000 Subject: [New-bugs-announce] [issue37116] Use PEP 570 syntax for positional-only parameters Message-ID: <1559314061.29.0.391357031696.issue37116@roundup.psfhosted.org> New submission from Serhiy Storchaka : Two PRs apply PEP 570 syntax to the existing code. PR 13700 contains only compatible changes and can be applied to 3.8. It removes existing *args or naming tricks and makes self and cls arguments positional-only (if needed). PR 12620 contains breaking changes and can be applied only to 3.9. It converts deprecation warnings introduced in 3.8 into errors. ---------- components: Library (Lib) messages: 344079 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Use PEP 570 syntax for positional-only parameters type: enhancement versions: Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 14:44:02 2019 From: report at bugs.python.org (Mario Corchero) Date: Fri, 31 May 2019 18:44:02 +0000 Subject: [New-bugs-announce] [issue37117] Simplify customization of the logging time through datefmt Message-ID: <1559328242.82.0.712821467925.issue37117@roundup.psfhosted.org> New submission from Mario Corchero : Users that want to provide a custom template for the timestamp part of logging cannot customize the milliseconds part of asctime. They can use the msecs attribute of the Logger but that effectively requires to break the formatting in two parts. Note that it is not necessary to provide msecs on the default template as there is code that handles it in https://github.com/python/cpython/blob/c8d5bf6c3fa09b43f6a5ee779d493d251dbcc53c/Lib/logging/__init__.py#L603 Something we can do to improve this situation is changing the default converter to produce a datetime rather than a timetuple, as strftime of datetime and time uses the same template format. This will allow changing the format including milliseconds through the datefmt argument. formatter = logging.Formatter("%(asctime)s %(message)s", datefmt="%Y%m%d %H:%M:%S.%f") Compare this to the current soltution: formatter = logging.Formatter("%(asctime)s%(msecs)d %(message)s", datefmt="%Y%m%d %H:%M:%S") Note how you need to split the formatting of time into two different parts. This becomes even more diserse if you also want to add something after the time formatting (Example: a Z to note UTC). One more reason why this is quite powerful is that once we get timezones added to the standard library we will be able to (in a backward compatible way) just pass the timezone when we do `datetime.fromtimestamp`, which will add the timezone information to the datetime and users will be able to use %z to add the offset of the logging timestamp (Hurray! timestamps with offsets). Sample implementation: https://github.com/mariocj89/cpython/commit/5047d730c0a0dcd6276f40c5b66762071dfcb448 If it looks interesting I can update the docs, add some further tests and send the PR. Wanted to confirm it is before putting time into that. I cannot come with any "downside" of not doing it and I think it will simplify considerably the way users can format timestamps in logs. With this change we will just be able to say "To change the formatting of timestamps in a formatter, just use the datefmt argument". Thoughts? ---------- components: Library (Lib) messages: 344102 nosy: mariocj89, p-ganssle, vinay.sajip priority: normal severity: normal status: open title: Simplify customization of the logging time through datefmt type: enhancement versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri May 31 16:49:02 2019 From: report at bugs.python.org (gary*atlanta) Date: Fri, 31 May 2019 20:49:02 +0000 Subject: [New-bugs-announce] [issue37118] Why is GIL on 2.7 so much faster than 3.7 Message-ID: <1559335742.69.0.897974337357.issue37118@roundup.psfhosted.org> Change by gary*atlanta : ---------- components: Interpreter Core nosy: gary*atlanta priority: normal severity: normal status: open title: Why is GIL on 2.7 so much faster than 3.7 versions: Python 3.7 _______________________________________ Python tracker _______________________________________