Mailman 3 April 2019 - Python-checkins

bpo-36157:Document PyInterpreterState_Main() (GH-12238)
by Miss Islington (bot) April 1, 2019

April 1, 2019

https://github.com/python/cpython/commit/35fc38e5e82d18ceec95af9af210331949… commit: 35fc38e5e82d18ceec95af9af2103319497e2eaf branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-04-01T08:15:10-07:00 summary: bpo-36157:Document PyInterpreterState_Main() (GH-12238) I have added documentation for `PyInterpreterState_Main()`. I chose to place it under Advanced Debugger Support together with … [View More]

1 0

bpo-36157:Document PyInterpreterState_Main() (GH-12238)
by Miss Islington (bot) April 1, 2019

April 1, 2019

https://github.com/python/cpython/commit/8c61739defd88c7f79e86537886c337458… commit: 8c61739defd88c7f79e86537886c33745843ce01 branch: master author: Joannah Nanjekye <33177550+nanjekyejoannah(a)users.noreply.github.com> committer: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> date: 2019-04-01T08:08:43-07:00 summary: bpo-36157:Document PyInterpreterState_Main() (GH-12238) I have added documentation for `PyInterpreterState_Main()`. I chose to place it … [View More]

1 0

bpo-36495: Fix two out-of-bounds array reads (GH-12641)
by Guido van Rossum April 1, 2019

April 1, 2019

https://github.com/python/cpython/commit/a4d78362397fc3bced6ea80fbc7b5f4827… commit: a4d78362397fc3bced6ea80fbc7b5f4827aec55e branch: master author: Brad Larsen <brad(a)bradfordlarsen.com> committer: Guido van Rossum <guido(a)python.org> date: 2019-04-01T07:36:05-07:00 summary: bpo-36495: Fix two out-of-bounds array reads (GH-12641) Research and fix by @bradlarsen. files: M Python/ast.c diff --git a/Python/ast.c b/Python/ast.c index e9154fecff06..913e53ad7937 100644 --- a/Python/… [View More]

1 0

bpo-20844: open script file with "rb" mode (GH-12616)
by Inada Naoki April 1, 2019

April 1, 2019

https://github.com/python/cpython/commit/8384670615a90418fc52c3881242b7c10d… commit: 8384670615a90418fc52c3881242b7c10d1f2b13 branch: 3.7 author: Inada Naoki <songofacandy(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-04-01T21:02:51+09:00 summary: bpo-20844: open script file with "rb" mode (GH-12616) (cherry picked from commit 10654c19b5e6efdf3c529ff9bf7bcab89bdca1c1) files: A Misc/NEWS.d/next/Core and Builtins/2019-03-29-18-47-50.bpo-20844.ge-7SM.rst M Doc/c-… [View More]api/veryhigh.rst M Lib/test/test_cmd_line_script.py M Modules/main.c diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index c891f6320f94..317093e95615 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -109,6 +109,10 @@ the same library that the Python runtime is using. (:func:`sys.getfilesystemencoding`). If *closeit* is true, the file is closed before PyRun_SimpleFileExFlags returns. + .. note:: + On Windows, *fp* should be opened as binary mode (e.g. ``fopen(filename, "rb")``. + Otherwise, Python may not handle script file with LF line ending correctly. + .. c:function:: int PyRun_InteractiveOne(FILE *fp, const char *filename) diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index 5ec9bbbb1230..b2632602d454 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -387,6 +387,23 @@ def test_issue8202_dash_m_file_ignored(self): script_name, script_name, script_dir, '', importlib.machinery.SourceFileLoader) + def test_issue20884(self): + # On Windows, script with encoding cookie and LF line ending + # will be failed. + with support.temp_dir() as script_dir: + script_name = os.path.join(script_dir, "issue20884.py") + with open(script_name, "w", newline='\n') as f: + f.write("#coding: iso-8859-1\n") + f.write('"""\n') + for _ in range(30): + f.write('x'*80 + '\n') + f.write('"""\n') + + with support.change_cwd(path=script_dir): + rc, out, err = assert_python_ok(script_name) + self.assertEqual(b"", out) + self.assertEqual(b"", err) + @contextlib.contextmanager def setup_test_pkg(self, *args): with support.temp_dir() as script_dir, \ diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-03-29-18-47-50.bpo-20844.ge-7SM.rst b/Misc/NEWS.d/next/Core and Builtins/2019-03-29-18-47-50.bpo-20844.ge-7SM.rst new file mode 100644 index 000000000000..22a400ae7c8f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-03-29-18-47-50.bpo-20844.ge-7SM.rst @@ -0,0 +1,2 @@ +Fix running script with encoding cookie and LF line ending +may fail on Windows. diff --git a/Modules/main.c b/Modules/main.c index 9011bd1f69cb..e3683b941754 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -1534,7 +1534,7 @@ pymain_open_filename(_PyMain *pymain) const _PyCoreConfig *config = &_PyGILState_GetInterpreterStateUnsafe()->core_config; FILE* fp; - fp = _Py_wfopen(pymain->filename, L"r"); + fp = _Py_wfopen(pymain->filename, L"rb"); if (fp == NULL) { char *cfilename_buffer; const char *cfilename; [View Less]

1 0

bpo-20844: open script file with "rb" mode (GH-12616)
by Inada Naoki April 1, 2019

April 1, 2019

https://github.com/python/cpython/commit/10654c19b5e6efdf3c529ff9bf7bcab89b… commit: 10654c19b5e6efdf3c529ff9bf7bcab89bdca1c1 branch: master author: Inada Naoki <songofacandy(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-04-01T18:35:20+09:00 summary: bpo-20844: open script file with "rb" mode (GH-12616) files: A Misc/NEWS.d/next/Core and Builtins/2019-03-29-18-47-50.bpo-20844.ge-7SM.rst M Doc/c-api/veryhigh.rst M Lib/test/test_cmd_line_script.py M Modules/main.c … [View More] diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index c891f6320f94..317093e95615 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -109,6 +109,10 @@ the same library that the Python runtime is using. (:func:`sys.getfilesystemencoding`). If *closeit* is true, the file is closed before PyRun_SimpleFileExFlags returns. + .. note:: + On Windows, *fp* should be opened as binary mode (e.g. ``fopen(filename, "rb")``. + Otherwise, Python may not handle script file with LF line ending correctly. + .. c:function:: int PyRun_InteractiveOne(FILE *fp, const char *filename) diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index 85d2a4be069b..d138ca027c68 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -409,6 +409,23 @@ def test_issue8202_dash_m_file_ignored(self): script_name, script_name, script_dir, '', importlib.machinery.SourceFileLoader) + def test_issue20884(self): + # On Windows, script with encoding cookie and LF line ending + # will be failed. + with support.temp_dir() as script_dir: + script_name = os.path.join(script_dir, "issue20884.py") + with open(script_name, "w", newline='\n') as f: + f.write("#coding: iso-8859-1\n") + f.write('"""\n') + for _ in range(30): + f.write('x'*80 + '\n') + f.write('"""\n') + + with support.change_cwd(path=script_dir): + rc, out, err = assert_python_ok(script_name) + self.assertEqual(b"", out) + self.assertEqual(b"", err) + @contextlib.contextmanager def setup_test_pkg(self, *args): with support.temp_dir() as script_dir, \ diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-03-29-18-47-50.bpo-20844.ge-7SM.rst b/Misc/NEWS.d/next/Core and Builtins/2019-03-29-18-47-50.bpo-20844.ge-7SM.rst new file mode 100644 index 000000000000..22a400ae7c8f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-03-29-18-47-50.bpo-20844.ge-7SM.rst @@ -0,0 +1,2 @@ +Fix running script with encoding cookie and LF line ending +may fail on Windows. diff --git a/Modules/main.c b/Modules/main.c index 42d2c3c2aeec..6a7f735ed692 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -283,7 +283,7 @@ static int pymain_run_file(_PyCoreConfig *config, PyCompilerFlags *cf) { const wchar_t *filename = config->run_filename; - FILE *fp = _Py_wfopen(filename, L"r"); + FILE *fp = _Py_wfopen(filename, L"rb"); if (fp == NULL) { char *cfilename_buffer; const char *cfilename; [View Less]

1 0

bpo-36026: make descr error message consistent (GH-11930)
by Inada Naoki April 1, 2019

April 1, 2019

https://github.com/python/cpython/commit/62f9588663ebfea1735e9d142ef527395a… commit: 62f9588663ebfea1735e9d142ef527395a6c2b95 branch: master author: Inada Naoki <songofacandy(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-04-01T17:56:11+09:00 summary: bpo-36026: make descr error message consistent (GH-11930) set.add(0) and set.add.__get__(0) now raise TypeError with same error message. files: M Lib/test/test_descr.py M Objects/descrobject.c diff --git a/Lib/… [View More]test/test_descr.py b/Lib/test/test_descr.py index 09eef8c56f30..e37a98417f50 100644 --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -1614,10 +1614,14 @@ class SubSpam(spam.spamlist): pass with self.assertRaises(TypeError) as cm: spam_cm(list) - self.assertEqual( - str(cm.exception), + expected_errmsg = ( "descriptor 'classmeth' requires a subtype of 'xxsubtype.spamlist' " "but received 'list'") + self.assertEqual(str(cm.exception), expected_errmsg) + + with self.assertRaises(TypeError) as cm: + spam_cm.__get__(None, list) + self.assertEqual(str(cm.exception), expected_errmsg) def test_staticmethods(self): # Testing static methods... @@ -1952,6 +1956,29 @@ class E(object): self.assertEqual(E().foo.__func__, C.foo) # i.e., unbound self.assertTrue(repr(C.foo.__get__(C(1))).startswith("<bound method ")) + @support.impl_detail("testing error message from implementation") + def test_methods_in_c(self): + # This test checks error messages in builtin method descriptor. + # It is allowed that other Python implementations use + # different error messages. + set_add = set.add + + expected_errmsg = "descriptor 'add' of 'set' object needs an argument" + + with self.assertRaises(TypeError) as cm: + set_add() + self.assertEqual(cm.exception.args[0], expected_errmsg) + + expected_errmsg = "descriptor 'add' for 'set' objects doesn't apply to a 'int' object" + + with self.assertRaises(TypeError) as cm: + set_add(0) + self.assertEqual(cm.exception.args[0], expected_errmsg) + + with self.assertRaises(TypeError) as cm: + set_add.__get__(0) + self.assertEqual(cm.exception.args[0], expected_errmsg) + def test_special_method_lookup(self): # The lookup of special methods bypasses __getattr__ and # __getattribute__, but they still can be descriptors. diff --git a/Objects/descrobject.c b/Objects/descrobject.c index ab4151ec93e0..0fe9a441b82c 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -78,8 +78,8 @@ descr_check(PyDescrObject *descr, PyObject *obj, PyObject **pres) } if (!PyObject_TypeCheck(obj, descr->d_type)) { PyErr_Format(PyExc_TypeError, - "descriptor '%V' for '%s' objects " - "doesn't apply to '%s' object", + "descriptor '%V' for '%.100s' objects " + "doesn't apply to a '%.100s' object", descr_name((PyDescrObject *)descr), "?", descr->d_type->tp_name, obj->ob_type->tp_name); @@ -99,7 +99,7 @@ classmethod_get(PyMethodDescrObject *descr, PyObject *obj, PyObject *type) else { /* Wot - no type?! */ PyErr_Format(PyExc_TypeError, - "descriptor '%V' for type '%s' " + "descriptor '%V' for type '%.100s' " "needs either an object or a type", descr_name((PyDescrObject *)descr), "?", PyDescr_TYPE(descr)->tp_name); @@ -108,8 +108,8 @@ classmethod_get(PyMethodDescrObject *descr, PyObject *obj, PyObject *type) } if (!PyType_Check(type)) { PyErr_Format(PyExc_TypeError, - "descriptor '%V' for type '%s' " - "needs a type, not a '%s' as arg 2", + "descriptor '%V' for type '%.100s' " + "needs a type, not a '%.100s' as arg 2", descr_name((PyDescrObject *)descr), "?", PyDescr_TYPE(descr)->tp_name, type->ob_type->tp_name); @@ -117,8 +117,8 @@ classmethod_get(PyMethodDescrObject *descr, PyObject *obj, PyObject *type) } if (!PyType_IsSubtype((PyTypeObject *)type, PyDescr_TYPE(descr))) { PyErr_Format(PyExc_TypeError, - "descriptor '%V' for type '%s' " - "doesn't apply to type '%s'", + "descriptor '%V' requires a subtype of '%.100s' " + "but received '%.100s'", descr_name((PyDescrObject *)descr), "?", PyDescr_TYPE(descr)->tp_name, ((PyTypeObject *)type)->tp_name); @@ -181,7 +181,7 @@ descr_setcheck(PyDescrObject *descr, PyObject *obj, PyObject *value, if (!PyObject_TypeCheck(obj, descr->d_type)) { PyErr_Format(PyExc_TypeError, "descriptor '%V' for '%.100s' objects " - "doesn't apply to '%.100s' object", + "doesn't apply to a '%.100s' object", descr_name(descr), "?", descr->d_type->tp_name, obj->ob_type->tp_name); @@ -239,9 +239,8 @@ methoddescr_call(PyMethodDescrObject *descr, PyObject *args, PyObject *kwargs) if (!_PyObject_RealIsSubclass((PyObject *)Py_TYPE(self), (PyObject *)PyDescr_TYPE(descr))) { PyErr_Format(PyExc_TypeError, - "descriptor '%V' " - "requires a '%.100s' object " - "but received a '%.100s'", + "descriptor '%V' for '%.100s' objects " + "doesn't apply to a '%.100s' object", descr_name((PyDescrObject *)descr), "?", PyDescr_TYPE(descr)->tp_name, self->ob_type->tp_name); @@ -278,9 +277,8 @@ _PyMethodDescr_FastCallKeywords(PyObject *descrobj, if (!_PyObject_RealIsSubclass((PyObject *)Py_TYPE(self), (PyObject *)PyDescr_TYPE(descr))) { PyErr_Format(PyExc_TypeError, - "descriptor '%V' " - "requires a '%.100s' object " - "but received a '%.100s'", + "descriptor '%V' for '%.100s' objects " + "doesn't apply to a '%.100s' object", descr_name((PyDescrObject *)descr), "?", PyDescr_TYPE(descr)->tp_name, self->ob_type->tp_name); [View Less]

1 0

[3.7] bpo-36492: Fix passing special keyword arguments to some functions. (GH-12637) (GH-12645)
by Serhiy Storchaka April 1, 2019

April 1, 2019

https://github.com/python/cpython/commit/a37f356de19828241bf19129f804369794… commit: a37f356de19828241bf19129f804369794c72ed3 branch: 3.7 author: Serhiy Storchaka <storchaka(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-04-01T10:59:24+03:00 summary: [3.7] bpo-36492: Fix passing special keyword arguments to some functions. (GH-12637) (GH-12645) The following arguments can be passed as keyword arguments for passing to other function if the corresponding required … [View More]argument is passed as positional: - "func" in functools.partialmethod(), weakref.finalize(), profile.Profile.runcall(), cProfile.Profile.runcall(), bdb.Bdb.runcall(), trace.Trace.runfunc() and curses.wrapper(). - "function" in unittest.addModuleCleanup() and unittest.TestCase.addCleanup(). - "fn" in the submit() method of concurrent.futures.ThreadPoolExecutor and concurrent.futures.ProcessPoolExecutor. - "callback" in contextlib.ExitStack.callback(), contextlib.AsyncExitStack.callback() and contextlib.AsyncExitStack.push_async_callback(). - "c" and "typeid" in multiprocessing.managers.Server.create(). - "obj" in weakref.finalize(). (cherry picked from commit 42a139ed88c487f325a241c6ee8b308b3c045975) files: A Misc/NEWS.d/next/Library/2019-03-31-10-21-54.bpo-36492.f7vyUs.rst M Lib/bdb.py M Lib/cProfile.py M Lib/concurrent/futures/_base.py M Lib/concurrent/futures/process.py M Lib/concurrent/futures/thread.py M Lib/contextlib.py M Lib/curses/__init__.py M Lib/functools.py M Lib/multiprocessing/managers.py M Lib/profile.py M Lib/test/test_concurrent_futures.py M Lib/test/test_contextlib.py M Lib/test/test_contextlib_async.py M Lib/test/test_functools.py M Lib/test/test_trace.py M Lib/test/test_weakref.py M Lib/trace.py M Lib/unittest/case.py M Lib/weakref.py diff --git a/Lib/bdb.py b/Lib/bdb.py index 25c6260c47c7..caf207733b73 100644 --- a/Lib/bdb.py +++ b/Lib/bdb.py @@ -616,11 +616,23 @@ def runctx(self, cmd, globals, locals): # This method is more useful to debug a single function call. - def runcall(self, func, *args, **kwds): + def runcall(*args, **kwds): """Debug a single function call. Return the result of the function call. """ + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor 'runcall' of 'Bdb' object " + "needs an argument") + elif 'func' in kwds: + func = kwds.pop('func') + self, *args = args + else: + raise TypeError('runcall expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + self.reset() sys.settrace(self.trace_dispatch) res = None diff --git a/Lib/cProfile.py b/Lib/cProfile.py index f6e423b3dd1a..9485d3d515e4 100755 --- a/Lib/cProfile.py +++ b/Lib/cProfile.py @@ -103,7 +103,19 @@ def runctx(self, cmd, globals, locals): return self # This method is more useful to profile a single function call. - def runcall(self, func, *args, **kw): + def runcall(*args, **kw): + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor 'runcall' of 'Profile' object " + "needs an argument") + elif 'func' in kw: + func = kw.pop('func') + self, *args = args + else: + raise TypeError('runcall expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + self.enable() try: return func(*args, **kw) diff --git a/Lib/concurrent/futures/_base.py b/Lib/concurrent/futures/_base.py index 61c81bb7fe68..0b847307a328 100644 --- a/Lib/concurrent/futures/_base.py +++ b/Lib/concurrent/futures/_base.py @@ -536,7 +536,7 @@ def set_exception(self, exception): class Executor(object): """This is an abstract base class for concrete asynchronous executors.""" - def submit(self, fn, *args, **kwargs): + def submit(*args, **kwargs): """Submits a callable to be executed with the given arguments. Schedules the callable to be executed as fn(*args, **kwargs) and returns @@ -545,6 +545,15 @@ def submit(self, fn, *args, **kwargs): Returns: A Future representing the given call. """ + if len(args) >= 2: + pass + elif not args: + raise TypeError("descriptor 'submit' of 'Executor' object " + "needs an argument") + elif 'fn' not in kwargs: + raise TypeError('submit expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + raise NotImplementedError() def map(self, fn, *iterables, timeout=None, chunksize=1): diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py index ce7d642b098a..8a0ed98b3e88 100644 --- a/Lib/concurrent/futures/process.py +++ b/Lib/concurrent/futures/process.py @@ -593,7 +593,19 @@ def _adjust_process_count(self): p.start() self._processes[p.pid] = p - def submit(self, fn, *args, **kwargs): + def submit(*args, **kwargs): + if len(args) >= 2: + self, fn, *args = args + elif not args: + raise TypeError("descriptor 'submit' of 'ProcessPoolExecutor' object " + "needs an argument") + elif 'fn' in kwargs: + fn = kwargs.pop('fn') + self, *args = args + else: + raise TypeError('submit expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + with self._shutdown_lock: if self._broken: raise BrokenProcessPool(self._broken) diff --git a/Lib/concurrent/futures/thread.py b/Lib/concurrent/futures/thread.py index 78359711d5d9..c7c9ef44c60b 100644 --- a/Lib/concurrent/futures/thread.py +++ b/Lib/concurrent/futures/thread.py @@ -142,7 +142,19 @@ def __init__(self, max_workers=None, thread_name_prefix='', self._initializer = initializer self._initargs = initargs - def submit(self, fn, *args, **kwargs): + def submit(*args, **kwargs): + if len(args) >= 2: + self, fn, *args = args + elif not args: + raise TypeError("descriptor 'submit' of 'ThreadPoolExecutor' object " + "needs an argument") + elif 'fn' in kwargs: + fn = kwargs.pop('fn') + self, *args = args + else: + raise TypeError('submit expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + with self._shutdown_lock: if self._broken: raise BrokenThreadPool(self._broken) diff --git a/Lib/contextlib.py b/Lib/contextlib.py index 1ff8cdf1cecf..2d745ea3e3c5 100644 --- a/Lib/contextlib.py +++ b/Lib/contextlib.py @@ -378,7 +378,8 @@ def _exit_wrapper(exc_type, exc, tb): return _exit_wrapper @staticmethod - def _create_cb_wrapper(callback, *args, **kwds): + def _create_cb_wrapper(*args, **kwds): + callback, *args = args def _exit_wrapper(exc_type, exc, tb): callback(*args, **kwds) return _exit_wrapper @@ -427,11 +428,23 @@ def enter_context(self, cm): self._push_cm_exit(cm, _exit) return result - def callback(self, callback, *args, **kwds): + def callback(*args, **kwds): """Registers an arbitrary callback and arguments. Cannot suppress exceptions. """ + if len(args) >= 2: + self, callback, *args = args + elif not args: + raise TypeError("descriptor 'callback' of '_BaseExitStack' object " + "needs an argument") + elif 'callback' in kwds: + callback = kwds.pop('callback') + self, *args = args + else: + raise TypeError('callback expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + _exit_wrapper = self._create_cb_wrapper(callback, *args, **kwds) # We changed the signature, so using @wraps is not appropriate, but @@ -540,7 +553,8 @@ def _create_async_exit_wrapper(cm, cm_exit): return _exit_wrapper @staticmethod - def _create_async_cb_wrapper(callback, *args, **kwds): + def _create_async_cb_wrapper(*args, **kwds): + callback, *args = args async def _exit_wrapper(exc_type, exc, tb): await callback(*args, **kwds) return _exit_wrapper @@ -575,11 +589,23 @@ def push_async_exit(self, exit): self._push_async_cm_exit(exit, exit_method) return exit # Allow use as a decorator - def push_async_callback(self, callback, *args, **kwds): + def push_async_callback(*args, **kwds): """Registers an arbitrary coroutine function and arguments. Cannot suppress exceptions. """ + if len(args) >= 2: + self, callback, *args = args + elif not args: + raise TypeError("descriptor 'push_async_callback' of " + "'AsyncExitStack' object needs an argument") + elif 'callback' in kwds: + callback = kwds.pop('callback') + self, *args = args + else: + raise TypeError('push_async_callback expected at least 1 ' + 'positional argument, got %d' % (len(args)-1)) + _exit_wrapper = self._create_async_cb_wrapper(callback, *args, **kwds) # We changed the signature, so using @wraps is not appropriate, but diff --git a/Lib/curses/__init__.py b/Lib/curses/__init__.py index 47378741acc2..b3373af70e88 100644 --- a/Lib/curses/__init__.py +++ b/Lib/curses/__init__.py @@ -60,7 +60,7 @@ def start_color(): # raises an exception, wrapper() will restore the terminal to a sane state so # you can read the resulting traceback. -def wrapper(func, *args, **kwds): +def wrapper(*args, **kwds): """Wrapper function that initializes curses and calls another function, restoring normal keyboard/screen behavior on error. The callable object 'func' is then passed the main window 'stdscr' @@ -68,6 +68,14 @@ def wrapper(func, *args, **kwds): wrapper(). """ + if args: + func, *args = args + elif 'func' in kwds: + func = kwds.pop('func') + else: + raise TypeError('wrapper expected at least 1 positional argument, ' + 'got %d' % len(args)) + try: # Initialize curses stdscr = initscr() diff --git a/Lib/functools.py b/Lib/functools.py index b734899b56de..1daa1d177591 100644 --- a/Lib/functools.py +++ b/Lib/functools.py @@ -323,7 +323,20 @@ class partialmethod(object): callables as instance methods. """ - def __init__(self, func, *args, **keywords): + def __init__(*args, **keywords): + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor '__init__' of partialmethod " + "needs an argument") + elif 'func' in keywords: + func = keywords.pop('func') + self, *args = args + else: + raise TypeError("type 'partialmethod' takes at least one argument, " + "got %d" % (len(args)-1)) + args = tuple(args) + if not callable(func) and not hasattr(func, "__get__"): raise TypeError("{!r} is not callable or a descriptor" .format(func)) diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py index 4ae8ddc77018..8e8d28f4b7cd 100644 --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -351,10 +351,30 @@ def shutdown(self, c): finally: self.stop_event.set() - def create(self, c, typeid, *args, **kwds): + def create(*args, **kwds): ''' Create a new shared object and return its id ''' + if len(args) >= 3: + self, c, typeid, *args = args + elif not args: + raise TypeError("descriptor 'create' of 'Server' object " + "needs an argument") + else: + if 'typeid' not in kwds: + raise TypeError('create expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) + typeid = kwds.pop('typeid') + if len(args) >= 2: + self, c, *args = args + else: + if 'c' not in kwds: + raise TypeError('create expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) + c = kwds.pop('c') + self, *args = args + args = tuple(args) + with self.mutex: callable, exposed, method_to_typeid, proxytype = \ self.registry[typeid] @@ -576,10 +596,13 @@ def _run_server(cls, registry, address, authkey, serializer, writer, util.info('manager serving at %r', server.address) server.serve_forever() - def _create(self, typeid, *args, **kwds): + def _create(*args, **kwds): ''' Create a new shared object; return the token and exposed tuple ''' + self, typeid, *args = args + args = tuple(args) + assert self._state.value == State.STARTED, 'server not yet started' conn = self._Client(self._address, authkey=self._authkey) try: diff --git a/Lib/profile.py b/Lib/profile.py index 0340a7907bfd..c26f0c8bfa93 100755 --- a/Lib/profile.py +++ b/Lib/profile.py @@ -425,7 +425,19 @@ def runctx(self, cmd, globals, locals): return self # This method is more useful to profile a single function call. - def runcall(self, func, *args, **kw): + def runcall(*args, **kw): + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor 'runcall' of 'Profile' object " + "needs an argument") + elif 'func' in kw: + func = kw.pop('func') + self, *args = args + else: + raise TypeError('runcall expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + self.set_cmd(repr(func)) sys.setprofile(self.dispatcher) try: diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py index 7e5a4edf80b2..59aa0f46f59f 100644 --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -49,6 +49,9 @@ def create_future(state=PENDING, exception=None, result=None): def mul(x, y): return x * y +def capture(*args, **kwargs): + return args, kwargs + def sleep_and_raise(t): time.sleep(t) raise Exception('this is an exception') @@ -658,6 +661,12 @@ def test_submit(self): def test_submit_keyword(self): future = self.executor.submit(mul, 2, y=8) self.assertEqual(16, future.result()) + future = self.executor.submit(capture, 1, self=2, fn=3) + self.assertEqual(future.result(), ((1,), {'self': 2, 'fn': 3})) + future = self.executor.submit(fn=capture, arg=1) + self.assertEqual(future.result(), ((), {'arg': 1})) + with self.assertRaises(TypeError): + self.executor.submit(arg=1) def test_map(self): self.assertEqual( diff --git a/Lib/test/test_contextlib.py b/Lib/test/test_contextlib.py index 2a44404a603e..30c2e27b3c7e 100644 --- a/Lib/test/test_contextlib.py +++ b/Lib/test/test_contextlib.py @@ -575,6 +575,7 @@ def test_callback(self): ((), dict(example=1)), ((1,), dict(example=1)), ((1,2), dict(example=1)), + ((1,2), dict(self=3, callback=4)), ] result = [] def _exit(*args, **kwds): @@ -597,6 +598,15 @@ def _exit(*args, **kwds): self.assertIsNone(wrapper[1].__doc__, _exit.__doc__) self.assertEqual(result, expected) + result = [] + with self.exit_stack() as stack: + with self.assertRaises(TypeError): + stack.callback(arg=1) + with self.assertRaises(TypeError): + self.exit_stack.callback(arg=2) + stack.callback(callback=_exit, arg=3) + self.assertEqual(result, [((), {'arg': 3})]) + def test_push(self): exc_raised = ZeroDivisionError def _expect_exc(exc_type, exc, exc_tb): diff --git a/Lib/test/test_contextlib_async.py b/Lib/test/test_contextlib_async.py index e34fc38d1180..cc38dcf8c45c 100644 --- a/Lib/test/test_contextlib_async.py +++ b/Lib/test/test_contextlib_async.py @@ -329,6 +329,15 @@ def setUp(self): self.assertEqual(result, expected) + result = [] + async with AsyncExitStack() as stack: + with self.assertRaises(TypeError): + stack.push_async_callback(arg=1) + with self.assertRaises(TypeError): + self.exit_stack.push_async_callback(arg=2) + stack.push_async_callback(callback=_exit, arg=3) + self.assertEqual(result, [((), {'arg': 3})]) + @_async_test async def test_async_push(self): exc_raised = ZeroDivisionError diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index a91c6348e709..a7625d609039 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -464,6 +464,7 @@ class A(object): positional = functools.partialmethod(capture, 1) keywords = functools.partialmethod(capture, a=2) both = functools.partialmethod(capture, 3, b=4) + spec_keywords = functools.partialmethod(capture, self=1, func=2) nested = functools.partialmethod(positional, 5) @@ -497,6 +498,8 @@ def test_arg_combinations(self): self.assertEqual(self.A.both(self.a, 5, c=6), ((self.a, 3, 5), {'b': 4, 'c': 6})) + self.assertEqual(self.a.spec_keywords(), ((self.a,), {'self': 1, 'func': 2})) + def test_nested(self): self.assertEqual(self.a.nested(), ((self.a, 1, 5), {})) self.assertEqual(self.a.nested(6), ((self.a, 1, 5, 6), {})) @@ -550,6 +553,13 @@ def test_invalid_args(self): with self.assertRaises(TypeError): class B(object): method = functools.partialmethod(None, 1) + with self.assertRaises(TypeError): + class B: + method = functools.partialmethod() + class B: + method = functools.partialmethod(func=capture, a=1) + b = B() + self.assertEqual(b.method(2, x=3), ((b, 2), {'a': 1, 'x': 3})) def test_repr(self): self.assertEqual(repr(vars(self.A)['both']), diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py index 63f474179d68..66fff1cef470 100644 --- a/Lib/test/test_trace.py +++ b/Lib/test/test_trace.py @@ -70,6 +70,9 @@ def traced_func_calling_generator(): def traced_doubler(num): return num * 2 +def traced_capturer(*args, **kwargs): + return args, kwargs + def traced_caller_list_comprehension(): k = 10 mylist = [traced_doubler(i) for i in range(k)] @@ -238,6 +241,14 @@ def test_simple_caller(self): } self.assertEqual(self.tracer.results().calledfuncs, expected) + def test_arg_errors(self): + res = self.tracer.runfunc(traced_capturer, 1, 2, self=3, func=4) + self.assertEqual(res, ((1, 2), {'self': 3, 'func': 4})) + res = self.tracer.runfunc(func=traced_capturer, arg=1) + self.assertEqual(res, ((), {'arg': 1})) + with self.assertRaises(TypeError): + self.tracer.runfunc() + def test_loop_caller_importing(self): self.tracer.runfunc(traced_func_importing_caller, 1) diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py index 1fac08dafc7d..ad7a6acfcc7d 100644 --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -1839,6 +1839,33 @@ def add(x,y,z): self.assertEqual(f.alive, False) self.assertEqual(res, [199]) + def test_arg_errors(self): + def fin(*args, **kwargs): + res.append((args, kwargs)) + + a = self.A() + + res = [] + f = weakref.finalize(a, fin, 1, 2, func=3, obj=4) + self.assertEqual(f.peek(), (a, fin, (1, 2), {'func': 3, 'obj': 4})) + f() + self.assertEqual(res, [((1, 2), {'func': 3, 'obj': 4})]) + + res = [] + f = weakref.finalize(a, func=fin, arg=1) + self.assertEqual(f.peek(), (a, fin, (), {'arg': 1})) + f() + self.assertEqual(res, [((), {'arg': 1})]) + + res = [] + f = weakref.finalize(obj=a, func=fin, arg=1) + self.assertEqual(f.peek(), (a, fin, (), {'arg': 1})) + f() + self.assertEqual(res, [((), {'arg': 1})]) + + self.assertRaises(TypeError, weakref.finalize, a) + self.assertRaises(TypeError, weakref.finalize) + def test_order(self): a = self.A() res = [] diff --git a/Lib/trace.py b/Lib/trace.py index 0ed7ba95b520..206bd2b689f1 100755 --- a/Lib/trace.py +++ b/Lib/trace.py @@ -452,7 +452,19 @@ def runctx(self, cmd, globals=None, locals=None): sys.settrace(None) threading.settrace(None) - def runfunc(self, func, *args, **kw): + def runfunc(*args, **kw): + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor 'runfunc' of 'Trace' object " + "needs an argument") + elif 'func' in kw: + func = kw.pop('func') + self, *args = args + else: + raise TypeError('runfunc expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + result = None if not self.donothing: sys.settrace(self.globaltrace) diff --git a/Lib/unittest/case.py b/Lib/unittest/case.py index 758924d80113..811f5df23dd1 100644 --- a/Lib/unittest/case.py +++ b/Lib/unittest/case.py @@ -442,12 +442,25 @@ def addTypeEqualityFunc(self, typeobj, function): """ self._type_equality_funcs[typeobj] = function - def addCleanup(self, function, *args, **kwargs): + def addCleanup(*args, **kwargs): """Add a function, with arguments, to be called when the test is completed. Functions added are called on a LIFO basis and are called after tearDown on test failure or success. Cleanup items are called even if setUp fails (unlike tearDown).""" + if len(args) >= 2: + self, function, *args = args + elif not args: + raise TypeError("descriptor 'addCleanup' of 'TestCase' object " + "needs an argument") + elif 'function' in kwargs: + function = kwargs.pop('function') + self, *args = args + else: + raise TypeError('addCleanup expected at least 1 positional ' + 'argument, got %d' % (len(args)-1)) + args = tuple(args) + self._cleanups.append((function, args, kwargs)) def setUp(self): diff --git a/Lib/weakref.py b/Lib/weakref.py index 753f07291e20..59b3aa5621a3 100644 --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -527,7 +527,27 @@ class finalize: class _Info: __slots__ = ("weakref", "func", "args", "kwargs", "atexit", "index") - def __init__(self, obj, func, *args, **kwargs): + def __init__(*args, **kwargs): + if len(args) >= 3: + self, obj, func, *args = args + elif not args: + raise TypeError("descriptor '__init__' of 'finalize' object " + "needs an argument") + else: + if 'func' not in kwargs: + raise TypeError('finalize expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) + func = kwargs.pop('func') + if len(args) >= 2: + self, obj, *args = args + else: + if 'obj' not in kwargs: + raise TypeError('finalize expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) + obj = kwargs.pop('obj') + self, *args = args + args = tuple(args) + if not self._registered_with_atexit: # We may register the exit function more than once because # of a thread race, but that is harmless diff --git a/Misc/NEWS.d/next/Library/2019-03-31-10-21-54.bpo-36492.f7vyUs.rst b/Misc/NEWS.d/next/Library/2019-03-31-10-21-54.bpo-36492.f7vyUs.rst new file mode 100644 index 000000000000..749e2a87ab6c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-03-31-10-21-54.bpo-36492.f7vyUs.rst @@ -0,0 +1,5 @@ +Arbitrary keyword arguments (even with names "self" and "func") can now be +passed to some functions which should accept arbitrary keyword arguments and +pass them to other function (for example partialmethod(), TestCase.addCleanup() +and Profile.runcall()) if the required arguments are passed as positional +arguments. [View Less]

1 0

bpo-36492: Deprecate passing some arguments as keyword arguments. (GH-12637)
by Serhiy Storchaka April 1, 2019

April 1, 2019

https://github.com/python/cpython/commit/42a139ed88c487f325a241c6ee8b308b3c… commit: 42a139ed88c487f325a241c6ee8b308b3c045975 branch: master author: Serhiy Storchaka <storchaka(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-04-01T09:16:35+03:00 summary: bpo-36492: Deprecate passing some arguments as keyword arguments. (GH-12637) Deprecated passing the following arguments as keyword arguments: - "func" in functools.partialmethod(), weakref.finalize(), profile.… [View More]Profile.runcall(), cProfile.Profile.runcall(), bdb.Bdb.runcall(), trace.Trace.runfunc() and curses.wrapper(). - "function" in unittest.addModuleCleanup() and unittest.TestCase.addCleanup(). - "fn" in the submit() method of concurrent.futures.ThreadPoolExecutor and concurrent.futures.ProcessPoolExecutor. - "callback" in contextlib.ExitStack.callback(), contextlib.AsyncExitStack.callback() and contextlib.AsyncExitStack.push_async_callback(). - "c" and "typeid" in the create() method of multiprocessing.managers.Server and multiprocessing.managers.SharedMemoryServer. - "obj" in weakref.finalize(). Also allowed to pass arbitrary keyword arguments (even "self" and "func") if the above arguments are passed as positional argument. files: A Misc/NEWS.d/next/Library/2019-03-31-10-21-54.bpo-36492.f7vyUs.rst M Doc/whatsnew/3.8.rst M Lib/bdb.py M Lib/cProfile.py M Lib/concurrent/futures/_base.py M Lib/concurrent/futures/process.py M Lib/concurrent/futures/thread.py M Lib/contextlib.py M Lib/curses/__init__.py M Lib/functools.py M Lib/multiprocessing/managers.py M Lib/profile.py M Lib/test/test_concurrent_futures.py M Lib/test/test_contextlib.py M Lib/test/test_contextlib_async.py M Lib/test/test_functools.py M Lib/test/test_trace.py M Lib/test/test_weakref.py M Lib/trace.py M Lib/unittest/case.py M Lib/unittest/test/test_runner.py M Lib/weakref.py diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index ccd8bbd81042..411f2299b290 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -594,6 +594,29 @@ Deprecated version they will be errors. (Contributed by Serhiy Storchaka in :issue:`36048`.) +* Deprecated passing the following arguments as keyword arguments: + + - *func* in :func:`functools.partialmethod`, :func:`weakref.finalize`, + :meth:`profile.Profile.runcall`, :meth:`cProfile.Profile.runcall`, + :meth:`bdb.Bdb.runcall`, :meth:`trace.Trace.runfunc` and + :func:`curses.wrapper`. + - *function* in :func:`unittest.addModuleCleanup` and + :meth:`unittest.TestCase.addCleanup`. + - *fn* in the :meth:`~concurrent.futures.Executor.submit` method of + :class:`concurrent.futures.ThreadPoolExecutor` and + :class:`concurrent.futures.ProcessPoolExecutor`. + - *callback* in :meth:`contextlib.ExitStack.callback`, + :meth:`contextlib.AsyncExitStack.callback` and + :meth:`contextlib.AsyncExitStack.push_async_callback`. + - *c* and *typeid* in the :meth:`~multiprocessing.managers.Server.create` + method of :class:`multiprocessing.managers.Server` and + :class:`multiprocessing.managers.SharedMemoryServer`. + - *obj* in :func:`weakref.finalize`. + + In future releases of Python they will be :ref:`positional-only + <positional-only_parameter>`. + (Contributed by Serhiy Storchaka in :issue:`36492`.) + API and Feature Removals ======================== diff --git a/Lib/bdb.py b/Lib/bdb.py index ec0f92c06a78..54aa98437450 100644 --- a/Lib/bdb.py +++ b/Lib/bdb.py @@ -618,11 +618,26 @@ def runctx(self, cmd, globals, locals): # This method is more useful to debug a single function call. - def runcall(self, func, *args, **kwds): + def runcall(*args, **kwds): """Debug a single function call. Return the result of the function call. """ + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor 'runcall' of 'Bdb' object " + "needs an argument") + elif 'func' in kwds: + func = kwds.pop('func') + self, *args = args + import warnings + warnings.warn("Passing 'func' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('runcall expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + self.reset() sys.settrace(self.trace_dispatch) res = None diff --git a/Lib/cProfile.py b/Lib/cProfile.py index 305e79e28049..2e449cc576ce 100755 --- a/Lib/cProfile.py +++ b/Lib/cProfile.py @@ -103,7 +103,22 @@ def runctx(self, cmd, globals, locals): return self # This method is more useful to profile a single function call. - def runcall(self, func, *args, **kw): + def runcall(*args, **kw): + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor 'runcall' of 'Profile' object " + "needs an argument") + elif 'func' in kw: + func = kw.pop('func') + self, *args = args + import warnings + warnings.warn("Passing 'func' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('runcall expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + self.enable() try: return func(*args, **kw) diff --git a/Lib/concurrent/futures/_base.py b/Lib/concurrent/futures/_base.py index 8b9dc507138e..ea16eef841c5 100644 --- a/Lib/concurrent/futures/_base.py +++ b/Lib/concurrent/futures/_base.py @@ -544,7 +544,7 @@ def set_exception(self, exception): class Executor(object): """This is an abstract base class for concrete asynchronous executors.""" - def submit(self, fn, *args, **kwargs): + def submit(*args, **kwargs): """Submits a callable to be executed with the given arguments. Schedules the callable to be executed as fn(*args, **kwargs) and returns @@ -553,6 +553,19 @@ def submit(self, fn, *args, **kwargs): Returns: A Future representing the given call. """ + if len(args) >= 2: + pass + elif not args: + raise TypeError("descriptor 'submit' of 'Executor' object " + "needs an argument") + elif 'fn' in kwargs: + import warnings + warnings.warn("Passing 'fn' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('submit expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + raise NotImplementedError() def map(self, fn, *iterables, timeout=None, chunksize=1): diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py index 9b85e7f33769..306e9ce47a6b 100644 --- a/Lib/concurrent/futures/process.py +++ b/Lib/concurrent/futures/process.py @@ -594,7 +594,22 @@ def _adjust_process_count(self): p.start() self._processes[p.pid] = p - def submit(self, fn, *args, **kwargs): + def submit(*args, **kwargs): + if len(args) >= 2: + self, fn, *args = args + elif not args: + raise TypeError("descriptor 'submit' of 'ProcessPoolExecutor' object " + "needs an argument") + elif 'fn' in kwargs: + fn = kwargs.pop('fn') + self, *args = args + import warnings + warnings.warn("Passing 'fn' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('submit expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + with self._shutdown_lock: if self._broken: raise BrokenProcessPool(self._broken) diff --git a/Lib/concurrent/futures/thread.py b/Lib/concurrent/futures/thread.py index 78359711d5d9..0a61e3a9ac1b 100644 --- a/Lib/concurrent/futures/thread.py +++ b/Lib/concurrent/futures/thread.py @@ -142,7 +142,22 @@ def __init__(self, max_workers=None, thread_name_prefix='', self._initializer = initializer self._initargs = initargs - def submit(self, fn, *args, **kwargs): + def submit(*args, **kwargs): + if len(args) >= 2: + self, fn, *args = args + elif not args: + raise TypeError("descriptor 'submit' of 'ThreadPoolExecutor' object " + "needs an argument") + elif 'fn' in kwargs: + fn = kwargs.pop('fn') + self, *args = args + import warnings + warnings.warn("Passing 'fn' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('submit expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + with self._shutdown_lock: if self._broken: raise BrokenThreadPool(self._broken) diff --git a/Lib/contextlib.py b/Lib/contextlib.py index c06ec73f489d..ae498a2b6ef5 100644 --- a/Lib/contextlib.py +++ b/Lib/contextlib.py @@ -377,7 +377,8 @@ def _create_exit_wrapper(cm, cm_exit): return MethodType(cm_exit, cm) @staticmethod - def _create_cb_wrapper(callback, *args, **kwds): + def _create_cb_wrapper(*args, **kwds): + callback, *args = args def _exit_wrapper(exc_type, exc, tb): callback(*args, **kwds) return _exit_wrapper @@ -426,11 +427,26 @@ def enter_context(self, cm): self._push_cm_exit(cm, _exit) return result - def callback(self, callback, *args, **kwds): + def callback(*args, **kwds): """Registers an arbitrary callback and arguments. Cannot suppress exceptions. """ + if len(args) >= 2: + self, callback, *args = args + elif not args: + raise TypeError("descriptor 'callback' of '_BaseExitStack' object " + "needs an argument") + elif 'callback' in kwds: + callback = kwds.pop('callback') + self, *args = args + import warnings + warnings.warn("Passing 'callback' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('callback expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + _exit_wrapper = self._create_cb_wrapper(callback, *args, **kwds) # We changed the signature, so using @wraps is not appropriate, but @@ -536,7 +552,8 @@ def _create_async_exit_wrapper(cm, cm_exit): return MethodType(cm_exit, cm) @staticmethod - def _create_async_cb_wrapper(callback, *args, **kwds): + def _create_async_cb_wrapper(*args, **kwds): + callback, *args = args async def _exit_wrapper(exc_type, exc, tb): await callback(*args, **kwds) return _exit_wrapper @@ -571,11 +588,26 @@ def push_async_exit(self, exit): self._push_async_cm_exit(exit, exit_method) return exit # Allow use as a decorator - def push_async_callback(self, callback, *args, **kwds): + def push_async_callback(*args, **kwds): """Registers an arbitrary coroutine function and arguments. Cannot suppress exceptions. """ + if len(args) >= 2: + self, callback, *args = args + elif not args: + raise TypeError("descriptor 'push_async_callback' of " + "'AsyncExitStack' object needs an argument") + elif 'callback' in kwds: + callback = kwds.pop('callback') + self, *args = args + import warnings + warnings.warn("Passing 'callback' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('push_async_callback expected at least 1 ' + 'positional argument, got %d' % (len(args)-1)) + _exit_wrapper = self._create_async_cb_wrapper(callback, *args, **kwds) # We changed the signature, so using @wraps is not appropriate, but diff --git a/Lib/curses/__init__.py b/Lib/curses/__init__.py index 47378741acc2..44a198428820 100644 --- a/Lib/curses/__init__.py +++ b/Lib/curses/__init__.py @@ -60,7 +60,7 @@ def start_color(): # raises an exception, wrapper() will restore the terminal to a sane state so # you can read the resulting traceback. -def wrapper(func, *args, **kwds): +def wrapper(*args, **kwds): """Wrapper function that initializes curses and calls another function, restoring normal keyboard/screen behavior on error. The callable object 'func' is then passed the main window 'stdscr' @@ -68,6 +68,17 @@ def wrapper(func, *args, **kwds): wrapper(). """ + if args: + func, *args = args + elif 'func' in kwds: + func = kwds.pop('func') + import warnings + warnings.warn("Passing 'func' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('wrapper expected at least 1 positional argument, ' + 'got %d' % len(args)) + try: # Initialize curses stdscr = initscr() diff --git a/Lib/functools.py b/Lib/functools.py index 426653f13f6d..1f1874db9b4c 100644 --- a/Lib/functools.py +++ b/Lib/functools.py @@ -354,7 +354,23 @@ class partialmethod(object): callables as instance methods. """ - def __init__(self, func, *args, **keywords): + def __init__(*args, **keywords): + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor '__init__' of partialmethod " + "needs an argument") + elif 'func' in keywords: + func = keywords.pop('func') + self, *args = args + import warnings + warnings.warn("Passing 'func' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError("type 'partialmethod' takes at least one argument, " + "got %d" % (len(args)-1)) + args = tuple(args) + if not callable(func) and not hasattr(func, "__get__"): raise TypeError("{!r} is not callable or a descriptor" .format(func)) diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py index 7973012b98d1..80c3ddb9154a 100644 --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -358,10 +358,36 @@ def shutdown(self, c): finally: self.stop_event.set() - def create(self, c, typeid, *args, **kwds): + def create(*args, **kwds): ''' Create a new shared object and return its id ''' + if len(args) >= 3: + self, c, typeid, *args = args + elif not args: + raise TypeError("descriptor 'create' of 'Server' object " + "needs an argument") + else: + if 'typeid' not in kwds: + raise TypeError('create expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) + typeid = kwds.pop('typeid') + if len(args) >= 2: + self, c, *args = args + import warnings + warnings.warn("Passing 'typeid' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + if 'c' not in kwds: + raise TypeError('create expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) + c = kwds.pop('c') + self, *args = args + import warnings + warnings.warn("Passing 'c' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + args = tuple(args) + with self.mutex: callable, exposed, method_to_typeid, proxytype = \ self.registry[typeid] @@ -583,10 +609,13 @@ def _run_server(cls, registry, address, authkey, serializer, writer, util.info('manager serving at %r', server.address) server.serve_forever() - def _create(self, typeid, *args, **kwds): + def _create(*args, **kwds): ''' Create a new shared object; return the token and exposed tuple ''' + self, typeid, *args = args + args = tuple(args) + assert self._state.value == State.STARTED, 'server not yet started' conn = self._Client(self._address, authkey=self._authkey) try: @@ -1261,15 +1290,25 @@ def __init__(self, *args, **kwargs): _SharedMemoryTracker(f"shmm_{self.address}_{getpid()}") util.debug(f"SharedMemoryServer started by pid {getpid()}") - def create(self, c, typeid, *args, **kwargs): + def create(*args, **kwargs): """Create a new distributed-shared object (not backed by a shared memory block) and return its id to be used in a Proxy Object.""" # Unless set up as a shared proxy, don't make shared_memory_context # a standard part of kwargs. This makes things easier for supplying # simple functions. + if len(args) >= 3: + typeod = args[2] + elif 'typeid' in kwargs: + typeid = kwargs['typeid'] + elif not args: + raise TypeError("descriptor 'create' of 'SharedMemoryServer' " + "object needs an argument") + else: + raise TypeError('create expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) if hasattr(self.registry[typeid][-1], "_shared_memory_proxy"): kwargs['shared_memory_context'] = self.shared_memory_context - return Server.create(self, c, typeid, *args, **kwargs) + return Server.create(*args, **kwargs) def shutdown(self, c): "Call unlink() on all tracked shared memory, terminate the Server." diff --git a/Lib/profile.py b/Lib/profile.py index 5df43604acdd..9a865d3f6f6e 100755 --- a/Lib/profile.py +++ b/Lib/profile.py @@ -425,7 +425,22 @@ def runctx(self, cmd, globals, locals): return self # This method is more useful to profile a single function call. - def runcall(self, func, *args, **kw): + def runcall(*args, **kw): + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor 'runcall' of 'Profile' object " + "needs an argument") + elif 'func' in kw: + func = kw.pop('func') + self, *args = args + import warnings + warnings.warn("Passing 'func' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('runcall expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + self.set_cmd(repr(func)) sys.setprofile(self.dispatcher) try: diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py index 01125c79ba57..903afbd2a4f6 100644 --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -49,6 +49,9 @@ def create_future(state=PENDING, exception=None, result=None): def mul(x, y): return x * y +def capture(*args, **kwargs): + return args, kwargs + def sleep_and_raise(t): time.sleep(t) raise Exception('this is an exception') @@ -658,6 +661,13 @@ def test_submit(self): def test_submit_keyword(self): future = self.executor.submit(mul, 2, y=8) self.assertEqual(16, future.result()) + future = self.executor.submit(capture, 1, self=2, fn=3) + self.assertEqual(future.result(), ((1,), {'self': 2, 'fn': 3})) + with self.assertWarns(DeprecationWarning): + future = self.executor.submit(fn=capture, arg=1) + self.assertEqual(future.result(), ((), {'arg': 1})) + with self.assertRaises(TypeError): + self.executor.submit(arg=1) def test_map(self): self.assertEqual( diff --git a/Lib/test/test_contextlib.py b/Lib/test/test_contextlib.py index 755d9b95a677..188a29d9f9fd 100644 --- a/Lib/test/test_contextlib.py +++ b/Lib/test/test_contextlib.py @@ -574,6 +574,7 @@ def test_callback(self): ((), dict(example=1)), ((1,), dict(example=1)), ((1,2), dict(example=1)), + ((1,2), dict(self=3, callback=4)), ] result = [] def _exit(*args, **kwds): @@ -596,6 +597,16 @@ def _exit(*args, **kwds): self.assertIsNone(wrapper[1].__doc__, _exit.__doc__) self.assertEqual(result, expected) + result = [] + with self.exit_stack() as stack: + with self.assertRaises(TypeError): + stack.callback(arg=1) + with self.assertRaises(TypeError): + self.exit_stack.callback(arg=2) + with self.assertWarns(DeprecationWarning): + stack.callback(callback=_exit, arg=3) + self.assertEqual(result, [((), {'arg': 3})]) + def test_push(self): exc_raised = ZeroDivisionError def _expect_exc(exc_type, exc, exc_tb): diff --git a/Lib/test/test_contextlib_async.py b/Lib/test/test_contextlib_async.py index 57716aea9059..492b226a0d54 100644 --- a/Lib/test/test_contextlib_async.py +++ b/Lib/test/test_contextlib_async.py @@ -352,6 +352,16 @@ def setUp(self): self.assertEqual(result, expected) + result = [] + async with AsyncExitStack() as stack: + with self.assertRaises(TypeError): + stack.push_async_callback(arg=1) + with self.assertRaises(TypeError): + self.exit_stack.push_async_callback(arg=2) + with self.assertWarns(DeprecationWarning): + stack.push_async_callback(callback=_exit, arg=3) + self.assertEqual(result, [((), {'arg': 3})]) + @_async_test async def test_async_push(self): exc_raised = ZeroDivisionError diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index 63a9ade54806..4b2b9ab61fa7 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -464,6 +464,7 @@ class A(object): positional = functools.partialmethod(capture, 1) keywords = functools.partialmethod(capture, a=2) both = functools.partialmethod(capture, 3, b=4) + spec_keywords = functools.partialmethod(capture, self=1, func=2) nested = functools.partialmethod(positional, 5) @@ -497,6 +498,8 @@ def test_arg_combinations(self): self.assertEqual(self.A.both(self.a, 5, c=6), ((self.a, 3, 5), {'b': 4, 'c': 6})) + self.assertEqual(self.a.spec_keywords(), ((self.a,), {'self': 1, 'func': 2})) + def test_nested(self): self.assertEqual(self.a.nested(), ((self.a, 1, 5), {})) self.assertEqual(self.a.nested(6), ((self.a, 1, 5, 6), {})) @@ -550,6 +553,14 @@ def test_invalid_args(self): with self.assertRaises(TypeError): class B(object): method = functools.partialmethod(None, 1) + with self.assertRaises(TypeError): + class B: + method = functools.partialmethod() + with self.assertWarns(DeprecationWarning): + class B: + method = functools.partialmethod(func=capture, a=1) + b = B() + self.assertEqual(b.method(2, x=3), ((b, 2), {'a': 1, 'x': 3})) def test_repr(self): self.assertEqual(repr(vars(self.A)['both']), diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py index 5c333b7a0a5e..afe790267661 100644 --- a/Lib/test/test_trace.py +++ b/Lib/test/test_trace.py @@ -70,6 +70,9 @@ def traced_func_calling_generator(): def traced_doubler(num): return num * 2 +def traced_capturer(*args, **kwargs): + return args, kwargs + def traced_caller_list_comprehension(): k = 10 mylist = [traced_doubler(i) for i in range(k)] @@ -270,6 +273,15 @@ def test_simple_caller(self): } self.assertEqual(self.tracer.results().calledfuncs, expected) + def test_arg_errors(self): + res = self.tracer.runfunc(traced_capturer, 1, 2, self=3, func=4) + self.assertEqual(res, ((1, 2), {'self': 3, 'func': 4})) + with self.assertWarns(DeprecationWarning): + res = self.tracer.runfunc(func=traced_capturer, arg=1) + self.assertEqual(res, ((), {'arg': 1})) + with self.assertRaises(TypeError): + self.tracer.runfunc() + def test_loop_caller_importing(self): self.tracer.runfunc(traced_func_importing_caller, 1) diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py index 1fac08dafc7d..50a46f817f9f 100644 --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -1839,6 +1839,35 @@ def add(x,y,z): self.assertEqual(f.alive, False) self.assertEqual(res, [199]) + def test_arg_errors(self): + def fin(*args, **kwargs): + res.append((args, kwargs)) + + a = self.A() + + res = [] + f = weakref.finalize(a, fin, 1, 2, func=3, obj=4) + self.assertEqual(f.peek(), (a, fin, (1, 2), {'func': 3, 'obj': 4})) + f() + self.assertEqual(res, [((1, 2), {'func': 3, 'obj': 4})]) + + res = [] + with self.assertWarns(DeprecationWarning): + f = weakref.finalize(a, func=fin, arg=1) + self.assertEqual(f.peek(), (a, fin, (), {'arg': 1})) + f() + self.assertEqual(res, [((), {'arg': 1})]) + + res = [] + with self.assertWarns(DeprecationWarning): + f = weakref.finalize(obj=a, func=fin, arg=1) + self.assertEqual(f.peek(), (a, fin, (), {'arg': 1})) + f() + self.assertEqual(res, [((), {'arg': 1})]) + + self.assertRaises(TypeError, weakref.finalize, a) + self.assertRaises(TypeError, weakref.finalize) + def test_order(self): a = self.A() res = [] diff --git a/Lib/trace.py b/Lib/trace.py index 3049e4ec6839..fd40fbae8505 100755 --- a/Lib/trace.py +++ b/Lib/trace.py @@ -451,7 +451,22 @@ def runctx(self, cmd, globals=None, locals=None): sys.settrace(None) threading.settrace(None) - def runfunc(self, func, *args, **kw): + def runfunc(*args, **kw): + if len(args) >= 2: + self, func, *args = args + elif not args: + raise TypeError("descriptor 'runfunc' of 'Trace' object " + "needs an argument") + elif 'func' in kw: + func = kw.pop('func') + self, *args = args + import warnings + warnings.warn("Passing 'func' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('runfunc expected at least 1 positional argument, ' + 'got %d' % (len(args)-1)) + result = None if not self.donothing: sys.settrace(self.globaltrace) diff --git a/Lib/unittest/case.py b/Lib/unittest/case.py index a157ae8a14bc..972a4658b17b 100644 --- a/Lib/unittest/case.py +++ b/Lib/unittest/case.py @@ -86,9 +86,21 @@ def _id(obj): _module_cleanups = [] -def addModuleCleanup(function, *args, **kwargs): +def addModuleCleanup(*args, **kwargs): """Same as addCleanup, except the cleanup items are called even if setUpModule fails (unlike tearDownModule).""" + if args: + function, *args = args + elif 'function' in kwargs: + function = kwargs.pop('function') + import warnings + warnings.warn("Passing 'function' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('addModuleCleanup expected at least 1 positional ' + 'argument, got %d' % (len(args)-1)) + args = tuple(args) + _module_cleanups.append((function, args, kwargs)) @@ -463,18 +475,44 @@ def addTypeEqualityFunc(self, typeobj, function): """ self._type_equality_funcs[typeobj] = function - def addCleanup(self, function, *args, **kwargs): + def addCleanup(*args, **kwargs): """Add a function, with arguments, to be called when the test is completed. Functions added are called on a LIFO basis and are called after tearDown on test failure or success. Cleanup items are called even if setUp fails (unlike tearDown).""" + if len(args) >= 2: + self, function, *args = args + elif not args: + raise TypeError("descriptor 'addCleanup' of 'TestCase' object " + "needs an argument") + elif 'function' in kwargs: + function = kwargs.pop('function') + self, *args = args + import warnings + warnings.warn("Passing 'function' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + raise TypeError('addCleanup expected at least 1 positional ' + 'argument, got %d' % (len(args)-1)) + args = tuple(args) + self._cleanups.append((function, args, kwargs)) @classmethod - def addClassCleanup(cls, function, *args, **kwargs): + def addClassCleanup(*args, **kwargs): """Same as addCleanup, except the cleanup items are called even if setUpClass fails (unlike tearDownClass).""" + if len(args) >= 2: + cls, function, *args = args + elif not args: + raise TypeError("descriptor 'addClassCleanup' of 'TestCase' object " + "needs an argument") + else: + raise TypeError('addClassCleanup expected at least 1 positional ' + 'argument, got %d' % (len(args)-1)) + args = tuple(args) + cls._class_cleanups.append((function, args, kwargs)) def setUp(self): diff --git a/Lib/unittest/test/test_runner.py b/Lib/unittest/test/test_runner.py index 2b475c2d8566..443b689dbea0 100644 --- a/Lib/unittest/test/test_runner.py +++ b/Lib/unittest/test/test_runner.py @@ -403,6 +403,22 @@ class Module(object): self.assertEqual(str(e.exception), 'CleanUpExc') self.assertEqual(unittest.case._module_cleanups, []) + def test_addModuleCleanup_arg_errors(self): + cleanups = [] + def cleanup(*args, **kwargs): + cleanups.append((args, kwargs)) + + class Module(object): + unittest.addModuleCleanup(cleanup, 1, 2, function='hello') + with self.assertWarns(DeprecationWarning): + unittest.addModuleCleanup(function=cleanup, arg='hello') + with self.assertRaises(TypeError): + unittest.addModuleCleanup() + unittest.case.doModuleCleanups() + self.assertEqual(cleanups, + [((), {'arg': 'hello'}), + ((1, 2), {'function': 'hello'})]) + def test_run_module_cleanUp(self): blowUp = True ordering = [] @@ -547,6 +563,50 @@ def tearDownClass(cls): 'tearDownModule', 'cleanup_good']) self.assertEqual(unittest.case._module_cleanups, []) + def test_addClassCleanup_arg_errors(self): + cleanups = [] + def cleanup(*args, **kwargs): + cleanups.append((args, kwargs)) + + class TestableTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.addClassCleanup(cleanup, 1, 2, function=3, cls=4) + with self.assertRaises(TypeError): + cls.addClassCleanup(function=cleanup, arg='hello') + def testNothing(self): + pass + + with self.assertRaises(TypeError): + TestableTest.addClassCleanup() + with self.assertRaises(TypeError): + unittest.TestCase.addCleanup(cls=TestableTest(), function=cleanup) + runTests(TestableTest) + self.assertEqual(cleanups, + [((1, 2), {'function': 3, 'cls': 4})]) + + def test_addCleanup_arg_errors(self): + cleanups = [] + def cleanup(*args, **kwargs): + cleanups.append((args, kwargs)) + + class TestableTest(unittest.TestCase): + def setUp(self2): + self2.addCleanup(cleanup, 1, 2, function=3, self=4) + with self.assertWarns(DeprecationWarning): + self2.addCleanup(function=cleanup, arg='hello') + def testNothing(self): + pass + + with self.assertRaises(TypeError): + TestableTest().addCleanup() + with self.assertRaises(TypeError): + unittest.TestCase.addCleanup(self=TestableTest(), function=cleanup) + runTests(TestableTest) + self.assertEqual(cleanups, + [((), {'arg': 'hello'}), + ((1, 2), {'function': 3, 'self': 4})]) + def test_with_errors_in_addClassCleanup(self): ordering = [] diff --git a/Lib/weakref.py b/Lib/weakref.py index 753f07291e20..285c70792e0b 100644 --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -527,7 +527,33 @@ class finalize: class _Info: __slots__ = ("weakref", "func", "args", "kwargs", "atexit", "index") - def __init__(self, obj, func, *args, **kwargs): + def __init__(*args, **kwargs): + if len(args) >= 3: + self, obj, func, *args = args + elif not args: + raise TypeError("descriptor '__init__' of 'finalize' object " + "needs an argument") + else: + if 'func' not in kwargs: + raise TypeError('finalize expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) + func = kwargs.pop('func') + if len(args) >= 2: + self, obj, *args = args + import warnings + warnings.warn("Passing 'func' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + if 'obj' not in kwargs: + raise TypeError('finalize expected at least 2 positional ' + 'arguments, got %d' % (len(args)-1)) + obj = kwargs.pop('obj') + self, *args = args + import warnings + warnings.warn("Passing 'obj' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + args = tuple(args) + if not self._registered_with_atexit: # We may register the exit function more than once because # of a thread race, but that is harmless diff --git a/Misc/NEWS.d/next/Library/2019-03-31-10-21-54.bpo-36492.f7vyUs.rst b/Misc/NEWS.d/next/Library/2019-03-31-10-21-54.bpo-36492.f7vyUs.rst new file mode 100644 index 000000000000..f294bd27b25d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-03-31-10-21-54.bpo-36492.f7vyUs.rst @@ -0,0 +1,5 @@ +Deprecated passing required arguments like *func* as keyword arguments +in functions which should accept arbitrary keyword arguments and pass them +to other function. Arbitrary keyword arguments (even with names "self" and +"func") can now be passed to these functions if the required arguments are +passed as positional arguments. [View Less]

1 0