Python-checkins
Threads by month
- ----- 2024 -----
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2004 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2003 -----
- December
- November
- October
- September
- August
June 2019
- 2 participants
- 665 discussions
Improve version added references in `typing` module docs (GH-13457)
by Miss Islington (bot) 01 Jun '19
by Miss Islington (bot) 01 Jun '19
01 Jun '19
https://github.com/python/cpython/commit/c76add7afd68387aa2481d672e1c0d7e7b…
commit: c76add7afd68387aa2481d672e1c0d7e7b4c9afc
branch: 3.7
author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com>
committer: GitHub <noreply(a)github.com>
date: 2019-06-01T17:23:40-07:00
summary:
Improve version added references in `typing` module docs (GH-13457)
(cherry picked from commit b7daabd711274a009e70556020efeae502a85f0b)
Co-authored-by: Anthony Sottile <asottile(a)umich.edu>
files:
A Misc/NEWS.d/next/Documentation/2019-05-20-22-21-17.bpo-36984.IjZlmS.rst
M Doc/library/typing.rst
diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
index 21258a58d0aa..12f4c03f4232 100644
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -555,7 +555,7 @@ The module defines the following classes, functions and decorators:
A generic version of :class:`collections.abc.Collection`
- .. versionadded:: 3.6
+ .. versionadded:: 3.6.0
.. class:: AbstractSet(Sized, Collection[T_co])
@@ -599,6 +599,7 @@ The module defines the following classes, functions and decorators:
A generic version of :class:`collections.deque`.
+ .. versionadded:: 3.5.4
.. versionadded:: 3.6.1
.. class:: List(list, MutableSequence[T])
@@ -648,6 +649,8 @@ The module defines the following classes, functions and decorators:
A generic version of :class:`collections.abc.Awaitable`.
+ .. versionadded:: 3.5.2
+
.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co])
A generic version of :class:`collections.abc.Coroutine`.
@@ -661,25 +664,33 @@ The module defines the following classes, functions and decorators:
async def bar() -> None:
x = await c # type: int
+ .. versionadded:: 3.5.3
+
.. class:: AsyncIterable(Generic[T_co])
A generic version of :class:`collections.abc.AsyncIterable`.
+ .. versionadded:: 3.5.2
+
.. class:: AsyncIterator(AsyncIterable[T_co])
A generic version of :class:`collections.abc.AsyncIterator`.
+ .. versionadded:: 3.5.2
+
.. class:: ContextManager(Generic[T_co])
A generic version of :class:`contextlib.AbstractContextManager`.
- .. versionadded:: 3.6
+ .. versionadded:: 3.5.4
+ .. versionadded:: 3.6.0
.. class:: AsyncContextManager(Generic[T_co])
A generic version of :class:`contextlib.AbstractAsyncContextManager`.
- .. versionadded:: 3.6
+ .. versionadded:: 3.5.4
+ .. versionadded:: 3.6.2
.. class:: Dict(dict, MutableMapping[KT, VT])
@@ -708,12 +719,14 @@ The module defines the following classes, functions and decorators:
A generic version of :class:`collections.Counter`.
+ .. versionadded:: 3.5.4
.. versionadded:: 3.6.1
.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT])
A generic version of :class:`collections.ChainMap`.
+ .. versionadded:: 3.5.4
.. versionadded:: 3.6.1
.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co])
@@ -778,7 +791,7 @@ The module defines the following classes, functions and decorators:
yield start
start = await increment(start)
- .. versionadded:: 3.5.4
+ .. versionadded:: 3.6.1
.. class:: Text
@@ -977,6 +990,7 @@ The module defines the following classes, functions and decorators:
raise RuntimeError('no way')
.. versionadded:: 3.5.4
+ .. versionadded:: 3.6.2
.. data:: Union
diff --git a/Misc/NEWS.d/next/Documentation/2019-05-20-22-21-17.bpo-36984.IjZlmS.rst b/Misc/NEWS.d/next/Documentation/2019-05-20-22-21-17.bpo-36984.IjZlmS.rst
new file mode 100644
index 000000000000..b26eeadb924a
--- /dev/null
+++ b/Misc/NEWS.d/next/Documentation/2019-05-20-22-21-17.bpo-36984.IjZlmS.rst
@@ -0,0 +1 @@
+Improve version added references in ``typing`` module - by Anthony Sottile.
1
0
01 Jun '19
https://github.com/python/cpython/commit/b7daabd711274a009e70556020efeae502…
commit: b7daabd711274a009e70556020efeae502a85f0b
branch: master
author: Anthony Sottile <asottile(a)umich.edu>
committer: Ivan Levkivskyi <levkivskyi(a)gmail.com>
date: 2019-06-02T01:13:25+01:00
summary:
Improve version added references in `typing` module docs (GH-13457)
files:
A Misc/NEWS.d/next/Documentation/2019-05-20-22-21-17.bpo-36984.IjZlmS.rst
M Doc/library/typing.rst
diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
index 2575a995817d..1a766c29a57a 100644
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -637,7 +637,7 @@ The module defines the following classes, functions and decorators:
A generic version of :class:`collections.abc.Collection`
- .. versionadded:: 3.6
+ .. versionadded:: 3.6.0
.. class:: AbstractSet(Sized, Collection[T_co])
@@ -681,6 +681,7 @@ The module defines the following classes, functions and decorators:
A generic version of :class:`collections.deque`.
+ .. versionadded:: 3.5.4
.. versionadded:: 3.6.1
.. class:: List(list, MutableSequence[T])
@@ -730,6 +731,8 @@ The module defines the following classes, functions and decorators:
A generic version of :class:`collections.abc.Awaitable`.
+ .. versionadded:: 3.5.2
+
.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co])
A generic version of :class:`collections.abc.Coroutine`.
@@ -743,25 +746,33 @@ The module defines the following classes, functions and decorators:
async def bar() -> None:
x = await c # type: int
+ .. versionadded:: 3.5.3
+
.. class:: AsyncIterable(Generic[T_co])
A generic version of :class:`collections.abc.AsyncIterable`.
+ .. versionadded:: 3.5.2
+
.. class:: AsyncIterator(AsyncIterable[T_co])
A generic version of :class:`collections.abc.AsyncIterator`.
+ .. versionadded:: 3.5.2
+
.. class:: ContextManager(Generic[T_co])
A generic version of :class:`contextlib.AbstractContextManager`.
- .. versionadded:: 3.6
+ .. versionadded:: 3.5.4
+ .. versionadded:: 3.6.0
.. class:: AsyncContextManager(Generic[T_co])
A generic version of :class:`contextlib.AbstractAsyncContextManager`.
- .. versionadded:: 3.6
+ .. versionadded:: 3.5.4
+ .. versionadded:: 3.6.2
.. class:: Dict(dict, MutableMapping[KT, VT])
@@ -790,12 +801,14 @@ The module defines the following classes, functions and decorators:
A generic version of :class:`collections.Counter`.
+ .. versionadded:: 3.5.4
.. versionadded:: 3.6.1
.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT])
A generic version of :class:`collections.ChainMap`.
+ .. versionadded:: 3.5.4
.. versionadded:: 3.6.1
.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co])
@@ -860,7 +873,7 @@ The module defines the following classes, functions and decorators:
yield start
start = await increment(start)
- .. versionadded:: 3.5.4
+ .. versionadded:: 3.6.1
.. class:: Text
@@ -1166,6 +1179,7 @@ The module defines the following classes, functions and decorators:
raise RuntimeError('no way')
.. versionadded:: 3.5.4
+ .. versionadded:: 3.6.2
.. data:: Union
diff --git a/Misc/NEWS.d/next/Documentation/2019-05-20-22-21-17.bpo-36984.IjZlmS.rst b/Misc/NEWS.d/next/Documentation/2019-05-20-22-21-17.bpo-36984.IjZlmS.rst
new file mode 100644
index 000000000000..b26eeadb924a
--- /dev/null
+++ b/Misc/NEWS.d/next/Documentation/2019-05-20-22-21-17.bpo-36984.IjZlmS.rst
@@ -0,0 +1 @@
+Improve version added references in ``typing`` module - by Anthony Sottile.
1
0
01 Jun '19
https://github.com/python/cpython/commit/ac60d1afd2b04f61fe4c965740fa32809f…
commit: ac60d1afd2b04f61fe4c965740fa32809f2b84ed
branch: 3.7
author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com>
committer: GitHub <noreply(a)github.com>
date: 2019-06-01T15:26:00-07:00
summary:
bpo-32411: IDLE: Remove line number sort in browser.py (GH-5011)
Insertion in line order makes sorting keys by line order unneeded.
(cherry picked from commit 1a4d9ffa1aecd7e750195f2be06d3d16c7a3a88f)
Co-authored-by: Cheryl Sabella <cheryl.sabella(a)gmail.com>
files:
A Misc/NEWS.d/next/IDLE/2017-12-25-18-48-50.bpo-32411.vNwDhe.rst
M Lib/idlelib/NEWS.txt
M Lib/idlelib/browser.py
M Lib/idlelib/idle_test/test_browser.py
diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt
index 7b5a13a21524..808c236bb590 100644
--- a/Lib/idlelib/NEWS.txt
+++ b/Lib/idlelib/NEWS.txt
@@ -3,6 +3,8 @@ Released on 2019-06-24?
======================================
+bpo-32411: Stop sorting dict created with desired line order.
+
bpo-37038: Make idlelib.run runnable; add test clause.
bpo-36958: Print any argument other than None or int passed to
diff --git a/Lib/idlelib/browser.py b/Lib/idlelib/browser.py
index 234883fe8605..e5b0bc53c662 100644
--- a/Lib/idlelib/browser.py
+++ b/Lib/idlelib/browser.py
@@ -29,9 +29,10 @@ def transform_children(child_dict, modname=None):
The dictionary maps names to pyclbr information objects.
Filter out imported objects.
Augment class names with bases.
- Sort objects by line number.
+ The insertion order of the dictonary is assumed to have been in line
+ number order, so sorting is not necessary.
- The current tree only calls this once per child_dic as it saves
+ The current tree only calls this once per child_dict as it saves
TreeItems once created. A future tree and tests might violate this,
so a check prevents multiple in-place augmentations.
"""
@@ -51,7 +52,7 @@ def transform_children(child_dict, modname=None):
supers.append(sname)
obj.name += '({})'.format(', '.join(supers))
obs.append(obj)
- return sorted(obs, key=lambda o: o.lineno)
+ return obs
class ModuleBrowser:
diff --git a/Lib/idlelib/idle_test/test_browser.py b/Lib/idlelib/idle_test/test_browser.py
index dfbab6dd6b5e..25d6dc663036 100644
--- a/Lib/idlelib/idle_test/test_browser.py
+++ b/Lib/idlelib/idle_test/test_browser.py
@@ -61,16 +61,16 @@ def test_close(self):
# Nested tree same as in test_pyclbr.py except for supers on C0. C1.
mb = pyclbr
module, fname = 'test', 'test.py'
-f0 = mb.Function(module, 'f0', fname, 1)
-f1 = mb._nest_function(f0, 'f1', 2)
-f2 = mb._nest_function(f1, 'f2', 3)
-c1 = mb._nest_class(f0, 'c1', 5)
-C0 = mb.Class(module, 'C0', ['base'], fname, 6)
-F1 = mb._nest_function(C0, 'F1', 8)
-C1 = mb._nest_class(C0, 'C1', 11, [''])
-C2 = mb._nest_class(C1, 'C2', 12)
-F3 = mb._nest_function(C2, 'F3', 14)
-mock_pyclbr_tree = {'f0': f0, 'C0': C0}
+C0 = mb.Class(module, 'C0', ['base'], fname, 1)
+F1 = mb._nest_function(C0, 'F1', 3)
+C1 = mb._nest_class(C0, 'C1', 6, [''])
+C2 = mb._nest_class(C1, 'C2', 7)
+F3 = mb._nest_function(C2, 'F3', 9)
+f0 = mb.Function(module, 'f0', fname, 11)
+f1 = mb._nest_function(f0, 'f1', 12)
+f2 = mb._nest_function(f1, 'f2', 13)
+c1 = mb._nest_class(f0, 'c1', 15)
+mock_pyclbr_tree = {'C0': C0, 'f0': f0}
# Adjust C0.name, C1.name so tests do not depend on order.
browser.transform_children(mock_pyclbr_tree, 'test') # C0(base)
@@ -87,12 +87,12 @@ def test_transform_module_children(self):
transform = browser.transform_children
# Parameter matches tree module.
tcl = list(transform(mock_pyclbr_tree, 'test'))
- eq(tcl, [f0, C0])
- eq(tcl[0].name, 'f0')
- eq(tcl[1].name, 'C0(base)')
+ eq(tcl, [C0, f0])
+ eq(tcl[0].name, 'C0(base)')
+ eq(tcl[1].name, 'f0')
# Check that second call does not change suffix.
tcl = list(transform(mock_pyclbr_tree, 'test'))
- eq(tcl[1].name, 'C0(base)')
+ eq(tcl[0].name, 'C0(base)')
# Nothing to traverse if parameter name isn't same as tree module.
tcl = list(transform(mock_pyclbr_tree, 'different name'))
eq(tcl, [])
diff --git a/Misc/NEWS.d/next/IDLE/2017-12-25-18-48-50.bpo-32411.vNwDhe.rst b/Misc/NEWS.d/next/IDLE/2017-12-25-18-48-50.bpo-32411.vNwDhe.rst
new file mode 100644
index 000000000000..a5522012923a
--- /dev/null
+++ b/Misc/NEWS.d/next/IDLE/2017-12-25-18-48-50.bpo-32411.vNwDhe.rst
@@ -0,0 +1,2 @@
+In browser.py, remove extraneous sorting by line number since dictionary was
+created in line number order.
1
0
01 Jun '19
https://github.com/python/cpython/commit/b7fade4f87e0d37d1686a4e8596141e55e…
commit: b7fade4f87e0d37d1686a4e8596141e55ecef099
branch: master
author: Raymond Hettinger <rhettinger(a)users.noreply.github.com>
committer: GitHub <noreply(a)github.com>
date: 2019-06-01T15:01:46-07:00
summary:
Put math.comb() docs is correct place alphabetically (GH-13734)
files:
M Doc/library/math.rst
diff --git a/Doc/library/math.rst b/Doc/library/math.rst
index 206b06edd2a2..8c6837050319 100644
--- a/Doc/library/math.rst
+++ b/Doc/library/math.rst
@@ -36,6 +36,21 @@ Number-theoretic and representation functions
:class:`~numbers.Integral` value.
+.. function:: comb(n, k)
+
+ Return the number of ways to choose *k* items from *n* items without repetition
+ and without order.
+
+ Also called the binomial coefficient. It is mathematically equal to the expression
+ ``n! / (k! (n - k)!)``. It is equivalent to the coefficient of the *k*-th term in the
+ polynomial expansion of the expression ``(1 + x) ** n``.
+
+ Raises :exc:`TypeError` if the arguments not integers.
+ Raises :exc:`ValueError` if the arguments are negative or if *k* > *n*.
+
+ .. versionadded:: 3.8
+
+
.. function:: copysign(x, y)
Return a float with the magnitude (absolute value) of *x* but the sign of
@@ -232,21 +247,6 @@ Number-theoretic and representation functions
:meth:`x.__trunc__() <object.__trunc__>`.
-.. function:: comb(n, k)
-
- Return the number of ways to choose *k* items from *n* items without repetition
- and without order.
-
- Also called the binomial coefficient. It is mathematically equal to the expression
- ``n! / (k! (n - k)!)``. It is equivalent to the coefficient of the *k*-th term in the
- polynomial expansion of the expression ``(1 + x) ** n``.
-
- Raises :exc:`TypeError` if the arguments not integers.
- Raises :exc:`ValueError` if the arguments are negative or if *k* > *n*.
-
- .. versionadded:: 3.8
-
-
Note that :func:`frexp` and :func:`modf` have a different call/return pattern
than their C equivalents: they take a single argument and return a pair of
values, rather than returning their second return value through an 'output
1
0
01 Jun '19
https://github.com/python/cpython/commit/fefdc009906c5ea8fb57383817b3c42d3b…
commit: fefdc009906c5ea8fb57383817b3c42d3b0634ad
branch: 3.7
author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com>
committer: GitHub <noreply(a)github.com>
date: 2019-06-01T14:58:33-07:00
summary:
Fix the error handling in bytesio_sizeof(). (GH-10459)
bytesio_sizeof() must check if an error has occurred in _PySys_GetSizeOf().
(cherry picked from commit 36dcaab7fde5d2e54cdeff5b705b5adcb27726dd)
Co-authored-by: Zackery Spytz <zspytz(a)gmail.com>
files:
M Modules/_io/bytesio.c
diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c
index e4d637cc3aea..8501f42ed800 100644
--- a/Modules/_io/bytesio.c
+++ b/Modules/_io/bytesio.c
@@ -942,8 +942,13 @@ bytesio_sizeof(bytesio *self, void *unused)
Py_ssize_t res;
res = _PyObject_SIZE(Py_TYPE(self));
- if (self->buf && !SHARED_BUF(self))
- res += _PySys_GetSizeOf(self->buf);
+ if (self->buf && !SHARED_BUF(self)) {
+ Py_ssize_t s = _PySys_GetSizeOf(self->buf);
+ if (s == -1) {
+ return NULL;
+ }
+ res += s;
+ }
return PyLong_FromSsize_t(res);
}
1
0
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714)
by Eric Snow 01 Jun '19
by Eric Snow 01 Jun '19
01 Jun '19
https://github.com/python/cpython/commit/6a150bcaeb190d1731b38ab9c7a5d1a352…
commit: 6a150bcaeb190d1731b38ab9c7a5d1a352847ddc
branch: master
author: Eric Snow <ericsnowcurrently(a)gmail.com>
committer: GitHub <noreply(a)github.com>
date: 2019-06-01T15:39:46-06:00
summary:
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714)
files:
A Misc/NEWS.d/next/Core and Builtins/2018-09-15-12-13-46.bpo-33608.avmvVP.rst
M Include/internal/pycore_ceval.h
M Include/internal/pycore_pystate.h
M Lib/test/test_capi.py
M Modules/_testcapimodule.c
M Modules/signalmodule.c
M Python/ceval.c
M Python/ceval_gil.h
M Python/pylifecycle.c
M Python/pystate.c
diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h
index 37170ed438f8..d44afdf4fa46 100644
--- a/Include/internal/pycore_ceval.h
+++ b/Include/internal/pycore_ceval.h
@@ -12,19 +12,22 @@ extern "C" {
#include "pycore_pystate.h"
#include "pythread.h"
-PyAPI_FUNC(void) _Py_FinishPendingCalls(_PyRuntimeState *runtime);
PyAPI_FUNC(void) _PyEval_Initialize(struct _ceval_runtime_state *);
PyAPI_FUNC(void) _PyEval_FiniThreads(
- struct _ceval_runtime_state *ceval);
+ struct _ceval_runtime_state *);
PyAPI_FUNC(void) _PyEval_SignalReceived(
- struct _ceval_runtime_state *ceval);
+ struct _ceval_runtime_state *);
PyAPI_FUNC(int) _PyEval_AddPendingCall(
PyThreadState *tstate,
- struct _ceval_runtime_state *ceval,
+ struct _ceval_runtime_state *,
+ struct _ceval_interpreter_state *,
+ unsigned long thread_id,
int (*func)(void *),
void *arg);
+PyAPI_FUNC(void) _PyEval_FinishPendingCalls(PyInterpreterState *);
PyAPI_FUNC(void) _PyEval_SignalAsyncExc(
- struct _ceval_runtime_state *ceval);
+ struct _ceval_runtime_state *,
+ struct _ceval_interpreter_state *);
PyAPI_FUNC(void) _PyEval_ReInitThreads(
_PyRuntimeState *runtime);
diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h
index 520a74b8a61f..aca5533022e3 100644
--- a/Include/internal/pycore_pystate.h
+++ b/Include/internal/pycore_pystate.h
@@ -25,7 +25,7 @@ struct pyruntimestate;
/* ceval state */
-struct _pending_calls {
+struct _ceval_pending_calls {
int finishing;
PyThread_type_lock lock;
/* Request for running pending calls. */
@@ -36,6 +36,7 @@ struct _pending_calls {
int async_exc;
#define NPENDINGCALLS 32
struct {
+ unsigned long thread_id;
int (*func)(void *);
void *arg;
} calls[NPENDINGCALLS];
@@ -53,15 +54,21 @@ struct _ceval_runtime_state {
int tracing_possible;
/* This single variable consolidates all requests to break out of
the fast path in the eval loop. */
+ // XXX This can move to _ceval_interpreter_state once all parts
+ // from COMPUTE_EVAL_BREAKER have moved under PyInterpreterState.
_Py_atomic_int eval_breaker;
/* Request for dropping the GIL */
_Py_atomic_int gil_drop_request;
- struct _pending_calls pending;
/* Request for checking signals. */
_Py_atomic_int signals_pending;
struct _gil_runtime_state gil;
};
+struct _ceval_interpreter_state {
+ struct _ceval_pending_calls pending;
+};
+
+
/* interpreter state */
typedef PyObject* (*_PyFrameEvalFunction)(struct _frame *, int);
@@ -136,6 +143,7 @@ struct _is {
uint64_t tstate_next_unique_id;
+ struct _ceval_interpreter_state ceval;
struct _warnings_runtime_state warnings;
PyObject *audit_hooks;
diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py
index 4dd78bb9a2fd..fabc821e5c3a 100644
--- a/Lib/test/test_capi.py
+++ b/Lib/test/test_capi.py
@@ -431,7 +431,7 @@ def pendingcalls_wait(self, l, n, context = None):
def test_pendingcalls_threaded(self):
#do every callback on a separate thread
- n = 32 #total callbacks
+ n = 32 #total callbacks (see NPENDINGCALLS in pycore_ceval.h)
threads = []
class foo(object):pass
context = foo()
diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-09-15-12-13-46.bpo-33608.avmvVP.rst b/Misc/NEWS.d/next/Core and Builtins/2018-09-15-12-13-46.bpo-33608.avmvVP.rst
new file mode 100644
index 000000000000..73a01a1f46bd
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2018-09-15-12-13-46.bpo-33608.avmvVP.rst
@@ -0,0 +1,5 @@
+We added a new internal _Py_AddPendingCall() that operates relative to the
+provided interpreter. This allows us to use the existing implementation to
+ask another interpreter to do work that cannot be done in the current
+interpreter, like decref an object the other interpreter owns. The existing
+Py_AddPendingCall() only operates relative to the main interpreter.
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
index b42f41cc8d8f..bf20e81a4ce8 100644
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -2677,6 +2677,7 @@ pending_threadfunc(PyObject *self, PyObject *arg)
Py_INCREF(callable);
Py_BEGIN_ALLOW_THREADS
+ /* XXX Use the internal _Py_AddPendingCall(). */
r = Py_AddPendingCall(&_pending_callback, callable);
Py_END_ALLOW_THREADS
diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
index 7698984ff3af..1964646da252 100644
--- a/Modules/signalmodule.c
+++ b/Modules/signalmodule.c
@@ -21,6 +21,7 @@
#include <process.h>
#endif
#endif
+#include "internal/pycore_pystate.h"
#ifdef HAVE_SIGNAL_H
#include <signal.h>
@@ -259,6 +260,7 @@ trip_signal(int sig_num)
/* Notify ceval.c */
_PyRuntimeState *runtime = &_PyRuntime;
PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
+ PyInterpreterState *interp = runtime->interpreters.main;
_PyEval_SignalReceived(&runtime->ceval);
/* And then write to the wakeup fd *after* setting all the globals and
@@ -299,7 +301,10 @@ trip_signal(int sig_num)
{
/* Py_AddPendingCall() isn't signal-safe, but we
still use it for this exceptional case. */
- _PyEval_AddPendingCall(tstate, &runtime->ceval,
+ _PyEval_AddPendingCall(tstate,
+ &runtime->ceval,
+ &interp->ceval,
+ runtime->main_thread,
report_wakeup_send_error,
(void *)(intptr_t) last_error);
}
@@ -318,7 +323,10 @@ trip_signal(int sig_num)
{
/* Py_AddPendingCall() isn't signal-safe, but we
still use it for this exceptional case. */
- _PyEval_AddPendingCall(tstate, &runtime->ceval,
+ _PyEval_AddPendingCall(tstate,
+ &runtime->ceval,
+ &interp->ceval,
+ runtime->main_thread,
report_wakeup_write_error,
(void *)(intptr_t)errno);
}
diff --git a/Python/ceval.c b/Python/ceval.c
index d9a71e942153..a092a2355641 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -101,66 +101,64 @@ static long dxp[256];
#endif
#endif
-#define GIL_REQUEST _Py_atomic_load_relaxed(&ceval->gil_drop_request)
-
/* This can set eval_breaker to 0 even though gil_drop_request became
1. We believe this is all right because the eval loop will release
the GIL eventually anyway. */
-#define COMPUTE_EVAL_BREAKER(ceval) \
+#define COMPUTE_EVAL_BREAKER(ceval_r, ceval_i) \
_Py_atomic_store_relaxed( \
- &(ceval)->eval_breaker, \
- GIL_REQUEST | \
- _Py_atomic_load_relaxed(&(ceval)->signals_pending) | \
- _Py_atomic_load_relaxed(&(ceval)->pending.calls_to_do) | \
- (ceval)->pending.async_exc)
+ &(ceval_r)->eval_breaker, \
+ _Py_atomic_load_relaxed(&(ceval_r)->gil_drop_request) | \
+ _Py_atomic_load_relaxed(&(ceval_r)->signals_pending) | \
+ _Py_atomic_load_relaxed(&(ceval_i)->pending.calls_to_do) | \
+ (ceval_i)->pending.async_exc)
-#define SET_GIL_DROP_REQUEST(ceval) \
+#define SET_GIL_DROP_REQUEST(ceval_r) \
do { \
- _Py_atomic_store_relaxed(&(ceval)->gil_drop_request, 1); \
- _Py_atomic_store_relaxed(&(ceval)->eval_breaker, 1); \
+ _Py_atomic_store_relaxed(&(ceval_r)->gil_drop_request, 1); \
+ _Py_atomic_store_relaxed(&(ceval_r)->eval_breaker, 1); \
} while (0)
-#define RESET_GIL_DROP_REQUEST(ceval) \
+#define RESET_GIL_DROP_REQUEST(ceval_r, ceval_i) \
do { \
- _Py_atomic_store_relaxed(&(ceval)->gil_drop_request, 0); \
- COMPUTE_EVAL_BREAKER(ceval); \
+ _Py_atomic_store_relaxed(&(ceval_r)->gil_drop_request, 0); \
+ COMPUTE_EVAL_BREAKER(ceval_r, ceval_i); \
} while (0)
/* Pending calls are only modified under pending_lock */
-#define SIGNAL_PENDING_CALLS(ceval) \
+#define SIGNAL_PENDING_CALLS(ceval_r, ceval_i) \
do { \
- _Py_atomic_store_relaxed(&(ceval)->pending.calls_to_do, 1); \
- _Py_atomic_store_relaxed(&(ceval)->eval_breaker, 1); \
+ _Py_atomic_store_relaxed(&(ceval_i)->pending.calls_to_do, 1); \
+ _Py_atomic_store_relaxed(&(ceval_r)->eval_breaker, 1); \
} while (0)
-#define UNSIGNAL_PENDING_CALLS(ceval) \
+#define UNSIGNAL_PENDING_CALLS(ceval_r, ceval_i) \
do { \
- _Py_atomic_store_relaxed(&(ceval)->pending.calls_to_do, 0); \
- COMPUTE_EVAL_BREAKER(ceval); \
+ _Py_atomic_store_relaxed(&(ceval_i)->pending.calls_to_do, 0); \
+ COMPUTE_EVAL_BREAKER(ceval_r, ceval_i); \
} while (0)
-#define SIGNAL_PENDING_SIGNALS(ceval) \
+#define SIGNAL_PENDING_SIGNALS(ceval_r) \
do { \
- _Py_atomic_store_relaxed(&(ceval)->signals_pending, 1); \
- _Py_atomic_store_relaxed(&(ceval)->eval_breaker, 1); \
+ _Py_atomic_store_relaxed(&(ceval_r)->signals_pending, 1); \
+ _Py_atomic_store_relaxed(&(ceval_r)->eval_breaker, 1); \
} while (0)
-#define UNSIGNAL_PENDING_SIGNALS(ceval) \
+#define UNSIGNAL_PENDING_SIGNALS(ceval_r, ceval_i) \
do { \
- _Py_atomic_store_relaxed(&(ceval)->signals_pending, 0); \
- COMPUTE_EVAL_BREAKER(ceval); \
+ _Py_atomic_store_relaxed(&(ceval_r)->signals_pending, 0); \
+ COMPUTE_EVAL_BREAKER(ceval_r, ceval_i); \
} while (0)
-#define SIGNAL_ASYNC_EXC(ceval) \
+#define SIGNAL_ASYNC_EXC(ceval_r, ceval_i) \
do { \
- (ceval)->pending.async_exc = 1; \
- _Py_atomic_store_relaxed(&(ceval)->eval_breaker, 1); \
+ (ceval_i)->pending.async_exc = 1; \
+ _Py_atomic_store_relaxed(&(ceval_r)->eval_breaker, 1); \
} while (0)
-#define UNSIGNAL_ASYNC_EXC(ceval) \
+#define UNSIGNAL_ASYNC_EXC(ceval_r, ceval_i) \
do { \
- (ceval)->pending.async_exc = 0; \
- COMPUTE_EVAL_BREAKER(ceval); \
+ (ceval_i)->pending.async_exc = 0; \
+ COMPUTE_EVAL_BREAKER(ceval_r, ceval_i); \
} while (0)
@@ -180,8 +178,8 @@ void
PyEval_InitThreads(void)
{
_PyRuntimeState *runtime = &_PyRuntime;
- struct _ceval_runtime_state *ceval = &runtime->ceval;
- struct _gil_runtime_state *gil = &ceval->gil;
+ struct _ceval_runtime_state *ceval_r = &runtime->ceval;
+ struct _gil_runtime_state *gil = &ceval_r->gil;
if (gil_created(gil)) {
return;
}
@@ -189,19 +187,15 @@ PyEval_InitThreads(void)
PyThread_init_thread();
create_gil(gil);
PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
- take_gil(ceval, tstate);
+ take_gil(ceval_r, tstate);
- struct _pending_calls *pending = &ceval->pending;
- pending->lock = PyThread_allocate_lock();
- if (pending->lock == NULL) {
- Py_FatalError("Can't initialize threads for pending calls");
- }
+ // The pending calls mutex is initialized in PyInterpreterState_New().
}
void
-_PyEval_FiniThreads(struct _ceval_runtime_state *ceval)
+_PyEval_FiniThreads(struct _ceval_runtime_state *ceval_r)
{
- struct _gil_runtime_state *gil = &ceval->gil;
+ struct _gil_runtime_state *gil = &ceval_r->gil;
if (!gil_created(gil)) {
return;
}
@@ -209,20 +203,24 @@ _PyEval_FiniThreads(struct _ceval_runtime_state *ceval)
destroy_gil(gil);
assert(!gil_created(gil));
- struct _pending_calls *pending = &ceval->pending;
- if (pending->lock != NULL) {
- PyThread_free_lock(pending->lock);
- pending->lock = NULL;
- }
+ // The pending calls mutex is freed in PyInterpreterState_Delete().
}
static inline void
exit_thread_if_finalizing(PyThreadState *tstate)
{
- _PyRuntimeState *runtime = tstate->interp->runtime;
- /* _Py_Finalizing is protected by the GIL */
+ PyInterpreterState *interp = tstate->interp;
+ // Stop if thread/interpreter inalization already stated.
+ if (interp == NULL) {
+ return;
+ }
+ _PyRuntimeState *runtime = interp->runtime;
+ if (runtime == NULL) {
+ return;
+ }
+ // Don't exit if the main thread (i.e. of the main interpreter).
if (runtime->finalizing != NULL && !_Py_CURRENTLY_FINALIZING(runtime, tstate)) {
- drop_gil(&runtime->ceval, tstate);
+ drop_gil(&runtime->ceval, &interp->ceval, tstate);
PyThread_exit_thread();
}
}
@@ -231,12 +229,12 @@ void
PyEval_AcquireLock(void)
{
_PyRuntimeState *runtime = &_PyRuntime;
- struct _ceval_runtime_state *ceval = &runtime->ceval;
+ struct _ceval_runtime_state *ceval_r = &runtime->ceval;
PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
if (tstate == NULL) {
Py_FatalError("PyEval_AcquireLock: current thread state is NULL");
}
- take_gil(ceval, tstate);
+ take_gil(ceval_r, tstate);
exit_thread_if_finalizing(tstate);
}
@@ -244,12 +242,21 @@ void
PyEval_ReleaseLock(void)
{
_PyRuntimeState *runtime = &_PyRuntime;
- PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
/* This function must succeed when the current thread state is NULL.
We therefore avoid PyThreadState_Get() which dumps a fatal error
in debug mode.
*/
- drop_gil(&runtime->ceval, tstate);
+ PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
+ // Fall back to the main interpreter if there is not active Python
+ // thread. This only affects the eval_breaker.
+ PyInterpreterState *interp = runtime->interpreters.main;
+ if (tstate != NULL) {
+ interp = tstate->interp;
+ if (interp == NULL) {
+ Py_FatalError("PyEval_ReleaseLock: NULL interpreter state");
+ }
+ }
+ drop_gil(&runtime->ceval, &interp->ceval, tstate);
}
void
@@ -258,14 +265,19 @@ PyEval_AcquireThread(PyThreadState *tstate)
if (tstate == NULL) {
Py_FatalError("PyEval_AcquireThread: NULL new thread state");
}
- assert(tstate->interp != NULL);
-
- _PyRuntimeState *runtime = tstate->interp->runtime;
- struct _ceval_runtime_state *ceval = &runtime->ceval;
+ PyInterpreterState *interp = tstate->interp;
+ if (interp == NULL) {
+ Py_FatalError("PyEval_AcquireThread: NULL interpreter state");
+ }
+ _PyRuntimeState *runtime = interp->runtime;
+ if (runtime == NULL) {
+ Py_FatalError("PyEval_AcquireThread: NULL runtime state");
+ }
+ struct _ceval_runtime_state *ceval_r = &runtime->ceval;
/* Check someone has called PyEval_InitThreads() to create the lock */
- assert(gil_created(&ceval->gil));
- take_gil(ceval, tstate);
+ assert(gil_created(&ceval_r->gil));
+ take_gil(ceval_r, tstate);
exit_thread_if_finalizing(tstate);
if (_PyThreadState_Swap(&runtime->gilstate, tstate) != NULL) {
Py_FatalError("PyEval_AcquireThread: non-NULL old thread state");
@@ -278,14 +290,20 @@ PyEval_ReleaseThread(PyThreadState *tstate)
if (tstate == NULL) {
Py_FatalError("PyEval_ReleaseThread: NULL thread state");
}
- assert(tstate->interp != NULL);
+ PyInterpreterState *interp = tstate->interp;
+ if (interp == NULL) {
+ Py_FatalError("PyEval_ReleaseThread: NULL interpreter state");
+ }
+ _PyRuntimeState *runtime = interp->runtime;
+ if (runtime == NULL) {
+ Py_FatalError("PyEval_ReleaseThread: NULL runtime state");
+ }
- _PyRuntimeState *runtime = tstate->interp->runtime;
PyThreadState *new_tstate = _PyThreadState_Swap(&runtime->gilstate, NULL);
if (new_tstate != tstate) {
Py_FatalError("PyEval_ReleaseThread: wrong thread state");
}
- drop_gil(&runtime->ceval, tstate);
+ drop_gil(&runtime->ceval, &interp->ceval, tstate);
}
/* This function is called from PyOS_AfterFork_Child to destroy all threads
@@ -296,15 +314,17 @@ PyEval_ReleaseThread(PyThreadState *tstate)
void
_PyEval_ReInitThreads(_PyRuntimeState *runtime)
{
- struct _ceval_runtime_state *ceval = &runtime->ceval;
- if (!gil_created(&ceval->gil)) {
+ struct _ceval_runtime_state *ceval_r = &runtime->ceval;
+ if (!gil_created(&ceval_r->gil)) {
return;
}
- recreate_gil(&ceval->gil);
+ recreate_gil(&ceval_r->gil);
PyThreadState *current_tstate = _PyRuntimeState_GetThreadState(runtime);
- take_gil(ceval, current_tstate);
+ take_gil(ceval_r, current_tstate);
- struct _pending_calls *pending = &ceval->pending;
+ // Only the main interpreter remains, so ignore the rest.
+ PyInterpreterState *interp = _PyRuntime.interpreters.main;
+ struct _ceval_pending_calls *pending = &interp->ceval.pending;
pending->lock = PyThread_allocate_lock();
if (pending->lock == NULL) {
Py_FatalError("Can't initialize threads for pending calls");
@@ -318,22 +338,28 @@ _PyEval_ReInitThreads(_PyRuntimeState *runtime)
raised. */
void
-_PyEval_SignalAsyncExc(struct _ceval_runtime_state *ceval)
+_PyEval_SignalAsyncExc(struct _ceval_runtime_state *ceval_r,
+ struct _ceval_interpreter_state *ceval_i)
{
- SIGNAL_ASYNC_EXC(ceval);
+ SIGNAL_ASYNC_EXC(ceval_r, ceval_i);
}
PyThreadState *
PyEval_SaveThread(void)
{
_PyRuntimeState *runtime = &_PyRuntime;
- struct _ceval_runtime_state *ceval = &runtime->ceval;
+ struct _ceval_runtime_state *ceval_r = &runtime->ceval;
PyThreadState *tstate = _PyThreadState_Swap(&runtime->gilstate, NULL);
if (tstate == NULL) {
Py_FatalError("PyEval_SaveThread: NULL tstate");
}
- assert(gil_created(&ceval->gil));
- drop_gil(ceval, tstate);
+ PyInterpreterState *interp = tstate->interp;
+ if (interp == NULL) {
+ Py_FatalError("PyEval_SaveThread: NULL interpreter state");
+ }
+
+ assert(gil_created(&ceval_r->gil));
+ drop_gil(ceval_r, &interp->ceval, tstate);
return tstate;
}
@@ -343,14 +369,20 @@ PyEval_RestoreThread(PyThreadState *tstate)
if (tstate == NULL) {
Py_FatalError("PyEval_RestoreThread: NULL tstate");
}
- assert(tstate->interp != NULL);
+ PyInterpreterState *interp = tstate->interp;
+ if (interp == NULL) {
+ Py_FatalError("PyEval_RestoreThread: NULL interpreter state");
+ }
+ _PyRuntimeState *runtime = interp->runtime;
+ if (runtime == NULL) {
+ Py_FatalError("PyEval_RestoreThread: NULL runtime state");
+ }
+ struct _ceval_runtime_state *ceval_r = &runtime->ceval;
- _PyRuntimeState *runtime = tstate->interp->runtime;
- struct _ceval_runtime_state *ceval = &runtime->ceval;
- assert(gil_created(&ceval->gil));
+ assert(gil_created(&ceval_r->gil));
int err = errno;
- take_gil(ceval, tstate);
+ take_gil(ceval_r, tstate);
exit_thread_if_finalizing(tstate);
errno = err;
@@ -381,17 +413,17 @@ PyEval_RestoreThread(PyThreadState *tstate)
*/
void
-_PyEval_SignalReceived(struct _ceval_runtime_state *ceval)
+_PyEval_SignalReceived(struct _ceval_runtime_state *ceval_r)
{
/* bpo-30703: Function called when the C signal handler of Python gets a
signal. We cannot queue a callback using Py_AddPendingCall() since
that function is not async-signal-safe. */
- SIGNAL_PENDING_SIGNALS(ceval);
+ SIGNAL_PENDING_SIGNALS(ceval_r);
}
/* Push one item onto the queue while holding the lock. */
static int
-_push_pending_call(struct _pending_calls *pending,
+_push_pending_call(struct _ceval_pending_calls *pending, unsigned long thread_id,
int (*func)(void *), void *arg)
{
int i = pending->last;
@@ -399,6 +431,7 @@ _push_pending_call(struct _pending_calls *pending,
if (j == pending->first) {
return -1; /* Queue full */
}
+ pending->calls[i].thread_id = thread_id;
pending->calls[i].func = func;
pending->calls[i].arg = arg;
pending->last = j;
@@ -407,7 +440,7 @@ _push_pending_call(struct _pending_calls *pending,
/* Pop one item off the queue while holding the lock. */
static void
-_pop_pending_call(struct _pending_calls *pending,
+_pop_pending_call(struct _ceval_pending_calls *pending, unsigned long *thread_id,
int (**func)(void *), void **arg)
{
int i = pending->first;
@@ -417,6 +450,7 @@ _pop_pending_call(struct _pending_calls *pending,
*func = pending->calls[i].func;
*arg = pending->calls[i].arg;
+ *thread_id = pending->calls[i].thread_id;
pending->first = (i + 1) % NPENDINGCALLS;
}
@@ -427,10 +461,12 @@ _pop_pending_call(struct _pending_calls *pending,
int
_PyEval_AddPendingCall(PyThreadState *tstate,
- struct _ceval_runtime_state *ceval,
+ struct _ceval_runtime_state *ceval_r,
+ struct _ceval_interpreter_state *ceval_i,
+ unsigned long thread_id,
int (*func)(void *), void *arg)
{
- struct _pending_calls *pending = &ceval->pending;
+ struct _ceval_pending_calls *pending = &ceval_i->pending;
PyThread_acquire_lock(pending->lock, WAIT_LOCK);
if (pending->finishing) {
@@ -445,20 +481,27 @@ _PyEval_AddPendingCall(PyThreadState *tstate,
_PyErr_Restore(tstate, exc, val, tb);
return -1;
}
- int result = _push_pending_call(pending, func, arg);
+ int result = _push_pending_call(pending, thread_id, func, arg);
+
+ /* signal loop */
+ SIGNAL_PENDING_CALLS(ceval_r, ceval_i);
PyThread_release_lock(pending->lock);
- /* signal main loop */
- SIGNAL_PENDING_CALLS(ceval);
return result;
}
+/* Py_AddPendingCall() is a simple wrapper for the sake
+ of backward-compatibility. */
int
Py_AddPendingCall(int (*func)(void *), void *arg)
{
_PyRuntimeState *runtime = &_PyRuntime;
+ PyInterpreterState *interp = runtime->interpreters.main;
PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
- return _PyEval_AddPendingCall(tstate, &runtime->ceval, func, arg);
+ return _PyEval_AddPendingCall(tstate,
+ &runtime->ceval, &interp->ceval,
+ runtime->main_thread,
+ func, arg);
}
static int
@@ -479,47 +522,69 @@ handle_signals(_PyRuntimeState *runtime)
return 0;
}
- struct _ceval_runtime_state *ceval = &runtime->ceval;
- UNSIGNAL_PENDING_SIGNALS(ceval);
+ struct _ceval_runtime_state *ceval_r = &runtime->ceval;
+ struct _ceval_interpreter_state *ceval_i = &interp->ceval;
+ UNSIGNAL_PENDING_SIGNALS(ceval_r, ceval_i);
if (_PyErr_CheckSignals() < 0) {
- SIGNAL_PENDING_SIGNALS(ceval); /* We're not done yet */
+ SIGNAL_PENDING_SIGNALS(ceval_r); /* We're not done yet */
return -1;
}
return 0;
}
static int
-make_pending_calls(_PyRuntimeState *runtime)
+make_pending_calls(PyInterpreterState *interp)
{
- static int busy = 0;
-
- /* only service pending calls on main thread */
- if (PyThread_get_thread_ident() != runtime->main_thread) {
- return 0;
+ if (interp == NULL) {
+ Py_FatalError("make_pending_calls: NULL interpreter state");
+ }
+ _PyRuntimeState *runtime = interp->runtime;
+ if (runtime == NULL) {
+ Py_FatalError("make_pending_calls: NULL runtime state");
+ }
+ PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
+ if (tstate == NULL) {
+ Py_FatalError("make_pending_calls: NULL thread state");
}
+ if (tstate->interp == NULL || tstate->interp != interp) {
+ Py_FatalError("make_pending_calls: thread state mismatch");
+ }
+ static int busy = 0;
/* don't perform recursive pending calls */
if (busy) {
return 0;
}
busy = 1;
- struct _ceval_runtime_state *ceval = &runtime->ceval;
+ struct _ceval_runtime_state *ceval_r = &runtime->ceval;
+ struct _ceval_interpreter_state *ceval_i = &interp->ceval;
/* unsignal before starting to call callbacks, so that any callback
added in-between re-signals */
- UNSIGNAL_PENDING_CALLS(ceval);
+ UNSIGNAL_PENDING_CALLS(ceval_r, ceval_i);
int res = 0;
/* perform a bounded number of calls, in case of recursion */
- struct _pending_calls *pending = &ceval->pending;
+ struct _ceval_pending_calls *pending = &ceval_i->pending;
+ unsigned long thread_id = 0;
for (int i=0; i<NPENDINGCALLS; i++) {
int (*func)(void *) = NULL;
void *arg = NULL;
/* pop one item off the queue while holding the lock */
PyThread_acquire_lock(pending->lock, WAIT_LOCK);
- _pop_pending_call(pending, &func, &arg);
+ _pop_pending_call(pending, &thread_id, &func, &arg);
PyThread_release_lock(pending->lock);
+ if (thread_id && PyThread_get_thread_ident() != thread_id) {
+ // Thread mismatch, so move it to the end of the list
+ // and start over.
+ _PyEval_AddPendingCall(tstate,
+ &runtime->ceval, &interp->ceval,
+ thread_id,
+ func, arg);
+ goto error;
+ }
+
/* having released the lock, perform the callback */
if (func == NULL) {
break;
@@ -535,17 +600,16 @@ make_pending_calls(_PyRuntimeState *runtime)
error:
busy = 0;
- SIGNAL_PENDING_CALLS(ceval);
+ SIGNAL_PENDING_CALLS(ceval_r, ceval_i);
return res;
}
void
-_Py_FinishPendingCalls(_PyRuntimeState *runtime)
+_PyEval_FinishPendingCalls(PyInterpreterState *interp)
{
assert(PyGILState_Check());
- PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
- struct _pending_calls *pending = &runtime->ceval.pending;
+ struct _ceval_pending_calls *pending = &interp->ceval.pending;
PyThread_acquire_lock(pending->lock, WAIT_LOCK);
pending->finishing = 1;
@@ -555,12 +619,19 @@ _Py_FinishPendingCalls(_PyRuntimeState *runtime)
return;
}
- if (make_pending_calls(runtime) < 0) {
- PyObject *exc, *val, *tb;
- _PyErr_Fetch(tstate, &exc, &val, &tb);
- PyErr_BadInternalCall();
- _PyErr_ChainExceptions(exc, val, tb);
- _PyErr_Print(tstate);
+ if (make_pending_calls(interp) < 0) {
+ _PyRuntimeState *runtime = interp->runtime;
+ if (runtime == NULL) {
+ runtime = &_PyRuntime;
+ }
+ PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
+ if (tstate != NULL) {
+ PyObject *exc, *val, *tb;
+ _PyErr_Fetch(tstate, &exc, &val, &tb);
+ PyErr_BadInternalCall();
+ _PyErr_ChainExceptions(exc, val, tb);
+ _PyErr_Print(tstate);
+ }
}
}
@@ -579,7 +650,8 @@ Py_MakePendingCalls(void)
return res;
}
- res = make_pending_calls(runtime);
+ PyInterpreterState *interp = _PyRuntime.interpreters.main;
+ res = make_pending_calls(interp);
if (res != 0) {
return res;
}
@@ -596,11 +668,11 @@ Py_MakePendingCalls(void)
int _Py_CheckRecursionLimit = Py_DEFAULT_RECURSION_LIMIT;
void
-_PyEval_Initialize(struct _ceval_runtime_state *state)
+_PyEval_Initialize(struct _ceval_runtime_state *ceval_r)
{
- state->recursion_limit = Py_DEFAULT_RECURSION_LIMIT;
+ ceval_r->recursion_limit = Py_DEFAULT_RECURSION_LIMIT;
_Py_CheckRecursionLimit = Py_DEFAULT_RECURSION_LIMIT;
- _gil_initialize(&state->gil);
+ _gil_initialize(&ceval_r->gil);
}
int
@@ -612,9 +684,9 @@ Py_GetRecursionLimit(void)
void
Py_SetRecursionLimit(int new_limit)
{
- struct _ceval_runtime_state *ceval = &_PyRuntime.ceval;
- ceval->recursion_limit = new_limit;
- _Py_CheckRecursionLimit = ceval->recursion_limit;
+ struct _ceval_runtime_state *ceval_r = &_PyRuntime.ceval;
+ ceval_r->recursion_limit = new_limit;
+ _Py_CheckRecursionLimit = ceval_r->recursion_limit;
}
/* the macro Py_EnterRecursiveCall() only calls _Py_CheckRecursiveCall()
@@ -663,7 +735,7 @@ _Py_CheckRecursiveCall(const char *where)
static int do_raise(PyThreadState *tstate, PyObject *exc, PyObject *cause);
static int unpack_iterable(PyThreadState *, PyObject *, int, int, PyObject **);
-#define _Py_TracingPossible(ceval) ((ceval)->tracing_possible)
+#define _Py_TracingPossible(ceval_r) ((ceval_r)->tracing_possible)
PyObject *
@@ -709,8 +781,10 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
PyObject *retval = NULL; /* Return value */
_PyRuntimeState * const runtime = &_PyRuntime;
PyThreadState * const tstate = _PyRuntimeState_GetThreadState(runtime);
- struct _ceval_runtime_state * const ceval = &runtime->ceval;
- _Py_atomic_int * const eval_breaker = &ceval->eval_breaker;
+ PyInterpreterState * const interp = tstate->interp;
+ struct _ceval_runtime_state * const ceval_r = &runtime->ceval;
+ struct _ceval_interpreter_state * const ceval_i = &interp->ceval;
+ _Py_atomic_int * const eval_breaker = &ceval_r->eval_breaker;
PyCodeObject *co;
/* when tracing we set things up so that
@@ -797,7 +871,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
#ifdef LLTRACE
#define FAST_DISPATCH() \
{ \
- if (!lltrace && !_Py_TracingPossible(ceval) && !PyDTrace_LINE_ENABLED()) { \
+ if (!lltrace && !_Py_TracingPossible(ceval_r) && !PyDTrace_LINE_ENABLED()) { \
f->f_lasti = INSTR_OFFSET(); \
NEXTOPARG(); \
goto *opcode_targets[opcode]; \
@@ -807,7 +881,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
#else
#define FAST_DISPATCH() \
{ \
- if (!_Py_TracingPossible(ceval) && !PyDTrace_LINE_ENABLED()) { \
+ if (!_Py_TracingPossible(ceval_r) && !PyDTrace_LINE_ENABLED()) { \
f->f_lasti = INSTR_OFFSET(); \
NEXTOPARG(); \
goto *opcode_targets[opcode]; \
@@ -1122,27 +1196,27 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
goto fast_next_opcode;
}
- if (_Py_atomic_load_relaxed(&ceval->signals_pending)) {
+ if (_Py_atomic_load_relaxed(&ceval_r->signals_pending)) {
if (handle_signals(runtime) != 0) {
goto error;
}
}
- if (_Py_atomic_load_relaxed(&ceval->pending.calls_to_do)) {
- if (make_pending_calls(runtime) != 0) {
+ if (_Py_atomic_load_relaxed(&ceval_i->pending.calls_to_do)) {
+ if (make_pending_calls(interp) != 0) {
goto error;
}
}
- if (_Py_atomic_load_relaxed(&ceval->gil_drop_request)) {
+ if (_Py_atomic_load_relaxed(&ceval_r->gil_drop_request)) {
/* Give another thread a chance */
if (_PyThreadState_Swap(&runtime->gilstate, NULL) != tstate) {
Py_FatalError("ceval: tstate mix-up");
}
- drop_gil(ceval, tstate);
+ drop_gil(ceval_r, ceval_i, tstate);
/* Other threads may run now */
- take_gil(ceval, tstate);
+ take_gil(ceval_r, tstate);
/* Check if we should make a quick exit. */
exit_thread_if_finalizing(tstate);
@@ -1155,7 +1229,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
if (tstate->async_exc != NULL) {
PyObject *exc = tstate->async_exc;
tstate->async_exc = NULL;
- UNSIGNAL_ASYNC_EXC(ceval);
+ UNSIGNAL_ASYNC_EXC(ceval_r, ceval_i);
_PyErr_SetNone(tstate, exc);
Py_DECREF(exc);
goto error;
@@ -1170,7 +1244,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
/* line-by-line tracing support */
- if (_Py_TracingPossible(ceval) &&
+ if (_Py_TracingPossible(ceval_r) &&
tstate->c_tracefunc != NULL && !tstate->tracing) {
int err;
/* see maybe_call_line_trace
diff --git a/Python/ceval_gil.h b/Python/ceval_gil.h
index 34d48c990c44..b44d0abad36b 100644
--- a/Python/ceval_gil.h
+++ b/Python/ceval_gil.h
@@ -141,9 +141,11 @@ static void recreate_gil(struct _gil_runtime_state *gil)
}
static void
-drop_gil(struct _ceval_runtime_state *ceval, PyThreadState *tstate)
+drop_gil(struct _ceval_runtime_state *ceval_r,
+ struct _ceval_interpreter_state *ceval_i,
+ PyThreadState *tstate)
{
- struct _gil_runtime_state *gil = &ceval->gil;
+ struct _gil_runtime_state *gil = &ceval_r->gil;
if (!_Py_atomic_load_relaxed(&gil->locked)) {
Py_FatalError("drop_gil: GIL is not locked");
}
@@ -163,12 +165,12 @@ drop_gil(struct _ceval_runtime_state *ceval, PyThreadState *tstate)
MUTEX_UNLOCK(gil->mutex);
#ifdef FORCE_SWITCHING
- if (_Py_atomic_load_relaxed(&ceval->gil_drop_request) && tstate != NULL) {
+ if (_Py_atomic_load_relaxed(&ceval_r->gil_drop_request) && tstate != NULL) {
MUTEX_LOCK(gil->switch_mutex);
/* Not switched yet => wait */
if (((PyThreadState*)_Py_atomic_load_relaxed(&gil->last_holder)) == tstate)
{
- RESET_GIL_DROP_REQUEST(ceval);
+ RESET_GIL_DROP_REQUEST(ceval_r, ceval_i);
/* NOTE: if COND_WAIT does not atomically start waiting when
releasing the mutex, another thread can run through, take
the GIL and drop it again, and reset the condition
@@ -181,13 +183,19 @@ drop_gil(struct _ceval_runtime_state *ceval, PyThreadState *tstate)
}
static void
-take_gil(struct _ceval_runtime_state *ceval, PyThreadState *tstate)
+take_gil(struct _ceval_runtime_state *ceval_r,
+ PyThreadState *tstate)
{
if (tstate == NULL) {
Py_FatalError("take_gil: NULL tstate");
}
+ PyInterpreterState *interp = tstate->interp;
+ if (interp == NULL) {
+ Py_FatalError("take_gil: NULL interp");
+ }
+ struct _ceval_interpreter_state *ceval_i = &interp->ceval;
- struct _gil_runtime_state *gil = &ceval->gil;
+ struct _gil_runtime_state *gil = &ceval_r->gil;
int err = errno;
MUTEX_LOCK(gil->mutex);
@@ -210,7 +218,7 @@ take_gil(struct _ceval_runtime_state *ceval, PyThreadState *tstate)
_Py_atomic_load_relaxed(&gil->locked) &&
gil->switch_number == saved_switchnum)
{
- SET_GIL_DROP_REQUEST(ceval);
+ SET_GIL_DROP_REQUEST(ceval_r);
}
}
_ready:
@@ -232,11 +240,11 @@ take_gil(struct _ceval_runtime_state *ceval, PyThreadState *tstate)
COND_SIGNAL(gil->switch_cond);
MUTEX_UNLOCK(gil->switch_mutex);
#endif
- if (_Py_atomic_load_relaxed(&ceval->gil_drop_request)) {
- RESET_GIL_DROP_REQUEST(ceval);
+ if (_Py_atomic_load_relaxed(&ceval_r->gil_drop_request)) {
+ RESET_GIL_DROP_REQUEST(ceval_r, ceval_i);
}
if (tstate->async_exc != NULL) {
- _PyEval_SignalAsyncExc(ceval);
+ _PyEval_SignalAsyncExc(ceval_r, ceval_i);
}
MUTEX_UNLOCK(gil->mutex);
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
index 6590ef8e9a27..3de5528811ae 100644
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -1147,15 +1147,31 @@ Py_FinalizeEx(void)
return status;
}
+ /* Get current thread state and interpreter pointer */
+ PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
+ PyInterpreterState *interp = tstate->interp;
+
// Wrap up existing "threading"-module-created, non-daemon threads.
wait_for_thread_shutdown();
// Make any remaining pending calls.
- _Py_FinishPendingCalls(runtime);
-
- /* Get current thread state and interpreter pointer */
- PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
- PyInterpreterState *interp = tstate->interp;
+ /* XXX For the moment we are going to ignore lingering pending calls.
+ * We've seen sporadic on some of the buildbots during finalization
+ * with the changes for per-interpreter pending calls (see bpo-33608),
+ * meaning the previous _PyEval_FinishPendincCalls() call here is
+ * a trigger, if not responsible.
+ *
+ * Ignoring pending calls at this point in the runtime lifecycle
+ * is okay (for now) for the following reasons:
+ *
+ * * pending calls are still not a widely-used feature
+ * * this only affects runtime finalization, where the process is
+ * likely to end soon anyway (except for some embdding cases)
+ *
+ * See bpo-37127 about resolving the problem. Ultimately the call
+ * here should be re-enabled.
+ */
+ //_PyEval_FinishPendingCalls(interp);
/* The interpreter is still entirely intact at this point, and the
* exit funcs may be relying on that. In particular, if some thread
@@ -1580,6 +1596,9 @@ Py_EndInterpreter(PyThreadState *tstate)
// Wrap up existing "threading"-module-created, non-daemon threads.
wait_for_thread_shutdown();
+ // Make any remaining pending calls.
+ _PyEval_FinishPendingCalls(interp);
+
call_py_exitfuncs(interp);
if (tstate != interp->tstate_head || tstate->next != NULL)
diff --git a/Python/pystate.c b/Python/pystate.c
index 2b7db0e48deb..a9f3389a0d83 100644
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -218,6 +218,13 @@ PyInterpreterState_New(void)
return NULL;
}
+ interp->ceval.pending.lock = PyThread_allocate_lock();
+ if (interp->ceval.pending.lock == NULL) {
+ PyErr_SetString(PyExc_RuntimeError,
+ "failed to create interpreter ceval pending mutex");
+ return NULL;
+ }
+
interp->eval_frame = _PyEval_EvalFrameDefault;
#ifdef HAVE_DLOPEN
#if HAVE_DECL_RTLD_NOW
@@ -345,6 +352,10 @@ PyInterpreterState_Delete(PyInterpreterState *interp)
if (interp->id_mutex != NULL) {
PyThread_free_lock(interp->id_mutex);
}
+ if (interp->ceval.pending.lock != NULL) {
+ PyThread_free_lock(interp->ceval.pending.lock);
+ interp->ceval.pending.lock = NULL;
+ }
PyMem_RawFree(interp);
}
@@ -1014,7 +1025,7 @@ PyThreadState_SetAsyncExc(unsigned long id, PyObject *exc)
p->async_exc = exc;
HEAD_UNLOCK(runtime);
Py_XDECREF(old_exc);
- _PyEval_SignalAsyncExc(&runtime->ceval);
+ _PyEval_SignalAsyncExc(&runtime->ceval, &interp->ceval);
return 1;
}
}
@@ -1444,7 +1455,7 @@ _PyObject_GetCrossInterpreterData(PyObject *obj, _PyCrossInterpreterData *data)
return 0;
}
-static void
+static int
_release_xidata(void *arg)
{
_PyCrossInterpreterData *data = (_PyCrossInterpreterData *)arg;
@@ -1452,42 +1463,21 @@ _release_xidata(void *arg)
data->free(data->data);
}
Py_XDECREF(data->obj);
-}
-
-static void
-_call_in_interpreter(struct _gilstate_runtime_state *gilstate,
- PyInterpreterState *interp,
- void (*func)(void *), void *arg)
-{
- /* We would use Py_AddPendingCall() if it weren't specific to the
- * main interpreter (see bpo-33608). In the meantime we take a
- * naive approach.
- */
- PyThreadState *save_tstate = NULL;
- if (interp != _PyRuntimeGILState_GetThreadState(gilstate)->interp) {
- // XXX Using the "head" thread isn't strictly correct.
- PyThreadState *tstate = PyInterpreterState_ThreadHead(interp);
- // XXX Possible GILState issues?
- save_tstate = _PyThreadState_Swap(gilstate, tstate);
- }
-
- func(arg);
-
- // Switch back.
- if (save_tstate != NULL) {
- _PyThreadState_Swap(gilstate, save_tstate);
- }
+ PyMem_Free(data);
+ return 0;
}
void
_PyCrossInterpreterData_Release(_PyCrossInterpreterData *data)
{
+ _PyRuntimeState *runtime = &_PyRuntime;
+
if (data->data == NULL && data->obj == NULL) {
// Nothing to release!
return;
}
- // Switch to the original interpreter.
+ // Get the original interpreter.
PyInterpreterState *interp = _PyInterpreterState_LookUpID(data->interp);
if (interp == NULL) {
// The intepreter was already destroyed.
@@ -1496,10 +1486,28 @@ _PyCrossInterpreterData_Release(_PyCrossInterpreterData *data)
}
return;
}
+ // XXX There's an ever-so-slight race here...
+ if (interp->finalizing) {
+ // XXX Someone leaked some memory...
+ return;
+ }
// "Release" the data and/or the object.
- struct _gilstate_runtime_state *gilstate = &_PyRuntime.gilstate;
- _call_in_interpreter(gilstate, interp, _release_xidata, data);
+ _PyCrossInterpreterData *copied = PyMem_Malloc(sizeof(_PyCrossInterpreterData));
+ if (copied == NULL) {
+ PyErr_SetString(PyExc_MemoryError,
+ "Not enough memory to preserve cross-interpreter data");
+ PyErr_Print();
+ return;
+ }
+ memcpy(copied, data, sizeof(_PyCrossInterpreterData));
+ PyThreadState *tstate = _PyRuntimeState_GetThreadState(runtime);
+ int res = _PyEval_AddPendingCall(tstate,
+ &runtime->ceval, &interp->ceval,
+ 0, _release_xidata, copied);
+ if (res != 0) {
+ // XXX Queue full or couldn't get lock. Try again somehow?
+ }
}
PyObject *
1
0
bpo-29414: Change 'the for statement is such an iterator' in Tutorial (GH-273)
by Raymond Hettinger 01 Jun '19
by Raymond Hettinger 01 Jun '19
01 Jun '19
https://github.com/python/cpython/commit/218e47b61862470477922e9aba1a23fd3d…
commit: 218e47b61862470477922e9aba1a23fd3dab18ae
branch: master
author: Marco Buttu <marco.buttu(a)gmail.com>
committer: Raymond Hettinger <rhettinger(a)users.noreply.github.com>
date: 2019-06-01T14:11:47-07:00
summary:
bpo-29414: Change 'the for statement is such an iterator' in Tutorial (GH-273)
files:
M Doc/tutorial/controlflow.rst
diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst
index 81a28a6e5325..79111f8518d9 100644
--- a/Doc/tutorial/controlflow.rst
+++ b/Doc/tutorial/controlflow.rst
@@ -139,18 +139,24 @@ but in fact it isn't. It is an object which returns the successive items of
the desired sequence when you iterate over it, but it doesn't really make
the list, thus saving space.
-We say such an object is *iterable*, that is, suitable as a target for
+We say such an object is :term:`iterable`, that is, suitable as a target for
functions and constructs that expect something from which they can
-obtain successive items until the supply is exhausted. We have seen that
-the :keyword:`for` statement is such an *iterator*. The function :func:`list`
-is another; it creates lists from iterables::
+obtain successive items until the supply is exhausted. We have seen that
+the :keyword:`for` statement is such a construct, while an example of function
+that takes an iterable is :func:`sum`::
+ >>> sum(range(4)) # 0 + 1 + 2 + 3
+ 6
- >>> list(range(5))
- [0, 1, 2, 3, 4]
+Later we will see more functions that return iterables and take iterables as
+arguments. Lastly, maybe you are curious about how to get a list from a range.
+Here is the solution::
-Later we will see more functions that return iterables and take iterables as argument.
+ >>> list(range(4))
+ [0, 1, 2, 3]
+In chapter :ref:`tut-structures`, we will discuss in more detail about
+:func:`list`.
.. _tut-break:
@@ -161,7 +167,7 @@ The :keyword:`break` statement, like in C, breaks out of the innermost enclosing
:keyword:`for` or :keyword:`while` loop.
Loop statements may have an :keyword:`!else` clause; it is executed when the loop
-terminates through exhaustion of the list (with :keyword:`for`) or when the
+terminates through exhaustion of the iterable (with :keyword:`for`) or when the
condition becomes false (with :keyword:`while`), but not when the loop is
terminated by a :keyword:`break` statement. This is exemplified by the
following loop, which searches for prime numbers::
@@ -188,8 +194,8 @@ following loop, which searches for prime numbers::
the :keyword:`for` loop, **not** the :keyword:`if` statement.)
When used with a loop, the ``else`` clause has more in common with the
-``else`` clause of a :keyword:`try` statement than it does that of
-:keyword:`if` statements: a :keyword:`!try` statement's ``else`` clause runs
+``else`` clause of a :keyword:`try` statement than it does with that of
+:keyword:`if` statements: a :keyword:`try` statement's ``else`` clause runs
when no exception occurs, and a loop's ``else`` clause runs when no ``break``
occurs. For more on the :keyword:`!try` statement and exceptions, see
:ref:`tut-handling`.
1
0
https://github.com/python/cpython/commit/36dcaab7fde5d2e54cdeff5b705b5adcb2…
commit: 36dcaab7fde5d2e54cdeff5b705b5adcb27726dd
branch: master
author: Zackery Spytz <zspytz(a)gmail.com>
committer: Serhiy Storchaka <storchaka(a)gmail.com>
date: 2019-06-02T00:07:45+03:00
summary:
Fix the error handling in bytesio_sizeof(). (GH-10459)
bytesio_sizeof() must check if an error has occurred in _PySys_GetSizeOf().
files:
M Modules/_io/bytesio.c
diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c
index 32427e44de5b..19e1ed8441e3 100644
--- a/Modules/_io/bytesio.c
+++ b/Modules/_io/bytesio.c
@@ -943,8 +943,13 @@ bytesio_sizeof(bytesio *self, void *unused)
Py_ssize_t res;
res = _PyObject_SIZE(Py_TYPE(self));
- if (self->buf && !SHARED_BUF(self))
- res += _PySys_GetSizeOf(self->buf);
+ if (self->buf && !SHARED_BUF(self)) {
+ Py_ssize_t s = _PySys_GetSizeOf(self->buf);
+ if (s == -1) {
+ return NULL;
+ }
+ res += s;
+ }
return PyLong_FromSsize_t(res);
}
1
0
bpo-20092. Use __index__ in constructors of int, float and complex. (GH-13108)
by Serhiy Storchaka 01 Jun '19
by Serhiy Storchaka 01 Jun '19
01 Jun '19
https://github.com/python/cpython/commit/bdbad71b9def0b86433de12cecca022eee…
commit: bdbad71b9def0b86433de12cecca022eee91bd9f
branch: master
author: Serhiy Storchaka <storchaka(a)gmail.com>
committer: GitHub <noreply(a)github.com>
date: 2019-06-02T00:05:48+03:00
summary:
bpo-20092. Use __index__ in constructors of int, float and complex. (GH-13108)
files:
A Misc/NEWS.d/next/Core and Builtins/2019-05-31-11-55-49.bpo-20092.KIMjBW.rst
M Doc/c-api/complex.rst
M Doc/c-api/float.rst
M Doc/library/functions.rst
M Doc/reference/datamodel.rst
M Doc/whatsnew/3.8.rst
M Lib/test/test_cmath.py
M Lib/test/test_complex.py
M Lib/test/test_float.py
M Lib/test/test_getargs2.py
M Lib/test/test_index.py
M Lib/test/test_int.py
M Objects/abstract.c
M Objects/complexobject.c
M Objects/floatobject.c
diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst
index 675bd013e892..06dbb2572725 100644
--- a/Doc/c-api/complex.rst
+++ b/Doc/c-api/complex.rst
@@ -129,4 +129,10 @@ Complex Numbers as Python Objects
If *op* is not a Python complex number object but has a :meth:`__complex__`
method, this method will first be called to convert *op* to a Python complex
- number object. Upon failure, this method returns ``-1.0`` as a real value.
+ number object. If ``__complex__()`` is not defined then it falls back to
+ :meth:`__float__`. If ``__float__()`` is not defined then it falls back
+ to :meth:`__index__`. Upon failure, this method returns ``-1.0`` as a real
+ value.
+
+ .. versionchanged:: 3.8
+ Use :meth:`__index__` if available.
diff --git a/Doc/c-api/float.rst b/Doc/c-api/float.rst
index 8a996422ce48..057ff522516a 100644
--- a/Doc/c-api/float.rst
+++ b/Doc/c-api/float.rst
@@ -47,9 +47,13 @@ Floating Point Objects
Return a C :c:type:`double` representation of the contents of *pyfloat*. If
*pyfloat* is not a Python floating point object but has a :meth:`__float__`
method, this method will first be called to convert *pyfloat* into a float.
+ If ``__float__()`` is not defined then it falls back to :meth:`__index__`.
This method returns ``-1.0`` upon failure, so one should call
:c:func:`PyErr_Occurred` to check for errors.
+ .. versionchanged:: 3.8
+ Use :meth:`__index__` if available.
+
.. c:function:: double PyFloat_AS_DOUBLE(PyObject *pyfloat)
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
index 425a985320fc..d5c9f18c79b7 100644
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -318,6 +318,11 @@ are always available. They are listed here in alphabetical order.
:class:`int` and :class:`float`. If both arguments are omitted, returns
``0j``.
+ For a general Python object ``x``, ``complex(x)`` delegates to
+ ``x.__complex__()``. If ``__complex__()`` is not defined then it falls back
+ to :meth:`__float__`. If ``__float__()`` is not defined then it falls back
+ to :meth:`__index__`.
+
.. note::
When converting from a string, the string must not contain whitespace
@@ -330,6 +335,10 @@ are always available. They are listed here in alphabetical order.
.. versionchanged:: 3.6
Grouping digits with underscores as in code literals is allowed.
+ .. versionchanged:: 3.8
+ Falls back to :meth:`__index__` if :meth:`__complex__` and
+ :meth:`__float__` are not defined.
+
.. function:: delattr(object, name)
@@ -584,7 +593,8 @@ are always available. They are listed here in alphabetical order.
float, an :exc:`OverflowError` will be raised.
For a general Python object ``x``, ``float(x)`` delegates to
- ``x.__float__()``.
+ ``x.__float__()``. If ``__float__()`` is not defined then it falls back
+ to :meth:`__index__`.
If no argument is given, ``0.0`` is returned.
@@ -609,6 +619,9 @@ are always available. They are listed here in alphabetical order.
.. versionchanged:: 3.7
*x* is now a positional-only parameter.
+ .. versionchanged:: 3.8
+ Falls back to :meth:`__index__` if :meth:`__float__` is not defined.
+
.. index::
single: __format__
@@ -780,7 +793,8 @@ are always available. They are listed here in alphabetical order.
Return an integer object constructed from a number or string *x*, or return
``0`` if no arguments are given. If *x* defines :meth:`__int__`,
- ``int(x)`` returns ``x.__int__()``. If *x* defines :meth:`__trunc__`,
+ ``int(x)`` returns ``x.__int__()``. If *x* defines :meth:`__index__`,
+ it returns ``x.__index__()``. If *x* defines :meth:`__trunc__`,
it returns ``x.__trunc__()``.
For floating point numbers, this truncates towards zero.
@@ -812,6 +826,9 @@ are always available. They are listed here in alphabetical order.
.. versionchanged:: 3.7
*x* is now a positional-only parameter.
+ .. versionchanged:: 3.8
+ Falls back to :meth:`__index__` if :meth:`__int__` is not defined.
+
.. function:: isinstance(object, classinfo)
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
index 44017d8a55df..fa47bf1c1619 100644
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -2394,11 +2394,9 @@ left undefined.
functions). Presence of this method indicates that the numeric object is
an integer type. Must return an integer.
- .. note::
-
- In order to have a coherent integer type class, when :meth:`__index__` is
- defined :meth:`__int__` should also be defined, and both should return
- the same value.
+ If :meth:`__int__`, :meth:`__float__` and :meth:`__complex__` are not
+ defined then corresponding built-in functions :func:`int`, :func:`float`
+ and :func:`complex` fall back to :meth:`__index__`.
.. method:: object.__round__(self, [,ndigits])
diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst
index 4c5a9bb0cdb9..591b45488372 100644
--- a/Doc/whatsnew/3.8.rst
+++ b/Doc/whatsnew/3.8.rst
@@ -250,6 +250,12 @@ Other Language Changes
compatible with the existing :meth:`float.as_integer_ratio` method.
(Contributed by Lisa Roach in :issue:`33073`.)
+* Constructors of :class:`int`, :class:`float` and :class:`complex` will now
+ use the :meth:`~object.__index__` special method, if available and the
+ corresponding method :meth:`~object.__int__`, :meth:`~object.__float__`
+ or :meth:`~object.__complex__` is not available.
+ (Contributed by Serhiy Storchaka in :issue:`20092`.)
+
* Added support of ``\N{name}`` escapes in :mod:`regular expressions <re>`.
(Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.)
@@ -868,7 +874,10 @@ Build and C API Changes
``__index__()`` method (like :class:`~decimal.Decimal` and
:class:`~fractions.Fraction`). :c:func:`PyNumber_Check` will now return
``1`` for objects implementing ``__index__()``.
- (Contributed by Serhiy Storchaka in :issue:`36048`.)
+ :c:func:`PyNumber_Long`, :c:func:`PyNumber_Float` and
+ :c:func:`PyFloat_AsDouble` also now use the ``__index__()`` method if
+ available.
+ (Contributed by Serhiy Storchaka in :issue:`36048` and :issue:`20092`.)
* Heap-allocated type objects will now increase their reference count
in :c:func:`PyObject_Init` (and its parallel macro ``PyObject_INIT``)
diff --git a/Lib/test/test_cmath.py b/Lib/test/test_cmath.py
index 43a074b4b663..a00185f43dbf 100644
--- a/Lib/test/test_cmath.py
+++ b/Lib/test/test_cmath.py
@@ -220,12 +220,11 @@ class NeitherComplexNorFloat(object):
pass
class NeitherComplexNorFloatOS:
pass
- class MyInt(object):
+ class Index:
def __int__(self): return 2
def __index__(self): return 2
- class MyIntOS:
+ class MyInt:
def __int__(self): return 2
- def __index__(self): return 2
# other possible combinations of __float__ and __complex__
# that should work
@@ -255,6 +254,7 @@ def __float__(self):
self.assertEqual(f(FloatAndComplexOS()), f(cx_arg))
self.assertEqual(f(JustFloat()), f(flt_arg))
self.assertEqual(f(JustFloatOS()), f(flt_arg))
+ self.assertEqual(f(Index()), f(int(Index())))
# TypeError should be raised for classes not providing
# either __complex__ or __float__, even if they provide
# __int__ or __index__. An old-style class
@@ -263,7 +263,6 @@ def __float__(self):
self.assertRaises(TypeError, f, NeitherComplexNorFloat())
self.assertRaises(TypeError, f, MyInt())
self.assertRaises(Exception, f, NeitherComplexNorFloatOS())
- self.assertRaises(Exception, f, MyIntOS())
# non-complex return value from __complex__ -> TypeError
for bad_complex in non_complexes:
self.assertRaises(TypeError, f, MyComplex(bad_complex))
diff --git a/Lib/test/test_complex.py b/Lib/test/test_complex.py
index 21c6eaed6054..fe1e566562de 100644
--- a/Lib/test/test_complex.py
+++ b/Lib/test/test_complex.py
@@ -368,6 +368,24 @@ def __float__(self):
self.assertAlmostEqual(complex(real=float2(17.), imag=float2(23.)), 17+23j)
self.assertRaises(TypeError, complex, float2(None))
+ class MyIndex:
+ def __init__(self, value):
+ self.value = value
+ def __index__(self):
+ return self.value
+
+ self.assertAlmostEqual(complex(MyIndex(42)), 42.0+0.0j)
+ self.assertAlmostEqual(complex(123, MyIndex(42)), 123.0+42.0j)
+ self.assertRaises(OverflowError, complex, MyIndex(2**2000))
+ self.assertRaises(OverflowError, complex, 123, MyIndex(2**2000))
+
+ class MyInt:
+ def __int__(self):
+ return 42
+
+ self.assertRaises(TypeError, complex, MyInt())
+ self.assertRaises(TypeError, complex, 123, MyInt())
+
class complex0(complex):
"""Test usage of __complex__() when inheriting from 'complex'"""
def __complex__(self):
diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py
index 5278d716de23..b656582538e8 100644
--- a/Lib/test/test_float.py
+++ b/Lib/test/test_float.py
@@ -223,6 +223,21 @@ def __float__(self):
with self.assertWarns(DeprecationWarning):
self.assertIs(type(FloatSubclass(F())), FloatSubclass)
+ class MyIndex:
+ def __init__(self, value):
+ self.value = value
+ def __index__(self):
+ return self.value
+
+ self.assertEqual(float(MyIndex(42)), 42.0)
+ self.assertRaises(OverflowError, float, MyIndex(2**2000))
+
+ class MyInt:
+ def __int__(self):
+ return 42
+
+ self.assertRaises(TypeError, float, MyInt())
+
def test_keyword_args(self):
with self.assertRaisesRegex(TypeError, 'keyword argument'):
float(x='3.14')
diff --git a/Lib/test/test_getargs2.py b/Lib/test/test_getargs2.py
index 07e2d1513791..1a73fa461580 100644
--- a/Lib/test/test_getargs2.py
+++ b/Lib/test/test_getargs2.py
@@ -457,6 +457,8 @@ def test_f(self):
with self.assertWarns(DeprecationWarning):
self.assertEqual(getargs_f(BadFloat2()), 4.25)
self.assertEqual(getargs_f(BadFloat3(7.5)), 7.5)
+ self.assertEqual(getargs_f(Index()), 99.0)
+ self.assertRaises(TypeError, getargs_f, Int())
for x in (FLT_MIN, -FLT_MIN, FLT_MAX, -FLT_MAX, INF, -INF):
self.assertEqual(getargs_f(x), x)
@@ -489,6 +491,8 @@ def test_d(self):
with self.assertWarns(DeprecationWarning):
self.assertEqual(getargs_d(BadFloat2()), 4.25)
self.assertEqual(getargs_d(BadFloat3(7.5)), 7.5)
+ self.assertEqual(getargs_d(Index()), 99.0)
+ self.assertRaises(TypeError, getargs_d, Int())
for x in (DBL_MIN, -DBL_MIN, DBL_MAX, -DBL_MAX, INF, -INF):
self.assertEqual(getargs_d(x), x)
@@ -511,6 +515,8 @@ def test_D(self):
with self.assertWarns(DeprecationWarning):
self.assertEqual(getargs_D(BadComplex2()), 4.25+0.5j)
self.assertEqual(getargs_D(BadComplex3(7.5+0.25j)), 7.5+0.25j)
+ self.assertEqual(getargs_D(Index()), 99.0+0j)
+ self.assertRaises(TypeError, getargs_D, Int())
for x in (DBL_MIN, -DBL_MIN, DBL_MAX, -DBL_MAX, INF, -INF):
c = complex(x, 1.0)
diff --git a/Lib/test/test_index.py b/Lib/test/test_index.py
index a2ac32132e23..cbdc56c801a4 100644
--- a/Lib/test/test_index.py
+++ b/Lib/test/test_index.py
@@ -60,7 +60,7 @@ def test_int_subclass_with_index(self):
# subclasses. See issue #17576.
class MyInt(int):
def __index__(self):
- return int(self) + 1
+ return int(str(self)) + 1
my_int = MyInt(7)
direct_index = my_int.__index__()
diff --git a/Lib/test/test_int.py b/Lib/test/test_int.py
index 307ca36bb4fa..6fdf52ef23f6 100644
--- a/Lib/test/test_int.py
+++ b/Lib/test/test_int.py
@@ -378,15 +378,23 @@ def __trunc__(self):
int(ExceptionalTrunc())
for trunc_result_base in (object, Classic):
- class Integral(trunc_result_base):
- def __int__(self):
+ class Index(trunc_result_base):
+ def __index__(self):
return 42
class TruncReturnsNonInt(base):
def __trunc__(self):
- return Integral()
- with self.assertWarns(DeprecationWarning):
- self.assertEqual(int(TruncReturnsNonInt()), 42)
+ return Index()
+ self.assertEqual(int(TruncReturnsNonInt()), 42)
+
+ class Intable(trunc_result_base):
+ def __int__(self):
+ return 42
+
+ class TruncReturnsNonIndex(base):
+ def __trunc__(self):
+ return Intable()
+ self.assertEqual(int(TruncReturnsNonInt()), 42)
class NonIntegral(trunc_result_base):
def __trunc__(self):
@@ -418,6 +426,21 @@ def __trunc__(self):
with self.assertRaises(TypeError):
int(TruncReturnsBadInt())
+ def test_int_subclass_with_index(self):
+ class MyIndex(int):
+ def __index__(self):
+ return 42
+
+ class BadIndex(int):
+ def __index__(self):
+ return 42.0
+
+ my_int = MyIndex(7)
+ self.assertEqual(my_int, 7)
+ self.assertEqual(int(my_int), 7)
+
+ self.assertEqual(int(BadIndex()), 0)
+
def test_int_subclass_with_int(self):
class MyInt(int):
def __int__(self):
@@ -431,9 +454,19 @@ def __int__(self):
self.assertEqual(my_int, 7)
self.assertEqual(int(my_int), 42)
- self.assertRaises(TypeError, int, BadInt())
+ my_int = BadInt(7)
+ self.assertEqual(my_int, 7)
+ self.assertRaises(TypeError, int, my_int)
def test_int_returns_int_subclass(self):
+ class BadIndex:
+ def __index__(self):
+ return True
+
+ class BadIndex2(int):
+ def __index__(self):
+ return True
+
class BadInt:
def __int__(self):
return True
@@ -442,6 +475,10 @@ class BadInt2(int):
def __int__(self):
return True
+ class TruncReturnsBadIndex:
+ def __trunc__(self):
+ return BadIndex()
+
class TruncReturnsBadInt:
def __trunc__(self):
return BadInt()
@@ -450,6 +487,17 @@ class TruncReturnsIntSubclass:
def __trunc__(self):
return True
+ bad_int = BadIndex()
+ with self.assertWarns(DeprecationWarning):
+ n = int(bad_int)
+ self.assertEqual(n, 1)
+ self.assertIs(type(n), int)
+
+ bad_int = BadIndex2()
+ n = int(bad_int)
+ self.assertEqual(n, 0)
+ self.assertIs(type(n), int)
+
bad_int = BadInt()
with self.assertWarns(DeprecationWarning):
n = int(bad_int)
@@ -462,6 +510,12 @@ def __trunc__(self):
self.assertEqual(n, 1)
self.assertIs(type(n), int)
+ bad_int = TruncReturnsBadIndex()
+ with self.assertWarns(DeprecationWarning):
+ n = int(bad_int)
+ self.assertEqual(n, 1)
+ self.assertIs(type(n), int)
+
bad_int = TruncReturnsBadInt()
with self.assertWarns(DeprecationWarning):
n = int(bad_int)
diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-05-31-11-55-49.bpo-20092.KIMjBW.rst b/Misc/NEWS.d/next/Core and Builtins/2019-05-31-11-55-49.bpo-20092.KIMjBW.rst
new file mode 100644
index 000000000000..7536dc33c9f1
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2019-05-31-11-55-49.bpo-20092.KIMjBW.rst
@@ -0,0 +1,4 @@
+Constructors of :class:`int`, :class:`float` and :class:`complex` will now
+use the :meth:`~object.__index__` special method, if available and the
+corresponding method :meth:`~object.__int__`, :meth:`~object.__float__`
+or :meth:`~object.__complex__` is not available.
diff --git a/Objects/abstract.c b/Objects/abstract.c
index 68d06edfa600..77d09143aa07 100644
--- a/Objects/abstract.c
+++ b/Objects/abstract.c
@@ -1373,6 +1373,13 @@ PyNumber_Long(PyObject *o)
}
return result;
}
+ if (m && m->nb_index) {
+ result = _PyLong_FromNbIndexOrNbInt(o);
+ if (result != NULL && !PyLong_CheckExact(result)) {
+ Py_SETREF(result, _PyLong_Copy((PyLongObject *)result));
+ }
+ return result;
+ }
trunc_func = _PyObject_LookupSpecial(o, &PyId___trunc__);
if (trunc_func) {
result = _PyObject_CallNoArg(trunc_func);
@@ -1480,6 +1487,18 @@ PyNumber_Float(PyObject *o)
Py_DECREF(res);
return PyFloat_FromDouble(val);
}
+ if (m && m->nb_index) {
+ PyObject *res = PyNumber_Index(o);
+ if (!res) {
+ return NULL;
+ }
+ double val = PyLong_AsDouble(res);
+ Py_DECREF(res);
+ if (val == -1.0 && PyErr_Occurred()) {
+ return NULL;
+ }
+ return PyFloat_FromDouble(val);
+ }
if (PyFloat_Check(o)) { /* A float subclass with nb_float == NULL */
return PyFloat_FromDouble(PyFloat_AS_DOUBLE(o));
}
diff --git a/Objects/complexobject.c b/Objects/complexobject.c
index a5f95186d625..f78c0fdf78de 100644
--- a/Objects/complexobject.c
+++ b/Objects/complexobject.c
@@ -984,7 +984,7 @@ complex_new_impl(PyTypeObject *type, PyObject *r, PyObject *i)
}
nbr = r->ob_type->tp_as_number;
- if (nbr == NULL || nbr->nb_float == NULL) {
+ if (nbr == NULL || (nbr->nb_float == NULL && nbr->nb_index == NULL)) {
PyErr_Format(PyExc_TypeError,
"complex() first argument must be a string or a number, "
"not '%.200s'",
@@ -996,7 +996,7 @@ complex_new_impl(PyTypeObject *type, PyObject *r, PyObject *i)
}
if (i != NULL) {
nbi = i->ob_type->tp_as_number;
- if (nbi == NULL || nbi->nb_float == NULL) {
+ if (nbi == NULL || (nbi->nb_float == NULL && nbi->nb_index == NULL)) {
PyErr_Format(PyExc_TypeError,
"complex() second argument must be a number, "
"not '%.200s'",
@@ -1052,7 +1052,7 @@ complex_new_impl(PyTypeObject *type, PyObject *r, PyObject *i)
/* The "imag" part really is entirely imaginary, and
contributes nothing in the real direction.
Just treat it as a double. */
- tmp = (*nbi->nb_float)(i);
+ tmp = PyNumber_Float(i);
if (tmp == NULL)
return NULL;
ci.real = PyFloat_AsDouble(tmp);
diff --git a/Objects/floatobject.c b/Objects/floatobject.c
index 2bf7061d4f62..15cbe5c9d8ba 100644
--- a/Objects/floatobject.c
+++ b/Objects/floatobject.c
@@ -246,6 +246,15 @@ PyFloat_AsDouble(PyObject *op)
nb = Py_TYPE(op)->tp_as_number;
if (nb == NULL || nb->nb_float == NULL) {
+ if (nb && nb->nb_index) {
+ PyObject *res = PyNumber_Index(op);
+ if (!res) {
+ return -1;
+ }
+ double val = PyLong_AsDouble(res);
+ Py_DECREF(res);
+ return val;
+ }
PyErr_Format(PyExc_TypeError, "must be real number, not %.50s",
op->ob_type->tp_name);
return -1;
1
0
01 Jun '19
https://github.com/python/cpython/commit/1a4d9ffa1aecd7e750195f2be06d3d16c7…
commit: 1a4d9ffa1aecd7e750195f2be06d3d16c7a3a88f
branch: master
author: Cheryl Sabella <cheryl.sabella(a)gmail.com>
committer: Terry Jan Reedy <tjreedy(a)udel.edu>
date: 2019-06-01T17:03:22-04:00
summary:
bpo-32411: IDLE: Remove line number sort in browser.py (#5011)
Insertion in line order makes sorting keys by line order unneeded.
files:
A Misc/NEWS.d/next/IDLE/2017-12-25-18-48-50.bpo-32411.vNwDhe.rst
M Lib/idlelib/NEWS.txt
M Lib/idlelib/browser.py
M Lib/idlelib/idle_test/test_browser.py
diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt
index e1bc009ae933..b260d4e5ef1b 100644
--- a/Lib/idlelib/NEWS.txt
+++ b/Lib/idlelib/NEWS.txt
@@ -3,6 +3,8 @@ Released on 2019-10-20?
======================================
+bpo-32411: Stop sorting dict created with desired line order.
+
bpo-37038: Make idlelib.run runnable; add test clause.
bpo-36958: Print any argument other than None or int passed to
diff --git a/Lib/idlelib/browser.py b/Lib/idlelib/browser.py
index 234883fe8605..e5b0bc53c662 100644
--- a/Lib/idlelib/browser.py
+++ b/Lib/idlelib/browser.py
@@ -29,9 +29,10 @@ def transform_children(child_dict, modname=None):
The dictionary maps names to pyclbr information objects.
Filter out imported objects.
Augment class names with bases.
- Sort objects by line number.
+ The insertion order of the dictonary is assumed to have been in line
+ number order, so sorting is not necessary.
- The current tree only calls this once per child_dic as it saves
+ The current tree only calls this once per child_dict as it saves
TreeItems once created. A future tree and tests might violate this,
so a check prevents multiple in-place augmentations.
"""
@@ -51,7 +52,7 @@ def transform_children(child_dict, modname=None):
supers.append(sname)
obj.name += '({})'.format(', '.join(supers))
obs.append(obj)
- return sorted(obs, key=lambda o: o.lineno)
+ return obs
class ModuleBrowser:
diff --git a/Lib/idlelib/idle_test/test_browser.py b/Lib/idlelib/idle_test/test_browser.py
index dfbab6dd6b5e..25d6dc663036 100644
--- a/Lib/idlelib/idle_test/test_browser.py
+++ b/Lib/idlelib/idle_test/test_browser.py
@@ -61,16 +61,16 @@ def test_close(self):
# Nested tree same as in test_pyclbr.py except for supers on C0. C1.
mb = pyclbr
module, fname = 'test', 'test.py'
-f0 = mb.Function(module, 'f0', fname, 1)
-f1 = mb._nest_function(f0, 'f1', 2)
-f2 = mb._nest_function(f1, 'f2', 3)
-c1 = mb._nest_class(f0, 'c1', 5)
-C0 = mb.Class(module, 'C0', ['base'], fname, 6)
-F1 = mb._nest_function(C0, 'F1', 8)
-C1 = mb._nest_class(C0, 'C1', 11, [''])
-C2 = mb._nest_class(C1, 'C2', 12)
-F3 = mb._nest_function(C2, 'F3', 14)
-mock_pyclbr_tree = {'f0': f0, 'C0': C0}
+C0 = mb.Class(module, 'C0', ['base'], fname, 1)
+F1 = mb._nest_function(C0, 'F1', 3)
+C1 = mb._nest_class(C0, 'C1', 6, [''])
+C2 = mb._nest_class(C1, 'C2', 7)
+F3 = mb._nest_function(C2, 'F3', 9)
+f0 = mb.Function(module, 'f0', fname, 11)
+f1 = mb._nest_function(f0, 'f1', 12)
+f2 = mb._nest_function(f1, 'f2', 13)
+c1 = mb._nest_class(f0, 'c1', 15)
+mock_pyclbr_tree = {'C0': C0, 'f0': f0}
# Adjust C0.name, C1.name so tests do not depend on order.
browser.transform_children(mock_pyclbr_tree, 'test') # C0(base)
@@ -87,12 +87,12 @@ def test_transform_module_children(self):
transform = browser.transform_children
# Parameter matches tree module.
tcl = list(transform(mock_pyclbr_tree, 'test'))
- eq(tcl, [f0, C0])
- eq(tcl[0].name, 'f0')
- eq(tcl[1].name, 'C0(base)')
+ eq(tcl, [C0, f0])
+ eq(tcl[0].name, 'C0(base)')
+ eq(tcl[1].name, 'f0')
# Check that second call does not change suffix.
tcl = list(transform(mock_pyclbr_tree, 'test'))
- eq(tcl[1].name, 'C0(base)')
+ eq(tcl[0].name, 'C0(base)')
# Nothing to traverse if parameter name isn't same as tree module.
tcl = list(transform(mock_pyclbr_tree, 'different name'))
eq(tcl, [])
diff --git a/Misc/NEWS.d/next/IDLE/2017-12-25-18-48-50.bpo-32411.vNwDhe.rst b/Misc/NEWS.d/next/IDLE/2017-12-25-18-48-50.bpo-32411.vNwDhe.rst
new file mode 100644
index 000000000000..a5522012923a
--- /dev/null
+++ b/Misc/NEWS.d/next/IDLE/2017-12-25-18-48-50.bpo-32411.vNwDhe.rst
@@ -0,0 +1,2 @@
+In browser.py, remove extraneous sorting by line number since dictionary was
+created in line number order.
1
0