Mailman 3 July 2023 - Python-checkins

[3.12] gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) (gh-107522)
by ericsnowcurrently 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/fc4532a55d23887bae49350d2f939c597d… commit: fc4532a55d23887bae49350d2f939c597d6b5b98 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ericsnowcurrently <ericsnowcurrently(a)gmail.com> date: 2023-07-31T17:25:18-06:00 summary: [3.12] gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) (gh-107522) gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) (cherry picked from commit db361a340af3970c279908c8746a6b9ed45f47b8) Co-authored-by: Eric Snow <ericsnowcurrently(a)gmail.com> files: M Doc/c-api/memory.rst M Doc/whatsnew/3.12.rst diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 8968b26b64320..1df8c2b911ca8 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -476,6 +476,10 @@ Customize Memory Allocators thread-safe: the :term:`GIL <global interpreter lock>` is not held when the allocator is called. + For the remaining domains, the allocator must also be thread-safe: + the allocator may be called in different interpreters that do not + share a ``GIL``. + If the new allocator is not a hook (does not call the previous allocator), the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the debug hooks on top on the new allocator. @@ -500,6 +504,8 @@ Customize Memory Allocators **must** wrap the existing allocator. Substituting the current allocator for some other arbitrary one is **not supported**. + .. versionchanged:: 3.12 + All allocators must be thread-safe. .. c:function:: void PyMem_SetupDebugHooks(void) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index cbf47e4ad835d..39e702bb012fb 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1922,6 +1922,13 @@ Porting to Python 3.12 * :c:func:`PyUnstable_Long_IsCompact` * :c:func:`PyUnstable_Long_CompactValue` +* Custom allocators, set via :c:func:`PyMem_SetAllocator`, are now + required to be thread-safe, regardless of memory domain. Allocators + that don't have their own state, including "hooks", are not affected. + If your custom allocator is not already thread-safe and you need + guidance then please create a new GitHub issue + and CC ``@ericsnowcurrently``. + Deprecated ----------

1 0

[3.12] gh-99113: Add a What's New Entry for PEP 684 (gh-107520) (gh-107523)
by ericsnowcurrently 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/31cd12ab21b305f19732372634cb50940f… commit: 31cd12ab21b305f19732372634cb50940f353e6e branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ericsnowcurrently <ericsnowcurrently(a)gmail.com> date: 2023-07-31T17:24:45-06:00 summary: [3.12] gh-99113: Add a What's New Entry for PEP 684 (gh-107520) (gh-107523) gh-99113: Add a What's New Entry for PEP 684 (gh-107520) (cherry picked from commit 79e479c45fc63b6001b206fec832064c31fc1f11) Co-authored-by: Eric Snow <ericsnowcurrently(a)gmail.com> files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index d7389f696ea2e..cbf47e4ad835d 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -70,6 +70,10 @@ New grammar features: * :pep:`701`: Syntactic formalization of f-strings +Interpreter improvements: + +* :ref:`whatsnew312-pep684` + New typing features: * :pep:`688`: Making the buffer protocol accessible in Python @@ -277,6 +281,36 @@ The new :class:`inspect.BufferFlags` enum represents the flags that can be used to customize buffer creation. (Contributed by Jelle Zijlstra in :gh:`102500`.) +.. _whatsnew312-pep684: + +PEP 684: A Per-Interpreter GIL +------------------------------ + +Sub-interpreters may now be created with a unique GIL per interpreter. +This allows Python programs to take full advantage of multiple CPU +cores. + +Use the new :c:func:`Py_NewInterpreterFromConfig` function to +create an interpreter with its own GIL:: + + PyInterpreterConfig config = { + .check_multi_interp_extensions = 1, + .gil = PyInterpreterConfig_OWN_GIL, + }; + PyThreadState *tstate = NULL; + PyStatus status = Py_NewInterpreterFromConfig(&tstate, &config); + if (PyStatus_Exception(status)) { + return -1; + } + /* The new interpeter is now active in the current thread. */ + +For further examples how to use the C-API for sub-interpreters with a +per-interpreter GIL, see :source:`Modules/_xxsubinterpretersmodule.c`. + +A Python API is anticipated for 3.13. (See :pep:`554`.) + +(Contributed by Eric Snow in :gh:`104210`, etc.) + New Features Related to Type Hints ================================== @@ -1744,6 +1778,12 @@ New Features (Contributed by Eddie Elizondo in :gh:`84436`.) +* :pep:`684`: Added the new :c:func:`Py_NewInterpreterFromConfig` + function and :c:type:`PyInterpreterConfig`, which may be used + to create sub-interpreters with their own GILs. + (See :ref:`whatsnew312-pep684` for more info.) + (Contributed by Eric Snow in :gh:`104110`.) + * In the limited C API version 3.12, :c:func:`Py_INCREF` and :c:func:`Py_DECREF` functions are now implemented as opaque function calls to hide implementation details.

1 0

[3.12] gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (GH-107403) (gh-107521)
by ericsnowcurrently 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/3f6afafa7e7294786913dbecf79fe42c8b… commit: 3f6afafa7e7294786913dbecf79fe42c8b510f85 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ericsnowcurrently <ericsnowcurrently(a)gmail.com> date: 2023-07-31T23:15:13Z summary: [3.12] gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (GH-107403) (gh-107521) gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (GH-107403) It was added in 3.12 for PEP 684 (per-interpreter GIL). (cherry picked from commit fb344e99aa0da5bef9318684ade69978585fe060) Co-authored-by: Eric Snow <ericsnowcurrently(a)gmail.com> files: M Doc/c-api/init.rst M Doc/c-api/module.rst M Doc/howto/isolating-extensions.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 52824b6d88afd..8b960b06871b1 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1555,6 +1555,7 @@ function. You can create and destroy them using the following functions: in any thread where the sub-interpreter is currently active. Otherwise only multi-phase init extension modules (see :pep:`489`) may be imported. + (Also see :c:macro:`Py_mod_multiple_interpreters`.) This must be ``1`` (non-zero) if :c:member:`~PyInterpreterConfig.use_main_obmalloc` is ``0``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 8ca48c852d4e6..2b6875d533c82 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -376,6 +376,37 @@ The available slot types are: If multiple ``Py_mod_exec`` slots are specified, they are processed in the order they appear in the *m_slots* array. +.. c:macro:: Py_mod_multiple_interpreters + + Specifies one of the following values: + + .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED + + The module does not support being imported in subinterpreters. + + .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED + + The module supports being imported in subinterpreters, + but only when they share the main interpreter's GIL. + (See :ref:`isolating-extensions-howto`.) + + .. c:macro:: Py_MOD_PER_INTERPRETER_GIL_SUPPORTED + + The module supports being imported in subinterpreters, + even when they have their own GIL. + (See :ref:`isolating-extensions-howto`.) + + This slot determines whether or not importing this module + in a subinterpreter will fail. + + Multiple ``Py_mod_multiple_interpreters`` slots may not be specified + in one module definition. + + If ``Py_mod_multiple_interpreters`` is not specified, the import + machinery defaults to ``Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED``. + + .. versionadded:: 3.12 + See :PEP:`489` for more details on multi-phase initialization. Low-level module creation functions diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index 60854c3c034d7..2551fbe87b5c2 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -1,5 +1,7 @@ .. highlight:: c +.. _isolating-extensions-howto: + *************************** Isolating Extension Modules ***************************

1 0

gh-99113: Add a What's New Entry for PEP 684 (gh-107520)
by ericsnowcurrently 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/79e479c45fc63b6001b206fec832064c31… commit: 79e479c45fc63b6001b206fec832064c31fc1f11 branch: main author: Eric Snow <ericsnowcurrently(a)gmail.com> committer: ericsnowcurrently <ericsnowcurrently(a)gmail.com> date: 2023-07-31T23:13:55Z summary: gh-99113: Add a What's New Entry for PEP 684 (gh-107520) files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index a5cb325cedb22..3c39476ced9f4 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -69,6 +69,10 @@ New grammar features: * :pep:`701`: Syntactic formalization of f-strings +Interpreter improvements: + +* :ref:`whatsnew312-pep684` + New typing features: * :pep:`688`: Making the buffer protocol accessible in Python @@ -276,6 +280,36 @@ The new :class:`inspect.BufferFlags` enum represents the flags that can be used to customize buffer creation. (Contributed by Jelle Zijlstra in :gh:`102500`.) +.. _whatsnew312-pep684: + +PEP 684: A Per-Interpreter GIL +------------------------------ + +Sub-interpreters may now be created with a unique GIL per interpreter. +This allows Python programs to take full advantage of multiple CPU +cores. + +Use the new :c:func:`Py_NewInterpreterFromConfig` function to +create an interpreter with its own GIL:: + + PyInterpreterConfig config = { + .check_multi_interp_extensions = 1, + .gil = PyInterpreterConfig_OWN_GIL, + }; + PyThreadState *tstate = NULL; + PyStatus status = Py_NewInterpreterFromConfig(&tstate, &config); + if (PyStatus_Exception(status)) { + return -1; + } + /* The new interpeter is now active in the current thread. */ + +For further examples how to use the C-API for sub-interpreters with a +per-interpreter GIL, see :source:`Modules/_xxsubinterpretersmodule.c`. + +A Python API is anticipated for 3.13. (See :pep:`554`.) + +(Contributed by Eric Snow in :gh:`104210`, etc.) + New Features Related to Type Hints ================================== @@ -1742,6 +1776,12 @@ New Features (Contributed by Eddie Elizondo in :gh:`84436`.) +* :pep:`684`: Added the new :c:func:`Py_NewInterpreterFromConfig` + function and :c:type:`PyInterpreterConfig`, which may be used + to create sub-interpreters with their own GILs. + (See :ref:`whatsnew312-pep684` for more info.) + (Contributed by Eric Snow in :gh:`104110`.) + * In the limited C API version 3.12, :c:func:`Py_INCREF` and :c:func:`Py_DECREF` functions are now implemented as opaque function calls to hide implementation details.

1 0

gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519)
by ericsnowcurrently 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/db361a340af3970c279908c8746a6b9ed4… commit: db361a340af3970c279908c8746a6b9ed45f47b8 branch: main author: Eric Snow <ericsnowcurrently(a)gmail.com> committer: ericsnowcurrently <ericsnowcurrently(a)gmail.com> date: 2023-07-31T23:11:15Z summary: gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) files: M Doc/c-api/memory.rst M Doc/whatsnew/3.12.rst diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 8968b26b64320..1df8c2b911ca8 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -476,6 +476,10 @@ Customize Memory Allocators thread-safe: the :term:`GIL <global interpreter lock>` is not held when the allocator is called. + For the remaining domains, the allocator must also be thread-safe: + the allocator may be called in different interpreters that do not + share a ``GIL``. + If the new allocator is not a hook (does not call the previous allocator), the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the debug hooks on top on the new allocator. @@ -500,6 +504,8 @@ Customize Memory Allocators **must** wrap the existing allocator. Substituting the current allocator for some other arbitrary one is **not supported**. + .. versionchanged:: 3.12 + All allocators must be thread-safe. .. c:function:: void PyMem_SetupDebugHooks(void) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 0ac4a3df8fefa..a5cb325cedb22 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1880,6 +1880,13 @@ Porting to Python 3.12 * :c:func:`PyUnstable_Long_IsCompact` * :c:func:`PyUnstable_Long_CompactValue` +* Custom allocators, set via :c:func:`PyMem_SetAllocator`, are now + required to be thread-safe, regardless of memory domain. Allocators + that don't have their own state, including "hooks", are not affected. + If your custom allocator is not already thread-safe and you need + guidance then please create a new GitHub issue + and CC ``@ericsnowcurrently``. + Deprecated ----------

1 0

gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (#107403)
by ericsnowcurrently 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/fb344e99aa0da5bef9318684ade6997858… commit: fb344e99aa0da5bef9318684ade69978585fe060 branch: main author: Eric Snow <ericsnowcurrently(a)gmail.com> committer: ericsnowcurrently <ericsnowcurrently(a)gmail.com> date: 2023-07-31T17:03:01-06:00 summary: gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (#107403) It was added in 3.12 for PEP 684 (per-interpreter GIL). files: M Doc/c-api/init.rst M Doc/c-api/module.rst M Doc/howto/isolating-extensions.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 60912a1ecaede..dd53fe2bc9569 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1287,6 +1287,7 @@ function. You can create and destroy them using the following functions: in any thread where the sub-interpreter is currently active. Otherwise only multi-phase init extension modules (see :pep:`489`) may be imported. + (Also see :c:macro:`Py_mod_multiple_interpreters`.) This must be ``1`` (non-zero) if :c:member:`~PyInterpreterConfig.use_main_obmalloc` is ``0``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index fa5e97846e7bc..e37505f6450a3 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -376,6 +376,37 @@ The available slot types are: If multiple ``Py_mod_exec`` slots are specified, they are processed in the order they appear in the *m_slots* array. +.. c:macro:: Py_mod_multiple_interpreters + + Specifies one of the following values: + + .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED + + The module does not support being imported in subinterpreters. + + .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED + + The module supports being imported in subinterpreters, + but only when they share the main interpreter's GIL. + (See :ref:`isolating-extensions-howto`.) + + .. c:macro:: Py_MOD_PER_INTERPRETER_GIL_SUPPORTED + + The module supports being imported in subinterpreters, + even when they have their own GIL. + (See :ref:`isolating-extensions-howto`.) + + This slot determines whether or not importing this module + in a subinterpreter will fail. + + Multiple ``Py_mod_multiple_interpreters`` slots may not be specified + in one module definition. + + If ``Py_mod_multiple_interpreters`` is not specified, the import + machinery defaults to ``Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED``. + + .. versionadded:: 3.12 + See :PEP:`489` for more details on multi-phase initialization. Low-level module creation functions diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index 60854c3c034d7..2551fbe87b5c2 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -1,5 +1,7 @@ .. highlight:: c +.. _isolating-extensions-howto: + *************************** Isolating Extension Modules ***************************

1 0

[3.11] gh-107507: Replace 'The goals of Argument Clinic' with a summary (GH-107508) (#107517)
by erlend-aasland 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/3be07c98b3e22601a5837ce25c04803ba6… commit: 3be07c98b3e22601a5837ce25c04803ba63a1aa6 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2023-07-31T21:43:16Z summary: [3.11] gh-107507: Replace 'The goals of Argument Clinic' with a summary (GH-107508) (#107517) Summarise the goals of Argument Clinic in a single sentence. Mention that Argument Clinic was introduced with PEP-436. (cherry picked from commit abb71c6a8f73482c910ffdf050a86089a48e0e60) Co-authored-by: Erlend E. Aasland <erlend(a)python.org> files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 1ec8984391bbe..913a3df7e3c48 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,8 +13,10 @@ Argument Clinic How-To .. topic:: Abstract Argument Clinic is a preprocessor for CPython C files. - Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins", + It was introduced in Python 3.4 with :pep:`436`, + in order to provide introspection signatures, + and to generate performant and tailor-made boilerplate code + for argument parsing in CPython builtins, module level functions, and class methods. This document is divided in four major sections: @@ -44,58 +46,6 @@ Argument Clinic How-To Background ========== - -The goals of Argument Clinic ----------------------------- - -Argument Clinic's primary goal -is to take over responsibility for all argument parsing code -inside CPython. This means that, when you convert a function -to work with Argument Clinic, that function should no longer -do any of its own argument parsing—the code generated by -Argument Clinic should be a "black box" to you, where CPython -calls in at the top, and your code gets called at the bottom, -with ``PyObject *args`` (and maybe ``PyObject *kwargs``) -magically converted into the C variables and types you need. - -In order for Argument Clinic to accomplish its primary goal, -it must be easy to use. Currently, working with CPython's -argument parsing library is a chore, requiring maintaining -redundant information in a surprising number of places. -When you use Argument Clinic, you don't have to repeat yourself. - -Obviously, no one would want to use Argument Clinic unless -it's solving their problem—and without creating new problems of -its own. -So it's paramount that Argument Clinic generate correct code. -It'd be nice if the code was faster, too, but at the very least -it should not introduce a major speed regression. (Eventually Argument -Clinic *should* make a major speedup possible—we could -rewrite its code generator to produce tailor-made argument -parsing code, rather than calling the general-purpose CPython -argument parsing library. That would make for the fastest -argument parsing possible!) - -Additionally, Argument Clinic must be flexible enough to -work with any approach to argument parsing. Python has -some functions with some very strange parsing behaviors; -Argument Clinic's goal is to support all of them. - -Finally, the original motivation for Argument Clinic was -to provide introspection "signatures" for CPython builtins. -It used to be, the introspection query functions would throw -an exception if you passed in a builtin. With Argument -Clinic, that's a thing of the past! - -One idea you should keep in mind, as you work with -Argument Clinic: the more information you give it, the -better job it'll be able to do. -Argument Clinic is admittedly relatively simple right -now. But as it evolves it will get more sophisticated, -and it should be able to do many interesting and smart -things with all the information you give it. - - Basic concepts --------------

1 0

[3.12] gh-107507: Replace 'The goals of Argument Clinic' with a summary (GH-107508) (#107516)
by erlend-aasland 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/fd6085529935afec085cea0729507ba6d4… commit: fd6085529935afec085cea0729507ba6d4b34ab3 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2023-07-31T21:42:58Z summary: [3.12] gh-107507: Replace 'The goals of Argument Clinic' with a summary (GH-107508) (#107516) Summarise the goals of Argument Clinic in a single sentence. Mention that Argument Clinic was introduced with PEP-436. (cherry picked from commit abb71c6a8f73482c910ffdf050a86089a48e0e60) Co-authored-by: Erlend E. Aasland <erlend(a)python.org> files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 7aafd48711b58..9c9a4f45dd0f5 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,8 +13,10 @@ Argument Clinic How-To .. topic:: Abstract Argument Clinic is a preprocessor for CPython C files. - Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins", + It was introduced in Python 3.4 with :pep:`436`, + in order to provide introspection signatures, + and to generate performant and tailor-made boilerplate code + for argument parsing in CPython builtins, module level functions, and class methods. This document is divided in four major sections: @@ -44,58 +46,6 @@ Argument Clinic How-To Background ========== - -The goals of Argument Clinic ----------------------------- - -Argument Clinic's primary goal -is to take over responsibility for all argument parsing code -inside CPython. This means that, when you convert a function -to work with Argument Clinic, that function should no longer -do any of its own argument parsing—the code generated by -Argument Clinic should be a "black box" to you, where CPython -calls in at the top, and your code gets called at the bottom, -with ``PyObject *args`` (and maybe ``PyObject *kwargs``) -magically converted into the C variables and types you need. - -In order for Argument Clinic to accomplish its primary goal, -it must be easy to use. Currently, working with CPython's -argument parsing library is a chore, requiring maintaining -redundant information in a surprising number of places. -When you use Argument Clinic, you don't have to repeat yourself. - -Obviously, no one would want to use Argument Clinic unless -it's solving their problem—and without creating new problems of -its own. -So it's paramount that Argument Clinic generate correct code. -It'd be nice if the code was faster, too, but at the very least -it should not introduce a major speed regression. (Eventually Argument -Clinic *should* make a major speedup possible—we could -rewrite its code generator to produce tailor-made argument -parsing code, rather than calling the general-purpose CPython -argument parsing library. That would make for the fastest -argument parsing possible!) - -Additionally, Argument Clinic must be flexible enough to -work with any approach to argument parsing. Python has -some functions with some very strange parsing behaviors; -Argument Clinic's goal is to support all of them. - -Finally, the original motivation for Argument Clinic was -to provide introspection "signatures" for CPython builtins. -It used to be, the introspection query functions would throw -an exception if you passed in a builtin. With Argument -Clinic, that's a thing of the past! - -One idea you should keep in mind, as you work with -Argument Clinic: the more information you give it, the -better job it'll be able to do. -Argument Clinic is admittedly relatively simple right -now. But as it evolves it will get more sophisticated, -and it should be able to do many interesting and smart -things with all the information you give it. - - Basic concepts --------------

1 0

gh-107507: Replace 'The goals of Argument Clinic' with a summary (#107508)
by erlend-aasland 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/abb71c6a8f73482c910ffdf050a86089a4… commit: abb71c6a8f73482c910ffdf050a86089a48e0e60 branch: main author: Erlend E. Aasland <erlend(a)python.org> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2023-07-31T23:33:13+02:00 summary: gh-107507: Replace 'The goals of Argument Clinic' with a summary (#107508) Summarise the goals of Argument Clinic in a single sentence. Mention that Argument Clinic was introduced with PEP-436. files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 7aafd48711b58..9c9a4f45dd0f5 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,8 +13,10 @@ Argument Clinic How-To .. topic:: Abstract Argument Clinic is a preprocessor for CPython C files. - Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins", + It was introduced in Python 3.4 with :pep:`436`, + in order to provide introspection signatures, + and to generate performant and tailor-made boilerplate code + for argument parsing in CPython builtins, module level functions, and class methods. This document is divided in four major sections: @@ -44,58 +46,6 @@ Argument Clinic How-To Background ========== - -The goals of Argument Clinic ----------------------------- - -Argument Clinic's primary goal -is to take over responsibility for all argument parsing code -inside CPython. This means that, when you convert a function -to work with Argument Clinic, that function should no longer -do any of its own argument parsing—the code generated by -Argument Clinic should be a "black box" to you, where CPython -calls in at the top, and your code gets called at the bottom, -with ``PyObject *args`` (and maybe ``PyObject *kwargs``) -magically converted into the C variables and types you need. - -In order for Argument Clinic to accomplish its primary goal, -it must be easy to use. Currently, working with CPython's -argument parsing library is a chore, requiring maintaining -redundant information in a surprising number of places. -When you use Argument Clinic, you don't have to repeat yourself. - -Obviously, no one would want to use Argument Clinic unless -it's solving their problem—and without creating new problems of -its own. -So it's paramount that Argument Clinic generate correct code. -It'd be nice if the code was faster, too, but at the very least -it should not introduce a major speed regression. (Eventually Argument -Clinic *should* make a major speedup possible—we could -rewrite its code generator to produce tailor-made argument -parsing code, rather than calling the general-purpose CPython -argument parsing library. That would make for the fastest -argument parsing possible!) - -Additionally, Argument Clinic must be flexible enough to -work with any approach to argument parsing. Python has -some functions with some very strange parsing behaviors; -Argument Clinic's goal is to support all of them. - -Finally, the original motivation for Argument Clinic was -to provide introspection "signatures" for CPython builtins. -It used to be, the introspection query functions would throw -an exception if you passed in a builtin. With Argument -Clinic, that's a thing of the past! - -One idea you should keep in mind, as you work with -Argument Clinic: the more information you give it, the -better job it'll be able to do. -Argument Clinic is admittedly relatively simple right -now. But as it evolves it will get more sophisticated, -and it should be able to do many interesting and smart -things with all the information you give it. - - Basic concepts --------------

1 0

Use tstate->interp to get the interpreter state in bytecodes.c (GH-107506)
by brandtbucher 31 Jul '23

31 Jul '23

https://github.com/python/cpython/commit/dfb55d9d7f9349582a2077f87e95c8f9d5… commit: dfb55d9d7f9349582a2077f87e95c8f9d5e2ebd4 branch: main author: Brandt Bucher <brandtbucher(a)microsoft.com> committer: brandtbucher <brandtbucher(a)gmail.com> date: 2023-07-31T14:18:38-07:00 summary: Use tstate->interp to get the interpreter state in bytecodes.c (GH-107506) files: M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 7320c309884ac..0bea7b59597a2 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -3245,7 +3245,7 @@ dummy_func( total_args++; } DEOPT_IF(total_args != 1, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.len, CALL); STAT_INC(CALL, hit); PyObject *arg = args[0]; @@ -3272,7 +3272,7 @@ dummy_func( total_args++; } DEOPT_IF(total_args != 2, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.isinstance, CALL); STAT_INC(CALL, hit); PyObject *cls = args[1]; @@ -3295,7 +3295,7 @@ dummy_func( ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); assert(method != NULL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(method != interp->callable_cache.list_append, CALL); DEOPT_IF(!PyList_Check(self), CALL); STAT_INC(CALL, hit); diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index c2f0d0a77b16d..ed7cf4c15ec4a 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -2154,7 +2154,7 @@ total_args++; } DEOPT_IF(total_args != 1, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.len, CALL); STAT_INC(CALL, hit); PyObject *arg = args[0]; @@ -2189,7 +2189,7 @@ total_args++; } DEOPT_IF(total_args != 2, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.isinstance, CALL); STAT_INC(CALL, hit); PyObject *cls = args[1]; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index dccd6da912376..e43b3dedf14f5 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3984,7 +3984,7 @@ total_args++; } DEOPT_IF(total_args != 1, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.len, CALL); STAT_INC(CALL, hit); PyObject *arg = args[0]; @@ -4020,7 +4020,7 @@ total_args++; } DEOPT_IF(total_args != 2, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.isinstance, CALL); STAT_INC(CALL, hit); PyObject *cls = args[1]; @@ -4050,7 +4050,7 @@ ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); assert(method != NULL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(method != interp->callable_cache.list_append, CALL); DEOPT_IF(!PyList_Check(self), CALL); STAT_INC(CALL, hit);

1 0