https://github.com/python/cpython/commit/dc10264eb880ed63fcf42c17057f3f5d879... commit: dc10264eb880ed63fcf42c17057f3f5d879a0a0c branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> committer: encukou <encukou@gmail.com> date: 2021-06-28T11:35:52+02:00 summary: bpo-40939: Remove documentation for `PyParser_*` & add porting notes (GH-26855) (GH-26898) I tried to be relatively thorough and give lots of links. One reason is that this wasn't deprecated very long; also it seems people running into this tend to not be familiar with similar APIs. (cherry picked from commit 29987f72650b7cccee4df216c8297e8484a44e6a) Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Co-authored-by: Petr Viktorin <encukou@gmail.com> files: A Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst M Doc/c-api/veryhigh.rst M Doc/data/refcounts.dat M Doc/faq/extending.rst M Doc/whatsnew/3.10.rst diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 0f760eaa7ad578..3354a2b976f343 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -185,42 +185,6 @@ the same library that the Python runtime is using. :c:func:`PyMem_RawRealloc`, instead of being allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. - -.. c:function:: struct _node* PyParser_SimpleParseString(const char *str, int start) - - This is a simplified interface to - :c:func:`PyParser_SimpleParseStringFlagsFilename` below, leaving *filename* set - to ``NULL`` and *flags* set to ``0``. - - -.. c:function:: struct _node* PyParser_SimpleParseStringFlags( const char *str, int start, int flags) - - This is a simplified interface to - :c:func:`PyParser_SimpleParseStringFlagsFilename` below, leaving *filename* set - to ``NULL``. - - -.. c:function:: struct _node* PyParser_SimpleParseStringFlagsFilename( const char *str, const char *filename, int start, int flags) - - Parse Python source code from *str* using the start token *start* according to - the *flags* argument. The result can be used to create a code object which can - be evaluated efficiently. This is useful if a code fragment must be evaluated - many times. *filename* is decoded from the :term:`filesystem encoding and - error handler`. - - -.. c:function:: struct _node* PyParser_SimpleParseFile(FILE *fp, const char *filename, int start) - - This is a simplified interface to :c:func:`PyParser_SimpleParseFileFlags` below, - leaving *flags* set to ``0``. - - -.. c:function:: struct _node* PyParser_SimpleParseFileFlags(FILE *fp, const char *filename, int start, int flags) - - Similar to :c:func:`PyParser_SimpleParseStringFlagsFilename`, but the Python - source code is read from *fp* instead of an in-memory string. - - .. c:function:: PyObject* PyRun_String(const char *str, int start, PyObject *globals, PyObject *locals) This is a simplified interface to :c:func:`PyRun_StringFlags` below, leaving diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat index 505f1203dd1bdd..29cd89efc6fbf7 100644 --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -1791,32 +1791,6 @@ PyObject_TypeCheck:int::: PyObject_TypeCheck:PyObject*:o:0: PyObject_TypeCheck:PyTypeObject*:type:0: -PyParser_SimpleParseFile:struct _node*::: -PyParser_SimpleParseFile:FILE*:fp:: -PyParser_SimpleParseFile:const char*:filename:: -PyParser_SimpleParseFile:int:start:: - -PyParser_SimpleParseFileFlags:struct _node*::: -PyParser_SimpleParseFileFlags:FILE*:fp:: -PyParser_SimpleParseFileFlags:const char*:filename:: -PyParser_SimpleParseFileFlags:int:start:: -PyParser_SimpleParseFileFlags:int:flags:: - -PyParser_SimpleParseString:struct _node*::: -PyParser_SimpleParseString:const char*:str:: -PyParser_SimpleParseString:int:start:: - -PyParser_SimpleParseStringFlags:struct _node*::: -PyParser_SimpleParseStringFlags:const char*:str:: -PyParser_SimpleParseStringFlags:int:start:: -PyParser_SimpleParseStringFlags:int:flags:: - -PyParser_SimpleParseStringFlagsFilename:struct _node*::: -PyParser_SimpleParseStringFlagsFilename:const char*:str:: -PyParser_SimpleParseStringFlagsFilename:const char*:filename:: -PyParser_SimpleParseStringFlagsFilename:int:start:: -PyParser_SimpleParseStringFlagsFilename:int:flags:: - PyRun_AnyFile:int::: PyRun_AnyFile:FILE*:fp:: PyRun_AnyFile:const char*:filename:: diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst index aecb56eaa4fd2f..3379e41d9de072 100644 --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -275,39 +275,8 @@ for more hints. However sometimes you have to run the embedded Python interpreter in the same thread as your rest application and you can't allow the -:c:func:`PyRun_InteractiveLoop` to stop while waiting for user input. The one -solution then is to call :c:func:`PyParser_ParseString` and test for ``e.error`` -equal to ``E_EOF``, which means the input is incomplete. Here's a sample code -fragment, untested, inspired by code from Alex Farber:: - - #define PY_SSIZE_T_CLEAN - #include <Python.h> - #include <node.h> - #include <errcode.h> - #include <grammar.h> - #include <parsetok.h> - #include <compile.h> - - int testcomplete(char *code) - /* code should end in \n */ - /* return -1 for error, 0 for incomplete, 1 for complete */ - { - node *n; - perrdetail e; - - n = PyParser_ParseString(code, &_PyParser_Grammar, - Py_file_input, &e); - if (n == NULL) { - if (e.error == E_EOF) - return 0; - return -1; - } - - PyNode_Free(n); - return 1; - } - -Another solution is trying to compile the received string with +:c:func:`PyRun_InteractiveLoop` to stop while waiting for user input. +A solution is trying to compile the received string with :c:func:`Py_CompileString`. If it compiles without errors, try to execute the returned code object by calling :c:func:`PyEval_EvalCode`. Otherwise save the input for later. If the compilation fails, find out if it's an error or just diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 7ed80fb903efd1..5e510d8f6ed3ae 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1696,9 +1696,9 @@ Removed that were only being used by the old parser, including ``node.h``, ``parser.h``, ``graminit.h`` and ``grammar.h``. -* Removed the Public C API functions :c:func:`PyParser_SimpleParseStringFlags`, - :c:func:`PyParser_SimpleParseStringFlagsFilename`, - :c:func:`PyParser_SimpleParseFileFlags` and :c:func:`PyNode_Compile` +* Removed the Public C API functions ``PyParser_SimpleParseStringFlags``, + ``PyParser_SimpleParseStringFlagsFilename``, + ``PyParser_SimpleParseFileFlags`` and ``PyNode_Compile`` that were deprecated in 3.9 due to the switch to the new PEG parser. * Removed the ``formatter`` module, which was deprecated in Python 3.4. @@ -1812,6 +1812,41 @@ Changes in the Python API also inherits the current builtins. (Contributed by Victor Stinner in :issue:`42990`.) +Changes in the C API +-------------------- + +* The C API functions ``PyParser_SimpleParseStringFlags``, + ``PyParser_SimpleParseStringFlagsFilename``, + ``PyParser_SimpleParseFileFlags``, ``PyNode_Compile`` and the type + used by these functions, ``struct _node``, were removed due to the switch + to the new PEG parser. + + Source should be now be compiled directly to a code object using, for + example, :c:func:`Py_CompileString`. The resulting code object can then be + evaluated using, for example, :c:func:`PyEval_EvalCode`. + + Specifically: + + * A call to ``PyParser_SimpleParseStringFlags`` followed by + ``PyNode_Compile`` can be replaced by calling :c:func:`Py_CompileString`. + + * There is no direct replacement for ``PyParser_SimpleParseFileFlags``. + To compile code from a ``FILE *`` argument, you will need to read + the file in C and pass the resulting buffer to :c:func:`Py_CompileString`. + + * To compile a file given a ``char *`` filename, explicitly open the file, read + it and compile the result. One way to do this is using the :py:mod:`io` + module with :c:func:`PyImport_ImportModule`, :c:func:`PyObject_CallMethod`, + :c:func:`PyBytes_AsString` and :c:func:`Py_CompileString`, + as sketched below. (Declarations and error handling are omitted.) :: + + io_module = Import_ImportModule("io"); + fileobject = PyObject_CallMethod(io_module, "open", "ss", filename, "rb"); + source_bytes_object = PyObject_CallMethod(fileobject, "read", ""); + result = PyObject_CallMethod(fileobject, "close", ""); + source_buf = PyBytes_AsString(source_bytes_object); + code = Py_CompileString(source_buf, filename, Py_file_input); + CPython bytecode changes ======================== diff --git a/Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst b/Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst new file mode 100644 index 00000000000000..2531ac1ba3c12c --- /dev/null +++ b/Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst @@ -0,0 +1 @@ +Removed documentation for the removed ``PyParser_*`` C API.