+cc: cython-devel background reading for those new to the thread: https://mail.python.org/archives/list/python-dev@python.org/thread/ZWTBR5ESY... On Tue, Aug 17, 2021 at 9:47 AM Victor Stinner <vstinner@python.org> wrote:
Since Cython is a common consumer of this C API, can somone please dig into Cython to see exactly what it needs in terms of API? How does Cython create all arguments of the __Pyx_PyCode_New() macro? Does it copy an existing function to only override some fields, something like CodeType.replace(field=new_value)?
If possible, I would prefer that Cython only uses the *public* C API. Otherwise, it will be very likely that Cython will break at every single Python release. Cython has a small team to maintain the code base, whereas CPython evolves much faster with a larger team.
Victor
I don't claim knowledge of Cython internals, but the two places it appears to call it's __Pyx_PyCode_New macro are: https://github.com/cython/cython/blob/master/Cython/Utility/Exceptions.c#L76... in __Pyx_CreateCodeObjectForTraceback() - this one already has a `#if CYTHON_COMPILING_IN_LIMITED_API` code path option in it. and https://github.com/cython/cython/blob/master/Cython/Compiler/ExprNodes.py#L9... in CodeObjectNode.generate_result_code() that creates PyCodeObject's for CyFunction instances per its comment. Slightly described in this comment http://google3/third_party/py/cython/files/Cython/Compiler/ExprNodes.py?l=39.... I don't see anything obvious mentioning the limited API in that code generator. it'd be best to loop in Cython maintainers for more of an idea of Cython's intents and needs with PyCode_New APIs. I've cc'd cython-devel@python.org. -Greg
On Tue, Aug 17, 2021 at 8:51 AM Gregory P. Smith <greg@krypto.org> wrote:
Doing a search of a huge codebase (work), the predominant user of
PyCode_New* APIs appears to be checked in Cython generated code (in all sorts of third_party OSS projects). It's in the boilerplate that Cython extensions make use of via it's __Pyx_PyCode_New macro. https://github.com/cython/cython/blob/master/Cython/Utility/ModuleSetupCode....
I saw very few non-Cython uses. There are some, but at a very quick
first glance they appear simple - easy enough to reach out to the projects with a PR to update their code.
The Cython use will require people to upgrade Cython and regenerate
their code before they can use the Python version that changes these. That is not an uncommon thing for Cython. It's unfortunate that many projects on ship generated sources rather than use Cython at build time, but that isn't _our_ problem to solve. The more often we change internal APIs that things depend on, the more people will move their projects towards doing the right thing with regards to either not using said APIs or rerunning an up to date code generator as part of their build instead of checking in generated unstable API using sources.
-gps
On Mon, Aug 16, 2021 at 8:04 PM Guido van Rossum <guido@python.org>
On Mon, Aug 16, 2021 at 4:44 PM Nick Coghlan <ncoghlan@gmail.com>
wrote:
[...] A cloning-with-replacement API that accepted the base code object and
Yes (I forgot to mention that).
Moving actual "from scratch" code object creation behind the
Py_BUILD_CORE guard with an underscore prefix on the name would also make sense, since it defines a key piece of the compiler/interpreter boundary.
Yeah, we have _PyCode_New() for that.
Cheers, Nick.
P.S. Noting an idea that won't work, in case anyone else reading the
That's the API style that _PyCode_New() uses (thanks to Eric who IIRC
wrote: the "safe to modify" fields could be a good complement to the API deprecation proposal. thread was thinking the same thing: a "PyType_FromSpec" style API won't help here, as the issue is that the compiler is now doing more work up front and recording that extra info in the code object for the interpreter to use. There is no way to synthesise that info if it isn't passed to the constructor, as it isn't intrinsically recorded in the opcode sequence. pushed for this and implemented it). You gave me an idea now: the C equivalent to .replace() could use the same input structure; one can leave fields NULL that should be copied from the original unmodified.
-- --Guido van Rossum (python.org/~guido) Pronouns: he/him (why is my pronoun here?) _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/NWYMCDAM...
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/67DMIW7N... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.