On 2020-06-15 13:47, M.-A. Lemburg wrote:
On 15.06.2020 12:53, Matti Picus wrote:
On 6/15/20 12:58 PM, M.-A. Lemburg wrote:
Sure, there are often workarounds which can be used, but we do need some way to tell the interpreter "the initialization had a problem, please remove the object from your internal management resources without calling the deallocator".
Perhaps a new public API could be created for this purpose, like you say.
I respectfully disagree. There is purposefully a separation between __new__ and __init__. A failure in __new__ should not call __del__, and a failure in __init__ should be gracefully handled by the __del__. It is not the interpreter's duty to provide an interface that breaks this construct. A c-extension class should not get a pass to do things differently from a python-level class. It is important that CPython not provide interfaces that will make other implementation's lives harder (PyPy, GraalPython, MicroPython). Anyone using C-API functions that start with a "_" should be very thoughtful since those are not meant to be public, usually for good reason.
In many cases, the above will work and you can signal your deallocator which parts of the object have already been initialized, but this is not always possible.
Do you have a concrete example where tis is not possible?
"_Py_ForgetReference and PyObject_Del will do normal deallocation of the object *except* calling the user-provided deallocator" strikes me as a very fragile assumption. Where is the assumption is coming from? (That's what I meant to ask when I asked "What does the _Py_ForgetReference function do?" – apologies for wording it in a misleading/loaded way.)
The fact that extension writers turn to APIs marked as internal is usually triggered by lack of public APIs for the purpose. This should be a signal to consider making an API public or creating a better interface to provide similar logic.
Unfortunately, this doesn't really work as a signal, because nothing is actually signalled back to CPython devs – until the private API in question stops working.
FWIW: I don't see problems for other implementations to provide the above API, but perhaps I'm missing something.
I see no documentation on what the function should do. I assume providing a function with unspecified semantics is always a problem.