Hmm... how would this be different than defining a HPy handle to be the exact same thing?
I sort of think of a handle as a lightweight proxy object (just like used in cpyext) to the union type, thereby providing another indirection to the union type. In addition to normal allocations, handles are allocated and updated too. However, I have to say, practically the difference may be questionable, one of those things you cant truly bet on without actually trying.
The harder problem is probably when a C function wants to keep ownership of a PyObject for longer than a function call.
I have not looked at cpyext in depth but I will. Optionally, we can use a mechanism of "references". Typically have a function that gets a Python value from the stack and returns an integer reference for instance to it. The same reference can later be used to get the value. We can similarly destroy the references. This can ease our work to some extent.
For the shorter case, we need to find a way of creating dynamic scope of some sort. We could borrow some ideas from Lua.
On Tue, Jun 8, 2021 at 9:47 AM Antoine Pitrou <antoine@python.org> wrote:
Le 08/06/2021 à 13:08, Joannah Nanjekye a écrit :
Hey all,
For my research, IBM wrote a tracing GC for CPython and I was trying out some ideas on how we would support the CAPI.
I know about handles used in HPy but I felt they can actually incur allocation overhead and use more memory.
Instead, I thought of changing the semantics of the union type (PyObject) to not point to internal structures and use a stack for sharing data between Python and C. There can be one push function for each Python type with a direct representation in C: Py_pushInteger for ints, etc. When a C function returns, all values in the stack are returned to Python as the results of the C function. Assuming we can have a way of returning multiple values in Python.
Specifically, change
typedef struct Object *PyObject;
To:
typedef unsigned int PyObject;
Hmm... how would this be different than defining a HPy handle to be the exact same thing?
Where now PyObject becomes an index into an internal array that stored all values that had to be given to. This means that when a value is in that array, it would not be collected by Python. When the C function returns its whole array is erased, and the values used by the function are collected.
The harder problem is probably when a C function wants to keep ownership of a PyObject for longer than a function call.
Other than that, it's difficult to think about the ramifications of your proposal :-)
Also, you may want to look at how cpyext defined their own CPython compatibility layer, since they have to solve a similar problem.
Regards
Antoine.
capi-sig mailing list -- capi-sig@python.org To unsubscribe send an email to capi-sig-leave@python.org https://mail.python.org/mailman3/lists/capi-sig.python.org/ Member address: nanjekyejoannah@gmail.com
-- Best, Joannah Nanjekye
*"You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program." Alan J. Perlis*