GC for native extensions
Hey all,
For my research, IBM wrote a tracing GC for CPython and I was trying out some ideas on how we would support the CAPI.
I know about handles used in HPy but I felt they can actually incur allocation overhead and use more memory.
Instead, I thought of changing the semantics of the union type (PyObject) to not point to internal structures and use a stack for sharing data between Python and C. There can be one push function for each Python type with a direct representation in C: Py_pushInteger for ints, etc. When a C function returns, all values in the stack are returned to Python as the results of the C function. Assuming we can have a way of returning multiple values in Python.
Specifically, change
typedef struct Object *PyObject;
To:
typedef unsigned int PyObject;
Where now PyObject becomes an index into an internal array that stored all values that had to be given to. This means that when a value is in that array, it would not be collected by Python. When the C function returns its whole array is erased, and the values used by the function are collected.
This setup gets us a reliable Union type (PyObject), the garbage collector can also move objects. I think that backward compatibility can easily be implemented using macros.
What is some feedback on this approach and am I overconfident of having reasonable backward compatibility? Also, can this experiment uncover any insights that CPython would find useful?
-- Best, Joannah Nanjekye
*"You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program." Alan J. Perlis*
On 08. 06. 21 13:08, Joannah Nanjekye wrote:
Hey all,
For my research, IBM wrote a tracing GC for CPython and I was trying out some ideas on how we would support the CAPI.
I know about handles used in HPy but I felt they can actually incur allocation overhead and use more memory.
Instead, I thought of changing the semantics of the union type (PyObject) to not point to internal structures and use a stack for sharing data between Python and C. There can be one push function for each Python type with a direct representation in C: Py_pushInteger for ints, etc. When a C function returns, all values in the stack are returned to Python as the results of the C function. Assuming we can have a way of returning multiple values in Python.
Sorry, but I don't understand this proposal. What do you mean by "returned to Python"?
Calling a C function is not a special case. It's the oposite: calling a Python function is done by calling a C funtion (one of https://docs.python.org/3/c-api/call.html#object-calling-api or their private variants).
Specifically, change
typedef struct Object *PyObject;
To:
typedef unsigned int PyObject;
Where now PyObject becomes an index into an internal array that stored all values that had to be given to.
Given to what?
This means that when a value is in that array, it would not be collected by Python. When the C function returns its whole array is erased, and the values used by the function are collected.
This setup gets us a reliable Union type (PyObject), the garbage collector can also move objects. I think that backward compatibility can easily be implemented using macros.
What is some feedback on this approach and am I overconfident of having reasonable backward compatibility? Also, can this experiment uncover any insights that CPython would find useful?
Sorry, but I don't understand this proposal. What do you mean by "returned to Python"?
Passing values from C to Python, is what I meant. The stack exists to handle this case.
Calling a C function is not a special case. It's the oposite: calling a Python function is done by calling a C funtion (one of https://docs.python.org/3/c-api/call.html#object-calling-api or their private variants).
I know this and was thinking the stack could be used for the C part to ease other things related to GC.
Given to what?
To C, the full sentence reads as below:
Where now PyObject becomes an index into an internal array that stores all values that had to be given to C.
On Tue, Jun 8, 2021 at 9:14 AM Petr Viktorin encukou@gmail.com wrote:
On 08. 06. 21 13:08, Joannah Nanjekye wrote:
Hey all,
For my research, IBM wrote a tracing GC for CPython and I was trying out some ideas on how we would support the CAPI.
I know about handles used in HPy but I felt they can actually incur allocation overhead and use more memory.
Instead, I thought of changing the semantics of the union type (PyObject) to not point to internal structures and use a stack for sharing data between Python and C. There can be one push function for each Python type with a direct representation in C: Py_pushInteger for ints, etc. When a C function returns, all values in the stack are returned to Python as the results of the C function. Assuming we can have a way of returning multiple values in Python.
Sorry, but I don't understand this proposal. What do you mean by "returned to Python"?
Calling a C function is not a special case. It's the oposite: calling a Python function is done by calling a C funtion (one of https://docs.python.org/3/c-api/call.html#object-calling-api or their private variants).
Specifically, change
typedef struct Object *PyObject;
To:
typedef unsigned int PyObject;
Where now PyObject becomes an index into an internal array that stored all values that had to be given to.
Given to what?
This means that when a value is in that array, it would not be collected by Python. When the C function returns its whole array is erased, and the values used by the function are collected.
This setup gets us a reliable Union type (PyObject), the garbage
collector > can also move objects. I think that backward compatibility can easily be > implemented using macros. > > What is some feedback on this approach and am I overconfident of having > reasonable backward compatibility? Also, can this experiment uncover any > insights that CPython would find useful?
capi-sig mailing list -- capi-sig@python.org To unsubscribe send an email to capi-sig-leave@python.org https://mail.python.org/mailman3/lists/capi-sig.python.org/ Member address: nanjekyejoannah@gmail.com
-- Best, Joannah Nanjekye
*"You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program." Alan J. Perlis*
Le 08/06/2021 à 13:08, Joannah Nanjekye a écrit :
Hey all,
For my research, IBM wrote a tracing GC for CPython and I was trying out some ideas on how we would support the CAPI.
I know about handles used in HPy but I felt they can actually incur allocation overhead and use more memory.
Instead, I thought of changing the semantics of the union type (PyObject) to not point to internal structures and use a stack for sharing data between Python and C. There can be one push function for each Python type with a direct representation in C: Py_pushInteger for ints, etc. When a C function returns, all values in the stack are returned to Python as the results of the C function. Assuming we can have a way of returning multiple values in Python.
Specifically, change
typedef struct Object *PyObject;
To:
typedef unsigned int PyObject;
Hmm... how would this be different than defining a HPy handle to be the exact same thing?
Where now PyObject becomes an index into an internal array that stored all values that had to be given to. This means that when a value is in that array, it would not be collected by Python. When the C function returns its whole array is erased, and the values used by the function are collected.
The harder problem is probably when a C function wants to keep ownership of a PyObject for longer than a function call.
Other than that, it's difficult to think about the ramifications of your proposal :-)
Also, you may want to look at how cpyext defined their own CPython compatibility layer, since they have to solve a similar problem.
Regards
Antoine.
Hmm... how would this be different than defining a HPy handle to be the exact same thing?
I sort of think of a handle as a lightweight proxy object (just like used in cpyext) to the union type, thereby providing another indirection to the union type. In addition to normal allocations, handles are allocated and updated too. However, I have to say, practically the difference may be questionable, one of those things you cant truly bet on without actually trying.
The harder problem is probably when a C function wants to keep ownership of a PyObject for longer than a function call.
I have not looked at cpyext in depth but I will. Optionally, we can use a mechanism of "references". Typically have a function that gets a Python value from the stack and returns an integer reference for instance to it. The same reference can later be used to get the value. We can similarly destroy the references. This can ease our work to some extent.
For the shorter case, we need to find a way of creating dynamic scope of some sort. We could borrow some ideas from Lua.
On Tue, Jun 8, 2021 at 9:47 AM Antoine Pitrou antoine@python.org wrote:
Le 08/06/2021 à 13:08, Joannah Nanjekye a écrit :
Hey all,
For my research, IBM wrote a tracing GC for CPython and I was trying out some ideas on how we would support the CAPI.
I know about handles used in HPy but I felt they can actually incur allocation overhead and use more memory.
Instead, I thought of changing the semantics of the union type (PyObject) to not point to internal structures and use a stack for sharing data between Python and C. There can be one push function for each Python type with a direct representation in C: Py_pushInteger for ints, etc. When a C function returns, all values in the stack are returned to Python as the results of the C function. Assuming we can have a way of returning multiple values in Python.
Specifically, change
typedef struct Object *PyObject;
To:
typedef unsigned int PyObject;
Hmm... how would this be different than defining a HPy handle to be the exact same thing?
Where now PyObject becomes an index into an internal array that stored all values that had to be given to. This means that when a value is in that array, it would not be collected by Python. When the C function returns its whole array is erased, and the values used by the function are collected.
The harder problem is probably when a C function wants to keep ownership of a PyObject for longer than a function call.
Other than that, it's difficult to think about the ramifications of your proposal :-)
Also, you may want to look at how cpyext defined their own CPython compatibility layer, since they have to solve a similar problem.
Regards
Antoine.
capi-sig mailing list -- capi-sig@python.org To unsubscribe send an email to capi-sig-leave@python.org https://mail.python.org/mailman3/lists/capi-sig.python.org/ Member address: nanjekyejoannah@gmail.com
-- Best, Joannah Nanjekye
*"You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program." Alan J. Perlis*
On Tue, Jun 8, 2021 at 3:22 PM Joannah Nanjekye nanjekyejoannah@gmail.com wrote:
I sort of think of a handle as a lightweight proxy object (just like used in cpyext)
It's more accurate to think of an HPy handle as a pointer or an index (much as in your original proposal). There is no proxy object in HPy at all. The handle is just an opaque reference -- i.e. an index or a pointer that one cannot use to access any memory layout or state directly.
It's more accurate to think of an HPy handle as a pointer or an index (much as in your original proposal). There is no proxy object in HPy at all. The handle is just an opaque reference -- i.e. an index or a pointer that one cannot use to access any memory layout or state directly. Yes, sorry for confusing you, am in agreement with this definition of handles, i.e just a reference to the object, And all we do is update the handle in case of moving etc
On Tue, Jun 8, 2021 at 11:14 AM Simon Cross hodgestar@gmail.com wrote:
On Tue, Jun 8, 2021 at 3:22 PM Joannah Nanjekye nanjekyejoannah@gmail.com wrote:
I sort of think of a handle as a lightweight proxy object (just like used in cpyext)
It's more accurate to think of an HPy handle as a pointer or an index (much as in your original proposal). There is no proxy object in HPy at all. The handle is just an opaque reference -- i.e. an index or a pointer that one cannot use to access any memory layout or state directly.
-- Best, Joannah Nanjekye
*"You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program." Alan J. Perlis*
Hi Joannah,
On Tue, Jun 8, 2021 at 1:09 PM Joannah Nanjekye nanjekyejoannah@gmail.com wrote:
typedef unsigned int PyObject;
This is almost exactly how an HPy handle is defined on PyPy.
On CPython, an HPy handle is just a PyObject * wrapped in a typedef (so that the compiler will generate an error if one attempts to access the elements of the PyObject struct).
Yours sincerely, Simon
Hi Joannah,
IMO you would save time by investing on HPy. It exists and solves your issue.
On Tue, Jun 8, 2021 at 1:09 PM Joannah Nanjekye nanjekyejoannah@gmail.com wrote:
Where now PyObject becomes an index into an internal array that stored all values that had to be given to. This means that when a value is in that array, it would not be collected by Python. When the C function returns its whole array is erased, and the values used by the function are collected.
The C API doesn't make any assumption about object lifetime. You should assume that objects are still used after the function exit.
Example (pseudo-code):
void store(PyObject list) { PyObject *obj = PyLong_FromLong(1); // obj is used after PyLong_FromLong() exited
PyList_Append(list, obj); Py_DECREF(obj);
// oh oh, obj is stored in list and must now remain valid // until list is destroyed }
By the way, how do you magically clear your "array" in store()? How do you inject code to clear it? In HPy, the array is "always there", it's not cleared: HPy_Close() simply punchs holes in it.
Changing PyObject structure to avoid direct access into the PyObject is the purpose of my PEP 620. This PEP remains controversial (it's not accepted) and introduces many incompatible changes in the C API. Until *all* structures are made opaque, I'm not sure that you can consider changing PyObject definition (to "unsigned int", or anything else). The practical problem is that the PyObject *structure* is "leaked" into all sub-types. For example, the PyUnicodeObject structure (str type) starts with "PyObject ob_base;". Right now, it remains possible to access directly the PyObject.ob_type member of a PyUnicodeObject using "((PyObject*)obj)->ob_type" (please don't do that, use Py_TYPE(obj) ;-)).
What is some feedback on this approach and am I overconfident of having reasonable backward compatibility?
It is simply totally backward incompatible and so impossible practically today.
Tons of code still access directly into the PyObject structure: see https://bugs.python.org/issue39573
By the way, my change to deny "Py_TYPE(obj) = new_type;" (direct access to PyObject.ob_type) has been reverted for the 2nd time ;-) (It broke Windows buildbots building Python in debug mode. Issue with the stack usage and function inlining.)
Victor
Night gathers, and now my watch begins. It shall not end until my death.
Thanks all for the feedback, I will heed it.
I hoped changing the semantics of PyObject in conjunction with a stack would give me a simpler design.
My only frustration with handles is overhead and high memory costs. Also if you don't get very mechanical when implementing the GC, you can have so much fragmentation when dealing with managing memory for the handles themselves. I wonder if the GC in PyPy has been edited to support HPy already?
On Tue, Jun 8, 2021 at 7:14 PM Victor Stinner vstinner@python.org wrote:
Hi Joannah,
IMO you would save time by investing on HPy. It exists and solves your issue.
On Tue, Jun 8, 2021 at 1:09 PM Joannah Nanjekye nanjekyejoannah@gmail.com wrote:
Where now PyObject becomes an index into an internal array that stored all values that had to be given to. This means that when a value is in that array, it would not be collected by Python. When the C function returns its whole array is erased, and the values used by the function are collected.
The C API doesn't make any assumption about object lifetime. You should assume that objects are still used after the function exit.
Example (pseudo-code):
void store(PyObject list) { PyObject *obj = PyLong_FromLong(1); // obj is used after PyLong_FromLong() exited
PyList_Append(list, obj); Py_DECREF(obj);
// oh oh, obj is stored in list and must now remain valid // until list is destroyed }
By the way, how do you magically clear your "array" in store()? How do you inject code to clear it? In HPy, the array is "always there", it's not cleared: HPy_Close() simply punchs holes in it.
Changing PyObject structure to avoid direct access into the PyObject is the purpose of my PEP 620. This PEP remains controversial (it's not accepted) and introduces many incompatible changes in the C API. Until *all* structures are made opaque, I'm not sure that you can consider changing PyObject definition (to "unsigned int", or anything else). The practical problem is that the PyObject *structure* is "leaked" into all sub-types. For example, the PyUnicodeObject structure (str type) starts with "PyObject ob_base;". Right now, it remains possible to access directly the PyObject.ob_type member of a PyUnicodeObject using "((PyObject*)obj)->ob_type" (please don't do that, use Py_TYPE(obj) ;-)).
What is some feedback on this approach and am I overconfident of having reasonable backward compatibility?
It is simply totally backward incompatible and so impossible practically today.
Tons of code still access directly into the PyObject structure: see https://bugs.python.org/issue39573
By the way, my change to deny "Py_TYPE(obj) = new_type;" (direct access to PyObject.ob_type) has been reverted for the 2nd time ;-) (It broke Windows buildbots building Python in debug mode. Issue with the stack usage and function inlining.)
Victor
Night gathers, and now my watch begins. It shall not end until my death.
-- Best, Joannah Nanjekye
*"You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program." Alan J. Perlis*
On Wed, Jun 9, 2021 at 1:01 AM Joannah Nanjekye nanjekyejoannah@gmail.com wrote:
My only frustration with handles is overhead and high memory costs.
I thought I explained that this was not the case and that you agreed, so I am confused by you restating it.
I wonder if the GC in PyPy has been edited to support HPy already?
All handles are short lived so they don't cause GC fragmentation. For example, if I reimplement Victor's example in HPy one has:
// pseudo-code
void store(HPyContext *ctx, HPy list) { HPy obj = HPyLong_FromLong(ctx, 1);
HPyList_Append(ctx, list, obj); HPy_Close(ctx, obj);
// obj is stored after store exits, but no *handles* escape. // how obj is tracked by the Python implementation GC is completely hidden by the HPy API }
- Simon
Thanks Simon.
On Wed, Jun 9, 2021, 04:07 Simon Cross hodgestar@gmail.com wrote:
On Wed, Jun 9, 2021 at 1:01 AM Joannah Nanjekye nanjekyejoannah@gmail.com wrote:
My only frustration with handles is overhead and high memory costs.
I thought I explained that this was not the case and that you agreed, so I am confused by you restating it.
I wonder if the GC in PyPy has been edited to support HPy already?
All handles are short lived so they don't cause GC fragmentation. For example, if I reimplement Victor's example in HPy one has:
// pseudo-code
void store(HPyContext *ctx, HPy list) { HPy obj = HPyLong_FromLong(ctx, 1);
HPyList_Append(ctx, list, obj); HPy_Close(ctx, obj);
// obj is stored after store exits, but no *handles* escape. // how obj is tracked by the Python implementation GC is completely hidden by the HPy API }
- Simon
I would also love to say my analysis is based off some published evaluation of handles titled, handles revisited, so HPy may be different obviously. I may have to look and see.
On Wed, Jun 9, 2021, 04:07 Simon Cross hodgestar@gmail.com wrote:
On Wed, Jun 9, 2021 at 1:01 AM Joannah Nanjekye nanjekyejoannah@gmail.com wrote:
My only frustration with handles is overhead and high memory costs.
I thought I explained that this was not the case and that you agreed, so I am confused by you restating it.
I wonder if the GC in PyPy has been edited to support HPy already?
All handles are short lived so they don't cause GC fragmentation. For example, if I reimplement Victor's example in HPy one has:
// pseudo-code
void store(HPyContext *ctx, HPy list) { HPy obj = HPyLong_FromLong(ctx, 1);
HPyList_Append(ctx, list, obj); HPy_Close(ctx, obj);
// obj is stored after store exits, but no *handles* escape. // how obj is tracked by the Python implementation GC is completely hidden by the HPy API }
- Simon
participants (5)
-
Antoine Pitrou -
Joannah Nanjekye -
Petr Viktorin -
Simon Cross -
Victor Stinner