On Sat, 2 Mar 2019 at 12:08, Neil Schemenauer <nas-python@arctrix.com> wrote:
If we implement PyHandle in CPython in a simple way (e.g. as a typecast from PyObject), I think we will have similar problems to what the Android runtime encountered around the release of Ice Cream Sandwich. See:
https://android-developers.googleblog.com/2011/11/jni-local-reference-changes-in-ics.htmlAgain, a possible solution might be a debug build option CPython that uses indirect pointers for the handles and checks that the API is used correctly. Or, if we went with a JNIEnv* like vtable, you could have a command-line flag that switched CPython to use indirect pointers for handles.
Given the long history of the existing CPython C API, I think any new handle-based design is still going to have to support the invariant that "c_handle_X == c_handle_Y" <=> "python_object_X is python_object_Y".
Anything else will be far too error prone (Consider that JNI has *never* worked that way, and people still get it wrong repeatedly based on assumption about how object identity should work in C/C++. How much worse would the problem be for us given that Python's C API *has* worked that way for the past ~30 years?)
That invariant doesn't require that Python level object IDs actually be C level memory addresses (even if CPython implements it that way), but it does require that there be a 1:1 mapping between live handles and the live objects they reference. While that does bring in the issues that Carl mentions with recycling of object IDs creating surprising false equivalences, those aren't new problems either (CPython's generous use of free lists for builtin types means that IDs get recycled all the time in the existing APIs).
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia