On Mon, Nov 26, 2018 at 4:10 PM Larry Hastings <larry@hastings.org> wrote:

On 11/23/18 5:15 AM, Armin Rigo wrote:
Also FWIW, my own 2 cents on the topic of changing the C API: let's
entirely drop ``PyObject *`` and instead use more opaque
handles---like a ``PyHandle`` that is defined as a pointer-sized C
type but is not actually directly a pointer.  The main difference this
would make is that the user of the API cannot dereference anything
from the opaque handle, nor directly compare handles with each other
to learn about object identity.  They would work exactly like Windows
handles or POSIX file descriptors.
Why would this be better than simply returning the pointer? Sure, it prevents ever dereferencing the pointer and messing with the object, it is true. So naughty people would be prevented from messing with the object directly instead of using the API as they should. But my understanding is that the implementation would be slightly slower--there'd be all that looking up objects based on handles, and managing the handle namespace too. I'm not convinced the nice-to-have of "you can't dereference the pointer anymore" is worth this runtime overhead.

Or maybe you have something pretty cheap in mind, e.g. "handle = pointer ^ 49"? Or even "handle = pointer ^ (random odd number picked at startup)" to punish the extra-naughty?

Heck, it'd be find if someones implementation (such as a simple shim for CPython's existing API) wants to internally keep a PyObject structure and have PyHandle's implementation just be a typecast from PyObject* to PyHandle. The real point is that a handle is opaque and cannot be depended on by any API _user_ as being a pointer. What it means behind the scenes of a given VM is left entirely up to the VM.

When an API returns a handle, that is an implicit internal INCREF if a VM is reference counting. When code calls an API that consumes a handle by taking ownership of it for itself (Py_DECREF could be considered one of these if you have a Py_DECREF equivalent API) that means "I can no longer using this handle".

Comparisons get documented as being invalid, pointing to the API to call for an identity check, but it is up to each implementation to decide if it wants to force the handles to be unique. Anyone depending on that behavior is being bad and should not be supported.

-gps

PS ... use C++ and you could actually make handle identity comparisons do the right thing...