[Python-Dev] C API changes

[Gregory P. Smith]

On Mon, Nov 26, 2018 at 4:10 PM Larry Hastings <larry at hastings.org> wrote:

> On 11/23/18 5:15 AM, Armin Rigo wrote:
>
> Also FWIW, my own 2 cents on the topic of changing the C API: let's
> entirely drop ``PyObject *`` and instead use more opaque
> handles---like a ``PyHandle`` that is defined as a pointer-sized C
> type but is not actually directly a pointer.  The main difference this
> would make is that the user of the API cannot dereference anything
> from the opaque handle, nor directly compare handles with each other
> to learn about object identity.  They would work exactly like Windows
> handles or POSIX file descriptors.
>
>
> Why would this be better than simply returning the pointer?  Sure, it
> prevents ever dereferencing the pointer and messing with the object, it is
> true.  So naughty people would be prevented from messing with the object
> directly instead of using the API as they should.  But my understanding is
> that the implementation would be slightly slower--there'd be all that
> looking up objects based on handles, and managing the handle namespace
> too.  I'm not convinced the nice-to-have of "you can't dereference the
> pointer anymore" is worth this runtime overhead.
>
> Or maybe you have something pretty cheap in mind, e.g. "handle = pointer ^
> 49"?  Or even "handle = pointer ^ (random odd number picked at startup)" to
> punish the extra-naughty?
>
Heck, it'd be find if someones implementation (such as a simple shim for
CPython's existing API) wants to internally keep a PyObject structure and
have PyHandle's implementation just be a typecast from PyObject* to
PyHandle.  The real point is that a handle is opaque and cannot be depended
on by any API _user_ as being a pointer.  What it means behind the scenes
of a given VM is left entirely up to the VM.

When an API returns a handle, that is an implicit internal INCREF if a VM
is reference counting.  When code calls an API that consumes a handle by
taking ownership of it for itself (Py_DECREF could be considered one of
these if you have a Py_DECREF equivalent API) that means "I can no longer
using this handle".

Comparisons get documented as being invalid, pointing to the API to call
for an identity check, but it is up to each implementation to decide if it
wants to force the handles to be unique. Anyone depending on that behavior
is being bad and should not be supported.

-gps

PS ... use C++ and you could actually make handle identity comparisons do
the right thing...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20181127/35f34448/attachment.html>