[Python-Dev] C API changes

[E. Madison Bray]

On Fri, Nov 23, 2018 at 2:22 PM Armin Rigo <armin.rigo at gmail.com> wrote:
>
> Hi Hugo, hi all,
>
> On Sun, 18 Nov 2018 at 22:53, Hugh Fisher <hugo.fisher at gmail.com> wrote:
> > I suggest that for the language reference, use the license plate
> > or registration analogy to introduce "handle" and after that use
> > handle throughout. It's short, distinctive, and either will match
> > up with what the programmer already knows or won't clash if
> > or when they encounter handles elsewhere.
>
> FWIW, a "handle" is typically something that users of an API store and
> pass around, and which can be used to do all operations on some
> object.  It is whatever a specific implementation needs to describe
> references to an object.  In the CPython C API, this is ``PyObject*``.
> I think that using "handle" for something more abstract is just going
> to create confusion.
>
> Also FWIW, my own 2 cents on the topic of changing the C API: let's
> entirely drop ``PyObject *`` and instead use more opaque
> handles---like a ``PyHandle`` that is defined as a pointer-sized C
> type but is not actually directly a pointer.  The main difference this
> would make is that the user of the API cannot dereference anything
> from the opaque handle, nor directly compare handles with each other
> to learn about object identity.  They would work exactly like Windows
> handles or POSIX file descriptors.  These handles would be returned by
> C API calls, and would need to be closed when no longer used.  Several
> different handles may refer to the same object, which stays alive for
> at least as long as there are open handles to it.  Doing it this way
> would untangle the notion of objects from their actual implementation.
> In CPython objects would internally use reference counting, a handle
> is really just a PyObject pointer in disguise, and closing a handle
> decreases the reference counter.  In PyPy we'd have a global table of
> "open objects", and a handle would be an index in that table; closing
> a handle means writing NULL into that table entry.  No emulated
> reference counting needed: we simply use the existing GC to keep alive
> objects that are referenced from one or more table entries.  The cost
> is limited to a single indirection.

+1

As another point of reference, if you're interested, I've been working
lately on the special purpose computer algebra system GAP.  It also
uses an approach like this: Objects are referenced throughout via an
opaque "Obj" type (which is really just a typedef of "Bag", the
internal storage reference handle of its "GASMAN" garbage collector
[1]).  A nice benefit of this, along with the others discussed above,
is that it has being relatively easy to replace the garbage collector
in GAP--there are options for it to use Boehm-GC, as well as Julia's
GC.

GAP has its own problems, but it's relatively simple and has been
inspiring to look at; I was coincidentally wondering just recently if
there's anything Python could take from it (conversely, I'm trying to
bring some things I've learned from Python to improve GAP...).

[1] https://github.com/gap-system/gap/blob/master/src/gasman.c