[Python-Dev] Let's update CObject API so it is safe and regular!

[Jim Fulton]

On Mar 31, 2009, at 3:14 PM, Larry Hastings wrote:

(Thanks for calling my attention to this. :)

>
> The CObject API has two flaws.
>
> First, there is no usable type safety mechanism.  You can store a void
> *object, and a void *description.  There is no established schema for
> the description; it could be an integer cast to a pointer, or it could
> point to memory of any configuration, or it could be NULL.  Thus users
> of the CObject API generally ignore it--thus working without any type
> safety whatsoever.  A programmer could crash the interpreter from pure
> Python by mixing and matching CObjects from different modules (e.g.  
> give
> "curses" a CObject from "_ctypes").

The description field wasn't in the original CObject implementation  
that I was involved with many years ago.  Looking at it now, I don't  
think it is intended as a type-safety mechanism at all, but as a way  
to pass data to the destructor.  I don't know what motivated this. (I  
don't know why it it's called "description". This name seems to be  
very confusing.)

The only type-safety mechanism for a CObject is it's identity.  If you  
want to make sure you're using the foomodule api, make sure the  
address of the CObject is the same as the address of the api object  
exported by the module.

The exporting module should automate use of the C API by providing an  
appropriate header file, as described in http://docs.python.org/extending/extending.html#providing-a-c-api-for-an-extension-module 
.

> Second, the destructor callback is defined as taking *either* one *or*
> two parameters, depending on whether the "descr" pointer is non- 
> NULL. One can debate the finer points of what is and isn't defined  
> behavior in
> C, but at its heart this is a sloppy API.

<shrug>

It was necessary for backward compatibility. I don't know what  
motivated this, so I don't know if the benefit was worth the ugliness.

> MvL and I discussed this last night and decided to float a revision of
> the API.  I wrote the patch, though, so don't blame Martin if you  
> don't
> like my specific approach.
>
> The color of this particular bike shed is:
> * The PyCObject is now a private data structure; you must use  
> accessors.
> I added accessors for all the members.

The original implementation didn't expose the structure. I don't know  
why it was exposed. It would be backward incompatible to hide it again  
now.

> * The constructors and the main accessor (PyCObject_AsVoidPtr) now all
> *require* a "const char *type" parameter, which must be a non-NULL C
> string of non-zero length.  If you call that accessor and the "type"
> is invalid *or doesn't match*, it fails.

That would break backward compatibility. Are you proposing this for  
Python 3?

What would be the gain in this? The CObject is already a type  
identifier for itself.  In any case, client code generally doesn't  
mess with CObjects directly anyway.

> * The destructor now takes the PyObject *, not the PyCObject *.  You
> must use accessors to get your hands on the data inside to free it.

It currently isn't passed the CObject, but the C pointer that it  
holds.  In any case, changing the API isn't practical, at least not  
for Python 2.

> Yes, you can easily skip around the "matching type" restriction by
> calling PyCObject_AsVoidPtr(cobj, PyCObject_GetType(cobj)).  The point
> of my API changes is to *encourage* correct use.
>
> I've posted a patch implementing this change in the 3.1 trunk to the
> bug tracker:
>
>   http://bugs.python.org/issue5630
>
> I look forward to your comments!


-1

I don't see that this gains anything.

1. All you're adding, afaict is a name for the API and the (address of  
the) CObject itself already provides this.

2. Only code provided by the module provider should be accessing the  
CObject exported by the module.

Jim

--
Jim Fulton
Zope Corporation