[Python-Dev] Extending types in C - help needed

[M.-A. Lemburg]

Jack Jansen wrote:
> 
> >> On Thursday, January 17, 2002, at 11:29  AM, M.-A. Lemburg wrote:
> >>
> >>> I am more in favour of
> >>> exposing the pickle reduce API through "O@", that is
> >>> have PyArgTuple_Parse() call the .__reduce__() method
> >>> of the object. This will then return (factory, state_tuple)
> >>> and these could then be exposed to the C function via two
> >>> PyObject*.
> >>
> >> You've suggested this before, but at that time I ignored it
> >> because it made absolutely no sense to me. "pickle" triggers one
> >> set of ideas for me, "reduce" triggers a different set, "factory
> >> function" yet another different set. None of these sets of ideas
> >> have the least resemblance to what I'm trying to do:-)
> >
> > The idea is simple but extends what you are trying to
> > achieve (I gave an example on how to use this somewhere
> > in the "wrapper" thread). Basically, you'll just want to
> > use the state tuple to access the underlying void* C pointer
> > via a PyCObject which does the wrapping of the pointer.
> > The "pickle" mechanism would store the PyCObject in the
> > state tuple which you could then access to get at the
> > C pointer.
> >
> I think you're missing a few points here. First of all, my objects
> aren't PyCObjects but other extension objects. 

I know. The idea is that either you add a .__reduce__ method
to the extension objects or register their types with a registry
comparable to copyreg.

> While the main pointer in
> the object could be wrapped in a PyCObject there may be other
> information in my objects that is important, such as a pointer to the
> dispose routine to call on the c-pointer when the Python object reaches
> refcount zero (and this pointer may change over time as ownership of,
> say, a button is passed from Python to the system). 

Note that PyCObjects support all of this. It's not important in this
context, though.  The PyCObject is only used to wrap the raw 
pointer; the factory function then takes this pointer and creates
one of your extension object out of it.

> The _New and
> _Convert routines will know how to get from the C pointer to the
> *correct* object, i.e. normally there will be only one Python object for
> every C object.

That's also possible using the "pickle" approach.
 
> Also, the method seems rather complicated for doing a simple thing. The
> only thing I really want is a way to refer to an _New or _Convert method
> from Python code. The most reasonable way to do that seems to be by
> creating a way to get from te type object (which is available in Python)
> to those routines. Thomas' suggestion looked very promising, and simple
> too, until Guido said that unfortunately it couldn't be done. Your
> suggestion, as far as I understand it, looks complicated and probably
> inefficient too (remember the code will have to go through all these
> hoops every time it needs to convert an object from Python to C or vice
> versa).

It is more complicated, but also more flexible. Plus it builds on
techniques which are already applied in Python's pickle 
mechanism.

Note that by adding a tp_reduce slot, the overhead of calling
a Python function could be kept reasonable. Helper functions
could aid in accessing the C pointer which is stored in
the state tuple.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/