[issue6251] c++ extension module implementation guide/example in extending/embedding documentation

[John O'Driscoll]

New submission from John O'Driscoll <biggerbumme at yahoo.co.in>:

feature: extension module C++ howto/example in extending-embedding/c-api
documentation


why:	The embedding/extension documentation states that module
implementation in c++ is possible, without providing any guidance beyond
this. Coders more familiar/comfortable with c++ than c, writing c++ code
to expose in python, might want to create the actual python class/module
implementations in c++ classes or structs. A basic guide can help
prevent wastage of energy reinventing the wheel, and also serve as a
guide to safe/preferred style. The method outlined here can be a
starting point in finding out what that style is. 
Also it seems, to my eyes at least, a little easier to 'visualise' and
apply than the equivalent in plain C (the Noddy module etc), a bit more
'systematic', and easier to understand the relation between c and python
objects. So after some trial and error, discovering the pitfalls, I am
finding it easy to create useful python classes with this recipe. Others
might also find it useful, even if only as a stimulus to find a Better Way.

Python is an object-oriented language noted for its clarity, so an
allegedly simple and clear OO implementation strategy for module classes
should be examined.


what:	I've written a module currently called 'cpeepee', containing a
basic python class ,(struct TestRealDestructor in C++,
'cpeepee.destructo' in python). It has been tested with python-2.5 and
g++-4.2 on ubuntu-'hardy heron' and python-2.5/g++-3.4 on FreeBSD-6.4. 

The c++ struct inherits from PyObject.(You could also inherit from
PyVarObject or other more specialised types). The most important
non-feature of this class is that it is not virtual, has no virtual
destructor or functions(in member objects virtual destructors and the
rest are OK however). Therefore the packing of ob_refcnt and friends is
correct, and casting to/from PyObject is safe(-and otherwise is not -
the PyObject* cast from such a type with virtual destructor is offset by
the size of a pointer -the vptr?-, but when python casts back to the
type - from void* or so?- the offset is not removed, leading to mayhem).

[You could also not inherit, simply put the PyObject_HEAD macro as the
first entry in the struct - could be a class also, as long as the
PyObject members(ob_refcnt, ob_type ... )were public - You would have to
write a new macro to fill in those members in the constructor's
initialiser list, but that doesn't look too hard. As it is the inherited
classes use a function and some macros (almost identical to
PyObject_HEAD_INIT(typo) ) to fill in the PyObject parent. Again, vfuncs
are out]

	The destructo method and member tables are static members of
TestRealDestructor, as is its type object.(Other optional tables etc
should also be static members if provided - makes for a simple
consistent setup)
	The objects constructor is called from
TestRealDestructor::type.tp_new() and passed the args and kwds arguments
it is passed, using placement new with memory obtained from
tp_alloc()(see TestRealDestructor::create() ). Being able to properly
call an object's constructor was the real motivation for writing the
code this derives from. Using C style one is stuck casting a char* to
your type and then filling its fields by hook or crook.
 
On error, either you could throw a c++ exception in constructor to catch
in tp_new, and convert to a python exception, or simply throw the python
exception in the constructor, and then check if PyErr_Occurred() in
tp_new() - the approach with destructo.

Since tp_new and the constructor take care of object creation, this
leaves tp_init not doing much at all. It could be used for extra checks,
or printing funky messages about preserved ham

Functions to be exposed in python API as class member functions should
static members of the class, and to do the work they call ordinary
member functions of the class object passed into the static function .
You could use global static functions(with the first arg
TestRealDestructor* say, rather than PyObject*), but that's less clear,
less systematic, less OO.

Everything can be placed in a namespace to avoid any pollution of global
namespace.

I've worked out the bugs that were obvious to me, so it should compile
and run as is, without error messages. The one compile warning I don't
know how to banish comes from the offsetof macro when setting member
offsets in the member table(as copied from the c example) - however the
object whose offset is so established works fine from python, so I think
it's a spurious warning. 

If you find any issues with my approach, I'm happy to work through them,
or if you know what to do then make whatever changes you think required.

I'm happy to put any code or words I contribute on this topic to go
under python's copyright as long as I'm credited somehow(however you
normally do that). Whatever, I'm happy to contribute to such a great
project as python

sincerely

John O'Driscoll

----------
assignee: georg.brandl
components: Documentation
files: CXXdemo-0.1.tar.gz
messages: 89190
nosy: georg.brandl, subgeometer
severity: normal
status: open
title: c++ extension module implementation guide/example in extending/embedding documentation
type: resource usage
versions: Python 2.5
Added file: http://bugs.python.org/file14253/CXXdemo-0.1.tar.gz

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6251>
_______________________________________