Mailman 3 July 2004 - Cplusplus-sig

RE: [C++-sig] Re: Wrapper for exception translation
by Gottfried.Ganssauge＠HAUFE.DE Dec. 13, 2004

Dec. 13, 2004

> > ... > >> > >> > Currently, in order to wrap user defined exceptions you need to > >> > > >> > - allocate a python exception using PyErr_NewException > >> > >> Well, no. I don't see any call to PyErr_NewException in > >> libs/python/test/exception_translator.cpp > > > > Right, you don't need to if you are content with RuntimeError > > exceptions. > > or any other pre-packaged exception type provided by Python. > > > My objective was another one: I wanted to create module exceptions > > that are derived from the python "Exception" class. > >> > >> > This function needs to be parametrized using a specific naming > >> > convention > >> > >> It's not clear to me what you're referring to. Are you > sure you mean > >> "naming convention"? > > Yes. > > From the documentation to PyErr_NewException: > > > > ... The __module__ attribute of the new class is set to > the first part > > (up to the last dot) of the name argument, and the > class name is set to > > the last part (after the last dot). > > ...yeah, that's a naming convention, but I don't see how that affects > the parameterization of any function. It is enforced by PyErr_NewException, so you must follow this convention. On the hand I don't think that any user of boost.python is aware of that convention, so I wanted to codify it within my facility. ... > > What I wanted to achieve was the following: > > Create an instance of a translated_exception_handler<> t > which will be > > registered with the library by calling > > register_exception_translator<>(). > > Sorry to be difficult, but I'm going to contradict you. You didn't > want any of that. What you wanted was to create a new Python > exception type in the module which would correspond to a particular > C++ exception type and would be raised at the boundary with Python > whenever a wrapped function threw the C++ exception. Right? Right. I actually described the way to do it within the current library design. > > > Normally t would be destructed by the end of exception_<>() thereby > > leaving a registration to a non-existing translator - very bad > > karma. > > I don't understand any of this. All you have to pass to > register_exception_translator is a function pointer, and functions > live forever. So where's the lifetime issue? A pointer to a function which has access to python exception instance e. As I didn't want to create separate entities I chose to supply a functor instead which has e as a member variable. > > > Because I didn't like to have an additional container for all > > translated_exception_handler<> instances, I chose to store t within > > e's class dictionary. At that moment it seemed to be the easiest > > way to create a wrapper class using class_<> instead of making a > > true python class by hand (and it even works!). You are right, it > > is way too complex, I will change it to the mechanism I used in > > opaque_pointer_converter. > > I don't know what that is, but before you consider doing something > which is again too complex, please consider: > > template <class E> > struct translate_exception > { > static char const* msg; This would need to be defined somewhere. > static object python_exception_type; ... and this would need to be defined somewhere as well. > > static void handler(E const&) > { > PyErr_SetString(python_exception_type.ptr(), msg.c_str()); > }; > > translate_exception(char const* python_exception_name, > char const* msg_) > { > // need fancier name manipulation here > python_exception_type > = object( > handle<>(PyErr_NewException(name))); > msg = msg_; > register_exception_translator<E>(handler); > } > }; > > > Simple. Yes. ... > > I'll make a redesign ... At times you don't see the forrest, just trees ... I think this time I have a simple facility which is also easy to use (no macros). It could be even simpler with a completely different design but frankly - I have no clue how to implement that: We would need a way to somehow derive (in the python sense) a class_<> from PyExc_Exception (the type of the python Exception class) or any other python class (PyErr_NewException() has a base parameter, which could be used for that). At the same time an exception_translator should be registered for that class_<> which uses PyErr_SetObject() to raise a copy of the exception caught. This would allow us to have true Python exceptions in a pythonic sense directly wrapped around the corresponding C++ exception. > > Thnx for your feedback. > > Cool! Looking forward to it. > > -- > Dave Abrahams > Boost Consulting > www.boost-consulting.com Cheers, Gottfried

3 7

Boost Python and Intel C++ 7.1 windows
by Xavier WARIN(Compte LOCAL) - I23 Aug. 16, 2004

Aug. 16, 2004

Hi, While compiling with Intel C++ 7.1 Boost Python , i get this kind of error on class.cpp and a lot of other Boost Python files : C:\boost-1.30.2\boost/type_traits/is_integral.hpp(39): error: class "boost::is_integral<unsigned short>" has already been defined BOOST_TT_AUX_BOOL_TRAIT_CV_SPEC1(is_integral,wchar_t,true) I think that there must be a define that i missed somewhere ! I don't find it Any idea ? Thank you Xavier Warin

2 2

Passing pointer to classes from C++ to Python
by Jesús Gumbau Portalès July 31, 2004

July 31, 2004

I am using the PyObject_CallObject function to call a method on a python script and passing a pointer to a class object as a parameter. I pass the address of that object as: PyInt_FromLong((long)&object); as a parameter. The problem is interpreting that address as a reference to a object from Python. (I am using SWIG so that python can know the class structure of the object). _________________________________________________________________ Descarga gratis la Barra de Herramientas de MSN http://www.msn.es/usuario/busqueda/barra?XAPID=2031&DI=1055&SU=http%3A//www…

1 0

Boost.python + PyQt?
by Robert Bell July 30, 2004

July 30, 2004

Hi All- I have some Qt widgets that I need exported to Python for use with PyQt. I've been looking into sip, but I'm also curious if boost.python is up to the task. Has anyone tried this? Bob

1 0

Boost wrapper converting Boost::multi_array to PyArray
by Ateljevich, Eli July 30, 2004

July 30, 2004

Hi. I am attempting to create a wrapper for a C++ method that returns a Boost::multi_array. I was hoping to convert this result to a PyArray without copying the underlying data. I have isolated my problem below, using a class with one method which creates a trivial multi_array, the converter code and the wrapper. I have included a very simple test.py file and its output. As the results show: 1. The results are correct for most elements of the array, but the first element of the array is getting overwritten. 2. Both the multi_array and the PyArray seem to have the correct value for the first element right before returning from the to_python_converter. So it seems like it must be happening either during or after the return. Can anyone shed any light on why this is happening? I am using a recent Enthought build of Python23 with mingw 3.2.3 compiler on a Windows XP machine. Thanks so much, Eli ///////////////////////File test.py import timeseries creator=timeseries.multi_array_creator() array=creator.getArray() print "\nIn Python array looks like: %s" % array /////////////////////// Output >>> execfile("test.py") in C++: a[0][0] = 0 a[0][1] = 1 a[0][2] = 2 a[1][0] = 10 a[1][1] = 11 a[1][2] = 12 ... On entry to convert(), a[0][0] = 0 After PyArray creation a[0][0] = 0 Before return, PyArrayObject first slot = 0 In Python array looks like: array([[9623160, 1, 2], [10, 11, 12], [20, 21, 22]], 'i') //////////////////////// File multi_array_creator.hpp #include <iostream> #include "boost/multi_array.hpp" using namespace std; class multi_array_creator{ public: typedef boost::multi_array<int, 2> ArrayType; multi_array_creator(){ import_array() } ArrayType getArray(){ ArrayType* a=new ArrayType(boost::extents[3][3]); for (int i=0;i<3;i++){ for (int j=0;j<3;j++) (*a)[i][j] = 10*i+j; } cout << "in C++:" << endl; for (int i=0;i<2;i++) for (int j=0;j<3;j++) cout << "a[" < #include <Numeric/arrayobject.h> #include <Python.h> #include <iostream> #include "multi_array_creator.hpp" using namespace std; using namespace boost; using namespace boost::python; struct MArrayToPYArray { static PyObject* convert(multi_array<int, 2> const& a) { cout << "On entry to convert(), a[0][0] = " << a[0][0] << endl; PyObject* po = PyArray_FromDimsAndData(2, (int*) a.shape(), PyArray_INT, (char*) (a.origin())); cout << "After PyArray creation a[0][0] = " << a[0][0] << endl; PyArrayObject* pao=(PyArrayObject*)po; cout <<"Before return, PyArrayObject first slot = " << *(int*)(pao->data)<< endl; return po; } }; void export_MultiArrayCreator(){ import_array(); class_<multi_array_creator>("multi_array_creator") .def("getArray", &multi_array_creator::getArray); to_python_converter<multi_array<int, 2>, MArrayToPYArray>(); } void export_MultiArrayCreator(); BOOST_PYTHON_MODULE(timeseries){ export_MultiArrayCreator(); }

1 0

Boost wrapper converting Boost::multi_array to PyArray
by Ateljevich, Eli July 30, 2004

July 30, 2004

Hi. I am attempting to create a wrapper for a C++ method that returns a Boost::multi_array. I was hoping to convert this result to a PyArray without copying the underlying data. I have isolated my problem below, using a class with one method which creates a trivial multi_array, the converter code and the wrapper. I have included a very simple test.py file and its output. As the results show: 1. The results are correct for most elements of the array, but the first element of the array is getting overwritten. 2. Both the multi_array and the PyArray seem to have the correct value for the first element right before returning from the to_python_converter. So it seems like it must be happening either during or after the return. Can anyone shed any light on why this is happening? I am using a recent Enthought build of Python23 with mingw 3.2.3 compiler on a Windows XP machine. Thanks so much, Eli ///////////////////////File test.py import timeseries creator=timeseries.multi_array_creator() array=creator.getArray() print "\nIn Python array looks like: %s" % array /////////////////////// Output >>> execfile("test.py") in C++: a[0][0] = 0 a[0][1] = 1 a[0][2] = 2 a[1][0] = 10 a[1][1] = 11 a[1][2] = 12 ... On entry to convert(), a[0][0] = 0 After PyArray creation a[0][0] = 0 Before return, PyArrayObject first slot = 0 In Python array looks like: array([[9623160, 1, 2], [10, 11, 12], [20, 21, 22]], 'i') //////////////////////// File multi_array_creator.hpp #include <iostream> #include "boost/multi_array.hpp" using namespace std; class multi_array_creator{ public: typedef boost::multi_array<int, 2> ArrayType; multi_array_creator(){ import_array() } ArrayType getArray(){ ArrayType* a=new ArrayType(boost::extents[3][3]); for (int i=0;i<3;i++){ for (int j=0;j<3;j++) (*a)[i][j] = 10*i+j; } cout << "in C++:" << endl; for (int i=0;i<2;i++) for (int j=0;j<3;j++) cout << "a[" < #include <Numeric/arrayobject.h> #include <Python.h> #include <iostream> #include "multi_array_creator.hpp" using namespace std; using namespace boost; using namespace boost::python; struct MArrayToPYArray { static PyObject* convert(multi_array<int, 2> const& a) { cout << "On entry to convert(), a[0][0] = " << a[0][0] << endl; PyObject* po = PyArray_FromDimsAndData(2, (int*) a.shape(), PyArray_INT, (char*) (a.origin())); cout << "After PyArray creation a[0][0] = " << a[0][0] << endl; PyArrayObject* pao=(PyArrayObject*)po; cout <<"Before return, PyArrayObject first slot = " << *(int*)(pao->data)<< endl; return po; } }; void export_MultiArrayCreator(){ import_array(); class_<multi_array_creator>("multi_array_creator") .def("getArray", &multi_array_creator::getArray); to_python_converter<multi_array<int, 2>, MArrayToPYArray>(); } void export_MultiArrayCreator(); BOOST_PYTHON_MODULE(timeseries){ export_MultiArrayCreator(); }

1 0

wrapping boost::optional
by Neal D. Becker July 30, 2004

July 30, 2004

I'm trying to wrap boost::optional<T>. I have an adequate solution here: #include <boost/optional.hpp> #include <boost/python/module.hpp> #include <boost/python/def.hpp> #include <boost/python/class.hpp> #include <boost/python/init.hpp> #include <boost/python/operators.hpp> using namespace boost::python; template<typename T> inline void set (boost::optional<T>& o, T x) { o = x; } template<typename T> inline T get (boost::optional<T>& o) { return o.get(); } #define OPTIONAL(T)\ class_<boost::optional<T> >("opt_"#T) \ .def ("set", &set<T>) \ .def ("get", &get<T>) \ .def (!(self)) \ ; BOOST_PYTHON_MODULE(optional_wrap) { OPTIONAL(int) OPTIONAL(double) } What would be better, though, is that if the option is not initialized, it should return None in response to get. That would be more pythonic, no? Problem is, I don't know how to do that. Any hints?

3 4

Interface slices between python and c++
by Neal D. Becker July 29, 2004

July 29, 2004

I now have some experience using boost::python with vector_indexing_suite to expose std::vector objects to python. Using this, it is easy to expose c++ algorithms written for stl-style iterators. Consider an algorithm F1: template<typename it_t> void F1 (it_t start, it_t end); We can do: template<typename cont> void wrap_f1 (cont c) { F1 (c.begin(), c.end()); } and then expose wrap_f1<vector<double> >, for example. But now, suppose we want to apply algorithm F1 to a *slice* of a vector? I noticed that in boost.cvs, there is "slice.hpp". AFAICT, vector_indexing_suite is not using it, though. OK, so my question is, what is a "good" approach to this? I would add, I think this must be a very common issue.

1 1

pyste patch for new-style polymorphism
by Roman Yakovenko July 28, 2004

July 28, 2004

Hi. I wrote pyste patch for new-style polymorphism. I testes it. It seems to work fine, without breaking anything else. Patch description: patch consist from 2 files. The first one is pyste_future.py. It purpose is similar to "import ??? from future". It defines call_dispatch boolean constant. The second one is edited ClassExporter.py. Also I send difference file. Roman. __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail # Copyright Bruno da Silva de Oliveira 2003. Use, modification and # distribution is subject to the Boost Software License, Version 1.0. # (See accompanying file LICENSE_1_0.txt or copy at # http:#www.boost.org/LICENSE_1_0.txt) import exporters from Exporter import Exporter from declarations import * from settings import * from policies import * from SingleCodeUnit import SingleCodeUnit from EnumExporter import EnumExporter from utils import makeid, enumerate import copy import exporterutils import re import pyste_future #============================================================================== # ClassExporter #============================================================================== class ClassExporter(Exporter): 'Generates boost.python code to export a class declaration' def __init__(self, info, parser_tail=None): Exporter.__init__(self, info, parser_tail) # sections of code self.sections = {} # template: each item in the list is an item into the class_<...> # section. self.wrapper_name = '' self.sections['template'] = [] # constructor: each item in the list is a parameter to the class_ # constructor, like class_<C>(...) self.sections['constructor'] = [] # inside: everything within the class_<> statement self.sections['inside'] = [] # scope: items outside the class statement but within its scope. # scope* s = new scope(class<>()); # ... # delete s; self.sections['scope'] = [] # declarations: outside the BOOST_PYTHON_MODULE macro self.sections['declaration'] = [] self.sections['declaration-outside'] = [] self.sections['include'] = [] # a list of Constructor instances self.constructors = [] # a list of code units, generated by nested declarations self.nested_codeunits = [] def ScopeName(self): return makeid(self.class_.FullName()) + '_scope' def Name(self): return self.info.name def SetDeclarations(self, declarations): Exporter.SetDeclarations(self, declarations) if self.declarations: decl = self.GetDeclaration(self.info.name) if isinstance(decl, Typedef): self.class_ = self.GetDeclaration(decl.type.name) if not self.info.rename: self.info.rename = decl.name else: self.class_ = decl self.class_ = copy.deepcopy(self.class_) else: self.class_ = None def ClassBases(self): all_bases = [] for level in self.class_.hierarchy: for base in level: all_bases.append(base) return [self.GetDeclaration(x.name) for x in all_bases] def Order(self): '''Return the TOTAL number of bases that this class has, including the bases' bases. Do this because base classes must be instantialized before the derived classes in the module definition. ''' num_bases = len(self.ClassBases()) return num_bases, self.class_.FullName() def Export(self, codeunit, exported_names): self.InheritMethods(exported_names) self.MakeNonVirtual() if not self.info.exclude: self.ExportBasics() self.ExportBases(exported_names) self.ExportConstructors() self.ExportVariables() self.ExportVirtualMethods(codeunit) self.ExportMethods() self.ExportOperators() self.ExportNestedClasses(exported_names) self.ExportNestedEnums(exported_names) self.ExportSmartPointer() self.ExportOpaquePointerPolicies() self.ExportAddedCode() self.Write(codeunit) exported_names[self.Name()] = 1 def InheritMethods(self, exported_names): '''Go up in the class hierarchy looking for classes that were not exported yet, and then add their public members to this classes members, as if they were members of this class. This allows the user to just export one type and automatically get all the members from the base classes. ''' valid_members = (Method, ClassVariable, NestedClass, ClassEnumeration) fullnames = [x.FullName() for x in self.class_] pointers = [x.PointerDeclaration(True) for x in self.class_ if isinstance(x, Method)] fullnames = dict([(x, None) for x in fullnames]) pointers = dict([(x, None) for x in pointers]) for level in self.class_.hierarchy: level_exported = False for base in level: base = self.GetDeclaration(base.name) if base.FullName() not in exported_names: for member in base: if type(member) in valid_members: member_copy = copy.deepcopy(member) member_copy.class_ = self.class_.FullName() if isinstance(member_copy, Method): pointer = member_copy.PointerDeclaration(True) if pointer not in pointers: self.class_.AddMember(member) pointers[pointer] = None elif member_copy.FullName() not in fullnames: self.class_.AddMember(member) else: level_exported = True if level_exported: break def IsValid(member): return isinstance(member, valid_members) and member.visibility == Scope.public self.public_members = [x for x in self.class_ if IsValid(x)] def Write(self, codeunit): indent = self.INDENT boost_ns = namespaces.python pyste_ns = namespaces.pyste code = '' # begin a scope for this class if needed nested_codeunits = self.nested_codeunits needs_scope = self.sections['scope'] or nested_codeunits if needs_scope: scope_name = self.ScopeName() code += indent + boost_ns + 'scope* %s = new %sscope(\n' %\ (scope_name, boost_ns) # export the template section template_args = self.sections['template'][:] if pyste_future.call_dispatch and self.wrapper_name: #in this case place it as first template argument template_args.remove( self.wrapper_name ) template_args.insert( 0, self.wrapper_name ) template_params = ', '.join(template_args) code += indent + boost_ns + 'class_< %s >' % template_params # export the constructor section constructor_params = ', '.join(self.sections['constructor']) code += '(%s)\n' % constructor_params # export the inside section in_indent = indent*2 for line in self.sections['inside']: code += in_indent + line + '\n' # write the scope section and end it if not needs_scope: code += indent + ';\n' else: code += indent + ');\n' for line in self.sections['scope']: code += indent + line + '\n' # write the contents of the nested classes for nested_unit in nested_codeunits: code += '\n' + nested_unit.Section('module') # close the scope code += indent + 'delete %s;\n' % scope_name # write the code to the module section in the codeunit codeunit.Write('module', code + '\n') # write the declarations to the codeunit declarations = '\n'.join(self.sections['declaration']) for nested_unit in nested_codeunits: declarations += nested_unit.Section('declaration') if declarations: codeunit.Write('declaration', declarations + '\n') declarations_outside = '\n'.join(self.sections['declaration-outside']) if declarations_outside: codeunit.Write('declaration-outside', declarations_outside + '\n') # write the includes to the codeunit includes = '\n'.join(self.sections['include']) for nested_unit in nested_codeunits: includes += nested_unit.Section('include') if includes: codeunit.Write('include', includes) def Add(self, section, item): 'Add the item into the corresponding section' self.sections[section].append(item) def ExportBasics(self): '''Export the name of the class and its class_ statement.''' class_name = self.class_.FullName() if not pyste_future.call_dispatch: self.Add('template', class_name) name = self.info.rename or self.class_.name self.Add('constructor', '"%s"' % name) def ExportBases(self, exported_names): 'Expose the bases of the class into the template section' hierarchy = self.class_.hierarchy exported = [] for level in hierarchy: for base in level: if base.visibility == Scope.public and base.name in exported_names: exported.append(base.name) if exported: break if exported: code = namespaces.python + 'bases< %s > ' % (', '.join(exported)) self.Add('template', code) def ExportConstructors(self): '''Exports all the public contructors of the class, plus indicates if the class is noncopyable. ''' py_ns = namespaces.python indent = self.INDENT def init_code(cons): 'return the init<>() code for the given contructor' param_list = [p.FullName() for p in cons.parameters] min_params_list = param_list[:cons.minArgs] max_params_list = param_list[cons.minArgs:] min_params = ', '.join(min_params_list) max_params = ', '.join(max_params_list) init = py_ns + 'init< ' init += min_params if max_params: if min_params: init += ', ' init += py_ns + ('optional< %s >' % max_params) init += ' >()' return init constructors = [x for x in self.public_members if isinstance(x, Constructor)] self.constructors = constructors[:] # don't export constructors if the class is abstract if self.class_.abstract: for cons in constructors: if cons.IsCopy(): constructors.remove(cons) break if not constructors: # declare no_init self.Add('constructor', py_ns + 'no_init') else: # write the constructor with less parameters to the constructor section smaller = None for cons in constructors: if smaller is None or len(cons.parameters) < len(smaller.parameters): smaller = cons assert smaller is not None self.Add('constructor', init_code(smaller)) constructors.remove(smaller) # write the rest to the inside section, using def() for cons in constructors: code = '.def(%s)' % init_code(cons) self.Add('inside', code) # check if the class is copyable if not self.class_.HasCopyConstructor() or self.class_.abstract: self.Add('template', namespaces.boost + 'noncopyable') def ExportVariables(self): 'Export the variables of the class, both static and simple variables' vars = [x for x in self.public_members if isinstance(x, Variable)] for var in vars: if self.info[var.name].exclude: continue name = self.info[var.name].rename or var.name fullname = var.FullName() if var.type.const: def_ = '.def_readonly' else: def_ = '.def_readwrite' code = '%s("%s", &%s)' % (def_, name, fullname) self.Add('inside', code) def OverloadName(self, method): 'Returns the name of the overloads struct for the given method' name = makeid(method.FullName()) overloads = '_overloads_%i_%i' % (method.minArgs, method.maxArgs) return name + overloads def GetAddedMethods(self): added_methods = self.info.__added__ result = [] if added_methods: for name, rename in added_methods: decl = self.GetDeclaration(name) self.info[name].rename = rename result.append(decl) return result def ExportMethods(self): '''Export all the non-virtual methods of this class, plus any function that is to be exported as a method''' declared = {} def DeclareOverloads(m): 'Declares the macro for the generation of the overloads' if (isinstance(m, Method) and m.static) or type(m) == Function: func = m.FullName() macro = 'BOOST_PYTHON_FUNCTION_OVERLOADS' else: func = m.name macro = 'BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS' code = '%s(%s, %s, %i, %i)\n' % (macro, self.OverloadName(m), func, m.minArgs, m.maxArgs) if code not in declared: declared[code] = True self.Add('declaration', code) def Pointer(m): 'returns the correct pointer declaration for the method m' # check if this method has a wrapper set for him wrapper = self.info[m.name].wrapper if wrapper: return '&' + wrapper.FullName() else: return m.PointerDeclaration() def IsExportable(m): 'Returns true if the given method is exportable by this routine' ignore = (Constructor, ClassOperator, Destructor) return isinstance(m, Function) and not isinstance(m, ignore) and not m.virtual methods = [x for x in self.public_members if IsExportable(x)] methods.extend(self.GetAddedMethods()) staticmethods = {} for method in methods: method_info = self.info[method.name] # skip this method if it was excluded by the user if method_info.exclude: continue # rename the method if the user requested name = method_info.rename or method.name # warn the user if this method needs a policy and doesn't have one method_info.policy = exporterutils.HandlePolicy(method, method_info.policy) # check for policies policy = method_info.policy or '' if policy: policy = ', %s%s()' % (namespaces.python, policy.Code()) # check for overloads overload = '' if method.minArgs != method.maxArgs and not method_info.wrapper: # add the overloads for this method DeclareOverloads(method) overload_name = self.OverloadName(method) overload = ', %s%s()' % (namespaces.pyste, overload_name) # build the .def string to export the method pointer = Pointer(method) code = '.def("%s", %s' % (name, pointer) code += policy code += overload code += ')' self.Add('inside', code) # static method if isinstance(method, Method) and method.static: staticmethods[name] = 1 # add wrapper code if this method has one wrapper = method_info.wrapper if wrapper and wrapper.code: self.Add('declaration-outside', wrapper.code) # export staticmethod statements for name in staticmethods: code = '.staticmethod("%s")' % name self.Add('inside', code) def MakeNonVirtual(self): '''Make all methods that the user indicated to no_override no more virtual, delegating their export to the ExportMethods routine''' for member in self.class_: if type(member) == Method and member.virtual: member.virtual = not self.info[member.name].no_override def ExportVirtualMethods(self, codeunit): # check if this class has any virtual methods has_virtual_methods = False for member in self.class_: if type(member) == Method and member.virtual: has_virtual_methods = True break holder = self.info.holder if has_virtual_methods: generator = _VirtualWrapperGenerator(self.class_, self.ClassBases(), self.info, codeunit) if holder: self.wrapper_name = holder(generator.FullName()) else: self.wrapper_name = generator.FullName() self.Add('template', self.wrapper_name ) for definition in generator.GenerateDefinitions(): self.Add('inside', definition) self.Add('declaration', generator.GenerateVirtualWrapper(self.INDENT)) else: if holder: self.Add('template', holder(self.class_.FullName())) # operators natively supported by boost BOOST_SUPPORTED_OPERATORS = '+ - * / % ^ & ! ~ | < > == != <= >= << >> && || += -= '\ '*= /= %= ^= &= |= <<= >>='.split() # create a map for faster lookup BOOST_SUPPORTED_OPERATORS = dict(zip(BOOST_SUPPORTED_OPERATORS, range(len(BOOST_SUPPORTED_OPERATORS)))) # a dict of operators that are not directly supported by boost, but can be exposed # simply as a function with a special name BOOST_RENAME_OPERATORS = { '()' : '__call__', } # converters which have a special name in python # it's a map of a regular expression of the converter's result to the # appropriate python name SPECIAL_CONVERTERS = { re.compile(r'(const)?\s*double$') : '__float__', re.compile(r'(const)?\s*float$') : '__float__', re.compile(r'(const)?\s*int$') : '__int__', re.compile(r'(const)?\s*long$') : '__long__', re.compile(r'(const)?\s*char\s*\*?$') : '__str__', re.compile(r'(const)?.*::basic_string<.*>\s*(\*|\&)?$') : '__str__', } def ExportOperators(self): 'Export all member operators and free operators related to this class' def GetFreeOperators(): 'Get all the free (global) operators related to this class' operators = [] for decl in self.declarations: if isinstance(decl, Operator): # check if one of the params is this class for param in decl.parameters: if param.name == self.class_.FullName(): operators.append(decl) break return operators def GetOperand(param): 'Returns the operand of this parameter (either "self", or "other<type>")' if param.name == self.class_.FullName(): return namespaces.python + 'self' else: return namespaces.python + ('other< %s >()' % param.name) def HandleSpecialOperator(operator): # gatter information about the operator and its parameters result_name = operator.result.name param1_name = '' if operator.parameters: param1_name = operator.parameters[0].name # check for str ostream = 'basic_ostream' is_str = result_name.find(ostream) != -1 and param1_name.find(ostream) != -1 if is_str: namespace = namespaces.python + 'self_ns::' self_ = namespaces.python + 'self' return '.def(%sstr(%s))' % (namespace, self_) # is not a special operator return None frees = GetFreeOperators() members = [x for x in self.public_members if type(x) == ClassOperator] all_operators = frees + members operators = [x for x in all_operators if not self.info['operator'][x.name].exclude] for operator in operators: # gatter information about the operator, for use later wrapper = self.info['operator'][operator.name].wrapper if wrapper: pointer = '&' + wrapper.FullName() if wrapper.code: self.Add('declaration-outside', wrapper.code) else: pointer = operator.PointerDeclaration() rename = self.info['operator'][operator.name].rename # check if this operator will be exported as a method export_as_method = wrapper or rename or operator.name in self.BOOST_RENAME_OPERATORS # check if this operator has a special representation in boost special_code = HandleSpecialOperator(operator) has_special_representation = special_code is not None if export_as_method: # export this operator as a normal method, renaming or using the given wrapper if not rename: if wrapper: rename = wrapper.name else: rename = self.BOOST_RENAME_OPERATORS[operator.name] policy = '' policy_obj = self.info['operator'][operator.name].policy if policy_obj: policy = ', %s()' % policy_obj.Code() self.Add('inside', '.def("%s", %s%s)' % (rename, pointer, policy)) elif has_special_representation: self.Add('inside', special_code) elif operator.name in self.BOOST_SUPPORTED_OPERATORS: # export this operator using boost's facilities op = operator is_unary = isinstance(op, Operator) and len(op.parameters) == 1 or\ isinstance(op, ClassOperator) and len(op.parameters) == 0 if is_unary: self.Add('inside', '.def( %s%sself )' % \ (operator.name, namespaces.python)) else: # binary operator if len(operator.parameters) == 2: left_operand = GetOperand(operator.parameters[0]) right_operand = GetOperand(operator.parameters[1]) else: left_operand = namespaces.python + 'self' right_operand = GetOperand(operator.parameters[0]) self.Add('inside', '.def( %s %s %s )' % \ (left_operand, operator.name, right_operand)) # export the converters. # export them as simple functions with a pre-determined name converters = [x for x in self.public_members if type(x) == ConverterOperator] def ConverterMethodName(converter): result_fullname = converter.result.FullName() result_name = converter.result.name for regex, method_name in self.SPECIAL_CONVERTERS.items(): if regex.match(result_fullname): return method_name else: # extract the last name from the full name result_name = makeid(result_name) return 'to_' + result_name for converter in converters: info = self.info['operator'][converter.result.FullName()] # check if this operator should be excluded if info.exclude: continue special_code = HandleSpecialOperator(converter) if info.rename or not special_code: # export as method name = info.rename or ConverterMethodName(converter) pointer = converter.PointerDeclaration() policy_code = '' if info.policy: policy_code = ', %s()' % info.policy.Code() self.Add('inside', '.def("%s", %s%s)' % (name, pointer, policy_code)) elif special_code: self.Add('inside', special_code) def ExportNestedClasses(self, exported_names): nested_classes = [x for x in self.public_members if isinstance(x, NestedClass)] for nested_class in nested_classes: nested_info = self.info[nested_class.name] nested_info.include = self.info.include nested_info.name = nested_class.FullName() exporter = self.__class__(nested_info) exporter.SetDeclarations(self.declarations) codeunit = SingleCodeUnit(None, None) exporter.Export(codeunit, exported_names) self.nested_codeunits.append(codeunit) def ExportNestedEnums(self, exported_names): nested_enums = [x for x in self.public_members if isinstance(x, ClassEnumeration)] for enum in nested_enums: enum_info = self.info[enum.name] enum_info.include = self.info.include enum_info.name = enum.FullName() exporter = EnumExporter(enum_info) exporter.SetDeclarations(self.declarations) codeunit = SingleCodeUnit(None, None) exporter.Export(codeunit, exported_names) self.nested_codeunits.append(codeunit) def ExportSmartPointer(self): smart_ptr = self.info.smart_ptr if smart_ptr: class_name = self.class_.FullName() smart_ptr = smart_ptr % class_name self.Add('scope', '%sregister_ptr_to_python< %s >();' % (namespaces.python, smart_ptr)) def ExportOpaquePointerPolicies(self): # check all methods for 'return_opaque_pointer' policies methods = [x for x in self.public_members if isinstance(x, Method)] for method in methods: return_opaque_policy = return_value_policy(return_opaque_pointer) if self.info[method.name].policy == return_opaque_policy: macro = exporterutils.EspecializeTypeID(method.result.name) if macro: self.Add('declaration-outside', macro) def ExportAddedCode(self): if self.info.__code__: for code in self.info.__code__: self.Add('inside', code) #============================================================================== # Virtual Wrapper utils #============================================================================== def _ParamsInfo(m, count=None): if count is None: count = len(m.parameters) param_names = ['p%i' % i for i in range(count)] param_types = [x.FullName() for x in m.parameters[:count]] params = ['%s %s' % (t, n) for t, n in zip(param_types, param_names)] #for i, p in enumerate(m.parameters[:count]): # if p.default is not None: # #params[i] += '=%s' % p.default # params[i] += '=%s' % (p.name + '()') params = ', '.join(params) return params, param_names, param_types class _VirtualWrapperGenerator(object): 'Generates code to export the virtual methods of the given class' def __init__(self, class_, bases, info, codeunit): self.class_ = copy.deepcopy(class_) self.bases = bases[:] self.info = info self.wrapper_name = makeid(class_.FullName()) + '_Wrapper' self.virtual_methods = None self._method_count = {} self.codeunit = codeunit self.GenerateVirtualMethods() SELF = 'py_self' def DefaultImplementationNames(self, method): '''Returns a list of default implementations for this method, one for each number of default arguments. Always returns at least one name, and return from the one with most arguments to the one with the least. ''' base_name = 'default_' + method.name minArgs = method.minArgs maxArgs = method.maxArgs if minArgs == maxArgs: return [base_name] else: return [base_name + ('_%i' % i) for i in range(minArgs, maxArgs+1)] def Declaration(self, method, indent): '''Returns a string with the declarations of the virtual wrapper and its default implementations. This string must be put inside the Wrapper body. ''' pyste = namespaces.pyste python = namespaces.python method_name = method.name rename = self.info[method.name].rename or method.name result = method.result.FullName() return_str = 'return ' if result == 'void': return_str = '' params, param_names, param_types = _ParamsInfo(method) constantness = '' if method.const: constantness = ' const' # call_method callback decl = indent + '%s %s(%s)%s%s {\n' % (result, method.name, params, constantness, method.Exceptions()) param_names_str = ', '.join(param_names) if param_names_str: param_names_str = ', ' + param_names_str self_str = self.SELF if not pyste_future.call_dispatch: decl += indent*2 + '%(return_str)s%(python)scall_method< %(result)s >' \ '(%(self_str)s, "%(rename)s"%(param_names_str)s);\n' % locals() else: decl += indent*2 + '%(return_str)sthis->get_override("%(method_name)s")' \ '(%(param_names_str)s);\n' % locals() decl += indent + '}\n' # default implementations (with overloading) def DefaultImpl(method, param_names): 'Return the body of a default implementation wrapper' indent2 = indent * 2 wrapper = self.info[method.name].wrapper if not wrapper: # return the default implementation of the class return indent2 + '%s%s(%s);\n' % \ (return_str, method.FullName(), ', '.join(param_names)) else: if wrapper.code: self.codeunit.Write('declaration-outside', wrapper.code) # return a call for the wrapper params = ', '.join(['this'] + param_names) return indent2 + '%s%s(%s);\n' % (return_str, wrapper.FullName(), params) if not pyste_future.call_dispatch: if not method.abstract and method.visibility != Scope.private: minArgs = method.minArgs maxArgs = method.maxArgs impl_names = self.DefaultImplementationNames(method) for impl_name, argNum in zip(impl_names, range(minArgs, maxArgs+1)): params, param_names, param_types = _ParamsInfo(method, argNum) decl += '\n' decl += indent + '%s %s(%s)%s {\n' % (result, impl_name, params, constantness) decl += DefaultImpl(method, param_names) decl += indent + '}\n' return decl def MethodDefinition(self, method): '''Returns a list of lines, which should be put inside the class_ statement to export this method.''' # dont define abstract methods pyste = namespaces.pyste rename = self.info[method.name].rename or method.name default_names = [] if not pyste_future.call_dispatch: default_names = self.DefaultImplementationNames(method) class_name = self.class_.FullName() wrapper_name = pyste + self.wrapper_name result = method.result.FullName() is_method_unique = method.is_unique constantness = '' if method.const: constantness = ' const' # create a list of default-impl pointers minArgs = method.minArgs maxArgs = method.maxArgs if method.abstract: default_pointers = [] elif is_method_unique: default_pointers = ['&%s::%s' % (wrapper_name, x) for x in default_names] else: default_pointers = [] for impl_name, argNum in zip(default_names, range(minArgs, maxArgs+1)): param_list = [x.FullName() for x in method.parameters[:argNum]] params = ', '.join(param_list) signature = '%s (%s::*)(%s)%s' % (result, wrapper_name, params, constantness) default_pointer = '(%s)&%s::%s' % (signature, wrapper_name, impl_name) default_pointers.append(default_pointer) # get the pointer of the method pointer = method.PointerDeclaration() if method.abstract: pointer = namespaces.python + ('pure_virtual(%s)' % pointer) # warn the user if this method needs a policy and doesn't have one method_info = self.info[method.name] method_info.policy = exporterutils.HandlePolicy(method, method_info.policy) # Add policy to overloaded methods also policy = method_info.policy or '' if policy: policy = ', %s%s()' % (namespaces.python, policy.Code()) # generate the defs definitions = [] # basic def if default_pointers: definitions.append('.def("%s", %s, %s%s)' % (rename, pointer, default_pointers[-1], policy)) for default_pointer in default_pointers[:-1]: definitions.append('.def("%s", %s%s)' % (rename, default_pointer, policy)) else: definitions.append('.def("%s", %s%s)' % (rename, pointer, policy)) return definitions def FullName(self): return namespaces.pyste + self.wrapper_name def GenerateVirtualMethods(self): '''To correctly export all virtual methods, we must also make wrappers for the virtual methods of the bases of this class, as if the methods were from this class itself. This method creates the instance variable self.virtual_methods. ''' def IsVirtual(m): if type(m) is Method: pure_virtual = m.abstract and m.virtual virtual = m.virtual and m.visibility != Scope.private return virtual or pure_virtual else: return False # extract the virtual methods, avoiding duplications. The duplication # must take in account the full signature without the class name, so # that inherited members are correctly excluded if the subclass overrides # them. def MethodSig(method): if method.const: const = ' const' else: const = '' if method.result: result = method.result.FullName() else: result = '' params = ', '.join([x.FullName() for x in method.parameters]) return '%s %s(%s)%s%s' % ( result, method.name, params, const, method.Exceptions()) already_added = {} self.virtual_methods = [] for member in self.class_: if IsVirtual(member): already_added[MethodSig(member)] = None self.virtual_methods.append(member) for base in self.bases: base_methods = [copy.deepcopy(x) for x in base if IsVirtual(x)] for base_method in base_methods: self.class_.AddMember(base_method) all_methods = [x for x in self.class_ if IsVirtual(x)] for member in all_methods: sig = MethodSig(member) if IsVirtual(member) and not sig in already_added: self.virtual_methods.append(member) already_added[sig] = 0 def Constructors(self): return self.class_.Constructors(publics_only=True) def GenerateDefinitions(self): defs = [] for method in self.virtual_methods: exclude = self.info[method.name].exclude # generate definitions only for public methods and non-abstract methods if method.visibility == Scope.public and not exclude: defs.extend(self.MethodDefinition(method)) return defs def GenerateVirtualWrapper(self, indent): 'Return the wrapper for this class' # generate the class code class_name = self.class_.FullName() if not pyste_future.call_dispatch: code = 'struct %s: %s\n' % (self.wrapper_name, class_name) else: code = 'struct %s: %s, %swrapper<%s>\n' % (self.wrapper_name, class_name, namespaces.python, class_name) code += '{\n' # generate constructors (with the overloads for each one) for cons in self.Constructors(): # only public constructors minArgs = cons.minArgs maxArgs = cons.maxArgs # from the min number of arguments to the max number, generate # all version of the given constructor cons_code = '' for argNum in range(minArgs, maxArgs+1): params, param_names, param_types = _ParamsInfo(cons, argNum) if params: params = ', ' + params cons_code += indent + '%s(PyObject* %s_%s):\n' % \ (self.wrapper_name, self.SELF, params) cons_code += indent*2 + '%s(%s), %s(%s_) {}\n\n' % \ (class_name, ', '.join(param_names), self.SELF, self.SELF) code += cons_code # generate the body body = [] for method in self.virtual_methods: if not self.info[method.name].exclude: body.append(self.Declaration(method, indent)) body = '\n'.join(body) code += body + '\n' # add the self member if not pyste_future.call_dispatch: code += indent + 'PyObject* %s;\n' % self.SELF code += '};\n' return code #this is experemental future of boost python #next lines will describe effect of this future #you have to enable it explicitly in order to get it. call_dispatch = True """ call_dispatch = False current behaviour struct Wrapper { Wrapper(PyObject* o) : mSelf(o) {} PyObject* mSelf; }; struct P { virtual ~P(){} virtual std::string f() = 0; std::string g() { return "P::g()"; } }; struct PWrapper : P, Wrapper { PWrapper (PyObject* self) : Wrapper(self) {} std::string f() { return call_method<std::string>(mSelf, "f"); } }; struct Q : virtual P { std::string f() { return "Q::f()"; } }; BOOST_PYTHON_MODULE_INIT(polymorphism_ext) { class_<P,boost::noncopyable, PWrapper>("P") .def("f", pure_virtual(&P::f)) ; class_<Q, bases >("Q") .def("g", &P::g) ; } call_dispatch = True next generation behaviour, in this case you will need boost 1.32 struct P { virtual ~P(){} virtual std::string f() = 0; std::string g() { return "P::g()"; } }; struct PWrapper : P, wrapper { char const* f() { return this->get_override("f")(); } }; struct Q : virtual P { std::string f() { return "Q::f()"; } }; BOOST_PYTHON_MODULE_INIT(polymorphism_ext) { class_<PCallback,boost::noncopyable>("P") .def("f", pure_virtual(&P::f)) ; class_<Q, bases >("Q") .def("g", &P::g) ; } """

2 1

New-style polymorphism
by David Abrahams July 28, 2004

July 28, 2004

I just checked in changes that improve the way we can wrap classes with virtual functions in Boost.Python. This is essentially what's described in http://www.boost.org/libs/python/todo.html#base-class-for-virtual-function-… The differences are most apparent if you compare libs/python/test/polymorphism.cpp and libs/python/test/polymorphism2.cpp. * you no longer need an initial "self" argument to yourconstructor. Instead you derive your class X from wrapper<X> (I'm looking for better names for that base class) * You no longer need to define a special "default" member function for the default behavior, and you only have to pass one member function pointer to "def()" * You no longer need to use call_method; you instead ask the wrapper<X> base class for the method override, and then call that instead. If there's no override, you call the default directly. On some broken compilers you may need to use call<result_type>(...) args for some of the calls (always on vc6/7, for reference return types only on some other compielrs). Most other reasonably conforming compilers will convert the result of calling the override directly into the function's return type. * It should be much faster in the case where you have a default implementation that isn't overridden in Python, because you'll usually not have to enter the Python interpreter. * Many problems with dangling references have been eliminated, because you don't have to take a round-trip through Python. You can wrap virtual functions that return char const* ! I hope to get this documented before the release, but in case I don't, the old method still works; I'd like people to get some experience with this new feature though, so I can see if it's working out as it should. It would be great if Pyste could be switched to generate the appropriate code. Thanks, -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

2 4