Mailman 3 July 2003 - Cplusplus-sig

smart pointers, polymorphism
by Dusty Leary July 9, 2003

July 9, 2003

Hello, I am converting a semi-large api over to boost.python. We have been using boost::intrusive_ptr for some time now. (I can't remember exactly why). I tried switching before mailing the list. With both intrusive_ptr and shared_ptr, I am running into "conversion to base" problems. I have quite a few abstract interfaces. Each one defines an overloaded templated static "cast" function, to make it easy to cast references. (e.g. foo::bar::MyInterface::cast(someReference)), which handles using dynamic_cast or boost::shared_dynamic_cast, etc... With my previous home-grown python interfaces (stubs generated from an idl), the binding for the cast function would just extract the anonymous pointer from the PyObject (after verifying that the PyObject's type was one of our interfaces), static_cast it to IObject* (even though it might not implement IObject), then dynamic_cast it to the interface type (in this case, foo::bar::MyInterface). I am not sure if static_casting it to some possibly unrelated polymorphic type, and then dynamic_casting it to our type is valid. (Although I was sure it would always be a polymorphic type). It worked, at least for my current environment. Anyway, I can't get the casting to work properly. I am always getting "TypeError: bad argument for built-in operation."... presumably, boost.python can't find a valid overload. I have tried forcing all interfaces in the system to extend a common base interface (IObject), with no luck. (so, I define "static shared_ptr<T> cast_iobject(IObject*)" in each interface class. Even though I have "IFoo : public IObject", a shared_ptr<IFoo> can't be passed into my cast function) Either, I need to special case this function, accepting PyObjects directly and doing the static_cast -> dynamic_cast thing like I did in my previous bindings, or I need to somehow make the compiler and/or boost.python infer the proper behaviour. With all of the boost::python::(converter, holder, pointee) stuff, that might be possible, but I can't make heads or tails of it. What is the proper way?

2 1

Pyste: problems with inheritance across headers.
by Prabhu Ramachandran July 9, 2003

July 9, 2003

Hi, I found a few more serious bugs with Pyste. I split up a simple class hierarchy over several header files and also made different pyste files to wrap these. Pyste has difficulty with executing the module functions in the right order (as I suspected in an earlier post) and also has difficulties with the base classes. The trouble being that the exported_names are grouped by headers and only the exported names grouped by header are known when generating the code for a class. So if a base class is in another header the bases will not be correctly set leading to problems. Possible resolution for this problem: 1. Compute all the exported names in one go and then use that information when generating the code. 2. Additionally, do not use grouping by headers to order the writes. A more sophisticated approach is needed to get the right sequence of module initialization. 3. Assume that all bases are exported by default instead of relying on exported_names. I guess this is not an option. :( The necessity of the exported_names destroys my plans of incremental generation of wrappers without generating an additional file with necessary information. Why? Well, because when you generate wrappers with a single pyste file it has no way of knowing the exported names from other pyste files and therefore will not set the bases correctly. So the only way of doing this would be to allow Pyste to generate the xml file into an _interface.xml in individual steps and then get Pyste to use the generated XML files touching only the wrapper files that changed. My guess is that this will reduce the time taken to generate the interface files. This is what Nicodemus was saying earlier but I learnt it from 'The University of Hard Knocks'. Essentially the --only-wrap and --only-main options are useless. Instead we need a --generate-xml and --use-xml option. --generate-xml simply generates the xml file(s) and --use-xml does not call gccxml and uses the generated xml file(s). Its up to the build system to determine when and how to generate the xml file when the interface file changes and that is quite easy to do. I've attached a tarball of the files that demonstrate the above problem with inheritance across several headers. Anyway, I'd like to know if the above approach for multiple files is possible or I'm missing something again. If it is, I can probably work on a patch sometime, unless Nicodemus has it covered. cheers, prabhu

3 8

Allocating objects on the heap by default.
by Prabhu Ramachandran July 9, 2003

July 9, 2003

Hi, I have a question that I'm not sure has much to do with Boost.Python itself but I'm wrapping some code via Boost.Python and need to know what the best approach is to solve the problem. The code is designed to be used by allocating objects on the heap (via new) and handing them off to very simple containers that manage these objects. The lifetime of the objects is also managed by the container. So here is a quick example. // -------------------- class Obj { public: Obj(); ~Obj(); }; class Mgr { public: Mgr(); ~Mgr(); // deletes memory for objects in container. void add(Obj* a); void popBack(); private: std::vector<Obj*> arr; }; void usage() { Obj *o = new Obj(); Mgr *m = new Mgr(); m->add(o); delete m; } // -------------------- Perhaps the design is totally stupid, but its used consistently everywhere and quite simple once you know you can't delete an object you created and passed to a manager. Juas as the user has to manage and remember to invoke delete for every new he has to remember that the manager manages memory as well. I don't see that as a bad decision but I guess purists would not agree with me. Anyway, I want to wrap the code above (except the usage function) and want to arrange for Boost.Python to allocate all of the objects, Obj and Mgr in the heap by default. After wrapping the above via Boost.Python and using it from Python this is what happens: >>> import test >>> a = test.Obj() >>> b = test.Mgr() >>> b.add(a) >>> del b Segmentation fault. Trouble here is that Obj is not allocated on the heap but on the stack, just as one would expect in C++. Yes, I read the penultimate FAQ entry and my question is: can I arrange the wrappers to automatically allocate Obj and Mgr objects on the heap and additionally be handled by a shared_ptr/auto_ptr without having to write a factory function like NewObj for every single class? I'd like to avoid writing factory functions just to do this since I have lots of classes like this. If the object is allocated on the heap by default and I manage the object via boost::shared_ptr then the above segfault will not happen. Is there a way that this can be handled easily? Alternatively can this be generated via Pyste/Python magic? FWIW and FYI, not that it matters but SWIG does this by default -- allocates everything via new and deals with the resulting opaque pointers. Also, is the above approach of creating objects and handing them off to a manager a bad thing to do in general? Whats a better approach? This is a generic c++ question but the issue arises in the context of wrapping so I thought it sufficient relevant to post here and not a c++ list. I'm sorry if this is inappropriate. Any suggestions will be much appreciated. Thanks! cheers, prabhu

3 20

'boost::STATIC_ASSERTION_FAILURE<false>' problem ...
by Pierre Barbier de Reuille July 9, 2003

July 9, 2003

Hello, I have a type I want to export but without function nor initialisation. The line defining the type is : class_<Delaunay2DGraph::vertex_descriptor>( "cell_descriptor", no_init ); This type is defined in an object defined in a library (the Boost Graph Library). Bur when I try to compile my module I have this error I don't understand : /home/barbier/apps/boost/boost/python/object/make_instance.hpp: In static member function `static PyObject* boost::python::objects::make_instance_impl<T, Holder, Derived>::execute(Arg&) [with Arg = const boost::reference_wrapper<void* const>, T = void*, Holder = boost::python::objects::value_holder<void*>, Derived = boost::python::objects::make_instance<void*, boost::python::objects::value_holder<void*> >]': /home/barbier/apps/boost/boost/python/object/class_wrapper.hpp:27: instantiated from `static PyObject* boost::python::objects::class_cref_wrapper<Src, MakeInstance>::convert(const Src&) [with Src = void*, MakeInstance = boost::python::objects::make_instance<void*, boost::python::objects::value_holder<void*> >]' /home/barbier/apps/boost/boost/python/converter/as_to_python_function.hpp:28: instantiated from `static PyObject* boost::python::converter::as_to_python_function<T, ToPython>::convert(const void*) [with T = void*, ToPython = boost::python::objects::class_cref_wrapper<void*, boost::python::objects::make_instance<void*, boost::python::objects::value_holder<void*> > >]' /home/barbier/apps/boost/boost/python/to_python_converter.hpp:34: instantiated from `boost::python::to_python_converter<T, Conversion>::to_python_converter() [with T = void*, Conversion = boost::python::objects::class_cref_wrapper<void*, boost::python::objects::make_instance<void*, boost::python::objects::value_holder<void*> > >]' /home/barbier/apps/boost/boost/python/class.hpp:101: instantiated from `void boost::python::detail::register_class_to_python(boost::mpl::bool_<true>, SelectHolder, T*) [with T = void*, SelectHolder = boost::python::objects::detail::select_value_holder<void*, void*>]' /home/barbier/apps/boost/boost/python/class.hpp:577: instantiated from `void boost::python::class_<T, X1, X2, X3>::register_() const [with T = void*, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified]' /home/barbier/apps/boost/boost/python/class.hpp:605: instantiated from `boost::python::class_<T, X1, X2, X3>::class_(const char*, boost::python::no_init_t) [with T = void*, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified]' editor_python.cpp:255: instantiated from here /home/barbier/apps/boost/boost/python/object/make_instance.hpp:25: `sizeof' applied to incomplete type `boost::STATIC_ASSERTION_FAILURE<false>' Any idea about what I need to do ? Thanks, -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68

2 3

Compiling for 2.2 and 2.3
by Achim Domma (ProCoders) July 9, 2003

July 9, 2003

Hi, I have a project which takes some time to compile and want to provide windows binaries for python 2.2 and 2.3. Do I have to do a 'bjam -a' each time I change the python version, or is there a faster way? It's not clear to me, how much which part of boost.python depends on the python version. regards, Achim

1 0

Pyste-generated extentsion Aborts Python
by Jim Wilson July 8, 2003

July 8, 2003

Nicodemus, I got the CVS, and it seemed to fix the "overloaded bar(char**)" problem I mentioned before. My goal was to wrap (at least a portion of) the Fast Light ToolKit (FLTK: http://www.fltk.org ). Others are attempting this with SWIG ( http://pyfltk.sourceforge.net/ ), but I was hoping Pyste would simplify the job. Alas, I'm not sure I can simplify the files I'm wrapping, but my wrapper is quite lightweight. Since I didn't want to figure out all those "needs- a-policy" complaints, I wrote a small header which exposed only the routines I needed from FLTK Version 1.1.4. I'm not sure you want this, but here it is: fltk.h: class Fl { // A class masquerading as a namespace, public: // Fl is essentially a buch of static methods static int run(); }; #include <.../Fl_Window.H> // This will be instanciated fltk.pyste: Class("Fl", "fltk.h") w = Class("Fl_Window", "fltk.h") exclude(w.icon) # Easier to exclude than use the exclude(w.child) # trick I did with class Fl, since exclude(w.current) # I may want to *do* something with exclude(w.array) # my Fl_Window! exclude(w.resizable) exclude(w._ddfdesign_kludge) (In fltk.h, <...> was the path prefix my the directory holding Fl_Window.h.) I pysted, beat bjam into submission, and got a clean compile and link. Now came python: >>>from fltk import Fl, Fl_Window >>>w = Fl_Window(0,0, 600,400) >>>w.show() >>>Fl.run() Abort The "w = ..." line, creates a window in the upper left corner (0,0) of size 600x400 pixels. The next draws it on the screen, the last one fires up the event loop. The window is momentarily displayed before it immediately vanishes and "Abort" appears (followed by the bash prompt). The running python is half as big as the SWIG version though ;D. I'd appreciate any advice. Jim Wilson

3 14

Pyste: Resolution of order of exports.
by Prabhu Ramachandran July 8, 2003

July 8, 2003

Hi, I was thinking of something along these lines: 1. While writing the module section for each class, also add a comment that contains all the bases in order (via ClassBases). Something like so: // bases = [B, C, D] void Export_A() 2. I'll then read this 'bases' information when generating _main.cpp and then for each base, ensure that the order is such that each of the bases or ancestral bases is exported earlier. I'm guessing that this should be fairly easy to do and does not seem too bad a hack to be considered inelegant. It also eliminates the need to generate a separate file. Is there a simpler way or an alternative? Thoughts? BTW, in your code you group the exporters based on the header used and then sort the list based on Order. I'm not sure that this approach is guaranteed to work all the time since if the base is in another header the sorting won't work. However, I might be wrong and if you are sure that it will work then we could do that, i.e. write the header info and then the 'Order' then do the same thing that you do. Will this approach work? What do you folks think? Thanks! cheers, prabhu

2 1

Strange behaviour for inheritance with C++ classes.
by Prabhu Ramachandran July 8, 2003

July 8, 2003

Hi, Here is a simple test case for some code that gives me strange behavior under (gcc version 2.95.4 20011002 (Debian prerelease)). Atleast the behavior is not what I expected. $ python -V Python 2.2.1 // -------- test.hpp ------ #include <iostream> class Pure { public: virtual ~Pure(){} virtual void f() = 0; }; class Derived : public Pure { public: Derived(){} ~Derived(){} void f() {cout << "This is Derived::f\n";} }; // -------- test.hpp ------ I wrapped this via Pyste using test = AllFromHeader('test.hpp') I then ran the following test under Python: >>> import test >>> class A(test.Derived): ... def __init__(self): ... test.Derived.__init__(self) ... >>> d = test.Derived() >>> print isinstance(d, test.Derived) 1 >>> print isinstance(d, test.Pure) 0 >>> test.func(d) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: bad argument type for built-in operation >>> a = A() >>> print isinstance(a, test.Derived) 1 >>> print isinstance(a, test.Pure) 0 >>> test.func(a) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: bad argument type for built-in operation >>> class B(test.Pure): ... def __init__(self): ... test.Pure.__init__(self) ... def f(self): ... print "B.f" ... >>> b = B() >>> test.func (b) B.f What surprises me is the following: 1. d is not an instance of test.Pure but if I did everything (i.e. defined all the classes) in pure Python thats the way it would be! 2. test.func should work with anything that is derived from test.Pure but neither did 'd' nor did 'a' work but 'b' did work! So what is going on here and what must I do to see the behaviour that one would expect? My apologies in advance if this is a silly mistake that I've made but I can't see why the above does not work. Perhaps Pyste is doing something incorrectly but I cant see whats wrong there either. Thanks. prabhu

4 5

Pyste: added new build options.
by Prabhu Ramachandran July 7, 2003

July 7, 2003

Hi, I've finished implementing a --only-wrap and --only-main option for Pyste. It works nicely in that I can now generate and build my wrappers incrementally, interface file by interface file. Something like this: pyste.py --only-wrap --module=test --out . --multiple file1.pyste pyste.py --only-wrap --module=test --out . --multiple file2.pyste [...] pyste.py --only-main --module=test --out . \ --multiple file1.pyste [...] fileN.pyste the last command completes in no time since it does not use GCCXML and writes a simple _main.cpp file. Nicodemus, I can mail the list another patch for this but I've already sent two other patches maybe so you'd prefer one big patch that incorporates my earlier two as well, i.e. (a) support for hold_with_*, add path to sys.path, (b) fix for the --multiple bug and (c) support for the --only-wrap and --only-main? Please let me know what you'd prefer and I'll send you the appropriate patch. cheers, prabhu

2 6

Pyste: serious bug with --multiple.
by Prabhu Ramachandran July 7, 2003

July 7, 2003

Hi, Found a serious bug with Pyste's --multiple option. It would not mix up the declarations and put them in wrong files. I've attached a patch that fixes this problem. Generating only one file per --multiple is nice but still has the old problem where if I change one interface file I need to wait several minutes for pyste (gccxml actually) to regenerate the files. Nicodemus suggested that a --xml-cache option would be useful. However even with that option you'd generate many xml files and will need to keep track of those files etc. Instead I have a much simpler approach that will solve much of our problems. 1. Add a --only-wrap option that will process any number of interface files and will generate the necessary wrapper code and *will not* generate a _main.cpp file. Further, it will add an extra function to each _interface_name.cpp file that reads something like this: void _interface_name_module_init() { Export_SomeClass(); Export_SomeHeader_H(); // ... } 2. The --multiple option with the --only-main option will simply create a _main.cpp with something like this: void _interface_name_module_init(); BOOST_PYTHON_MODULE(module_name) { _interface_name_module(); // ... } Note that the above does not need to use gccxml at all since it only needs the interface file names. This should make it possible to selectively compile individual interfaces via --multiple --only-wrap and then generate a _main.cpp separately using all the interfaces very quickly. There are other issues not related to the above also but I won't go into them here. I'm going to go ahead and implement --only-main and --only-wrap on my copy of the code here and will submit it to the list if it works nicely. I'll let you folks decide if it goes into CVS or not. cheers, prabhu diff -ur Pyste.orig/HeaderExporter.py Pyste/HeaderExporter.py --- Pyste.orig/HeaderExporter.py Mon Jul 7 18:07:09 2003 +++ Pyste/HeaderExporter.py Mon Jul 7 13:06:43 2003 @@ -7,6 +7,7 @@ from declarations import * import os.path import exporters +import MultipleCodeUnit #============================================================================== # HeaderExporter @@ -64,7 +65,10 @@ exporter = exporter_type(info) exporter.SetDeclarations(self.declarations) exporter.SetParsedHeader(self.parser_header) - codeunit.SetCurrent(self.interface_file, exporter.Unit()) + if isinstance(codeunit, MultipleCodeUnit.MultipleCodeUnit): + codeunit.SetCurrent(self.interface_file, exporter.Unit()) + else: + codeunit.SetCurrent(exporter.Unit()) exporter.GenerateCode(codeunit, exported_names) diff -ur Pyste.orig/MultipleCodeUnit.py Pyste/MultipleCodeUnit.py --- Pyste.orig/MultipleCodeUnit.py Mon Jul 7 18:06:26 2003 +++ Pyste/MultipleCodeUnit.py Mon Jul 7 16:42:10 2003 @@ -79,24 +79,27 @@ self._CreateOutputDir(); # order all code units by filename, and merge them all codeunits = {} # filename => list of codeunits - # the main_unit holds all the include, declaration and declaration-outside sections - # to keep them all in the top of the source file - main_unit = None + + # While ordering all code units by file name, the first code + # unit in the list of code units is used as the main unit + # which dumps all the include, declaration and + # declaration-outside sections at the top of the file. for (filename, _), codeunit in self.codeunits.items(): - if filename not in codeunits: + if filename not in codeunits: + # this codeunit is the main codeunit. codeunits[filename] = [codeunit] - main_unit = codeunit - main_unit.Merge(self.all) + codeunit.Merge(self.all) else: + main_unit = codeunits[filename][0] for section in ('include', 'declaration', 'declaration-outside'): main_unit.code[section] = main_unit.code[section] + codeunit.code[section] codeunit.code[section] = '' codeunits[filename].append(codeunit) - # write all the codeunits, merging first the contents of - # the special code unit named __all__ - for codeunits in codeunits.values(): + + # Now write all the codeunits appending them correctly. + for file_units in codeunits.values(): append = False - for codeunit in codeunits: + for codeunit in file_units: codeunit.Save(append) if not append: append = True

2 3