Mailman 3 November 2006 - Cplusplus-sig

Heap cleanup in python extension
by Alexander Eisenhuth Nov. 16, 2006

Nov. 16, 2006

Hello Together, hope it's ok to post a bit beside c++. Shortly what I'm doing: - Extending python with boost.pthon extension - Using python C-Api for datatypes in the extension - extension has a thread (that can be stopped/started) - thread started: extension updates a dict (given as parameter to the extension) every 20 ms - the dict looks like: {int:{int:[float,float,float] int:[float,float,float], ...}} Now my question: I create the dict every 20 ms on the heap in the following way: ------------------------------ static PyObject* createPTDict() { static int aNumIter = 0; PyObject* nRetDictPy = PyDict_New(); // dicto to return PyObject* nToolIdPy = PyInt_FromLong(1234567); PyObject* nLedDict = PyDict_New(); // dict containing 5 key,values for (int aLedNum=0; aLedNum < 5; aLedNum++) { PyObject* aLedNumPy = PyInt_FromLong(aLedNum); PyObject* aLedList = PyList_New(0); for ( int aXYZ=0; aXYZ < 3; aXYZ++) { PyObject* aLedPosXYZ = PyFloat_FromDouble(1.0); PyList_Append(aLedList, aLedPosXYZ); } PyDict_SetItem(nLedDict, aLedNumPy, aLedList); } PyDict_SetItem(nRetDictPy, nToolIdPy, nLedDict); aNumIter ++; return nRetDictPy; } ------------------------------ I give it to python with: ------------------------------ PyObject *aKey, *aValue; int aPos = 0; PyObject* aNewToolDict = createPTDict(); PyDict_Next(aNewToolDict, &aPos, &aKey, &aValue); // DictInPython passed prior to extension PyDict_SetItem(<DictInPython>, aKey, aValue); ------------------------------ And now, what would be a proper way to cleanup the heap? 1) Does PyDict_Clear(aNewToolDict) also decrement the ref counting for the containing list, int, ... PyObjects? 2) Do I need to Py_CLEAR( PyObject *o) for every created PyObject? Thanks a lot for every answer/hint. Regards Alexander

1 0

pyCXX example of a callback
by apocalypznow Nov. 16, 2006

Nov. 16, 2006

Hi, Can anyone post a simple pyCXX example of a callback into python? My python function should allow a string passed as a parameter, and returns a string, ie: def somepythonCB(s_in): s_out = dosomethingtoinputstringandreturnoutputstring(s_in) return s_out Much appreciated, thanks.

3 3

vector<int> -> list
by Johan Johansson Nov. 15, 2006

Nov. 15, 2006

Dear list, What would be the best way to return a rather largish vector of integers to Python from a C++ function? And while I'm at it, what would be the best way of iterating through a boost::python::list? Obviously two very basic questions, but neither google nor grep seems to know the answers. Thanks in advance, Johan

2 1

__str__ method for wrapper classes
by Noah Schwartz Nov. 14, 2006

Nov. 14, 2006

Hi, I have an inheritance scheme in my C++ library. I expose the base class to python using the method described in the tutorial (a struct that inherits from my class and from wrapper). I have the extraction operator implemented for the base class (lets call the base class MyClass). I want to be able to print MyClass as a string in python. I added .def(str(self)) to the class object for the wrapper (MyClassWrap) thinking it would work but, python complains that the __str__ method did not match the C++ signature: Boost.Python.ArgumentError: Python argument types in MyClass.__str__(MyClass) did not match C++ signature: __str__(MyClassWrap {lvalue}) I think the problem is that I defined the __str__ method for MyClassWrap and not for MyClass. Since MyClassWrap also inherits from wrapper, it can't use MyClass's extraction operator. I don't have a class_ object for MyClass. I only have it for MyClassWrap and the other classes that inherit from MyClass. How do I fix this? Noag

2 2

from_python converter problem, unable to match C++ signature
by Mark Kirk Nov. 13, 2006

Nov. 13, 2006

Hi, I have a Python-to-C++ argument conversion problem that I've been scratching my head over for a few days. I'm hoping David or another knowledgeable community member can help me. I'm using boost::python to wrap a C++ lib that I've written. It's gone well for 95% of the lib (and, yeah, boost::python is truly fantastic), but some functions and a constructor have arguments which are a pointer to an in-house C struct. Its library has a corresponding Python extension that was wrapped using the Python C Api, and an exposed function is provided for extracting the pointer to the C struct from the PyObject. Furthermore, the header for the Python extension has an extern declaration for the C struct's corresponding PyTypeObject. This Python extension makes its symbols globally available through a call to sys.setdlopenflags(dl.RTLD_NOW | dl.RTLD_GLOBAL). I am unable to convert function arguments from Python extension objects of the in-house data type to the corresponding C++ pointers; The extension builds (with caveats below), but I get a runtime Boost.Python.ArgumentError telling me that the args do not match the C++ signature. With me so far? I've boiled this down to a simple example which exhibits the problem. For purposes of this discussion: mytest - the example C++ lib that I'm extending via boost::python foo - C library for in-house data type FOO - the in-house C data type inhouse.foo - Python C Api extension of foo library The converter strategy I'm using is similar to boost_1_32_0/libs/python/test/m1.cpp. Here are some code snippets: mytest.h (trivial mytest.cc omitted for brevity) ------------------------------------------------ #include <foo.h> class mytest { public: mytest(); mytest(FOO *aFoo); ~mytest(); void f(FOO *aFoo) const; private: int mX; }; pymytest.cc (boost::python wrapper) ----------------------------------- #include <boost/python.hpp> #include <python/fooobject.h> // from Python extension for FOO #include "mytest.h" #include "foo_internal.h" // *see note1 using namespace boost::python; struct foo_from_python { static FOO*& execute(PyObject& p) { static FOO* a = FooObject_ptr(&p); return a; } }; BOOST_PYTHON_MODULE(_mytest) { lvalue_from_pytype<foo_from_python, &FooObject_Type>(); class_<mytest>("mytest", init<>()) .def("mytest", init<FOO*>()) // *see note2 .def("f", &mytest::f); } __init__.py for mytest extension -------------------------------- # need inhouse.foo's symbols loaded first import inhouse.foo from _mytest import * python/fooobject.h ------------------ // extern "C", etc. #include <Python.h> #include <foo.h> FOO* FooObject_ptr(PyObject* object); // performs a PyObject_TypeCheck() // then returns the embedded FOO* extern PyTypeObject FooObject_Type; __init__.py excerpt for foo extension ------------------------------------- # The author of this extension added some functionality in Python # and chose to inherit from the actual C extension class Foo(_foo.Foo): * note1: This file should *not* be included, but I can't build without it. It defines the FOO type. I get a number of compiler errors from boost::python::type_id() not being able to do its thing because FOO is an undefined type. * note2: The constructor causes a number of compiler errors difficult to summarize or understand beginning in def_vistor_access::visit(). I've left it commented out trying to address the other errors first. Using gcc3.4.3 on RHEL4 Linux with boost 1.32.0, Python 2.4.2. If I include the foo_internal.h header and comment out the constructor taking the FOO* argument, the extensions builds cleanly, but I still have a problem. Here's what happens in Python: Python 2.4.2 (#104, Sep 18 2006, 18:00:37) [GCC 3.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import inhouse.foo >>> from inhouse import mytest >>> fooObj = inhouse.foo.Foo() >>> mytestObj = mytest.mytest() >>> mytestObj.f(fooObj) Traceback (most recent call last): File "<stdin>", line 1, in ? Boost.Python.ArgumentError : Python argument types in mytest.f(mytest, Foo) did not match C++ signature: f(mytest {lvalue}, FOO*) OK, after all that exposition, here are some questions: 1) Is lvalue_from_pytype<> the correct or best way to perform the conversion? Some other converter type? It's easy to write thin wrappers for the member functions which have FOO* args (though tedious) and doing this avoids needing to include the foo_internal.h header for the type_id() problem. However, I don't know how I would wrap the constructor. Beyond that, the FOO data type is heavily used, so a proper converter is desirable. The foo_from_python extractor class above is equivalent to: extract_member<PyFOOObject, FOO*, &PyFOOObject::foo>, &FooObject_Type>() However, PyFOOObject is defined in the .c file of the foo Python extension (though I might be able to convince the author to move it to the header): typedef struct { PyObject_HEAD PyObject* userArgs; FOO* foo; } PyFOOObject; 2) I verified that my converter is being registered by calling converter::registry::query(type_info(typeid(ANI*))), but it seems that it's never used. Have I mismatched a type somehow? I believe boost::python iterates through registered lvalue converters until it finds one that returns non-NULL. Correct? Again, I think my converter is never invoked. I looked into the opaque pointer converters, but it seemed like the wrong path. I can't extract the FOO* from the PyObject using them either, correct? 3) I was concerned that the inhouse.foo.Foo Python class was a problem since the "real" Python FOO type is inhouse.foo._foo.Foo. However, the PyObject_TypeCheck macro appears to properly check subtypes. Also, if I pass in an inhouse.foo._foo.Foo object, I get the same ArgumentError. And, I verified the PyObject's ob_type field is the address of FooObjectType. Despite all of this, does anyone think there might be a problem here? I'm frustrated and forced to admit I'm stuck on this one. If anyone can enlighten me or has a *solution*, I would be incredibly grateful. I feel like this must be a common task, yet I've found few relevant threads addressing converters. This leads me to wonder if I'm missing something obvious. I'm happy to provide any further info. And, if you made it through this, I appreciate your patience! thanks, mark

2 2

Creating a python class from c++
by James Healey Nov. 12, 2006

Nov. 12, 2006

Can any body tell me how you create a python class from within c++? so in my .py file I have class MyClass: def f(self): return 'hello world' And from my c++ code i want to be able to do.... x = new MyClass x.f() oh and im using boost as well, if that makes it any easier. Regards. Jimmy. ___________________________________________________________ Inbox full of spam? Get leading spam protection and 1GB storage with All New Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html

2 1

ArgumentError: Python argument types in None.None
by Anand, Kumar Nov. 12, 2006

Nov. 12, 2006

I have a base class and a derived class (in say userTypes.h & userTypes.cpp) and this is built into a static library (say libuserTypes.a) ---------------------------- class Base { public: int x; Base(int i); }; class Derived : public Base { public: int y; Derived(int i, int j); }; ---------------------------- I exposed both these classes to python (using Boost Python) in a Boost_UserTypes.so module. (Note: My .so links against the static libuserTypes.a). All this works perfectly and I am able to import the module in a python script and use it. Now I built another module Boost_UserInterfaces.so which exposes a C++ interface (using the types from libuserTypes.a). Code is as follows. ---------------------------- namespace TestFrmwkEg_Wrapper { Base* createDerivedTest(int baseX, int derivedY) { return new Derived(baseX, derivedY); } } BOOST_PYTHON_MODULE (Boost_UserInterfaces) { def("createDerivedTest", &TestFrmwkEg_Wrapper::createDerivedTest, bp::return_value_policy<bp::manage_new_object>()); } ---------------------------- Note: My function returns a base class pointer to derived class instance. Boost_UserInterfaces.so also links against the static libuserTypes.a. Now in my python script when I do the following: ---------------------------- import Boost_UserInterfaces import Boost_UserTypes dr = Boost_UserInterfaces.createDerivedTest(3, 8) print dr.y ---------------------------- I get the following error when I try to access attribute 'y' of the derived object. Boost.Python.ArgumentError: Python argument types in None.None(Derived) did not match C++ signature: None(Derived {lvalue}) -Can anyone explain to me why this is happening? I used pdb.set_trace to examine the dr variable and everything looks ok. (Pdb) dir(dr) ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__instance_size__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'x', 'y'] (Pdb) print dr <Boost_UserTypes.Derived object at 0x2b13d969b0c0> I understand that linking the static .a library twice (once in every .so) is a bad thing to do, but I would still like to understand the reasons for this error. If print and dir command are showing the correct derived object as expected, why is the access to the derived class attribute failing? Thanks Kumar

1 0

indexing suite and std::vector<std::string>
by Johan Rydberg Nov. 11, 2006

Nov. 11, 2006

Hi, One of the methods I wrap return a std::vector<std::string>, which needs to be wrapped as well. I was told about the indexing suite and gave it a try; class_<vector<string> >("std_string_vec") .def (vector_indexing_suite<vector<string> > ()); With the following result; >>> repr(x) '<std_string_vec object at 0xb7e37d2c>' >>> len(x) 5 >>> x[0] Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: No Python class registered for C++ class std::string I was under the impression that strings should be magically wrapped by boost.python. Any ideas anyone? ~j

1 1

pyste, abstract classes, policy inheritance
by Will Welch Nov. 10, 2006

Nov. 10, 2006

hi, i'm using pyste on a chain of classes that derive from an abstract base. one of the member functions returns a pointer type that needs a policy declared for it. i'm noticing that i must do the set_policy for this function in each class along the inheritance chain, rather than just once for the abstract base. is there a workaround for this? i'm brand new to boost.python and pyste, unsure if this is a boost thing or a pyste-generated wrapper thing. i don't actually want to ever call that function, by the way, but using exclude() nets me errors about it being an abstract virtual function (presumably because it is left out of the wrapper class), and that causes its wrapper's m_held declaration to fail. how does one use exclude() in this situation? finally, speaking of pyste-generated wrappers and abstract classes: i've noticed that having no virtual functions in a class that derives from an abstract class, means that the needed wrapper for the derived class isnt produced by pyste. current workaround is to add a fake virtual function declaration to the derived class. is there a fix that does not involve changing the c++ source? thanks, -- will

2 1

Boost.Python call policies at runtime...
by Kevin Jones Nov. 9, 2006

Nov. 9, 2006

Hi all, Given a class called Variant that can (at runtime) contain either an int or a T*. I.E. class Variant { public: int type; union { int i; T* t; }; }; How can I create a call policy that uses return_by_value when Variant is an int or manage_new_object when Variant is a T* (based on the value of "type" at runtime)? Note: I need to wrap either the int as a python object or T* as a python object and not Variant. Any help or sugestions would be most appreciated. Best regards, Kevin Jones ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com

5 6