Mailman 3 May 2003 - Cplusplus-sig

RE: [C++-sig] Converting CString to/from std::string
by Kirsebom Nikolai Sept. 25, 2003

Sept. 25, 2003

-----Original Message----- From: Ralf W. Grosse-Kunstleve [mailto:rwgk@yahoo.com] Sent: 27. mai 2003 08:46 To: c++-sig(a)python.org Subject: Re: ÆC++-sigÅ Converting CString to/from std::string --- Nikolai Kirsebom <nikolai(a)micon.no> wrote: > I have a rather large set of C++ classes I want to make accessible > (initially only parts of some of the classes) from Python. I cannot > change the classes as they are defined and used throughout a large > application. Some of the methods take CString as parameters and others > return CString values. > > Does anyone have suggestion on how I can convert these parameters so > that I can use Boost.Python to generate the wrapper code. What do you want your CStrings to be in Python? Regular Python strings? If you think "yes" remind yourself that Python strings are immutable. I.e. if you have function signatures involving non-const references or pointers to CString as arguments or return values the mapping from C++ CStrings to Python strings will not work very well. More depending on your response. Ralf Some time ago I made a 'socket based' system (using BISON/FLEX for parsing) where I expose C++ objects to python. In this system CStrings are regular Python strings. So as an example I can write: --------------------------------------------- C++: class IF { CString MenuText; CString GetHelpText(); int SetHelpText(CString s); }; In Python: o = IF() # Behind the 'curtains' create a 'link' to a C++ object print o.MenuText s = o.GetHelpText(o.MenuText + "some text") ---------------------------------------------- So I would like something similar. I'm very new to the Boost system and do not really have experience in C++ Template development so I find the Reference Manual somewhat difficult to understand. This may explain if I request 'dead easy' or 'impossible' things. Thanks for any help Nikolai

4 6

Threads and Boost.Python
by Patrick Hartling June 15, 2003

June 15, 2003

I was able to sort out the multi-threading issues I was having earlier this week with Boost.Python thanks to David Abrahams' suggestions. What I came up with is a relatively simple guard object that is instantiated whenever C++ code calls into the Python interpreter. To make this guard work, I had to deal with two key issues: 1. Ensuring that threads created by C/C++ bootstrapped themselves with the Python interpreter. 2. Preventing deadlocks if the same thread tried to acquire the GIL more than once. Issue #1 was straightforward enough to handle. The C++ code base that I am exposing to Python includes a cross-platform data structure for thread-specific data (TSD). Within the guard, there is a static data member that is an instance of this TSD structure. Our TSD handler is a templated type, and the struct I gave it contained two fields: struct State { State() : isLocked(false), pyState(NULL) {} bool isLocked; PyThreadState* pyState; }; The boolean field is used to deal with Issue #2 from above; the thread state object needs to be different for each unique thread calling into the Python interpreter. The nice thing about our TSD handler is that a new instance of the above structure will be instantiated automatically for each thread, so whenever a new thread creates an instance of the guard, the guard constructor handles filling in the State instance fields. The code from my guard constructor is this: Guard::Guard() : mMyLock(false) { if ( NULL == mState->pyState ) { mState->pyState = PyThreadState_New(PyInterpreterState_New()); } if ( ! mState->isLocked ) { // Acquire the GIL. PyEval_AcquireThread(mState->pyState); mState->isLocked = true; mMyLock = true; } } Here, Guard::mState is the static data member that is an instance of our TSD handler. The destructor is this: Guard::~Guard() { if ( mMyLock && mState->isLocked ) { // Release the GIL. PyEval_ReleaseThread(mState->pyState); mState->isLocked = false; } } The last detail is the data member Guard::mMyLock. That is a boolean to handle keep track of which guard instance actually acquired the GIL so that it is the one to release it upon destruction. Dealing with the recursive lock problem was probably the trickiest part. It's too bad that the GIL cannot be locked twice by the same thread, but perhaps that behavior doesn't exist for all the threading implementations upon which the Python interpreter runs. On the whole, this guard concept is working very well for me, though I haven't even been using it for 48 hours yet. Conceptually, I think something like a "synchronized call" (to steal a Java concept) in Boost.Python would help automate this process. At this time, however, I don't know how well my guard will apply to arbitrary calls into the Python interpreter from a thread. It works like a charm for boost::python::call_method<>(), and that's what I care about the most right now. Two side issues I ran into involved deciding when to call PyEval_InitThreads() and where to put the two macros for allowing and disallowing threads. With the way our software works, the main thread basically doesn't do anything once it starts a microkernel's control thread. The primordial thread just sits and blocks on a semaphore waiting for the microkernel to shut down. From there on, all calls into the Python interpreter from C++ are guaranteed to come from a thread other than the one that started the interpreter. So, I put Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS in a wrapper function that is exposed to Python where the Python "main" function makes the sempahore wait call. There will be other cases I'll have to hunt down as we expand our use of Boost.Python, but at least I have the general idea of what to watch out for. -Patrick -- Patrick L. Hartling | Research Assistant, VRAC patrick(a)vrac.iastate.edu | 2624 Howe Hall: 1.515.294.4916 http://www.137.org/patrick/ | http://www.vrac.iastate.edu/

2 3

long long unsigned issue...
by Milind Patil June 8, 2003

June 8, 2003

Hi, I have used boost libraries to wrap a large c++ library and it has worked very well. However, I stumbled on the issue of long long conversion to class problem. The problem has been reduced to the minimum code below. I would like to expose to python a class that has constructors from int and long long unsigned int among others: class Y { public: Y() : y(0L) { } Y(int y) : y(y) { } Y(long long unsigned int y) : y(y) { } Y(Y const& rhs) : y(rhs.y) { } virtual ~Y() { } operator int() const { return y; } void operator=(Y const& y) { this->y = y.y; } long long unsigned int y; }; I use boost thus: #include <boost/python/module.hpp> #include <boost/python/def.hpp> #include <boost/python.hpp> #include <hello.h> using namespace boost::python; BOOST_PYTHON_MODULE(hello) { class_< Y >("Y", init< >()) .def(init< int >()) .def(init< long long unsigned int >()) .def(init< const Y & >()) .def_readwrite("y", &Y::y) .def("__int__", &Y::operator int) ; implicitly_convertible<int, Y>(); implicitly_convertible<long long unsigned int, Y>(); } and use it in python thus: import hello x = hello.Y(4294967295) print x.y y = hello.Y(0xFFFFFFFFFFFFFFFF) print y.y However the code craps out: Traceback (most recent call last): File "p.py", line 3, in ? x = hello.Y(4294967295) OverflowError: long int too large to convert to int Why wouldn't it try yo convert to long long unsigned int before trying out int? I am seeking a behaviour where python int is converted to c++ int before constructing to c++ object Y, and python long is converted to c++ long long unsigned int before constructing to c++ object Y. It is possible that there are ways to write explicit boost converters from long to Y, but that is too hairy for me. But before I go into it, I was wondering if I have made a silly presumption or missed a simple obvious way to do this. Any pointers will be a great help. Thanks, Milind

4 35

Projects Page Reminder
by David Abrahams June 6, 2003

June 6, 2003

If you have a project using Boost.Python, and you wouldn't mind if it were listed at http://www.boost.org/libs/python/doc/projects.html, please send or post the text of an entry for that page! Thank you very much for your support, -- Dave Abrahams Boost Consulting www.boost-consulting.com

5 9

missing boost::ref in pyste generated code?
by Giulio Eulisse June 4, 2003

June 4, 2003

With the following piece of code: --CUT-HERE-- #include <iostream> using std::cout; using std::endl; class Ev { public: Ev(int ii) : i(ii) { cout << "EV " << i << endl; } virtual ~Ev() { cout << "dead Ev " << i << endl; } int id() const { return i;} virtual void me() const=0; private: int i; }; class REv : public Ev { public: REv(int ii) : Ev(ii){} virtual void me() const { cout << "ReV " << id() << endl; } }; class AA { public: AA() : i(1) { cout << "AA " << i << endl; } virtual ~AA() { cout << "dead AA " << i << endl; } int id() const { return i;} virtual void dump() { cout << "I'm a AA " << i << endl; } virtual void here(Ev* ev) const=0; virtual void there(Ev const&) const=0; private: int i; }; class BB : public AA { public: BB(int jj=2) : j(jj) { cout << "BB " << j << endl; } virtual ~BB() { cout << "dead BB " << j << endl; } int jd() const { return j;} virtual void dump() { cout << "I'm a BB " << j << endl; } virtual void here(Ev* ev) const { cout << "I'm a BB " << j <<" "; ev->me(); } virtual void there(Ev const& ev) const { cout << "I'm a BB " << j <<" "; ev.me(); } private: int j; }; inline void hello(AA* a, int k) { REv v(k); a->here(&v);} inline void hello2(AA* a, int k) { REv v(k); a->there(v);} ---END-CUT--- when pystified with ---CUT-HERE--- Ev = Class("Ev","Utilities/UtExamples/python/btest.hh") REv = Class("REv","Utilities/UtExamples/python/btest.hh") AA = Class("AA","Utilities/UtExamples/python/btest.hh") BB = Class("BB","Utilities/UtExamples/python/btest.hh") hello = Function("hello","Utilities/UtExamples/python/btest.hh") hello2 = Function("hello2","Utilities/UtExamples/python/btest.hh") ---CUT-HERE--- I get the following boostified code: // Includes ==================================================================== #include <boost/python.hpp> #include <Utilities/UtExamples/python/btest.hh> #include <Utilities/UtExamples/python/stest.hh> // Using ======================================================================= using namespace boost::python; // Declarations ================================================================ namespace { struct AA_Wrapper: AA { AA_Wrapper(PyObject* self_, const AA & p0): AA(p0), self(self_) {} AA_Wrapper(PyObject* self_): AA(), self(self_) {} void dump() { call_method< void >(self, "dump"); } void default_dump() { AA::dump(); } void here(Ev * p0) const { call_method< void >(self, "here", p0); //should be //boost::ref(*p0) } void there(const Ev & p0) const { call_method< void >(self, "there", p0); //should be //boost::ref(p0) } PyObject* self; }; struct Ev_Wrapper: Ev { Ev_Wrapper(PyObject* self_, const Ev & p0): Ev(p0), self(self_) {} Ev_Wrapper(PyObject* self_, int p0): Ev(p0), self(self_) {} void me() const { call_method< void >(self, "me"); } PyObject* self; }; struct BB_Wrapper: BB { BB_Wrapper(PyObject* self_, const BB & p0): BB(p0), self(self_) {} BB_Wrapper(PyObject* self_): BB(), self(self_) {} BB_Wrapper(PyObject* self_, int p0): BB(p0), self(self_) {} void dump() { call_method< void >(self, "dump"); } void default_dump() { BB::dump(); } void here(Ev * p0) const { call_method< void >(self, "here", p0); //should be //boost::ref(*p0) } void default_here(Ev * p0) const { BB::here(p0); } void there(const Ev & p0) const { call_method< void >(self, "there", p0); } void default_there(const Ev & p0) const { BB::there(p0); } PyObject* self; }; struct REv_Wrapper: REv { REv_Wrapper(PyObject* self_, const REv & p0): REv(p0), self(self_) {} REv_Wrapper(PyObject* self_, int p0): REv(p0), self(self_) {} void me() const { call_method< void >(self, "me"); } void default_me() const { REv::me(); } PyObject* self; }; }// namespace // Module ====================================================================== BOOST_PYTHON_MODULE(btest) { class_< AA, boost::noncopyable, AA_Wrapper >("AA", init< >()) .def("id", &AA::id) .def("dump", &AA::dump, &AA_Wrapper::default_dump) ; class_< Ev, boost::noncopyable, Ev_Wrapper >("Ev", init< int >()) .def("id", &Ev::id) ; class_< BB, bases< AA > , BB_Wrapper >("BB", init< const BB & >()) .def(init< optional< int > >()) .def("jd", &BB::jd) .def("dump", &BB::dump, &BB_Wrapper::default_dump) .def("here", &BB::here, &BB_Wrapper::default_here) .def("there", &BB::there, &BB_Wrapper::default_there) ; class_< REv, bases< Ev > , REv_Wrapper >("REv", init< const REv & >()) .def(init< int >()) .def("me", &REv::me, &REv_Wrapper::default_me) ; class_< T >("T", init< >()) .def(init< const T & >()) ; def("hello", &hello); def("hello2", &hello2); } ---END--- this fails when doing the following in python: >>> from PyUtExamples import * >>> >>> b = BB() AA 1 BB 2 >>> hello2(b,4) EV 4 dead Ev 4 Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: No to_python (by-value) converter found for C++ type: 2Ev >>> as it seem to pass Ev by value, instead of passing it by reference. We fixed it by adding boost::ref where indicated in the sourcecode as otherwise it tries to pass things by value. Is this a bug or am I doing something wrong? Ciao, Giulio

4 8

custom less than operator? to_python_converter for multiple destination types?
by Gavin Doughtie June 3, 2003

June 3, 2003

So, I'm wrapping a class that defines "<" as an internal class with a bool operator() like so: struct Baz { struct Less { bool operator() (const Baz& a, const Baz& b) const; } }; Inconveniently, it also does NOT define operator < in Baz. How would I wrap this to make Baz objects in Python use Baz::Less::operator() as the < operator? -- Gavin Doughtie DreamWorks SKG (818) 695-3821

4 9

[newbie] Failing to test boost.python under cygwin
by Gilles Orazi June 3, 2003

June 3, 2003

Hello, I am trying to build the Boost.Python examples under cygwin and faces some problems. I set the environement variables like that (taken from a post in the archives) : export PYTHON='E:\cygwin\bin\python.exe' export PYTHON_ROOT='E:\cygwin\lib\python2.2' export PYTHON_VERSION='2.2' export PYTHON_INCLUDES='E:\cygwin\usr\include\python2.2' export PYTHON_LIB_PATH='E:\cygwin\lib\python2.2\config' export PYTHON_STDLIB_PATH='E:\cygwin\lib\python2.2' export GCC_PYTHON_ROOT='E:\cygwin\usr' This is what I get : $ bjam -sTOOLS=gcc test don't know how to make <libs!python!example>test_getting_started1.py don't know how to make E:\cygwin\bin\python.exe don't know how to make <libs!python!example>test_getting_started2.py ...found 1922 targets... ...can't find 3 targets... ...can't make 2 targets... ...skipped <libs!python!example/test1.test/gcc/debug/runtime-link-dynamic>test1. test for lack of <libs!python!example>test_getting_started1.py... ...skipped <libs!python!example/test2.test/gcc/debug/runtime-link-dynamic>test2. test for lack of <libs!python!example>test_getting_started2.py... ...skipped 2 targets... What's going wrong here ? Thanks by advance for your help. --- Gilles

2 2

set_wrapper
by Giulio Eulisse June 3, 2003

June 3, 2003

If I have a simple class class A {} and a function void f_A(void){...} pyste's set_wrapper(A.f,"f_A") should, IMHO, generate class_< A >("A", init< >()) .def(init< const A & >()) .def("f", &f_A) ; even if A::f() is not present. Is there a reason why it doesn't? Thanks in advance, Giulio

3 7

Custom exceptions
by Gavin Doughtie June 1, 2003

June 1, 2003

So, let's say I want to register a custom exception which python code can catch, thusly: try: doSomething() except mymodule.MyException e: print e Is there anything in boost python that makes setting this up easy? I've got a working exception translator and everything, but I don't see anything equivalent to putting "PyErr_NewException("mymodule.MyException", NULL, NULL)" into the system dictionary during module initialization. Or am I just working too hard again? -- Gavin Doughtie DreamWorks SKG

3 2

Re: [newbie] Question on embedding/extending with boost::python
by Dirk Gerrits May 30, 2003

May 30, 2003

Cross-posted to the C++ Sig. Please post all further replies there. Chris Heald wrote: > Hi all -- > > I'm working on an app that is going to be a C++ code engine which will be > heavily extended through a series of Python scripts (objects, really). My > desired requirements are as follows: [snip] > having trouble is with c and d - how do I give a Python app hooks into an > existing C++ state machine? I know I can write an extension easily enough, > but by my understanding, that will simply give the Python object access to a > series of functions, and not the objects contained in the state machine that > the interpreter is running under. I could use "pull" operations, wherein the > C++ app polls the objects for changed state in a loop and updates its sate > accordingly, but that seems that it wouldn't perform nearly so well as if > the Python scripts were able to update the C++ state machine themselves, so > I'm avoiding that route. I'm not entirely sure what your problem is. Have you looked at libs/python/test/embedding.cpp? It shows how your embedded interpreter can get access to the internals of the running C++ program. (Specifically, its classes.) Perhaps you can use that as a starting point? > I had a co-worker suggest that I use Python as my primary execution > environment instead of C++. While this would admittedly make some things > easier, how much performance would I sacrifice with this route? I'm building > a 3D application, and it doesn't seem that Python would be well-suited to > such an application in terms of performance. There is potential for > literally tens of thousands of instances of these objects to exist in this > state machine, and I certainly don't want to hinder performance by moving > the primary execution into an interpreted environment rather than a compiled > environment. Well, my common sense tells me that you may very well be right. But the important point is that there's really no way to tell, except for trying it out. (Profiling is the key.) And surely, writing the whole thing in Python (except for the critical sections) takes less time then writing the whole thing in C++ with just some things written in Python? Taking this road could very well lead to disappointing performance as you say. But then you'd still have gained experience from making the Python prototype, and its C++ bits can most-likely be recycled anyway. I don't really feel qualified to advise you to do one thing or the other. Embedding is sometimes the right/efficient way to do things, but such situations are becoming more and more rare. I am guessing that yours is such a situation, but you could argue that making a decision on such a guess is premature opimization (and therefore evil ;)). Perhaps others here can provide additional insights. Regards, Dirk Gerrits

2 4