Mailman 3 March 2013 - Cplusplus-sig

boost::python embedding error - runtime - Mac OS X - 1.38
by Igor Karpov Oct. 1, 2014

Oct. 1, 2014

Hi, I am trying to run the following program: #include <iostream> #include <Python.h> #include <boost/python.hpp> using namespace boost::python; using namespace std; int main(int argc, char** argv) { { // Using Python/C Py_Initialize(); PyObject* main_module = PyImport_ImportModule("__main__"); PyObject* globals = PyEval_GetGlobals(); PyObject* locals = PyEval_GetLocals(); PyRun_SimpleString("print 'Hello World, from Python/C!'\n"); Py_Finalize(); } { // Using boost::python Py_Initialize(); object main_module = import("__main__"); object main_namespace = main_module.attr("__dict__"); try { object ignored = exec("print 'Hello World, from boost::python!'\n", main_namespace); Py_Finalize(); } catch (error_already_set const& e) { PyErr_Print(); return 1; } } return 0; } on Linux (both x86 and x86_64), with Ubuntu's default version of Boost (1.34.1) installed, I am able to do so as expected: $ ./embed Hello World, from Python/C! Hello World, from boost::python! on Mac OS X, I get a runtime error: $ ./embed Hello World, from Python/C! Bus Error Running with gdb, this is the stack trace: Hello World, from Python/C! Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_PROTECTION_FAILURE at address: 0x00000000 0x00000000 in ?? () (gdb) where #0 0x00000000 in ?? () #1 0x00bb618f in PyEval_GetGlobals () #2 0x00bce0dd in PyImport_Import () #3 0x00bce2b0 in PyImport_ImportModule () #4 0x008127d4 in boost::python::import (name=@0xbffff4a8) at libs/python/src/import.cpp:20 #5 0x00004446 in main (argc=1, argv=0xbffff510) at embed.cc:22 For this Mac, boost version is latest (1.38) and python version is: Python 2.5.4 (r254:67917, Dec 23 2008, 14:57:27) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Is this a bug? I attached the source and the cmake file I am using to build, for convenience. Thanks, --Igor.

4 6

sendkeys method
by sam Sept. 30, 2013

Sept. 30, 2013

Hello, Is there away(method) in C/C++ programming that make the software think that someone pressed the OK button, similar to sendkeys in VBS (WshShell.SendKeys ("{Enter}")) Thank you, Sam

4 5

question on boost.python exception mechanisms
by Holger Joukl April 30, 2013

April 30, 2013

Hi, I'm wrapping a C++ library that's actually just a thin wrapper around a C lib. Through a dispatch() method of a Dispatcher class there's an invocation of callbacks which get implemented on the Python side, by subclassing callback classes exposed through Boost.Python. Now, for performing the actual dispatch the C++ library calls into the underlying C library. This hasn't been a problem so far as we've been targetting Solaris using Sun/Oracle studio compilers. However, on Linux compiled with GCC, if an exception gets raised in the Python callback the program segfaults. Obviously the C part of the vendor library has not been compiled with (GCC-) exception support for Linux. Unfortunately this isn't under my control and the library provider seems not to be willing to make the library robust against exceptions in user callback code. So I need to keep the Boost C++ exception that "signals" the Python exception from passing through the C parts. Instead of introducing some custom error/exception notification scheme I *think* I can simply do this: As I need wrapper classes for the virtual method override mechanisms anyway I can just ignore/suppress the exception in the callback wrapper layer and re-throw the exception again in the dispatcher, provided that in between we do not return to Python and no other call into Python is made: I.e. The callback wrapper simply suppresses the C++ exception "signalled" for the Python exception: class CallbackWrap : public Callback, public bp::wrapper<Callback> { // pure virtual so no need for default implementation // virtual void onMsg(MsgListener* listener, Msg& msg) = 0; virtual void onMsg(MsgListener* listener, Msg& msg) { try { this->get_override("onMsg")(bp::ptr(listener), boost::ref (msg)); } catch (const bp::error_already_set& e) { if (PyErr_Occurred()) { // Do nothing: The Dispatcher that invoked the callback // MUST check if a Python error has been set and re-throw } } } }; The Dispatcher is then responsible for re-throwing, to not let the Python exception go undetected when returning to the Python world: // virtual Status dispatch (); virtual Status dispatch() { Status ret; if (bp::override f = this->get_override("dispatch")) { ret = f(); // *note* } else { ret = Queue::dispatch(); } if (PyErr_Occurred()) { // Re-throw the error that stems from the Python exception in the dispatched callback bp::throw_error_already_set(); } return ret; } Does this sound sane? Any hints appreciated, Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart

3 14

Segfaults in object deallocation
by Alex Leach April 17, 2013

April 17, 2013

Dear list, I've started using Boost.Python to wrap a 3rd party C++ library, and whilst running my unit tests, the Python interpreter segfault's during garbage collection. With explicit object deletion: $ gdb -q --args python -c 'import mylib; obj = mylib.MyObj(); del(obj)' ... *** Error in `/usr/bin/python': free(): invalid next size (fast): 0x0000000000806fd0 *** ======= Backtrace: ========= /usr/lib/libc.so.6(+0x7ab06)[0x7ffff74d3b06] /usr/lib/libc.so.6(+0x7b883)[0x7ffff74d4883] /usr/lib/libboost_python.so.1.53.0(_ZN5boost6python15instance_holder10deallocateEP7_objectPv+0x15)[0x7ffff004f555] /usr/lib/libboost_python.so.1.53.0(+0x265a1)[0x7ffff004f5a1] ... ======= Memory map: ======== ... or, leaving it to the garbage collector:- $ gdb -q --args python -c 'import mylib; obj = mylib.obj() ' ... Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7b2b0b0 in visit_decref () from /usr/lib/libpython2.7.so.1.0 (gdb) Quick question: What do I need to do to fix this? ========================== --- The wrapper class --- The wrapper class I've written is fairly simple; I haven't explicitly added a 'PyObject* _self' attribute, nor done any of the wiki recommendations[3] for managing ownership, as I don't see anywhere in the tutorial that recommends any of these methods. ------------------------------------ --- Using boost::python::wrapper --- The wrapped class (in 3rd party library) does have one virtual method (that is protected and I haven't exposed to Python) and a virtual destructor. I gather the right thing to do with virtual classes is to also inherit from 'boost::python::wrapper<Base>', but this doesn't affect the seg-fault. I guess I don't need to inherit from wrapper<Base>, as I am not exposing any virtual methods, nor using 'this->get_override(...)' anywhere in the class definition. Another very similar class is in the same source file though (virtual destructor and one unexposed virtual method), but this one has a public copy constructor. If I inherit from 'wrapper<Base>', I get a compilation error regarding an argument mismatch when Boost.Python passes the derived class to the copy constructor of the base class. Of course I can add noncopyable, to the class_<..> registration, but I'd like the exposed class to stay copyable, and I might like to use some of the wrapper template class' functionality. Should I wrap the copy constructor in the wrapper class too? Would it be possible to add this functionality to the wrapper<> template, when the class is not noncopyable? ------------------------------------ --- Calling thread-safe methods --- I don't think this is directly relevant, but the classes in question use the 3rd party library's own mutex implementations to ensure thread-safety. I've just got these working by writing a simple utility class that releases the GIL in the constructor and reacquires it in its destructor, using the C-Python API PyGILState_* functions[1]. The general tactic was described on the boost.python wiki[2], but the PyEval_(Save|Restore)Thread functions described caused the same assertion errors that I was getting earlier, with the pthreads mutex. This happened when the 3rd party library tried to lock the global mutex. Although this is mentioned in the C-Python manual, I would have found it useful if thread-safety was specifically mentioned in the tutorial or reference manual somewhere... I attach the solution I wrote to this problem, in case someone would want to add it and document it in Boost.Python somewhere. Usage: { MakeThreadsafe scope; //< GIL acquired // call function that acquires pthreads mutex. } //< GIL released when leave scope ------------------------------------ --- Garbage collection segfaul --- Any help with this would be really appreciated! I thought that registering a wrapped class with class_<...> was enough, to ensure an object's ref count was incremented and decremented properly. I guess that's the fundamentals of the problem, but am not sure the easiest / best / most efficient way of solving the problem.. Again, some specific documentation, on when and why this problem occurs and how to solve it, in the tutorial would be great! Looking forward to hearing back... Kind regards, Alex [1] - http://docs.python.org/2.7/c-api/init.html#non-python-created-threads [2] - http://wiki.python.org/moin/boost.python/HowTo#Multithreading_Support_for_m… [3] - http://wiki.python.org/moin/boost.python/HowTo#ownership_of_C.2B-.2B-_object -- Using Opera's mail client: http://www.opera.com/mail/

5 13

const_iterator type
by Alex Leach March 29, 2013

March 29, 2013

Hi, I'm trying to wrap an iterable class, which returns a custom iterator typedef'd as const_iterator, from its begin and end methods. When using boost::python::(iterator|iterators|range) to create an iterator in the exposed __iter__ function in the exposed class_ def, I get a load of compile errors:- /usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:166:53: error: no type named 'iterator_category' in 'class mylib::MyConstIterator' /usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:167:53: error: no type named 'value_type' in 'class mylib::MyConstIterator' /usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:168:53: error: no type named 'difference_type' in 'class mylib::MyConstIterator' /usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:169:53: error: no type named 'pointer' in 'class mylib::MyConstIterator' /usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:170:53: error: no type named 'reference' in 'class mylib::MyConstIterator' Here's a simple example demonstrating the problem:- // const_iterator.cpp #include <boost/python/class.hpp> #include <boost/python/copy_const_reference.hpp> #include <boost/python/iterator.hpp> #include <boost/python/module.hpp> // External lib declaration // namespace notmylib { // forward declaration class MyConstIterator; class MyData { public: MyData(void); virtual ~MyData(void); // ... typedef MyConstIterator const_iterator; const_iterator begin(void) const; const_iterator end(void) const; }; class MyConstIterator { public: typedef std::pair<int, int> TRange; // constructors MyConstIterator(void); MyConstIterator(const MyData); // dtor ~MyConstIterator(void) {} // copy constructor MyConstIterator(const MyConstIterator& iter); // assignment operator MyConstIterator& operator= (const MyConstIterator& iter); // Comparison operators bool operator==(const MyConstIterator& iter) const; bool operator!=(const MyConstIterator& iter) const; // Get Data Range TRange GetRange(void) const; // Go backwards one step void Rewind(void); private: MyData m_data; size_t m_index; }; } BOOST_PYTHON_MODULE(const_iterator) { boost::python::class_<notmylib::MyData, boost::noncopyable>("Data") .def("__iter__", boost::python::iterator< const notmylib::MyData, boost::python::return_value_policy< boost::python::copy_const_reference> >() ) // ... ; } Although this doesn't have the member function definitions, so won't work as a complete example from Python, it completely replicates the compile error I get with the actual class I'm trying to wrap. Command to compile, using GCC 4.7.2:- g++ -c -L/usr/lib -g -O1 -fPIC -I/usr/include/python2.7 const_iterator.cpp -o const_iterator.o -lboost_python I couldn't find any examples using const_iterator in the test directory, or elsewhere, so can't find anything to go by. I thought to manually add those typedef's the compilers complaining about, to a Python wrapper class derived from MyIterator. Is there another class I can inherit from, in boost or the STL, to do this for me? Any alternative suggestions or recommendations would be appreciated! Kind regards, Alex --

2 3

Re: [C++-sig] Segfaults in object deallocation
by Jeffrey Van Voorst March 21, 2013

March 21, 2013

This could be misinformation, but some of the boost libraries have compile time flags that depend on whether or not threading is enabled. As an example, I was running into issues with an invalid free() with respect to shared_ptr (in the cctbx project). Unfortunately, (for your case) the fix was to download a more recent version of the cctbx project that no longer exhibited this problem. In short, the boost smart pointers, boost spirit, etc. implementations vary depending on the compile time options. I don't know if boost.python has options. Also, I found that with these issues it is generally easier to pinpoint the problem when the code is compile without optimization. --Jeff

2 2

Re: [C++-sig] Segfaults in object deallocation
by Alex Leach March 20, 2013

March 20, 2013

On Wed, 20 Mar 2013 19:27:34 -0000, Jim Bosch <talljimbo(a)gmail.com> wrote: > (btw, this reply seemed to only go to me rather than the list) oh, sorry... Need to get into a habit of double-checking the reply address.. > > On 03/20/2013 02:16 PM, Alex Leach wrote: >> void PyEnv::ResetPyList(const boost::python::list& envp) >> { >> int envc = boost::python::len(envp); >> const char** pyenvp = new const char*[envc+1]; >> boost::python::stl_input_iterator<std::string> begin(envp), end; >> int i=0; >> while (begin != end) >> { >> pyenvp[i++] = (*begin).c_str(); >> begin++; >> } >> pyenvp[i] = '\0'; >> const char* const* _envp = &pyenvp[0]; >> Environment::Reset(_envp); >> delete[] pyenvp; >> } > > This actually looks like it could be your problem, and I think the dict > version has the same issue. > > When the stl_input_iterator makes a std::string from the Python object > you've passed in, it's a temporary, and when you extract a c_str() from > it, the pointer that's returned is invalidated as soon as the > std::string is destroyed (which is probably as soon as the iterator is > incremented). This would probably explain an error I was getting yesterday, actually.. Thanks! Environment::Reset uses 'strchr' to search for the "=" character, and prints a diagnostic message if it's not found. I kept seeing that message, and it showed that the char* it found was garbage. However, the above method completes its call to Environment::Reset without issue now (later got segfaults because of no NULL pointer at the end of the array). > > So you're passing a whole lot of invalid char* pointers into > Environment::Reset. Note that they're invalid before you pass them in, > so it doesn't matter if Environment::Reset does a deep copy of what's > passed in. > > This wouldn't explain why you're seeing the problem in the constructor > instead of the destructor, but sometimes memory issues are like that. err, isn't it the other way around? The constructor's complete successfully (verified with plenty of Python print and C++ printf statements), only the destructor seg-faults... Thanks for the help though; appreciated! Kind regards, Alex

1 0

Extracting constant pointer to char* array from list
by Alex Leach March 20, 2013

March 20, 2013

Hi, I was wondering the best way to solve this problem:- void WrapperClass::parse_args(boost::python::list argv) { int argc = boost::python::len(argv); const char* const* pargv = boost::python::extract<const char* const*>(argv); /// <---- WrappedClass::parse_args(argc, pargv); } This doesn't work unfortunately. (Sorry, that's not very helpful, I know; I can't remember the exact error right now). I've had a look through boost/python/extract.hpp and frankly have no idea how I'd extend it for this use case, but I have hacked together a rather bloated and repetitive solution that I copy and paste each time I have to do the same thing. e.g. void WrapperClass::parse_args(const boost::python::list& argv) { int argc = boost::python::len(argv); // - Could be a one-liner call to extract<>():- const char** pyargv = new const char*[argc+1]; boost::python::stl_input_iterator<std::string> begin(argv), end; int i=0; while (begin != end) { pyargv[i++] = (*begin).c_str(); begin++; } pyargv[i] = '\0'; const char* const* p_argv = &pyargv[0]; /// ---------- WrappedClass::parse_args(argc, p_argv); delete[] pyargv; } This took quite a lot of trial and error to get running correctly, but is there a way to perform this type of extraction already? Could I extend boost::python::extract to do it for me? If so, please could someone show me how I could? Cheers, Alex -- Using Opera's mail client: http://www.opera.com/mail/

1 0

Exposing c++ class template to Python (known template instances)
by dbarak March 17, 2013

March 17, 2013

Assume I have this Interface: class FooInterface { public: ... }; class FooFilePolicy : public FooInterface { ... }; class FooIoPolicy : public FooInterface { ... }; And I have this template using the upper interface policy: template< typename FooPolicy > class BarWithPolicy { ... FooPolicy* policy; ... }; How can I expose BarWithPolicy to python that can be generically instantiate with both FooIoPolicy and FooFilePolicy ? I don't want to do this : BarWithPolicy<FooFilePolicy >& GetBarWithPolicyFile() { return new BarWithPolicy<FooFilePolicy >; } BarWithPolicy<FooIoPolicy>& GetBarWithPolicyIo() { return new BarWithPolicy<FooIoPolicy >; } BOOST_PYTHON_MODULE(PyBar) { def("GetBarWithPolicyFile", &GetBarWithPolicyFile, return_value_policy<reference_existing_object>(), "" ); def("GetBarWithPolicyIo", &GetBarWithPolicyIo, return_value_policy<reference_existing_object>(), "" ); } I would like to have some thing like this in python: >>> barPolicy = GetBarWithPolic(Io) Or >>> barPolicy = GetBarWithPolic(File) Thanks -Dotan -- View this message in context: http://boost.2283326.n4.nabble.com/Exposing-c-class-template-to-Python-know… Sent from the Python - c++-sig mailing list archive at Nabble.com.

3 4

CallPolicy for operators
by Michael Wild March 15, 2013

March 15, 2013

Hi all Is there a way to apply a CallPolicy to operator definitions? In particular, I'm interested in the inplace operators (+=, -=, *=, /= and friends). To give a bit more context: The library I'm trying to wrap exposes some static const objects. So far, I have been able to wrap modifying function by having a registration facility for these static const objects and a custom CallPolicy that raises an AttributError when one tries to modify one of the registered objects. One option I see is to directly define __iadd__ etc. instead of using the built-in convenience operators. However, I'm not sure whether that has unintended side-effects. I haven't completely understood the operators implementation in bp, but it looks fairly involved. Thanks Michael

3 3