Injecting void* and HWND handling code from Py++
Hello all! I had a question about how to handle void* and HWNDs from within Python using Boost::Python and Py++. Reading through the Py++ documentation, I'm loving how I can slice and dice my classes/wrappers and seemingly do whatever I want with them, but there's still A LOT I haven't figured out. I've found a previous set of similar questions here: http://boost.2283326.n4.nabble.com/Wrapping-and-Passing-HWND-with-Boost-Pyth... but didn't fully understand the automated implementation details. All the relevant code is posted at the end, but my main questions were as follows. (1) Using the win32gui Python library, HWNDs are passed around as ints describing the address of the window. These can be reinterpret_cast'ed into HWNDs in C++ and that seems to work fine. However, using Py++, how can I automatically inject a reinterpret_cast into Foo's constructor, or the initializer list of a constructor wrapper? I've found these commands, .add_declaration_code(...) .add_registration_code(...) .add_wrapper_code(...) but they don't place code where I need it to go. So, is there a command that I should be using? My goal is to not touch my Foo.hpp code (as in, no added code, no helper code), so I would like everything to stay within the Boost::Python generated wrapper code (FooWrapper.cpp). (2) From Python, I would like to manipulate the memory of a buffer stored within Foo. In my REAL code, the only accessor I can use is Address() which returns a void pointer, which Py++/BP flips into an opaque pointer (which I don't think I can use to manipulate m_buffer's memory in Python). One option is ctypes, but I frankly just don't know how to combine that with BP for the purpose of manipulating data behind a void*, even after having looked at the ctypes integration code in the Py++ docs. The other option is to do something like what I did with the fake VoidPtr() accessor. I could create accessors like VoidPtrAsUChar() for example, and then expose those using the return_addressof call policy (hopefully!). If this is, in fact, a valid approach, then I have the same question as above... Given that I can't change the Foo.hpp code, is there a clean way of injecting those AsUChar, AsInt, As... helper methods straight into the wrapper code, so that they call Address() internally and then reinterpret_cast the returned void* into something more useful and accessible in Python? Thanks! -SJ -------------------------------------------------- Here is my trivial Foo class: #include <windows.h> class Foo { public: Foo( const HWND window ) : m_wnd( window ) { const unsigned int size = 10; m_buffer = new unsigned char[size]; for( unsigned int i = 0; i < size; ++i ) m_buffer[i] = i+1; } ~Foo() { delete[] m_buffer; } int IsWindow() { if ( ::IsWindow( m_wnd ) ) return 1; return -1; } void* Address() // In my real code, this is the interface to access memory { return reinterpret_cast< void* >( m_buffer ); } // Fake example accessors void* VoidPtr() { return reinterpret_cast< void* >( m_buffer ); } int* VoidPtrAsInt() { return reinterpret_cast< int* >( m_buffer ); } unsigned char* VoidPtrAsUChar() { return reinterpret_cast< unsigned char* >( m_buffer ); } char* VoidPtrAsChar() { return reinterpret_cast< char* >( m_buffer ); } private: HWND m_wnd; unsigned char* m_buffer; // This buffer could be of any type in real code Foo(); Foo(const Foo&); }; -------------------------------------------------- Here is my Py++ code: mb = module_builder.module_builder_t( files=fileList , gccxml_path=pathToGccXml , working_directory=workingDirectory , compiler='msvc9' ) mb.member_functions( return_type='char *' ).call_policies = call_policies.return_value_policy( call_policies.return_addressof ) mb.member_functions( return_type='int *' ).call_policies = call_policies.return_value_policy( call_policies.return_addressof ) mb.member_functions( return_type='unsigned char *' ).call_policies = call_policies.return_value_policy( call_policies.return_addressof ) mb.build_code_creator( module_name='FooWrapper' ) FooWrapper = os.path.join( os.path.abspath('.'), 'FooWrapper.cpp' ) if os.path.exists( FooWrapper ): os.remove( FooWrapper ) mb.write_module( FooWrapper ) -------------------------------------------------- Which produces this generated code: namespace bp = boost::python; BOOST_PYTHON_MODULE(FooWrapper){ { //::Foo typedef bp::class_< Foo, boost::noncopyable > Foo_exposer_t; Foo_exposer_t Foo_exposer = Foo_exposer_t( "Foo", bp::init< HWND__ *
(( bp::arg("window") )) ); bp::scope Foo_scope( Foo_exposer ); bp::implicitly_convertible< HWND const, Foo >(); { //::Foo::Address
typedef void * ( ::Foo::*Address_function_type )( ) ; Foo_exposer.def( "Address" , Address_function_type( &::Foo::Address ) , bp::return_value_policy< bp::return_opaque_pointer >() ); } { //::Foo::IsWindow typedef int ( ::Foo::*IsWindow_function_type )( ) ; Foo_exposer.def( "IsWindow" , IsWindow_function_type( &::Foo::IsWindow ) ); } { //::Foo::VoidPtr typedef void * ( ::Foo::*VoidPtr_function_type )( ) ; Foo_exposer.def( "VoidPtr" , VoidPtr_function_type( &::Foo::VoidPtr ) , bp::return_value_policy< bp::return_opaque_pointer >() ); } { //::Foo::VoidPtrAsChar typedef char * ( ::Foo::*VoidPtrAsChar_function_type )( ) ; Foo_exposer.def( "VoidPtrAsChar" , VoidPtrAsChar_function_type( &::Foo::VoidPtrAsChar ) , bp::return_value_policy< pyplusplus::call_policies::return_addressof >() ); } { //::Foo::VoidPtrAsInt typedef int * ( ::Foo::*VoidPtrAsInt_function_type )( ) ; Foo_exposer.def( "VoidPtrAsInt" , VoidPtrAsInt_function_type( &::Foo::VoidPtrAsInt ) , bp::return_value_policy< pyplusplus::call_policies::return_addressof >() ); } { //::Foo::VoidPtrAsUChar typedef unsigned char * ( ::Foo::*VoidPtrAsUChar_function_type )( ) ; Foo_exposer.def( "VoidPtrAsUChar" , VoidPtrAsUChar_function_type( &::Foo::VoidPtrAsUChar ) , bp::return_value_policy< pyplusplus::call_policies::return_addressof >() ); } } } -- View this message in context: http://boost.2283326.n4.nabble.com/Injecting-void-and-HWND-handling-code-fro... Sent from the Python - c++-sig mailing list archive at Nabble.com.
On Mon, May 28, 2012 at 6:37 AM, Suresh Joshi <sureshj@gmail.com> wrote:
Hello all!
I had a question about how to handle void* and HWNDs from within Python using Boost::Python and Py++. Reading through the Py++ documentation, I'm loving how I can slice and dice my classes/wrappers and seemingly do whatever I want with them, but there's still A LOT I haven't figured out.
I've found a previous set of similar questions here: http://boost.2283326.n4.nabble.com/Wrapping-and-Passing-HWND-with-Boost-Pyth... but didn't fully understand the automated implementation details.
All the relevant code is posted at the end, but my main questions were as follows.
(1) Using the win32gui Python library, HWNDs are passed around as ints describing the address of the window. These can be reinterpret_cast'ed into HWNDs in C++ and that seems to work fine. However, using Py++, how can I automatically inject a reinterpret_cast into Foo's constructor, or the initializer list of a constructor wrapper?
I've found these commands, .add_declaration_code(...) .add_registration_code(...) .add_wrapper_code(...) but they don't place code where I need it to go.
So, is there a command that I should be using? My goal is to not touch my Foo.hpp code (as in, no added code, no helper code), so I would like everything to stay within the Boost::Python generated wrapper code (FooWrapper.cpp).
What you can do is to generate global function, which takes HWND as integer, does casting and returns your object (via std::auto_ptr). Then, you can add registration code, that use boost::python::make_constructor function to create your class.
(2) From Python, I would like to manipulate the memory of a buffer stored within Foo. In my REAL code, the only accessor I can use is Address() which returns a void pointer, which Py++/BP flips into an opaque pointer (which I don't think I can use to manipulate m_buffer's memory in Python).
One option is ctypes, but I frankly just don't know how to combine that with BP for the purpose of manipulating data behind a void*, even after having looked at the ctypes integration code in the Py++ docs.
The other option is to do something like what I did with the fake VoidPtr() accessor. I could create accessors like VoidPtrAsUChar() for example, and then expose those using the return_addressof call policy (hopefully!). If this is, in fact, a valid approach, then I have the same question as above...
Given that I can't change the Foo.hpp code, is there a clean way of injecting those AsUChar, AsInt, As... helper methods straight into the wrapper code, so that they call Address() internally and then reinterpret_cast the returned void* into something more useful and accessible in Python?
I think Py++ repository contains ctypes + boost.python integration code example. It worse to check it. Regards, Roman.
On 27 May 2012 at 20:37, Suresh Joshi wrote:
(2) From Python, I would like to manipulate the memory of a buffer stored within Foo. In my REAL code, the only accessor I can use is Address() which returns a void pointer, which Py++/BP flips into an opaque pointer (which I don't think I can use to manipulate m_buffer's memory in Python).
One option is ctypes, but I frankly just don't know how to combine that with BP for the purpose of manipulating data behind a void*, even after having looked at the ctypes integration code in the Py++ docs.
The other option is to do something like what I did with the fake VoidPtr() accessor. I could create accessors like VoidPtrAsUChar() for example, and then expose those using the return_addressof call policy (hopefully!). If this is, in fact, a valid approach, then I have the same question as above...
Given that I can't change the Foo.hpp code, is there a clean way of injecting those AsUChar, AsInt, As... helper methods straight into the wrapper code, so that they call Address() internally and then reinterpret_cast the returned void* into something more useful and accessible in Python?
If you really, really want to do this, I would have the wrapper of your Address method return an array-like interface e.g. like a std::vector<>. Look into the indexing suite - a method of implementing direct image bitmap access from python came up on this list a few years back. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/
I'm having a problem with some python objects derived in python. I have a Clone method that's supposed to make a full copy of these python inherited objects, but it isn't working. The problem is my wrapper class' PyObject* self isn't getting copied, so when I use call_method<>(self, "method name"), self will never point to the copied instance. Here's a brief example showing the issue C++ file: #include <boost/python.hpp>#include <scripting/Python/ScriptHelpers.h>using namespace boost::python; class BaseClass{public: typedef boost::shared_ptr<BaseClass> ClonePtr; BaseClass() : ID(++IDCounter) { } BaseClass(const BaseClass& cp) : ID(++IDCounter) { } virtual boost::shared_ptr<BaseClass> Clone() { return boost::shared_ptr<BaseClass>(new BaseClass(*this)); } virtual int GetID() const { return ID; } virtual void Test() { std::cout << "C++ side ID: " << ID << "\n"; }protected: int ID;private: static int IDCounter;}; class BaseClassWrap : public BaseClass{public: BaseClassWrap(PyObject* self_) : self(self_) { } BaseClassWrap(PyObject* self_, const BaseClass& cp) : self(self_), BaseClass(cp) { } BaseClassWrap(PyObject* self_, const BaseClassWrap& cp) : self(self_), BaseClass(cp) { } virtual int GetID() const override { return call_method<int>(self, "GetID"); } virtual void Test() { call_method<void>(self, "Test"); } void TestDefault() { BaseClass::Test(); } int GetIDDefault() const { return BaseClass::GetID(); } boost::shared_ptr<BaseClass> Clone() override { return call_method<ClonePtr>(self, "Clone"); } boost::shared_ptr<BaseClass> CloneDefault() const { return boost::shared_ptr<BaseClass>(new BaseClassWrap(*this)); } PyObject* self;private:}; BOOST_PYTHON_MODULE(TestModule){ class_<BaseClass, boost::shared_ptr<BaseClassWrap> >("BaseClass", init<>()) .def(init<const BaseClass&>()) .def(init<const BaseClassWrap&>()) .def("GetID", &BaseClassWrap::GetIDDefault) .def("Clone", &BaseClassWrap::CloneDefault) .def("Test", &BaseClassWrap::TestDefault) .def("__copy__", &generic__copy__< BaseClassWrap >) .def("__deepcopy__", &generic__deepcopy__< BaseClassWrap >) ;} int BaseClass::IDCounter = 0; int main(){ PyImport_AppendInittab("TestModule", initTestModule); Py_Initialize(); try { auto EngineModule = object( (handle<>(PyImport_ImportModule("TestModule"))) ); auto MainModule = object((handle<>(borrowed(PyImport_AddModule("__main__"))))); auto MainNamespace = MainModule.attr("__dict__"); MainNamespace["TestModule"] = EngineModule; object result = exec_file("D:\\FinalReport\\Test.py", MainNamespace, MainNamespace); object test = MainNamespace["test"]; boost::shared_ptr<BaseClass> basec = extract<boost::shared_ptr<BaseClass> >(test()); if (basec.get() != nullptr) { auto cl = basec->Clone(); basec->Test(); cl->Test(); std::cout << "Original ID " << basec->GetID() << "\n"; std::cout << "Clone ID " << cl->GetID() << "\n"; } } catch (boost::python::error_already_set) { PyErr_Print(); } return 0;} generic__copy__ and generic__deep__copy are in ScriptHelpers and shown below. They were pulled from an old thread on this mailing list and look like: #define PYTHON_ERROR(TYPE, REASON) \ { \ PyErr_SetString(TYPE, REASON); \ throw error_already_set(); \ } template<class T> inline PyObject * managingPyObject(T *p) { return typename manage_new_object::apply<T *>::type()(p); } template<class Copyable> object generic__copy__(object copyable) { Copyable *newCopyable(new Copyable(extract<const Copyable &>(copyable))); object result(boost::python::detail::new_reference(managingPyObject(newCopyable))); extract<dict>(result.attr("__dict__"))().update( copyable.attr("__dict__")); return result; } template<class Copyable> object generic__deepcopy__(object copyable, dict memo) { object copyMod = import("copy"); object deepcopy = copyMod.attr("deepcopy"); Copyable *newCopyable(new Copyable(extract<const Copyable &>(copyable))); object result(boost::python::detail::new_reference(managingPyObject(newCopyable))); // HACK: copyableId shall be the same as the result of id(copyable) //in Python - // please tell me that there is a better way! (and which ;-p) int copyableId = (int)(copyable.ptr()); memo[copyableId] = result; extract<dict>(result.attr("__dict__"))().update( deepcopy(extract<dict>(copyable.attr("__dict__"))(), memo)); return result; } and the python file: from TestModule import *import copy class Der(BaseClass): def Test(self): print "Python Side Test. ID: " + str(self.GetID()) def Clone(self): print "Python Clone" return copy.deepcopy(self) def test(): return Der() Sorry about the length, just want to make sure I got everything relevant in. When I run this code, it prints: Python Clone Python Side Test. ID: 1Python Side Test. ID: 1Original ID 1Clone ID 1 this is wrong because Clone ID should be 2 (inspecting the object confirms it is 2). Like I said I'm pretty sure the problem is that the wrapper class' copy constructor only makes a copy of the PyObject* self, not a full copy. This is how every reference doc I saw was doing class wrapping, but isn't this wrong? Should we be making a full copy of the PyObject*? How would I go about doing that, or otherwise modifying my classes to get the behaviour I expect. Thanks
On 05/31/2012 08:49 PM, Jay Riley wrote:
I'm having a problem with some python objects derived in python. I have a Clone method that's supposed to make a full copy of these python inherited objects, but it isn't working. The problem is my wrapper class' PyObject* self isn't getting copied, so when I use call_method<>(self, "method name"), self will never point to the copied instance. Here's a brief example showing the issue
<snip>
Sorry about the length, just want to make sure I got everything relevant in. When I run this code, it prints:
Python Clone Python Side Test. ID: 1 Python Side Test. ID: 1 Original ID 1 Clone ID 1
this is wrong because Clone ID should be 2 (inspecting the object confirms it is 2). Like I said I'm pretty sure the problem is that the wrapper class' copy constructor only makes a copy of the PyObject* self, not a full copy. This is how every reference doc I saw was doing class wrapping, but isn't this wrong? Should we be making a full copy of the PyObject*? How would I go about doing that, or otherwise modifying my classes to get the behaviour I expect.
There are a couple of things here that probably need to be addressed: - Be wary of relying on copy-constructor side effects to demonstrate whether something is working; the compiler is allowed to elide copy constructors under many conditions, which might result in misleading results. That said, I don't think that's what's causing your problem here (or at least not all of it). - Your implementations of __copy__ and __deepcopy__ won't work with polymorphic wrapper classes, because they construct a BaseClass instance and tell Boost.Python to wrap it by managing the new reference. That necessarily makes a Python object that holds a BaseClass object, not one that holds a BaseClassWrap object. If you want to get a BaseClassWrap Python object, you want to pass it by value to Boost.Python; that should invoke the BaseClassWrap copy constructor and pass it a new PyObject* to manage (so it doesn't have to copy the PyObject* itself). I think that just means you can remove the "managingPyObject" function, and instead use: object result(extract<const Copyable &>(copyable)()); HTH Jim
participants (5)
-
Jay Riley -
Jim Bosch -
Niall Douglas -
Roman Yakovenko -
Suresh Joshi