Francois Ostiguy wrote:

Here is what I tried (in slightly simplified form): object PythonInstanceOfA = PythonTypeA( py_get_existing() );

object PythonInstanceOfA = py_get_existing(); py_get_existing is already a factory function returning (python) instance of A. HTH, Mike

Thank You Dave, Max and Chez for your insights. I am getting closer to my goal, but it is not quite working yet. In the process I am slowly gaining more understanding of the way boost.python works. I have to say that I never cease to be amazed at how carefully crafted boost.python is. It is truly a remarkable piece of work. Back to my problem: Last Friday, following Max and Dave's suggestions I tried (in simplified form): class A; class_<A, A*, boost::noncopyable> PythonTypeA; A* get_existing() { return aptr; } ... A* aptr = new A(); object py_get_existing = make_function( &get_existing, return_value_policy<reference_existing_object>() ); object PythonInstanceOfA = PythonTypeA( py_get_existing() ); ... Another poster pointed out that make_function is a factory for python objects and therefore the last line was incorrect. Dave added that it was "bogus". When I remove this last line, everything compiles properly, but of course when I print the "py_get_existing" object in python I get <Boost.Python.function object at 0x834cb28>; which is not a wrapped class A object. After thinking about this a bit, it occured to me that what I needed was more along the lines of object PythonInstanceOfA = PythonTypeA( aptr ); /// If I understand correctly, this should instantiate an object using the automatic converter defined by the extended python type class_<A, A*, boost::noncopyable> PythonTypeA. The code compiles without a hitch, but I get a core dump when the statement is executed. I am still investigating what might be going on (*** see note below ***) ). The approach suggested by Chez (chezdique AT yahoo.com) appears more promising. I have yet to try his code, but a cursory look gives me the impression that what he is doing is not fundamentally not different from what I just described. He defines an an empty ObjectWrapper<T> and a correcponding get_ptr() function. Once again, correct me if I am wrong, but this appers superflous since T& and T* converters should automatically be generated by boost.python. An ObjectWrapper class would be useful to support some kind of smart pointers. I apologize for the lengthy post. I hope this is not too confusing and I would greatly appreciate more comments/feedback. -Francois ---------------------------------------------------------------------------- Dr. Jean-Francois OSTIGUY voice: (630) 840-2231 Beam Physics Dept MS220 FAX: (630) 840-6039 Fermi National Accelerator Laboratory email: ostiguy@fnal.gov Batavia IL 60510-0500 WWW:www-ap.fnal.gov/~ostiguy *** note: one complication is that I want to keep all the extended python type definitions together. This means that somehow the PythonTypeA object has to be global. Since there are typically a large number of types declared in each modules, the simplest thing seemed to use a global class_<A, A*, boost::noncopyable>* PythonTypeAPtr and set PythonTypeAPtr = &PythonTypeA.

As Mike and I wrote, just do this:

object PythonInstanceOfA = py_get_existing();

Unfortunately, that does not seem to do quite what I want. I constructed a simple complete toy example to demonstrate. Note the following: (1) creating python_object_a forces an undesired instantiation of a type A (i.e. in addition to the the existing instance a) (2) if the interface defines a python object without an __init__ function, the result is a runtime error: "Object cannot by created from python." (3) if you compile the example, here is what you get when the module is imported from python Python 2.3.3 (#1, Jul 7 2004, 13:43:38) [GCC 3.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information.

...

...

from inject import * a <inject.A object at 0xb733b10c> dir() ['A', '__builtins__', '__doc__', '__name__', 'a'] dir (a) ['A', '__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__instance_size__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__']

Note that the object instance "a" has been injected. However, for a reason that escapes me, the method "dummy" which is part of the specified interface is not correctly exported. Again, my goal is to wrap an existing instance of A, without having to create a spurious object (basically without having to call the A constructor more than once). Comments, suggestions ??? -Francois ---------------------------------------------------------------------------- Dr. Jean-Francois OSTIGUY voice: (630) 840-2231 Beam Physics Dept MS220 FAX: (630) 840-6039 Fermi National Accelerator Laboratory email: ostiguy@fnal.gov Batavia IL 60510-0500 WWW:www-ap.fnal.gov/~ostiguy /////// *************** example: inject.cpp **************** ///////// #include <boost/python.hpp> class A { public: A(int d) { _dummy = d; } int dummy() { return _dummy;} private: int _dummy; }; A* a = 0; A* get_existing() { return a; } void inject_object(); using namespace boost::python; class_ <A, A*, boost::noncopyable>* PythonTypeAPtr; BOOST_PYTHON_MODULE( inject ) { static class_ <A, A*, boost::noncopyable> PythonTypeA("A", init<int>()); //*** note: using no_init here will cause a runtime error !! PythonTypeA.def("A", &A::dummy); /***** this does not seem to export the dummy function to python. Why ? PythonTypeAPtr = &PythonTypeA; inject_object(); } void inject_object() { a = new A(99); object py_get_existing = make_function( &get_existing, return_value_policy< reference_existing_object>() ); object python_object_a = (*PythonTypeAPtr)(2); // <<<<<<========== //****** instantiation of a spurious object python_object_a = py_get_existing(); handle<> main_module ( boost::python::borrowed( PyImport_AddModule("__main__") ) ); handle<> main_dict ( boost::python::borrowed( PyModule_GetDict(main_module.get()) )); PyDict_SetItemString ( main_dict.get(), "a", python_object_a.ptr() ); }

Francois Ostiguy <ostiguy@fnal.gov> writes:

...

As Mike and I wrote, just do this:

object PythonInstanceOfA = py_get_existing();

Unfortunately, that does not seem to do quite what I want. I constructed a simple complete toy example to demonstrate.

Note the following:

(1) creating python_object_a forces an undesired instantiation of a type A (i.e. in addition to the the existing instance a)

So don't do that. It's not needed and it wasn't in the code Mike posted. Please just read his directions very carefully and don't add anything else.

(2) if the interface defines a python object without an __init__ function, the result is a runtime error: "Object cannot by created from python."

The result of what?

(3) if you compile the example, here is what you get when the module is imported from python

Python 2.3.3 (#1, Jul 7 2004, 13:43:38) [GCC 3.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information.

...

...

...

from inject import * a <inject.A object at 0xb733b10c> dir() ['A', '__builtins__', '__doc__', '__name__', 'a'] dir (a) ['A', '__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__instance_size__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__']

Note that the object instance "a" has been injected. However, for a reason that escapes me, the method "dummy" which is part of the specified interface is not correctly exported.

Because you called it "A", not "dummy".

Again, my goal is to wrap an existing instance of A, without having to create a spurious object (basically without having to call the A constructor more than once).

Comments, suggestions ???

Read and follow directions more carefully. Sorry to be blunt, but that is the best I can do. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Francois Ostiguy wrote:

class_< A, A*, boost:noncopyable>

registers a type converter from A* to Python and it is through this converter that the object becomes an instance of the Python type A.

It's (a) create a pytype wrapper class and (b) registers a type converter unless you specify boost:noncopyable as you did.

Does this mean that only one Python type can/should be declared as holder of a A* ? In other words, is it possible to declare two Python types, say A1 and A2, both holding a A* but with different exported interfaces ?

You can't have multiple converters. You'll get a boost compile error trying.

object PythonInstanceOfA( a ) ; // a is a raw ptr to A

does not work.

boost:noncopyable prevents converting c++ value to python. My .02. Let's Dave correct me. Mike

Francois Ostiguy <ostiguy@fnal.gov> writes:

...

...

...

As Mike and I wrote, just do this:

object PythonInstanceOfA = py_get_existing();

Unfortunately, that does not seem to do quite what I want. I constructed a simple complete toy example to demonstrate.

Dave, Mike, and all the others that took the time to reply to my posts:

Your patience is greatly appreciated.

I tried too many things ... and I should have tested my toy example more carefully before posting it. The bottom line is that by following your instructions I got my example to work just as expected. Although I think I understand what it going on, I would be interested in your feedback.

Tell me if I got this right:

I understand that

object PythonInstanceOfA = py_get_existing();

[ or alternatively, object PythonInstanceOfA( py_get_existing() ); ]

creates python object wrapper and initializes it with the the object returned by py_get_existing(). Since neither py_get_existing() or PythonInstanceOfA explicitly know anything about the "A" object exported type, I presume this means that instantiating

class_< A, A*, boost:noncopyable> ^^^^ this should be completely unneccessary

registers a type converter from A* to Python and it is through this converter that the object becomes an instance of the Python type A.

Not quite. It registers type converters from A to Python, but to handle A* (because the pointer might be owned or just a reference as in your case) you need a return value policy. So the conversion is handled by the make_function call that created py_get_existing. However, you need class_<A...> in order to create the Python class object corresponding to A.

Does this mean that only one Python type can/should be declared as holder of a A* ? In other words, is it possible to declare two Python types, say A1 and A2, both holding a A* but with different exported interfaces ?

Maybe. It would probably at least warn you at runtime, though, and it's probably a bad idea.

Finally, the py_get_existing() gymnastics was required because

object PythonInstanceOfA( a ) ; // a is a raw ptr to A

does not work.

Right, it tries to _copy_ a into the A object held by PythonInstanceOfA.

But, apparently

object PythonInstanceOfA( aw ) ; // aw is an instance of a wrapper class which defines get_pointer() does work.

Can you briefly explain what is happening ?

Boost.Python recognizes that it's a wrapper class that contains a PyObject* referring to a Python object, and just extracts that pointer. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Francois Ostiguy <ostiguy@fnal.gov> writes:

Dave, Mike (and others):

Thank you for the explanations. This is truly very helpful.

More questions if I may:

...

...

class_< A, A*, boost:noncopyable> ^^^^ this should be completely unneccessary

If I omit A*, how does class A infer that I am trying to wrap a pointer ?

It doesn't. I explained where the pointer-to-Python conversion comes from in my previous message.

...

Boost.Python recognizes that it's a wrapper class that contains a PyObject* referring to a Python object, and just extracts that pointer.

I am confused by the above statement. Which wrapper class are you talking about ?

Consider the class (presented in another post):

class Wrapper<T> {

public:

typedef T element_type;

Wrapper(T * a): _a(a) { } T* get() const { return _a ;} T* operator->() const { return _a ;}

... etc ..

private:

T* _a;

};

this class does not contain a PyObject* ...

That's not a wrapper class; it's a smart pointer. A wrapper class is something like BaseWrap shown at http://www.boost.org/libs/python/doc/tutorial/doc/class_virtual_functions.ht...

Yet it is OK to do

class_< A, Wrapper<A>, boost:noncopyable>("A", no_init); ...

A* a = new A();

object PythonTypeA ( Wrapper<A>(a) );

In that case the smart-pointer-to-Python conversion is registered by the class_<A, Wrapper<A> ...> construction.

Finally, when an object holds a pointer to an existing instance, destroying the python wrapper object also deletes the held pointer.

Not if py_get_existing is created with return_value_policy<reference_existing_object>. In that case it's just held by a plain dumb pointer. If you want the Python wrapper object to manage the lifetime of the C++ object, you need return_value_policy<manage_new_object>.

If the c++ object pointed to by the held pointer is deleted first, when the python object reference count goes to 0, the held c++ is invalid. Any suggestion on how to deal with this issue ?

Hold the c++ object by shared_ptr, or if you need strict single-owner semantics, by auto_ptr. The latter is the result of using return_value_policy<manage_new_object> anyway. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

That's not a wrapper class; it's a smart pointer.

Dave = I apologize for the the poor wording. One might argue that the particular smart pointer in my example is a "wrapper" around a regular pointer. I agree that "wrapper class", especially in the context of the boost.python library, has a more precise meaning. Let me try to ask my questions differently. Q1 - Your explanation of how the conversion is handled is "So the conversion is handled by the make_function call that created py_get_existing. However, you need class_<A...> in order to create the Python class object corresponding to A." When make_function() converts a c++ A* into a python object "A", "conversion" implies (1) knowing what the python type A interface is, which is why class_<A,..> is needed. (2) deciding what will be held in that object. In my case, the return value policy implies that I want the object to hold a dumb pointer. What determines the valid held types ? Do I understand correctly that A*, A&, auto_ptr<A> are automatically valid held types and therefore explicitly specifying A* is uncessary ? In other words, the only held types that need to be explicitly specified are those that are not already supported ? Q2 - When I do ... class<A, boost:noncopyable>("A", no_init); class<B,SmartPointer<B>, boost:noncopyable>("B", no_init); A* a; B* b; ... object PythonInstanceofA ( a ); // error [1] object PythonInstanceofB ( SmartPointer<B>(b) ); // OK [2] according to your previous explanations: [1] "fails because there is an attempt to copy "a" directly into the A object held by PythonInstanceofA" object is generic object. By what mechanism would it know prior to construction that it should be holding an instance of A ? I would think that PythonInstanceofA can only acquire its identity as a type "A" by infering it from the argument in the constructor. So I do not understand the statement "copy "a" directly into the A object held by PythonInstanceofA". [2] "succeeds because Boost.Python recognizes that it's a wrapper class that contains a PyObject* referring to a Python object, and just extracts that pointer" Which "wrapper" class are you refering to ? SmartPointer<T> ? Does class<B, SmartPointer<B>, boost:noncopyable> register a converter from SmartPointer<B> to Python ? Thank you in advance. Your patience is greatly appreciated ;-) -Francois ---------------------------------------------------------------------------- Dr. Jean-Francois OSTIGUY voice: (630) 840-2231 Beam Physics Dept MS220 FAX: (630) 840-6039 Fermi National Accelerator Laboratory email: ostiguy@fnal.gov Batavia IL 60510-0500 WWW:www-ap.fnal.gov/~ostiguy

Does the following sample code let you do what you want? Essentially you can inject any C++ instance into Python namespace, as long as you have defined the 'class_<>' for the C++ class. Note particular the use of ObjectWrap. //---START--- // An ObjectWrap to wrap any object which is to be manipulated by Python. template<class T> class ObjectWrap { public: typedef T element_type; ObjectWrap(T& t) : p_(&t) { } ObjectWrap(T* p) : p_(p) { } virtual ~ObjectWrap() { } T* get() const { return p_; } T& operator*() const { return *p_; } T* operator->() const { return p_; } private: T * p_; }; namespace boost { // Some internal boost functions require get_pointer to be defined. // Define this before including <boost/python.hpp> template<class T> inline T * get_pointer(ObjectWrap<T> const & p) { return p.get(); } } #include <iostream> #include <boost/python.hpp> using namespace boost::python; /// A Singleton class which makes embedding Python easy. class Python { public: Python() { Py_Initialize(); mainModule = object(handle<>(borrowed(PyImport_AddModule("__main__") ))); mainNamespace = mainModule.attr("__dict__"); } ~Python() { //Py_Finalize(); // XXX } // Inject a C++ instance into the current Python module. template <typename T> void addObject(const char* name, T& obj) { ObjectWrap<T> o(&obj); currentNamespace[name] = object(o); } // Run Python script. void run(const char* script) const { try { object result(handle<>(PyRun_String(script, Py_file_input, mainNamespace.ptr(), mainNamespace.ptr()))); } catch(error_already_set) { PyErr_Print(); } } // Get current module. object getModule() const { return currentModule; } // Set current module. void setModule(const char* module = 0) { if (module == 0) // __main__ module { currentModule = mainModule; currentNamespace = mainNamespace; } else { currentModule = object(handle<>(borrowed(PyImport_AddModule(const_cast<char*>(module))))); currentNamespace = currentModule.attr("__dict__"); } } static Python& Instance() { static Python p; return p; } private: object mainModule; object mainNamespace; object currentModule; object currentNamespace; }; class A { public: A(int v) : value(v) { } int getValue() const { return value; } void setValue(int v) { value = v; } void add(int v) { value += v; } private: int value; }; void addObject(const char* name, A& a) { static bool isRegistered = false; if (!isRegistered) // Only register the class once. { // Make sure we define the class in the current module. scope s(Python::Instance().getModule()); class_<A, ObjectWrap<A>, boost::noncopyable>("A", no_init) .add_property("value", &A::getValue, &A::setValue) .def("add", &A::add) ; isRegistered = true; } Python::Instance().addObject(name, a); } int main() { Python::Instance().setModule("mymodule"); A* a = new A(10); A* b = new A(20); addObject("a1", *a); addObject("a2", *a); addObject("b", *b); std::cout << "In C++:" << std::endl; std::cout << "a: " << a->getValue() << std::endl; std::cout << "b: " << b->getValue() << std::endl << std::endl; Python::Instance().run( "import mymodule\n" "print 'Now we are in Python:'\n" "print 'dir():'\n" "print dir()\n" "print 'dir(mymodule):'\n" "print dir(mymodule)\n" "print\n" "print 'mymodule.a1, mymodule.a2 are " "references to the same \\'a\\' instance.'\n" "print 'a1:', mymodule.a1.value\n" "print 'a2:', mymodule.a2.value\n" "print\n" "mymodule.a1.value -= 2\n" "print 'After mymodule.a1.value -= 2:'\n" "print 'a1:', mymodule.a1.value\n" "print 'a2:', mymodule.a2.value\n" "print\n" "print 'b is different from a1, a2, of course.'\n" "mymodule.b.add(-19)\n" "print 'After mymodule.b.add(-19):'\n" "print 'b:', mymodule.b.value\n" "print\n" "mymodule.a2.add(3)\n" "print 'After mymodule.a2.add(3):'\n" "print 'a1:', mymodule.a1.value\n" "print 'a2:', mymodule.a2.value\n" "print 'a1 == a2?', mymodule.a1 == mymodule.a2\n" "print\n" ); std::cout << "Now we are back to C++ again:" << std::endl; std::cout << "a: " << a->getValue() << std::endl; std::cout << "b: " << b->getValue() << std::endl; delete a; return 0; } //---END--- --- Francois Ostiguy <ostiguy@fnal.gov> wrote:

Dave and Max -

Many thanks. With your suggestions, I am getting much closer ... but it is not quite working yet.

Here is what I tried (in slightly simplified form):

...

class A;

class_<A, A*, boost::noncopyable> PythonTypeA;

A* aptr = new A();

A* get_existing() { return aptr; }

object py_get_existing = make_function( &get_existing, return_value_policy<reference_existing_object>() );

object PythonInstanceOfA = PythonTypeA( py_get_existing() );

...

The code compiles cleanly. Unfortunately, I get a core dump when the last line is executed. The debugger tells me that the problem originates from line 154 in return_from_python.hpp

151 inline T return_object_manager_from_python<T>::operator()(PyObject* obj) const 152 { 153 return T( 154 object_manager_traits<T>::adopt(expect_non_null(obj)) 155 ); 156 } 157 } 158 159 }}} // namespace boost::python::converter 160

obj is null and expect_non_null(obj) throws an exception. This function is called from line 20 in object_call.cpp

13 14 template <BOOST_PP_ENUM_PARAMS_Z(1, N, class A)> 15 typename detail::dependent<object, A0>::type 16 operator()(BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, A, const& a)) const 17 { 18 typedef typename detail::dependent<object, A0>::type obj; 19 U const& self = *static_cast<U const*>(this); 20 return call<obj>(get_managed_object(self, tag), BOOST_PP_ENUM_PARAMS_Z(1, N, a)); 21 } 22

so somehow, calling the object returned by py_get_existing appears to return a null pointer.

-Francois

---------------------------------------------------------------------------- Dr. Jean-Francois OSTIGUY voice: (630) 840-2231 Beam Physics Dept MS220 FAX: (630) 840-6039 Fermi National Accelerator Laboratory email: ostiguy@fnal.gov Batavia IL 60510-0500 WWW:www-ap.fnal.gov/~ostiguy

_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig

__________________________________ Do you Yahoo!? Yahoo! Mail Address AutoComplete - You start. We finish. http://promotions.yahoo.com/new_mail

Ah... it never comes to my mind that I can define it elsewhere. Perhaps I was illuded by the fact that several get_pointer functions were all defined in namespace boost. Anyway VC7 still chokes if I put it in the global namespace. The good news is that I have upgraded to VC++ Toolkit 2003 recently... --- David Abrahams <dave@boost-consulting.com> wrote:

dique <chezdique@yahoo.com> writes:

...

namespace boost { // Some internal boost functions require get_pointer to be defined.

But not in namespace boost. Never, ever make a definition in a library's namespace unless the documentation tells you to. Put this in the same namespace as ObjectWrap, or if your compiler is broken like vc6/7, put it in the global namespace.

...

// Define this before including <boost/python.hpp> template<class T> inline T * get_pointer(ObjectWrap<T> const & p) { return p.get(); } }

-- Dave Abrahams Boost Consulting http://www.boost-consulting.com

_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig

__________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail