From nick at ilm.com  Tue Nov  8 03:19:56 2011
From: nick at ilm.com (Nick Rasmussen)
Date: Mon, 7 Nov 2011 18:19:56 -0800
Subject: [C++-sig] Inheriting from a python base class/exception translation
Message-ID: <20111108021956.GI31565@maxima.lucasfilm.com>

I'm cleaning up some of our python bindings for public release as
part of the openexr distribution, and wanted to update the python
bindings of our exception library, Iex, to use boost::python.

The first thing I did was the trivial binding of the class hierarchy
out to python, and performed register_exception_translator<> for each
type with a translator like:

void
translateExc(const BaseExc &exc)
{
    PyErr_SetObject(pytype,boost::python::object(exc).ptr());
}

where pytype is the pointer to the type object for BaseExc's
python binding.  This allows instances of the exception types
to be raised in c++, translated into python, and caught.

However, the problem I'm having is this: to allow instances of
the exception types to be raised in python (or in fact to have
a try/except/else block finalize properly), the python objects
need to be derived from the python Exception type or one of
its subclasses.

I saw this thread from several years ago:

http://mail.python.org/pipermail/cplusplus-sig/2006-April/010320.html

but, at least with python 2.6, objects raised from within
a script must derive from Exception or else a TypeError will
be raised:

testing raising existing BaseExc exception object:
Traceback (most recent call last):
  File "testexc.py", line 24, in <module>
    raise e1
TypeError: exceptions must be classes or instances, not BaseExc

So my question is this: how do I bind a c++ type out to
boost::python such that the python bindings for the type
inherits from a type defined in python?

What I've tried so far was to reflect RuntimeError out to c++
in a manner similar to how the other python builtin object
types (e.g. list/dict) are handled in boost::python.  The
reflection was successful, and I was getting correct translation
of RuntimeErrors across the python/C boundary (both for raising
and passing instances of RuntimeError objects).  I didn't
particularly expect this to work because of the object layouts, but
at least by using the c++ RuntimeError type as the parameter to
bases<>, isinstance identified the instances as being derived from
RuntimeError, and the instances were raiseable.  However, whenever
the objects were deallocated, the deallocation would crash:

#0  0x0000000000479e9f in _PyWeakref_GetWeakrefCount (head=0x30) at Objects/weakrefobject.c:16
#1  0x000000000047ca50 in PyObject_ClearWeakRefs (object=0x892960) at Objects/weakrefobject.c:898
#2  0x000000000046a2bd in subtype_dealloc (self=0x892960) at Objects/typeobject.c:968

Looking in valgrind, there were unsurprisingly plenty of other
errors before the crash as well. :)

One other approach that seemed like it may work would be to define the
entire exception hierarchy in python, mirroring the c++ hierarchy,
and reflect each class individually to c++.  That doesn't seem
particularly boost-pythonic, so I'm hoping that there's a better way
such that I can just add a small fix-up in the base of the bindings
for the exception hierarchy (perhaps a custom PyTypeObject?).

Any ideas?  I've included a condensed example and testcase below:

-nick

exctestmodule.cpp:

#include <Python.h>
#include <boost/python.hpp>
#include <boost/format.hpp>

using namespace boost::python;

namespace ExcTest {

template <class Exc>
struct ExcTranslator
{
    static PyObject *pytype;
    static const char *module;
    static const char *name;

    static void translateExc(const Exc &exc)
    {
        PyErr_SetObject(pytype,object(exc).ptr());
    }

    static std::string
    repr(const Exc &exc)
    {
        return (boost::format("%s.%s('%s')")%module%name%exc.what()).str();
    }
};

template <class Exc, class Base>
class_<Exc,bases<Base> >
registerExc()
{
    class_<Exc,bases<Base> > exc_class(ExcTranslator<Exc>::name,init<const std::string &>());
    exc_class
        .def("__repr__",&ExcTranslator<Exc>::repr)
        ;
    ExcTranslator<Exc>::pytype = exc_class.ptr();
    register_exception_translator <Exc>(&ExcTranslator<Exc>::translateExc);
    return exc_class;
}

struct BaseExc : public std::exception
{
    explicit BaseExc(const std::string &message) : _message(message) {}
    virtual ~BaseExc() throw() {}
    virtual const char *what() const throw() { return _message.c_str(); }
    std::string _message;
};

struct ArgExc : public BaseExc
{
    explicit ArgExc(const std::string &message) : BaseExc(message) {}
    virtual ~ArgExc() throw() {}
};

void
testException (int idx)
{
    if (idx == 1)
        throw ArgExc("ArgExc from c++");
    throw BaseExc("BaseExc from c++");
}

#define PY_DEFINE_EXC(ExcType,ModuleName,ExcName)                     \
template <> PyObject *ExcTranslator<ExcType>::pytype = 0;             \
template <> const char *ExcTranslator<ExcType>::module = #ModuleName; \
template <> const char *ExcTranslator<ExcType>::name = #ExcName;

PY_DEFINE_EXC(BaseExc,exctest,BaseExc)
PY_DEFINE_EXC(ArgExc,exctest,ArgExc)


} // namespace ExcTest

using namespace ExcTest;

BOOST_PYTHON_MODULE(exctest)
{
    def("testException", &testException);

    class_<BaseExc> myExc_class("BaseExc",init<const std::string &>());
    myExc_class
        .def("__str__",&BaseExc::what)
        .def("__repr__",&ExcTranslator<BaseExc>::repr)
        ;
    ExcTranslator<BaseExc>::pytype = myExc_class.ptr();
    register_exception_translator <BaseExc>(&ExcTranslator<BaseExc>::translateExc);

    registerExc<ArgExc,BaseExc>();
}

testexc.py:

#!/usr/bin/env python2.6
import exctest
import traceback

print 'testing BaseExc exception creation:'
e1 = exctest.BaseExc('BaseExc from python')
assert str(e1) == 'BaseExc from python'
assert repr(e1) == "exctest.BaseExc('BaseExc from python')"
#assert isinstance(e1,RuntimeError)
#del e1
print '  pass: %s' % (repr(e1))

print 'testing ArgExc exception creation:'
e2 = exctest.ArgExc('ArgExc from python')
assert str(e2) == 'ArgExc from python'
assert repr(e2) == "exctest.ArgExc('ArgExc from python')"
#assert isinstance(e2,RuntimeError)
assert isinstance(e2,exctest.BaseExc)
#del e2
print '  pass: %s' % (repr(e2))

print 'testing raising existing BaseExc exception object:'
try:
   raise e1
except exctest.ArgExc, e:
   traceback.print_exc()
   assert False
except exctest.BaseExc, e:
   print '  pass: %s' % (repr(e))
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing raising existing ArgExc exception object:'
try:
   raise e2
except exctest.ArgExc, e:
   print '  pass: %s' % (repr(e))
except exctest.BaseExc, e:
   traceback.print_exc()
   assert False
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing BaseExc exception translation:'
try:
   exctest.testException(0)
except exctest.ArgExc, e:
   traceback.print_exc()
   assert False
except exctest.BaseExc, e:
   print '  pass: %s' % (repr(e))
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing ArgExc exception translation:'
try:
   exctest.testException(1)
except exctest.ArgExc, e:
   print '  pass: %s' % (repr(e))
except exctest.BaseExc, e:
   traceback.print_exc()
   assert False
except:
   traceback.print_exc()
   assert False
else:
   assert False


print 'testing BaseExc raise:'
try:
   raise exctest.BaseExc('new BaseExc from python')
except exctest.ArgExc, e:
   traceback.print_exc()
   assert False
except exctest.BaseExc, e:
   print '  pass: %s' % (repr(e))
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing ArgExc raise:'
try:
   raise exctest.ArgExc('new ArgExc from python')
except exctest.ArgExc, e:
   print '  pass: %s' % (repr(e))
except exctest.BaseExc, e:
   traceback.print_exc()
   assert False
except:
   traceback.print_exc()
   assert False
else:
   assert False

print "done"


From super24bitsound at hotmail.com  Tue Nov  8 07:12:23 2011
From: super24bitsound at hotmail.com (Jay Riley)
Date: Tue, 8 Nov 2011 01:12:23 -0500
Subject: [C++-sig] Passing ownership to C++ through pointer
In-Reply-To: <20111108021956.GI31565@maxima.lucasfilm.com>
References: <20111108021956.GI31565@maxima.lucasfilm.com>
Message-ID: <BAY158-W65AAD064686755D5C4A6CBB4DE0@phx.gbl>


I'm trying to pass an object from python to C++. I want C++ to take control of the object.
The object I'm trying to change ownership on is defined in it's module as follows (superfluous code removed):

BOOST_PYTHON_MODULE(ComponentModule)		{
			class_<Components::Component, std::auto_ptr<Components::Component> >("Component")				.def(init<>())				.def(init<bool, boost::python::optional<const int> >())				.def(init<const std::string&, int, boost::python::optional<bool, const int>  >())				.def(init<const std::string&, const std::string&, boost::python::optional<bool, const int>  >())				;
			class_<Components::UseComponent, std::auto_ptr<UseComponentWrap>, bases<Components::Component> >("UseComponent")				.def(init<>())				.def(init<const std::string&,bool, bool>())				.def(init<const Components::UseComponent&>())				;		}	}
And The object I want to take control is:
BOOST_PYTHON_MODULE(ItemModule)		{						class_<Items::Item, bases<Entity> >("Item")				.def(init<Items::Item&>())				.def(init<const std::string&>())				.def("RegisterComponent",&Items::Item::RegisterComponent )				;		}
Register Component is the function that takes control of the component. It's full definition is:
void Item::RegisterComponent(const std::string& indexName, Components::Component* component)		{			auto it = ComponentCollection.find(indexName);			if (it != ComponentCollection.end())                        {				delete it->second;                                ComponentCollection.erase(it);			ComponentCollection[indexName] = component;		}
The item posses the component until destroyed/overwritten.
A sample piece of python code would be:
healer = UseComponent("HealUseModule", True, True)print "Adding Healing Component"CurrentItem.RegisterComponent("Healing Component", healer)
where CurrentItem is an item. The problem here is that when the component goes out of scope in the python file, it get's deleted adn invalidated since C++ hasn't taken ownership of it. I tried following the advice here
http://www.boost.org/doc/libs/1_36_0/libs/python/doc/v2/faq.html#ownership
which is why I wrapped Component/UseComponent in an auto_ptr. Additionally fro, this advice, I changed to RegisterComponent wrap to
.def("RegisterComponent", &RegisterComp)
where RegisterComp is 
void RegisterComp(Items::Item& item, const std::string& compName, std::auto_ptr<Components::UseComponent> comp)//, void* comp)		{			item.RegisterComponent(compName, comp.get());			comp.release();		}
this fails, saying the Python argument types did not match the C++ signature. I'm certain it's the std::auto_ptr<Components::UseComponent> comp argument that is invalid because the call succeeds when that argument is removed but item and name are left.
Am I doing something wrong? base on that snippet it should work. Is there another way to do this? I know I should probably switch to smart_ptrs and I'm considering it, but Id like to be able to know how to do this, even if its just for future reference. Any help would be appreciated.
Thanks 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111108/be6b5ad9/attachment.html>

From super24bitsound at hotmail.com  Tue Nov  8 07:17:54 2011
From: super24bitsound at hotmail.com (Jay Riley)
Date: Tue, 8 Nov 2011 01:17:54 -0500
Subject: [C++-sig] Passing ownership to C++ through pointer
In-Reply-To: <BAY158-W65AAD064686755D5C4A6CBB4DE0@phx.gbl>
References: <20111108021956.GI31565@maxima.lucasfilm.com>
	<BAY158-W65AAD064686755D5C4A6CBB4DE0@phx.gbl>
Message-ID: <snt0-eas32046D0E630BBD7EE78C827B4DE0@phx.gbl>

Sorry just as a quick append, UseComponentWrap inherits from UseComponent to allow overriding of a few virtual methods

On 2011-11-08, at 1:13 AM, "Jay Riley" <super24bitsound at hotmail.com> wrote:

> I'm trying to pass an object from python to C++. I want C++ to take control of the object.
> 
> The object I'm trying to change ownership on is defined in it's module as follows (superfluous code removed):
> 
> 
> BOOST_PYTHON_MODULE(ComponentModule)
> 		{
> 
> 			class_<Components::Component, std::auto_ptr<Components::Component> >("Component")
> 				.def(init<>())
> 				.def(init<bool, boost::python::optional<const int> >())
> 				.def(init<const std::string&, int, boost::python::optional<bool, const int>  >())
> 				.def(init<const std::string&, const std::string&, boost::python::optional<bool, const int>  >())
> 				;
> 
> 			class_<Components::UseComponent, std::auto_ptr<UseComponentWrap>, bases<Components::Component> >("UseComponent")
> 				.def(init<>())
> 				.def(init<const std::string&,bool, bool>())
> 				.def(init<const Components::UseComponent&>())
> 				;
> 		}
> 	}
> 
> And The object I want to take control is:
> 
> BOOST_PYTHON_MODULE(ItemModule)
> 		{
> 			
> 			class_<Items::Item, bases<Entity> >("Item")
> 				.def(init<Items::Item&>())
> 				.def(init<const std::string&>())
> 				.def("RegisterComponent",&Items::Item::RegisterComponent )
> 				;
> 		}
> 
> Register Component is the function that takes control of the component. It's full definition is:
> 
> void Item::RegisterComponent(const std::string& indexName, Components::Component* component)
> 		{
> 			auto it = ComponentCollection.find(indexName);
> 			if (it != ComponentCollection.end())
>                         {
> 				delete it->second;
>                                 ComponentCollection.erase(it);
> 			ComponentCollection[indexName] = component;
> 		}
> 
> The item posses the component until destroyed/overwritten.
> 
> A sample piece of python code would be:
> 
> healer = UseComponent("HealUseModule", True, True)
> print "Adding Healing Component"
> CurrentItem.RegisterComponent("Healing Component", healer)
> 
> where CurrentItem is an item. The problem here is that when the component goes out of scope in the python file, it get's deleted adn invalidated since C++ hasn't taken ownership of it. I tried following the advice here
> 
> http://www.boost.org/doc/libs/1_36_0/libs/python/doc/v2/faq.html#ownership
> 
> which is why I wrapped Component/UseComponent in an auto_ptr. Additionally fro, this advice, I changed to RegisterComponent wrap to
> 
> .def("RegisterComponent", &RegisterComp)
> 
> where RegisterComp is 
> 
> void RegisterComp(Items::Item& item, const std::string& compName, std::auto_ptr<Components::UseComponent> comp)//, void* comp)
> 		{
> 			item.RegisterComponent(compName, comp.get());
> 			comp.release();
> 		}
> 
> this fails, saying the Python argument types did not match the C++ signature. I'm certain it's the std::auto_ptr<Components::UseComponent> comp argument that is invalid because the call succeeds when that argument is removed but item and name are left.
> 
> Am I doing something wrong? base on that snippet it should work. Is there another way to do this? I know I should probably switch to smart_ptrs and I'm considering it, but Id like to be able to know how to do this, even if its just for future reference. Any help would be appreciated.
> 
> Thanks
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111108/1395dbdf/attachment-0001.html>

From talljimbo at gmail.com  Tue Nov  8 16:55:00 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 08 Nov 2011 10:55:00 -0500
Subject: [C++-sig] Inheriting from a python base class/exception
	translation
In-Reply-To: <20111108021956.GI31565@maxima.lucasfilm.com>
References: <20111108021956.GI31565@maxima.lucasfilm.com>
Message-ID: <4EB950D4.7060606@gmail.com>

On 11/07/2011 09:19 PM, Nick Rasmussen wrote:
> I'm cleaning up some of our python bindings for public release as
> part of the openexr distribution, and wanted to update the python
> bindings of our exception library, Iex, to use boost::python.
>
> The first thing I did was the trivial binding of the class hierarchy
> out to python, and performed register_exception_translator<>  for each
> type with a translator like:
>
> void
> translateExc(const BaseExc&exc)
> {
>      PyErr_SetObject(pytype,boost::python::object(exc).ptr());
> }
>
> where pytype is the pointer to the type object for BaseExc's
> python binding.  This allows instances of the exception types
> to be raised in c++, translated into python, and caught.
>
> However, the problem I'm having is this: to allow instances of
> the exception types to be raised in python (or in fact to have
> a try/except/else block finalize properly), the python objects
> need to be derived from the python Exception type or one of
> its subclasses.
>

This is a tough problem, and one I think Boost.Python wasn't designed to 
do, since most of it was in place before one could even derive from 
Python builtins using the C-API.  It's definitely something I'd like to 
fix in the future.

In the meantime, I've essentially followed the "parallel pure-Python 
exception hierarchy" method, but you can do some metaprogramming tricks 
to keep from having to actually build the Python hierarchy yourself.  In 
other words, write some Python code that inspects your 
Boost.Python-wrapped C++ exception hierarchy and calls type(...) to make 
corresponding Python types that derive from Exception.  The pure-Python 
exceptions can then be set as class members of the Boost.Python-wrapped 
C++ exceptions, so your translate functions can do something like this:

void translateExc(const BaseExc & exc) {
     bp::object wrappedExc(exc);
     bp::object pyExcType = wrappedExc.attr("_pyExcType")
     // assume the constructor for the Python exception accepts a
     // C++ exception; useful if the C++ exception has data
     bp::object pyExc = pyExcType(wrappedExc);
     PyErr_SetObject(pyExcType.ptr(), pyExc.ptr());
}

To finish it off, you could add a from-Python converter that converts 
the pure-Python exception back to the appropriate C++ exception when 
it's passed to Boost.Python-wrapped functions, but that isn't always 
necessary.

I don't have any code to do the metaprogramming bits on hand, but I'm 
happy to help further if you have questions.

By the way, I do hope someone else has an even better idea - my approach 
is more complicated than it ought to be, but at least it's all overhead 
rather than something that scales with the number of exceptions you have.

Good luck!

Jim

From talljimbo at gmail.com  Tue Nov  8 17:02:23 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 08 Nov 2011 11:02:23 -0500
Subject: [C++-sig] Passing ownership to C++ through pointer
In-Reply-To: <BAY158-W65AAD064686755D5C4A6CBB4DE0@phx.gbl>
References: <20111108021956.GI31565@maxima.lucasfilm.com>
	<BAY158-W65AAD064686755D5C4A6CBB4DE0@phx.gbl>
Message-ID: <4EB9528F.7000200@gmail.com>

On 11/08/2011 01:12 AM, Jay Riley wrote:
> I'm trying to pass an object from python to C++. I want C++ to take
> control of the object.

<snip>

> Am I doing something wrong? base on that snippet it should work. Is
> there another way to do this? I know I should probably switch to
> smart_ptrs and I'm considering it, but Id like to be able to know how to
> do this, even if its just for future reference. Any help would be
> appreciated.
>

I suspect the inheritance is getting in the way, and Boost.Python isn't 
being as smart as it could be about converting auto_ptr<T> to 
auto_ptr<U> when T* is convertible to U*.

You might want to try having your register function wrapper accept 
auto_ptr<UseComponentWrap> (and if necessary, providing an overload that 
takes auto_ptr<UseComponent>).

Before that, however, you might want to just try using "auto_ptr &" 
rather than simple "auto_ptr".  It'd be unfortunate if that was 
necessary, but it's a possibility.

Good luck!

Jim

From nick at ilm.com  Wed Nov  9 00:11:54 2011
From: nick at ilm.com (Nick Rasmussen)
Date: Tue, 8 Nov 2011 15:11:54 -0800
Subject: [C++-sig] Inheriting from a python base class/exception
 translation
In-Reply-To: <4EB950D4.7060606@gmail.com>
References: <20111108021956.GI31565@maxima.lucasfilm.com>
	<4EB950D4.7060606@gmail.com>
Message-ID: <20111108231154.GO31565@maxima.lucasfilm.com>

On Tue, 08 Nov 2011, Jim Bosch wrote:

> In the meantime, I've essentially followed the "parallel pure-Python
> exception hierarchy" method, but you can do some metaprogramming
> tricks to keep from having to actually build the Python hierarchy
> yourself.  In other words, write some Python code that inspects your
> Boost.Python-wrapped C++ exception hierarchy and calls type(...) to
> make corresponding Python types that derive from Exception.  The
> pure-Python exceptions can then be set as class members of the
> Boost.Python-wrapped C++ exceptions, so your translate functions can
> do something like this:
> 
> void translateExc(const BaseExc & exc) {
>     bp::object wrappedExc(exc);
>     bp::object pyExcType = wrappedExc.attr("_pyExcType")
>     // assume the constructor for the Python exception accepts a
>     // C++ exception; useful if the C++ exception has data
>     bp::object pyExc = pyExcType(wrappedExc);
>     PyErr_SetObject(pyExcType.ptr(), pyExc.ptr());
> }
> 
> To finish it off, you could add a from-Python converter that
> converts the pure-Python exception back to the appropriate C++
> exception when it's passed to Boost.Python-wrapped functions, but
> that isn't always necessary.
> 

Thanks, I ended up implementing something along those lines (which
was actually pretty close to what the original Python/C bindings
were doing).  I've got the basic funcionality working (exception
translation, from python conversion, to python conversion), but it
still seems a bit fragile, and only supports exceptions with a
single string.

I've attached the latest iteration of the module and testcase below.
If anyone can suggest any improvements, or a way to get it working
with standard boost wrappings of the c++ types, I'd appreciate it.

-nick

exctestmodule.cpp:

#include <Python.h>
#include <boost/python.hpp>
#include <boost/format.hpp>

using namespace boost::python;

namespace ExcTest {

template <class Exc>
struct ExcTranslator
{
    static PyObject *pytype;
    static const char *module;
    static const char *name;

    // to python
    static PyObject *convert(const Exc &exc)
    {
        return incref(object(handle<>(borrowed(pytype)))(exc.what()).ptr());
    }

    static PyTypeObject *get_pytype()
    {
        return (PyTypeObject *)pytype;
    }

    // from python
    static void *convertible(PyObject *exc)
    {
        if (!PyType_IsSubtype(Py_TYPE(exc),(PyTypeObject *)pytype)) return 0;
        return exc;
    }

    static void construct(PyObject* raw_exc, converter::rvalue_from_python_stage1_data* data)
    {
        object exc(handle<>(borrowed(raw_exc)));
        std::string s = extract<std::string>(exc.attr("__str__")());
        void *storage = ((converter::rvalue_from_python_storage<Exc>*)data)->storage.bytes;
        new (storage) Exc(s);
        data->convertible = storage;
    }

    // translate exception
    static void translate(const Exc &exc)
    {
        PyErr_SetObject(pytype,convert(exc));
    }
};

template <class Exc, class Base>
void
registerExc()
{
    std::string classname = ExcTranslator<Exc>::name;
    std::string module = ExcTranslator<Exc>::module;
    std::string basename = ExcTranslator<Base>::name;
    std::string basemodule = ExcTranslator<Base>::module;

    dict tmpDict;
    tmpDict["__builtins__"] = handle<>(borrowed(PyEval_GetBuiltins()));

    std::string definition;
    if (basemodule != module)
    {
        definition += (boost::format("import %s\n") % basemodule).str();
        basename = (boost::format("%s.%s") % basemodule % basename).str();
    }
    else
    {
        // bind in the base class type into the tmp dict
        tmpDict[basename] = object(handle<>(borrowed(ExcTranslator<Base>::pytype)));
    }

    definition += (boost::format("class %s (%s):\n"
                                 "  def __init__ (self, v=''):\n"
                                 "    super(%s,self).__init__(v)\n"
                                 "  def __repr__ (self):\n"
                                 "    return \"%s.%s('%%s')\"%%(self.args[0])\n")
                   % classname % basename % classname % module % classname).str();

    handle<> tmp(PyRun_String(definition.c_str(),Py_file_input,tmpDict.ptr(),tmpDict.ptr()));
    object exc_class = tmpDict[classname];
    scope().attr(classname.c_str()) = exc_class;
    ExcTranslator<Exc>::pytype = exc_class.ptr();

    // to python
    to_python_converter<Exc,ExcTranslator<Exc>,true>();

    // from python
    converter::registry::push_back(&ExcTranslator<Exc>::convertible,
                                   &ExcTranslator<Exc>::construct,type_id<Exc>());

    // exception translation
    register_exception_translator <Exc>(&ExcTranslator<Exc>::translate);
}

// not registered, but define RuntimeError so that BaseExc can have the appropriate
// python base type
struct RuntimeError {};
template<> PyObject *ExcTranslator<RuntimeError>::pytype   = PyExc_RuntimeError;
template<> const char *ExcTranslator<RuntimeError>::name   = "RuntimeError";
template<> const char *ExcTranslator<RuntimeError>::module = "__builtin__";

struct BaseExc : public std::exception
{
    explicit BaseExc(const std::string &message) : _message(message) {}
    virtual ~BaseExc() throw() {}
    virtual const char *what() const throw() { return _message.c_str(); }
    std::string _message;
};

struct ArgExc : public BaseExc
{
    explicit ArgExc(const std::string &message) : BaseExc(message) {}
    virtual ~ArgExc() throw() {}
};

void
testException (int idx)
{
    if (idx == 1)
        throw ArgExc("ArgExc from c++");
    throw BaseExc("BaseExc from c++");
}

#define PY_DEFINE_EXC(ExcType,ModuleName,ExcName)                     \
template <> PyObject *ExcTranslator<ExcType>::pytype   = 0;           \
template <> const char *ExcTranslator<ExcType>::module = #ModuleName; \
template <> const char *ExcTranslator<ExcType>::name   = #ExcName;

PY_DEFINE_EXC(BaseExc,exctest,BaseExc)
PY_DEFINE_EXC(ArgExc,exctest,ArgExc)

std::string
baseExcString(const BaseExc &exc)
{
    return exc.what();
}

std::string
argExcString(const ArgExc &exc)
{
    return exc.what();
}

BaseExc
makeBaseExc(const std::string &s)
{
    return BaseExc(s);
}

ArgExc
makeArgExc(const std::string &s)
{
    return ArgExc(s);
}

} // namespace ExcTest

using namespace ExcTest;

BOOST_PYTHON_MODULE(exctest)
{
    def("testException", &testException);
    def("baseExcString", &baseExcString);
    def("argExcString", &argExcString);
    def("makeBaseExc", &makeBaseExc);
    def("makeArgExc", &makeArgExc);

    registerExc<BaseExc,RuntimeError>();
    registerExc<ArgExc,BaseExc>();
}

testexc.py:

#!/usr/bin/env python2.6
import exctest
import traceback

print 'testing BaseExc exception creation:'
e1 = exctest.BaseExc('BaseExc from python')
assert str(e1) == 'BaseExc from python'
assert repr(e1) == "exctest.BaseExc('BaseExc from python')"
#assert isinstance(e1,RuntimeError)
#del e1
print '  pass: %s' % (repr(e1))

print 'testing ArgExc exception creation:'
e2 = exctest.ArgExc('ArgExc from python')
assert str(e2) == 'ArgExc from python'
assert repr(e2) == "exctest.ArgExc('ArgExc from python')"
#assert isinstance(e2,RuntimeError)
assert isinstance(e2,exctest.BaseExc)
#del e2
print '  pass: %s' % (repr(e2))

print 'testing raising existing BaseExc exception object:'
try:
   raise e1
except exctest.ArgExc, e:
   traceback.print_exc()
   assert False
except exctest.BaseExc, e:
   print '  pass: %s' % (repr(e))
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing raising existing ArgExc exception object:'
try:
   raise e2
except exctest.ArgExc, e:
   print '  pass: %s' % (repr(e))
except exctest.BaseExc, e:
   traceback.print_exc()
   assert False
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing BaseExc exception translation:'
try:
   exctest.testException(0)
except exctest.ArgExc, e:
   traceback.print_exc()
   assert False
except exctest.BaseExc, e:
   print '  pass: %s' % (repr(e))
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing ArgExc exception translation:'
try:
   exctest.testException(1)
except exctest.ArgExc, e:
   print '  pass: %s' % (repr(e))
except exctest.BaseExc, e:
   traceback.print_exc()
   assert False
except:
   traceback.print_exc()
   assert False
else:
   assert False


print 'testing BaseExc raise:'
try:
   raise exctest.BaseExc('new BaseExc from python')
except exctest.ArgExc, e:
   traceback.print_exc()
   assert False
except exctest.BaseExc, e:
   print '  pass: %s' % (repr(e))
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing ArgExc raise:'
try:
   raise exctest.ArgExc('new ArgExc from python')
except exctest.ArgExc, e:
   print '  pass: %s' % (repr(e))
except exctest.BaseExc, e:
   traceback.print_exc()
   assert False
except:
   traceback.print_exc()
   assert False
else:
   assert False

print 'testing exception conversion:'
be = exctest.makeBaseExc('testStr')
assert (isinstance(be,exctest.BaseExc))
assert (be.__class__ == exctest.BaseExc)
ae = exctest.makeArgExc('testStr')
assert (isinstance(ae,exctest.BaseExc))
assert (isinstance(ae,exctest.ArgExc))
assert (ae.__class__ == exctest.ArgExc)
assert (exctest.baseExcString(be) == 'testStr')
assert (exctest.baseExcString(ae) == 'testStr')
assert (exctest.argExcString(ae) == 'testStr')

print "done"


From super24bitsound at hotmail.com  Wed Nov  9 02:11:14 2011
From: super24bitsound at hotmail.com (Jay Riley)
Date: Tue, 8 Nov 2011 20:11:14 -0500
Subject: [C++-sig] Passing ownership to C++ through pointer
In-Reply-To: <4EB9528F.7000200@gmail.com>
References: <20111108021956.GI31565@maxima.lucasfilm.com>,
	<BAY158-W65AAD064686755D5C4A6CBB4DE0@phx.gbl>,
	<4EB9528F.7000200@gmail.com>
Message-ID: <BAY158-W496F3F564D0606B7C86E5B4DF0@phx.gbl>


Okay I think something else is majorly wrong; my component objects are being seen at Item onjects. I'll try to keep the code I show to a minimum. 
In my python file, I have the following:
def ItemFactory(ItemNode):    ItemName = ItemNode.attrib["name"]    if (True):    #if ItemName == "Elixir":        CurrentItem = Item(ItemName)        print ItemName        componentNode = ItemNode.find("Components");        if (componentNode != None):            components = componentNode.getchildren()            for component in components:                if component.tag == "HealingComponent":                    healer = UseComponent("HealUseModule", True, True)                    print "Adding Healing Component"                    CurrentItem.RegisterComponent("Healing Component", healer)        ItemModule.ItemLibrary.AddItem(CurrentItem)
ItemNode is just an XML node that I parse for properties. Each Item node gets sent to this function. As per Jim's advice, RegisterComponent looks as follows:
void RegisterComp(Items::Item& item, const std::string& compName, std::auto_ptr<UseComponentWrap> comp)//, void* comp){	item.RegisterComponent(compName, comp.get());	comp.release();}
Now, the components are registering. The problem is, when I try do the following:
try	{		boost::shared_ptr<Items::Item> item = Game.GetItemLibrary().GetItem("Elixir");		Components::Component* heal = item->GetComponent("Healing Component");				Components::UseComponent* healertrue = dynamic_cast<Components::UseComponent*>(heal);		if (healertrue != nullptr)			healertrue->Use(std::vector<Character::BaseCharacter*>(), std::vector<Character::BaseCharacter*>());	}	catch (boost::python::error_already_set)	{		PyErr_Print();	}
it prints out this error:
AttributeError: 'Item' object has no attribute 'Use'
this is really confusing. How could my UseComponent have become an Item?
I was testing it, and set the AddItem function as:
static std::vector<Components::Component*> test;void ItemDatabase::AddItem(boost::shared_ptr<Item> item)	{		if (item.get() != nullptr)		{			IDToItemMap[item->GetID()] = item;			NameToItemMap[item->GetName()] = item;			auto healComp = item->GetComponent("Healing Component");			if (healComp != nullptr)			{				test.push_back(healComp);				int x = 10;			}		}	}
Here's the weird thing. While I'm adding items, I used x = 10 as a breakpoint to watch healComp's be added to test. The first healComp get's inserted correctly, and the PyObject's tp_name says UseComponent. When the second item with a healComp is being added, I used the vector to check the first healComp inserted and it somehow became an item (tp_name says Item). This is really weird. I thought it might be a scoping problem in my python code, but I can't see anything wrong with it. I'm not sure if it's relevant, but I'm including my UseComponent wrapping
//Class definitionclass UseComponentWrap : public Components::UseComponent		{		public:			UseComponentWrap(PyObject* self_);			UseComponentWrap(PyObject* self_, const std::string& name, bool hasUse, bool hasBattleUse);			UseComponentWrap(PyObject* self_, const Components::UseComponent& cmp);			bool Use(std::vector<Character::BaseCharacter*>& Users, std::vector<Character::BaseCharacter*>& Targets) override;			bool UseDefault(std::vector<Character::BaseCharacter*>& Users, std::vector<Character::BaseCharacter*>& Targets);			bool BattleUse(std::vector<Battle::BattleCharacter*>& Users, std::vector<Battle::BattleCharacter*>& Targets, Battle::BattleField* field) override;			bool BattleUseDefault(std::vector<Battle::BattleCharacter*>& Users, std::vector<Battle::BattleCharacter*>& Targets, Battle::BattleField* field);			Components::UseComponent::ClonePtr Clone() const override;			Components::UseComponent::ClonePtr CloneDefault() const;			PyObject* self;

		private:
		};
//Python Wrapping
class_<Components::UseComponent, std::auto_ptr<UseComponentWrap>, bases<Components::Component> >("UseComponent")				.def(init<>())				.def(init<const std::string&,bool, bool>())				.def(init<const Components::UseComponent&>())				.def("BattleUse", &Components::UseComponent::BattleUse, &UseComponentWrap::BattleUseDefault)				.def("Use", &Components::UseComponent::Use, &UseComponentWrap::UseDefault)				.def("HasUse", &Components::UseComponent::HasUse)				.def("HasBattleUse", &Components::UseComponent::HasBattleUse)				.def("Clone", &Components::UseComponent::Clone, &UseComponentWrap::CloneDefault, return_value_policy<reference_existing_object>())				.def("__copy__", &generic__copy__<Components::UseComponent>)				.def("__deepcopy__", &generic__deepcopy__<Components::UseComponent>)				;
I really don't see how this can get turned into an Item, Any ideas?
Thanks for the help so far
> Date: Tue, 8 Nov 2011 11:02:23 -0500
> From: talljimbo at gmail.com
> To: cplusplus-sig at python.org
> Subject: Re: [C++-sig] Passing ownership to C++ through pointer
> 
> On 11/08/2011 01:12 AM, Jay Riley wrote:
> > I'm trying to pass an object from python to C++. I want C++ to take
> > control of the object.
> 
> <snip>
> 
> > Am I doing something wrong? base on that snippet it should work. Is
> > there another way to do this? I know I should probably switch to
> > smart_ptrs and I'm considering it, but Id like to be able to know how to
> > do this, even if its just for future reference. Any help would be
> > appreciated.
> >
> 
> I suspect the inheritance is getting in the way, and Boost.Python isn't 
> being as smart as it could be about converting auto_ptr<T> to 
> auto_ptr<U> when T* is convertible to U*.
> 
> You might want to try having your register function wrapper accept 
> auto_ptr<UseComponentWrap> (and if necessary, providing an overload that 
> takes auto_ptr<UseComponent>).
> 
> Before that, however, you might want to just try using "auto_ptr &" 
> rather than simple "auto_ptr".  It'd be unfortunate if that was 
> necessary, but it's a possibility.
> 
> Good luck!
> 
> Jim
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111108/327123eb/attachment-0001.html>

From talljimbo at gmail.com  Wed Nov  9 03:16:07 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 08 Nov 2011 21:16:07 -0500
Subject: [C++-sig] Passing ownership to C++ through pointer
In-Reply-To: <BAY158-W496F3F564D0606B7C86E5B4DF0@phx.gbl>
References: <20111108021956.GI31565@maxima.lucasfilm.com>,
	<BAY158-W65AAD064686755D5C4A6CBB4DE0@phx.gbl>,
	<4EB9528F.7000200@gmail.com>
	<BAY158-W496F3F564D0606B7C86E5B4DF0@phx.gbl>
Message-ID: <4EB9E267.1060709@gmail.com>

I think stepping through this with a debugger is indeed the best way to 
figure out what's going on.  I'm afraid there's both too little and too 
much information here for me to take a good guess at what the problem is.

I will say that exposing std::vectors of raw pointers to Python seems 
like a very scary thing to do, in terms of ensuring that you have the 
right memory management happening.  Weird bugs can happen when things 
get destroyed unexpectedly.

If the context you're running this code permits, I'd also recommend 
running this through valgrind.

Jim



On 11/08/2011 08:11 PM, Jay Riley wrote:
> Okay I think something else is majorly wrong; my component objects are
> being seen at Item onjects. I'll try to keep the code I show to a minimum.
>
> In my python file, I have the following:
>
> def ItemFactory(ItemNode):
> ItemName = ItemNode.attrib["name"]
> if (True):
> #if ItemName == "Elixir":
> CurrentItem = Item(ItemName)
> print ItemName
> componentNode = ItemNode.find("Components");
> if (componentNode != None):
> components = componentNode.getchildren()
> for component in components:
> if component.tag == "HealingComponent":
> healer = UseComponent("HealUseModule", True, True)
> print "Adding Healing Component"
> CurrentItem.RegisterComponent("Healing Component", healer)
> ItemModule.ItemLibrary.AddItem(CurrentItem)
>
> ItemNode is just an XML node that I parse for properties. Each Item node
> gets sent to this function. As per Jim's advice, RegisterComponent looks
> as follows:
>
> void RegisterComp(Items::Item& item, const std::string& compName,
> std::auto_ptr<UseComponentWrap> comp)//, void* comp)
> {
> item.RegisterComponent(compName, comp.get());
> comp.release();
> }
>
> Now, the components are registering. The problem is, when I try do the
> following:
>
> try
> {
> boost::shared_ptr<Items::Item> item =
> Game.GetItemLibrary().GetItem("Elixir");
> Components::Component* heal = item->GetComponent("Healing Component");
> Components::UseComponent* healertrue =
> dynamic_cast<Components::UseComponent*>(heal);
> if (healertrue != nullptr)
> healertrue->Use(std::vector<Character::BaseCharacter*>(),
> std::vector<Character::BaseCharacter*>());
> }
> catch (boost::python::error_already_set)
> {
> PyErr_Print();
> }
>
> it prints out this error:
>
> AttributeError: 'Item' object has no attribute 'Use'
>
> this is really confusing. How could my UseComponent have become an Item?
>
> I was testing it, and set the AddItem function as:
>
> static std::vector<Components::Component*> test;
> void ItemDatabase::AddItem(boost::shared_ptr<Item> item)
> {
> if (item.get() != nullptr)
> {
> IDToItemMap[item->GetID()] = item;
> NameToItemMap[item->GetName()] = item;
> auto healComp = item->GetComponent("Healing Component");
> if (healComp != nullptr)
> {
> test.push_back(healComp);
> int x = 10;
> }
> }
> }
>
> Here's the weird thing. While I'm adding items, I used x = 10 as a
> breakpoint to watch healComp's be added to test. The first healComp
> get's inserted correctly, and the PyObject's tp_name says UseComponent.
> When the second item with a healComp is being added, I used the vector
> to check the first healComp inserted and it somehow became an item
> (tp_name says Item). This is really weird. I thought it might be a
> scoping problem in my python code, but I can't see anything wrong with
> it. I'm not sure if it's relevant, but I'm including my UseComponent
> wrapping
>
> //Class definition
> class UseComponentWrap : public Components::UseComponent
> {
> public:
> UseComponentWrap(PyObject* self_);
> UseComponentWrap(PyObject* self_, const std::string& name, bool hasUse,
> bool hasBattleUse);
> UseComponentWrap(PyObject* self_, const Components::UseComponent& cmp);
> bool Use(std::vector<Character::BaseCharacter*>& Users,
> std::vector<Character::BaseCharacter*>& Targets) override;
> bool UseDefault(std::vector<Character::BaseCharacter*>& Users,
> std::vector<Character::BaseCharacter*>& Targets);
> bool BattleUse(std::vector<Battle::BattleCharacter*>& Users,
> std::vector<Battle::BattleCharacter*>& Targets, Battle::BattleField*
> field) override;
> bool BattleUseDefault(std::vector<Battle::BattleCharacter*>& Users,
> std::vector<Battle::BattleCharacter*>& Targets, Battle::BattleField* field);
> Components::UseComponent::ClonePtr Clone() const override;
> Components::UseComponent::ClonePtr CloneDefault() const;
> PyObject* self;
>
>
> private:
>
> };
>
> //Python Wrapping
>
> class_<Components::UseComponent, std::auto_ptr<UseComponentWrap>,
> bases<Components::Component> >("UseComponent")
> .def(init<>())
> .def(init<const std::string&,bool, bool>())
> .def(init<const Components::UseComponent&>())
> .def("BattleUse", &Components::UseComponent::BattleUse,
> &UseComponentWrap::BattleUseDefault)
> .def("Use", &Components::UseComponent::Use, &UseComponentWrap::UseDefault)
> .def("HasUse", &Components::UseComponent::HasUse)
> .def("HasBattleUse", &Components::UseComponent::HasBattleUse)
> .def("Clone", &Components::UseComponent::Clone,
> &UseComponentWrap::CloneDefault,
> return_value_policy<reference_existing_object>())
> .def("__copy__", &generic__copy__<Components::UseComponent>)
> .def("__deepcopy__", &generic__deepcopy__<Components::UseComponent>)
> ;
>
> I really don't see how this can get turned into an Item, Any ideas?
>
> Thanks for the help so far
>
>  > Date: Tue, 8 Nov 2011 11:02:23 -0500
>  > From: talljimbo at gmail.com
>  > To: cplusplus-sig at python.org
>  > Subject: Re: [C++-sig] Passing ownership to C++ through pointer
>  >
>  > On 11/08/2011 01:12 AM, Jay Riley wrote:
>  > > I'm trying to pass an object from python to C++. I want C++ to take
>  > > control of the object.
>  >
>  > <snip>
>  >
>  > > Am I doing something wrong? base on that snippet it should work. Is
>  > > there another way to do this? I know I should probably switch to
>  > > smart_ptrs and I'm considering it, but Id like to be able to know
> how to
>  > > do this, even if its just for future reference. Any help would be
>  > > appreciated.
>  > >
>  >
>  > I suspect the inheritance is getting in the way, and Boost.Python isn't
>  > being as smart as it could be about converting auto_ptr<T> to
>  > auto_ptr<U> when T* is convertible to U*.
>  >
>  > You might want to try having your register function wrapper accept
>  > auto_ptr<UseComponentWrap> (and if necessary, providing an overload that
>  > takes auto_ptr<UseComponent>).
>  >
>  > Before that, however, you might want to just try using "auto_ptr &"
>  > rather than simple "auto_ptr". It'd be unfortunate if that was
>  > necessary, but it's a possibility.
>  >
>  > Good luck!
>  >
>  > Jim
>  > _______________________________________________
>  > Cplusplus-sig mailing list
>  > Cplusplus-sig at python.org
>  > http://mail.python.org/mailman/listinfo/cplusplus-sig
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig


From wmamrak at gmail.com  Thu Nov 10 12:40:04 2011
From: wmamrak at gmail.com (Wojciech Mamrak)
Date: Thu, 10 Nov 2011 12:40:04 +0100
Subject: [C++-sig] Converting existing C++ data to python using Boost.Python
Message-ID: <CAMhLx000nh24Guu-2jdAJQdRpue0acB4+w1Y4g8WOj74YMqNQQ@mail.gmail.com>

Hello,

I am aware about the possibility to register converters to- and from-
Python, which enable the implicit conversions between user-defined and
build-in types in Python.
My question is different: Is it possible to use Boost.Python to
convert existing data structures from C++ to Python?

Simple example:

//C++ struct:
struct Foo {
    int x, int y;
};

class_<Foo>("Foo")
    .def_readwrite("x", &Foo::x)
    .def_readwrite("y", &Foo::y);

//instance of Foo
Foo foo = {1, 2};

Now I would like to pass foo object to some Python method. Obviously I
need to create a Python version of this object.
Python is aware of the Foo definition thanks to Boost.Python, but
still, is there any other method of transferring the data than the
one, which uses some glue function which creates a Python Foo instance
based on all members of C++ foo passed separately to it? As far as I
am aware, Python converters are not intended to help here.

The docs claim, that BOOST::PYTHON::Object can not be used as a
replacement of PyObject, i.e. e.g. it can't be passed to some Python
function and used inside of it. Hence I assume it won't be helpful in
this case.

thanks!

From dave at boostpro.com  Thu Nov 10 14:31:19 2011
From: dave at boostpro.com (Dave Abrahams)
Date: Thu, 10 Nov 2011 08:31:19 -0500
Subject: [C++-sig] Converting existing C++ data to python using
	Boost.Python
References: <CAMhLx000nh24Guu-2jdAJQdRpue0acB4+w1Y4g8WOj74YMqNQQ@mail.gmail.com>
Message-ID: <m2vcqsgl3s.fsf@pluto.luannocracy.com>


on Thu Nov 10 2011, Wojciech Mamrak <wmamrak-AT-gmail.com> wrote:

> Hello,
>
> I am aware about the possibility to register converters to- and from-
> Python, which enable the implicit conversions between user-defined and
> build-in types in Python.
> My question is different: Is it possible to use Boost.Python to
> convert existing data structures from C++ to Python?
>
> Simple example:
>
> //C++ struct:
> struct Foo {
>     int x, int y;
> };
>
> class_<Foo>("Foo")
>     .def_readwrite("x", &Foo::x)
>     .def_readwrite("y", &Foo::y);
>
> //instance of Foo
> Foo foo = {1, 2};
>
> Now I would like to pass foo object to some Python method. 

If you get the python method into a boost::python::object, you can just
do:

   boost::python::object the_target
   the_target.attr("the_method")(foo);

> Obviously I need to create a Python version of this object.  

Boost.Python can handle that for you behind the scenes.

> Python is aware of the Foo definition thanks to Boost.Python, but
> still, is there any other method of transferring the data than the
> one, which uses some glue function which creates a Python Foo instance
> based on all members of C++ foo passed separately to it? As far as I
> am aware, Python converters are not intended to help here.
>
> The docs claim, that BOOST::PYTHON::Object can not be used as a
> replacement of PyObject, i.e. e.g. it can't be passed to some Python
> function and used inside of it. 

Where?!


-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com


From wmamrak at gmail.com  Thu Nov 10 20:16:18 2011
From: wmamrak at gmail.com (Wojciech Mamrak)
Date: Thu, 10 Nov 2011 20:16:18 +0100
Subject: [C++-sig] Converting existing C++ data to python using
	Boost.Python
In-Reply-To: <m2vcqsgl3s.fsf@pluto.luannocracy.com>
References: <CAMhLx000nh24Guu-2jdAJQdRpue0acB4+w1Y4g8WOj74YMqNQQ@mail.gmail.com>
	<m2vcqsgl3s.fsf@pluto.luannocracy.com>
Message-ID: <CAMhLx02z995U4PfERuD9KH2Yz-LwpDXWmcpUTOVgsR05aW1v6A@mail.gmail.com>

Thank you Dave,

> If you get the python method into a boost::python::object, you can just
> do:
>
>   boost::python::object the_target
>   the_target.attr("the_method")(foo);
>
>> Obviously I need to create a Python version of this object.
>
> Boost.Python can handle that for you behind the scenes.

Oh, I wasn't aware of that. Epic.

> Where?!
Sorry, my bad, misunderstanding of docs. Also ptr() method of Object
class enables its use in low-level Python C API functions.

Thanks for help!

2011/11/10 Dave Abrahams <dave at boostpro.com>:
>
> on Thu Nov 10 2011, Wojciech Mamrak <wmamrak-AT-gmail.com> wrote:
>
>> Hello,
>>
>> I am aware about the possibility to register converters to- and from-
>> Python, which enable the implicit conversions between user-defined and
>> build-in types in Python.
>> My question is different: Is it possible to use Boost.Python to
>> convert existing data structures from C++ to Python?
>>
>> Simple example:
>>
>> //C++ struct:
>> struct Foo {
>> ? ? int x, int y;
>> };
>>
>> class_<Foo>("Foo")
>> ? ? .def_readwrite("x", &Foo::x)
>> ? ? .def_readwrite("y", &Foo::y);
>>
>> //instance of Foo
>> Foo foo = {1, 2};
>>
>> Now I would like to pass foo object to some Python method.
>
> If you get the python method into a boost::python::object, you can just
> do:
>
> ? boost::python::object the_target
> ? the_target.attr("the_method")(foo);
>
>> Obviously I need to create a Python version of this object.
>
> Boost.Python can handle that for you behind the scenes.
>
>> Python is aware of the Foo definition thanks to Boost.Python, but
>> still, is there any other method of transferring the data than the
>> one, which uses some glue function which creates a Python Foo instance
>> based on all members of C++ foo passed separately to it? As far as I
>> am aware, Python converters are not intended to help here.
>>
>> The docs claim, that BOOST::PYTHON::Object can not be used as a
>> replacement of PyObject, i.e. e.g. it can't be passed to some Python
>> function and used inside of it.
>
> Where?!
>
>
> --
> Dave Abrahams
> BoostPro Computing
> http://www.boostpro.com
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From super24bitsound at hotmail.com  Fri Nov 11 03:46:37 2011
From: super24bitsound at hotmail.com (Jay Riley)
Date: Thu, 10 Nov 2011 21:46:37 -0500
Subject: [C++-sig] Python freezing
Message-ID: <BAY158-W74317AEC5E6B4C0DF07DBB4DD0@phx.gbl>


I'm doing some testing on my C++ wrappers, and I'm hitting a point where python just freezes. The debugger stops, no exception is thrown and my program stops responding. I'm not entirely sure what code to show, so if it's not enough, let me know. Here's the python code:
class HealingComponent(UseComponent):    def __init__(self, FieldHealMap, BattleHealMap = None):        self.FieldHealMap = FieldHealMap        self.BattleHealMap = BattleHealMap        UseComponent.__init__(self, "HealUseModule", True, True)    def Use(self, action):        print action.GetName()        return False    def BattleUse(self, action, field):        print "BattleTest"    def Clone(self):        return copy.deepcopy(self)
Action is defined as:
class Action : public Entity		{		public:			Action();			Action(const std::string& name, const int type = EntityTypes::ActionEntity);			std::vector<ActiveAction> Users;			std::vector<ActiveAction> Targets;
			ClonePtr Clone() const override;		protected:
		};
and wrapped as:
class_<Battle::Action, boost::shared_ptr<ActionWrap>, bases<Entity> >("Action")				.def(init<>())				.def(init<const std::string&>())				.def(init<const Battle::Action&>())				.def_readwrite("Users", &Battle::Action::Users)				.def_readwrite("Targets", &Battle::Action::Targets)				.def("Clone", &Battle::Action::Clone, &ActionWrap::CloneDefault)				.def("__copy__", &generic__copy__< ActionWrap >)				.def("__deepcopy__", &generic__deepcopy__< ActionWrap >)				;
Entity (stripped down) is defined as:
class Entity : public Cloneable<Entity> 	{	public:		const std::string& GetName() const;
		virtual ClonePtr Clone() const override;
		bool operator==(const Entity& entity) const;		bool operator!=(const Entity& entity) const;		Entity& operator=(const Entity& entity);	private:		std::string Name;		static int EntityIDCounter;		int NameHash;		int Type;		int UID;		boost::unordered_map<std::string, boost::shared_ptr<Components::Component> > ComponentCollection;	};
and wrapped as:
class_<Entity, boost::shared_ptr<Entity> >("Entity")				.def("GetName", &Entity::GetName, boost::python::return_value_policy<boost::python::reference_existing_object>())				.def("Clone", &Entity::Clone)//, &EntityWrap::CloneDefault)				.def("__eq__", &Entity::operator ==)				.def("__neq__", &Entity::operator !=)				.def("__copy__", &generic__copy__<Entity>)				.def("__deepcopy__", &generic__deepcopy__<Entity>)				;Finally, action is passed in as follows:
bool UseComponentWrap::Use(Battle::Action* action)		{			return call_method<bool>(self, "Use", ptr(action));//, boost::ref(Targets));		}
and from my main:
Battle::Action act = Battle::Action("Test");				if (healertrue != nullptr)		{			healertrue->Use(&act);		}

the program freezes on action.GetName() in the python file. In fact, it freezes on any attempt to access action's properties. If I don't attemp to access action, the fuction appears to operate correctly. With no exception to catch and no error message, I'm pretty stuck of what is causing this. It'd seem theres a problem wrapping my Action class, but I can't see any errors there. 
When I trace through with my debugger, GetName from Entity is definitely getting called correctly.
When I trace the caller of GetName its being called from
template <class RC, class F, class TC BOOST_PP_ENUM_TRAILING_PARAMS_Z(1, N, class AC)>inline PyObject* invoke(invoke_tag_<false,true>, RC const& rc, F& f, TC& tc BOOST_PP_ENUM_TRAILING_BINARY_PARAMS_Z(1, N, AC, & ac) ){    return rc( (tc().*f)(BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, ac, () BOOST_PP_INTERCEPT)) );}
in invoke.hpp. Continuin the step through, it goes into to_python_indirect.hpp
struct to_python_indirect{    template <class U>    inline PyObject*    operator()(U const& ref) const    {        return this->execute(const_cast<U&>(ref), is_pointer<U>());    }
on this->execute, if I step over, it immediately takes me into errors.cpp
 catch(const boost::python::error_already_set&)    {        // The python error reporting has already been handled.    } 
to be exact, I can trace the error's origin to his code segment:
BOOST_PYTHON_DECL PyTypeObject* registration::get_class_object() const{    if (this->m_class_object == 0)    {        ::PyErr_Format(            PyExc_TypeError            , const_cast<char*>("No Python class registered for C++ class %s")            , this->target_type.name());            throw_error_already_set();    }        return this->m_class_object;}when i check this->target_type, it appears as
_M_d_name = 0x00745d88 ".?AV?$basic_string at DU?$char_traits at D@std@@V?$allocator at D@2@@std@@"
so it seems as though string isn't in the registry? and the error gets digested silently. This is really weird and I don't quite understand why returning a string causing a problem. Is it because it's returned as a const std::string&? I'm almost positive I've returned that type before in another program without a problem. Could there be something wrong with action? it'd seem weird I can call member functions on if it's not exposed correctly, but I really don't know.
If someone can help me out I'd appreciate it.
Thanks
Thanks. 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111110/ab335a74/attachment-0001.html>

From pknudsgaard at rgbnetworks.com  Fri Nov 11 03:53:12 2011
From: pknudsgaard at rgbnetworks.com (Per Knudsgaard)
Date: Fri, 11 Nov 2011 02:53:12 +0000
Subject: [C++-sig] [Boost.Python] How to adjust Python reference counts for
 hybrid objects?
Message-ID: <5606173E2DE93342AB70B54621721AE44D17797E@msg10003.rgbnetworks.com>

   Hi,

   I have run into a small problem with my understanding of Python reference counts with Python/C++ hybrid objects.  I have an abstract interface called A and an associated factory interface called AFactory.  I am looking to implement various A+AFactory combinations in both C++ and Python.  I am using boosts intrusive smart pointers to make life a little more interesting.
   I have attached two files that illustrate my problem (they are a little long, sorry about that but I couldn't come up with a better example).  On linux, compile the cpp file with:

$ g++ foo.cpp -g -fPIC -shared -I/usr/include/python2.7 -lpython2.7 -lboost_python -o module.so

   Run the py file with a single argument ('crash' to produce a segfault, anything else to have it work).

   What am I doing?  The abstract factory interface looks roughly like this (with most of the pointer stuff removed):

class AFactory {
  public:
    virtual A::Ptr newA() = 0;
};

   A single function that returns an intrusive pointer to the A interface.  I have an AFactoryWrap class that will forward the call to a python implementation.  I derive it in Python as AFactoryDerived (where ADerived is the python implementation of the A interface):

tmp = ADerived()
class AFactoryDerived( module.AFactory ):
    def __init__( self ):
        module.AFactory.__init__( self )

    def newA( self ):
        if( sys.argv[1] == 'crash' ):
            return ADerived() # segfaults
        else:
            return tmp        # works

   I then trigger the whole thing from a simple C++ function (that I also call from python, just to add to the fun):

void f2( const AFactory::Ptr &factory ) {
    A::Ptr a = factory->newA();
    a->f();
}

   Get a new instance of A from the factory.  Call a virtual function, implemented in Python, on it that prints "In f" on stdout.

   It works if the instance of ADerived is persistent (global variable) and segfaults if it is temporary (local to the newA call).  That makes me suspect that the problem is with Python reference counts and that I somehow need to maintain a reference from the c++ side so it doesn't get garbage collected.  The boost::python::handle<> class would seem like the way to go, except I can't seem to find any decent examples of how to use it.

   Am I on the right track?  I would assume that the handle<> should be added to the Wrapper constructors, and the ptrRelease function enhanced to count C++ and Python references, but the syntax of creating a handle and obtaining python reference counts is escaping me.

   Thanks,

   -- Per.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111111/ddbeda43/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: foo.cpp
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111111/ddbeda43/attachment.ksh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: foo.py
Type: application/octet-stream
Size: 528 bytes
Desc: foo.py
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111111/ddbeda43/attachment.obj>

From dave at boostpro.com  Fri Nov 11 04:00:44 2011
From: dave at boostpro.com (Dave Abrahams)
Date: Thu, 10 Nov 2011 22:00:44 -0500
Subject: [C++-sig] Python freezing
References: <BAY158-W74317AEC5E6B4C0DF07DBB4DD0@phx.gbl>
Message-ID: <m2ehxffjmr.fsf@pluto.luannocracy.com>


on Thu Nov 10 2011, Jay Riley <super24bitsound-AT-hotmail.com> wrote:

> so it seems as though string isn't in the registry? and the error
> gets digested silently. This is really weird and I don't quite
> understand why returning a string causing a problem. Is it because
> it's returned as a const std::string&? 

Yep.  Use a thin wrapper that returns by value.


-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com


From super24bitsound at hotmail.com  Fri Nov 11 04:24:43 2011
From: super24bitsound at hotmail.com (Jay Riley)
Date: Thu, 10 Nov 2011 22:24:43 -0500
Subject: [C++-sig] Python freezing
In-Reply-To: <m2ehxffjmr.fsf@pluto.luannocracy.com>
References: <BAY158-W74317AEC5E6B4C0DF07DBB4DD0@phx.gbl>,
	<m2ehxffjmr.fsf@pluto.luannocracy.com>
Message-ID: <BAY158-W5101E6E03AE29E1CC30A23B4DD0@phx.gbl>


Well that was easy, Thanks

Out of curiosity is there a reason const std::string& doesn't work? Would it just be more trouble then it's worth to support it?

> To: cplusplus-sig at python.org
> From: dave at boostpro.com
> Date: Thu, 10 Nov 2011 22:00:44 -0500
> Subject: Re: [C++-sig] Python freezing
> 
> 
> on Thu Nov 10 2011, Jay Riley <super24bitsound-AT-hotmail.com> wrote:
> 
> > so it seems as though string isn't in the registry? and the error
> > gets digested silently. This is really weird and I don't quite
> > understand why returning a string causing a problem. Is it because
> > it's returned as a const std::string&? 
> 
> Yep.  Use a thin wrapper that returns by value.
> 
> 
> -- 
> Dave Abrahams
> BoostPro Computing
> http://www.boostpro.com
> 
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111110/d60761e8/attachment.html>

From pknudsgaard at rgbnetworks.com  Fri Nov 11 23:41:25 2011
From: pknudsgaard at rgbnetworks.com (Per Knudsgaard)
Date: Fri, 11 Nov 2011 22:41:25 +0000
Subject: [C++-sig] [Boost.Python] How to adjust Python reference counts
 for hybrid objects?
In-Reply-To: <5606173E2DE93342AB70B54621721AE44D17797E@msg10003.rgbnetworks.com>
References: <5606173E2DE93342AB70B54621721AE44D17797E@msg10003.rgbnetworks.com>
Message-ID: <5606173E2DE93342AB70B54621721AE44D17845A@msg10003.rgbnetworks.com>

   After playing around with this some more, I seem to be right in thinking that this is a problem with Python reference count.  I have tried 3 different approaches to solving the problem:


1.       My first thought was to get a hold of a PyObject *self pointer in the constructor and incrementing its reference count forcing the Python object to persist despite going out of scope in Python.  I found two ways to do that, described in [1] and [2].  They seem to fail for reasons described in the answers to [2].

2.       My second thought was to increment the reference count the moment the object enters into C++.  This stopped the crash, but if I instrument the __del__ method in the Python object, it gets called before I increment the reference count.  I think that is a strong indication that this method left me with memory corruption so I am hesitant to follow this path.

3.       The third choice is that I implement a C++ method that marks the object as "shared" and increments its reference count.  I have to make an explicit call to that method in the Python constructor and I check if the object has been marked shared before passing it into C++.  I have added the appropriate smart pointer logic to maintain the extra reference count as long as there are C++ pointers around.

   The third option seems to work.  Constructors and destructors are called in reasonable ways indicating that objects are neither get deleted prematurely nor do they stick around for too long.  I just don't like it because I would prefer the integration to be transparent, the person implementing the derived class in python should not need to know that (s)he is deriving it from a C++ class.

   Which returns me to the question of how one is supposed to handle this case?  My current solution is neither crashing nor leaking memory, but it isn't very elegant.  It kind of stands out against the elegance of Boost and Boost.Python.

   Thanks,

   -- Per.

   [1] http://wiki.python.org/moin/boost.python/HowTo#ownership_of_C.2B-.2B-_object_extended_in_Python
   [2] http://mail.python.org/pipermail/cplusplus-sig/2007-March/011790.html

From: cplusplus-sig-bounces+pknudsgaard=rgbnetworks.com at python.org [mailto:cplusplus-sig-bounces+pknudsgaard=rgbnetworks.com at python.org] On Behalf Of Per Knudsgaard
Sent: Thursday, November 10, 2011 6:53 PM
To: cplusplus-sig at python.org
Subject: [C++-sig] [Boost.Python] How to adjust Python reference counts for hybrid objects?

   Hi,

   I have run into a small problem with my understanding of Python reference counts with Python/C++ hybrid objects.  I have an abstract interface called A and an associated factory interface called AFactory.  I am looking to implement various A+AFactory combinations in both C++ and Python.  I am using boosts intrusive smart pointers to make life a little more interesting.

   I have attached two files that illustrate my problem (they are a little long, sorry about that but I couldn't come up with a better example).  On linux, compile the cpp file with:

$ g++ foo.cpp -g -fPIC -shared -I/usr/include/python2.7 -lpython2.7 -lboost_python -o module.so

   Run the py file with a single argument ('crash' to produce a segfault, anything else to have it work).

   What am I doing?  The abstract factory interface looks roughly like this (with most of the pointer stuff removed):

class AFactory {
  public:
    virtual A::Ptr newA() = 0;
};

   A single function that returns an intrusive pointer to the A interface.  I have an AFactoryWrap class that will forward the call to a python implementation.  I derive it in Python as AFactoryDerived (where ADerived is the python implementation of the A interface):

tmp = ADerived()
class AFactoryDerived( module.AFactory ):
    def __init__( self ):
        module.AFactory.__init__( self )

    def newA( self ):
        if( sys.argv[1] == 'crash' ):
            return ADerived() # segfaults
        else:
            return tmp        # works

   I then trigger the whole thing from a simple C++ function (that I also call from python, just to add to the fun):

void f2( const AFactory::Ptr &factory ) {
    A::Ptr a = factory->newA();
    a->f();
}

   Get a new instance of A from the factory.  Call a virtual function, implemented in Python, on it that prints "In f" on stdout.

   It works if the instance of ADerived is persistent (global variable) and segfaults if it is temporary (local to the newA call).  That makes me suspect that the problem is with Python reference counts and that I somehow need to maintain a reference from the c++ side so it doesn't get garbage collected.  The boost::python::handle<> class would seem like the way to go, except I can't seem to find any decent examples of how to use it.

   Am I on the right track?  I would assume that the handle<> should be added to the Wrapper constructors, and the ptrRelease function enhanced to count C++ and Python references, but the syntax of creating a handle and obtaining python reference counts is escaping me.

   Thanks,

   -- Per.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111111/10b767bf/attachment.html>

From adam.preble at gmail.com  Sat Nov 12 17:34:49 2011
From: adam.preble at gmail.com (Adam Preble)
Date: Sat, 12 Nov 2011 10:34:49 -0600
Subject: [C++-sig] Fwd: Passing Python-derived class back into a native C++
 class doesn't match the C++ signature
In-Reply-To: <CAE-m0_kNyorNDSZSvwMUgAbsreApN-muPjo7_=f5CKXUtoMnPQ@mail.gmail.com>
References: <CAE-m0_kNyorNDSZSvwMUgAbsreApN-muPjo7_=f5CKXUtoMnPQ@mail.gmail.com>
Message-ID: <CAE-m0_koVfo6SU2GM2y6-qe-YdKmZg1j4WdO-0TPAgwRgXKcug@mail.gmail.com>

I am seeing this come up in many variations, with people doing slightly
different things.  I haven't yet seen an example doing something in this
exact progression.  I apologize here because I feel like I'm repeating a
topic that has come up on the list before--even just a few weeks back.
Here's what I'm trying to do:

1. Expose a base class with a pure virtual method.
2. Expose an acceptor class that accepts that base class as an argument for
a method.
3. Implement the base class in Python
4. Pass that implemented class to the acceptor

I am using Boost 1.42.

I have some source snippets.  I am using the wrapper method, though without
it I had the same result:

    class BaseTest
    {
    public:
        virtual int GimmeNumber() = 0;
        virtual ~BaseTest() {}
    };

    class BaseTestWrap : public BaseTest, public wrapper<BaseTest>
    {
        int GimmeNumber()
        {
            return this->get_override("GimmeNumber")();
        }
    };

    class ITakeFoo
    {
    public:
        void DoStuff(BaseTest* test)
        {
            cout << "Whoo calling BaseTest to do stuff: " <<
test->GimmeNumber() << endl;
        }
    };

...

    class_<BaseTestWrap, boost::noncopyable>("BaseTest")
        .def("GimmeNumber", pure_virtual(&BaseTest::GimmeNumber))   // I
have tried &BaseTestWrap here too.  Not sure what to do but it doesn't
solve the issue at hand
        ;

    class_<ITakeFoo>("ITakeFoo", init<>())
        .def("DoStuff", &ITakeFoo::DoStuff)
        ;


In Python:
class DerivedTest(BaseTest):
    def __init__(self):
        pass

    def GimmeNumber(self):
        return 100

dt = DerivedTest()
ITF = ITakeFoo()
ITF.DoStuff(dt)

The DoStuff() call fails:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Boost.Python.ArgumentError: Python argument types in
    ITakeFoo.DoStuff(ITakeFoo, DerivedTest)
did not match C++ signature:
    DoStuff(ITakeFoo {lvalue}, BaseTest*)

I can appreciate there being a memory-management-related policy issue but I
can't see what might have to be done.  What else am I missing?  I guess I
should add the documentation I've found would be nice to go this last mile
and show how to do this step in the Boost documentation.  The wrapper stuff
was fine enough but they don't see it getting called back into the native
C++ code.

As it stands, if I do type.mro(DerivedTest), I see:
[<class '__main__.DerivedTest'>, <class 'BaseTest'>, <type
'Boost.Python.instance'>, <type 'object'>]

It appears to some extent it got it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111112/9491b590/attachment.html>

From nat at lindenlab.com  Sat Nov 12 20:02:33 2011
From: nat at lindenlab.com (Nat Goodspeed)
Date: Sat, 12 Nov 2011 14:02:33 -0500
Subject: [C++-sig] Fwd: Passing Python-derived class back into a native
	C++ class doesn't match the C++ signature
In-Reply-To: <CAE-m0_koVfo6SU2GM2y6-qe-YdKmZg1j4WdO-0TPAgwRgXKcug@mail.gmail.com>
References: <CAE-m0_kNyorNDSZSvwMUgAbsreApN-muPjo7_=f5CKXUtoMnPQ@mail.gmail.com>
	<CAE-m0_koVfo6SU2GM2y6-qe-YdKmZg1j4WdO-0TPAgwRgXKcug@mail.gmail.com>
Message-ID: <89C1B59C-6F25-4573-BABD-5F067611F13F@lindenlab.com>

On Nov 12, 2011, at 11:34 AM, Adam Preble <adam.preble at gmail.com> wrote:

> I am seeing this come up in many variations, with people doing slightly different things. 

I have no direct experience with this issue myself, but I can parrot a bit of advice I've read here before.

> In Python:
> class DerivedTest(BaseTest):
>     def __init__(self):
        BaseTest.__init__(self)
>         pass
> 
>     def GimmeNumber(self):
>         return 100

From adam.preble at gmail.com  Sat Nov 12 20:12:27 2011
From: adam.preble at gmail.com (Adam Preble)
Date: Sat, 12 Nov 2011 13:12:27 -0600
Subject: [C++-sig] Fwd: Passing Python-derived class back into a native
 C++ class doesn't match the C++ signature
In-Reply-To: <89C1B59C-6F25-4573-BABD-5F067611F13F@lindenlab.com>
References: <CAE-m0_kNyorNDSZSvwMUgAbsreApN-muPjo7_=f5CKXUtoMnPQ@mail.gmail.com>
	<CAE-m0_koVfo6SU2GM2y6-qe-YdKmZg1j4WdO-0TPAgwRgXKcug@mail.gmail.com>
	<89C1B59C-6F25-4573-BABD-5F067611F13F@lindenlab.com>
Message-ID: <CAE-m0_mJDxifc0SUc4uhZVVu385wDmtRAYe2TdjxRvm1iyFjKg@mail.gmail.com>

Woooah that looks like it does work, although I had to interpret it like so:

class DerivedTest(BaseTest):
  def __init__(self):
    BaseTest.__init__(self)

  def GimmeNumber(self):
    return 100

I don't really consider myself an expert on these things, though I was
Googling around on the proper way to handle the inheritance construction in
Python.  Is this kind of the normal rigors when subclassing in Python
normally, or is this something I must particularly pay attention to when
using Boost?

On Sat, Nov 12, 2011 at 1:02 PM, Nat Goodspeed <nat at lindenlab.com> wrote:

> On Nov 12, 2011, at 11:34 AM, Adam Preble <adam.preble at gmail.com> wrote:
>
> > I am seeing this come up in many variations, with people doing slightly
> different things.
>
> I have no direct experience with this issue myself, but I can parrot a bit
> of advice I've read here before.
>
> > In Python:
> > class DerivedTest(BaseTest):
> >     def __init__(self):
>         BaseTest.__init__(self)
> >         pass
> >
> >     def GimmeNumber(self):
> >         return 100
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111112/f2c0e4ef/attachment.html>

From talljimbo at gmail.com  Sat Nov 12 20:25:42 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Sat, 12 Nov 2011 14:25:42 -0500
Subject: [C++-sig] Fwd: Passing Python-derived class back into a native
 C++ class doesn't match the C++ signature
In-Reply-To: <CAE-m0_mJDxifc0SUc4uhZVVu385wDmtRAYe2TdjxRvm1iyFjKg@mail.gmail.com>
References: <CAE-m0_kNyorNDSZSvwMUgAbsreApN-muPjo7_=f5CKXUtoMnPQ@mail.gmail.com>
	<CAE-m0_koVfo6SU2GM2y6-qe-YdKmZg1j4WdO-0TPAgwRgXKcug@mail.gmail.com>
	<89C1B59C-6F25-4573-BABD-5F067611F13F@lindenlab.com>
	<CAE-m0_mJDxifc0SUc4uhZVVu385wDmtRAYe2TdjxRvm1iyFjKg@mail.gmail.com>
Message-ID: <4EBEC836.9030909@gmail.com>

On 11/12/2011 02:12 PM, Adam Preble wrote:
> Woooah that looks like it does work, although I had to interpret it like so:
>
> class DerivedTest(BaseTest):
>    def __init__(self):
>      BaseTest.__init__(self)
>
>    def GimmeNumber(self):
>      return 100
>
> I don't really consider myself an expert on these things, though I was
> Googling around on the proper way to handle the inheritance construction
> in Python.  Is this kind of the normal rigors when subclassing in Python
> normally, or is this something I must particularly pay attention to when
> using Boost?
>

Calling the base class __init__ is less important in pure Python (i.e. 
sometimes it's not necessary), but it's still definitely good practice; 
it depends on whether the base class __init__ does anything important. 
Python does not ever call it automatically, and in Boost.Python it 
definitely does something important.

Jim

From nat at lindenlab.com  Sat Nov 12 21:41:04 2011
From: nat at lindenlab.com (Nat Goodspeed)
Date: Sat, 12 Nov 2011 15:41:04 -0500
Subject: [C++-sig] Fwd: Passing Python-derived class back into a native
	C++ class doesn't match the C++ signature
In-Reply-To: <4EBEC836.9030909@gmail.com>
References: <CAE-m0_kNyorNDSZSvwMUgAbsreApN-muPjo7_=f5CKXUtoMnPQ@mail.gmail.com>
	<CAE-m0_koVfo6SU2GM2y6-qe-YdKmZg1j4WdO-0TPAgwRgXKcug@mail.gmail.com>
	<89C1B59C-6F25-4573-BABD-5F067611F13F@lindenlab.com>
	<CAE-m0_mJDxifc0SUc4uhZVVu385wDmtRAYe2TdjxRvm1iyFjKg@mail.gmail.com>
	<4EBEC836.9030909@gmail.com>
Message-ID: <6F8B9857-3A83-4BDD-BFC8-C9E017E2A7CF@lindenlab.com>

On Nov 12, 2011, at 2:25 PM, Jim Bosch <talljimbo at gmail.com> wrote:

> On 11/12/2011 02:12 PM, Adam Preble wrote:
>> Woooah that looks like it does work, although I had to interpret it like so:
>> 
>> class DerivedTest(BaseTest):
>>   def __init__(self):
>>     BaseTest.__init__(self)

Yes, sorry the email quoting confused my intended indentation.

>> Is this kind of the normal rigors when subclassing in Python
>> normally, or is this something I must particularly pay attention to when
>> using Boost?
> 
> Calling the base class __init__ is less important in pure Python (i.e. sometimes it's not necessary), but it's still definitely good practice; it depends on whether the base class __init__ does anything important. Python does not ever call it automatically, and in Boost.Python it definitely does something important.

That's right: this is NOT specific to Boost.Python. If your Python subclass constructor doesn't explicitly call the base-class constructor, the base class remains uninitialized. As Jim says, with Boost.Python the bad effect of an uninitialized base can include that type recognition failure.

fwiw, if the only logic in your subclass constructor is to forward all the same arguments to the base-class constructor, you could omit it entirely and just let Python implicitly call the inherited base-class constructor. I only define a subclass constructor when it must perform additional logic beyond initializing the base class. Again, that's a general rule in Python rather than anything specific to Boost.Python.
> 

From talljimbo at gmail.com  Sat Nov 12 22:19:19 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Sat, 12 Nov 2011 16:19:19 -0500
Subject: [C++-sig] [Boost.Python v3] handle, object, and PyObject*
Message-ID: <4EBEE2D7.2040206@gmail.com>

Boost.Python has two smart-pointer-like classes for holding PyObject*. 
"handle<>" is more or less a customized intrusive_ptr, with the ability 
to be templated on other PyObject "derived" types (like PyTypeObject). 
The "object" hierarchy provides a much higher-level API, and defaults to 
Python's None rather than a null pointer.

In terms of actual usage, I see:

  - "object" is widely used in the public Boost.Python API, and is 
extremely useful there.

  - "handle<>" is rarely used except as a bridge to create object from 
PyObject*.  The internal API tends to use raw PyObject* instead.

  - "handle<>" is almost never used with a non-default template parameter.

Is there any reason why the internal API uses raw PyObject* instead of 
handle<>?

If not, I'm inclined to start using handle<> there (or even object, for 
features that have become widely used like rvalue-converters).

And since handle<> is almost never used as a template, I'm inclined to 
make it a regular class - or even just a typedef of intrusive_ptr<PyObject>.

Any thoughts?

Jim

From pknudsgaard at rgbnetworks.com  Tue Nov 15 03:54:54 2011
From: pknudsgaard at rgbnetworks.com (Per Knudsgaard)
Date: Tue, 15 Nov 2011 02:54:54 +0000
Subject: [C++-sig] [Boost.Python] How to adjust Python reference counts
 for hybrid objects?
In-Reply-To: <5606173E2DE93342AB70B54621721AE44D17845A@msg10003.rgbnetworks.com>
References: <5606173E2DE93342AB70B54621721AE44D17797E@msg10003.rgbnetworks.com>
	<5606173E2DE93342AB70B54621721AE44D17845A@msg10003.rgbnetworks.com>
Message-ID: <5606173E2DE93342AB70B54621721AE44D17956B@msg10003.rgbnetworks.com>

   I have gotten (3) to work, but I am wondering about two things.  Since I couldn't get the constructor to work, I went for a modifier that returns the object.  That allows me to do the following (where NewObject is a Python object that derives from a C++ object):

class Factory:
   def create( self ):
       return NewObject().share()

   The share function looks like this:

PyObject *share() {
    PyObject *owner = boost::python::detail::wrapper_base_::get_owner( *this );
    Py_INCREF(owner);
    Py_INCREF(owner);
    // Fix the C++ reference count

    return owner;
}

   So, two questions:

*         Is this a legal/recommended way to do a modifier?

*         If I only do a single INCREF, then the object is destroyed while executing the NewObject().share() call.  Why do I need two INCREFs?

   Anyway, the unittests are passing, I now have intrusive pointers with shared ownership between C++ and Python.  Being forced to call the share method is a little annoying, but it works.  Thanks for making it possible.

   -- Per.

From: cplusplus-sig-bounces+pknudsgaard=rgbnetworks.com at python.org [mailto:cplusplus-sig-bounces+pknudsgaard=rgbnetworks.com at python.org] On Behalf Of Per Knudsgaard
Sent: Friday, November 11, 2011 2:41 PM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] [Boost.Python] How to adjust Python reference counts for hybrid objects?

   After playing around with this some more, I seem to be right in thinking that this is a problem with Python reference count.  I have tried 3 different approaches to solving the problem:


1.       My first thought was to get a hold of a PyObject *self pointer in the constructor and incrementing its reference count forcing the Python object to persist despite going out of scope in Python.  I found two ways to do that, described in [1] and [2].  They seem to fail for reasons described in the answers to [2].

2.       My second thought was to increment the reference count the moment the object enters into C++.  This stopped the crash, but if I instrument the __del__ method in the Python object, it gets called before I increment the reference count.  I think that is a strong indication that this method left me with memory corruption so I am hesitant to follow this path.

3.       The third choice is that I implement a C++ method that marks the object as "shared" and increments its reference count.  I have to make an explicit call to that method in the Python constructor and I check if the object has been marked shared before passing it into C++.  I have added the appropriate smart pointer logic to maintain the extra reference count as long as there are C++ pointers around.

   The third option seems to work.  Constructors and destructors are called in reasonable ways indicating that objects are neither get deleted prematurely nor do they stick around for too long.  I just don't like it because I would prefer the integration to be transparent, the person implementing the derived class in python should not need to know that (s)he is deriving it from a C++ class.

   Which returns me to the question of how one is supposed to handle this case?  My current solution is neither crashing nor leaking memory, but it isn't very elegant.  It kind of stands out against the elegance of Boost and Boost.Python.

   Thanks,

   -- Per.

   [1] http://wiki.python.org/moin/boost.python/HowTo#ownership_of_C.2B-.2B-_object_extended_in_Python
   [2] http://mail.python.org/pipermail/cplusplus-sig/2007-March/011790.html

From: cplusplus-sig-bounces+pknudsgaard=rgbnetworks.com at python.org<mailto:cplusplus-sig-bounces+pknudsgaard=rgbnetworks.com at python.org> [mailto:cplusplus-sig-bounces+pknudsgaard=rgbnetworks.com at python.org] On Behalf Of Per Knudsgaard
Sent: Thursday, November 10, 2011 6:53 PM
To: cplusplus-sig at python.org<mailto:cplusplus-sig at python.org>
Subject: [C++-sig] [Boost.Python] How to adjust Python reference counts for hybrid objects?

   Hi,

   I have run into a small problem with my understanding of Python reference counts with Python/C++ hybrid objects.  I have an abstract interface called A and an associated factory interface called AFactory.  I am looking to implement various A+AFactory combinations in both C++ and Python.  I am using boosts intrusive smart pointers to make life a little more interesting.

   I have attached two files that illustrate my problem (they are a little long, sorry about that but I couldn't come up with a better example).  On linux, compile the cpp file with:

$ g++ foo.cpp -g -fPIC -shared -I/usr/include/python2.7 -lpython2.7 -lboost_python -o module.so

   Run the py file with a single argument ('crash' to produce a segfault, anything else to have it work).

   What am I doing?  The abstract factory interface looks roughly like this (with most of the pointer stuff removed):

class AFactory {
  public:
    virtual A::Ptr newA() = 0;
};

   A single function that returns an intrusive pointer to the A interface.  I have an AFactoryWrap class that will forward the call to a python implementation.  I derive it in Python as AFactoryDerived (where ADerived is the python implementation of the A interface):

tmp = ADerived()
class AFactoryDerived( module.AFactory ):
    def __init__( self ):
        module.AFactory.__init__( self )

    def newA( self ):
        if( sys.argv[1] == 'crash' ):
            return ADerived() # segfaults
        else:
            return tmp        # works

   I then trigger the whole thing from a simple C++ function (that I also call from python, just to add to the fun):

void f2( const AFactory::Ptr &factory ) {
    A::Ptr a = factory->newA();
    a->f();
}

   Get a new instance of A from the factory.  Call a virtual function, implemented in Python, on it that prints "In f" on stdout.

   It works if the instance of ADerived is persistent (global variable) and segfaults if it is temporary (local to the newA call).  That makes me suspect that the problem is with Python reference counts and that I somehow need to maintain a reference from the c++ side so it doesn't get garbage collected.  The boost::python::handle<> class would seem like the way to go, except I can't seem to find any decent examples of how to use it.

   Am I on the right track?  I would assume that the handle<> should be added to the Wrapper constructors, and the ptrRelease function enhanced to count C++ and Python references, but the syntax of creating a handle and obtaining python reference counts is escaping me.

   Thanks,

   -- Per.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111115/5f246f84/attachment-0001.html>

From olivier.voyer at gmail.com  Tue Nov 15 14:53:05 2011
From: olivier.voyer at gmail.com (Olivier Voyer)
Date: Tue, 15 Nov 2011 08:53:05 -0500
Subject: [C++-sig] Multiple modules in a single pyd
Message-ID: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>

Hi everyone,

Is it possible to have multiple modules linking to one single pyd file? I'm
using SWIG with VS2010 and I can't find a way of doing that.

Thank you,

Olivier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111115/77307013/attachment.html>

From talljimbo at gmail.com  Tue Nov 15 15:24:01 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 15 Nov 2011 09:24:01 -0500
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
Message-ID: <CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>

On Nov 15, 2011 8:53 AM, "Olivier Voyer" <olivier.voyer at gmail.com> wrote:
>
> Hi everyone,
>
> Is it possible to have multiple modules linking to one single pyd file?
I'm using SWIG with VS2010 and I can't find a way of doing that.
>

I believe this is not supported by the Python C-API itself, regardless of
what wrapper generator or library you use.  The name of the module import
function needs to be related to the loadable module file name, and you
generally can't have two functions for which that's true.

Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111115/16ee0be2/attachment.html>

From olivier.voyer at gmail.com  Tue Nov 15 16:00:08 2011
From: olivier.voyer at gmail.com (Olivier Voyer)
Date: Tue, 15 Nov 2011 10:00:08 -0500
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
Message-ID: <CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>

Jim, thank you for your answer.

What if I have this big C++ project that I cannot split in multiple smaller
projects? I have no choice but to create a big Python module exposing all
the functions/classes? What is the common practice?

Regards,

Olivier

On Tue, Nov 15, 2011 at 9:24 AM, Jim Bosch <talljimbo at gmail.com> wrote:

> On Nov 15, 2011 8:53 AM, "Olivier Voyer" <olivier.voyer at gmail.com> wrote:
> >
> > Hi everyone,
> >
> > Is it possible to have multiple modules linking to one single pyd file?
> I'm using SWIG with VS2010 and I can't find a way of doing that.
> >
>
> I believe this is not supported by the Python C-API itself, regardless of
> what wrapper generator or library you use.  The name of the module import
> function needs to be related to the loadable module file name, and you
> generally can't have two functions for which that's true.
>
> Jim
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111115/708a3bb6/attachment.html>

From fraca7 at free.fr  Tue Nov 15 15:51:23 2011
From: fraca7 at free.fr (=?iso-8859-1?Q?J=E9r=F4me_Laheurte?=)
Date: Tue, 15 Nov 2011 15:51:23 +0100
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
Message-ID: <54FF6DC2-2DA7-4092-A6BF-EC8A9AB6AE29@free.fr>


Le 15 nov. 2011 ? 15:24, Jim Bosch a ?crit :

> On Nov 15, 2011 8:53 AM, "Olivier Voyer" <olivier.voyer at gmail.com> wrote:
> >
> > Hi everyone,
> >
> > Is it possible to have multiple modules linking to one single pyd file? I'm using SWIG with VS2010 and I can't find a way of doing that.
> >
> 
> I believe this is not supported by the Python C-API itself, regardless of what wrapper generator or library you use.  The name of the module import function needs to be related to the loadable module file name, and you generally can't have two functions for which that's true.

But you can put several submodules in an extension module, as a workaround (see code below). I always wondered if that was possible with boost::python or SWIG ?


#include <Python.h>

static PyObject *myfunc(PyObject *self, PyObject *args)
{
    if (!PyArg_ParseTuple(args, "|myfunc"))
        return NULL;

    printf("Hello, world\n");

    Py_INCREF(Py_None);
    return Py_None;
}

static PyMethodDef functions[] = {
    { "myfunc", (PyCFunction)myfunc, METH_VARARGS, "" },
    { NULL }
};

static PyMethodDef nofunctions[] = {
    { NULL }
};

void inittestmod()
{
    PyObject *mdl, *submdl;

    if (!(mdl = Py_InitModule3("testmod", nofunctions, "")))
        return;

    if (!(submdl = Py_InitModule3("foo", functions, "")))
        return;

    Py_INCREF(submdl);
    PyModule_AddObject(mdl, "foo", submdl);
}

After building,

>>> import testmod
>>> testmod.foo.myfunc()
Hello, world
>>>


From talljimbo at gmail.com  Tue Nov 15 16:20:03 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 15 Nov 2011 10:20:03 -0500
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
Message-ID: <4EC28323.4080803@gmail.com>

On 11/15/2011 10:00 AM, Olivier Voyer wrote:
> Jim, thank you for your answer.
>
> What if I have this big C++ project that I cannot split in multiple
> smaller projects? I have no choice but to create a big Python module
> exposing all the functions/classes? What is the common practice?
>

It sounds like you just want to make a Python package - a directory with 
an __init__.py file that contains all your submodules, which can each be 
compiled separately.

Is there some reason you felt you needed to put all the modules in the 
same pyd file?

Jim

From talljimbo at gmail.com  Tue Nov 15 16:20:04 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 15 Nov 2011 10:20:04 -0500
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <54FF6DC2-2DA7-4092-A6BF-EC8A9AB6AE29@free.fr>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<54FF6DC2-2DA7-4092-A6BF-EC8A9AB6AE29@free.fr>
Message-ID: <4EC28324.8090101@gmail.com>

On 11/15/2011 09:51 AM, J?r?me Laheurte wrote:
>
> But you can put several submodules in an extension module, as a
> workaround (see code below). I always wondered if that was possible
> with boost::python or SWIG ?

Interesting, I hadn't thought of that.

Anyhow, I believe it should be possible (but not very useful) with both; 
you can make the inner module, but it's just a Python C-API module and 
you have to manually add SWIG or Boost.Python-wrapped functions and 
types to it.

Jim


>
>
> #include<Python.h>
>
> static PyObject *myfunc(PyObject *self, PyObject *args) { if
> (!PyArg_ParseTuple(args, "|myfunc")) return NULL;
>
> printf("Hello, world\n");
>
> Py_INCREF(Py_None); return Py_None; }
>
> static PyMethodDef functions[] = { { "myfunc", (PyCFunction)myfunc,
> METH_VARARGS, "" }, { NULL } };
>
> static PyMethodDef nofunctions[] = { { NULL } };
>
> void inittestmod() { PyObject *mdl, *submdl;
>
> if (!(mdl = Py_InitModule3("testmod", nofunctions, ""))) return;
>
> if (!(submdl = Py_InitModule3("foo", functions, ""))) return;
>
> Py_INCREF(submdl); PyModule_AddObject(mdl, "foo", submdl); }
>
> After building,
>
>>>> import testmod testmod.foo.myfunc()
> Hello, world
>>>>
>
> _______________________________________________ Cplusplus-sig mailing
> list Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig


From olivier.voyer at gmail.com  Tue Nov 15 16:42:34 2011
From: olivier.voyer at gmail.com (Olivier Voyer)
Date: Tue, 15 Nov 2011 10:42:34 -0500
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <4EC28323.4080803@gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
	<4EC28323.4080803@gmail.com>
Message-ID: <CA+_ahhzneSSfxOH6eHv8pdNETO_U2jKNuadc0x+iY-GEKtvwwg@mail.gmail.com>

Yes, that's exactly what I want to do. But, from what I understand, each
module or submodule (.py file) must link to its own .pyd file, ie
module1.py -> _module1.pyd, module2.py -> _module2.pyd.

What I would really love to have is:

myPackage/
myPackage/__init__.py
myPackage/module1.py -> linked to _mySingleAPI.pyd
myPackage/module2.py -> linked to _mySingleAPI.pyd

I just need a way to have some sort of modularity within my application. I
cannot split my main project into smaller projects, it would require a huge
amount of time.

On Tue, Nov 15, 2011 at 10:20 AM, Jim Bosch <talljimbo at gmail.com> wrote:

> On 11/15/2011 10:00 AM, Olivier Voyer wrote:
>
>> Jim, thank you for your answer.
>>
>> What if I have this big C++ project that I cannot split in multiple
>> smaller projects? I have no choice but to create a big Python module
>> exposing all the functions/classes? What is the common practice?
>>
>>
> It sounds like you just want to make a Python package - a directory with
> an __init__.py file that contains all your submodules, which can each be
> compiled separately.
>
> Is there some reason you felt you needed to put all the modules in the
> same pyd file?
>
>
> Jim
> ______________________________**_________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/**mailman/listinfo/cplusplus-sig<http://mail.python.org/mailman/listinfo/cplusplus-sig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111115/c6ac964c/attachment-0001.html>

From wichert at wiggy.net  Tue Nov 15 16:08:26 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Tue, 15 Nov 2011 16:08:26 +0100
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <54FF6DC2-2DA7-4092-A6BF-EC8A9AB6AE29@free.fr>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<54FF6DC2-2DA7-4092-A6BF-EC8A9AB6AE29@free.fr>
Message-ID: <4EC2806A.8050507@wiggy.net>

On 11/15/2011 03:51 PM, J?r?me Laheurte wrote:
> Le 15 nov. 2011 ? 15:24, Jim Bosch a ?crit :
>
>> On Nov 15, 2011 8:53 AM, "Olivier Voyer"<olivier.voyer at gmail.com>  wrote:
>>> Hi everyone,
>>>
>>> Is it possible to have multiple modules linking to one single pyd file? I'm using SWIG with VS2010 and I can't find a way of doing that.
>>>
>> I believe this is not supported by the Python C-API itself, regardless of what wrapper generator or library you use.  The name of the module import function needs to be related to the loadable module file name, and you generally can't have two functions for which that's true.
> But you can put several submodules in an extension module, as a workaround (see code below). I always wondered if that was possible with boost::python or SWIG ?

You can. For example:


void exportModule1() {
     object module(handle<>(borrowed(PyImport_AddModule("mypkg.module1"))));
     scope().attr("module1") = module.
     scope module_scope = module;

    // put your def/class_/etc things here
}

void exportModule2() {
     object module(handle<>(borrowed(PyImport_AddModule("mypkg.module2"))));
     scope().attr("module2") = module.
     scope module_scope = module;

    // put your def/class_/etc things here
}

void export() {
     object package = scope();
     package.attr("__path__") = "mypkg";
     exportModule1();
     exportModule2();
}




From wichert at wiggy.net  Tue Nov 15 16:47:55 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Tue, 15 Nov 2011 16:47:55 +0100
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <CA+_ahhzneSSfxOH6eHv8pdNETO_U2jKNuadc0x+iY-GEKtvwwg@mail.gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
	<4EC28323.4080803@gmail.com>
	<CA+_ahhzneSSfxOH6eHv8pdNETO_U2jKNuadc0x+iY-GEKtvwwg@mail.gmail.com>
Message-ID: <4EC289AB.4010304@wiggy.net>

On 11/15/2011 04:42 PM, Olivier Voyer wrote:
> Yes, that's exactly what I want to do. But, from what I understand, 
> each module or submodule (.py file) must link to its own .pyd file, ie 
> module1.py -> _module1.pyd, module2.py -> _module2.pyd.
>
> What I would really love to have is:
>
> myPackage/
> myPackage/__init__.py
> myPackage/module1.py -> linked to _mySingleAPI.pyd
> myPackage/module2.py -> linked to _mySingleAPI.pyd

Why not make mySingleAPI.pyd a single extension myPackage.pyd module 
that creates myPackage, myPackage.module1, etc. all at once? You don't 
need to have separate files for each module if you do that.

Wichert.


From olivier.voyer at gmail.com  Tue Nov 15 16:52:28 2011
From: olivier.voyer at gmail.com (Olivier Voyer)
Date: Tue, 15 Nov 2011 10:52:28 -0500
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <4EC289AB.4010304@wiggy.net>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
	<4EC28323.4080803@gmail.com>
	<CA+_ahhzneSSfxOH6eHv8pdNETO_U2jKNuadc0x+iY-GEKtvwwg@mail.gmail.com>
	<4EC289AB.4010304@wiggy.net>
Message-ID: <CA+_ahhxM=HTP4ASLq4-Hw_463RDYu0UkStN+Uf3L3B=CE8L-zg@mail.gmail.com>

Bingo! But I'm not sure how to do that... at this moment I'm using SWIG as
a C++ wrapper. Do I only need to modify the __init__py file at the root of
my package?

On Tue, Nov 15, 2011 at 10:47 AM, Wichert Akkerman <wichert at wiggy.net>wrote:

> On 11/15/2011 04:42 PM, Olivier Voyer wrote:
>
>> Yes, that's exactly what I want to do. But, from what I understand, each
>> module or submodule (.py file) must link to its own .pyd file, ie
>> module1.py -> _module1.pyd, module2.py -> _module2.pyd.
>>
>> What I would really love to have is:
>>
>> myPackage/
>> myPackage/__init__.py
>> myPackage/module1.py -> linked to _mySingleAPI.pyd
>> myPackage/module2.py -> linked to _mySingleAPI.pyd
>>
>
> Why not make mySingleAPI.pyd a single extension myPackage.pyd module that
> creates myPackage, myPackage.module1, etc. all at once? You don't need to
> have separate files for each module if you do that.
>
> Wichert.
>
>
> ______________________________**_________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/**mailman/listinfo/cplusplus-sig<http://mail.python.org/mailman/listinfo/cplusplus-sig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111115/7a157a64/attachment.html>

From wichert at wiggy.net  Tue Nov 15 16:54:41 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Tue, 15 Nov 2011 16:54:41 +0100
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <CA+_ahhxM=HTP4ASLq4-Hw_463RDYu0UkStN+Uf3L3B=CE8L-zg@mail.gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
	<4EC28323.4080803@gmail.com>
	<CA+_ahhzneSSfxOH6eHv8pdNETO_U2jKNuadc0x+iY-GEKtvwwg@mail.gmail.com>
	<4EC289AB.4010304@wiggy.net>
	<CA+_ahhxM=HTP4ASLq4-Hw_463RDYu0UkStN+Uf3L3B=CE8L-zg@mail.gmail.com>
Message-ID: <4EC28B41.9080700@wiggy.net>

On 11/15/2011 04:52 PM, Olivier Voyer wrote:
> Bingo! But I'm not sure how to do that... at this moment I'm using 
> SWIG as a C++ wrapper. Do I only need to modify the __init__py file at 
> the root of my package?

Personally I don't have a single .py file for my extensions, I just drop 
in a single myPackage.so file (I don't use windows, but your equivalent 
is probably a .pyd file) which exports all modules.

Wichert.

From fraca7 at free.fr  Tue Nov 15 17:15:33 2011
From: fraca7 at free.fr (=?iso-8859-1?Q?J=E9r=F4me_Laheurte?=)
Date: Tue, 15 Nov 2011 17:15:33 +0100
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <4EC28323.4080803@gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
	<4EC28323.4080803@gmail.com>
Message-ID: <00DB7B92-9BB3-4159-A312-0090598AB2CF@free.fr>


Le 15 nov. 2011 ? 16:20, Jim Bosch a ?crit :

> On 11/15/2011 10:00 AM, Olivier Voyer wrote:
>> Jim, thank you for your answer.
>> 
>> What if I have this big C++ project that I cannot split in multiple
>> smaller projects? I have no choice but to create a big Python module
>> exposing all the functions/classes? What is the common practice?
>> 
> 
> It sounds like you just want to make a Python package - a directory with an __init__.py file that contains all your submodules, which can each be compiled separately.
> 
> Is there some reason you felt you needed to put all the modules in the same pyd file?

To share common structures/classes without having to use a PyCObject. Actually that's rather theoretical, I never actually needed this, just curious.


From fraca7 at free.fr  Tue Nov 15 17:16:56 2011
From: fraca7 at free.fr (=?iso-8859-1?Q?J=E9r=F4me_Laheurte?=)
Date: Tue, 15 Nov 2011 17:16:56 +0100
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <4EC2806A.8050507@wiggy.net>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<54FF6DC2-2DA7-4092-A6BF-EC8A9AB6AE29@free.fr>
	<4EC2806A.8050507@wiggy.net>
Message-ID: <2BF613D1-F776-44A8-A88A-EACE682ACEB4@free.fr>


Le 15 nov. 2011 ? 16:08, Wichert Akkerman a ?crit :

> On 11/15/2011 03:51 PM, J?r?me Laheurte wrote:
>> Le 15 nov. 2011 ? 15:24, Jim Bosch a ?crit :
>> 
>>> On Nov 15, 2011 8:53 AM, "Olivier Voyer"<olivier.voyer at gmail.com>  wrote:
>>>> Hi everyone,
>>>> 
>>>> Is it possible to have multiple modules linking to one single pyd file? I'm using SWIG with VS2010 and I can't find a way of doing that.
>>>> 
>>> I believe this is not supported by the Python C-API itself, regardless of what wrapper generator or library you use.  The name of the module import function needs to be related to the loadable module file name, and you generally can't have two functions for which that's true.
>> But you can put several submodules in an extension module, as a workaround (see code below). I always wondered if that was possible with boost::python or SWIG ?
> 
> You can. For example:
> 
> 
> void exportModule1() {
>    object module(handle<>(borrowed(PyImport_AddModule("mypkg.module1"))));
>    scope().attr("module1") = module.
>    scope module_scope = module;
> 
>   // put your def/class_/etc things here
> }
> 
> void exportModule2() {
>    object module(handle<>(borrowed(PyImport_AddModule("mypkg.module2"))));
>    scope().attr("module2") = module.
>    scope module_scope = module;
> 
>   // put your def/class_/etc things here
> }
> 
> void export() {
>    object package = scope();
>    package.attr("__path__") = "mypkg";
>    exportModule1();
>    exportModule2();
> }

Nice, thanks. I'll keep this around.


From fraca7 at free.fr  Tue Nov 15 17:18:05 2011
From: fraca7 at free.fr (=?iso-8859-1?Q?J=E9r=F4me_Laheurte?=)
Date: Tue, 15 Nov 2011 17:18:05 +0100
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <00DB7B92-9BB3-4159-A312-0090598AB2CF@free.fr>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>
	<CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
	<4EC28323.4080803@gmail.com>
	<00DB7B92-9BB3-4159-A312-0090598AB2CF@free.fr>
Message-ID: <892075A7-517D-4125-89E1-FB7C3921E99A@free.fr>


Le 15 nov. 2011 ? 17:15, J?r?me Laheurte a ?crit :

> 
> Le 15 nov. 2011 ? 16:20, Jim Bosch a ?crit :
> 
>> On 11/15/2011 10:00 AM, Olivier Voyer wrote:
>>> Jim, thank you for your answer.
>>> 
>>> What if I have this big C++ project that I cannot split in multiple
>>> smaller projects? I have no choice but to create a big Python module
>>> exposing all the functions/classes? What is the common practice?
>>> 
>> 
>> It sounds like you just want to make a Python package - a directory with an __init__.py file that contains all your submodules, which can each be compiled separately.
>> 
>> Is there some reason you felt you needed to put all the modules in the same pyd file?
> 
> To share common structures/classes without having to use a PyCObject. Actually that's rather theoretical, I never actually needed this, just curious.

Nevermind, that's a silly reason. Of course I can just use the same classes from different .so files (as long as I rebuild them all when I change a declaration).


From s_sourceforge at nedprod.com  Tue Nov 15 18:54:20 2011
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Tue, 15 Nov 2011 17:54:20 -0000
Subject: [C++-sig] Multiple modules in a single pyd
In-Reply-To: <CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
References: <CA+_ahhxg93A6ZB4VbRAUrXoiU=cW4ktTb3NoW3cos2r94H2ifQ@mail.gmail.com>,
	<CAN+0q25qNYDTm1uFaHj_24FwF9c=bOoH8mgaF1eDn8B5VX7e7w@mail.gmail.com>,
	<CA+_ahhwvBhnKWW8+9O+pA9RmnQopoGb2em8Dpcu6bvdNQTp9NQ@mail.gmail.com>
Message-ID: <4EC2A74C.30764.2DB94EA@s_sourceforge.nedprod.com>

On 15 Nov 2011 at 10:00, Olivier Voyer wrote:

> What if I have this big C++ project that I cannot split in multiple smaller
> projects? I have no choice but to create a big Python module exposing all
> the functions/classes? What is the common practice?

You are aware, I assume, that the python wrappings can live in a 
separate DLL/SO than the thing you are wrapping?

So, you can have a monolithic big C++ project DLL/SO several dozen 
megabytes in size, but with multiple wrapper DLL/SO's wrapping just a 
portion of the APIs provided by the monolithic DLL/SO. Each can be 
loaded, as needed, by the python runtime.

Generally speaking, one wants to try and keep symbol counts low when 
possible. Dynamic linkers have become much better in recent years at 
becoming O(N) with symbol count, but there still a few O(N^2) 
behaviours in there. A very large DLL/SO therefore typically will be 
much slower to load in than many smaller DLL/SOs even if they 
represent the same amount of code. And on POSIX, don't forget to make 
judicious use of "-fvisibility=hidden" and 
"-fvisibility-inlines-hidden" as they can very dramatically reduce 
symbol count for the linker.

HTH,
Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From talljimbo at gmail.com  Wed Nov 16 00:43:21 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 15 Nov 2011 18:43:21 -0500
Subject: [C++-sig] [Boost.Python] How to adjust Python reference counts
 for hybrid objects?
In-Reply-To: <5606173E2DE93342AB70B54621721AE44D17956B@msg10003.rgbnetworks.com>
References: <5606173E2DE93342AB70B54621721AE44D17797E@msg10003.rgbnetworks.com>
	<5606173E2DE93342AB70B54621721AE44D17845A@msg10003.rgbnetworks.com>
	<5606173E2DE93342AB70B54621721AE44D17956B@msg10003.rgbnetworks.com>
Message-ID: <4EC2F919.6090605@gmail.com>

On 11/14/2011 09:54 PM, Per Knudsgaard wrote:
> I have gotten (3) to work, but I am wondering about two things.
> Since I couldn't get the constructor to work, I went for a modifier
> that returns the object.

Sorry no one responded to your questions earlier. I havne't looked too
closely yet, but I thought I'd point out that you have basically written 
a big workaround for the fact that you're using intrusive_ptr instead of 
shared_ptr. Pretty much all of the things you're trying to do work 
out-of-the-box with shared_ptr, and you don't have to worry about the 
reference count yourself at all.

Unfortunately, similar support for intrusive_ptr isn't really present in 
Boost.Python, but that's because intrusive_ptr isn't nearly as flexible.

>
> So, two questions:
>
> *         Is this a legal/recommended way to do a modifier?

Legal, yes.  I think.  If switching to shared_ptr is a possibility, I'd 
consider that the recommended way to deal with the entire problem. 
Without it, I think you're in rather uncharted territory.

> *         If I only do a single INCREF, then the object is destroyed
> while executing the NewObject().share() call.  Why do I need two
> INCREFs?

I'm a bit stumped by this, too, but I haven't looked as closely as I 
might have if you didn't seem to have a satisfactory solution or if I 
didn't think shared_ptr was really the way to go here.  Part of the 
explanation might be the fact that Boost.Python doesn't hold a reference 
to what you get out of get_owner(), so you don't have one either, and 
that might do funny things when the only Python reference is a temporary 
object you're calling the share() method on.  But I still would have 
expected that to be safe.

HTH

Jim

From pknudsgaard at rgbnetworks.com  Wed Nov 16 04:50:20 2011
From: pknudsgaard at rgbnetworks.com (Per Knudsgaard)
Date: Wed, 16 Nov 2011 03:50:20 +0000
Subject: [C++-sig] [Boost.Python] How to adjust Python reference counts
 for hybrid objects?
In-Reply-To: <4EC2F919.6090605@gmail.com>
References: <5606173E2DE93342AB70B54621721AE44D17797E@msg10003.rgbnetworks.com>
	<5606173E2DE93342AB70B54621721AE44D17845A@msg10003.rgbnetworks.com>
	<5606173E2DE93342AB70B54621721AE44D17956B@msg10003.rgbnetworks.com>
	<4EC2F919.6090605@gmail.com>
Message-ID: <5606173E2DE93342AB70B54621721AE44D179EFB@msg10003.rgbnetworks.com>

   Thanks for your answer.

   I generally prefer intrusive pointers for a number of reasons:  They are safer (no problems converting between raw and managed).  They are generally faster (no cache misses because the reference count is embedded in your object).  They don't require extra allocations (saving a few bytes for every object).  None of those are really that significant here, but since I have no libraries or third party objects in my application, I don't encounter the most obvious downside.

   That said, after writing the email last night, I realized that I could make a minor change to PtrBase which eliminates the need for the share function.  It has a slight performance cost (one indirection but it is also in the pure C++ case which bugs me a little) and it made the code much cleaner.  The strange thing is that I no longer need to call the Py_INCREF function twice.  Ah, well.

   I have mostly used SWIG in the past so I am on a learning curve with Boost.Python.  Apart from somewhat cryptic error messages, I like it.  At some point I will start digging in the code to see how the details are working.  Maybe I can make some charts as I go along :)

   -- Per.

-----Original Message-----
From: Jim Bosch [mailto:talljimbo at gmail.com] 
Sent: Tuesday, November 15, 2011 3:43 PM
To: Development of Python/C++ integration
Cc: Per Knudsgaard
Subject: Re: [C++-sig] [Boost.Python] How to adjust Python reference counts for hybrid objects?

On 11/14/2011 09:54 PM, Per Knudsgaard wrote:
> I have gotten (3) to work, but I am wondering about two things.
> Since I couldn't get the constructor to work, I went for a modifier 
> that returns the object.

Sorry no one responded to your questions earlier. I havne't looked too closely yet, but I thought I'd point out that you have basically written a big workaround for the fact that you're using intrusive_ptr instead of shared_ptr. Pretty much all of the things you're trying to do work out-of-the-box with shared_ptr, and you don't have to worry about the reference count yourself at all.

Unfortunately, similar support for intrusive_ptr isn't really present in Boost.Python, but that's because intrusive_ptr isn't nearly as flexible.

>
> So, two questions:
>
> *         Is this a legal/recommended way to do a modifier?

Legal, yes.  I think.  If switching to shared_ptr is a possibility, I'd consider that the recommended way to deal with the entire problem. 
Without it, I think you're in rather uncharted territory.

> *         If I only do a single INCREF, then the object is destroyed
> while executing the NewObject().share() call.  Why do I need two 
> INCREFs?

I'm a bit stumped by this, too, but I haven't looked as closely as I might have if you didn't seem to have a satisfactory solution or if I didn't think shared_ptr was really the way to go here.  Part of the explanation might be the fact that Boost.Python doesn't hold a reference to what you get out of get_owner(), so you don't have one either, and that might do funny things when the only Python reference is a temporary object you're calling the share() method on.  But I still would have expected that to be safe.

HTH

Jim

From super24bitsound at hotmail.com  Mon Nov 21 15:08:57 2011
From: super24bitsound at hotmail.com (Jay Riley)
Date: Mon, 21 Nov 2011 09:08:57 -0500
Subject: [C++-sig] Boost python and boost signal
Message-ID: <BAY158-W40064CAF29DAB86180BA65B4CB0@phx.gbl>





I was wondering if it's possible to hook python functions as slots into boost::signal or boost::signal2 signals? I found an old thread that said it wasn't directly possible, but it was from 2004.  		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111121/17fd3930/attachment.html>

From stefan at seefeld.name  Mon Nov 21 15:25:57 2011
From: stefan at seefeld.name (Stefan Seefeld)
Date: Mon, 21 Nov 2011 09:25:57 -0500
Subject: [C++-sig] Boost python and boost signal
In-Reply-To: <BAY158-W40064CAF29DAB86180BA65B4CB0@phx.gbl>
References: <BAY158-W40064CAF29DAB86180BA65B4CB0@phx.gbl>
Message-ID: <4ECA5F75.2080304@seefeld.name>

On 11/21/2011 09:08 AM, Jay Riley wrote:
> I was wondering if it's possible to hook python functions as slots
> into boost::signal or boost::signal2 signals? I found an old thread
> that said it wasn't directly possible, but it was from 2004.

Given that it's possible to wrap Python functions in ordinary C++
functions / function objects, I don't see why it shouldn't be possible
to use Python functions wherever you'd be able to use C++ functions.
Just be careful with proper Python runtime initialization and thread safety.

    Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From s_sourceforge at nedprod.com  Mon Nov 21 19:06:17 2011
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Mon, 21 Nov 2011 18:06:17 -0000
Subject: [C++-sig] Boost python and boost signal
In-Reply-To: <BAY158-W40064CAF29DAB86180BA65B4CB0@phx.gbl>
References: <BAY158-W40064CAF29DAB86180BA65B4CB0@phx.gbl>
Message-ID: <4ECA9319.15127.21CCABCE@s_sourceforge.nedprod.com>

On 21 Nov 2011 at 9:08, Jay Riley wrote:

> I was wondering if it's possible to hook python functions as slots into
> boost::signal or boost::signal2 signals? I found an old thread that said
> it wasn't directly possible, but it was from 2004.  		 	   		  

Integrating this properly - or rather an enhanced version of it - was 
one of the ideas mooted for the next version of Boost.Python some 
months ago. Some clever metaprogramming can even allow python to work 
with arbitrary function pointers for callbacks :) However I don't 
think it's on the cards for what Jim is currently planning.

Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From stefan at seefeld.name  Mon Nov 21 19:16:04 2011
From: stefan at seefeld.name (Stefan Seefeld)
Date: Mon, 21 Nov 2011 13:16:04 -0500
Subject: [C++-sig] Boost python and boost signal
In-Reply-To: <4ECA9319.15127.21CCABCE@s_sourceforge.nedprod.com>
References: <BAY158-W40064CAF29DAB86180BA65B4CB0@phx.gbl>
	<4ECA9319.15127.21CCABCE@s_sourceforge.nedprod.com>
Message-ID: <4ECA9564.6050404@seefeld.name>

On 11/21/2011 01:06 PM, Niall Douglas wrote:
> On 21 Nov 2011 at 9:08, Jay Riley wrote:
>
>> I was wondering if it's possible to hook python functions as slots into
>> boost::signal or boost::signal2 signals? I found an old thread that said
>> it wasn't directly possible, but it was from 2004.  		 	   		  
> Integrating this properly - or rather an enhanced version of it - was 
> one of the ideas mooted for the next version of Boost.Python some 
> months ago. Some clever metaprogramming can even allow python to work 
> with arbitrary function pointers for callbacks :) However I don't 
> think it's on the cards for what Jim is currently planning.

Out of curiosity: what are the issues preventing this from "Just Work" ?

    Stefan


-- 

      ...ich hab' noch einen Koffer in Berlin...


From s_sourceforge at nedprod.com  Mon Nov 21 23:05:43 2011
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Mon, 21 Nov 2011 22:05:43 -0000
Subject: [C++-sig] Boost python and boost signal
In-Reply-To: <4ECA9564.6050404@seefeld.name>
References: <BAY158-W40064CAF29DAB86180BA65B4CB0@phx.gbl>,
	<4ECA9319.15127.21CCABCE@s_sourceforge.nedprod.com>,
	<4ECA9564.6050404@seefeld.name>
Message-ID: <4ECACB37.2037.22A7E35F@s_sourceforge.nedprod.com>

On 21 Nov 2011 at 13:16, Stefan Seefeld wrote:

> >> I was wondering if it's possible to hook python functions as slots into
> >> boost::signal or boost::signal2 signals? I found an old thread that said
> >> it wasn't directly possible, but it was from 2004.  		 	   		  
> > Integrating this properly - or rather an enhanced version of it - was 
> > one of the ideas mooted for the next version of Boost.Python some 
> > months ago. Some clever metaprogramming can even allow python to work 
> > with arbitrary function pointers for callbacks :) However I don't 
> > think it's on the cards for what Jim is currently planning.
> 
> Out of curiosity: what are the issues preventing this from "Just Work" ?

Well, as a minimum, my code would need to be recoded to use Boost 
rather than my own metaprogramming library. It would then need to 
pass peer review etc.

However, as a wider issue, deep thought would need to be put into how 
exactly BPL traverses between "python" and "not python" and a 
strictly defined API decided upon. Right now, this boundary isn't 
tightly defined e.g. when you iterate through a wrapped container, 
exactly how far outside python is one venturing etc. Defining it 
tightly and not accidentally introducing problems for yourself later 
would be real tough though. Hence the deep thought being necessary.

Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From adam.preble at gmail.com  Thu Nov 24 07:07:04 2011
From: adam.preble at gmail.com (Adam Preble)
Date: Thu, 24 Nov 2011 00:07:04 -0600
Subject: [C++-sig] Boost.Python: How do I handle object destruction of a
	Python-created object?
Message-ID: <CAE-m0_k=8x=is6A8CME81-cLFjGt20EZnuYe=jAuFExTESsn=Q@mail.gmail.com>

I'm using boost 1.47 on Windows Vista, with Visual Studio 2010.

My situation is something like this:

1. I have a base interface in C++ to which I've exposed a wrapper in Python.
2. In the C++ source, I have a class that contains multiple instantiations
of said interface.  We can call it Foo.  Foo has been wrapped too.
3. In Python, I've implemented this interface.
4. I have an instance of the object floating around that I created in
Python, contained within a Foo instantiate in Python.
5. The code goes out of scope, and Foo's destructor starts.  Foo's
destructor will try to delete its implementations of the interface.
6. Upon running into the Python implementation of the interface, we crash.

Actually, with gcc in Linux I don't see anything bad happen at all.  It
isn't until I try this on Windows that all hell breaks loose.  What I've
found with my code is that this isn't Visual Studio 2010's fault.  It's
calling me out on something stupid that gcc/Linux let slide.  I think the
fundamental question is: how I can get the resource freed through Python
correctly?

If I let the Python class implementation go out of scope on its own, I
don't see any problems.  Furthermore, Foo is content on its own.  It's the
merging of the two that brings me problems.  Particularly, the program
crashes when I try to delete the Python-instantiated object.  I'm not
entirely surprised by this, but I have no real good way to try to tackle
this problem.  I don't even have good, crisp source code to show the
problem yet.  The code where this has happened has been running fine on
Linux for awhile so I don't have an isolated case to post.  I can work
towards that if I've confused everybody.

One last note.  I did put some test print statements in the virtual
destructor I had written for the interface.  I do see that code does run,
so I think it ultimately comes down to the memory freeing negotiated
between Python and my C++ runtime.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111124/f7804452/attachment.html>

From s_sourceforge at nedprod.com  Thu Nov 24 14:16:22 2011
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Thu, 24 Nov 2011 13:16:22 -0000
Subject: [C++-sig] Boost.Python: How do I handle object destruction of a
	Python-created object?
In-Reply-To: <CAE-m0_k=8x=is6A8CME81-cLFjGt20EZnuYe=jAuFExTESsn=Q@mail.gmail.com>
References: <CAE-m0_k=8x=is6A8CME81-cLFjGt20EZnuYe=jAuFExTESsn=Q@mail.gmail.com>
Message-ID: <4ECE43A6.22136.30366514@s_sourceforge.nedprod.com>

Shouldn't you be wrapping a managed pointer to your C++ instance? 
That stages the destruction properly by keeping the virtual function 
table around long enough to call a virtualised destructor.

Search the archives of this mailing list. There are lots of examples.

Niall


On 24 Nov 2011 at 0:07, Adam Preble wrote:

> I'm using boost 1.47 on Windows Vista, with Visual Studio 2010.
> 
> My situation is something like this:
> 
> 1. I have a base interface in C++ to which I've exposed a wrapper in Python.
> 2. In the C++ source, I have a class that contains multiple instantiations
> of said interface.  We can call it Foo.  Foo has been wrapped too.
> 3. In Python, I've implemented this interface.
> 4. I have an instance of the object floating around that I created in
> Python, contained within a Foo instantiate in Python.
> 5. The code goes out of scope, and Foo's destructor starts.  Foo's
> destructor will try to delete its implementations of the interface.
> 6. Upon running into the Python implementation of the interface, we crash.
> 
> Actually, with gcc in Linux I don't see anything bad happen at all.  It
> isn't until I try this on Windows that all hell breaks loose.  What I've
> found with my code is that this isn't Visual Studio 2010's fault.  It's
> calling me out on something stupid that gcc/Linux let slide.  I think the
> fundamental question is: how I can get the resource freed through Python
> correctly?
> 
> If I let the Python class implementation go out of scope on its own, I
> don't see any problems.  Furthermore, Foo is content on its own.  It's the
> merging of the two that brings me problems.  Particularly, the program
> crashes when I try to delete the Python-instantiated object.  I'm not
> entirely surprised by this, but I have no real good way to try to tackle
> this problem.  I don't even have good, crisp source code to show the
> problem yet.  The code where this has happened has been running fine on
> Linux for awhile so I don't have an isolated case to post.  I can work
> towards that if I've confused everybody.
> 
> One last note.  I did put some test print statements in the virtual
> destructor I had written for the interface.  I do see that code does run,
> so I think it ultimately comes down to the memory freeing negotiated
> between Python and my C++ runtime.
> 


-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From edxxgardo at gmail.com  Fri Nov 25 12:54:13 2011
From: edxxgardo at gmail.com (Edgardo C.)
Date: Fri, 25 Nov 2011 12:54:13 +0100
Subject: [C++-sig] Problem with importing shared library into Python
Message-ID: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>

Hello everybody,

I am having problems with importing a .so shared library I just compiled
following the example giving in the book:
API Design for C++ : www.apibook.com (Chapter 11)

I downloaded the Boost library and I compiled it (fullw) into a
subdirectory called /compilation, where I get the /include and /lib
directories. No error during the compilation (Ubuntu 11.10).
Then I downloaded the code from the book examples and I compiled it without
errors and I get the phonebook.so in the same directory as where I launch
python interpreter, but the import in python trigger this error:


ImportError: /home/pablo/phonebook/phonebook.so: undefined symbol:
_ZN5boost6python6detail11init_moduleEPKcPFvvE

what I think is because python does not find the library.

I try to play around with the LD_LIBRARY_PATH variable, ldconfig, I copied
the phonebook.so library into /usr/lib and /usr/local/lib, and nothing ...
I get the same error.

Can anybody explain me what is the problem? thanks a lot in advance,

N.B: the version of python used during the boost compilation and .cpp
compilation are the same, 2.7
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111125/9caede4c/attachment.html>

From talljimbo at gmail.com  Fri Nov 25 15:29:06 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Fri, 25 Nov 2011 09:29:06 -0500
Subject: [C++-sig] Problem with importing shared library into Python
In-Reply-To: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>
References: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>
Message-ID: <4ECFA632.9000408@gmail.com>

On 11/25/2011 06:54 AM, Edgardo C. wrote:
> Hello everybody,
>
> I am having problems with importing a .so shared library I just compiled
> following the example giving in the book:
> API Design for C++ : www.apibook.com <http://www.apibook.com> (Chapter 11)
>
> I downloaded the Boost library and I compiled it (fullw) into a
> subdirectory called /compilation, where I get the /include and /lib
> directories. No error during the compilation (Ubuntu 11.10).
> Then I downloaded the code from the book examples and I compiled it
> without errors and I get the phonebook.so in the same directory as where
> I launch python interpreter, but the import in python trigger this error:
>
>
> ImportError: /home/pablo/phonebook/phonebook.so: undefined symbol:
> _ZN5boost6python6detail11init_moduleEPKcPFvvE
>
> what I think is because python does not find the library.
>
> I try to play around with the LD_LIBRARY_PATH variable, ldconfig, I
> copied the phonebook.so library into /usr/lib and /usr/local/lib, and
> nothing ... I get the same error.
>
> Can anybody explain me what is the problem? thanks a lot in advance,
>

I think you just need to link your library against the boost_python 
library, and make sure that's also somewhere on your dynamic library 
path.  With gcc, that means adding a "-lboost_python" to the command 
line options.

Jim


From edxxgardo at gmail.com  Fri Nov 25 15:52:33 2011
From: edxxgardo at gmail.com (Edgardo C.)
Date: Fri, 25 Nov 2011 15:52:33 +0100
Subject: [C++-sig] Problem with importing shared library into Python
In-Reply-To: <4ECFA632.9000408@gmail.com>
References: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>
	<4ECFA632.9000408@gmail.com>
Message-ID: <CAGFAtdC1dA6n=pHk0iSLwexorj6MRf70BvAH-m7_+4BPc1Qr-A@mail.gmail.com>

Hello Jim,

I really appreciate your feedback. I did what you mention, and I did it
downloading manually the boost library, compiling, etc, etc. Then, I
thought, ok, maybe I did something wrong and I downloaded the already
compiled boost packages using the synaptic tool in Ubuntu and then I
compiled the source code against the Ubuntu boost isntalled packages. The
compilation works fine, no errors but I get the same error.

What I do not know is whether the problem is the compilation of the shared
library or python. I spent the whole day on this,
agrrrrrrrrrrrrrrrrrrrrrrrrr

this guy :

http://www.mentby.com/vivek-60/


had a similar problem. But, this command:

$ nm /usr/local/lib/libboost_python3.so.1.46.1 | c++filt | grep init_module

show no symbols for me. Any other idea?



2011/11/25 Jim Bosch <talljimbo at gmail.com>

> On 11/25/2011 06:54 AM, Edgardo C. wrote:
>
>> Hello everybody,
>>
>> I am having problems with importing a .so shared library I just compiled
>> following the example giving in the book:
>> API Design for C++ : www.apibook.com <http://www.apibook.com> (Chapter
>> 11)
>>
>>
>> I downloaded the Boost library and I compiled it (fullw) into a
>> subdirectory called /compilation, where I get the /include and /lib
>> directories. No error during the compilation (Ubuntu 11.10).
>> Then I downloaded the code from the book examples and I compiled it
>> without errors and I get the phonebook.so in the same directory as where
>> I launch python interpreter, but the import in python trigger this error:
>>
>>
>> ImportError: /home/pablo/phonebook/**phonebook.so: undefined symbol:
>> _ZN5boost6python6detail11init_**moduleEPKcPFvvE
>>
>> what I think is because python does not find the library.
>>
>> I try to play around with the LD_LIBRARY_PATH variable, ldconfig, I
>> copied the phonebook.so library into /usr/lib and /usr/local/lib, and
>> nothing ... I get the same error.
>>
>> Can anybody explain me what is the problem? thanks a lot in advance,
>>
>>
> I think you just need to link your library against the boost_python
> library, and make sure that's also somewhere on your dynamic library path.
>  With gcc, that means adding a "-lboost_python" to the command line options.
>
> Jim
>
> ______________________________**_________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/**mailman/listinfo/cplusplus-sig<http://mail.python.org/mailman/listinfo/cplusplus-sig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111125/3ae9717f/attachment.html>

From edxxgardo at gmail.com  Fri Nov 25 16:06:09 2011
From: edxxgardo at gmail.com (Edgardo C.)
Date: Fri, 25 Nov 2011 16:06:09 +0100
Subject: [C++-sig] Problem with importing shared library into Python
In-Reply-To: <4ECFA632.9000408@gmail.com>
References: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>
	<4ECFA632.9000408@gmail.com>
Message-ID: <CAGFAtdBaeuZAf+HutQXeL-RUtRnZd+utCd5jDuXCsTgC84GBTA@mail.gmail.com>

We have this situation(remember that the source code is from the book API
C++ design www.apibook.com):

Source code:

- phonebook.h
- phonebook.cpp
- phonebook_wrap.cpp

Boost is download, compiled and installed into:
- /Desktop/boost_1_48_0/compilation/

which means that /include and /lib are the subfolders of this with all the
headers and libraries into it.

Now:

g++ -c
phonebook.cpp
(compiles only phonebook.cpp, without error)
g++ -c phonebook_wrap.cpp -I<boost_includes> -I<python_include>  (compiles
the wrapper against the /include subfolder mentioned before and the
python2.7 headers in /usr/...)
g++ -shared -o phonebook.so phonebook.o phonebook_wrap.o -lboost_python
-lpython (links the objects agains the /lib subfolder mentioned before and
and python2.7 libraries in /usr/...)

After this I get the phonebook.so shared library, but when I try to import
it in python I get the error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: ./phonebook.so: undefined symbol:
_ZN5boost6python6detail11init_moduleEPKcPFvvE


Hope this can help better to understand (maybe) what is going on. Again, I
follow the same procedure after installing the precompiled packages from
Ubuntu using synaptic, and the same error. Hope that somebody can help.




















2011/11/25 Jim Bosch <talljimbo at gmail.com>

> On 11/25/2011 06:54 AM, Edgardo C. wrote:
>
>> Hello everybody,
>>
>> I am having problems with importing a .so shared library I just compiled
>> following the example giving in the book:
>> API Design for C++ : www.apibook.com <http://www.apibook.com> (Chapter
>> 11)
>>
>>
>> I downloaded the Boost library and I compiled it (fullw) into a
>> subdirectory called /compilation, where I get the /include and /lib
>> directories. No error during the compilation (Ubuntu 11.10).
>> Then I downloaded the code from the book examples and I compiled it
>> without errors and I get the phonebook.so in the same directory as where
>> I launch python interpreter, but the import in python trigger this error:
>>
>>
>> ImportError: /home/pablo/phonebook/**phonebook.so: undefined symbol:
>> _ZN5boost6python6detail11init_**moduleEPKcPFvvE
>>
>> what I think is because python does not find the library.
>>
>> I try to play around with the LD_LIBRARY_PATH variable, ldconfig, I
>> copied the phonebook.so library into /usr/lib and /usr/local/lib, and
>> nothing ... I get the same error.
>>
>> Can anybody explain me what is the problem? thanks a lot in advance,
>>
>>
> I think you just need to link your library against the boost_python
> library, and make sure that's also somewhere on your dynamic library path.
>  With gcc, that means adding a "-lboost_python" to the command line options.
>
> Jim
>
> ______________________________**_________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/**mailman/listinfo/cplusplus-sig<http://mail.python.org/mailman/listinfo/cplusplus-sig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111125/97c86c3d/attachment-0001.html>

From stefan at seefeld.name  Fri Nov 25 16:26:00 2011
From: stefan at seefeld.name (Stefan Seefeld)
Date: Fri, 25 Nov 2011 10:26:00 -0500
Subject: [C++-sig] Problem with importing shared library into Python
In-Reply-To: <CAGFAtdBaeuZAf+HutQXeL-RUtRnZd+utCd5jDuXCsTgC84GBTA@mail.gmail.com>
References: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>
	<4ECFA632.9000408@gmail.com>
	<CAGFAtdBaeuZAf+HutQXeL-RUtRnZd+utCd5jDuXCsTgC84GBTA@mail.gmail.com>
Message-ID: <4ECFB388.9070307@seefeld.name>

On 11/25/2011 10:06 AM, Edgardo C. wrote:
> We have this situation(remember that the source code is from the book
> API C++ design www.apibook.com <http://www.apibook.com>):
>
> Source code:
>
> - phonebook.h
> - phonebook.cpp
> - phonebook_wrap.cpp
>
> Boost is download, compiled and installed into:
> - /Desktop/boost_1_48_0/compilation/
>
> which means that /include and /lib are the subfolders of this with all
> the headers and libraries into it.
>
> Now:
>
> g++ -c
> phonebook.cpp                                                                   
> (compiles only phonebook.cpp, without error)
> g++ -c phonebook_wrap.cpp -I<boost_includes> -I<python_include> 
> (compiles the wrapper against the /include subfolder mentioned before
> and the python2.7 headers in /usr/...)
> g++ -shared -o phonebook.so phonebook.o phonebook_wrap.o
> -lboost_python -lpython (links the objects agains the /lib subfolder
> mentioned before and and python2.7 libraries in /usr/...)

Not sure whether this has anything to do with the symptoms you are
observing, but I believe you should not link to the Python runtime, as
that will be provided by the interpreter. (Some exotic platforms such as
cygwin seem to require to link against -lpython anyhow, but certainly
not GNU/Linux)

FWIW,
        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From anders.e.e.wallin at gmail.com  Fri Nov 25 17:42:29 2011
From: anders.e.e.wallin at gmail.com (Anders Wallin)
Date: Fri, 25 Nov 2011 18:42:29 +0200
Subject: [C++-sig] Problem with importing shared library into Python
In-Reply-To: <CAGFAtdBaeuZAf+HutQXeL-RUtRnZd+utCd5jDuXCsTgC84GBTA@mail.gmail.com>
References: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>
	<4ECFA632.9000408@gmail.com>
	<CAGFAtdBaeuZAf+HutQXeL-RUtRnZd+utCd5jDuXCsTgC84GBTA@mail.gmail.com>
Message-ID: <CAPnVNRXdRiVWjax3Sb3xq0Cf7sM0Mqveuh0Ej_+zxnEkTWm-WA@mail.gmail.com>

> Hope this can help better to understand (maybe) what is going on. Again, I
> follow the same procedure after installing the precompiled packages from
> Ubuntu using synaptic, and the same error. Hope that somebody can help.

FWIW with this CMakeLists.txt the example-code from the book compiles
and runs fine on my Ubuntu 11.10 system:
http://pastebin.com/YrWVEmxX

If I run make with "make VERBOSE=1" it gives the following output
(this might help if you don't want to use cmake...):

/usr/bin/c++   -Dphonebook_EXPORTS -fPIC -I/usr/include/python2.7
-o CMakeFiles/phonebook.dir/phonebook.o -c
/home/anders/Desktop/apibook-apibook-tip/11_Scripting/python/phonebook.cpp

/usr/bin/c++   -Dphonebook_EXPORTS -fPIC -I/usr/include/python2.7
-o CMakeFiles/phonebook.dir/phonebook_wrap.o -c
/home/anders/Desktop/apibook-apibook-tip/11_Scripting/python/phonebook_wrap.cpp

/usr/bin/c++  -fPIC    -shared -Wl,-soname,phonebook.so -o
phonebook.so CMakeFiles/phonebook.dir/phonebook.o
CMakeFiles/phonebook.dir/phonebook_wrap.o -lboost_python


hth,
Anders

From edxxgardo at gmail.com  Fri Nov 25 20:51:06 2011
From: edxxgardo at gmail.com (Edgardo C.)
Date: Fri, 25 Nov 2011 20:51:06 +0100
Subject: [C++-sig] Problem with importing shared library into Python
In-Reply-To: <CAPnVNRXdRiVWjax3Sb3xq0Cf7sM0Mqveuh0Ej_+zxnEkTWm-WA@mail.gmail.com>
References: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>
	<4ECFA632.9000408@gmail.com>
	<CAGFAtdBaeuZAf+HutQXeL-RUtRnZd+utCd5jDuXCsTgC84GBTA@mail.gmail.com>
	<CAPnVNRXdRiVWjax3Sb3xq0Cf7sM0Mqveuh0Ej_+zxnEkTWm-WA@mail.gmail.com>
Message-ID: <CAGFAtdCHK7CuwYG3akgtvnjcE716UNMP4Lh_zVA4_UNDepsZgw@mail.gmail.com>

Hello to all again and I really appreciate the help. Finally it works now
!!!!!!!!!!!!!!!

Unfortunately i happened just 10 minutes before leaving the office and
until Monday I will not touch again the code.
@Anders: I will try what you suggest on Monday.


What I did was:

$ g++ -c phonebook_wrap.cpp -I/usr/include/python2.7
$ g++ -shared  -o phonebook.so phonebook.o phonebook_wrap.o -lboost_python
-L/usr/bin/python.27

Previously I used variables to access the folder (header and libraries),
now I use the folder explicitly. Now I am compiling against the boot
libraries installed using synaptic and against the python executable
located in /usr/bin/python.27.
Honestly I expected to compiled against -lpython, but there is not python
library like "libpython ... .so":

$ g++ -shared  -o phonebook.so phonebook.o phonebook_wrap.o -lboost_python
-lpython
/usr/bin/ld: cannot find -lpython
collect2: ld returned 1 exit status


I compiled against the executable, which is an ELF file and shared library
as well, as noticed after running "file" command and it worked perfectly.
Now I can import the phonebook.so from Python and run all the code. I still
want to review all what I did again because there is something that is not
clear what went wrong. Until Monday I would not be able to touch the code
again, but thanks a lot for all your suggestions, I really appreciate it.
Keep you informed on Monday.

Have a nice week end to you all,
Cheers,








2011/11/25 Anders Wallin <anders.e.e.wallin at gmail.com>

> > Hope this can help better to understand (maybe) what is going on. Again,
> I
> > follow the same procedure after installing the precompiled packages from
> > Ubuntu using synaptic, and the same error. Hope that somebody can help.
>
> FWIW with this CMakeLists.txt the example-code from the book compiles
> and runs fine on my Ubuntu 11.10 system:
> http://pastebin.com/YrWVEmxX
>
> If I run make with "make VERBOSE=1" it gives the following output
> (this might help if you don't want to use cmake...):
>
> /usr/bin/c++   -Dphonebook_EXPORTS -fPIC -I/usr/include/python2.7
> -o CMakeFiles/phonebook.dir/phonebook.o -c
> /home/anders/Desktop/apibook-apibook-tip/11_Scripting/python/phonebook.cpp
>
> /usr/bin/c++   -Dphonebook_EXPORTS -fPIC -I/usr/include/python2.7
> -o CMakeFiles/phonebook.dir/phonebook_wrap.o -c
>
> /home/anders/Desktop/apibook-apibook-tip/11_Scripting/python/phonebook_wrap.cpp
>
> /usr/bin/c++  -fPIC    -shared -Wl,-soname,phonebook.so -o
> phonebook.so CMakeFiles/phonebook.dir/phonebook.o
> CMakeFiles/phonebook.dir/phonebook_wrap.o -lboost_python
>
>
> hth,
> Anders
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20111125/da8b4285/attachment.html>

From stefan at seefeld.name  Fri Nov 25 21:05:35 2011
From: stefan at seefeld.name (Stefan Seefeld)
Date: Fri, 25 Nov 2011 15:05:35 -0500
Subject: [C++-sig] Problem with importing shared library into Python
In-Reply-To: <CAGFAtdCHK7CuwYG3akgtvnjcE716UNMP4Lh_zVA4_UNDepsZgw@mail.gmail.com>
References: <CAGFAtdBO46whP5ZnQuVJW_kLzfe6JZ__h1Aocp_COPVw7Hr0+Q@mail.gmail.com>
	<4ECFA632.9000408@gmail.com>
	<CAGFAtdBaeuZAf+HutQXeL-RUtRnZd+utCd5jDuXCsTgC84GBTA@mail.gmail.com>
	<CAPnVNRXdRiVWjax3Sb3xq0Cf7sM0Mqveuh0Ej_+zxnEkTWm-WA@mail.gmail.com>
	<CAGFAtdCHK7CuwYG3akgtvnjcE716UNMP4Lh_zVA4_UNDepsZgw@mail.gmail.com>
Message-ID: <4ECFF50F.3080605@seefeld.name>

On 11/25/2011 02:51 PM, Edgardo C. wrote:
> Hello to all again and I really appreciate the help. Finally it works
> now !!!!!!!!!!!!!!!

Good !

>
> Unfortunately i happened just 10 minutes before leaving the office and
> until Monday I will not touch again the code. 
> @Anders: I will try what you suggest on Monday.
>
>
> What I did was:
>
> $ g++ -c phonebook_wrap.cpp -I/usr/include/python2.7
> $ g++ -shared  -o phonebook.so phonebook.o phonebook_wrap.o
> -lboost_python -L/usr/bin/python.27

That last argument ("-L/usr/bin/python.27") doesn't make any sense. You
should remove it.

>
> Previously I used variables to access the folder (header and
> libraries), now I use the folder explicitly. Now I am compiling
> against the boot libraries installed using synaptic and against the
> python executable located in /usr/bin/python.27.
> Honestly I expected to compiled against -lpython, but there is not
> python library like "libpython ... .so":

As I said in my last mail, you shouldn't need to link against the Python
runtime itself when building extension modules, as it will already be
loaded by the interpreter.


>
> $ g++ -shared  -o phonebook.so phonebook.o phonebook_wrap.o
> -lboost_python -lpython
> /usr/bin/ld: cannot find -lpython
> collect2: ld returned 1 exit status
>
>
> I compiled against the executable, which is an ELF file and shared
> library as well, as noticed after running "file" command and it worked
> perfectly.

That doesn't make sense. You can't link against an executable. The link
command likely succeeded simply because it already found all symbols it
needed without that. Use `ldd phonebook.so` to see what libraries your
module depends on.


Regards,
        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...