From mikeowen at llnl.gov  Thu Jul  2 20:38:22 2009
From: mikeowen at llnl.gov (J. Michael Owen)
Date: Thu, 2 Jul 2009 11:38:22 -0700
Subject: [C++-sig] pybindgen: allow_subclassing option causing reference
	parameters to be copied?
In-Reply-To: <a467ca4f0906300629y62b03396pf2b8d0c5f30db0c0@mail.gmail.com>
References: <A1EAE3BB-FF1C-4492-AE10-76637EA656C4@llnl.gov>
	<a467ca4f0906241531v663e33ew98c0285c98adf247@mail.gmail.com>
	<C2F723FD-D5E1-48D4-B424-00C392A96F8C@llnl.gov>
	<a467ca4f0906241703s21eabcb7jeac2ce932fa28512@mail.gmail.com>
	<a467ca4f0906280819s638e0d06td0439f304c6323a3@mail.gmail.com>
	<B69AC2CB-26C4-4997-9EAF-1A28AD76A9CC@llnl.gov>
	<a467ca4f0906300629y62b03396pf2b8d0c5f30db0c0@mail.gmail.com>
Message-ID: <765D6C96-5340-4A57-9333-DC53D362C39B@llnl.gov>

I pulled down your bazaar branch and merged it with my own changes (I  
need to use the container methods, inplace numeric operators, and  
numeric operators with built in python  types, so I have to keep using  
my changes for now), and now reference parameters are working just  
fine.  Great!

Since the code I'm binding has probably thousands of reference  
parameters to expose, I've written little helper methods as shorthand  
for the long parameter definition.  This is probably a little  
specialized for what you want, but it might be helpful to include  
something like these (akin to the "param" method you already have in  
utils):

def refparam(cppobj, argname):
     return Parameter.new("%s&" % cppobj, argname,  
direction=Parameter.DIRECTION_INOUT)

def constrefparam(cppobj, argname):
     return Parameter.new(ref("const %s&" % cppobj), argname,  
direction=Parameter.DIRECTION_INOUT)

So on those changes I have for the pybindgen code (allowing the stuff  
I mention above), I'd be happy to modify them to fit in better with  
how you want pybindgen to look if you'd like to give me an idea what  
you're looking for.

On Jun 30, 2009, at 6:29 AM, Gustavo Carneiro wrote:

>
> Ah, right.  The thing is, the code tries to make a copy of the  
> object to give to the python wrapper in case it appears the python  
> code kept a reference to the wrapper.
>
> I need to fix pybindgen to, if the class has no copy constructor  
> keep the wrapper->obj pointer NULL instead of giving it a copy.
>
> The fix has been pushed to the bazaar branch.  See https://  
> code.launchpad.net/~gjc/pybindgen/trunk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090702/9d7dd5c5/attachment.htm>

From kranar at hotmail.com  Thu Jul  2 21:50:06 2009
From: kranar at hotmail.com (Kamal Mansouri)
Date: Thu, 2 Jul 2009 15:50:06 -0400
Subject: [C++-sig] Boost.Python and Boost.Any
Message-ID: <SNT105-W56C4C40F2560CF8EAC4CE7CA2F0@phx.gbl>


Hello,

I am currently working on a way to convert boost::any to/from Python/C++.

So far the idea is that for any C++ type T exported to Python via Boost.Python,
you also register a function that converts an instance of boost::any storing a
value of type T to Python as well.

In other words, if you had a function f and exported it to Python as follows:

// C++ code:
boost::any f() {
  int x = 5;
  boost::any a = x;
  return a;
}

Then in Python it is possible to do this:

x = f()
print x

And it will print "5", the result of f() automatically gets converted into a Python
integer instead of an instance of boost::any storing an integer.

This part I got working, thankfully, my problem is doing the flip side, passing the value
5 from Python to a C++ function that takes a boost::any as a parameter and having
that boost::any store an int.

The problem with the solution I have now is that when I pass an arbitrary Python value
to a function taking a boost::any, the boost::any holds an instance of type PyObject*.

My AnyFromPython converter looks like this:

struct AnyFromPython {
  AnyFromPython() {
    converter::registry::push_back(&convertible, &construct, type_id<any>());
  }

  static void* convertible(PyObject* value) {
    return value;
  }

  static void construct(PyObject* value,
      converter::rvalue_from_python_stage1_data* data) {
    if(value == 0) {
      throw_error_already_set();
    }
    void* storage = ((converter::rvalue_from_python_storage<any>*)
      data)->storage.bytes;
    new (storage) any(value);
    data->convertible = storage;
  }
};


The question is... is it possible to convert the parameter "value" into what it
would have otherwise been converted to in C++, and then take that result and
wrap it with a boost::any, as opposed to what it's doing now, which is just
wrapping a raw PyObject*.

If I could do this, then boost::any would work seamlessly with Python, and users
of my library would never need to know that boost::any is being used behind the
scenes in the C++ world.

Any help would be appreciated,
Kamal

_________________________________________________________________
Attention all humans. We are your photos. Free us.
http://go.microsoft.com/?linkid=9666046
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090702/7f2acee9/attachment.htm>

From paul at gingernut.tv  Fri Jul  3 13:15:10 2009
From: paul at gingernut.tv (Paul Scruby)
Date: Fri, 3 Jul 2009 12:15:10 +0100
Subject: [C++-sig] boost::python and threads
Message-ID: <h2kp7s$m69$1@ger.gmane.org>

I am having some problems using boost::python with boost::thread.  I'm using 
threads because I want to run some tasks in the background when I'm using 
the Python's interactive shell.  However, when I use get_override() to call 
a Python method from  another boost::thread it crashes internally.  For 
example:

    #include <boost/python.hpp>
    #include <boost/thread/thread.hpp>
    #include <boost/thread/xtime.hpp>

    using namespace boost::python;

    class Ticker
        :    public wrapper<Ticker>
    {
    private:
        bool run_;
        volatile bool * running_;
        boost::thread * thread_;
        boost::xtime xt_;
    public:
        Ticker() : running_(&run_) { *running_ = false; }

        void operator()()
        {
            while (*running_)
            {
                boost::xtime_get(&xt_, boost::TIME_UTC);
                ++xt_.sec;
                boost::thread::sleep(xt_);
                onTick();
            }
        }

        void run()
        {
            if (*running_ == false)
            {
                *running_ = true;
                thread_ = new boost::thread(*this);
            }
        }

        void stop()
        {
            if (*running_ == true)
            {
                *running_ = false;
                thread_->join();
                delete thread_;
            }
        }

        virtual void onTick() { get_override("onTick")(); }
        void default_onTick() {}
    };

    BOOST_PYTHON_MODULE(tick)
    {
        class_<Ticker, boost::noncopyable> ("Ticker")
            .def("run", &Ticker::run)
            .def("stop", &Ticker::stop)
            .def("onTick", &Ticker::default_onTick);
    }

Here is a test script that which will crash when you import it into Python's 
interactive shell.

    from tick import Ticker

    class MyTicker(Ticker):
        def onTick(self):
        print "Each second"

    myticker = MyTicker()
    myticker.run()

I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler on 
Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008 on 
Windows XP.

The call-stack in dbx on Solaris:

    >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in 
PyErr_Restore at 0xfef38fa1
    0xfef38fa1: PyErr_Restore+0x0031:       movl     0x00000028(%edi),%ecx
    Current function is boost::python::override::operator()
       99           detail::method_result x(

    (dbx) where
    current thread: t at 2
      [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e, 
0x80652fc), at 0xfef38fa1
      [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
      [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
      [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
      [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at 
0xfef2bf02
      [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
    =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in 
"override.hpp"
      [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
      [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
      [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 
56 in "thread.hpp"
      [11] thread_proxy(0x810a288), at 0xfea78ce4
      [12] _thr_setup(0xfe670200), at 0xfee159b9
      [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e, 
0x80652fc, 0x80f1220), at 0xfee15ca0

The call-stack in Visual Studio 2008:

    python26.dll!1e013595()
    [Frames below may be incorrect and/or missing, no symbols loaded for 
python26.dll]
    python26.dll!1e09ee7d()
  > tick.pyd!boost::python::override::operator()()  Line 103 + 0x16 bytes 
C++
    00f3fd64()
    tick.pyd!Ticker::operator()()  Line 27 + 0xe bytes C++
    tick.pyd!boost::detail::thread_data<Ticker>::run()  Line 57 C++
    tick.pyd!boost::`anonymous namespace'::thread_start_function(void * 
param=0x00245f30)  Line 168 C++
    msvcr90d.dll!_callthreadstartex()  Line 348 + 0xf bytes C
    msvcr90d.dll!_threadstartex(void * ptd=0x00d46938)  Line 331 C
    kernel32.dll!7c80b729()


Have a missed a trick using the wrapper, or does boost::python not support 
threading?

Many thanks,

Paul




From merlin66b at gmail.com  Sat Jul  4 13:19:22 2009
From: merlin66b at gmail.com (Thomas Berg)
Date: Sat, 4 Jul 2009 13:19:22 +0200
Subject: [C++-sig] boost::python and threads
In-Reply-To: <h2kp7s$m69$1@ger.gmane.org>
References: <h2kp7s$m69$1@ger.gmane.org>
Message-ID: <c472dffc0907040419t634e7970n10aecac0a385f7b@mail.gmail.com>

Hi,

On Fri, Jul 3, 2009 at 1:15 PM, Paul Scruby<paul at gingernut.tv> wrote:
> I am having some problems using boost::python with boost::thread. ?I'm using
> threads because I want to run some tasks in the background when I'm using
> the Python's interactive shell. ?However, when I use get_override() to call
> a Python method from ?another boost::thread it crashes internally. ?For
> example:
>

>
> Have a missed a trick using the wrapper, or does boost::python not support
> threading?
>
> Many thanks,
>
> Paul

Short answer: boost::python does not support threading. There is more
information here, and a few interesting links (with partial
solutions): http://www.boost.org/doc/libs/1_39_0/libs/python/todo.html#full-threading-support

Hope this helps,
Thomas

From wladwig at wdtinc.com  Sat Jul  4 20:34:00 2009
From: wladwig at wdtinc.com (William Ladwig)
Date: Sat, 4 Jul 2009 13:34:00 -0500
Subject: [C++-sig] boost::python and threads
In-Reply-To: <h2kp7s$m69$1@ger.gmane.org>
References: <h2kp7s$m69$1@ger.gmane.org>
Message-ID: <765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com>

It looks to me like you have a garbage collection problem going on.  If you create a wrapped c++ object in python, then python is going to own the object and will destroy it when its reference count goes to 0.  In your python example script at the bottom, you call the Ticker's run() function, which from the python point of view, returns quickly and the script ends.  Python has no idea that you spawned off a new thread from the C++ side, so when the script ends, python destroys the object and now you have a problem.  One way that you can check to see if this is what is going on is to add this to the bottom of the test script and see if the crashes go away:

# Warning, you'll need to kill this script manually
import time
while True:
    time.sleep(1)

Generally when I want to fire off a new C++ thread, I hold any objects that the thread needs in an auto_ptr (see held type for class wrappers), take ownership of them in C++ (see the FAQ in the documentation) and start the thread from the C++ side.  Or, you can also create C++ wrappers which accept shared_ptr arguments, while holding your classes in shared_ptrs, and this should handle the reference counting as well.  Unfortunately, this may require some interface changes to what you have already written (or possibly some clever wrapping).  

Hope this helps,
Bill


________________________________________
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul Scruby [paul at gingernut.tv]
Sent: Friday, July 03, 2009 6:15 AM
To: cplusplus-sig at python.org
Subject: [C++-sig] boost::python and threads

I am having some problems using boost::python with boost::thread.  I'm using
threads because I want to run some tasks in the background when I'm using
the Python's interactive shell.  However, when I use get_override() to call
a Python method from  another boost::thread it crashes internally.  For
example:

    #include <boost/python.hpp>
    #include <boost/thread/thread.hpp>
    #include <boost/thread/xtime.hpp>

    using namespace boost::python;

    class Ticker
        :    public wrapper<Ticker>
    {
    private:
        bool run_;
        volatile bool * running_;
        boost::thread * thread_;
        boost::xtime xt_;
    public:
        Ticker() : running_(&run_) { *running_ = false; }

        void operator()()
        {
            while (*running_)
            {
                boost::xtime_get(&xt_, boost::TIME_UTC);
                ++xt_.sec;
                boost::thread::sleep(xt_);
                onTick();
            }
        }

        void run()
        {
            if (*running_ == false)
            {
                *running_ = true;
                thread_ = new boost::thread(*this);
            }
        }

        void stop()
        {
            if (*running_ == true)
            {
                *running_ = false;
                thread_->join();
                delete thread_;
            }
        }

        virtual void onTick() { get_override("onTick")(); }
        void default_onTick() {}
    };

    BOOST_PYTHON_MODULE(tick)
    {
        class_<Ticker, boost::noncopyable> ("Ticker")
            .def("run", &Ticker::run)
            .def("stop", &Ticker::stop)
            .def("onTick", &Ticker::default_onTick);
    }

Here is a test script that which will crash when you import it into Python's
interactive shell.

    from tick import Ticker

    class MyTicker(Ticker):
        def onTick(self):
        print "Each second"

    myticker = MyTicker()
    myticker.run()

I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler on
Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008 on
Windows XP.

The call-stack in dbx on Solaris:

    >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
PyErr_Restore at 0xfef38fa1
    0xfef38fa1: PyErr_Restore+0x0031:       movl     0x00000028(%edi),%ecx
    Current function is boost::python::override::operator()
       99           detail::method_result x(

    (dbx) where
    current thread: t at 2
      [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
0x80652fc), at 0xfef38fa1
      [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
      [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
      [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
      [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
0xfef2bf02
      [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
    =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
"override.hpp"
      [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
      [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
      [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
56 in "thread.hpp"
      [11] thread_proxy(0x810a288), at 0xfea78ce4
      [12] _thr_setup(0xfe670200), at 0xfee159b9
      [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
0x80652fc, 0x80f1220), at 0xfee15ca0

The call-stack in Visual Studio 2008:

    python26.dll!1e013595()
    [Frames below may be incorrect and/or missing, no symbols loaded for
python26.dll]
    python26.dll!1e09ee7d()
  > tick.pyd!boost::python::override::operator()()  Line 103 + 0x16 bytes
C++
    00f3fd64()
    tick.pyd!Ticker::operator()()  Line 27 + 0xe bytes C++
    tick.pyd!boost::detail::thread_data<Ticker>::run()  Line 57 C++
    tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
param=0x00245f30)  Line 168 C++
    msvcr90d.dll!_callthreadstartex()  Line 348 + 0xf bytes C
    msvcr90d.dll!_threadstartex(void * ptd=0x00d46938)  Line 331 C
    kernel32.dll!7c80b729()


Have a missed a trick using the wrapper, or does boost::python not support
threading?

Many thanks,

Paul



_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From wladwig at wdtinc.com  Sat Jul  4 22:03:20 2009
From: wladwig at wdtinc.com (William Ladwig)
Date: Sat, 4 Jul 2009 15:03:20 -0500
Subject: [C++-sig] boost::python and threads
In-Reply-To: <765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com>
References: <h2kp7s$m69$1@ger.gmane.org>,
	<765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com>
Message-ID: <765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com>

Whoops, I think this problem is a little uglier than I thought, since you overrode the onTick() function in python with a call to print, which needs access to the interpreter in your new thread.  See the link Thomas posted for dealing with the GIL (along with worrying about any possible garbage collection issues).  Now I remember why I kept my C++ threads isolated from Python stuff....sorry, it's been a while....

Bill 
________________________________________
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of William Ladwig [wladwig at wdtinc.com]
Sent: Saturday, July 04, 2009 1:34 PM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] boost::python and threads

It looks to me like you have a garbage collection problem going on.  If you create a wrapped c++ object in python, then python is going to own the object and will destroy it when its reference count goes to 0.  In your python example script at the bottom, you call the Ticker's run() function, which from the python point of view, returns quickly and the script ends.  Python has no idea that you spawned off a new thread from the C++ side, so when the script ends, python destroys the object and now you have a problem.  One way that you can check to see if this is what is going on is to add this to the bottom of the test script and see if the crashes go away:

# Warning, you'll need to kill this script manually
import time
while True:
    time.sleep(1)

Generally when I want to fire off a new C++ thread, I hold any objects that the thread needs in an auto_ptr (see held type for class wrappers), take ownership of them in C++ (see the FAQ in the documentation) and start the thread from the C++ side.  Or, you can also create C++ wrappers which accept shared_ptr arguments, while holding your classes in shared_ptrs, and this should handle the reference counting as well.  Unfortunately, this may require some interface changes to what you have already written (or possibly some clever wrapping).

Hope this helps,
Bill


________________________________________
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul Scruby [paul at gingernut.tv]
Sent: Friday, July 03, 2009 6:15 AM
To: cplusplus-sig at python.org
Subject: [C++-sig] boost::python and threads

I am having some problems using boost::python with boost::thread.  I'm using
threads because I want to run some tasks in the background when I'm using
the Python's interactive shell.  However, when I use get_override() to call
a Python method from  another boost::thread it crashes internally.  For
example:

    #include <boost/python.hpp>
    #include <boost/thread/thread.hpp>
    #include <boost/thread/xtime.hpp>

    using namespace boost::python;

    class Ticker
        :    public wrapper<Ticker>
    {
    private:
        bool run_;
        volatile bool * running_;
        boost::thread * thread_;
        boost::xtime xt_;
    public:
        Ticker() : running_(&run_) { *running_ = false; }

        void operator()()
        {
            while (*running_)
            {
                boost::xtime_get(&xt_, boost::TIME_UTC);
                ++xt_.sec;
                boost::thread::sleep(xt_);
                onTick();
            }
        }

        void run()
        {
            if (*running_ == false)
            {
                *running_ = true;
                thread_ = new boost::thread(*this);
            }
        }

        void stop()
        {
            if (*running_ == true)
            {
                *running_ = false;
                thread_->join();
                delete thread_;
            }
        }

        virtual void onTick() { get_override("onTick")(); }
        void default_onTick() {}
    };

    BOOST_PYTHON_MODULE(tick)
    {
        class_<Ticker, boost::noncopyable> ("Ticker")
            .def("run", &Ticker::run)
            .def("stop", &Ticker::stop)
            .def("onTick", &Ticker::default_onTick);
    }

Here is a test script that which will crash when you import it into Python's
interactive shell.

    from tick import Ticker

    class MyTicker(Ticker):
        def onTick(self):
        print "Each second"

    myticker = MyTicker()
    myticker.run()

I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler on
Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008 on
Windows XP.

The call-stack in dbx on Solaris:

    >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
PyErr_Restore at 0xfef38fa1
    0xfef38fa1: PyErr_Restore+0x0031:       movl     0x00000028(%edi),%ecx
    Current function is boost::python::override::operator()
       99           detail::method_result x(

    (dbx) where
    current thread: t at 2
      [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
0x80652fc), at 0xfef38fa1
      [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
      [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
      [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
      [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
0xfef2bf02
      [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
    =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
"override.hpp"
      [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
      [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
      [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
56 in "thread.hpp"
      [11] thread_proxy(0x810a288), at 0xfea78ce4
      [12] _thr_setup(0xfe670200), at 0xfee159b9
      [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
0x80652fc, 0x80f1220), at 0xfee15ca0

The call-stack in Visual Studio 2008:

    python26.dll!1e013595()
    [Frames below may be incorrect and/or missing, no symbols loaded for
python26.dll]
    python26.dll!1e09ee7d()
  > tick.pyd!boost::python::override::operator()()  Line 103 + 0x16 bytes
C++
    00f3fd64()
    tick.pyd!Ticker::operator()()  Line 27 + 0xe bytes C++
    tick.pyd!boost::detail::thread_data<Ticker>::run()  Line 57 C++
    tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
param=0x00245f30)  Line 168 C++
    msvcr90d.dll!_callthreadstartex()  Line 348 + 0xf bytes C
    msvcr90d.dll!_threadstartex(void * ptd=0x00d46938)  Line 331 C
    kernel32.dll!7c80b729()


Have a missed a trick using the wrapper, or does boost::python not support
threading?

Many thanks,

Paul



_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From renatox at gmail.com  Sat Jul  4 22:15:52 2009
From: renatox at gmail.com (Renato Araujo)
Date: Sat, 4 Jul 2009 17:15:52 -0300
Subject: [C++-sig] boost::python and threads
In-Reply-To: <765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com>
References: <h2kp7s$m69$1@ger.gmane.org>
	<765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com>
	<765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com>
Message-ID: <95291a80907041315k41b7ad88o32d2111ae8fe1e91@mail.gmail.com>

Hi Paul

In my bindings I had a problem like this, to solve I created a simple
class like that:

class thread_locker
{
thread_locker()
{
    if (thread_support::enabled())
        m_gstate = PyGILState_Ensure();
}

~thread_locker()
{
    if (thread_support::enabled())
        PyGILState_Release(m_gstate);
}
};

then in my wrapper virtual implementation I did this:

...
void wrapper::virtual_func(..)
{
thread_locker lock;
.. my code ..
}
....

this solve my problems with call of virtual functions in thread enviroment.

BR




On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<wladwig at wdtinc.com> wrote:
> Whoops, I think this problem is a little uglier than I thought, since you overrode the onTick() function in python with a call to print, which needs access to the interpreter in your new thread. ?See the link Thomas posted for dealing with the GIL (along with worrying about any possible garbage collection issues). ?Now I remember why I kept my C++ threads isolated from Python stuff....sorry, it's been a while....
>
> Bill
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of William Ladwig [wladwig at wdtinc.com]
> Sent: Saturday, July 04, 2009 1:34 PM
> To: Development of Python/C++ integration
> Subject: Re: [C++-sig] boost::python and threads
>
> It looks to me like you have a garbage collection problem going on. ?If you create a wrapped c++ object in python, then python is going to own the object and will destroy it when its reference count goes to 0. ?In your python example script at the bottom, you call the Ticker's run() function, which from the python point of view, returns quickly and the script ends. ?Python has no idea that you spawned off a new thread from the C++ side, so when the script ends, python destroys the object and now you have a problem. ?One way that you can check to see if this is what is going on is to add this to the bottom of the test script and see if the crashes go away:
>
> # Warning, you'll need to kill this script manually
> import time
> while True:
> ? ?time.sleep(1)
>
> Generally when I want to fire off a new C++ thread, I hold any objects that the thread needs in an auto_ptr (see held type for class wrappers), take ownership of them in C++ (see the FAQ in the documentation) and start the thread from the C++ side. ?Or, you can also create C++ wrappers which accept shared_ptr arguments, while holding your classes in shared_ptrs, and this should handle the reference counting as well. ?Unfortunately, this may require some interface changes to what you have already written (or possibly some clever wrapping).
>
> Hope this helps,
> Bill
>
>
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul Scruby [paul at gingernut.tv]
> Sent: Friday, July 03, 2009 6:15 AM
> To: cplusplus-sig at python.org
> Subject: [C++-sig] boost::python and threads
>
> I am having some problems using boost::python with boost::thread. ?I'm using
> threads because I want to run some tasks in the background when I'm using
> the Python's interactive shell. ?However, when I use get_override() to call
> a Python method from ?another boost::thread it crashes internally. ?For
> example:
>
> ? ?#include <boost/python.hpp>
> ? ?#include <boost/thread/thread.hpp>
> ? ?#include <boost/thread/xtime.hpp>
>
> ? ?using namespace boost::python;
>
> ? ?class Ticker
> ? ? ? ?: ? ?public wrapper<Ticker>
> ? ?{
> ? ?private:
> ? ? ? ?bool run_;
> ? ? ? ?volatile bool * running_;
> ? ? ? ?boost::thread * thread_;
> ? ? ? ?boost::xtime xt_;
> ? ?public:
> ? ? ? ?Ticker() : running_(&run_) { *running_ = false; }
>
> ? ? ? ?void operator()()
> ? ? ? ?{
> ? ? ? ? ? ?while (*running_)
> ? ? ? ? ? ?{
> ? ? ? ? ? ? ? ?boost::xtime_get(&xt_, boost::TIME_UTC);
> ? ? ? ? ? ? ? ?++xt_.sec;
> ? ? ? ? ? ? ? ?boost::thread::sleep(xt_);
> ? ? ? ? ? ? ? ?onTick();
> ? ? ? ? ? ?}
> ? ? ? ?}
>
> ? ? ? ?void run()
> ? ? ? ?{
> ? ? ? ? ? ?if (*running_ == false)
> ? ? ? ? ? ?{
> ? ? ? ? ? ? ? ?*running_ = true;
> ? ? ? ? ? ? ? ?thread_ = new boost::thread(*this);
> ? ? ? ? ? ?}
> ? ? ? ?}
>
> ? ? ? ?void stop()
> ? ? ? ?{
> ? ? ? ? ? ?if (*running_ == true)
> ? ? ? ? ? ?{
> ? ? ? ? ? ? ? ?*running_ = false;
> ? ? ? ? ? ? ? ?thread_->join();
> ? ? ? ? ? ? ? ?delete thread_;
> ? ? ? ? ? ?}
> ? ? ? ?}
>
> ? ? ? ?virtual void onTick() { get_override("onTick")(); }
> ? ? ? ?void default_onTick() {}
> ? ?};
>
> ? ?BOOST_PYTHON_MODULE(tick)
> ? ?{
> ? ? ? ?class_<Ticker, boost::noncopyable> ("Ticker")
> ? ? ? ? ? ?.def("run", &Ticker::run)
> ? ? ? ? ? ?.def("stop", &Ticker::stop)
> ? ? ? ? ? ?.def("onTick", &Ticker::default_onTick);
> ? ?}
>
> Here is a test script that which will crash when you import it into Python's
> interactive shell.
>
> ? ?from tick import Ticker
>
> ? ?class MyTicker(Ticker):
> ? ? ? ?def onTick(self):
> ? ? ? ?print "Each second"
>
> ? ?myticker = MyTicker()
> ? ?myticker.run()
>
> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler on
> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008 on
> Windows XP.
>
> The call-stack in dbx on Solaris:
>
> ? ?>>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
> PyErr_Restore at 0xfef38fa1
> ? ?0xfef38fa1: PyErr_Restore+0x0031: ? ? ? movl ? ? 0x00000028(%edi),%ecx
> ? ?Current function is boost::python::override::operator()
> ? ? ? 99 ? ? ? ? ? detail::method_result x(
>
> ? ?(dbx) where
> ? ?current thread: t at 2
> ? ? ?[1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
> 0x80652fc), at 0xfef38fa1
> ? ? ?[2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
> ? ? ?[3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
> ? ? ?[4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
> ? ? ?[5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
> 0xfef2bf02
> ? ? ?[6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
> ? ?=>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
> "override.hpp"
> ? ? ?[8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
> ? ? ?[9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
> ? ? ?[10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
> 56 in "thread.hpp"
> ? ? ?[11] thread_proxy(0x810a288), at 0xfea78ce4
> ? ? ?[12] _thr_setup(0xfe670200), at 0xfee159b9
> ? ? ?[13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
> 0x80652fc, 0x80f1220), at 0xfee15ca0
>
> The call-stack in Visual Studio 2008:
>
> ? ?python26.dll!1e013595()
> ? ?[Frames below may be incorrect and/or missing, no symbols loaded for
> python26.dll]
> ? ?python26.dll!1e09ee7d()
> ?> tick.pyd!boost::python::override::operator()() ?Line 103 + 0x16 bytes
> C++
> ? ?00f3fd64()
> ? ?tick.pyd!Ticker::operator()() ?Line 27 + 0xe bytes C++
> ? ?tick.pyd!boost::detail::thread_data<Ticker>::run() ?Line 57 C++
> ? ?tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
> param=0x00245f30) ?Line 168 C++
> ? ?msvcr90d.dll!_callthreadstartex() ?Line 348 + 0xf bytes C
> ? ?msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) ?Line 331 C
> ? ?kernel32.dll!7c80b729()
>
>
> Have a missed a trick using the wrapper, or does boost::python not support
> threading?
>
> Many thanks,
>
> Paul
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho

From alexey.akimov85 at gmail.com  Sun Jul  5 01:50:34 2009
From: alexey.akimov85 at gmail.com (Alexey Akimov)
Date: Sat, 4 Jul 2009 18:50:34 -0500
Subject: [C++-sig] how to extract an instance of arbitrary c++ class from
	python object?
Message-ID: <382d227f0907041650w6c5ed691red40d7f05fe02996@mail.gmail.com>

Hi,
Here is a question.(see Subject title) I am particularly interested in such
implementation using python.boost library.
For simple cases I've seen examples, such as:

int x = extract<int>(obj.attr("Attribute_name"))

However, when a type T in extract<T>() becomes user-specified things become
harder. For example,
suppose I have a type Vector wich is something like:

class Vector{
public:
double x,y,z;
};

and want to extract information from python object obj to c++ obect of class
Vector IN ONE OPERATION.
I've found that there are some converters for some standard types, but I'm
not able to manage how to do this
for my class. What should I do in order to make the extract<> function to
understand how it should convert a python
object (represented as boost::python::object - something like this) into
Vector object?
Again, variant like:
Vector r;
r.x = extract<double>(obj.attr("x"))
... and so on
are not interested. I need to make it in one operation.
Thanks

Alexey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090704/090e877f/attachment.htm>

From renatox at gmail.com  Sun Jul  5 02:27:23 2009
From: renatox at gmail.com (Renato Araujo)
Date: Sat, 4 Jul 2009 21:27:23 -0300
Subject: [C++-sig] how to extract an instance of arbitrary c++ class
	from python object?
In-Reply-To: <382d227f0907041650w6c5ed691red40d7f05fe02996@mail.gmail.com>
References: <382d227f0907041650w6c5ed691red40d7f05fe02996@mail.gmail.com>
Message-ID: <95291a80907041727q5c274ddavf40de93577189412@mail.gmail.com>

Hi Alexey,

you can use this example to see how create converter rules in
boost:pyhton: http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/faq.html#custom_string

or you can expose you class to pyhon using boost:python in normal way.
http://www.boost.org/doc/libs/1_39_0/libs/python/doc/tutorial/doc/html/python/exposing.html

BR

On Sat, Jul 4, 2009 at 8:50 PM, Alexey Akimov<alexey.akimov85 at gmail.com> wrote:
> Hi,
> Here is a question.(see Subject title) I am particularly interested in such
> implementation using python.boost library.
> For simple cases I've seen examples, such as:
>
> int x = extract<int>(obj.attr("Attribute_name"))
>
> However, when a type T in extract<T>() becomes user-specified things become
> harder. For example,
> suppose I have a type Vector wich is something like:
>
> class Vector{
> public:
> double x,y,z;
> };
>
> and want to extract information from python object obj to c++ obect of class
> Vector IN ONE OPERATION.
> I've found that there are some converters for some standard types, but I'm
> not able to manage how to do this
> for my class. What should I do in order to make the extract<> function to
> understand how it should convert a python
> object (represented as boost::python::object - something like this) into
> Vector object?
> Again, variant like:
> Vector r;
> r.x = extract<double>(obj.attr("x"))
> ... and so on
> are not interested. I need to make it in one operation.
> Thanks
>
> Alexey
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho

From divinekid at gmail.com  Sun Jul  5 16:34:36 2009
From: divinekid at gmail.com (Haoyu Bai)
Date: Sun, 5 Jul 2009 22:34:36 +0800
Subject: [C++-sig] Boost.Python py3k support mid-term status report
Message-ID: <1d7983e80907050734v27cc0a8em3bdc696c6b71f260@mail.gmail.com>

Hi,

There is a list of milestones we have reached for the GSoC py3k support project:

1. Made Boost.Python compiled successfully with Python 3
2. Almost all the test cases passed, except some test cases need to be
clean up to compatible with Python 3.
3. Basic unicode support by using Python API.
4. The bjam based BPL test suite runner will automatically call 2to3
to convert the source code of test cases to Python 3 code when needed.

I also have found some Python bugs when working on this project
(mainly 2to3 and document), and submitted reports to Python issue
tracker, and they have been fixed quickly. I also gained basic
knowledge about Bjam, so the next hacking on Boost.Build would be
easier.

I'm currently working on the test suite clean up. doctests are heavily
used in Boost.Python test suite, and 2to3 is not clever enough to deal
with all of that. (For example, the output part of doctest will be
untouched by 2to3, so if the expected output is "42L", then it will
reamain 42L after 2to3.)

The next step will be focus on the building system. The concept of
Bjam is complex and much different from the build systems I'm familiar
with. So I'd expect to put more effort on it.

Now we only have basic unicode support. The PyUnicode and C++ string
conversion is simply done by the PyUnicode_AsUTF8String and
PyUnicode_FromStringAndSize API. Converting Unicode to const char* is
a bit more complex - it is done by an API _PyUnicode_AsString, which
is an API marked only for Python internal use. When asked why this API
is not public, Benjamin Peterson answered "because implicitly encoding
unicode is bad practice in Python 3" [1]. However, use this API is
almost the only way to keep the Boost.Python PyUniocde -> const char*
converting semantic unchanged. I'll look into the Boost Unicode
project to see if we can make use of this library.

Stefan suggested me to look into the Py_Finalize issue. I did a little
experiments but didn't see the problem occurred. Can anyone help to
provide an example that produce the issue or point out details about
this issue? Thanks!

The boost SVN branch of this projec is located at the following address:

https://svn.boost.org/svn/boost/sandbox-branches/bhy/py3k/

If you are interesting, please checkout it, review it and  try it.
I'll happily provide help if you want to migrate your extension module
to py3k.

Thank you!

Reference:
[1] http://bugs.python.org/issue6223

-- Haoyu Bai

From sipickles at googlemail.com  Sun Jul  5 17:55:33 2009
From: sipickles at googlemail.com (Simon Pickles)
Date: Sun, 05 Jul 2009 16:55:33 +0100
Subject: [C++-sig] boost::python::make_tuple crashing
Message-ID: <4A50CCF5.9070103@googlemail.com>

Hi,

I can't figure why the test program is crashing. I am comparing speeds 
of boost::python::tuple and boost::any as variable length application 
message wrappers.
////////////////////////////////////////////////
#include "boost//any.hpp"
#include "boost//python.hpp"
#include <string>
#include <vector>
// RakNet::GetTime for accurate millisecond timimg
#include "GetTime.h"
#include <stdio.h>
#include <iostream>
using namespace std;

void HandleBoostAny( const vector<boost::any>& ba)
{
    const string s = boost::any_cast<string>(ba[0]);
    const float f = boost::any_cast<float>(ba[1]);
    const int ui = boost::any_cast<int>(ba[2]);
}
void HandleBoostPythonTuple( const boost::python::tuple& b)
{
    const string s = boost::python::extract<std::string>(b[0]);
    const float f = boost::python::extract<float>(b[0]);
    const int ui = boost::python::extract<int>(b[2]);
}
int _tmain(int argc, _TCHAR* argv[])
{
    int count = 0;
    int target = 100000;
    string s = "spam";
    unsigned char i = 42;
    float f = 3.14f;

    //cout << "Starting boost.any test" << endl;
    //RakNetTime baStart = RakNet::GetTime();
    //for ( count = 0; count < target; ++count )
    //{
    //    vector<boost::any> ba;
    //    ba.push_back(boost::any(s));
    //    ba.push_back(boost::any(f));
    //    ba.push_back(boost::any(i));
    //    HandleBoostAny(ba);
    //}
    //RakNetTime baEnd = RakNet::GetTime();
    //cout << "Duration: " << (baEnd - baStart) << endl;
    ////////////////////////
    cout << "Starting boost.python.tuple test" << endl;
    RakNetTime bptStart = RakNet::GetTime();
    for ( count = 0; count < target; ++count )
    {
        boost::python::tuple bpt = boost::python::make_tuple(s,f);

        HandleBoostPythonTuple(boost::python::make_tuple(f));
    }
    RakNetTime bptEnd = RakNet::GetTime();
    cout << "Duration: " << (bptEnd - bptStart) << endl;
    return 0;
}
////////////////////////////////////////////////////

I am getting an access violation here, on WinXP SP3, boost 1.38:

     python25.dll!1e07de69()    
     [Frames below may be incorrect and/or missing, no symbols loaded 
for python25.dll]   
 >    
speedTest.exe!boost::python::converter::arg_to_python<int>::arg_to_python<int>(const 
int & x=0)  Line 113 + 0x31 bytes    C++
     
speedTest.exe!boost::python::api::object_initializer_impl<0,0>::get<int>(const 
int & x=0, boost::mpl::bool_<0> __formal={...})  Line 374 + 0xf bytes    C++
     
speedTest.exe!boost::python::api::object_base_initializer<int>(const int 
& x=0)  Line 296 + 0x10 bytes    C++
     speedTest.exe!boost::python::api::object::object<int>(const int & 
x=0)  Line 316 + 0x2c bytes    C++
     
speedTest.exe!boost::python::api::object_operators<boost::python::api::object>::operator[]<int>(const 
int & key=0)  Line 53 + 0xf bytes    C++
     speedTest.exe!HandleBoostPythonTuple(const boost::python::tuple & 
b={...})  Line 21 + 0x27 bytes    C++
     speedTest.exe!main()  Line 59 + 0x35 bytes    C++
     speedTest.exe!__tmainCRTStartup()  Line 586 + 0x19 bytes    C
     speedTest.exe!mainCRTStartup()  Line 403    C
     kernel32.dll!7c817077()    

I can use make tuple in other programs, but cant see why not here.

Thanks for any advice

Simon

From sipickles at googlemail.com  Sun Jul  5 18:27:11 2009
From: sipickles at googlemail.com (Simon Pickles)
Date: Sun, 05 Jul 2009 17:27:11 +0100
Subject: [C++-sig] boost::python::make_tuple crashing
Message-ID: <4A50D45F.9000404@googlemail.com>

Sorry, had a typo or two:

#include "boost//any.hpp"
#include "boost//python.hpp"
#include <string>
#include <vector>
// RakNet::GetTime for accurate millisecond timimg
#include "GetTime.h"
#include <stdio.h>
#include <iostream>

using namespace std;

void HandleBoostPythonTuple( const boost::python::tuple& b)
{
    const string s = boost::python::extract<std::string>(b[0]);
    const float f = boost::python::extract<float>(b[1]);
    const int ui = boost::python::extract<int>(b[2]);
}

int main()
{
    int count = 0;
    int target = 100000;

    string s = "spam";
    unsigned char i = 42;
    float f = 3.14f;

    cout << "Starting boost.python.tuple test" << endl;
    RakNetTime bptStart = RakNet::GetTime();
    for ( count = 0; count < target; ++count )
    {
        HandleBoostPythonTuple(boost::python::make_tuple(s,f,i));
    }
    RakNetTime bptEnd = RakNet::GetTime();
    cout << "Duration: " << (bptEnd - bptStart) << endl;
    return 0;
}


From merlin66b at gmail.com  Sun Jul  5 18:39:53 2009
From: merlin66b at gmail.com (Thomas Berg)
Date: Sun, 5 Jul 2009 18:39:53 +0200
Subject: [C++-sig] boost::python::make_tuple crashing
In-Reply-To: <4A50D45F.9000404@googlemail.com>
References: <4A50D45F.9000404@googlemail.com>
Message-ID: <c472dffc0907050939k1f9f000an810ce6c9ee2fc424@mail.gmail.com>

On Sun, Jul 5, 2009 at 6:27 PM, Simon Pickles<sipickles at googlemail.com> wrote:
> Sorry, had a typo or two:
>
> int main()
> {
> ? int count = 0;
> ? int target = 100000;
>
> ? string s = "spam";
> ? unsigned char i = 42;
> ? float f = 3.14f;
[snip]
> ? cout << "Starting boost.python.tuple test" << endl;
> ? RakNetTime bptStart = RakNet::GetTime();
> ? for ( count = 0; count < target; ++count )
> ? {
> ? ? ? HandleBoostPythonTuple(boost::python::make_tuple(s,f,i));
> ? }
> ? RakNetTime bptEnd = RakNet::GetTime();
> ? cout << "Duration: " << (bptEnd - bptStart) << endl;
> ? return 0;
> }


You seem to miss Py_Initialize() in your example at least. The code
then runs on my machine (VS2005, boost 1.37)

Cheers,
Thomas

From paul at gingernut.tv  Mon Jul  6 11:36:04 2009
From: paul at gingernut.tv (Paul Scruby)
Date: Mon, 6 Jul 2009 10:36:04 +0100
Subject: [C++-sig] boost::python and threads
References: <h2kp7s$m69$1@ger.gmane.org><765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com><765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com>
	<95291a80907041315k41b7ad88o32d2111ae8fe1e91@mail.gmail.com>
Message-ID: <h2sgic$ad4$1@ger.gmane.org>

Hiya,

That's fantastic, all I needed to do was to put PyGILState_Ensure(); before 
my virtual function calls into python from another thread and my program no 
longer crashes.  Problem solved, isn't boost::python great!

Many thanks,

Paul


"Renato Araujo" <renatox at gmail.com> wrote in message 
news:95291a80907041315k41b7ad88o32d2111ae8fe1e91 at mail.gmail.com...
Hi Paul

In my bindings I had a problem like this, to solve I created a simple
class like that:

class thread_locker
{
thread_locker()
{
    if (thread_support::enabled())
        m_gstate = PyGILState_Ensure();
}

~thread_locker()
{
    if (thread_support::enabled())
        PyGILState_Release(m_gstate);
}
};

then in my wrapper virtual implementation I did this:

...
void wrapper::virtual_func(..)
{
thread_locker lock;
.. my code ..
}
....

this solve my problems with call of virtual functions in thread enviroment.

BR




On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<wladwig at wdtinc.com> wrote:
> Whoops, I think this problem is a little uglier than I thought, since you 
> overrode the onTick() function in python with a call to print, which needs 
> access to the interpreter in your new thread. See the link Thomas posted 
> for dealing with the GIL (along with worrying about any possible garbage 
> collection issues). Now I remember why I kept my C++ threads isolated from 
> Python stuff....sorry, it's been a while....
>
> Bill
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org 
> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of William 
> Ladwig [wladwig at wdtinc.com]
> Sent: Saturday, July 04, 2009 1:34 PM
> To: Development of Python/C++ integration
> Subject: Re: [C++-sig] boost::python and threads
>
> It looks to me like you have a garbage collection problem going on. If you 
> create a wrapped c++ object in python, then python is going to own the 
> object and will destroy it when its reference count goes to 0. In your 
> python example script at the bottom, you call the Ticker's run() function, 
> which from the python point of view, returns quickly and the script ends. 
> Python has no idea that you spawned off a new thread from the C++ side, so 
> when the script ends, python destroys the object and now you have a 
> problem. One way that you can check to see if this is what is going on is 
> to add this to the bottom of the test script and see if the crashes go 
> away:
>
> # Warning, you'll need to kill this script manually
> import time
> while True:
> time.sleep(1)
>
> Generally when I want to fire off a new C++ thread, I hold any objects 
> that the thread needs in an auto_ptr (see held type for class wrappers), 
> take ownership of them in C++ (see the FAQ in the documentation) and start 
> the thread from the C++ side. Or, you can also create C++ wrappers which 
> accept shared_ptr arguments, while holding your classes in shared_ptrs, 
> and this should handle the reference counting as well. Unfortunately, this 
> may require some interface changes to what you have already written (or 
> possibly some clever wrapping).
>
> Hope this helps,
> Bill
>
>
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org 
> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul 
> Scruby [paul at gingernut.tv]
> Sent: Friday, July 03, 2009 6:15 AM
> To: cplusplus-sig at python.org
> Subject: [C++-sig] boost::python and threads
>
> I am having some problems using boost::python with boost::thread. I'm 
> using
> threads because I want to run some tasks in the background when I'm using
> the Python's interactive shell. However, when I use get_override() to call
> a Python method from another boost::thread it crashes internally. For
> example:
>
> #include <boost/python.hpp>
> #include <boost/thread/thread.hpp>
> #include <boost/thread/xtime.hpp>
>
> using namespace boost::python;
>
> class Ticker
> : public wrapper<Ticker>
> {
> private:
> bool run_;
> volatile bool * running_;
> boost::thread * thread_;
> boost::xtime xt_;
> public:
> Ticker() : running_(&run_) { *running_ = false; }
>
> void operator()()
> {
> while (*running_)
> {
> boost::xtime_get(&xt_, boost::TIME_UTC);
> ++xt_.sec;
> boost::thread::sleep(xt_);
> onTick();
> }
> }
>
> void run()
> {
> if (*running_ == false)
> {
> *running_ = true;
> thread_ = new boost::thread(*this);
> }
> }
>
> void stop()
> {
> if (*running_ == true)
> {
> *running_ = false;
> thread_->join();
> delete thread_;
> }
> }
>
> virtual void onTick() { get_override("onTick")(); }
> void default_onTick() {}
> };
>
> BOOST_PYTHON_MODULE(tick)
> {
> class_<Ticker, boost::noncopyable> ("Ticker")
> .def("run", &Ticker::run)
> .def("stop", &Ticker::stop)
> .def("onTick", &Ticker::default_onTick);
> }
>
> Here is a test script that which will crash when you import it into 
> Python's
> interactive shell.
>
> from tick import Ticker
>
> class MyTicker(Ticker):
> def onTick(self):
> print "Each second"
>
> myticker = MyTicker()
> myticker.run()
>
> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler on
> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008 on
> Windows XP.
>
> The call-stack in dbx on Solaris:
>
> >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
> PyErr_Restore at 0xfef38fa1
> 0xfef38fa1: PyErr_Restore+0x0031: movl 0x00000028(%edi),%ecx
> Current function is boost::python::override::operator()
> 99 detail::method_result x(
>
> (dbx) where
> current thread: t at 2
> [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
> 0x80652fc), at 0xfef38fa1
> [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
> [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
> [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
> [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
> 0xfef2bf02
> [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
> =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
> "override.hpp"
> [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
> [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
> [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
> 56 in "thread.hpp"
> [11] thread_proxy(0x810a288), at 0xfea78ce4
> [12] _thr_setup(0xfe670200), at 0xfee159b9
> [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
> 0x80652fc, 0x80f1220), at 0xfee15ca0
>
> The call-stack in Visual Studio 2008:
>
> python26.dll!1e013595()
> [Frames below may be incorrect and/or missing, no symbols loaded for
> python26.dll]
> python26.dll!1e09ee7d()
> > tick.pyd!boost::python::override::operator()() Line 103 + 0x16 bytes
> C++
> 00f3fd64()
> tick.pyd!Ticker::operator()() Line 27 + 0xe bytes C++
> tick.pyd!boost::detail::thread_data<Ticker>::run() Line 57 C++
> tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
> param=0x00245f30) Line 168 C++
> msvcr90d.dll!_callthreadstartex() Line 348 + 0xf bytes C
> msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) Line 331 C
> kernel32.dll!7c80b729()
>
>
> Have a missed a trick using the wrapper, or does boost::python not support
> threading?
>
> Many thanks,
>
> Paul
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho 




From paul at gingernut.tv  Mon Jul  6 12:39:39 2009
From: paul at gingernut.tv (Paul Scruby)
Date: Mon, 6 Jul 2009 11:39:39 +0100
Subject: [C++-sig] boost::python and threads
References: <h2kp7s$m69$1@ger.gmane.org><765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com><765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com><95291a80907041315k41b7ad88o32d2111ae8fe1e91@mail.gmail.com>
	<h2sgic$ad4$1@ger.gmane.org>
Message-ID: <h2ska4$l0m$1@ger.gmane.org>

Hello again,

Sorry, I spoke too soon.  The good news is that wrapping my virtual method 
calls into Python between PyGILState_Ensure() and PyGILState_Release() fixed 
the crash under Python2.6.2 on Windows, but not when I tested it again under 
Python 2.4.4 on Solaris.  Under Python 2.4.4 on SolarisSolaris it now calls 
and exits the virtual method in Python sucessfully, but then crashes when 
C++ trys to release the global interpretor lock.

The call-stack with the Sun C++ 5.9 compiler under Solaris is
    t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid 
at 0xfec453ed
    0xfec453ed: sem_invalid+0x0013: movzwl   0x00000006(%eax),%eax
    Current function is Ticker::operator()
    28                           PyGILState_Release(state_);
    (dbx) where
    current thread: t at 2
    [1] sem_invalid(0x0), at 0xfec453ed
    [2] _sem_post(0x0), at 0xfec454c4
    [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0, 
0xfe77ef38, 0xfef441b5), at 0xfef492dc
    [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec, 
0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
    [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1, 
0x0, 0x80a3140), at 0xfef43eba
    [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
  =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
    [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56 
in "thread.hpp"
    [9] thread_proxy(0x810a288), at 0xfea78ce4
    [10] _thr_setup(0xfe670200), at 0xfee159b9
    [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0, 
0xfe77ef14), at 0xfee15ca0

Do you think it's worth repeating this test using gcc/linux, or do you think 
that this is just a limitation of using Python with threads?

Thanks again,

Paul


t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid at 
0xfec453ed
0xfec453ed: sem_invalid+0x0013: movzwl   0x00000006(%eax),%eax
Current function is Ticker::operator()
   28                           PyGILState_Release(state_);
(dbx) where
current thread: t at 2
  [1] sem_invalid(0x0), at 0xfec453ed
  [2] _sem_post(0x0), at 0xfec454c4
  [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0, 
0xfe77ef38, 0xfef441b5), at 0xfef492dc
  [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec, 
0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
  [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1, 
0x0, 0x80a3140), at 0xfef43eba
  [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
=>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
  [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56 in 
"thread.hpp"
  [9] thread_proxy(0x810a288), at 0xfea78ce4
  [10] _thr_setup(0xfe670200), at 0xfee159b9
  [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0, 
0xfe77ef14), at 0xfee15ca0



"Paul Scruby" <paul at gingernut.tv> wrote in message 
news:h2sgic$ad4$1 at ger.gmane.org...
> Hiya,
>
> That's fantastic, all I needed to do was to put PyGILState_Ensure(); 
> before my virtual function calls into python from another thread and my 
> program no longer crashes.  Problem solved, isn't boost::python great!
>
> Many thanks,
>
> Paul
>
>
> "Renato Araujo" <renatox at gmail.com> wrote in message 
> news:95291a80907041315k41b7ad88o32d2111ae8fe1e91 at mail.gmail.com...
> Hi Paul
>
> In my bindings I had a problem like this, to solve I created a simple
> class like that:
>
> class thread_locker
> {
> thread_locker()
> {
>    if (thread_support::enabled())
>        m_gstate = PyGILState_Ensure();
> }
>
> ~thread_locker()
> {
>    if (thread_support::enabled())
>        PyGILState_Release(m_gstate);
> }
> };
>
> then in my wrapper virtual implementation I did this:
>
> ...
> void wrapper::virtual_func(..)
> {
> thread_locker lock;
> .. my code ..
> }
> ....
>
> this solve my problems with call of virtual functions in thread 
> enviroment.
>
> BR
>
>
>
>
> On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<wladwig at wdtinc.com> wrote:
>> Whoops, I think this problem is a little uglier than I thought, since you 
>> overrode the onTick() function in python with a call to print, which 
>> needs access to the interpreter in your new thread. See the link Thomas 
>> posted for dealing with the GIL (along with worrying about any possible 
>> garbage collection issues). Now I remember why I kept my C++ threads 
>> isolated from Python stuff....sorry, it's been a while....
>>
>> Bill
>> ________________________________________
>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org 
>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of 
>> William Ladwig [wladwig at wdtinc.com]
>> Sent: Saturday, July 04, 2009 1:34 PM
>> To: Development of Python/C++ integration
>> Subject: Re: [C++-sig] boost::python and threads
>>
>> It looks to me like you have a garbage collection problem going on. If 
>> you create a wrapped c++ object in python, then python is going to own 
>> the object and will destroy it when its reference count goes to 0. In 
>> your python example script at the bottom, you call the Ticker's run() 
>> function, which from the python point of view, returns quickly and the 
>> script ends. Python has no idea that you spawned off a new thread from 
>> the C++ side, so when the script ends, python destroys the object and now 
>> you have a problem. One way that you can check to see if this is what is 
>> going on is to add this to the bottom of the test script and see if the 
>> crashes go away:
>>
>> # Warning, you'll need to kill this script manually
>> import time
>> while True:
>> time.sleep(1)
>>
>> Generally when I want to fire off a new C++ thread, I hold any objects 
>> that the thread needs in an auto_ptr (see held type for class wrappers), 
>> take ownership of them in C++ (see the FAQ in the documentation) and 
>> start the thread from the C++ side. Or, you can also create C++ wrappers 
>> which accept shared_ptr arguments, while holding your classes in 
>> shared_ptrs, and this should handle the reference counting as well. 
>> Unfortunately, this may require some interface changes to what you have 
>> already written (or possibly some clever wrapping).
>>
>> Hope this helps,
>> Bill
>>
>>
>> ________________________________________
>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org 
>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul 
>> Scruby [paul at gingernut.tv]
>> Sent: Friday, July 03, 2009 6:15 AM
>> To: cplusplus-sig at python.org
>> Subject: [C++-sig] boost::python and threads
>>
>> I am having some problems using boost::python with boost::thread. I'm 
>> using
>> threads because I want to run some tasks in the background when I'm using
>> the Python's interactive shell. However, when I use get_override() to 
>> call
>> a Python method from another boost::thread it crashes internally. For
>> example:
>>
>> #include <boost/python.hpp>
>> #include <boost/thread/thread.hpp>
>> #include <boost/thread/xtime.hpp>
>>
>> using namespace boost::python;
>>
>> class Ticker
>> : public wrapper<Ticker>
>> {
>> private:
>> bool run_;
>> volatile bool * running_;
>> boost::thread * thread_;
>> boost::xtime xt_;
>> public:
>> Ticker() : running_(&run_) { *running_ = false; }
>>
>> void operator()()
>> {
>> while (*running_)
>> {
>> boost::xtime_get(&xt_, boost::TIME_UTC);
>> ++xt_.sec;
>> boost::thread::sleep(xt_);
>> onTick();
>> }
>> }
>>
>> void run()
>> {
>> if (*running_ == false)
>> {
>> *running_ = true;
>> thread_ = new boost::thread(*this);
>> }
>> }
>>
>> void stop()
>> {
>> if (*running_ == true)
>> {
>> *running_ = false;
>> thread_->join();
>> delete thread_;
>> }
>> }
>>
>> virtual void onTick() { get_override("onTick")(); }
>> void default_onTick() {}
>> };
>>
>> BOOST_PYTHON_MODULE(tick)
>> {
>> class_<Ticker, boost::noncopyable> ("Ticker")
>> .def("run", &Ticker::run)
>> .def("stop", &Ticker::stop)
>> .def("onTick", &Ticker::default_onTick);
>> }
>>
>> Here is a test script that which will crash when you import it into 
>> Python's
>> interactive shell.
>>
>> from tick import Ticker
>>
>> class MyTicker(Ticker):
>> def onTick(self):
>> print "Each second"
>>
>> myticker = MyTicker()
>> myticker.run()
>>
>> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler 
>> on
>> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008 
>> on
>> Windows XP.
>>
>> The call-stack in dbx on Solaris:
>>
>> >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
>> PyErr_Restore at 0xfef38fa1
>> 0xfef38fa1: PyErr_Restore+0x0031: movl 0x00000028(%edi),%ecx
>> Current function is boost::python::override::operator()
>> 99 detail::method_result x(
>>
>> (dbx) where
>> current thread: t at 2
>> [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
>> 0x80652fc), at 0xfef38fa1
>> [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
>> [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
>> [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
>> [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
>> 0xfef2bf02
>> [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
>> =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
>> "override.hpp"
>> [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
>> [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
>> [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
>> 56 in "thread.hpp"
>> [11] thread_proxy(0x810a288), at 0xfea78ce4
>> [12] _thr_setup(0xfe670200), at 0xfee159b9
>> [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
>> 0x80652fc, 0x80f1220), at 0xfee15ca0
>>
>> The call-stack in Visual Studio 2008:
>>
>> python26.dll!1e013595()
>> [Frames below may be incorrect and/or missing, no symbols loaded for
>> python26.dll]
>> python26.dll!1e09ee7d()
>> > tick.pyd!boost::python::override::operator()() Line 103 + 0x16 bytes
>> C++
>> 00f3fd64()
>> tick.pyd!Ticker::operator()() Line 27 + 0xe bytes C++
>> tick.pyd!boost::detail::thread_data<Ticker>::run() Line 57 C++
>> tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
>> param=0x00245f30) Line 168 C++
>> msvcr90d.dll!_callthreadstartex() Line 348 + 0xf bytes C
>> msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) Line 331 C
>> kernel32.dll!7c80b729()
>>
>>
>> Have a missed a trick using the wrapper, or does boost::python not 
>> support
>> threading?
>>
>> Many thanks,
>>
>> Paul
>>
>>
>>
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>
>
>
>
> -- 
> Renato Araujo Oliveira Filho 




From renatox at gmail.com  Mon Jul  6 15:19:47 2009
From: renatox at gmail.com (Renato Araujo)
Date: Mon, 6 Jul 2009 10:19:47 -0300
Subject: [C++-sig] boost::python and threads
In-Reply-To: <h2ska4$l0m$1@ger.gmane.org>
References: <h2kp7s$m69$1@ger.gmane.org>
	<765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com>
	<765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com>
	<95291a80907041315k41b7ad88o32d2111ae8fe1e91@mail.gmail.com>
	<h2sgic$ad4$1@ger.gmane.org> <h2ska4$l0m$1@ger.gmane.org>
Message-ID: <95291a80907060619u5bff0dcey56947d1ac848cc8e@mail.gmail.com>

I'm using gcc/linux and python >= 2.4 and works fine for me.



On Mon, Jul 6, 2009 at 7:39 AM, Paul Scruby<paul at gingernut.tv> wrote:
> Hello again,
>
> Sorry, I spoke too soon. ?The good news is that wrapping my virtual method
> calls into Python between PyGILState_Ensure() and PyGILState_Release() fixed
> the crash under Python2.6.2 on Windows, but not when I tested it again under
> Python 2.4.4 on Solaris. ?Under Python 2.4.4 on SolarisSolaris it now calls
> and exits the virtual method in Python sucessfully, but then crashes when
> C++ trys to release the global interpretor lock.
>
> The call-stack with the Sun C++ 5.9 compiler under Solaris is
> ? ?t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid
> at 0xfec453ed
> ? ?0xfec453ed: sem_invalid+0x0013: movzwl ? 0x00000006(%eax),%eax
> ? ?Current function is Ticker::operator()
> ? ?28 ? ? ? ? ? ? ? ? ? ? ? ? ? PyGILState_Release(state_);
> ? ?(dbx) where
> ? ?current thread: t at 2
> ? ?[1] sem_invalid(0x0), at 0xfec453ed
> ? ?[2] _sem_post(0x0), at 0xfec454c4
> ? ?[3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
> ? ?[4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
> ? ?[5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
> 0x0, 0x80a3140), at 0xfef43eba
> ? ?[6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
> ?=>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
> ? ?[8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56
> in "thread.hpp"
> ? ?[9] thread_proxy(0x810a288), at 0xfea78ce4
> ? ?[10] _thr_setup(0xfe670200), at 0xfee159b9
> ? ?[11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
> 0xfe77ef14), at 0xfee15ca0
>
> Do you think it's worth repeating this test using gcc/linux, or do you think
> that this is just a limitation of using Python with threads?
>
> Thanks again,
>
> Paul
>
>
> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid at
> 0xfec453ed
> 0xfec453ed: sem_invalid+0x0013: movzwl ? 0x00000006(%eax),%eax
> Current function is Ticker::operator()
> ? 28 ? ? ? ? ? ? ? ? ? ? ? ? ? PyGILState_Release(state_);
> (dbx) where
> current thread: t at 2
> ?[1] sem_invalid(0x0), at 0xfec453ed
> ?[2] _sem_post(0x0), at 0xfec454c4
> ?[3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
> ?[4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
> ?[5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
> 0x0, 0x80a3140), at 0xfef43eba
> ?[6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
> ?[8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56 in
> "thread.hpp"
> ?[9] thread_proxy(0x810a288), at 0xfea78ce4
> ?[10] _thr_setup(0xfe670200), at 0xfee159b9
> ?[11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
> 0xfe77ef14), at 0xfee15ca0
>
>
>
> "Paul Scruby" <paul at gingernut.tv> wrote in message
> news:h2sgic$ad4$1 at ger.gmane.org...
>> Hiya,
>>
>> That's fantastic, all I needed to do was to put PyGILState_Ensure();
>> before my virtual function calls into python from another thread and my
>> program no longer crashes. ?Problem solved, isn't boost::python great!
>>
>> Many thanks,
>>
>> Paul
>>
>>
>> "Renato Araujo" <renatox at gmail.com> wrote in message
>> news:95291a80907041315k41b7ad88o32d2111ae8fe1e91 at mail.gmail.com...
>> Hi Paul
>>
>> In my bindings I had a problem like this, to solve I created a simple
>> class like that:
>>
>> class thread_locker
>> {
>> thread_locker()
>> {
>> ? ?if (thread_support::enabled())
>> ? ? ? ?m_gstate = PyGILState_Ensure();
>> }
>>
>> ~thread_locker()
>> {
>> ? ?if (thread_support::enabled())
>> ? ? ? ?PyGILState_Release(m_gstate);
>> }
>> };
>>
>> then in my wrapper virtual implementation I did this:
>>
>> ...
>> void wrapper::virtual_func(..)
>> {
>> thread_locker lock;
>> .. my code ..
>> }
>> ....
>>
>> this solve my problems with call of virtual functions in thread
>> enviroment.
>>
>> BR
>>
>>
>>
>>
>> On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<wladwig at wdtinc.com> wrote:
>>> Whoops, I think this problem is a little uglier than I thought, since you
>>> overrode the onTick() function in python with a call to print, which
>>> needs access to the interpreter in your new thread. See the link Thomas
>>> posted for dealing with the GIL (along with worrying about any possible
>>> garbage collection issues). Now I remember why I kept my C++ threads
>>> isolated from Python stuff....sorry, it's been a while....
>>>
>>> Bill
>>> ________________________________________
>>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org
>>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of
>>> William Ladwig [wladwig at wdtinc.com]
>>> Sent: Saturday, July 04, 2009 1:34 PM
>>> To: Development of Python/C++ integration
>>> Subject: Re: [C++-sig] boost::python and threads
>>>
>>> It looks to me like you have a garbage collection problem going on. If
>>> you create a wrapped c++ object in python, then python is going to own
>>> the object and will destroy it when its reference count goes to 0. In
>>> your python example script at the bottom, you call the Ticker's run()
>>> function, which from the python point of view, returns quickly and the
>>> script ends. Python has no idea that you spawned off a new thread from
>>> the C++ side, so when the script ends, python destroys the object and now
>>> you have a problem. One way that you can check to see if this is what is
>>> going on is to add this to the bottom of the test script and see if the
>>> crashes go away:
>>>
>>> # Warning, you'll need to kill this script manually
>>> import time
>>> while True:
>>> time.sleep(1)
>>>
>>> Generally when I want to fire off a new C++ thread, I hold any objects
>>> that the thread needs in an auto_ptr (see held type for class wrappers),
>>> take ownership of them in C++ (see the FAQ in the documentation) and
>>> start the thread from the C++ side. Or, you can also create C++ wrappers
>>> which accept shared_ptr arguments, while holding your classes in
>>> shared_ptrs, and this should handle the reference counting as well.
>>> Unfortunately, this may require some interface changes to what you have
>>> already written (or possibly some clever wrapping).
>>>
>>> Hope this helps,
>>> Bill
>>>
>>>
>>> ________________________________________
>>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org
>>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul
>>> Scruby [paul at gingernut.tv]
>>> Sent: Friday, July 03, 2009 6:15 AM
>>> To: cplusplus-sig at python.org
>>> Subject: [C++-sig] boost::python and threads
>>>
>>> I am having some problems using boost::python with boost::thread. I'm
>>> using
>>> threads because I want to run some tasks in the background when I'm using
>>> the Python's interactive shell. However, when I use get_override() to
>>> call
>>> a Python method from another boost::thread it crashes internally. For
>>> example:
>>>
>>> #include <boost/python.hpp>
>>> #include <boost/thread/thread.hpp>
>>> #include <boost/thread/xtime.hpp>
>>>
>>> using namespace boost::python;
>>>
>>> class Ticker
>>> : public wrapper<Ticker>
>>> {
>>> private:
>>> bool run_;
>>> volatile bool * running_;
>>> boost::thread * thread_;
>>> boost::xtime xt_;
>>> public:
>>> Ticker() : running_(&run_) { *running_ = false; }
>>>
>>> void operator()()
>>> {
>>> while (*running_)
>>> {
>>> boost::xtime_get(&xt_, boost::TIME_UTC);
>>> ++xt_.sec;
>>> boost::thread::sleep(xt_);
>>> onTick();
>>> }
>>> }
>>>
>>> void run()
>>> {
>>> if (*running_ == false)
>>> {
>>> *running_ = true;
>>> thread_ = new boost::thread(*this);
>>> }
>>> }
>>>
>>> void stop()
>>> {
>>> if (*running_ == true)
>>> {
>>> *running_ = false;
>>> thread_->join();
>>> delete thread_;
>>> }
>>> }
>>>
>>> virtual void onTick() { get_override("onTick")(); }
>>> void default_onTick() {}
>>> };
>>>
>>> BOOST_PYTHON_MODULE(tick)
>>> {
>>> class_<Ticker, boost::noncopyable> ("Ticker")
>>> .def("run", &Ticker::run)
>>> .def("stop", &Ticker::stop)
>>> .def("onTick", &Ticker::default_onTick);
>>> }
>>>
>>> Here is a test script that which will crash when you import it into
>>> Python's
>>> interactive shell.
>>>
>>> from tick import Ticker
>>>
>>> class MyTicker(Ticker):
>>> def onTick(self):
>>> print "Each second"
>>>
>>> myticker = MyTicker()
>>> myticker.run()
>>>
>>> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler
>>> on
>>> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008
>>> on
>>> Windows XP.
>>>
>>> The call-stack in dbx on Solaris:
>>>
>>> >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
>>> PyErr_Restore at 0xfef38fa1
>>> 0xfef38fa1: PyErr_Restore+0x0031: movl 0x00000028(%edi),%ecx
>>> Current function is boost::python::override::operator()
>>> 99 detail::method_result x(
>>>
>>> (dbx) where
>>> current thread: t at 2
>>> [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
>>> 0x80652fc), at 0xfef38fa1
>>> [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
>>> [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
>>> [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
>>> [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
>>> 0xfef2bf02
>>> [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
>>> =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
>>> "override.hpp"
>>> [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
>>> [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
>>> [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
>>> 56 in "thread.hpp"
>>> [11] thread_proxy(0x810a288), at 0xfea78ce4
>>> [12] _thr_setup(0xfe670200), at 0xfee159b9
>>> [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
>>> 0x80652fc, 0x80f1220), at 0xfee15ca0
>>>
>>> The call-stack in Visual Studio 2008:
>>>
>>> python26.dll!1e013595()
>>> [Frames below may be incorrect and/or missing, no symbols loaded for
>>> python26.dll]
>>> python26.dll!1e09ee7d()
>>> > tick.pyd!boost::python::override::operator()() Line 103 + 0x16 bytes
>>> C++
>>> 00f3fd64()
>>> tick.pyd!Ticker::operator()() Line 27 + 0xe bytes C++
>>> tick.pyd!boost::detail::thread_data<Ticker>::run() Line 57 C++
>>> tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
>>> param=0x00245f30) Line 168 C++
>>> msvcr90d.dll!_callthreadstartex() Line 348 + 0xf bytes C
>>> msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) Line 331 C
>>> kernel32.dll!7c80b729()
>>>
>>>
>>> Have a missed a trick using the wrapper, or does boost::python not
>>> support
>>> threading?
>>>
>>> Many thanks,
>>>
>>> Paul
>>>
>>>
>>>
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>
>>
>>
>>
>> --
>> Renato Araujo Oliveira Filho
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho

From alexey.akimov85 at gmail.com  Tue Jul  7 01:07:54 2009
From: alexey.akimov85 at gmail.com (Alexey Akimov)
Date: Mon, 6 Jul 2009 18:07:54 -0500
Subject: [C++-sig] exposing pointer to Python
Message-ID: <382d227f0907061607x68a00129y9136a21b8432fd1@mail.gmail.com>

I have some class X that contains a variable which is a pointer to some
other class OtherClass (for simplicity we may imagine it can be some
standard c++ type - int, double, etc.):

class X{

public:
     OtherClass* variable;

};

my question is: how can I expose it to python such that user may access to
variable using [] operator?
like:

>>> import module
>>> module.X[i] = i

I've found several examples on this topic:
http://mail.python.org/pipermail/cplusplus-sig/2005-January/008189.html
http://osdir.com/ml/python.c++/2003-01/msg00042.html

but still can not make this trick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090706/a8847f35/attachment.htm>

From nicolaslara at gmail.com  Tue Jul  7 01:25:49 2009
From: nicolaslara at gmail.com (Nicolas Lara)
Date: Mon, 6 Jul 2009 23:25:49 +0000
Subject: [C++-sig] Building with Scons
Message-ID: <46ef4bec0907061625w7bd5f8e5ye81dc79b6aab0104@mail.gmail.com>

Hello,

I am trying to build a python extension with scons but the resulting
library cannot be loaded:
   ImportError: dynamic module does not define init function (initcylucene)

I am using the following SConstruct file:

FILES = ['typeconversion.cpp', 'document.cpp', 'search.cpp', 'queryparser.cpp',
         'analysis.cpp', 'index.cpp', 'store.cpp', 'util.cpp']
def python_tool(env):
    pybase = 'python%s' % sys.version[0:3]
    env.Append(CPPPATH=[os.path.join(sys.prefix, 'include', pybase)],
               LIBPATH=[os.path.join(sys.prefix, 'lib', pybase, 'config')],
               LIBS=['lib%s' % pybase])
    if env['PLATFORM'] not in ['cygwin', 'win32']:
        env.Append(LIBS=['util'])

def boost_python_tool(env):
    env.Append(CPPDEFINES=['BOOST_PYTHON_DYNAMIC_LIB',
                           'BOOST_PYTHON_DYNAMIC_MODULE'],
               CPPPATH=['$boostIncludes'],  # boostIncludes is a PathOption
               LIBS=['boost_python'])

def clucene_tool(env):
    env.Append(CPPPATH=['/usr/local/lib/CLucene/',
                        '/usr/local/lib/'],
               LIBS=['clucene'])

import os
env = Environment(ENV=os.environ, tools=['default', python_tool,
                                         boost_python_tool, clucene_tool])
env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')


Previously I was using the following Makefile, which worked but was
statically linked:

SHELL = /bin/bash
SRC = ./src/
BUILD = ./build/
INCLUDES = -I/usr/include/python2.6/ -I/usr/local/lib/ -I/usr/local/lib/CLucene/
LIBS = -lboost_python -lpython2.6 -lclucene
CFLAGS = -shared -fPIC -g -pedantic -Wall -Wextra
OBJECTS = document.o search.o queryparser.o analysis.o index.o store.o util.o
LOBJECTS = $(BUILD)typeconversion.o $(BUILD)document.o
$(BUILD)search.o $(BUILD)queryparser.o $(BUILD)analysis.o
$(BUILD)index.o $(BUILD)store.o $(BUILD)util.o

all: cylucene

cylucene: $(OBJECTS)
	mkdir -p $(BUILD) && cp $(SRC)initfile.py $(BUILD)__init__.py
	$(CC) $(SRC)cylucene.cpp $(LOBJECTS) $(INCLUDES) $(LIBS) $(CFLAGS) -o
$(BUILD)cylucene.so

document.o: typeconversion.o
	mkdir -p $(BUILD)
	$(CC) $(SRC)document.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)document.o

queryparser.o: typeconversion.o
	mkdir -p $(BUILD)
	$(CC) $(SRC)queryparser.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o
$(BUILD)queryparser.o

analysis.o:
	mkdir -p $(BUILD)
	$(CC) $(SRC)analysis.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)analysis.o

search.o:
	mkdir -p $(BUILD)
	$(CC) $(SRC)search.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)search.o

index.o:
	mkdir -p $(BUILD)
	$(CC) $(SRC)index.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)index.o

store.o:
	mkdir -p $(BUILD)
	$(CC) $(SRC)store.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)store.o

util.o:
	mkdir -p $(BUILD)
	$(CC) $(SRC)util.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)util.o

typeconversion.o:
	mkdir -p $(BUILD)
	$(CC) $(SRC)typeconversion.cpp $(CFLAGS) -o $(BUILD)typeconversion.o

clean:
	rm -Rf build/


Does anyone have experience working with scons and boost::python? Can
anyone help?

Thanks in advance!


-- 
Nicolas Lara
Linux user #380134
http://nicolas-lara.blogspot.com/
Public key id: 0x152e7713 at http://subkeys.pgp.net/

From renatox at gmail.com  Tue Jul  7 04:00:18 2009
From: renatox at gmail.com (Renato Araujo)
Date: Mon, 6 Jul 2009 23:00:18 -0300
Subject: [C++-sig] exposing pointer to Python
In-Reply-To: <382d227f0907061607x68a00129y9136a21b8432fd1@mail.gmail.com>
References: <382d227f0907061607x68a00129y9136a21b8432fd1@mail.gmail.com>
Message-ID: <95291a80907061900g6248daa4g7d9a02e0db22e605@mail.gmail.com>

take a look at:
http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/indexing.html

BR


On Mon, Jul 6, 2009 at 8:07 PM, Alexey Akimov<alexey.akimov85 at gmail.com> wrote:
> I have some class X that contains a variable which is a pointer to some
> other class OtherClass (for simplicity we may imagine it can be some
> standard c++ type - int, double, etc.):
>
> class X{
>
> public:
> ???? OtherClass* variable;
>
> };
>
> my question is: how can I expose it to python such that user may access to
> variable using [] operator?
> like:
>
>>>> import module
>>>> module.X[i] = i
>
> I've found several examples on this topic:
> http://mail.python.org/pipermail/cplusplus-sig/2005-January/008189.html
> http://osdir.com/ml/python.c++/2003-01/msg00042.html
>
> but still can not make this trick.
>
>
>
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho

From wladwig at wdtinc.com  Tue Jul  7 04:53:12 2009
From: wladwig at wdtinc.com (William Ladwig)
Date: Mon, 6 Jul 2009 21:53:12 -0500
Subject: [C++-sig] Building with Scons
In-Reply-To: <46ef4bec0907061625w7bd5f8e5ye81dc79b6aab0104@mail.gmail.com>
References: <46ef4bec0907061625w7bd5f8e5ye81dc79b6aab0104@mail.gmail.com>
Message-ID: <765CBD9053EA2B438625895F30D1856F023806D4D6@storm.wdtinc.com>

Does the name of the module defined in your BOOST_PYTHON_MODULE section match the name of the .so file (assuming you are using Linux)?  That's usually the error I get when I have a name mismatch.  

Also, I haven't really used scons, but shouldn't this:

env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')

be:

env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX=''") ?


Bill


________________________________________
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Nicolas Lara [nicolaslara at gmail.com]
Sent: Monday, July 06, 2009 6:25 PM
To: Development of Python/C++ integration
Subject: [C++-sig] Building with Scons

Hello,

I am trying to build a python extension with scons but the resulting
library cannot be loaded:
   ImportError: dynamic module does not define init function (initcylucene)

I am using the following SConstruct file:

FILES = ['typeconversion.cpp', 'document.cpp', 'search.cpp', 'queryparser.cpp',
         'analysis.cpp', 'index.cpp', 'store.cpp', 'util.cpp']
def python_tool(env):
    pybase = 'python%s' % sys.version[0:3]
    env.Append(CPPPATH=[os.path.join(sys.prefix, 'include', pybase)],
               LIBPATH=[os.path.join(sys.prefix, 'lib', pybase, 'config')],
               LIBS=['lib%s' % pybase])
    if env['PLATFORM'] not in ['cygwin', 'win32']:
        env.Append(LIBS=['util'])

def boost_python_tool(env):
    env.Append(CPPDEFINES=['BOOST_PYTHON_DYNAMIC_LIB',
                           'BOOST_PYTHON_DYNAMIC_MODULE'],
               CPPPATH=['$boostIncludes'],  # boostIncludes is a PathOption
               LIBS=['boost_python'])

def clucene_tool(env):
    env.Append(CPPPATH=['/usr/local/lib/CLucene/',
                        '/usr/local/lib/'],
               LIBS=['clucene'])

import os
env = Environment(ENV=os.environ, tools=['default', python_tool,
                                         boost_python_tool, clucene_tool])
env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')


Previously I was using the following Makefile, which worked but was
statically linked:

SHELL = /bin/bash
SRC = ./src/
BUILD = ./build/
INCLUDES = -I/usr/include/python2.6/ -I/usr/local/lib/ -I/usr/local/lib/CLucene/
LIBS = -lboost_python -lpython2.6 -lclucene
CFLAGS = -shared -fPIC -g -pedantic -Wall -Wextra
OBJECTS = document.o search.o queryparser.o analysis.o index.o store.o util.o
LOBJECTS = $(BUILD)typeconversion.o $(BUILD)document.o
$(BUILD)search.o $(BUILD)queryparser.o $(BUILD)analysis.o
$(BUILD)index.o $(BUILD)store.o $(BUILD)util.o

all: cylucene

cylucene: $(OBJECTS)
        mkdir -p $(BUILD) && cp $(SRC)initfile.py $(BUILD)__init__.py
        $(CC) $(SRC)cylucene.cpp $(LOBJECTS) $(INCLUDES) $(LIBS) $(CFLAGS) -o
$(BUILD)cylucene.so

document.o: typeconversion.o
        mkdir -p $(BUILD)
        $(CC) $(SRC)document.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)document.o

queryparser.o: typeconversion.o
        mkdir -p $(BUILD)
        $(CC) $(SRC)queryparser.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o
$(BUILD)queryparser.o

analysis.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)analysis.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)analysis.o

search.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)search.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)search.o

index.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)index.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)index.o

store.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)store.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)store.o

util.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)util.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)util.o

typeconversion.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)typeconversion.cpp $(CFLAGS) -o $(BUILD)typeconversion.o

clean:
        rm -Rf build/


Does anyone have experience working with scons and boost::python? Can
anyone help?

Thanks in advance!


--
Nicolas Lara
Linux user #380134
http://nicolas-lara.blogspot.com/
Public key id: 0x152e7713 at http://subkeys.pgp.net/
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From wladwig at wdtinc.com  Tue Jul  7 04:54:10 2009
From: wladwig at wdtinc.com (William Ladwig)
Date: Mon, 6 Jul 2009 21:54:10 -0500
Subject: [C++-sig] Building with Scons
In-Reply-To: <765CBD9053EA2B438625895F30D1856F023806D4D6@storm.wdtinc.com>
References: <46ef4bec0907061625w7bd5f8e5ye81dc79b6aab0104@mail.gmail.com>,
	<765CBD9053EA2B438625895F30D1856F023806D4D6@storm.wdtinc.com>
Message-ID: <765CBD9053EA2B438625895F30D1856F023806D4D7@storm.wdtinc.com>

Never mind, those single quotes looked like one double quote on my screen....


________________________________________
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of William Ladwig [wladwig at wdtinc.com]
Sent: Monday, July 06, 2009 9:53 PM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] Building with Scons

Does the name of the module defined in your BOOST_PYTHON_MODULE section match the name of the .so file (assuming you are using Linux)?  That's usually the error I get when I have a name mismatch.

Also, I haven't really used scons, but shouldn't this:

env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')

be:

env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX=''") ?


Bill


________________________________________
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Nicolas Lara [nicolaslara at gmail.com]
Sent: Monday, July 06, 2009 6:25 PM
To: Development of Python/C++ integration
Subject: [C++-sig] Building with Scons

Hello,

I am trying to build a python extension with scons but the resulting
library cannot be loaded:
   ImportError: dynamic module does not define init function (initcylucene)

I am using the following SConstruct file:

FILES = ['typeconversion.cpp', 'document.cpp', 'search.cpp', 'queryparser.cpp',
         'analysis.cpp', 'index.cpp', 'store.cpp', 'util.cpp']
def python_tool(env):
    pybase = 'python%s' % sys.version[0:3]
    env.Append(CPPPATH=[os.path.join(sys.prefix, 'include', pybase)],
               LIBPATH=[os.path.join(sys.prefix, 'lib', pybase, 'config')],
               LIBS=['lib%s' % pybase])
    if env['PLATFORM'] not in ['cygwin', 'win32']:
        env.Append(LIBS=['util'])

def boost_python_tool(env):
    env.Append(CPPDEFINES=['BOOST_PYTHON_DYNAMIC_LIB',
                           'BOOST_PYTHON_DYNAMIC_MODULE'],
               CPPPATH=['$boostIncludes'],  # boostIncludes is a PathOption
               LIBS=['boost_python'])

def clucene_tool(env):
    env.Append(CPPPATH=['/usr/local/lib/CLucene/',
                        '/usr/local/lib/'],
               LIBS=['clucene'])

import os
env = Environment(ENV=os.environ, tools=['default', python_tool,
                                         boost_python_tool, clucene_tool])
env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')


Previously I was using the following Makefile, which worked but was
statically linked:

SHELL = /bin/bash
SRC = ./src/
BUILD = ./build/
INCLUDES = -I/usr/include/python2.6/ -I/usr/local/lib/ -I/usr/local/lib/CLucene/
LIBS = -lboost_python -lpython2.6 -lclucene
CFLAGS = -shared -fPIC -g -pedantic -Wall -Wextra
OBJECTS = document.o search.o queryparser.o analysis.o index.o store.o util.o
LOBJECTS = $(BUILD)typeconversion.o $(BUILD)document.o
$(BUILD)search.o $(BUILD)queryparser.o $(BUILD)analysis.o
$(BUILD)index.o $(BUILD)store.o $(BUILD)util.o

all: cylucene

cylucene: $(OBJECTS)
        mkdir -p $(BUILD) && cp $(SRC)initfile.py $(BUILD)__init__.py
        $(CC) $(SRC)cylucene.cpp $(LOBJECTS) $(INCLUDES) $(LIBS) $(CFLAGS) -o
$(BUILD)cylucene.so

document.o: typeconversion.o
        mkdir -p $(BUILD)
        $(CC) $(SRC)document.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)document.o

queryparser.o: typeconversion.o
        mkdir -p $(BUILD)
        $(CC) $(SRC)queryparser.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o
$(BUILD)queryparser.o

analysis.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)analysis.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)analysis.o

search.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)search.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)search.o

index.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)index.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)index.o

store.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)store.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)store.o

util.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)util.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)util.o

typeconversion.o:
        mkdir -p $(BUILD)
        $(CC) $(SRC)typeconversion.cpp $(CFLAGS) -o $(BUILD)typeconversion.o

clean:
        rm -Rf build/


Does anyone have experience working with scons and boost::python? Can
anyone help?

Thanks in advance!


--
Nicolas Lara
Linux user #380134
http://nicolas-lara.blogspot.com/
Public key id: 0x152e7713 at http://subkeys.pgp.net/
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From paul at gingernut.tv  Tue Jul  7 12:54:56 2009
From: paul at gingernut.tv (Paul Scruby)
Date: Tue, 7 Jul 2009 11:54:56 +0100
Subject: [C++-sig] Passing Python classes derived from C++ back into C++
Message-ID: <h2v9i4$s6g$1@ger.gmane.org>

Hiya,

I'm trying to pass a C++ class that I've extended in Python to a C++ 
function.  I found an example of how to do this in David Abrahams article on 
Building Hybrid Systems with Boost Python.
http://www.boostpro.com/writing/bpl.html#virtual-functions

However, I've not managed to get this example to compile:

  #include <string>
  #include <boost/python.hpp>
  using namespace boost::python;

  class Base
  {
   public:
      virtual int f(std::string x) { return 42; }
      virtual ~Base() {}
  };

  int calls_f(Base & b, std::string x) { return b.f(x); }

  struct BaseWrap : Base
  {
      BaseWrap(PyObject * self_) : self(self_) {}
      PyObject * self;
      int f_default(std::string x) { return this->Base::f(x); }
      int f(std::string x) { return call_method<int>(self, "f", x); }
  };

  BOOST_PYTHON_MODULE(mytest)
  {
      def("calls_f", calls_f);
      class_<Base, BaseWrap>("Base")
          .def("f", &Base::f, &BaseWrap::f_default);
  }

_________________

  from mytest import *

  class Derived(Base):
      def f(self, s):
          return len(s)

  calls_f(Base(), 'foo')
  calls_f(Derived(), 'forty-two')


With Boost 1.38 and Visual Studio 8 compiler I get the error:

  boost_1_38\boost\python\object\make_instance.hpp(68) : see reference to 
function template instantiation 
'boost::python::objects::value_holder_back_reference<Value,Held>::value_holder_back_reference<boost::reference_wrapper<T>>(PyObject 
*,A0)' being compiled
  with
  [
      Value=Base,
      Held=BaseWrap,
      T=const Base,
      A0=boost::reference_wrapper<const Base>
  ]

With Boost 1.63 and Sun C++ 5.9 compiler I get the error:

  boost-1_36/boost/python/object/class_metadata.hpp", line 232:
  Error: Overloading ambiguity between "static 
boost::python::objects::class_metadata<Base, BaseWrap, 
boost::python::detail::not_specified, 
boost::python::detail::not_specified>::maybe_register_pointer_to_python(void*, 
void*, void*)"
  and "static boost::python::objects::class_metadata<Base, BaseWrap, 
boost::python::detail::not_specified, 
boost::python::detail::not_specified>::maybe_register_pointer_to_python(void*, 
void*, mpl_::bool_<1>*)".

I also tried extending the boost::python::wrapper<T> using get_override() 
instead of call_method<T>(), but as it can not be constructed with a 
PyObject * so the C++ base class can't be extracted from a Python 
derivation.

Has anyone managed to get this working?  Any help will be gratefully 
received.

Cheers,

Paul




From nicolaslara at gmail.com  Tue Jul  7 14:54:32 2009
From: nicolaslara at gmail.com (Nicolas Lara)
Date: Tue, 7 Jul 2009 12:54:32 +0000
Subject: [C++-sig] Building with Scons
In-Reply-To: <765CBD9053EA2B438625895F30D1856F023806D4D6@storm.wdtinc.com>
References: <46ef4bec0907061625w7bd5f8e5ye81dc79b6aab0104@mail.gmail.com> 
	<765CBD9053EA2B438625895F30D1856F023806D4D6@storm.wdtinc.com>
Message-ID: <46ef4bec0907070554r13e8dd5dh8a1d71facdc08b1@mail.gmail.com>

Thanks for the reply,

Yes, I checked the name of the module. It matches the name of the
generated file. I also tried changing the name to include "lib" (since
scons also generates called libcylucene.so) but it doesnt work.
My module looks like this:

void init_util();

BOOST_PYTHON_MODULE(cylucene)
{
   init_util();
}

On Tue, Jul 7, 2009 at 2:53 AM, William Ladwig<wladwig at wdtinc.com> wrote:
> Does the name of the module defined in your BOOST_PYTHON_MODULE section match the name of the .so file (assuming you are using Linux)? ?That's usually the error I get when I have a name mismatch.
>
> Also, I haven't really used scons, but shouldn't this:
>
> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')
>
> be:
>
> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX=''") ?
>
>
> Bill
>
>
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Nicolas Lara [nicolaslara at gmail.com]
> Sent: Monday, July 06, 2009 6:25 PM
> To: Development of Python/C++ integration
> Subject: [C++-sig] Building with Scons
>
> Hello,
>
> I am trying to build a python extension with scons but the resulting
> library cannot be loaded:
> ? ImportError: dynamic module does not define init function (initcylucene)
>
> I am using the following SConstruct file:
>
> FILES = ['typeconversion.cpp', 'document.cpp', 'search.cpp', 'queryparser.cpp',
> ? ? ? ? 'analysis.cpp', 'index.cpp', 'store.cpp', 'util.cpp']
> def python_tool(env):
> ? ?pybase = 'python%s' % sys.version[0:3]
> ? ?env.Append(CPPPATH=[os.path.join(sys.prefix, 'include', pybase)],
> ? ? ? ? ? ? ? LIBPATH=[os.path.join(sys.prefix, 'lib', pybase, 'config')],
> ? ? ? ? ? ? ? LIBS=['lib%s' % pybase])
> ? ?if env['PLATFORM'] not in ['cygwin', 'win32']:
> ? ? ? ?env.Append(LIBS=['util'])
>
> def boost_python_tool(env):
> ? ?env.Append(CPPDEFINES=['BOOST_PYTHON_DYNAMIC_LIB',
> ? ? ? ? ? ? ? ? ? ? ? ? ? 'BOOST_PYTHON_DYNAMIC_MODULE'],
> ? ? ? ? ? ? ? CPPPATH=['$boostIncludes'], ?# boostIncludes is a PathOption
> ? ? ? ? ? ? ? LIBS=['boost_python'])
>
> def clucene_tool(env):
> ? ?env.Append(CPPPATH=['/usr/local/lib/CLucene/',
> ? ? ? ? ? ? ? ? ? ? ? ?'/usr/local/lib/'],
> ? ? ? ? ? ? ? LIBS=['clucene'])
>
> import os
> env = Environment(ENV=os.environ, tools=['default', python_tool,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? boost_python_tool, clucene_tool])
> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')
>
>
> Previously I was using the following Makefile, which worked but was
> statically linked:
>
> SHELL = /bin/bash
> SRC = ./src/
> BUILD = ./build/
> INCLUDES = -I/usr/include/python2.6/ -I/usr/local/lib/ -I/usr/local/lib/CLucene/
> LIBS = -lboost_python -lpython2.6 -lclucene
> CFLAGS = -shared -fPIC -g -pedantic -Wall -Wextra
> OBJECTS = document.o search.o queryparser.o analysis.o index.o store.o util.o
> LOBJECTS = $(BUILD)typeconversion.o $(BUILD)document.o
> $(BUILD)search.o $(BUILD)queryparser.o $(BUILD)analysis.o
> $(BUILD)index.o $(BUILD)store.o $(BUILD)util.o
>
> all: cylucene
>
> cylucene: $(OBJECTS)
> ? ? ? ?mkdir -p $(BUILD) && cp $(SRC)initfile.py $(BUILD)__init__.py
> ? ? ? ?$(CC) $(SRC)cylucene.cpp $(LOBJECTS) $(INCLUDES) $(LIBS) $(CFLAGS) -o
> $(BUILD)cylucene.so
>
> document.o: typeconversion.o
> ? ? ? ?mkdir -p $(BUILD)
> ? ? ? ?$(CC) $(SRC)document.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)document.o
>
> queryparser.o: typeconversion.o
> ? ? ? ?mkdir -p $(BUILD)
> ? ? ? ?$(CC) $(SRC)queryparser.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o
> $(BUILD)queryparser.o
>
> analysis.o:
> ? ? ? ?mkdir -p $(BUILD)
> ? ? ? ?$(CC) $(SRC)analysis.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)analysis.o
>
> search.o:
> ? ? ? ?mkdir -p $(BUILD)
> ? ? ? ?$(CC) $(SRC)search.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)search.o
>
> index.o:
> ? ? ? ?mkdir -p $(BUILD)
> ? ? ? ?$(CC) $(SRC)index.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)index.o
>
> store.o:
> ? ? ? ?mkdir -p $(BUILD)
> ? ? ? ?$(CC) $(SRC)store.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)store.o
>
> util.o:
> ? ? ? ?mkdir -p $(BUILD)
> ? ? ? ?$(CC) $(SRC)util.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)util.o
>
> typeconversion.o:
> ? ? ? ?mkdir -p $(BUILD)
> ? ? ? ?$(CC) $(SRC)typeconversion.cpp $(CFLAGS) -o $(BUILD)typeconversion.o
>
> clean:
> ? ? ? ?rm -Rf build/
>
>
> Does anyone have experience working with scons and boost::python? Can
> anyone help?
>
> Thanks in advance!
>
>
> --
> Nicolas Lara
> Linux user #380134
> http://nicolas-lara.blogspot.com/
> Public key id: 0x152e7713 at http://subkeys.pgp.net/
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Nicolas Lara
Linux user #380134
http://nicolas-lara.blogspot.com/
Public key id: 0x152e7713 at http://subkeys.pgp.net/

From wladwig at wdtinc.com  Tue Jul  7 15:43:46 2009
From: wladwig at wdtinc.com (William Ladwig)
Date: Tue, 7 Jul 2009 08:43:46 -0500
Subject: [C++-sig] Building with Scons
In-Reply-To: <46ef4bec0907070554r13e8dd5dh8a1d71facdc08b1@mail.gmail.com>
References: <46ef4bec0907061625w7bd5f8e5ye81dc79b6aab0104@mail.gmail.com>
	<765CBD9053EA2B438625895F30D1856F023806D4D6@storm.wdtinc.com>,
	<46ef4bec0907070554r13e8dd5dh8a1d71facdc08b1@mail.gmail.com>
Message-ID: <765CBD9053EA2B438625895F30D1856F023806D4DB@storm.wdtinc.com>

You definitely want the name of the extension .so file to be cylucene.so, so you will want to drop the 'lib' prefix.  Is your BOOST_PYTHON_MODULE contained in a file named cylucene.cpp?  It looks like you are using a cylucene.cpp file in the final step of your makefile, but it is not in your list of "FILES" for your scons script, so I don't think it is ever getting compiled and linked (that would be consistent with the error you are getting).  If that's the case, you can probably just add it to the FILES list and everything will come to life.

Also, here is a link from the boost.python howto which has an example of building with scons:

http://wiki.python.org/moin/boost.python/BuildingExtensions

Bill


________________________________________
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Nicolas Lara [nicolaslara at gmail.com]
Sent: Tuesday, July 07, 2009 7:54 AM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] Building with Scons

Thanks for the reply,

Yes, I checked the name of the module. It matches the name of the
generated file. I also tried changing the name to include "lib" (since
scons also generates called libcylucene.so) but it doesnt work.
My module looks like this:

void init_util();

BOOST_PYTHON_MODULE(cylucene)
{
   init_util();
}

On Tue, Jul 7, 2009 at 2:53 AM, William Ladwig<wladwig at wdtinc.com> wrote:
> Does the name of the module defined in your BOOST_PYTHON_MODULE section match the name of the .so file (assuming you are using Linux)?  That's usually the error I get when I have a name mismatch.
>
> Also, I haven't really used scons, but shouldn't this:
>
> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')
>
> be:
>
> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX=''") ?
>
>
> Bill
>
>
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Nicolas Lara [nicolaslara at gmail.com]
> Sent: Monday, July 06, 2009 6:25 PM
> To: Development of Python/C++ integration
> Subject: [C++-sig] Building with Scons
>
> Hello,
>
> I am trying to build a python extension with scons but the resulting
> library cannot be loaded:
>   ImportError: dynamic module does not define init function (initcylucene)
>
> I am using the following SConstruct file:
>
> FILES = ['typeconversion.cpp', 'document.cpp', 'search.cpp', 'queryparser.cpp',
>         'analysis.cpp', 'index.cpp', 'store.cpp', 'util.cpp']
> def python_tool(env):
>    pybase = 'python%s' % sys.version[0:3]
>    env.Append(CPPPATH=[os.path.join(sys.prefix, 'include', pybase)],
>               LIBPATH=[os.path.join(sys.prefix, 'lib', pybase, 'config')],
>               LIBS=['lib%s' % pybase])
>    if env['PLATFORM'] not in ['cygwin', 'win32']:
>        env.Append(LIBS=['util'])
>
> def boost_python_tool(env):
>    env.Append(CPPDEFINES=['BOOST_PYTHON_DYNAMIC_LIB',
>                           'BOOST_PYTHON_DYNAMIC_MODULE'],
>               CPPPATH=['$boostIncludes'],  # boostIncludes is a PathOption
>               LIBS=['boost_python'])
>
> def clucene_tool(env):
>    env.Append(CPPPATH=['/usr/local/lib/CLucene/',
>                        '/usr/local/lib/'],
>               LIBS=['clucene'])
>
> import os
> env = Environment(ENV=os.environ, tools=['default', python_tool,
>                                         boost_python_tool, clucene_tool])
> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')
>
>
> Previously I was using the following Makefile, which worked but was
> statically linked:
>
> SHELL = /bin/bash
> SRC = ./src/
> BUILD = ./build/
> INCLUDES = -I/usr/include/python2.6/ -I/usr/local/lib/ -I/usr/local/lib/CLucene/
> LIBS = -lboost_python -lpython2.6 -lclucene
> CFLAGS = -shared -fPIC -g -pedantic -Wall -Wextra
> OBJECTS = document.o search.o queryparser.o analysis.o index.o store.o util.o
> LOBJECTS = $(BUILD)typeconversion.o $(BUILD)document.o
> $(BUILD)search.o $(BUILD)queryparser.o $(BUILD)analysis.o
> $(BUILD)index.o $(BUILD)store.o $(BUILD)util.o
>
> all: cylucene
>
> cylucene: $(OBJECTS)
>        mkdir -p $(BUILD) && cp $(SRC)initfile.py $(BUILD)__init__.py
>        $(CC) $(SRC)cylucene.cpp $(LOBJECTS) $(INCLUDES) $(LIBS) $(CFLAGS) -o
> $(BUILD)cylucene.so
>
> document.o: typeconversion.o
>        mkdir -p $(BUILD)
>        $(CC) $(SRC)document.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)document.o
>
> queryparser.o: typeconversion.o
>        mkdir -p $(BUILD)
>        $(CC) $(SRC)queryparser.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o
> $(BUILD)queryparser.o
>
> analysis.o:
>        mkdir -p $(BUILD)
>        $(CC) $(SRC)analysis.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)analysis.o
>
> search.o:
>        mkdir -p $(BUILD)
>        $(CC) $(SRC)search.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)search.o
>
> index.o:
>        mkdir -p $(BUILD)
>        $(CC) $(SRC)index.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)index.o
>
> store.o:
>        mkdir -p $(BUILD)
>        $(CC) $(SRC)store.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)store.o
>
> util.o:
>        mkdir -p $(BUILD)
>        $(CC) $(SRC)util.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)util.o
>
> typeconversion.o:
>        mkdir -p $(BUILD)
>        $(CC) $(SRC)typeconversion.cpp $(CFLAGS) -o $(BUILD)typeconversion.o
>
> clean:
>        rm -Rf build/
>
>
> Does anyone have experience working with scons and boost::python? Can
> anyone help?
>
> Thanks in advance!
>
>
> --
> Nicolas Lara
> Linux user #380134
> http://nicolas-lara.blogspot.com/
> Public key id: 0x152e7713 at http://subkeys.pgp.net/
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



--
Nicolas Lara
Linux user #380134
http://nicolas-lara.blogspot.com/
Public key id: 0x152e7713 at http://subkeys.pgp.net/
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From renatox at gmail.com  Tue Jul  7 16:08:04 2009
From: renatox at gmail.com (Renato Araujo)
Date: Tue, 7 Jul 2009 11:08:04 -0300
Subject: [C++-sig] Passing Python classes derived from C++ back into C++
In-Reply-To: <h2v9i4$s6g$1@ger.gmane.org>
References: <h2v9i4$s6g$1@ger.gmane.org>
Message-ID: <95291a80907070708p66393027nde354f5b7cc3ab1a@mail.gmail.com>

Hi Paul,

You can  extend from boost::python::wrapper<T>  and use this function
to get PyObject from c++ object.
namespace detail
{
..
  namespace wrapper_base_ // ADL disabler
  {
    inline PyObject* get_owner(wrapper_base const volatile& w);
....

BR



On Tue, Jul 7, 2009 at 7:54 AM, Paul Scruby<paul at gingernut.tv> wrote:
> Hiya,
>
> I'm trying to pass a C++ class that I've extended in Python to a C++
> function. ?I found an example of how to do this in David Abrahams article on
> Building Hybrid Systems with Boost Python.
> http://www.boostpro.com/writing/bpl.html#virtual-functions
>
> However, I've not managed to get this example to compile:
>
> ?#include <string>
> ?#include <boost/python.hpp>
> ?using namespace boost::python;
>
> ?class Base
> ?{
> ? public:
> ? ? ?virtual int f(std::string x) { return 42; }
> ? ? ?virtual ~Base() {}
> ?};
>
> ?int calls_f(Base & b, std::string x) { return b.f(x); }
>
> ?struct BaseWrap : Base
> ?{
> ? ? ?BaseWrap(PyObject * self_) : self(self_) {}
> ? ? ?PyObject * self;
> ? ? ?int f_default(std::string x) { return this->Base::f(x); }
> ? ? ?int f(std::string x) { return call_method<int>(self, "f", x); }
> ?};
>
> ?BOOST_PYTHON_MODULE(mytest)
> ?{
> ? ? ?def("calls_f", calls_f);
> ? ? ?class_<Base, BaseWrap>("Base")
> ? ? ? ? ?.def("f", &Base::f, &BaseWrap::f_default);
> ?}
>
> _________________
>
> ?from mytest import *
>
> ?class Derived(Base):
> ? ? ?def f(self, s):
> ? ? ? ? ?return len(s)
>
> ?calls_f(Base(), 'foo')
> ?calls_f(Derived(), 'forty-two')
>
>
> With Boost 1.38 and Visual Studio 8 compiler I get the error:
>
> ?boost_1_38\boost\python\object\make_instance.hpp(68) : see reference to
> function template instantiation
> 'boost::python::objects::value_holder_back_reference<Value,Held>::value_holder_back_reference<boost::reference_wrapper<T>>(PyObject
> *,A0)' being compiled
> ?with
> ?[
> ? ? ?Value=Base,
> ? ? ?Held=BaseWrap,
> ? ? ?T=const Base,
> ? ? ?A0=boost::reference_wrapper<const Base>
> ?]
>
> With Boost 1.63 and Sun C++ 5.9 compiler I get the error:
>
> ?boost-1_36/boost/python/object/class_metadata.hpp", line 232:
> ?Error: Overloading ambiguity between "static
> boost::python::objects::class_metadata<Base, BaseWrap,
> boost::python::detail::not_specified,
> boost::python::detail::not_specified>::maybe_register_pointer_to_python(void*,
> void*, void*)"
> ?and "static boost::python::objects::class_metadata<Base, BaseWrap,
> boost::python::detail::not_specified,
> boost::python::detail::not_specified>::maybe_register_pointer_to_python(void*,
> void*, mpl_::bool_<1>*)".
>
> I also tried extending the boost::python::wrapper<T> using get_override()
> instead of call_method<T>(), but as it can not be constructed with a
> PyObject * so the C++ base class can't be extracted from a Python
> derivation.
>
> Has anyone managed to get this working? ?Any help will be gratefully
> received.
>
> Cheers,
>
> Paul
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho

From alexey.akimov85 at gmail.com  Tue Jul  7 16:16:15 2009
From: alexey.akimov85 at gmail.com (Alexey Akimov)
Date: Tue, 7 Jul 2009 09:16:15 -0500
Subject: [C++-sig] exposing pointer to Python
In-Reply-To: <95291a80907061900g6248daa4g7d9a02e0db22e605@mail.gmail.com>
References: <382d227f0907061607x68a00129y9136a21b8432fd1@mail.gmail.com>
	<95291a80907061900g6248daa4g7d9a02e0db22e605@mail.gmail.com>
Message-ID: <382d227f0907070716j46e87308s8b278fb487b47dc0@mail.gmail.com>

Thank you, Renato

However this is still not clear how to wrap a pointer to double using that
indexing_suites. For types like vector<double> it is pretty much clear, but
what do i need to do in order to wrap just a simple poiner to double?

Also i have related question - how can i wrap an overloaded indexing
operator - []? For numerical operators there are some signatures like
.def(self+self) ,etc, but for operator [] i've never seen any such usage
signatures? Is there any way to do this?


2009/7/6 Renato Araujo <renatox at gmail.com>

> take a look at:
> http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/indexing.html
>
> BR
>
>
> On Mon, Jul 6, 2009 at 8:07 PM, Alexey Akimov<alexey.akimov85 at gmail.com>
> wrote:
> > I have some class X that contains a variable which is a pointer to some
> > other class OtherClass (for simplicity we may imagine it can be some
> > standard c++ type - int, double, etc.):
> >
> > class X{
> >
> > public:
> >      OtherClass* variable;
> >
> > };
> >
> > my question is: how can I expose it to python such that user may access
> to
> > variable using [] operator?
> > like:
> >
> >>>> import module
> >>>> module.X[i] = i
> >
> > I've found several examples on this topic:
> > http://mail.python.org/pipermail/cplusplus-sig/2005-January/008189.html
> > http://osdir.com/ml/python.c++/2003-01/msg00042.html
> >
> > but still can not make this trick.
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Cplusplus-sig mailing list
> > Cplusplus-sig at python.org
> > http://mail.python.org/mailman/listinfo/cplusplus-sig
> >
>
>
>
> --
> Renato Araujo Oliveira Filho
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090707/11b65853/attachment.htm>

From nicolaslara at gmail.com  Tue Jul  7 16:22:15 2009
From: nicolaslara at gmail.com (Nicolas Lara)
Date: Tue, 7 Jul 2009 14:22:15 +0000
Subject: [C++-sig] Building with Scons
In-Reply-To: <765CBD9053EA2B438625895F30D1856F023806D4DB@storm.wdtinc.com>
References: <46ef4bec0907061625w7bd5f8e5ye81dc79b6aab0104@mail.gmail.com> 
	<765CBD9053EA2B438625895F30D1856F023806D4D6@storm.wdtinc.com> 
	<46ef4bec0907070554r13e8dd5dh8a1d71facdc08b1@mail.gmail.com> 
	<765CBD9053EA2B438625895F30D1856F023806D4DB@storm.wdtinc.com>
Message-ID: <46ef4bec0907070722t6cb8f21evc3de6c74cc0524b8@mail.gmail.com>

Of course! Silly mistake... thank you!, that worked perfectly...

On Tue, Jul 7, 2009 at 1:43 PM, William Ladwig<wladwig at wdtinc.com> wrote:
> You definitely want the name of the extension .so file to be cylucene.so, so you will want to drop the 'lib' prefix. ?Is your BOOST_PYTHON_MODULE contained in a file named cylucene.cpp? ?It looks like you are using a cylucene.cpp file in the final step of your makefile, but it is not in your list of "FILES" for your scons script, so I don't think it is ever getting compiled and linked (that would be consistent with the error you are getting). ?If that's the case, you can probably just add it to the FILES list and everything will come to life.
>
> Also, here is a link from the boost.python howto which has an example of building with scons:
>
> http://wiki.python.org/moin/boost.python/BuildingExtensions
>
> Bill
>
>
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Nicolas Lara [nicolaslara at gmail.com]
> Sent: Tuesday, July 07, 2009 7:54 AM
> To: Development of Python/C++ integration
> Subject: Re: [C++-sig] Building with Scons
>
> Thanks for the reply,
>
> Yes, I checked the name of the module. It matches the name of the
> generated file. I also tried changing the name to include "lib" (since
> scons also generates called libcylucene.so) but it doesnt work.
> My module looks like this:
>
> void init_util();
>
> BOOST_PYTHON_MODULE(cylucene)
> {
> ? init_util();
> }
>
> On Tue, Jul 7, 2009 at 2:53 AM, William Ladwig<wladwig at wdtinc.com> wrote:
>> Does the name of the module defined in your BOOST_PYTHON_MODULE section match the name of the .so file (assuming you are using Linux)? ?That's usually the error I get when I have a name mismatch.
>>
>> Also, I haven't really used scons, but shouldn't this:
>>
>> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')
>>
>> be:
>>
>> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX=''") ?
>>
>>
>> Bill
>>
>>
>> ________________________________________
>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Nicolas Lara [nicolaslara at gmail.com]
>> Sent: Monday, July 06, 2009 6:25 PM
>> To: Development of Python/C++ integration
>> Subject: [C++-sig] Building with Scons
>>
>> Hello,
>>
>> I am trying to build a python extension with scons but the resulting
>> library cannot be loaded:
>> ? ImportError: dynamic module does not define init function (initcylucene)
>>
>> I am using the following SConstruct file:
>>
>> FILES = ['typeconversion.cpp', 'document.cpp', 'search.cpp', 'queryparser.cpp',
>> ? ? ? ? 'analysis.cpp', 'index.cpp', 'store.cpp', 'util.cpp']
>> def python_tool(env):
>> ? ?pybase = 'python%s' % sys.version[0:3]
>> ? ?env.Append(CPPPATH=[os.path.join(sys.prefix, 'include', pybase)],
>> ? ? ? ? ? ? ? LIBPATH=[os.path.join(sys.prefix, 'lib', pybase, 'config')],
>> ? ? ? ? ? ? ? LIBS=['lib%s' % pybase])
>> ? ?if env['PLATFORM'] not in ['cygwin', 'win32']:
>> ? ? ? ?env.Append(LIBS=['util'])
>>
>> def boost_python_tool(env):
>> ? ?env.Append(CPPDEFINES=['BOOST_PYTHON_DYNAMIC_LIB',
>> ? ? ? ? ? ? ? ? ? ? ? ? ? 'BOOST_PYTHON_DYNAMIC_MODULE'],
>> ? ? ? ? ? ? ? CPPPATH=['$boostIncludes'], ?# boostIncludes is a PathOption
>> ? ? ? ? ? ? ? LIBS=['boost_python'])
>>
>> def clucene_tool(env):
>> ? ?env.Append(CPPPATH=['/usr/local/lib/CLucene/',
>> ? ? ? ? ? ? ? ? ? ? ? ?'/usr/local/lib/'],
>> ? ? ? ? ? ? ? LIBS=['clucene'])
>>
>> import os
>> env = Environment(ENV=os.environ, tools=['default', python_tool,
>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? boost_python_tool, clucene_tool])
>> env.SharedLibrary(target='cylucene', source=FILES, SHLIBPREFIX='')
>>
>>
>> Previously I was using the following Makefile, which worked but was
>> statically linked:
>>
>> SHELL = /bin/bash
>> SRC = ./src/
>> BUILD = ./build/
>> INCLUDES = -I/usr/include/python2.6/ -I/usr/local/lib/ -I/usr/local/lib/CLucene/
>> LIBS = -lboost_python -lpython2.6 -lclucene
>> CFLAGS = -shared -fPIC -g -pedantic -Wall -Wextra
>> OBJECTS = document.o search.o queryparser.o analysis.o index.o store.o util.o
>> LOBJECTS = $(BUILD)typeconversion.o $(BUILD)document.o
>> $(BUILD)search.o $(BUILD)queryparser.o $(BUILD)analysis.o
>> $(BUILD)index.o $(BUILD)store.o $(BUILD)util.o
>>
>> all: cylucene
>>
>> cylucene: $(OBJECTS)
>> ? ? ? ?mkdir -p $(BUILD) && cp $(SRC)initfile.py $(BUILD)__init__.py
>> ? ? ? ?$(CC) $(SRC)cylucene.cpp $(LOBJECTS) $(INCLUDES) $(LIBS) $(CFLAGS) -o
>> $(BUILD)cylucene.so
>>
>> document.o: typeconversion.o
>> ? ? ? ?mkdir -p $(BUILD)
>> ? ? ? ?$(CC) $(SRC)document.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)document.o
>>
>> queryparser.o: typeconversion.o
>> ? ? ? ?mkdir -p $(BUILD)
>> ? ? ? ?$(CC) $(SRC)queryparser.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o
>> $(BUILD)queryparser.o
>>
>> analysis.o:
>> ? ? ? ?mkdir -p $(BUILD)
>> ? ? ? ?$(CC) $(SRC)analysis.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)analysis.o
>>
>> search.o:
>> ? ? ? ?mkdir -p $(BUILD)
>> ? ? ? ?$(CC) $(SRC)search.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)search.o
>>
>> index.o:
>> ? ? ? ?mkdir -p $(BUILD)
>> ? ? ? ?$(CC) $(SRC)index.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)index.o
>>
>> store.o:
>> ? ? ? ?mkdir -p $(BUILD)
>> ? ? ? ?$(CC) $(SRC)store.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)store.o
>>
>> util.o:
>> ? ? ? ?mkdir -p $(BUILD)
>> ? ? ? ?$(CC) $(SRC)util.cpp $(INCLUDES) $(LIBS) $(CFLAGS) -o $(BUILD)util.o
>>
>> typeconversion.o:
>> ? ? ? ?mkdir -p $(BUILD)
>> ? ? ? ?$(CC) $(SRC)typeconversion.cpp $(CFLAGS) -o $(BUILD)typeconversion.o
>>
>> clean:
>> ? ? ? ?rm -Rf build/
>>
>>
>> Does anyone have experience working with scons and boost::python? Can
>> anyone help?
>>
>> Thanks in advance!
>>
>>
>> --
>> Nicolas Lara
>> Linux user #380134
>> http://nicolas-lara.blogspot.com/
>> Public key id: 0x152e7713 at http://subkeys.pgp.net/
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>
>
>
>
> --
> Nicolas Lara
> Linux user #380134
> http://nicolas-lara.blogspot.com/
> Public key id: 0x152e7713 at http://subkeys.pgp.net/
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Nicolas Lara
Linux user #380134
http://nicolas-lara.blogspot.com/
Public key id: 0x152e7713 at http://subkeys.pgp.net/

From gjcarneiro at gmail.com  Tue Jul  7 17:27:40 2009
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Tue, 7 Jul 2009 16:27:40 +0100
Subject: [C++-sig] pybindgen: allow_subclassing option causing reference
	parameters to be copied?
In-Reply-To: <765D6C96-5340-4A57-9333-DC53D362C39B@llnl.gov>
References: <A1EAE3BB-FF1C-4492-AE10-76637EA656C4@llnl.gov>
	<a467ca4f0906241531v663e33ew98c0285c98adf247@mail.gmail.com>
	<C2F723FD-D5E1-48D4-B424-00C392A96F8C@llnl.gov>
	<a467ca4f0906241703s21eabcb7jeac2ce932fa28512@mail.gmail.com>
	<a467ca4f0906280819s638e0d06td0439f304c6323a3@mail.gmail.com>
	<B69AC2CB-26C4-4997-9EAF-1A28AD76A9CC@llnl.gov>
	<a467ca4f0906300629y62b03396pf2b8d0c5f30db0c0@mail.gmail.com>
	<765D6C96-5340-4A57-9333-DC53D362C39B@llnl.gov>
Message-ID: <a467ca4f0907070827s619fb884lb07ae1b68649418@mail.gmail.com>

2009/7/2 J. Michael Owen <mikeowen at llnl.gov>

> I pulled down your bazaar branch and merged it with my own changes (I need
> to use the container methods, inplace numeric operators, and numeric
> operators with built in python  types, so I have to keep using my changes
> for now), and now reference parameters are working just fine.  Great!
>

I just pushed your merged contribution with some fixes and unit tests to
bazaar trunk.


>
> Since the code I'm binding has probably thousands of reference parameters
> to expose, I've written little helper methods as shorthand for the long
> parameter definition.  This is probably a little specialized for what you
> want, but it might be helpful to include something like these (akin to the
> "param" method you already have in utils):
>

To be frank I don't like this. refparam does not give any hint that it
specifies direction inout for the parameter.  I don't think it's worth to
merge this.


>
> def refparam(cppobj, argname):
>     return Parameter.new("%s&" % cppobj, argname,
> direction=Parameter.DIRECTION_INOUT)
>
> def constrefparam(cppobj, argname):
>     return Parameter.new(ref("const %s&" % cppobj), argname,
> direction=Parameter.DIRECTION_INOUT)
>
> So on those changes I have for the pybindgen code (allowing the stuff I
> mention above), I'd be happy to modify them to fit in better with how you
> want pybindgen to look if you'd like to give me an idea what you're looking
> for.
>
> On Jun 30, 2009, at 6:29 AM, Gustavo Carneiro wrote:
>
>
> Ah, right.  The thing is, the code tries to make a copy of the object to
> give to the python wrapper in case it appears the python code kept a
> reference to the wrapper.
>
> I need to fix pybindgen to, if the class has no copy constructor keep the
> wrapper->obj pointer NULL instead of giving it a copy.
>
> The fix has been pushed to the bazaar branch.  See https://
> code.launchpad.net/~gjc/pybindgen/trunk<https://+code.launchpad.net/%7Egjc/pybindgen/trunk>
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090707/a381946b/attachment-0001.htm>

From roman.yakovenko at gmail.com  Tue Jul  7 21:36:22 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 7 Jul 2009 22:36:22 +0300
Subject: [C++-sig] exposing pointer to Python
In-Reply-To: <382d227f0907070716j46e87308s8b278fb487b47dc0@mail.gmail.com>
References: <382d227f0907061607x68a00129y9136a21b8432fd1@mail.gmail.com>
	<95291a80907061900g6248daa4g7d9a02e0db22e605@mail.gmail.com>
	<382d227f0907070716j46e87308s8b278fb487b47dc0@mail.gmail.com>
Message-ID: <7465b6170907071236x353e9e79m52018180a53be7f9@mail.gmail.com>

On Tue, Jul 7, 2009 at 5:16 PM, Alexey Akimov<alexey.akimov85 at gmail.com> wrote:
> Thank you, Renato
>
> However this is still not clear how to wrap a pointer to double using that
> indexing_suites. For types like vector<double> it is pretty much clear, but
> what do i need to do in order to wrap just a simple poiner to double?

The short answer: you can not. The long answer - use Google

> Also i have related question - how can i wrap an overloaded indexing
> operator - []? For numerical operators there are some signatures like
> .def(self+self) ,etc, but for operator [] i've never seen any such usage
> signatures? Is there any way to do this?

Yes. Take a look on one of my call policies:

http://language-binding.net/pyplusplus/documentation/functions/call_policies/return_range.html

This should give you an idea and direction.


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From mikeowen at llnl.gov  Wed Jul  8 02:19:22 2009
From: mikeowen at llnl.gov (J. Michael Owen)
Date: Tue, 7 Jul 2009 17:19:22 -0700
Subject: [C++-sig] pybindgen: allow_subclassing option causing reference
	parameters to be copied?
In-Reply-To: <a467ca4f0907070827s619fb884lb07ae1b68649418@mail.gmail.com>
References: <A1EAE3BB-FF1C-4492-AE10-76637EA656C4@llnl.gov>
	<a467ca4f0906241531v663e33ew98c0285c98adf247@mail.gmail.com>
	<C2F723FD-D5E1-48D4-B424-00C392A96F8C@llnl.gov>
	<a467ca4f0906241703s21eabcb7jeac2ce932fa28512@mail.gmail.com>
	<a467ca4f0906280819s638e0d06td0439f304c6323a3@mail.gmail.com>
	<B69AC2CB-26C4-4997-9EAF-1A28AD76A9CC@llnl.gov>
	<a467ca4f0906300629y62b03396pf2b8d0c5f30db0c0@mail.gmail.com>
	<765D6C96-5340-4A57-9333-DC53D362C39B@llnl.gov>
	<a467ca4f0907070827s619fb884lb07ae1b68649418@mail.gmail.com>
Message-ID: <5B00A3C9-EAC1-4740-9AB0-BE2A5F669444@llnl.gov>

Hello Gustavo,

I have some further evolved changes for the sequence methods that you  
may want to use over my previous version.  This is more general, and  
involves less code as I realized I could register the sequence methods  
to just call the wrapped methods pybindgen is already providing.  This  
also  also corrects a bug I had with exposing containers of pointers.   
Since this code is more general and smaller, it seems like an  
improvement over what I gave you before.

I'm also including some minor extensions to your name mangling code in  
utils.py here, which I found necessary 'cause I'm using templated  
functions with things like "add_function_as_method".

These changes are current with your latest Bazaar branch, and I've  
verified that all tests are passing.  There are three altered files  
I'm including here: cppclass.py, pytypeobject.py, and utils.py.

Mike.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: patched_files.tbz
Type: application/octet-stream
Size: 18195 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090707/3ff6cf80/attachment-0001.obj>
-------------- next part --------------





On Jul 7, 2009, at 8:27 AM, Gustavo Carneiro wrote:
>
> I just pushed your merged contribution with some fixes and unit  
> tests to bazaar trunk.

From paul at gingernut.tv  Wed Jul  8 12:03:09 2009
From: paul at gingernut.tv (Paul Scruby)
Date: Wed, 8 Jul 2009 11:03:09 +0100
Subject: [C++-sig] Passing Python classes derived from C++ back into C++
References: <h2v9i4$s6g$1@ger.gmane.org>
	<95291a80907070708p66393027nde354f5b7cc3ab1a@mail.gmail.com>
Message-ID: <h31qt8$sq4$1@ger.gmane.org>

Hi Rento,

Got it working now.  I did not realize that the wrapper<> automatically 
handled downcast conversion as well.

Thanks,

Paul

"Renato Araujo" <renatox at gmail.com> wrote in message 
news:95291a80907070708p66393027nde354f5b7cc3ab1a at mail.gmail.com...
Hi Paul,

You can  extend from boost::python::wrapper<T>  and use this function
to get PyObject from c++ object.
namespace detail
{
..
  namespace wrapper_base_ // ADL disabler
  {
    inline PyObject* get_owner(wrapper_base const volatile& w);
....

BR



On Tue, Jul 7, 2009 at 7:54 AM, Paul Scruby<paul at gingernut.tv> wrote:
> Hiya,
>
> I'm trying to pass a C++ class that I've extended in Python to a C++
> function. I found an example of how to do this in David Abrahams article 
> on
> Building Hybrid Systems with Boost Python.
> http://www.boostpro.com/writing/bpl.html#virtual-functions
>
> However, I've not managed to get this example to compile:
>
> #include <string>
> #include <boost/python.hpp>
> using namespace boost::python;
>
> class Base
> {
> public:
> virtual int f(std::string x) { return 42; }
> virtual ~Base() {}
> };
>
> int calls_f(Base & b, std::string x) { return b.f(x); }
>
> struct BaseWrap : Base
> {
> BaseWrap(PyObject * self_) : self(self_) {}
> PyObject * self;
> int f_default(std::string x) { return this->Base::f(x); }
> int f(std::string x) { return call_method<int>(self, "f", x); }
> };
>
> BOOST_PYTHON_MODULE(mytest)
> {
> def("calls_f", calls_f);
> class_<Base, BaseWrap>("Base")
> .def("f", &Base::f, &BaseWrap::f_default);
> }
>
> _________________
>
> from mytest import *
>
> class Derived(Base):
> def f(self, s):
> return len(s)
>
> calls_f(Base(), 'foo')
> calls_f(Derived(), 'forty-two')
>
>
> With Boost 1.38 and Visual Studio 8 compiler I get the error:
>
> boost_1_38\boost\python\object\make_instance.hpp(68) : see reference to
> function template instantiation
> 'boost::python::objects::value_holder_back_reference<Value,Held>::value_holder_back_reference<boost::reference_wrapper<T>>(PyObject
> *,A0)' being compiled
> with
> [
> Value=Base,
> Held=BaseWrap,
> T=const Base,
> A0=boost::reference_wrapper<const Base>
> ]
>
> With Boost 1.63 and Sun C++ 5.9 compiler I get the error:
>
> boost-1_36/boost/python/object/class_metadata.hpp", line 232:
> Error: Overloading ambiguity between "static
> boost::python::objects::class_metadata<Base, BaseWrap,
> boost::python::detail::not_specified,
> boost::python::detail::not_specified>::maybe_register_pointer_to_python(void*,
> void*, void*)"
> and "static boost::python::objects::class_metadata<Base, BaseWrap,
> boost::python::detail::not_specified,
> boost::python::detail::not_specified>::maybe_register_pointer_to_python(void*,
> void*, mpl_::bool_<1>*)".
>
> I also tried extending the boost::python::wrapper<T> using get_override()
> instead of call_method<T>(), but as it can not be constructed with a
> PyObject * so the C++ base class can't be extracted from a Python
> derivation.
>
> Has anyone managed to get this working? Any help will be gratefully
> received.
>
> Cheers,
>
> Paul
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho 




From gjcarneiro at gmail.com  Wed Jul  8 12:07:09 2009
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Wed, 8 Jul 2009 11:07:09 +0100
Subject: [C++-sig] pybindgen: allow_subclassing option causing reference
	parameters to be copied?
In-Reply-To: <5B00A3C9-EAC1-4740-9AB0-BE2A5F669444@llnl.gov>
References: <A1EAE3BB-FF1C-4492-AE10-76637EA656C4@llnl.gov>
	<a467ca4f0906241531v663e33ew98c0285c98adf247@mail.gmail.com>
	<C2F723FD-D5E1-48D4-B424-00C392A96F8C@llnl.gov>
	<a467ca4f0906241703s21eabcb7jeac2ce932fa28512@mail.gmail.com>
	<a467ca4f0906280819s638e0d06td0439f304c6323a3@mail.gmail.com>
	<B69AC2CB-26C4-4997-9EAF-1A28AD76A9CC@llnl.gov>
	<a467ca4f0906300629y62b03396pf2b8d0c5f30db0c0@mail.gmail.com>
	<765D6C96-5340-4A57-9333-DC53D362C39B@llnl.gov>
	<a467ca4f0907070827s619fb884lb07ae1b68649418@mail.gmail.com>
	<5B00A3C9-EAC1-4740-9AB0-BE2A5F669444@llnl.gov>
Message-ID: <a467ca4f0907080307if549d1dw798935079853c301@mail.gmail.com>

2009/7/8 J. Michael Owen <mikeowen at llnl.gov>

> Hello Gustavo,
>
> I have some further evolved changes for the sequence methods that you may
> want to use over my previous version.  This is more general, and involves
> less code as I realized I could register the sequence methods to just call
> the wrapped methods pybindgen is already providing.  This also  also
> corrects a bug I had with exposing containers of pointers.  Since this code
> is more general and smaller, it seems like an improvement over what I gave
> you before.
>
> I'm also including some minor extensions to your name mangling code in
> utils.py here, which I found necessary 'cause I'm using templated functions
> with things like "add_function_as_method".
>
> These changes are current with your latest Bazaar branch, and I've verified
> that all tests are passing.  There are three altered files I'm including
> here: cppclass.py, pytypeobject.py, and utils.py.
>

Pushed these changes, with memory leaks fixed.

Thanks.


>
> Mike.
>
>
>
>
>
>
>
> On Jul 7, 2009, at 8:27 AM, Gustavo Carneiro wrote:
>
>>
>> I just pushed your merged contribution with some fixes and unit tests to
>> bazaar trunk.
>>
>
>


-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090708/f85e9e83/attachment.htm>

From paul at gingernut.tv  Wed Jul  8 17:48:29 2009
From: paul at gingernut.tv (Paul Scruby)
Date: Wed, 8 Jul 2009 16:48:29 +0100
Subject: [C++-sig] boost::python and threads
References: <h2kp7s$m69$1@ger.gmane.org><765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com><765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com><95291a80907041315k41b7ad88o32d2111ae8fe1e91@mail.gmail.com><h2sgic$ad4$1@ger.gmane.org>
	<h2ska4$l0m$1@ger.gmane.org>
	<95291a80907060619u5bff0dcey56947d1ac848cc8e@mail.gmail.com>
Message-ID: <h32f4m$8ip$1@ger.gmane.org>

Hi Renato,

Okay, I have installed Python 2.4.2 with gcc 4.1.2 on Linux and it's 
crashing in the same place as on Solaris.

  Program received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 0x41074940 (LWP 12004)]
  0x000000307740c850 in sem_post () from /lib64/libpthread.so.0
  (gdb) where
  #0  0x000000307740c850 in sem_post () from /lib64/libpthread.so.0
  #1  0x000000308b2b78e9 in PyThread_release_lock ()
     from /usr/lib64/libpython2.4.so.1.0
  #2  0x00002b46531d9f82 in Ticker::operator() (this=0xbd0f478) at 
ticker.cc:28
  #3  0x00002b46531d9fb0 in boost::detail::thread_data<Ticker>::run (
      this=0xbd0f370)
      at /opt/atm/include/boost-1_39/boost/thread/detail/thread.hpp:56
  #4  0x00002b46533ee14b in thread_proxy ()
     from /opt/atm/lib64/libboost_thread-gcc41-mt-1_39.so.1.39.0
  #5  0x0000003077406367 in start_thread () from /lib64/libpthread.so.0
  #6  0x00000030768d2f7d in clone () from /lib64/libc.so.6

With the global interpret lock added my code now looks like this...

  #include <boost/python.hpp>
  #include <boost/thread/thread.hpp>
  #include <boost/thread/xtime.hpp>
  using namespace boost::python;

  class Ticker
      :    public wrapper<Ticker>
  {
  private:
      bool run_;
      volatile bool * running_;
      boost::thread * thread_;
      boost::xtime xt_;
      PyGILState_STATE state_;
  public:
      Ticker() :running_(&run_) { *running_ = false; }

      void operator()()
      {
          while (*running_)
          {
              boost::xtime_get(&xt_, boost::TIME_UTC);
              ++xt_.sec;
              boost::thread::sleep(xt_);
              state_ = PyGILState_Ensure();
              onTick();
              PyGILState_Release(state_);
          }
      }

      void run()
      {
          if (*running_ == false)
          {
              *running_ = true;
              thread_ = new boost::thread(*this);
          }
      }

      void stop()
      {
          if (*running_ == true)
          {
              *running_ = false;
              thread_->join();
              delete thread_;
          }
      }

      virtual void onTick() { get_override("onTick")(); }
      void default_onTick() {}
  };

  BOOST_PYTHON_MODULE(tick)
  {
      class_<Ticker, boost::noncopyable> ("Ticker")
          .def("run", &Ticker::run)
          .def("stop", &Ticker::stop)
          .def("onTick", &Ticker::default_onTick);
  }


Thanks again,

Paul


"Renato Araujo" <renatox at gmail.com> wrote in message 
news:95291a80907060619u5bff0dcey56947d1ac848cc8e at mail.gmail.com...
I'm using gcc/linux and python >= 2.4 and works fine for me.



On Mon, Jul 6, 2009 at 7:39 AM, Paul Scruby<paul at gingernut.tv> wrote:
> Hello again,
>
> Sorry, I spoke too soon. The good news is that wrapping my virtual method
> calls into Python between PyGILState_Ensure() and PyGILState_Release() 
> fixed
> the crash under Python2.6.2 on Windows, but not when I tested it again 
> under
> Python 2.4.4 on Solaris. Under Python 2.4.4 on SolarisSolaris it now calls
> and exits the virtual method in Python sucessfully, but then crashes when
> C++ trys to release the global interpretor lock.
>
> The call-stack with the Sun C++ 5.9 compiler under Solaris is
> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid
> at 0xfec453ed
> 0xfec453ed: sem_invalid+0x0013: movzwl 0x00000006(%eax),%eax
> Current function is Ticker::operator()
> 28 PyGILState_Release(state_);
> (dbx) where
> current thread: t at 2
> [1] sem_invalid(0x0), at 0xfec453ed
> [2] _sem_post(0x0), at 0xfec454c4
> [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
> [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
> [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
> 0x0, 0x80a3140), at 0xfef43eba
> [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
> [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56
> in "thread.hpp"
> [9] thread_proxy(0x810a288), at 0xfea78ce4
> [10] _thr_setup(0xfe670200), at 0xfee159b9
> [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
> 0xfe77ef14), at 0xfee15ca0
>
> Do you think it's worth repeating this test using gcc/linux, or do you 
> think
> that this is just a limitation of using Python with threads?
>
> Thanks again,
>
> Paul
>
>
> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid at
> 0xfec453ed
> 0xfec453ed: sem_invalid+0x0013: movzwl 0x00000006(%eax),%eax
> Current function is Ticker::operator()
> 28 PyGILState_Release(state_);
> (dbx) where
> current thread: t at 2
> [1] sem_invalid(0x0), at 0xfec453ed
> [2] _sem_post(0x0), at 0xfec454c4
> [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
> [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
> [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
> 0x0, 0x80a3140), at 0xfef43eba
> [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
> [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56 in
> "thread.hpp"
> [9] thread_proxy(0x810a288), at 0xfea78ce4
> [10] _thr_setup(0xfe670200), at 0xfee159b9
> [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
> 0xfe77ef14), at 0xfee15ca0
>
>
>
> "Paul Scruby" <paul at gingernut.tv> wrote in message
> news:h2sgic$ad4$1 at ger.gmane.org...
>> Hiya,
>>
>> That's fantastic, all I needed to do was to put PyGILState_Ensure();
>> before my virtual function calls into python from another thread and my
>> program no longer crashes. Problem solved, isn't boost::python great!
>>
>> Many thanks,
>>
>> Paul
>>
>>
>> "Renato Araujo" <renatox at gmail.com> wrote in message
>> news:95291a80907041315k41b7ad88o32d2111ae8fe1e91 at mail.gmail.com...
>> Hi Paul
>>
>> In my bindings I had a problem like this, to solve I created a simple
>> class like that:
>>
>> class thread_locker
>> {
>> thread_locker()
>> {
>> if (thread_support::enabled())
>> m_gstate = PyGILState_Ensure();
>> }
>>
>> ~thread_locker()
>> {
>> if (thread_support::enabled())
>> PyGILState_Release(m_gstate);
>> }
>> };
>>
>> then in my wrapper virtual implementation I did this:
>>
>> ...
>> void wrapper::virtual_func(..)
>> {
>> thread_locker lock;
>> .. my code ..
>> }
>> ....
>>
>> this solve my problems with call of virtual functions in thread
>> enviroment.
>>
>> BR
>>
>>
>>
>>
>> On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<wladwig at wdtinc.com> wrote:
>>> Whoops, I think this problem is a little uglier than I thought, since 
>>> you
>>> overrode the onTick() function in python with a call to print, which
>>> needs access to the interpreter in your new thread. See the link Thomas
>>> posted for dealing with the GIL (along with worrying about any possible
>>> garbage collection issues). Now I remember why I kept my C++ threads
>>> isolated from Python stuff....sorry, it's been a while....
>>>
>>> Bill
>>> ________________________________________
>>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org
>>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of
>>> William Ladwig [wladwig at wdtinc.com]
>>> Sent: Saturday, July 04, 2009 1:34 PM
>>> To: Development of Python/C++ integration
>>> Subject: Re: [C++-sig] boost::python and threads
>>>
>>> It looks to me like you have a garbage collection problem going on. If
>>> you create a wrapped c++ object in python, then python is going to own
>>> the object and will destroy it when its reference count goes to 0. In
>>> your python example script at the bottom, you call the Ticker's run()
>>> function, which from the python point of view, returns quickly and the
>>> script ends. Python has no idea that you spawned off a new thread from
>>> the C++ side, so when the script ends, python destroys the object and 
>>> now
>>> you have a problem. One way that you can check to see if this is what is
>>> going on is to add this to the bottom of the test script and see if the
>>> crashes go away:
>>>
>>> # Warning, you'll need to kill this script manually
>>> import time
>>> while True:
>>> time.sleep(1)
>>>
>>> Generally when I want to fire off a new C++ thread, I hold any objects
>>> that the thread needs in an auto_ptr (see held type for class wrappers),
>>> take ownership of them in C++ (see the FAQ in the documentation) and
>>> start the thread from the C++ side. Or, you can also create C++ wrappers
>>> which accept shared_ptr arguments, while holding your classes in
>>> shared_ptrs, and this should handle the reference counting as well.
>>> Unfortunately, this may require some interface changes to what you have
>>> already written (or possibly some clever wrapping).
>>>
>>> Hope this helps,
>>> Bill
>>>
>>>
>>> ________________________________________
>>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org
>>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul
>>> Scruby [paul at gingernut.tv]
>>> Sent: Friday, July 03, 2009 6:15 AM
>>> To: cplusplus-sig at python.org
>>> Subject: [C++-sig] boost::python and threads
>>>
>>> I am having some problems using boost::python with boost::thread. I'm
>>> using
>>> threads because I want to run some tasks in the background when I'm 
>>> using
>>> the Python's interactive shell. However, when I use get_override() to
>>> call
>>> a Python method from another boost::thread it crashes internally. For
>>> example:
>>>
>>> #include <boost/python.hpp>
>>> #include <boost/thread/thread.hpp>
>>> #include <boost/thread/xtime.hpp>
>>>
>>> using namespace boost::python;
>>>
>>> class Ticker
>>> : public wrapper<Ticker>
>>> {
>>> private:
>>> bool run_;
>>> volatile bool * running_;
>>> boost::thread * thread_;
>>> boost::xtime xt_;
>>> public:
>>> Ticker() : running_(&run_) { *running_ = false; }
>>>
>>> void operator()()
>>> {
>>> while (*running_)
>>> {
>>> boost::xtime_get(&xt_, boost::TIME_UTC);
>>> ++xt_.sec;
>>> boost::thread::sleep(xt_);
>>> onTick();
>>> }
>>> }
>>>
>>> void run()
>>> {
>>> if (*running_ == false)
>>> {
>>> *running_ = true;
>>> thread_ = new boost::thread(*this);
>>> }
>>> }
>>>
>>> void stop()
>>> {
>>> if (*running_ == true)
>>> {
>>> *running_ = false;
>>> thread_->join();
>>> delete thread_;
>>> }
>>> }
>>>
>>> virtual void onTick() { get_override("onTick")(); }
>>> void default_onTick() {}
>>> };
>>>
>>> BOOST_PYTHON_MODULE(tick)
>>> {
>>> class_<Ticker, boost::noncopyable> ("Ticker")
>>> .def("run", &Ticker::run)
>>> .def("stop", &Ticker::stop)
>>> .def("onTick", &Ticker::default_onTick);
>>> }
>>>
>>> Here is a test script that which will crash when you import it into
>>> Python's
>>> interactive shell.
>>>
>>> from tick import Ticker
>>>
>>> class MyTicker(Ticker):
>>> def onTick(self):
>>> print "Each second"
>>>
>>> myticker = MyTicker()
>>> myticker.run()
>>>
>>> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler
>>> on
>>> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008
>>> on
>>> Windows XP.
>>>
>>> The call-stack in dbx on Solaris:
>>>
>>> >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
>>> PyErr_Restore at 0xfef38fa1
>>> 0xfef38fa1: PyErr_Restore+0x0031: movl 0x00000028(%edi),%ecx
>>> Current function is boost::python::override::operator()
>>> 99 detail::method_result x(
>>>
>>> (dbx) where
>>> current thread: t at 2
>>> [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
>>> 0x80652fc), at 0xfef38fa1
>>> [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
>>> [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
>>> [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
>>> [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
>>> 0xfef2bf02
>>> [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
>>> =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
>>> "override.hpp"
>>> [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
>>> [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
>>> [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
>>> 56 in "thread.hpp"
>>> [11] thread_proxy(0x810a288), at 0xfea78ce4
>>> [12] _thr_setup(0xfe670200), at 0xfee159b9
>>> [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
>>> 0x80652fc, 0x80f1220), at 0xfee15ca0
>>>
>>> The call-stack in Visual Studio 2008:
>>>
>>> python26.dll!1e013595()
>>> [Frames below may be incorrect and/or missing, no symbols loaded for
>>> python26.dll]
>>> python26.dll!1e09ee7d()
>>> > tick.pyd!boost::python::override::operator()() Line 103 + 0x16 bytes
>>> C++
>>> 00f3fd64()
>>> tick.pyd!Ticker::operator()() Line 27 + 0xe bytes C++
>>> tick.pyd!boost::detail::thread_data<Ticker>::run() Line 57 C++
>>> tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
>>> param=0x00245f30) Line 168 C++
>>> msvcr90d.dll!_callthreadstartex() Line 348 + 0xf bytes C
>>> msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) Line 331 C
>>> kernel32.dll!7c80b729()
>>>
>>>
>>> Have a missed a trick using the wrapper, or does boost::python not
>>> support
>>> threading?
>>>
>>> Many thanks,
>>>
>>> Paul
>>>
>>>
>>>
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>
>>
>>
>>
>> --
>> Renato Araujo Oliveira Filho
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho 




From renatox at gmail.com  Wed Jul  8 19:23:53 2009
From: renatox at gmail.com (Renato Araujo)
Date: Wed, 8 Jul 2009 14:23:53 -0300
Subject: [C++-sig] boost::python and threads
In-Reply-To: <h32f4m$8ip$1@ger.gmane.org>
References: <h2kp7s$m69$1@ger.gmane.org>
	<765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com>
	<765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com>
	<95291a80907041315k41b7ad88o32d2111ae8fe1e91@mail.gmail.com>
	<h2sgic$ad4$1@ger.gmane.org> <h2ska4$l0m$1@ger.gmane.org>
	<95291a80907060619u5bff0dcey56947d1ac848cc8e@mail.gmail.com>
	<h32f4m$8ip$1@ger.gmane.org>
Message-ID: <95291a80907081023w5c1959d4u9bb7d5a4258d9240@mail.gmail.com>

Ok I made 2 modifications and I got this working here.

Try this.

51,54c50
<      virtual void onTick() {
<          if (object o = get_override("onTick"))
<              o();
<       }
---
>      virtual void onTick() { get_override("onTick")(); }
60d55
<      PyEval_InitThreads();
65a61
>

BR




On Wed, Jul 8, 2009 at 12:48 PM, Paul Scruby<paul at gingernut.tv> wrote:
> Hi Renato,
>
> Okay, I have installed Python 2.4.2 with gcc 4.1.2 on Linux and it's
> crashing in the same place as on Solaris.
>
> ?Program received signal SIGSEGV, Segmentation fault.
> ?[Switching to Thread 0x41074940 (LWP 12004)]
> ?0x000000307740c850 in sem_post () from /lib64/libpthread.so.0
> ?(gdb) where
> ?#0 ?0x000000307740c850 in sem_post () from /lib64/libpthread.so.0
> ?#1 ?0x000000308b2b78e9 in PyThread_release_lock ()
> ? ? from /usr/lib64/libpython2.4.so.1.0
> ?#2 ?0x00002b46531d9f82 in Ticker::operator() (this=0xbd0f478) at
> ticker.cc:28
> ?#3 ?0x00002b46531d9fb0 in boost::detail::thread_data<Ticker>::run (
> ? ? ?this=0xbd0f370)
> ? ? ?at /opt/atm/include/boost-1_39/boost/thread/detail/thread.hpp:56
> ?#4 ?0x00002b46533ee14b in thread_proxy ()
> ? ? from /opt/atm/lib64/libboost_thread-gcc41-mt-1_39.so.1.39.0
> ?#5 ?0x0000003077406367 in start_thread () from /lib64/libpthread.so.0
> ?#6 ?0x00000030768d2f7d in clone () from /lib64/libc.so.6
>
> With the global interpret lock added my code now looks like this...
>
> ?#include <boost/python.hpp>
> ?#include <boost/thread/thread.hpp>
> ?#include <boost/thread/xtime.hpp>
> ?using namespace boost::python;
>
> ?class Ticker
> ? ? ?: ? ?public wrapper<Ticker>
> ?{
> ?private:
> ? ? ?bool run_;
> ? ? ?volatile bool * running_;
> ? ? ?boost::thread * thread_;
> ? ? ?boost::xtime xt_;
> ? ? ?PyGILState_STATE state_;
> ?public:
> ? ? ?Ticker() :running_(&run_) { *running_ = false; }
>
> ? ? ?void operator()()
> ? ? ?{
> ? ? ? ? ?while (*running_)
> ? ? ? ? ?{
> ? ? ? ? ? ? ?boost::xtime_get(&xt_, boost::TIME_UTC);
> ? ? ? ? ? ? ?++xt_.sec;
> ? ? ? ? ? ? ?boost::thread::sleep(xt_);
> ? ? ? ? ? ? ?state_ = PyGILState_Ensure();
> ? ? ? ? ? ? ?onTick();
> ? ? ? ? ? ? ?PyGILState_Release(state_);
> ? ? ? ? ?}
> ? ? ?}
>
> ? ? ?void run()
> ? ? ?{
> ? ? ? ? ?if (*running_ == false)
> ? ? ? ? ?{
> ? ? ? ? ? ? ?*running_ = true;
> ? ? ? ? ? ? ?thread_ = new boost::thread(*this);
> ? ? ? ? ?}
> ? ? ?}
>
> ? ? ?void stop()
> ? ? ?{
> ? ? ? ? ?if (*running_ == true)
> ? ? ? ? ?{
> ? ? ? ? ? ? ?*running_ = false;
> ? ? ? ? ? ? ?thread_->join();
> ? ? ? ? ? ? ?delete thread_;
> ? ? ? ? ?}
> ? ? ?}
>
> ? ? ?virtual void onTick() { get_override("onTick")(); }
> ? ? ?void default_onTick() {}
> ?};
>
> ?BOOST_PYTHON_MODULE(tick)
> ?{
> ? ? ?class_<Ticker, boost::noncopyable> ("Ticker")
> ? ? ? ? ?.def("run", &Ticker::run)
> ? ? ? ? ?.def("stop", &Ticker::stop)
> ? ? ? ? ?.def("onTick", &Ticker::default_onTick);
> ?}
>
>
> Thanks again,
>
> Paul
>
>
> "Renato Araujo" <renatox at gmail.com> wrote in message
> news:95291a80907060619u5bff0dcey56947d1ac848cc8e at mail.gmail.com...
> I'm using gcc/linux and python >= 2.4 and works fine for me.
>
>
>
> On Mon, Jul 6, 2009 at 7:39 AM, Paul Scruby<paul at gingernut.tv> wrote:
>> Hello again,
>>
>> Sorry, I spoke too soon. The good news is that wrapping my virtual method
>> calls into Python between PyGILState_Ensure() and PyGILState_Release()
>> fixed
>> the crash under Python2.6.2 on Windows, but not when I tested it again
>> under
>> Python 2.4.4 on Solaris. Under Python 2.4.4 on SolarisSolaris it now calls
>> and exits the virtual method in Python sucessfully, but then crashes when
>> C++ trys to release the global interpretor lock.
>>
>> The call-stack with the Sun C++ 5.9 compiler under Solaris is
>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid
>> at 0xfec453ed
>> 0xfec453ed: sem_invalid+0x0013: movzwl 0x00000006(%eax),%eax
>> Current function is Ticker::operator()
>> 28 PyGILState_Release(state_);
>> (dbx) where
>> current thread: t at 2
>> [1] sem_invalid(0x0), at 0xfec453ed
>> [2] _sem_post(0x0), at 0xfec454c4
>> [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
>> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
>> [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
>> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
>> [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
>> 0x0, 0x80a3140), at 0xfef43eba
>> [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
>> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
>> [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56
>> in "thread.hpp"
>> [9] thread_proxy(0x810a288), at 0xfea78ce4
>> [10] _thr_setup(0xfe670200), at 0xfee159b9
>> [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
>> 0xfe77ef14), at 0xfee15ca0
>>
>> Do you think it's worth repeating this test using gcc/linux, or do you
>> think
>> that this is just a limitation of using Python with threads?
>>
>> Thanks again,
>>
>> Paul
>>
>>
>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid at
>> 0xfec453ed
>> 0xfec453ed: sem_invalid+0x0013: movzwl 0x00000006(%eax),%eax
>> Current function is Ticker::operator()
>> 28 PyGILState_Release(state_);
>> (dbx) where
>> current thread: t at 2
>> [1] sem_invalid(0x0), at 0xfec453ed
>> [2] _sem_post(0x0), at 0xfec454c4
>> [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
>> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
>> [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
>> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
>> [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
>> 0x0, 0x80a3140), at 0xfef43eba
>> [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
>> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
>> [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56 in
>> "thread.hpp"
>> [9] thread_proxy(0x810a288), at 0xfea78ce4
>> [10] _thr_setup(0xfe670200), at 0xfee159b9
>> [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
>> 0xfe77ef14), at 0xfee15ca0
>>
>>
>>
>> "Paul Scruby" <paul at gingernut.tv> wrote in message
>> news:h2sgic$ad4$1 at ger.gmane.org...
>>> Hiya,
>>>
>>> That's fantastic, all I needed to do was to put PyGILState_Ensure();
>>> before my virtual function calls into python from another thread and my
>>> program no longer crashes. Problem solved, isn't boost::python great!
>>>
>>> Many thanks,
>>>
>>> Paul
>>>
>>>
>>> "Renato Araujo" <renatox at gmail.com> wrote in message
>>> news:95291a80907041315k41b7ad88o32d2111ae8fe1e91 at mail.gmail.com...
>>> Hi Paul
>>>
>>> In my bindings I had a problem like this, to solve I created a simple
>>> class like that:
>>>
>>> class thread_locker
>>> {
>>> thread_locker()
>>> {
>>> if (thread_support::enabled())
>>> m_gstate = PyGILState_Ensure();
>>> }
>>>
>>> ~thread_locker()
>>> {
>>> if (thread_support::enabled())
>>> PyGILState_Release(m_gstate);
>>> }
>>> };
>>>
>>> then in my wrapper virtual implementation I did this:
>>>
>>> ...
>>> void wrapper::virtual_func(..)
>>> {
>>> thread_locker lock;
>>> .. my code ..
>>> }
>>> ....
>>>
>>> this solve my problems with call of virtual functions in thread
>>> enviroment.
>>>
>>> BR
>>>
>>>
>>>
>>>
>>> On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<wladwig at wdtinc.com> wrote:
>>>> Whoops, I think this problem is a little uglier than I thought, since
>>>> you
>>>> overrode the onTick() function in python with a call to print, which
>>>> needs access to the interpreter in your new thread. See the link Thomas
>>>> posted for dealing with the GIL (along with worrying about any possible
>>>> garbage collection issues). Now I remember why I kept my C++ threads
>>>> isolated from Python stuff....sorry, it's been a while....
>>>>
>>>> Bill
>>>> ________________________________________
>>>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org
>>>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of
>>>> William Ladwig [wladwig at wdtinc.com]
>>>> Sent: Saturday, July 04, 2009 1:34 PM
>>>> To: Development of Python/C++ integration
>>>> Subject: Re: [C++-sig] boost::python and threads
>>>>
>>>> It looks to me like you have a garbage collection problem going on. If
>>>> you create a wrapped c++ object in python, then python is going to own
>>>> the object and will destroy it when its reference count goes to 0. In
>>>> your python example script at the bottom, you call the Ticker's run()
>>>> function, which from the python point of view, returns quickly and the
>>>> script ends. Python has no idea that you spawned off a new thread from
>>>> the C++ side, so when the script ends, python destroys the object and
>>>> now
>>>> you have a problem. One way that you can check to see if this is what is
>>>> going on is to add this to the bottom of the test script and see if the
>>>> crashes go away:
>>>>
>>>> # Warning, you'll need to kill this script manually
>>>> import time
>>>> while True:
>>>> time.sleep(1)
>>>>
>>>> Generally when I want to fire off a new C++ thread, I hold any objects
>>>> that the thread needs in an auto_ptr (see held type for class wrappers),
>>>> take ownership of them in C++ (see the FAQ in the documentation) and
>>>> start the thread from the C++ side. Or, you can also create C++ wrappers
>>>> which accept shared_ptr arguments, while holding your classes in
>>>> shared_ptrs, and this should handle the reference counting as well.
>>>> Unfortunately, this may require some interface changes to what you have
>>>> already written (or possibly some clever wrapping).
>>>>
>>>> Hope this helps,
>>>> Bill
>>>>
>>>>
>>>> ________________________________________
>>>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org
>>>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul
>>>> Scruby [paul at gingernut.tv]
>>>> Sent: Friday, July 03, 2009 6:15 AM
>>>> To: cplusplus-sig at python.org
>>>> Subject: [C++-sig] boost::python and threads
>>>>
>>>> I am having some problems using boost::python with boost::thread. I'm
>>>> using
>>>> threads because I want to run some tasks in the background when I'm
>>>> using
>>>> the Python's interactive shell. However, when I use get_override() to
>>>> call
>>>> a Python method from another boost::thread it crashes internally. For
>>>> example:
>>>>
>>>> #include <boost/python.hpp>
>>>> #include <boost/thread/thread.hpp>
>>>> #include <boost/thread/xtime.hpp>
>>>>
>>>> using namespace boost::python;
>>>>
>>>> class Ticker
>>>> : public wrapper<Ticker>
>>>> {
>>>> private:
>>>> bool run_;
>>>> volatile bool * running_;
>>>> boost::thread * thread_;
>>>> boost::xtime xt_;
>>>> public:
>>>> Ticker() : running_(&run_) { *running_ = false; }
>>>>
>>>> void operator()()
>>>> {
>>>> while (*running_)
>>>> {
>>>> boost::xtime_get(&xt_, boost::TIME_UTC);
>>>> ++xt_.sec;
>>>> boost::thread::sleep(xt_);
>>>> onTick();
>>>> }
>>>> }
>>>>
>>>> void run()
>>>> {
>>>> if (*running_ == false)
>>>> {
>>>> *running_ = true;
>>>> thread_ = new boost::thread(*this);
>>>> }
>>>> }
>>>>
>>>> void stop()
>>>> {
>>>> if (*running_ == true)
>>>> {
>>>> *running_ = false;
>>>> thread_->join();
>>>> delete thread_;
>>>> }
>>>> }
>>>>
>>>> virtual void onTick() { get_override("onTick")(); }
>>>> void default_onTick() {}
>>>> };
>>>>
>>>> BOOST_PYTHON_MODULE(tick)
>>>> {
>>>> class_<Ticker, boost::noncopyable> ("Ticker")
>>>> .def("run", &Ticker::run)
>>>> .def("stop", &Ticker::stop)
>>>> .def("onTick", &Ticker::default_onTick);
>>>> }
>>>>
>>>> Here is a test script that which will crash when you import it into
>>>> Python's
>>>> interactive shell.
>>>>
>>>> from tick import Ticker
>>>>
>>>> class MyTicker(Ticker):
>>>> def onTick(self):
>>>> print "Each second"
>>>>
>>>> myticker = MyTicker()
>>>> myticker.run()
>>>>
>>>> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler
>>>> on
>>>> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008
>>>> on
>>>> Windows XP.
>>>>
>>>> The call-stack in dbx on Solaris:
>>>>
>>>> >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
>>>> PyErr_Restore at 0xfef38fa1
>>>> 0xfef38fa1: PyErr_Restore+0x0031: movl 0x00000028(%edi),%ecx
>>>> Current function is boost::python::override::operator()
>>>> 99 detail::method_result x(
>>>>
>>>> (dbx) where
>>>> current thread: t at 2
>>>> [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
>>>> 0x80652fc), at 0xfef38fa1
>>>> [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
>>>> [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
>>>> [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
>>>> [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
>>>> 0xfef2bf02
>>>> [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
>>>> =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
>>>> "override.hpp"
>>>> [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
>>>> [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
>>>> [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
>>>> 56 in "thread.hpp"
>>>> [11] thread_proxy(0x810a288), at 0xfea78ce4
>>>> [12] _thr_setup(0xfe670200), at 0xfee159b9
>>>> [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
>>>> 0x80652fc, 0x80f1220), at 0xfee15ca0
>>>>
>>>> The call-stack in Visual Studio 2008:
>>>>
>>>> python26.dll!1e013595()
>>>> [Frames below may be incorrect and/or missing, no symbols loaded for
>>>> python26.dll]
>>>> python26.dll!1e09ee7d()
>>>> > tick.pyd!boost::python::override::operator()() Line 103 + 0x16 bytes
>>>> C++
>>>> 00f3fd64()
>>>> tick.pyd!Ticker::operator()() Line 27 + 0xe bytes C++
>>>> tick.pyd!boost::detail::thread_data<Ticker>::run() Line 57 C++
>>>> tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
>>>> param=0x00245f30) Line 168 C++
>>>> msvcr90d.dll!_callthreadstartex() Line 348 + 0xf bytes C
>>>> msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) Line 331 C
>>>> kernel32.dll!7c80b729()
>>>>
>>>>
>>>> Have a missed a trick using the wrapper, or does boost::python not
>>>> support
>>>> threading?
>>>>
>>>> Many thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Cplusplus-sig mailing list
>>>> Cplusplus-sig at python.org
>>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>> _______________________________________________
>>>> Cplusplus-sig mailing list
>>>> Cplusplus-sig at python.org
>>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>> _______________________________________________
>>>> Cplusplus-sig mailing list
>>>> Cplusplus-sig at python.org
>>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>>
>>>
>>>
>>>
>>> --
>>> Renato Araujo Oliveira Filho
>>
>>
>>
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>
>
>
>
> --
> Renato Araujo Oliveira Filho
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho

From xavier.benech at t-immersion.com  Thu Jul  9 00:38:40 2009
From: xavier.benech at t-immersion.com (=?ISO-8859-1?Q?Xavier_B=E9nech?=)
Date: Thu, 09 Jul 2009 00:38:40 +0200
Subject: [C++-sig] sys.exit() and PyRun_SimpleFileExFlags()
Message-ID: <4A551FF0.2040408@t-immersion.com>


Hi,

There is a behaviour I do not understand of PyRun_SimpleFileExFlags(), 
normally when it executes a python script containing a sys.exit(), it 
results by ending the calling application.

I have got this behaviour with PyRun_SimpleFileExFlags() when I call it 
from the main thread of a GUI application (so I use PyRun_FileExFlags() 
in this case).

But in another application when it is called 
(PyRun_SimpleFileExFlags())) from a working thread (not the main one) of 
a console application, everything just go fine without exiting the 
application nor the thread. It returns the exit code specified in 
sys.exit() just fine.

Last things, I am working on Windows. And for the story, I started to 
make a small function to wrap some embedded python script call using the 
simple way with PyRun_SimpleFileExFlags() for the second application, 
but when I integrate it in the first one (with GUI) got some problems 
(:)) so I redo things with a cleaner PyRun_FileExFlags() call.

So, my question is: why do I have different behaviour of 
PyRun_SimpleFileExFlags() in this two cases?

Regards,
Xavier Benech.


From paul at gingernut.tv  Thu Jul  9 20:46:57 2009
From: paul at gingernut.tv (Paul Scruby)
Date: Thu, 9 Jul 2009 19:46:57 +0100
Subject: [C++-sig] boost::python and threads
References: <h2kp7s$m69$1@ger.gmane.org><765CBD9053EA2B438625895F30D1856F023806D4D3@storm.wdtinc.com><765CBD9053EA2B438625895F30D1856F023806D4D4@storm.wdtinc.com><95291a80907041315k41b7ad88o32d2111ae8fe1e91@mail.gmail.com><h2sgic$ad4$1@ger.gmane.org>
	<h2ska4$l0m$1@ger.gmane.org><95291a80907060619u5bff0dcey56947d1ac848cc8e@mail.gmail.com><h32f4m$8ip$1@ger.gmane.org>
	<95291a80907081023w5c1959d4u9bb7d5a4258d9240@mail.gmail.com>
Message-ID: <h35dvd$t9r$1@ger.gmane.org>

Hi Renato,

It's all working now, tested on gcc, sun C++ and visual studio.

Thanks again,

Paul


"Renato Araujo" <renatox at gmail.com> wrote in message 
news:95291a80907081023w5c1959d4u9bb7d5a4258d9240 at mail.gmail.com...
Ok I made 2 modifications and I got this working here.

Try this.

51,54c50
<      virtual void onTick() {
<          if (object o = get_override("onTick"))
<              o();
<       }
---
>      virtual void onTick() { get_override("onTick")(); }
60d55
<      PyEval_InitThreads();
65a61
>

BR




On Wed, Jul 8, 2009 at 12:48 PM, Paul Scruby<paul at gingernut.tv> wrote:
> Hi Renato,
>
> Okay, I have installed Python 2.4.2 with gcc 4.1.2 on Linux and it's
> crashing in the same place as on Solaris.
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x41074940 (LWP 12004)]
> 0x000000307740c850 in sem_post () from /lib64/libpthread.so.0
> (gdb) where
> #0 0x000000307740c850 in sem_post () from /lib64/libpthread.so.0
> #1 0x000000308b2b78e9 in PyThread_release_lock ()
> from /usr/lib64/libpython2.4.so.1.0
> #2 0x00002b46531d9f82 in Ticker::operator() (this=0xbd0f478) at
> ticker.cc:28
> #3 0x00002b46531d9fb0 in boost::detail::thread_data<Ticker>::run (
> this=0xbd0f370)
> at /opt/atm/include/boost-1_39/boost/thread/detail/thread.hpp:56
> #4 0x00002b46533ee14b in thread_proxy ()
> from /opt/atm/lib64/libboost_thread-gcc41-mt-1_39.so.1.39.0
> #5 0x0000003077406367 in start_thread () from /lib64/libpthread.so.0
> #6 0x00000030768d2f7d in clone () from /lib64/libc.so.6
>
> With the global interpret lock added my code now looks like this...
>
> #include <boost/python.hpp>
> #include <boost/thread/thread.hpp>
> #include <boost/thread/xtime.hpp>
> using namespace boost::python;
>
> class Ticker
> : public wrapper<Ticker>
> {
> private:
> bool run_;
> volatile bool * running_;
> boost::thread * thread_;
> boost::xtime xt_;
> PyGILState_STATE state_;
> public:
> Ticker() :running_(&run_) { *running_ = false; }
>
> void operator()()
> {
> while (*running_)
> {
> boost::xtime_get(&xt_, boost::TIME_UTC);
> ++xt_.sec;
> boost::thread::sleep(xt_);
> state_ = PyGILState_Ensure();
> onTick();
> PyGILState_Release(state_);
> }
> }
>
> void run()
> {
> if (*running_ == false)
> {
> *running_ = true;
> thread_ = new boost::thread(*this);
> }
> }
>
> void stop()
> {
> if (*running_ == true)
> {
> *running_ = false;
> thread_->join();
> delete thread_;
> }
> }
>
> virtual void onTick() { get_override("onTick")(); }
> void default_onTick() {}
> };
>
> BOOST_PYTHON_MODULE(tick)
> {
> class_<Ticker, boost::noncopyable> ("Ticker")
> .def("run", &Ticker::run)
> .def("stop", &Ticker::stop)
> .def("onTick", &Ticker::default_onTick);
> }
>
>
> Thanks again,
>
> Paul
>
>
> "Renato Araujo" <renatox at gmail.com> wrote in message
> news:95291a80907060619u5bff0dcey56947d1ac848cc8e at mail.gmail.com...
> I'm using gcc/linux and python >= 2.4 and works fine for me.
>
>
>
> On Mon, Jul 6, 2009 at 7:39 AM, Paul Scruby<paul at gingernut.tv> wrote:
>> Hello again,
>>
>> Sorry, I spoke too soon. The good news is that wrapping my virtual method
>> calls into Python between PyGILState_Ensure() and PyGILState_Release()
>> fixed
>> the crash under Python2.6.2 on Windows, but not when I tested it again
>> under
>> Python 2.4.4 on Solaris. Under Python 2.4.4 on SolarisSolaris it now 
>> calls
>> and exits the virtual method in Python sucessfully, but then crashes when
>> C++ trys to release the global interpretor lock.
>>
>> The call-stack with the Sun C++ 5.9 compiler under Solaris is
>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid
>> at 0xfec453ed
>> 0xfec453ed: sem_invalid+0x0013: movzwl 0x00000006(%eax),%eax
>> Current function is Ticker::operator()
>> 28 PyGILState_Release(state_);
>> (dbx) where
>> current thread: t at 2
>> [1] sem_invalid(0x0), at 0xfec453ed
>> [2] _sem_post(0x0), at 0xfec454c4
>> [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
>> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
>> [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
>> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
>> [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
>> 0x0, 0x80a3140), at 0xfef43eba
>> [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
>> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
>> [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56
>> in "thread.hpp"
>> [9] thread_proxy(0x810a288), at 0xfea78ce4
>> [10] _thr_setup(0xfe670200), at 0xfee159b9
>> [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
>> 0xfe77ef14), at 0xfee15ca0
>>
>> Do you think it's worth repeating this test using gcc/linux, or do you
>> think
>> that this is just a limitation of using Python with threads?
>>
>> Thanks again,
>>
>> Paul
>>
>>
>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in sem_invalid at
>> 0xfec453ed
>> 0xfec453ed: sem_invalid+0x0013: movzwl 0x00000006(%eax),%eax
>> Current function is Ticker::operator()
>> 28 PyGILState_Release(state_);
>> (dbx) where
>> current thread: t at 2
>> [1] sem_invalid(0x0), at 0xfec453ed
>> [2] _sem_post(0x0), at 0xfec454c4
>> [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
>> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
>> [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
>> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
>> [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
>> 0x0, 0x80a3140), at 0xfef43eba
>> [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
>> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
>> [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56 in
>> "thread.hpp"
>> [9] thread_proxy(0x810a288), at 0xfea78ce4
>> [10] _thr_setup(0xfe670200), at 0xfee159b9
>> [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
>> 0xfe77ef14), at 0xfee15ca0
>>
>>
>>
>> "Paul Scruby" <paul at gingernut.tv> wrote in message
>> news:h2sgic$ad4$1 at ger.gmane.org...
>>> Hiya,
>>>
>>> That's fantastic, all I needed to do was to put PyGILState_Ensure();
>>> before my virtual function calls into python from another thread and my
>>> program no longer crashes. Problem solved, isn't boost::python great!
>>>
>>> Many thanks,
>>>
>>> Paul
>>>
>>>
>>> "Renato Araujo" <renatox at gmail.com> wrote in message
>>> news:95291a80907041315k41b7ad88o32d2111ae8fe1e91 at mail.gmail.com...
>>> Hi Paul
>>>
>>> In my bindings I had a problem like this, to solve I created a simple
>>> class like that:
>>>
>>> class thread_locker
>>> {
>>> thread_locker()
>>> {
>>> if (thread_support::enabled())
>>> m_gstate = PyGILState_Ensure();
>>> }
>>>
>>> ~thread_locker()
>>> {
>>> if (thread_support::enabled())
>>> PyGILState_Release(m_gstate);
>>> }
>>> };
>>>
>>> then in my wrapper virtual implementation I did this:
>>>
>>> ...
>>> void wrapper::virtual_func(..)
>>> {
>>> thread_locker lock;
>>> .. my code ..
>>> }
>>> ....
>>>
>>> this solve my problems with call of virtual functions in thread
>>> enviroment.
>>>
>>> BR
>>>
>>>
>>>
>>>
>>> On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<wladwig at wdtinc.com> 
>>> wrote:
>>>> Whoops, I think this problem is a little uglier than I thought, since
>>>> you
>>>> overrode the onTick() function in python with a call to print, which
>>>> needs access to the interpreter in your new thread. See the link Thomas
>>>> posted for dealing with the GIL (along with worrying about any possible
>>>> garbage collection issues). Now I remember why I kept my C++ threads
>>>> isolated from Python stuff....sorry, it's been a while....
>>>>
>>>> Bill
>>>> ________________________________________
>>>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org
>>>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of
>>>> William Ladwig [wladwig at wdtinc.com]
>>>> Sent: Saturday, July 04, 2009 1:34 PM
>>>> To: Development of Python/C++ integration
>>>> Subject: Re: [C++-sig] boost::python and threads
>>>>
>>>> It looks to me like you have a garbage collection problem going on. If
>>>> you create a wrapped c++ object in python, then python is going to own
>>>> the object and will destroy it when its reference count goes to 0. In
>>>> your python example script at the bottom, you call the Ticker's run()
>>>> function, which from the python point of view, returns quickly and the
>>>> script ends. Python has no idea that you spawned off a new thread from
>>>> the C++ side, so when the script ends, python destroys the object and
>>>> now
>>>> you have a problem. One way that you can check to see if this is what 
>>>> is
>>>> going on is to add this to the bottom of the test script and see if the
>>>> crashes go away:
>>>>
>>>> # Warning, you'll need to kill this script manually
>>>> import time
>>>> while True:
>>>> time.sleep(1)
>>>>
>>>> Generally when I want to fire off a new C++ thread, I hold any objects
>>>> that the thread needs in an auto_ptr (see held type for class 
>>>> wrappers),
>>>> take ownership of them in C++ (see the FAQ in the documentation) and
>>>> start the thread from the C++ side. Or, you can also create C++ 
>>>> wrappers
>>>> which accept shared_ptr arguments, while holding your classes in
>>>> shared_ptrs, and this should handle the reference counting as well.
>>>> Unfortunately, this may require some interface changes to what you have
>>>> already written (or possibly some clever wrapping).
>>>>
>>>> Hope this helps,
>>>> Bill
>>>>
>>>>
>>>> ________________________________________
>>>> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org
>>>> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul
>>>> Scruby [paul at gingernut.tv]
>>>> Sent: Friday, July 03, 2009 6:15 AM
>>>> To: cplusplus-sig at python.org
>>>> Subject: [C++-sig] boost::python and threads
>>>>
>>>> I am having some problems using boost::python with boost::thread. I'm
>>>> using
>>>> threads because I want to run some tasks in the background when I'm
>>>> using
>>>> the Python's interactive shell. However, when I use get_override() to
>>>> call
>>>> a Python method from another boost::thread it crashes internally. For
>>>> example:
>>>>
>>>> #include <boost/python.hpp>
>>>> #include <boost/thread/thread.hpp>
>>>> #include <boost/thread/xtime.hpp>
>>>>
>>>> using namespace boost::python;
>>>>
>>>> class Ticker
>>>> : public wrapper<Ticker>
>>>> {
>>>> private:
>>>> bool run_;
>>>> volatile bool * running_;
>>>> boost::thread * thread_;
>>>> boost::xtime xt_;
>>>> public:
>>>> Ticker() : running_(&run_) { *running_ = false; }
>>>>
>>>> void operator()()
>>>> {
>>>> while (*running_)
>>>> {
>>>> boost::xtime_get(&xt_, boost::TIME_UTC);
>>>> ++xt_.sec;
>>>> boost::thread::sleep(xt_);
>>>> onTick();
>>>> }
>>>> }
>>>>
>>>> void run()
>>>> {
>>>> if (*running_ == false)
>>>> {
>>>> *running_ = true;
>>>> thread_ = new boost::thread(*this);
>>>> }
>>>> }
>>>>
>>>> void stop()
>>>> {
>>>> if (*running_ == true)
>>>> {
>>>> *running_ = false;
>>>> thread_->join();
>>>> delete thread_;
>>>> }
>>>> }
>>>>
>>>> virtual void onTick() { get_override("onTick")(); }
>>>> void default_onTick() {}
>>>> };
>>>>
>>>> BOOST_PYTHON_MODULE(tick)
>>>> {
>>>> class_<Ticker, boost::noncopyable> ("Ticker")
>>>> .def("run", &Ticker::run)
>>>> .def("stop", &Ticker::stop)
>>>> .def("onTick", &Ticker::default_onTick);
>>>> }
>>>>
>>>> Here is a test script that which will crash when you import it into
>>>> Python's
>>>> interactive shell.
>>>>
>>>> from tick import Ticker
>>>>
>>>> class MyTicker(Ticker):
>>>> def onTick(self):
>>>> print "Each second"
>>>>
>>>> myticker = MyTicker()
>>>> myticker.run()
>>>>
>>>> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler
>>>> on
>>>> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008
>>>> on
>>>> Windows XP.
>>>>
>>>> The call-stack in dbx on Solaris:
>>>>
>>>> >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
>>>> PyErr_Restore at 0xfef38fa1
>>>> 0xfef38fa1: PyErr_Restore+0x0031: movl 0x00000028(%edi),%ecx
>>>> Current function is boost::python::override::operator()
>>>> 99 detail::method_result x(
>>>>
>>>> (dbx) where
>>>> current thread: t at 2
>>>> [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
>>>> 0x80652fc), at 0xfef38fa1
>>>> [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
>>>> [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
>>>> [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
>>>> [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
>>>> 0xfef2bf02
>>>> [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
>>>> =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 
>>>> in
>>>> "override.hpp"
>>>> [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
>>>> [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
>>>> [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
>>>> 56 in "thread.hpp"
>>>> [11] thread_proxy(0x810a288), at 0xfea78ce4
>>>> [12] _thr_setup(0xfe670200), at 0xfee159b9
>>>> [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
>>>> 0x80652fc, 0x80f1220), at 0xfee15ca0
>>>>
>>>> The call-stack in Visual Studio 2008:
>>>>
>>>> python26.dll!1e013595()
>>>> [Frames below may be incorrect and/or missing, no symbols loaded for
>>>> python26.dll]
>>>> python26.dll!1e09ee7d()
>>>> > tick.pyd!boost::python::override::operator()() Line 103 + 0x16 bytes
>>>> C++
>>>> 00f3fd64()
>>>> tick.pyd!Ticker::operator()() Line 27 + 0xe bytes C++
>>>> tick.pyd!boost::detail::thread_data<Ticker>::run() Line 57 C++
>>>> tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
>>>> param=0x00245f30) Line 168 C++
>>>> msvcr90d.dll!_callthreadstartex() Line 348 + 0xf bytes C
>>>> msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) Line 331 C
>>>> kernel32.dll!7c80b729()
>>>>
>>>>
>>>> Have a missed a trick using the wrapper, or does boost::python not
>>>> support
>>>> threading?
>>>>
>>>> Many thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Cplusplus-sig mailing list
>>>> Cplusplus-sig at python.org
>>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>> _______________________________________________
>>>> Cplusplus-sig mailing list
>>>> Cplusplus-sig at python.org
>>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>> _______________________________________________
>>>> Cplusplus-sig mailing list
>>>> Cplusplus-sig at python.org
>>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>>
>>>
>>>
>>>
>>> --
>>> Renato Araujo Oliveira Filho
>>
>>
>>
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>
>
>
>
> --
> Renato Araujo Oliveira Filho
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho 




From jvansanten at gmail.com  Sat Jul 11 23:49:36 2009
From: jvansanten at gmail.com (Jakob van Santen)
Date: Sat, 11 Jul 2009 23:49:36 +0200
Subject: [C++-sig] Boost.Python indexing suite version 2?
Message-ID: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>

Hello,

While attempting to wrap a vector of objects where the element class  
doesn't define operator()==, I ran across a mention of a "Version 2"  
of the indexing suite from 2003:

http://mail.python.org/pipermail/cplusplus-sig/2003-October/005453.html

While it's mentioned on the Py++ page, it doesn't seem to have made it  
in to the main boost distribution in the intervening 5 years. Can  
anyone tell me what happened to this project?

Thanks,
Jakob van Santen

From gjcarneiro at gmail.com  Sun Jul 12 18:55:50 2009
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Sun, 12 Jul 2009 17:55:50 +0100
Subject: [C++-sig] Announce: PyBindGen 0.11 released
Message-ID: <a467ca4f0907120955g74e1565fq57d4e938ac05cae5@mail.gmail.com>

To integrate some recent contributions, here's a new PyBindGen 0.11 release.

About: PyBindGen is a Python bindings generator written in Python.

Download:  http://code.google.com/p/pybindgen/

Summary of changes:

    - Generate code that supports GCC's -fvisibility=hidden

    - Add rudimentary support for std::map containers

    - Some inplace numeric operators now supported (J. Michael Owen)

    - Partial sequence protocol support: __len__, __getitem__,
      and __setitem__ (J. Michael Owen)

    - Partially support pointer-to-container parameters

    - Call traceback.extract_stack() less often (faster on win32)

    - No need for GIL locking in attribute setters (thanks Stuart Stock)

    - Allow wrapping enum given list of (name, value) pairs

    - New foreign_cpp_namespace option, for wrapping classes outside
      our module's C++ namespace

    - Add a docstring parameter to CppMethod constructor (Robin Gilks)
      Also add a docstring parameter for CppClass

    - Better support for typedefs

    - Implement support for enum reference parameters

    - Add support for size_t "by-value" type

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090712/70fed8c7/attachment.htm>

From cadchrk at narod.ru  Mon Jul 13 04:19:57 2009
From: cadchrk at narod.ru (Tyomich on the AIR)
Date: Mon, 13 Jul 2009 06:19:57 +0400
Subject: [C++-sig] Boost.Python is slow?
Message-ID: <362261247451597@webmail47.yandex.ru>

Hello everyone! I'm currently embedding Python and have a problem. I have a struct (point) which looks like

struct TPoint
{
    int x, y;
};

And I want it to be usable from Python. I made it in two ways: the first one was writing all the warpping by myself according to Python/C API, and the second was wrap it with Boost.Python. So it looks like

class_<TPoint>("TPoint")
    .def_readwrite("x", &TPoint::x)
    .def_readwrite("y", &TPoint::y);

It wraps ok but here is a problem: I create a huge (1000000 items) list of objects of that type:

def cr_fig(n):
res = []
while n>0:
res.append(TPoint())
n -= 1
return res

And then when I use a simple function which just increments the x member of every object, this function takes about 4 seconds to process the whole list (1000000 items) of Boost-wrapped objects and less than a second to process objects of my own wrapping! How this can be possible? Is Boost.Python much more slower than Python itself?

Regards, Artyom Chirkov.

From kranar at hotmail.com  Mon Jul 13 04:48:31 2009
From: kranar at hotmail.com (Kamal Mansouri)
Date: Sun, 12 Jul 2009 22:48:31 -0400
Subject: [C++-sig] Boost.Python is slow?
In-Reply-To: <362261247451597@webmail47.yandex.ru>
References: <362261247451597@webmail47.yandex.ru>
Message-ID: <SNT105-W587953BD57926EAB6B6EE2CA220@phx.gbl>


For something like this, it would be better to write a function that performs the operation on the million+ Points in C++ and expose that function to Python.  This would be a better alternative to both using Boost.Python or wrapping it by hand.

Kamal

> From: cadchrk at narod.ru
> To: cplusplus-sig at python.org
> Date: Mon, 13 Jul 2009 06:19:57 +0400
> Subject: [C++-sig] Boost.Python is slow?
> 
> Hello everyone! I'm currently embedding Python and have a problem. I have a struct (point) which looks like
> 
> struct TPoint
> {
>     int x, y;
> };
> 
> And I want it to be usable from Python. I made it in two ways: the first one was writing all the warpping by myself according to Python/C API, and the second was wrap it with Boost.Python. So it looks like
> 
> class_<TPoint>("TPoint")
>     .def_readwrite("x", &TPoint::x)
>     .def_readwrite("y", &TPoint::y);
> 
> It wraps ok but here is a problem: I create a huge (1000000 items) list of objects of that type:
> 
> def cr_fig(n):
> res = []
> while n>0:
> res.append(TPoint())
> n -= 1
> return res
> 
> And then when I use a simple function which just increments the x member of every object, this function takes about 4 seconds to process the whole list (1000000 items) of Boost-wrapped objects and less than a second to process objects of my own wrapping! How this can be possible? Is Boost.Python much more slower than Python itself?
> 
> Regards, Artyom Chirkov.
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig

_________________________________________________________________
Attention all humans. We are your photos. Free us.
http://go.microsoft.com/?linkid=9666046
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090712/2966abad/attachment.htm>

From cadchrk at narod.ru  Mon Jul 13 06:40:56 2009
From: cadchrk at narod.ru (Tyomich on the AIR)
Date: Mon, 13 Jul 2009 08:40:56 +0400
Subject: [C++-sig] Boost.Python is slow?
In-Reply-To: <SNT105-W587953BD57926EAB6B6EE2CA220@phx.gbl>
References: <362261247451597@webmail47.yandex.ru>
	<SNT105-W587953BD57926EAB6B6EE2CA220@phx.gbl>
Message-ID: <49011247460056@webmail65.yandex.ru>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090713/af0c8b3e/attachment.htm>

From email at christian-gleinser.de  Tue Jul 14 16:47:48 2009
From: email at christian-gleinser.de (Rock Lobster)
Date: Tue, 14 Jul 2009 07:47:48 -0700 (PDT)
Subject: [C++-sig]  Boost.Python: same class in several modules
Message-ID: <24479797.post@talk.nabble.com>


Hello,

I tried to do the following:
I've got two Python modules which are both wrapped by boost.python, and both
of them share some header files, so there are several classes which are used
by both modules.

As an easy example:
- first module is called "videolib" and second module is called "videofx"
- both modules use a class called "VideoFile", which is inside a single
"videofile.h" (included by both modules)
- videolib module has a function that returns a VideoFile*.
- videofx module has a class with a method that accepts VideoFile* as a
parameter.

Now I'd like to use both modules in Python, and call e.g.
videolib.createVideoFile() and then use the returned object to put it into
the, let's say, videofx.doSomethingWith(vf) method.

But the problem is that Python doesn't know that both VideoFile* types are
exactly the same, so he says "Python argument types did not match C++
signature".

Is there anything I can do to manage this situation? Or would I have to
re-engineer the library structures?

Nice greetings and thanks in advance
Chris
-- 
View this message in context: http://www.nabble.com/Boost.Python%3A-same-class-in-several-modules-tp24479797p24479797.html
Sent from the Python - c++-sig mailing list archive at Nabble.com.


From wladwig at wdtinc.com  Tue Jul 14 17:15:57 2009
From: wladwig at wdtinc.com (William Ladwig)
Date: Tue, 14 Jul 2009 10:15:57 -0500
Subject: [C++-sig] Boost.Python: same class in several modules
In-Reply-To: <24479797.post@talk.nabble.com>
References: <24479797.post@talk.nabble.com>
Message-ID: <765CBD9053EA2B438625895F30D1856F023821C4E5@storm.wdtinc.com>

Are you using the static or dynamic version of the boost python library?  This looks similar to a problem a coworker had and switching to the dynamic version of the library fixed his problem.  According to the documentation, the dynamic version of the library "contains a type conversion registry. Because one registry is shared among all extension modules, instances of a class exposed to Python in one dynamically-loaded extension module can be passed to functions exposed in another such module."

Also, if you organize your extension modules to be used within packages, you can use __init__.py magic to load any extension classes before using them.  I had to do this once when I defined a base class in one extension module and subclassed it in another.  I believe the dynamic version of the boost python library is required for this to work.

Regards, 

Bill



-----Original Message-----
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [mailto:cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Rock Lobster
Sent: Tuesday, July 14, 2009 9:48 AM
To: cplusplus-sig at python.org
Subject: [C++-sig] Boost.Python: same class in several modules


Hello,

I tried to do the following:
I've got two Python modules which are both wrapped by boost.python, and both
of them share some header files, so there are several classes which are used
by both modules.

As an easy example:
- first module is called "videolib" and second module is called "videofx"
- both modules use a class called "VideoFile", which is inside a single
"videofile.h" (included by both modules)
- videolib module has a function that returns a VideoFile*.
- videofx module has a class with a method that accepts VideoFile* as a
parameter.

Now I'd like to use both modules in Python, and call e.g.
videolib.createVideoFile() and then use the returned object to put it into
the, let's say, videofx.doSomethingWith(vf) method.

But the problem is that Python doesn't know that both VideoFile* types are
exactly the same, so he says "Python argument types did not match C++
signature".

Is there anything I can do to manage this situation? Or would I have to
re-engineer the library structures?

Nice greetings and thanks in advance
Chris
-- 
View this message in context: http://www.nabble.com/Boost.Python%3A-same-class-in-several-modules-tp24479797p24479797.html
Sent from the Python - c++-sig mailing list archive at Nabble.com.

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From renatox at gmail.com  Tue Jul 14 19:22:35 2009
From: renatox at gmail.com (Renato Araujo)
Date: Tue, 14 Jul 2009 14:22:35 -0300
Subject: [C++-sig] Boost.Python: same class in several modules
In-Reply-To: <765CBD9053EA2B438625895F30D1856F023821C4E5@storm.wdtinc.com>
References: <24479797.post@talk.nabble.com>
	<765CBD9053EA2B438625895F30D1856F023821C4E5@storm.wdtinc.com>
Message-ID: <95291a80907141022y4ebd09fex922de8376d36708f@mail.gmail.com>

Verify your library linkage order, because python boost library need
to be the first one, for some reasons of types resolve.

BR

On Tue, Jul 14, 2009 at 12:15 PM, William Ladwig<wladwig at wdtinc.com> wrote:
> Are you using the static or dynamic version of the boost python library? ?This looks similar to a problem a coworker had and switching to the dynamic version of the library fixed his problem. ?According to the documentation, the dynamic version of the library "contains a type conversion registry. Because one registry is shared among all extension modules, instances of a class exposed to Python in one dynamically-loaded extension module can be passed to functions exposed in another such module."
>
> Also, if you organize your extension modules to be used within packages, you can use __init__.py magic to load any extension classes before using them. ?I had to do this once when I defined a base class in one extension module and subclassed it in another. ?I believe the dynamic version of the boost python library is required for this to work.
>
> Regards,
>
> Bill
>
>
>
> -----Original Message-----
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [mailto:cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Rock Lobster
> Sent: Tuesday, July 14, 2009 9:48 AM
> To: cplusplus-sig at python.org
> Subject: [C++-sig] Boost.Python: same class in several modules
>
>
> Hello,
>
> I tried to do the following:
> I've got two Python modules which are both wrapped by boost.python, and both
> of them share some header files, so there are several classes which are used
> by both modules.
>
> As an easy example:
> - first module is called "videolib" and second module is called "videofx"
> - both modules use a class called "VideoFile", which is inside a single
> "videofile.h" (included by both modules)
> - videolib module has a function that returns a VideoFile*.
> - videofx module has a class with a method that accepts VideoFile* as a
> parameter.
>
> Now I'd like to use both modules in Python, and call e.g.
> videolib.createVideoFile() and then use the returned object to put it into
> the, let's say, videofx.doSomethingWith(vf) method.
>
> But the problem is that Python doesn't know that both VideoFile* types are
> exactly the same, so he says "Python argument types did not match C++
> signature".
>
> Is there anything I can do to manage this situation? Or would I have to
> re-engineer the library structures?
>
> Nice greetings and thanks in advance
> Chris
> --
> View this message in context: http://www.nabble.com/Boost.Python%3A-same-class-in-several-modules-tp24479797p24479797.html
> Sent from the Python - c++-sig mailing list archive at Nabble.com.
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho

From email at christian-gleinser.de  Thu Jul 16 10:32:18 2009
From: email at christian-gleinser.de (Rock Lobster)
Date: Thu, 16 Jul 2009 01:32:18 -0700 (PDT)
Subject: [C++-sig] Boost.Python: same class in several modules
In-Reply-To: <24479797.post@talk.nabble.com>
References: <24479797.post@talk.nabble.com>
Message-ID: <24512473.post@talk.nabble.com>


Thanks for your answers, I tried both but it didn't work. However, later it
turned out that both libraries didn't link against exactly the same boost
version, so that was the problem. Now everything works fine :)
-- 
View this message in context: http://www.nabble.com/Boost.Python%3A-same-class-in-several-modules-tp24479797p24512473.html
Sent from the Python - c++-sig mailing list archive at Nabble.com.


From david.roy.perso at gmail.com  Mon Jul 20 16:32:34 2009
From: david.roy.perso at gmail.com (David Roy)
Date: Mon, 20 Jul 2009 07:32:34 -0700 (PDT)
Subject: [C++-sig]  Calling function on object of unknown type
Message-ID: <24570986.post@talk.nabble.com>


Hi all,

I'm extending C++ with Boost.Python.
At some point I need to call a Python function on an object of unknown
Python type.
I found those 2 solutions, but they seem so simple that they make me wonder
about the dangerosity of doing this (I'm a newbie to Boost.Python):

static void test_func_attr(object pyObj, object arg)
{
	pyObj.attr("test")(arg);		
}

BOOST_PYTHON_MODULE(my_module)
{
	def("test_func_attr", &test_func_attr);	
}

Called from :

class MyClass(object):
     def test(self,str):
        print str

class = MyClass()
test_func_attr(class, "hello")

The second solution is very close:

static void test_func_attr(object pyObj, object arg)
{
    object func = pyObj.attr("test");
    call<void>(extract<PyObject*>(func), arg);		
}

Both are working. Am I doing something wrong here? And which one is the best
(advantage of #1 is that I don't need to know the return type of test())

Thanks in advance
-David 
-- 
View this message in context: http://www.nabble.com/Calling-function-on-object-of-unknown-type-tp24570986p24570986.html
Sent from the Python - c++-sig mailing list archive at Nabble.com.


From Matthew.Scouten at tradingtechnologies.com  Mon Jul 20 19:51:33 2009
From: Matthew.Scouten at tradingtechnologies.com (Matthew Scouten (TT))
Date: Mon, 20 Jul 2009 12:51:33 -0500
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>
Message-ID: <32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>

I would like to second this question.

Several times recently, I have needed to wrap STL-like containers that are almost-vectors, just different enough that vector_indexing_suite does not work. This is a minor PITA with the standard indexing_suite and looks like it will be much easier with the new one. Due to technical/political reasons, it is difficult for us to use anything not on the mainline, in a numbered boost distribution. It would be a big help if this got on the mainline.  

Is there still a good reason not to?

PS: Jakob, I have some code that may help with the 'no operator()== problem":

//Thanks to Andreas Kl?ckner on "Development of Python/C++ integration" <C++-sig at python.org> for this code.
template <class T>
class no_compare_indexing_suite : public bp::vector_indexing_suite<T, false, no_compare_indexing_suite<T> > 
{
  public:
    static bool contains(T &container, typename T::value_type const &key)
    { 
        PYTHON_ERROR(PyExc_NotImplementedError, "containment checking not supported on this container"); 
    } 
};

Just use this where you would otherwise use vector_indexing_suite.

-----Original Message-----
From: cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org [mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org] On Behalf Of Jakob van Santen
Sent: Saturday, July 11, 2009 4:50 PM
To: cplusplus-sig at python.org
Subject: [C++-sig] Boost.Python indexing suite version 2?

Hello,

While attempting to wrap a vector of objects where the element class  
doesn't define operator()==, I ran across a mention of a "Version 2"  
of the indexing suite from 2003:

http://mail.python.org/pipermail/cplusplus-sig/2003-October/005453.html

While it's mentioned on the Py++ page, it doesn't seem to have made it  
in to the main boost distribution in the intervening 5 years. Can  
anyone tell me what happened to this project?

Thanks,
Jakob van Santen
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From roman.yakovenko at gmail.com  Mon Jul 20 20:31:14 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 20 Jul 2009 21:31:14 +0300
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>
Message-ID: <7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>

On Mon, Jul 20, 2009 at 8:51 PM, Matthew Scouten
(TT)<Matthew.Scouten at tradingtechnologies.com> wrote:
> I would like to second this question.

Right now, I am, with help from other people and  from the author
support this suite.

Here( http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/
) you can find an improve version of originally proposed suite.

The important change is that today this library( indexing suite v2) is
header only and could be installed\used without modifying Boost
directories. I also added few unit tests and it is used in a few large
projects without known issues.

For more information read the following document:
http://language-binding.net/pyplusplus/documentation/containers.html

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From troy at resophonic.com  Mon Jul 20 22:34:42 2009
From: troy at resophonic.com (troy d. straszheim)
Date: Mon, 20 Jul 2009 16:34:42 -0400
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>	<32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>
	<7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>
Message-ID: <4A64D4E2.7010600@resophonic.com>

Roman Yakovenko wrote:
> On Mon, Jul 20, 2009 at 8:51 PM, Matthew Scouten
> (TT)<Matthew.Scouten at tradingtechnologies.com> wrote:
>> I would like to second this question.
> 
> Right now, I am, with help from other people and  from the author
> support this suite.
> 
> Here( http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/
> ) you can find an improve version of originally proposed suite.
> 
> The important change is that today this library( indexing suite v2) is
> header only and could be installed\used without modifying Boost
> directories. I also added few unit tests and it is used in a few large
> projects without known issues.
> 
> For more information read the following document:
> http://language-binding.net/pyplusplus/documentation/containers.html
> 

I suppose this is a good time to follow up on this old post, advertising 
the std_map_indexing_suite that Jakob and I put together:

   http://article.gmane.org/gmane.comp.python.c%2B%2B/13642

I'm wondering if the v2 interface supports what is listed there, and if 
not, is there interest in adding support?

-t


From roman.yakovenko at gmail.com  Tue Jul 21 06:42:00 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 21 Jul 2009 07:42:00 +0300
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <4A64D4E2.7010600@resophonic.com>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>
	<7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>
	<4A64D4E2.7010600@resophonic.com>
Message-ID: <7465b6170907202142n18e213c9xf499bd736781d483@mail.gmail.com>

On Mon, Jul 20, 2009 at 11:34 PM, troy d. straszheim<troy at resophonic.com> wrote:

> I suppose this is a good time to follow up on this old post, advertising the
> std_map_indexing_suite that Jakob and I put together:
>
> ?http://article.gmane.org/gmane.comp.python.c%2B%2B/13642
>
> I'm wondering if the v2 interface supports what is listed there, and if not,
> is there interest in adding support?

Indexing suite V2 doesn't support some of them, but it should not be
to hard to extend it. I did it few times.

Today this indexing suite supports the following containers: vector,
deque, list, map, multimap, hash_map, set, hash_set.  It could be nice
to get support for the rest of containers too( queue, priority_queue,
stack, mutliset, hash_multiset, stack ).

I think, we can join the efforts and complete it.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From basti.kr at gmail.com  Wed Jul 22 16:16:05 2009
From: basti.kr at gmail.com (Sebastian Kraemer)
Date: Wed, 22 Jul 2009 16:16:05 +0200
Subject: [C++-sig] wrapping reference members
Message-ID: <4A671F25.5040309@gmail.com>

Hi all,
I'm a beginner of c++ and boost::python, so this may be a stupid
question. When I have the following class:

class Foo{
    public:
        float &a;
    };

Is there some way to expose "a" in python? I would like to use it like a
normal python attribute, e.g. set it's value with "Foo().a = 3.1".

Thanks for any help,
Sebastian


From Matthew.Scouten at tradingtechnologies.com  Wed Jul 22 16:30:18 2009
From: Matthew.Scouten at tradingtechnologies.com (Matthew Scouten (TT))
Date: Wed, 22 Jul 2009 09:30:18 -0500
Subject: [C++-sig] wrapping reference members
In-Reply-To: <4A671F25.5040309@gmail.com>
References: <4A671F25.5040309@gmail.com>
Message-ID: <32490DFF7774554A85D65D23A9F0F9380B0B7019@chiex01>

I would think that .def_readwrite would do what you need. If it doesn't
look at using .def_property and get/set pair. Caveat: making sure that
'a' stays a valid reference is your problem. Python will happily crash
horribly (if you are lucky) in response to bogus pointers or references.


-----Original Message-----
From:
cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org
[mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at py
thon.org] On Behalf Of Sebastian Kraemer
Sent: Wednesday, July 22, 2009 9:16 AM
To: cplusplus-sig at python.org
Subject: [C++-sig] wrapping reference members

Hi all,
I'm a beginner of c++ and boost::python, so this may be a stupid
question. When I have the following class:

class Foo{
    public:
        float &a;
    };

Is there some way to expose "a" in python? I would like to use it like a
normal python attribute, e.g. set it's value with "Foo().a = 3.1".

Thanks for any help,
Sebastian

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From basti.kr at gmail.com  Wed Jul 22 16:56:46 2009
From: basti.kr at gmail.com (Sebastian Kraemer)
Date: Wed, 22 Jul 2009 16:56:46 +0200
Subject: [C++-sig] wrapping reference members
In-Reply-To: <4A671F25.5040309@gmail.com>
References: <4A671F25.5040309@gmail.com>
Message-ID: <4A6728AE.7060608@gmail.com>

> I would think that .def_readwrite would do what you need. If it doesn't
> look at using .def_property and get/set pair. Caveat: making sure that
> 'a' stays a valid reference is your problem. Python will happily crash
> horribly (if you are lucky) in response to bogus pointers or references.
Thanks for your fast answer! I tried it with following code:

class_<Foo>("Foo", init<>())
    .def_readwrite("a", &Foo::a)
    ;

But it raises the error:

error: cannot create pointer to reference member 'Foo::a'

And it seems it's really not possible to create a pointer to a reference in c++ ;) So what else do I have to write instead of "&Foo::a"?

Cheers,
Sebastian


From Matthew.Scouten at tradingtechnologies.com  Wed Jul 22 17:18:05 2009
From: Matthew.Scouten at tradingtechnologies.com (Matthew Scouten (TT))
Date: Wed, 22 Jul 2009 10:18:05 -0500
Subject: [C++-sig] wrapping reference members
In-Reply-To: <4A6728AE.7060608@gmail.com>
References: <4A671F25.5040309@gmail.com> <4A6728AE.7060608@gmail.com>
Message-ID: <32490DFF7774554A85D65D23A9F0F9380B0B707F@chiex01>

Try something like this:

float get_a(const Foo& self) {return self.a}
void  set_a(Foo& self, float a_new) {self.a = a_new }

class_<Foo>("Foo", init<>())
    .def_property("a", &get_a, &set_a)
    ;



-----Original Message-----
From:
cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org
[mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at py
thon.org] On Behalf Of Sebastian Kraemer
Sent: Wednesday, July 22, 2009 9:57 AM
To: cplusplus-sig at python.org
Subject: Re: [C++-sig] wrapping reference members

> I would think that .def_readwrite would do what you need. If it
doesn't
> look at using .def_property and get/set pair. Caveat: making sure that
> 'a' stays a valid reference is your problem. Python will happily crash
> horribly (if you are lucky) in response to bogus pointers or
references.
Thanks for your fast answer! I tried it with following code:

class_<Foo>("Foo", init<>())
    .def_readwrite("a", &Foo::a)
    ;

But it raises the error:

error: cannot create pointer to reference member 'Foo::a'

And it seems it's really not possible to create a pointer to a reference
in c++ ;) So what else do I have to write instead of "&Foo::a"?

Cheers,
Sebastian

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From wladwig at wdtinc.com  Wed Jul 22 17:20:12 2009
From: wladwig at wdtinc.com (William Ladwig)
Date: Wed, 22 Jul 2009 10:20:12 -0500
Subject: [C++-sig] wrapping reference members
In-Reply-To: <4A6728AE.7060608@gmail.com>
References: <4A671F25.5040309@gmail.com> <4A6728AE.7060608@gmail.com>
Message-ID: <765CBD9053EA2B438625895F30D1856F02382DFB98@storm.wdtinc.com>

You may need to create a getter and a setter in your wrapper, and then you can use the "make_function" function to change the CallPolicy when you use create a property.  I think you want your call policy to be return_internal_reference<> and that will handle the lifetime correctly.  Here is a reference link that that technique:

http://wiki.python.org/moin/boost.python/HowTo#getterandsettermethodsasaproperty

There may be an easier solution to your problem, but I know that the technique above will work.

Hope this helps,

Bill

-----Original Message-----
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [mailto:cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Sebastian Kraemer
Sent: Wednesday, July 22, 2009 9:57 AM
To: cplusplus-sig at python.org
Subject: Re: [C++-sig] wrapping reference members

> I would think that .def_readwrite would do what you need. If it doesn't
> look at using .def_property and get/set pair. Caveat: making sure that
> 'a' stays a valid reference is your problem. Python will happily crash
> horribly (if you are lucky) in response to bogus pointers or references.
Thanks for your fast answer! I tried it with following code:

class_<Foo>("Foo", init<>())
    .def_readwrite("a", &Foo::a)
    ;

But it raises the error:

error: cannot create pointer to reference member 'Foo::a'

And it seems it's really not possible to create a pointer to a reference in c++ ;) So what else do I have to write instead of "&Foo::a"?

Cheers,
Sebastian

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From basti.kr at gmail.com  Wed Jul 22 17:25:10 2009
From: basti.kr at gmail.com (Sebastian Kraemer)
Date: Wed, 22 Jul 2009 17:25:10 +0200
Subject: [C++-sig] wrapping reference members
In-Reply-To: <32490DFF7774554A85D65D23A9F0F9380B0B707F@chiex01>
References: <4A671F25.5040309@gmail.com> <4A6728AE.7060608@gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B707F@chiex01>
Message-ID: <4A672F56.5020001@gmail.com>

Thanks for the answer!

From Matthew.Scouten at tradingtechnologies.com  Wed Jul 22 22:00:01 2009
From: Matthew.Scouten at tradingtechnologies.com (Matthew Scouten (TT))
Date: Wed, 22 Jul 2009 15:00:01 -0500
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com><32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>
	<7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>
Message-ID: <32490DFF7774554A85D65D23A9F0F9380B0B728B@chiex01>

I am looking into whether this will satisfy my needs. It looks good so
far but I have a question:
How am I supposed to use container_proxy? The only thing given for an
example is a link to an empty file. Using it as a drop-in replacement
for container_suite does not work.  

-----Original Message-----
From:
cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org
[mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at py
thon.org] On Behalf Of Roman Yakovenko
Sent: Monday, July 20, 2009 1:31 PM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] Boost.Python indexing suite version 2?

On Mon, Jul 20, 2009 at 8:51 PM, Matthew Scouten
(TT)<Matthew.Scouten at tradingtechnologies.com> wrote:
> I would like to second this question.

Right now, I am, with help from other people and  from the author
support this suite.

Here(
http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/index
ing_suite_v2/
) you can find an improve version of originally proposed suite.

The important change is that today this library( indexing suite v2) is
header only and could be installed\used without modifying Boost
directories. I also added few unit tests and it is used in a few large
projects without known issues.

For more information read the following document:
http://language-binding.net/pyplusplus/documentation/containers.html

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From roman.yakovenko at gmail.com  Thu Jul 23 06:57:22 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Thu, 23 Jul 2009 07:57:22 +0300
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <32490DFF7774554A85D65D23A9F0F9380B0B728B@chiex01>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>
	<7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B728B@chiex01>
Message-ID: <7465b6170907222157h74f13b14v2f8b37a785b7cda6@mail.gmail.com>

On Wed, Jul 22, 2009 at 11:00 PM, Matthew Scouten
(TT)<Matthew.Scouten at tradingtechnologies.com> wrote:
> I am looking into whether this will satisfy my needs. It looks good so
> far but I have a question:
> How am I supposed to use container_proxy? The only thing given for an
> example is a link to an empty file. Using it as a drop-in replacement
> for container_suite does not work.

Unfortunately, I never used this class before. If you will send
**small and complete** example of what you are trying to do, I will
take a look on it.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Robert.Smallshire at roxar.com  Thu Jul 23 16:17:53 2009
From: Robert.Smallshire at roxar.com (Robert Smallshire)
Date: Thu, 23 Jul 2009 16:17:53 +0200
Subject: [C++-sig] boost::python::str and Python's str and unicode types
Message-ID: <446E8416AAF61D47B271B4E2FAE19BE66B5A33@svgw2k15.roxardomain.roxar.com>


Hello,

I'm looking for some confirmation that my understanding of what
boost.python is doing is correct.

>From what I understand boost::python::str represents the str
(PyStringObject) type in Python.  Unless I'm missing something, here
doesn't seem to be anything like boost::python::unicode representing the
unicode type (PyUnicodeObject).

Boost.Python does have a built-in conversion from std::wstring to
object, but this creates a str in Python, so presumably there is some
(lossy?) decoding going on here.

It seems that to directly create a PyUnicodeObject from C++ I will need
to do one or more of the following:

1) Directly use the Python C API to create and manipulate
PyUnicodeObject.
2) Create my own subclass of boost::python::object called for example
'unicode', which would be similar to the str class provided out of the
box, but would use PyUnicodeObject underneath and deal with wchar_t
rather than char.

Correct?

Many thanks,

Rob

DISCLAIMER:
This message contains information that may be privileged or confidential and is the property of the Roxar Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorised to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

From seefeld at sympatico.ca  Thu Jul 23 17:22:13 2009
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Thu, 23 Jul 2009 11:22:13 -0400
Subject: [C++-sig] boost::python::str and Python's str and unicode types
In-Reply-To: <446E8416AAF61D47B271B4E2FAE19BE66B5A33@svgw2k15.roxardomain.roxar.com>
References: <446E8416AAF61D47B271B4E2FAE19BE66B5A33@svgw2k15.roxardomain.roxar.com>
Message-ID: <4A688025.3000007@sympatico.ca>

On 07/23/2009 10:17 AM, Robert Smallshire wrote:
> Hello,
>
> I'm looking for some confirmation that my understanding of what
> boost.python is doing is correct.
>
> > From what I understand boost::python::str represents the str
> (PyStringObject) type in Python.  Unless I'm missing something, here
> doesn't seem to be anything like boost::python::unicode representing the
> unicode type (PyUnicodeObject).
>
> Boost.Python does have a built-in conversion from std::wstring to
> object, but this creates a str in Python, so presumably there is some
> (lossy?) decoding going on here.
>
> It seems that to directly create a PyUnicodeObject from C++ I will need
> to do one or more of the following:
>
> 1) Directly use the Python C API to create and manipulate
> PyUnicodeObject.
> 2) Create my own subclass of boost::python::object called for example
> 'unicode', which would be similar to the str class provided out of the
> box, but would use PyUnicodeObject underneath and deal with wchar_t
> rather than char.
>    

You might want to talk to Haoyu Bai, who is presently working on the 
Python 3 port of boost.python. I wonder whether some of his work would 
be useful with Python 2, notably the unicode stuff. Haoyu, any thoughts 
on this ?

Regards,
         Stefan

-- 

       ...ich hab' noch einen Koffer in Berlin...


From Matthew.Scouten at tradingtechnologies.com  Thu Jul 23 17:59:01 2009
From: Matthew.Scouten at tradingtechnologies.com (Matthew Scouten (TT))
Date: Thu, 23 Jul 2009 10:59:01 -0500
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <7465b6170907222157h74f13b14v2f8b37a785b7cda6@mail.gmail.com>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com><32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01><7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com><32490DFF7774554A85D65D23A9F0F9380B0B728B@chiex01>
	<7465b6170907222157h74f13b14v2f8b37a785b7cda6@mail.gmail.com>
Message-ID: <32490DFF7774554A85D65D23A9F0F9380B0B7506@chiex01>

This is a sketch of what I want to do. This seems to work seamlessly
with the 
included indexing suite. I cannot figure out how the make it work with
the advanced one. 

---------------------- c++
----------------------------------------------
struct foo
{
    int bar;
};

class_<foo>("foo")
    .def_readwrite("bar", &foo::bar)
    ;

#if 1
//this does not work the way I want
class_< std::vector<foo> >("class_< std::vector<foo> >("VectorOfFoo") )
    .def( indexing::container_suite< std::vector<foo> >() )
    ;
#else
//I think this should work the way I want, but it will not compile.
class_< std::vector<foo> >("VectorOfFoo")
    .def( indexing::container_proxy< std::vector<foo> >() )
    ;
#endif

----------------------Python--------------------------------------------
--
v = VectorOfFoo()
f = foo()
f.bar = 0
v.append(f)
assert v[0].bar == 0
v[0].bar = 10 #This returns the foo by value. That does not happen with
the bp::vector_indexing_suite.
assert v[0].bar == 10 # this fails. v[0].bar is still 0


-----Original Message-----
From:
cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org
[mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at py
thon.org] On Behalf Of Roman Yakovenko
Sent: Wednesday, July 22, 2009 11:57 PM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] Boost.Python indexing suite version 2?

On Wed, Jul 22, 2009 at 11:00 PM, Matthew Scouten
(TT)<Matthew.Scouten at tradingtechnologies.com> wrote:
> I am looking into whether this will satisfy my needs. It looks good so
> far but I have a question:
> How am I supposed to use container_proxy? The only thing given for an
> example is a link to an empty file. Using it as a drop-in replacement
> for container_suite does not work.

Unfortunately, I never used this class before. If you will send
**small and complete** example of what you are trying to do, I will
take a look on it.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From roman.yakovenko at gmail.com  Sat Jul 25 21:19:17 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sat, 25 Jul 2009 22:19:17 +0300
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <32490DFF7774554A85D65D23A9F0F9380B0B7506@chiex01>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>
	<7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B728B@chiex01>
	<7465b6170907222157h74f13b14v2f8b37a785b7cda6@mail.gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B7506@chiex01>
Message-ID: <7465b6170907251219r66481766ra5c0e3ad31d21a74@mail.gmail.com>

On Thu, Jul 23, 2009 at 6:59 PM, Matthew Scouten
(TT)<Matthew.Scouten at tradingtechnologies.com> wrote:
> This is a sketch of what I want to do. This seems to work seamlessly
> with the
> included indexing suite. I cannot figure out how the make it work with
> the advanced one.

Okey. By default the indexing suite v2 returns objects by value (
default call policy ).
It is very simple to change this behavior:

        namespace bp = boost::python;

        typedef bp::class_< std::vector< foo > > foo_vector_exposer_t;
        foo_vector_exposer_t foo_vector_exposer =
foo_vector_exposer_t( "foo_vector");
        bp::scope foo_vector_scope( foo_vector_exposer );
        foo_vector_exposer.def( bp::indexing::vector_suite<
std::vector< foo > >::with_policies(bp::return_internal_reference<
>()) );

And than all your code will work as expected.

I suggest you not to use "container_suite":
* the compilation is longer
* it is also unable to guess/resolve between map and multimap containers.

P.S. Let me know if you still want\need to use container_proxy. I
never used that class, so it could take me some time to understand it.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Matthew.Scouten at tradingtechnologies.com  Mon Jul 27 18:29:17 2009
From: Matthew.Scouten at tradingtechnologies.com (Matthew Scouten (TT))
Date: Mon, 27 Jul 2009 11:29:17 -0500
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <7465b6170907251219r66481766ra5c0e3ad31d21a74@mail.gmail.com>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com><32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01><7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com><32490DFF7774554A85D65D23A9F0F9380B0B728B@chiex01><7465b6170907222157h74f13b14v2f8b37a785b7cda6@mail.gmail.com><32490DFF7774554A85D65D23A9F0F9380B0B7506@chiex01>
	<7465b6170907251219r66481766ra5c0e3ad31d21a74@mail.gmail.com>
Message-ID: <32490DFF7774554A85D65D23A9F0F9380B1360C5@chiex01>

The documentation points out (with very good reason) that just using
"::with_policies(bp::return_internal_reference)" can cause major
problems if the container in question reallocates things between the
time that you extract the object and the time you try to access it. 

v = VectorofFoo()
v.extend([Foo() for x in range(10)])
f = v[5] #returns a reference to a foo on the heap
f.extend([Foo() for x in range(1000)]) #almost certainly causes
reallocation of underlying array in c++

f.function() #accesses a no longer valid reference, sky falls.

The standard indexing suite uses clever proxy magic to work around this.


On second Look, it looks like the intended usage of container_proxy is
something like this:

class_< container_proxy <std::vector<foo> > >("class_< std::vector<foo>
>("VectorOfFoo") )
    .def( indexing::container_suite< container_proxy <std::vector<foo> >
>() )
    ;

Which seems kind of clunky to me.

I think defaulting to by-value is the wrong behavior. The idiom
"v[n].some_attribute = blah" works seamlessly on python lists, and c++
vectors. The indexing suite V1 is the default behavior, and can be
disabled if needed. 


I am using container_suite on a "std::map<std::string,
std::vector<std::string> >" and the only problem I have is a compiler
warning about a very long decorated name. using foo_vector_exposer_t
directly seems kind of clunky to me. 

While I was trying to use indexing suite v2 to wrap a non-STL but
STL-like container class, I noticed that the documentation for doing so
is badly out of date. This is the main (almost the only) advantage that
V2 has over V1, so these docs badly need to be updated. 


-----Original Message-----
From:
cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org
[mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at py
thon.org] On Behalf Of Roman Yakovenko
Sent: Saturday, July 25, 2009 2:19 PM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] Boost.Python indexing suite version 2?

On Thu, Jul 23, 2009 at 6:59 PM, Matthew Scouten
(TT)<Matthew.Scouten at tradingtechnologies.com> wrote:
> This is a sketch of what I want to do. This seems to work seamlessly
> with the
> included indexing suite. I cannot figure out how the make it work with
> the advanced one.

Okey. By default the indexing suite v2 returns objects by value (
default call policy ).
It is very simple to change this behavior:

        namespace bp = boost::python;

        typedef bp::class_< std::vector< foo > > foo_vector_exposer_t;
        foo_vector_exposer_t foo_vector_exposer =
foo_vector_exposer_t( "foo_vector");
        bp::scope foo_vector_scope( foo_vector_exposer );
        foo_vector_exposer.def( bp::indexing::vector_suite<
std::vector< foo > >::with_policies(bp::return_internal_reference<
>()) );

And than all your code will work as expected.

I suggest you not to use "container_suite":
* the compilation is longer
* it is also unable to guess/resolve between map and multimap
containers.

P.S. Let me know if you still want\need to use container_proxy. I
never used that class, so it could take me some time to understand it.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From rogeeff at gmail.com  Mon Jul 27 21:20:23 2009
From: rogeeff at gmail.com (Gennadiy Rozental)
Date: Mon, 27 Jul 2009 19:20:23 +0000 (UTC)
Subject: [C++-sig] Boost.Python module initialization failure
Message-ID: <loom.20090727T191936-796@post.gmane.org>

What is the right way to report error from the Boost.Python extension module
init function?

Gennadiy


From roman.yakovenko at gmail.com  Mon Jul 27 21:43:24 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 27 Jul 2009 22:43:24 +0300
Subject: [C++-sig] Boost.Python indexing suite version 2?
In-Reply-To: <32490DFF7774554A85D65D23A9F0F9380B1360C5@chiex01>
References: <F7C57D77-A408-421C-93D9-28B5C0D1AADD@gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B6A3A@chiex01>
	<7465b6170907201131h2b45177dj51b70423f70b6065@mail.gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B728B@chiex01>
	<7465b6170907222157h74f13b14v2f8b37a785b7cda6@mail.gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B0B7506@chiex01>
	<7465b6170907251219r66481766ra5c0e3ad31d21a74@mail.gmail.com>
	<32490DFF7774554A85D65D23A9F0F9380B1360C5@chiex01>
Message-ID: <7465b6170907271243r5b664f40kfd77cc85626499c9@mail.gmail.com>

On Mon, Jul 27, 2009 at 7:29 PM, Matthew Scouten
(TT)<Matthew.Scouten at tradingtechnologies.com> wrote:
> The documentation points out (with very good reason) that just using
> "::with_policies(bp::return_internal_reference)" can cause major
> problems if the container in question reallocates things between the
> time that you extract the object and the time you try to access it.
>
> v = VectorofFoo()
> v.extend([Foo() for x in range(10)])
> f = v[5] #returns a reference to a foo on the heap
> f.extend([Foo() for x in range(1000)]) #almost certainly causes
> reallocation of underlying array in c++
>
> f.function() #accesses a no longer valid reference, sky falls.
>
> The standard indexing suite uses clever proxy magic to work around this.
>

Okey, that is why v2 provides container_proxy class. There is an
advantage of not mixing "proxy magic" with container suite: simple
implementation.
>From my experience, this limitation was never a problem. After all,
users understands that they deal with wrapper and should be careful.

>
> On second Look, it looks like the intended usage of container_proxy is
> something like this:
>
> class_< container_proxy <std::vector<foo> > >("class_< std::vector<foo>
>>("VectorOfFoo") )
> ? ?.def( indexing::container_suite< container_proxy <std::vector<foo> >
>>() )
> ? ?;
>
> Which seems kind of clunky to me.

I don't think the author had a choice here. Can you propose an
alternative interface?

One possible solution is to add helper function: ( pseudo code)

template< class TContainer >
void register_container_proxy( const char* name, boost::type< TContainer > ){
    class_< container_proxy < TContainer > >( name )
    .def( indexing::container_suite< container_proxy < TContainer > >>() ) ;
}


> I think defaulting to by-value is the wrong behavior. The idiom
> "v[n].some_attribute = blah" works seamlessly on python lists, and c++
> vectors. The indexing suite V1 is the default behavior, and can be
> disabled if needed.

May be it is a little bit confusing, but

>> The documentation points out (with very good reason) that just using
>> "::with_policies(bp::return_internal_reference)" can cause major
>> problems if the container in question reallocates things between the
>> time that you extract the object and the time you try to access it.

makes it reasonable and consistent.

>
> I am using container_suite on a "std::map<std::string,
> std::vector<std::string> >" and the only problem I have is a compiler
> warning about a very long decorated name. using foo_vector_exposer_t
> directly seems kind of clunky to me.

You can write the code as you wish. I like this way, mainly because it
is easier to understand a compiler errors. At least it points you
exactly to the problematic line. The syntax

class_ < ... >
 .def (...)
 .def (...);

is cool but it has its price.

> While I was trying to use indexing suite v2 to wrap a non-STL but
> STL-like container class, I noticed that the documentation for doing so
> is badly out of date. This is the main (almost the only) advantage that
> V2 has over V1, so these docs badly need to be updated.

You are welcome to contribute. My English and explanation skills are
not as good as I want.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Robert.Smallshire at roxar.com  Tue Jul 28 16:11:35 2009
From: Robert.Smallshire at roxar.com (Robert Smallshire)
Date: Tue, 28 Jul 2009 16:11:35 +0200
Subject: [C++-sig] boost::python::str and Python's str and unicode types
In-Reply-To: <4A688025.3000007@sympatico.ca>
References: <446E8416AAF61D47B271B4E2FAE19BE66B5A33@svgw2k15.roxardomain.roxar.com>
	<4A688025.3000007@sympatico.ca>
Message-ID: <446E8416AAF61D47B271B4E2FAE19BE66B5A63@svgw2k15.roxardomain.roxar.com>


> On 07/23/2009 10:17 AM, Robert Smallshire wrote:
> > Hello,
> >
> > I'm looking for some confirmation that my understanding of what
> > boost.python is doing is correct.
> >
> > > From what I understand boost::python::str represents the str
> > (PyStringObject) type in Python.  Unless I'm missing
> something, here
> > doesn't seem to be anything like boost::python::unicode
> representing
> > the unicode type (PyUnicodeObject).
> >
> > Boost.Python does have a built-in conversion from std::wstring to
> > object, but this creates a str in Python, so presumably
> there is some
> > (lossy?) decoding going on here.
> >
> > It seems that to directly create a PyUnicodeObject from C++ I will
> > need to do one or more of the following:
> >
> > 1) Directly use the Python C API to create and manipulate
> > PyUnicodeObject.
> > 2) Create my own subclass of boost::python::object called
> for example
> > 'unicode', which would be similar to the str class provided
> out of the
> > box, but would use PyUnicodeObject underneath and deal with wchar_t
> > rather than char.

I have modified my local build of boost.python to include a
boost::python::unicode class, together with appropriate conversions from
wchar, const wchar_t* and std::wstring, together with additional tests
in the boost.python test suite.  As you would expect, the code is
closely derived from the existing boost::python::str code but uses the
Python C API for PyUnicode_Type rather than PyString_Type. There are no
breaking changes to the existing boost.python API.

I'd like to contribute these changes back to boost.python so they can be
hopefully integrated into future official Boost releases.  What is the
procedure for this?  Where do I start?

Cheers,

Robert Smallshire

DISCLAIMER:
This message contains information that may be privileged or confidential and is the property of the Roxar Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorised to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

From merlin66b at gmail.com  Tue Jul 28 17:43:32 2009
From: merlin66b at gmail.com (Thomas Berg)
Date: Tue, 28 Jul 2009 17:43:32 +0200
Subject: [C++-sig] boost::python::str and Python's str and unicode types
In-Reply-To: <446E8416AAF61D47B271B4E2FAE19BE66B5A63@svgw2k15.roxardomain.roxar.com>
References: <446E8416AAF61D47B271B4E2FAE19BE66B5A33@svgw2k15.roxardomain.roxar.com>
	<4A688025.3000007@sympatico.ca>
	<446E8416AAF61D47B271B4E2FAE19BE66B5A63@svgw2k15.roxardomain.roxar.com>
Message-ID: <c472dffc0907280843k68119af4k997ea835b27228e7@mail.gmail.com>

On Tue, Jul 28, 2009 at 4:11 PM, Robert
Smallshire<Robert.Smallshire at roxar.com> wrote:
> I have modified my local build of boost.python to include a
> boost::python::unicode class, together with appropriate conversions from
> wchar, const wchar_t* and std::wstring, together with additional tests
> in the boost.python test suite. ?As you would expect, the code is
> closely derived from the existing boost::python::str code but uses the
> Python C API for PyUnicode_Type rather than PyString_Type. There are no
> breaking changes to the existing boost.python API.
>
> I'd like to contribute these changes back to boost.python so they can be
> hopefully integrated into future official Boost releases. ?What is the
> procedure for this? ?Where do I start?
>
> Cheers,
>
> Robert Smallshire

Just a side note, I think Haoyu Bai is already adding unicode support
to boost.python in his GSoC project. His work should at least be
available through svn, according to his post from 4 weeks ago:
http://mail.python.org/pipermail/cplusplus-sig/2009-July/014664.html

Cheers,
Thomas Berg

From rwgk at yahoo.com  Tue Jul 28 19:26:02 2009
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Tue, 28 Jul 2009 10:26:02 -0700 (PDT)
Subject: [C++-sig] boost::python::str and Python's str and unicode types
In-Reply-To: <446E8416AAF61D47B271B4E2FAE19BE66B5A63@svgw2k15.roxardomain.roxar.com>
References: <446E8416AAF61D47B271B4E2FAE19BE66B5A33@svgw2k15.roxardomain.roxar.com>
	<4A688025.3000007@sympatico.ca>
	<446E8416AAF61D47B271B4E2FAE19BE66B5A63@svgw2k15.roxardomain.roxar.com>
Message-ID: <323989.89424.qm@web111409.mail.gq1.yahoo.com>

I'd be happy to help out, but I don't know if/how this could interfere with Haoyu Bai's work.
Could you post your patches to the trac system at svn.boost.org so we can have a look
and figure out a way forward?
Ralf




----- Original Message ----
From: Robert Smallshire <Robert.Smallshire at roxar.com>
To: Development of Python/C++ integration <cplusplus-sig at python.org>
Sent: Tuesday, July 28, 2009 7:11:35 AM
Subject: Re: [C++-sig] boost::python::str and Python's str and unicode types


> On 07/23/2009 10:17 AM, Robert Smallshire wrote:
> > Hello,
> >
> > I'm looking for some confirmation that my understanding of what
> > boost.python is doing is correct.
> >
> > > From what I understand boost::python::str represents the str
> > (PyStringObject) type in Python.  Unless I'm missing
> something, here
> > doesn't seem to be anything like boost::python::unicode
> representing
> > the unicode type (PyUnicodeObject).
> >
> > Boost.Python does have a built-in conversion from std::wstring to
> > object, but this creates a str in Python, so presumably
> there is some
> > (lossy?) decoding going on here.
> >
> > It seems that to directly create a PyUnicodeObject from C++ I will
> > need to do one or more of the following:
> >
> > 1) Directly use the Python C API to create and manipulate
> > PyUnicodeObject.
> > 2) Create my own subclass of boost::python::object called
> for example
> > 'unicode', which would be similar to the str class provided
> out of the
> > box, but would use PyUnicodeObject underneath and deal with wchar_t
> > rather than char.

I have modified my local build of boost.python to include a
boost::python::unicode class, together with appropriate conversions from
wchar, const wchar_t* and std::wstring, together with additional tests
in the boost.python test suite.  As you would expect, the code is
closely derived from the existing boost::python::str code but uses the
Python C API for PyUnicode_Type rather than PyString_Type. There are no
breaking changes to the existing boost.python API.

I'd like to contribute these changes back to boost.python so they can be
hopefully integrated into future official Boost releases.  What is the
procedure for this?  Where do I start?

Cheers,

Robert Smallshire


From Robert.Smallshire at roxar.com  Tue Jul 28 21:44:42 2009
From: Robert.Smallshire at roxar.com (Robert Smallshire)
Date: Tue, 28 Jul 2009 21:44:42 +0200
Subject: [C++-sig] boost::python::str and Python's str and unicode types
References: <446E8416AAF61D47B271B4E2FAE19BE66B5A33@svgw2k15.roxardomain.rox
	ar.com><4A688025.3000007@sympatico.ca><446E8416AAF61D47B271B4E2FAE19BE66B5A
	63@svgw2k15.roxardomain.roxar.com> 
	<c472dffc0907280843k68119af4k997ea835b27228e7@mail.gmail.com>
Message-ID: <446E8416AAF61D47B271B4E2FAE19BE65E6DCD@svgw2k15.roxardomain.roxar.com>

An embedded message was scrubbed...
From: "Robert Smallshire" <Robert.Smallshire at roxar.com>
Subject: RE: [C++-sig] boost::python::str and Python's str and unicode types
Date: Tue, 28 Jul 2009 21:44:42 +0200
Size: 10475
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090728/0361967d/attachment-0001.eml>
-------------- next part --------------
DISCLAIMER:
This message contains information that may be privileged or confidential and is the property of the Roxar Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorised to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. 

From Paul_Kunz at slac.stanford.edu  Thu Jul 30 18:39:25 2009
From: Paul_Kunz at slac.stanford.edu (Paul F. Kunz)
Date: Thu, 30 Jul 2009 09:39:25 -0700
Subject: [C++-sig] Trouble converting numpy.float32 to C++
Message-ID: <200907301639.n6UGdPJL023642@ki-ls01.slac.stanford.edu>

I'm using Boost.Python to wrap a C++ class that takes a
std::vector<double> & as argument..
Using scitbx/include/scitbx/boost_python/container_conversions.h as
suggested by the Boost.Python
FAQ, I can correctly convert and use a Python sequence generated by numpy with

numpy.arange(10, dtype=numpy.float64 )

But when I change float64 to float32 I get an exception with message

"No registered converter was able to produce a C++ rvalue of type
double from this Python object of type numpy.float32"

Disappointing but maybe understandable.   So I change my C++ class to
accept vector <float> and still get
exception with

"No registered converter was able to produce a C++ rvalue of type
float from this Python object of type numpy.float32"
^^^^^

Strange, not the result I expected.   Even stranger, if I change the
python sequence to float64 while keeping the C++ with vector<float> it
works fine; no exception.

Any suggestions on how to get numpy.float32 handled?   Also
numpy.int64 converted to a double.

From hans_meine at gmx.net  Thu Jul 30 19:43:15 2009
From: hans_meine at gmx.net (Hans Meine)
Date: Thu, 30 Jul 2009 19:43:15 +0200
Subject: [C++-sig] Using symbols from other python extension modules?
Message-ID: <200907301943.17649.hans_meine@gmx.net>

Hi,

when developing boost::python or SIP-based extension modules, we have 
repeatedly had the need to use C++ classes or functions from other modules.  
(E.g. I might have a foocmodule and a corresponding unit test module.)
A good use case is a boost::python-like registry (with a slightly different 
purpose), which should be shared across multiple modules.

However, I have always been facing the problem that this seemed to require a 
common shared library (? la libboost_python), but this is
a) overkill e.g. in the case of only a single required function
   (get_registry()), and
b) not supported (AFAIK) by distutils/setuptools (i.e. building shared
   libraries requires lengthy, possibly non-portable hacks).

It would be great if I could somehow tell the dynamic linker to resolve 
against the function definition in the other module (and in fact it works at 
least on Linux to link against libfoocmodule.so, but IIRC that's not 
portable).

What are your recommendations/experiences?

Thanks,
  Hans

From sahasr at naman.ms  Fri Jul 31 10:52:53 2009
From: sahasr at naman.ms (Sahasranaman MS)
Date: Fri, 31 Jul 2009 14:22:53 +0530
Subject: [C++-sig] boost::python::import crashes when called from a dll
Message-ID: <ec3874c70907310152w512390e1x63d0d5575a941bb1@mail.gmail.com>

I am having a problem. I am running Windows XP, Python 2.5.4.

I have a custom library which uses python's urllib2 to download data. The
import statement inside the call looks like this:

    boost::python::object urllib = boost::python::import("urllib2");

This works fine when I use my custom library in an executable application.
When I use the library in a dll, while executing, it raises this exception:

    WindowsError: exception: access violation reading 0x00000008

It is happening exactly at this import statement. I tried importing the
'sys' module first, setting library search path and then run, but its
failing to import even the sys module, and fails with the same error. Any
advice on how I could fix this?


--
Sahasranaman MS
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090731/0b9d6b12/attachment.htm>