From zf.yuan.y at gmail.com  Wed Jan  7 15:19:57 2015
From: zf.yuan.y at gmail.com (=?UTF-8?B?6KKB5oyv6aOe?=)
Date: Wed, 7 Jan 2015 22:19:57 +0800
Subject: [C++-sig] boost python wrapping boost message queue problem
Message-ID: <CADTdNXniyDPU0FOhpoh4yJQx_hPXB_ifevxs4k0rVY=NisZWzg@mail.gmail.com>

Hi all.

I wrapped the boost ipc message queue into Python using Boost.Python.
Everything goes fine except I found that

1. the try_receive / try_send will block python when the mq is empty / full.
2. when I use the wrapped message queue in one Python thread, any
other Python thread would be blocked and won't execute.

I'd appreciate a lot for your help!

Below is my C++ wrapping codes.

===============================================

class message_queue {
public:
  message_queue(unsigned int, const char*, size_t, size_t);
  ~message_queue();
  void send(boost::python::str str_buffer, size_t m_size);
  void try_send(boost::python::str str_buffer, size_t m_size);
  boost::python::str receive(size_t m_size);
  boost::python::str try_receive(size_t m_size);
private:
  bi::message_queue* p_message_queue;
};

message_queue::message_queue(unsigned int flag, const char* mq_name,
size_t mq_length = 1, size_t m_size = 1) {
  switch(flag) {
  case 0:
    p_message_queue = new bi::message_queue(bi::create_only, mq_name,
mq_length, m_size);
    break;
  case 1:
    p_message_queue = new bi::message_queue(bi::open_or_create,
mq_name, mq_length, m_size);
    break;
  case 2:
    p_message_queue = new bi::message_queue(bi::open_only, mq_name);
    break;
  }
}

message_queue::~message_queue() {
  delete p_message_queue;
}

void message_queue::send(boost::python::str str_buffer, size_t m_size) {
  assert(m_size ==
boost::python::extract<size_t>(str_buffer.attr("__len__")()));
  const char* buffer = boost::python::extract<const char*>(str_buffer);

  p_message_queue->send(buffer, m_size, 0);
}

void message_queue::try_send(boost::python::str str_buffer, size_t m_size) {
  assert(m_size ==
boost::python::extract<size_t>(str_buffer.attr("__len__")()));
  const char* buffer = boost::python::extract<const char*>(str_buffer);
  p_message_queue->try_send(buffer, m_size, 0);
  return;
}

boost::python::str message_queue::receive(size_t m_size) {
  char* buffer = new char[m_size];
  bi::message_queue::size_type recvd_size;
  unsigned int priority;
  p_message_queue->receive(buffer, m_size, recvd_size, priority);
  return boost::python::str(buffer, m_size);
}

boost::python::str message_queue::try_receive(size_t m_size) {
  char* buffer = new char[m_size];
  bi::message_queue::size_type recvd_size;
  unsigned int priority;
  p_message_queue->try_receive(buffer, m_size, recvd_size, priority);
  return boost::python::str(buffer, m_size);
}

bool shared_memory_object_remove(const char* shm_name) {
  return bi::shared_memory_object::remove(shm_name);
}

BOOST_PYTHON_MODULE(boost_ipc) {
  using namespace boost::python;
  class_<message_queue>("message_queue", init<unsigned int, const
char*, optional<size_t, size_t> >())
    .def("send", &message_queue::send)
    .def("receive", &message_queue::receive)
    .def("try_send", &message_queue::send)
    .def("try_receive", &message_queue::receive);
  def("message_queue_remove", message_queue_remove);
  scope().attr("O_CREATE_ONLY") = 0;
  scope().attr("O_OPEN_OR_CREATE") = 1;
  scope().attr("O_OPEN_ONLY") = 2;
}

===========================================

Below is the Python Script that will block on try_receive
function.

===========================================

#!/usr/bin/python

import boost_ipc

boost_ipc.message_queue_remove("test_mq")
mq1 = boost_ipc.message_queue(boost_ipc.O_CREATE_ONLY, "test_mq", 1, 5)
mq2 = boost_ipc.message_queue(boost_ipc.O_OPEN_ONLY,      "test_mq")

mq1.try_send("hello", 5)
print mq2.receive(5)
print mq2.try_receive(5)

From Holger.Joukl at LBBW.de  Fri Jan  9 09:40:07 2015
From: Holger.Joukl at LBBW.de (Holger Joukl)
Date: Fri, 9 Jan 2015 09:40:07 +0100
Subject: [C++-sig] boost python wrapping boost message queue problem
In-Reply-To: <CADTdNXniyDPU0FOhpoh4yJQx_hPXB_ifevxs4k0rVY=NisZWzg@mail.gmail.com>
References: <CADTdNXniyDPU0FOhpoh4yJQx_hPXB_ifevxs4k0rVY=NisZWzg@mail.gmail.com>
Message-ID: <OFD3127A15.4A7260A3-ONC1257DC8.002AE473-C1257DC8.002F9F31@LBBW.de>

Hi,

> I wrapped the boost ipc message queue into Python using Boost.Python.
> Everything goes fine except I found that
>
> 1. the try_receive / try_send will block python when the mq is empty /
full.
> 2. when I use the wrapped message queue in one Python thread, any
> other Python thread would be blocked and won't execute.

I don't know about the boost ipc message queue but my guess is try_receive
(...) and
try_send(...) are blocking calls(?) and you'll need to allow Python threads
to
run while blocking.

I.e. you'd need to release Python's global interpreter lock (GIL) before
the blocking call and reacquire it when the call returns by using the
Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS macros.

Looks to me like you could do so in your message_queue class's methods,
something similar to

boost::python::str message_queue::try_receive(size_t m_size) {
  char* buffer = new char[m_size];
  bi::message_queue::size_type recvd_size;
  unsigned int priority;
  Py_BEGIN_ALLOW_THREADS;
  p_message_queue->try_receive(buffer, m_size, recvd_size, priority);
  Py_END_ALLOW_THREADS;
  return boost::python::str(buffer, m_size);
}

(fully untested code!)

Note that there might be additional intricacies with regard to waking up
the
blocking call when a message arrives in the queue and/or signal handling,
e.g. making
the program interruptible while in the blocking call.
This depends on the queue implementation and your requirements (play well
with
signal handlers registered on the Python side, ....)

The snippets below are samples from our wrapping of a message queue of the
commercial
messaging middleware TIB/Rendezvous - it's old code and could be improved
but works for us and might just give you an idea plus some documentation.

Hope it helps,
Holger


// file helpers/debug_config.hpp
#ifndef DEBUG_CONFIG_HPP
#define DEBUG_CONFIG_HPP

// __GNUC__ is defined for both C and C++
#if defined(__GNUC__)
#define __PRETTY_FUNCTION__ __PRETTY_FUNCTION__
#elif (defined(__SUNPRO_C) || defined(__SUNPRO_CC))
#define __PRETTY_FUNCTION__ __func__
#else
#define __PRETTY_FUNCTION__ __func__
#endif // if defined(__GNUC__)

#include <iostream>


// See proposed solution by Graham Dumpleton here for debug macros:
// http://www.velocityreviews.com/forums/t280342-debugging-macros.html
// IMHO this has 2 advantages:
// - integrates much nicer with code formatting than #ifdef DEBUG ...
#endif
// - gets compiled even if DEBUG is unset so the debug code gets checked
//   by the compiler (and optimized away, then, hopefully :)
//FIXME: Does this *really* get optimized away? For gcc (4.6.1) the unused
//FIXME: output code is left out (checked using the strings cmd), but for
//FIXME: Solaris studio i can grep the debug code in the resulting object
//FIXME: file in a simple example program... Must we care?


// DEBUG switches on all debugging output
#ifdef DEBUG
#define DEBUG_TRACE
#define DEBUG_FUNC
#define DEBUG_CTOR
#define DEBUG_DTOR
#define DEBUG_MODULES
#endif


// use these switches to switch on specific debug tracing
#ifdef DEBUG_TRACE
#define DEBUGTRACER_TRACE if (0) ; else std::cout
#define DEBUGTRACER_CONDITIONAL if (0) ; else
#else
#define DEBUGTRACER_TRACE if (1) ; else std::cout
#define DEBUGTRACER_CONDITIONAL if (1) ; else
#endif

#ifdef DEBUG_FUNC
#define DEBUGTRACER_FUNC if (0) ; else std::cout
#else
#define DEBUGTRACER_FUNC if (1) ; else std::cout
#endif

#ifdef DEBUG_CTOR
#define DEBUGTRACER_CTOR if (0) ; else std::cout
#else
#define DEBUGTRACER_CTOR if (1) ; else std::cout
#endif

#ifdef DEBUG_DTOR
#define DEBUGTRACER_DTOR if (0) ; else std::cout
#else
#define DEBUGTRACER_DTOR if (1) ; else std::cout
#endif

#ifdef DEBUG_MODULES
#define DEBUGTRACER_MODULES if (0) ; else std::cout
#else
#define DEBUGTRACER_MODULES if (1) ; else std::cout
#endif

#endif //DEBUG_CONFIG_HPP




// file tibrvqueue.hpp
// See tibrvque.cpp for extensive background documentation

#ifndef _ThreadedTibrvQueue_hpp
#define _ThreadedTibrvQueue_hpp

#include <tibrvcpp.h>
#include <cassert>
#include <list>
#include <signal.h>

#include "helpers/debug_config.hpp"

class ThreadedTibrvQueue : public TibrvQueue
{
    public:
        ThreadedTibrvQueue();
        void waitEvent();
        bool waitEvent(double timeout);

        TibrvStatus destroy();
        TibrvStatus destroy(TibrvQueueOnComplete* completeCB, const void*
closure = NULL);

    private:
        static void _myHook(tibrvQueue eventQueue, void* closure);
        static void sighandler(int);
        static void installSignalHandlers();
        static void removeSignalHandlers();
    private:
        pthread_cond_t m_cond;
        pthread_mutex_t m_mutex;
        typedef std::list<ThreadedTibrvQueue*> instance_list;
        static instance_list m_instances;
        static pthread_mutex_t m_instances_mutex;
        static int m_waiting;
        typedef void (*sighandler_t)(int);
        static sighandler_t m_sighandler[NSIG];
};

#endif



// file tibrvqueue.cpp
// ThreadedTibrvQueue: A TibrvQueue subclass to enable multithreaded python
// programs with Rendezvous
//
// As the TIB/Rv library does not know anything about Python, its
// functions/methods do of course not release the python GIL, even in
situations
// where this would be desired or even essential. E.g.
//  * TibrvQueue::dispatch()
//  * TibrvQueue::timedDispatch()
// won't release the GIL prior to blocking (either endlessly or for a set
interval).
//
// This means during waiting for an TIB/Rv event (incoming UDP message
(which is
// also I/O), timer event, I/O Event (platform-dependant)) no other thread
will
// be allowed to run, as the thread that called dispatch/timedDispatch
(usually
// the main thread in its main loop) holds the GIL, blocking other threads
from
// doing their workings.
// Therefore, it is essential to release the GIL in such situations.
//
// The ThreadedTibrvQueue solves this by introducing
//  * the waitEvent() method which
//    1. installs a signal handler to catch "break" signals
//    2. releases the GIL
//    3. starts waiting for the condition member variable m_cond (i.e. goes
to
//       sleep)
//    and
//  * a hook callback method _myHook() (which gets called by the TIB/Rv
event
//    mechanisms whenever an event is put into the queue) that signals the
//    condition variable m_cond. This notifies the waiting processing in
//    waitEvent() to continue running, which will then remove the signal
handler
//    and reacquire the GIL.
// The signal handler is needed to make the program interruptible e.g. by
// ctrl-c while waiting for an event to be put in the queue; it will also
signal
// the m_cond condition variable to allow waitEvent() to proceed.
//
// Used together with the normal dispatch/timedDispatch methods this allows
for
// running multithreaded programs. You'll have to use this in combination,
i.e.
// the dispatch call must be preceded by a waitEvent() call so your main Rv
loop
// implementation will basically look like this:
//
// queue = ThreadedTibrvQueue()
// while 1:
//     queue.waitEvent()
//     queue.dispatch()
//
// Possible improvements:
//  * make waitEvent() a private method and override the
//    dispatch()/timedDispatch() methods (poll() also?) to call the private
//    waitEvent(). This would make for a better API interface.
//  * unify ThreadedTibrvQueue constructor behaviour with standard
TibrvQueue
//    behaviour: ThreadedTibrvQueue performs a create() call in the
constructor
//    while TibrvQueue does not and you have to to this in a second step
//
// Alternatives:
// It is essentially possible to run multithreaded programs with the
standard
// Tibrv queue if you forgo using blocking dispatch; however, caution is
// recommended. Imagine you try to run a threaded program with another
running
// thread using a main thread loop like
//
// sys.setcheckinterval(1) # minimal "tick" count for thread switching
// queue = Tibrv.defaultQueue()
// while 1:
//     queue.timedDispatch(0.1)
//     print "..."
//
// This *will not work*, i.e. the other running thread will not be able to
run
// properly! The reason for this is that when timedDispatch() returns,
while
// the main loop thread will release the GIL it is itself waiting to
reaqcuire
// it immediately. This means we now have 2 threads waiting to proceed: Our
// other thread and the main thread wanting to continue with the next loop
// iteration. The way thread switching is implemented (at least up to
Python 2.6)
// one or the other may be scheduled to run, depending on OS scheduler
decisions
// (see http://www.dabeaz.com/python/UnderstandingGIL.pdf for a thorough
// explanation). This can result in the other thread not ever getting the
chance
// to continue.
// It is still possible to support threaded programs with a construct like
//
// queue = Tibrv.defaultQueue()
// while 1:
//     queue.timedDispatch(0.1)
//     time.sleep(0.0001)
//
// Blocking I/O forces release of the GIL, as does time.sleep() (the Python
C
// macro Py_BEGIN_ALLOW_THREADS basically does that: "hey, I~m entering
some
// blocking operation, here~s the GIL back" [quoted from
//
http://jessenoller.com/2009/02/01/python-threads-and-the-global-interpreter-lock/,
// which is also an excellent introduction to threading and the GIL]). So
this
// really gives our 2nd thread the possibility to run.
// Note, however, that while the main thread waits in timedDispatch(X) for
X
// seconds
//  * no signals will be retrieved to allow program interruption
//  * the other thread(s) are not able to run
// which means X must be reasonably small. In consequence, such a solution
would
// be suboptimal at best.

#include <boost/python.hpp> // silence  warning: "_FILE_OFFSET_BITS"
redefined
#include "tibrvqueue.hpp"
#include <queue.h>
#include <boost/python/errors.hpp>
//#include <boost/python/objects.hpp>
//#include <boost/python/conversions.hpp>
#include <boost/format.hpp>
#include <iostream>
#include <string>
#include <pythread.h>
#include <pthread.h>
#include <math.h>
#include <signal.h>
#include <string.h>
#include <errno.h>
#include <sys/time.h>
//#undef DEBUG
//#define DEBUG


#ifdef DEBUG
// FIXME: This is provided on Solaris but not Linux; is there some
equivalent#
// we could use instead?
#ifdef SIG2STR_MAX
std::ostream& operator<<(std::ostream& to, const sigset_t *sigset)
{
    char buf[SIG2STR_MAX];
    for (int s=0;s<MAXSIG;s++) {
        sig2str(s, buf);
        to << buf << "(" << s << "):" << (sigismember(sigset, s)) <<
std::endl;
    }
    return to;
}

std::ostream &operator<<(std::ostream &to,const struct sigaction& sa)
{
  to << "sa_flags=" << sa.sa_flags << "(";
  to << ( (sa.sa_flags & SA_SIGINFO ) ? "SA_SIGINFO " : "" );
  to << ( (sa.sa_flags & SA_RESTART ) ? "SA_RESTART " : "" );
  to << ( (sa.sa_flags & SA_ONSTACK ) ? "SA_ONSTACK " : "" );
  to << ( (sa.sa_flags & SA_NODEFER ) ? "SA_NODEFER " : "" );
  to << ( (sa.sa_flags & SA_RESETHAND ) ? "SA_RESETHAND " : "" );
  to << ") sa_handler=" << (void*)sa.sa_handler << " sa_sigaction=" <<
(void*)sa.sa_sigaction;
  return to;
}

void dumpsigs()
{
    sigset_t sigset;
    pthread_sigmask(0, 0, &sigset);
    std::cout << "thread: " << pthread_self() << " sigset: " << &sigset <<
std::endl;

    struct sigaction sa;
    sigaction(SIGINT, 0, &sa);
    std::cout << "SIGINT:" << sa << std::endl;
    sigaction(SIGTERM, 0, &sa);
    std::cout << "SIGTERM:" << sa << std::endl;
    sigaction(SIGQUIT, 0, &sa);
    std::cout << "SIGQUIT:" << sa << std::endl;
}
#endif // ifdef SIG2STR_MAX
#endif // ifdef DEBUG

ThreadedTibrvQueue::instance_list ThreadedTibrvQueue::m_instances;
pthread_mutex_t ThreadedTibrvQueue::m_instances_mutex;
int ThreadedTibrvQueue::m_waiting = 0;
ThreadedTibrvQueue::sighandler_t ThreadedTibrvQueue::m_sighandler[NSIG];


ThreadedTibrvQueue::ThreadedTibrvQueue() : TibrvQueue()
{
    // this section is guarded by the global interpreter lock
    if (m_instances.size() == 0) {
        pthread_mutex_init(&m_instances_mutex, 0);
    }
    // prevent signal handlers from iterating m_instances at this point in
time
    pthread_mutex_lock( &m_instances_mutex );
    m_instances.push_back(this);
    pthread_mutex_unlock( &m_instances_mutex );

    TibrvStatus status = create();
    if (status != TIBRV_OK) {
        throw std::runtime_error((boost::format("TibrvQueue::create: %1%")
% status.getText()).str());
    }
    pthread_mutex_init(&m_mutex, 0);
    pthread_cond_init(&m_cond, 0);
    status = tibrvQueue_SetHook(getHandle(), &_myHook, this);
    if (status != TIBRV_OK) {
        throw std::runtime_error((boost::format("tibrvQueue_SetHook: %1%")
% status.getText()).str());
    }
}


TibrvStatus ThreadedTibrvQueue::destroy()
{
    DEBUGTRACER_FUNC << "--> " << __PRETTY_FUNCTION__ << std::endl;
    return destroy(NULL, NULL);
}


// must not repeat closure's default arg value here:
// in tibrvqueue.hpp:  TibrvStatus destroy(TibrvQueueOnComplete*
completeCB, const void* closure=NULL);
TibrvStatus ThreadedTibrvQueue::destroy(TibrvQueueOnComplete* completeCB,
const void* closure)
{
    DEBUGTRACER_FUNC << "--> " << __PRETTY_FUNCTION__ << std::endl;

    if (getHandle() != TIBRV_INVALID_ID) {
        TibrvStatus status = tibrvQueue_RemoveHook(getHandle());
        pthread_cond_destroy(&m_cond);
        pthread_mutex_destroy(&m_mutex);
        if (status != TIBRV_OK) {
            throw std::runtime_error((boost::format("tibrvQueue_RemoveHook:
%1%") % status.getText()).str());
        }
        pthread_mutex_lock( &m_instances_mutex );
        m_instances.remove(this);
        pthread_mutex_unlock( &m_instances_mutex );
    }
    DEBUGTRACER_FUNC << "<-- " << __PRETTY_FUNCTION__ << std::endl;
    return TibrvQueue::destroy(completeCB, closure);
}


void ThreadedTibrvQueue::waitEvent()
{
    DEBUGTRACER_FUNC << "--> " << __PRETTY_FUNCTION__ << std::endl;

    tibrv_u32 available = 0;
    TibrvStatus status;
    int cond_status = 0;
    assert(m_waiting>=0);
    if (!m_waiting) installSignalHandlers();
    m_waiting++;

    Py_BEGIN_ALLOW_THREADS;
    DEBUGTRACER_TRACE << __PRETTY_FUNCTION__ << ": pthread_mutex_lock" <<
std::endl;

    pthread_mutex_lock( &m_mutex );

    status = getCount(available);
//     while (!available && cond_status == 0 && status == TIBRV_OK) {
    if (!available && status == TIBRV_OK) {
        cond_status = pthread_cond_wait(&m_cond, &m_mutex);
        status = getCount(available);
        DEBUGTRACER_TRACE << __PRETTY_FUNCTION__ << "available = " <<
available << std::endl;
    }
//     assert(available);
    DEBUGTRACER_TRACE << __PRETTY_FUNCTION__ << ": pthread_mutex_unlock" <<
std::endl;

    pthread_mutex_unlock( &m_mutex );
    Py_END_ALLOW_THREADS;

    m_waiting--;
    assert(m_waiting>=0);
    if (!m_waiting) removeSignalHandlers();

    if (status != TIBRV_OK) {
        throw std::runtime_error((boost::format("TibrvQueue::getCount:
%1%") % status.getText()).str());
    }
    if (cond_status != 0) {
        //throw std::runtime_error(std::string("pthread_cond_wait
failed"));
        // to raise a standard python exception, either create a custom
exception
        // and register an exception translator, or do it manually like
this:
        // FIXME: Should go to a helper lib that takes the message and the
Exception
        const char message[] = "pthread_cond_wait failed";
        PyErr_SetString(PyExc_OSError, message);
        boost::python::throw_error_already_set();
    }
#if 0
    if (!available) {
        boost::python::tuple exc_value(boost::python::ref
(BOOST_PYTHON_CONVERSION::to_python(EINTR)),
                                       boost::python::ref
(BOOST_PYTHON_CONVERSION::to_python(strerror(EINTR))));
        //throw boost::python::OSError(exc_value.reference().get());
        std::string message = (boost::format("%1%") % exc_value.reference
().get()).str();
        PyErr_SetString(PyExc_OSError, message.c_str());
        boost::python::throw_error_already_set();
    }
#endif

    DEBUGTRACER_TRACE << __PRETTY_FUNCTION__ << ": " << available <<
std::endl;
}

static void set_interval(struct timespec* timeout_spec, double timeout)
{
    double integral, fractional;
    time_t sec; long nsec;
    fractional = modf(timeout, &integral);
    sec = (time_t)integral;
    nsec = (long)(fractional*1.0e9);

#ifdef sun
    clock_gettime(CLOCK_REALTIME, timeout_spec);
#else
    struct timeval tv;
    gettimeofday(&tv, 0);
    timeout_spec->tv_sec = tv.tv_sec;
    timeout_spec->tv_nsec = tv.tv_usec*1000;
#endif
    timeout_spec->tv_sec += (sec + (timeout_spec->tv_nsec + nsec) /
1000000000);
    timeout_spec->tv_nsec = (timeout_spec->tv_nsec + nsec) % 1000000000;
}


bool ThreadedTibrvQueue::waitEvent(double timeout)
{
    DEBUGTRACER_FUNC << "--> " << __PRETTY_FUNCTION__ << std::endl;

    tibrv_u32 available = 0;
    TibrvStatus status;
    int cond_status = 0;

    assert(m_waiting>=0);
    if (!m_waiting) installSignalHandlers();
    m_waiting++;

    Py_BEGIN_ALLOW_THREADS;
    pthread_mutex_lock( &m_mutex );
    struct timespec timeout_spec;

    set_interval(&timeout_spec, timeout);
    status = getCount(available);

//     while (!available && cond_status != ETIMEDOUT && status == TIBRV_OK)
{
    if (!available && status == TIBRV_OK) {
        cond_status = pthread_cond_timedwait(&m_cond, &m_mutex,
&timeout_spec);
        status = getCount(available);
    }

//     assert(available || cond_status == ETIMEDOUT || status != TIBRV_OK);
    pthread_mutex_unlock( &m_mutex );
    Py_END_ALLOW_THREADS;
    m_waiting--;
    assert(m_waiting>=0);
    if (!m_waiting) removeSignalHandlers();

    if (status != TIBRV_OK) {
        throw std::runtime_error((boost::format("TibrvQueue::getCount:
%1%") % status.getText()).str());
    }
    if (cond_status != 0 && cond_status != ETIMEDOUT) {
        // FIXME: Should go to a helper lib
        const char message[] = "pthread_cond_timedwait failed";
        PyErr_SetString(PyExc_OSError, message);
        boost::python::throw_error_already_set();
    }

    DEBUGTRACER_FUNC << "<-- " << __PRETTY_FUNCTION__ << ": " << available
<< std::endl;

    return available != 0;
}

void ThreadedTibrvQueue::_myHook( tibrvQueue eventQueue, void* closure)
{
    DEBUGTRACER_FUNC << "--> " << __PRETTY_FUNCTION__ << std::endl;

    ThreadedTibrvQueue* queue = (ThreadedTibrvQueue*)closure;
    assert(queue->getHandle() == eventQueue);
    pthread_mutex_lock( &(queue->m_mutex) );
    pthread_cond_signal( &(queue->m_cond) );
    pthread_mutex_unlock( &(queue->m_mutex) );

    DEBUGTRACER_FUNC << "<-- " << __PRETTY_FUNCTION__ << std::endl;
}

void ThreadedTibrvQueue::sighandler(int signum)
{
    DEBUGTRACER_FUNC << "--> " << __PRETTY_FUNCTION__ << std::endl;
    DEBUGTRACER_TRACE << "signal " << signum << std::endl;

    pthread_mutex_lock( &m_instances_mutex );
    for (instance_list::iterator iter = m_instances.begin();
iter !=m_instances.end(); iter++) {
        ThreadedTibrvQueue* queue = *iter;
        pthread_mutex_lock( &(queue->m_mutex) );
        DEBUGTRACER_TRACE << __PRETTY_FUNCTION__ << ": pthread_cond_signal"
<< std::endl;
        pthread_cond_signal( &(queue->m_cond) );

        DEBUGTRACER_TRACE << __PRETTY_FUNCTION__ << ":
pthread_mutex_unlock" << std::endl;
        pthread_mutex_unlock( &(queue->m_mutex) );
    }
    pthread_mutex_unlock( &m_instances_mutex );
    sighandler_t oldhandler = m_sighandler[signum];

#ifdef DEBUG
#ifdef SIG2STR_MAX
    char buf[SIG2STR_MAX];
    sig2str(signum, buf);
    DEBUGTRACER_TRACE << __PRETTY_FUNCTION__ << "oldhandler for " << buf <<
"=" << (void*)(oldhandler) << std::endl;
#endif // ifdef SIG2STR_MAX
#endif // ifdef DEBUG
    if (oldhandler != SIG_DFL && oldhandler != SIG_IGN && oldhandler !=
SIG_ERR
#ifdef SIG_HOLD
        && oldhandler != SIG_HOLD
#endif
        ) {
        (*oldhandler)(signum);
    }
    DEBUGTRACER_FUNC << "<-- " << __PRETTY_FUNCTION__ << std::endl;

}


void ThreadedTibrvQueue::installSignalHandlers()
{
    DEBUGTRACER_FUNC << "--> " << __PRETTY_FUNCTION__ << std::endl;

    m_sighandler[SIGINT] = signal(SIGINT, sighandler);
    m_sighandler[SIGTERM] = signal(SIGTERM, sighandler);
    m_sighandler[SIGQUIT] = signal(SIGQUIT, sighandler);

    DEBUGTRACER_FUNC << "<-- " << __PRETTY_FUNCTION__ << std::endl;
}

void ThreadedTibrvQueue::removeSignalHandlers()
{
    DEBUGTRACER_FUNC << "--> " << __PRETTY_FUNCTION__ << std::endl;

    signal(SIGINT, m_sighandler[SIGINT]);
    signal(SIGTERM, m_sighandler[SIGTERM]);
    signal(SIGQUIT, m_sighandler[SIGQUIT]);

   DEBUGTRACER_FUNC << "<-- " << __PRETTY_FUNCTION__ << std::endl;
}


Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart


From coutinho at esrf.fr  Thu Jan  8 18:42:14 2015
From: coutinho at esrf.fr (Tiago Coutinho)
Date: Thu, 08 Jan 2015 18:42:14 +0100
Subject: [C++-sig] [Boost Python] Cannot call base method from within python
 extended class method
Message-ID: <1420738934.22828.4.camel@bcu01ctrl.esrf.fr>

Hello,

My problem is the following:

I am trying to bind a third party C++ library to python using
boost-python.
This library has a class Element and a class Owner (the later takes
ownership 
of an Element object).

To make things interesting, Element defines a pure virtual method
"get_float".
The Element should be sub-classed in python. Here is a simplified
version of the
library:

// m1.hpp
---------------------------------------------------------------------
// m1.hpp

class Element {
public:
    int get_int() { return -1; }
    virtual float get_float() = 0;
};

class Owner {
    Element *m_element;

public:

    Owner(): m_element(NULL) {}
    ~Owner() { delete m_element; }
    
    void set_element(Element* e) {
	    delete m_element;
        m_element = e;
    };

    Element* get_element() { return m_element; }
};


and here is how I am binding to python:

// m1_py.cpp
----------------------------------------------------------------

class ElementWrap : public Element,
		    public wrapper<Element>
{
public:
    PyObject* self;

    ElementWrap(PyObject* self_): Element(), self(self_)
    { Py_INCREF(self); } 
    
    ~ElementWrap() { Py_DECREF(self); }
    
    virtual float get_float() { return call_method<float>(self,
"get_float"); }

};

void set_element(Owner& owner, auto_ptr<Element> e) {
    owner.set_element(e.get());
    e.release();
}

BOOST_PYTHON_MODULE(m1) {
    
    class_<Element, auto_ptr<ElementWrap>,
boost::noncopyable>("Element", init<>())
	.def("get_int", &Element::get_int)
	.def("get_float", &Element::get_float)
	;
    register_ptr_to_python<auto_ptr<Element> >();
    implicitly_convertible<auto_ptr<ElementWrap>, auto_ptr<Element> >();


    class_<Owner>("Owner", init<>())
	.def("set_element", &set_element)
	.def("get_element", &Owner::get_element, return_internal_reference<>())
    ;
}

... but when I use it like this:

# m2.py
-----------------------------------------------------------------------

import m1

class SubElement(m1.Element):

    def get_float(self):
        return 4.5678 + self.get_int()

element = SubElement()
owner = m1.Owner()
owner.set_element(element)
element = owner.get_element()

# ops! 
print (element.get_float())


I get an exception in the last line:
Traceback (most recent call last):
  File "m2.py", line 16, in <module>
    print (element.get_float())
  File "m2.py", line 8, in get_float
    return 1.0 + self.get_int()
Boost.Python.ArgumentError: Python argument types in
    Element.get_int(SubElement)
did not match C++ signature:
    get_int(Element {lvalue})

It seems that when I passed ownership to python I lost the ability to
call
a method from the base class.

Can anyone help me?

Thanks in advance

Cheers
Tiago



From p.omta at np-komplete.com  Mon Jan 12 07:40:25 2015
From: p.omta at np-komplete.com (Paul Omta)
Date: Mon, 12 Jan 2015 07:40:25 +0100
Subject: [C++-sig] [Boost Python] Cannot call base method from within
 python extended class method
In-Reply-To: <1420738934.22828.4.camel@bcu01ctrl.esrf.fr>
References: <1420738934.22828.4.camel@bcu01ctrl.esrf.fr>
Message-ID: <54B36C59.4000100@np-komplete.com>

Hi Tiago,

For ElementWrap::get_float, try:
float get_float() { return this->get_override("get_float")(); }

Binding to Python of that function should then be:
.def("get_float", pure_virtual(&Element::get_float))

See also: 
http://www.boost.org/doc/libs/1_57_0/libs/python/doc/tutorial/doc/html/python/exposing.html#python.class_virtual_functions

With regards,

     Paul

P.S. Unrelated to your question: Try to use boost::shared_ptr instead of 
auto_ptr (it's deprecated).
Regrettably, you can't use std::shared_ptr, since Boost Python doesn't 
support it.

On 08/01/15 18:42, Tiago Coutinho wrote:
> Hello,
>
> My problem is the following:
>
> I am trying to bind a third party C++ library to python using
> boost-python.
> This library has a class Element and a class Owner (the later takes
> ownership
> of an Element object).
>
> To make things interesting, Element defines a pure virtual method
> "get_float".
> The Element should be sub-classed in python. Here is a simplified
> version of the
> library:
>
> // m1.hpp
> ---------------------------------------------------------------------
> // m1.hpp
>
> class Element {
> public:
>      int get_int() { return -1; }
>      virtual float get_float() = 0;
> };
>
> class Owner {
>      Element *m_element;
>
> public:
>
>      Owner(): m_element(NULL) {}
>      ~Owner() { delete m_element; }
>      
>      void set_element(Element* e) {
> 	    delete m_element;
>          m_element = e;
>      };
>
>      Element* get_element() { return m_element; }
> };
>
>
> and here is how I am binding to python:
>
> // m1_py.cpp
> ----------------------------------------------------------------
>
> class ElementWrap : public Element,
> 		    public wrapper<Element>
> {
> public:
>      PyObject* self;
>
>      ElementWrap(PyObject* self_): Element(), self(self_)
>      { Py_INCREF(self); }
>      
>      ~ElementWrap() { Py_DECREF(self); }
>      
>      virtual float get_float() { return call_method<float>(self,
> "get_float"); }
>
> };
>
> void set_element(Owner& owner, auto_ptr<Element> e) {
>      owner.set_element(e.get());
>      e.release();
> }
>
> BOOST_PYTHON_MODULE(m1) {
>      
>      class_<Element, auto_ptr<ElementWrap>,
> boost::noncopyable>("Element", init<>())
> 	.def("get_int", &Element::get_int)
> 	.def("get_float", &Element::get_float)
> 	;
>      register_ptr_to_python<auto_ptr<Element> >();
>      implicitly_convertible<auto_ptr<ElementWrap>, auto_ptr<Element> >();
>
>
>      class_<Owner>("Owner", init<>())
> 	.def("set_element", &set_element)
> 	.def("get_element", &Owner::get_element, return_internal_reference<>())
>      ;
> }
>
> ... but when I use it like this:
>
> # m2.py
> -----------------------------------------------------------------------
>
> import m1
>
> class SubElement(m1.Element):
>
>      def get_float(self):
>          return 4.5678 + self.get_int()
>
> element = SubElement()
> owner = m1.Owner()
> owner.set_element(element)
> element = owner.get_element()
>
> # ops!
> print (element.get_float())
>
>
> I get an exception in the last line:
> Traceback (most recent call last):
>    File "m2.py", line 16, in <module>
>      print (element.get_float())
>    File "m2.py", line 8, in get_float
>      return 1.0 + self.get_int()
> Boost.Python.ArgumentError: Python argument types in
>      Element.get_int(SubElement)
> did not match C++ signature:
>      get_int(Element {lvalue})
>
> It seems that when I passed ownership to python I lost the ability to
> call
> a method from the base class.
>
> Can anyone help me?
>
> Thanks in advance
>
> Cheers
> Tiago
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> https://mail.python.org/mailman/listinfo/cplusplus-sig


From Holger.Joukl at LBBW.de  Tue Jan 13 12:13:52 2015
From: Holger.Joukl at LBBW.de (Holger Joukl)
Date: Tue, 13 Jan 2015 12:13:52 +0100
Subject: [C++-sig] [Boost Python] Cannot call base method from within
 python extended class method
In-Reply-To: <54B36C59.4000100@np-komplete.com>
References: <1420738934.22828.4.camel@bcu01ctrl.esrf.fr>
 <54B36C59.4000100@np-komplete.com>
Message-ID: <OFB0B8551F.BB7C58E8-ONC1257DCC.003B5FDA-C1257DCC.003DB229@LBBW.de>

Hi,

> For ElementWrap::get_float, try:
> float get_float() { return this->get_override("get_float")(); }
>
> Binding to Python of that function should then be:
> .def("get_float", pure_virtual(&Element::get_float))
>
> See also:
> http://www.boost.org/doc/libs/1_57_0/libs/python/doc/tutorial/doc/
> html/python/exposing.html#python.class_virtual_functions
>
> With regards,
>
>      Paul

Doesn't solve this problem:

$ cat m1_py.cpp
// m1_py.cpp
#include "boost/python.hpp"
#include <memory>
#include "m1.hpp"



class ElementWrap : public Element, public boost::python::wrapper<Element>
{
public:
    // Element::get_float is pure virtual so no need for default
implementation
    // if no python override found
    virtual float get_float() {
        boost::python::override py_override = this->get_override
("get_float");
        if (!py_override) {
            // want a nice error message if no Python override of the
            // pure virtual method has been found
            boost::python::detail::pure_virtual_called();
        }
        return py_override();
    }
};



void set_element(Owner& owner, std::auto_ptr<ElementWrap> e)
{
    owner.set_element(e.get());
    e.release();
}


BOOST_PYTHON_MODULE(m1) {

    boost::python::class_<ElementWrap, std::auto_ptr<ElementWrap>,
                          boost::noncopyable>(
                                  "Element",
                                   boost::python::init<>())
    .def("get_int", &Element::get_int)
    .def("get_float", boost::python::pure_virtual(&Element::get_float))
    ;


    boost::python::class_<Owner>("Owner", boost::python::init<>())
    .def("set_element", &set_element)
    .def("get_element", &Owner::get_element,
boost::python::return_internal_reference<>())
    ;
}
0 $ cat m2.py
# m2.py

import m1

class SubElement(m1.Element):

    def get_float(self):
        return 4.5678 + self.get_int()

element = SubElement()
print "element:", element
owner = m1.Owner()
print "element.get_int():",
print element.get_int()
print "owner.set_element(element)..."
owner.set_element(element)
print "element:", element
try:
    print "element.get_int():",
    print element.get_int()
except Exception as e:
    print "*** Exception caught:"
    print e
print "element = owner.get_element(element)..."
element = owner.get_element()
print "element:", element

# ops!
print (element.get_float())

0 $ PYTHONPATH=./bin/gcc-4.6.1/debug/ python2.7 m2.py
element: <__main__.SubElement object at 0x7fef49783100>
element.get_int(): -1
owner.set_element(element)...
element: <__main__.SubElement object at 0x7fef49783100>
element.get_int(): *** Exception caught:
Python argument types in
    Element.get_int(SubElement)
did not match C++ signature:
    get_int(Element {lvalue})
element = owner.get_element(element)...
element: <__main__.SubElement object at 0x7fef49783100>
Traceback (most recent call last):
  File "m2.py", line 29, in <module>
    print (element.get_float())
  File "m2.py", line 8, in get_float
    return 4.5678 + self.get_int()
Boost.Python.ArgumentError: Python argument types in
    Element.get_int(SubElement)
did not match C++ signature:
    get_int(Element {lvalue})


We don't even get to the code to invoke a Python override here.

Note how after owner.set_element() the SubElement instance becomes
"inoperational".

I don't quite understand the boost.python inner workings but the call to
void set_element(Owner& owner, std::auto_ptr<ElementWrap> e)
transfers ownership to the local auto_ptr e, so the auto_ptr held from
the Python side gets modified (released ownership, now pointing to null).

Much the same result as with the original implementation (although I seem
to get the same object id for the get_element() return value here but that
might be coincidence).

Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart


From coutinho at esrf.fr  Tue Jan 13 15:10:06 2015
From: coutinho at esrf.fr (Tiago Coutinho)
Date: Tue, 13 Jan 2015 15:10:06 +0100
Subject: [C++-sig] [Boost Python] Cannot call base method from within
 python extended class method
In-Reply-To: <OFB0B8551F.BB7C58E8-ONC1257DCC.003B5FDA-C1257DCC.003DB229@LBBW.de>
References: <1420738934.22828.4.camel@bcu01ctrl.esrf.fr>
 <54B36C59.4000100@np-komplete.com>
 <OFB0B8551F.BB7C58E8-ONC1257DCC.003B5FDA-C1257DCC.003DB229@LBBW.de>
Message-ID: <1421158206.7120.36.camel@bcu01ctrl.esrf.fr>

Hi,

On Tue, 2015-01-13 at 12:13 +0100, Holger Joukl wrote:
> Hi,
> 
> > For ElementWrap::get_float, try:
> > float get_float() { return this->get_override("get_float")(); }
> >
> > Binding to Python of that function should then be:
> > .def("get_float", pure_virtual(&Element::get_float))
> >
> > See also:
> > http://www.boost.org/doc/libs/1_57_0/libs/python/doc/tutorial/doc/
> > html/python/exposing.html#python.class_virtual_functions
> >
> > With regards,
> >
> >      Paul
> 
> Doesn't solve this problem:

I confirm what Holger said. Paul's reply doesn't solve the problem.
In fact, if I change the ElementWrap::get_float to:
virtual float get_float() { return this->get_override("get_float")(); }

I get in python:
Traceback (most recent call last):
  File "m2.py", line 18, in <module>
    print (element.get_float())
TypeError: 'NoneType' object is not callable

Which is very odd. It seems that boost get_override is not able to find
the overloaded python SubElement.get_float() method.

For reference I am using a Debian 7 machine 64bits.
Python 2.7.3 
GCC 4.7.2
boost-python 1.49.0

Thanks for all your help.

Kind regards,
  Tiago

> 
> $ cat m1_py.cpp
> // m1_py.cpp
> #include "boost/python.hpp"
> #include <memory>
> #include "m1.hpp"
> 
> 
> 
> class ElementWrap : public Element, public boost::python::wrapper<Element>
> {
> public:
>     // Element::get_float is pure virtual so no need for default
> implementation
>     // if no python override found
>     virtual float get_float() {
>         boost::python::override py_override = this->get_override
> ("get_float");
>         if (!py_override) {
>             // want a nice error message if no Python override of the
>             // pure virtual method has been found
>             boost::python::detail::pure_virtual_called();
>         }
>         return py_override();
>     }
> };
> 
> 
> 
> void set_element(Owner& owner, std::auto_ptr<ElementWrap> e)
> {
>     owner.set_element(e.get());
>     e.release();
> }
> 
> 
> BOOST_PYTHON_MODULE(m1) {
> 
>     boost::python::class_<ElementWrap, std::auto_ptr<ElementWrap>,
>                           boost::noncopyable>(
>                                   "Element",
>                                    boost::python::init<>())
>     .def("get_int", &Element::get_int)
>     .def("get_float", boost::python::pure_virtual(&Element::get_float))
>     ;
> 
> 
>     boost::python::class_<Owner>("Owner", boost::python::init<>())
>     .def("set_element", &set_element)
>     .def("get_element", &Owner::get_element,
> boost::python::return_internal_reference<>())
>     ;
> }
> 0 $ cat m2.py
> # m2.py
> 
> import m1
> 
> class SubElement(m1.Element):
> 
>     def get_float(self):
>         return 4.5678 + self.get_int()
> 
> element = SubElement()
> print "element:", element
> owner = m1.Owner()
> print "element.get_int():",
> print element.get_int()
> print "owner.set_element(element)..."
> owner.set_element(element)
> print "element:", element
> try:
>     print "element.get_int():",
>     print element.get_int()
> except Exception as e:
>     print "*** Exception caught:"
>     print e
> print "element = owner.get_element(element)..."
> element = owner.get_element()
> print "element:", element
> 
> # ops!
> print (element.get_float())
> 
> 0 $ PYTHONPATH=./bin/gcc-4.6.1/debug/ python2.7 m2.py
> element: <__main__.SubElement object at 0x7fef49783100>
> element.get_int(): -1
> owner.set_element(element)...
> element: <__main__.SubElement object at 0x7fef49783100>
> element.get_int(): *** Exception caught:
> Python argument types in
>     Element.get_int(SubElement)
> did not match C++ signature:
>     get_int(Element {lvalue})
> element = owner.get_element(element)...
> element: <__main__.SubElement object at 0x7fef49783100>
> Traceback (most recent call last):
>   File "m2.py", line 29, in <module>
>     print (element.get_float())
>   File "m2.py", line 8, in get_float
>     return 4.5678 + self.get_int()
> Boost.Python.ArgumentError: Python argument types in
>     Element.get_int(SubElement)
> did not match C++ signature:
>     get_int(Element {lvalue})
> 
> 
> We don't even get to the code to invoke a Python override here.
> 
> Note how after owner.set_element() the SubElement instance becomes
> "inoperational".
> 
> I don't quite understand the boost.python inner workings but the call to
> void set_element(Owner& owner, std::auto_ptr<ElementWrap> e)
> transfers ownership to the local auto_ptr e, so the auto_ptr held from
> the Python side gets modified (released ownership, now pointing to null).
> 
> Much the same result as with the original implementation (although I seem
> to get the same object id for the get_element() return value here but that
> might be coincidence).
> 
> Holger
> 
> Landesbank Baden-Wuerttemberg
> Anstalt des oeffentlichen Rechts
> Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
> HRA 12704
> Amtsgericht Stuttgart
> 
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> https://mail.python.org/mailman/listinfo/cplusplus-sig



From Holger.Joukl at LBBW.de  Tue Jan 13 17:07:51 2015
From: Holger.Joukl at LBBW.de (Holger Joukl)
Date: Tue, 13 Jan 2015 17:07:51 +0100
Subject: [C++-sig] [Boost Python] Cannot call base method from within
 python extended class method
In-Reply-To: <1421158206.7120.36.camel@bcu01ctrl.esrf.fr>
References: <1420738934.22828.4.camel@bcu01ctrl.esrf.fr>
 <54B36C59.4000100@np-komplete.com>
 <OFB0B8551F.BB7C58E8-ONC1257DCC.003B5FDA-C1257DCC.003DB229@LBBW.de>
 <1421158206.7120.36.camel@bcu01ctrl.esrf.fr>
Message-ID: <OF01E50010.29A10955-ONC1257DCC.00518BF7-C1257DCC.00589C7D@LBBW.de>

> > > For ElementWrap::get_float, try:
> > > float get_float() { return this->get_override("get_float")(); }
> > >
> > > Binding to Python of that function should then be:
> > > .def("get_float", pure_virtual(&Element::get_float))
> > >
> > > See also:
> > > http://www.boost.org/doc/libs/1_57_0/libs/python/doc/tutorial/doc/
> > > html/python/exposing.html#python.class_virtual_functions
> > >
> > > With regards,
> > >
> > >      Paul
> >
> > Doesn't solve this problem:
>
> I confirm what Holger said. Paul's reply doesn't solve the problem.
> In fact, if I change the ElementWrap::get_float to:
> virtual float get_float() { return this->get_override("get_float")(); }
>
> I get in python:
> Traceback (most recent call last):
>   File "m2.py", line 18, in <module>
>     print (element.get_float())
> TypeError: 'NoneType' object is not callable

Is your element object actually a SubElement instance or a m1.Element
instance here?
You should normally get this only when calling the get_float method of an
m1.Element
instance. This is basically a call to a pure virtual method which is really
only
possible because the original C++ element class is represented to Python by
the derived ElementWrap class, which overrides the pure virtual to make
callbacks from C++ to Python possible.

I put the
boost::python::detail::pure_virtual_called();
line into my example code to have a nicer error message for this case.

No step closer to solving your actual problem, though...

Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart


From vincent.vande.vyvre at telenet.be  Tue Jan 27 07:20:41 2015
From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre)
Date: Tue, 27 Jan 2015 07:20:41 +0100
Subject: [C++-sig] Unicode string in different version of libboost-python
Message-ID: <54C72E39.5030903@telenet.be>

Hi,

I'm working on a python wrapper for the lib libexiv2

This is the port to Python 3 of pyexiv2 written for Python 2

I've two machines, one 32 bits with these versions:

vincent at tiemoko:~$ dpkg --list | grep libboost
ii libboost-python-dev         1.48.0.2            Boost.Python Library 
development files (default version)
ii libboost-python1.46-dev     1.46.1-7ubuntu3     Boost.Python Library 
development files
ii libboost-python1.46.1       1.46.1-7ubuntu3     Boost.Python Library
ii libboost-python1.48.0       1.48.0-3            Boost.Python Library
ii libboost1.46-dev            1.46.1-7ubuntu3     Boost C++ Libraries 
development files

and one 64 bits (this is my main devel machine):

vincent at djoliba:~$ dpkg --list | grep libboost
ii  libboost-date-time1.54.0:amd64  1.54.0-4ubuntu3.1   amd64  set of 
date-time libraries
ii  libboost-python-dev             1.54.0.1ubuntu1     amd64 
Boost.Python Library
ii  libboost-python1.54-dev:amd64   1.54.0-4ubuntu3.1   amd64 
Boost.Python Library development files
ii  libboost-python1.54.0:amd64     1.54.0-4ubuntu3.1   amd64 
Boost.Python Library
rc  libboost-serialization1.46.1    1.46.1-7ubuntu3     amd64 
serialization library for C++
ii  libboost-system1.54.0:amd64     1.54.0-4ubuntu3.1   amd64 Operating 
system (e.g. diagnostics support) library
ii  libboost1.54-dev                1.54.0-4ubuntu3.1   amd64  Boost C++ 
Libraries development files

It's seems these different versions have different behaviours with the 
python string.

On the more recent version, I encode my python string in utf-8 before 
send it to the wrapper and that's works fine.

On the 32 bits machine I've this error:
TypeError: No registered converter was able to produce a C++ rvalue of 
type std::string from this Python object of type bytes

If I send directly the unicode string to the wrapper, without encoding, 
that's solve the problem.

Example:

python:

# xmp tag values are embedded into a list
value = ['d?j? vu']
try:
     tag._setArrayValue([v.encode('utf-8') for v in value])
except TypeError:
     # old libboost-python version
     tag._setArrayValue(value)

-------------------------------------------------------------------------
exiv2wrapper_python.cpp

#include "exiv2wrapper.hpp"

#include "exiv2/exv_conf.h"
#include "exiv2/version.hpp"

#include <boost/python.hpp>

using namespace boost::python;

using namespace exiv2wrapper;

BOOST_PYTHON_MODULE(libexiv2python)
{
     scope().attr("exiv2_version_info") = \
         boost::python::make_tuple(EXIV2_MAJOR_VERSION,
                                   EXIV2_MINOR_VERSION,
                                   EXIV2_PATCH_VERSION);

register_exception_translator<Exiv2::Error>(&translateExiv2Error);

     // Swallow all warnings and error messages written by libexiv2 to 
stderr
     // (if it was compiled with DEBUG or without SUPPRESS_WARNINGS).
     // See https://bugs.launchpad.net/pyexiv2/+bug/507620.
     std::cerr.rdbuf(NULL);

     [skip]

         .def("_setArrayValue", &XmpTag::setArrayValue)

---------------------------------------------------------------------------
exiv2wrapper.cpp

#include "exiv2wrapper.hpp"

#include "boost/python/stl_iterator.hpp"

#include <fstream>

     [skip]

  void XmpTag::setArrayValue(const boost::python::list& values)
{
     // Reset the value
     _datum->setValue(0);

     for(boost::python::stl_input_iterator<std::string> iterator(values);
         iterator != boost::python::stl_input_iterator<std::string>();
         ++iterator)
     {
         _datum->setValue(*iterator);
     }
}
-------------------------------------------------------------------------


Is this difference into two versions is normal ?

I can adapt my python code for that, but how to know the version of 
libboost ?

Here I use "dpkg --list | grep libboost" but that's for debian 
pacckaging system only and my lib must be run on any os (Windows include).

Thanks for all advices.

Vincent

From a.huebl at hzdr.de  Wed Jan 28 15:21:30 2015
From: a.huebl at hzdr.de (Huebl, Axel)
Date: Wed, 28 Jan 2015 15:21:30 +0100
Subject: [C++-sig] Boost.Python: Review of Patches
In-Reply-To: <54C8EBB7.60902@hzdr.de>
References: <54C8EBB7.60902@hzdr.de>
Message-ID: <54C8F06A.9080508@hzdr.de>

Oh I just realized
  boost-langbinding

is the better place to report this.

Sry for cross-posting, I will migrate my question there.


Best,
Axel

On 28.01.2015 15:01, Huebl, Axel wrote:
> Hi,
> 
> I was directed here from the Boost groups homepage
>   http://www.boost.org/community/groups.html#cplussig
> 
> so I hope my question is well placed.
> 
> I was wondering if, how and when patches for Boost.Python (specially bug
> fixes) are merged.
> 
> When I reported two bugs (including patches) beginning of the month
> (that sounds a bit impatient - so if I am just pushing to much just tell
> me) in both trac and on GitHub:
>   https://svn.boost.org/trac/boost/ticket/10932
>   https://svn.boost.org/trac/boost/ticket/10933
> 
> I noticed that there are other pull requests since at least two boost
> stable releases on GitHub
>   https://github.com/boostorg/python/pulls
>   (some of them also with a trac issue)
> 
> that have not yet been reviewed/included/commented-on/closed in the code
> base even if they are most of the time straight forward fixes.
> 
> My background:
>   I am combined Boost.Python with via cmake with an open source, (mpi)
> parallel, many-GPGPU code [1] compiled with nvcc 6.5 and noted some
> minor compile errors in Boost.Python that can be fixed quite easily.
> 
> (Btw: it works like a charm, thank you so much!)
> 
> 
> Best regards,
> Axel Huebl
> 
> [1] http://picongpu.hzdr.de
> 

-- 

Axel Huebl
Phone +49 351 260 3582
https://www.hzdr.de/crp
Computational Radiation Physics
Laser Particle Acceleration Division
Helmholtz-Zentrum Dresden - Rossendorf e.V.

Bautzner Landstrasse 400, 01328 Dresden
POB 510119, D-01314 Dresden
Vorstand: Prof. Dr.Dr.h.c. R. Sauerbrey
          Prof. Dr.Dr.h.c. P. Joehnk
VR 1693 beim Amtsgericht Dresden

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5099 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20150128/40778430/attachment.bin>

From a.huebl at hzdr.de  Wed Jan 28 15:01:27 2015
From: a.huebl at hzdr.de (Huebl, Axel)
Date: Wed, 28 Jan 2015 15:01:27 +0100
Subject: [C++-sig] Boost.Python: Review of Patches
Message-ID: <54C8EBB7.60902@hzdr.de>

Hi,

I was directed here from the Boost groups homepage
  http://www.boost.org/community/groups.html#cplussig

so I hope my question is well placed.

I was wondering if, how and when patches for Boost.Python (specially bug
fixes) are merged.

When I reported two bugs (including patches) beginning of the month
(that sounds a bit impatient - so if I am just pushing to much just tell
me) in both trac and on GitHub:
  https://svn.boost.org/trac/boost/ticket/10932
  https://svn.boost.org/trac/boost/ticket/10933

I noticed that there are other pull requests since at least two boost
stable releases on GitHub
  https://github.com/boostorg/python/pulls
  (some of them also with a trac issue)

that have not yet been reviewed/included/commented-on/closed in the code
base even if they are most of the time straight forward fixes.

My background:
  I am combined Boost.Python with via cmake with an open source, (mpi)
parallel, many-GPGPU code [1] compiled with nvcc 6.5 and noted some
minor compile errors in Boost.Python that can be fixed quite easily.

(Btw: it works like a charm, thank you so much!)


Best regards,
Axel Huebl

[1] http://picongpu.hzdr.de
-- 

Axel Huebl
Phone +49 351 260 3582
https://www.hzdr.de/crp
Computational Radiation Physics
Laser Particle Acceleration Division
Helmholtz-Zentrum Dresden - Rossendorf e.V.

Bautzner Landstrasse 400, 01328 Dresden
POB 510119, D-01314 Dresden
Vorstand: Prof. Dr.Dr.h.c. R. Sauerbrey
          Prof. Dr.Dr.h.c. P. Joehnk
VR 1693 beim Amtsgericht Dresden

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5099 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20150128/4bdee7c4/attachment.bin>