[C++-sig] Boost Python v2 - extending and embedding - memory leaks, compiler warnings

David Abrahams dave at boost-consulting.com
Sun Nov 10 05:46:40 CET 2002

Tex Riddell <texrmex at yahoo.com> writes:

> I posted this question to the boost-users yahoo group, but I realized
> from the response to another message on that group regarding
> Boost.Python that I should send boost python questions to this list
> instead.


> First, I'm excited about the new work that's been done with Boost
> Python v2.  I can't wait to see if it's enough to overcome the
> pitfalls that prevented me from getting past the evaluation stage
> with v1.

Me neither <said with some trepidation ;-)>...

> I'm using VC7 IDE to build my project and link to python22.lib and
> boost_python.lib (built with VC7 command line tools using bjam).  

Proceed at your own risk. Many people report problems which are due to
getting the wrong project configuration using the IDE. I strongly
suggest you use the -n -a option with some of the bjam examples to see
how the command-line options come out, so you can match them.

> I'm attempting to extend and embed in my existing project, so I'm
> not building a dll, but and exe.  First of all, can I do this?  

Yes. However, there are some caveats. Boost.Python keeps some Python
objects alive in global variables (it has some registries and a few
other resources it needs). There is currently no integration with
PyFinalize() to get these cleaned up, so if you call PyFinalize() your
application may crash on exit as the reference counts are decremented
and the dead Python interpreter is invoked. This is something we plan
to address in a future version.

> When it links it says "Creating library ....lib and object ....exp",
What is "it"?

> and still creates an exe.  If I don't include my test module (and
> thus boost/python.hpp) I don't get this message.  I'm assuming that
> by defining a module, boost defines exports and the linker then
> creates the lib and exp, since there were exports?  Although I've
> built dlls before, I really don't know the inner workings of the
> process.

If you're talking about what you're building in the IDE, I have no
clue and no control over that. Boost.Python itself lives in a DLL,
though if you're determined to you there are ways to link it into your
app directly. Normally, the code your module gets from boost.python
generates only /imports/ from boost_python.dll, so I really have no idea.

> The following is a brief example of my module.  At the bottom is a
> simple test function that I'm calling just to test importing of the
> module.  When I call TestModule(), I get 2 memory leaks from the module
> definition section, one from the class definition

Yes, extension modules are never unloaded, and classes in the
extension module are kept alive additionally by Boost.Python's

> , and the other from the method definition.

Of course, since the class holds a reference to its method, the class
keeps the method alive also.


> In addition to the memory leaks, I get a number of compiler warnings,
> similar to the following:
> d:\Boost\boost\python\instance_holder.hpp(18) : warning C4275: non
> dll-interface class 'boost::noncopyable' used as base for dll-interface
> struct 'boost::python::instance_holder'
>         d:\Boost\boost\utility.hpp(50) : see declaration of
> 'boost::noncopyable'
>         d:\Boost\boost\python\instance_holder.hpp(17) : see declaration
> of 'boost::python::instance_holder'

You can ignore this.

> *and*
> d:\Boost\boost\python\detail\exception_handler.hpp(34) : warning C4251:
> 'boost::python::detail::exception_handler::m_impl' : class
> 'boost::function2<R,T0,T1,Policy,Mixin,Allocator>' needs to have
> dll-interface to be used by clients of struct
> 'boost::python::detail::exception_handler'
>         with
>         [
>             R=bool,
>             T0=const boost::python::detail::exception_handler &,
>             T1=const
> boost::function0<void,boost::empty_function_policy,boost::empty_function_mixin,int>
> &,
>             Policy=boost::empty_function_policy,
>             Mixin=boost::empty_function_mixin,
>             Allocator=int
>         ]

You can ignore this, too.

> I was wondering if my project is not set up properly, 

Maybe your warning level is too high ;-)
Actually, I'm kidding. Someone should add #pragmas to suppress these
warnings. Patches graciously accepted.

> and if these
> warnings might have something to do with the memory leaks.


> In v1 there was a vc project you could use for building boost
> python.  This works well if you are planning on linking to a vc
> built project, since you can make sure your settings, paths and libs
> are consistent between the boost python lib and your project.  I
> currently have 3 different sets of build tools (yes, 3 compilers) on
> this box, with several different versions of standard and other
> libraries, so making sure I have control of the paths and settings
> used is very important.

That's why we created Boost.Build. I'm currently building and testing
with 11 different compilers on this machine alone, and many others on
other machines.

> This is one reason I'm weary of the jam environment.  It's yet
> another build environment, one that I'm not familiar with, and one
> that I'm not going to be using with any other projects except to
> build boost python.  

Your loss ;-)

> So a question would be: how easy would it be for me to set up a VC
> project for building boost python?  Has this already been done?  Or
> am I barking up the wrong tree?

Personally I think you're barking up the wrong tree, since maintaining
platform-specific solutions like that is difficutl, but other people
have done it and I believe I'm probably going to end up having to
supply one eventually :(

                       David Abrahams
   dave at boost-consulting.com * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution

More information about the Cplusplus-sig mailing list