[C++-sig] Memory Leaks In VS.NET 2003 With BOOST_PYTHON_MODULE

mjkeyes at sbcglobal.net mjkeyes at sbcglobal.net
Mon Oct 10 12:45:05 CEST 2005


Ralf (et. al),

(note - I hope my newsgroup reader doesn't wrap all this in one line)

I've been playing around with this and I'm still thinking boost leaks
memory in an embedded and extended application.  I use Visual Leak
Detector 1.0 to track memory leaks, and here is where the leak comes
from (in my code):

BOOST_PYTHON_MODULE(PyInputHandler)
{
	
class_<Extensions::PythonInputHandler,boost::noncopyable>("PythonInputHa
ndler")
		.def("HandleInput",
pure_virtual(&SocketLib::InputHandler<std::string>::HandleInput))
		.def("Enter",
pure_virtual(&SocketLib::InputHandler<std::string>::Enter))
		.def("Leave",
pure_virtual(&SocketLib::InputHandler<std::string>::Leave))
		.def("Hungup",
pure_virtual(&SocketLib::InputHandler<std::string>::Hungup))
		.def("Flooded",
pure_virtual(&SocketLib::InputHandler<std::string>::Flooded))
		.def("NoRoom",
pure_virtual(&SocketLib::InputHandler<std::string>::NoRoom))
		.def("Quit", &Extensions::PythonInputHandler::Quit)
		.def("ShutDown",
&Extensions::PythonInputHandler::ShutDown)
		.def("SendGlobal",
&Extensions::PythonInputHandler::SendGlobal)
		.def("AddInputHandler",
&Extensions::PythonInputHandler::AddInputHandler)
		;
}

If I insert VLDDisable() before this class_ block and VLDEnable() after
(which temporarily disables the leak detection for this function call),
I get zero leaks.  With it turned on, here is what I see (repeated 17
times):

(note - the block varies for each leak)
---------- Block 166 at 0x003A7520: 8 bytes ----------
  Call Stack:
    f:\vs70builds\3077\vc\crtbld\crt\src\newop.cpp (12): operator new
    d:\devltool\boost\boost_1_33_0\boost\python\object\py_function.hpp
(121):
boost::python::objects::py_function::py_function<boost::python::detail::
caller<void (__cdecl*)(_object
*),boost::python::default_call_policies,boost::mpl::vector2<void,_object
*> > >
    d:\devltool\boost\boost_1_33_0\boost\python\make_function.hpp (62):
boost::python::detail::make_function_aux<void (__cdecl*)(_object
*),boost::python::default_call_policies,boost::mpl::vector2<void,_object
*>,boost::mpl::int_<0> >
 
d:\devltool\boost\boost_1_33_0\boost\python\detail\make_keyword_range_fn
.hpp (32): boost::python::detail::make_keyword_range_function<void
(__cdecl*)(_object *),boost::python::default_call_policies>
 
d:\devltool\boost\boost_1_33_0\boost\python\detail\make_keyword_range_fn
.hpp (64):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
    d:\devltool\boost\boost_1_33_0\boost\python\init.hpp (338):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
    d:\devltool\boost\boost_1_33_0\boost\python\init.hpp (399):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
    d:\devltool\boost\boost_1_33_0\boost\python\init.hpp (177):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
    d:\devltool\boost\boost_1_33_0\boost\python\def_visitor.hpp (32):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
    d:\devltool\boost\boost_1_33_0\boost\python\def_visitor.hpp (67):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
    d:\devltool\boost\boost_1_33_0\boost\python\class.hpp (226):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
    d:\devltool\boost\boost_1_33_0\boost\python\class.hpp (502):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
    d:\devltool\boost\boost_1_33_0\boost\python\class.hpp (631):
boost::python::detail::make_keyword_range_constructor<boost::mpl::vector
0<boost::mpl::na>,boost::mpl::size<boost::mpl::vector0<boost::mpl::na>
>,boost::python::objects::value_holder<Extensions::PythonInputHandler>,b
oost::python::default_call_policies>
 
d:\devl\muds\testbed\completelibtest\thirdlibtest\lib\pythoninputhandler
.h (153): init_module_PyInputHandler
    0x10006C2B (File and line number not available):
boost::python::detail::exception_handler::operator=
    0x1000EC23 (File and line number not available):
boost::python::detail::init_module
    0x1E0682B1 (File and line number not available): PyImport_FindModule
    0x48747570 (File and line number not available): (Function name
unavailable)
  Data:
	... (whatever data leaked)

It appears in py_function.hpp, the following line is the culprit:

struct py_function
{
    template <class Caller>
    py_function(Caller const& caller)
(line 120 - leak):        : m_impl(new
caller_py_function_impl<Caller>(caller))
    {}
/*...*/
};

This m_impl is a std::auto_ptr<py_function_impl_base>, and the
constructor is given a new caller_py_function_impl.  However, I don't
know that this is ever deallocated.

I could be terribly wrong here, but if I run my code without this
BOOST_INIT_MODULE call, I get no leaks.  Is this something that Python
will deallocate?

Any advice is appreciated!

Thanks!
Matt

"Ralf W. Grosse-Kunstleve" <rwgk at yahoo.com> wrote in message
news:<20051007232909.9702.qmail at web31514.mail.mud.yahoo.com>...
> --- Matt <mjkeyes at sbcglobal.net> wrote:
> > handle<> hInitPtr(PyImport_ImportModule( "Python.SnakeMUD" ));
> 
> I think this is correct code, and I know for sure the handle<> 
> constructor doesn't introduce memory leaks. But I don't know what 
> "Python.SnakeMUD" is. What happens if you import "sys" instead?
> 
> > However, when I uncomment even just the first line (the hInitPtr), I

> > get 17 leaks.
> 
> How do you measure memory leaks?
> E.g. dynamically loading modules is irreversible in C Python. I could 
> imagine some associated allocation overhead that is not de-allocated 
> until the process finishes. If I suspect a leak somewhere my usual 
> response is to call the corresponding functions repeatedly in an 
> infinite loop and manually monitor the total allocation of the process

> (e.g. "top" under Linux or the Task Manager under Windows). With this 
> method even tiny leaks usually show up very quickly on the radar, 
> while one-time static allocations connected to imports or other static

> initializers don't create any visible noise.
> 
> Cheers,
>         Ralf
> 
> 
> 
> 		
> __________________________________
> Yahoo! Mail - PC Magazine Editors' Choice 2005 
> http://mail.yahoo.com




More information about the Cplusplus-sig mailing list