boost: python may cause memory leaks?
Hi, friends, I use VS 7.1 and I have following code: #include "stdafx.h" #include <crtdbg.h> using namespace boost::python; class cPart { public: void set_text( const std::string &t ) { _text = t; } std::string text( void ) { return _text; } private: std::string _text; }; BOOST_PYTHON_MODULE(hello) { class_<cPart, boost::noncopyable >("cPart") .add_property( "text", cPart::text, cPart::set_text ); } int _tmain(int argc, _TCHAR* argv[]) { _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); if (PyImport_AppendInittab("hello", inithello) == -1) return 1; Py_Initialize(); PyRun_SimpleString( "from hello import *\n" "p = cPart()\n" "p.text = 'hello'\n" "print p.text\n" ); Py_Finalize(); getchar(); return 0; } I compiled the code and ran it. Everything was ok except I got memory leaks from the output window of the VS like these: Detected memory leaks! Dumping objects -> {54} normal block at 0x003E5AD8, 8 bytes long. Data: < RE A > 10 52 45 00 83 BE 41 00 {53} normal block at 0x003E5A90, 8 bytes long. Data: <(RE A > 28 52 45 00 D6 B6 41 00 {52} normal block at 0x003E5A48, 8 bytes long. Data: <lRE l A > 6C 52 45 00 6C B2 41 00 Object dump complete. I just wonder if I really got memory leaks or it was just a illusion. Any suggestion would be appreciated. Leo Yee
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 22 Dec 2003 at 22:07, Leo Yee wrote:
I compiled the code and ran it. Everything was ok except I got memory leaks from the output window of the VS like these:
I just wonder if I really got memory leaks or it was just a illusion. Any suggestion would be appreciated.
I personally have not seen any leaks originating from boost.python. If you want MSVC to print where the leaked memory was allocated, do the following: Place this at the start of each .cpp file: static const char *_fxmemdbg_current_file_ = __FILE__; Include the following header: #include <crtdbg.h> #define _FXMEMDBG_NEW_ new(1, _fxmemdbg_current_file_, __LINE__) #define _FXMEMDBG_MALLOC_(x) _malloc_dbg(x, 1, _fxmemdbg_current_file_, __LINE__) #define new _FXMEMDBG_NEW_ #define malloc(x) _FXMEMDBG_MALLOC_(x) Now MSVC will print which file and line number the leak was allocated. This can be very useful. Obviously define only for debug builds. If you want a real working example, see FXMemDbg.h in my TnFOX library http://www.nedprod.com/TnFOX/ Cheers, Niall -----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2 iQA/AwUBP+ePk8EcvDLFGKbPEQJvzACgpiy+NzLuXxEzdOKEKRibsROfb7oAoOgM 82BqUaZjewHhG1FNUO8srAlH =aViX -----END PGP SIGNATURE-----
If we insert code like this, _CrtSetBreakAlloc( 54 ); just after _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); we can see where boost::python allocates the memory and I found one of them was: struct py_function { template <class Caller> py_function(Caller const& caller) : m_impl(new caller_py_function_impl<Caller>(caller)) {} ... This is what I saw. If I insert memory leak dectetion code in your way, it founds nothing. It is realy confusing. Leo Yee "Leo Yee" <surffirst@yahoo.com> ÐŽÈëÓÊŒþ news:bs6tml$r5u$1@sea.gmane.org...
Hi, friends,
I use VS 7.1 and I have following code:
#include "stdafx.h" #include <crtdbg.h>
using namespace boost::python;
class cPart { public:
void set_text( const std::string &t ) { _text = t; } std::string text( void ) { return _text; }
private: std::string _text;
};
BOOST_PYTHON_MODULE(hello) {
class_<cPart, boost::noncopyable >("cPart") .add_property( "text", cPart::text, cPart::set_text ); }
int _tmain(int argc, _TCHAR* argv[]) { _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );
if (PyImport_AppendInittab("hello", inithello) == -1) return 1;
Py_Initialize();
PyRun_SimpleString( "from hello import *\n" "p = cPart()\n" "p.text = 'hello'\n" "print p.text\n" );
Py_Finalize();
getchar();
return 0; }
I compiled the code and ran it. Everything was ok except I got memory leaks from the output window of the VS like these:
Detected memory leaks! Dumping objects -> {54} normal block at 0x003E5AD8, 8 bytes long. Data: < RE A > 10 52 45 00 83 BE 41 00 {53} normal block at 0x003E5A90, 8 bytes long. Data: <(RE A > 28 52 45 00 D6 B6 41 00 {52} normal block at 0x003E5A48, 8 bytes long. Data: <lRE l A > 6C 52 45 00 6C B2 41 00 Object dump complete.
I just wonder if I really got memory leaks or it was just a illusion. Any suggestion would be appreciated.
Leo Yee
--- Leo Yee <surffirst@yahoo.com> wrote:
Py_Finalize();
I don't use embedding myself and therefore I could be wrong, but I believe I saw messages saying that you cannot use Py_Finalize() if you use Boost.Python. Ralf __________________________________ Do you Yahoo!? New Yahoo! Photos - easier uploading and sharing. http://photos.yahoo.com/
You are right. This seems caused by using if (PyImport_AppendInittab("hello", inithello) == -1) return 1; before Py_Initialize(); However, I can't find a solution to this problem. There is no documentation telling me how to remove these memeory leaks. Leo Yee "Ralf W. Grosse-Kunstleve" <rwgk@yahoo.com> ???? news:20031223143710.79105.qmail@web20205.mail.yahoo.com...
--- Leo Yee <surffirst@yahoo.com> wrote:
Py_Finalize();
I don't use embedding myself and therefore I could be wrong, but I believe I saw messages saying that you cannot use Py_Finalize() if you use Boost.Python. Ralf
__________________________________ Do you Yahoo!? New Yahoo! Photos - easier uploading and sharing. http://photos.yahoo.com/
--- Leo Yee <surffirst@yahoo.com> wrote:
You are right. This seems caused by using
if (PyImport_AppendInittab("hello", inithello) == -1) return 1;
before Py_Initialize();
However, I can't find a solution to this problem. There is no documentation telling me how to remove these memeory leaks.
Have you tried it the "normal" way, i.e. Py_Initialize() first and then PyImport_ImportModule()? And without calling Py_Finalize()? __________________________________ Do you Yahoo!? New Yahoo! Photos - easier uploading and sharing. http://photos.yahoo.com/
"Leo Yee" <surffirst@yahoo.com> writes:
You are right. This seems caused by using
if (PyImport_AppendInittab("hello", inithello) == -1) return 1;
before Py_Initialize();
That's not the problem. The problem is that Boost.Python hangs on to some Python references even after Py_Finalize is called. When the application exits, those reference counts are decremented, even though there's no interpreter running.
However, I can't find a solution to this problem. There is no documentation telling me how to remove these memeory leaks.
I doubt you can do anything about it without solving the problem described above. Those references are keeping memory from being deallocated. -- Dave Abrahams Boost Consulting www.boost-consulting.com
Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> writes:
Have you tried it the "normal" way, i.e. Py_Initialize() first and then PyImport_ImportModule()? And without calling Py_Finalize()?
I compiled the code to DLLs and import it from a python script but the memory leaks were still there. I guess if we put the following at the beginning of a program we will always get these memory leaks. int _tmain(int argc, _TCHAR* argv[]) { _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); ... "David Abrahams" <dave@boost-consulting.com> writes:
I doubt you can do anything about it without solving the problem described above. Those references are keeping memory from being deallocated.
I guess you mean that's not a problem and the memory leaks wouldn't made my programs mal-functional. Merry Chiristmas. Leo Yee
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 24 Dec 2003 at 8:03, David Abrahams wrote:
That's not the problem. The problem is that Boost.Python hangs on to some Python references even after Py_Finalize is called. When the application exits, those reference counts are decremented, even though there's no interpreter running.
Can't you at least do for those held references specially if(ref && Py_IsInitialized()) decref()? It does seem a shame to leak memory. I guess it's not so important in modern days of process boundaries, but I guess a non-leaking program shows someone has taken time and care which can only be a good thing. Certainly when certain software vendors who I won't mention publicly choose to ship debug builds of their retail software because the MSVC CRT debug lib doesn't crash!!! :( Cheers, Niall -----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2 iQA/AwUBP+nefMEcvDLFGKbPEQIp6ACghIV+NbkBMytCe2LEdmfN2DAGY6gAn0S5 v1YpRyr5IaMRJ0n99jisWBoJ =uF7j -----END PGP SIGNATURE-----
participants (4)
-
David Abrahams -
Leo Yee -
Niall Douglas -
Ralf W. Grosse-Kunstleve