[Python.NET] Memory leak problem in function "AsManagedObject"
Denis Akhiyarov
denis.akhiyarov at gmail.com
Fri Jan 29 01:59:49 EST 2016
I created a pull to fix this. Such calls should be inside using blocks,
like it was for other methods in PyObject. This particular one was missed.
It would be nice if any static analyzers revealed this. I tried PVS-Studio
and it did not reveal any problems on pythonnet, except handling of
Double.MaxValue & Double.MinValue.
On Wed, Jan 20, 2016 at 2:53 AM, Tony Roberts <tony at pyxll.com> wrote:
> Hi Eliana,
>
> great, glad you were able to find the problem. I'll make that change in
> github.
>
> thanks,
> Tony
>
> On Tue, Jan 19, 2016 at 8:46 PM Eliana Mendes <eliana.mendesp at gmail.com>
> wrote:
>
>> Thank you all for the answers!
>>
>>
>> I got the source code from the githup and built it myself, as you
>> suggested. Unfortunately the memory leak problem remains even with this
>> latest branch. But this time I was able to figure out where the problem is!
>>
>>
>>
>> The problem is not inside the “AsMangedObject”” as I thought before, but
>> instead it is inside the “GetItem(int index)” method which is inside the "
>> *pyobject*" class. This method creates a new instance of *PyInt *and a
>> new instance of *PyObject*. Those are exactly the objects that my memory
>> profiler was showing. In order to fix my problem I had to first edit
>> this method to be like this:
>>
>>
>> public virtual PyObject GetItem(int index) {
>>
>> PyInt key = new PyInt(index);
>>
>> PyObject item = GetItem((PyObject)key);
>>
>> key.Dispose();
>>
>> return item;
>>
>> }
>>
>>
>>
>> After doing that, in my code, I had to add a dispose for myTuple[i]. So I
>> changed my code to look something similar to this:
>>
>>
>>
>> PyObject tupleResult = myTuple[0];
>>
>> results[0] =
>> (double)tupleResult.AsManagedObject(typeof(double));
>>
>> tupleResult.Dispose();
>>
>>
>>
>> By doing those changes the problem was entirely fixed!
>>
>>
>>
>> I’m not sure who to contact exactly about changing the source code, but I
>> would highly recommend in making those changes in the “GetItem” method in
>> order to avoid someone else having the same memory problem.
>>
>>
>>
>> Thanks a lot for all your help!
>>
>>
>> Best regards,
>>
>>
>> Eliana
>>
>> 2016-01-18 16:54 GMT-06:00 Denis Akhiyarov <denis.akhiyarov at gmail.com>:
>>
>>> Your binaries are rather outdated..
>>>
>>> Here is the latest branch:
>>>
>>> https://github.com/pythonnet/pythonnet/tree/develop
>>>
>>> And binaries here:
>>>
>>> https://pypi.python.org/pypi/pythonnet/2.1.0.dev1
>>>
>>> http://www.lfd.uci.edu/~gohlke/pythonlibs/#pythonnet
>>>
>>> On Mon, Jan 18, 2016, 4:47 PM Eliana Mendes <eliana.mendesp at gmail.com>
>>> wrote:
>>>
>>>> Hello Tony,
>>>>
>>>>
>>>>
>>>> Thanks a lot for your answer! I actually just got directly the
>>>> “Python.Runtime.dll” from the download link in sourceforge:
>>>> http://sourceforge.net/projects/pythonnet/files/. But from the
>>>> description I see that the last update of that one was done in 2013. Would
>>>> there be a newer version somewhere else that I could get? Perhaps this
>>>> version I’m using is out of date and you have a newer one. Let me know if
>>>> that’s the case.
>>>>
>>>>
>>>>
>>>> I tried your suggestion of putting the “using” statement for the tuple
>>>> objects, and also for “myTuple”, and the memory is still increasing a lot.
>>>> I’ve made a memory profile to see which objects are being retained and
>>>> after I ran that statement 5 thousand times I got this:
>>>>
>>>>
>>>> [image: Imagem inline 1]
>>>>
>>>>
>>>> As you can see more that 12 thousand instances of “PyInt” and more than
>>>> 6 thousand of “PyObject” are created . Apparently it is creating these
>>>> “PyInt” and “PyObjects” somewhere inside the “AsManagedObject”, because I
>>>> don’t have PyInt objects anywhere in my code and this just happens when I
>>>> call “AsManagedObject”.
>>>>
>>>>
>>>>
>>>> Let me know if there is a newer version that could probably have that
>>>> problem fixed which I am not aware of.
>>>>
>>>>
>>>> I appreciate a lot your help!
>>>>
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Eliana
>>>>
>>>> 2016-01-17 6:05 GMT-06:00 Tony Roberts <tony at pyxll.com>:
>>>>
>>>>> Hi Eliana,
>>>>>
>>>>> which version of pythonnet are you using? when you say you're using
>>>>> the latest are you building it yourself from the develop branch on github?
>>>>>
>>>>> I had a quick look at the code that converts from a python object to a
>>>>> double, and I don't see any obvious memory leaks there. Perhaps the leak is
>>>>> the objects resulting from indexing into the tuple are never getting
>>>>> disposed? Try something like this instead and see if it helps (and report
>>>>> back as it may help improve the code if we know exactly what the problem
>>>>> is).
>>>>>
>>>>> PyTuple myTuple = PyTuple.AsTuple(result);
>>>>>
>>>>> double result0;
>>>>> using (var item0 = myTuple[0])
>>>>> result0 = (double)item0.AsManagedObject(typeof(double));
>>>>>
>>>>> double result1;
>>>>> using (var item1 = myTuple[1])
>>>>> result1 = (double)item1.AsManagedObject(typeof(double));
>>>>>
>>>>> myTuple.Dispose();
>>>>>
>>>>> I would also use a using statement for the myTuple as well, just to be
>>>>> sure dispose is called in the case an exception is thrown somewhere.
>>>>>
>>>>> Regards,
>>>>> Tony
>>>>>
>>>>>
>>>>> On Fri, Jan 15, 2016 at 8:06 PM Eliana Mendes <
>>>>> eliana.mendesp at gmail.com> wrote:
>>>>>
>>>>>> Hello experts,
>>>>>>
>>>>>>
>>>>>>
>>>>>> I'm having a memory leak problem when using the
>>>>>> function AsManagedObject(typeof(double)). Basically I have something like
>>>>>> this:
>>>>>>
>>>>>>
>>>>>>
>>>>>> PyTuple myTuple = PyTuple.AsTuple(result);
>>>>>>
>>>>>> double result0 = (double)myTuple[0].AsManagedObject(typeof(double));
>>>>>>
>>>>>> double result1 = (double)myTuple[1].AsManagedObject(typeof(double));
>>>>>>
>>>>>> myTuple.Dispose();
>>>>>>
>>>>>>
>>>>>>
>>>>>> where "result" is just a PyObject that returned from a python
>>>>>> function. I simplified the code above just so you can understand better,
>>>>>> but the thing is that the line that calls "AsManagedObject” is executed
>>>>>> thousands of times and it is increasing significantly the memory heap (it
>>>>>> goes over 3 GB of memory in my scenario and it’s not released after
>>>>>> execution). If I don't call just this specific function the memory remains
>>>>>> stable. But I don’t know any other way to convert the PyObject to "double"
>>>>>> unless using the “AsManagedObject” function.
>>>>>>
>>>>>> It sounds to me that some objects are allocated inside the
>>>>>> "AsManagedObject" method and they are not being released. Maybe it’s a bug
>>>>>> there. Any ideas? I'm using latest version of python for .NET.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>>
>>>>>> Eliana Mendes
>>>>>>
>>>>>> Software Engineer
>>>>>>
>>>>>> _________________________________________________
>>>>>> Python.NET mailing list - PythonDotNet at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pythondotnet
>>>>>
>>>>>
>>>>> _________________________________________________
>>>>> Python.NET mailing list - PythonDotNet at python.org
>>>>> https://mail.python.org/mailman/listinfo/pythondotnet
>>>>>
>>>>
>>>> _________________________________________________
>>>> Python.NET mailing list - PythonDotNet at python.org
>>>> https://mail.python.org/mailman/listinfo/pythondotnet
>>>
>>>
>>> _________________________________________________
>>> Python.NET mailing list - PythonDotNet at python.org
>>> https://mail.python.org/mailman/listinfo/pythondotnet
>>>
>>
>> _________________________________________________
>> Python.NET mailing list - PythonDotNet at python.org
>> https://mail.python.org/mailman/listinfo/pythondotnet
>
>
> _________________________________________________
> Python.NET mailing list - PythonDotNet at python.org
> https://mail.python.org/mailman/listinfo/pythondotnet
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythondotnet/attachments/20160129/6826e638/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 70883 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pythondotnet/attachments/20160129/6826e638/attachment-0001.png>
More information about the PythonDotNet
mailing list