[Baypiggies] Watching how much memory my Python program is taking up

[Paul Hoffman]

I've never had good luck with psutil; seemed flaky both times I tried it.

Also, I looked at the resource modules, but they don't give useful
information. That is, if you watch "top -o %MEM" in one window and run
Python CLI in another, then eat up a huge amount of memory with something
like a=bytearray(2**32), then free the memory with a=2, you see the memory
be released in "top", but resource.getrusage(resource.RUSAGE_SELF)[2] still
shows the larger amount because it is showing the maximum size used, not
the current.

Other thoughts?

--Paul Hoffman

On Thu, Feb 19, 2015 at 11:24 AM, Luke Gotszling <gotszlin at gmail.com> wrote:

> Hi Paul,
>
> You can try one of these methods:
> http://fa.bianp.net/blog/2013/different-ways-to-get-memory-consumption-or-lessons-learned-from-memory_profiler/
>
> Also look at the resource module:
> https://docs.python.org/3.4/library/resource.html
>
> Best, Luke
>
> On Thu, Feb 19, 2015 at 8:08 AM, Paul Hoffman <paul.hoffman at gmail.com>
> wrote:
>
>> Greetings. I have a Python 3.4 program that will keep eating up memory
>> until I tell it to stop; in short, it uses asyncio to launch as many
>> coroutines as the machine can handle. I want to determine approximately how
>> much memory I am using at this moment so I know when to stop consuming more
>> and having the box run out of memory (I already have swapping turned off).
>>
>> Are there well-known patterns for this? I looked in a bunch of places and
>> found nothing, which kinda surprised me. I recognize that it is unlikely
>> that there is a cross-OS method for this, but I would be happy with just
>> something that says "I'm now taking up a total of 3.7 gig" and I can do the
>> "how much is too much" calculation by hand.
>>
>> Pointers are appreciated.
>>
>> --Paul Hoffman
>>
>> _______________________________________________
>> Baypiggies mailing list
>> Baypiggies at python.org
>> To change your subscription options or unsubscribe:
>> https://mail.python.org/mailman/listinfo/baypiggies
>>
>
>
>
> --
>
> [image: --]
> Luke Gotszling
> [image: http://]about.me/luke
> <http://about.me/luke?promo=email_sig>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/baypiggies/attachments/20150219/21e1ad2b/attachment.html>