What is heating the memory here? hashlib?

Paulo da Silva p_s_d_a_s_i_l_v_a_ns at netcabo.pt
Sat Feb 13 20:44:55 EST 2016


Às 22:45 de 13-02-2016, Chris Angelico escreveu:
> On Sun, Feb 14, 2016 at 9:26 AM, Paulo da Silva
> <p_s_d_a_s_i_l_v_a_ns at netcabo.pt> wrote:
>> I meant eating! :-)
> 
> Heh, "heating" works too - the more you use memory, the more it heats up :)
:-) It is heating my head!
...

> 
> What happens if, after hashing each file (and returning from this
> function), you call gc.collect()? If that reduces your RAM usage, you
> have reference cycles somewhere.
> 
I have used gc and del. No luck.

The most probable cause seems to be hashlib not correctly handling big
buffers updates. I am working in a computer and testing in another. For
the second part may be somehow I forgot to transfer the change to the
other computer. Unlikely but possible.

Anyway it is doing its job right now with bfsz=16KB (this takes a few
hours). No memory leakages anymore.
I'll address this problem in a near future, may be when I move to the
new kubuntu 16.04 LTS. This will bring new SW releases and the problem
may have been fixed.

Thanks
Paulo



More information about the Python-list mailing list