Zipimport leaks memory?

ashwin.u.rao at nokia.com ashwin.u.rao at nokia.com
Thu Aug 20 12:02:26 CEST 2009


Hi,

We are currently trying to identify and fix all the memory leaks by just doing Py_Initialize-PyRun_SimpleFile(some simple script)-Py_Finalize and found that there are around 70 malloc-ed blocks which are not freed. One of the significant contributor to this number is the 'files' object in ZipImporter.  I am not able to identify the reason for this leak and was wondering if anyone on this list would help me out here. 

So, here goes :

Since we have a zip file in our sys.path, this object is initialized and added to the zip_directory_cache dict during Py_Initialize. One point to note here is that there is no DECREF on the 'files' object after adding it to the zip_directory_cache dict.

When a module in a directory is imported(encoding.alias) then the reference count of 'files' is increased. When this module is unloaded during Py_Finalize-PyImport_Cleanup, the ref count of files object is decremented properly. So at the end of Py_Finalize the files object still has one reference count which is a result of it being an entry in the zip_directory_cache.

To summarize :

ZipImporter->files - zipimporter_init function:

Py_Initialize - 
Read files from zip - ref_count -1
Add files to zip_directory_cache - ref count - 2
Import some modules from different directory in zip - ref count - 3

Py_Finalze - 
PyImport_Cleanup - sub-directory - ref-count - 2
PyImport_Cleanup - main directory - ref-count - 1

So the reference count of 'files' is still 1 after Py_Finalize. The zip_directory_cache is the dict that has this 'files' object and this dict is added to the module during zipimport module initialization. This dict is destroyed during _PyModule_Clear(called by PyImport_Cleanup) but the 'files' object's reference count is not decremented. 

Please excuse me if I have misinterpreted the whole logic. Any pointers towards solving this would be great. 

Thank you.

~Ashwin




More information about the Python-list mailing list