Dear Paul,

Cannot reproduce the failure on Cython 0.19.1. If you are using an earlier version, please try that instead.

We also used Cython 0.19.1. If the unit test runs through, this means that the memory leak is there. I'm sorry for the ambiguity. Please change the lines if not cause_leak: n_memory_views_expexted -= 1 into just n_memory_views_expexted -= 1 Then the test will fail if the memory leak exists. I also just noticed that unittest.main() doesn't work in Cython files. The tests are just ignored. Please call the tests this way instead: python -m unittest memoryview_leak For your convenience, here's the updated code: -------- File memoryview_leak.pyx --------------------------------------------------------------- """ Cython module exposing memory leaks due to memoryview attributes in cdef classes in Cython 0.19.1 Compile:: python setup.py build_ext --inplace Run:: python -m unittest memoryview_leak """ from __future__ import print_function, division import unittest import gc import numpy as np cdef class LeakTestSuper: """ Superclass for the leak test in :class:`LeakTest1` and :class:`LeakTest2`. """ pass cdef class LeakTest1(LeakTestSuper): """ Class for memory leak testing. Holds a memoryview which is extracted from an :class:`numpy.ndarray` at initialization. This class avoids the leak by resetting the memoryview to None in the method :meth:`__dealloc__`. ``__del__()`` wouldn't work, because it is not supported in Cython's `cdef` classes. """ cdef double[:] _s def __init__(self): self._s = np.empty(2) def __dealloc__(self): self._s = None cdef class LeakTest2(LeakTestSuper): """ Class for memory leak testing. Holds an array that is allocated at initialization. This class does not avoid the leak and thus exposes the problem with memoryviews in Cython. """ cdef double[:] _s def __init__(self): self._s = np.empty(2) class TestLeakTest(unittest.TestCase): def n_objects_by_type(self, typename): return len([ obj for obj in gc.get_objects() if type(obj).__name__ == typename]) def test_gc(self): # Make sure to clear all old memoryviews (circularly referenced). gc.collect() # Now there mustn't be any memoryviews left. n_memory_views_expexted = 0 for cause_leak in [False, True, False, True]: # Create an object of LeakTest1 or LeakTest2 allocating a memoryview # internally. leaktest = LeakTest2() if cause_leak else LeakTest1() # Check the number of allocated memory views. n_memory_views_expexted += 1 n_memory_views = self.n_objects_by_type('memoryview') self.assertEqual(n_memory_views, n_memory_views_expexted) # Delete the reference to leaktest and let the garbage collector do # its thing. del leaktest gc.collect() # Check for leaks by counting the memoryviews again. if True: # not cause_leak: n_memory_views_expexted -= 1 n_memory_views = self.n_objects_by_type('memoryview') self.assertEqual(n_memory_views, n_memory_views_expexted) ----------------------------------------------------------------------------------------------------- Here's the corresponding setup file: -------- File setup.py ---------------------------------------------------------------------------- from __future__ import print_function, division from distutils.core import setup from Cython.Build import cythonize setup( name = "memoryview_leak", ext_modules = cythonize('memoryview_leak.pyx'), ) --------------------------------------------------------------------------------------------------------- Best regards, Bruno Daniel From: cython-devel [cython-devel-bounces+bruno.daniel=blue-yonder.com@python.org] on behalf of cython-devel-request@python.org [cython-devel-request@python.org] Sent: Wednesday, September 25, 2013 12:00 PM To: cython-devel@python.org Subject: cython-devel Digest, Vol 32, Issue 10 Send cython-devel mailing list submissions to cython-devel@python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/cython-devel or, via email, send a message with subject or body 'help' to cython-devel-request@python.org You can reach the person managing the list at cython-devel-owner@python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of cython-devel digest..." Today's Topics: 1. Re: Memory leak of memoryview attributes in cdef subclasses (Pauli Virtanen) ---------------------------------------------------------------------- Message: 1 Date: Tue, 24 Sep 2013 20:43:14 +0300 From: Pauli Virtanen <pav@iki.fi> To: cython-devel@python.org Subject: Re: [Cython] Memory leak of memoryview attributes in cdef subclasses Message-ID: <l1siva$rg5$1@ger.gmane.org> Content-Type: text/plain; charset=ISO-8859-1 Hi, 18.09.2013 16:14, Daniel, Bruno kirjoitti:

We encountered memory leaks when using memoryview attributes in cdef subclasses in Cython code. They can be avoided by adding a dealloc method setting the value of the memoryview attribute to None. The problem does not occur in topmost cdef classes.

Here's an example: (See the module docstring on how to compile and run the example.)

Cannot reproduce the failure on Cython 0.19.1. If you are using an earlier version, please try that instead. One bug with memory leaks in cdef classes was fixed in 0.18 IIRC. -- Pauli Virtanen ------------------------------ Subject: Digest Footer _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel ------------------------------ End of cython-devel Digest, Vol 32, Issue 10 ********************************************