[Cython] Bug - PyPy tuple leak in __Pyx_PyObject_CallOneArg?

[Jason Madden]

Hello,

I'm one of the maintainers of the gevent concurrency library, which has recently been ported to run on PyPy. Under PyPy, a small portion of the code is compiled with Cython in order to get the desired atomic semantics (specifically, a semaphore class). We recently had a user report an easily reproducible leak of tuples of one element. Tracking it down, it appears that __Pyx_PyObject_CallOneArg creates a new tuple under PyPy, but neglects to free it. This was tested with Cython 0.23.3 and PyPy 2.6.1.

Our Cython code contained a loop like this, and every iteration of the loop leaked a tuple:

    for link in links:
        link(self)

The C output for that last line looked like this:

    __Pyx_PyObject_CallOneArg(__pyx_t_10, ((PyObject *)__pyx_v_self));...

And, under PyPy, the implementation of __Pyx_PyObject_CallOneArg is different than it is under CPython. Specifically, with Cython 0.23.3 this is the code:

    #if CYTHON_COMPILING_IN_CPYTHON
    ...
    #else
    static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) {
        PyObject* args = PyTuple_Pack(1, arg);
        return (likely(args)) ? __Pyx_PyObject_Call(func, args, NULL) : NULL;
    }
    #endif

PyTuple_Pack is documented as returning a new reference (https://docs.python.org/2/c-api/tuple.html#c.PyTuple_Pack), so it seems to me like this code should be decrementing the reference to the tuple before returning (much like the __Pyx__PyObject_CallOneArg function does under CPython).

Changing the loop to avoid the use of CallOneArg seemed to resolve the tuple leak, presumably at some execution time cost:

   args = (self,)
   for link in links:
       link(*self)

There's some additional background at https://bitbucket.org/pypy/pypy/issues/2149/memory-leak-for-python-subclass-of-cpyext#comment-22347393

Am I interpreting this correctly to be a bug, or could there be something wrong in the way we're handling callbacks? Please let me know if there's any further information I can provide.

Thanks,
Jason