Battle of the garbage collectors, or ARGGHHHHHH!!!!
CFK
cfkaran2 at gmail.com
Sat Sep 2 20:44:21 EDT 2017
On Wed, Apr 26, 2017 at 10:38 PM, Cem Karan <cfkaran2 at gmail.com> wrote:
>
> On Apr 24, 2017, at 8:54 PM, Jon Ribbens <jon+usenet at unequivocal.eu>
> wrote:
>
> > On 2017-04-24, CFK <cfkaran2 at gmail.com> wrote:
> >> Long version: I'm trying to write bindings for python via ctypes to
> control
> >> a library written in C that uses the bdwgc garbage collector (
> >> http://www.hboehm.info/gc/). The bindings mostly work, except for when
> >> either bdwgc or python's garbage collector decide to get into an
> argument
> >> over what is garbage and what isn't, in which case I get a segfault
> because
> >> one or the other collector has already reaped the memory.
> >
> > Make your Python C objects contain a pointer to a
> > GC_MALLOC_UNCOLLECTABLE block that contains a pointer to the
> > bwdgc object it's an interface to? And GC_FREE it in tp_dealloc?
> > Then bwdgc won't free any C memory that Python is referencing.
>
> OK, I realized today that there was a miscommunication somewhere. My
> python code is all pure python, and the library is pure C, and it is not
> designed to be called by python (it's intended to be language neutral, so
> if someone wants to call it from a different language, they can). That
> means that tp_dealloc (which is part of the python C API) is probably not
> going to work.
>
> I got interrupted (again) so I didn't have a chance to try the next trick
> and register the ctypes objects as roots from which to scan in bdwgc, but
> I'm hoping that roots aren't removed. If that works, I'll post it to the
> list.
>
> Thanks,
> Cem Karan
Right, apparently I win at the 'late reply' game. That said, I wanted to
give Jon Ribbens credit for his idea, because it was very close to what I
used in the end. The only difference is that I also used weakref.finalize()
to tie a finalizer to the lifetime of the ctypes pointer that I was using.
The finalizer called GC_FREE() to free the uncollectable block, which
allowed the C allocator to cleanup memory.
The only thing I never figured out was how to get a C block to hold onto
python memory. I didn't need it, but it felt like it would make for a nice
duality with the method above.
Thanks,
Cem Karan
More information about the Python-list
mailing list