[pypy-dev] Reference counting

[Maciej Fijalkowski]

On Sat, Jan 7, 2012 at 1:46 PM, Marko Tasic <mtasic85 at gmail.com> wrote:
> Hi,
>
> I've been carefully following your mailing list for years, and using PyPy
> for
> different kind of projects, mostly for highly distributed and decentralized
> systems,
> and everything is just great compared to CPython, Jython and IronPython.
> I've even written partial ctypes wrapper for GObject Introspection, so I
> could
> use Gtk as GUI toolkit on top of PyPy. Today, I cannot imagine writing
> Python
> code without executing it on PyPy because it just feels natural to use it as
> default
> implementation instead of an alternative CPython :)

Hi Marko, we're very glad to hear that!

>
> I know that you don't feel that using reference counting as GC mechanism is
> good
> idea because of overhead of maintaining reference counts to each object and
> non concurrent nature of it, but can you give me any idea where to start and
> how
> I can implement reference counting for PyPy. Have in mind that I am not new
> to
> Python, and low level stuff, just want to measure performance and possibly
> implement alternative reference counting strategies.

it is a very valid question. In fact, we already do have a refcounting
implementation which you get by passing --gc=ref to the translate.py
script. Refcounting has several ups and downs and it's definitely
valid to experiment and see overheads. However, at least current
approach and pypy in general has several problems with that:

* refcounting is implemented as a transformation of flow graphs. See
pypy/rpython/memory/gctransform/refcounting.py for details. This
approach works very well in the sense we don't have to maintain
refcounts by hand in the interpreter (we'll never do that). This is
all well and good, however it also means there is a significant
redundancy in references. You would need to either implement a smarter
policy or a refcount-removing optimization (typically run after
inlining) in order for that to be viable.

* we generally never cared about avoiding cycles in the python
interpreter itself. I don't think pypy's refcounting implementation
comes with cycle detector included, but don't quote me on that

* there is not JIT support. Typically JIT requires a bit of support
from the GC to cooperate. In order to get good results, you would
probably also need to implement an optimization that removes
unnecessary refcounts in traces.

I hope that helps clarify some things, feel free to ask more if not!

Cheers,
fijal