[pypy-dev] stmgc: rethinking the GC?

Remi Meier remi.meier at gmail.com
Tue Dec 22 05:24:15 EST 2015


Hi Armin,

I think we need to quantify how bad the minor GCs really are. They are
thread-local and they should also take less time for shorter
transactions.

The last benchmark I looked at had a different problem that I would
summarize as (1) time spent creating backup copies, (2) huge commit
log entry size, (3) time spent pulling the changes in, (4) major
collections. All of these need to be amortized with longer
transactions, too.

I mention this because these problems come from writing to *old*
objects and at least (3) and (4) are non-thread-local issues that
prevent scaling. If we go for a different GC type, we need to make
sure that we still have ways to reduce the cost of these problems
(currently with overflow objs and ignoring young objs).

I don't know if minor GCs are really that high on the list of things
that cause overhead. It may just be that short transactions produce
more old objs relative to their runtime, and these old objs cause more
work in following transactions. So I currently see two important
directions to look into: (1) quantify the cost of minor GCs and
estimate the advantage of a non-moving GC, and (2) think about ways to
make old objs cheaper (e.g. by doing less work for thread-local objs).

But I'll first have to read up on that LuaJIT GC over the holidays :)

Cheers,
Remi

On 22 December 2015 at 08:30, Armin Rigo <arigo at tunes.org> wrote:
> Hi Remi,
>
> Just thinking about stmgc: currently it requires large transactions in
> order to work efficiently.  I think it is mostly caused by the fact
> that transaction commits always force a minor collection.  Maybe we
> should rethink that at some point, by changing the GC completely...
> There are moving GCs that are really concurrent but they tend to be an
> advanced mess.  Maybe a better alternative would be to experiment
> again with non-moving GCs.  Non-moving generational GCs are not very
> common as far as I know, but they do exist; for example see
> http://wiki.luajit.org/New-Garbage-Collector .
>
>
> A bientôt,
>
> Armin.


More information about the pypy-dev mailing list